We will create EMR HBase clusters using the tooling provided by Amazon:
http://elasticmapreduce.s3.amazonaws.com/elastic-mapreduce-ruby.zip

Note: As you might see in commands.rb the default_hadoop_version is set to 0.20(.x), but as our tests found using Hadoop in version 1.0.3 has significant performance gain. Therefore when creating EMR cluster, we will explicitly set this version.

Let’s create one:

elastic-mapreduce --create \
--hbase \
--name "EMR HBase YCSB" \
--num-instances 2 \
--instance-type m1.large \
--hadoop-version 1.0.3
Created job flow j-1PP3JU6UJ0HQ1

elastic-mapreduce --list --active
j-1PP3JU6UJ0HQ1     WAITING
ec2-23-22-19-48.compute-1.amazonaws.com          EMR HBase YCSB
 COMPLETED      Start HBase

Build the project (HBase master server variables should now defaults to localhost (127.0.0.1)).

git clone git@github.com:dataminelab/YCSB.git
cd YCSB
export MAVEN_OPTS="-Xmx512m -Xms128m -Xss2m"

(check http://jira.codehaus.org/browse/MASSEMBLY-549 why…)

mvn clean install -Dcheckstyle.skip=true
cd distribution/target
scp -i ~/.ssh/dataminelab-ec2.pem ycsb-0.1.5-SNAPSHOT.tar.gz \
hadoop@ec2-23-22-19-48.compute-1.amazonaws.com:/home/hadoop/ycsb.tar.gz 
ssh -i ~/.ssh/dataminelab-ec2.pem \
hadoop@ec2-23-22-19-48.compute-1.amazonaws.com
tar xvzf ycsb.tar.gz
ln -s ycsb-0.1.5-SNAPSHOT ycsb
cd ycsb

Create the working table in HBase (aleady pre-split):

hbase org.apache.hadoop.hbase.util.RegionSplitter usertable -c 200 -f family

Hard to be perfect – because of https://issues.apache.org/jira/browse/HBASE-4163 is still not in place – please vote! :)
But it still seems to be better than no split at all!

You might spot:

12/08/25 13:39:16 ERROR metrics.MetricsSaver:
Failed SaveRecords hdfs:/mnt/var/lib/hadoop/metrics/raw/i-694c4712_04272_raw.bin
Shutdown in progress

as in https://forums.aws.amazon.com/thread.jspa?threadID=100643 but it doesn’t seem to hurt us…

hbase shell
scan '.META.', {COLUMNS => 'info:regioninfo'}
exit

Load initial data into HBase

./bin/ycsb load hbase -p columnfamily=family -P workloads/workloada | tee load.log

Check for your own eyes that the data is loaded into HBase

hbase shell

hbase(main):001:0> count 'usertable'
Current count: 1000, row: user995698996184959679
1000 row(s) in 2.3210 seconds

And run the tests – only as a warm-up:

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=10000 \
-s \
-threads 10 | tee warm-up-tests.log

And now the real tests with 10 threads:

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 10 | tee real-tests-workload-a.log

cat real-tests-workload-a.log

[OVERALL], RunTime(ms), 47132.0
[OVERALL], Throughput(ops/sec), 2121.700755325469
[UPDATE], Operations, 50209
[UPDATE], AverageLatency(us), 186.93305980999423

And also 10 threads, but for another workload type.

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloadf \
-p columnfamily=family \
-p operationcount=100000 \
-s -threads 10 | tee real-tests-workload-f.log
cat real-tests-workload-f.log

[OVERALL], RunTime(ms), 52748.0
[OVERALL], Throughput(ops/sec), 1895.8064760749223
[UPDATE], Operations, 50018
[UPDATE], AverageLatency(us), 11.925006997480907

Now we might check how these workload scenarios behave when increasing thread number.
Starting with 100 threads.

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 100 | tee real-tests-workload-a-100t.log
cat real-tests-workload-a-100t.log

[OVERALL], RunTime(ms), 24234.0
[OVERALL], Throughput(ops/sec), 4126.433935792688
[UPDATE], Operations, 50063
[UPDATE], AverageLatency(us), 1076.5547010766434

500 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 500 | tee real-tests-workload-a-500t.log
cat real-tests-workload-a-500t.log

[OVERALL], RunTime(ms), 20706.0
[OVERALL], Throughput(ops/sec), 4829.518014102193
[UPDATE], Operations, 50099
[UPDATE], AverageLatency(us), 6167.192359128925

1000 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 1000 | tee real-tests-workload-a-1kt.log
cat real-tests-workload-a-1kt.log

[OVERALL], RunTime(ms), 21484.0
[OVERALL], Throughput(ops/sec), 4654.626698938745
[UPDATE], Operations, 49988
[UPDATE], AverageLatency(us), 9423.208390013604

2000 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 2000 | tee real-tests-workload-a-2kt.log
cat real-tests-workload-a-2kt.log

[OVERALL], RunTime(ms), 24358.0
[OVERALL], Throughput(ops/sec), 4105.427374989737
[UPDATE], Operations, 49957
[UPDATE], AverageLatency(us), 7786.985767760274

And the same for the other workload scenario now:
100 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloadf \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 100 | tee real-tests-workload-f-100t.log
cat real-tests-workload-f-100t.log

[OVERALL], RunTime(ms), 33924.0
[OVERALL], Throughput(ops/sec), 2947.7655936799906
[UPDATE], Operations, 50136
[UPDATE], AverageLatency(us), 17.44125977341631

1000 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloadf \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 1000 | tee real-tests-workload-f-1kt.log
cat real-tests-workload-f-1kt.log

[OVERALL], RunTime(ms), 29309.0
[OVERALL], Throughput(ops/sec), 3411.921252857484
[UPDATE], Operations, 50127
[UPDATE], AverageLatency(us), 16.611586570111914

2000 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloadf \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 2000 | tee real-tests-workload-f-2kt.log
cat real-tests-workload-f-2kt.log

[OVERALL], RunTime(ms), 29311.0
[OVERALL], Throughput(ops/sec), 3411.688444611238
[UPDATE], Operations, 49951
[UPDATE], AverageLatency(us), 59.80148545574663

3000 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloadf \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 3000 | tee real-tests-workload-f-3kt.log
cat real-tests-workload-f-3kt.log

[OVERALL], RunTime(ms), 32314.0
[OVERALL], Throughput(ops/sec), 3063.6875657609703
[UPDATE], Operations, 49492
[UPDATE], AverageLatency(us), 20.00127293299927

4000 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloadf \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 4000 | tee real-tests-workload-f-4kt.log
cat real-tests-workload-f-4kt.log

[OVERALL], RunTime(ms), 35051.0
[OVERALL], Throughput(ops/sec), 2852.985649482183
[UPDATE], Operations, 50095
[UPDATE], AverageLatency(us), 38.50611837508733

Let’s now try more instances instead just one slave – 4 slaves, same type as before.

elastic-mapreduce --create \
--hbase \
--name "EMR HBase YCSB" \
--num-instances 5 \
--instance-type m1.large \
--hadoop-version 1.0.3
Created job flow j-OE7G6YUHMD2I

elastic-mapreduce --list --active
j-OE7G6YUHMD2I      WAITING
ec2-50-17-100-242.compute-1.amazonaws.com         EMR HBase YCSB
COMPLETED      Start HBase

Now just copy already built test suite:

scp -i ~/.ssh/dataminelab-ec2.pem ycsb-0.1.5-SNAPSHOT.tar.gz \
hadoop@ec2-50-17-100-242.compute-1.amazonaws.com:/home/hadoop/ycsb.tar.gz
ssh -i ~/.ssh/dataminelab-ec2.pem \
hadoop@ec2-50-17-100-242.compute-1.amazonaws.com

tar xvzf ycsb.tar.gz
ln -s ycsb-0.1.5-SNAPSHOT ycsb
cd ycsb

Initialize table:

hbase org.apache.hadoop.hbase.util.RegionSplitter usertable -c 200 -f family

Load initial data:

./bin/ycsb load hbase \
-p columnfamily=family \
-P workloads/workloada | tee load.log

And run tests:
warm-up

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=10000 \
-s \
-threads 10 | tee warm-up-tests.log

10 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 10 | tee real-tests-workload-a.log
cat real-tests-workload-a.log

[OVERALL], RunTime(ms), 42609.0
[OVERALL], Throughput(ops/sec), 2346.9220117815485
[UPDATE], Operations, 50073
[UPDATE], AverageLatency(us), 117.53685618996265

100 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 100 | tee real-tests-workload-a-100t.log
cat real-tests-workload-a-100t.log

[OVERALL], RunTime(ms), 23500.0
[OVERALL], Throughput(ops/sec), 4255.31914893617
[UPDATE], Operations, 49837
[UPDATE], AverageLatency(us), 1089.7759295302687

500 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 500 | tee real-tests-workload-a-500t.log
cat real-tests-workload-a-500t.log

[OVERALL], RunTime(ms), 19763.0
[OVERALL], Throughput(ops/sec), 5059.960532307848
[UPDATE], Operations, 50196
[UPDATE], AverageLatency(us), 4854.259104311101

1000 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 1000 | tee real-tests-workload-a-1kt.log
cat real-tests-workload-a-1kt.log

[OVERALL], RunTime(ms), 20028.0
[OVERALL], Throughput(ops/sec), 4993.0097862991815
[UPDATE], Operations, 49904
[UPDATE], AverageLatency(us), 9582.977617024688

2000 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 2000 | tee real-tests-workload-a-2kt.log
cat real-tests-workload-a-2kt.log

[OVERALL], RunTime(ms), 22608.0
[OVERALL], Throughput(ops/sec), 4423.2130219391365
[UPDATE], Operations, 49988
[UPDATE], AverageLatency(us), 6244.29357045691

5000 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 5000 | tee real-tests-workload-a-5kt.log
cat real-tests-workload-a-5kt.log

[OVERALL], RunTime(ms), 24861.0
[OVERALL], Throughput(ops/sec), 4022.3643457624394
[UPDATE], Operations, 50100
[UPDATE], AverageLatency(us), 8150.377125748503

10k threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 10000 | tee real-tests-workload-a-10kt.log
cat real-tests-workload-a-10kt.log

[OVERALL], RunTime(ms), 25336.0
[OVERALL], Throughput(ops/sec), 3946.9529523208084
[UPDATE], Operations, 50176
[UPDATE], AverageLatency(us), 8851.578204719388

workload f, 10 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloadf \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 10 | tee real-tests-workload-f.log
cat real-tests-workload-f.log

[OVERALL], RunTime(ms), 53310.0
[OVERALL], Throughput(ops/sec), 1875.8206715438005
[UPDATE], Operations, 49867
[UPDATE], AverageLatency(us), 12.18058034371428

100 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloadf \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 100 | tee real-tests-workload-f-100t.log
cat real-tests-workload-f-100t.log

[OVERALL], RunTime(ms), 30991.0
[OVERALL], Throughput(ops/sec), 3226.7432480397533
[UPDATE], Operations, 50145
[UPDATE], AverageLatency(us), 13.73040183467943

1k threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloadf \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 1000 | tee real-tests-workload-f-1kt.log
cat real-tests-workload-f-1kt.log

[OVERALL], RunTime(ms), 29185.0
[OVERALL], Throughput(ops/sec), 3426.4176803152304
[UPDATE], Operations, 50047
[UPDATE], AverageLatency(us), 29.82979998801127

2k threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloadf \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 2000 | tee real-tests-workload-f-2kt.log
cat real-tests-workload-f-2kt.log

[OVERALL], RunTime(ms), 31906.0
[OVERALL], Throughput(ops/sec), 3134.206732276061
[UPDATE], Operations, 50111
[UPDATE], AverageLatency(us), 24.55253337590549

3k threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloadf \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 3000 | tee real-tests-workload-f-3kt.log
cat real-tests-workload-f-3kt.log

[OVERALL], RunTime(ms), 34410.0
[OVERALL], Throughput(ops/sec), 2877.070619006103
[UPDATE], Operations, 49607
[UPDATE], AverageLatency(us), 23.37424153849255

Now let’s see how even more serious instances offered by AWS would behave in this scenario!
m1.xlarge (2 x more memory, 2 x more CPU than m1.large)

elastic-mapreduce --create \
--hbase \
--name "EMR HBase YCSB" \
--num-instances 5 \
--instance-type m1.xlarge \
--hadoop-version 1.0.3
Created job flow j-2ICBS9029MJAV

./elastic-mapreduce --list --active
j-2ICBS9029MJAV      WAITING
ec2-107-21-130-111.compute-1.amazonaws.com         EMR HBase YCSB
COMPLETED      Start HBase

scp -i ~/.ssh/dataminelab-ec2.pem ycsb-0.1.5-SNAPSHOT.tar.gz \
hadoop@ec2-107-21-130-111.compute-1.amazonaws.com:/home/hadoop/ycsb.tar.gz
ssh -i ~/.ssh/dataminelab-ec2.pem \
hadoop@ec2-107-21-130-111.compute-1.amazonaws.com

tar xvzf ycsb.tar.gz
ln -s ycsb-0.1.5-SNAPSHOT ycsb
cd ycsb

hbase org.apache.hadoop.hbase.util.RegionSplitter usertable -c 200 -f family

./bin/ycsb load hbase \
-p columnfamily=family \
-P workloads/workloada | tee load.log

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=10000 \
-s \
-threads 10 | tee warm-up-tests.log

10 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 10 | tee real-tests-workload-a.log
cat real-tests-workload-a.log

[OVERALL], RunTime(ms), 39481.0
[OVERALL], Throughput(ops/sec), 2532.8639092221574
[UPDATE], Operations, 49981
[UPDATE], AverageLatency(us), 62.85440467377604

100 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 100 | tee real-tests-workload-a-100t.log
cat real-tests-workload-a-100t.log

[OVERALL], RunTime(ms), 17877.0
[OVERALL], Throughput(ops/sec), 5593.779716954747
[UPDATE], Operations, 50100
[UPDATE], AverageLatency(us), 640.4568662674651

1k threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s -threads 1000 | tee real-tests-workload-a-1kt.log
cat real-tests-workload-a-1kt.log

[OVERALL], RunTime(ms), 13986.0
[OVERALL], Throughput(ops/sec), 7150.00715000715
[UPDATE], Operations, 49750
[UPDATE], AverageLatency(us), 8759.566291457286

2k threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 2000 | tee real-tests-workload-a-2kt.log
cat real-tests-workload-a-2kt.log

[OVERALL], RunTime(ms), 14783.0
[OVERALL], Throughput(ops/sec), 6764.526821348847
[UPDATE], Operations, 50118
[UPDATE], AverageLatency(us), 26718.534857735744

3k threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 3000 | tee real-tests-workload-a-3kt.log
cat real-tests-workload-a-3kt.log

[OVERALL], RunTime(ms), 15477.0
[OVERALL], Throughput(ops/sec), 6396.588486140725
[UPDATE], Operations, 49465
[UPDATE], AverageLatency(us), 12066.01403012231

4k threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 4000 | tee real-tests-workload-a-4kt.log
cat real-tests-workload-a-4kt.log

[OVERALL], RunTime(ms), 15261.0
[OVERALL], Throughput(ops/sec), 6552.650547146321
[UPDATE], Operations, 49883
[UPDATE], AverageLatency(us), 22551.664294449012

another workload, 10 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloadf \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 10 | tee real-tests-workload-f.log
cat real-tests-workload-f.log

[OVERALL], RunTime(ms), 45751.0
[OVERALL], Throughput(ops/sec), 2185.744573889095
[UPDATE], Operations, 49950
[UPDATE], AverageLatency(us), 9.801721721721721

500 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloadf \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 500 | tee real-tests-workload-f-500t.log
cat real-tests-workload-f-500t.log

[OVERALL], RunTime(ms), 21870.0
[OVERALL], Throughput(ops/sec), 4572.473708276178
[UPDATE], Operations, 49678
[UPDATE], AverageLatency(us), 11.18187125085551

1k threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloadf \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 1000 | tee real-tests-workload-f-1kt.log
cat real-tests-workload-f-1kt.log

[OVERALL], RunTime(ms), 19207.0
[OVERALL], Throughput(ops/sec), 5206.435153850159
[UPDATE], Operations, 49879
[UPDATE], AverageLatency(us), 11.812406022574631

2k threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloadf \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 2000 | tee real-tests-workload-f-2kt.log
cat real-tests-workload-f-2kt.log

[OVERALL], RunTime(ms), 20493.0
[OVERALL], Throughput(ops/sec), 4879.715024642561
[UPDATE], Operations, 50114
[UPDATE], AverageLatency(us), 12.770423434569182

And for now, more CPU power!
c1.xlarge (same memory, 5 x more CPU than m1.large)

elastic-mapreduce --create \
--hbase \
--name "EMR HBase YCSB" \
--num-instances 5 \
--instance-type c1.xlarge \
--hadoop-version 1.0.3
Created job flow j-3KZHQRG2D74AY

./elastic-mapreduce --list --active
j-3KZHQRG2D74AY     WAITING
ec2-75-101-255-226.compute-1.amazonaws.com          EMR HBase YCSB
COMPLETED      Start HBase

scp -i ~/.ssh/dataminelab-ec2.pem ycsb-0.1.5-SNAPSHOT.tar.gz \
hadoop@ec2-75-101-255-226.compute-1.amazonaws.com:/home/hadoop/ycsb.tar.gz
ssh -i ~/.ssh/dataminelab-ec2.pem \
hadoop@ec2-75-101-255-226.compute-1.amazonaws.com

tar xvzf ycsb.tar.gz
ln -s ycsb-0.1.5-SNAPSHOT ycsb
cd ycsb

hbase org.apache.hadoop.hbase.util.RegionSplitter usertable -c 200 -f family

./bin/ycsb load hbase \
-p columnfamily=family \
-P workloads/workloada | tee load.log

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=10000 \
-s \
-threads 10 | tee warm-up-tests.log

10 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 10 | tee real-tests-workload-a.log
cat real-tests-workload-a.log

[OVERALL], RunTime(ms), 32121.0
[OVERALL], Throughput(ops/sec), 3113.228106223343
[UPDATE], Operations, 49973
[UPDATE], AverageLatency(us), 71.10029415884577

100 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 100 | tee real-tests-workload-a-100t.log
cat real-tests-workload-a-100t.log

[OVERALL], RunTime(ms), 15076.0
[OVERALL], Throughput(ops/sec), 6633.059166887769
[UPDATE], Operations, 50167
[UPDATE], AverageLatency(us), 644.8327187194769

1k threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 1000 | tee real-tests-workload-a-1kt.log
cat real-tests-workload-a-1kt.log

[OVERALL], RunTime(ms), 12864.0
[OVERALL], Throughput(ops/sec), 7773.63184079602
[UPDATE], Operations, 50240
[UPDATE], AverageLatency(us), 9889.390306528663

2k threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 2000 | tee real-tests-workload-a-2kt.log
cat real-tests-workload-a-2kt.log

[OVERALL], RunTime(ms), 14889.0
[OVERALL], Throughput(ops/sec), 6716.367788300087
[UPDATE], Operations, 50216
[UPDATE], AverageLatency(us), 41222.41986617811

3k threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 3000 | tee real-tests-workload-a-3kt.log
cat real-tests-workload-a-3kt.log

[OVERALL], RunTime(ms), 14461.0
[OVERALL], Throughput(ops/sec), 6845.9995850909345
[UPDATE], Operations, 49451
[UPDATE], AverageLatency(us), 51852.53568178601

5k threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 5000 | tee real-tests-workload-a-5kt.log
cat real-tests-workload-a-5kt.log

[OVERALL], RunTime(ms), 17072.0
[OVERALL], Throughput(ops/sec), 5857.544517338331
[UPDATE], Operations, 49835
[UPDATE], AverageLatency(us), 82378.54861041436

10k threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloada \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 10000 | tee real-tests-workload-a-10kt.log
cat real-tests-workload-a-10kt.log

[OVERALL], RunTime(ms), 20226.0
[OVERALL], Throughput(ops/sec), 4944.131316127757
[UPDATE], Operations, 50113
[UPDATE], AverageLatency(us), 49147.25219005049

another workload, 10 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloadf \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 10 | tee real-tests-workload-f.log
cat real-tests-workload-f.log

[OVERALL], RunTime(ms), 40801.0
[OVERALL], Throughput(ops/sec), 2450.920320580378
[UPDATE], Operations, 49966
[UPDATE], AverageLatency(us), 12.13715326421967

400 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloadf \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 400 | tee real-tests-workload-f-400t.log
cat real-tests-workload-f-400t.log

[OVERALL], RunTime(ms), 17856.0
[OVERALL], Throughput(ops/sec), 5600.358422939068
[UPDATE], Operations, 50071
[UPDATE], AverageLatency(us), 14.301591739729584

500 threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloadf \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 500 | tee real-tests-workload-f-500t.log
cat real-tests-workload-f-500t.log

[OVERALL], RunTime(ms), 17909.0
[OVERALL], Throughput(ops/sec), 5583.784689262382
[UPDATE], Operations, 50210
[UPDATE], AverageLatency(us), 16.105915156343357

1k threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloadf \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 1000 | tee real-tests-workload-f-1kt.log
cat real-tests-workload-f-1kt.log

[OVERALL], RunTime(ms), 16982.0
[OVERALL], Throughput(ops/sec), 5888.5879166175955
[UPDATE], Operations, 50088
[UPDATE], AverageLatency(us), 15.313268647180962

2k threads

./bin/ycsb run hbase \
-p columnfamily=family \
-P workloads/workloadf \
-p columnfamily=family \
-p operationcount=100000 \
-s \
-threads 2000 | tee real-tests-workload-f-2kt.log
cat real-tests-workload-f-2kt.log

[OVERALL], RunTime(ms), 17219.0
[OVERALL], Throughput(ops/sec), 5807.538184563564
[UPDATE], Operations, 49989
[UPDATE], AverageLatency(us), 17.61469523295125

Even after running these simple scenarios we are able to check how for given configuration the number of threads used influences the throughput for each of workload type:

• workload a:
  
• workload f:
  

You can now play with other instance types and instance numbers. You can also mix multiple nodes running YCSB benchmark code and observe possible saturation, either from master’s CPU or network layer.

We also invite you to play with the code or even contribute features and improvements, so that others can benefit from them too – have fun!

There is an increasing number of monthly meetups: BigData London, HUG UK, Data Science London, London R, Cassandra London, Neo4j London, London MongoDB User Group, Oracle BigData, Data Visualisation London, Big Data Debate, DeNormalised London, LonData, CloudComputing.

Upcoming conferences that are worth mentioning:

• Skillsmatter Progressive NoSQL Tutorial (9-11th May 2012)
• Berlin Buzzworlds – only couple hours from London via Eurostar (4-5th June 2012)
• Big Data Analytics 2012 (20th June 2012)
• Big Data Summit 2012 (28th June 2012)
• Big Data World Europe (19-20th September 2012)
• DeNormalised NoSQL Conference London (20-21st September 2012)
• Big Data Congress (7th November 2012)

We just had a London BigData week that was full of meetings and hackatons dedicated to Hadoop, Visualisations and NoSQL. In case you missed the last Big Data week you are for a treat – simply like us on Facebook to have a chance of winning one ticket (worth £495) for 3 days of SkillsMatter NoSQL tutorials.

There are as well few online places where every data scientist can improve or challenge their skills:

• Stanford Machine Learning – online Machine Learning course
• Harvard Open Learning Initiative – online computer science courses from Harvard
• iTunes U – growing library of online courses
• BigData University – few basic tutorials on BigData
• Kaggle – test your skills and win prizes

If you know of anything interesting coming up in London, let us know in the comments.

Bioinformatics is the science of applying information technology to biology in order to understand the latter. There are numerous ways in which computers can aid biologists. In this particular project, we have been using microarrays to find the connection between different genes. The use of microarray technologies has enabled us to detect changes to gene expression across the genome in thousands of experiments with hundreds of species. However, interpreting the changes identified in these experiments has been hampered by a lack of knowledge of the gene function. Even in highly studied genomes, approximately 50-60% of genes will be assigned functions, yet less than 30% will be annotated with a highly specific function. Little of the annotation will have been observed in experiments conducted with the species of interest, as most gene function annotation is based on annotations assigned to orthologous genes taken from experiments done with other species, such as yeast and mammalian cell culture.

We are interested in building a better understanding of gene function in the worm C. elegans by harnessing the large quantity of experimental microarray data in the public database. Currently, we have a database of over fifty curated experiments. With this, we attempt to assign putative functions to genes based on the expression profile across experiments in the public repositories. My role in this project is to help expand the number of curated experiments in the database and study the functions of approximately 1000 genes known to be regulated in long-lived worms, to try to understand the functions of these genes, e.g. by showing experimental evidence of a role in nutrient sensing, innate immunity or stress response.

Here are the slides from the presentation. Refer to slides 10 and 11 to see how to migrate your R application to the cloud in just 3 lines of code:

Oh, and did I mention how cool our lab is? Have a look at the following ad, which was made at UCL  just a couple of metres from my desk.

Full disclosure: DataMine Lab is in no way affiliated with Birkbeck or UCL and the above project is part of my individual bioinformatics studies.

A couple of events worth mentioning are:

• NoSQL eXchange (2 November, 2011)
• Predictive Analytics (30 November – 1 December 2011)

NoSQL Exchange should offer an interesting overview on various NoSQL technologies, including MongoDB, Cassandra, CouchDB and Riak. Tom Wilkie (from Acunu) will give a tour on the future of NOSQL. Data Mine Lab is a sponsor of NoSQL Exchange and we are giving away one free ticket (worth £195). Simply like us on Facebook before 21nd of October for a chance to win that ticket. Every new follower within that timeframe will be entered into a drawing.

There are also some recurring events on the topics of information, big data and visualisation:

• Creative Mornings – the last talk by David McCandless from Information is Beautiful was a treat, and the next one looks great for data geeks as well.
• Quantified Self – an interesting take on analysing the data from your life.
• BigData London – a good mix of like-minded people working with a range of big data technologies.
• HUG UK – another Meetup group, this time focussed on Hadoop.
• Cassandra London – for users of Apache Cassandra.
• Neo4j London – a graph database group.
• London MongoDB User Group – is featuring talks on MongoDB and NoSQL.

If you’re as busy as us, you probably find it difficult to keep up with international events such as OSCON and Strata conferences. Fortunately, thanks to O’Reilly you can enjoy the talks even if you missed the conference in the first place. There’s plenty of interesting material on the website and it’s definitely worth a look:

• OSCON Data Sessions 2011
• Strata Conference 2011

This list isn’t intended to be exhaustive and undoubtedly there are data events happening that we’re unaware of. If you know of anything interesting coming up in London, let us know in the comments.

One of the most important website metrics is the number of unique visitors. However, it is also one of the most difficult to calculate. In this post, I will review a sampling strategy which produces a very good estimate of unique users, yet is computationally cheap.

Non-additive data

It is relatively easy to calculate small numbers of unique visitors: all you need to do is perform a single SQL query.

To calculate the number of unique records in Hive, run the following:

[gist id=1159244]

However, once the number of records in the table “page_views” becomes very large, this query may result in OOM errors. If this happens, there are other ways to calculate the exact number of unique visitors. Alternatively, it is possible to generate useful figures by using a sample.

Sampling

In practice, estimating the unique visitors metric gives pretty close results. In our tests on tens of millions of records, the results came within 0.1% of real values. One thing to remember is to ensure you sample visitors and not page views. The presented sampling method is a simple Bernoulli Sampling.

Having a sample can sometimes be even more useful than calculating the exact number. You can build a data warehouse around the sample and slice and dice on unique visitors — something which cannot be done on pre-calculated non-additive data. I will show at the end of this post how to create a cube that can be used to visualise unique visitors data.

Hashing

In order to sample users, we need to get every n-th user randomly from the population of records. One way to do it is to calculate the visitor hash for every record using a uniform hashing function (such as Md5). Md5 generates a random hexadecimal string on which we can filter only those users whose hash finishes with an arbitrary string, such as ’00’. Notice that since this is a uniform hashing function, the probability that the user hash finishes with ‘0’ is 1/16, and so the probability that it finishes with ’00’ is 1/256.

Note that Hive (at the time of writing, version 0.7) does not implement an Md5 function, so feel free to use the following code to add an Md5 hash function to Hive:

[gist id=1050002]

Alternatively you may patch your Hive distribution with the code from the following ticket HIVE-1262.

HiveQL

The following query will generate a unique visitors sample:

[gist id=1049918]

Pentaho

There are many other issues with unique visitors, such as how to present non-additive results to the end user. BI tools (such as Pentaho Mondrian) allow you to do this with the distinct aggregate function:

[gist id=1159282]

After loading the sample to your aggregate, the OLAP tools will allow you to report on it in a similar way to how you would report on standard additive data. See below:

Unique Visitors Cube

You should have a good knowledge of algorithms and TDD, and be experienced in building real-world high traffic applications.

We constantly test new technology, using solutions such as:

• Hadoop / Hive / HBase
• Amazon Cloud
• Mahout
• and many others

Experience with Hadoop is an asset, but more important is your ability to quickly learn new technology, commitment to quality, attention to detail, and enthusiasm.

How to apply
If you think you are right for this role, please send your CV to info@dataminelab.com with the subject line [Developer], explaining why you would like to work with us.

Look forward to hearing from you!

I often asked myself this question while studying towards an MSc in Cognitive and Decisions Sciences. This course covered a broad range of topics, ranging from computer science, AI and neuroscience, to psychology and philosophy. I was always tempted to see if I could use some of this research in our work.

You may ask what cognitive science has in common with online advertising. The answer is quite a lot, as some of our customers have already noticed. It can help to better understand the interests of online visitors and their decision processes, and can ultimately help website owners to serve more relevant web content and advertising. I decided to explore this question in more detail during my MSc thesis.

This project was motivated by a paper published by scientists from Microsoft Research (Yan et al., 2009) who found that behavioural targeting in search advertising could yield up to a 670% increase in the overall CTR (click-through ratio). We performed a systematic study of the clickstream logs of a commercial ad network and found that the overall CTR could be increased by as much as 909%.

This study considered the impact of behavioural targeting techniques on online display advertising. Specifically, we investigated whether simulating delivery of traffic to chosen clusters of users would increase the overall CTR of all ads. We examined the data using different evaluation metrics, such as user similarity, precision, recall and F-measure, then we used the t-test to confirm the significance of the results. The experimental design was implemented with the help of scalable data mining libraries, which allowed a successful analysis of the large body of data.

You may download the paper here (the source code is included): MSc Thesis – How much behavioural targeting can help online advertising

I used many data technologies in this project, starting with Hadoop and Hive and Amazon Cloud. All programming was done with Java and Python. The biggest challenge was posed by the amount of data which needed to be clustered, but thanks to Apache Mahout this project was finished in less than a month.

Abstract

Online advertising has exploded during the past few years; the current UK market (as of the middle of 2010) is evaluated at more than £3.5 billion. Such advertising grew dramatically — by about 2200% — during the 2000s. Behavioural targeting (BT) is largely regarded as one of the most effective techniques in optimizing online advertising. However, despite the impressive numbers involved in this industry, there are only a few academic studies performed on real world click-stream data (e.g. Yan, Liu, Wang, Zhang, Jiang & Chen 2009; Ratnaparkhi 2010; Chen, Pavlov, & Canny 2009). This may be linked to the extreme demands on system resources caused by the huge amount of advertising data available.

Yan et al. (2009) confirmed that BT could significantly increase the effectiveness of one specific type of online advertising (so-called search advertising). In this work we investigate whether techniques linked to BT may be beneficial to online display advertising. Using data from a major commercial ad network, we show that a simple BT technique (such as user clustering) could improve click-through ratio by more than 900%.

Furthermore, from a software engineering perspective, we provide support for using distributed open source technologies to tackle the complex analysis of advertising data.

References

• Yan , J., & Liu, N., & Wang, G., & Zhang, W., & Jiang, Y., & Chen, Z. (2009). How much can Behavioural Targeting Help Online Advertising? Proceedings of the 18th international conference on World Wide Web. Madrid, Spain.
• Ratnaparkhi, A. (2010). Finding predictive search queries for behavioral targeting. In ADKDD’10, The 4th International Workshop on Data Mining and Audience Intelligence for Advertising.
• Chen, Y., Pavlov, D., & Canny, J.F. (2009). Large-scale behavioral targeting. In KDD ’09: Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, 209-218.

DataMine Lab is a new kind of a software consulting company that works with large data projects. Traditionally to analyse big data, big investments were needed. Times have changed and thanks to cloud computing and mature open source technology we can do this now on a budget and in record time. We love open source software and use it to our clients’ advantage.

We would love to know your opinion about our website, please let us know what you think at info@dataminelab.com.