Wondering around

Testing javascript with Play Framework 2.0 and JsTestDriver

2012-02-16T00:12:00.000-08:00

At qwallet we're building an extensive javascript application backed up by a Scala on Play 2.0 backend. Since I'm using IntelliJ IDEA for the scala development, I'm using it for javascript as well. The nice bonus that comes with that is IntelliJ's integration with JsTestDriver.
While working on the javascript app I have it constantly tested on three captured browsers:

The test console shows the test results (super fast) and any console output that got fetched from the browser, e.g:

In the above test example you can see all tests are passing on firefox and chrome, but fail on safari. The sample has three test cases, the expanded one has two test methods. If there's an error, the stack trace is clickable and intellij will take you right to the line of the error. The tests are pretty fast, 6ms on chrome for this small subset means there's some room to grow.
The extra sweetness of Play2.0 is the auto compilation of javascript files. In dev mode the JsTestDriver is serving the library files and test cases to the browser and the application files are served through the Play2.0 application. Play will recompile the files as it detects change. It gives an extra coverage as in many cases the compiler catches things that I may have missed when writing the test.

On CI mode there is no need to pipe the javascript files through play as the files are compiled once. Not that its much faster this way, but it reduces the need of having play running in the background. As we're going to have qwallet operating in Continuous Deployment mode, testing all parts of the application is crucial. JsTestDriver bundled with the closure compiler provides a nice component in the testing strategy with its fast dev mode unit testing.

BTW, we're hiring. If you're interested, shoot us an email to jobs@qwallet.

Throw undeclared checked exception in Java, take II

2011-11-01T10:46:00.000-07:00

Wrapping is in many times a necessity as you may not be able to throw a declared exception for contract restrictions or want to reduce boiler plate code. Wrapping an exception with a RuntimeException is sub optimal, takes more resources and clutter the logs with unnecessary stack traces, makes you look for the root cause. Loved Alexey's idea about throwing undeclared checked exception in Java. Here is a small twist on that AnyThrow class: Now I can do So the compiler and the reader knows I'm breaking the flow with an exception. The stack trace does not have the AnyThrow in it and the "return null" there is only to trick the compiler.

Testing Guice can init servlets

2011-10-04T15:57:00.000-07:00

Sequel to the prev post, binding servlets is not enough. You must make sure that Guice can initiate them. For example, a common error is not binding servlets as singleton, there you get exceptions like

ServletException: Servlets must be bound as singletons. Key[type=my.servlet.FooServlet, annotation=[none]] was not bound in singleton scope.

Here's a way to test for all servlets in Guice context When calling init on the filter it would init all servlets in context.

Testing Guice Servlet bindings

2011-10-04T14:46:00.000-07:00

Evan if you work with Servlet (for the record, I hate j2ee) and Jersey you can still enjoy Guice.
Unfortunately its not that evident how to test that the ServletBinder is binding your servlets to the right place. Here is a small Assert class that does that.

Continuous Deployment at kaChing

2010-07-24T12:34:00.000-07:00

In my last visit to Israel I visited IBM Research, Outbrain and Answers.com. Over there we discussed kaChing's culture, working methodologies and tools. The talk focused on continuous deployment and lean startup and was based on Pascal's and David's posts at kaChing Eng Blog.
Here is the presentation, using the fantastic Prezi tool.

Continuous Deployment on Prezi

Scala on Eclipse without the plugin

2010-06-20T21:06:00.000-07:00

The Scala supported IDE is one of the few pain points of developers who want to start using Scala in their Java project. On existing long term project developed by a team its hard to step in and introduce a new language that is not supported by the existing IDE. On way to go about it is to hid the fact that you use Scala from the Java world by using one way dependency injection. Still, if you wish to truly absorb Scala into your existing java environment then you'll soon introduced cross language dependencies.

Most of our team is using Eclipse as the main IDE, its incrimental compilation in Java with its tight JUnit integration are great for fast TDD programming. Unfortunately the Eclipse Scala plugin is not there yet, it may hangs the IDE and messes up Java compilation - especially in large (more then 1000 source files) Java/Scala projects. Though the plugin is getting better over time some developers would find the plugin as a majore drag on their productivity.
For developers who do not write Scala at all or rather edit Scala with other editors, you can use this alternate path which lets them work on their Java or Scala code without messing with the plugin.

Compile Scala Eclipse project without Scala Plugin

1. Add a compilation script to your project.
The Following script is using the Fast Scala Compiler (fsc). The fsc is a compilation server which always run in the background, as in a warm scalac always ready to receive new work. Is will reduce compilation time dramatically.
The classpath for compilation is taken from the Eclipse project .classpath file. You may take the source directory from there as well if you wish (exercise to the reader).
The params are not passed to the fsc in the command line since in my project's case the line is too long for the OS to handle. The alternative is to put it into a file and let fsc handle it for you.

#!/bin/sh
# Create a classpath from the eclipse .classpath file
lib=`grep classpathentry .classpath | grep "kind=\"lib\"" | tr -d '\t' | sed 's/.*path=\"\(.*[^src]\.jar\)\".*/\1/' | grep -v classpathentry`
CLASSPATH=`echo ${lib} | sed 's/ /:/g' `  

# point SCALA_HOME to Scala home (might want to add it to your project as well)  
export SCALA_HOME=lib-tools/scala-2.8.0

# java opts for your compilation server
export JAVA_OPTS="-client -Xmx1024M -Xms256M -XX:PermSize=128m -Xss2M -XX:MaxPermSize=256m -Xverify:none -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled"

DEST=target/bin-scala
mkdir -p $DEST

SCALAC_PROPERTIES=scalac.properties
echo "-classpath $CLASSPATH:target/bin" > $SCALAC_PROPERTIES
echo "-d $DEST -deprecation" >> $SCALAC_PROPERTIES
find src srctest -name *.java -o -name *.scala >> scalac.properties

time $SCALA_HOME/bin/fsc @$SCALAC_PROPERTIES &

2. Eclipse classpath
Add the scalac destination directory ($DEST) to the .eclipse classpath

3. Add the fsc builder to your project
Open the project properties and add the script above to the list of builders.
Set the working directory to be your project root directory.

5. Set the builder properties
Run it in the background when needed:

Add Scala syntax highlighting editor for Eclipse

Even without the Scala plugin refactoring and debugging capabilities you can write some decent Scala code with the help of syntax highlighting for better readability. There are two options:

The color editor which use jEdit modes. Daniel Spiewak added a Scala mode to it, unfortunately the plugin doesn't seem to work with Eclipse 3.6.
The Colorer take5 project which works fine on my Eclipse on the Mac. It does not have a Scala mode yet but you can grab one at http://github.com/eishay/eclipsecolorer_scala

It gives you a lightweight way (with its pros and cons) to write Scala in Eclipse.

InfoQ talk "Absorbing Scala in the Java Ecosystem"

2010-05-28T00:08:00.000-07:00

About six months after, InfoQ released the video of the "Absorbing Scala in the Java Ecosystem" talk I gave at QCon San Francisco 2009.
The event was great though it feels funny looking at myself talking.

Findbugs, Hudson and Pizza Driven Development (PDD)

2010-04-15T20:54:00.000-07:00

Original post at the kaChing Eng Blog

As you may know, kaChing is an test driven engineering organization. Test driven is not an option, its a must. We move fast and push code to production few dozens of times a day in a five minutes release cycle, so we must have high confidence in our code.
In complex systems there is no end to testings, each test system is an another line of defense which eventually gets broken but the more you have, the less chances bugs will reach production. We do not have QA team and do not want to have one, the reasoning is that if a human is involved in testing then there is a higher chance of missing things and you simply can't test all the site dozens of times a day.

Lately we decided to add a yet another line of defense: Code Static Analysis (e.g. Findbugs, PMD and CPD). We decided to start with Findbugs which has a great ANT task and Hudson plugin (both of these we use). The problem with these tools is that they're producing tons of warnings and most organizations ignore them after since they're too noisy to deal with.

David V. from testdriven.com recommended: "Pizza Driven Development" (aka PDD) which works as following:

Step one: Order Pizza. Most engineers would commit on doing something if it will get them Pizza. For the minority that would not be seduced by Pizza, good old fashion violence would do.
Step two: Give each member of the team two cards. Go over the list of rules with the team and have them vote on them. Voting is done using the cards where:

No cards: I think the rule is stupid and we should filter it out in the findbugsExclude.xml
One card: The rule is important but not critical.
Two cards: The rule is super important and we should fix it right away.

The test sherif them creates three lists using the voting and the team works on fixing the "must fix" list which should have no more then few dozens of issues so it can be all fixed in couple of hours. Once you have that done the findbugs build is green and we're ready to go.

Next step is having hudson run findbugs on every post commit so the build is considered to be broken if a new findbugs issue is introduced. The engineer who introduced the issue must either filter the class from that rule in the xml file or fix the bug as a first priority. Since the engineers get a notification few minutes after the commit then they are probably still messing with the code and its easy for them to fix it on the spot.

In the next few weeks we are adding a new rule from the "not critical" list every few days. The goal is to have all the rules we think are important without the common "its to noisy, lets ignore it" approche. Only after we're done with that we're going to add the next static analysis tool to build. The good thing about these tools and hudson is that you can run them in parallel to the unit/integration tests, on another machine, so they won't slow down the overall release cycle.

Testing with Hibernate

2010-04-15T20:51:00.000-07:00

Original post at the kaChing Eng Blog

Some of the unit tests I wrote lately involved setting up a data setup in the DB and testing the code using and mutating it. To make it clear, the DB is an in memory DB so the setup is super fast without IO slowdowns. Its very easy to do the setup using hibernate but the problem comes when you set a large collection of objects and a "java.sql.BatchUpdateException: failed batch" is being thrown. The frustrating point there is that hibernate won't let you know what exactly went wrong, even if you set your logger to "trace" level.

In order to solve it you can add the following to the system properties of that test:

-Dhibernate.statement_cache.size=0 
-Dhibernate.jdbc.batch_size=0 
-Dhibernate.jdbc.use_scrollable_resultset=false 
-Dhibernate.bytecode.use_reflection_optimizer=false 
-Dhibernate.show_sql=true

It will execute the statements one by one and will give more precise details of what went wrong.
Note: you do NOT want to use these properties in production or in the continuance integration (CI) environment.

Parallelizing JUnit test runs

2010-02-20T19:46:00.000-08:00

Original post at the kaChing engineering blog.

Test runs should be as fast as possible in order to allow a lean development cycle. One of the applications is a Continuous Deployment (see Lean Startup).

Using strong multi-core machines to run tests is not enough since most unit tests are using a single thread per test. Apart of reducing IO to minimum, having tests running in parallel is required to squeeze the juice of of the machine. Additional value from Parallelizing JUnit test runs is ensuring that tests have no dependency between each other.

The way we implemented parallel test runs is creating an ANT target with a parallel task containing N JUnit tasks where N == number of cores. For example:

<parallel>
      <junit printsummary="yes" haltonfailure="yes" fork="true" maxmemory="${maxmemory}" showoutput="yes">
        <jvmarg value="-XX:MaxPermSize=${permMem}"/>
        <jvmarg value="-Xms${minMem}"/>
        <jvmarg value="-Xmx${maxmemory}"/>
        <classpath refid="classpath.test" />
        <formatter type="xml" usefile="true" />
        <test name="com.kaching.GroupedTests$GroupA" todir="${testTargetJunit}" unless="testcase"/>
      </junit>
      <junit printsummary="yes" haltonfailure="yes" fork="true" maxmemory="${maxmemory}" showoutput="yes">
        <jvmarg value="-XX:MaxPermSize=${permMem}"/>
        <jvmarg value="-Xms${minMem}"/>
        <jvmarg value="-Xmx${maxmemory}"/>
        <classpath refid="classpath.test" />
        <formatter type="xml" usefile="true" />
        <test name="com.kaching.GroupedTests$GroupB" todir="${testTargetJunit}" unless="testcase"/>
      </junit>
  ...
     </parallel>

The GroupedTests$GroupX are classes extending TestCase with a public static Test suite(). The suite() method creates a JUnit test suite on the fly by loading all the tests in scope and filtering them out. If for example there are four cores, therefore you would like to have a group of four suites each running fourth of the tests. The suits are using a java.util.Random seeded with the commit revision number. Using the Random object we decide on placing test cases in suits.

random.nextInt(numOfTestSuites) == testSuiteId

Therefore a testcase goes to a single suite and they are evenly distributed between suites. Randomness in assigning test cases to suites (and therefore to processes) gives us some reassurance that there are no dependencies between tests.

As a result we have the tests running about twice as fast as they did before, in the range of 2 min, 20 sec for about 4.6k tests for one of our components, including code fetch from source repository, clean, build, and test suite setup time. It gives us a nice 100% test machine utilization and faster commit to production deployment cycle (about four minutes).

Benchmarking Voldemort on SSD with production data

2010-02-01T22:24:00.000-08:00

Its not a secret kaChing is using Voldemort in production. As our data set and grows larger we wish to figure out what is the best bang for the buck in terms of hardware investment. Would it be migrating our 32bit machines to 64bit ones and have more memory available for them or using SSD? There are good reasons why one may be better then the other but the only way to know is to test. Yes, we believe that Test Driven Development (TDD) applies to building system as well as coding.

The time John invested the time in creating the tool (written in Scala btw) to parse the logs and get the graphs had quickly payed itself. In John's post Voldemort in the Wild he wrote about the initial settings he created to test the changes before we actually did them and some of the initial result. The results are unique to our own system, which is the best thing about them. You may get different results for different load, usage patterns, data size, cluster setup.

There are more tunings we played with, waiting to John's next post.

Complement TDD with MDA

2010-01-09T23:49:00.000-08:00

Original post at the kaChing Eng Blog.

Test Driven Development (aka TDD) is on the rise. Good developers understand that code with no proper testing is dead code. You can't trust it to do what you want and its hard to change.
I'm a strong believer in Dijkstra's observation that "Program testing can be a very effective way to show the presence of bugs, but it is hopelessly inadequate for showing their absence."

Dijkstra's statement doesn't contradict TDD. The test is testing a limited state machine. We do hope will cover the bloody battlefield of production confronting live data from users but if when we find the users did something unexpected which broke our software, we add a test emulating the users behavior and fix the problem.

Introducing Monitoring Driven Architecture (aka MDA)!
MDA is a second line of defense for TDD. MDA means that you bake monitoring into your architecture. Once you have MDA and the software is written in a monitorable (it is a word) way, you can have a faster detection of problems and auto roll back of faulty code. On the other hand, it is not uncommon that a small number of users suffer from a problem which manifests itself in some NPE thrown in one of the logs once in a blue moon and the operations team finds about it after a long while.

This is why I'm so excited about John's new Flexible Log Monitoring with Scribe, Esper, and Nagios deployment. It means that when we do find a problem we'll of course fix it but in addition make sure we express it in the logs and have our monitoring tools pick it up and send alerts about its existent without counting on anyone to manually look at the logs.

Outbrain's Password Recovery Mail

2010-01-06T22:21:00.000-08:00

I forgot my Outbrain password and got this nice recovery email in return:

From: Outbrain CEO <****@outbrain.com>
Hi eishay,

Click on the following link to reset your password:
http://www.outbrain.com/ResetPassword?ticket=****
You will be prompted to choose a new password for your account.

Feel free to drop me a note with any questions you have.

Thanks,
Yaron Galai,
outbrain CEO

First time I notice a "system" email coming from a human that you can actually reply to, nevertheless the CTO of the company. Its an interesting concept, especially since I'm not a paying costumer. Yaron explained their take on it in this tweet:

@eishay Cool!... we have a rule here @Outbrain - all system emails must go out from an email of a human being. I hate all the info@ BS...

I found the approach very appealing, but can it scale? What is the max # of users it can support?
I assume its about the type of uses as well.

Reflection on 2009

2010-01-06T21:39:00.000-08:00

This post is only to myself. Nothing interesting, just few personal notes.

The last year was very interesting!
Working in both LinkedIn and kaChing was/is a great experience. These are two fantastic companies, with very bright future, leading in their own fields and with amazingly talented engineers.

The SiliconValley CodeCamp '09 was a lot of fun. Had a full room and got high speaker evaluation and some candidates to kaChine (yes, we're still hiring).

The other talk was at QCon which was exciting. There where so many people that we had to move to a larger room that was packed as well :-)
Got some nice feedback via twitter twit twit twit twit twit twit twit twit twit.

It was great to do three sessions with the Reversim podcast (in Hebrew) about Scala, scalability and startups.

The open source serialization comparison project had significantly grown, it now has a dozen committers who contributed something in some point in time and still helping to update the project.
Overall I had lots of fun, and feel like 2010 is going to be even better !

Subversion Backup

2010-01-02T20:45:00.000-08:00

Posted on the kaChing Eng Blog.

Yes, we're using Subversion. I know that distributed version control systems (e.g. Git) are cool and we might get there sometime, but for misc reasons we're still using SVN. For the records, some of us are using GIT-SVN and we're working and releasing from trunk (part of a the lean startup methodology) so the branching merging is less of an issue.
I did some work to migrate our repository and spent some time to setup our SVN repo. Here are some bits and pieces I collected from scattered sites or made up myself to facilitate the SVN backup. Hope it will help anyone starting from scratch.

For the backup I'm using the great svnbackup script. Here are parts of our script (launched by crontab):

now=$(date +%F)
svnbackup.sh --out-dir $OUT_DIR --file-name $FILE_NAME -v $REPO_LOCATION
RETVAL=$?
if [ $RETVAL -ne 0 ]; then
    mail -s "ERROR: SVN backup on $now" $KACHING_OPS
    exit 1
fi

Then the script sync's up the backup directory with S3 and verifies that the content of the last_saved file matches the last revision from SVN which it gets using

last_revision=$(svn -q --limit=1 log https://chb2.kcprod.info:4242/svn/backend | head -2 | tail -1 | cut -c 2-6)

Backup is not enough, we must constantly test that when the time comes we'll be able to use it. Therefor we added a script, triggered by Nagios, to run on another machine and try to do a full repo rebuild from scratch.
The first thing the script is doing is to brute force clean up the repo:

rm -rf $SVN_REPO
svnadmin create $SVN_REPO

Then do a S3 sync to get all the backup files and load the files into the svn repo in the right order:

for file in $(ls $SVN_BACKUP_FILES_DIR/*.bzip2 | sort -t '-' -k 4 -n)
do 
  bzip2 -dc $file | svnadmin load $SVN_REPO
done

Next step is getting few revisions and checking that their attributes (e.g. comments) match in both live and backup test repos.

p.s.
Just because I'm paranoid we're also have an svn sync on an SVN slave server our second data center where every commit is backed-up on the fly and some of our systems (e.g. WebSVN) are reading from it.

Attaching a Java debugger to the Scala REPL

2009-12-29T00:59:00.000-08:00

Originally posted on the kaChing Eng Blog.

I'm using the Scala REPL to play around with java libraries and check their runtime behaviors. One of the things I'm using it for is to check how Voldemort's client is behaving in different setups. For one of the checks I wanted to trace the client threads with an IDE debugger.
To attach a debugger to the Scala REPL all you need to do is to export the debugger values into JAVA_OPTS:

export JAVA_OPTS='-Xdebug -Xrunjdwp:transport=dt_socket,address=1773,server=y,suspend=n'

Run your scala REPL and attach your debugger to port 1773. Done.

(*) Tested with scala 2.7

Baking availability SLA into the code

2009-12-09T00:53:00.000-08:00

This is a copy of the post I made on the kaChing engineering blog.

Availability and Partition Tolerance are essential for many distributed systems. A simple (though not comprehensive) way to measure both is using response time SLAs between services as implied from Jeff Darcy's observation:

Lynch (referring to the 2002 SIGACT paper) also makes the point that unbounded delay is indistinguishable from failure. Time is therefore an essential component of these definitions (a point made even more explicitly in the Dynamo paper).

At kaChing we think that SLAs is something that should be baked into the code so the developer will have to think of while creating the service contact. For that reason we created a @ResponseTime annotation for out internal services:

@Retention(RUNTIME)
@Target({ TYPE })
public @interface ResponseTime {
  long warn();
  long error();
  TimeUnit unit() default MILLISECONDS;
}

A typical service query for online requests is annotated with

@ResponseTime(warn = 100, error = 200)

Where the time depends on the services constraints, access to resources for example. A ping query for example has a

@ResponseTime(warn = 2, error = 4)

and an offline analytics call may take hours

@ResponseTime(warn = 10, error = 15, unit = HOURS)

Nevertheless, every request should have an SLA and the developer must think of it when writing the implementation.
Once we have this tool the SLA of subsequent services (B, C & D) a service (A) needs to call can statically be computed to verify that no path in the tree of subsequent services calls exceeds the root service (A) SLA. In other other words, for service queries A, B, C & D. If A calls (B and B calls C) and in parallel A calls D then we should have SLA(A) > max (SLA(B) + SLA(C), SLA (D)).
kaChing's service query container sample the time it takes for every call using perf4j and logs times and SLA violations to local disk. If the query's time exceeds the warning threshold it will be logged accordingly, but if the error threshold is broken then the container will try to terminate the query. Terminating the query is a bit harsh, but the since our client timeout is using the SLA as well then most chances are that the client gave up on the query and either retried (service queries are Idempotent) or aborted. Another reason to shoot down a runaway query that exceeds the SLA error time is that it may be locking or consuming resources from other queries and slowing the whole system down.
The perf4j messages are also piped through a log4j socket appender (async buffers in each side of the pipe) to a central hub. The hub then does statistics on the aggregated times, it loads the SLAs of the queries and checks that the cluster is not violating its SLA. The central hub can then send a daily report on SLAs and real time alerts pinpointing a slower then expected service. Keeping the reports and comparing them to historical reports are helping to see an improvement or regression in every part of the site.

This monitoring technique is only a small portion of the automated system operation a small startup must have in order to stay flexible. Stay tuned for more monitoring automation posts.

Looking for a passionate Systems Engineer !

2009-10-10T13:08:00.000-07:00

Not a typical post, but I might as well use this platform to promote it. Come join the best startup ever !

kaChing is an investing talent marketplace where individual investors gain access to the best investing talent on the Web. An SEC registered investment adviser, kaChing enables customers to mirror automatically the trades of the "kaChing Geniuses" who have an Investing IQ of 140 or greater. The company's investors include Marc Andreessen, Jeff Jordan, CEO of Open Table (OPEN) and former president of PayPal, and retired partners from Benchmark Capital and Kleiner Perkins Caufield & Byers.

Join a stellar engineering team whose team members worked on massive backend infrastructure, Gmail's high performance frontend, the Native Scala Compiler with Prof. Odersky and more. We are test-driven and have a continuous integration process that lets us achieve a 5 minute build and release cycle.

Job description:

In charge of the backend of a revolutionary web site doing large scale high availability deployments and configuration management using open source software.
Keeping the system having a fully automated release cycle, continues testing and monitoring.
Build tools for monitoring, security, systems management and deployment if the available ones are not enough.
Help engineers architect the next levels of the system and making sure it stays agile and easily absorb high capacity.

Qualifications

Extremely smart and self motivated
Passion for test-driven development and automated functional QA
Live and breathe performance
Thinks out of the conventional box of IT management
Experience with databases, preferable MySQL
Good knowledge about monitoring and deployment tools
Good knowledge about network and distributed computing
Expert in at least one scripting language and know at least one more
Ability to work independently
A BS in computer science or related fields

I'll soon add a link to the official page.

Speeding up with Voldemort and Protobuf

2009-10-07T22:35:00.000-07:00

In order to supply some of the analytics behind kaChing we have a some nice number crunching processes working over piles of financial data. The end results are displayed on the site (e.g. the find investors page).

The post is about one of such services which for the sake of this post I'll refer to as the SuperCruncher. In the first iteration we grabbed the data straight from the DB into the SuperCruncher. Needless to say, relational databases are not handling stress nicely and a pattern of fast iteration over all of the DB kills it very fast. Results of first iteration: 25 hours of computing.

Since we couldn't manage to squeeze more hours into the day we run a second iteration.
Voldemort & Protobuf for the rescue!
Though we have piles of data to compute, some of the data does not change though we do need to read it each time. We shoved the data into a Protobuf data structure which made the binary size considerably smaller, and pushed the protobuf into Voldemort. We used the Voldamort ProtoBufSerializer which provides an extremely simple way to store and use protobufs.
So essentially Voldemort in this case is used as a persistent cache to store normalized data. When reading the date the SuperCruncher first check it it exist in the naturally sharded Voldemort cluster and gets the delta from the DB.
Using Voldemort and Protobuf the I/O problem vanished, the SuperCruncher became four times faster (!) and bounced the performance ball back to the CPU. Cutting down CPU time is usually easier then cutting on I/O and indeed we managed to make the running time considerably faster in later iterations (hint: Joda Time & GC).

Shameless promo: kaChing is hiring!
We are looking for an world class Systems Engineer.

Our release cycle is 5 minute long, we release from the branch and have 100% tests passing on all time therefore we do not need a typical sysops person. We are looking for an excellent engineer to run, architect and automate the system.

Scala @ Silicon Valley Code Camp

2009-10-03T21:23:00.000-07:00

Had a great day at the Silicon Valley Code Camp. Gave couple of talks about Scala and met lots of interesting people. Here are the slides for the talks I gave.

Absorbing Scala Into Java Ecosystem

A First Look at Scala on Google App Engine

kaChing on Yahoo! Homepage

2009-10-02T17:21:00.000-07:00

println("Hello kaChing!")

Off to a new adventure

2009-09-20T20:25:00.000-07:00

Had a great couple of years at LinkedIn and now I'm off to a new adventure in the valley of silicon. This time its a great web startup called KaChing which I'm sure you'll hear more about in the next few months.

So I might be a bit quiet for a while working with great technology but if you happen to be around come and meet me at the Silicon Valley Code Camp!
I'll go back now to work on the Voldamort & Protobuf integration.

Protobuf JSON Serializer is great, but a bit slow

2009-08-10T23:16:00.000-07:00

Run a small benchmark with the fresh new protobuf-java-format. The ProtobufJsonSerializer code is nice and trivial, greatly increasing the portability of protobuf and its capabilities to communicate with services that can't use the protobuf native parsing libraries.
Alas, it also comes with some price. Unfortunately, this new (still early) version of the library is very slow compared to other serialization libraries. See the full benchmarking wiki page for more info.

               Obj create, Serialization, Deserializtn,  Total Time, Size
protobuf      , 421.14250,    3738.75000,   2471.25000,  6631.14250, 217
json (jackson), 235.45000,    3196.50000,   4934.25000,  8366.20000, 304
JsonMarshaller, 237.79000,   18107.00000,  24715.75000, 43060.54000, 298
protobuf-json , 431.94000,   19933.25000, 102577.75000,122942.94000, 389

EasyMock IArgumentMatcher with Scala Howto

2009-08-10T21:53:00.000-07:00

Its pretty trivial if you're a Scala savvy, but for the learners it might be a bit tricky. First thing, you should probably first get familiar with the java way of using IArgumentMatcher. To use it you should have a static method that calls EasyMock.reportMatcher with your implementation of IArgumentMatcher. To archive that you must create the method in an object. Its a workaround for not having static methods in Scala. Then implement the matcher, pattern matching is lots of help here, and remember that in Scala AnyRef is equivalent to Java's Object.

object TestMyClazz {
  /**
   * for EZMock argument matcher
   */
  def eqMyClazz(obj: MyClazz) : MyClazz = {
    EasyMock.reportMatcher(new MyClazzMatcher(obj))
    null
  }
  
  class MyClazzMatcher(obj: MyClazz) extends IArgumentMatcher {
    def matches(actual: AnyRef): Boolean = actual match {
      case actual: MyClazz  => //check things...
      case null => //is null ok?
      case _ => false
    }
    def appendTo(buffer: StringBuffer) : Unit = buffer append "some text..."
  }
}

Done, now you just need to call eqMyClazz() with your expected value.

someObj.useMyClazz(eqMyClazz(expectedObjectOfMyClazz))

Speaking at the QCon San Francisco '09 "Absorbing Scala"

2009-08-10T21:23:00.000-07:00

Funny thing happened to me,
I checked out the new Google search engine on their sandbox url.
On of the things I searched for is my own name (very modest of me, I know...). I was a bit surprised to see I'm going to talk at QCon San Francisco 2009 about "Absorbing Scala". I remember discussing it a long while ago but it kinda trailed off and I forgot about it. They did got my current title wrong, not that it matters too much.

Anyway, its a yet another practical use of google search :-)

Hope there'll be more Scala nuts in the crowd to quiet down the other Language Zealot (by the way, I do love the groovy).

See you there!