Tech Voice

What to expect while Migrating to Spring 3

2010-03-24T18:40:00.006-04:00

Recently, I migrated one of my applications to Spring 3. The project was previously using Spring 2.0.8. Spring 2 was released in 2006 so it was high time that we migrated to a later version. That and the fact that there are plenty of reasons to migrate to Spring 3. Although Spring 3 was released in December 2009, the milestone and other pre-releases of Spring 3 have been around and being actively used by the Spring community for more than a year.

The application that we wanted to migrate was fairly large with 150k lines of code (tests not included). Since this was a heavily used application, we couldn't afford to have many regression errors. Luckily, we had about 6500 unit tests and about 1700 functional tests running constantly in a continuous integration system. Having so many tests execute every time we changed something basically ensured that we don't have too many regression errors. It also considerably boosts the confidence level when we want to make a change to the application.

The first thing you will notice when upgrading to Spring 3 from Spring 2 is that it is packaged in a bunch of jars (or maven dependencies). This changed in Spring 2.5 and is generally regarded a good move although it makes upgrading a bit of a pain (even with maven). The good part is, the jar/dependency names are pretty self descriptive. It'll take a couple of hit and miss tries to get all the dependencies into the classpath.

Now, lets move on to using Spring 3.
API Compatibility:
Spring 3 is generally API compatible with the older versions. The only major difference you might find is your compiler might warn about generics. Older classes like JdbcTemplate got a generics refit to their APIs.
For example: The return values of methods like queryForList got changed to give a more meaningful signature.
Before:
List<Map> userDetails = jdbcTemplate.queryForList

After:
List<Map<String, Object>> userDetails = jdbcTemplate.queryForList

Apart from these, there are also a few minor API changes where
Example: AbstractBeanFactory.getMergedBeanDefinition(..) returns BeanDefinition instead of RootBeanDefinition.
Most of these API changes are minor and should be easy to resolve.

LifeCycle:
If you are using the any of these beans and you had any of these declared as lazy-init, you may be in for a small surprise.
DefaultMessageListenerContainer
GenericMessageEndpointManager
JmsMessageEndpointManager
SchedulerFactoryBean
SimpleMessageListenerContainer

These components extend a new SmartLifecycle interface and are eligible for autostartup. Previous versions of Spring lacked a nice way to start services in a given order. The depends-on attribute doesn't really help if there are a lot of components to be started, especially if they are spread a bunch of xml files. The Lifecycle interface provided a crude way of achieving ordered bean startup albeit with some manual coding. The Phase, SmartLifecycle interfaces take it a step further and give an ability to automatically start your services when Spring starts up. More on this in a later blog post.

A SmartLifeCycle bean can declare that it needs to be autoStarted (by returning true in the isAutoStartup() method). When the ApplicationContext starts, it looks for any SmartLifeCycle beans that are present in the context, and initializes all of them, even if they are marked as lazy. It then selectively starts the ones that are marked to be autoStarted. This means that if any of these beans are previously declared to be lazy-init, they will be initialized after application context has initialized. This is not really a bug as the isAutoStartup() method is available to Spring only after the bean has been initialized. The consolation if any is that these beans are initialized in a later phase, during the startup of SmartLifecycle beans.

In our application, we were using a few of these beans. While testing, we were counting on these beans not getting initialized due to the lazy-init attribute. But, with Spring 3 these beans were all getting initialized, which was undesirable. The correct resolution was to move these beans out of the bean context into a separate context xml file that is never loaded during testing.

We've been running Spring 3 in production for a little over 2 weeks now, with no issues. Overall, the Spring 3 upgrade was pretty painless and straightforward. I think the Spring team did a good job of trying to preserve API compatibility and to make sure that there were no regression errors in the APIs.

If you are using an older version of Spring, it is time to move to Spring 3.

Ofcourse, you should follow me on twitter.

The first Date sucked. Can a second Date fix it? JSR 310

2010-03-05T20:01:00.005-05:00

Well technically, this is the third Date API. But, it forms a much better title!

JSR 310 is the new Date Time API specification. I'm overjoyed by the fact that Java will finally get the long overdue revamp to the infamous java.util.Date and java.util.Calendar classes. It is nice to see that JSR 310 builds on Joda Time API unlike some other APIs that didn't follow this approach (cough: java.util.logging).

The JSR 310 is in early draft review. And, they are looking for feedback about the API. So, I decided to take a crack at it.

There are a lot of things done right with this API. One of the nicest things is not having to mess around with the pesky Calendar.add(int,int) method. Among a lot of other things, JSR 310 introduces three new classes for date/time handling: LocalDate, LocalTime, LocalDateTime.

Now lets consider what could be better:

You got the money, now give me my constructors:
One thing I don't like is the absence of usable constructors in the three classes LocalDate, LocalTime, LocalDateTime. There are static factory methods that you can use to create the objects but having constructors for the simple usecases would be just nice.

Since LocalDate, LocalTime and LocalDateTime have similar interfaces, I'm going to pick on only one of them. The one that will take the beating is: LocalDate :-)

Compare the following:
Joda Time:
LocalDate date = new LocalDate(2010, 2, 20);

JSR 310:
LocalDate date = LocalDate.of(2010, 2, 20)
This may not be such a big deal to many folks, but I don't see why the constructors shouldn't be exposed especially given that this class is final!

Keep it simple 1: Printing Dates
Joda Time:
LocalDate date = ...
String dateString = date.toString("yyyy/MM/dd");

JSR 310:
LocalDate date = ...
DateTimeFormatter formatter = new DateTimeFormatterBuilder().appendPattern("yyyy/MM/dd").toFormatter()
String dateString = formatter.print(date);

Looks like we could use an overloaded method in the LocalDate class that works like Joda Time. I can imagine that it won't consider all the cases that DateTimeFormatterBuilder handles. But, it can at least address the most general usecase. I could be wrong, I haven't had a chance to use the API much. If so, please feel free to comment and let me know.

Keep it simple 2: Parsing Dates
There is the static method parse in LocalDate
LocalDate date = LocalDate.parse("2010-2-20");
The format that you have to pass it is fixed. For the same reasons as above, it'll be nice to see an overloaded method that takes a string format.

DateAdjusters is awesome:
Example:
LocalDate friday = ...
LocalDate monday = DateAdjusters.nextNonWeekendDay().adjustDate(friday);

Consider a case where we want to obtain the next valid working/custom day instead of the next non weekend day. DateAdjuster is an interface, we could write a custom implementation, but providing a method where developers can hook into the DateAdjusters class would be great.

LocalDate nextWorkingDate = DateAdjusters.nextWorkingDay(new WorkingDayDeterminer() {
public boolean isWorkingDay(LocalDate date) {
return ...;
}
}).adjustDate(friday);

So lets start using it:
Not so fast there. We typically use Date/Time with APIs like JDBC (does anyone use straight JDBC anymore?), Quartz etc. Unless these APIs upgrade to using JSR 310 classes, we won't reap full benefits of using these new classes.

Also, the original Date and Calendar classes are the IE6 of the Java API now. They must die! Or at least be deprecated, so the adoption for these new classes increases.

Finally, what is with the javax.time package name? Not that it bothers me too much, but java.time has an unmatched sex appeal to it.

Ofcourse, you should follow me on twitter.

Live Google IO notes

2009-05-27T16:07:00.006-04:00

Effective GWT: Developing a complex, high-performance app with Google Web Toolkit

Effective GWT: Developing a complex, high-performance app with Google Web Toolkit
Lombardi Software - Blueprint
GWT What and Why

Generates optimized javascript (like escape analysis etc)

High Fidelity Mockup

Done in photoshop
More expensive
Finalize the icons and colors etc

Going to code

Be involved in the design
You need to know css and HTML DOM
What is the appropriate DOM structure
How to create and manipulate GWT

Design

Design outer layer with divs
Faster way is to do html panel and divs ? (what does that mean?)
DOM structure is created by GWT decorator panel
And, you can apply css on them

Handling Window Resizing

Goal is to handle browser window resizing
Static HTML you're limited to what you can achieve in css
Listen to ResizeEvent from window and propagate sizes down to children
Because they only do fixed sized row, they can do background images to do styling in table rows

Animation

Not all browsers do CSS3 ?
Helps users understand the behavior of application (provides visual feedback)
Done all in java in GWT

Original Implementation

Iterate through your objects, create widgets and add it to containers
Javascript - object creation and GC is expensive especially in IE6

New Implementation

Generate raw HTML in Javascript
Use flyweight pattern for event handling
They create html inside java (javascript) and do a DOM.setInnerHTML()

Event Handling
When All Else Fails

They dual compile code to Java and Javascript
If they find that the browser's javascript engine is slow, they render it on the server and sent to the client
So based on performance, they can dynamically move rendering between server and client

Compiling GWT code is slow

By default, GWT compiles code to 5 different browsers
You can tell GWT to compile code only for a single browser - locale, this speeds up development time
Well, you can run hosted mode and that never compiles :-) or use GWT 2.0 it never compiles and supports out of process hosted mode !
Instead of doing DOM manipulation over objects like Element.getStyle().setProperty('css property'), put that property in a css file
Checkout Episodes plugin from the creator of YSlow. It sends client performance numbers back to server.

This presentation is more of a war story. It deals with the Blueprint product. Because most of their clients run it in IE6, Lombadri had to go through extra steps to optimize their application to rely less on IE6's javascript engine. These techniques also apply when you have a really rich GWT application.

Transactions Across Datacenters (and Other Weekend Projects)

Consistency

Talked about Weak Consistency, Eventual Consistency (thanks to Werner for making this popular), Strong consistency (AppEngine datastore, File systems, RDBMSes, Azure tables)

Transactions
Why across datacenters?

Catastrophic failures, expected failures, routing maintenance, geo locality (CDN, edge caching etc)
Basically vertically partitioning your application
Packet roundtrip from west to east coast is 30ms

Why not across datacenters

Within a datacenter, it costs much lesser to communicate, low latency (1ms within rack, 1-5 ms across)
Outside datacenter

Expensive
High latency

Multihoming

????
As soon as you write across multiple locations, you will have consistency problems
Realtime writes is always the hardest
Don't do it
A datacenter in silicon valley went down and twitter and friendfeed went down for more than 2 hrs. Both did not have multihoming

Option 2:

Better but not ideal

Have multiple datacenters, have primary and secondary
Mediocre at catastrophic failure
window of lost data because of asynchronous replication

Examples:

Amazon Web Services
Banks, Brokerages etc

Depending on systems, all your slaves can serve reads

Option 3: True Multihoming

Simultaneous writes in different data centers
Two way: hard
NASDAQ does 2 datacenters and does 2phase commit across them for transactions
Expensive and definitely slower

Techniques and Tradeoffs

Backups

Make a copy
Dog Fooding - they make other teams use their internal systems so they get to iterate and then release the API

Maser Slave replication

Usually asynchronous

Good for throughput, latency

Most RDBMSs do binary log based replication
AppEngine also follows this model.
AppEngine write is much slower than a relational db
But, it is geared for read more than write

Multi Master Replication

Support writes at multiple locations and then merge them
Asynchronous, eventual consistency (Amazon's shopping cart service does this)
You cannot rely on a global clock
Because of this, you cannot do global transactions
Another way of thinking about this is this is like mutlithreading without locks

Two Phase Commit

Heavyweight, synchronous, high latency
Semi distributed as there is a coordinator

Paxos

Fully distributed consensus protocol
No single master like 2PC
Still has longer latency
Gives a better throughput than 2PC

Paxos for the Datastore

Closer datacenter? not really because you are doing two round trips
Same datacenter? no
Opt In...

Paxos for AppEngine

They use that to coordinate when moving between datacenters
Use a lock server
Managing memcache

Conclusion

No silver bullet
Embracing tradeoffs
Consistency is app driven, the platform cannot make that choice.
AppEngine is going to support options in consistency models in future (Nice)

Building Scalable Complex apps on AppEngine:

List Property

Property has multiple values
Maintains it's order
Queried with an equals filter
Densely pack information instead of denormalizing it
Cut across all data and query on one of the values in the list property
select * from FavoriteColors where color = 'yellow' where color is a list property
Saves space to use list property
Uses more CPU to serialize and deserialize the list property
Never have composite index between two list properties because it creates a cartesian product index

Concrete Example: Microblogging

Fanout of messages can be inefficient in terms of space
Message sending by reference
You would use list properties instead of joins
select * from messages where receiver = 'user'

Problem with List Property

selects load all of the list properties

Relational Index Entity

Split the message into two entities (message index and message)
We put them into same entity group and make message index a child of the message
There is a key only query it lets you fetch just the fetch
Reads are 10 times faster and cheaper than with just plain list properties

Merge Join

AppEngine supports self joins
Data mining like operations
Don't have to build indexes in advance before this query
Can be used to test set membership

How does Merge Join work?

Because they don't have histograms (RDBMSs use histograms to make a query plan)
They store all property indexes in sorted order
Uses zigzag algorithm
If we are using 2 filters, it scans the first property to find a match, then moves the second one to find a match. Then, if the keys don't match, it moves the first one until both the property and key match
select * from animal where legs = 4 and type = 'cow'
Scales with number of filters
Can't apply sort orders - must sort in memory

Keynote 2

New Google Product Google Wave

Platform
Product
Protocol

A wave is a conversation between multiple people.
Wave can be viewed has an enhanced twitter or a hybrid (e-mail, IM, and word document)

Comment Support - Allows for inline comments or your typical comments at the end of the wave/posting.
Real-time updates from others - The document changes real-time while others are updating it.
Spell Check - A very sweet inline real-time spell checking and takes the word context into account. i.e., "Can I have some been soup" and it offered the follow "Can I have some bean soup"
Play back changes - You can play back the changes in the conversation to see what was done and in what order. Very much like playing back a video.
Private replies - Supports for private conversations between users hidden from others on the wave.
Drag and Drop - Wave supports D&D from iPhoto.
Uses Google Contacts - Integrates with the your GMail and GTalk contacts
Wave cloning - Wave allows for cloning of an existing Wave to create a new wave. Reminds me of Git :)

When this occurs, all subscribers or people in the wave are notified.

Inline editing - Supports inline editing from Wave and external websites that are using the Wave plug-ins.

When changes are made, the document is marked up to reflect where the changes were done, ONLY from the last time *you* viewed it.

Collaboration Editing - Awesome support for changing content in the same document in the same area and changes are reflected in different colors.
Open Social Integration

Any open social app can live inside wave

Developer API -

A demo was given of a custom widget that allows a user to vote "Yes, Maybe, No"
Sudoku Widget that allows multiple players to play with one another.
Chess Widget that allows others to play one another and uses the playback feature. Nice integration.
Google Maps Widget that shows all other users where you're looking. Also, draw regions, add pins and more! Very sweet!!

Search - You can search your contacts or use their built in Google search to search the web and actually use the results in the document. i.e., search for an image and select it to embed it into the document.
Multiple languages - Supports multiple languages.

Real-time language translation using a program called "Rosy". This was very sweet!

Polls - A nice little extension allows for creating polls.
System Federation between Wave 'systems'

Wave systems can collaborate between one another.
Private Waves between people within the same 'wave system' are never sent to other wave servers in the federation.

Forms are native to Wave
They are going to Open Source the 'lion' share of the code.
Written in GWT and HTML5
Demos

There was a demo of dragging and dropping a file into the browser to create an attachment. This is not supported in HTML5 yet, it is a prototype.
A nice demo of how the API is used to write an external application like a blog. Very cool demo.
Orkut demo with using their embedded API.
Nice twitter integration that signs in to twitter and actually will post tweets.
Very sweet code.google.com integration tool.

All attendees at Google I/O will obtain an account to use Google Wave before it's released. Hence, come see Google IO. Did I mention we also got a free Cell phone with a full month of unlimited service?
Comes with a developers API (We're talking about Google, it's expected! :)
Minor bug occurred when doing the demo. Hey, we're talking about live demos (turned out to be a wrong configured browser proxy).
Google Wave is great for team collaboration by adding inline comments, embed images, viewing changes, live changes, remove the conversation nose and release a final product and more.
Open Social Integration

Any open social app can live inside wave.

Website URLs

http://wave.google.com - Main website for Wave
http://code.google.com/apis/wave - API website
http://waveprotocol.org - The protocol website

Offline Processing on App Engine: A Look Ahead

By: Brett
Live Notes by @dushyanth

Motivation

AppEngine is great for request based database backed applications
Cron is good for periodic jobs, but not good enough

Problems with Polling

Wasted work as it is not event driven
Workers stay resident when there is no work wasting resources
Fixed number of workers. Or admins must manually add workers
Limited amount of optimization possible

Long lived hanging connectons

Existing task queue like systems

MQ, Amazon SQS, Azure Queues, Starling (getting popular these days)

Task Queue API

Part of AppEngine Labs (API may change until it graduates from Labs)
Asynchronous execution for a first in first out queue.
If execution fails, work will be retried until successful
Tasks are light weight to store. They are 3 times faster than storing in the datastore.
Tasks are scalable. The tasks can be started across a lot of machines.
Implements queueing. NOT pub-sub
Goals: High throughput, maximizing data throughput
Pushes tasks to the app. No polling
Uses Web hooks (It is a RESTful push-based interface for doing work)
Task is submitted as a web hook. If you get a 200 back, it succeeds.
Essentially combines queuing over REST.
Integrated into admin console as normal requests
Supports config driven throttling

Can be used to Prevent web services (external) from getting overloaded
Stay inside budget per hour etc

How task Queue Works

Tasks enqueue in a queue
Queue Moderator pulls from the head of the queue
It submits the task to the workers. Queue Moderator has capability to create new workers (threads).
Max number of threads depends on throughput
When a task is submitted, it could be running even before the enqueue request API call returns :-)

EdgeCases

Tasks have to be idempotent
Possible for a task to spuriously run twice even without failures.
You could use memcache or database to avoid it running twice, but that responsibility is on the developer

Working with TaskQueues

Each task added to a single queue
You can create multiple queues per application

Working with ETA (Estimated time of Arrival)

How long until the task is executed
Different than "visibility timeouts" in other systems

Working with tasks: Names

Tasks can be named. If a task is not named, it is auto generated

Prevents tasks from accidentally being submitted multiple times

Concrete Example: Write behind cache

Minimizes writes with repeated cache flushing

Write new data to cache
Periodically read cache and persist to disk

To implement, user submits data to cache and a task to task queue
When the task queue is processed, task is dispatched and the task does a periodic read from the cache and writes to the datastore. Essentially using the TaskQueue as an executor

Python only at first. Java comes next
Java support in the works

Webhooks, JMS

The Future

Batch Processing

Task Queue is good for small daasets (<100k>
More tools needed for parallelization

Map Reduce in future

Eventually
Want it to work with small and large (Terabyte scale) datasets

The Softer Side of Schemas - Mapping Java Persistence Standards to the Google App Engine Datastore

Live Notes by @dushyanth

Datastore is

Transactional
Natively Partitioned - developer does not have to worry about scaling
Hierarchical - every entity can have a notion of parent
Schemaless - no restricted structure
Based on BigTable
Not a relational database
Not a SQL Engine

Simplifying Storage

Simplifies

Development
Management of applications

Scale always matter

Request volume
Data volume

Datastore Storage Model

Entity consists of

Kind
Key
Entity Group
0..n properties
If entity group == key, the entity is a parent

Heterogeneous property types. Properties can be of different types in different entities
Supports multi valued properties
Variable property - Having the same properties between entities is not needed
Soft Schema

It is a schema whose constraints are enforced only in the application layer
Simpler development process

Rapid typesafe prototype

Can be enforced by JDO or JPA metadata mappings

Transactions

Only transact within an entity

Relationship Management

JDO and JPA are not just about object relationships

Transparent persistence
Object view of your data
Centralized mapping
Big maintainability win
AppEngine decides and manages which entity group the entity belongs to
Uses ownership to enfore entity group colocation

Future JDO/JPA work

Support unowned relationships

Bringing existing code to App Engine

Datastore is not a drop in replacement for RDBMS
Plan for data migration
Primary Keys

Single property keys: Straight forward way to map single property keys
Composite keys:Can map to ancestor chain
Mapping table: Can be represented using multi-value properties

And can be queried with set memebership

Transactions

Identify roots in the data model
Identify operations that transact on multiple roots
Analyze impact of partial success

Refactor
Run compensating logic

Queries

Shift processing from reads to writes.

Denormalize
Expensive write and cheap reads

Google Wave Client: Powered by GWT

Live Notes by @dushyanth

Wave UI requirements

It got to be fast
Stunning
Optimistic UI

JSNI

Java can call javascript

Client Architecture

Bidirectional communication channel - keep alive http
Protocol Compiler

Generates interfaces, client + server implementations

Code Heavy

Can use UIBinder to plop GWT components into html

Most bugs are from CSS

Style Injector + CssResource
Looks like Minification + Image Spriting is done by GWT
Allows modularization of CSS
Different CSS for different browsers

Inefficient JSON handling

JSO - Javascript object structure
Didn't quite get it

Hosted mode isn't quite browser like

OOPHM (Out Of Process Hosted Mode) - Browser plugin to debug in eclipse

Download Size

runAsync(dynamic loading of code)
Download lazily

No transparency between javascript and java

SOYC (Story of your compile) reports
Java package to javascript breakdown report

JSOs cannot implement interfaces

SingleJsoImpl
In order to inline, JSOs cannot have polymorphic dispatch
Atmost one JSO class being implementing one interface

Improving Gears

Client side thumbnailing

They create a thumbnail using the workerpool before uploading the image to server.

Desktop drag n drop
Resumable uploading

Performance

Startup

runAsync
fast start
inline images + css
smaller download
stats collection
server-side script selection

Server sends down the correct javascript + css files based on http headers

Loaded Client

Optimistic UI (trying to guess what the user will click next)
Prefetching
Flyweight pattern
Rendering tricks

Mobile Client

Deferred binding saves the day
iPhone browser is always running
It loads faster than native apps

Testing

Use Model View Presenter design pattern - how is it different from MVC?
Prefer JUnit tests over GWTTestCase
Browser automation - WebDriver
Web driver is a developer focused tool for browser automation
Has native keyboard and mouse events, rather than synthesised via JS
iPhone Driver - automated testing on iPhone
Remote Web Driver - so web testing can be farmed out into a grid

Java on the AppEngine

2009-04-08T08:47:00.002-04:00

Google has released support for Java on the AppEngine.

Here are some areas that really interest me:
Languages:
Since the support is at a Java Platform level, other languages like JRuby, Scala, Clojure, Groovy will be able to run on the AppEngine.

Enterprise Support:
AppEngine has been around for about a year with support for python. Typical usecase of AppEngine was simple webapps (backed by Google's BigTable). There was no support for batch jobs and no support for bulk loading of data. Yesterday's announcement provides support for those two and includes support for a Secure Data Connector that lets you access your enterprise's data that is present behind a firewall.

There were rumors about Java support on AppEngine for a few weeks now. Typically, AppEngine's features are aimed at a much higher level on the technology stack. If you look at python support for AppEngine, You will notice that it ships with a custom web framework that you can use to deploy applications. Java's support comes at a much lower level. It is much closer to the Servlet and JSP specification. These are not the exact tools you would choose to run your web apps in Java these days. Looks like Google has been counting on enabling developers run popular web frameworks on top of AppEngine. They have been working with people from the Java community enabling them to build support for AppEngine in their products.

I have been playing with the java support in AppEngine, it integrates very nicely into Eclipse through a plugin, supports a one click deployment. Not that you will want to deploy to your production server from Eclipse ;-) But, developers can deploy to their dev versions of the application using it.

Here are some quick stats on the environment that your programs will be running in:
Environment:
java.specification.version=1.6
java.vendor=Sun Microsystems Inc.
line.separator=\n
java.class.version=50.0 (I use a Mac OS X, AppEngine supports both Java 5 and Java 6)
java.util.logging.config.file=WEB-INF/logging.properties
java.specification.name=Java Platform API Specification
java.vendor.url=http\://java.sun.com/
java.vm.version=1.6.0_13
os.name=Linux java.version=1.6.0_13
java.vm.specification.version=1.0
user.dir=/base/data/home/apps//1.332645749880898171
java.specification.vendor=Sun Microsystems Inc.
java.vm.specification.name=Java Virtual Machine Specification
java.vm.vendor=Sun Microsystems Inc.
file.separator=/
path.separator=\:
java.vm.specification.vendor=Sun Microsystems Inc.
java.vm.name=Java HotSpot(TM) Client VM
file.encoding=ANSI_X3.4-1968

Other Properties:
availableProcessors: 1337 (humor ;-)
totalMemory: 104857600 (bytes)
maxMemory: 104857600
freeMemory: 6293011

I definitely intend to dig deeper into Java(and other JVM languages) on the AppEngine, and I'll be blogging my experiences as I go. You've been warned ;-)

My Joel Test

2009-03-30T02:14:00.008-04:00

I used to give client teams the Joel Test. It was good for the most part, but it does contain some pretty basic questions that almost all teams succeeded at. I retained a few from the Joel Test, and added some questions that I found were really essential to form a productive team.

Retained these from the Joel Test:
1. Can you make a build in one step?
2. Do you fix bugs before writing new code?
3. Do you use the best tools money can buy?
4. Do new candidates write code during their interview?

Added these to my list:
5. Do you use Scrum/XP?
6. Does your team do your own releases? (or is there a central release team that does that)
7. Can your team start/stop your prod servers and batch jobs (ofcourse with proper audits)?
8. Do you insist on measuring round trip times before and after a performance enhancement was made?
9. Do you have realtime alerts comparing inputs and outputs of your system?
10. In enterprise integration scenarios, do you automate the validation of inputs from other teams?
11. Do developers get to interact frequently with the end users of the system?
12. Does your team get some choice in the frameworks they use or does your company mandate a standard one?
13. Do your developers participate/give tech talks? Do you sponsor if needed?

A word on Joel's Tests:
Joel's tests have 12 questions. I have retained four. The rest got the axe because I think they are too common (using a source control) or I still have see them in action (like hallway usability studies)

5. Do you use Scrum/XP?
Without going into the pros & cons of Scrum/XP, the only point I would like to add here is, both of them tend to keep the end user involved in product development. This cuts waste and gives direction.

6. Does your team do your own releases? (or is there a central release team that does that?)
In general, teams that can release and upgrade their applications tend to be more productive especially during release cycles. I've seen enough teams which were forced to go through a central release team. These poor teams often email/send the upgrade commands to the release team and twiddle their thumbs. When the release breaks (ofcourse that happens), the team is forced to 'remote debug' for the release team. The release team usually has no time/expertise to deal with these app specific issues. This brings the morale down, makes sure that upgrades are a major pain for everyone in the team. Hence, they tend to lump upgrades together and not to frequent releases at all.

7. Can your developers start/stop your prod servers and batch jobs (ofcourse with proper audits)?
Your team designed the app, architected it, and built it. They fix the bugs. If you trust them and give them responsibility of keeping the server up, they will/can handle it. If they need to bounce the server or force kick off a failed job, they have to be able to do it. If they are not able to do these tasks, during production outages etc, they execute these manually defeating the entire purpose. I do agree the need for proper audits, but I strongly believe that depends on the manager doing enough checks.

8. Do you insist on measuring round trip times before and after a performance enhancement was made?
Nothing speaks like numbers in performance situations. Performance is a non-functional feature. You can only see the effectiveness of the fix only if you measure the before and after effects.

9. Do you have realtime alerts comparing inputs and outputs of your system?
Nothing beats having checks that constantly compare the outputs with expected output for the real time input. When it goes out of sync, an alarm would be raised. This has the benefit of catching the error as soon as it occurs (failfast). This also has a benefit that you catch your error as soon as the user encountered it. This ensures a very quick turnaround time.

10. In enterprise integration scenarios, do you automate the validation of inputs from other teams?
Suppose you rely on a service from another team. Having automated input validations to the values returned by the service (sometimes proactively) ensures that you catch data setup errors in systems that you depend on as well.

11. Do developers get to interact frequently with the end users of the system?
Above all, this makes your developers take responsibility for their code & bugs. No one likes appearing sloppy. If developers can see that their product is being used by end users who frequently talk to them it makes them put their best code forward.

12. Does your team get some choice in the frameworks they use or does your company mandate a standard one?
Too many companies force their developers to use a single framework/API for a task. This is generally made by someone 'up there' who believes too much in consistency. While I agree that having too much choice is bad, having a bit of choice is generally good. Using the right tool for the job makes all the difference.

13. Do your developers participate/give tech talks? Do you sponsor if needed?
Tough times, I know. But, the truly committed developer usually involves himself in attending (or better giving) talks. And, most of these talks usually pay for the speaker's expenses which makes it a no brainer.

Free the Air Waves

2008-08-19T20:42:00.003-04:00

As more and more TV stations move to digital only broadcasting, they vacate the frequencies used for analog transmission. Also, the analog channels are already spaced too wide apart to minimize interference. This decision was made a few decades ago.

Clearly, with the current advances in technology, it is possible to use all this white space to check what your favorite blogger is doing on twitter :-). Yepp. It can be used as a "wifi on steroids" as Larry put it.

As always with the govt, endless campaigns are needed to convince the FCC. So, go ahead, do yourself a favor and sign a petition.
Free The Air Waves

While you are at it, digg it.

iPhone 3G Zagg discount

2008-07-12T00:08:00.004-04:00

One time use 20% discount codes for Invisible Shield aka Zagg
Invisible shield for iPhone 3G

Works for iPhone 3G invisible shield too.
ch788u
n4ut8y

If you have more, please add them in the comments for others :-)

p.s: I don't work for them, I just love their products...

Google IO 2008 Day Two

2008-05-29T18:48:00.005-04:00

HTML 5 - Gears

From a vendor's perspective: You don't win users by being super standards compliant. At best, you don't lose users. Gears targets developers, not users. So, Gears considers standards to be a profit center rather than a cost center. Gears is a playground for new web APIs. One such example is SQL in Gears, which once proven successful in Gears, was included in HTML5.

Dalvik VM Internals:
Dalvik is designed to run on a slow CPU, with relatively low RAM, and on an OS without swap space while powered by a battery. The system library takes around 10MB. Thats a fairly large library at the application developer's disposal. Memory is saved via minimal repetition, per-type pools, and implicit labeling. Zygote is a pre-initialized process + pre-warmed Dalvik VM. This ensures responsiveness. When requested, it forks and returns the new process. The advantage is that the application shares the API classes in memory so that the memory foot print is less. Android manages multiple processes which have separate heaps, and separate GCs. When an application is installed, it is verified so that it does not violate the constriants of the system. The application is also optimized( static linking, inlining some native methods etc) so that when the application is run, it runs faster. Oh btw: the Dalvik is not a stack based machine but it is a register based machine. Theoretically, it is defined to be an infinite register machine. This results in fewer instructions to perform operations.

Inside the Android Application Framework:

Lifecycle:
Android provides a lot of hooks for application developers to plug-in custom code during the lifecycle. In my APIs, I try to give my developers a powerful yet simple looking APIs. Granted it is a tricky decision to make, but it is all about drawing a line and finalizing on the API. The Android lifecycle hooks are pretty complex and remind you of EJB lifecycle. Hopefully, a lot of developers will not get confused with differences between methods like onCreate(), onStart() and onStop(), onDestroy()

Threads:
Each process has a thread. Each thread has a Looper which handles a message queue. Any events are posted to this message queue. And, are processed by the thread through the Looper. Loopers cannot accommodate multi-threaded access. Loopers support a message handler to handle multi-threaded access. Looper also handles calling local service calls. Local services are ones defined in the same process.

Processes:
Each process is given it's own user ID. The only processes that run as root as Zygot, runtime, and init. A process has Intents, Services, and ContentProviders among other things. A Service is used to expose some functionality to other applications. To expose data, a ContentProvider is used.

Google IO 2008 Day One

2008-05-29T12:01:00.000-04:00

Keynote by Vic Gundotra
3 areas that Google is concentrating on are Client, Connectivity, Cloud.

Client:
The client from Google's perspective is the browser. Google Gears is spearheading to improve the browser capabilities. The Gears team is a major contributor in HTML 5.

Alan, Vice President of engineering from myspace showed an interesting usage of Gears by utilizing the local computing power of the PC. When a user with Gears performs a search on myspace, the results are served in realtime by the gears server on the local machine and using it's power and not having to make a trip to myspace's servers. Better experience for the user, lesser load for myspace. This is an interesting usage as Gears was initially designed to solved the disconnected client problem.

Connectivity:
Nice to findout that Webkit browser built into android. And, yes it is embeddable into Android applications. When cell applications and bandwidth of the mobile phone industry mature enough, browser will play a major part in how we deploy and run applications on the mobile. Nice to note that google is working on Gears support for Android. Next was an Android demo. Browser, Maps, Notification, Games, built in Compass et al. Can't wait to get my hands on an Android powered phone !!!

Cloud:
Kevin Gibbs the Tech Lead of App Engine gave a talk about the current capabilities of the App Engine platform (checkout my previous blog about App Engine) and the future direction for the project. One of the features that google is working on is offline processing for AppEngine. The ability to import and export data and to perform batch processing. Oh yes, App Engine now supports memcache. Google released a pricing for AppEngine and oh yes, AppEngine is public now. So, go and sign up if you already haven't. But, no Java/Groovy support yet. I spoke to a few Googlers about Java support for AppEngine. As we all know it is in the works. But, they were pretty tight lipped about it.

GWT: Bruce Johnson - Engineering Manager GWT
For me, being able to use GWT on mobile phones is the ultimate. Development in GWT is quite nice but, some things are not as productive as they could be. For example, having to define an interface for each component, using an custom Interface to access resource bundles. After looking at the buttoned down approach that Rails and Grails take for web development, I think GWT will really fly if it supports a scripting language(jvm based obviously) to develop.

Introduction to Android - Jason Chen
Looks like there are 1.1 billion PC based internet users and over 3 billion mobile users. And, most of the mobile market is fragmented with more than a dozen platforms to develop applications on. Jason discussed the android application stack which you can find here.

Anatomy & Physiology of an Android Application
Linux Kernel: Android built on linux kernel but it is not linux. All android specific functionality is built as drivers that run on top of the kernel. Thanks to the power of microkernel architecture. Android has custom linux drivers like Alarm, Ashmem (shared memory), Binder (IPC), low memory killer, power management, kernel debugger and logger modules.

Binder: Reduces IPC(Inter Process Communication) overhead and manages security issues. Uses shared memory (data between applications is shared). Manages per process thread pool. Performs Reference counting and mapping of object references so that shared objects are tracked and cleaned up. It also supports synchronous calling between applications to enable IPC. When process A calls method foo on service stub, binder driver proxies the object and sends it to the service. Since there is no serialization, there is no overhead that is encountered by a typical IPC.

Power Management:
Mobile devices run on battery power which have limited capacity. So, Android builds on linux's power management to manage power better. It does not replace linux power management. Android uses wake locks (partial, full) which expose the platform' power management to the developer. An Application requests a wake lock, PowerManager makes sure that the device is powered on until the application releases the wakelock. Alternative way of implementing wakeness is userActivity which takes a time period and keeps the application processing part of android alive.

Sitting on the linux kernel are libraries written in C++. Like Bionic (the custom libc implementation) Function libraries like Webkit browser, SQLlite and Native libraries like the SurfaceManager. Applications write to surfaces and the surface manager coalesces the different surfaces and outputs it to the frame buffer (display). Audio Manager works similarly. Audio Flinger routes different audios to different audio devices like speaker, headset, ear piece etc.

Hardware Abstraction Library: These are native libraries that sit above the linux kernel. This layer defines APIs for developers to port android to different hardwares.

Android Physiology:
When the Android platform starts, it starts the linux daemons like USB, Debug Bridge etc. It them starts the Zygote process which is basically a template for future processes. Next, it starts the service manager as the default binder. Overall, this talk rocked.

Under the covers of App Engine Datastore - Ryan Barrett

The AppEngine datastore is powered by BigTable. BigTable is basically a sharded sorted array. Supports sort of test and set operations. They call it single row transactions (read-update)

All entities are stored in a single table called Entities table. Entity keys have a field name:value format(hierarchial). The entities table is ordered by key. So, accessing an entity with a key or iterating on adjacent keys is easy. Although, the only way to re-parent an entity is to delete it and re-create it.

Composite Index:
You define the index yourself in index.yaml Or, when the app runs in the dev area, app engine looks at it's queries and creates indices by default. The flip side of auto-index is if a query has not been hit in dev mode, the prod refuses to handle it, and throws an exception. So, run all ur code paths in dev mode.

Transactions: AppEngine datastore supports atleast read-committed using a last committed timestamp embedded into the data.

Oh yes, there was a 4 hour party at the end of the day. Good music, good food and got to network with Geeks and Higher-ups alike.

The Rise of App Engine

2008-04-11T19:25:00.007-04:00

Google released AppEngine, and there were a bunch of reviews comparing it with Amazon EC2. Not quite the same. From a manager's perspective who is trying to get a web project done probably yes, you can achieve the same goal in this case with both...

But, the real difference is in the level at which these are operating. AppEngine, Heroku et al are I would say at a service level. EC2 bascially lets you write what the heck ever code you want, and provides an abstraction at the machine level. Instead of running on the bare metal, you would run on EC2 and get all the goodies that come with the classic "Add another layer of abstraction" rule.

The importance of AppEngine is the productivity gain that it would bring into a very focused but large set of applications. Yes, there is no support for batch and so, lot of other cleanup jobs etc cannot be really run on the platform as of now. But, it is still in it's initial stages and too early to comment. I don't see it ever brimming with additional features though. If you have noticed with Google's products, it is not the feature set that they try to win the market with. Example: Yahoo Mail and Yahoo Messenger kick GMail and GTalk's butt respectively in terms of feature set. But, it is ease of use, attacking the problems that matter (like spam, being able to find what you wrote, and storage size)

Also, I really don't see them ever supporting my current favorite web framework Grails. Why? Grails is tied to hibernate which I don't see running on a non relational database like the one google has as the data store of AppEngine. Not to say they are not working on a "Write your Grails, we port it to something else and run on AppEngine" virtual machine.

There have been free services that ran JSP/ASP based web applications out there for a while. But, those were more of a "write your app, run on our server" kind of model. App Engine is essentially a scalability/availability mantra for the masses. Google has enforced the scalability best practices by restricting the developers to use a subset of features which lend themselves to scaling.

On an un-related note, I could not find a way to get either blogger or feedburner to offer Tag specific feeds. I had to jump through hoops to achieve it. More on that in a later blog. Stay tuned...

TSSJS 2008 Day Three - Synopsis

2008-03-28T18:30:00.004-04:00

Session II – eBay Market Place Architecture – Randy Shoup

What happened to Session I ‘s coverage? Lets just blame it on the night before :-)

We all know eBay doesn’t do transactions. Well at least no client side transactions. Their databases run on auto commit. One way they deal with it is by carefully ordering database transactions (example: inserting the slave record and then inserting the master to ensure a consistent master). They also have reconciliary jobs that go through the database and cleanse it periodically.

Strategies for scalability used by eBay
1. Partition
2. Asynchrony
3. Automate Everything
4: Remember Everything Fails

1. Partition

Obviously, they don’t use sessions and they don’t cache business objects (surprisingly). As expected, they use URL rewriting and cookies to track the user. If the data they have to keep about the user is larger than will fit in these two schemes, they use a scratch database. Since they don’t cache business / user related data, they do hammer the databases for all their queries. To handle this situation, they use a custom sharding solution over their ORM and partition their database based on functional divides in the application.

Search: Search queries come to an aggregator which Is actually a scatter-gather (from Enterprise Integration Patterns). This component forwards the search requests to individual nodes which are responsible for indexing and searching just a part of the entire data space. And, then return the results to the aggregator which aggregates (da !!!) and displays them.

2. Asynchrony

The really hard part of massaging systems is guaranteeing once only delivery. If you loosen this restriction, it is a lot easier to scale. They deal with duplicate events by modeling event processing to be idempotent. They deal with out of ordering by making the consumer go to a service that returns the latest state of the event once the consumer receives the event.

3. Automate Everything

Part of that is adaptive configuration: The consumers that dynamically adjusts to meet the SLA by changing parameters like event polling size , number of threads etc. The adaptive configuration also adapts to changes in number of consumer instances.

He gives an example of an adaptive search experience. They have a feedback loop and in an offline way, they analyze it , create a metadata out of it and feed it to the system that uses it to change it’s behavior. Perturbation is the idea that 90% of the time they recommend the optimal. 10% of the time they recommend new options B, C, D etc.. So that if D becomes popular, it will become the dominant recommended value. They also overweigh the negative feedback so that the oscillations are dampened. Pretty slick.

Strategy 4: Remember Everything Fails

Some of the failure patterns used are failure detection, rollback and graceful degradation. Applications log to a message bus and they have listeners that automate the failure detection. It also allows them to detect historical data and it is used from a capacity planning perspective. They get about 1.5TB of log messages every day :-) grep that.

Code rollout/rollback: They have a policy. NO changes to the site that cannot be undone. Each feature has a rollout plan. And, there is a monster rollout plan for the 2 weeks. There is an automated tool that rolls out the dependencies in the reverse dependencies. The automated tool also does rollbacks.

Here is a cool feature. Every feature has a on and off state. It allows them to turn features off rather than redeploying code that lacks that feature. This allows them to deploy features off and then start them later. They are decoupling code deployment from feature deployment. From a developer perspective, they check for feature availability. To blow my own horn a bit, I have built features in the past which can be turned on and off at runtime. I know what you’re thinking (don’t freak out, I don’t) . This is similar to OSGi. Nope not quite. OSGi is about deploying services and controlling their existence. I would say this may be similar from an implementation perspective, but the intent is quite different.

When the resource fails, and it is not critical, it is safely ignored. If a critical service fails, they go to an async mode (and do the processing later) or do failover. When a service does come back up, you don’t want all clients hitting it at once. They have a phased way of letting clients to hit it.

Overall, this talk was very informative. Randy went through a lot of concepts in great detail at a very high pace. I felt there was so much more information left that the talk could have gone for one more hour at the same pace.

Session III – The Busy Java Developer’s Guide to Scala – Ted Neward

A pure functional language has no side effects. But you knew that already. This talk focused on giving an Scala intro to a Java developer. Again, I am not going to cover much of this talk as you can read about Scala yourself.

Lunch Keynote Panel: Patrick Linskey, Ted Neward and others with Eugnee Ciurana as the mike boy ;-)

The conversation went towards the over abundance of frameworks in java. One good point made was, you should never sit to write a framework. You build an application and then extract a framework. In that case, YAGNI will be inherent in that effort. One of the very few web frameworks that was built like that is Rails.

Another point is, it has to be usable before it is reusable. Good one.

In answer to whether the appearance of free type(terse) languages (where syntax does not matter much) java will allow syntax to be optional, it was pointed out that if you try to shoe horn additional features into the language, it will fail under it’s own weight. The main thing in Java is the platform and the APIs and frameworks that are available. Additional languages that run on java platform but support a whole new set of features will see the light.

Session IV: Map Reduce – Why does it Matter – Eugene Ciurana

We looked at Map Reduce and worked out through implementation scenarios in various business domains and the problems or hurdles that we would face in arriving at the solutions using Map Reduce. Audience got to participate and overall, it was a good exercise. (There, I can talk in bullshit business lingo too :-) )

I'm sure you've seen this already, but it is too cool not to point out
http://members.aol.com/matt999h/bullshit.htm

TSSJS 2008 Day Two - Synopsis

2008-03-27T19:11:00.002-04:00

Session I - Concurrency: Past and Present – Brian Goetz

I have heard Brian’s talks in the past in the No Fluff Just Stuff conferences, and they have always been enlightening.

Brian recommends these papers about concurrency:
coping with parallelism treiber 1986
why threads are a bad idea, osterhout 1995
the problem with threads, edward lee – 2006

The talk is a bit more targeted at junior developers than I had hoped. Brian dealt with deadlocks and deadlock avoiding mechanisms. Brian talked about STM (Software Transactional Memory) and why he thinks it is not the silver bullet. He does not mention the reasons. He says he is thinking of presenting them in a talk in JavaOne.

Session II- Performance Puzzlers: Kirk Pepperdine & Brian Goetz

What followed were general commandments of performance (at least at this day and age). Release memory variables soon. Perofrm benchmarks on a dedicated machine.

One interesting point that was made was that in a benchmark with a system using OR mapper and a similar one that didn’t the one with the OR mapper performed better. The reason was that the concurrent garbage collector was essentially stealing processor cycles to clean up all the objects created by the OR mapper and ended up throttling the system against flooding the database with requests. When the OR mapper was missing, the database was thrashing leading to worser performance. So, even though both the direct JDBC implementation and the ORM implementation hit the database with similar number of requests, the ORM implementation is not flooding the database and is providing throttling for the requests. Interesting, but this is one of the things that happens by accident.

Session III- Implementing an ESB solution using Mule – Ross Mason

Services can be categorized as being in one of three layers.
Task based – represents a business task – example: buy a product – little reuse
Entity Based – represents a sub task – example: bill a customer – some reuse
Utility based – represents an atomic independent service – example: credit card processing – most reuse

Reusability of a service usually depends on which layer the service is classified into. With most reuse coming form the lowest. (as expected)

Mule provides support for creating functional test cases which, I think are more useful than testing one single class in isolation. More about unit testing vs functional testing in a later post.

Mule Expression framework is a way to evaluate expressions on a mule message. This makes content based routing easier. The expressions can be in xpath, groovy and a whole lot of other types.

Lunch Keynote: Why the Next Five Years will be about languages – Ted Neward

Ted built a case of language oriented programming. Object oriented programming is not the pinnacle in the evolution of languages. I seriously doubt if we will ever reach the pinnacle :-). One of the required features of a language will be tool support. These days it is a lot easier to get tool support for new languages. And, many of these languages can be made to run on the same platform that we currently run on. In the near future, we will see a lot more languages prop up into our mainstream development. Overall, the keynote was great.

Session IV: You got Your Ruby in my Java – Chris Nelson

Chris gave an introduction of Ruby. This talk is basically a Ruby introduction to the Java developer. I am not going to list the content of the talk as you can google it up and read about Ruby. It is nice that we as an industry are finally breaking the language stalemate that we hit and are willing to look at other languages to improve productivity.

Fire Side Chat: Concurrent Programming with Java and Erlang:

This was the first fireside chat that I’ve been to. The closest I’ve been to a fire side chat is viewing a chat on Google video :-). This free form of discussion with the right participants can really touch on various topics. Something like the discussion on TSS without the flame wars :-).

Fire Side Chat: Mission Critical deployment at Leap Frog Systems - Eugene Ciruana

I have to admit that I don't exactly remember the name of the presentation. But, this is by far the best one I've attended at TSSJS. The crown jewel if you will. To give an analogy, if you've read Patterns of Enterprise Application Architecture - Martin Fowler, it is the embodiment of years (if not decades) of rich software development expertise condensed in a book. This talk was the embodiment of years of practical, i repeat, practical software development experience condensed in a 1 hr talk. This was a fire side chat and not a typical presentation. We the audience, got to ask all sorts of questions about Leap Frog's infrastructure and more importantly, the design decisions that went into selecting the software solutions. It was like discussing the battle plans and reviewing the war with a war general.

No matter how many subject matter experts you talk to, it is the experience in the field and in the trenches that really teaches you and really counts.

TSSJS 2008 Day One - Synopsis

2008-03-26T21:07:00.003-04:00

Key Note: Neal Ford, Thoughtworks

Neal started with an assault on Ruby of Rails developers(and later justified the reason why ruby on rails developers brag). I think the reason why they brag so much is because it is so productive. Thankfully, on the java side, we have Grails to the rescue. Anybody who hasn’t taken a look at Grails absolutely should.

Next, he builds a case for DSLs. One of the strong points of DSLs is that the context is implicit. You don’t have to keep reminding the runtime of the context:

Example: Consider the following java code

Car car = new Car();
car.setColor(Color.BLACK);
car.setTransmission(Transmission.MANUAL);
car.setPrice(30000);

Now take a look at this in a dsl that I just made up

create Car:car
color:black
transmission:manual
price:30000

Now that is much more clearer as we don’t have to keep ourselves repeating the context which in this case is car.

One thing you have to give to Neal Ford is, he knows how to give presentations. In his presentation, he normally uses just one sentence per slide, Sometimes just one word. And lots of pictures. These pictures are typically not from the computer science domain. These are usually analogies from the real world that help the listener in understanding the underlying concepts. Another technique I noticed him use is when he wants the user to concentrate on his talk and not on the screen, he leaves the screen blank.

Also, a template for your presentation is not needed. Leave it dark. It works magic with the audience. Templates and light colored background distracts the listener from what you want to show. This style of presentation is also visible in Steve Job’s presentation. He never uses a template.

Ok that was an un-intended tangent. Now back to the talk: As a programmer, it adds real value to write programs that are close to how the user talks about the problem. It is a huge advantage if the users can read what the programmer can produce..

The final point was that the development stack of the future would be a structured programming language at the bottom. A dynamic programming language on top of it to improve development productivity and then a DSL on top to stay close to the problem space.

Tactical Design by Glenn Vanderburg:

Presentation tip: Try to build on key aspects of prior presentations (the ones you like) so that the audience get a transitionary feel.

His talk concentrated on how to improve the design skills of poor designers and make them good designers. I think it is obvious that DRY and sticking to one level of abstraction thought the method is a good way to improve design. This is analogous to saying “use Design Patterns”. It is also ironic that he took the same topics and repeated them over and over. Overall, the talk was sub mediocre. Luckily, I had my laptop with me and caught up on some grails reference documentation.

p.s: Lenovo sucks. But having a built in upgraded battery back that gives you 6 hours of alive time rocks.

Also, if ure in a talk that you are not sure of, sit at the last so that you can sneak out if the talk sucks. (Also iterated by Hani in one of the previous years of TSSJS)

Self Scaling Java Based Cloud Architecures – Jinesh Varia, Evangalist at Amazon

I can say I hate talks given by evangelists as most of them tend to be biased. The talk sounded like it was a product presentation and less about giving insight. It takes a visionary to be able to give insight. I thought I can stick around as I wanted to know more about Amazon Web Services. Basically it was a vendor talk, and does not delve into the methodology or mindset that leads to cloud computing.

After 30 minutes, I could not bear it anymore and snuck out.

Next stop: Building REST-ful web services with the JAX-RS API – Mark Hansen

This talk turned into a ‘show and tell’ and I snuck out as soon as I typed the title.

Speed: Kirk Pepperdine

This was the third talk I went to in an hour. You can’t go wrong with a talk about JVM J. The talk covered improvements in garbage collection times and algorithms. One of the cool things that Java 7 (sun jvm) will do is look for local references and allocate them on the register. So that they never see RAM.

After the talk, I caught up to Kirk and asked him a few questions about when you would do volatile (for primitives) vs when you would do AtomicBoolean(and other atomic classes). More on this on a separate blog.

Designing for scalability: Patrick Linskey

This talk was about horizontal vs vertical scalability. Running into contention because of shared state. And, about partitioning the application at various levels to achieve scalability.

An application can be partitioned in multiple ways. Some of which include:

Partitioning along application bottlenecks.

Patritioning along data set “fault lines”. Example: geographic collocation so that only relevant information is close to where it is being used. This is an example of a stateful service being partitioned for scalability.

Also using asynchronous execution to increase scalability. Nothing ground breaking. One good point made was that asynchronous architectures inherently support throttling so that the system can catch up on processing in off peak hours.

Boldly Go where the java language has never gone before – Geert Bevin

This is an interesting frame of mind that you do not need to leave the java language. The language can be instrumented or transformed into a different execution platform so that the developer skills can be reused. Geert gives example of Terracota which performs dynamic byte code manipulation of classes to provide clustering without the developer having to use a vendor specific API.

The next example Geert touches is GWT. GWT lets the developer code in java but generates javascript under the covers in production. I would think the intent of GWT is nice. Free the developer from having to worry about javascript incompatibilities between different browsers. The implementation is based on java, which I increasingly think is a wrong tool to create web based/GUI applications. The Java syntax is too buttoned up. Grails on the other hand is a wonderful tool to create web application. I think a grails implementation of GWT would really shine.

The next example is Android. This is my favorite. Android, as you might already know lets the developer write code in java, but gets compiled into an executable that gets interpreted by the Dalvik virtual machine on the cell phone. Although the developer is coding in java, like in GWT, what gets executed has nothing to do with the JVM. I have not done any serious Android development. My take is that UI development is better left to templates. I think it is just a matter of time that a UI templating framework is released on top of Android.

The scalability pitfalls of the realtime web- Jonas Jacobi & John Fallows

The presenters build a case for a event driven server which is basically a reverse ajax. And started listing the pitfalls and solutions(or hacks depending on your perspective) to these pitfalls. Snoozfest. And yes, I got out as I could.

There was only one other presentation that was remotely interesting, which was a use case on SCA.

Next Generation Payment Systems using SCA

My take on SCA is that it is trying to make the same mistake that EJBs did with the “thy shalt give thy java interface to the client”. I am a big proponent of simple and non language specific contracts between multiple parties in a distributed platform. When you expose a webservice (even REST based ones) you are essentially creating a contract with the end developer about the nature of the data interchange or service invocation. The problem with a language specific contract is that apart from being tied to the language, you now get into headaches with versioning. Most programming languages that we currently have do not work well with versioning. So much so, that these days, we have had to build specifications like OSGi to handle multiple versions of an object existing in the same virtual machine.