With the 7.0 release of GigaSpaces, I thought I would write some blogs about the new and exciting new features that are part of the release. The first feature that I would like to talk about is GigaSpaces executors support (which premiered in 6.6) and one of the cool things you can do with it.

In essence, GigaSpaces executor support allows to define custom Tasks that will be executed within GigaSpaces in a collocated manner with data. Aside from collocation, one of the nice feature of it is the fact that the code does not have to be predefined within the GigaSpaces cluster, but instead it is loaded on demand. Here is an example of such a simple task:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

public static class TradeValueTask implements Task<Float> {
 
    private long tradeId;
 
    @TaskGigaSpace
    private transient GigaSpace gigaSpace;
 
    public TradeValueTask(long tradeId) {
        this.tradeId = tradeId;
    }
 
    @SpaceRouting
    public long getTradeId() {
        return tradeId;
    }
 
    public Float execute() throws Exception {
        Trade trade = gigaSpace.readById(Trade.class, tradeId);
        return trade.getValue();
    }
}

The above task allows one to actually get *just* the trade value for a given trade id, instead of reading the whole trade and extracting the value from it on the client side.

The execution will be directed to the same partition that trade exists on (assuming that the trade id is also the routing field for Trade) and executed there. This means that the readByid operation will be executed in a collocated manner with the space instance.

Here is an example of how it can be executed:

1
2
3

// gigaSpace here is a clustered proxy of the whole cluster
AsyncFuture<Float> result = gigaSpace.execute(new TradeValueTask(1));
float value = result.get();

Now, lets assume that calculating the trade value is complex and we already have a service that knows how to configure it in the Processing Unit descriptor file:

1

<bean id="traveValueCalulator" class="eg.TradeValueCalulator" />

We would love to be able to use it in order to calculate the trade value from within the processing unit. What we actually want to do is be able to get a handle to the TradeValueCalculator from within our class. There are several ways to do so, but one of the nicest is to actually use Spring autowiring capabilities to inject it, which we can easily do:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

@AutowiredTask
public static class TradeValueTask implements Task<Float> {
 
    private long tradeId;
 
    @TaskGigaSpace
    private transient GigaSpace gigaSpace;
 
    @Autowired
    private transient TradeValueCalulator tradeValueCalulator;
 
    public TradeValueTask(long tradeId) {
        this.tradeId = tradeId;
    }
 
    @SpaceRouting
    public long getTradeId() {
        return tradeId;
    }
 
    public Float execute() throws Exception {
        Trade trade = gigaSpace.readById(Trade.class, tradeId);
        return tradeValueCalculator.calculate(trade);
    }
}

By marking the class with the @AutowiredTask annotation, it will automatically get autowired with the beans defined within the processing unit. Pretty cool, no?

(As a side note, of course, autowiring takes time, which might be of essence ;). Another option is just implement ApplicatiobnContextAware, and use the ApplicationContext to get the bean based on its id).

So we can autowire beans into a Task executed using GigaSpaces. One last thing before we end this post, I would like to show another nifty feature of GigaSpaces executors, which is the executor builder.

Lets assume that we now want to calculate the sum of several trades by their ids. We could have gone ahead and execute each task, wait on all the futures, and sum the results. But, why work hard when we (GigaSpaceians) can work hard for you (my personal motto):

1
2
3
4
5
6
7

AsyncFuture<Float> result = 
          gigaSpace.executorBuilder(new SumReducer<Float, Float>(Float.class))
                                 .add(new TradeValueTask(1)) 
                                 .add(new TradeValueTask(2)) 
                                 .add(new TradeValueTask(3))
                                 .execute(); 
float value = result.get();

All three tasks will be executed in parallel, in a none blocking mode, each on its respective partition, and the results will be automatically reduced using our built in SumReducer.

Enjoy!

One of quantum physics’ crazier notions is that two particles seem to communicate with each other instantly, even when they’re billions of miles apart. Albert Einstein, arguing that nothing travels faster than light, dismissed this as impossible “spooky action at a distance.”

The great man may have been wrong. A series of recent mind-bending laboratory experiments has given scientists an unprecedented peek behind the quantum veil, confirming that this realm is as mysterious as imagined.

Based off this theory, there has been several computer science related experiments, especially revolving around Quantum Computer and different encryption algorithms that I won’t get into in this blog post. What I would like to suggest is how Quantum Physics can revolutionize the area of (In Memory ;) ) Data Grids.

One of the main problems of Data Grids is the ability to replicate state changes from one instance to the other, especially when using WAN. This highly ties into Brewer’s CAP Theorem, which states that:

When designing distributed web services, there are three properties that are commonly desired: consistency, availability, and partition tolerance. It is impossible to achieve all three. In this note, we prove this conjecture in the asynchronous network model, and then discuss solutions to this dilemma in the partially synchronous model.

What I am suggesting is that once the ability (and it very close, there are already companies building highly secure computer systems using Quantum Physics), CAP theorem will no longer be applicable.

As a thought experiment, imagine that a photon has an up spin, and that represents binary 1 (I know, it can get much more advance than that, I am simplifying things). A down spin represent binary 0. Once photons are “entangled” (we bring up our data grid), and then we separate them (across the building or across the ocean, does not really matter), we can get “instantaneous replication”. Once we change the state of one photon, the other will change its state instantaneously (only when we check its state, but that is when we really care about it ;) ). By exhibiting this behavior, we actually can get all three properties of CAP theorem.

Imagine as well the ability to store a “local cache” of the data. Since the size of data that can be “store” is exponentially bigger than current technology, and the fact that “state change” is not bounded by current technology (no need for wires), most people can have most of the data locally most of the time (which in itself, is relative). Once local data is changed, there is no need for 2PC or something like that in order to update the master data. For all intent and purposes, we hold the master data :).

Quantum Physics is going to revolutionize the way we go about and use technology. What I talked about is just the tip of the iceberg, but I personally believe that once the technology starts maturing, the impact it will have on the world will dwarf the arrival of computers, the industrial revolution, or any other major event that occurred in our not so long history.

One of Einstein famous quotes regarding Quantum physics is “God doesn’t play dice with the universe”. I personally like better Neils Bohr, a big proponent of quantum uncertainty, rebuttal: “Quit telling God what to do.”

Seems like several Compass users did not like the move to Google groups (can’t access it from work and so on), so we are back to the old forums. Sorry for the trouble…

Cheers,
Shay

Google App Engine was released yesterday for Java. It provides the ability to deploy Java based web applications onto the App Engine, and provides a data store service (with JPA and JDO on top of it), memcached, and others.

So, what I decided to do is try and get Compass integrated with Google App Engine to allow for easy full text search of Google App Engine apps (yea, the irony). Basically, it took me a couple of hours (most of them spent on this bug in gae), and we have something ready to use. Here is a screencast showing how the Guestbook app can be made searchable easily (use full screen mode when code is shown):

What are the steps needed?

• Make your domain model searchable.
• Create a Compass instance. Configure it to store the index using GAE data store (there is also support for native Lucene applications). Configure it not to use threads.
• Create a Compass Gps, have it automatically mirror changes done using the JDO/JPA API to the search engine. Call index operation to completely reindex your application.

Thats it. Pretty simple no? Especially since these are the steps you use to enable any Java application with full text search using Compass :).

Of course, there are some leftovers. The first is better support for JPA (built on top of Compass generic support for JPA). Another enhancement is to use memcache service to cache heavily used index portions.

Update: Questions regarding the integration are best asked on the Compass forums. Its hard to have a conversation using the blog comments.

Enjoy!

Compass version 2.2.0 GA released. There are many features included in the release which I covered in the different milestone and release candidates announcements. Here is a link to the user group announcement.

I already have several things lined up for Compass 3.0, with the first thing on the plan is a complete restructuring of the codebase to a more modular one (should not be a breaking change, maybe some package renaming, will try to keep that to a minimum). Compass integrates with many libraries now, and its time to make the structure/build more modular (it is already mostly modular internally). Will report back on how it goes.

Enjoy!

p.s. Compass is now on maven repo central, all maven lovers rejoice!.

Compass version 2.1.4 released. The version fixes one major bug revolving around how Compass manages read committed transactions, it is highly recommended to upgrade to it.

Enjoy!

Compass forums have moved to Google Groups, the group name is compass-project-user, and the link is http://groups.google.com/group/compass-project-user.

Why? Well, basically, Jive is no more, and it seems like Clear Space or something like that is the new way for the company to go. Also, no more open source license. So, I thought it was a good time to move to Google groups.

I managed to migrate most of the forum posts to Google groups, but, if you want to be sure that I will answer a question, please repost a question you have again. Sorry for the inconvenience.

Cheers,
Shay

Compass version 2.2.0 RC2 released. The version fixes several bugs, including the two bugs described in the 2.1.3 post.

Full release notes can be found here.

Enjoy!

Compass version 2.1.3 released. The version fixes two important bugs. The first fixes a problem when using Compass support unmarshall flag set to false (OSEM) and using analyzer property controller or boost property controller (basically, it did not work…).

The second one revolves around Compass integration with TopLink and EclipseLink. Basically, with complex hierarchy of mappings, objects would be created twice in the index. It boiled down to how TopLink and EclipseLink fire up events for such cases. An event on class B that extends A is also fired for A…, so Compass needs to take that into account when registering event listeners. Surprisingly, there was an extra fix required for EclipseLink, as the logic of firing events changed a bit between EclipseLink and TopLink.

Full release notes can be found here.

Enjoy!.

Compass version 2.2.0 RC1 released. Main features include simplified index and search session, mt transaction processor, and dynamic properties. Full release notes can be found here.

Enjoy!