I haven’t read the book yet (just downloaded on Kindle), but based on the teaser text at Amazon, I was disappointed to see that the author felt the need to tie his already interesting topic into the current financial malaise. The jacket text suggests he is blaming the quants for the 2008-2009 financial crisis. This will probably sell more books, but is an intellectually simplistic/sensationalistic/lazy argument in my opinion. I will reserve full judgment until I have read the book though.

Amazing how being anti-anything seems to be the preferred approach in books, newspapers and TV appearances these days.

The CIO makes what looks like the right declaration: “We WILL reduce power, no matter what”. Then good, honest projects come along that boost the top or bottom line and become exceptions to the “no matter what” part of the edict.

It is well documented that CIOs tend to last 2-3 years, and a typical IT worker will outlive several CIOs. Through time, these declarations take on the flavor of The Who’s immortal observation — meet the new boss, same as the old boss. Go on, click and waste the few minutes listening to vintage era Who. You know you want to, and it’s more energizing than drinking 2 cups of coffee. Not only that, but it immediately erases the memory of their Superbowl performance.


The best way we have seen to get the foot soldiers aligned with management objectives is to tie a meaningful piece of their compensation to it. Then everyone is aligned to find a way to make both kinds of “green” (more money and less power) happen. The good news is when half your annual pay is tied to anything, it becomes your focus. The bad news is this only works in financial services where bonuses are large enough to motivate. Having half of a government workers 10% bonus tied to power usage won’t change anyone’s behavior. They will become a bigger star, and get a bigger raise for delivering better or faster than they will for delivering cheaper.

In 2020, data centres worldwide will consume about 450 billion kWh and their CO2 emissions (at about 330 million tones) will be equal to those of Portugal, Switzerland, Greece and Sweden combined. Their electricity bills will amount to nearly $45 billion (£26bn).

The demand for computing is always growing, so the only practical way to get more green is to do more work per watt.

This is a big reason why many customers are rolling out Solace technology. We generally get in the door on speeds and feeds, but we get picked and purchased based on our ability to reduce costs and shrink the datacenter. One Solace message router can replace the work being done by 10-30 servers running software-based messaging middleware, while using the power equivalent to about 1.5 servers, a power savings of 85-95%. That’s just power, remember, and doesn’t even reflect other cost-efficiencies like a smaller datacenter footprint and reductions in software licenses, software maintenance, linux maintenance contracts, and the server hardware itself.

It’s a win for everybody: faster applications for the operations teams and lower costs for the pencil pushers. Software-based middleware can’t demonstrate these results, and in fact is one of the key drivers of the “server sprawl” problem because scaling a software-based solution means splitting the work across more cores, more machines, etc.

The server vendors will forever push replacement cycles with a 10-15% more efficient machine, but it seems pretty clear that the most effective green strategy is finding ways to decommission the server racks altogether.

Last week Greg MacSweeney wrote a good article in Wall Street and Technology highlighting the back office as a new battleground for efficiency. After years of chasing zero latency for the front-office, the back office is comparatively archaic and badly in need of updating.

Greg predicts that 2010 will be the year the back office gets some love, and his analysis is spot on. Electronic trading has accelerated the rate of initiating trades so much that most mid- and back-office systems struggle to assess risk and execute trades before someone else gets there first. The mid and back office have to keep up with risk and make sure trade execution and settlement don’t experience bottlenecks or disruptions. But you can’t just apply front-office advances to the frumpier back office. If the front office is (mostly) about latency, the mid and back office are primarily about cost-effective, predictable and reliable behavior.

In financial services the cornerstone technology for making information flow is messaging middleware, and for front-to-back messaging, Solace’s answer is its Unified Messaging Platform (UMP). Solace’s UMP gives you one API that is best-of-breed for performance, reliability and operational simplicity across the front, middle and back office. It’s proven with big investment banks, financial information providers, hedge funds, high frequency trading shops and exchanges. I’m not talking about the future either, over half the companies we’re working with today look to us for mid and back office advantage in addition to speeding up the front office. Consolidating these traditionally segregated environments is changing the game in terms of value delivered for our customers.

With apologies to Justin Timberlake, we’re bringing sexy (to the) back (office)!

We look forward to discussing the announcements we made last week (DHS/DNDO as a customer, Thermo Fisher as a partner, and our new geospatial routing capability) and seeing what else is hot in the world of information exchange and critical incident preparedness.

In addition, we’ll be participating in a demonstration designed to show off how a range of technologies can help authorities manage a multi-faceted critical incident. In the scenario, an ammonia leak is detected in the Philadelphia Flyers’ arena shortly before game time, just as a severe weather front is moving into town. The scenario includes the routing of alerts and information as citizens are notified about the situation and given appropriate instructions depending on their location, hazmat teams are dispatched to the site of the ammonia leak, injured parties are sent to the best hospital for treatment, a tornado warning is issued, and more. The scenario includes practices and protocols such as geospatial routing, Common Alerting Protocol (CAP), Emergency Alert System (EAS), DM-OPEN, EDXL, Commercial Mobile Alerts System (CMAS) and EDXL Hospital Availability Exchange (HAVE). It should be an impressive display of the kind of information sharing being streamlined through the efforts of the National Information Exchange Model (NIEM).

Sensor Networks are Coming of Age

Sensor networks have been with us for years, but the combination of ubiquitous wireless networks, higher bandwidth, cheap storage, improved battery life and solar power are driving more and more applications towards data collection using sensors of one kind or another. These systems get very complex very quickly. Just consider:

• The sensors are usually distributed and heterogeneous
• Aggregate sensor data production rates are sky high (number of sensors times sample rate per sensor) particularly when combined with images and video
• The sensors can be fixed or mobile
• The people applications interested in the meaning of the sensor data can be fixed or mobile
• What is deemed important within the sensor data is fluid and constantly changing

How do you make sense of a massive stream of information, distributed across a large geography where the relationships between sensors and their surroundings can literally be changing minute to minute?

Why You Can’t Just Dump it in the Data Warehouse

Traditionally, the solution to problems with this much data has been to dump it into a data warehouse and let the data mining tools search for meaning among the gazillions of records. This is a proven way to index, apply rules and report on what happened, but it falls short in a variety of ways:

• It takes time to index and report, giving you an “after the fact” picture, not a current one
• Everything needs to be part of the master data model before it can be indexed and searched
• It’s notoriously hard to change what you are looking for, that is, to change the search and reporting rules

Data warehouses have their place and serve an essential function, but they don’t help you understand what’s happening right now. For example:

• A GPS sensor in a car could send data off to help city engineers understand traffic patterns and where to construct new lanes next year, but it’d be more satisfying if it could re-route you around the parade that’s blocking traffic right now.
• It’s good to know that a high radiation level sensor reading happened at the entry to the Holland tunnel an hour ago, but it’s a lot better to know that in real time when a truck is actually entering the tunnel. Similarly, it’s important to differentiate in real time between expected transportation of a medical device to a hospital (which can produce a positive radiation reading) and a radiation reading that is unexpected.
• Courier handheld scanners can feed a data warehouse with tracking information so you can look up the status of your shipment, but wouldn’t it be better if your mobile phone updated them on where you are and the package came to you? At work, at home, at your kid’s soccer match – no more “Sorry we missed you!” notes on your door. Science fiction? Nope, the day is coming.

Right Now: Understanding What, Where, When of Sensor Networks

What if sensors just spit out information and the infrastructure that was responsible for capturing and carrying that data could contextually filter and analyze the data to find the important real-time needles-in-the-haystack? What if that system was fully distributed in terms of geography, content and control so users of the system could define what they are interested in and many applications could find different meaning in the same data?

Today’s announcements make that vision a reality. Summarizing, Solace announced:

• The addition of a geospatial routing blade, making the high-volume processing of location-based information a core capability of our Unified Messaging Platform.
• A partnership with Thermo Fisher Scientific, the market leader in supplying sensors for a wide range of government and industrial uses, announcing that we have integrated Solace’s API with Thermo Fisher’s ViewPoint software. This means that any Thermo Fisher sensor that talks to ViewPoint is able to feed a Solace messaging backbone directly.
• And finally, the addition of Department of Homeland Security/Domestic Nuclear Detection Office as a new customer. Solace has been selected as part of a platform designed to enhance nuclear threat monitoring and response capabilities of local, state and federal emergency organizations.

Solace’s high throughput, content-aware messaging protocols effectively play matchmaker between the streams of data coming in from the sensors or other sources and the thousands, or even millions, of rules that define what to look for. It’s the size and scale of problem that is custom made for hardware middleware.

The Sky is the Limit

Today’s announcement focuses on emergency response at DHS/DNDO, but this same approach and architecture has virtually endless uses. For example:

• Environmental monitoring and study
• Command and control
• Manufacturing automation and controls
• RFID
• Smart grids
• Advanced surveillance and monitoring
• Home health monitoring
• Twitter or Facebook updates
• GPS data from cell phones

In all of these use cases, data starts out life as a stream of updates from hundreds or millions of endpoints flowing into a network. In each case, the difference between knowing what is happening now, and knowing what happened a while ago can be huge.

Some pieces of infrastructure – particularly those that are part of the network – are so critical (and so very underappreciated) – that they simply cannot be exposed to the kind of risk that comes from “sharing” resources in any model.

I can’t say her never ever kind of experience matches with what we have seen with hardware middleware. When it comes to sharing, our customers fall into two main camps:

• Those that re-buy gear for each major business unit because of concerns over traffic from one interfering with another. This is how they have always deployed software, and they’re not ready to do it differently in hardware. It has more to do with CYA than technology, which is consistent with Lori’s point.
• Those that are blending market data for many asset classes (across departments), and/or a combination of market data, order routing and risk management in the same infrastructure. Generally the motivation is reducing costs, but it can also reduce risks when the shared environment is both faster and more stable than the standalone software it replaces.

When people start to think of hardware middleware infrastructure as similar to their IP network infrastructure (which they almost always share across divisions and applications), many of the walls put up during the software era come down pretty quickly. The biggest reason people hesitate to share is, as Lori points out, fear of figuring out what happened when there is a problem or outage. The reality is that hardware infrastructure provides more visibility, not less, since stats tracking and logging are done in parallel without impacting performance. This level of visibility (which they are NOT getting in software) is key to establishing confidence that consolidating middleware does not increase risk.

I agree with Lori that many firms are not ready to hand over critical functions to the cloud where what is happening is truly smushy, but when the firm owns the specialized hardware, and can confidently manage traffic flow, sharing infrastructure is increasingly happening today.

We are delighted to add BCC Group to our growing list of partners.

Check out the bouncing eaves in this second one

Enjoy the holiday break, see you in January.

“A misunderstood dynamic of high-frequency trading is that it thrives off volatility, thereby reducing it. The clear winners in the revolution are small investors, who have seen their trading costs fall remarkably and markets price shares far more efficiently.“

The article also highlights the degree to which these firms keep tabs on and try to one-up each other:

“Lotus Capital Management LP of New York earlier this year realized that a competitor was beating it to a trade it had programmed by exactly 3 microseconds, day after day. The loss meant Lotus was forfeiting about $1,000 in daily revenue on that particular trading strategy. Lotus, a quantitative trading firm that uses high-frequency strategies, invested and tinkered, eventually shaving five microseconds from the router and two microseconds from the execution server.”

I don’t know if that particular story is true or folklore, but it underscored the effort that goes into being in the pole position for a given trading strategy.