The was plenty of market data cloud buzz at the 2010 SIFMA Technology Conference and Exhibition with NASDAQ and Xignite right in the center. Using the XigniteOnDemand Market Data Cloud Platform, NASDAQ plans to launch Data-on-Demand in the second half of 2010 to provide easy and flexible access to large amounts of detailed historical NASDAQ Level 1 trade and quote data for all U.S.-listed securities. Tick data is increasingly in demand for back testing of algorithmic trading strategies as the securities industry pushes the limits of high frequency trading.

Randall Hopkins, NASDAQ OMX’s Senior Vice President of Global Data Products, is quoted in the press release as saying: “Today our customers spend a large amount on technology infrastructure, not the market data itself. With Data-on-Demand, we want to drastically cut data management costs by running the technology infrastructure on the cloud for our clients and delivering to them the data they need, when they need it, and how they need it.”

NASDAQ discusses it’s market data cloud initiative with Max Bowie
of Inside Market Data at the 2010 SIFMA Financial Services Technology Expo

Obtaining and collecting tick data can be onerous and time-consuming as firms are required to establish feeds and maintain large amounts of data on-hand. On-demand market data distribution gives applications a way to cherry pick the specific subset of data that the application needs with pinpoint accuracy. Instead of combing through very large data sets of historical tick data, developers will be able to program their applications to select very specific data sets and obtain them on-demand and process them instantly. Data-on-Demand will also allow clients to download large tick data subsets on a scheduled basis.

We’re proud to announce that CME Group, the world’s leading and most diverse derivatives marketplace, is jumping on the cloud computing bandwagon and has agreed to use the XigniteOnDemand market data cloud platform to deliver OTC data from the cloud.

Brian McElligott, Managing Director of Information Products at CME Group is quoted in the press release as saying “With the ongoing development of our multiple OTC product and service offerings, and the need to deliver this data to customers with continued market transparency, the Xignite platform will be a quick and cost-effective way to get OTC pricing and reference data to our global market participants.”

Pete Harris of the A-Team Group interviews CME Group, BG Cantor and Xignite.
The A-Team Group gave the cloud prime coverage in their recent Q2 issue,
and Pete is launching a new “Market Data Cloud” channel, so stay tuned to that.

CME Group plans to offer on-demand access to end-of-day OTC settlement, volume and open interest data to support markets available through CME ClearPort®, a set of flexible clearing services open to OTC market participants to substantially mitigate counterparty risk and provide neutral settlement prices across asset classes. CME ClearPort currently clears more than 500,000 contracts daily, including Credit Default Swaps (CDS), energy, metals, agricultural commodities and foreign currencies. The service brings together more than 10,000 global users across the world including banks, hedge funds, trading entities, Inter Dealer Brokers (IDBs), Future Commission Merchants (FCMs), and clearing firms.

Henry Ford once said: “Any customer can have a car painted any color that he wants, so long as it is black.” Then, in 1923 Alfred Sloan came along and cleaned his clock by offering a tremendous variety in colors and models. But, Sloan didn’t do it one customer at a time. GM redesigned its manufacturing line with the flexibility to produce a multitude of models and colors without compromising the inherent economies-of-scale of Ford’s assembly line innovation—a practice that today has evolved into the concepts of flexible manufacturing and mass customization.

What does any of this have to do with cloud computing? And for that matter, what does it have to do with financial market data? This is the third post in a series called “Thinking Out Cloud” with the aim of helping financial services and market data IT professionals charged with developing cloud computing strategies separate the cloud buzz from the cloud reality. This post explores the important idea of mass customization in the cloud and its relevance to market data management.

Mass Customization and Market Data

According to Wikipedia, mass customization is about providing “services to meet individual customer’s needs with near mass production efficiency” by offering a “tremendous increase in variety and customization without a corresponding increase in costs.” Or rather, have it your way at a cost you can afford. In the last installment in this series entitled Thinking Out Cloud – The Market Data Sweet Spot, I pointed out that the best way to identify market data that can benefit from cloud computing is to look for data that is hard-to-use, hard-to-manage, and hard-to-access. Mass customization on the cloud strikes at the core of the “hard-to-use” problem by replacing inflexible data feeds and files in static formats with financial Web services that allow applications to select and consume highly customized market data sets on the fly.

Mass Customization, Meta Data and Web Services

In software, mass customization is achieved by converting hard-coded application functions into meta-data, so that the functional behavior of the software can be controlled through data on an ad-hoc basis. For example, the only real difference between this Xignite Wordpress blog and a million other Wordpress blogs comes down to a few theme files and the store of data that comprises the brilliant blog posts within. The posts in turn get shuffled around the Internet using RSS (really simple syndication). The RSS standard lies at the heart of the Web 2.0 revolution, powering the distribution of widely varied content such as blogs, news, friends, tweets, music, video, etc., all because a few abstract meta-data elements like “title”, “description”, and “link” can be generalized to contain an enormous and disparate variety of data. Swapping out this data can literally change one blog to any other blog (or one friend to any other friend!).

In cloud computing, meta-data abstraction is realized through Web services (like RSS). Web services take the concept of configurability through meta-data abstraction to its natural limit, because every operation of a Web service is inherently abstracted to meta-data in the XML inputs and outputs of the API. For example, the Xignite delayed stock quote Web service allows applications to retrieve a single quote for a single symbol, multiple quotes for multiple symbols, intraday tick-by-tick prices, market volume and news, simply by varying the Web service operation and the input parameters, i.e., the meta data. Providing access to such a wide variety of information in an ad-hoc fashion from a statically formatted feed or file is simply not possible without lots of hardware, software and custom development–most likely to create a Web service.

Wrap it Up, I’ll Take It!

More plainly, market data Web services are built for mass customization, because they cleanly wrap the underlying market data and data management infrastructure in meta-data enabling highly customized output based on variable input. Therefore, applications that require highly varied, ad-hoc market data subsets will be better served by Web services over the static formats of data feeds and files. Or, as we like to say at Xignite: “There’s an op[eration] for that!”

If you look at our new corporate bond master service, XigniteBondMaster, you’ll quickly see that you can use it to get information for over 125,000 U.S. corporate bonds, including issuer information, maturity dates, coupon rates and so on.  But you’re not likely to see one of the best features of the service until you roll up your sleeves and start digging around in the data:  the ability to look up official CUSIP identifiers with remarkable ease.

CUSIP identifiers are highly valued throughout the industry as the de facto standard for uniquely identifying over 9.1 million financial instruments.  And now you can easily look up CUSIPs for thousands of corporate bonds using XigniteBondMaster.  Interested in a particular issue, but not sure what its CUSIP is?  Try looking it up by the bond issuer!  Or just look it up by bond name.

All CUSIP data comes from CUSIP Global Services, an organization managed by Standard & Poor’s on behalf of the American Bankers Association specifically to oversee the official set of CUSIP identifiers.

You do need to have an agreement in place with CUSIP Global Services in order to receive CUSIP data, since this is highly valued data that must be licensed first.  But once you have an agreement in place, CUSIPs will automatically start appearing in your XigniteBondMaster outputs.

It’s a rare day when stock traders on Wall Street, can hear a bell struck all the way on the West Coast. Well, it happened today, when NASDAQ’s opening and closing bells rang from the heart of the Silicon Valley. Today’s ceremonies celebrated NASDAQ’s 20-year history in the valley, and was attended by NASDAQ CEO Robert Greifeld, NASDAQ EVP Bruce Aust, San Jose Mayor Chuck Reed, the San Jose City Council, local business executives, national press affiliates and myself.

While I have always supported the Silicon Valley as a hub of global innovation, it was surprising to learn that there are currently 513 NASDAQ listed companies in California, with 210 of them in the Silicon Valley.  I was amazed that this little sliver of the world has an impressive $1 trillion USD in combined market capitalization. With these facts fresh in my mind, I ran back to the office to dig into our web services and learn more about these companies.

Being a 30 year resident of the valley, even I was surprised by the high number of NASDAQ listed companies within 20 minutes of my office; Adobe Systems (ADBE -0.47), Apple (AAPL +0.71), Applied Materials (AMAT +0.22), Cisco (CSCO +0.07), eBay (EBAY +0.51), Electronic Arts (ERTS +0.28), Google (GOOG +2.02), Intuit (INTU +0.01), LAM Research (LRCX +1.06), SanDisk (SNDK +0.36), Symantec (SYMC -0.01), Yahoo (YHOO -0.0608) and 200 other wonderful Silicon Valley companies.

While conducting this impromptu research, I was reminded how quick it is to obtain exactly the data sets that I needed, and how much I love the ease and simplicity of financial web services. Overall, it was a good day for local corporations, a wonderful day for NASDAQ, and a sign that the Silicon Valley will continue to be a hub of technical innovation for decades to come.

Closing Bell Gallery (by Xignite): HERE
Closing Bell Video (by NASDAQ): HERE
Stock Data Sourced From: Xignite NASDAQLastSale

It takes 5 minutes to beat cream into butter, write half of your name in Elvish, or create a homemade tattoo gun.  Or if you want to get really wild, you could spend your next 5 minutes creating a Windows application to get real-time precious metal quotes.  If that’s a little too wild for you, you could also live vicariously through us by watching a video of us doing it.  But if you’re ready to throw caution to the wind and give it a try for yourself, here’s how:

Step 1: Start a new Windows application project in Visual Studio using VB.NET as the language.  (You can use any language, but this example just happens to use VB.NET.)

Step 2: Create a web reference to http://www.xignite.com/xMetals.asmx?WSDL and call it XigniteMetals.

Step 3: On the Form1 design, drag and drop 2 text boxes, 3 labels, and a button, arranging them as shown in the picture.  Give the left-most text box a name of tbMetal and default text of XAU.  Give the right-most text box a name of tbCurrency and a default text of USD.  From left to right, the labels should be given the following names and default text values:  lIntro defaults to “Right now ”, lCosts defaults to “ costs ”,  PriceLabel defaults to “0.00.”  The button should have a default text value of “Get Quote.”

Step 4: Double-click the button to create its click event handler.  Then add the following lines of code inside the event handler:

Dim xHeader As New XigniteMetals.Header()

xHeader.Username = “MYAUTHENTICATIONKEY”

Dim XigniteMetalsWebService As New XigniteMetals.XigniteMetals()

XigniteMetalsWebService.HeaderValue = xHeader

Dim Quote As XigniteMetals.MetalQuote = _

XigniteMetalsWebService.GetRealTimeMetalQuote(_

DirectCast([Enum].Parse(GetType(XigniteMetals.MetalTypes), tbMetal.Text), XigniteMetals.MetalTypes), _

DirectCast([Enum].Parse(GetType(XigniteMetals.MetalCurrencyTypes), tbCurrency.Text), XigniteMetals.MetalCurrencyTypes))

PriceLabel.Text = Quote.Rate.ToString(“#,###.00″)

Step 5: Replace “MYAUTHENTICATIONKEY” in the second line of code with your Xignite authentication key.  Don’t have one?  Just sign up for a free trial and use your e-mail address as your initial authentication key.

Step 6: Compile and run.

Congratulations! You might make a lousy tattooed elf chef, but you’ll be the life of the party with precious metal traders.

In the next 5 minutes, you could get $5 worth of plumbing services, $30 worth of legal advice or $17 worth of psychotherapy. Or, you could just spend your next five minutes building a real-time stock quote widget. Don’t think it can be done in 5 minutes? Check out a video showing us doing it in less time.

Here’s how you can do the same thing using PHP and Eclipse, although the same basic process can be used for any language and any IDE:

Step 1: Subscribe to XigniteRealTime to get access to real-time, Level I stock quotes. If you’re not ready to subscribe yet, you can also sign up for a free trial and get access to other real-time stock quote web services such as XigniteNASDAQLastSale and XigniteBATSLastSale. You’ll need to make a few adjustments to the sample code below to point to a different web service, but nothing major.

Step 2: Start a new PHP project in Eclipse.

Step 3: Create your index.php page with the following code:

<html><title>Get Real-Time Quote</title><body><form>

<input type=“text” name=“symbol” />

<input type=“submit” value=“Go” />

</form></body></html>

<?php

$client = new SoapClient(‘http://www.xignite.com/xRealTime.asmx?WSDL’);

$quote = $client->GetRealQuote(array(‘Exchange’ => “INET”, ‘Symbol’ => $_GET["Symbol"], ‘IncludeBidAsk’ => “false”));

echo “Last price: “, $quote->GetRealQuoteResult->Last;

?>

Step 4: Open a browser and load your index.php page to give your new stock quote widget a spin.

Congratulations! You just built a real-time stock quote widget in PHP and made a plumber, a lawyer and a psychotherapist very disappointed in you for such a ridiculous waste of your time.

Market data providers and IT professionals have tough jobs. Every day financial markets spew out huge fountains of data that must be captured, routed, scrubbed, reconciled, stored and redistributed with dizzying speed and accuracy. The diversity of data is staggering, from low-latency pricing data for algorithmic trading to intermittent corporate actions such as stock splits, and from globally dispersed real-time currency exchange rates to aggregated end-of-day VWAP and NAV calculations. Optimizing and tuning the market data systems that keep this crucial information flowing smoothly and cost effectively is no easy task. What, if anything, can cloud computing offer to ease the challenge?

This is the second post in a series called “Thinking Out Cloud” with the aim of helping financial services and market data IT professionals charged with developing cloud computing strategies separate the cloud buzz from the cloud reality. This post explores the types of market data that naturally lend themselves to cloud computing (and those that do not) in order to identify the market data sweet spot for cloud computing.

First, it’s important to recognize that it is not the market data that is being outsourced to the cloud, but the on-premise market data management infrastructure. Therefore, the market data sweet spot for cloud computing is the intersection of data management systems that offer little competitive advantage, but are costly and difficult to maintain in-house. Both relative competitive advantage and relative level of difficulty will vary from firm to firm by business focus and IT capability respectively; however, there are aspects of the market data itself that can contribute significantly to the cost and complexity of maintaining an in-house data management system.

Hard to Use Market Data

If the market data comes in original feed formats that are not well suited to the particular use of the data in the final application, then considerable effort must be expended to make the market data application-ready. For example, a real-time streaming exchange feed is great for creating a stock ticker, but not so great when the goal is to analyze historical tick data or simply get an ad-hoc real-time quote for a single symbol, then there can be lots of programming involved to parse the feed, store the data, continuously refresh the database and create a data access layer that applications can easily utilize. Cloud computing is built on Web services that allow for multiple interfaces to the market data, so it is especially good at tailoring the data format to the specific needs of the application on the fly. For example, a Web service request can be for a single price, multiple prices, or simply for symbol validation against master data.

Hard to Maintain Market Data

If the market data in question is stored and refreshed often due to daily activity, such as historical time series and tick-by-tick data, then it can entail a complex update process that must be maintained and monitored. Quality testing must be put in place to ensure data quality and alerts to ensure that update processes run successfully to completion. As market data accumulates, regular backups, purges and capacity upgrades must be carried out to ensure efficient operation. Also, market data that is particularly complex, such as corporate actions, can consume significant resources scrubbing, mapping and updating the data even if the volume is not as heavy as price data.

On the other hand, market data that is infrequently updated in large batches that replace the entire data set may be easier to receive and maintain internally as a simple flat file. Similarly, market data that is streamed for continuous presentation and immediately discarded should be relatively easy to handle in-house.

Hard to Access Market Data

Technical and geographic barriers can conspire to make certain kinds of market data extremely difficult to access, let alone store and use. Getting access to market data that is not in high demand in your geographic location can be very difficult when local data providers do not support it, such as low volume market niches and new products, or market data that is created on the other side of the world. If data needs to be made available throughout a dispersed global organization, it can be quite costly to receive it centrally and build the necessary network and services infrastructure to distribute it globally to consuming applications. Cloud computing provides direct application access to market data over the Internet, so both the geographic and technical barriers to access are significantly reduced. Web services standards ensure that the technical hurdle is very low, so as long as sufficient Internet bandwidth is available, access is no more difficult than pointing a Web browser to a Web page.

The Cloud Computing Sour Spot for Market Data

If hard to access, hard to maintain, and hard to use define the market data cloud computing sweet spot, then what is the sour spot? When boiled down to its essence, all of the “hard to’s” above are made “easy to’s” by the Internet. So, the sour spot is found when the Internet has a significant negative impact on market data delivery. The three biggest limitations, in most likely order of importance are…

Too Risky

The market data is unique and proprietary with significant competitive, security or privacy concerns that preclude storing or distributing it using a public network, even with encryption.

Too Fast

The latency requirements of the data are very strict and have a low tolerance for variation. Typically less than about 50ms on average which is about the best that can be consistently expected from a high performance Internet connection. In addition, Internet latency can vary significantly depending on the number of network hops between the cloud provider and the consuming application. This number improves every year (back in 2000 we were talking seconds). But it is safe to say that latency requirements measured in microseconds or nanoseconds won’t be a good fit for the cloud.

Too Fat

The market data volume involved implies a network transfer time that exceeds the requirements of the consuming application given the best expected performance over the Internet. This may be alleviated by getting a direct network link to the cloud provider if the cost of the additional bandwidth is justified by the overall cost savings from outsourcing the relevant data management infrastructure. In some cases, this can also be resolved by moving the application itself to the same cloud computing infrastructure. However, an in-house application that requires frequent updates with large volumes of data can quickly exceed the best performance currently available over the Internet.

Cloud computing has graduated from technical curiosity to technology trend. According to a recent Gartner survey, cloud computing and virtualization sit squarely at the top of the list of 2010 CIO priorities. However, most financial services IT professionals are still taking a cautious approach to cloud computing. This is quite understandable given the high performance, security and compliance requirements of financial applications. Unlike Twitter, you can’t just post the fail whale when a financial application has an issue, because real money is at stake not just tweets. So, what financial applications are good candidates for cloud computing and why? To cloud or not to cloud? That is the question.

This is the first post in an series entitled Thinking Out Cloud with the aim of helping financial services IT and market data professionals charged with developing cloud computing strategies separate the cloud buzz from the cloud reality.

Focus on Competitive Advantage

Realistically, somewhere between 75% and 95% of the software used by most businesses today is a commodity. That is, it offers no competitive advantage because the competition has access to the exact same capabilities from a plethora of software vendors, SaaS vendors, open-source projects and home-grown systems. This means that if your financial services IT department does everything in-house, pretty much 75%-95% of your IT department’s hard work and budgetary expense offers a competitive advantage of exactly zero. Only the 5%-25% spent on systems that help your business stand out from the competition make a real difference to top line revenue. Yet, I’ve never seen a financial services IT department that had extra time on its hands. Therefore, the first consideration in any analysis of cloud computing should be to weigh the competitive importance of the various systems under IT management, and to look for opportunities to free up resources by pushing systems that do not offer competitive advantage to the cloud.

Leverage Cloud Computing to Lower Costs

The primary financial benefit of cloud computing is lower total cost of ownership created through the economies-of-scale of shared computing resources. It makes no difference if it is infrastructure from Amzaon AWS, CRM from Salesforce.com, or market data from Xignite, the underlying common thread is that these vendors can do it significantly cheaper by the dozen than customers can do it in-house one at a time. This cost savings is then passed onto the customer in the form of an ongoing usage-based subscription. Conveniently, this prospect of lower overall costs fits nicely with the idea of outsourcing commodity systems. Therefore, the next consideration in an analysis of the benefits of cloud computing for a particular system should be the relative costs savings. It is critical at this point to compare the total cost of ownership of each internal system to a cloud vendor’s subscription pricing, not just the cost of the software and hardware. More often than not, the largest costs are hidden in the IT staff time that the systems absorb in the form of ongoing maintenance, upgrade projects, support and bug fixing, etc. Since these activities are taken on by the cloud vendor, they should be included to ensure an apples-to-apples comparison in the on-premise vs. cloud decision.

Reduce Operational Risk or at Least Stay Risk Neutral

Risk is easily the biggest barrier to cloud adoption for financial services applications. Financial risk, regulatory risk, security risk, performance risk…risk, risk, risk. The risk implications of cloud computing often appear so daunting that some financial services IT professionals would prefer to avoid the cloud entirely—which is a shame. To be sure, putting customer investment account information on the cloud is a non-starter. And, even something as simple as company email entails significant compliance risks. However, the one mistake most IT departments make when evaluating the cloud is to equate risk reduction with internal control. It is often the case that the focused expertise and scale of a cloud computing vendor provides significantly lower risk than an in-house system. This gut reaction can be compared to fear of flying: just because you’d feel safer if you were flying the plane, doesn’t mean you’d actually be safer. Therefore, it is important to evaluate the risks of cloud computing as objectively as possible. Does the vendor publish an SLA and public performance statistics? Who are its current customers and what do they say about the vendor? What are the vendor’s security and disaster recovery policies? What kind of support is available? All these questions should be asked of a cloud vendor. And, all of these questions should be asked of the comparable internal IT operation. If you’re not a pilot, you might just be better off taking a commercial flight.

Market Data: To Cloud or Not to Cloud?

There are two important aspects of market data systems to consider when evaluating the potential for moving them to the cloud. The first is to recognize that it is not the data that is being outsourced with cloud computing, but the data management infrastructure. In most cases, the data is already outsourced in the form of a traditional data feed, so the on-premise vs. cloud comparison actually centers around the internal software, hardware, databases, development and administrative IT staff required to maintain the internal data management infrastructure that shuffles the data from a feed to a final application. The second aspect is that unlike investment account data, market data is not about the company or its customers, and thereby sidesteps the number one barrier to cloud computing adoption in financial services: data security. But, it doesn’t make it a slam dunk. It is still important to carry out a full evaluation along the dimensions of competitive advantage focus, cost reduction and risk reduction. Does the data management system in question create competitive advantage or simply maintain competitive parity? Does the cloud offer significant cost reduction over the total cost of internal data management systems? And finally, does the cloud offer neutral to lower risk given the nature of the market data and the relative sophistication of the cloud vendor’s platform vis-a-vis on-premise operations.

Every five minutes, a business in the U.S. catches fire, someone in the UK swears, and a clock that took 5 years and £1 million to build before being unveiled by none other than Stephen Hawking, finally tells the time accurately for a second or two.

What will you do with your next 5 minutes?

Here’s a thought: Why not build a real-time forex currency converter? Sound like fun? If so, give it a try using the steps below. Or, if you’d rather leave all the fun to someone else, why not just watch a video of us doing it?

Here’s how it’s done:

Step 1: Start a new Windows application project in Visual Studio.

Step 2: Create a Web reference to http://www.xignite.com/xCurrencies.asmx?WSDL and call it XigniteCurrenciesService.

Step 3: On the Form1 design, drag and drop 3 text boxes, 2 labels, and a button, arranging them as shown in the picture. From left to right, give the text boxes the following names and default text: inputBaseAmount defaulting to 1000, inputBaseSymbol defaulting to USD, inputQuoteSymbol defaulting to EUR. One label should have a big equal sign in it and the other should have the name of labelQuoteAmount and zero for its text property. The button should read “Convert.”

Step 4: Double-click the button to create its click event handler, and give it the following 3 lines of code:

XigniteCurrenciesService.XigniteCurrencies XigniteCurrenciesWebService = new XigniteCurrenciesService.XigniteCurrencies();

XigniteCurrenciesService.ExchangeConversion Quote = XigniteCurrenciesWebService.ConvertRealTimeValue(inputBaseSymbol.Text, inputQuoteSymbol.Text, double.Parse(inputBaseAmount.Text));

labelQuoteAmount.Text = Quote.Result.ToString();

Step 5: Compile and run.

Authentication trouble? If you get an authentication error, it means you’re not being authenticated from a cookie or your IP address. Don’t worry. Just add 3 more lines after the first line of code to pass your authentication key through SOAP. Be sure to replace MYAUTHENTICATIONKEY with your own Xignite account authentication key. (Don’t have an authentication key? Just sign up to get one. Your e-mail will be your initial authentication key.)

XigniteCurrenciesService.Header objHeader = new XigniteCurrenciesService.Header();

objHeader.Username = “MYAUTHENTICATIONKEY”;

XigniteCurrenciesWebService.HeaderValue = objHeader;

Congratulations! You just spent the last five minutes far more productively than one U.S. business, one citizen of the UK, and one very expensive but very inaccurate clock.