ETALAB (France), data.gov.uk (UK), data.dc.gov (Washington, DC, US), whitehouse.gov/open (US), and countless other local and national governments have open their data coffers.  In the case of DC for instance, the cost of publishing the data was $50K for the city. The DC government expected it to spur the creation of a few new ventures, and a bit of private investments.  Instead, 50 startups were born and $3M invested.  There is a world of open data coming to the private software industry.

Open Government data is also going to be Big Data.  The size of data collected is by definition larger larger than traditional “enterprise data” for instance (especially at the national level).  The tools being developed for big data will solve some of the issues with access and real time analytics that exit with government data.  Exorbyte MatchMaker is one of these tools.  That’s why government agencies have already chosen MatchMaker for their search and data access challenges (2 national European census agencies, German Finance Ministry, and more).

Are you ready for open government data?  Any ideas what would make sense to build with this data?

IBM, Oracle, SAP, Microsoft, SalesForce.com, and others are all aiming their development efforts at Big Data (see vldb.org).  The amount of data produced, collected and stored by online activities to which companies, their customers, their partners, and their sales channels participate has grown enormously.  Tools are being developed that allow affordable long-term storage.  New columnar in-memory database formats have emerged that enable near-real-time analytics.  Fast growing stratups and open source solutions have also converged with their own new NoSQL formats (InfiniDB, LucidDB, InfoBright, Hadoop, NoSQL, etc.)     MatchMaker, Exoryte’s Universal Search platform, is the perfect answer to search within Big Data.

The challenges of search within big data are:

• Searching Big Data though SQL queries is simply too slow and inflexible – fuzzy or advanced search requires a search indexer layer or  something different than traditional on-disk relational DB formats.
• Indexing large databases can be long, disruptive to normal database operation and require complex hardware infrastructures.
• Running complex queries and fuzzy logic requires so much calculation and lookups that new search strategies are required.

Exorbyte MatchMaker is made to address these challenges and our professional services team has proven repeatedly tht they can be addressed:  Allianz (the world’s 12th-largest financial services group),  German Finance Ministry, and more blue chip and government organizations tun o us each year for that very expertise.

What do you think of Big Data?

The potential is huge with Magento already in the lead of independent ecommerce platforms.  The power of eBay’s marketing engine and PayPal’s payment platform should ensue some interesting contender for Yahoo! Store and other integrated platforms (Volusion, Amazon, etc.) .  Watch the X.Commerce Innovate Conference this week for the latest!

x.commerce mission statement

“10% of our queries are misspelled”.  We often see this going up in te 20-30% range on online shops.

So if anyone out there still doubts that string matching algorithmic fuzzy search is a nice-to-have in site search, I challenge you.  Yes indeed, by nature site search or ecommerce shop search has two problems when exact matching or low grade fuzzy methods (stemming, etc.) are used:

1. Greater chance of returning zero results with misspellings.  Google almost always has something to return even with exact matching on misspelling.  It’s returning results from among billions of web pages after all.
2. Low success rates in search overall:  Misspellings will invariably miss their target even if they sometimes match on stemming or misspelled content here and there.

At Exorbyte, we have the very best structured data error-tolerance you can find anywhere.  We have beaten some of the biggest names in the business in enterprise search purchase.  If you are about CRO (conversion rate optimization), you need error-tolerance and your best option is Exorbyte.

Steve Jobs Flew Away

http://www.apple.com/stevejobs/

See more here:

http://www.businessinsider.com/oracle-hp-autonomy-2011-9

http://www.marketwatch.com/story/oracle-fuels-fire-over-h-ps-autonomy-deal-2011-09-29

http://articles.businessinsider.com/2011-09-29/tech/30216980_1_oracle-qatalyst-slides

==== 10/18/2011 – Update ====

http://www.forbes.com/sites/ericsavitz/2011/10/18/oracle-buys-endeca-targeting-unstructured-data/ That’s it.  Oracle announces they are buying Endeca.  This is big news.  Another confirmation for us that Search is a hot category for enterprise software but also that deeper integration of search is in the works.  While this acquisition of may be a good thing for those Endeca users with full-text and unstructured  data applications, other with massive structured data (ecommerce, CRM, etc.) will need to turn to other solution providers possibly.  We are ready and waiting.

“Andreessen kicked off BoxWorks, the first-ever customer conference for cloud collaboration provider Box.net, this morning in San Francisco.

His firm invested in Box, he says, partly because he found that a lot of the other startups they were funding already used Box’s product.

As he put it, “Ten years ago, it was a joke: you’d raise $20 million in venture capital and write a $4 or $5 million check to Oracle, Sun, BEA, and EMC….When it started, Salesforce looked like a toy compared with Siebel. Look ahead five years later, it’s obviously better. Not a single one of our startups uses Oracle.”

Read more: Marc Andreessen: The “Clock Is Ticking” On Oracle - http://www.businessinsider.com/boxnet-2011-9#comment-4e84d76aecad04784000002d#ixzz1ZNOjydsW

Oracle is indeed under siege big time. But this cloud computing debacle is just one more front for them to fight on. SAP, their main rival in the DB business, has been taking effective swings at them regarding the lack of believe they have in in-memory databases (which are by-the-way the way forward for cloud computing also). Hasso Plattner from SAP has a field day here with this:

Our company sells SaaS site search for ecommerce and we also did away with Oracle from the start. Better yet, the vast majority of our customers are the same and this is a growing market!

Good luck Larry!

BUSTED: Oracle Publishes Slide Deck To Prove Autonomy CEO Is Lying
Read more: http://www.businessinsider.com/oracle-hp-autonomy-2011-9#ixzz1ZNOVjCVb

9/29/2011 – 2:16PM  - I will just add that no matter what Autonomy and HP say to justify that acquisition, I just can’t see it why they ended up above $10B, especially after looking at these slides.  Good luck.