However, a Twitter check revealed that Tony Bain is familiar with Groovy Corp and the Groovy SQL Switch (apparently they started out in Australia, where he lives and works, and he evidently knows the guys).  Tony’s take, in summary, is (emphasis mine):

They are an in memory RDBMS

They have worked with Intel to architect from the ground up for large multi processor concurrency

Initially they are launching as a multi-core appliance

They claim 200,000 sql operations per second from a single box

They are proprietary (not built on MySQL or any other open source database) which means they have had a lot of control around their architecture

They are a pretty cool company with some interesting people

There’s a little more detail at the above link.

• 20 names of actual customers.
• One quote citing a competitive win (over Netezza)
• One quote citing a ~50X speedup of one query “without manual tuning”
• One quote citing consistent 10-72X query performance speedups
• One quote citing a speedup from “days” to “minutes”

Unless I missed it, none of the quotes implied Exadata was actually in production, and none compared hardware between the old/slow/production and Exadata/fast/test systems.

Well, guess what?  In a throwaway line in a comment on Daniel Abadi’s blog, Barry Zane of ParAccel pointed out

we posted a load rate of almost 9TB/hour, which is, of course record breaking on its own

Quite right.

I hope the nonsense stops there, but I’m not optimistic …

Barry Zane of ParAccel has — finally! — started a blog.  Barrry’s first post, probably in connection with ParAccel’s recent TPC-H submission and subsequent brouhaha, consisted mainly of metaphor + very elementary and well-known arguments for column stores. Barry’s second post, however, was in direct response to Daniel Abadi’s speculation about ParAccel’s architecture.  That post also promises a follow-up addressing the TPC-H in a more substantive way.

Barry’s points include:

• ParAccel never used the row-oriented Postgres execution engine. This is contrary to Daniel’s speculation.
• ParAccel previously used an adaption of the Postgres cost-based optimizer, but now has written a new one from scratch.
• ParAccel has designed its optimizer to handle lots and lots of joins. One reason Barry offers is that ParAccel wants to run customers’ old schemas unaltered, whether or not those are really optimal for the ParAccel DBMS.  That approach is somewhat in contrast to Vertica, which originally focused entirely on star schemas.   And it goes well with ParAccel’s interest in appealing to customers who at least think they want to run ParAccel in Oracle or SQL Server emulation mode.

Also in the post, Barry:

• Makes an extremely silly marketing exaggeration by referring to ” the only other vendor that was able to run the 30TB TPC-H” (emphasis mine).
• Makes the more excusable marketing exaggeration “Publishing the benchmark with unmatched performance is simply one way to demonstrate robustness and flexibility.  Nothing more, nothing less.”
• Makes the very clear marketing claim “For customers, the real test will be their own bake-offs, where our performance has never been beaten.” (Emphasis mine.) That last one directly contradicts what I’ve been told by at least two ParAccel competitors, so I’ll be curious to see what they come up with to substantiate their version of the story.

Anyhow, it’s great to see ParAccel retreating from its obsessive secrecy, which in my opinion has been even worse than Netezza’s used to be.

*Thanks to Dave Kellogg for tipping me off to Plattner’s paper. I only went to two SIGMOD sessions, neither of which was Plattner’s. Nobody actually mentioned Plattner’s talk to me when I was down at SIGMOD.

Perhaps the most interesting part is Plattner’s claim that what’s demanding about OLTP isn’t database updating per se, but rather maintaining aggregates for quick-response analytics. In his main example of that point, Plattner proposes a real-life “more than 18″ table schema, of which 2 are base tables, and (most of?) the rest are materialized views that his proposed database architecture dispenses with (because analytic performance is sufficiently good without them). Thus, Plattner’s core columnar argument seemingly is

columnar –> natively fast analytics –> no need to maintain aggregates –> much lower update burden.

That said — if Plattner’s paper contained a clear statement of how much more expensive it is to insert or update a single row in a columnar vs. row-based system, I overlooked it. Instead, Plattner seems to be arguing that the volume of base-table updates is low enough that — whatever it may be — column-store update overhead is an acceptable price to pay.  (At one point he claims that only 5% of the data inserted in a financial application ever gets changed.) That may actually be true in a financial accounting system, but seems more questionable in a sufficiently large application that gets its updates from automatic devices, or from the consumer web.

Other highlights include:

• Like most modern observers, Plattner believes Postgres-style timestamping beats update-in-place.
• Plattner also offers a less common reason for liking timestamped inserts over updates-in-place — he thinks timestamps are helpful in planning-oriented applications. In particular, he wants timestamp-aware SQL extensions.
• Plattner claims columnar designs have a 10:1 compression advantage over row stores — specifically 20X vs. 2X — at least using compression schemes that allow for updating at reasonable speed. That seems exaggerated.
• Plattner seemed to drop various references to memory-centric structures SAP already uses. (SAP has long done a lot in-memory, in both the OLTP and planning areas. Years ago SAP told me of a customer that was buying >1 TB of RAM just to run SAP’s planning software. SAP also bragged that >99% of transactions never hit disk, in some sense of “transaction”. )
• There are lots of references to “tenants”, SaaS, and/or SAP’s SaaS product line. So SaaS is evidently a design point. That makes sense. First, SaaS is one of SAP’s biggest vulnerabilities. Second, the toughest customization a SaaS customer might want is to add a few columns to standard tables, which might be easier to accomodate with a columnar approach.

But now Daniel Abadi has done a brilliant, detailed, speculative analysis of ParAccel’s publications.  Here’s the meat, emphasis mine:

(1) Why did they configure their TPC-H application with such a high amount of disk I/O throughput capabilty when they are a column-store? (Stonebraker’s question)
(2) Why did queries spend seemingly 6X more time doing I/O than a column-store should have to do?
(3) Why are they worried about queries with thousands of joins?
(4) Why do they think TPC-H/TPC-DS queries have 42 joins?

And then a theory that answers all four questions at the same time came to me. Perhaps ParAccel directly followed my advice (see option 1) on “How to create a new column-store DBMS product in a week“. They’re not a column-store. They’re a vertically partitioned row-store (this is how column-stores were built back in the 70s before we knew any better). Each column is stored in its own separate table inside the row-store (PostgreSQL in ParAccel’s case). Queries over the original schema are then automatically rewritten into queries over the vertically partitioned schema and the row-store’s regular query execution engine can be used unmodified. But now, every attribute accessed by the query now adds an additional join to the query plan (since the vertical partitions for each column in a table have to be joined together).

This immediately explains why they are worried about queries with hundreds to thousands of joins (questions 3 and 4). But it also explains why they seem to be doing much more I/O than a native column-store. Since each vertical partition is its own table, then each tuple in a vertical partition (which contains just one value) is preceded by the row-store’s tuple header. In PostgreSQL this tuple header is on the order of 27 bytes. So if the column width is 4 bytes, then there is a factor of 7 extra space used up for the tuple header relative to actual user data. And if the implementation is super naive, they also will need an additional 4 bytes to store a tuple identifier for joining vertical partitions from the same original table with each other. This answers questions 1 and 2, as the factor of 6 worse I/O efficiency is now obvious.

It will be interesting to see whether ParAccel comments, but even it does, I wouldn’t necessarily take ParAccel’s statements as dispositive.  For example — and illustrative of my view of ParAccel’s trustworthiness — I believe ParAccel’s competition who tell me that ParAccel’s claim to have won or at least tied all POCs on performance is flat-out untrue.

In a comment to his post on node scalability, Daniel Abadi argued that disk space is a better metric to use in marketing than (presumably compressed) user data.  Well, I imagine he didn’t quite mean to say that, but that’s actually what he wound up saying, starting from the accurate observation that compression ratios vary wildly from one data set to another, even more than they vary from product to product on the same data.

Nonetheless, I favor user data as a metric because:

• That’s what users care about.
• That’s how a number of analytic DBMS vendors, including Vertica, actually price.