2. Numerous vendors are blending SQL and JSON management in their short-request DBMS. It will take some more work for me to have a strong opinion about the merits/demerits of various alternatives.

The default implementation — one example would be Clustrix’s — is to stick the JSON into something like a BLOB/CLOB field (Binary/Character Large Object), index on individual values, and treat those indexes just like any others for the purpose of SQL statements. Drawbacks include:

• You have to store or retrieve the JSON in whole documents at a time.
• If you are spectacularly careless, you could write JOINs with odd results.

IBM DB2 is one recent arrival to the JSON party. Unfortunately, I forgot to ask whether IBM’s JSON implementation was based on IBM DB2 pureXML when I had the chance, and IBM hasn’t gotten around to answering my followup query.

3. Nor has IBM gotten around to answering my followup queries on the subject of BLU, an interesting-sounding columnar option for DB2.

4. Numerous clients have asked me whether they should be active in DBaaS (DataBase as a Service). After all, Amazon, Google, Microsoft, Rackspace and salesforce.com are all in that business in some form, and other big companies have dipped toes in as well.

I’m skeptical that one can succeed both in that market and in selling database software, for reasons including:

• Nobody I can think of has done so.
• The value propositions are different.
  • DBaaS is about having administration be so easy that you the customer doesn’t need to worry about it.
  • Database software is about one or more of:
    • Development ease.
    • Price/performance/throughput.
    • Big-enterprise/legacy-vendor considerations.

I’m also skeptical about service-only DBaaS strategies, because users will naturally resist vendor lock-in.

But despite all my skepticism, DBaaS is an area I should probably learn more about.

5. I plan to spend more time looking at machine learning and other advanced analytics. I doubt they’ll soon match the past few years’ hype about “big data analytics”, but even the reality of modern analytics looks like it’s getting more interesting. Ditto if somebody has an interesting twist on more traditional predictive analytics.

6. Three years ago,  I wrote:

• It is inevitable* that governments and other constituencies will obtain huge amounts of information, which can be used to drastically restrict everybody’s privacy and freedom.
• To protect against this grave threat, multiple layers of defense are needed, technical and legal/regulatory/social/political alike.
• One particular layer is getting insufficient attention, namely restrictions upon the use (as opposed to the acquisition or retention) of data.

*And indeed in many ways even desirable

It is now frighteningly obvious that the US is becoming a high-surveillance society. The Boston Marathon bombing added three new elements to an already snowballing trend:

• A revelation that the FBI could track Tamerlan Tsarnaev’s communication content without any known warrant.
• A further revelation that the police know how to put on large paramilitary displays of force (and that the public generally approves).
• An increased belief that widespread video surveillance of public places is a Good Thing.

I need to write more about privacy.

0. Decision makers are motivated by two emotions above all — fear and greed. In the case of enterprise IT, that equates roughly to saying they want to buy stuff that:

• Is safe.
• Will confer benefits.

1. For a marketing message to succeed, whatever its goals are, the “confer benefits” part of the story needs to be:

• Compelling
• Believed

2. The “safe” part needs to be believed too. Rational belief in the safety of doing business with you is good. Blind faith is even better, but usually is enjoyed only by the most established of vendors.

In some cases, that may be the greatest competitive strength they have.

3. To be believed, enterprise IT messaging generally needs to be:

• Credible
• Clear

A certain amount of exaggeration is expected, and easily shrugged off. It’s also possible to get away with a certain amount of vagueness, whether in a fear/safety story or when pitching something as new/innovative/exciting. But don’t overdo either.

One common way to overdo your exaggeration — make an obviously false claim of uniqueness.

4. Please note: Deficiencies in the consistency of your messages can undermine credibility and clarity alike.

5. Messaging can become distorted in many ways, both accidental and deliberate. For example:

• Your salespeople get a few hours of sales training per month. Then you send them out on sales calls. Do you really think every nuance of every message will be delivered perfectly every time?
• That’s largely accidental. But rival salespeople will distort your messages on purpose.
• So will rival marketers when talking with press, analysts, and other influencers.
• Even influencers who believe your story will abbreviate and distort it when passing it onward. Many lack the detailed domain knowledge to fully understand it and put it in context anyway.

Message transmission is lossy, or worse.

6. So how do you combat message loss? My top tactics are:

• Break your story into clear, simple parts.
• Make sure your story has ENOUGH parts.

If enough parts get through accurately, then perhaps the target will correctly reassemble the overall message.

7. As one would hope, the layered messaging model performs well by these criteria:

• Its whole point is to help you credibly assert benefits.
• It calls for you to make numerous different claims …
• … each of which can be individually simple.
• It enforces consistency among the different parts of your story.

8. To also help punch messages through the noise, I commonly emphasize that vendors should use multiple proof points.

Any one proof point can be dismissed or discounted. An impressive-sounding reference account could have gotten your product for free, or might have a CIO who’s buddies with your founder. A single impressive feature can be sort-of matched by a competitor’s kludgy alternative. But if you say that 10 Fortune 100 enterprises are using your product, that’s hard to ignore. Ditto if you can recite multiple impressive features the competition can’t match.

Yes, I believe you should use customers as proof points even when you’re not allowed to use their names. A blog is a great vehicle for doing that.

Related links

• One consistency rule that’s often forgotten — don’t declare a position today you can’t walk back from.
• Even your pricing algorithm should be a simple function — specifically minimum() — of individually simple elements.

• A low-quantity list price.
• A standard volume discount (typically 50%ish, assuming negligible cost of goods sold).
• The real negotiated price.

But the whole process has to start with some concept of a single-unit price.

What kind of price? Well, for appliances, you usually should just charge a one-time fee for whatever is in the carton, plus annual maintenance; most alternatives are gimmicks. But for packaged software, there are numerous choices. The easy part is timing:

• All software can be charged for on an annual license.
• Some can be sold on a perpetual license as well. (Exceptions include: Open source, SaaS, and perhaps other software whose main competition is reasonably-priced subscriptions.)
• Annual maintenance is usually 20-22% of the perpetual license fee.
• When there are both perpetual and annual options, the annual fee is usually 40-60% of the perpetual one.

Tougher is deciding what kind of “unit” you should price by. My standard advice has become:

• There should be 2 or more simple pricing algorithms, so that …
  
• … the price for any given customer is the lowest of those choices.

Generally one pricing algorithm will be suited for most of your customers, while the others will be meant for minority or edge cases.

By “simple pricing approaches” I primarily mean the usual-suspect proxies for valuable work done, such as:

• So much per unit of computing capacity (server, core, virtual machine, etc.).
• So much per user (named, simultaneous log-on, whatever).
• So much per unit of data (amount of raw data,  capacity of RAM, volume streaming through).

Alternatively, some pricing schemes focus more on development effort averted — e.g., ETL vendors may charge according to the number of different databases you connect to.

For example:

• You could price business intelligence software:
  • Per server core to most of your customers.
  • Per virtual machine to those who virtualize it.
• You could price sophisticated analytic software:
  • Per named user to most of your customers.
  • Per server, for those few organizations who want widespread/occasional access to it.
  • Premium (i.e. expensively) either way.
• You could price database software:
  • Per terabyte stored …
  • … but with a per-server cap that keeps you competitive with appliances even when a database is highly compressible.
• You could price ETL software:
  • Per database connection …
  • … with a per-server cap …
  • … and a per-megabyte option that probably only makes sense for a few awkwardly sharded cloud deployments.

Virtues of such approaches include:

• Simplicity. Your salesman on the account should be able to quickly determine which pricing approach will apply. The prospect should be comfortable that there won’t be hard-to-foresee “gotcha” charges.
• Fairness and match to use case. For any particular prospect, there probably will be a pricing scheme that fits well.
• Competitive flexibility. Nothing in this strategy puts much of a floor or ceiling on your pricing. You can do whatever you think is economically best.

My advice is similar for SaaS (Software as a Service), for similar reasons, with variations such as:

• There’s no natural concept of a perpetual license.
• There may be more choices (with appropriate quantity discounts) for length of term.
• Pricing could be by unit-of-work — e.g. transactions or other operations.

In most cases, there’s a whole other dimension of pricing complexity — you want to carve your offering into tiers, e.g.:

• Paid product/free trial version.
• Base product/chargeable options.
• Full feature set/limited features for casual users.
• Good support/best support.

I’m not going to address those in detail now, but I’ll leave you with one cardinal rule:

Don’t provide anybody with software that gives them a bad experience.

Even your crippleware should be good.

Related links

• Pricing choices by many specific vendors

Yes, I’m asking via a blog post, even thought the core problem is that people who want to see my posts via e-mail aren’t getting them. Please work with me on this anyway.

My two basic questions are:

• What should be the frequency of delivery? To date, it’s been nightly (at least in theory).
• What delivery technology should be used? To date, it’s been FeedBlitz.

1. The nightly scheduling has been an artifact of an RSS-to-email link that no longer seems stable. So I’m thinking of just manually pasting each post into a list email, in which case:

• Posts could be sent without delay.
• Every post would be delivered by separate mail. (As opposed to having only one post per night be mailed, while others just get linked to.)

It’s a bit more work for me, but probably nothing dire. Does lower latency sound good to everybody?

2. The main technical options seem to be:

• Free services oriented to discussion lists, such as Yahoo Groups, but set to announce-only. These have very basic functionality.
• Commercial services oriented to marketing email lists, such as Aweber or MailChimp. Does anybody have favorable or unfavorable experience with particular services? Most vendors surely use one or another, but it’s tough to guess which they’ve selected just based on their spam and pabulum informative communications, given the customizability those services provide.

Any thoughts would be most welcomed.

3. And while I’m at it — what I should I do for social/sharing buttons? Presumably, if I included buttons that made it easy for you to tweet links to my posts, submit them to Hacker News, etc., more of you would do so. Which specific options would you like to use?

• Twitter?
• LinkedIn?
• Google +?
• Facebook?
• Slashdot?
• Hacker News?
• dzone?
• Digg?

Anything else? I’d like to omit the more dubious possibilities, as offering everything could be a lot of clutter …

Amazon Redshift

Amazon did a deal with ParAccel that amounted to:

• Amazon got a very cheap license to a limited subset of ParAccel’s product …
• … so that it could launch a service called Amazon Redshift.
• Amazon also invested in ParAccel.

Some argue that this is great for ParAccel’s future prospects. I’m not convinced.

No doubt there are and will be Redshift users, evidently including Infor. But so far as I can tell, Redshift uses very standard SQL, so it doesn’t seed a ParAccel market in terms of developer habits. The administration/operation story is similar. So outside of general validation/bragging rights, Redshift is not a big deal for ParAccel.

OEMs and bragging rights

It’s not just Amazon and Infor; there’s also a MicroStrategy deal to OEM ParAccel — I think it’s the real ParAccel software in that case — for a particular service, MicroStrategy Wisdom. But unless I’m terribly mistaken, HP Vertica, Sybase IQ and even Infobright each have a lot more OEMs than ParAccel, just as they have a lot more customers than ParAccel overall.

This OEM success is a great validation for the idea of columnar analytic RDBMS in general, but I don’t see where it’s an advantage for ParAccel vs. the columnar leaders.

Concurrency

As I admitted in the comment thread to my first Actian/ParAccel post, I’m confused about what kind of concurrent usage ParAccel can really support. The data I have, e.g. in the link immediately above, is not conclusive. Googling suggests that VectorWise was at one user per core a couple of years ago, supportive of my hypothesis that it doesn’t have some big concurrency edge on ParAccel. But to repeat — I don’t really know.

DBMS acquisitions in the past

My history blog on DBMS acquisitions yielded more favorable examples than I was expecting. (Of course, I omitted a lot of small and boring failures.) And DBMS conglomerates are the rule more than the exception, with IBM, Sybase, Teradata and Oracle all adopting acquisition-aided multi-DBMS strategies, at least to some extent.

That said, Sybase is the main example of a vendor of a slow-growth DBMS (Adaptive Server Enterprise) doing well with a faster-growing one (Sybase IQ). Perhaps not coincidentally, Actian’s latest management team draws significantly on Sybase. So yes; ParAccel is now owned by a company run by guys who know something about selling columnar DBMS.

But the whole thing would be more convincing if Ingres had shown more life under Actian’s ownership, or indeed at any point in the past 20 years. My bottom line is that Actian was floundering badly in the DBMS market 1 1/2 years ago, and not a lot of favorable news has emerged in the interim — except, quite arguably, for the management changes and acquisitions themselves.

In the pre-relational days, it was common practice to buy products that hadn’t succeeded yet, and grow with them. Often these were programs written at enterprises, rather than third-party packages. Most of Cullinet’s product line, including its flagship DBMS IDMS, was came into the company that way. ADR, if memory serves, acquired the tiny vendor who created DATACOM/DB.

Then things slowed down. A Canadian insurance company oddly bought Computer Corporation of America, to utter non-success. (At least I got an investment banking finder’s fee on the deal.) Computer Associates, which did brilliantly in acquiring computer operations software, had a much rockier time with DBMS. It acquired Cullinet, Applied Data Research, and ASK/Ingres — among others — and didn’t have much growth or other joy with any of them.

Indeed, Ingres has been acquired three times, and hasn’t accomplished much for any of the acquirers (ASK, Computer Associates, Actian).

I used to think that Oracle’s acquisition of RDB provided key pieces of what became Oracle’s own extensibility technology. Andy Mendelsohn, however, disputed this vehemently — at least by his standards of vehemence — and his sources are better than mine. Rather, I now believe as I wrote in 2011:

… while Oracle’s track record with standalone DBMS acquisitions is admirable (DEC RDB, MySQL, etc.), Oracle’s track record of integrating DBMS acquisitions into the Oracle product itself is not so good. (Express? Essbase? The text product line? None of that has gone particularly well.)

Experiences were similar for some other relational DBMS pioneers. 

• Sybase did well with its acquisitions of what became the standalone products Sybase IQ and Adaptive Server Anywhere.
• Informix did well with its acquisition of what became Informix’s parallel offering XPS (the same technology Ingres passed up), but terribly with Illustra (which it unwisely tried to integrate into its other products).
• Microsoft has done very well with Sybase Adaptive Server Enterprise’s source code, which formed the basis for SQL Server.

IBM’s acquisition of Informix, however, didn’t accomplish much that I’ve been able to discern. Ditto various small deals such as Oracle/Sleepycat, Oracle/TimesTen, or IBM/solidDB. And no acquisition of an object-oriented DBMS vendor — of which there have been many — has succeeded in igniting that niche market.

Finally, let’s consider the recent merger wave in the analytic RDBMS sector.

• Microsoft went first, acquiring DATAllegro. Integration of DATAllegro and SQL Server technology didn’t go well; while PDW (Parallel Data Warehouse) has finally come to market, I believe it’s much less based on DATAllegro than Microsoft first hoped.
• IBM has sold a lot of Netezza into its installed base. Otherwise Netezza seems to be lagging. And it’s generally assumed that most noteworthy Netezza people have been or can be hired away. (Big exceptions: Phil Francisco, perhaps also John Metzger.) Wisely, IBM has made no moves to combine DB2 and Netezza into a single product.
• EMC/Greenplum has been the flashiest of these deals. Some early bumps notwithstanding, EMC poured resources into Greenplum, and EMC/Greenplum have been correspondingly active.  Partnerships (VMware, GE), name changes (Pivotal) and so on have also kept the pot stirred.
• For the first 2 years after being acquired by HP, Vertica proceeded fairly independently, with what seems like decent growth, but also without a Greenplum-like flood of resources enjoyed by EMC Greenplum. I expect somewhat more integration going forward, perhaps an appliance strategy that somebody actually notices.
• The Teradata/Aster case is much like IBM/Netezza — separate products, focused on the Teradata customer base. Details, of course, differ.

• ParAccel scales out.
• ParAccel has added analytic platform capabilities.
• I don’t know for sure, but I’d guess ParAccel has more mature management/plumbing capabilities as well.

One might conjecture that ParAccel is bad at highly concurrent, single-node use cases, and VectorWise is better at them — but at the link above, ParAccel bragged of supporting 5,000 concurrent connections. Besides, if one is just looking for a high-use reporting server, why not get Sybase IQ?? Anyhow, Actian hasn’t been investing enough in VectorWise to make it a major market player, and they’re unlikely to start now that they own ParAccel as well.

But I expect ParAccel to fail too. Reasons include:

• ParAccel’s small market share and traction.
• The disruption of any acquisition like this one.
• My general view of Actian as a company.

2 years after being acquired, Vertica — which conceptually has always been ParAccel’s closest competitor — has finally taken major hits on engineering staffing. Even so, I expect HP Vertica to reopen what was once a large technology and momentum gap vs. ParAccel.

My views on Actian start:

• Actian is attempting to build a database software conglomerate on the cheap, starting with Ingres, ParAccel, VectorWise, Pervasive (itself a small conglomerate) and Versant.
• Actian hasn’t accomplished much with Ingres, its original acquisition.
• Actian hasn’t accomplished much with VectorWise.
• Actian’s brief, embarrassing pivot away from database software was a joke. (The comments at that link also show VectorWise’s positioning as very different in September, 2011 than it is now.)
• I’ve had some very bad experiences with Actian management, although it seems to have largely turned over since then.
• I can’t identify the folks to make this work at the acquired pieces either (even though I think well of a few of them, e.g. Mike Hoskins and Rick Glick).

I.e., building a database conglomerate is hard, and Actian isn’t up to the challenge.

Actian has three main paths it can follow for synergy:

• Acquire a lot of pieces and flip the whole thing for more money to a foolish buyer. This strategy worked splendidly for Autonomy, and to some extent for Sybase as well. But it’s a longshot, and not necessarily a win for customers even if investors do well.
• Sell a bunch of disparate products through the same sales force. Tough to execute. And at best it raises sales coverage up to the level of that for the most successful product — and Actian doesn’t really have successful new products.
• Integrate the technologies. Blech. You don’t integrate DBMS with wildly different architectures, as Informix died trying in the 1990s.

I don’t see enough opportunity there for the whole thing to work out, with sales synergy being the best opportunity to prove me wrong.

Related links

• Doug Henschen and Derrick Harris offer quotes and numbers about the deal.
• VectorWise’s academic founders Peter Boncz and Marcin Zukowski seem to have left the company.

1. So far, the buzzword of the year is “real-time analytics”, generally with “operational” or “big data” included as well. I hear variants of that positioning from NewSQL vendors (e.g. MemSQL), NoSQL vendors (e.g. AeroSpike), BI stack vendors (e.g. Platfora), application-stack vendors (e.g. WibiData), log analysis vendors (led by Splunk), data management vendors (e.g. Cloudera), and of course the CEP industry.

Yeah, yeah, I know — not all the named companies are in exactly the right market category. But that’s hard to avoid.

Why this gold rush? On the demand side, there’s a real or imagined need for speed. On the supply side, I’d say:

• There are vast numbers of companies offering data-management-related technology. They need ways to differentiate.
• Doing analytics at short-request speeds is an obvious data-management-related challenge, and not yet comprehensively addressed.

2. More generally, most of the applications I hear about are analytic, or have a strong analytic aspect. The three biggest areas — and these overlap — are:

• Customer interaction
• Network and sensor monitoring
• Game and mobile application back-ends

Also arising fairly frequently are:

• Algorithmic trading
• Anti-fraud
• Risk measurement
• Law enforcement/national security
• Healthcare
• Stakeholder-facing analytics

I’m hearing less about quality, defect tracking, and equipment maintenance than I used to, but those application areas have anyway been ebbing and flowing for decades.

3. Much of customer interaction revolves around recommendation and personalization. In connection with that I’ll remind you:

• Multiple sources say that 5 millisecond response is a real need. Srini Srinivasan explained why in a January comment.
• The results of the recommendation and personalization can be delivered in many different ways — product recommendations, ads, special offers, email, snail mail, call center scripts and more. This is the paradigmatic example for my skepticism about complete analytic applications.

4. Networks and sensors emit the epitome of machine-generated data. Data sources include web logs, network logs (in the IT sense), telecommunication networks, other utilities (e.g. electric), vehicle fleets, and more. Application themes include:

• Human monitoring, via some kind of real-time business intelligence view. I hear about that a lot.
• Various kinds of automated response. (Security is an obvious example.)
• Integration with other kinds of application, data source, or use case.

As one example of the last point, Oliver Ratzesberger told me years ago that eBay had up-to-the-minute BI cubes integrating customer response and log data, for the purpose of quickly detecting technology problems. Acunu recently told me that similar applications are one of their sales focuses.

5. In another example, games and mobile applications can be a lot like websites in terms of the analytics that support them (all the more so if we’re talking about games with in-app purchases). Two special features come up repeatedly, however — leaderboards for games, and geospatial data sent by mobile devices.

6. Algorithmic trading is flashy because of the sums of money involved, and because of what is often hyper-low latency; I’ve even heard 50 microseconds, and that’s a slightly out of date figure for a sequence of several atomic operations. But otherwise it’s not one of the more interesting areas to me, for at least two reasons:

• It depends on a lot of latency-specific stuff, such as hand-crafted hardware.
• The participants are secretive — understandably so as they’re literally in a race with each other –and don’t reveal much.

Another reason I don’t study it much is that high-frequency trading could be devastated at any time by some simple regulatory changes.

7. I finally figured out one of the big drivers for better risk analysis. Banks need to keep capital lying around to cover a fraction of the risk they take on. If they can estimate the risk more precisely, and come up with a lower number, then they need to keep less capital. That’s a lot like finding large bags of money.

8. Anti-fraud applications arise in many industries, with many different kinds of data and latency requirement. For example:

• Insurers don’t want to pay bogus claims. They usually have weeks to think about that problem.
• Telcos don’t want to provision services for customers who will defraud them. They have to decide at call-center speed.
• Similarly, retailers don’t want to accept bogus returns.
• Stockbrokers don’t want rogue traders to defeat their controls. A lot of data and analysis go into that mission, as billions of dollars — literally — can be at stake.

9. And finally, the recent Boston Marathon bombing has brought law-enforcement/anti-terrorism applications to the fore. The Boston Globe criticized difficulties in information sharing, but the money quote is:

The FBI followed up by checking government databases and looking for things such as “derogatory telephone communications, possible use of online sites associated with the promotion of radical activity, associations with other persons of interest, travel history and plans, and education history,” according to FBI Supervisory Agent Jason J. Pack. “The FBI also interviewed Tamerlan Tsarnaev and family members. The FBI did not find any terrorism activity.”

Neither the telephone intercept nor the web-surfing tracking is a capability the government routinely admits, unless there was something like a wiretap order that I so far haven’t seen reported.

Related links

• Government surveillance is even more inevitable than when I wrote in 2010 that freedom can only be preserved by limiting government USES of data.
• Stakeholder-facing analytics isn’t much better understood than when I wrote about it in 2010.
• I wrote up a different list of analytic use cases back in 2006.
• The continued drop in high-frequency trading latency strengthens my 2009 contrast between the speed of a turtle and the speed of light; we’re now over a 3 * 10^10 difference between the speed of trading and the speed of generic planning, and many turtles walk well faster than 1 cm/sec.

MemSQL’s flagship reference is Zynga, across 100s of servers. Beyond that, the company claims (to quote a late draft of the press release):

Enterprises are already using distributed MemSQL in production for operational analytics, network security, real-time recommendations, and risk management.

All four of those use cases fit MemSQL’s positioning in “real-time analytics”. Besides Zynga, MemSQL cites penetration into traditional low-latency markets — financial services (various subsectors) and ad-tech.

Highlights of MemSQL’s new distributed architecture start:

• There are two kinds of MemSQL node — “aggregator” and “leaf”.
  • Aggregators are a kind of head node. You can have a bunch of them.
  • Leafs run full single-server MemSQL. You can have a bunch of them too.
• MemSQL has two query optimizers. One kind runs on the aggregator nodes, and thinks about the whole cluster. The other runs on the leafs, and only thinks about its own node.
• Much of the join and aggregation work is done on the aggregator nodes, but I didn’t pursue that issue in much detail.
• It is good policy — and supported — to replicate small dimension/reference tables across the cluster. These are replicated to aggregator and leaf nodes alike. (This tells us that some joins are indeed done on the leafs. )
• MemSQL replication can be synchronous or asynchronous. It can be used for high availability.

Also:

• MemSQL writes (whether primary or replicated) go to a buffer. The buffer size can be 0 or positive, in a tradeoff of durability vs. the likelihood of a disk I/O bottleneck.
• MemSQL has many virtual nodes on each physical (leaf) node. (This is pretty much an industry-standard best practice, as it helps with elasticity, recovery from node failure, and so on.)
• Compression is still a future feature.
• So is online schema change.
• Leaf nodes have cost-based optimizers.
• MemSQL’s aggregator (cluster-wide) optimizer is mainly heuristic, but is supposed to get more cost-based in future releases.
• In some releases it will be possible to keep MemSQL running while upgrading the software. But that’s not a promise for releases that change how replication works.

And which not-easily-parallelized aggregate did MemSQL implement first? The same one Platfora did — COUNT DISTINCT.

Tokutek’s news is that they’re open sourcing much of TokuDB, but holding back hot backup for their paid version. I approve of this strategy — “doesn’t lose data” is an important feature, and well worth paying for.

I kid, I kid. Any system has at least a bad way to do backups — e.g. one that involves slowing performance, or perhaps even requires taking applications offline altogether. So the real points of good backup technology are:

• To keep performance steady.
• To make the whole thing as easy to manage as possible.

GenieDB is announcing a Version 2, which is basically a performance release. So in lieu of pretending to have much article-worthy news, GenieDB is taking the opportunity to remind folks of its core marketing messages, with catchphrases such as “multi-regional self-healing MySQL”. Good choice; indeed, I wish more vendors would adopt that marketing tactic.

Along the way, I did learn a bit more about GenieDB. In particular:

• GenieDB is now just backed by a hacked version of InnoDB (no more Berkeley DB Java Edition).
• Why hacked? Because GenieDB appends a Lamport timestamp to every row, which somehow leads to a need to modify how indexes and caching work.
• Benefits of the chamge include performance and simpler (for the vendor) development.
• An arguable disadvantage of the switch is that GenieDB no longer can use Berkeley DB’s key-value interface — but MySQL now has one of those too.

I also picked up some GenieDB company stats I didn’t know before — 9 employees and 2 paying customers.

Related links

• Recent posts about TokuDB and GenieDB

• Teradata tends to get 35-55% (roughly speaking) annual performance improvements, as measured by its internal blended measure Tperf. A big part of this is exploiting new-generation Intel processors.
• This year the figure is around 40%.
• The 6700 is based on Intel’s Sandy Bridge.
• Teradata previously told me that Ivy Bridge — the next one after Sandy Bridge — could offer a performance “discontinuity”. So, while this is just a guess, I expect that next year’s Teradata performance improvement will beat this year’s.
• Teradata has now largely switched over to InfiniBand.

Teradata is also talking about data integration and best-of-breed systems, with buzzwords such as:

• Teradata Unified Data Architecture.
• Fabric-based computing, even though this isn’t really about storage.
• Teradata SQL-H.

The upshot is that Teradata has at least 6 kinds of rack or cabinet it wants to sell you — along with software to connect them — of which it really thinks you should get at least 3:

• The 4 main Teradata-software appliances:
  • Active Enterprise Data Warehouse (the new 6700). Teradata thinks every sufficiently large enterprise should have one of these.
  • Extreme Performance Appliance (Teradata 4xxx), based on solid-state drives (which are also used in the 6xxx systems). At least I think so; the 4xxx wasn’t in the most recent slide deck I saw.
  • Data Warehouse Appliance (Teradata 2700).
  • Extreme Data Appliance (Teradata 1650).
• The Teradata Aster Big Analytics Appliance, running Aster and Hadoop software. Teradata basically thinks everybody should have one of these too.
• A separate cabinet for special-purpose “Teradata Managed Servers”. While there’s some space for Managed Servers in other Teradata appliances, Teradata now offers so many such capabilities that it thinks you will likely need a separate rack for those as well. These include (partial list):
  • Viewpoint system management.
  • Backup.
  • Teradata Unity.
  • Data movement, which is not the same thing as Teradata Unity.
  • Data loading, which is yet something else.
  • Generic compute (notably, to run SAS).

Even that doesn’t exhaust the possibilities:

• The 36 InfiniBand ports Teradata can fit into a cabinet aren’t enough, it suggests and presumably will sell you free-standing Mellanox switches as an alternative.
• That slide deck split the Big Analytics Appliance back out into Aster and Hadoop options.
• There also seems to be a SAS-specific modeling appliance.

And you can have — or in some cases must have — Teradata Managed Server nodes in other kinds of Teradata appliance.

Finally, Teradata also offers a stand-alone single- or several-node Teradata 670 Data Mart Appliance, notes on which include:

• The Teradata 670′s entry price is under $1/2 million, if you want to use it as your first Teradata system (something that evidently is happening, mainly outside the Americas).
• Another use for the Teradata 670 is for physical — as opposed to virtual — data mart spin-out.
• The primary use for the Teradata Data Mart Appliance, however, seems to be test/development for larger Teradata systems.
• The Teradata Data Mart Appliance is one of the options for placing in a separate managed-server Teradata rack.

Related links

• My recent musings on the variety of clusters and appliances an enterprise could have.
• A March, 2012 post on various vendors’ admissions that multiple analytic database systems are needed.

*Speaking of confusion — Teradata Aster seems to use the spellings SQL/MR and SQL-MR interchangeably.

… except that now there’s also a SQL-H for regular Teradata systems as well. While it has the same general features and benefits as SQL-H for Teradata Aster, the details are different, since the underlying systems are.

I hope that’s clear.

• DeepDB’s indexes can help you with analytic queries; hence, DeepDB is marketed as supporting OLTP (OnLine Transaction Processing) and analytics in the same system.
• DeepDB is marketed as “designed for big data and the cloud”, with reference to “Volume, Velocity, and Variety”. What I could discern in support of that is mainly:
  • DeepDB has been tested at up to 3 terabytes at customer sites and up to 1 billion rows internally.
  • Like most other NewSQL and NoSQL DBMS, DeepDB is append-only, and hence could be said to “stream” data to disk.
  • DeepDB’s indexes could at some point in the future be made to work well with non-tabular data.*
  • The Deep guys have plans and designs for scale-out — transparent sharding and so on.

*For reasons that do not seem closely related to product reality, DeepDB is marketed as if it supports “unstructured” data today.

Other NewSQL DBMS seem “designed for big data and the cloud” to at least the same extent DeepDB is. However, if we’re interpreting “big data” to include multi-structured data support — well, only half or so of the NewSQL products and companies I know of share Deep’s interest in branching out. In particular:

• Akiban definitely does. (Note: Stay tuned for some next-steps company news about Akiban.)
• Tokutek has planted a small stake there too.
• Key-value-store-backed NuoDB and GenieDB probably leans that way. (And SanDisk evidently shut down Schooner’s RDBMS while keeping its key-value store.)
• VoltDB, Clustrix, ScaleDB and MemSQL seem more strictly tabular, except insofar as text search is a requirement for everybody. (Edit: Oops; I forgot about Clustrix’s approach to JSON support.)

Edit: MySQL has some sort of an optional NoSQL interface, and hence so presumably do MySQL-compatible TokuDB, GenieDB, Clustrix, and MemSQL.

Also, some of those products do not today have the transparent scale-out that Deep plans to offer in the future.

Among the 10 people listed as part of Deep Information Sciences’ team, I noticed 2 who arguably had DBMS industry experience, in that they worked at virtualization vendor Virtual Iron, and stayed on for a while after Virtual Iron was bought by Oracle. One of them, Chief Scientist & Architect Tom Hazel, also was at Akiban for a few months, where he did actually work on a DBMS. Other Deep Information Sciences notes include:

• Deep has 25 or so people in all.
• Deep had a recent $10 million funding round.
• Deep Information Sciences is the former Cloudtree, which as of February, 2011 was pursuing quite a different strategy. (Evidently there was a pivot.) Deep was founded in 2010.
• There are 2 paying customers for DeepDB, even though it’s still in beta, and 8 trials. A similar number of trials and strategic partners are queued up.
• DeepDB general availability is expected later this quarter.

Although our call was blessedly technical, we didn’t have a chance to go through the DeepDB architecture in great detail. That said, DeepDB seems to store data in all of 3 ways:

• An in-memory row store.
• An on-disk row store with a very different architecture.
• Indexes, which can also serve as a column store.

Notes on that include:

• DeepDB’s in-memory row store is designed to manage single rows as much as possible, rather than pages. Indeed, there are “aspects of tries”, although we didn’t drill down into what exactly that meant.
• Indexes are streamed to disk no less than once every 15 seconds, by default, and perhaps with latency as low as 10 milliseconds.
• Perhaps the most important point I didn’t grasp is “segments”. The data and indexes on disk are stored in segments, which can be of different sizes, and which may each carry some summary data/metadata/whatever. Somehow, this is central to DeepDB’s design.
• In what is evidently a design focus, DeepDB tries to get the benefit of “in-memory data” that isn’t actually taking up RAM. B-trees can point at rows that aren’t actually in memory. Segments evicted from cache can leave some metadata or summary data behind.
• DeepDB’s compression story seems to be a work in progress.
  • There’s prefix compression already, at least in the indexes, which Deep just calls “compaction”.
  • Other compression is working in the lab, but not scheduled for Version 1.0.
    • Block compression seems to be in play.
    • Delta compression was mentioned once
    • Dictionary compression wasn’t mentioned at all.
  • DeepDB apparently will keep compressed data in cache, then decompress it to operate on it.
  • Different segments can be compressed/uncompressed differently.
• DeepDB’s on-disk row store is append-only. Time-travel is being worked on. While I forgot to ask, it seems likely that DeepDB has MVCC (Multi-Version Concurrency Control).

And finally: DeepDB in its current form is a “drop-in” InnoDB replacement, but not necessarily bug-compatible.

I think positioning boils down to:

• Product category, even though product categorizations are never precise.
• Orientation, along multiple attributes.* Hence positionings are more complex than vendors commonly realize.
• (Optionally, but it’s a common option) Target customer group.

When positioning is framed that way, we can say that the primary goals of messaging are to communicate, emphasize or try to change aspects of your positioning.

*I used to say “dimensions” instead of “attributes” — but most likely the attributes aren’t all orthogonal to each other and also aren’t all measured on a continuous scale.

The modern concept of “positioning” was formulated and popularized by Jack Trout, starting in the 1960s, and can be stated as (filling) a “location in the customer’s mind”. In practice, a Trout positioning combines a product category with a single-attribute orientation such as “safe”, “powerful”, or “fun”. But I think that’s too simple for B2B or technology contexts.

I like the Geoffrey Moore formulation better, in which he offers a positioning template:

For (target customers)
Who (have the following problem)
Our product is a (describe the product or solution)
That provides (cite the breakthrough capability).
Unlike (reference competition)
Our product/solution (describe the key point of competitive differentiation)

But while those are all good questions — compare them to my own strategy worksheet — Moore’s version is flawed too; in conflating positioning and messaging, he oversimplifies them both. 

What people — evidently including Trout and Moore — often overlook is that every product in a category is positioned along the same set of attributes, starting with those found on evaluation checklists. If any (sufficiently visible) competitor in a category claims to be strong in an attribute, then every other product in the category will be rated according to that attribute too.

For example, every analytic RDBMS is positioned by prospects and influencers, whether or not the vendor wishes it to be, according to whether it’s MPP (and in what sense), what kinds of concurrent workloads it handles, which SQL it does or doesn’t execute, how well it compresses, whether it has a true-columnar option and so on and so forth. Further and more important, each analytic RDBMS is positioned along summary attributes such as “enterprise-proven”, “handles large databases”, “ease of administration” and the like. B2B technology products are positioned not only by their strengths, but by their competitive weaknesses as well.

Specific principles I rely on when working with clients include:

• Your uniqueness claims should actually be true. If you say “Our big differentiation is D”, and your audience doesn’t think D is differentiating — then where are you positioned?
• Position in terms of summary attributes. People sometimes buy DBMS because they’re fast. They rarely buy them, however, because there’s a good optimizer; rather, a good optimizer is evidence that a DBMS is well-engineered and fast.
• Category names shouldn’t make a misleading first impression. By “misleading” I don’t necessarily mean “dishonest”; I’m just referring to what people’s natural assumptions and expectations will be. For example:
  • “Metadata” means several different things; hence it confuses people; hence vendors have learned not to use it in category names.
  • “Relational” evokes RDBMS so strongly that any other use of the word should probably be avoided.
• Leave a path to walk back. “Our X is amazingly good for segment Y” lets you later say “Now it’s great for Z as well.” But “We’re entirely focused on segment Y” — while it can be a powerful story — puts you in a trap you may not escape later on.

Above all — don’t bother saying things nobody will care about. That principle — which gets violated many times each day — is central to messaging and positioning alike.

1. I will not fund any entrepreneur who uses the word “disruptive”, unless she has actually read at least one book by Clayton Christensen.
2. I will not fund any entrepreneur who mentions “market projections” in other than ironic terms. Nobody who talks of market projections with a straight face should be trusted.
3. I will not fund any software entrepreneur who is unfamiliar with “The Mythical Man-Month”.
4. I will not fund any software whose primary feature is that it is implemented in the “cloud” or via “SaaS”. A me-too product on a different platform is still a me-too product.
5. I will not fund any pitch that emphasizes the word “elastic”. Elastic is an important feature of underwear and pajamas, but even in those domains it does not provide differentiation.
6. I will hire a 16 year old intern of moderately above-average intelligence. I will not sign or propose any contract that intern finds difficult to understand.
7. I will hire a second intern of moderately below-average intelligence. I will not fund any product whose documentation that intern finds difficult to understand. Exceptions may be made for products sold to orienteering athletes, crossword puzzle solvers, or engineers.
8. When a board on which I sit approves revenue targets for the year, I will further stipulate that the year-ending sales pipeline must comprise more than a Chinese hair salon, an Italian pushcart vendor, the CEO’s brother-in-law and a bankrupt bait shop in Nome.
9. I will only hire a CEO who can explain the technology at his previous company. A CEO who doesn’t know what his products do can’t sell or market them either.
10. I will only hire a CEO who can also walk me through a sales cycle at her previous company. A CEO who doesn’t know how a customer buys may well have trouble producing revenue.
11. I will support any plan that I agree is good for a company I have invested in, nor matter how modest or how bold. I will participate in any funding round that I think is profitable for my limited partners.
12. I will remember that a board of directors has a fiduciary responsibility to all shareholders, and not just to the preferred ones.

Please offer your suggestions below. An associate will get back to you with our decision.

Related links

• The original “When I am an Evil Overlord” list
• A rival list
• The most comprehensive such list I know of (with additional sections for heroes, sidekicks, beautiful daughters, etc.)

1. Pitch at a suitable level of detail.
2. Treat your top influencers as individuals.
3. For every news item, ask yourself — who cares?
4. Don’t pigeonhole your company or product.
5. Use a proofreader or copy editor.
6. Use short(er) sentences.
7. Blog.

I shall explain.  

1. Marketing pitches can be on at least three levels:

• Industry/sector. “There’s something big going on, and I’m here to tell you about it.” Such pitches sometimes work well in webinars and other lead-generation events. But they usually fail in PR.
• Company. “We’re an awesomely well-suited company to do X.” These pitches have their place, for example:
  • When pitching investors or prospective employees.
  • When telling a focus and commitment story.
  • In stealth mode, when you don’t have anything else to say …
  • … but not at first product launch, when you finally do.
• Product. This is usually the right way to go.

A common error is to make your product pitch in such general terms that it’s really a sector pitch in disguise.

2. There are many kinds of influencer, who often need to be handled in different ways. Some of the differences can be handled just by asking how they like to work (for example, I have a whole how to pitch me post). Beyond that, the right person to lead an important relationship is:

• Usually somebody who can truly speak for your company, and specifically:
  • Has the knowledge and ability to respond to pushback.
  • Knows the influencer well enough to argue back in turn.
• Occasionally an in-house press or analyst relations staffer.
• Almost never an outside PR person.

As one tech journalist put it:

If you are a small startup with innovative technology, put as little as possible between your own people who can talk with passion about the stuff, and whoever you’re trying to get coverage from.

1 of my top 20 vendor relationships — Teradata — is led by an in-house “relations” specialist. 0 of them are led by outside PR.

3. Enhancing your product is good. But if all you’re doing is playing catch-up in areas that you lag competitors, you have two main choices for marketing the enhancements:

• Assume nobody other than your current users cares.
• Emphasize your leading features, and only then add “Now with fewer drawbacks!”

And nobody — except perhaps in the affected regions — cares that you opened a sales office in Lake Wobegon, added a distributor in Grand Fenwick, or have a CEO named Elbonian Entrepreneur of the Year.

4. Focus is good. But if you claim extreme focus, there will still be a record of those claims when you later want to branch out, which can and will be used against you in sales and marketing situations.

5. A significant fraction of all marketing collateral is written by committees, by non-native English speakers, by engineers, or in a hurry. In all those cases, outside proofreading or even copy editing could be useful.

Lack of budget is no excuse; such services can be amazingly cheap.

6. In particular, sentence bloat is endemic, which is why my comments on press release drafts often say “Sentence from hell!” As I write this:

• The first sentence of the top press release on IBM.com contains 54 words; the first 3 combined contain 131.
• The analogous figures for Kognitio are 40 and 128.

Strunk and White weep.

7. Every vendor should have a blog. Period. There are no exceptions to that rule, because blogs serve one universal need — saying things that are inconvenient to express in other formats. Examples of things easier to do in a blog than elsewhere include:

• Tell customer success stories when you aren’t allowed to say the name of the customer(s).
• Explain specific technical points.
• Answer commonly raised sales objections. (But don’t be defensive when you do!)

Vendor blog dos and don’ts include:

• Do have multiple blog authors. One person never keeps up; besides, different posts should likely be written with different emphases.
• Don’t worry about frequency. Write when you have something to say.
• Don’t worry about driving/building traffic. If everybody who reads a post is somebody to whom you personally emailed a link, the blog is still extremely worthwhile.
• Don’t worry much about look and feel. A fairly generic WordPress blog is fine.
• Do include a link back to your main website. It’s amazing how many vendors forget to do that.

Related links:

• Our overview of marcom essentials
• Our comprehensive post on PR theory and tips
• Our IT vendor strategy and execution worksheets
• Tips (and a rant) on your initial meeting with a PR target

• A small oligopoly dominating the conjoined businesses of mobile device software and search. The companies most obviously positioned for membership are Google and Apple.
• The continued and growing combination of search, advertisement/recommendation, and alerting. The same user-specific data will be needed for all three.
• A whole lot of privacy concerns.

My reasoning starts from several observations:

• Enterprise search is greatly disappointing. My main reason for saying that is anecdotal evidence — I don’t notice users being much happier with search than they were 15 years ago. But business results are suggestive too:
  • HP just disclosed serious problems with Autonomy.
  • Microsoft’s acquisition of FAST was a similar debacle.
  • Lesser enterprise search outfits never prospered much. (E.g., when’s the last time you heard mention of Coveo?)
  • My favorable impressions of the e-commerce site search business turned out to be overdone. (E.g., Mercado’s assets were sold for a pittance soon after I wrote that, while Endeca and Inquira were absorbed into Oracle.)
  • Lucene/Solr’s recent stirrings aren’t really in the area of search.
• Web search, while superior to the enterprise kind, is disappointing people as well. Are Google’s results any better than they were 8 years ago? Google’s ongoing hard work notwithstanding, are they even as good?
• Consumer computer usage is swinging toward mobile devices. I hope I don’t have to convince you about that one.

In principle, there are two main ways to make search better:

• Understand more about the documents being searched over. But Google’s travails, combined with the rather dismal history of enterprise search, suggest we’re well into the diminishing-returns part of that project.
• Understand more about what the searcher wants.

The latter, I think, is where significant future improvement will be found.

So how does a search engine understand what you want? It can listen to you directly, parsing your search string. It can ask for more clarity, through some kind of disambiguation interface. Or it can make inferences, based on — well, based on just about any kind of information that might exist about you and your online behavior.

Search strings are short, typically four words or less. That doesn’t leave room for a lot of innovative parsing. Not a lot of progress can be made until search strings get a lot longer, and that is unlikely except perhaps through the convenience of speech recognition.

Faceted/parameterized selection has its place. For example, when I search on Amazon.com, the site encourages me to also select a department from its dropdown menu; otherwise, it refuses to rank the search results. And when I buy shirts from Land’s End, I just click through and never search at all. Still, Google’s been around for 15 years, and about all its successes in searcher-does-the-work disambiguation boil down to is:

• A list of a few major subcategories to search (News, YouTube, etc.).
• Spelling correction.
• A desultory list of related/more specific searches, perhaps just longer search strings other people have recently entered.
• Well-hidden “Advanced Search” features, which look much like AltaVista’s and AllTheWeb’s similar features did late in the 20th Century.

Whatever the user attitudes and behaviors are that constrain Google’s or its competitors’ success in this area, I can’t imagine them changing much — except, once again, in the event that speech recognition leads to richer human-computer conversations.

I’ve now highlighted two different ways in which there’s a search-interface challenge that will be tough to beat without turning to speech recognition. But the case for speech recognition is even stronger than that. We’re moving to small, mobile devices, and:

• Traditional search interfaces work worse on mobile devices than on desktop computers. Typing is harder. So is dealing with picky forms.
• Speech may work as well or better on mobile devices than at your desk. If you have upgraded your Apple device to IOS 6, you have both a microphone and Siri. The same may not be true of your desktop gear.

And so I conclude that speech recognition is a big part of the future of search.

What will that allow? Since talking is easier than typing, speech is a way to get longer text strings as search inputs, or more of them. It’s plausible that people might speak queries as complex as:

• “I want to buy a recharger for an iPad 3 with delivery this week.”
• “Where is 10gen’s Northern California office?” … “Which nearby restaurants have good Yelp reviews?”
• “Tell me about the David Reed who went to the Kennedy School of Government around 1977, went to Dartmouth before that, and worked for the Federal Communications Commission.”

Getting search engines to the point that they can handle such queries will be difficult but straightforward — but even more progress is needed. Search results for various queries will be greatly improved if the search engine “knows” things like:

• The location of your home and office, and the distance you’re willing to go from them to eat or shop.
• Your tastes in food, clothing, and gadgetry.
• The level of sophistication at which you like to read about medicine, finance, or electronics.
• Which people are or might be in your extended social network.

And that will cement internet search squarely in the world of — for once I approve of the term — big data.

Let’s start with a breezy overview, setting the stage for more detailed posts to follow. As I see it, there have been four eras at Oracle, which between them reflect just about every tech company management theory I can think of.

Startup: This period comprised initial development, custom contract with the US military (CIA, I think, even though the demo database was always naval), and initial product release. This is the one phase of Oracle’s history I didn’t witness personally. But it seems to have been pretty much a story of “build a minimum viable product for a great vision, and hustle until somebody buys it.”

Hypergrowth: Roughly speaking, Oracle grew 100% per year on its way from $5 million in revenue to $1 billion. This period formed much of the basis for Geoffrey Moore’s famous “Crossing the Chasm” series of books. In line with Moore’s later observations, Oracle’s priorities in this period were:

• Sell.
• Ship good enough product to sell.
• Worry about the details later.

By the backhanded “good enough … to sell”, I mainly mean two things. First, Oracle quality was pretty questionable, across the board. Second, as I observed in a post on Oracle’s arch-rival Ingres,

Ingres was first to market with new features such as a 4GL or a truly distributed DBMS. Oracle, however, was the first to market with the features customers most cared about, at a level of completeness they found acceptable.

Plateau, Professionalization, and Conquest: Oracle then hit a wall. New “adult supervision” management came in to clean up everything from product quality to accounting. This is when Oracle got serious about competing with IBM and EDS. It’s when Mike Fields and I coined the label “Enterprise Data Babysitter”.

Regaining momentum, Oracle pulled irreversibly past the other independent DBMS vendors of the era, and in essence past IBM mainframes as well. Its application business also finally got some traction, albeit with a long way still to go. And most interesting to me, Oracle triumphed with a blend of product and professional services efforts in a way that hasn’t been seen before or since.

This was also the era during which I was most closely involved with Oracle myself. Oracle was my biggest consulting client for multiple years. I double-dated with Larry Ellison. Geoff Squire bought me an obscene statuette in Indonesia, and presented it to me at an Oracle analyst day.* I stayed for a week in Oracle PR chief Gail Snider’s house, and a night in Marc Benioff’s. Oracle VP Bob Jesse explained to me what raves were. (He later left Oracle to start a charitable foundation promoting drug use in the context of religion.) And Oracle analyst relations chief Daniel Sagalowicz didn’t even bother talking to me, on the theory I was getting along just fine without his help. Fun times.

*It was a lovely carving of two pigs fornicating. He said he saw it and instantly thought of me. I presume this was a reference to my Wall Street background.

Empire expansion: Now Oracle was atop the heap, selling complex, expensive products to complex, deep-pocketed customers. This left Oracle in what Clayton Christensen would call the Innovator’s Dilemma position, subject to disruption from below. And so Oracle adopted the Innovator’s Solution with a vengeance, by:

• Thickening its stack, to ensure that wherever the profit opportunity went, Oracle would be there.
• Going to great lengths to buy a leading disrupter, MySQL. (Oracle endured  expensive delays in the Sun acquisition it could have averted by divesting MySQL.)

In particular, Oracle bought huge numbers of software vendors — PeopleSoft, Siebel, BEA, and many more. And then Oracle went further, bundling hardware as well — but that brings us pretty much to the present day.

Related link

• Wikipedia’s timeline for Oracle’s history is pretty reasonable overall, but there certainly are errors and omissions. For example, Wikipedia seems to think Mike Fields and Ray Lane joined Oracle the same year, when in fact Mike’s (and Geoff’s) replacement by Ray Lane was a key event in Oracle history.

Hagia Sophia seen through the Sultan Ahmed fountain, by Linda Barlow.

Interior view of the Rüstem Pasha Mosque, from a nice page by an outfit trying to sell Turkish language classes.

Tile outside the Rüstem Pasha Mosque showing Mecca, for those who can’t physically make the pilgrimage, by Linda Barlow.

Linda lived in Turkey for a while with her first husband, and speaks excellent Turkish. (In general, the Barlow women have an amazing talent for languages.)

If you’ve never been to Istanbul, it must be seen to be believed. From a hills and water standpoint, imagine 10 San Franciscos, but with many of the buildings being 500+ years old. The whole thing is wrapped around the Bosphorus, in which at any moment you can see 2-3 tankers, a whole lot of commuter ferries, and generally more ship traffic than I imagine can be found in any other similar expanse of water in the world (the Panama Canal area perhaps excepted). And there are plenty of places from which to get awesome views, most notably on the water itself. If you’re ever in Istanbul, seize every pretext you can find to be out on the water.

When it comes to great religious buildings, Istanbul may be my favorite city in the world, ahead of Rome, Paris, and even Kyoto. Reasons include:

• There are a lot of them here, since in the Ottoman Empire there weren’t many ways for the rich and powerful to use their wealth except by endowing buildings, most notably mosques (in particular, they could pass little wealth on to their heirs). The Ottomans built feverishly for about 400 years.
• Since Islam frowns on figurative art, the whole thing is very geometric, which I like.
• Mimar Sinan, the Ottoman analogue to Michelangelo or Leonardo da Vinci, got along fabulously with his patrons. (Suleiman the Magnificent called Sinan the only person who had “never disappointed” him.) Those patrons had much of the wealth of the Ottoman Empire to play with. Consequently, he had huge budgets and staff, and built many beautiful things.
• When the Turks conquered Constantinople, they found Hagia Sophia there, and developed much of their own style in mosques based on it. I think that style works beautifully on the outside of buildings just as on the inside, more than is the case for – for example – most Western cathedrals.
• Many of Istanbul’s great buildings can still be seen from the outside, at a distance, more than is the case in any other city I can think of. (The hills are a big help with that.)

The standard list of “must-see” sights in Istanbul would probably start with:

• Hagia Sophia.
• The Blue Mosque.
• Topkapi Palace.

They’re all great, but I’d put ahead of them:

• The Süleymaniye Mosque (by Sinan). To see why, just do a Google image search on its name.
• The Rüstem Pasha Mosque (also by Sinan). The big reason is the tiles, which – unlike in bigger mosques – are right in your face. Tiles are the greatest decoration in mosques, just as stained glass windows are in churches. I don’t find Rüstem Pasha quite as beautiful as Sainte-Chapelle in Paris – but the analogy isn’t far-fetched.

There also are several very nice tiled pavilions in Topkapi, but I’d put the Rüstem Pasha Mosque well ahead of any one of them.

Also on my mustn’t-miss list are:

• The final room in the Turkish and Islamic Arts Museum, aka the Ibrahim Pasa Palace. It’s a crafts museum, of course, since they didn’t have much figurative art in the Ottoman Empire. The final room has what must be the greatest collection of rugs in the world, many of them around 30 feet long. Don’t miss it, even if you only have 20 minutes to spend. It’s extremely close to the Blue Mosque, and hence to Hagia Sophia as well.
• The aforementioned boat ride(s).
• Topkapi Palace and the Blue Mosque.

Frankly, I like Hagia Sophia better from the outside than the inside, for beauty. It’s one of the world’s great feats of architectural engineering, unmatched in grandeur for about a millennium after it was built. But when the Muslims converted it from a church to a mosque (it’s now a secular museum, thanks to Ataturk), they covered up the art and made the whole interior rather plain. Also not on my can’t-miss list are the famous Covered and Egyptian/Spice Bazaars, although I really like the latter. More on those in a future post.

Istanbul traffic is a catastrophe, especially since streets are one lane wide and never point in the direction you wish to go at the moment. But fortunately, the absolutely-can’t-miss sights are all within walking distance of each other in the old city (i.e., the portion south of the Golden Horn inlet), and there’s a tram route looping around to help you out further. Also in that area is Eminönü, Istanbul’s busiest port area for boat departures. And if you do want to be driven around, distances are so short that, even though it’s slower than walking, it’s doable, and you do see extra stuff out the window as the vehicle creeps along.

A great Istanbul guide is very nice to have (but not required), and I have one to recommend: a freelancer named Tayfun Diker – pronounced like and with the same etymology as “typhoon” – whose Hotmail address is m+FirstName+LastName. Benefits of having a guide include:

• Lots of context – religious, historical, and architectural.
• Turkish language skills.
• Help finding stuff and getting around (and if you do want car/driver, it’s as cheap to get one with a guide as without).

Best of all, guides don’t have to be all that expensive; our hotel charged us 100 Euros for half a day with Tayfun, a van, and a driver – before tips — and perhaps if one contacted him directly one could do at least as well as that.

Not everybody can afford the time to spend a week or two in Istanbul. But if you can shoehorn even 2 days of Istanbul sightseeing into your schedule, I would highly recommend arranging to do so.

A spectacularly failed upgrade has brought down DBMS2.com. I’m going to try to reproduce the error — or hopefully not — and also work toward a fix by messing with my other blogs. Please forgive any chaos that ensues.

Most commonly, my answer is a variant on:

• Of course you need basic website content. For starters, your website should at least feature:
  • Answers of one paragraph or less to the top four strategic worksheet questions.
  • A several-paragraph description of your product/technology.
  • Management bios, contact information, and other obvious stuff.
• You also need a fairly technical company white paper. At some point in your sales cycle, there will be a technical evaluation. A white paper can answer a lot of early questions. What’s more, many of your early sales will be driven by people who think new technology is cool. Make it easy and appealing for them to learn about your cool new tech.
• Many people like videos. Whether it’s a link to a conference presentation or a white board talk or whatever, it’s good to have some kind of video. Some people, however — I’m one of them — don’t like videos, so don’t do anything essential in your videos you don’t also convey in writing.
• I further favor having a low-post-count blog. Notes on that include:
  • Almost nobody has the time to do a lot of blogging.
  • Even so, a blog is the most flexible and best way to communicate things that seem harder to say in other formats.
  • In particular, this can be a “poor man’s” way to make up for what is surely a distressing lack of resources in pre-sales support personnel, other collateral, and so on.
  • The goal isn’t to build a consistent readership. (You’re not going to invest enough effort for that.) The goal is to put up a few posts, then call influencers’ and prospects’ attention to them by email.

Beyond that, I’d say:

• Of course you want to generate leads. I don’t have strong opinions as to whether to make some of the items mentioned above require registration. But beware of the absurdly extreme position that says marketing serves solely to feed the sales pipeline.
• Supervise your PR very closely. Do much of it yourself. Indeed, strongly consider doing without a PR firm altogether.

Where, by way of contrast, do I favor being frugal?

• If you look at the PR-related links just above, I’m skeptical of investing a lot of money in PR. PR campaigns rarely provide high, positive value on your investment in them. Often, the “value” they provide is downright negative.
• Websites do not have to be lengthy or glitzy.
• Business-oriented white papers are an early-stage luxury. Non-technical businesspeople are interested in being “educated” by IBM, of which they’ve actually heard. But they aren’t as likely to read a paper from ZNewTek.
• Lead generation programs are expensive. In a direct-sales business that requires overcoming multiple sales objections, it’s easy to over-invest in leads.

Some of my reasoning behind these views includes:

1. Basically, the end goals of marketing communication are:

• Encourage customer organizations to give you money.
• Encourage engineers and other desirable employees to work for you.
• Encourage investors to give you money as well.

In various combinations, those audiences all need to be persuaded that:

• You have and will continue to have desirable product offerings …
• … and in fact customers will buy those offerings in enough quantity for your business to be healthy.
• (Highly desirable even if not totally necessary). Your product has a “cool” aspect. Engineers care about that directly, and the rest of your story is easier to sell if there’s a cool hook on which to hang it.

My suggestions touch those bases.

2. Credibility is everything. (If you can fake that, you’ve got it made!) There should be enough detail behind your stories for them to seem real. You shouldn’t oversell. Excessive marketing glitz can feel like overselling. So can overwrought PR.

3. A basic paradox of marketing communications is that they both:

• Need to be (sufficiently) concise, so as not to bore or scare your prospects before they’re ever engaged.
• Need to be (sufficiently) detailed, so as to be credible.

The answer is NOT the old sexist joke about “Short enough to be interesting but long enough to cover the interesting bits.” Rather, right from the getgo, your marketing communications should have both concise and detailed parts.

4. You shouldn’t address all audiences. You don’t want to sell to a prospect who’s unlikely to buy. That goes for influencers too.

My best example this week (it’s only Tuesday morning) is an email from Vitria, which reads in part:

The world’s first Operational Intelligence (OI) app …

While it seems like everyone is jumping on the big data bandwagon, only OI can claim to be purposely built for tackling big data in motion …

That’s utter nonsense. We’ve had a CEP/stream processing industry for years. We’ve had stock-quote and network-monitoring systems for decades. Maybe Vitria has a good story, but the core claims in their email are obviously false. If you think I’m overreacting, it’s only because so many other companies also pitch blatantly untrue claims.

So do I want to talk with them? Well, their email suggests that if I do, they’re likely to start out by emphatically saying untrue things. Blech. I think most serious reporters, bloggers and analysts would feel much as I do on the matter. Even the ones who do take a briefing are likely to go in with a more negative attitude than they might if the pitch email had been more closely based on reality.

And if I do ever talk with Vitria anyway, they’ll need to start by climbing out of a credibility hole.

• Actual or potential investors.
• Influencers.

More rarely mentioned is a fourth audience — actual or potential employees.  That’s a pity, in that marketing to them is a Really Big Deal. This should be obvious as soon as you consider:

• Recruiting is a hugely important form of sales.
• Where there are sales, there also is (or should be) marketing.

Each of these four “audiences” actually has many subgroups — see, for example, our taxonomy of influencers, or the new-vs.-repeat comments in our execution worksheet. But for simplicity I’m pretending today that they’re one audience each.

In the case of potential hires, this really sank in for me as I picked up more stealth-mode clients, with whom I’d have discussions like:

• “Why launch through TechCrunch? How will a post there help you get more customers, leads, or technical respect?”
• “It probably won’t — but it will help us attract engineers.”

In the case of existing technical staff, I hear more and more that senior management cares a lot about how external coverage affects their engineers. That, I was told, is what lay at the heart of my Greenplum debacle. More positively, it’s sometimes why companies go out of their way to brief me, even in sectors where I rarely influence customer buying decisions. In essence, companies market to key employees by marketing on their behalf.

I also think that academic papers and open source contributions can be viewed through a similar prism. Among other virtues, they show off smart staff and the impressive projects they’re working on, which is good for recruitment and morale alike.

Of course, it’s not just about engineers. Organizations need to encourage and recruit all kinds of employees. This is a key role of leadership. At sufficiently large enterprises, is may be a specialized human resource function as well.

And finally, you need to market to your associates just as you do to other audiences, so that they may:

• Act in accordance with your messages (i.e., prioritize making them be true!), in matters such as service delivery, development or business ethics.
• Communicate your messages to outsiders in a not-too-inaccurate fashion.

If employees (current and future) aren’t your single most important marketing target, then at least they’re close to the top.

There are three kinds of benefits performance can offer:

• It can allow you to do things more simply and/or cost-effectively (e.g., with less hardware or less tuning).
• It can allow you to do things better. Examples include:
  • Faster-loading web pages for your customers.
  • Faster-responding queries for your business analysts.
  • Better prices on your algorithmic trading.
  • Better analytic results, perhaps from:
    • Using more data.
    • Running more queries.
• It can allow you to do something that would be impractical otherwise (usually because of expense).

These benefits are easily confused. When a prospect says “I can’t do X with existing technology”, what she really means is often “I can’t afford to do X well enough to matter.” When a vendor says “We make it cheap and easy to do Y”, what prospects hear is commonly “Great! Now we’ll be able to do Y within our resources and budget.”

Given the breadth of the subject, it’s hard to generalize comprehensively about the marketing of performance claims. But my observations include: 

1. For “cheaper” to be a strong message, you have to be significantly cheaper in TCO (Total Cost of Ownership), not just in system acquisition/floor space/power costs. But raw performance is often not the biggest driver of cost, given:

• The cost and risk of technology adoption in immature markets, when experience and expertise are hard to come by.
• The cost and risk of technology adoption in mature markets, when expectations and switching costs may both be high.

2. Hardware/software purchase/license cost is a directly important performance consideration mainly to two classes of users:

• Those who want to do a lot more of something than they have been doing to date, and quail at the expense unless they change vehicles. But that’s in the “better/more” category of my taxonomy, not the “simpler/cheaper” one.
• Those who want to do something they couldn’t afford before. That goes in the “impractical” section.

I have real trouble thinking of a pure “we out-benchmark the other guys and so we’re cheaper” story that ever has won.

3. But “simpler” is a benefit that should not be overlooked. It speaks to all of operational cost, operational risk, and resource availability. Analytic RDBMS vendors brag about how little tuning their systems require. In both the Hadoop and NoSQL/NewSQL markets, ease of scaled-out cluster management is a major criterion.

4. An important sub-case of “better” is “do lots more”. Scenarios I run across frequently include (and these overlap a lot):

• We want to analyze a lot more data!
• We want to do a lot more analysis on our data.
• We’re hitting a wall with Oracle Standard Edition, and Oracle Enterprise Edition/Exadata cost much more than we want to pay.

5. If you think your story TRULY is “Our performance is so great it makes the otherwise impossible possible,” you’re kidding yourself. First, you have competitors, who also make it possible. Second, if what you’re newly making possible is all that bloody important, then probably people have already been making do to get it done as best they can, even in an inferior way.

Yes, I know there are a few exceptions. I invite you to mention them in the comment thread.

If you want to say “It can’t be done without us” as part of your marketing flair, be my guest. But please remember that what you’re saying isn’t actually true.

6. Overall, the most fruitful performance-related business-benefit positioning usually straddles “better” and “impractical without us.” For the richer or more sophisticated buyers, you’re “better”. For the laggards, you’re taking them by the hand and leading them to the Promised Land.

7. Actually, the middle layer of the layered messaging model may be more important than the top one. Your “metric” kinds of benefits may be clearer than your business benefit stories.

8. Anyhow, it’s hard to market on performance only, since performance stories are often hard to differentiate from each other. So the rest of your technical benefits may be what sets you apart from your close competitors. As just two examples:

• Multiple generations of data management technologies have been differentiated in large part by their associated development tools. Examples include prerelational DBMS in the early 1980s, relational DBMS in the later 1980s, and CEP/streaming tools in the present era.
• Memory-centric business intelligence tools are rarely differentiated from each other on performance grounds.

When I put it like that, it sounds straight out of Seth Godin. But Haidt says it in a different — and to me more compelling — way (emphasis mine):

Self-interest, political scientists have found, is a surprisingly weak predictor of people’s views on specific issues. Parents of children in public school are not more supportive of government aid to schools than other citizens. People without health insurance are not more likely to favor government-provided health insurance than are people who are fully insured.

Despite what you might have learned in Economics 101, people aren’t always selfish. In politics, they’re more often groupish. When people feel that a group they value — be it racial, religious, regional or ideological — is under attack, they rally to its defense, even at some cost to themselves. We evolved to be tribal, and politics is a competition among coalitions of tribes.

The key to understanding tribal behavior is not money, it’s sacredness. The great trick that humans developed at some point in the last few hundred thousand years is the ability to circle around a tree, rock, ancestor, flag, book or god, and then treat that thing as sacred. People who worship the same idol can trust one another, work as a team and prevail over less cohesive groups. So if you want to understand politics, and especially our divisive culture wars, you must follow the sacredness.

A good way to follow the sacredness is to listen to the stories that each tribe tells about itself and the larger nation.

Haidt then goes on to cite detailed explications of the core stories of the US Left and the US Right, and to back up his claims with a couple of examples from very recent politics:

It’s one thing for the government to insist that people have a right to buy a product that their employer abhors. But it’s a rather direct act of sacrilege (for many Christians) for the government to force religious institutions to pay for that product. The outraged reaction galvanized the Christian right and gave a lift to Rick Santorum’s campaign.

It’s one thing for a state government to make abortions harder to get (as with a waiting period). But it’s a rather direct act of sacrilege (for nearly all liberals as well as libertarians) for a state to force a doctor to insert a probe into a woman’s vagina. The outraged reaction galvanized the secular left and gave a lift to President Obama.

It would be interesting to try a similar contrast of stories for central IT vs. web-company hackers, or for relational hardliners vs. NoSQL fans. They might come out just as different-sounding as the liberal-vs.-conservative comparison in Haidt’s article.

Historically, the essence of enterprise applications has been data management — they capture business information, then show it to you. User interfaces are typically straightforward in the UI technology of the era — forms, reports, menus, and the like. The hard part of building enterprise applications is getting the data structures right. That was all true in the 1970s; it’s all still true today.

Indeed, for many years, the essence of an application software acquisition was the database design. Maintenance streams were often unimportant; code would get thrown out and rewritten. But the application’s specific database structure would be adapted into an extension to the acquirer’s own.

Examples that come to mind from the pre-relational era include:

• Bill of materials planning. This was before even my time, but it seems to have been a big part of what kicked off the whole DBMS industry, even though manufacturing applications then spent a decade not being DBMS-based.
• Order entry/accounts receivable. This was a tough problem from the mid 1970s though the mid 1980s or so. In particular, accounts receivable stumped John Landry at three consecutive companies — McCormack & Dodge, Distribution Management Systems, and Cullinet — before he claimed to finally have figured it out.
• Multi-currency support, about which I had an exchange that may be paraphrased as:
  • Pat Tinley of Ross Systems: “I’ve finally figured out how to do multi-currency right.”
  • Me: “Didn’t you tell me that at MSA?”
  • Pat: “I was wrong then.”
• Process manufacturing, and the co-products/byproducts it entailed. This led to the one significant patent suit outcome in enterprise software history, in which Marcam really did chase Ross Systems’ product off the market.

A shining relational-era example is SAP’s inclusion of workflow as a central aspect of 1990s application design.

The resulting apps, however, are cumbersome — very cumbersome. They’re cumbersome to use. They’re cumbersome to install. They’re cumbersome to change. People who use enterprise applications feel trapped in a bureaucratic hell. That is why I agree with the sentiment that operational enterprise applications are the verge of significant change.

To quote the owner of a popular sports fan discussion forum (emphasis mine):

The problem is several of the provisions in SOPA will force ISPs hosting websites (ie: the company that hosts our servers) to potentially disconnect us from the Internet if there’s a claim – unsubstantiated or not – that we’re infringing against copyright, regardless of if it has not been fully proved in court. The argument is that this would make it easy for someone to make false or weak claims against the site to take a us offline until we went to court.

That’s a headache I’m not prepared to deal with. The number of threats I get each year via e-mail from angry members from other teams we remove are pretty unreal and obviously you guys don’t see them, so giving any additional ammunition backed up by a law like this would be a potentially huge issue. I’ve been talking with other sites and it’s a very real concern that we’re all potentially going to be faced with if this goes through, unless it’s rewritten to better target the sites that are really the ones they’re looking to address.

And that’s just from the passions of sports fandom. The passions of the politics — or the commercial interests of those being criticized — are of even greater concern.

Indeed, SOPA-like legislation creates an easy way to take down any forum, blog, or other site that allows user-generated content: flood it with copyrighted content, then run to the regulators. We must never, ever, ever accept a legal regime in which publishers may be censored before they are PROVED to be guilty of wrongdoing.

the site-of-record for that startup root-system and everything that springs up from it, cycle-after-cycle

and mentor/investor/board member Mike Arrington simply called

to be the paper of record for Silicon Valley

That, I believe, is in the form a journalistic mission statement should take:

• “We (will) offer the best X about Y”, where …
• … “X” is something like news or analysis or opinion and …
• … “Y” is a particular subject area.

But there’s a problem with that template. One would ideally wish a mission statement of the form “We do the best A” to be followed up by “and, obviously, people will pay lots of money for A”. Journalistic mission statements don’t have that nice property.

Fortunately, at least in the case of tech blogging, they do tend to have a nice substitute. Let me explain.

TechCrunch and Pando Daily seem to have the same business plan:

• Create a popular and respected blog.
• Use the access provided by that popularity and respect to populate great conferences.
• Use the readership provided by that blog to promote the conferences.
• Ka-ching.

I have an analogous plan for DBMS 2:

• Create a popular and respected blog.
• Use the access provided by that popularity and respect to inform great consulting.
• Use the readership provided by that blog to promote the consulting.
• Ka-ching.

Other business models, such as GigaOm’s, would seem to be a hybrid of our two. All are what could be called “freemium” models, even if the other guys (and gals) sell a few ads as well. All seem to work.

Here’s what I think is the non-obvious part of our models:

Different parts of our readership are important for different reasons.

To a first approximation:

• Everybody who reads our work and benefits from it makes us feel good, and motivates us to do more.
• Everybody who reads our work and is influenced by it makes tech vendors want to be on our good side, talk to us, give us insight, please us by speaking at our events, and so on.
• A moderate fraction of our readers help us expand our readership by word-of-mouth.
• Only a small fraction of our readers chip in with helpful blog comments, insightful/tip-off e-mail, and the like, or by publicly throwing us links/tweets.
• Only a small fraction of our readers are likely to ever give us money.

I think a lot of successful journalistic (or quasi-journalistic) business models will be similarly layered.

• Historical notes on analytics — the pre-computer era
• Historical notes on analytic terminology (in which many terms used in this post are defined)
• Historical notes on analytics — departmental adoption (this post)
  

What set off my “history of analytics” posting kick is, simply put:

• Most interesting analytic software has been adopted first and foremost at the departmental level.
• People seem to be forgetting that fact.

In particular, I would argue that the following analytic technologies started and prospered largely through departmental adoption:

• Fourth-generation languages (the analytically-focused ones, which in fact started out being consumed on a remote/time-sharing basis)
• Electronic spreadsheets
• 1990s-era business intelligence
• Dashboards
• Fancy-visualization business intelligence
• Planning/budgeting
• Predictive analytics
• Text analytics
• Rules engines

If we leave out data management/system technologies* — e.g. data warehouse appliances or Hadoop — that’s pretty much everything that succeeded (and a couple that perhaps didn’t). I don’t know what to put on an “Enterprise-wide from the get-go” list except for a couple of duds like executive information systems and balanced scorecards.

*”System software” technologies such as DBMS often do eventually fall under the purview of central IT. But even for them there’s typically a multi-year period during which departments take the initiative in bringing them in.

Of course, this should surprise nobody; information technology is almost always adopted departmentally first, with the exceptions arising mainly in cases where departmental adoption makes no sense. Reasons include:

• The problem being solved is department-specific.
• The expertise and specific techniques to solve the problem are (or seem) subject/department-specific.
• The budget to solve the problem is department-specific.
• The best reasons to centralize technology often involve integration among departments, and new technology is rarely expected to start out being all that integrated.

Departments most likely to be early adopters (relative to others) of analytic technology seem to be:

• Finance/planning, especially in the old days when analytic technology was newer (nowadays finance might be more involved in trying to push reporting/analysis discipline out to a whole company).
• Sales/marketing, because they often have more data than other departments (actual purchase transactions, other customer contacts, and also a lot of external data).
• Investment research, because financial analysis is almost literally their core product. (Ditto trading, for very similar reasons.)

Three examples, for me, serve to bring all this home.

Business PC use famously started with individuals and departments just acquiring PCs, outside of the IT department’s control or even knowledge, way back in the day of the Apple II. Most commonly, the reason to get the PC was to run an electronic spreadsheet, generally VisiCalc.

10-15 years ago, when business intelligence vendors banged the drum for enterprise-wide BI/dashboard adoption, I’d ask them “So, do you have an enterprise-wide dashboard yourselves?” Invariably, they didn’t — but they did have departmental dashboards for sales and/or marketing. It became clear that this was a general pattern in BI adoption.

Multiple generations of technologies that one might think of as having to do with artificial intelligence – e.g. expert systems, predictive analytics* and text analytics — have wound up with applications being concentrated in the same few areas:

• Marketing
• Quality/maintenance
• Scientific/engineering research
• National security/law enforcement/anti-fraud
• Underwriting/investing/risk assessment
• Publishing (for the text-oriented technologies)

Those categories comprise 90%+ of the applications I can think of for the golly-gee-whiz technologies of their day. (You could add simulation to the list as well.) And outside of the publishing and criminal-catching sectors, those apps are pretty departmental in nature.

*I think predictive analytics has evolved into a blend of statistics and (other) machine learning, and machine learning can be viewed as a kind of AI.

So why do I think you should care about all this? Two reasons:

• History is cool.
• It has relevance to current issues in analytic technology adoption, which I plan to write more about soon.

I hope you agree.

• Historical notes on analytics — the pre-computer era
• Historical notes on analytic terminology (this post)
  
• Historical notes on analytics — departmental adoption

Discussions of the history of analytic technology are complicated by the broad variety of product category names that have been used over the decades. So let me collect here in one place some notes on how (and when) various terms have been used, specifically:

• Management information systems
• Decision support (systems)
• Report writer
• Fourth-generation language
• Executive information system
• Business intelligence
• OLAP (OnLine Analytic Processing)

Obviously, I can’t cover everything in this post. Omissions include but are not limited to:

• Anything in the data warehouse/data mart area. (For one thing, I don’t want to deal with the whole Inmon/Kimball dispute.)
• Anything in the predictive analytics area (but see the first point in a 2009 SPSS post).
• Terms I’ve recently sponsored, such as investigative analytics or machine-generated data.
• Big data (analytics) — I just discussed that mess a week ago.

The first prevalent term I recall for “information technology” was management information systems (MIS). I mention that mainly to note that it actually sounded a bit analytics-oriented, and hence to point out that in the old days — 1960s and so on — it didn’t seem necessary to name a separate category that amounted to “analytics”.

Meanwhile, the first prevalent term I recall that covered much of what we’d now call “analytics” was decision support, or decision support systems (DSS). I think DSS was always ill-defined, with multiple subcategories, just as analytics is today. The heyday of this term was in the 1970s/1980s.

Report writers were around in various forms for decades; consider for example the early 1970s history of Cullinane/Cullinet. By the time I became an analyst in the early 1980s, these were mainframe tools that let you specify paper reports, and the market leaders were probably Pansophic’s EASYTRIEVE and Informatics’ Mark IV. According to marketing, they could be used by non-programmers; in reality, they were a much easier way for programmers to do what end users asked. They were used both for one-shot queries and, as their main design point, repetitive reporting.

The report writer category then survived into the era of business intelligence (BI). (Indeed, Cognos’ big integrated BI tool early this century was called ReportNet.) More on that in the BI discussion below.

The term fourth-generation language (4GL) was widely used from the 1970s through the first part of the 1990s. Usually, a 4GL was:

• A vendor-specific programming language …
• … sold in connection with an interpreter …
• … that was particularly good at database manipulation.

In particular, the original 4GLs:

• Were primarily sold and used for analytics.
• Were often sold/used on a remote/timeshared basis.
• Often had some kind of (pre-relational) DBMS bundled in.

Classic examples of such 4GLs included FOCUS (the core product of Information Builders), RAMIS, and NOMAD; SAS arguably started out as a product of that kind too. Starting in the 1980s, however, 4GLs were used more generally, and indeed survived as an OLTP (OnLine Transaction Processing) technology long after they were supplanted by BI tools for most analytic purposes.

The last pre-BI term I want to mention is executive information system (EIS). EIS was in essence the 1980s term for “dashboard”, although the technology was much more primitive than it is today.

The term business intelligence was coined in the 1950s and then reinvented in the 1980s; however, it has described a major category only from the 1990s onward, specifically starting when GUIs (Graphical User Interfaces) became prevalent.. “Business intelligence” is sometimes used to comprise all of analytics; more commonly, however, it refers to tools focused on data selection and presentation.

These days, most of what we’d call BI comes in a single integrated package, focused on a dashboard; most of the exceptions are somewhat old-fashioned report writers. In the 1990s and early 2000s, however, business intelligence had several distinct subcategories …

… one of which was called OLAP (OnLine Analytic Processing). Actually,  the term “OLAP” has been confusingly been used to mean several different things, including:

• Pretty much all of analytics.
• A particular non-relational DBMS architecture that I prefer to refer to as MOLAP (Multidimensional OLAP).
• An integrated suite of DBMS, 4GL, and perhaps other tools around a MOLAP architecture. (Examples: The IRI Express and Arbor/Hyperion Essbase products Oracle bought.)
• A client-side BI tool with a little MOLAP DBMS built in. (Example: Cognos’ erstwhile flagship BI product PowerPlay.)

I hate the term “OLAP” with a passion, in part due to that confusion, and in part due to the specific way the confusion came about: Ted Codd introduced the term, allegedly objectively, but actually as a marketing shill for Arbor Software, which had an obvious business incentive to pretend that its specific technologies solved a broader class of problems than they actually did.

OK. With that too cleared away, I feel ready to write about the actual history of analytic technology.

• Historical notes on analytics — the pre-computer era (this post)
  
• Historical notes on analytic terminology
• Historical notes on analytics — departmental adoption

Sometimes, what people describe as being “New, new, new!!!” in analytics has actually been happening since before they were born, or even before their parents were. Occasionally, I point this out. I think it’s time to collect some of those observations into a short series of posts.

Before getting to the history of actual analytic software, I can’t resist racing through some really old stuff. In a 2004 white paper, I wrote:

Transactional business processes have been around literally since the beginning of recorded history. Some of the oldest known writings are clay tablets that record merchants’ tallies in Sumerian cuneiform, complete with seals to enforce transaction integrity. Analytic business processes date back nearly as long, especially in military applications; the first chapter of Sun Tzu’s The Art of War is called “Calculations,” or in some translations “Laying Plans.”*

As enterprise complexity increased, so did the sophistication of analytic business processes. Almost two centuries ago, Nathan Rothschild made an investment fortune from early news about the Battle of Waterloo, and several decades later Florence Nightingale** introduced statistics to the study of public health. With the invention of machines to tabulate information in the late 19th Century, analysis began to blossom.

*Back when I wrote that, I also considered including some of the accounting Caesar cited in his Commentaries on the Gallic Wars, but eventually decided that was a “production application” rather than anything we’d recognize as “analytics”.

**Florence Nightingale is, simply put, one of the more impressive women in history. Unfortunately, a couple of other statistical greats were associated with the deplorable subject of eugenics. I am thinking specifically of Francis Galton, who invented several really basic statistical concepts, and seems to generally have been a brilliant guy, and Karl Pearson. (Hat-tip to John Verostek for, um, tipping me off to Galton.)

Statistics also seems to have led the way in business applications of analytics. Specifically, statistical quality control dates back to the 1920s or so, pioneered by Walter Shewart and given greater visibility by his protégé W. Edwards Deming. On the monitoring side, various organizations collected industry-wide numbers by the 1930s or so. For example, movie box office receipts were reported at least as far back as 1939, perhaps by Variety; the New York Times bestseller list seems to have started slightly later, in 1942; and a rather superficial site on media history first gives hard numbers for the decade of the 1930s.

Still, to continue quoting the same white paper:

What utterly transformed both transactional and analytic business processes was the advent of electronic computing in the 20th Century. In particular, the volume of available data exploded. Even more important to analytic processes was the superhuman increase in the speed of computation. Various types of software emerged — business intelligence (BI) tools, spreadsheets, statistical packages, and the like – permitting kinds of analysis that had been infeasible without computers.

And with that out of the way, let’s return to discussing computer-era analytics.

Denis Pombriant obviously attended the same Marc Benioff session I did. Dion Hinchcliffe blogged the whole story in considerable detail.

It’s a cool story, and worthy of attention. But I’d like to step back and remind us that there are numerous different ways to use social technology in the enterprise, which probably shouldn’t be confused with each other. And then I’d like to discuss one area of social technology that’s relatively new to me: integration between social and operational applications.

Suppose we split up social technology use cases by saying it can help you:

• Communicate and collaborate internally …
• … and also with small groups of outsiders, such as your supply chain.
• Observe, listen to, and interact with consumers (and the world at large).

The biggest buzz, of course, is around social technology that reaches out to the buying public or world at large. You can use social technology to:

• Observe and listen to consumers — i.e., classic Voice of the Customer/Voice of the Market text analytics.
• Publish to consumers, influencers, etc., via blogging, broadcast-oriented Twitter, and other social media, or go even further and …
• … communicate with consumers interactively, whether through loosely-structured interaction (e.g. Twitter), or in the more structured ways that Attensity and others provide.

I support all that, and indeed participate ferociously myself. But for now, let’s move on.

On the internal collaboration/communication side, I’d say:

• Any communication tool useful for communicating with the public may be valuable internally as well — portals, blogs, Twitter-imitators, and so on.
• Pure email “push” may not always be the best tool for point-to-point internal communication.
• Text analytics on internal communication can have a variety of uses, e.g:
  • Compliance (yet another privacy intrusion, but sometimes a legitimate one).
  • Internal expert-finding. (In principle, this is the traditional genuine benefit of elaborate “knowledge management” implementations, but without the burdens of traditional knowledge management. In practice, that didn’t work out so great for Tacit Software.)
  • Project management.

That all gives plenty of scope for useful adoption, on both the email-replacement and text-analytic sides. But again, let’s keep going.

The relatively new to me — notwithstanding the “portals” link above — part of the social technology story is integration between social and operational applications. While at Dreamforce, I talked with two manufacturing application SaaS vendors — Kenandy and Rootstock Software. In both cases I asked “So what are you doing that’s an advance over where MRP was 20 years ago?” In both cases the main answer was “Now users can use social technology to track and communicate about particular orders or issues.”

*MRP stood for “Material Requirements Planning” and then “Manufacturing Resources Planning”, and is essentially the  forerunner of ERP. By “Kenandy” I specifically mean Kenandy’s founder — ASK Computer Systems founder and thus MRP legend Sandy Kurtzig.

Good point. Of course, it can be generalized; one can communicate and collaborate around almost any kind of business process. I’ve mentioned this before in analytic contexts; it’s an important concept on the monitoring-oriented side of business intelligence and — if Oliver Ratzesberger is to be believed — in investigative analytics as well. But the operational side may actually be more important.

Some things one does in the business world actually involve using one’s body, from manufacturing products to repairing power stations to standing in a store and serving customers. Most of the rest fits into one or more of three buckets:

• Creating (a product, a marketing plan, a marketing document, a compensation plan, a program for internal use, an analytic insight, …)
• Relating (to an employee, a sales prospect, a reporter, …)
• Participating in a fairly routine business process (data entry, accounting, mortgage approval, parts ordering, …)

And why can’t we just automate those routine business processes away? Because there’s so often a need for manual intervention. And when there’s a need for manual intervention, there’s usually also an element of communicating with other people. This is almost always true in cases of trouble-shooting or exception-handling (an order is late, a system is down, the automated result violates common sense). It may be present in other cases as well (the new account calls for a personal thank you note, the food order needs to be annotated with special requests). General email is commonly an awkward medium for these communications; automated messages are worse. Newer social technologies, however, have the potential to do much better.

So what do you think? Have I drunk too much Kool-Aid, or is this stuff for real?

• Oracle started out doing what amounted to custom development for government (military/intelligence) clients.
• Even when Oracle said it had productized its software, the stuff didn’t work very well without services to get it running.
• Oracle and Ingres both got a huge fraction of their early revenue* from deals to port their software to various brands of hardware.** That’s a lot like professional services.
• Oracle’s huge Tools Group grew out of professional services, if I have the story straight. Indeed, its first product was written by later long-time group chief Sohaib Abbasi when he was a consultant.

*Revenue recognition rules were rather different back then. Multi-million payments or guarantees for ports could be recognized as lump-sum revenue up front.

**Ingres once ran on more hardware platforms than it had employees, when both numbers were somewhere in the 40s. Most of the boxes on which the porting was tested were in one small office.

The benefits for a young software company of being in the professional services business include:

• It can be high-margin, especially if it shares the cost of sales with your software offering.
• It allows you to say “yes” to whatever the customer asks for. (More precisely, it takes you closer to that goal that you’d be without a service offering.)
• It allows you to fund capable staff. Or to put it even more bluntly …
• … professional services brings in revenue that pays your bills.
• It gets you involved with customers, learning stuff about their needs, and specifically addressing them.

The disadvantages of professional services generally boil down to various forms of defocus; you can screw up your development schedule, your development priorities, your sales priorities, your partnering efforts, your market positioning, your burn rate or just about anything else.

Many software companies pursue substantial professional services when they’re young. Many don’t. Both strategic choices can make sense.

*This is although I’ve spoken at four prior versions of the event, and responded to their request for free consulting as recently as this year.

OK, that might have been personal in some way — but Nick Patience tweets a very similar story. Even Seth himself tweets that

They have a business model that does not apply well to the IT conference space.

The Text Analytics Summit has been troubled for years, but evidently things have gotten worse.

This is more than an incidental problem. Interest in text data is exploding, and marketplace confusing about text analytic technology abounds. More clarity is needed, but too few folks have found an economic model for providing it. (The industry shares some of the blame for that.) I’m glad Seth is doing other conference work — notably on sentiment analysis — but yet more is needed.

If I get into the conference business — and it seems natural that I would — I’ll try to help fill the gap. But if somebody else beats me to the punch, more power to you, and please let me know how I can help.

This was all slightly before my time — I only entered the industry and met Software AG in 1981. So does anybody else out there recall more of the story than I do?

• Understanding the opinions of customers, prospects, or other groups. This can be based on any combination of documents the user organization controls (email, surveys, warranty reports, call center logs, etc.) — in which case — or public-domain documents such as blogs, forum posts, and tweets. The former is usually called Voice of the Customer (VotC), while the latter is Voice of the Market (VotM).
• Detecting and identifying problems. This can happen across many domains — VotC, VotM, diagnosing equipment malfunctions, identifying bad guys (from terrorists to fraudsters), or even getting early warnings of infectious disease outbreaks.
• Aiding text search, custom publishing, and other electronic document-shuffling use cases, often via document augmentation.

For several years, I’ve been distressed at the lack of progress in text analytics or, as it used to be called, text mining. Yes, the rise of sentiment analysis has been impressive, and higher volumes of text data are being processed than were before. But otherwise, there’s been a lot of the same old, same old. Most actual deployed applications of text analytics or text mining go something like this:

• A bunch of documents are analyzed to ascertain the ideas expressed in them.
• A count is made as to how many times each idea turns up.
• The application user notices any surprisingly large numbers, and as result of noticing pays attention to the corresponding ideas.

Often, it seems desirable to integrate text analytics with business intelligence and/or predictive analytics tools that operate on tabular data is. Even so, such integration is most commonly weak or nonexistent. Apart from the usual reasons for silos of automation, I blame this lack on a mismatch in precision, among other reasons. A 500% increase in mentions of a subject could be simple coincidence, or the result of a single identifiable press article. In comparison, a 5% increase in a conventional business metric might be much more important.

But in fairness, the text analytics innovation picture hasn’t been quite as bleak as what I’ve been painting so far. While standalone, passively-reported text analytics is indeed the baseline, there are some interesting exceptions. For example:

• I once confirmed that SPSS customer Cablecom‘s statistical models for churn and the like absolutely included text data; Cablecom even assigned different weights to the same apparent level of emotion depending on whether the text was in German, French, or Italian. Vertica recently told me of a Vertica/Hadoop customer doing something similar, except for the multilingual aspect. And the end of a 2008 SAS-based paper makes similar claims.
• There long* have been some examples of fact extraction that don’t really fit into my three buckets above. For example, researchers mine collections of articles to try to determine biochemical or biological pathways that would not be apparent from examining single research studies alone.
• It also has long* been the case that some bad-guy-finding applications — especially in the anti-terrorism area — used text analytics to populate state-of-the-art graph-oriented data analysis tools.

*When it comes to text analytics, “long” means “at least for the past several years.”

In more recent examples:

• Greenplum built a document recommender for law firms that does hard-core statistical analysis to determine which .1% of a document set lawyers might actually want to see, and which then learns from users’ feedback after they respond to initial result sets.
• Information extracted from investment news gets included into automated trading algorithms. This was unusual technology a couple of years ago, but is more common today.
• After a series of mergers, Attensity now uses marketing-oriented text analytics in at least three different ways:
  • Attensity text analytics feeds marketing dashboards just as it always did.
  • Attensity text analytics triggers alerts, as I wish dashboards and business intelligence tools more often did, the false positives problem notwithstanding.
  • Attensity text analytics triggers concrete workflows, for example routing specific social media hits for priority response.
  • And in one example that did not actually get into production, a very large social networking company correlated word usage (e.g., choice among different synonyms) against user characteristics such as age and gender.

Finally there are some applications that, while fitting the standard template, just strike me as getting to unusually sophisticated levels of analysis. For example, Vertica told me of another Vertica/Hadoop case where VotM document analysis is carried out to the level of observing which order brand names appear in, and adjusting that for whether or not it was just an alphabetical list.

I suspect text analytics is about to become more interesting again.

Related links

• The enabling technology for text/tabular data integration has existed for years.
• In 2006, I listed major application areas for data mining/predictive analytics. It overlaps pretty closely with the similar list for text mining/text analytics.
• Before being acquired by IBM, SPSS boasted a rather large text mining user base.

• TechCrunch got sold, GigaOm raised money, and VentureBeat/MediaBeat provided a good starting link for both those stories and more.  Since TechCrunch and GigaOm are/were both private, financial details are murky, but:
  • TechCrunch is variously reported as having revenue in the $6-10 million range, probably mainly from events. (If you believe that they sell ~3000 total tickets at ~$2000 each to two annual versions of TechCrunch Disrupt, that makes sense.)
  • GigaOm reports >10,000 subscribers to market research sevice (sort of) GigaOm Pro, at $199, apparently concentrated on the vendor side.
• John Gruber straightforwardly posts both ad rates and circulation for his blog. It’s a simple $5000/week for readership that exceeds mine by >1 order of magnitude.
• The New Yorker points out Gawker Media may not yet have crossed $20 million in revenue.
• An “ASCAP for news” seems to finally be on the way.
• Business Week/Bloomberg notices a trend that social-media/Voice of the Customer/Voice of the Market text analytics firms are getting acquired by bigger marketing-oriented firms. Seth Grimes, however, argues that the same trend is already passe’.
• TechCrunch accused the Wall Street Journal of killing a story about sister company MySpace, then quickly running it after TechCrunch caught them.
• LinkedIn has a really cool-looking tech blog. One recent post describes LinkedIn’s approach to socially-informed search. I read about it in a thoughtful post on Daniel Tunkelang’s blog.
• Bill Simmons took 3843 words to explain the story of a two-word tweet — “moss Vikings.” Somewhere in there are a few interesting ruminations about media in the current age.
• Some notes and links that actually belong here instead went up on DBMS 2 a few weeks ago.
• About half of what I write about liberty and privacy is highly relevant to the subjects of this blog, including almost all of today’s post.

Dear friend, How are you recently? I bought a laptop from a China company T0SHPD last week(the site is :www.toshpd1.com), and I received it now. The products are high quality with a very low price. They also sell mobile phones, TV, games, and so on. They are from Korea, Japan. You can go to their site to have a look, I am sure you will get many surprise and benefits. Best regards. h–)

It is in her Yahoo Mail Sent folder. It is not in the Sent folder for her desktop client (Outlook Express). Her computer passes a malware scan.

The site named first in the text looks to sell the kind of merchandise described. It has Whois record:

Domain Name: T0SHPD.COM
Registrar: MELBOURNE IT, LTD. D/B/A INTERNET NAMES WORLDWIDE
Whois Server: whois.melbourneit.com
Referral URL: http://www.melbourneit.com
Name Server: NS1.DNSPOOD.NET
Name Server: NS2.DNSPOOD.NET
Status: ok
Updated Date: 14-aug-2010
Creation Date: 14-aug-2010
Expiration Date: 14-aug-2011

Its Google footprint seems to be a few dozen copies of that spam message, plus the default hits one gets for any domain with a live site.

The site actually linked via the URL has Whois record:

Domain Name: TOSHPD1.COM
Registrar: MELBOURNE IT, LTD. D/B/A INTERNET NAMES WORLDWIDE
Whois Server: whois.melbourneit.com
Referral URL: http://www.melbourneit.com
Name Server: NS1.DNSPOOD.NET
Name Server: NS2.DNSPOOD.NET
Status: ok
Updated Date: 23-sep-2010
Creation Date: 23-sep-2010
Expiration Date: 23-sep-2011

Its Google footprint is very small.

The recipients are:

• Linda’s husband, daughter, daughter’s boyfriend, sister
• Linda’s close friend
• A mailing list to which Linda posts
• A friend with whom Linda has emailed exactly once in the past 3 decades
• A person whose name and address Linda doesn’t recognize

There’s no news of a Yahoo Mail attack going around that I could detect.

The password on Linda’s Yahoo Mail account (since changed) was not ridiculous, not brilliant, and not specific to that site alone. So the simplest theory is that her account was hit randomly, with her password being either:

• Guessed
• Repurposed from some other site she registered at.

Do any other plausible theories come to mind?

Be careful out there, people.

In particular, he argues:

• The value of journalism can only be assessed in connection with how people use it …
• … and their lack of enthusiasm about New Media news is a warning sign.
• Technology and form factor matter; imitating old media is likely not the best way to go.
• Personalization and targeting need to be a lot better. In particular:
  • What’s most important is getting stories to the people who are likely to want to act on what’s in them. The true value of journalism lies in informing people’s choices and actions. (By contrast, he seems to denigrate the other main benefits of news, which are pure entertainment and/or the facilitation of social interaction.)
  • It’s OK and natural that the people inclined to act — on  a given story or indeed at all — are only a small fraction of the overall population.

I am in vehement agreement with much of what Stray has to say, although I think he understates the importance of general knowledge and the often serendipitous benefits of pursuing same. For example:

• I tend to assume that what we write affects people’s choices by supporting their informed judgments.
• I think it is neither necessary nor acceptable to let investigative reporting wane.
• I have witheringly negative opinions about vacuous “news.”

And I indeed try to practice what Stray preaches. Most of my own posts — especially when you weight them by length and/or time spent researching and writing them — are designed to help at least some people make on-the-job decisions.

• I do just mean “help,” the assumption being that people read my work as part of a general research process.
• That lots of you read more for general interest or education is great. I suspect you still like the standard of quality to which I aspire, namely that what I write should in most cases actually be informative even to people who have reason to be well-informed in the area already.

There are still some things we need to do to preserve and extend the societal benefits of investigative reporting. But they are straightforward and very likely to happen.

Specifically, I recommend: 

• Public-spirited law-oriented types should do a better job of popularizing tips for how to get information out of government (Freedom of Information Act and all that). And back it up with more pro bono or charitably-funded legal assistance – not just for specific causes, but for general corruption investigations as well.
  • I’m sure quite a bit of that is happening, but it should be much more visible and active.
• Domain-specific websites should be created and promoted that seek out and call attention to negative stories in their particular areas, especially for specific industries or geographical regions.
  • A lot of those exist targeted at specific large companies people have grudges against, but otherwise they’re much too hard to find.
• Reporters need to be in the habit of seeking out stories first uncovered by other people.
  • They do this already, but they need to get better.

Below, at considerable length, is why I think those developments are both necessary and sufficient to carry the tradition of investigative journalism forward into the new media era.

For there to be public benefit from reporting, three things generally need to occur:

• Disclosure or discovery of the raw facts. Without that, you don’t have reporting or news.
• Analysis or interpretation. This stage can be optional when the purpose of news is entertainment, societal bonding, or whatever. But it’s pretty central to investigative journalism.
• Distribution and popularization. It doesn’t do much good to uncover an important story unless people notice and care about it. Old media, with its emphases on writing, curation, and physical distribution, almost defines itself by this stage. (E.g., “paper” is part of the word “newspaper.”)

Disclosure and discovery come in two main forms:

• Serendipity.
• Spadework.

The serendipity part often seems to work well in the new media. Let’s go to some examples.

• Wikileaks is a hugely successful case – people send Wikileaks documents or other files (a process that only makes sense with modern technology), and Wikileaks posts them.
  • Note: There was an  article yesterday about “internal strife” at Wikileaks – but the gist turned out to be that Wikileaks, already highly influential, could be doing even more than it already is.
• Michael Arrington found out about a meeting of major angel investors – perhaps originally via a tweet – and kicked off a major technology industry news story now known as “Angelgate”.
• An anonymous tipster spent 2 ½ hours IMing with me to reveal the true cause of the JP Morgan Chase site outages.
  • Motivation: Because s/he felt Chase’s technology organization was being unfairly maligned by prior coverage.
  • Why me: Because my previous speculative post about the JP Morgan Chase outages had shown up in the search engines and looked pretty credible.)
  • Result: Enough accurate tech details of a major consumer embarrassment to create a “teachable moment,” even though the concerned parties were trying to cover them up.
• An assisted living/nursing home in Dublin, Ohio called Friendship Village misbehaved toward my parents and me. I blogged about the problem, and it’s in the search engines now. If this turns out to be a pattern of behavior rather than an isolated incident, they’ll have some deserved trouble.

The story on the spadework side is more mixed. For example, there’s evidence I did as good a job on the JP Morgan Chase story as conventional media could today – Computerworld ran a story based on my post, without being able to uncover a single detail I hadn’t already found. But perhaps in the old-media-economics days, perhaps Computerworld would have had the resources to try harder and find something I didn’t. (E.g., I screwed up and didn’t actually get the details of the specific Oracle bug.) A bigger problem is outlined in this story on the uncovering of massive corruption in the California town of Bell. To wit (emphasis mine):

The new media ecosystem, in which citizen bloggers, small news outlets and big old-school media outlets effectively draw upon one another’s work to collaborate, didn’t quite work out in this case.

One blogger actually has anonymously and exhaustively alleged corruption in Bell for years …

The paper’s reporters say the blogger gave them tips. Though he’s a bit frustrated not to get more credit, he says the newspaper’s reporting muscle and much bigger audience gave life to the story in a way his website simply couldn’t. He counts his readers in the scores; The L.A. Times has hundreds of thousands of subscribers …

… some residents said they had gone to city hall to get their own answers. In essence, they were trying to do their own reporting on why their tax bills were so high and on rumors city officials were making a ton of money.

They got nowhere. …

“As a common citizen, I don’t know what my rights are with the city. I don’t know really how to attack them,” Sanchez said. “The Times, they have their legal departments. Of course, they’re able to get it more than a regular Joe like me.”

The citizens of Bell needed some place to turn for help, other than the overworked LA Times reporters who eventually uncovered the story on their own. Hence my first recommendation near the top of this post.

In many ways, analysis and interpretation work well in the new media era already. After all, there’s a whole world wide-web of self-appointed volunteer analysts on any issue you’d care to name! Yes, there are legitimate concerns about fragmentation and echo chambers, in which people only listen to the analysis of those folks who shared their biases to begin with. But those are hardly a barrier to muckraking – if anything, quite the contrary, as illustrated by the bogus ACORN prostitute/pimp advice scandal. (If your politics lean to the conservative side, think instead of something like a Michael Moore film.)

Or returning to the examples above:

• Wikileaks’ biggest leaks are widely analyzed by all sorts of commentators, including top-flight mainstream media people and a broad variety of online commentators alike. I’ll confess I didn’t find any analysis of Wikileaks’ revelations about, say, Iceland or the Turks & Caicos Islands, but I’ll also confess to not looking very hard.
• For the technology news uncovered respectively by Arrington and me, pretty much the ideal people to analyze it were, respectively – well, they were Arrington and me.
  • In the case of Angelgate, much other analysis (and news) ensued.
  • Analysis of the JP Morgan Chase outage details hasn’t yet gone all that far past me – but I already turned it into a “don’t make the same mistake JP Morgan Chase did” lesson.
• The Friendship Village case is being used as a cornerstone of my slowly-unfolding analysis of the general problem with medical records.

And that brings us to distribution and popularization. The most brilliant sleuthing in the world doesn’t help people very much if they – or their lawmakers/regulators/advisers/whatever – don’t find out about it.

• Wikileaks has that problem solved for its biggest leaks, but perhaps not for the others.
• Arrington’s TechCrunch is a top news outlet in his area, so the problem was automatically solved for him.
• DBMS 2 is a fairly serious outlet for database-related news. But in any case the JP Morgan Chase story was picked up by general trade press and financial-industry-specific press alike.
• As noted in the story on Bell, CA, nobody was paying attention to a blogger who apparently had worked quite a bit of it out.
• And if there’s anything you found lacking in my list of analysis/interpretation examples – well, if a story were picked up more broadly, then analysis/interpretation might also be stronger as well.

Almost nobody would ever see my Friendship Village story if I didn’t happen to own some websites with strong search engine authority. And how high it stays in the rankings as it ages still remains to be seen.

Possible answers take two main forms:

• Aggregation and curation,* in which various contributions are bundled together at go-to websites or the like.
• A reporting feeding chain, in which journalists with broader reach:
  • Steal/borrow/take ideas from more specialized contributors.
  • Repackage them.
  • Perhaps add additional value in reporting, analysis, or presentation. (Several examples of this may be found in the links above.)

Investigative reporting needs more of each.

*The latter is the more high-falutin’ version of the former.

Consider my story about Friendship Village. Standing alone, it’s not going to influence much of anybody, except insofar as I can personally influence the course of medical database design or privacy law. But suppose one person each reported similar things at 20 different institutions. A journalist who wrote a story based on those reports could carry a lot of sway, perhaps:

• Influencing the course of medical information exchange in the United States, or at least
• Alerting people to the lengths they have to go to get proper information about and before their sick relatives.

Similarly, suppose there were a go-to website for complaints about assisted living facilities. Well, people considering moving into Friendship Village would have a little concern to address. Even better, the very existence of that site might help motivate people to share more stories. Bad institutions would need to reform, and bad practices might be reformed under the spotlight of public scrutiny.

If this isn’t my longest blog post ever, it’s surely close. So while I have much more to say on these subjects, I’ll stop here. Comments and examples are warmly encouraged.

To promote the Progress of Science and useful Arts, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries;

but that’s just a longer way of saying the same thing.

Why does “securing … exclusive Right[s]” to the creators of things that are patented, copyrighted, or trademarked help make it a good decision for them to create stuff? Because it averts competition from copiers, thus making the creator a monopolist in what s/he has created, allowing her to at least somewhat value-price her creation.

I.e., the core point of intellectual property rights is to prevent copying-based competition. By way of contrast, any other kind of intellectual property “right” should be viewed with great suspicion.

Examples of my views include:

• In a recent comment I pooh-poohed an expansive interpretation of the GPL, even as I supported the GPL in core cases.
• I believe that most kinds of software patents are or should be invalid, but I’m willing to make an exception for innovative user experiences.
• I believe in copyright, even though I agree with consensus that in many cases copyright-holders’ business models will evolve away from the licensing of intellectual property. For example, I’m mightily annoyed when somebody claims my words as their own. But I give mine away for free. I just want to get the reputational benefit of what I write, and also to aggregate comments on my original blog posts rather than having them go to some other site.

The embeds of the future will work for the company, and be paid by the company to provide news about the company in a multitude of formats. Print, newsletter, video, blog, podcast, moving billboards, tattoos — whatever it takes. Because the bits and pieces of Corporate America that have a story to tell will still have their stories – just no ready outlets.

How is this different than what you have today? Surely there are corporate PR departments and external agencies already doing these things, right?

No.

What is required is an internal producer who writes in external voice — like the neutral point-of-view so often described by Wikipedia. People can smell marketing and propaganda coming around the corner, and they know when the pitches and puff pieces are missing that edge of neutrality. An accurate and fair piece is accurate and fair, no matter who writes it.

It’s an interesting theory, but it seems to presuppose dual marketing communication efforts, with separate departments of “Straightforwardness” and “Hype”. That may work at some companies, but in most cases I think it will be more practical to try to infuse straightforwardness through multiple parts of the marcom effort.

My more specific quick responses include:

• That sure sounds a lot like Robert Scoble in his Microsoft days.
• It also sounds like “community managers” at MMO game companies. (Both of the MMOs I’ve played have had great ones.) They often only use one or two channels (forums and the associated general website), but otherwise they fit the bill.
• Ike’s views fit very well with mine on the future of the information ecosystem.
• I’m getting ever more sympathetic to the idea that you need people whose main job is external communication of a straightforward kind. Reasons include:
  • Senior executives who write great blogs commonly don’t keep them up. And even when they’re active, the blogging is pretty sparse. E.g., among companies I follow closely, Vertica, Aster Data, and Netezza have all done some outstanding blogging in the past, but do very little of it now. Only Dave Kellogg at Mark Logic really keeps going.
  • It’s not obvious that senior executives are wrong to spend their time at something other than blogging. One of the greatest vendor blogs ever was Jonathan Schwartz’s at Sun.  Umm — how sure are we that he actually did much good for his company with that effort?
  • I frequently tell vendors “If you tell Story X in your own words, I’ll gladly point to it or post it for you.” They usually agree this is a wonderful idea — but then usually don’t free up the rather limited resources that would be required to take me up on it.
• That said, the kinds of people who provide customer support (pre- or post-sales) are often very well suited to fill the role Ike is describing. At least, that’s the case in enterprise tech companies.
• The media mix isn’t really as complex as Ike was suggesting. It basically falls into two groups: Text, and audio/video.
• That said, text/graphics and audio/video media people are increasingly the same. (Just think of sports media, where the newspaper folks make their big bucks on radio or TV. That’s a harbinger of the future. Or think again of Scoble.)
• One flaw of Ike’s idea is that in its pure form it only makes sense for companies large enough to have multi-person PR staffs. Other firms would have to use part-timers, or outsource.  And if you’re going to do that, might it not make more sense to pay part of the cost of sponsoring, you guessed it, an independent blog?
• I know that’s text/graphics-only, or at least text/graphics-mainly, but I happen to think audio/visual business news/PR is minor anyway. People may give enough attention to, for example, listen to audio from a company if it purports to teach them something. But news ABOUT a company? Who’s so interested in that to sit still for audio/video, unless they happen to be employees, or investors in its stock?

Bottom line: I think he’s wrong about some of his detailed views, but Ike Pigott is directionally very right in suggesting that newsmakers will increasingly become content creators for news about themselves.

• For starters, let me link to some of the posts I’ve made pointing to April Fool’s pranks in past years, including:
  • (2009) A wonderful spoof of the analyst business
  • (2009) Donald Farmer’s hilarious version of business intelligence
  • (2009) The Guardian’s translation of its news and archives to tweets (“OMG Hitler invades Poland, allies declare war see tinyurl.com/b5×6e for more”)
  • (2009) Google’s world-dominating, blog-writing AI with the personality of a pre-adolescent girl
  • (2009) Expedia’s space-travel offering
  • (2008) Netezza’s green box
  • (2008) LOTRO’s spoof quests — like other MMO folks, the Lord Of The Rings Online guys can be really funny. (But in retrospect I’m not so sure they were spoofs so much as a new comedic option in the game introduced on a cleverly-chosen date.)
  • (2007) My attempt to one-up Scoble et al., without much success.
  • (2002) A classic: Google PigeonRank
• I found a couple of sites that catalog April Fool’s pranks around the world (not just techie ones). The Museum of Hoaxes offers a curated approach, so their list is pretty funny. Another site lists just about every web hoax anybody bothers to submit, so it’s quality is more mixed (and a lot of the links now don’t work).
• While thinking about this post, I recalled and posted about some software industry pranks. The MSA/M&D ones still boggle my mind, but I couldn’t think of much else to match them.
• And then, of course, there was the time this blonde joke made, as it were, the rounds.

Stancle had been accused of creating a Facebook profile belonging to a nonexistent teenage girl and then, between approximately the spring of 2007 and November of 2008, using it to convince more than 30 of his male classmates to send in nude photos or videos of themselves.

Stancl then reportedly threatened to post the photos or videos on the Internet if they didn’t engage in some sort of sexual activity with him. At least seven of them have said they were coerced into sex acts, which Stancl documented with a cell phone camera.

Stancl’s victims were teenage boys focused on sex — not exactly the world’s clearest thinkers. Even so, I find it remarkable that multiple people would:

1. Send nude photographs of themselves to a stranger.
2. Be so concerned about those photographs getting published online that they would submit to sexual blackmail.
3. Allow the results of their sexual blackmail to be photographed.

Literally — WTF??

• The generalities I posted a few years ago still apply (and, I think, are a good read in any case about the realities of analyst coverage).
• The updates a year and a half ago are still very accurate, although I might name different specific clients today.
• The partial client list from half a year ago is still pretty accurate, although Microsoft and Kognitio have dropped off, Clearpace changed its name to RainStor, and non-RDBMS analytic data management/analysis contenders Cloudera and Splunk have been added.
• While I have user clients, I have nothing to disclose about them.

One new development is that for the first time since 2001, I’ve taken stock in a private company. It’s Petascan, a seed/stealth-stage outfit with some very innovative ideas about how to use Flash memory in support of analytic data processing.  I’d like to do more of this, with conflicts evaluated on a case-by-case basis.  For example, I bet I could bring a lot of value to vertically-oriented analytics start-ups, who would at worst compete with only a small fraction each of the business of the more horizontally-oriented companies I generally write about.

For the past several years, we’ve had an annual refresh of our vendor service offerings, always unveiled in the fall. This year has seen more change than usual, and so I’d like to share some of the highlights with you here. A revampimg of our services for users is in the works as well, and I’ll share that too with you when it is finalized.

Aspects that haven’t changed much include:

• We ask all vendor clients to join a program called the Monash Advantage.
• Monash Advantage members get effectively unmetered quick-inquiry consulting, and more in-depth advice sessions as well.
• Our speaking and writing services, which vendors like to use for lead generation and general image-buffing, are generally restricted to Monash Advantage members
• The entry-level Monash Advantage price is $10,000/year

The biggest change from prior years is that there are now three tiers of the Monash Advantage, up from one.

• The Monash Advantage Lite is for small, tightly-focused companies with severe budget constraints. We offer suggestions and help them think through their most pressing issues, a few times each year.
• The Monash Advantage Basic is for more typical technology companies. We help them with anything and everything.
• The Monash Advantage Custom is for companies that want us to serve as core strategic advisors.

The early response to this tiering has been very positive, and we have had multiple sign-ups for 2010 at each of the three levels.

Another change is that we no longer require companies to join the Monash Advantage on a strict calendar-year basis. Now, it’s calendar quarters, and for Custom members we’re completely flexible.

Finally, we’re open to doing stock deals with seed-stage companies, at least ones that don’t compete closely with our other clients. For example, I’ve just started advising one stealth start-up in a hardware area that complements analytic DBMS, and I’m having a blast.  I’ll disclose the names of any companies I have private stock in, as well as offering at least a capsule of what is publicly known about what they’re pursuing.

]]> Monash Research highlights http://feedproxy.google.com/~r/monashreport/feed/~3/A9rIapThYD8/ http://feedproxy.google.com/~r/MonashInformationServices/~3/ArGcDB-FYsU/ I prepare most of my blog posts in OpenOffice. Most of the rest I write directly online in WordPress. I almost never use Microsoft Word. The reason, simply put, is cruft. When I copy a post from OpenOffice to WordPress, I invariably get a line at the top that looks like <!–         @page [...] http://www.monashreport.com/?p=208 Thu, 03 Sep 2009 10:09:53 +0000 I prepare most of my blog posts in OpenOffice. Most of the rest I write directly online in WordPress. I almost never use Microsoft Word.

The reason, simply put, is cruft.

When I copy a post from OpenOffice to WordPress, I invariably get a line at the top that looks like

<!–         @page { size: 8.5in 11in; margin: 0.79in }         P { margin-bottom: 0.08in }     –>

I delete that, which according to OpenOffice stats amounts to exactly 100 characters; I fiddle with the bullet points a bit; I add a title, categories, and a MORE separator; and I’m basically good to go.

By way of contrast, in a recent post I copied a sentence from a press release I’d recieved across Google Mail in .DOC format, forgetting to stage it into OpenOffice first.  The cruft I needed to delete consisted of 6489 characters, namely:

<!– /* Font Definitions */ @font-face {font-family:Wingdings; panose-1:5 0 0 0 0 0 0 0 0 0; mso-font-charset:2; mso-generic-font-family:auto; mso-font-pitch:variable; mso-font-signature:0 268435456 0 0 -2147483648 0;} @font-face {font-family:”MS Mincho”; panose-1:2 2 6 9 4 2 5 8 3 4; mso-font-alt:”ＭＳ 明朝“; mso-font-charset:128; mso-generic-font-family:modern; mso-font-pitch:fixed; mso-font-signature:-536870145 1791491579 18 0 131231 0;} @font-face {font-family:Tahoma; panose-1:2 11 6 4 3 5 4 4 2 4; mso-font-charset:0; mso-generic-font-family:swiss; mso-font-format:other; mso-font-pitch:auto; mso-font-signature:3 0 0 0 1 0;} @font-face {font-family:Verdana; panose-1:2 11 6 4 3 5 4 4 2 4; mso-font-charset:0; mso-generic-font-family:swiss; mso-font-pitch:variable; mso-font-signature:-1593833729 1073750107 16 0 415 0;} @font-face {font-family:”\@MS Mincho”; panose-1:2 2 6 9 4 2 5 8 3 4; mso-font-charset:128; mso-generic-font-family:modern; mso-font-pitch:fixed; mso-font-signature:-536870145 1791491579 18 0 131231 0;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-parent:”"; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; mso-layout-grid-align:none; text-autospace:none; font-size:12.0pt; font-family:Arial; mso-fareast-font-family:”MS Mincho”; mso-bidi-font-family:”Times New Roman”; mso-fareast-language:JA;} h2 {mso-style-next:Normal; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; mso-outline-level:2; mso-layout-grid-align:none; text-autospace:none; font-size:12.0pt; font-family:Arial; mso-fareast-font-family:”MS Mincho”; mso-bidi-font-family:”Times New Roman”; mso-fareast-language:JA; font-weight:normal;} h3 {mso-style-next:Normal; margin-top:12.0pt; margin-right:0in; margin-bottom:3.0pt; margin-left:0in; mso-pagination:widow-orphan; page-break-after:avoid; mso-outline-level:3; mso-layout-grid-align:none; text-autospace:none; font-size:13.0pt; font-family:Arial; mso-fareast-font-family:”MS Mincho”; mso-fareast-language:JA; font-weight:bold;} p.MsoCommentText, li.MsoCommentText, div.MsoCommentText {margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; mso-layout-grid-align:none; text-autospace:none; font-size:10.0pt; font-family:Arial; mso-fareast-font-family:”MS Mincho”; mso-bidi-font-family:”Times New Roman”; mso-fareast-language:JA;} p.MsoHeader, li.MsoHeader, div.MsoHeader {margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; tab-stops:center 3.0in right 6.0in; font-size:12.0pt; mso-bidi-font-size:10.0pt; font-family:”Times New Roman”; mso-fareast-font-family:”Times New Roman”;} p.MsoFooter, li.MsoFooter, div.MsoFooter {margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; tab-stops:center 3.0in right 6.0in; mso-layout-grid-align:none; text-autospace:none; font-size:12.0pt; font-family:Arial; mso-fareast-font-family:”MS Mincho”; mso-bidi-font-family:”Times New Roman”; mso-fareast-language:JA;} span.MsoCommentReference {mso-ansi-font-size:8.0pt; mso-bidi-font-size:8.0pt;} a:link, span.MsoHyperlink {color:blue; text-decoration:underline; text-underline:single;} a:visited, span.MsoHyperlinkFollowed {color:#606420; text-decoration:underline; text-underline:single;} p {margin-right:0in; mso-margin-top-alt:auto; mso-margin-bottom-alt:auto; margin-left:0in; mso-pagination:widow-orphan; font-size:12.0pt; font-family:”Times New Roman”; mso-fareast-font-family:”Times New Roman”;} p.CommentSubject, li.CommentSubject, div.CommentSubject {mso-style-name:”Comment Subject”; mso-style-parent:”Comment Text”; mso-style-next:”Comment Text”; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; mso-layout-grid-align:none; text-autospace:none; font-size:10.0pt; font-family:Arial; mso-fareast-font-family:”MS Mincho”; mso-bidi-font-family:”Times New Roman”; mso-fareast-language:JA; font-weight:bold;} p.BalloonText, li.BalloonText, div.BalloonText {mso-style-name:”Balloon Text”; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; mso-layout-grid-align:none; text-autospace:none; font-size:8.0pt; font-family:Tahoma; mso-fareast-font-family:”MS Mincho”; mso-bidi-font-family:”Times New Roman”; mso-fareast-language:JA;} span.stylearial10ptblack {mso-style-name:stylearial10ptblack;} span.body1 {mso-style-name:body1; mso-ansi-font-size:7.5pt; mso-bidi-font-size:7.5pt; mso-ascii-font-family:Verdana; mso-hansi-font-family:Verdana;} @page Section1 {size:8.5in 11.0in; margin:1.0in 1.25in 1.0in 1.25in; mso-header-margin:.5in; mso-footer-margin:.5in; mso-paper-source:0;} div.Section1 {page:Section1;} /* List Definitions */ @list l0 {mso-list-id:-2; mso-list-type:simple; mso-list-template-ids:1315765622;} @list l0:level1 {mso-level-start-at:0; mso-level-number-format:bullet; mso-level-text:*; mso-level-tab-stop:none; mso-level-number-position:left; margin-left:0in; text-indent:0in;} @list l1 {mso-list-id:508636615; mso-list-type:hybrid; mso-list-template-ids:775071060 1270362110 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;} @list l1:level1 {mso-level-number-format:bullet; mso-level-text:; mso-level-tab-stop:.25in; mso-level-number-position:left; margin-left:.25in; text-indent:-.25in; mso-ansi-font-size:9.0pt; font-family:Symbol; color:windowtext; mso-ansi-font-weight:normal; mso-ansi-font-style:normal;} @list l2 {mso-list-id:809203047; mso-list-type:hybrid; mso-list-template-ids:1745241986 1270362110 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;} @list l2:level1 {mso-level-number-format:bullet; mso-level-text:; mso-level-tab-stop:.25in; mso-level-number-position:left; margin-left:.25in; text-indent:-.25in; mso-ansi-font-size:9.0pt; font-family:Symbol; color:windowtext; mso-ansi-font-weight:normal; mso-ansi-font-style:normal;} @list l3 {mso-list-id:1376927774; mso-list-type:hybrid; mso-list-template-ids:323012810 -557689488 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;} @list l3:level1 {mso-level-number-format:bullet; mso-level-text:; mso-level-tab-stop:.5in; mso-level-number-position:left; text-indent:-.25in; mso-ansi-font-size:12.0pt; mso-bidi-font-size:11.0pt; font-family:Symbol; color:windowtext;} @list l0:level1 lfo1 {mso-level-numbering:continue; mso-level-text:; mso-level-tab-stop:none; mso-level-number-position:left; mso-level-legacy:yes; mso-level-legacy-indent:.25in; mso-level-legacy-space:0in; margin-left:0in; text-indent:0in; font-family:Symbol;} ol {margin-bottom:0in;} ul {margin-bottom:0in;} –>

That’s a whole other level of annoying.

Meanwhile, this is as good a time as any to offer lyrics and music/video for the classic Leslie Fish filk song “Banned from Argo.”

Our proper, cool first officer was drugged with something green
And hauled into an alley, where he suffered things obscene
He sobered up in sickbay and he’s none the worse for wear
Except he’s somehow taught the bridge computer how to swear

Actually, this version has better sound and image quality, but the video part doesn’t speak to me.

Also:

• A collection of Dr. McCoy clips.
• A song combining Star Trek and The Hobbit.
• A video (large download) to Julia Ecklar’s beautiful song God Lives on Terra, with Star Trek:TNG clips interspersed with views of Wrentham, MA.  (I previously linked a hilarious parody of that song.)
• Another Bob Kanefsky parody song, this one based on a specific Star Trek episode. (Melody and performance by Leslie Fish.)

Overall, the interview went a lot better than my last one with the same station.