Mark Logic CEO Blog

Frost & Sullivan Webinar Slides: The FDA Mandates XML

2009-11-05T08:49:00.001-08:00

Embedded below please find the slides from a webinar hosted by Frost & Sullivan today entitled The FDA Mandates XML: Maximize Your Investments Through Dynamic Publishing. The webinar was sponsored by Quark and EMC and had participation from PAR, Pfizer, and Frost & Sullivan.

From a Mark Logic perspective, the question this begs is: where do you then put all the XML that you will be creating to comply with these FDA regulations and initiatives?

At Mark Logic, we believe it should be stored not in a relational database and not in a content management system, but in an XML server (e.g., MarkLogic Server) which provides for high-performance storage, search, analysis and delivery of XML content.

At one level more sophistication: while we believe you may wish to use a content management system (e.g., EMC/Documentum, Alfresco) to control the creation process for this content when it comes to what Frost & Sullivan calls "dynamic publishing" of that content, that you would want to turn to an XML server, such as MarkLogic.

Here are the slides from the webinar.

Frost & Sullivan Webinar: FDA Mandates XML

View more documents from Dave Kellogg.

Mark Logic in One Adjective-Filled Sentence by Jason Hunter

2009-11-03T18:39:00.000-08:00

Embedded below please find the short slide deck that Jason Hunter presented at the recent Oakland NoSQL Meetup. I thought I'd share the deck here because Jason's mission was unique: we were sponsoring a meeting of people largely opposed to commercial software and definitely opposed to SQL databases, so he had to tread lightly, say what he had to say, and get out.

Here's what he said.

Jason Hunter Slides from NoSQL Oakland meetup

View more documents from Dave Kellogg.

A Few Discounted Seats for MarkLogic Developer Training 11/10-11/12/09 in NYC

2009-11-02T15:00:00.000-08:00

I'm trying an experiment here in helping to move the most perishable of all inventory: training seats.

We have a few remaining seats in our upcoming developer training 11/10 - 11/12/09 at the Roosevelt Hotel in New York.

If you register in response to this blog post, we'll provide a $500 discount. To register, send a note to training@marklogic.com and mention this post.

IDC Names Mark Logic Innovative Information Access Company to Watch

2009-10-26T15:28:00.000-07:00

I'm very happy to report that IDC has named Mark Logic "an innovation information access company under $100M to watch." IDC's press release is here and a copy of the ($3500 a la carte paid) report is here.

Excerpt:

New technologies are eliminating boundaries between content and data to enable pervasive access to all relevant information. Contributing to this innovation is a group of small companies with the vision and technology to have an impact on the IT marketplace. IDC invited search, business intelligence, and content management companies with less than $100 million in revenue in 2008 to enter our Innovation Awards contest.

The complete list of innovative information access companies follows, in alphabetical order:

Carefx, a pretty interesting company that provides healthcare information systems.

Connotate, a company that aims to provide KM and BI on content using intelligent software agent technology.

Exalead, a French enterprise search company that seems to be benefiting from the vacuum created by the acquisitions in and around enterprise search -- e.g., Microsoft's acquisition of Fast, Autonomy's acquisition of Verity, and the general loss of focus on enterprise search at Autonomy resulting from their transition to a more consolidation / financial engineering strategy (e.g., the Zantaz and Interwoven acquisitions).

FatWire, a mid-tier WCM company who currently positions as web experience management and who seems to be enjoying a resurgence under my old TPC colleague, Yogesh Gupta. These guys also seem to be benefiting from consolidation of the tier 1 players above them (e.g., EMC buying Documentum, Oracle buying Stellent) as well as from good, old-fashioned improved execution.

Jaspersoft, a provider of open source BI, led by Brian Gentile, former marketing chief at Informatica and Brio. In its current incarnation (I think it's been through a few different strategies), it competes for market leadership with Pentaho, whose marketing is run by the very able Lance Walter, a member of my marketing team at Business Objects.

Mark Logic, the leading provider of XML servers, a type of enterprise infrastructure software for information applications.

NetBase, a content intelligence vendor whose message sounds a lot like Mark Logic's but which uses very different, and much more semantic, technology. These were the folks who had the tragicomical healthBase demo incident a few months back. Accidents aside, the technology looks interesting.

Recommind, who makes search-powered information risk management software who, as I understand things, has a strong focus on legal with an emphasis on e-discovery, classification, and compliance. They presumably compete with my friend Aaref Hilaly's company, Clearwell Systems, who have successfully carved out a leadership position in several boxes of the e-discovery reference model.

Vivismo, an enterprise search vendor whose initial market assault was based on dynamic clustering technology, but who now positions much more as a regular enterprise search vendor (e.g., search done right).

I'd like to thank IDC for the recognition in being selected to this list. We are honored to have been chosen and fully agree that we are a company to watch in changing the information technology landscape moving forward.

Mark Logic Profiled in San Jose / Silicon Valley Business Journal

2009-10-23T15:15:00.001-07:00

Just a quick post to highlight a story in the San Jose / Silicon Valley Business Journal entitled Mark Logic in San Carlos Saves Customers from Drowning in Data.

Excerpt:

The San Carlos technology firm offers software that helps businesses store, organize and filter all the data they’ve accumulated, including e-mails and Web pages and other so-called unstructured information that isn’t easily manageable. It’s an increasing need for enterprises, as the nature of information changes.

“It’s for the brave new world of unstructured information,” said David Kellogg, CEO of Mark Logic. “It’s a next-generation system for storing data.”

Check out how the photographer flashed off the glass logo in our lobby to create a cool reflection off the wall.

The full story is available here, but is subscription only.

Judge Does Not Decide on Dimissing ZL Technologies Complaint Against Gartner

2009-10-23T13:55:00.000-07:00

(Revised: confirmed that the source, MS&L, is ZL's PR firm.)

While I don't have official verification of this, I did learn the following this afternoon, regarding the lawsuit filed against Gartner by ZL Technologies over its treatment in their magic quadrants about which I posted earlier this week and which is also covered here, here, and here.

"In today’s hearing on Gartner’s motion to dismiss ZL’s complaint, the court did not come to a decision."

Precisely because I couldn't find any reference to this online (yet), I figured it was breaking news and should share it via the blog. I learned this information via an email from David Schraeder of MS&L, the PR firm representing ZL Technologies.

Given my limited understanding of the law, a non-decision is a decision. That is, by not deciding to throw out the case, I take to mean that the case is on and will proceed. More information will undoubtedly come out later, but I wanted to share this while it was hot.

It's not every day I can break a story on the Mark Logic CEO blog!

XML: YAFF, YADT, or Whole World?

2009-10-22T06:06:00.000-07:00

If you have a bunch of XML and are looking for of a place to put it, then I think I may have come up with a simple test that might be helpful.

In talking with prospective vendors of XML repositories (definition: software that lets you store, search, analyze and deliver XML), try to establish what I'll call "XML vision compatibility." Quite simply, try to figure out if the vendor's vision of XML is consistent with your own. To help with that exercise, I'll define what I see as the three common XML vendor visions:

YAFF (yet another file format)
YADT (yet another data type)
Whole world

YAFF Vendors
Vendors with the YAFF vision view XML as yet another file format. ECM vendors clearly fall into this category ("oh yes, XML is one of the 137 file formats you can manage in our system"). So do enterprise search vendors ("oh yes, we have filters for XML formatted files which clear out all those nasty tags and feed our indexing engine the lovely text.")

For example, let's look at how EMC Documentum -- one of the more XML-aggressive ECM vendors -- handles XML on its website.

Hmm. There's no XML on that page. But lots of information about records management, digital asset management, document capture, collaboration and document managent (it's not there either). Gosh, I wonder where it is? SAP integration? Don't think so. Hey, let's try Documentum Platform, whatever that is.

Not there, either. Now that's surprising because I really have no idea where else it might be. Oh, wait a minute. I didn't scroll the page down. Let's try that.

There we go. We finally found it. I knew they were committed to XML. What's going on here is that EMC has a huge, largely vendor consolidation-driven (e.g., Documentum, Captiva, Document Sciences, x-Hive, Kazeon) vision of what content management is. And XML is just one tiny piece of that vision. XML is, well, yet another file format among the scores that they have manage, archive, capture, and provide workflow, compliance, and process management against. The vision isn't about XML. It's about content. That's nice if you have an ECM problem (and a lot of money to solve it); t's not so nice if you have an XML problem, or more precisely a problem that can be solved with XML.

YADT Vendors
Vendors with the YADT vision view XML as yet another data type. These are the relational database management system vendors (e.g., Oracle) who have decided that the best way to handle XML is to make it a valid datatype for a column in a table.

The roots of this approach go back to the late 1980s and Ingres 6.3 (see this semi-related blast from the past) which was the first commercial DBMS to provide support for user-defined datatypes. All the primitives for datatyping were isolated from the core server code and made extensible through standard APIs. So, for example, if you wanted to store complex numbers of the form (a, bi) all you had to do was to write some primitives so the server would know:

What they look like -- i.e., (a, bi)
Any range constraints (the biggest, the smallest)
What operators should be available (e.g., +, -)
How to implement those operators -- (a, bi) + (c, di) = (a+c, (b+d)i)

It was -- far as I remember -- yet another clever idea from the biggest visionary in database management systems after Codd himself: Michael Stonebraker then of UC Berkeley and now of MIT. After founding Ingres, Stonebraker went on found Illustra which was all about "datablades" -- a sexy new name for user-defined types. Datablades, in turn, became sexy bait for Informix to buy the company with an eye towards leveraging the technology towards unseating Oracle from its leadership position. It didn't happen.

User-defined datatypes basically didn't work. There were two key problems:

You had user-written code running in the same address space as the database server. This made it nearly impossible to determine fault when the server crashed. Was it a database server bug, or did the customer cause problem in implementing a UDT? While RDBMS customers were well qualified to write applications and SQL, writing server-level was quite another affair. This was a bad idea.

Indexing and query processing performance. It's fairly simple to say that, for example, a text field looks like a string of words and the + operator means concatenate. It's basically impossible for a end customer to tell the query optimizer how to process queries involving those text fields and how to build indexes that maximize query performance. If getting stuff into UDTs was a level-5 challenge, getting stuff back out quickly was a level-100 one.

So while the notion of end users adding types to a DBMS basically failed, when XML came along the database vendors dusted off this approach, in saying effectively: let use all those hooks we put in to build support for XML types ourselves. And they did. Hence what I call the "XML column" approach to storing XML in a relational database.

After all, if your only data modeling element's a table, then every problem looks like a column.

Now this approach isn't necessarily bad. If, for example, you have a bunch of resumes and want to store attribute data in columns (e.g., name, address, phone, birthdate) and keep an XML copy of the resume alongside, then this might be a reasonable way to do things. That is, if you have a lot of data and a touch of XML, this may be the right way to do things.

So again, it comes down to vision alignment. If XML is just another type of data that you want to store in a column, then this might work for you. Bear in mind you'll:

Probably have to setup separate text and pre-defined XML path indexes (a hassle on regular schemas, an impossibility on irregular ones),
Face some limitations in how those indexes can be combined and optimized in processing queries,
Need to construct frankenqueries that mix SQL and XQuery, whose mixed-language semantics are sometimes so obscure that I've seen experts argue for hours about what the "correct" answer for a given queries is,
And suffer from potentially crippling performance problems as you scale to large amounts of XML.

But if those aren't problems, then this approach might work for you.

This is what it looks like when a vendor has a YADT vision. Half the fun in storing XML in an RDBMS is figure out which query language and which store options you want to use. See the table that starts on page 9, spans four pages, and considers nearly a dozen criteria to help you decide which of the three primary storage options you should use:

xmldb_11g_twp

See this post from IBM for more Oracle-poking on the complexity of storage options available. Excerpt:

Oracle has long claimed that the fact that Oracle Database has multiple different ways to store XML data is an advantage. At last count, I think they have something like seven different options:

Unstructured
XML-Object-Relational, where you store repeating elements in CLOBs
XML-Object-Relational, where you store repeating elements in VARRAY as LOBs
XML-Object-Relational, where you store repeating elements in VARRAY as nested tables
XML-Object-Relational, where you store repeating elements in VARRAY as XMLType pointers to BLOBs
XML-Object-Relational, where you store repeating elements in VARRAY as XMLType pointers to nested tables
XML-Binary

Their argument is that XML has diverse use cases and you need different storage methods to handle those diverse use cases. I don’t know about you, but I find this list to be a little bewildering. How do you decide among the options? And what happens if you change your mind and want to change storage method?

Such is life in the land of putting XML in tables because your database management system has columns.

Whole World Vendors
Vendors with the whole world vision view XML as, well, their whole world.

And when I say XML, I don't mean information that's already in XML. I mean information that is either already in XML (e.g., documents, information in any horizontal or industry-specific XML standard) or that is best modeled in XML (e.g., sparse data, irregular information, semi-structured information, information in no, multiple, and/or time-varying schemas).

"Whole world" vendors don't view XML as one format, but as a plethora: docbook, DITA, s1000d, xHMTL, TEI, XBRL, the HL7 standards in healthcare, the Acord standards in insurance, Microsoft's Open Office XML format, Open Document Format, Adobe's IDML, chemical markup lanuage, MathML, the DoD's DDMS metadata standard, semantic web standards like RDF and OWL, and scores of others.

Whole world vendors don't view XML tags as "something that get in the way of the text" and thus they don't provide filters for XML files. Nor do they require schema adherence because they know that XML schema compliance, in real life, tends to be more of an aspiration than a reality. So they allow you load and index XML, as is, avoiding the first step's a doozy problem, and enabling lazy clean-up of XML information.

Whole world vendors don't try to model XML in tables simple because they have a legacy tabular data model. Instead, their native modeling element (NME) is the XML document. That is:

In a hierarchical DBMS the NME is the hierarchy
In a network DBMS the NME is the graph
In a relational DBMS the NME is the table
In an object DBMS the NME is the object class hierarchy
In an OLAP, or multi-dimensional, DBMS the NME is the hypercube
And in an XML server, or native XML, DBMS the NME is the XML document

Whole world vendors don't bolt a search engine to a DBMS because they know XML is often document-centric, making search an integral function, and requiring a fundamentally hybrid search/database -- as opposed to a bolted-together search/database -- approach.

Here is what it looks like when you encounter a whole world vendor:

Gartner Sued Over Magic Quadrant for Alleged Damages of $132M plus Punitives of $1.3B

2009-10-20T17:37:00.000-07:00

I found this story today, entitled Gartner’s Magic Quadrant Goes to Court, about ZL Technologies who, citing damages of $132M, has decided to sue Gartner over its Magic Quadrants. From ZL’s web page on the suit:

ZL Technologies, a San Jose-based IT company specializing in ... enterprise software solutions for e-mail and file archiving, is challenging Gartner Group and the legitimacy of Gartner’s “Magic Quadrant.” In a complaint ... ZL claims that Gartner’s use of their proprietary “Magic Quadrant” is misleading and favors large vendors with large sales and marketing budgets over smaller innovators such as ZL that have developed higher performing products.

The complaint alleges: defamation; trade libel; false advertising; unfair competition; and negligent interference with prospective economic advantage.

“Sour grapes” spring to mind as an immediate reaction. In fact, ZL concedes that they’ve been ranked in the “niche” segment of every email archiving quadrant since 2005. (Ouch!) But they nevertheless argue that bigger stakes are in play and that this is not only about ZL, but Gartner itself, technological innovation, and very nearly preservation of the American way of life. Excerpt, edited for brevity:

Regardless of how the court may decide the First Amendment arguments, ZL hopes to achieve the following …
Fair Disclosure on Conflicts of Interest. Gartner generates its revenues from payments made by the same vendors whose products it evaluates. …
Fair Disclosure on Evaluation Scores. The tech industry would benefit if Gartner were required to disclose more data in its evaluation process and disclose component scores so vendors know exactly where they are lacking and by how much and take corrective action. …
Better Oversight. Gartner currently has an employee act as ombudsman to handle disagreements. The conflict of interest is self-evident in the way ZL’s concerns were summarily dismissed with little supporting evidence. ZL believes that Gartner’s immense heft and power in the marketplace necessitate careful checks and balances against abuse of power. ZL believes that if IT innovation is to remain a driver for the US economy, there must be assurances that ratings agencies such as Gartner do not subvert the competitive forces which drive innovation.

I remember a long time ago CA boycotted all Gartner research after some research-related dispute. It certain did nothing to help them: picking a fight with the movie critics always seems a risky strategy for a producer.

But it is hard to argue that Magic Quadrants are good for competition. They are inherently subjective in their assessments, they two-dimensionalize an N-dimensional problem, they encourage mental laziness on the part of customers, and –- heck –- some of us work in sectors that don’t even have a magic quadrant. What’s worse, ZL? Getting a poor ranking on an existing quadrant, or selling in a software category that Gartner doesn’t even recognize?

Since I think it’s fun to read court filings (when I have the time), let’s dig down a little deeper. The court documents are here, and I’ll embed them along the way as well. Here’s the initial complaint.

One of the many arguments made in the complaint is that Gartner doesn’t do “a single minute of independent testing of the products it purports to evaluate.” When I was younger in my career, I used to buy that argument. As I gotten older, I now realize (think: Wisdom of Crowds) that it is indeed possible to get a pretty good picture of a product’s strengths and weaknesses simply by talking to lots of people who use it.

And that’s what Gartner does. Yes, there are no guys in lab coats doing Consumer Reports style testing. But, sometimes the guys in the lab coats measure the wrong things anyway. So while Gartner does not, to my knowledge, do hands-on testing, they neither claim to do so nor, in my estimation, is such testing strictly necessary to develop an informed opinion.

That said, a pathological case of that research model is when a vendor has very small market share. If research is done primarily through talking and there's no one to talk to, then you're not going to get on the map very easily.

On the other hand, I love their brass tacks description of the reality behind being labeled a “niche player”:

These MQ placements were, and are, derogatory because they are understood by technology purchasers as a warning, by Gartner, that ZL and ZL products are not good choices for enterprise email archive applications.

Yep.

Also of interest was this statement by Gartner’s ombudsman:

My sense is that there has been a relationship issue for many years with [archiving analyst] Carolyn DiCenzio and at this point it’s come down to level of trust and respect."

I suppose there's some logical consistency at least -- if you're going to declare war on the #1 analyst firm, well, why not make it personal as well. :-)

Let’s move on. Here's Gartner's response, a motion to dismiss, which is much tougher reading and more techno-legal:

Gartner Motion to Dismiss

Clearly, Gartner's response is based on opinion and freedom of speech. Excerpt:

Whether plaintiff’s opinions about its product are correct, comprehensible or sincere has no legal significance; what matters is that the Complaint fails to state a claim because it attacks opinions expressed by Gartner, Inc. These opinions are constitutionally protected, in part to discourage lawsuits like this one, which are aimed at chilling the free expression of ideas and opinions.

While Gartner marketing may not love that response (imagine: "could we please defend the research as well as our right to have opinions?") it's not a terribly surprising one.

If nothing else, you can being to see why lawsuits cost so much money. Bear in mind the legal meters are probably running at $600/hour and they're still debating whether the case should be immediately thrown out: it's like dropping $20K standing on the starting line fighting about the start time for the race.

Here is ZL's opposition to Gartner's motion for dismissal, another 32-pages:

ZL Opposition to Gartner Motion to Dismiss

If you didn't jump into the document above, let me pull out the first zinging paragraph (bolding mine):

This is a commercial case about a dominant industry player’s baseless defamation of an independent startup whose growth prospects have been crushed by the defendant’s unfair business practices. Defendant Gartner, Inc. (“Gartner”), which advises businesses on information technology decisions, exercises hegemonic control over the purchases made by a wide swath of the international corporate and governmental market. The technology Gartner says to buy is bought; what Gartner says not to buy languishes unsold, leaving its developers scrambling for the leftover market share Gartner does not dictate. The problem arises when Gartner exercises its market power recklessly, maliciously or—because of its tremendous influence—negligently. When that occurs, as it has here, innovation and competition are stifled, to the detriment of small companies who lack the resources to challenge Gartner, and to the consuming public at large.

Wow, someone turned up the rhetoric meter! At this point things are quickly getting over my legal head. There arguments seem to be largely about what is fact vs. opinion. Since I'm unable to comment on the legal issue, I'll move on.

Finally, for the strong of legal stomach, here is Gartner's reply to the opposition to the motion to dismiss. (Say that ten times fast.)

Gartner Reply to ZL Opposition to Motion to Dismiss

Here's a nice summary of the counter-argument:

Try as it might, ZL cannot create a dispute where there is none. ZL alleges at great length in its Complaint (and recapitulates in its Opposition) that it has a strong product and satisfied customers. The Magic Quadrant reports do not say otherwise; the real point of contention here is not the quality of ZL’s product, but instead the subjective analytical model Gartner used to assess ZL’s market position and prospects. ZL does not contest Gartner’s basic assessments of ZL—that it has a good product but needs to expand its sales and marketing—but ZL challenges its placement on the Magic Quadrant Report because Gartner uses a “misguided analytical model” that gives “undue weight to sales and marketing.”

I have no idea how this will end. Will it be quickly thrown out of court? Will it a long drawn-out case? I don't know. I would say that Gartner's quadrants wield enormous power and that vendors go to great lengths to maxmize their position on them. And I'd say that you can't judge a vendor by the quality of its technology alone. While Ingres arguably had the best database technology in the 1980s, Oracle's sales and marketing prowess caused it to win the market and any analyst who -- focused solely on the technology -- would have recommended Ingres at that time would have done his customers a disservice.

I don't know how the movie here ends, but I at least expect it to be interesting.

Dear CIO: Stop Writing Big Checks for Commodity (Database) Software

2009-10-14T10:11:00.000-07:00

Dear CIO,

What’s wrong this picture?

At 50%+, Oracle’s operating margins have never been higher

The differentiation of Oracle’s database technology, however, has never been lower and the number of both core and specialized alternatives has never been greater.

So what’s going on? You, kind Sir or Madam, are being milked. What’s worse is that you, in an example of collective behavioral dysfunction, have inadvertently played a role in setting up the milking. What happened?

Like all smart CIOs you followed a bit of herd mentality when it came to core technology. Pity the poor fools who, back in the day, bet big on Ingres or Sybase. You played it safe and went with Oracle, IBM, or if your requirements weren’t too heavy, Microsoft.

The problem is, of course, that everyone executed the same strategy you did. Hence, the market created a system of increasing returns where the strong vendors got stronger and the weak ones died. The result: the RDBMS market is an (order of magnitude) $10B/year market, structured as an oligopoly with 3 players. Most other software markets worked out the same way.

You were focused on standardization. You realized that through a combination of decentralized IT decision making and growth-by-acquisition your organization had become a kitchen sink of enterprise software. You had everything. In order to reduce the administrative, training, and license acquisition costs, you fought tooth and nail with your divisions to standardize the environment. You said, “Heck, it’s all the same stuff in the end, folks, so let’s make Oracle our DBMS standard, Business Objects our BI standard, Documentum our ECM standard, and SAP our ERP standard.”

And you won. Mostly. There’s still some Cognos in finance. And marketing didn’t totally give up on Interwoven. But, for the most part, you won. You reduced the entropy of your IT environment and drove cost savings for your organization.

The problem is you’ve won the battle but lost the war. Why? Because if, as you say, the “stuff really is all the same” you shouldn’t standardize on the most expensive product. You should standardize on the cheapest.

Do you really need to be paying those big fees to Oracle for enterprise licenses? Wouldn’t MySQL do?

Are you really using all the functionality of that $1M/year Documentum ECM system? Wouldn’t SharePoint or Alfresco do?

For BI, do you need all the bells and whistles of BusinessObjects? Wouldn’t Pentaho or Qlikview do a fine job, at a fraction of the cost?

But these alternatives are obvious. Heck, even "the establishment" (i.e, Gartner) says it’s safe to tread in the open source water. So the question is, what’s holding you back?

Switching costs. It’s hard to move off Oracle or Documentum and you don’t want to pay the nut to do so.

Organizational inertia. Your whippersnapper DBAs who were in their 30s in the 1980s are now in their 50s. They’re thinking that change devalues their knowledge and experience; some just want to cruise into retirement. But that’s their personal agenda, not your enterprise one.

Accounting: you made it free for your divisions to keep using Documentum, Oracle, or BusinessObjects because you bought an enterprise license. While this appeared to “save” you money on a per-license basis, and it helped support your standardization initiative, it squashed innovation in your divisions, reinforced the organization inertia, and has a lot of people using the wrong tool for the job, resulting in projects that either take more or more expensive hardware than necessary (Oracle is good at this), that take too long to develop, or that simply fail.

So, what do I recommend doing about all this? I suggest that you adopt these policies, which –- for full disclosure, are at least partially in the self-interest of this blog’s author:

Stop writing big checks for commodity software. Every time a big check comes along, ask yourself: is this software differentiated or commoditized? Be willing to pay a premium for differentiated software, and price shop commodity software. Call a group of your smartest staff together periodically to help you make the commodity versus differentiated call.

When you see a big check coming for commodity software, make a migration plan. My hunch is that most of the time, you can create a nice 3-year ROI in the transition from premium to cheaper software. (This reminds me of the time I visited an investment bank’s CIO asking about their Documentum strategy. The answer: “our Documentum strategy is to get off Documentum,” because we're paying too much and using too little.)

Stop doing enterprise agreements that create poor economic incentives within your organization. Don’t pay $XM at the enterprise level, spread that as a “tax” across your divisions, and then make use of certain software “free.” It distorts project reality, creates false incentives, squashes innovation, and generates lots of hidden costs. If you want to negotiate a master agreement and discount rate, that’s fine. Shoot for centralized discounts without central planning.

Don’t worry that the prior policies will create mayhem. While I understand that you don’t want arbitrary taste differences increasing the entropy of your enterprise software portfolio, recognize that with the first policy you’ve solved that problem already. If you deem a category (e.g., core RDBMS, enterprise search) commoditized, then you are going to force people to pick on cost. You’ll get standardization on the commodity categories –- just on the least expensive alternatives. The only entropy you’ll need to manage will be on the differentiated software which, having dispatched the commodity majority, you’ll have time to explore, study, and exploit.

Why I am taking the time to write this note to you? Back in the 1980s I was a foot soldier in the relational database revolution, and today I’m the CEO of one specialized DBMS company and on the board of another.

Mark Logic makes an XML server which can save great amounts of time and money in creating applications against unstructured information, replacing the combination of an RDBMS, an enterprise search engine, and an application server. Not only can Mark Logic manage 100s of TB of XML, the system eliminates the object / relational/ hierarchical impedance mismatch between Java, SQL, and XML that hampers developer productivity. Mark Logic was recently named the fourth fastest-growing IT company in Silicon Valley.

Aster Data makes a specialized data warehouse DBMS that runs on low-cost commodity hardware with a shared nothing architecture and leverages in-database MapReduce technology for parallelism and high scalability.

And during the past 25 years or so I've watched the market evolve. While I fully understand the policies and market forces that have led us to where we are, I feel like we've come full circle. Vendor power is now concentrated in the big three. Vendor margins top 50%. Big vendors don't innovate; they consolidate. Inertia has set in customer organizations. And there's a major platform shift in progress; last time it was mainframe to minicomputer, this time it's cloud.

Things feel a lot to me the way they did in 1985, just past dawn of the relational revolution. So in one way I'm writing to point out the oft-overlooked obvious: stop paying premium prices for commodity items. And in another way I'm saying, take the money you save in so doing and invest it in innovation technologies that:

Drive competitive advantage (which will matter again as we come out of the Great Recession)

Enable the Internet-scale applications you'll need to face the coming information deluge

Reform the application development stack in ways that make sense for the coming generation of information applications, not that made sense for the last generation of data-centric ones.

Thank you for reading my note. If you have any questions or comments, please give me a ping at dave-dot-kellogg-at-marklogic-com or comment on this post.

Sincerely,

Dave Kellogg

docGenix: MarkLogic Unleashed on Contracts (with Webinar on 10/15/09)

2009-10-13T15:25:00.001-07:00

As a technology platform provider, you inevitably have certain visions of what applications your customers will one day build with your product. For example, Mark Logic Founder Christopher Lindblad originally envisioned that customers would spider and enrich Internet content, enabling an XQuery Internet search box as opposed to the typical keyword-oriented one. Christopher’s vision became reality a few years back when we started working in open source intelligence.

Being the “business guy” half of the Mark Logic equation, my vision was a little less, well, visionary. I’d always wanted someone to do something with contracts. They’re high value content, particularly when you’re either being sued or evaluating risk. They’re semi-structured; they have lots of sections and subsections and entities. But they’re typically messy, not in any type of explicit schema, per se. And, if you’re really good, they have lots of revisions that you can watch change over time.

Net/net: contracts are important, there’s a lot of structure and complexity associated with them, and -–to be blunt -– most organizations have effectively no idea what’s in them. So I always viewed them as a ripe target for MarkLogic Server.

I’m happy to report that docGenix has made that vision a reality. Founded by lawyer, self-taught programmer, and “legal rebel” Michael Will, docGenix specializes in the management of derivatives contracts: the pair-wise master agreements between parties that govern the trading of the their derivatives contracts. Among other things, these contracts help organizations evaluate risk and perform scenario analyses –- e.g., what would happen to our portfolio if counter-party X had their credit rating dropped to BBB+? And a typical financial institution will have tens of thousands of these agreements.

Here’s an excerpt on the origins of docGenix from the ABA Journals story, linked above:

Michael Will probably wasn’t the first finance attorney to look at a derivatives security and think, “There’s got to be an easier way of doing this.” But he certainly is one of the first attorneys to actually try to find that easier way.

In a chance hallway encounter in Allen & Overy’s London office, an information technology manager challenged Will to use a piece of development software. Will went home that night and then “didn’t sleep for six months. I did my regular job during the day and taught myself to program at night.”

Nearly a decade later, Will is the founder and chief technology officer for two Allen & Overy affiliates: Derivative Services, which opened in 2002, and docGenix, which opened in 2008. Derivative Services summarizes and delivers attorney analyses of legal opinions used in derivatives transactions. DocGenix takes that process a step further, relying on computers to analyze derivatives agreements, improve risk management and streamline the process for reviewing, updating and modifying derivatives agreements.

In addition to having a built a nice UI for searching these contracts, and a system for importing them, docGenix has paid particular attention to the idiosyncratic way in various facts, or “data points,” are expressed. For example, there are numerous ways to express a “ratings trigger” -– rating to exceed BBB+, rating not to drop below A-, rating to be A- or above, and on and on. docGenix reports that they typically identify between 1,500 to 8,500 data points per contract. Once identified, these data points are identified in XML in a canonical form and become queryable.

If this story’s piqued your interest, you should know that we are hosting a webinar with docGenix as the featured guest on Thursday, October 15th, 2009 at 8:00 AM PDT / 11:00 AM EDT, featuring John Berry, general counsel at docGenix and Steve Mount from Mark Logic. Here’s an except from the registration page:

Leveraging MarkLogic Server’s high-performance, scalable XML capabilities, docGenix Synopsys is a powerful, user-friendly application that provides financial institutions with deeper visibility and a more holistic view across their ISDA Agreement portfolios. Synopsys harnesses the power of XML and MarkLogic Server, to give financial institutions the agility to effectively analyze and respond to the threats and opportunities associated with today’s OTC derivatives markets.

For more information and/or to register for the webinar, please go here.

Meantime, if you’d like to hear more from Michael Will, I’ve embedded a video of him talking about docGenix, below.

XML, “The Necessary Ingredient” Research from Outsell

2009-10-13T10:15:00.000-07:00

Just a quick post to highlight some (fairly) recent research from Outsell entitled XML: The Necessary Ingredient for Information Publishing (paid, $695) based on a survey run early this year. While the survey had only 30 respondents, so you must generalize with care, I still think the results are interesting and illustrative.

Excerpt from the introduction on how XML can help information providers go web 2.0:

In our view, XML should be a key ingredient in any publisher’s or information provider’s toolkit. It is virtually impossible to envision a content purveyor that couldn’t gain significant advantage form XML usage, expert perhaps for a one product, paper-only publisher -- and such a vendor would be an anachronism.

Another excerpt, this one more of a primer on the basic problems solved by XML technology:

Some examples of [the problems solved with] XML in action

Content integration, bringing together and normalizing structured and unstructured content …

Content repurposing, using content in multiple products …

Multiple delivery formats, satisfying clients’ demands for custom content delivery …

Fine-grained searching, … more granularity than a simple keywords

Content syndication, partnering with other players requires the ability to integrate, normalize, and repurpose content from multiple sources

My single favorite piece of the report was this table [edited for brevity]:

Business Problem XML Solution

Product content exists in multiple formats and cannot be repurposed XML can be used to create a canonical set of tags that rationalize disparate formats. Automated tools can take Word, PDF, text, and other file formats, and tag their structure.

Publication processes are “hardwired” for each product due to product-specific format XML enables creation of an “author once, publish many” process where content is created once then tagged for use in multiple products

Clients demand custom delivery formats and each variation requires its own software XML schemas can be used to tailor content delivery to client-requested formats

Content and format are intertwined, making it difficult to change either XML enables separation of content and format using CSS or XHTML.

Search engines return long lists of documents with actual search results buried inside the documents. Search is limited to key words and phrases. XQuery is XML’s standard query language, enabling subdocument search and construction of very specific searches and queries.

The report then continues on to provide some results of the survey (50% of those surveyed have 50%+ of their content in XML), and then provides case studies of four XML user organizations.

There are brief write-ups on several vendors: Mark Logic, Really Strategies, Nstein, and ePublishing.

The report concludes with some business-oriented action items for those interested in the power of XML technology. The report was written by Outsell’s Marc Strohlein who has the unique background of having worked as both an media industry technology analyst as well as a media industry CIO.

Save The Date: Mark Logic 2009 Federal Summit –- 11/18/09

2009-10-12T17:33:00.001-07:00

Just a quick post to remind people to save the date for the Mark Logic Federal Summit to be held on November 18th, 2009 at the historic Willard Hotel in Washington, DC.

The Willard has a truly amazing history, having hosted every President –- as a sleeping guest or a guest at a social function -– since Zachary Taylor in 1850. For example, the Reverend Dr. Martin Luther King, Jr. wrote his “I Have a Dream” speech at the Willard on August 28th, 1963. To see a PDF of the Willard’s history brochure, go here.

The agenda for the event is here. Expect a fun, information-filled day discussing topics such as Mark Logic technology, unstructured information management, enterprise search, and next-generation database management. Learn about customer applications in areas like metadata catalogs, information sharing environments, open source intelligence, community data layers, and large-scale archiving.

Please join us at this event. For more information, please go here.

Update on the Renaming Effort

2009-10-12T10:25:00.000-07:00

Just a quick post to provide an update on the renaming effort.

Of the two finalists – Kellblog and Kellogic – I preferred Kellogic (and would like to thank Darren Cunningham for suggesting it). But the URL was already taken, so I was considering moving to Kellbog, instead. But, as someone with a marketing background, ambiguity about how to spell it proved to be a deal-killer (e.g., Kel-Blog, KelBlog, KellBlog, Kellblogg). There were just too many possibilities.

As was once said of Mark Logic's original name, Cerisent: "it's French for I don't know how to spell it."

Most importantly I became worried about the raw amount of time and energy required to move the blog over to a different platform, rename it, break my inbound links, break my intra-post links, leave the old blog in place to preserve them, and then worry that Google could incorrectly identify the new blog as blog-scraped spam and destroy its PageRank.

So, those fears aside, renaming a blog seems easy to do. Hence, until such time as I can properly research the hazards before beginning the journey, we will remain the Mark Logic CEO Blog.

Back in the Mark Logic CEO Blog Day

2009-10-12T07:39:00.000-07:00

Ever since I changed my sharing pattern to use bit.ly more, I’ve been using blogger less. This was put to me rather poignantly the other day when a prospective customer said: “why did you stop blogging because, well, back in the day, you used to have the best blog in the industry.”

Best blog? I blush.
Used to? Whoa.

It was a nice compliment, but it was also a wake-up call. Had my new found love of bit.ly destroyed me as a blogger? In fact, I hadn’t “stopped” blogging. I did 9 posts/month during 3Q09, but that was only half the 18 posts/month I did in 3Q08.

This got me thinking about why, which in turn lead me to start thinking about the major types of posts that I (and others) write:

1. Essays, which are often argumentative and sometimes rants. A blog composed of them is like an ongoing op-ed column. Each post typically cites multiple posts/articles on the topic and synthesizes an argument. These are hard to write.

2. Announcements, which announce upcoming speeches, marketing events, user conferences, et cetera. These are pretty straightforward, largely factual, and not hard to write.

3. Highlights. These typically highlight a single cool post/article, often with one excerpt and a pithy quote. Reality Check by Peter O’Kelly is largely composed of this type of post. These blogs deliver value in two ways: commentary and pre-filtering (i.e., highlighting the best-of’s in a mountain of content read by the author). I believe that the value usually comes 80% from pre-filtering and 20% from commentary.

4. Highlights posing as essays. Some posts are basically type 3, but pretend to be type 1 by dressing things up a bit, adding more wrapper commentary and taking multiple excerpts. But when you boil them down, they not argumentative essays. They’re just long, cool article highlights.

Type 3 posts are the most interesting -- and most relevant to my blog post frequency drop -- because there are effectively 3 ways to create them:

Via a blog such as Reality Check.

Via a tweetstream (an RSS feed of tweets) that include bit.ly-like shortened URLs with necessarily pithy commentary due to the short character limit.

Via a shared items feed, such as the very good feed from Jill O’Neil at NFAIS. While these lack the ability to comment, they do include as much of the underlying posts as the feeds allow. Bizarrely, while I should prefer this method given my frequent off-line consumption, I find it rather old fashioned and shy away in favor of more hip services like Twitter and bit.ly. (Who says there’s no fashion in tech?)

What’s led me to cut my post frequency in half is that I’ve infinitely increased my tweet frequency: from 0 tweets/day to ~10 tweets/day – i.e., ~300/month. In effect, I do way more type 3 “posts” than before; I just don’t do them on blogger and thus dress what could have been tweets as posts. So, in a “what have you done for the community lately” sense, I’d say that I’m sharing radically more, but I’m indeed analyzing less.

Since I know that many folks enjoy the blog precisely because of the essays, I will make an effort to increase the number of analytical, argumentative, and controversial posts going forward. But if you want to see my full information stream you’ll need to follow both my blog RSS feed and my tweetstream (via Twitter directly or via an RSS feed of my tweets).

Since I've got time to burn here over Greenland on my way back from Europe, and since I’m analyzing my web 2.0 participation, I’ll share some other fun observations.

On FriendFeed and Social Networking Worlds
I’m changing my usage of FriendFeed as part of strategy to build a bit of a wall between my personal friends (Facebook) and my work friends and colleagues (LinkedIn). Going forward, I believe that most people will do the same thing, ending up with N social networks and “hard walls” between them. I believe that few people will trust a friendship-type field to protect what’s truly personal from what’s work. They’d rather just play it safe and use totally different social networks. As a concrete example, a friend status’ed something akin to this the other day on Facebook:

Jane is reminding herself that when projects are spinning out of control and that things at work are a mess that she needs to focus herself on what she realistically can and cannot control.

While many people, including me, “liked” the status, I can assure you that Jane does not want her boss reading this and would never make this her LinkedIn “what are you working on” status.

So FriendFeed, in my estimation, has two problems. First, it doesn’t work well with Facebook -- it pollutes your feed with excessive updates which Facebook then over-corrects by downgrading FriendFeed updates to rarely seen second-class citizens. In reality, my Facebook friends don’t want to see my business-oriented Twitter tweets, and I don’t think my blog readers want to read my restaurant reviews. (To prove me wrong, go here.) Second, FriendFeed tears down exactly the wall I’m trying to build. So my strategy going forward to is build two social computing worlds: (1) a personal world centered on Facebook, which will use FriendFeed to link-in sites like Yelp and (2) a business world centered on LinkedIn, my blog, and Twitter.

On User-Generated Content
The problems with user-generated content (UGC) sites are clearer than ever. Yelp was recently in a mini-scandal where businesses could supposedly pay to determine which reviews came up first. I personally have more trouble using Yelp to pick restaurants because it’s starting to suffer from the TripAdvisor problem where businesses spam-up the site with positive reviews and everyone who's had a bad experience punishes the hotel with a scathing one. The result: 100 people say it’s the best hotel in the world and 100 people say it’s the worst, leaving me nowhere but confused.

Yelp Evolution: Dating?
You heard it here first, but I think Yelp might well end up a better dating website than Match or the other deliberately dating-oriented sites. I think many people will prefer the old-fashioned serendipity of meeting on Yelp and Yelp is already oriented towards social events which further the cause. ("Oh you love Chanterelle, I do too!") My spider sense is telling me that Yelp is figuring this out and running with it in a quiet way.

Yelp continues to fascinate me as a site for many reasons: food is indeed a basic compatibility between people, I have friends who been invited to fellow Yelper's houses based on getting to know each through reviews and comments, and I’ve even had friends invited to restaurants to sample new dishes for the menu based on the quality of the reviews. Yelp will evolve; into what I’m not sure, but I’m guessing it’s more social and less restaurant.

On Twitter: How URL Shortening Changed Everything
Twitter’s evolution is also fascinating. Remember, when Twitter was launched as a micro-blogging, it didn’t have URL shortening. Since most URLs are so long and since Tweets are so short, few people used it for post/article highlighting. It was primarily personal status updates: “I’m in Saint Louis having a beer.”

I think Twitter's original vision involved a lot of mobile phone use and a lot of personal updates as a result. With the advent of one feature -- URL shortening -- first with TinyURL and later with bit.ly, information sharing is now a key use of Twitter. I read about a recent report that said 80% of tweets are personal status updates (so-called "meformers") but information sharing tweets were 20% and growing fast.

Also, remember that Twitter is for “old people.” It’s one of the few super-hyped sites that isn’t used by the teen/twenties generation. Never forget that. If you think you’re in touch with the next generation because you use Twitter, you’re not. Kids use Facebook or MySpace and – as a great example of network effects – entire schools tend to go one way or the other in a binary fashion. My kids have never used Twitter. It’s all about Facebook and text messaging.

Den-Mark: Inaugural Denver Mark Logic User Group 10/12/09

2009-09-28T19:09:00.000-07:00

We continue to have success with local Mark Logic user group events.

The first meeting of Den-Mark (the Mark Logic user group in Denver) will take place on October 12, 2009 at 7:00 PM. The main speaker will be Clark Richey, Mark Logic community champion who will talk about MarkLogic Application Services.

The event is organized by Michael Fagan of Booz Allen Hamilton. This is an opportunity learn more about MarkLogic Server and collaborate with other MarkLogic users.

Pizza and Soft drinks will be provided and everyone will be entered into drawings for several giveaways provided by Mark Logic.

Event Details:

Date: Monday, October 12, 2009
Time: 7PM
Location: Auraria Campus, Tivoli Bldg (900 Walnut St. Denver CO. 80204) Rm 320 Section B - Baerresen Ballroom
Topic: MarkLogic Application Services
Presenter: Clark Richey, Mark Logic Community Champion
Subscribe to the Den-MARK mailing list

Mark Logic User Groups are informal volunteer events for users to meet, discuss, and share best practices regionally. This is an easy way to meet peers and learn how to get more out of MarkLogic Server.

Learn more on our user groups page. and come join us at our next event!

Save The Date: Mark Logic Digital Publishing Summit 12/10/09 at The Plaza

2009-09-16T17:35:00.000-07:00

I'm pleased to announce that the 3rd annual Mark Logic Digital Publishing Summit will be held on Thursday, December 10, 2009 at none other than The Plaza hotel in New York City.

More information will be undoubtedly be forthcoming from our marketing department, but I wanted to get the date out for calendar-blocking purposes as soon as possible.

[Revision: more information is now available here.]

This should be an outstanding event, with I'd guess well over 200 people in attendance, discussing the top issues in the publishing and media business. Mark Logic customers, partners and both technology and media industry analysts and pundits will be in attendance.

We're working on lining up an outstanding program to precede what should be an even more outstanding cocktail reception.

I look forward to seeing you there.

DHT, Listen To Your Heart, and Flanking Strategy

2009-09-15T13:14:00.001-07:00

In 1988, Listen To Your Heart was the third single issued in the USA by the Swedish pop duo Roxette. In this post, I'll use the song as fun example of flanking marketing strategy. First, the original song/video.

(Sorry, but EMI disallows embedding, so you'll have to press the above link to hear it.)

Then, in 2005 the Belgian dance group DHT released several trance cover versions (i.e., remakes) of the song. Here is one such example, the "hardcore remix":

Now it starts to get interesting. Based on the success of the trance cover versions, later that year DHT then releases an acoustic version of the song, which goes on to receive substantial airplay. Here it is:

Now, not being an music expert, I can't say anything for sure, but what are the odds that DHT would have done anywhere near as well by entering the market with a directly competitive Listen To Your Heart? My guess: quite low. Heck, even Alvin and the Chipmunks have remade this song.

In my estimation, the magic -- the reason it all worked -- was they entered the market on a non-competitive flank (club music) and then, once established, bridged from there into making a what I consider a better-than-original acoustic version of the song.

But "better" isn't the point. Simply having a better version on the initial attack would have gotten them nowhere. But building a strong position in the trance genre (flanking) and then bridging made all the difference.

Mercury News Reaches Out For Help

2009-09-14T08:23:00.000-07:00

I've always enjoyed Mike Cassidy, one of a seemingly dwindling number of columnists at the San Jose Mercury News, the self-proclaimed newspaper of Silicon Valley. So I had to respect his cover story column in Sunday's paper entitled It's Time For a Frank Talk with Readers about The Mercury News.

Excerpt:

Newspapers across the country are in trouble. The union representing editorial and business employees at the San Francisco Chronicle, which had significant layoffs in the spring, said recently that more job cuts are expected. Some papers have closed. Others will. Five years ago, San Jose without the Mercury News was unthinkable. Now, well, now we know the newspaper business is changing very rapidly — and not for the better.
So, it's time to talk. But this conversation shouldn't be about the problems facing the newspaper industry and the Mercury News. This should be about solutions — new, brilliant, creative solutions. The Mercury News remains a powerhouse of talent, intellect and dedication. The question is: what do you want us to do with what we have assembled right here and right now?
Yes, there is some urgency.
The Mercury News is already greatly diminished. As advertising revenue has declined in recent years, we've killed features and sections that aren't coming back. We've closed bureaus. And we've lost hundreds of talented journalists through layoffs and resignations. A news operation that bustled with about 400 journalists in 2000 is now down to about 125.

I was happy to see a newspaper asking its readers about where to go from here. In an industry that is often all too product centric, it's great to see people who are thinking about the customer.

To me, answering how to save newspapers is pretty much the same as answering how to save any business:

What needs can it meet better than anyone in the world?
How can it make money in so doing?

The Mercury News is an information business. So what information needs can it meet better than anyone else? None, you might think. In an era of free, real-time, Internet-delivered news, there's no room for any newspaper, you might say.

Not so fast. I'll take myself as a case study. Here's what I read, news-wise:

Print magazines: Economist, Business Week, Forbes
RSS feeds: over 100, usually technology business or software focused
Portal: MyYahoo, usually 95% redundant to the aggregate content of my RSS feeds, but for some reason I skim it anyway
Newspapers: The New York Times, San Jose Mercury News (aka, The Merc), Los Altos Town Crier

When I lived in Manhattan, I read The Wall Street Journal as my "local" paper because I worked on Wall Street. When I lived in Paris, I read Libération, largely because my French tutor convinced me its writing was best matched to my level of French. (I sometimes wonder if it wasn't a clever hoax to get someone in la direction reading a socialist newspaper.) But I digress.

My point is I've consistently had room in my life for three newspapers: (1) a big national/world one, (2) a local one, and (3) a hyper-local one. Whether it's now with the New York Times, The Merc, and the Town Crier or back in college with The Wall Street Journal, the San Francisco Chronicle, and The Daily Californian, the mix is still there.

So let's assume there's meaning in that pattern. I have three broad classes of information needs. So what should the Mercury News do?

Leave national/world reporting to someone else. They cannot compete.
Leave hyper-local reporting to someone else. They will never beat the Town Crier at covering my daughter's soccer team or the local school board.

What's left? Local. And what does local mean in Silicon Valley? Happily, two very clear things:

Business. Silicon Valley lives for business. Venture capital. Startups. Big titans and their stumbles and conquests. Projects spinning out of Stanford and Cal. New disruptive technologies. New business models.

Sports. The San Jose Sharks. The San Jose Earthquakes. The San Francisco Giants. The Oakland As. The 49ers. The Raiders. The FC Pride.

To me, it's a classic case of where across-the-board cuts will destroy you. When The Merc cuts back on overall resources, it should be increasing the relative resources -- and probably the absolute ones as well -- devoted to the areas where they can provide unique value for their readers.

Many are the days when I think about canceling my subscription to a paper that's routinely so thin it doubles over under the pressure of the rubber band, and when I hear the New York Times will be launching an SF Bay Area edition this Fall. But why don't I cancel? Because I'm a season ticket holder to the Sharks and I love the Merc's Shark coverage. And I'm a participant in Silicon Valley and they (used to) have pretty good coverage of local business.

My sense is The Merc is cutting wrong. They are going across-the-board. The business section is badly anemic of late and I view it as one of their two hopes going forward.

Once they've decide on "a strategy" -- i.e., which few things they can reasonably do better than anyone in the world -- then they can charge for it.

Information "doesn't want to be free." Information doesn't want anything at all. The people who produce information want to produce it because they see a return worth their investment. Sometimes that's direct compensation, as in paid subscription. Sometimes it's indirect compensation through advertising. Sometimes, it's fame which is either intrinsically valued or used as a means to land high-paid speeches and consulting gigs. Sometimes, it's a hope that you'll (quite indirectly) end up selling more software as a result.

I think newspapers need to forget the obsession of "do we charge?" for online vs. print. The model should be:

We make great content.
We charge for it.
We don't care how you want it delivered.

Think: check here to subscribe to The Merc's (content). Check here to pay additional supplement if you want us to print it and throw it squarely under the middle of your car each morning.

And charging for content doesn't mean firewalling off everything. Sure they should experiment with various forms of teasers (e.g., selected free stories, free first few paragraphs, perhaps free archives). But the basic model should be: we make great content, better than you can get anywhere else in our chosen domains, and we charge for it.

The more newspapers that do this, the healthier the newspaper business will become. And those who can't charge for content will no longer be able to blame the Internet. The light of day will show that lack of value-added, not the Internet, was their undoing. (Think: many sectors of the B2B trade press.)

While I'm at it, what shouldn't the folks at The Merc be doing?

Trying to figure out how to charge for information that's free elsewhere
Kidding themselves about where they can and cannot really provide information that's better than anyone else in the world
Thinking about the past, the good old days, et cetera. It's all about going forward.

In the end, I don't know if the business I've described is bigger or smaller than than their old one. It's probably smaller initially, but with creativity, you might be able to build it into something bigger. But the one thing I do know is that it's a business. It's sustainable. It has competitive advantage. And it's not predicated on the assumptions of a disappearing past.

Mark Logic and Employees Donate Over $12,000 to the Wounded Warrior Project

2009-09-11T18:39:00.000-07:00

I'm pleased to say that our first annual Federal benefit golf tournament was a great success with over 50 guests and about 15 Mark Logicians enjoying a fine morning of scramble golf in beautiful Northern Virginia. The first-place plaque was won by a threesome with an amazing score of 56.

We were thrilled to have Dan Nevins of the PGA and the Wounded Warrior Project golf with us in the morning and accept the donation in the afternoon. Dan is a truly inspirational guy, we felt honored to have him in attendance, and he is #1 top of the invitee list for next year. Not to mention, he's an outstanding golfer.

I encourage everyone to take a moment to learn more about and consider supporting the Wounded Warrior Project.

Below is a 30-second video featuring Dan.

NetBase Tragicomedy: The Perils of "Magic" and Language Processing

2009-09-03T13:35:00.000-07:00

It's no secret that I'm not a big fan of "magic" in software. You could argue I'm still bearing the scars from BusinessMiner, one of our few failed products, at Business Objects. You could argue that for some tasks, magic is a necessary evil, and I wouldn't argue back too hard. Many Mark Logic customers rely on "magic" to automatically enrich content, adding XML tags that identify entities (e.g., people, places, geopolitical organizations), sentiment (e.g., positive, negative or neutral), or even geo-code content with latitude and longitude that we then index, thus enabling geo-queries against content.

While I confess to some ignorance about how the magical tools work, it's my perception that on a bad day they're 50% accurate and on a good one they're 80%. Now one could argue that content that's enriched at 80% accuracy is way more valuable than unenriched content, and you'd be right. All I'm saying is I'm glad I'm not in the business of making the software that does that, because -- customers being customers -- nobody wants to hear that 80% is great and 100% is unattainable. Perhaps it's my lack of deep expertise in the field. Or perhaps it's my belief that humans are uncomfortable around black boxes.

The other reason I don't like magic is that it can fail in truly spectacular ways. What's the expression? To err is human. To really foul things up requires natural language processing.

This happened today with NetBase, a company whose high-level messaging is fairly similar to Mark Logic's though happily with very different technology and business strategy.

NetBase recently launched healthBase, "a new health research showcase to find treatments, causes, and complications of any condition [and the] pros and cons of any drug, food, or treatment."

Sounds nice. But, today they were slaughtered on TechCrunch with a story headlined: NetBase Thinks You Can Get Rid of Jews with Alcohol and Salt. Excerpt:

Several of our readers tested out the site and found that healthBase’s semantic search engine has some major glitches (see the comments). One of the most unfortunate examples is when you type in a search for “AIDS,” one of the listed causes of the disease is “Jew.” Really.
The ridiculousness continues. When you click on Jew, you can see proper “Treatments” for Jews, “Drugs And Medications” for Jews and “Complications” for Jews. Apparently, “alcohol” and “coarse salt” are treatments to get rid of Jews, as is Dr. Pepper! Who knew?

Here's a great demo of why I don't want to sell semantic processing technology. Here's the reply Netbase gave TechCrunch:

This is an unfortunate example of homonymy, i.e., words that have different meanings.
The showcase was not configured to distinguish between the disease “AIDS” and the verb “aids” (as in aiding someone). If you click on the result “Jew” you see a sentence from a Wikipedia page about 7th Century history: “Hispano-Visigothic king Egica accuses the Jews of aiding the Muslims, and sentences all Jews to slavery. ” Although Wikipedia contains a lot of great health information it also contains non-health related information (like this one) that is hard to filter out.

I hate to be pedestrian, but isn't that just a fancy way of saying it doesn't work? It reminds me of the quip about Autonomy, where, when the Bayesian and Shanon's Information Theory magic isn't working, they simply tell the customer that they're not smart enough to understand why. Nice.

Now, for the hapless NetBase, the AIDS query was just the beginning. They get destroyed in the blog comments, which quickly turned into a contest to find the silliest results.

The treatment for venture capital is funding. The cons is fool.
Masturbation causes insanity and is cured by cocaine.
The treatment for Twitter is Facebook. (This one might be right.)
The treatment for Microsoft is Viagra
Babies are caused by smoking and brain damage

It goes on and on. Now yes, many of the silly queries are out of the health domain, but there has to be better way to answer them.

One active commenter, Dave, who coined the "tragicomedy" description and who isn't me, had this to offer:

The tragi-comic failure of Netbase can teach a lot to every company in the Semantic space.

Lesson 1
Don’t even try to boil the ocean of the WWW with these technologies. [The] Internet is full of valuable information but crap (or opinions) is 90% [of it] , the cost of getting rid of this crap and save only the good stuff is very high, [and] that’s [what] makes [it] so hard to succed even for Google and Microsoft with billions [of dollars].

Lesson 2
Linguistic approaches are likely going to fail because search engines (and machines) can’t distinguish joke/seriousness, sarcasm/shame and sentiments in general. The semantic meaning is right there not in the words of a text.

Lesson 3

If you choose to apply such approaches to one specific topic like Medicine (good choice) then stick to that topic , that means accept as INPUT only medical terms and provide as OUTPUTS only medical terms.
...
This last point requires human intervention and predefined taxonomies/ontologies but Netbase claims that they don’t need them both, ]i.e., that] their engine is fully automatic —> the failure too.

Gordon Crovitz on Brands and Newspapers

2009-09-03T09:50:00.000-07:00

Here's a nice video clip of Gordon Crovitz, former publisher of The Wall Street Journal, talking about some of the peculiarities of the newspaper business and branding.

I've seen Gordon speak several times and think he's a fascinating speaker. We're trying to get him to speak at our Digital Publishing Summit this Fall/Winter. Hopefully, we'll succeed. Meantime, here's a little tidbit.

Why Mark Logic Isn't Bigger Than Oracle. Or Is It?

2009-09-02T17:42:00.001-07:00

I had a great meeting with a Silicon Valley technology executive this afternoon. He was a very clever chap with all the right credentials: an engineering master's, a top MBA, experience at Google. The works.

He asked us the standard set of questions that tech executives do.

Is this a database or search engine? (Yes. As in both.)
Does it have transaction consistency? (Yes.)
Does it provide real-time search? (Yes.)
Does it have a query language? (Yes. XQuery.)
Does it scale? (Yes, to 100+ TB today)
Does it cluster on cheap hardware? (Yes.)
Do you require schema adherence? (No.)
Can you handle semi-structured content? (Yes.)
Is it native XML? (Yes.)
What database does it run on? (Itself. i.e., it is a DBMS.)
Do you use Lucene? (No. It has our own built-in search engine.)
Is it a database bolted to a search engine? (No. It's both but as an integral hybrid.)

We went to cover common customer use-cases, talking about the generic reasons why customers buy Mark Logic:

Messy XML. (Various structures, unknown structures, changing structure.)

Big content. (Tens to hundreds of terabytes and nothing else will perform.)

DBMS/search-engine integration fatigue. (Tired of paying the costs of development, maintenance, and synchronization.)

Semi-structured data. (Such as the wide, sparse table problem.)

"Special" search requirements, where "special" could mean any combination of: structured, parametric, real-time, language-specific, or geo-coded search.

Then, suddenly, he caught us off-guard with his next question:

If you actually do all this, then why aren't you bigger than Oracle?

In the meeting, we stumbled: "uh, gosh, well, that's a question we don't hear often." Now, had I brought my A-game to the meeting, here's what I would have said:

On an age-adjusted basis, we are.

For my source, check out this New York Times post, How Long Does It Take to Build a Technology Empire? The post measures various high-tech companies on their historical revenue ramps, specifically addressing the question how long did it take to get to $50M in revenues?

The answer for Oracle is 10 years. In Oracle's year six, according to the graphic -- which is interactive, you should play around with it -- they were a mere $5M in inflation-adjusted revenues. Mark Logic, in year six (from its A-round funding) is many times that and should break $50M, in my estimation and with continued good fortune, in year 7.

Here's to Focus: Mark Logic Named Temis Partner of the Year

2009-09-02T16:46:00.001-07:00

I'm passionate about excellence. Because it's fun. Because it requires focus and intensity. And because I think it's the key to success in Silicon Valley.

To me the success formula is simple:

Pick a problem domain
Solve it better than anyone else in the world
Build up and out from there

Bullet #2, of course, is the tricky point. You don't just need to do it, whatever it is. You need to do it better than anyone else in the world. For real. You can't kid yourself. You need to do it better than the 20 guys buried somewhere within Oracle, IBM, and Microsoft who inevitably are doing the same thing. Better than the dozen other startups funded by herdish venture capitalists. Better also, perhaps, than an open source community attacking the same problem.

Excellence is how two 27 year olds in France launched a company that beat Oracle in Oracle-based reporting tools: Business Objects. Excellence, for that matter, is how Oracle crushed not only prior-generation DBMSs like IMS and IDMS, but also its relational competitors such as Ingres, Sybase, and Informix.

There's always a temptation in high-tech to build what I call a "moldly sandwich." Well, the ham's a bit old, the mayonnaise is turning, the cheese has little spots on it, and the bread is starting to mold. None of the components are the best, but heck, it's a sandwich -- and isn't that what people want to buy? Not ham, bread, and cheese. People want solutions.

Well yes. And no. It depends what they're made of and how hard they are to assemble.

At Mark Logic, to continue my metaphor, we make great ham. My friend Eric Bregand at Temis makes great cheese (text mining). Our other partners make great bread, great mayonnaise, and great pickles. Our services partners are the chefs, putting those ingredients together to make great sandwichs.

That's why I'm happy to have been selected as Temis Partner of the Year 2009. To me, it's a testament to focus, excellence, and partnership. Mark Logic, to quote the body paintees at our company picnic, makes the biggest, baddest, fastest XQuery implementation. Temis makes world-class text mining. Together, they're a great combination.

Is anyone else getting hungry?

The Perils of Text-Only Search

2009-09-01T17:34:00.000-07:00

You won't be surprised to know that I use a series of Google Alerts to help me track events relevant to Mark Logic. I often have two problems with them:

Old content is mis-identified as new. You can ask any Mark Logician about the number of times I've forwarded a story that I thought was a hot news item only to discover it was four years old and that Google had nevertheless "alerted" me to its existence. I highlight this here because it bugs me, but I will not drill into it.

Content is mis-parsed, resulting in erroneous matches and alerts.

For example, today I received a Google Alert on "Mark Logic" for this Reading Eagle story, entitled Douglass Township Man Waives Hearing on Charges He Fired Gun in Neighborhood. Wondering if we had a wayward employee, I read the story which is about a man named Jeffrey W. Logic, who is charged with firing several shots near a group of people assembled in a neighbor's front yard.

Here's the text that generated the hit. (Bolding mine.)

Logic pulled out a gun and fired several shots into one of the car's tires. He also fired a shot into the pavement, and a stone or bullet fragment ricocheted and struck the driver in the neck, causing a red mark.

Logic started to walk away when two men who had been at the party approached him. Logic pointed the gun at one of them but the man swatted it away. Logic then fired into the ground once more.

What happened? The words mark and logic are sequentially related in the text. But they're not in the same paragraph, let alone the same sentence. Clearly, if you'll pardon the pun, this result is a misfire, but it highlights an important problem with full-text search engines: they understand neither the structure nor the semantics of the content they are indexing.

For example, in an XML representation, you might indicate structure by using <para> tags to indicate paragraphs and <sentence> tags to indicate sentences. When searching, you could then say "find all the <sentences> that contain the phrase 'Mark Logic" and you wouldn't get the false match that Google returned.

Awareness of structural markup is important not only because it eliminates false matches, but because it enables you to express more powerful queries (from both a search and retrieval perspective), such as:

Find all the <figures> that have <captions> that contain the phrase "survival rate"

Return the <authors> and <abstracts> of articles that contain the word "lymphoma" and have <captions> that contain the phrase "survival rate"

Or, more powerfully, perform a citation analysis:

Return the <authors> and <abstracts> of <articles> with <citations> to <articles> written by <author> "Sandra Horning"

But, even better is the ability for the system to understand semantic markup, for example, coming from a taxonomy or an automatic entity extraction tool.

And find them only in the <articles> that contain references to the <drug> "Rituxan" which is a <monoclonal antibody> which the system knows is also called "Rituximab" and "MabThera."

And which contain the <disease> diffuse large b-cell lymphoma, which the system knows is a <b-cell lymphoma> which is a <lymphoma> which is a <blood cancer> which is a <cancer>

Then think of the simple permutations of this query you can run:

Against all monoclonal antibodies, not just Rituxan
Against all lymphomas, not just diffuse large b-cell. Or against all blood cancers.
Against not just those citing Horning, but against those citing any other author.

And then think of all the queries you can run against this same corpus when you apply any number of any combination of full-text, structural, and semantic constraints.

And pause.

If you wonder why I say that Mark Logic lets you run database-style queries against content* you hopefully now understand why.

It's not just about catching that Mark is the last word of sentence 20 and Logic is the first word in sentence 21. It's about combining structural, semantic, and full-text constraints and in virtually any combination. And that unleashes a mind-boggling amount of query power. A power, by the way, that we're accustomed to against data, but are now only beginning to understand against content.

---
* If you want to search only within items the author bolded for emphasis, you can do that, too!

** Or if you want to search only within footnotes, as they sometimes do in finance, you can do that, too!

I Just Bought Storage at $500/GB and Other Musings

2009-08-22T17:06:00.000-07:00

(Revised: Math Error at the End Fixed.)

With a 1 GB flash card costing ~$5, how in the world did I end up buying storage today at price of $500/GB?

The answer: I had to buy 3.5" floppy disks. A pack of 10 cost me $7 and holds 1.4 MB x 10 = 14 MB. If I did the math right, that's $500/GB, about 100x what it would cost as flash memory. (Frankly, I was happy to pay it because Symantec PartitionMagic required the creation of "rescue" floppies, which had to be floppes.) Office Depot had exactly 1 SKU, the ten-pack.

A funny thing happened while I was at the store. A woman said "Sir, do you know anything about memory?" I always wonder why I'm mistaken for retail clerks. Do I look smart or helpful? Or, perhaps I need to upgrade my weekend wardrobe from Cabela's to Faconnable?

Now, what I am supposed to say, "Ma'am, I'm CEO of a software company; I don't do hardware." No, that won't do. What I say, of course, is "yes."

"How many gigabytes should I buy (on a memory card) to hold a 200-page document? My daughter thinks I need 32?" she asks. My reactions are first that you should be questioning me in megabytes, not gigabytes, and second how we've completely lost touch with how big things are.

"No embedded images or figures," I asked. "Just text?"

"Yes."

I told her 1 GB should do fine. While the link above suggests a whole novel can fit in one megabyte, that strikes me as tight. Perhaps that's true for the .TXT version of it, but what would it be in Word, I wondered. So I quickly lorem ipsum'ed up a 200-page Word document to find out.

The answer: it contained 690K characters including spaces, and the Word file was 804K. She could have put it more than 36,000 times on a 32 GB card.

If you're interested overall in the costs of storage over time, check out this page, which has a great history from 1956 ($10M/GB) through to 2004 ($1/GB).

Business Problem	XML Solution
Product content exists in multiple formats and cannot be repurposed	XML can be used to create a canonical set of tags that rationalize disparate formats. Automated tools can take Word, PDF, text, and other file formats, and tag their structure.
Publication processes are “hardwired” for each product due to product-specific format	XML enables creation of an “author once, publish many” process where content is created once then tagged for use in multiple products
Clients demand custom delivery formats and each variation requires its own software	XML schemas can be used to tailor content delivery to client-requested formats
Content and format are intertwined, making it difficult to change either	XML enables separation of content and format using CSS or XHTML.
Search engines return long lists of documents with actual search results buried inside the documents. Search is limited to key words and phrases.	XQuery is XML’s standard query language, enabling subdocument search and construction of very specific searches and queries.