Cambridge Semantics

Google Knowledge Graph and the Semantic Web

2012-05-17T17:49:39Z

This week Google announced the Knowledge Graph. First of all, I love the slogan "Things Not Strings." That about sums it up. When you can take advantage of the meaning trapped in text, you can do quite a bit.

Even though Google is late to the party on this particular trend, it's great to see them making real progress. I had previously used Google rich search results as an example of semantic search on the Web, and the Knowledge Graph takes it to the next level.

From the description:

Search is a lot about discovery—the basic human need to learn and broaden your horizons. But searching still requires a lot of hard work by you, the user. So today I’m really excited to launch the Knowledge Graph, which will help you discover new information quickly and easily.

Take a query like [taj mahal]. For more than four decades, search has essentially been about matching keywords to queries. To a search engine the words [taj mahal] have been just that—two words.

But we all know that [taj mahal] has a much richer meaning. You might think of one of the world’s most beautiful monuments, or a Grammy Award-winning musician, or possibly even a casino in Atlantic City, NJ. Or, depending on when you last ate, the nearest Indian restaurant. It’s why we’ve been working on an intelligent model—in geek-speak, a “graph”—that understands real-world entities and their relationships to one another: things, not strings.

The Knowledge Graph and the Semantic Web

So what does this mean for the Semantic Web? It depends.

It sounds like Google has a curated Knowledge Graph that they control, and they use it to map documents on the web to concepts in the Knowledge Graph. It can then use the Knowledge Graph to expand the search results. For example, searching for something like "Tim Berners-Lee student" Google would know that TBL is a person and would also know who his students have been over the years, and return information on them instead of on TBL. That kind of thing is basically impossible without something like a Knowledge Graph, and enables much richer querying behaviors.

If this is the case, then hopefully metadata in web pages exposed via Schema.org (also backed by Google) vocabularies or popular Semantic Web ontologies such as FOAF makes it easier for Google to index and consume rich data on Web pages. Those that add the extra metadata are treated better in search results, and everyone would be encouraged to add more metadata to their pages. That would be a big step forward in the right direction. Explicit metadata on pages takes much of the guesswork out.

Another way it could benefit the Semantic Web is if Google were to publish the Knowledge Graph (via a SPARQL endpoint naturally!) making it part of the Linked Open Data Cloud. I can see it very quickly surpassing DBpedia in popularity. More importantly, I can imagine plugins that enable you to mouse over words on web pages and have a Knowledge Graph-powered panel popping up for more detail and other such low hanging usability improvements to information pages.

Conclusions

What do you all think?

Regardless of what happens next, this underscores the continuing trend beyond keyword search to actual meaning, and that is another step towards the vision of the Semantic Web.

How is semantic technology more flexible than relational technology?

2012-05-09T21:54:13Z

Michael from SemanticArts started this great thread on Linked In a couple weeks back. The question: How is semantic technology more flexible than relational technology?

There is a lot of researchy fluff in there, but I wanted to highlight a misconception that is widely held in the community.

He starts with an initial overview that I liked quite a bit (emphasis on the good stuff is mine):

For me the flexibility of semantic technologies comes from when you classify or organize data.

In a traditional relational database you define a schema (a classification/organizational tool), and you put data into the database organized according to that schema. This has one advantage -- it allows for speedy data access. But it also has one disadvantage. Your database contains data organized according to your understanding of your business at a fixed point in time -- but since things change, your business will immediately diverge from that understanding.

(BTW the same argument applies to OO databases, and OO programming languages, indeed any situation where you chose the organization scheme first)

Contrast this to semantic technologies. You put data into a triple store in a somewhat unorganized manner (it's not completely unorganized as you have to chose names for properties, classes etc, and decide what you want to record). You do very little classification/organization on data insertion. Only when you pull data out of the triple store do you organize it according to some ontology (here's where reasoning helps!). As a result you are always pulling data out organized according to your most recent need for and understanding of your business. Of course there is a downside -- speed.

A good analogy for the semantic approach is an archeologist at a new dig. He lays out a grid so he can say where objects are located, and he has some basic vocabulary like "made of wood", "made of ceramic", etc. He then simply writes down facts about the things he finds -- "Artifact 12 made of wood", "Artifact 12 found at location XYZ". He doesn't organize his data beyond this. Later he may add other facts like "Artifact 12 dated to 650AD", etc. Later when he wants to study some of the contents of the dig, he defines what he is interested in and then gets his army of graduate students (his reasoner!), to organize the relevant artifacts (i.e. to build his schema) according to his definition of what he wants. The archeologist has to work this way since a priori he doesn't know how to organize his findings -- he only knows the organization he wants as new facts come to light.

Another way to look at the schema vs no-schema debate is as points on a spectrum. At one end is flexible, at the other is speed (the constant tension in IT ...). If you really need speed go for a schema based approach. If you really need flexibility go for a no-schema based approach. If you need a mix of both, chose some point in the middle -- it's an engineering decision.

But then makes the fallacy (emphasis mine):

I must admit that the analogy isn't wholly my idea. I gave a talk about semantic web for OO programmers at the San Diego JUG and an actual archeologist was in the audience. He came up to me afterwards to ask lots of questions. His questions sparked the idea for the analogy.

It raises a question though -- if you "mess up" your choice of initial vocabulary, then semantic systems have similar sorts of problems to relational systems, in that you may have to restructure the vocabulary at a later date (just like you have to restructure relational schemas), and hence restructure the data.

My experience suggests that such restructuring happens (far?) less often with semantic systems than with relational systems, and that the restructuring is easier, since you can always treat your data as one big list of triples.

But how do you avoid "messing up" the initial choice of vocabulary. Somehow you have to choose your vocabulary to capture only the "atomic" ideas of interest -- i.e. ideas that cannot be derived from other ideas. The archeologist knows how to do this from years of experience of course, not just his, but the collective years of experience of the field as a whole.

A professional ontologist somehow manages to choose a vocabulary in such a way as to largely avoid restructuring issues -- choosing it to avoid ontological commitment as much as possible. Are there any metrics/thoughts on how they do this? Also are there any metrics/thoughts on why restructuring semantic systems is easier?

I think this is very key, and this is how I responded:

I actually take a different view. I believe that messing up ontologies is unavoidable. A simple example is two people publishing similar data sets at different times without reusing each other's ontologies. This happens all the time.

The power that RDF, OWL, and SPARQL give you in this circumstance is to quickly build maps between related constructs in a way that is not possible in the relational world (you'd be in ETL hell). In some cases, a simple owl:sameAs gets you there. In others, a basic SPARQL CONSTRUCT expression and you're golden. Either way, the translation layer is cheap.

To get back to your example, if somehow you mess up the model in some fundamental way, it's relatively (as compared to the relational world) cheap to make the modification, do the translation, and (*this* part is key) maintain an interface that is consistent for existing consumers.

TL;DR: with Semantics change is cheap, so mistakes are OK.

Introduction to Unstructured Data

2012-04-30T18:50:13Z

Expressing, communicating, and understanding meaning is natural to us but opaque to computers. Computers lack shared understandings, norms, languages, and common sense. They are, after all, merely tools. Today, people have to communicate in ways a computer understands for it to be able to work with the meaning, but there are small signs of this changing, like with Apple's Siri and IBM's Watson, but those examples are few and they are siloed.

Where files and streams are not generally intelligible by computers, they are unstructured data in IT parlance.

Unstructured data represents the majority of enterprise data and it continues to grow faster than people can consume it. It is emails, snailmails, and voicemails. It is handwritten document scans, photographs, and video. It's Word documents, PDF files, and PowerPoint presentations. It's tweets, message board postings, and online reviews. It runs broad and deep.
And it holds a lot of locked-away value to organizations.

Most importantly, it contains the most adaptive, fluid information, just at the edge of being gleaned organizationally.

Market sentiment: whether for stocks or stockings.
Breaking news: everything affects you in one way or another.
The voice of your customer: will you listen?
Competitors' announcements and the respective buzz: will you compete?

More often than structured data does, it contains leading indicators.

Manual processing of unstructured data happens implicitly, most often without anyone realizing that's what's being done. Summarizing a customer's diatribe, forwarding on a letter to the appropriate department, being cognizant of relevant product recalls, or entering dealers' quotes into a spreadsheet.

However, manual processes are seldom scalable, repeatable, or that fast. But why hope that a machine can understand these sources at all?

Using unstructured data properly, computers help bring order to messes, bring people together, and bring ideas and resources together. They can help people share and build on knowledge, can help people find links between things, and can help people care about the implications of things. Conceptually-linked structured or semi-structured metadata and related content could be explicitly linked, enabling enhanced navigation as well as some level of automated inferencing. The endgame is not replacing knowledge workers, but letting them work smarter, with more knowledge being made available to them faster: people can focus more on more value-added contributions rather than spend time on the mechanics.

Making this possible, artificial intelligence and related fields have made great strides in the past couple of decades. Computational linguistics, natural language processing, machine learning, knowledge representation, and big data analytics have all been breaking new ground in theory and practice; but there is still no magic bullet to software understanding everything that a person would. Software can however make sense of large quantities of unstructured data with specific goals in mind like for the above-mentioned, now manual, processes.

Methods of Dealing with Unstructured Data

We'll survey contemporary functionality and then we'll peek at what's on the horizon. Common things done with unstructured data today include search, faceting, clustering, summarization, tagging, and information extraction. Collectively, these are often called text analytics.

Search is the most common way we interact with unstructured content today. Sometimes we search on a phrase, like finding relevant pages with traditional web search engines. Sometimes we search, or facet, on a topic, like when we go to the Health news section of a modern news aggregator. Sometimes we implicitly search for what people actually do with the subject at hand, like when we view a product on Amazon just so we can see what people who viewed that item actually bought. Different technologies underlie these examples, but they are all focused on organizing a corpus, or body, of documents, in a way that makes sense in a particular context.

Documents can be automatically summarized, where the software either figures out the most representative sentences within the document, or alternatively, generates an entirely new paraphrasing from what it was able to understand using linguistics, context, topic mentions, and some level of semantic framing. A similar operation is automatically finding apropos topics to tag onto a document, which can then be fed back to faceting workflows.

Information extraction is the extraction of structured concepts or facts from unstructured data. Things like people, phone numbers, organizations, web addresses, and medications, are examples of classes of what are collectively considered entities. Relationships between entities, such as hirings, product releases, protein-protein interactions, and more complicated events, are another important general class of information that can be extracted and put to direct use. Semantic frames, not to be confused with other semantic technologies, are advanced meaning-oriented and somewhat context-aware tags and structure on words and phrases (but this raw technique is nearly impossible for an end-user to use in isolation). For applications like brand monitoring on social media, sentiment analysis, whether broad-stroked, or nuanced and attributable to certain things, is another common type of extraction.

When information becomes structured, and knowledge is represented, in ways that can contextually and semantically make sense to a program, it can be reasoned over with automatic inference and user-defined rules, often an impetus for information extraction. Taking the concept of information extraction to its logical conclusion, one may wonder why software can't simply understand everything in any arbitrary text with some kind of general knowledge modeling. To really address this, we need to consider the field of knowledge representation. I have purposely kept the technical discussion of what underlies these approaches out of this article, so the topic of our second installment in this series will be on the various approaches to knowledge representation as it relates to unstructured data.

Greater Than The Sum Of Its Parts

Solutions that combine basic unstructured functionalities in novel ways provide more robust options than any technique in isolation. Some impressive text mining programs, for instance, do upfront indexing of entities, and let users search for specific new classes of relationship on the fly, combining information extraction and search.

Taking this power of combination to the n^th level, a powerful approach on the horizon is to leverage as many appropriate techniques and technologies as possible, in conjunction, from natural language processing, computational linguistics, information retrieval, information extraction, and machine learning, and to do this in a flexible and easy way that lets you adapt to the goal at hand.

When you can plug in any combination of unstructured technologies and enable them to cooperate in novel ways to solve your problems, technologies can focus on their respective strengths, the system can relate the functionality and output of disparate technologies together, accuracy can improve, you can relate unstructured data back to your structured data, you can automate or structure workflows, and your users gain multiple poignant and empowering perspectives on newly unlocked information.

When information is in the terms that make sense to you, it is knowledge. When it is salient, it becomes important. What you do with it becomes insight. Systems must be able to manage unstructured information throughout its lifecycle—from origination to action—to really make use of it, and to do that requires a combination and correlation of existing techniques to a level unavailable in a single product today.

NoSQL Equals NoSecurity: Sometimes

2012-04-09T18:35:25Z

Over at InformationWeek, Michael Davis wrote an impassioned post lambasting the lack of proper security in the NoSQL World.

Clearly, the developers driving the NoSQL bus just don't get it. The only thing we've gotten from years of pushing to secure Hadoop and other big data technologies is integration with authentication frameworks such as Kerberos. Excuse us if we don't swoon with gratitude.

As technologies like Hadoop and NoSQL go mainstream, this situation must be addressed.

His big concern is that people use NoSQL to store and manage financial data, such as transactions, which occur in volumes too large to be effectively managed by traditional database technologies.

I've written about this kind of issue before in What Happened to NoSQL for the Enterprise. Developers are using NoSQL systems to solve specific problems, but leave lots of traditional database features on the table to make that bargain. To get to significant scale and easy cluster management, they give up on transactions or, in this case, security.

Proper security is not only tricky to implement, but typically has a performance cost. This cost goes against the main reason to use NoSQL databases: blazing scale and performance. What Michael identifies is that as soon as you start storing sensitive information—personal data on customers, financial information, medical data—you shouldn't make this bargain. And I agree.

As I've said before, Semantic Web technologies represent a very interesting NoSQL solution for the enterprise exactly because they don't jettison database best practices in order to get NoSQL benefits. Anzo, for example, supports transactions, logging, data provenance, encryption, fact-level security (think cell-level), etc. And we're not alone in this space. Revelytix has done a great deal of work at the US DoD, for example, and the DoD certainly takes security seriously.

There are certainly lots of applications where this is not important. But we can't just ignore security requirements to play with trendy technologies. For enterprises looking for serious alternatives to SQL systems—especially where flexibility and the ability to use unstructured data are concerned—Semantic Web systems represent the most mature databases around today.

SWiPE: An Example of Easier Semantic Web Software

2012-04-04T01:59:13Z

I was thrilled to come across ZDNet's coverage of SWiPE, a query-by-example approach to structured searching of wikipedia. SWiPE is being developed by Maurizio Atzori and Carlo Zaniolo and will be presented later thismonth at the demo track of the WWW2012 conference in Lyon. While information about the SWiPE design and implementation are presented in excruciating detail in Maurizio and Carlo's conference paper, the formality and detail of the paper belies the simplicity and ease-of-use of the tool.

Maurizio and Carlo are building on top of DBPedia, the Semantic Web representation of information (mostly) from Wikipedia infoboxes, but you wouldn't know that from watching the tool in action. There's no URIs, no mention of RDF, no mention of SPARQL; finding specific answers to a question is straightforward. Go ahead and watch the demo of SWiPE in action. It's only 23 seconds, I'll wait:

Got it? SWiPE lets you fill in values in existing infoboxes to find any entries on wikipedia that match the information you supply. It's simple and obvious to use, and it doesn't require any new context or user interface beyond the infoboxes that we all know very well already.

SWiPE is a fantastic example of what I've written about before: the need for Semantic Web software that is easy to use. As long as people need to learn SPARQL or learn how to use a linked data browser or learn how to use a circles-and-arrows-based query tool, the vast majority of people who could benefit from the power of DBPedia's structured representation of Wikipedia data were going to be missing out. By making Wikipedia search simple, though, SWiPE has the potential to bring this benefit to a far greater audience. I, for one, can't wait to try it out myself.

Semantics in the Real World: Where to Begin?

2012-04-03T19:47:23Z

A lot of the chatter in the semantics community is jargon-heavy, very technical, or both. To a newcomer trying to evaluate what problem, if any, might benefit from semantic technologies (and even which to apply!), this can be very confusing.

As part of Semantic University, we decided to include some introductory content to help people identify what kinds of appilcations could benefit from Semantic Web technologies, as free from jargon as possible. This set of four articles starts with what characteristics a use case might have to make it a good target for semantic technology application, and then continues to provide surveys of semantic technology applications on the web and in the enterprise, finishing with some short case studies.

Feel free to send these along to anyone that says, "So what?" or "Why?"

Introducing Semantic University

2012-04-02T14:12:53Z

Today I'm very proud to announce the launch of Semantic University.

We are creating Semantic University to be the most accessible and most complete place to learn about Semantic Web and other semantic technologies. Although there is only one lesson posted today, we have dozens in various stages of production, with many more ready to go. Subscribe to the mailing list to get updates when new lessons come out.

The Need

Let's face it: there needs to be better curated material about semantic technologies. This is especially true for people new to the space, but also applies to those with a basic understanding and looking to do something using them.

Put yourself in the shoes of someone who just heard about semantics—a colleague told you, or you saw a presentation or demo at a conference, or someone forwarded Tim Berners-Lee's TED talk to you. You would go to Google and search for "What is the Semantic Web?" How many pages would you have to go through to get to something suitable for a real beginner?

Or suppose you're a non-technical buyer who is being told that Microsoft's SQLServer 2012 has a "semantic modeling layer," how are you to know how that's different?

We experience this as a vendor and as a business all the time. We recently hired a company to help us out with our website, and in the second meeting they said to us, "Listen, we spent a couple hours trying to figure out what the heck the Semantic Web is, and are still a little confused. We think we need to at least have some idea to work with you; can you take some time to give us a tutorial?" They're not wrong.

The learning curve today is too steep. Although there are blogs and articles that tackle pieces of the problem here and there, you really have to be motivated to sift through it all yourself and piece together an understanding. Getting to the point where you know when to apply the tools and when not to is a daunting task.

Compared to Other Communities

When I look at other communities that are taking off, such as Node.js, MongoDB, Ruby on Rails, etc., they all have really terrific places for people to learn. They also do a great job integrating into existing ecosystems and patterns of behavior in their examples. If you look at MongoDB, for example, they have tutorials on using Mongo with many different web CMS systems, and in almost any language, and the tutorials get you up and running on a simple app in under an hour.

Where can you go for this experience in Semantic Web technologies? A book isn't sufficient, since a person is already committed to learning the technology by the time he makes the purchase.

Beyond the Basics

Even when you get beyond an initial understanding, the RDF tutorials out tend to focus on RDF/XML syntax, which is a tough place to start. There are few SPARQL tutorials that walk you through the process from the very beginning—download a SPARQL client, try this query, etc. There are more debates about RDFa vs. Microdata than there are helpful guides on getting started with RDFa.

Not enough focus on doing something real. They don't do a great job of putting the technologies into a relevant-enough context to let people take them out of the lab and into the real world, except through big vendor deals.

The Mission

This has to change. Our community has to grow. The time is right for the Semantic Web to take off. Schema.org, Facebook Open Graph, the growth of the Linked Data Cloud: it feels like we are so close and just need something to tip.

An accessible place to get started is key to that.

Today we launch the beginning of Semantic University. If you would like to contribute, we are actively looking for more writers and people to create hands-on exercises. Just drop us a line.

Existing Materials

This is not to say that there are not existing tutorials, presentations, videos, blog posts, and documentation that is high quality and relevant to the Semantic Web. There absolute is. The major problems are that they tend to be tough for beginners to grok, and that they are scattered all over the place. Having a central location from which to start is, itself, valuable.

Where existing materials exist and are relevant, we will link to them from the "Further Reading" bar on the side of each lesson. If you have a blog post or tutorial that is not mentioned and that you think is relevant to a topic, please let us know.

One More Thing

Lastly, while we are a company that sells a software product based on Semantic Web technology, this space is not a place where we're going to be advertising Anzo. We strongly believe that a rising tide will raise all ships in this community. Learning material associated with our own product line will remain separated in our own online documentation and forums.

Local vs. Global Semantics

2012-03-12T14:17:22Z

One of the big stories around the Microsoft’s SQL Server 2012 release is the inclusion of its new Business Intelligence Semantic Model (BISM). This isn’t anything groundbreaking in and of itself, as traditional BI tools such as Business Objects have incorporated a “Semantic Model” of sorts for years. For the record, these BI “semantic models” are not based on Semantic Web technologies, but that’s not what’s interesting to me. What’s interesting is that it got me thinking about the Continuum of Locality of Semantics in general.

The Continuum ranges from Local to Global semantics. Where you are on the continuum dramatically impacts your ability to communicate with other systems without requiring a translation layer.

On one end of the Continuum, the semantic layer is extremely localized and really only exists to support a single application or set of data. BISM is an example of this kind of local semantics: it’s meant to allow a multi-dimensional BI model to co-exist in a single server with the standard tabular data one expects in a relational database.

On the other end of the Continuum are semantic standards that are national or international, and often mandated. VCard is one example of this. Even if you don’t like some aspect of VCard, it’s not like you would choose any another contact format to accomplish something similar; that’s foolish. VCard is everywhere and has won.

In the middle are things like MDM systems, which are closer to the global semantics than local in spirit in that they attempt to define one set of semantics for an entire ecosystem. MDM systems are, in a way, top-down and authoritarian in their demands, and they somewhat naively assume that such mandates can be maintained over time. Businesses, partnerships, contracts, and competition move faster than IT can keep up with an MDM system that is used by multiple departments, and at some point it becomes dated, and so users produce data that doesn’t conform.

My observation here is that most technologies are built specifically to perform well at one specific location on the continuum. MDM doesn’t make much sense at the very local end or at a global level beyond an organization, just as BISM doesn’t make any sense outside of a single SQL Server instance.

This all brings me to my realization: Semantic Web technologies can play all along the continuum! By their nature, RDF-based applications can be both top-down and bottom-up at the same time.

For example, Linked Open Vocabularies (LOV), Schema.org, or the Facebook Open Graph vocabularies are very much global in scope, and top-down in nature. However—and this is the cool part—if I have a local CRM system built on Semantic Web technologies (which we do) I can choose to reuse LOV concepts, or simply describe how my data relates to them. I’m not mandated to follow a set of rules, but I am aiding in data interchange by following locally those that make sense.

It gets even better. A Semantic Web application that exists at one point in the continuum can very easily talk to applications that are either more global or more local with much cheaper translations than are required between, say, an MDM implementation and SQL Server. For example, SPARQL is perfect for this sort of lightweight, fast translation that’s required between two systems built by two people in two contexts but that deal with similar-enough data.

So the Continuum of Locality of Semantics is real, but is easily traversable by Semantic Web technologies and not by traditional “semantic” models.

The New Era of Just-in-Time Compliance

2012-03-06T22:18:40Z

Federal Reserve Chairman Ben Bernanke: “We have asked the banks to essentially do stress tests and ask, looking at all their positions, all their hedges, what would be the effect on their capital be if Greece defaulted?”

US FDA: All drug companies have to address “safety changes in labeling for some cholesterol –lowering drugs”. (One of many growing compliance requirements...)

US Health & Human Services: "We have heard from many in the provider community who have concerns about the administrative burdens they face in the years ahead” (...as it relates to complying with the implementation of the new ICD-10 standard and other compliance rules...)

The Need for Just-in-Time Compliance

I recently met with Richard Soley, CEO of OMG, one of the world’s leading standards setting groups. Richard mentioned, “This is the year for semantic technology.” I was pleased to hear that but asked, “Why?”

Richard felt that companies in the financial services, healthcare, and government all are looking at semantic technology to help them rapidly search, aggregate and manage data from varied sources. He felt that compliance is a key area where Semantic technology is being used. One data point: banks are increasingly creating the Chief Data Officer role, often reporting directly to the CEO. A key job requirement for these CDOs is to answer such questions such as Chairman Bernanke’s Greece exposure question. All these officers (or at least the ones we have talked with) feel that semantic technology is the technology to address global risk and event impact questions.

Note: You can hear directly from these Chief Data Officers and other senior business executives from several major financial institutions at OMG’s March 13^th Financial Services Semantics Conference. At the conference, business leaders will discuss how semantic technology can help with governance, risk, compliance management and other issues. (See Lee Feigenbaum’s post from March 2^nd for more details, including a discount registration code).

What is the issue? When regulatory agencies issue new or updated rules, it sets in motion a complex chain of activities and custom development projects for affected organizations. These are almost entirely manual activities, and so they are costly, time-consuming, and error-prone. While there is no shortage of traditional off-the-shelf compliance software packages, none of them allow for easy integration and management of changing environments. Current available compliance software tend to be inflexible: they are not meant for responding to unanticipated regulator requests (at least that is what one of the major money centered banks told us). The result is excessive regulatory management expense, incomplete data, inefficient processes and periodic fire-fighting exercises.

What’s the Solution?

As we’ve seen, just-in-time compliance requires a great deal of flexibility. We’ve found that Semantic Web technologies provide the necessary flexibility for this kind of problem.

Semantics allows end users to create their own data model to access, aggregate and analyze data from different databases and text sources. If the user needs to collaborate with someone else, then they can easily map their data models to a new jointly agreed data model and all people can share and use the data.

In the compliance realm, this means that different business units can have different models for what constitutes a policy, a detective control, or a preventative control, yet these different models can still all map back up to corporate compliance’s notion of a control (& so of the compliance results that come from that control). Similarly, the flexibility of the semantic model means that compliance teams can start to link together information on rules, policies, controls, tests, compliance, geographies, business units and more to facilitate exploratory analyses to more quickly detect potential compliance issues.

Net Result: Compliance management in the world of ever-changing regulation does not look so scary.

What Next?

To get a great, deep dive into how semantic technology is being used in financial services, check out OMG’s March 13^th Financial Services Semantics Conference. (Make sure to use discount code FSSC for a registration discount.) I'll be around to continue this conversation in person.

Semantics in Financial Services on March 13 in NYC

2012-03-02T15:57:54Z

OMG is hosting a one-day conference on semantics in financial services on March 13. We'll be participating to share some of our thoughts and experiences about why semantic technologies (in general) and Semantic Web technologies (specifically) are well-suited to address many of the challenges facing the financial services industry.

We've seen a marked uptick in interest in using semantics to tackle data challenges in financial companies in the past 12 months, and this conference is another data point in the area. Speakers at the event are from Citi, Bank of America, Wells Fargo, and HSBC, along with the US Treasury and the US Department of Defense. The topics we'll be discussing at the conference are fundamental to the industry and include:

The increasing role of Chief Data Officers and their need for semantics
The interplay between semantics and Big Data (a la the thoughts expressed here and here)
The use of semantic models to enable business interoperability between multiple financial services organizations

The conference will also focus on use cases and case studies for semantics in financial services. We'll be talking about using semantics for compliance information management and competitive/customer intelligence (more on those topics in future posts), and there will also be talks about trade lifecycle management, vocabulary management, business reporting,

If the topic interests you and you'll be in the NYC area in a couple of weeks, consider registering. You can use promotion code FSSC for a 30% discount when registering.

WhySQL? Evernote’s Boring Old Reliable Architecture

2012-02-28T17:19:43Z

In my last blog post I argued that Semantic Web databases have the flexibility inherent in NoSQL systems plus the transactional semantics of a relational database systems, and I argued this was a major reason for their growing adoption by enterprises.

Hours later, Evernote’s blog had a post called “WhySQL?” outlining why they didn’t go with a NoSQL system.

For those of you who don’t know, Evernote is a hot web startup that’s been growing like crazy with 8-figure annual revenue in 2011. However, unlike (seemingly) every other web startup in the world they are using a boring old SQL relational database (in their particular case it was MySQL ~~PostgreSQL~~). Not only that, but they’re not even using cloud hosting! What is this, 1999, you ask?

I thought it was a fascinating case of a high traffic web company going with traditional technology for exactly the reason I called out in my last piece: relational databases are reliable and predictable. You know what you get with their transactional guarantees. In the article, Dave says (emphasis mine):

Each of these coarse-grained API calls is implemented through single SQL transaction, which ensures that a client can completely trust any reply given by the server. The ACID-compliant database ensures…[Atomicity, Consistency, Durability]…

It gets even more interesting. Evernote holds tons of data, much of it multimedia. The data is not very structured. It serves (what I would guess) is a large amount of web traffic. Wouldn’t this be a perfect place to employ a hot, whizbang NoSQL database that offers greater performance?

In fact, Dave even outlines why NoSQL databases have such a great appeal:

The ACID benefits of a transactional database make it very hard to scale out a data set beyond the confines of a single server. Database clustering and multi-master replication are scary dark arts, and key-value data stores provide a much simpler approach to scale a single storage pool out across commodity boxes.

Right before saying that they can avoid it altogether by using a clever partitioning scheme (emphasis mine):

Fortunately, this is a problem that Evernote doesn’t currently need to solve. Even though we have nearly a billion Notes and almost 2 billion Resource files within our servers, these aren’t actually a single big data set. They’re cleanly partitioned into 20 million separate data sets, one per user.

That is, Dave & Evernote would rather deal with managing 20 million separate data sets than go with, what he admits, is a “much simpler approach to scale” (the NoSQL way) because SQL systems have a stronger transactional guarantee.

This is exactly what I was getting at in the previous post. The advantages of NoSQL systems to date are wonderful, but the lack of strong transactional guarantees make them impossible for enterprises to use for storing mission critical information.

Full disclosure: I’m a huge fan of Evernote, and am an Evernote Premium subscriber.

What Happened to NoSQL for the Enterprise?

2012-02-27T09:10:29Z

It’s no big secret that people have found better substitutes for the traditional relational (SQL) database for all kinds of use cases. My absolute favorite public example of this—just based on number of technologies involved—is Instagram’s infrastructure, which uses PostgreSQL on the backend, but also employs Redis on the front-end.

Anyone who’s trying to build a scalable website today relies heavily on various NoSQL databases, such as MongoDB, Redis, Riak, Cassandra, and Amazon’s Dynamo, to name just a few of the most popular ones.

However, enterprise penetration has been limited. I want to talk about that.

The Benefits of NoSQL: Scale & Performance

The reason that the usage of NoSQL databases has exploded on the web is that they execute some operations blazingly fast, such as atomic document lookups without joins. Furthermore, many NoSQL databases attack the Big Data problem head-on by coming with out-of-the-box support for distributing a database across a cluster of machines (which is very tricky to accomplish with traditional relational databases).

For example, a website like Pinterest (yes, the obligatory Pinterest mention since it seems illegal not to mention it these days), serving 10,000,000 visits a day, with a catalog of data growing exponentially, simply cannot be successful on a traditional, relational back-end. They need layers of caching and persistence to ensure a reliably interactive user experience.

This kind of scale is very different than what you experience in the enterprise, where you get fewer users with different speed expectations.

The Benefits of NoSQL: Flexibility

Aside from performance & scalability, the other major advantage of NoSQL systems is data flexibility.

SQL systems require that you create a schema before doing anything else. Want to build an application? First, build your model. Then start coding. Need to change your model? Good luck, since you have to change every single thing that ever might have depended on your first model.

NoSQL systems turn this on its head. When working with MongoDB, for example, you can start coding your app, storing things in the database as you learn you need to.

Lots of changes are so much easier. If you need a property to be multi-assigned, just do it! You don’t have to worry about creating entire link tables and adding joins and redoing your business logic all over the place just to make this change work.

No Enterprise for NoSQL

Despite these advantages, enterprise penetration of NoSQL databases has been pretty limited to date. Some technical reasons include:

Poor support for ACID transactions.
Loose guarantees of data consistency across a grid.
Limited support for aggregations.
Limited support for joins.

Basically, many of the things that are needed to maintain consistency of mission critical data do not hold true for the NoSQL databases.

Said another way, if you’re in IT and you’re trying to build a system in support of a mission-critical application, you have been trained to rely on these types of guarantees. It’s mentally unsettling to think about different, softer software guarantees such as “eventually consistency” of some NoSQL databases.

So you have to think about when you don’t need the rock solid ACID transactions of the relational world. And without the pressure of user scale as on the web there is much less motivation to actually go through this exercise.

This leaves enterprises going with newer SQL technologies like Vertica or Attivio that scale very well and are less confusing than NoSQL systems. Or, if they’re really adventurous, using Hadoop for a specific Big Data problem.

Semantic Web Databases: Flexible NoSQL for the Enterprise

One kind of NoSQL system that has been seeing penetration in the enterprise is Semantic Web databases. They don’t offer the same kind of performance and scale that the web-based NoSQL variety does, but instead provide much more flexibility than traditional relational systems while maintaining security & transactional integrity.

Getting back to our IT guy analogy. If you’re deciding between a relational database and a Semantic Web database, it no longer has to be about rock solid data integrity and transactional guarantees, because both systems provide them. It becomes about flexibility and tooling compatibility, which are easier things to wrap your head around.

For example, if you’re dealing with lots of unstructured data, then use a Semantic Web database. If you’re dealing with a schema that you expect to change over time (to incorporate new information types or sources), then use a Semantic Web database.

Thus you can start having a reasonable conversation when you limit the number of differences between the different style systems, instead of being overwhelmed by a class of systems that is somewhat alien. It’s easier to compare Semantic Web databases to relational databases than it is to compare something like Riak to a relational database. I believe this is one reason why Semantic Web databases have made progress in the enterprise where other NoSQL technologies have not.

Polyglot Persistence

So what it comes down to is that for decades we’ve had one standard way to store and query important data, and today there are new choices. As with any choice, there are tradeoffs, and for some applications NoSQL databases, including Semantic Web databases, can enable organizations to get more done in less time and with less hardware than relational databases. The trick is to know when and how to deploy these new tools.

Martin Fowler called this Polyglot Persistence (also the source for this post's image), which I think describes the future fantastically. Our job is made both easier and more difficult by the new world of database technology choices available to us.

Best of Both Worlds: Enterprise Data Management & Familiar Tools

2012-02-20T15:16:25Z

In a recent post at Dataversity, Jay Zaidi asked Should Users Switch from Office Productivity Tools to a Commercial Data Quality Tool? . Jay writes in his intro:

Due to the proliferation of software tools within companies, end users tend to cringe when there is talk of a new tool that they should use for data analysis, data validation and data transformation. They are very comfortable using office productivity tools or other data processing tools that are at their disposal, so the initial reaction to change is negative. Such tools are relatively easy to use and are familiar to them, since most users have had years of hands-on experience with them.

To paraphrase, why should I learn a new, unfamiliar tool when I’ve gotten by OK with Excel all these years? After all, as my colleague Rob is fond of saying, solving a problem with Excel requires only two steps:

Open Excel
Type stuff in

And the stuff can be anything you want it to be. It doesn’t need to fit into an existing data model or conform to a schema or stay the same tomorrow as it was today. It can just be whatever you need to do your job. That’s why Excel ends up holding all sorts of business-critical data: it gets the job done without requiring any thinking about what tool, what web page, or what database you need to use.

Jay then does a good job of outlining ten reasons to use a dedicated tool for data analysis, quality, and validation. Broadly speaking, Jay promotes:

The ability to work with data from heterogeneous formats, whether its tabular data, XML data, or relational data.
Out-of-the-box capabilities of dedicated tools for analytics, transformation, reporting, and data reconciliation. These are capabilities that come “for free” from nearly any information management or BI tool that you get only to a limited extent from Excel for a single spreadsheet and have to work pretty hard to get in Excel once your data starts coming from multiple sources.
The benefits of an underlying repository, including the ability to perform historical analysis, to store data quality & data validation rules outside of code, performance and scalability, governance and automation.

These benefits are all very real and very valuable. But on their own they don’t tell the whole story; on their own, they leave us in an unappealing position of evaluating trade-offs. Do we continue to use Excel as we see fit while muddling through our data integration and validation needs and foregoing historical analysis, data governance, and more sophisticated analysis? Do we switch to a dedicated tool to realize all the benefits that Jay speaks of but forgo the flexibility of using Excel and other office productivity tools to simply and quickly collect and update data by just typing stuff in? Do we try to bridge the two worlds and copy data between Excel and a dedicated data management tool, or constrain ourselves to only certain spreadsheet layouts, or treat spreadsheets as simple data extracts rather than as dynamic, interactive, living documents?

In the face of these tradeoffs, many people still choose to use Excel. This gives rise to shadow data environments, in which key business information exists outside of any governed system. Shadow data is extremely useful for its owner, but it can’t easily be shared with colleagues or reused for other purposes, and it’s also a huge risk to the business as it’s not backed up, secured, validated, or harmonized with other sources of data.

At Cambridge Semantics, we think you can have your cake and eat it too. This shadow data problem was one of the original motivating factors when we built Anzo. It’s the reason that we leveraged the flexibility of Semantic Web technologies to build the Anzo for Microsoft Excel plug-in. This plug-in lets you continue to use Excel and all of its flexibility to collect and share data, while still leveraging the rest of the Anzo software suite to derive all of the data management, integration and analysis benefits that Jay points out.

Without using Semantic Web technologies as the foundation of Anzo, I suspect we would have had a great deal of trouble flexibly accommodating people’s normal spreadsheet usage/behavior (that goes way beyond simple, static tabular CSV-style spreadsheets). We wouldn’t have been able to handle changing and evolving data models while minimizing any data preparation needed to integrate additional kinds of data. In short, Semantic Web technologies are the secret sauce that has allowed us to build an information management platform suite of tools that brings users the benefits of dedicated data analysis tools without losing the ability to use familiar & efficient office productivity tools. (Of course, this secret sauce brings other capabilities to the picture, like the ability to operationalize insights and to handle unstructured text documents; but those are topics for another day…)

It's All About the Data Model

2012-02-06T09:13:33Z

A data model is a way of representing data along with the logical operations that can be performed on that data. It is the foundation on which everything else is built in enterprise applications because at their core they store, access and manipulate data. The data model you choose can make things easy or very difficult. It can add complexity or not. It can require lots of expert setup and maintenance or very little.

Let’s talk about two extremely common data models—that of spreadsheets and that of relational databases. I will later introduce a newer data model—that of RDF, which combines the best of both.

Spreadsheet and Relational Data Models

The spreadsheet has a very simple data model. A single spreadsheet is broken into discrete cells whose values are referenced by a Row and Column index. That’s it! Yes, most of the time the Column has a heading telling you what kind of data it contains, but that is just a suggestion and is not enforced. What can you put into each cell? Anything! In the worst case you end up with a pesky error, but you can always go back and fix it, that is if you are aware of it. Who can enter data into each cell? Anyone! Sure, there are some security measures that you can use, but they are easily circumvented.

So the spreadsheet data model is completely flexible (the good news), but not very manageable because the association of what each cell represents and how it’s to be used is left entirely to the user…and is mostly in the user's head (the bad news). Said another way, you can't be sure of what the data represents and if it is accurate.

The relational data model, in contrast, is very manageable. In fact, that’s its most highly regarded trait! A relational database is composed of a series of pre-defined grids which have very strict rules regarding what is allowed in each cell, under what circumstances it might be modified, and how it might be combined with data residing elsewhere in the database. The key feature of the set of grids in the relational database is that there is a formal logic (relational algebra) that can operate on the columns and rows of the grids. This means that they can be managed with the certainty of understanding of what data is being dealt with at all times. This is the key to data integrity.

The drawback of this enforced structure is that it is very difficult to make even simple structural changes to a relational database. Any change requires experienced database administrators and, worse, might require changes to applications that rely on the database because a lot of the meaning of the data (what the rows represent, for example) is built into each of the applications. Said another way: although the relational database strictly enforces what is stored and how it can be operated on, it has very limited knowledge of what it means or how it is used. That is left to the applications, and gets very complicated as multiple applications share the same data.

A New Data Model in Town

There is a new data model called RDF—the data model of the Semantic Web—which combines the best of both worlds: the flexibility of a spreadsheet and the manageability and data integrity of a relational database. Based on standards set by the World Wide Web Consortium (W3C) to enable data combination on the Web, RDF defines each data cell by the entity it applies to (row) and the attribute it represents (column). Each cell is self-describing and not locked into a grid, in other words the data doesn't have to be "regular". Further, it has formal operations that can be performed on it, much like relational algebra, but clearly at a more atomic level. Unlike a relational database, each cell knows what entity it is describing at all times and through all data operations and it knows about the relationship each entity has to other entities, which in the relational data model have to be done by join statements at run-time.

The RDF data model is also called the Semantic Data Model since the word semantic means "meaning" and the meaning of each cell is attached to it. Each cell can be associated without any application logic to any other data cell representing the same entity, or more dramatically, any data cell or cells representing entities to which that entity is related. For example, it would know automatically that information about a company applies to each of its employees and vice versa. Applications can take advantage of this, without building in the association logic themselves; rather RDF will navigate through the data for them. And since it doesn't rely on data being "regular", new types of data of any shape or form can be added on the fly and quite easily.

So how important is the data model that is used? How important is the foundation for a building?

Big Data... or Right Data?

2012-01-31T16:10:28Z

What matters most, Big Data or Right Data? One look at all the IT headlines these days would suggest that Big Data is the most important data issue today. After all, with lots of computing power and better database storage techniques it is now practical to analyze petabytes of data. However, is that really the most compelling need that end users have? I don’t think so. Instead, I would claim that the issue most end users have is getting together the right data to help them do their jobs better, not analyzing billions of individual transactions.

Databases of the petabyte size mostly represent billions of individual transactions, for example individual telephone calls or ATM transactions. No one would argue against analyzing that data to look for "nuggets" of insights that can only be found at that detailed transaction level. However, that kind of analysis really should be left to the professional analysts. It requires sophisticated models and statistical techniques, and in the wrong hands can lead to all the classic errors of statistical analysis (e.g. correlation is not causation, and 5% of the time, random events will be statistically significant at the 95% level).

What most end users want is the result of these analyses, not the raw data. They want to build decision rules relevant to them based on these insights. Plus, they want to combine the insights with only that part of the raw data that is relevant to them in the context of doing their job. For example, a customer rep dealing directly with a customer needs the full details of that customer's previous interactions with the business that are relevant to the current call. This isn't as much a matter of filtering through petabytes of Big Data as it is of getting the Right Data at the right time. The rep needs data about the customer's interactions from lots of different data sources (purchase histories, support issues, call and email logs, payments, etc.), but he also needs to limit the information he sees to only the information that's contextually appropriate. My colleague, Rob Gonzalez, calls this the horizontal big data problem; it’s the challenge of combining data not only from big centralized databases, but also from small local databases, spreadsheets, documents, and from the Web.

Let's face it, well over half of the relevant data end users need is locked in data sources other than the centralized relational databases. So the key is tapping into these data sources, finding and mapping together commensurate data, and presenting that to the end user is an easy to digest form. Once you’ve accomplished this, you can apply decision rules to help the end user make better operational decisions, and in many cases even automate this for greater efficiency. Isn't this the real end user data issue?

What the Semantic Web and Digital Cameras have in Common

2012-01-25T17:20:54Z

Flexible, Agile, Easy to Use...words on every website for seemingly every database, business intelligence, or operational intelligence product on the market! Customers simply do not take these claims seriously anymore because theyave been promised again and again by vendors who have consistently failed to deliver on these promises.

How then can I describe something that really is a major change in all of those directions -something that is truly revolutionary? I ask because Semantic-driven data based on Semantic Web technology really is a fundamental shift in how to organize data that does deliver greater flexibility and ease of use.

IT old-timers like me remember the initial promises of relational database technology: flexibility and end user accessibility. Even decades ago, we were using those same words! Well, the latter sure never panned out. Even experts have trouble with SQL. As to flexibility, if you need a new view on data, or some new information included, end users are completely reliant on IT (or more likely, do their analysis in Excel, or not at all).

Of course, Business Intelligence products provide a major upgrade in end user accessibility and analytic power. However, they are rarely able to penetrate beyond Analysts, leaving the vast majority of knowledge workers still unable to effectively use all available data resources. And if structural changes in data schemas are required, even the Analysts require significant IT support.

Semantic Web technology is the first major advancement in data technology that Iave seen that may truly enable end users to directly access and manipulate data without requiring significant IT support. I've come up with an analogy to illustrate the difference between current data technologies and Semantic Web technology for data access and analysis. It is like the difference in photography between analog and digital.

Analog photography went through lots of phases of dramatic improvement, becoming a mass-market technology. But...no matter how far it went it was limited in its flexibility. Every picture was pretty much as you took it. Any modification required real experts, with specialist equipment and working in a dark room. With the advent of digital photography we have achieved extreme flexibility. The picture you take is simply the starting point to create the picture you want, and the end users themselves can make the changes with easy to use tools.

Semantic Web technology represents the same dramatic shift from the traditional technologies. It liberates each and every data value (think of them as pixels) regardless of its source to be accessed, associated, combined, or otherwise manipulated as an end user sees fit. Each data value sits independently and is self-describing. Data is liberated from its rigid grid, and yet can still be controlled, managed, and tracked as required by modern IT. It really is the best of both worlds between manageable and scalable relational (grid) databases, and the flexibility and ease of use of spreadsheets.

This sure sounds like a major paradigm shift to me.

Domain-driven Design with Anzo

2012-01-25T17:17:54Z

Domain-driven design (DDD) is an approach to developing software for complex needs by deeply connecting the implementation to an evolving model of the core business concepts.[1] The premise of domain-driven design is the following:

Placing the project's primary focus on the core domain and domain logic

Basing complex designs on a model

Initiating a creative collaboration between technical and domain experts to iteratively cut ever closer to the conceptual heart of the problem.

...from Wikipedia

Our own needs here at Cambridge Semantics have driven us to add capabilities to Anzo to enable domain-driven design. As Ben posted in Our Own Dog Food Tastes Pretty Darn Good, we continue to use Anzo more and more as our system of record for almost all aspects of our company business. To facilitate the development of Java applications, weave developed a simple to use JPA interface to Anzo which operates via standard Spring integration mechanisms. With these interfaces in place, a junior Spring developer can start writing applications connected to our enterprise data immediately using the skills he already has and without knowing anything about RDF, OWL, or SPARQL.

The domain objects are automatically and immediately setup for any ontology that is defined within Anzo. So as soon as anyone imports an ontology or creates one using the ontology editor available in Anzo for Microsoft Excel, client applications immediately have access to a full set of domain objects to interact with their data. The domain object maintains a live connection with the enterprise Anzo database so when your application does a CRUD operation to the object, those modifications affect the underlying data and are viewable across the enterprise.

What does this mean for an enterprise?

A business user can define the domain model with no developer interaction;
Others in the enterprise can enhance the model if necessary and the business owner can be alerted when this modification occurs;
From then on everyone works with the same domain model, so there is no translation code to move the domain model objects between systems. This avoids lots of complexity in individual applications;
Domain rules can be enforced at the enterprise store level;
Existing Java skill sets can be utilized efficiently to create applications that interact with the enterprise store.

From a developer's perspective, it's very straightforward to incorporate this domain-driven design approach into an application, simply by adding a maven artifact dependency. The only configuration that needs to be done is to specify the IP address of an Anzo server within the maven configuration. Once this configuration is done and the maven build is run, the domain model will be pulled in from Anzo and immediately become available for the developer to program against. We use this at Cambridge Semantics with JSF and Spring to write Web applications that interact with our enterprise data.

Anzo's support for domain-driven design gives our users a direct correlation with an enterprise domain model and actual application development, and it lowers the learning curve necessary to start developing semantic applications. If you're interested in learning more, please drop us a note.

Why Semantic Web Software Must Be Easy(er) to Use

2012-01-31T16:11:26Z

Over on my personal blog, I've written a couple of posts that outline two key thoughts on the transformative effects that Semantic Web technologies can have in the enterprise:

Why Semantic Web Technologies: Are We Asking the Wrong Question? -- Semantic Web technologies take enterprise problems that would take too long or too many resources to solve with traditional technologies and make them tractable, so you end up being able to apply technology and automation to far more day-to-day business problems then you could before.
Saving Months, Not Milliseconds: Do More Faster with the Semantic Web -- The flexibility of Semantic Web technologies mean that you can save days, weeks, or months of calendar time compared to approaches that use traditional technologies.

There's a key corrollary of these two observations that you need to keep in mind when building, browsing, or buying Semantic Web software. Semantic Web software must be easy to use.

On the surface, this sounds a bit trite. Surely we should demand that all software be easy to use, right? While ease of use is clearly an important goal in software design in general, I'd argue that it's absolutely crucial to successfully realizing the value from Semantic Web software. Here's why:

Software that is hard to use has two main effects. First, it frustrates and annoys the user. Users won't choose to use frustrating software for any more tasks than they absolutely have to. Second, hard to use software limits the audience who can benefit from it. This is particularly true for software that is hard to use because it is complicated. If a software application lacks an intuitive user experience, demands that users be familiar with URIs, RDF, or OWL, or requires knowledge of (or learning) an analytics or data access language like MDX or SQL, then it is immediately limiting its use to IT professionals, a small fraction of a company's employees. If a business manager wants to accomplish something with this hard-to-use software, they've no other option but to schedule time with IT, define their requirements, and wait for the results. And all of this takes significant calendar time; enough time, in fact, that it marginalizes the calendar time benefits promised by the flexibility of Semantic Web technologies.

In short, if Semantic Web software is hard to use, then many of the benefits of using these technologies in the first place are immediately lost.

Conversely, if Semantic Web software is easy to use, on the other hand, then the benefits of Semantic Web technologies' flexibility are brought directly to the end user, the business user. The business manager can bring together new data sets for analysis today, rather than a week for now. An analyst can setup triggers and alerts to monitor for key business indicators today, rather than waiting 3 months. A senior scientist can begin looking for correlations within ad-hoc sets of data today, rather than next year.

We try our best to take this to heart at Cambridge Semantics. We call it reach, and it forms one of our core guiding principles as we develop Anzo. Simply stated, reach means that all capabilities of the software should be usable by as broad a universe of users as possible. This is one of the reasons that we built one of our very first applications around collaborating with Microsoft Excel. But we also bring reach to other areas, from allowing end users to easily connect database data, to driving analytics via simple Excel formulas, to giving users the ability to define their own rules and alerts on the fly, to providing Anzo on the Web as an easy to use, self-service dashboard application for interacting with Semantic Web data.

Our Own Dog Food Tastes Pretty Darn Good

2012-01-25T17:28:30Z

This summer our company began using Anzo internally to run various aspects of the business. In particular, we ditched Salesforce.com for a home grown, pure Anzo-based CRM solution. But as a technical founder of Cambridge Semantics, I understand Anzo Enterprise Server top to bottom, down to the lowliest RDF statement. So I was skeptical that I would personally appreciate the advantages of our end-user products applied to trivial spreadsheet applications, and I was slow to adopt our "Dogfood CRM." However, in my current role, I do spend quite a bit of time managing business partner relationships and customer solution implementations so I was excited to see how Dogfood CRM would help my day-to-day activities. But was I too close to everything to fully realize the benefits espoused by those in the company already excitedly chowing down? Let's see.

Tracking billable hours without Dogfood was admittedly a bit of a pill. Entering the hours into a spreadsheet was easy enough. But making sure I used the right customer and project names was tricky since the drop-downs in my local spreadsheet did not always contain the latest options. The monthly "check-in" of my hours was the real pain. I would need to post my copy of the hours spreadsheet onto the file share, but check (probably not as carefuly as I should have done) that I wasn't clobbering any changes that Lee had made when billing customers the previous month. With dogfood, the process was almost cathartic in comparison. I could open any copy of the spreadsheet, login to Anzo, and enter just the hours I was worrying about at the moment and I was done. The drop-downs were all populated from live master data in Anzo so I never had to worry about using the wrong name for something. On top of that, there was no monthly aggregation required for me or Lee.

Tracking partner relationships before Dogfood was virtually impossible. Salesforce offered a tedious UI for adding interaction events that sapped all motivation and excitement from an engineer just getting his feet wet with the business aspects of a software startup. Left without any solution, to see what my last interaction with a partner I had to go back through emails and try to piece it together. With Dogfood CRM, I actually enjoy these management tasks. The data entry is so easy and the dashboards and views so rich, that I get the warm fuzzies talked about by my AP Computer Science teacher. I open our customer/partner DCT workbook, pick the company from a drop down, and Anzo auto-fills all the information about the partner into various tabs in the workbook, including "Comments" and "Deals" sections where I record my interactions. I simply type in my name, date and the summary of the interaction, and click "Submit". I can then use a customized Anzo on the Web report to view my most recent interactions with all partners or a particular partner. At a glance, I can see which partners I ought to follow up with at any given time. Finally, if I need to add a new partner, I can fill in the required information in the very same workbook, click "Submit" and the new partner is added to the system.

The combination of Anzo Data Collection Templates and customized views, I have streamlined and powerful method to track all of my customer and partner work. In the end, this Dogfood tastes pretty darn good.

Semantics - The Promise, Finally Fulfilled

2012-04-17T15:37:37Z

In the 20+ years I worked as a high tech management consultant around the world, I saw the businesses go through several stages of evolution. ERP promised huge benefits of never missing a customer order while concurrently have no cash-draining excess inventory. Unfortunately the big economic benefits were often not realized. Companies were required to mold their businesses around how the software tools worked, vs. what was optimal for them to compete. Aerospace companies were told they were implementing "best practices solutions" from the vendor's latest sales, but were often not informed that said sale was to a business operating in a completely different industry with separate challenges.

The larger companies often got around this by tailoring the vendor's application to fit their needs exactly. However this was at significant cost both to implement and maintain. Millions were spent on system integration firms and consultants to get the system to work just right, and it was right—for the time. As soon as the something in the business changed, it wasn't "just right" any longer. Things did change; they changed all the time. Mergers, acquisitions, new markets, new rules, updated regulatory requirements, compliance reporting requirements, government filings, etc., etc. Ultimately, businesses were at the mercy of their service provider or had to hire a significant IT support staff. Smaller to mid-sized organizations changed their business processes and practices to take advantage of their ERP solutions as best they could. By not customizing the application, they were able to implement the modules much faster, however at the sacrifice of usability. As a result, both large and smaller businesses were left with brittle, inflexible solutions that required much manual work-around with other tools like Excel spreadsheets and Access databases.

Finally with the advent of semantic technologies combined with dramatic improvements in hardware technology (networking, solid state drives, RAM, etc) and real end-user tools, normal business people are able to do things that previously would have taken too long and cost too much. Indeed, semantics is the technology for enabling everyday workers to understand the meaning of data that is accessible to them across the enterprise. People can now quickly and easily find and process information into reports and actions, regardless of where it is located and of the form that it is in. For example, a finance manager can now look across various departments and compare actual expenditures to what was budgeted by accessing spreadsheets, web-based dashboards, or PowerPoint presentations. Supply chain people can now combine information from their ERP system with those of other business partners like Contract Manufacturing firms and 3PL providers easily even when those partners are on totally different systems. Sales managers can quickly integrate SAP, Salesforce.com, an individual spreadsheets to understand how each of their reps are doing across geographies, by product, and major customer to develop more accurate forecasts for financial planning and demand forecasting.

In summary, semantics is empowering knowledge workers to improve their businesses, and do so without extensive and costly IT support. Is it the answer for everything? No, but it does a lot for a little and often generates great returns.