In this post, we show how you can access the data in a space as RDF and linked data. It should primarily be of interest to semweb geeks who are interested in playing around with Linked data.

To access the LinkedData URI’s in a workspace,
1. Open a workspace with some data and go to the explore tab (or you could just use this sample space)
3. By default, the anchor links next to entities in the table take you to the webpage from which the data was gathered.
4. To get the URI’s for these entities, you need to click on the WoD command (in the top-left command bar along with Save, Share etc).
5. In this mode, the anchor links next to each entity give you the RDF for this entity. Just click on it to get the rdf file.
6. The URI’s are all deferenceable – ie if you put it in the browser address bar, you should get its data inturn.
7. You can also browse these links in a linked-data browser. Here is what you should see in a linked-data browser. eg (Filter Squad)

You could watch the video below to see how this works

As a disclaimer, I need to mention that the linked-data that Semgel is publishing now is fairly rudimentary. There are several things that can be done to improve this including support for multiple formats, labels for all URI’s, sameas uris to other datasets, ontology URI’s which are independent of a specific workspace and more.

Edd Dumbill has published a indepth survey of data markets on the Radar blog. One of the important things that data markets can do is to provide mechanisms for integrating the data from multiple publishers. In this post, I try to assess how various sites fare in this regard. I am also take a wider view and look at these websites as potential portals to data on the web.

Semgel's objective is to empower the average spreadsheet user to gather and analyze data from the widest possible set of data sources on the web. One of the keys to realizing this objective will be the emergence of quality data portals.

Data portals can broadly be defined as sites that enable users to discover and access data from diverse sources in a unified manner. The key word for me is ‘unified’. To assess the degree to which different sites provide unified access, it would be useful to organize these sites along two dimensions.

Where is the data stored?

• Centralized : data is stored in and fetched from a local store. Access is faster and can potentially allow querying across sources.
• Federated : data is fetched from remote data source using their Web API's when requested by user. Access is slower querying across sources is often not feasible.

How is the data queried?

• Integrated : users can directly query data across ALL the datasets. Requires schema and data to be mapped.
• Contained : users can query data across individual datasets only.

The technology to provide unified access for federated and centralized data are significantly different. The holy grail for a web data portal would be to have website that offered comprehensive and integrated query capability across all data, centralized and federated.

Historical and slow changing data like people profiles would be centralized (ie stored locally) while fast changing data like twitter feeds, stock prices etc. would be federated (ie fetched on demand using web apis).

Achieving unification of this kind on a global scale clearly belongs to the realms of fantasy. However, it does help orient us when we seek to assess the various data portals in this regard.

Google Freebase : This was perhaps one product that did want to go after the holy grail. The plan was to build a global database seeded by data from wikipedia. It adopted a wiki approach to editing data. Anyone could add to their own datasets as well. It offers truly integrated data access with rich xpath style data queries. They also had plans to combine this data with data fetched via API's. However, its not clear as to where they are headed post their Google acquisition.

Wolfram Alpha : This is definitely the most sophisticated offering. The focus is not merely to be a data repository, but instead to compute and provide answers based on the data. They are however planning to offer api access to their expert curated data store. Further, the emphasis is less on entity relationships (as in Freebase), but perhaps more on numerical attributes of entities (not surprising given their computational background).

Yahoo Data Tables : This product was one of the more visionary offerings from the Yahoo stable. It offered unified access to a large collection of web apis using a sql-like query language (yql). However, they still did not seek to directly map the schema's across the various sources. While the product's future is uncertain, it could serve as a interesting model for other data portals.

Factual : This product, like freebase, adopted the wiki approach to building a data repository. While data across tables is indeed integrated, it is only exposed in a limited manner. Their current focus is on providing access to individual datatables through a API.

Kasabi : This is innovative offering using a fundamentally different technology stack. The objective is to enable users to upload highly structured data (RDF) and offer rich querying capabilities (sparql). The choice of linked data technologies allows them to easily support integrated query capabilities. However, they are still getting started and we need to see how they evolve.

Azure Data Market : This product allows users to upload datasets into a relational schema (instead of flat tables) and allow full SQL querying. But querying is contained to individual datasets. They also support remote data sources, but do provide some kind of unified access by mapping API's to the Odata protocol. The objective here is to enable cloud computing and not to create a integrated data portal.

Data Market : This offering is similar to Swivel (which has closed down) and Socrata (which has re-positioned itself as a publishing tool instead of as a data portal). The focus is on independent data sets instead of creating a integrated data pool. The data is however stored in a structured form allowing for basic visualization and computation. I rather like the sharp focus and clean interface!

Infochimps : The company has repositioned itself as a bigdata computation platform and its primary focus in not on being a data portal. It does offer acess to API's (similar to azure data market) as well as hosted datasets. However the hosted datasets are raw files made available for download ' you cannot query these datasets.

And finally, Socrata and Mashery need to be mentioned in this context. Although they not position themselves as data portals, they do offer a service to data publishers and are thus part of the ecosystem. Conceptually, they have a lot in common with the products mentioned above.

Data Portals have a long way to go. There are big technical challenges in terms of scalability as well issues in creating a viable business model. But big challenges often equate to big opportunities!

When the geek crowd discovers Semgel, they are perplexed. They see a application that takes data from Crunchbase, transforms it and presents it in a bunch of pretty tables and charts. What’s the big deal?

Well, here’s the thing. Crunchbase is simply a test case. What we are demonstrating is how we can fundamentally simplify the process of gathering and analyzing data from the web. This post describes the core motivation for developing Semgel.

Over the years, we have seen a web of data emerge to complement the web of pages. The web today offers hundreds of vast data repositories through simple data API’s. This data is hugely valuable to a vast array of knowledge activity.

Now, for the vast majority of knowledge workers, the spreadsheet is the tool of choice for analyzing data of any kind. As such, why is it that spreadsheets are rarely used to directly access and analyze data from the vast number of data API’s out there?

Broadly speaking, there are two kinds of datasets. Simple, flat datasets, typically formatted as csv’s, contain one type of item along with related data. Complex, nested datasets on the other hand, contain multiple types of interconnected data (typically formatted as xml/json). Most API’s provide the later.

Fig: Simple/Flat datasets

Fig: Complex/Nested datasets

While spreadsheets are perfectly adequate for dealing with simple datasets, they don-t do so well when we need to collate and analyze multiple complex datasets. Spreadsheets deal with complex datasets by denormalizing them. This significantly limits what you can do with the data. Some of the issues are

• It creates duplicate data making it difficult to comprehend or edit the raw data.
• It is difficult to get a consolidated/deduplicated list of a particular kind of item in the dataset
• Aggregations (sum’s, avg’s etc) would be incorrect due to the duplication of value cells

To overcome these issues, Semgel’s under pins the traditional spreadsheet model with a semantic database model, a model with a implicit understanding of data types and relationships.

• When users add datasets to Semgel, the app automatically seggregates different types of data into a collection of tables while retaining the links between the data.
• Users can seamlessly navigate between different types of data. They can interactively sort, filter, join, aggregate, or visualize the different types of data.
• Users can perform a set of data operations and save the result as a view. These views can be reused with different data sets.

Fig: Spreadsheets denormalize complex data

Fig: Semgel organizes complex data into a collection of interconnected tables

Now, with this model, if for example, you want a list of investors who have invested in ‘consumer’ companies who have made ‘a’ round invesments, you are just a couple of clicks away!

Semgel however requires a small adaptor for each API it supports. This is essential to ensure that Semgel is able to clearly identify the different types and relationships in the datasets offered by the API. Over time, we expect standards to emerge so as to remove the need for a adaptor. This is analogous to feedreaders using RSS to consume data from news sites without requiring site-specific adaptors.

You can also learn more about Semgel by going here or you could just signup to try it out now.

The standard Crunchbase page for a investor provides a list of their investments with links to the investee companies. However, if you want to get the big picture about the geography, stage or size focus of the investor, you will need to consolidate and analyze the data about all the investee companies.

With Semgel, you can do this entire exercise in just a couple of minutes. Just search for the investor and click a button ' Semgel will pull all the data together and present you with a set of standard views that provide the answers you are looking for.

For this post, let’s look at the portfolio of Union Square Ventures. After analyzing all their investments listed in Crunchbase (in over 48 companies), here is what you will get.

A consolidated table of all investee companies with key details.

A table of co-investors for Union Square Ventures sorted by the number of deals they have co-invested in.

Piecharts showing distribution of investments by geography (US state) and round.

Barchart showing distribution of investments over time.

Network visualization of co-investors. The investors are in blue and the investees are in orange. Only the top co-investors are included in the chart.

While you would need to do a lot more research into a investor to make a effective pitch, the facts above should clearly be your starting point for the background research.

You can also learn more about Semgel by going here or you could just signup to try it out now.

• Emerging phase: when the first companies in this segment are founded and perhaps receive small amounts seed funding.
• Evolving phase : when the segment is fairly well defined with some companies beginning to receive a or b round fundings.
• Mature phase : when the segment has well established leaders, is receiving late stage fundings and has seen some consolidation with companies being acquired or shutting down.

If you are enterpreneur investigating a market entry, looking to raise money or exploring a exit, you would want a have this big picture in mind so that you could tailor your narrative accordingly. And if you are investor researching a opportunity, the big picture is critical for making the right call.

In this post, we will look at News Aggregators' this segment has been around for a while and has seen some consolidation.

You begin by gathering data about the companies that belong to the segment. You could do this using keyword searches or by searching for and adding specific companies. Once you have done this, Semgel automatically organizes all this data and creates a set of standard views or reports.

One of these views is a table of all the key milestones in the journey of the startup. You can filter/sort this table to look at earliest foundings or fundings of companies.

Further, to get a better sense of the time interval between events, you can visualize these events as a timeline. You should now be able understand how the segment emerged, evolved and possible matured ' starting with foundings, going on to seeing fundings and perhaps ending with some acquisitions.

However, if there are a large numbers of events, it would still be difficult to quickly get the big picture. What would be ideal is to simple see a count of each type of event distributed across time. This can be created by exporting the table as a csv file and visualizing it as a pivot table.

If you have chosen a segment that is reaching maturity, you should clearly see three humps ' foundings, fundings and acquisitions which align with the three phases for this segment.

As an entrepreneur or investor you would gain but not only studying your own segment but looking at the evolution of other segments from which you can get a sense of the time taken for a segment to take a definitive shape.

You can also learn more about Semgel by going here or just signup to try it out now.

The vast majority of these companies are startups. And the most valuable data that is available about them is their journey so far ' their founding, fundings and exit. The chart below shows these milestones along with their associated data.

There are thousands of companies in crunchbase which have the above data

If you are an entrepreneur or a investor, you can analyze this data to gain insights into

• the evolution of a segment : when did we begin to see the first companies in this segment? what was the most active period of fundings? have there already been successful exits?
• the investments in a segment : what is the total funding for the segment, what has been the avg size of funding for a given round? who are the top investors in this segment?
• investor portfolios : distribution of a investor’s portfolio by category/geography/ stage etc. Success rate? Co-investors?
• overall investment trends : what the distribution of investments by sector or geography and stage in the last quarter? most active investors? Biggest deals?

These insights play a critical role in making decisions about market entry, fund raising and more. And with Semgel, you could get these very insights in a matter of minutes!

You can learn more about Semgel here or you can signup to try it out now. You can also subscribe to the RSS feed for this blog or follow us on twitter.

The Crunchbase website allows users to search for and quickly look up detailed profile pages for individual companies. This is perfectly adequate for the casual user who just wants to know a bit more about a company.

However, if you happen to be a entrepreneur looking to raise money or a investor scouting for investment opportunities, you typically need to gather data about a bunch of related companies and analyze the same. Semgel was designed to meet this very need.

Semgel is a web application that enables you to quickly gather and analyze data about Crunchbase. With a just a few clicks, Semgel can create a seamless database of just the companies you are interested in. You can then go on to analyze this database in a intuitive manner.

What’s more, Semgel automatically creates a set of standard views such as consolidated investment tables, activity in the sector, key people etc which would be immediately useful for you.

In future blog posts, we will show how you can use Semgel to gain insights, discover opportunities and make informed decisions faster. If you are a enterpreneur, investor or marketeer in the technology sector, we would encourage you to follow this blog via our rss feed or twitter.

You can also learn more about Semgel by going here or just signup to try it out now.