July 23rd, 2007
A bumper crop of new mashup platforms
While application developers tend to roll their eyes at the concept of end-user mashups, they remain one of the more promising new trends in software development this year. And while it’s certainly true it’s early days yet for mashups, the tools that enable them remaining rather limited, seems to be changing as I regularly come across compelling new mashup platforms as well as upgrades to existing ones that show what will be possible soon. And for now, as evidenced recently in the McKinsey Web 2.0 in business survey where 21% of organizations globally said they are using or planning to use mashups, there appears to be considerable demand for mashups at the enterprise level even though the majority of existing offerings are primarily aimed at the consumer space. Is this disconnect resolving with the current crop of offerings? Let’s take a look.
In today’s mashup world, the apparent business potential of highly accessible and easy-to-use mashup creation tools like Yahoo! Pipes and Microsoft’s PopFly is still undermined by their apparent lack of readiness for the enterprise. Mashups could theoretically allow business users to move — when appropriate — from their current so-called “end-user development tools” such as Microsoft Excel that are highly isolated and poorly integrated to much more deeply integrated models that are more Web-based and hence more open, collaborative, reusable, shareable, and in general make better use of existing sources of content and functionality. Remember, business workers still spend a significant amount of time manually integrating together the data in their ever increasing number of business applications. Tools that could let thousands of workers solve their situational software integration problems on the spot themselves, instead of waiting (sometimes forever) for IT to provide a solution, is indeed a potent vision.
So what’s typically missing from today’s mashup platforms to make them both useful and desirable in the enterprise? While no one knows for sure, since mashups are just starting to be considered seriously in many organizations, it generally boils down to 1) deep access to existing enterprise services and data/content repositories, 2) SaaS-style Web-based mashup assembly and use, 3) assembly models that are truly end-user friendly with very little training required, 4) a credible management and maintenance story for IT departments that must support a flood of public end-user built and integrated apps, and last but certainly not least, 5) mashup products that address important questions about mashups and enterprise security. None of these are particularly easy to solve, which is most likely why mashups haven’t been more prevalent before now.
This latter issue of security — in terms of reliably securing applications that are created largely out of other services and applications — can’t be understated and will likely determine whether an mashup platform can even be considered for adoption in a given organization. This is particularly crucial since the Global SOA, the vast landscape of open functionality and content on the Web, now provides a truly massive yet rather security-challenged set of source of material for enterprise mashups. The question here is whether Web apps that are assembled by users — and not developers or security experts — and that combine capabilities from a wide variety of sources including the open Web can ever be made safe enough for most businesses? That’s an important open question and one that few of the mashup platforms listed below spend much time addressing.
Are mashups really a major new development model? Read a detailed discussion.
The answers to these questions will inevitably shake out as the existing mashups products get applied to real business problems and the industry collectively learns what capabilities and approaches are needed for them to be successful. And I don’t expect it will be a one size-fits-all either; mashups can be approached many different ways, from the pure service mashup models of RSSBus and Kapow’s RoboMaker to the innovative yet very end-user friendly wiki model that IBM’s QEDWiki takes.
I’ve been been tracking many of these new or evolving mashup platforms and thought I’d compile my take of the leading players in the mashup space today, particularly given the number of new or significantly upgraded products in the last few months. To make the cut, all the products listed below had to allow live integration of functionality or content (data) over a network, provide an easy-to-use development model that is theoretically accessible by end-users, be available in at least beta form, and either consume and/or produce Web-based applications and services. Using this refined selection model, you’ll see this list looks a bit different from last year’s round-up of mashup platforms. Yet despite the removal of a few products, the list is bigger than ever with 17 products currently available that offer credible mashup assembly capabilities today.
Seventeen Mashup Platforms
Apatar is a mashup product designed to help “users and mashup developers to join their on-premise data sources with the Web and Office 2.0 applications without coding.” While Apatar lacks a Web-based IDE and uses a native downloadable application to help users deliver on their on-demand data integration scenarios, it’s a good complement to existing mashup tools such as QEDWiki (described below) since it’s a primarily data-level mashup tool. Apatar’s core value proposition centers around connecting the Global SOA to the enterprise SOA. It also has connectors to enterprise databases and a few popular CRM applications such as Salesforce and SugarCRM, thereby offering the potential via self-service to eliminate the “cut-and-paste hell” that too many workers are caught in today. Apatar is open source and runs on Windows and Linux.
BEA’s Aqualogic Pages product is one of the few full-blown end-user mashup tools currently available as a commercial, supported product from a leading software company. AquaLogic Pages is an application builder that gives end-users a “palette of drag-and-drop components that help participants create simple Web pages, blogs, and wikis, as well as more powerful data-driven Web applications.” BEA has also picked up on the consumer mashup attitude and states that creating Web applications is now as “easy as creating a document” and the product’s main page says that Pages can be used by IT to “distribute the load of Web building to participants of all kind”, more clear communication that BEA appears to “get” mashups. Product Focus: The Long Tail of Software Demand that mashups can theoretically tap. Along with a data integration builder, AquaLogic Pages provides a robust set of tools to build and manage mashups made from structured and unstructured data using the now-familiar “spaces” model. BEA AquaLogic Pages is a commercial product and is available on the Windows platform only and works best with a BEA infrastructure.
Dapper has been around for a little while now and made last year’s list and the product continues to improve. It’s 100% Web-based both for data mashup creation as well as use. The attractive and capable Web-based mashup wizard, called Dapp Factory, can turn just about any type of content located anywhere into formats such as XML, RSS, Google Gadget, Netvibes Module, iCalendar, and more. Dapper also has a good sized mashup community and you can see many of the Dapps, their name for a data mashup instance, that users have created and shared publicly in their mashup gallery. Note that Dapper is primarily a data-swizzling service, though they do provide a “face” to the data pulled from remote sites and services by delivery via visual widgets such as Google’s Gadgets and Netvibe’s Modules. For now, Dapper is a free and entirely Web-based service though they intend on providing revenue generators in the near future.
Applibase’s DataMashups site is one of the more complete and sophisticated Web-based mashup builders available today. What DataMashups sometimes lacks in visual style it more than provides in substance with some of the most compelling examples of real, usable business mashup that I’ve seen. DataMashups can be externally or internally hosted and you can try out their Web-based mashup builder online to build your own situational applications. DataMashups provides end-to-end tools to build robust Web apps complete with user interfaces and their mashup examples gallery shows what’s possible. Their best example app, built entirely using the Web-based tool is the feature rich OnTrack Issue Tracker sporting fairly sophisticated Google integration, charting, and relationship database support on the server-side. This entire app was built using mashup technology. DataMashups supports external 3rd party widgets and the back-end components run in Java on Windows and Linux. Relational database access from a mashup is limited to mySql today though theoretically most databases supported by the Java platform will work.
Another entry in the data mashup category is Denodo’s data mashup product suite. Featuring the ability to pull data from a range of source including the Web, the local intranet, electronic documents, unstructured data, databases, data warehouses, XML repositories, SAP, Seibel, and even e-mail, Denodo’s has assembled a formidable and sophisticated product that will do live, real-time integration of back-end business data into new, seamless services that can then be consumed by other user interface mashup builders. Many end-user data integration scenarios don’t require a visual mashup and just need data to flow seamlessly under the covers and Denodo delivers this with more enterprise context than most, particularly around connectors to enterprise systems. Data mashups created with Denodo can in turn be shared as Web services, or even via JDBC or Java APIs. Denodo’s attempt to simplify mashups by encouraging a unified data model across input services is going to require that extensive upfront work be done by IT before it can be used by sophisticated users to build the integrations they need. Denodo’s mashup products are commercial applications that run on most platforms that support Java, though certain features only run on Windows.
Extensio is a relatively new entrant in the mashup space and their product takes advantages of the latest developments in Web gadgets and widgets. Extensio provides a “end-to-end Information Delivery Platform that enables enterprises to extract data from multiple data sources, assemble them as information services and deliver them on multiple user front-ends.” Like Denodo, Extensio must be installed and configured by experts, almost certainly the IT department, before it can be used by end-users. The core of the Extensio product is the Symphony SOA integrator with various optional extentions that project the mashed-up data and user experience out to the Web, desktop applications, or mobile devices. This creates a picture for enterprise integration that is even bigger than Web mashups and can form the basis of a ambitious enterprise-wide integration strategy. End-users come into the picture with Extensio because they can use the various extended data sources in a location of their choosing, from their mobile phone, Excel, and even Web widgets. Data sources can include relational databases, CRM/ERP systems, and more. The real novelty in Extensio however is it’s support for user customizable widgets that allow users to put displays of data wherever they need to be, without technical assistance, which I’ve likened in the past as the new “visual SOA.” The Extensio widget gallery shows widgets that are powered by Extensio Symphony from back end services — including relational databases — but can be configured and then deployed by users with a cut-and-paste. Extensio is a commercial product that runs on the J2EE platform.
Microsoft has entered the situational software arena with a fascinating new mashup development tool called Popfly. One of the best looking mashup tools by far, and also one of the most engaging and easy-to-use, Popfly pushes the envelope in a number of areas including its advanced 3D building block visualization, extremely rich user interface based on Silverlight (the RIA technology formerly known as WPF/E), social networking-style mashup gallery, and consistent focus on enabling true user generated software. While still in closed beta, Popfly is getting near release and seems to have a lot of the right ingredients for a consumer-focused mashup ecosystem. Microsoft has long understood the ecosystem effect of 3rd party developers and makes it easy for a developer to create a new “block”, or piece of reusable functionality, accessible to the rest of the Popfly community that anyone can then incorporate into their mashup. Popfly sounds great, so what’s missing? Specifically, enterprise context including support for portals as well as direct access to relational data, to name a couple of items. Though to be fair, Microsoft SQL Server — and most other commercial databases — can turn just about any stored procedure into a Web service that can then be used with Popfly. Expect it to make a splash when it’s released to the public, though its lack of on-premises capability so far will be one significant impediment to enterprise use, along with its requirement for Silverlight plug-in installation on the client, at least until Windows Vista makes a larger corporate penetration than it has so far.
JackBe’s Presto series of enterprise mashup solutions is one of the most complete and compelling solutions available today for organizations that want to move the excitement and results of the consumer mashup story into their day-to-day business. Originally an Ajax framework provider, JackBe has matured their products to encompass a vision for the effective and safe employment of mashup techniques by developers and end-users alike. Providing a situational software development platform that is geared towards several key audiences inside of enterprises; Presto provides either the familiar Eclipse IDE model for mashup creation that developers will insist on, or an end-user friendly browser-based version that workers can use to build their own applications out of the services that have been provisioned by IT in the JackBe mashup infrastructure. That’s one of the key advantages of JackBe’s enterprise-friendly approach; governance and manageability have been designed into the platform at its core. JackBe’s Presto line, which includes Edge for users at the “edge” of the network as well as Studio for developers is arguably one of the most forward-thinking and advanced enterprise mashup platforms with only IBM’s QEDWiki nearly at the same level. JackBe is a commercial product that runs on the Java platform.
Kapow is the one of the most venerable — and capable — data mashup platforms on the market. With both an open community known as OpenKapow as well as a powerful desktop-based graphic mashup builder, Kapow has some of the best performing and most stable capabilities in the mashup world. Kapow can be hosted internally or externally (at the aforementioned OpenKapow) and can pull data and content from one of the widest varieties of sources of any product that include the usual like Web services and XML to content management systems, PDFs, and more. One of the trickiest parts of live integration of code and data is what happens when things go wrong and this is one key area where Kapow has maturity that few have, including robust error handling and exception processing. While many data mashup platforms are just standing on their wobbly legs, Kapow has been in production at many large consumer sites and enterprises for years. Kapow is a commercial product, though the OpenKapow community version is essentially free of charge but has limitations.
Proto is another interested new mashup development platform that claims to bring “a fresh look and industrial-strength desktop application development to knowledge workers.” While not Web-based, Proto has capabilities that support the industries that tend to have a very high percentage of knowledge workers, specifically those that work in the financial industry. But despite this, Proto has a consumer mashup attitude and their application gallery gives one a good sense that they understand how to build applications that make the most of corporate information combined with functionality from the open Web. Proto is a commercial product, though it’s free for personal use, and runs in conjunction with Microsoft’s Visual Basic for Applications and Adobe’s Flash platforms.
IBM’s QEDWiki is one of the most impressive mashup platforms presently available and is the de facto poster child for how the end-user development model of enterprise mashups might look. Based on the wiki model, QEDWiki smartly provides a end-user development approach that matches the traditional software development model: each change to a wiki page — which is also the unit of application development in QEDWiki — is versioned so that users are already using safe best practices in configuration management automatically as they wire in widgets and feeds to build the online apps they need on the fly. IBM has been quiet about the future of QEDWiki and though it has a well thought out development lifecycle, component palette, and much more, it’s unclear what will happen to it given that it’s currently on Alphaworks, where many IBM projects go to become open source. My 15 minute screencast that built a working business app from end-to-end earlier this year shows how QEDWiki shines when it comes to end-user-friendly development, but solutions like JackBe are beginning to show how QEDWiki is still lacking in deep enterprise readiness. QEDWiki is regularly updated and continues to evolve nicely and will be a key mashup platform to watch.
From the company with the name that looks like a typo but is really a semi-obscure programmer’s reference, \n software’s RSSBus takes the Yahoo! Pipes school thought to building data mashups. RSSBus turns just about any kind of data into RSS feeds that can then be consumed by anything that can process RSS, which is an impressive list of software these days (almost everything). RSSBus can access and pull data from the usual litany of sources: documents, databases, spreadsheets, file systems. It also comes with an decent set of connectors for many of the more popular open Web services such as Amazon’s eCommerce APIs and credit card processing services as well as more enterprise specific services such as SMTP. While it’s one of the more end-user unfriendly tools on this list, RSSBus does provide a powerful admin application that puts a lot of power in the hands of empowered users and small enterprise staffers. While Yahoo! Pipes might do the job for some tasks, one of the biggest selling points of RSSBus is that it can be deployed and managed internally for maximum control and security. RSSBus runs on .NET 2.0 and is free for single-user applications, with a multi-user commercial license available.
While genuine open source mashup solutions are still rare, SnapLogic manages to provide a solution that delivers a community-based data mashup server that can create the inputs needed for visual mashups. Even more unusual for an open source product, which often emphasize command-line based tools, SnapLogic has a fairly advanced graphical mashup IDE. With recently added support for JSON and RSS, SnapLogic advertises that it’s ready to provide the data for Ajax-based applications, which is increasingly how SOAs are projecting services within their organizations. SnapLogic supports connectors to SugarCRM, QuickBooks, and Salesforce among others. While SnapLogic also tends to skirt the edge of what users or even analysts might use in terms of tools, the foundation is there for them to provide tools that can break out of the IT back office, or least becomes a joint facilitator between IT and business, thereby getting users out of the manual integration business. SnapLogic supports Windows and Linux, although not Windows Vista currently.
SOA Express from StrikeIron is one of those mashups tools that is the exception that proves the rule. While entirely Excel-based, SOA Express allows the data contained in the enterprise SOA — or even from services out on the open Web — to be pulled in and integrated directly by end-users using the spreadsheet model. SOA Express purports to enable anyone to “build rich composite applications quickly and easily within Microsoft Excel without any programming.” While it makes the cut on this list because it’s currently available and work, it also shows why the previous model for user generated software, namely easy to understand models of rows and columns filled with data and formulas, has been hard to displace and we still have probably not hit upon a good replacement model with mashups yet, QEDWiki excepted. It’s worth quoting a bit more from the SOA Express documentation that says the product can “put the power of live data directly to the desktop without any risk to internal data controls. And business users can build ‘live’ Excel workbooks with real-time data available all over the organization and combine them with data that is available over the Web.” That’s unmistakably the mashup model of ad hoc integration of data and functionality and it shows how our older user-controlled data integration platforms might have life in them yet and move successfully into the mashup era.
Teqlo is a different concept in mashups and one that I’ve been tracking for a while. Based on a kind of Web 3.0 concept,Teqlo provides a mashup development service that can largely let an application build itself. Sporting one of the slickest and most Mac-like graphical mashup builders I’ve seen yet, Teqlo combines the go-anywhere roaming of a Web start page with a dashboard that can host all of the situational software that you’ve built. Teqlo also understands the importance of offering widgets containing commonly used references to external services and 3rd party functionality and has some of the most important ones already provisioned. I’ve been tracking them for over a year now and the product, one that was once only promising, has begun to show real capability as an enabler of situational apps and smart, almost Semantic Web-style data integration. Teqlo is a free consumer app in beta at the moment.
One of the highest profile mashup tools available today, Yahoo! Pipes is primarily a data mashup, not a user interface mashup tool. Despite this, its compelling Web-based IDE provides such state-of-the-art Ajax impressiveness and wow factor, that the promise of these types of tools shows through clearly in this product. Pipes continues to be improved and updated and lives up to its tagline of “an interactive data aggregator and manipulator that lets you mashup your favorite online data sources.” Complete with a community library of data mashups to select from (and learn from), the clone feature is a great way to find something that’s close to what you need, copy it, and tweak it with the changes you need. This potentially enables the fast, viral recombination and spread of good ideas and otherwise difficult to recreate from scratch integrations and is one of their most important features. While far from enterprise ready, Yahoo! Pipes does an admirable job doing what it’s supposed to do and is already Web-scale, meaning that it can do what few of the others can do, provide performance even in large-scale business situations. It’s limitations for enterprise use mean that only a small subset of enterprise data mashups could be built with it, but it may fit the bill for a subset of your business integration problems. Watch Pipes closely for improvement and changes, but it can provide the indispensable input to user interface mashup tools which so badly need the finessed data that Pipes can supply.
The last mashup environment on this list is also one of the other open source mashup solutions. Released just this month, the WSO2 Mashup Server is yet another data mashup service that provides the ability to compose and recompose services into forms that are more consumable. Specifically, the Mashup Server says it can provide “human-consumable results through a variety of user interfaces including Web pages, portals, e-mail, Instant Messenger service, Short Message Service (SMS), etc.” With an Apache license and a brand-new community portal to share data mashups, it’s unclear how popular it will be. However, it is apparent that even the open source community is finding the need to facilitate the combination and management of the increasingly torrent of services that are available to build upon. Also, it’s fairly unclear how easy the Mashup Server is to use but the home page promises that it is indeed a “platform for creating, deploying, and consuming Web services Mashups in the simplest fashion possible” and so it’s listed here with the benefit of the doubt.
Wrap-Up
One interesting aspect of this list is the relatively low prevalence of open source products. Only three of the seventeen mashup platforms listed above was based on an open source license. In addition, there is a clear dividing line emerging between products that only provide data integration and recombination from remote Web services and tools which provide a way in which to build a real user interface for a mashup, with a two-way flow of data from the underlying services. For now, commercial products remain far ahead of their open source brethren and data mashups are just as popular as user interface mashups.
Several other important trends stand out: Most mashup platforms have not yet fully seized on the power of widgets and gadgets to provide an effective model for sourcing components and composing mashups. The existing generation of platforms is also probably making it too hard for communities of user to form and share solutions that are easily cloned and tweaked. This could providing a Darwinian-like model of rapid, viral software evolution that provides thousands of tiny improvements on a continuous basis from a large user community, allowing complex applications to emerge that naturally fit their environments. A far-fetched sounding idea but one in which the large networks of our enterprises and particularly the Web have more than enough motivated participants to make this happen, if only the tools would focus on it. This may yet be another significant advantage of the mashup model vs. traditional development model and Yahoo! Pipes comes closer than anyone to achieving it.
In this way, it’s likely that the most successful mashup platforms will, like all things that appear to succeed on the Web, play to the intrinsic strengths of the network platform. And one of these strengths is that the network can provide the proverbial eyeballs that will make all bugs, lacking features, and other obstacles to great software shallower than any other way we know of making it. It’s a bright future, we just have to fix the five things listed at the beginning of this post to get there.
Will mashups unleash end-users to create the software they want, just like they are now creating the content they want?
Update: A few people have pointed out that Google’s interesting new mashup entry, the Google Mashup Editor (GME) was not on the list. I considered adding it and may soon do that when I get access to try it out. But all indications are right now that it’s entirely developer-oriented with no portion of it that could be employed by end-users.
A veteran of software development, Dion Hinchcliffe has been working for two decades with leading-edge methods to accelerate project schedules and raise the bar for software quality. See his full profile and disclosure of his industry affiliations.
