The Red Hat Summit also took place this week, prompting  announcements of new collaborations and service offerings. Read ahead for news on open government, the Red Hat Summit and more!

• Kevin Merritt’s NextGov op-ed discusses how the open data movement in government already has and will continue to help fuel innovation and entrepreneurship around solving problems and providing meaningful changes to society: “Op-Ed: Open Data Is Re-Defining Government in the 21st Century.”
• Loek Essers reported on Berlin’s open source software migration proposal getting voted down for a second time, but finding agreement on open standards in the InfoWorld article, “Berlin Won’t Migrate to Open Source, Looks to Open Standards Instead.”
• “Red Hat CEO: Open Source is Not Just About Cost” was Sean Michael Kerner’s Datamation article on Jim Whitehurst’s opening keynote during this week’s Red Hat Summit in Boston.
• In other Red Hat news, on SiliconANGLE Ryan Cox reported on the company’s newly announced collaboration and integration with Hortonworks as an effort to enable more open source big data projects within the Apache Hadoop community: “Open Source Big Data Storage on Apache Hadoop.”
• Steven J. Vaughan-Nichols reported on the OpenShift Platform-as-a-Service (PaaS) cloud offering coming out of beta after two years, in the ZDNet article, “Red Hat Opens OpenShift PaaS Cloud for Business.”
• On USA TODAY, Mark Veverka’s article, “Unplugged: Cassandra Could Wreak Havoc on Oracle,” compared the open source database Cassandra to the Linux operating system as a game changer for enterprise-scale database demands.
• In the ComputerWeekly blog, “Open Source HTML5 Secure File Sync for Android,” Adrian Bridgwater shares details around SpiderOak and how open sourcing its code has worked to improve the tool.
• “Like a Good Student, edX Finishes Open Source Project Ahead of Schedule” was Bryan Behrenshausen’s opensource.com interview with edX Vice President of Engineering and Educational Services, Rob Rubin.

Are you ready to hack your Sony SmartWatch?

Commercial enterprise software companies like SAP, Oracle, Microsoft and others spend thousands of development hours and millions of dollars annually to ensure that the software they deliver supports both cross-compatibility and backward-compatibility. Consumers of these products depend on this important element to support their migration and evolutionary efforts as they move their businesses forward in leveraging the latest technologies, while continuing to maintain long-tail systems that support significant segments of their businesses.

Software products that have been around for five or more years can spend upwards to 25 percent of engineering investments on compatibility. As a long-time product manager, I’ve regularly wanted to invest the majority of R&D budgets on net-new features, but without fail, have always compromised new features for compatibility to ensure seamless support for the upgrade path of broad, complex implementations.

But what about open source software?

This past week, I attended JAXConf, a popular Java developer conference in Santa Clara. While I regularly speak with enterprise Java developers about their challenges in utilizing open source software, (including selection, licensing and integration) I had a compelling conversation this week with an experienced enterprise architect from a major health care processing company about backward-compatibility.

He shared with me that, while the open source opportunity was top-of-mind, backward-compatibility was a key concern and one that has all too often created challenges for his long term use of open source. Exploring this issue further, he said that within his organization, there is significant interest in leveraging the open source opportunity, but wonders why the open source community isn’t taking a more serious, longer-term view on compatibility.

My own experiences have been mixed with open source and backward compatibility. Many of the more popular, mainstream projects that are used in binary form seem to actually provide pretty good backward-compatibility support. But these projects represent a small fraction of the broader open source opportunity. Is the open source user base less demanding then the typical commercial enterprise software user?  I think not. Maybe it’s more about the drive to move forward quickly, without wanting to spend precious development time looking back to ensure seamless upgrades. Or maybe it’s just a mindset that some projects focus on, while others do not.

My experiences with projects from the major open source foundations including Apache, Eclipse and Linux, have all been really good, so maybe it’s a reflection on the level of organized efforts versus the small-time independent development team.

What do you think? How important is backward-compatibility to your organization? And if you are an open source developer, how important is backward-compatibility to the success of your project?

• Following up on a big story from last week, ZDNet’s Steven J. Vaughn-Nichols shared details around Mozilla’s newly announced backer for its Firefox OS, electronics manufacturer Foxconn: “Foxconn Backs Firefox OS Play.”
• Derrick Harris’s GigaOM article, “IBM Throws Its Weight Behind MongoDB For Mobile Apps,” covered IBM’s work with 10gen on a MongoDB API based standard for mobile apps built on the NoSQL database.
• “My Year Of Living Open Source” was Sam Muirhead’s opinion piece on CNN, updating the world on his yearlong mission to live completely open source, without using any copyrighted or patented products.
• DJ Walker-Morgan’s The H article, “Big Switch Leaves OpenDaylight,” reported the reasoning on the behind Big Switch’s decision to leave the OpenDaylight project it helped found.
• With the feel-good story of the week, on opensource.com Michael D Roden told of how DrupalCon attendee’s came together in a volunteer effort to help FEMA create the Help4OK.org site as an information hub for victims of the recent Oklahoma tornado: “Disaster Relief Now From DrupalCon.”
• On InfoWorld, Brandon Butler reported on the Yarn workload management tool being included in Hadoop’s 2.0 release, making it easier to build and run apps on its platform: “Get Ready For A Flood Of New Hadoop Apps”
• Michael Fitzgerald’s InformationWeekly article, “EdX Goes Open Source To Woo MOOC Developers,” described how the online education platform (started by Harvard and MIT) released its full source code to encourage more contributors.

Have you contributed to the kickstarter campaign to help the Smart Citizen open source, crowdsourced environmental monitoring platform?

I’ve been reminded numerous times since that conversation about the list and how comprehensively it covers the space of software today. But, nagging in the back of my head somewhere has always been the desire to see those four threads weaved together. Which is why Kristof Kloecker’s keynote at this year’s IBM Innovate conference was so appealing. It was the first attempt I’ve heard to provide a Unified Field Theory, if you will, for the current state of software. He didn’t completely nail it and I’ve been thinking about it a lot in the last few days to the point that I can’t distinguish where I’ve enhanced what he said. So, here’s my weaving with Kristof’s inspiration.

First, a new thought IBM was presenting at the conference was Time to Customer Feedback as a substitute for Time to Market. This dovetails beautifully into concepts of The Lean Startup, which author Eric Ries presented at the conference as well. About 100 pages in, I’m hooked on the idea that designing fast learning loops is the formula for development success in the face of extreme uncertainty. I believe it was strategic planning wiz Arie de Geus of Royal Dutch Shell who said that the only sustainable competitive advantage is to learn faster. Ries tells you how to do it with a simple framework: Build, Measure, Learn…repeat. The key is to focus on that and doing it as fast as you can.

Back to the big four themes: Woven together, they enable Enterprises to spin the loops, learn fast and thus enable sustainable competitive advantage in today’s crazy, uncertain business environment.

Cloud provides the velocity in building. Mashing up services, tried and true open source components, frameworks, PaaS…these are the pieces developers can quickly assemble to get prototypes and/or minimum viable products and, now, via devops tools, to quickly deploy them.

The ubiquity of Mobile devices and the Social networks that stitch them all together enable measurement by giving companies unprecedented access to customers. Focus groups and consumer testing have historically been limited and costly. Mobile devices provide the communication vehicle for companies to access the behavior the mass of consumers and to do so economically. Social Media provide a ready communication channel for the exchange of feedback to the company as well as between consumers.

The good news/bad news is by broadly engaging customers through Mobile Devices and Social Media a company can generate massive amounts of data. That’s where Big Data comes into play. To learn, companies need to make sense of the mass of (often unstructured) data and that’s the promise of big data.

As Eric Ries says, in the old days the question was “Can we build it?” and now it’s “What should we build?” Woven together Cloud, Mobile, Social and Big Data, all driven by open source, both enable Enterprises to build anything as well as to scientifically navigate through great uncertainly to make sure they are building the right thing.

One of the challenges as a core supplier in the embedded software supply chain is providing license compliance information in a single common format that all our customers can readily consume and share. The challenge today stems from the reality that different organizations insist on obtaining different kinds of information in different preferred formats that support different internal processes. Each organization performs the task of combing through huge repositories of open source software in order to collect up relevant license information. This task would be independently and repeatedly performed by each of the different vendors in the supply chain for the same software, as it is passed from one vendor to the next. The missing piece was a standard licensing reporting format that would ensure the information is clear, consistent and of high quality (regardless of who prepared it and independent of any specific process).

The Software Package Exchange (SPDX) is the Linux Foundation’s answer to this problem. SPDX is neither a tool nor a process, but a specification, which describes a specific file format for recording licensing and copyright information of a software package. A simple but useful analogy is the Portable Document Format (PDF). PDF, like SPDX, is a standard format design to enable sharing of content independent of the tools or process that created it. Although many tools can create and read PDF files, PDF is just a format description (file data layout). You do not need to understand the details of the PDF format to view and benefit from a document stored in the PDF format. Therefore, like the PDF specification, one is not required to read and understand the SPDX specification to benefit from the information contained in an SPDX file. The SPDX specification details are important to developers who build tools that do the analysis to create, store, retrieve, display and edit SPDX content (the producers). And as with PDF, it is expected that the large majority of us (the consumers of SPDX data) will access the content using a viewer that displays SPDX data in a friendly and useful manner.

The idea of using a standard record exchange format within a supply chain is not new. For example, the automotive industry has been using Electronic Data Interchange (EDI) protocol within the automotive supply chain to share parts ordering information for decades. Like EDI, SPDX provides a key missing infrastructural piece for exchanging software components within the embedded software supply chain.

As companies that share the agenda of helping systems development organizations be compliant, both Black Duck and Wind River are active participants in the Linux Foundation’s SPDX working group providing input and assisting with the adoption of SPDX. View our jointly delivered webinar about best practices in managing license compliance in embedded software solutions, SPDX being one.

Today’s procurement professionals can play a key role in helping organizations realize additional cost savings from the use of FOSS. As new requests for third-party commercial software are submitted, procurement specialists need to be able to assess if open source alternatives are available that could potentially replace the need for a commercial software purchase. To accomplish this, they’ll need to understand where to find and evaluate FOSS projects and become familiar with FOSS selection criteria, including support, license, security vulnerabilities, and more. This requires a working knowledge of the many public FOSS resources, including how to navigate within them to discover needed information supporting the selection process.

And then there’s the question of why procurement professionals aren’t spending more time helping development organizations choose the “best” open source projects. Like all purchases of goods and services, there’s more than just price involved in the selection. Today’s procurement experts can help development teams choose projects that fit into broader open source policies, reducing potential risks and longer-term maintenance costs. So while there is technically no direct “cost” for the license to use open source, there are indeed costs associated with the deployment and implementation of open source software, similar to “closed source”/proprietary cousins. And many of the same questions used for proprietary software solutions apply, including size of development team, release schedules, security vulnerability history, etc.  So it makes sense that procurement ought to be involved in what software solutions, open source or not, are brought into their organizations.

But it seems impractical for procurement to be involved in all open source selection, so where should they draw the line?  What about the 100s and perhaps 1000s of open source components, snippets, and files that developers bring in that become part of an open source or closed source product or solution?  Should procurement look at each and every component that leaks in through every developer’s desktop?  A good starting point would be to focus only on the “standalone” solutions that might be considered as alternatives to closed-source/proprietary software, projects including WordPress, Drupal, SugarCRM, Alfresco, and hundreds more.

There are several resources available today that can help procurement professionals navigate the world of open source alternatives, in addition to methods and tools to facilitate its review.  OSS Watch recommends that procurement consider the use of frameworks including the OS Maturity Model (OSMM); the Qualification and Selection of Open Source software (QSOS); or the Open Business Readiness Rating model (Open BRR).  Procurement will also want to consider open source support organizations like credativ that can assess and offer ongoing maintenance for popular FOSS solutions.

Procurement teams have long helped organizations acquire goods and services to meet their business needs. With companies spending hundreds of millions of dollars annually on software purchases, there are significant potential savings associated with an open source savvy procurement specialist! And when procurement professionals engage in helping out with the selection of open source, developers can stay focused on evaluating features sets and writing code, bringing projects to market faster.

Becoming involved with open source solution acquisition ought to be a “given” for today’s procurement organizations.  So why are so few procurement organizations involved in the acquisition of FOSS?  Is your procurement team involved?

• On GigaOm, Barb Darrow shared Dell’s change of heart over creating an OpenStack based public cloud, in the article, “Dell Backs Away From OpenStack Public Cloud, Steps Up To Enstratius.”
• “Android Dramatically Extends Lead With Open Source Developers” was Matt Asay’s ReadWrite article reporting on mobile data from Black Duck Software that revealed new open source projects from Android outstripped new iOS open source projects by a factor of four in 2012.
• Jennifer Cloer interviewed Samsung’s Ibrahim Haddad about the company’s efforts to attract the best open source developer talent, in the Linux.com article, “Samsung Talks About Its Aggressive Linux Talent Recruitment Strategy.”
• Also on Linux.com, Joe ‘Zonker’ Brockmeier caught up with Luke Kanies, CEO of Puppet Labs, to discuss how he manages running a company and project community at the same time: “Puppet Labs CEO: How to Grow an Authentic Open Source Community.”
• On opensource.com, Adam Firestone shared the “Top 5 Misconceptions About Open Source in Government Programs.”
• Nick Summers reported on the first open source Sailfish OS based mobile device announced by Jolla, in The Next Web article, “Jolla unveils its first Sailfish OS smartphone, launching in Europe before end of 2013 for €399.99.”
• “Mozilla, Foxconn May Launch Firefox OS Tablet Next Week” was the CIO Today headline for Barry Levine’s article anticipating that Mozilla and Foxconn will announce a new Firefox tablet during their upcoming press conference scheduled for June 3^rd.
• Jeremy Kirk covered the Australian government’s draft proposal for software compatibility with OpenDocument Format across all of its agencies in the ComputerWorld article, “Government Mulls Requiring OpenDocument Format Compatibility.”
• The Register’s Gavin Clarke reported on the details around Drupal.org and groups.drupal.org recently being hacked, “Drupal Hacked, Resets Passwords After Millions of Accounts Exposed.”
• In the InformationWeek article “Rackspace Meets VMware Halfway On Hybrid Cloud” Charles Babcock provided Rackspace President Lee Moorman’s perspective on different approaches to private and public cloud offerings.

Who’s ready for a slice of NASA’s 3D printed pizza?

Image credit to FastCompany Co.Design

The average in-house practitioner can’t be blamed for a lack of knowledge about OSS and its impact on business with today’s dizzying pace of technological advancement. Most lawyers are trained to deal with transactions, regulations and contracts; few are trained to understand and deal with the profound impact software technology has on the businesses they counsel and guide. For most lawyers of a certain age, open source is a topic that wasn’t properly discussed during law school or early years on the job.  Compounding this difficulty is the acronym soup of licenses (GPL, MIT, BSD) particular to OSS and the strange, inverted application of legal principles (e.g., copyleft) vs. principles introduced in traditional legal training (e.g., copyright).

The lawyer who dismisses OSS based on uncertainty or doubt stemming from a GPL-based fear is missing a tremendous opportunity to add value to their organization.  Ideally, the forward thinking lawyer will actively promote adoption of open source in his or her company.  At the very least, the in-house lawyer must not be a roadblock.  And this is where “the talk” comes in.

Just as parents must prepare themselves to have the essential (and sometimes awkward) coming of age conversation with their children as they mature, it is critically important for general counsel to be ready to have “the talk” with their IT departments. It will have to happen eventually; it’s just a question of when. In preparation for this talk, let’s highlight the three big questions about OSS and its use a general counsel should be able to answer.

The Big 3 OSS Questions

 1.  Why should I care about open source?

Open source software use is mainstream in companies large and small, public and private, technology-enabled and technology-producing. For example in 2012 IDC reported that open source makes up 30% of the code in Global2000 organizations. The in-house lawyer may not realize it, but open source is probably being used right now by developers, who are building and improving your company’s development tools and products. If the general counsel doesn’t know about the company’s open source use – or worse, thinks it isn’t happening – the business has a real problem.

 2.  Which OSS licenses should I counsel the business to use?

If your developers are using open source software to create or improve your company’s products, the best place to start is by determining what you want to do with the OSS code. Match the license choice to your business goals. If the OSS code will end up in a product your company will distribute, it is typically wise to opt for a permissive license (e.g., MIT or one of the BSD family of licenses) or one of the licenses that strike a compromise (e.g., the GNU Lesser General Public License (L-GPL) or the Mozilla Public License).

With respect to releasing some of your company’s code under an OSS license, license choice can be determined by simply deciding how much you do or do not want to encourage downstream reuse of your code. If your company wants to minimize commercial reuse of its original code, choose a restrictive or copyleft license, perhaps one from the GPL family. If you are content to let others add to your work including commercial distribution in the future, perhaps under even more restrictive provisions (as long as they retain copyright notices), use a permissive license, e.g., MIT or one of the BSD family of licenses.

There are more than 2,200 OSS licenses in use and fewer than 70 are approved by the Open Source Initiative’s (OSI) license review process. In choosing an OSS license to release your company’s code under, it’s good practice to use an OSI-approved license.

3.  What do I need to do to prepare for OSS use?

The most important step to using OSS is to create an open source policy and stick to it. Involve your development team in the process and ask for their input into the policy. But first, audit your code base to see what’s in it now. You might be surprised to find how much OSS is already present. Know what your obligations are and, where necessary, take remedial action,

Do what you can to make life easy on your development teams and the legal department by automating open source software use and management, since many organizations find the volume of code or speed of development quickly outpaces manual methods. By helping to streamline the tracking and management of open source use in your organization, you as a general counsel will truly become a partner to development, rather than an obstacle to be overcome.

For more details around how general counsel can prepare for “the talk” with IT departments around open source use, read my new whitepaper: The Talk –What a General Counsel Needs to Know about Open Source Software.

Without revealing confidential details: A partner is doing some cool stuff with Ohloh, essentially using the free, public directory of open source software (OSS) to verify the OSS cred of developers in order to qualify them for a special offer. They’ve done this manually in the past, requiring the developers Ohloh ID and looking up their commit history. However they want to scale the program and needed to automate this manual process.

The Ohloh API supports most of the functionality that the partner was looking for, but to really ensure good user experience, they wanted to use OAuth, the open protocol for authorization, functionality that wasn’t supported. On the morning of our call to tell the partner that we would have to look at this as a future addition,  Ohloh’s development manager pulled up…well Ohloh, on the chance he might find something that could help. And he found omniauth-ohloh, a component that did exactly what the partner needed and was already integrated with Ohloh. So the answer essentially turned around 180 degrees to…yes we do support OAuth.

OK, it’s not quite that simple. The component is in Ruby and needs to be rewritten in Python and there’s some testing and integration to be done. But the big point holds: There are a million projects out there, ranging from complete apps to widgets, that perform the most arcane functions…in this case, just the arcane function we needed. If you have a problem, it’s worth looking around and Ohloh is a great place to start. For this one you could search either on project metadata or code via Ohloh code search and find just the right thing.

As an aside, if you have any interest, like the aforementioned partner does, in providing resources to legit open source committers, or to vet committers for any other reason, we would love to support.

The spur to collaboration

One theme in the slides is the formation of consortia that develop software for entire industries. One recent example everybody knows about is OpenStack, but many industries have their own impressive collaboration projects, such as GENIVI in the auto industry.

What brings competitors together to collaborate? In the case of GENIVI, it’s the impossibility of any single company meeting consumer demand through its own efforts. Car companies typically take five years to put a design out to market, but customers are used to product releases more like those of cell phones, where you can find something enticingly new every six months. In addition, the range of useful technologies—Bluetooth, etc.—is so big that a company has to become expert at everything at once. Meanwhile, according to Vescuso, the average high-end car contains more than 100 million lines of code. So the pace and complexity of progress is driving the auto industry to work together.

All too often, the main force uniting competitors is the fear of another vendor and the realization that they can never beat a dominant vendor on its own turf. Open source becomes a way of changing the rules out from under the dominant player. OpenStack, for instance, took on VMware in the virtualization space and Amazon.com in the IaaS space. Android attracted phone manufacturers and telephone companies as a reaction to the iPhone.

A valuable lesson can be learned from the history of the Open Software Foundation, which was formed in reaction to an agreement between Sun and AT&T. In the late 1980s, Sun had become the dominant vendor of Unix, which was still being maintained by AT&T. Their combination panicked vendors such as Digital Equipment Corporation and Apollo Computer (you can already get a sense of how much good OSF did them), who promised to create a single, unified standard that would give customers increased functionality and more competition.

The name Open Software Foundation was deceptive, because it was never open. Instead, it was a shared repository into which various companies dumped bad code so they could cynically claim to be interoperable while continuing to compete against each other in the usual way. It soon ceased to exist in its planned form, but did survive in a fashion by merging with X/Open to become the Open Group, an organization of some significance because it maintains the X Window System. Various flavors of BSD failed to dislodge the proprietary Unix vendors, probably because each BSD team did its work in a fairly traditional, closed fashion. It remained up to Linux, a truly open project, to unify the Unix community and ultimately replace the closed Sun/AT&T partnership.

Collaboration can be driven by many things, therefore, but it usually takes place in one of two fashions. In the first, somebody throws out into the field some open source code that everybody likes, as Rackspace and NASA did to launch OpenStack, or IBM did to launch Eclipse. Less common is the GENIVI model, in which companies realize they need to collaborate to compete and then start a project.

A bigger pie for all

The first thing on most companies’ minds when they adopt open source is to improve interoperability and defend themselves against lock-in by vendors. The Future of Open Source survey indicates that the top reasons for choosing open source is its quality (slide 13) and security (slide 15). This is excellent news because it shows that the misconceptions of open source are shattering, and the arguments by proprietary vendors that they can ensure better quality and security will increasingly be seen as hollow.

Going beyond these considerations is an important sign of strength for an open source project: the creation of a community that extends the project. OpenStack has clearly reached this stage, with so many new contributions that hardly anyone can keep track of them. Drupal offers another stunning success in this regard.

Users are starting to recognize the value of crowdsourcing for development. Note the interest both in using APIs and in offering APIs to code the users have produced themselves (slides 22 and 23). Two-thirds of open source developers surveyed work for a large company (IBM, Ford, etc.) and get paid to do open source coding.

Ultimately, the increased interoperability and community input lead to open source’s best contribution: it makes a bigger pie to divide up than proprietary technologies.

Cloud computing is an excellent example. Pretty much since Amazon launched AWS—a fine product that has altered the hosting equation for thousands of companies—analysts and business leaders have been warning of lock-in in the cloud. Other issues, such as security, performance, and SLAs, come up regularly too, but potential cloud customers are fundamentally driven away by the fear that data will be lost or the cloud provider will simply go out of business. OpenStack addressed this fear by guaranteeing that data and compute instances could be moved freely between providers. Although other services emulating AWS have come into being, no proprietary provider could give the same guarantee as an open source platform. And consequently, an open platform will hugely expand the customer base for cloud computing.

Open development may be the key to breaking the logjam in the market for electronic health records, most of which are priced so astronomically that only major institutions can afford them. (SaaS providers such as athenahealth and Practice Fusion are filling the gap, but they are not open source and therefore suffer some of the problems of cloud computing.) The Department of Veterans Affairs’ VistA software has gotten a lot of praise for health care management, a notable feat in a field where users slam proprietary offerings as difficult and unfriendly. Although VistA was released many years ago to the public, it was slow to catch on, partly because three or four companies took it in different directions and made incompatible platforms while fighting each other for a small market. VistA was nominally open, but was not being developed in an open manner like Linux or Drupal.

Finally, a couple years ago, the VA set VistA on a firmer open ground by putting it under the care of an independent non-profit organization, OSEHRA. Part of their mandate is to develop community, the importance of which I’ve already discussed. In particular, they’re trying to get all the companies to develop VistA-based products to work together on a common code base. OSEHRA, incidentally, collaborated on the Future of Open Source survey.

The survey

A few other intriguing results from the Future of Open Source survey include:

• Government is the largest adopter of open source, ahead of any particular industry (slide 11)
• The number of healthcare open source projects is rapidly increasing, although the slides don’t indicate how widespread their use is (slide 12)
• SaaS is on its way to becoming the preferred way to deliver open source software (slide 41)

I spent a good deal of time asking Yeaton and Vescuso how the survey was conducted, to find out how diverse the respondents were. Thirty of Black Duck’s collaborators distributed the survey to their customer base, and broad input was sought through social media and mention by Forrester Research. Many of the 800 respondents came through an appeal by open source vendors. Two-thirds of the respondents were companies who were mainly consumers of open source. So I would guess that a large proportion of respondents were happy users of open source, but that the field of software users as a whole were reasonably well represented.

This blog was originally posted on O’Reilly Media.