Port 25: Open Source Community at Microsoft

OSCON 2009

Peter Galli — Tue, 30 Jun 2009 00:29:00 GMT

As Microsoft continues to support and participate in open source communities, the company is again a proud sponsor of the annual O'Reilly Open Source Convention (OSCON), which is being held in San Jose from July 20 to July 24.

In addition to having a booth on the show floor, Tony Hey, the Corporate Vice President for Microsoft External Research, will deliver a keynote address on Thursday July 23, titled "Open Tools and Services on Microsoft Platforms," which will examine the far-reaching changes open research tools and services will have to support every stage of the research process.

Erik Meijer, one of Microsoft's Principal Architects, will also give a keynote talk on Friday July 24 and titled "Fundamentalist Functional Programming."

His talk will argue that fundamentalist functional programming - that is, radically eliminating all side effects from programming languages, including strict evaluation - is what it takes to conquer the concurrency and parallelism dragon.

Following his keynote, Erik is also presenting on using the LiveLabs Reactive Framework to democratize the cloud.

Vijay Rajagopalan, a Principal Architect in Microsoft's Interoperability group, is also giving a talk on Wednesday July 22 in the Product and Services Track, titled "Interoperability - Build Mission Critical Applications in PHP, Ruby, Java and Eclipse Using Microsoft Software & Services."

During his presentation, Vijay will talk about how Microsoft has delivered multiple technologies that focus on interoperability with non-Microsoft and Open Source technologies. He will also show how developers can, today, use Eclipse tools to build Silverlight applications that run on PCs and Macs, as well as how they can develop using combinations of PHP, Java and Ruby in addition to the standard Microsoft languages.

In addition to all the talking, we also expect to do a lot of "showing," and a number of product groups will be represented in the Microsoft booth, including folk from the Education, External Research, Open Source Technology Center, Interoperability and CodePlex parts of the company, all of whom will be giving technical demos and chatting to attendees..

An analyst/partner roundtable discussion is also on the cards, as is a broader interoperability discussion. You won't want to miss any of it.

Crafting a Better PHP Build Process on Windows – Part IV

Garrett Serack — Wed, 24 Jun 2009 00:12:00 GMT

In the previous post, I discussed what it took to use PGO on the Windows PHP build. That led to me building automated build scripts...

Automation as the root of all evil

"Anything that can be done for you, automatically, can be done to you, automatically." - David C. Wyland

First, I had to get the entire dependency stack into the mix. While some of the dependent libraries had VCProject files, some didn't. Worse, even if they had them, you couldn't tell with a degree of certainty that they were compiled with the same settings which would enable them to take advantage of PGO optimization.

I began taking each project, updating (or creating, using the Trace and mkProject tools) the Visual C++ project files that would use the same settings as the rest, and eventually came up with a solution file that had 74 projects in it - some of the projects generated more than one binary.

Next, I had to actually automate the process of creating the vcproject files. Once you've got the right dependencies, the PHP build process cranks out over 30 binaries when you include the PHP extensions that get built as part of the core.

After what seemed like a million compile-verify-tweak iterations, I had the tools that could generate VCProject files for the core PHP and all the extensions, provided it was all in the right place.

Next I wrote a .cmd batch script that went step-by-step, checking out the source, compiling the dependent libraries, building the PHP makefile, compiling PHP like the community did - and logging what it was doing, then switching to instrumentation, rebuilding the dependencies again, building the stack, PGO training it with test data and some applications (Wordpress, MediaWiki and phpBB) and then relinking it with optimization.

I got the .cmd script almost working, but it was fairly fragile. At that point I decided to switch batch scripting strategies and, in about a week, rewrote the batch script in JScript, which was far more flexible, and a lot more reliable.

What's next...

"The future always arrives too fast... and in the wrong order." - Alvin Toffler

During this process, I tweaked the build process that is generated quite a bit, adding in a few more applications to the PGO training, which cranks the performance up more and more.

Now, I can add in more scripts to assist with the training pretty trivially, but it still takes some effort to package up an entire application like MediaWiki or Wordpress and include it into the build process.

Even once I've added in an application, I end up doing a whole slew of comparative testing to see what impact it has on the final executables.

As time goes by, I'm sure there will be more tweaking to be done but, in all likelihood, any significant performance gains are going to be the result some modification of the PHP codebase itself.

Crafting a Better PHP Build Process on Windows – Part III

Garrett Serack — Wed, 17 Jun 2009 20:47:00 GMT

Previously, I talked about using PGO in the PHP build process. In order to use it I had to observe...

The Heisenberg build process

"A process cannot be understood by stopping it. Understanding must move with the flow of the process, must join it and flow with it." - The First Law of Mentat, quoted by Paul Atreides to Reverend Mother Gaius Helen Mohiam

Really, what I needed was a tool in two parts. The first would watch what happens during the build process, and the second would take that data and spit out some .vcproj files.

When I want to see what's happening on my own system I use ProcMon - a Sysinternals tool that monitors processes, what files they touch, what commands get executed, etc. I grabbed that and tried to watch what happens when you run NMake on the makefile when building PHP. It turns out that are a few problems with that - ProcMon isn't very scriptable (making it tricky to automate) and even if it was, it has problems chopping off the command line in its log files when it's past a certain length.

I found nothing else that did quite what I needed, so I started thinking about how to write a tool that does the same thing. In the past I have used Detours (an API detouring library built by Microsoft Research) to build a couple of quick-and-dirty snoop/debugging tools. Starting with a sample that came from the Detours library, I cobbled together a tool that would watch a process and its children, recording every file written or read, every command issued, and dump it into an XML file which I could process later.

Creating the project files

At the same time, I began working on a tool that would generate .vcproj files from the data gathered during the make process. I first tried just putting together a tool which assembled the .vcproj XML file from what I knew about the layout of the project file but, as the build got trickier, the xml was getting harder to make sure it came out the way that Visual Studio expected. I turned to the Visual Studio SDK to see if there were any COM objects I could use to manipulate project files - there were, but they aren't documented in great detail, and they were really designed to be used to inside Visual Studio for automation. Having scoured the planet, I found some examples of using the VCProjectEngine to generate project files.

For a couple of weeks solid, I worked on the tool to generate project files, compiling, testing, tweaking, etc. I finally reached a point where I generated a project file completely that would compile the php.exe and php5.dll . Having finally arrived at this point, I built PHP using PGO instrumentation, ran the bench.php script from the PHP source directory, and then re-linked the project. This first time, I saw about an 18% improvement in speed over the previous version!

That moment

"It ain't over 'til it's over, and maybe not then, either. " - Slovotsky's Law #29

Well, as anyone who's done software development will tell you, there's the moment when you finally get your program to do what you want under very controlled conditions, and then - quite some time later - there's the moment that you can give the fruits of that labor to someone else so they can do the same thing.

Now that I had passed the point where I'd finally proven that it was worth the effort to build a PGO-optimized version of PHP, I had to get it scripted so that it could be done in an automated fashion, not just on my computer or a computer in our Lab.

In the final part, I wrap up with the automation of the build and look to where we might go next in PHP.

Crafting a Better PHP Build Process on Windows – Part II

Garrett Serack — Thu, 11 Jun 2009 20:10:00 GMT

I talked about getting started in building the PHP stack in my last post, now I'm taking it...

One step further

"We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil." - Donald Knuth

A chance conversation I had last summer at OSCON with Trent Nelson - who was building Python on Windows - planted the seeds of how to get PHP on Windows optimized further. Trent was using the PGO features of Visual Studio to generate Python binaries that run faster.

Rather than spend a lot of time optimizing all the little bits of PHP itself, I thought that this would be an ideal way to improve the overall speed of PHP, provided I could find the right scenarios to train PHP with. Little did I know that finding the right scenarios wasn't the hardest part.

What is PGO? (from Wikipedia)
Profile-guided optimization (PGO) is a compiler optimization technique in computer programming to improve program runtime performance. In contrast to traditional optimization techniques that solely use the source code, PGO uses the results of test runs of the instrumented program to optimize the final generated code. The compiler is used to access data from a sample run of the program across a representative input set. The data indicates which areas of the program are executed more frequently, and which areas are executed less frequently. All optimizations benefit from profile-guided feedback because they are less reliant on heuristics when making compilation decisions.

Adding PGO to the existing build process

"I have not failed, I've just found 10,000 ways that won't work." - Thomas Edison

I had downloaded the source to the dependent libraries off the PHP wiki, checked out the PHP source code, and begun the process of adding in PGO support to the existing build process. This proved to be extremely difficult.

Even limiting the scope to just the core of PHP itself - without the dependent libraries - I ran into trouble trying to compile using PGO instrumentation and then re-linking after running some tests. The make file that gets generated by the configure.js script (a JScript version of the automake configure script for the Windows platform) was just not built with what I had in mind.

I spent the better part of two weeks trying different approaches to tweaking the makefile so that I could use PGO to improve the PHP executable, but I kept running into roadblocks. Worse, the closer I got to a makefile that did what I wanted, the farther away from the current build process I was getting, and I wasn't sure that what I would end up with would even be close to what was being built today.

The long dark winter road

"Only the meek get pinched. The bold survive." - Ferris Bueller

I came to the conclusion that I'd have to build new Visual Studio project files from scratch. What worried me was that this would end up to be a completely different build process, and I'd never get the community to abandon what was already working, so I'd better be able to rebuild these new project files easily.

I started looking (inside Microsoft and out) for any tools which generated Visual C++ project files. I found someone internally who had used some JScript to create project files from text files, but after some experimentation, I found this was nowhere near what I needed. What I really needed was a way to convert the generated Makefile into a .vcproj file-and not just 'wrap' it.

Once I found there was no such tool* , I began trying to figure out how to create one. I had this idea a few times in the last decade or so: watch how a program is compiled, and create a project file that does the same thing. Having tossed around the idea in my head before, I knew it wasn't going to be trivial but, without it, I couldn't do what needed to be done.

* Let me tell you: you never want to think about writing a tool to parse out what a makefile does. It's rather like making a tool that tells you how sausage is made, in excruciating detail. Ugh.

In Part III, I'll talk about the trouble with observing the build process.

Crafting a Better PHP Build Process on Windows – Part I

Garrett Serack — Tue, 09 Jun 2009 18:22:00 GMT

The last several months, I've been working very deeply with PHP - specifically, compiling the PHP core itself, and looking for avenues for optimization. This is the first of four posts about the journey I've been on with PHP.

I get started building PHP

"It is a bad plan that admits of no modification" - Publilius Syrus

I started working with building PHP itself about a year ago. Initially, I was trying to put together an environment to compile up the PHP stack so that I could do some debugging, and track down a few faults that we were encountering in some of the PHP applications that we were trying to modify to use the SQL Server PHP driver that the SQL Server team here at Microsoft was creating.

Once I began to work with the source code, I found out very quickly that on top of having a hard time recreating the exact same binaries that the community build process generated, there were a large number of dependent libraries that were available in binary-only form and which were kept in a zip file that was passed around from developer to developer. That seemed a little odd for an open-source project, but I can certainly understand that over time, unless someone is working hard to keep it all together, these things happen.

Around the same time, the community had started to invest time and effort to 'clean up' the dependencies for building PHP on Windows, and move towards supporting VC9 (Visual Studio 2008) as an officially supported compiler.

In order to help in this process, I built out some testing environments in our Lab, which would let me compile up PHP on Windows and Linux, in order to get decent and reliable test results which we could use to identify any shortcomings that we could then address. This includes benchmarking not just the core PHP executable, but replicable and comparable testing of PHP applications such as Wordpress, MediaWiki, Gallery and phpBB.

PHP 5.3 on Windows: Not your father's PHP

"I'm looking for a lot of men who have an infinite capacity to not know what can't be done." - Henry Ford

For PHP 5.3, Pierre (and others) had gone out and found up-to-date versions of all the dependencies, brought them together, and managed to get them compiling with VC6 and VC9. They had posted these in binary and source form to the PHP Windows Internals site, which allows anyone to rebuild the PHP stack on Windows and, theoretically, get the same results as the 'official' build.

Jumping in at that point was much easier than it had been, as all you had to do was download the binaries of the libraries, check out the source code, run a few commands at the command line and, presto, you had your PHP executables.

At this point Pierre and I played around with the build flags on VC9 and found some settings that gave some pretty significant improvements to the speed of PHP vs. the speed of the VC6 version -and a lot of speed improvements vs. the old 5.2x line of PHP.

In Part II, I'll talk about going one step further with optimization.

Microsoft, NASA and Open Source

Peter Galli — Tue, 02 Jun 2009 15:05:00 GMT

There has been some renewed media interest in the NASA Space Act Agreement with Microsoft, which was signed earlier this year.

Microsoft signed that agreement so as to provide an umbrella framework of contractual terms that allows for a variety of cooperative projects. At the same time, we also entered into a project agreement that had an initial project to write code that would allow for the conversion of a lot of data about the planets (the planetary data system) and Martian survey information (LROC) into the World Wide Telescope (WWT) format.

WWT is an online program that pulls together images from across space and which lets users use their computer screens to traverse the universe.

Conversion of the information into the WWT format Tessellated Octahedral Adaptive Subdivision Transform (TOAST), allows the data to be viewed in both the WWT client and a newly developed Silverlight version of the client that supports multiple platforms, while other clients can also implement TOAST. The WWT is a freely downloadable Windows client application.

TOAST displays flat images, like those from telescopes,on representations of spherical objects like planets and moons, on a computer screen. One of the primary reasons for adapting the TOAST format for WWT was that it can accurately render the celestial poles with little distortion.

The traditional Mercator projection that is used by Google Earth and Sky in Google Earth cannot accurately render terrain or the sky within 15 degrees of the poles. TOAST is able to accurately render the sky and polar regions of the sky, Earth and planets with little distortion, which was important to both WWT and NASA.

The intent all along was also to make the conversion utilities that were developed for this initial project available under an open source licensed distribution. This is part of Microsoft's Open Edge strategy, which allows for the extension of the platforms that we provide to the community, by the community, in cooperation with the leading domain authorities - in this case NASA.

The WWT platform is the best astronomical data visualization technology available, and it makes sense that the most knowledgeable members of the community should be able to extend the platform with a variety of components under a mixture of licensing models.

As Alyssa Goodman, a Professor of Astronomy at Harvard University, says: "If only we could travel faster than the speed of light, we could leave our solar system, go past the nearby stars of our galaxy, leave the Milky Way and visit the many galaxies beyond. Until then more and more incredible telescopes, including this WorldWide Telescope, will continue to let us marvel at the wonders of the Universe."

Real Mission Critical

Mark Stone — Mon, 01 Jun 2009 16:15:00 GMT

The 1.0 release of WinBioinfTools might seem like a modest event; as of this writing, the project has
44 downloads. High Performance Computing (HPC) is a small community, granted, and the number of HPC
applications for bioinformatics is a small subset of that. Let's not confuse popularity with importance,
however.

We use the phrase "mission critical" very frequently and somewhat casually within software development. In
talking to a friend about the swine flu outbreak, I was reminded that the phrase has its origin in
military history: an aspect of a mission so critical that failure in that aspect would result in the
loss of life. In the developing world where medical infrastructure can be a fragile thing, information about
the origins or genetic makeup of a virus can be vital. It can be mission critical.

Historically, the developing world has been dependent on developed western countries to do their research for them.
Open source is beginning to level that playing field, though. Using a cluster environment and software
projects like CoCoNUT for gene sequencing and comparison, even university research centers with modest x86
server environments can play in the HPC space. This is important because the research priorities for a
university in a developing country may be very different from the research priorities of a major western
research university. At its best, this is exactly the kind of lowering of barriers to entry that open
source should facilitate.

For all its value, CoCoNUT has two significant limitations. Its license is an academic license, not a fully
open source license. And it runs only on Linux/Unix systems. The latter is particularly important. Research
scientists are not IT professionals, and they should not have to care about the underlying platform on which
their software runs. The spirit of open source is to make software as widely available as possible, and there
is no way to meet that spirit without including Windows Server among the target platforms. Mission critical
demands no less.

So WinBioinfTools makes important steps forward on both fronts. The team at Nile University has released a
GPL-licensed project that "contains a number of programs for Bioinformatics running over Windows Cluster running
Windows HPC server 2008. The current version includes the CoCoNUT system for pairwise genome comparison,
parallel global sequence alignment, and parallel BLAST."

This is a great example of a local software community using open source to make their needs a priority,
and delivering a project that will benefit local software communities in other developing countries with
similar needs. WinBioinfTools puts us one step closer to making scientific computing software platform
neutral, and closer to making Windows Server a first class citizen in the open source world of HPC.

Migrating PHP Apps to Windows

Peter Galli — Fri, 29 May 2009 15:32:00 GMT

Microsoft Malaysia is helping sponsor a competition, known as LAMP2WIN, designed to help ensure that PHP applications run well on both Windows and open source platforms.

The competition, themed ‘World of Interoperability,' is being organized by PHP.net.my and involves migrating PHP applications from the Linux-Apache-MySQL-PHP (LAMP) stack to the Windows platform.

All 20 applications selected were developed for LAMP, with either partial or no support for the Windows platform.

LAMP2WIN contestants will be assigned one of 20 Open Source applications on a round-robin basis across five different categories - blog, forum, eCommerce, wiki and CMS/portal.

Each of the more than 40 contestants will be given a hosted space that supports PHP, IIS and SQL Server, and they will then be able to upload and operationalize their migrated applications to this space at any time during the competition.

These uploaded and operational entries will be viewed and judged by both the public and the official judges. Contestants are also encouraged to blog about their experience while performing the migration, which will help the judges understand the development processes taken.

PHP.net.my founder Ahmad Amran was recently quoted in MIS Asia saying that "as PHP specialists, we realise and utilise the powerful capabilities of PHP, but the shortcomings of some open-source applications available only on certain platforms set off a light bulb in our heads. Why limit ourselves to only one platform when the ecosystem is derived of so many other platforms?"

"This is why we are calling all Malaysian PHP developers to showcase their skills through LAMP2WIN to demonstrate the relevance of interoperability. The underlying idea of the competition is not about selecting the best application, but to contribute back to the Malaysian software ecosystem by providing choice to the people," Amran said.

LAMP2WIN follows a similar competition earlier this year in Japan, designed to get competitive LAMP engineers to increase the volume of technical information around PHP/IIS and application compatibility. The competition was titled "Install Maniax 2008".

A total of 100 engineers were chosen to compete and seeded with Dell server hardware and the Windows Web Server 2008 operating system. They were then required to deploy Windows Server/IIS and make the Web Server accessible from the Internet. They also had to run popular PHP/Perl applications on IIS and publish technical documentation on how to configure those applications to run on IIS.

A total of 71 applications out of the targeted 75 were ported onto IIS, of which 47 were newly ported to IIS, and related new "how to" documents were published to the Internet. Some 24 applications were also ported onto IIS based on existing "how to" documents.

PHP|Tek in Chicago

hjanssen — Thu, 28 May 2009 00:29:00 GMT

Last week I got the perfect excuse to get out of the Planning and Budget process that we are going through right now, attending PHP|Tek, which was a welcome escape as planning and budgeting in any company is usually enough fun to make a grown man cry!

So last week I went to PHP|Tek in Chicago to speak and meet folk from the PHP community. As always, I greatly enjoy meeting the people who write and use PHP, and I have been to and spoken with enough of the speakers at past events that I know a lot of the core people by first name.

Kind of funny that we now have gotten to the point inside of Microsoft that we are almost old hats at Open Source conferences

There were two days prior to the conference where a group of core PHP developers and community people talked about the state - past, present and future - of PHP. It was super cool to be invited to that one!

Unfortunately I was only able to join one of those two days: amazing that flying from Seattle to Chicago takes the better part of a day!

The discussions there where very wide ranging, from whether there will be a PHP 5.4, what 6.0 will bring, which bugs are current show stoppers, where PDO is going, etc. etc.

For me PHP|Tek remains a very nice ‘community' conference, where the focus is on the community of PHP and not the business/vendors of PHP.

These kinds of conferences are the best way to network, and it would take too long to talk about all the people I spoke to. But Elizabeth Smith and I talked about us writing documentation for php.net (I have been wanting to write the ‘how to build PHP for Windows' part) so hopefully look for more documentation written by Microsoft for php.net soon.

As always I talked to a lot of the usual suspects: Scott MacVicar, Andrei Zmievski, Derick Rethans, Sebastian Bergmann, Chris Shiflett, Cal Evans and others.

Oh, and if you are really bored, check out the latest May issue of php architect, which has a bunch of really cool articles about PHP and Windows. Some of them were even co-written by me, which gives you an idea how far php | architect has sunk to have people write articles for them

I just checked out the Website, and the May issue is not posted yet. But everybody who attended PHP|Tek got a copy of that issue in their goodies bag.

I always enjoy giving sessions and the session I did give at PHP|Tek was ‘PHP 5.3 The best PHP on Windows Yet' , and I got some really good feedback. I think I had about 40+ people in my session. People are always surprised to see Microsoft's involvement with PHP and what we have done with the community so far.

It is a talk I have given before. It starts with describing what the organization I belong to (the Microsoft Open Source Technology Center) does and how we work inside of Microsoft. After that I go into some detail about why PHP 5.3 is the best PHP on Windows.

Did you know that, for example, with PHP releases prior to 5.3, the code was build with libraries that were more than 10 years old and for which nobody really had any idea where the source code went? So it was built - linked rather - with object files that were more than 10 years old.

It makes it really hard to fix/improve stuff that you do not have the source code for

Well, pretty much all the issues of the past are now gone. I will make sure I write a blog about what truly went into PHP 5.3 for Windows soon, if the budgeting and planning process doesn't kill me before that point. In the meantime, here is a link to phpfreaks where, a few weeks ago, I posted a bunch of what we have been doing.

One really interesting thing is that there were a lot of Microsoft people at this conference, specifically from the DPE (Developer Platform Evangelism) side of Microsoft.

These are the people who are very much field and customer focused. From my conversations with them, they enjoyed the conference and were glad to get the opportunity to speak with a lot of the OS crowd. It is amazing how much we all have in common once we talk about technology.

Thanks to the people who put on the conference: of course Marco Tabini, the man behind PHP|Tek, but especially Elizabeth Naramore, who is the unsung hero that is the real driver behind keeping PHP|Tek running smoothly!

More Open Source Goodness

Peter Galli — Wed, 20 May 2009 20:05:00 GMT

Tony Hey, the corporate vice president of Microsoft Research's External Research group, used the Open Repositories Conference to announce today the public availability of Zentity and the second version of the Article Authoring Add-in for Word 2007, both of which will be released as open source.

Zentity, previously called Research-Output Repository Platform and code-named Famulus, is a platform that allows institutions to store all of their digital scholarship: papers, lecture, presentations, videos-anything that might be collected by the university as part of the digital output of their researchers and scholars.

Over the past nine months two betas for this have been released, which refreshed the user interfaces and added new controls, and complement the services provided in the package.

The second version of the Article Authoring Add-in for Word 2007 includes new functionality, including the ability to upload directly into a repository - Microsoft's or those of others - via the SWORD [Simple Web Operation for Repository Deposit] protocol.

Support for authoring Object Reuse and Exchange resource maps within the Word environment has also been added, as well as the ability to perform literature searches and to import the bibliographic information in Word with one click, which makes it very simple to quickly add citations into a paper.

A key element of the Microsoft External Research vision is to support the scholarly communications lifecycle with software and services so that data and information flow in a coordinated and seamless fashion.

With regard to the plan to open source these tools, Lee Dirks, the director of the Education and Scholarly Communication team, said that "first and foremost, we're releasing the binaries, but soon thereafter, we'll release both of these as open source. Once they are available, our big push over the next 12 to 18 months will be to build a worldwide community around these assets."

You can find a lot more information on these announcements here.

These moves also follow the March release by Microsoft and the Creative Commons of an add-in for Microsoft Word 2007 that enables authors to easily insert scientific hyperlinks or ontologies as semantic annotations to their documents and research papers.

Microsoft is also making the source code available for the Creative Commons Add-in for Word 2007 free of charge to open source communities on CodePlex through the OSI-approved Microsoft Public License, which lets developers tailor it for specific industries using domain-specific language.

Microsoft Teams up With Black Duck Software

Peter Galli — Mon, 18 May 2009 20:30:00 GMT

Microsoft and Black Duck Software this morning announced an agreement under which projects from CodePlex will be fed automatically into Black Duck's open source KnowledgeBase repository, and which will also will be searchable through Koders.com, a search engine for open source and other downloadable code.

This means that those customers who use the Black Duck KnowledgeBase to leverage, manage and detect the use of open source components in software application development projects, will now get comprehensive coverage of CodePlex-hosted projects, many of which are Windows .NET based.

Developers will also now be able to use Black Duck's Koders.com search search engine for CodePlex projects. While not all of the 9,000 CodePlex projects will be searchable in Koders as of today, most are expected to be by the end of June.

You can read the news release here.

Given that Black Duck's KnowledgeBase is a useful resource for development managers tasked with managing open source code in mixed-source development environments, the addition of CodePlex projects makes this a more powerful development resource, said Sam Ramji, Microsoft's Senior Director of Platform Strategy.

Black Duck, which is a Microsoft Visual Studio Industry and Windows Embedded Partner, scours the Internet, collecting open source and other downloadable code into its KnowledgeBase, a searchable repository of more than 200,000 open source projects collected from more than 4,100 Internet sites. More than 40,000 new projects have been added to the KnowledgeBase since January 2009.

As CodePlex is one of the fastest-growing open source hosting sites, this agreement will make it easier and faster for Black Duck to manage the steady stream of new projects on the site, said Peter Vescuso, Black Duck's Executive Vice president of marketing and business development.

CodePlex currently hosts 9,000 projects and adds about 100 new ones each week. "By teaming with Microsoft, we are assured of comprehensive, ongoing coverage of CodePlex projects in the KnowledgeBase," Vescuso said.

Announcing the PHP SDK for Windows Azure

Peter Galli — Wed, 13 May 2009 00:29:00 GMT

Vijay Rajagopalan, a Principal Architect here at Microsoft, is at TechEd India, where he will demo later this week a new set of interoperability projects related to PHP.

These projects include the PHP SDK for Windows Azure, an open source effort for which Microsoft has provided funding, with development by RealDolmen, whose goal is to provide high-level abstractions that enable PHP developers to interoperate readily with Windows Azure.

The PHP SDK for Windows Azure focuses on REST and provides PHP classes for Windows Azure blobs, tables and queue, helper classes for HTTP transport, AuthN/AuthZ, REST and error management, as well as manageability, instrumentation and logging support.

Rajagopalan will also announce the launch of a series of projects that offer samples and a toolkit that enable PHP developers to include Silverlight controls, Microsoft Virtual Earth maps and IE Webslices and Accelerators in PHP web applications; as well as automatically generated a simple "Create, Read, Update, Delete (CRUD)" PHP application from a table in SQL Server.

These projects, for which Microsoft has provided funding and which are available on Codeplex under a BSD license, are yet another proofpoint of the company's commitment to interoperability, and developers will be happy to know that the first batch of these have already been developed by Accenture.

Read Rajagopalan's full blog here for all the details.

The Azure Services Platform has been designed to be open, standards-based and interoperable, and its support for XML, REST and SOAP standards means that any of the Azure services can be called from other platforms and programming languages.

Microsoft has provided funding for two other SDKs that support third party programming languages: Java SDK for Microsoft .NET Services and Ruby SDK for Microsoft .NET Services so as to facilitate interoperability between the Azure Services Platform and non-Microsoft languages and technologies.

The inclusion of FastCGI in Windows Azure's hosting environment was announced at MIX 2009, and the protocol enables developers to run web applications on Windows Azure that were written using third party programming languages, including PHP. This opens up new options for PHP developers to deploy their applications.

A Technology Preview of the PHP SDK for Windows Azure will be released under a BSD license, while a functionally complete version of the SDK, which will support tables and queues, should be available for download by this fall of 2009, but the team is calling on developers to provide feature requests, test the toolkit, and join the user forum.

So, stay tuned, as there's a whole lot more to come!

Apache Stonehenge: Interoperability at Work

Peter Galli — Tue, 12 May 2009 17:24:00 GMT

The Stonehenge incubator project is approaching its first milestone: deploying the first set of samples and making them work together.

This is a really exciting development and continues to deliver on the project's primary goal: to provide practical applications that span languages and platforms and demonstrate how to achieve interoperability.

Multiple implementations of the Stonehenge Stocktrader sample application, including .NET, Java, PHP, Python and Ruby, have been committed to the repository. You can check the code here.

From a simplified architecture point of view, the Stonehenge Stocktrader application is built as follows:

A User Interface layer delivering the web front end (HTML)

A middle tier layer including a Business Services layer (login, account processing) and an Order Processing layer (buy/sell transactions)

A Data Access layer to provide access to the database for the middle tier layer (Business Services and Order Processing)

And, finally, the database where the application data lives

The work thus far has focused on the .NET, PHP, and Java interoperability scenarios, and the three Stocktrader implementations have been deployed in multiple configurations.

A series of tests were then run, mixing and matching the layers from the three implementations, playing with the configurations and leveraging the Web Services standards, including WS-Security, to provide message integrity and security.

A detailed "interoperability walkthrough" explaining all the different configurations has been posted here, while the full blog post by Kamaljit Bath, a Principal Program Manager in the Interoperability Technical Strategy Team at Microsoft, can be found here.

"Microsoft is pleased with its participation and the progress so far, and this new outcome from the Stonehenge project is very encouraging. With the implementation of the WS-* Standards, we get the benefit of distributed applications and platforms. We recognized that it is not always easy to achieve these goals, but I really feel this type of practical guidance will be helpful for these types of scenarios," said Bath.

The team is also actively soliciting comments and feedback, and encouraging both developers and users to participate in the project to ensure that the project continues to move in a direction that meets real people's needs.

PHP 5.3 RC2 Highly Optimized for Windows

Garrett Serack — Mon, 11 May 2009 16:52:00 GMT

Howdy,

I've been working for many months with Pierre Joye - well, really many people in the PHP community - on getting PHP to run faster on Windows.

Pierre has been working rapidly on upgrading libraries (Pierre pioneered the work to get PHP and its hoard of dependent libraries updated and properly compiling on Windows), replacing old POSIX-emulation code with native calls, patching bugs, and about a million other things, all of which had a huge impact on performance and stability of PHP on Windows.

For my part, I've been spending my time behind the scenes by feeding information to Pierre that he needs, testing, analyzing, and finally by constructing a new build process that enables us to take advantage of some pretty sweet optimization technology in Visual Studio.

Starting today, you can find snapshot builds of PHP 5.3 that are built using my optimized build process on the windows.php.net site.

A few notes:

Over the course of the next couple of weeks, I'll be explaining how this build process works, and making available the tools that make it all possible.

Only the non-thread-safe version is available, so you need to use FastCGI with IIS in order to use it.

Since this is a radically different build than the ones that had been traditionally used to create the Windows PHP binaries, you should download the binaries and test with them, but you probably should avoid using them in production just yet.

If you have any feedback about the builds, leave me a comment, or email me.

Helping Facilitate Open Government

Peter Galli — Wed, 06 May 2009 19:03:00 GMT

Microsoft will announce on May 7 an initiative to help government agencies and developers publish and interact with their data in Windows Azure, the company's cloud computing platform.

One of these is the Open Government Data Initiative (OGDI), a cloud-based approach to housing public government data in Windows Azure, making it accessible in a programmatic manner via open standard protocols and application programming interfaces.

The source code for OGDI is being made publicly available through CodePlex, Microsoft's open source hosting site, so that developers may reuse it and provide feedback. Sample code is also being provided for technologies widely used on the Web, including PHP, Python, Flash, JavaScript, and Silverlight.

This initiative helps to provide government with tools focused on increasing responsiveness and access to critical services, streamlined processes and services.

For their part, Microsoft and its partners have developed a robust enterprise architecture approach that enables agencies to meet the technology requirements of government mandates with a familiar set of tools - built on an enterprise-ready, scalable, and easily-managed software-powered architecture.

So, in short, the goal of ODGI is to reduce the cost of publishing government data, and simplifying data access by leveraging cloud computing and open standards.

More information on Microsoft's Open Government Data Initiative can be found here.

To see an implementation of a data service in Windows Azure, using a sample of publicly available government data, visit this reference beta site.

These moves are part of Microsoft's ongoing open government efforts aimed at helping government organizations meet goals of transparency, participation and collaboration, particularly as an ever increasing amount of data becomes necessary and available.

As such, new methods need to be opened up to allow interaction with that data, and Microsoft's OGDI is designed to help public sector entities meet these goals.

This software, which underscores the importance of programmatic access to government data rather than having to download it, will give developers the ability to write programs that access data via Web-friendly programming methods without having to download or host the data; and let them write applications using any technology via open standards.

It also provides easier access to a broad array of government datasets, enabling the building of new and unique applications, while governments will be able to automatically refresh data without having to buy and maintain servers.

Cloud computing is the ideal platform for government data, and the technology is finally available to make it happen, says John Miri, Senior Fellow at the Center for Digital Government.

"The qualities that government looks for in an information management platform - things like flexibility, scalability, security, performance, and cost efficiency - are all better in a cloud model. As we see demands for government to become more transparent, collaborative, and interactive, a shift like this in technology architecture just has to happen, " he says.

For governments to become truly open, citizen access to public data in standards-based and interoperable ways is essential at all levels of government.

Given that most federal, state, local and education entities implement the American Recovery and Reinvestment Act (ARRA), they can meet open government goals of oversight, transparency and accountability through cloud and on-premises solutions such as Microsoft Stimulus360, which helps public sector agencies track, measure, and share information about federal stimulus programs through graphical dashboards and maps.