Software Development Times Blog

Microsoft: Open source protects Silverlight investments

drubinstein.nospam@nospam.bzmedia.com (dworthington) — Fri, 20 Nov 2009 09:58:00 -0500

Microsoft and the Mono project are working to increase interoperability between Silverlight and Mono Moonlight, said Brian Goldfarb, director of developer and user experience platforms at Microsoft, during an interview at Professional Developers Conference this week. Moonlight is an open-source implementation of Silverlight.

Goldfarb and Mono project lead Miguel de Icaza had a lively discussion about some of the legal technicalities concerning the work at a Tuesday evening cocktail party. Goldfarb told SD Times that he and de Icaza were commited to driving the work forward, and that an announcement will be made in the coming weeks (or months).

When asked whether Silverlight was part of the open Web, Microsoft executives including chief software architect Ray Ozzie and Server and Tools Business head Bob Muglia pointed to Silverlight as a testament to the platform's openess. Goldfarb added that Mono Moonlight protects customers' investments in Silverlight, so that vendor lock in will "never apply."

Whether Microsoft will actually open up all of its Silverlight/.NET libraries remains to be seen.

We are the big data problem

drubinstein.nospam@nospam.bzmedia.com (ahandy) — Thu, 19 Nov 2009 16:53:00 -0500

Like a kid waiting for Christmas to come, I have been watching the Mahout project with great anticipation. When you toss around the concepts of map/reduce and machine learning, there's an awful lot of potential for radical ninjas to ensue. While the magical science-fiction world of artificial intelligence is still in a fetal state within our reality, it is, nonetheless, a growing science.

One of the things humans are discovering about making real thinking machines is that a wealth of experience goes a long way. It's the same for humans and machines: The more memories we have, the more we are able to structure thought processes based on those memories, and to learn from them. For computers, memories can be thought of as datasets. And the bigger the dataset, the more understanding can be extracted.

We all know about the BIG DATA PROBLEM. But if I were going to write something here in the guise of a souped-up VC, or grizzled startup veteran, I'd say the big data problem, when observed from the right angle, is more like a big data opportunity.

Imagine how much optimization information you could pull from six years of user logs? Need metrics, anyone? Try juicing all of your user stats. Not just this month's stats, but all of them. Why not throw in the databases of old user info you got from that newly acquired company you're still digesting? Hadoop is the place to put all of this stuff, as we should all know by now. But the data in Hadoop is only as good as the people who extract meaning from it. And with petabytes of data available at once, what human being could ever comprehend, much less query that mound of information? We can, and do, poke at it, and pull massive amounts of data from it. But there is the potential for infinitely more meaning to be derived from our data.

The big data problem is not a problem with the machines anymore. They're ready, thanks to Doug Cutting, the Apache Foundation, Yahoo, Facebook, Cloudera, and all the hordes of other Hadoop committers out there. The big data problem is a problem with us. These datasets are just getting too big, and we can't spend our days reformatting them, writing connectors for them, or passing them through oodles of filters.

Today, enterprises spend most of their time integrating, not coding. They're making this database work with that database, and wrapping this system in that one. Most of these activities are performed to transform data from one form into another. The customer database has to be able to exchange information with the new databases brought in through company mergers. The HR system has to talk to the ERP system, and both have to be backed up according to processes that change daily. It's all about taking information, transforming it programatically, and passing it on. And the way we do it now is ludicrously inefficient.

No, the future is not in "integrating" big data. The future is in teaching the machines to understand big data for us. We've already done this in places that aren't generally associated with artifical intelligence. Take a look at any corporate firewall or load balancing system, and you can see how the rules have moved away from being simple laws into ever more complex Turing-complete languages of their own. Most of the business processes you work with every day are essentially organically grown machine rules; they've evolved out of experience with what works for the organization. Companies are, after all, organic hive minds with mechanical arms. Large distributed systems, as it were.

If we can build enough common tools to create machine learning on top of big data, humanity as a whole will be remarkably changed for the better. Imagine machines able to identify cancer trends through the analysis of patient data correlated with weather patterns and bird migration. Who knows what sort of broad connections like that could exist in our world? Perhaps someone will teach the machines to look through our software repositories and learn how to write code by understanding every check-in, every rewrite, every refactorization...

Of course, this is all still pie-in-the sky. Mmmm, pie. Pie is good. But you can't make pie without first making dough. And before you make dough, you have to crush grain into flour. And harvest the grain. We all currently exist somewhere in the latter portion of that metaphor, as far as building big learning machines is concerned.

No matter: The folks behind the Mahout project are working on making that dough. Mahout began as a sub-Lucene project, and has morphed into the first real attempt at building a foundation for machine learning on top of Hadoop. Version 0.2 of Mahout was officially released on Wednesday, and there's a number of interesting new features:

Significant performance increase (and API changes) in the collaborative filtering engine
K-nearest-neighbor and SVD recommenders
Much code cleanup, bug fixing
Random forests, frequent pattern mining using parallel FP growth
Latent Dirichlet Allocation
Updates for Hadoop 0.20.x

The point is: Pay attention to Mahout. Maybe go contribute some code. It's an interesting project that, I feel, is really working towards an inevitable future.

APIs movin' on up

drubinstein.nospam@nospam.bzmedia.com (jfeinman) — Wed, 18 Nov 2009 15:53:00 -0500

I attended the Business of APIs Conference earlier this week in New York City, held by API services provider Mashery, and heard some interesting thoughts on what APIs are all about in today's world (literally today, not two years ago or in the ancient times of 5-10 years ago in Technology Land).

ProgrammableWeb founder John Musser was the first speaker of the day, and later in the afternoon, he told me that there's often a fear or risk involved in opening up an API. To combat that fear, organizations should open APIs gradually.

"Open up pieces of functionality one by one and see how it goes," Musser said. "And then take the next step. You don’t have to go straight in to the point where you’re opening yourself up to things you might not be prepared to handle."

Chad Dickerson, CTO of online handbag marketplace Etsy and former Yahoo employee, made the point that there is always room for improvement when it comes to APIs. "A good API is never finished," he said.

"In the same way that you launch a website and are always adding to it, I think the same thing is true for APIs. If you’re starting an API, you can’t just leave it. It needs to grow as your business grows."

More on Silverlight 4

drubinstein.nospam@nospam.bzmedia.com (dworthington) — Wed, 18 Nov 2009 02:29:00 -0500

Multiple sources have said that Microsoft will be revealing Silverlight 4 at Professional Developers Conference tomorrow. Some features include expanded video support and more desktop functionality. Expect more access to local resources. The release can be described as 'Silverlight 3 but better.' There may also be an announcement involving Mono Moonlight.

IE9 and Silverlight 4 will surface at PDC

drubinstein.nospam@nospam.bzmedia.com (dworthington) — Mon, 16 Nov 2009 22:43:00 -0500

Microsoft will be outlining Internet Explorer 9 and Silverlight 4 at Professional Developers Conference. Expect more depth about Silverlight. Silverlight 4 is Wednesday's keynote.

Gartner Magic Quadrant case dismissed

drubinstein.nospam@nospam.bzmedia.com (ahandy) — Thu, 05 Nov 2009 14:55:00 -0500

The lawsuit filed by ZL Technoogies alleging that Gartner favors its paid clients when choosing who gets into the magic quadrant was dismissed by a judge before the real trial began, yesterday. The dismissal, however, can be refiled with some amendments, according to the judge. Here's what ZL Technologies CEO, Kon Leong, has to say about the decision:

"While we are disappointed that the court has dismissed our lawsuit as filed, we are pleased that it has given us leave to amend our complaint, over Gartner opposition. We believe the market should take note that the defense on which Gartner prevailed was its argument that its reports contain “pure opinions,” namely, opinions which are not based on objective facts. In ZL’s view, that is directly contrary to the statements Gartner makes to its customers when selling its allegedly sound research. ZL intends to amend its complaint and refile within 30 days.

"ZL believes that Gartner’s overwhelming influence on large corporations’ purchasing decisions, and its inaccurate ratings, including its bias in favor of large vendors, combine to pose major competitive hurdles that hurt smaller innovative vendors across all technology sectors. The harm falls not only on new and innovative companies like ZL, but on the enterprise customers who receive faulty purchasing advice, and as a result overspend on inferior technology."

ApacheCon Live

drubinstein.nospam@nospam.bzmedia.com (ahandy) — Wed, 04 Nov 2009 20:39:00 -0500

Just a quick note to all of you out there in the tubes that ApacheCon is going on now. I am off to the 10th anniversary party, here in my home town of Oakland. It should be a fun bash, especially because Microsoft and HP are some of the sponsors. The conference continues tomorrow, but the party goes all night tonight! I've uploaded our pictures of the event. they can be seen on Flickr. Oakland Mayor Ron Dellums also declared today officially ASF Day in Oakland.

Beware of ldd

drubinstein.nospam@nospam.bzmedia.com (ahandy) — Mon, 26 Oct 2009 13:37:00 -0500

If you use Linux and you program, you've probably used the ldd command to track down a dependency or two. If you use it with any frequency, you definitely need to read this article. Peteris Krumins explains that ldd is not an innocuous little utility, devoid of malicious possibilties. From the blog entry:

For example, you can put a malicious executable in ~/app/bin/exec and have it loaded by ~/app/lib/loader.so. If someone does `ldd /home/you/app/bin/exec` then it’s game over for them. They just ran the nasty code you had put in your executable. You can do some social engineering to get the sysadmin to execute `ldd` on your executable allowing you to gain the control over the box.

Proposal to solidify Python

drubinstein.nospam@nospam.bzmedia.com (ahandy) — Wed, 21 Oct 2009 16:24:00 -0500

Guido van Rossum sent an e-mail to the Python-Ideas mailing list today, in which he proposed that Python 3 be the Python forever. Well, almost forever. I'll let him explain in his own words, below:

I propose a moratorium on language changes. This would be a period of several years during which no changes to Python's grammar or language semantics will be accepted. The reason is that frequent changes to the language cause pain for implementors of alternate implementations (Jython, IronPython, PyPy, and others probably already in the wings) at little or no benefit to the average user (who won't see the changes for years to come and might not be in a position to upgrade to the latest version for years after).

The main goal of the Python development community at this point should be to get widespread acceptance of Python 3000. There is tons of work to be done before we can be comfortable about Python 3.x, mostly in creating solid ports of those 3rd party libraries that must be ported to Py3k before other libraries and applications can be ported. (Other work related to Py3k acceptance might be tools to help porting, tools to help maintaining multiple versions of a codebase, documentation about porting to Python 3, and so on. Also, work like that going on in the distutils-sig is very relevant.)

Note, the moratorium would only cover the language itself plus built-in functions, not the standard library. Development in the standard library is valuable and much less likely to be a stumbling block for alternate language implementations. I also want to exclude details of the CPython implementation, including the C API from being completely frozen -- for example, if someone came up with (otherwise acceptable) changes to get rid of the GIL I wouldn't object.

But the moratorium would clearly apply to proposals for anonymous blocks, "yield from" (PEP 380), changes to decorator syntax, and the like. (I'm sure it won't stop *discussion* of those proposals, and that's not the purpose of the moratorium; but at least it will stop worries elsewhere that such proposals might actually be *accepted* anytime soon.)

--Guido van Rossum (home page: http://www.python.org/~guido/

Darl McBride ousted from SCO

drubinstein.nospam@nospam.bzmedia.com (jfeinman) — Wed, 21 Oct 2009 10:36:00 -0500

Looks like the sun has set on Darl McBride's controversial tenure with The SCO Group.

As part of its restructuring under a U.S. bankruptcy court trustee, SCO has given CEO Darl McBride his walking papers. McBride was seen as the main driving force behind the seemingly endless lawsuits against Novelll, IBM and Red Hat over who is the true owner of UNIX. COO Jeff Hunsaker, CFO Ken Nielsen and General Counsel Ryan Tibbitts remain, at least for now.

We will have more shortly.