benjaminm's blog

Office Server 2007: Can it help me write less code?

Wed, 14 Jun 2006 13:47:32 GMT

Recently I've been doing a "tour of the trenches", helping a major client with their web-based applications for the Insurance industry. It's taught me that there's a lot of development work that goes on in companies that doesn't relate to the overall aim of delivering business value through software, specifically around the building of "platforms". Brad Abrams noted something similar when he went on his road trip, there are a lot of companies building custom platforms.

From my experience a lot of these platforms turn out badly since most companies can't afford the time or money to build and create successful platforms. Rather than writing platforms, I'm looking for ways to use Other Peoples' Code to build solutions that deliver better business value. At TechEd this week I'm interested in evaluating Office 2007, and particularly the server platform, as a platform to build on.

Looking at the domain of Insurance applications it's easy to see why lots of developers want to write a platform. The basic flow is similar in structure to many other types of financial trading:

An application form, with one or more wizard-like pages, containing controls with single- and cross-field validation.
The application form is sent to a rating engine (usually authored by Underwriters) which determines whether the business will generate a quote and if so, what the value of that quote is.
After a quote is generated there are various sequential and state-based workflows around that can occur (e.g. accepting the quote, revising the quote, "binding" the quote to make it a policy).

With that in mind I'm currently looking at Office Server 2007 as a possible platform to build on. Here's a sketch of what I'm looking:

Create the application form in InfoPath (giving me the single- and cross-field validation), then publish it as a HTML form using Office Forms Server (there was a session at PDC as well).
Use Excel Services to host the spreadsheet that contains all of the logic for calculating the Quote. Once we agree on a set of named cells/ranges in the spreadsheet, the underwriters can keep control of it.
Use SharePoint's support for Workflow to implement the Workflow.

Obviously I'm just sketching at this stage (I still have questions about licensing, performance and flexibility are key) and I need to do some prototyping, but overall I'm interested in the possibility of being able to build basic Insurance trading applications on top of Office 2007 as a platform, dramatically reducing the amount of code I will need to write.

Server-side Excel: changing the face of financial apps?

Mon, 12 Jun 2006 16:47:15 GMT

I'm currently pondering whether the new server-side capability of Excel 2007 could radically change the face of many financial application. Excel is so widely used in finance that many companies could describe it as their platform. I'm currently involved in writing a web application for a group who use Excel extensively and I frequently think if we could work more harminously with this spreadsheet we could build the application for much lower cost but equal or higher business value.

In the keynote last night they demonstrated the Windows Compute Cluster Server 2003 running Excel Services to perform Monte Carlo simulations using Microsoft Excel. Initially I thought this was implausible, imagining the response if someone asked, "Can we set up a cluster server in order to run my Excel spreadsheets?", but on reflection I think this could make a great deal of sense. Even though I'm suspicious about the efficiency of this kind of solution, viewed from a different perspective I think it could represent great business value.

It's amazing to think that Windows and Office could lead to a scenario where a front-office trader could write software that can be executed in a cluster - an arena that previously seemed the domain of C++ and hard-core developers. On the one hand it scares me, since there will be less custom C# development for me to do, but on the other hand I like it a lot since re-writing functionality that Excel already implements takes a long time and doesn't give as big a "bang for the buck".

Obviously I need to do a lot more research on Excel Services. There's a good introduction here and some more technical details on how you can call Excel through web services.

Ray Ozzie: Services Disruption and the need for 'Client Server Service Synergy'

Mon, 12 Jun 2006 16:13:41 GMT

TechEd Boston started Sunday night with Ray Ozzie admonishing us that there's a 'Services Disruption' coming and proposing the slightly tongue-twisting 'Client Server Service Synergy' (and the lesser-used companion phrase 'Client Server Service Symmetry') as the way forward, with Microsoft offering more services in future.

Ray with the usual dramatic structure of a presentation (much like what is recommended in the "beyond bullets" book), setting up that we're heading for a new disruption in the way that we work. He did the standard mainframe -> mini computers -> micro computers -> client/server -> the internet ("the mother of all disruptions") -> peer-peer set of disruptions, personalised through his own life experiences with Lotus Notes and Groove. There was the standard argument that we need to be on the lookout for new disruptions.

The drivers that are changing things for today are the promise of multiprocessors (32 to 1000 CPUs inside a single machine - up to "oodles" in Ray's words) the reduction in storage space and the growing ubiquity of bandwidth. The interesting point that Ray highlighted was the fact that in the past it was research and corporate environments that drove datacentres, whereas today these were being driven by experience with consumer market through search, advertising and consumer shopping sites. Ray's point was that the benefits of investments in these environments will be felt more widely.

He wasn't terribly specific about how this would happen, though he was trying to position Microsoft's approach as a mix between the client services approach and external services in a "client server service synergy". He showed windows desktop search searching over the internet, SharePoint sites and the local machine as example of this. He also showed Microsoft Dynamics using 'business mashups' with Windows Live virtual earth. He also mentioned the online/offline architecture of Groove. There was much mention of the Windows Live set of products. There was a Windows Live Identity that I hadn't heard of before - I'm assuming it's a rename of Passport? It might have been the jetlag but it felt like another moment of marketing fatigue (apparently MOM will become System Server Operations Manager as well).

The presentation hall felt very much like an aircraft hangar and the regular sound of planes overhead reinforced this. Ray was the first keynote speaker I've seen wearing a suit jacket on one of these talks. Obviously the chip-implant wasn't powerful enough to convert him to the standard casual 'uniform' of a blue shirt and a pair of chinos.

DDD3: Deadline for speaker submissions extended until Monday

Thu, 16 Mar 2006 12:14:59 GMT

We've extended the deadline for speaker submissions until this Monday 20th March to allow final submissions to be put forward. We have an excellent list of submissions so far, but we'd like to allow everyone enough time to submit a session. Attendee voting will start on Tuesday.

DDD III on Sat 3 June: Call for Speakers!

Wed, 15 Feb 2006 23:05:44 GMT

Developer Developer Developer, the free day-long conference for developers, by developers is back for a third time on Saturday 3 June. The call for speakers is open. If you've got a topic that you'd love to share with other developers in the community, please submit a session.

Attendee registration will open later, but you can always view some of the videos from DDD II last October.

I can see clearly now: NUnitAddin and NCoverBrowser

Fri, 03 Feb 2006 13:03:11 GMT

Jamie Cansdale's done it again- he's added Test With -> Coverage to the TestDriven.NET Visual Studio add-in. A picture's worth 1,000 words:

Once the tests are run it launches Grant Drake's (aka KiwiDude) re-write of Jeff Key's NCoverExplorer. The NCoverExplorer then displays full coverage stats as well as illustrating the coverage of code blocks. Here's the output from a run over the NUnit code:

This is awesome work that will make it easier to understand and work with code.

Microsoft getting Agile with Windows

Mon, 26 Sep 2005 21:35:00 GMT

I came across this interesting article on how Jim Allchin has championed developing Windows Vista in a more agile way. He mentioned several times in his PDC keynote that Microsoft had changed the way they were developing software, but didn't provide many details. Here's an example:

We feel very confident about broad availability [of Windows Vista] by the end of 2006. Now, why do we feel that confidence? We feel the confidence because we re-did the way we were building Windows. During the last two years, we completely re-engineered engineering

Microsoft have always prided itself that it understood how to ship software, so it's interesting to see them reflecting on the problems with their current approaches and how they can be improved.

Here are some interesting points:

They have reduced the amount of time spent integrating components together by sorting out the dependency between components. Knowing these dependencies, and working on minimising them (sounds like architecture to me), is enabling Microsoft to leave up components that don't pass 'quality gates'
They are reducing the idea that 'pulling an all nighter' and being a heroic developer is something to admire:

In 2001 Microsoft made a documentary film celebrating the creation of Windows XP, which remains the latest full update of Windows. When Mr. Allchin previewed the film, it confirmed some of his misgivings about the Windows culture. He saw the eleventh-hour heroics needed to finish the product and get it to customers. Mr. Allchin ordered the film to be burned.

Bill Gates being was worried about upsetting the status-quo, 'taxing' the developers with process and was concerned about backlash from the developer teams.
They are finding a lot of value in automated testing tools. One thought that has been bouncing around my head since the PDC is the comment that Jeffrey Richter made about the problem where a method was added to a class as an instance method, when all other methods were static and the constructor was private, which meant that the instance method could never been used. As Jeffrey Richter said, it's clear that the developer had not written a single test to check his (I know it was as a he - the guy owns up to the mistake in Brad Abrams' excellent new book 'Framework Design Guidelines') work! Clearly Microsoft have a way to go in getting test infected.
Reducing the build time of the product has meant they are getting greater feedback more quickly.

Recently I've been in the position of working with an ex-IBM Architect and an ex-Microsoft Dev Lead. It's been interesting to see the differences in backgrounds and perspectives, which broadly fall into the differences between Architecture and low-level system Development. An example was a discussion about finding the distance between a point and a rectangle. The Developer had come up with a patented approach to solving the problem in an extremely efficient way, but the Architect wanted to know why the problem only considered rectangles, and whether it couldn't have been abstracted further to consider many different types of shapes.

The discussions have helped me see the value of both approaches, but overall, the need to architect a solution well, since it is easier to find faster algorithms, but if things haven't been architected well it's a great burden to refactor the structure of the code.

Indigo helps end the POX/REST debate

Wed, 14 Sep 2005 19:52:58 GMT

In the keynote yesterday, Don Box and Chris Anderson showed how Indigo can be used in a REST style by sending Plain Old XML (POX) over HTTP. It’s a great testament to the quality of the Indigo extensibility architecture that they could do this. By supporting both the REST and SOAP styles it takes the heat out of the debate that one is better than the other.

One of the Hands On Labs goes through the code they used in detail. The example uses a HTTP GET request to return a plain XML payload, in this case a RSS feed. An extra level of detail was the fact that the RSS added a custom element indicating an 'adult content rating' system, which was signed. There were several interesting points from the Hands On Lab.

One complexity around supporting a REST style with Indigo comes from the fact that the HTTP GET request does not have any WS-Address-style action. To get around this you can use the mapAddresssHeadersToHttpHeaders attribute of the HttpBindingElelement to make the URI of the HTTP GET request the To value on the Indigo Message object.

Since no action is specified on the HTTP GET request the ServiceContract needs to trap any unmatched action using the OperationContract's Action named parameter:

public interface IRestStyleRequestResponse
{
[OperationContract(Action="*")]
Message GetRequest(Message request);
}

Since the HTTP GET request on the HTTP transport in Indigo usually returns a description of the service, this needs to be turned off using the Description property of an instance of the ServiceHost type:

// service is an instance of ServiceHost
// stop the HTTP GET returning the site description.
service.Description.Behaviors.Remove(typeof(ServiceMetadataBehavior));

To hook in this binding a custom BindingElement extension was written. The lab walked through creating a IChannelFactory and IListenerFactory and the rest of the necessary objects as part of the WCF stack. Just before the transport binding element in the Binding stack there's a custom encoder which does the job of writing out the Message to the wire as plain XML, rather than a SOAP message (as Don Box said yesterday, 'it lathers the SOAP off the message on the way out and SOAPs up the message on the way back in').

Bill Gates' PDC Keynote: Office 12 and the end of menus

Tue, 13 Sep 2005 19:46:42 GMT

Office 12 was unvieled for the first time during the Bill Gates' PDC keynote. The main thing of note was a simplified user interface devoid of menus and toolbars and replaced with task-focussed tabs and the 'ribbon' bar. The big question: does this represent a great leap forward in goal-focussed usability or will this be the 'New Coke' moment for Office?

Bill's initial comments:

Promise of previous PDCs has been realised. 2000 was about .NET and XML Web Service, 2003 was Avalon (Windows Presentation Foundation) and Indigo (Windows Communication Framework) and Longhorn (Vista). A shame that Hailstorm and ObjectSpaces have been so quickly forgotten, but software is a cut throat game.
.NET is the most popular development platform in the word
XML has been core and has gone through three phases: surface support in terms of tools that worked with it, then platform level support in terms of .NET and XML web services, then core in terms of integration with SQL Server and Office.
XML has become persuasive. RSS has given us notifications, XML Schemas are being used to create industry schemas and now with WS-* the enabling of protocols to handle security and other requirements.
Standard comments about 'an exciting time', 'still not halfway through the PC revolution'. "Best industry in the world", "exciting times"
PC shipments are up 15% on last year to 200 million units a year.

Funny things that came up on the closed captions on the video screen as Bill spoke:

'Intel a mir a cash mode' - 'IntelliMirror cache mode'
'Things like our asses are driving up to do this' (RSS)
Sin yer jistic (synergistic)

Overall I was incredible impressed that the subtitles kept pace and did such an accurate job.

Windows Vista Demo

"Clarity" Demos:

Task bar now shows preview of documents when you mouse over them.
Alt+Tab now shows a 'flip view' that shows a strip with the same document previews.
Windows Key and Space Bar shows a 'stacked' window view with the windows stacked three deep.
Quick search shows in all explorer views, plus the desktop sidebar. Interestingly it now appears at the bottom of the start bar. It will show search results in place and search the desktop and the internet.
The thumbnail of files shows up in explorer view.
The search on the explorer will search document metadata (e.g. author) and text - although unfortunately there was no visual highlighting of where the search appeared.
Virtual folders are supported. They are defined with XML files (simple elements like cope and tags) and allow you to see all documents on the PC. These can be organised by metadata in a couple of new ways (stack, keywords, author).
Documents can be 'painted' with metadata by dragging and dropping.
The side bar (think side bar, same clock as PDC 2003) hosts gadgets, written in DHTML, script or WCF).
Gadgets are a new feature that allows information to be displayed on a laptop lid (such as flight information, or ability to control music). Some need the laptop to be powered, some don’t.

"Confidence" Demos:

Phishing sites have increased 500%. IE 7 has some new features to show phishing sites. For example, if the site uses an IP address the address bar turns yellow and there is a security icon that warns this may be a phishing site.
Clicking on the security warning allows you to report this site as a potential phishing site.
Microsoft is going to host a phishing site review group that will investigate the possible phishing site votes (and similar, 'this is not a phishing site' link)
The dynamic protection service which powers this feature is opt-in. If a site has been reported and judged to be a phishing site then the address bar goes red and comes up with a warning.
IE7 also has tab, as well as an 'all tabs view' where you have the ability to view all open tabs in a thumbnail view. They can be saved as a common set - so you could save your favourite 5 sites as a set and open them up each morning.
RSS is built into the browser. This works with the platform RSS store so the sidebar can display the RSS feeds as well. Viewing the xml feed behind the RSS shows up with a nicely formatted page.
Microsoft CRM is using RSS to send notifications via RSS.

Office 12 Demos
Word

Word used to have 1500 commands and 35 toolbars. 9 out of 10 request for features in Office were features already in the product.
No more toolbars or menus! There's now a 'ribbon' area that is task based - so you might be 'insert'ing in word or using tables.
The keyboard shortcuts are still available, they just don't seem visible on any of the screens.
The team think that 60 - 80% of the features are available in the new tabs.
The tabs do make some tasks, like adding a header or footer much easier to use.
There are improved tool tips with more text to explain the features.
Using the Tables tab in Excel allows you to format tables in Excel. Selecting the table formatting example shows the formatting in place (I didn't see how to undo this command)
Apparently they are finding people don't need much training to use the product. They 'don't need training wheels'. It will be interesting to see how this flies with large company's IT directors.
Word has a nice feature to insert drop-downs, showing the formatting in place.
There is still a File menu. It has useful features like Finalize that let you clear up hidden text, comments and other features.

PowerPoint

Improved shape support - automatically converts bullet points into different graphics (cycle of boxes, graph with points on them).

Outlook

Still has the command bars!
Is now focused around a to-do list. Now easier to create tasks off email. Ability to flag email with a time to follow up so that they appear in the tasks. Should improve some of the Getting Things Done style work.
RSS feeds are built in.
Support for different office documents and SharePoint emails.
Attachments can be viewed in place.

Thoughts on the new interface
It will be interesting to see the responses since it is quite a change from the current Office user interface, though I think it is a positive change - ever time I visit my dad he always has 17 toolbars in word, with one button on each of them because the toolbars terrify him so much. The new user interface will protect him from that kind of advanced customisation.

The user interface definitely seems to work much better than it would appear when initially viewed. It will be interesting to see how corporate customers react and whether they think the new interface will be as intuitive as it is claimed, or whether the possible re-training costs might be an inhibitor to adoption.

Why writing an Operating System is Hard

Mon, 12 Sep 2005 18:16:59 GMT

At the end of Jeffrey Richter's talk yesterday he showed the depth to which Microsoft has gone to ensure that Windows runs all applications well. He an exe file called armymen.exe (here's another mention of it) that was actually just a renamed copy of Notepad.exe. It had three subfolders, each with a zero byte file in it. When he clicked on it, Windows XP resized the screen to 640 x 480 at 16 colours and disabled that ALT + Tab task switching. Closing the task returned the screen to normal.

Under the covers, Windows has some hard-coded logic that checks for the name 'armymen' and the related directories and files and adjusts the screen accordingly.

This highlights the lengths that Microsoft have gone to in order that all applications continue to run. It also makes me glad that I'm not writing operating systems.

Jim Johnson on System.Transactions

Mon, 12 Sep 2005 15:49:44 GMT

Jim Johnson showed how System.Transacations will allow developers to use Transactions to write more reliable software in simple way that will work across different transaction coordinators while ensuring that performance cost of distributed transactions is only paid if the code uses them. I had a great time talking with Jim at the last PDC and it was exciting to see him demonstrating this technology that will ship with .NET 2.0, as well as a demo of the transactional file system that's coming with Longhorn Server.

Transactions: Ensuring reliability and resilience
Jim spoke about the philosophy behind the System.Transactions work. He views transactions as a reliability aid that helps developer write resilient applications that can recover from errors and return to a consistent state, even in the face of scalability and concurrency challenges.

Jim showed two methods that were both written to correctly handle multithreading and concurrency, but that they may not work correctly if one depends on the other, since it would require code to trap errors and handle rollbacks. He argued that transactions provide a simple way of ensuring that the two could work together in an atomic way, behaving correctly even in the case of errors, without a lot of complex code, which is the major use case for System.Transactions.

System.Transactions: an efficient, unified managed code object model
The goals behind System.Transaction are to make transactions ubiquitous. In order to do that they had to make using transactions simpler for developers and overcome the perception that they were too slow. They have answered the complexity problem by designing a managed code object model in System.Transactions that uses the same code to work with both local in-memory transactions and distributed transactions. They have reduced the performance issues by ensuring that transactions only enlist the necessary resource managers.

Using a transaction involves some fairly simple code:

using (TransactionScope scope = new TransactionScope())
{
   // Do something

   // Ensure that the transaction doesn't roll back
   Scope.Complete()
}

The using statement provides a convenient language wrapper around the use of the TransactionScope object. The developer only has to explicitly signal the success of the transaction, otherwise the transaction will automatically be rolled back once the using statement block exits. The benefit to developers is that the code can be dramatically simplified since the developer doesn't have to write special-case clean up code in the catch block to recover from a problem.

The same code can be used to work with in-memory transactions and database or MSMQ transactions. With .NET 2.0 it is possible to use transactions without having to use the MSDTC, which makes transactions incredibly fast. If the code uses resources that involve distributed transaction coordinators these will automatically be brought in. So in the above code, if there was a call to a database then it would automatically pick up the existing transaction. >

// This transaction scope is similar to the COM+ Transaction.Required - it will inherit an existing transaction or create a new one if necessary
using (TransactionScope scope = new TransactionScope())
{
    // The SqlConnection will pick up the existing ambient transaction
    using (SqlConnection conn = new SqlConnection())
    {
        // ...
    }
    scope.Complete();
}

Longhorn: transactional registry and file system
Jim showed a demo of the transactional file system in Longhorn that used the same code as above but ensured that a file was only written to the file system if the transaction was successful. This will be a great development benefit, since doing this today requires that developers have to write their own compensating resource transaction. Currently it still has dependency on the DTC with a Kernel Ambient object, but this will be factored out by launch.

Overall I think that System.Transactions is an excellent addition to the .NET developers toolkit, since it makes it easy to guarantee that components will behave correctly even when faced with an error.>

Jeffrey Richter: The Good, The Bad, The Ugly of C# 2.0 and .NET 2.0

Sun, 11 Sep 2005 22:41:51 GMT

Jeffrey Richter combines a deep technical knowledge of .NET with a great sense of humour and honesty about where things went bad, to provide a useful overview of what's new in C# 2.0 and .NET 2.0. I find sessions like this useful in learning the story or intention behind features in the new technology, in a way that helps me understand them better.

The morning session covered an introduction to the new features in .NET 2.0 mostly around generics and anonymous methods. Seeing some demos on the new yield statement helped me understand more about how it works. He showed how you could use the yield statement to recursively walk all of the files and directories in the file system. The demo was simply writing out the paths to the console window and as it was running there were obvious pauses in its performance. Jeffrey said that these pauses were likely to be due to garbage collections, which highlighted that the ease of use of the yield statement came at the cost of performance (a class was created per directory/file which put a lot of pressure on the working set and garbage collection). Jeffrey mentioned the C# team have a solution which reduces the performance costs of iterators but it wont ship until after C# 2.0.

He mentioned that another approach to getting the flexibility provided by the yield statement without the poor performance, was to use generics and anonymous methods instead. For example, many of the generic collection types in .NET such as Array and List have useful generic delegate types, such as ForEach(Action<T>), FindAll(Predicate<T>) as Sort(Comparison<T>).

As an example, I can sort a list of string types and print them to the screen in just two lines of code using this techniques:

List<string> names = new List<string>(new String[]{ "Timothy", "Benjamin", "Samuel" });

names.Sort(String.Compare);

names.ForEach(Console.WriteLine);

Whilst I've enjoyed these methods before I did enjoy hearing Jeffrey's positioning of these techniques as a pattern to use instead of the yield statement. Jeffrey also mentioned that Microsoft had received many requests to add richer generic functionality to the generic collection classes. His company, Wintellect, have been engaged by Microsoft to write these collections and make them available for free download, and they are currently available as the PowerCollection library. Jeffrey believes that this functionality might eventually be made part of the Framework Class Library in future versions of .NET.

The great thing about Jeffrey's presentation is that draws on his experience as a consultant to the CLR team. For instance, he mentioned that having different security levels on properties, a 'new' feature for C# 2.0, was originally part of .NET 1.0 Beta 1 but that it was removed since the C# team viewed properties as just a special kind of field. From this perspective, having a different accessor made no sense. But as Jeffrey said, after a large amount of customer features, the C# team has put this feature back. it just took them five years.

Jeffrey is also not afraid to mention the things that the CLR team have done badly. He spoke about the justification for the static class feature of .NET 2.0. Apparently in .NET 1.0 the Environment class that only had static methods, but late in the beta cycle a developer added a HasShutDownStarted method but forgot to make it static. On top of this the Environment class had a private constructor so it was impossible for anyone to create a new instance of the Environment class and call this method. The worst part of this is that it showed that the developer had not written a single test to check the method that he added! If they had written a single test they would have seen that they could not have compiled the code that tried to write this method.

The session helped me see the benefits in understanding how a particular language feature works, particularly understanding whether it involves the language compiler or involves the CLR. This knowledge provides the background behind some of the constraints on the features (such as why you can have a catch block inside an iterator). The tension between the language compilers and the CLR was highlighted in the recent post-beta 2 changes to more fully support Nullable types in the CLR. Don Box's idea last year was that many of the changes going forward are going to be driven by 'syntactic sugar' using the language compilers, while Jeffrey was more upbeat that there's still innovation happening inside .NET and the CLR.

Jeffrey is razor-sharp technically and very funny. My favourite comments so was "Indigo loves using attributes. They use attributes so much that you no longer have to write code"

The afternoon session started with a coverage of partial types, before spending a long time on CLR hosting and how the SQL team helped ensure that the CLR could be hosted in a secure and reliable way. While Jeffrey is able to explain this stuff really well, it was a hard slog to keep up with the section on Constrained Execution Regions ... so I've nipped off to watch David Solomon and Mark Russinovich's Windows Internals section (or Sysinternals tools seminar as it seems to be).