Steve Morgan - Technology Spot

Writing about the Cloud for PlatformCloud9

Steve Morgan — Thu, 08 Mar 2012 19:08:00 GMT

I was recently invited, through work, to start writing about the technical side of designing and developing applications for the cloud on PlatformCloud9.com.

PlatformCloud9 is an industry-specific site from the editorial team from BusinessCloud9.com and is expected to launch soon (at the time of writing, at least). I've started with some articles on developing multi-tenant applications for Windows Azure. While the PlatformCloud9 site is being established, those articles will appear on BusinessCloud9.

BusinessCloud9 is a well-respected site with insight into cloud computing from a number of experts across the IT industry. The material is typically aimed at CxO's. The sister site, PlatformCloud9, aims to build on that with guidance of an more technical nature from subject matter experts with hands-on experience actually delivering cloud-based solutions.

It's a great opportunity for me. As you'll see, my posts here have been extremely sporadic. Writing for the site gives me a great incentive to knuckle down and create some interesting new posts. The feedback I've had so far has been extremely encouraging. The exposure I gain is no bad thing, either.

A less fortunate consequence is that I'm likely to be posting here even more sporadically; if such a thing was possible. But please join me over at PlatformCloud9 or BusinessCloud9 and hopefully you'll find something of interest.

Howto: Use ASP.NET SQL Server Session State Provider with Entity Framework Code-First with SQL Azure - Part II

Steve Morgan — Fri, 12 Aug 2011 08:22:00 GMT

This is the second and final part of a two-part post. In Part I, I described some of the issues with combining EF Code-First and ASP.NET SQL Server Session State in the same database and by the end of the article, I'd described how I'd got Entity Framework managing the two tables that support the SQL Server Session State Provider.

All access to the state database is implemented using a significant number of SQL Server Stored Procedures. Stored procs seem to be falling out of favour, as developers put their faith in abstraction layers such as Entity Framework to magically infer the best way to interact with their data. I'm still a big fan of stored procs and despite arguments to the contrary, my experience suggests that I'm better at optimising data access that EF is. An unfortunate consequence is that, in certain areas (and so far, I see EF Code-First as one of them), stored procs are very much under-supported. The Reverse Engineer Code First Power Tool that I described in Part I will happily reverse-engineer your database tables, but gives no consideration to those all-important stored procedures. As a consequence, when EF recreates the database, it will create the state tables, but no stored procs to accompany them. That just won't do.

The approach I took was to get Entity Framework to automatically create the stored procedures when it recreated the database. The magic is in an Initializer class; in this case called SchoolInitializer. Here's the original source of that class, with various lines of database population code removed for brevity:

public class SchoolInitializer : DropCreateDatabaseIfModelChanges<SchoolContext>
{
    protected override void Seed(SchoolContext context)
    {
        var students = new List<Student>
        {
            new Student { FirstMidName = "Carson",   LastName = "Alexander", EnrollmentDate = DateTime.Parse("2005-09-01") },
            new Student { FirstMidName = "Meredith", LastName = "Alonso",    EnrollmentDate = DateTime.Parse("2002-09-01") }
        };
        students.ForEach(s => context.Students.Add(s));
        context.SaveChanges();

        // Other content removed for brevity
    }
}

There are two points of note, here. Firstly, SchoolInitializer is derived from DropCreateDatabaseIfModelChanges<SchoolContext>. This tells EF to automatically recreate the database if the model associated with the context changes in code. That's not a bad approach for Development and Test environments (a bit risky for Production ones, though). The problem at this point is that it relies on the presence of a table in the database called EdmMetadata and as our database only has the ASP State tables in it at the moment, it will fail.

Secondly, the Seed() method is called to populate the database with initial data when it is recreated. The example uses this method to add test data. I shall use the method to solve my stored procedure problem... (I should say, there may well be a better place to do this, but it'll do for now).

I decided to approach the problem by having the Seed() method run a SQL script to generate the stored procs. I have a SQL script, of course, which is the InstallSqlState.SQL script that I used originally to create the ASP State tables.

There are some changes required, though...

Firstly, I needed to remove the script that created the ASP state tables. EF takes care of those, now.

The next problem got me scratching my head for a few minutes, though. The ASPStateTempSessions table makes use of default values for a couple of its columns:

CREATE TABLE dbo.ASPStateTempSessions (
    SessionId        nvarchar(88)    NOT NULL PRIMARY KEY,
    Created          datetime        NOT NULL DEFAULT GETUTCDATE(),
    Expires          datetime        NOT NULL,
    LockDate         datetime        NOT NULL,
    LockDateLocal    datetime        NOT NULL,
    LockCookie       int             NOT NULL,
    Timeout          int             NOT NULL,
    Locked           bit             NOT NULL,
    SessionItemShort VARBINARY(7000) NULL,
    SessionItemLong  image           NULL,
    Flags            int             NOT NULL DEFAULT 0,
)

Note the use of DEFAULT GETUTCDATE() for the Created column and DEFAULT 0 for Flags. The problem here is that EF is now responsible for creating our tables but EF and default values do not make happy bed fellows. Fortunately, the solution was simple enough. I updated the stored procs to explicitly supply the values instead of relying on the presence of the defaults. There's another point in favour of stored procedures!

As an example, this stored procedure:

CREATE PROCEDURE dbo.TempInsertStateItemLong
    @id         nvarchar(88),
    @itemLong  image,
    @timeout    int
AS    
    DECLARE @now AS datetime
    DECLARE @nowLocal AS datetime
    
    SET @now = GETUTCDATE()
    SET @nowLocal = GETDATE()

    INSERT dbo.ASPStateTempSessions 
        (SessionId, 
         SessionItemLong, 
         Timeout, 
         Expires, 
         Locked, 
         LockDate,
         LockDateLocal,
         LockCookie) 
    VALUES 
        (@id, 
         @itemLong, 
         @timeout, 
         DATEADD(n, @timeout, @now), 
         0, 
         @now,
         @nowLocal,
         1)

    RETURN 0

became

CREATE PROCEDURE dbo.TempInsertStateItemLong
    @id         nvarchar(88),
    @itemLong  image,
    @timeout    int
AS    
    DECLARE @now AS datetime
    DECLARE @nowLocal AS datetime
    
    SET @now = GETUTCDATE()
    SET @nowLocal = GETDATE()

    INSERT dbo.ASPStateTempSessions 
        (SessionId, 
         SessionItemLong, 
         Timeout, 
         Expires, 
         Locked, 
         LockDate,
         LockDateLocal,
         LockCookie,
         Flags,
         Created) 
    VALUES 
        (@id, 
         @itemLong, 
         @timeout, 
         DATEADD(n, @timeout, @now), 
         0, 
         @now,
         @nowLocal,
         1,
         0,
         @now)

    RETURN 0

The subtle changes, for the less-than-eagle-eyed are that I now explicitly set the Flags and Created columns. Each insert into ASPStateTempSessions was reviewed and updated where necessary. I've provided the updated script for you to download.

The next step was to get the Seed() method to run this updated SQL script whenever the database was created. There's a complication to running SQL scripts from code. In our application, we can use

context.Database.ExecuteSqlCommand("string of SQL");

to execute a SQL instruction against the database. We might consider reading our SQL script into a string, then executing it with ExecuteSqlCommand(), but that won't work. The problem is that our SQL script is not one SQL command, but many. Each is delimited by the

GO

instruction. ExecuteSqlCommand will baulk at commands being provided in this way. There are several methods that we can employ to resolve the problem.

Spawn a new process running osql.exe or sqlcmd.exe passing the script file as a parameter
Use SQL Server Management Objects to run the script
Break the script into commands that can be run using ExecuteSqlCommand()

I hate spawning out to another process and waiting for it to complete. This isn't Unix and running up processes is an expensive and time-consuming operation. It's also, shall we say, a little 'inelegant'.

Ordinarily, I'd use SQL Server Management Objects to run a script. However, in this case, I didn't want to add further dependencies to the application but rather wanted to keep it lighter weight. So in this case, I chose the third option and wrote a routine that would chop up my SQL script into individual statements and execute them one-by-one.

Here it is, in its entirety:

private void RunSQLScript(DbContext context, string scriptFile)
{
    var reader = File.OpenText(scriptFile);

    StringBuilder commandBuilder = new StringBuilder();
    string line;

    while (!reader.EndOfStream)
    {
        line = reader.ReadLine();

        if (line.Trim().ToLower() == "go")
        {
            context.Database.ExecuteSqlCommand(commandBuilder.ToString());
            commandBuilder.Clear();
        }
        else
        {
            commandBuilder.Append(line);
            commandBuilder.Append("\r\n");
        }
    }

    if (commandBuilder.Length > 0)
    {
        context.Database.ExecuteSqlCommand(commandBuilder.ToString());
    }
}

The method assumes that statements are delimited by a "GO" instruction on a line by itself. It builds up the SQL command in a StringBuilder and executes it when it encounters a "GO". The extra call to ExecuteSqlCommand() at the end is there to cater for scripts that don't have a "GO" as the last instruction. As a refinement, I should really get the code to skip blank lines (and maybe comments).

Armed with my new ability to run a SQL script, I updated the Seed() method as follows:

protected override void Seed(SchoolContext context)
{
    var students = new List<Student>
    {
        new Student { FirstMidName = "Carson",   LastName = "Alexander", EnrollmentDate = DateTime.Parse("2005-09-01") },
        new Student { FirstMidName = "Meredith", LastName = "Alonso",    EnrollmentDate = DateTime.Parse("2002-09-01") }
    };
    students.ForEach(s => context.Students.Add(s));
    context.SaveChanges();

    // Code yet again removed for brevity

    RunSQLScript(context, @"C:\ContosoUniversity\ContosoUniversity\ContosoUniversity\DBScripts\InstallSqlStateProcs.SQL");
}

I was feeling somewhat lazy by this point, so specified the full path to the SQL script. In reality, this should be picked up from the relevant location.

That takes care of getting EF to set my database up to support the ASP.NET SQL Server Session State Provider. All that remains is to configure the application to use SQL Server Session State:

<system.web>
    <sessionState mode="SQLServer" sqlConnectionString="Data Source=.\SQLExpress;Initial Catalog=School;Integrated Security=SSPI" cookieless="false" timeout="20" allowCustomSqlDatabase="true" />
</system.web>

The sample Contoso University application doesn't use Session State at all, so to test, I added a couple of lines of code to the CoursesController...

public ActionResult Create()
{
    Session["SomethingInSession"] = "Hello!";
    PopulateDepartmentsDropDownList();
    return View();
}

[HttpPost]
public ActionResult Create(Course course)
{
    System.Diagnostics.Debug.Assert(Session["SomethingInSession"] == "Hello!");

    // Snip
}

So, that's it. I won't be surprised to find further issues as we move through the lifecycle, but this seems like a good start. I hope you find it of use, too.

Howto: Use ASP.NET SQL Server Session State Provider with Entity Framework Code-First with SQL Azure - Part I

Steve Morgan — Thu, 11 Aug 2011 21:52:00 GMT

This article was prompted in response to a question on my favourite programmers' site, StackOverflow.com.

As part of my work on an accelerated engineering process for Windows Azure applications, I've been implementing SQL Server (and SQL Azure) databases using a database-first approach. However, it's hoped that moving to EF Code-First can further improve the engineering process and the latest code generators that my team and I are working with implement business databases this way.

Another design issue to deal with is that of session state. While there are Azure providers that use Azure Table Storage, I've had more success with the standard SQL Server Session State Provider, as I've found the Table Storage Provider to be somewhat unreliable in the development fabric.

There's an argument that using Table Storage for session state is more cost-effective that using SQL Server Session State with Windows Azure, as you have to pay for SQL Azure databases. That argument is weakened somewhat if you're using SQL Azure, anyway, but it still holds true if you have to pay for a separate SQL Azure database, just to hold the state.

The ideal, then, in my case, is to use the main application SQL Azure database for both business data and session state.

Using our previous code generators, with a database-first implementation, that was straightforward enough. Setting up the database for session state involves simply running a SQL script. The standard InstallSqlState.SQL script doesn't work, as it fails against some of the limitations of SQL Azure. I found a modified script, over at MSDN Forums. That was a good starting point, but it needs some modifications. Not least, I removed references to the [AspState] database and the creation of users. You can download a copy of my modified script here. It turns out that more changes would be required, but more later.

The problem is that if you simply run this script against your EF Code-First database, it will be lost if EF subsequently regenerates the database. And it turns out, it's not happy if the database contains tables that it doesn't know about.

The trick, then, is to have EF manage the tables along with those required to contain the business data.

Because I didn't want to risk breaking an existing application, I headed over to MSDN, downloaded the Contoso University ASP.NET MVC Application Using Entity Framework Code First sample and used this as the basis for my investigations.

I created a new database under SQL Express (it's easy enough to migrate it to SQL Azure using the SQL Azure Migration Wizard later). Then, using SQL Server Management Studio, I ran the modified InstallSqlState.SQL to set up the tables for the SQL Server Session State Provider. It creates numerous stored procedures, too, but these ultimately get thrown away.

Next, I downloaded and installed the EF Power Tools CTP from within Visual Studio 2010 (Tools -> Extension Manager and search the online gallery for Entity Framework Power Tools).

After restarting Visual Studio, one the installation was complete, my solution was reloaded.

I right-clicked on the ContosoUniversity project and selected Entity Framework -> Reverse Engineer Code First.

Then, you simply connect to the new database and let the Power Tool do its stuff:

The import process creates a new context class (called SchoolContext.cs in my case, because the database was called School), an Entities folder with classes for the ASP State tables and a Mapping folder containing mapping classes). It also adds a connection string for the database to web.config.

I then set to work pulling the wizard's work apart an integrating into the application. First step was to add the new tables to the SchoolContext class. Because the wizard generates its context class in the root of the project, it was simple enough to copy-and-paste between the two before deleting the generated class.

     public class SchoolContext : DbContext
    {
        public DbSet<ASPStateTempApplication> ASPStateTempApplications { get; set; }
        public DbSet<ASPStateTempSession> ASPStateTempSessions { get; set; }
        public DbSet<Course> Courses { get; set; }
        public DbSet<Department> Departments { get; set; }
        public DbSet<Enrollment> Enrollments { get; set; }
        public DbSet<Instructor> Instructors { get; set; }
        public DbSet<Student> Students { get; set; }
        public DbSet<Person> People { get; set; }
        public DbSet<OfficeAssignment> OfficeAssignments { get; set; }

        protected override void OnModelCreating(DbModelBuilder modelBuilder)
        {
            modelBuilder.Conventions.Remove<PluralizingTableNameConvention>();
            modelBuilder.Configurations.Add(new ASPStateTempApplicationMap());
            modelBuilder.Configurations.Add(new ASPStateTempSessionMap());
            modelBuilder.Entity<Instructor>()
                .HasOptional(p => p.OfficeAssignment).WithRequired(p => p.Instructor);
            modelBuilder.Entity<Course>()
                .HasMany(c => c.Instructors).WithMany(i => i.Courses)
                .Map(t => t.MapLeftKey("CourseID")
                    .MapRightKey("PersonID")
                    .ToTable("CourseInstructor"));
            modelBuilder.Entity<Department>()
                .HasOptional(x => x.Administrator);
        }
    }

Next, some modifications were required to the ASPStateTempSessionMap class, to ensure that the table generated by EF matched that created by the script (as far as we can, at least).

public class ASPStateTempSessionMap : EntityTypeConfiguration<ASPStateTempSession>
{
    public ASPStateTempSessionMap()
    {
        // Original content removed for brevity...
        this.Property(t => t.SessionItemLong).HasColumnName("SessionItemLong").HasColumnType("image");
        // ...
    }
}

Here, I've added .HasColumnType("image"), because by default, EF creates the table with a varbinary column, instead.

Then, a temporary change to the SchoolInitializer class, to force EF to recreate the database...

I changed

public class SchoolInitializer : DropCreateDatabaseIfModelChanges<SchoolContext>

public class SchoolInitializer : DropCreateDatabaseAlways<SchoolContext>

But once I've finished, I'll revert that change.

Running the application now causes EF to create the database afresh, including the requisite ASPStateTempSessions and ASPStateTempApplications tables. But before I could use the SQL Server Session State Provider, I had to address the problem of the numerous missing stored procedures.

I address that issue in Part II of this article.

Did DisplayPad crash my Mac OS X Lion upgrade?

Steve Morgan — Thu, 21 Jul 2011 15:12:00 GMT

Well, those were a good few hours of my life that I'm not going to get back!

Like, I suspect, many others, I downloaded Apple's OS X 10.7 Lion upgrade for my MacBook Pro from the App Store yesterday. To hell with my download allowance, I thought; I'll just not watch YouTube for the rest of the month.

The installation went smoothly enough. It did, however, identify one piece of incompatible software, DisplayPad, which it dutifully moved to the Incompatible Software folder at the root of my hard disk.

A quick search on my Search Engine of Choice revealed an updated version of DisplayPad, 1.0.2, which claims to be compatible with Lion. So I downloaded this latest version and installed it. Given the nature of DisplayPad, I expected it to install some kind of display driver or OS extension to allow the UI that would be tagetted at a second monitor to be relayed to the iPad. This, the installer did and on succesful completion told me that a restart was required.

I restarted the machine.

At the initial Apple Logo screen, the progress spinner sat for a while before moving on. The next screen that I saw was the 'fabric-look' background to the Lion logon screen. And this is where it all went wrong. Despite waiting for many minutes, the login icons never appeared (I should make it clear at this point that I've disabled automatic login for security reasons). My MacBook Pro had crashed!

After a while, I powered off the machine, restarted and confirmed that the behaviour remained.

I powered off again and this time, restarted into Safe Mode by holding the Shift key down. I was able to login successfully and did a bit of poking around, but couldn't see anything obvious. I also wasn't sure how to uninstall DisplayPad. Dragging it to the Trash Bin doesn't seem sufficient; no surprise, given that it's installing that display driver. Again, my Search Engine of Choice yielded no suggestions and the FAQ on the Clean Cut Code site (the producers of DisplayPad) contained only two, less that useful, questions and answers.

After an hour or so of trying, I gave up and this time restarted using Command-R to bring up the recovery tools.

Fortunately, my MacBook Pro is regularly backed up to an external hard disk that is attached to my Airport Extreme (my current Wireless Access Point of choice) using Time Machine. The Recovery Tools give you the option of restoring from a Time Machine backup and so I selected the last Snow Leopard backup taken shortly before I upgraded to Lion.

What followed was a restore lasting around 13 hours. I should consider myself lucky; the original estimate was 21 hours!

Once the restore was complete, I rebooted and started again. Microsoft Office wanted to rebuild its internal identity store (it warned me that this usually happens when you restore from Time Machine). I was also disappointed to find that Fingerprint, software to support AirPrint with non-AirPrint compatible printers, had reverted to Trial mode and I can't find my license key. That's a separate issue, though, and not of concern here.

So guess what I did? I let DisplayPad update again. Like a fool!

When I rebooted, I had exactly the same symptoms. The machine got as far as the login screen, but displayed no icons, no prompts, nothing.

However, in my frustration, I must admit to having a bit of a rattle on the keyboard. Not a vigorous one, obviously, as this is a MacBook Pro and warrants some respect. I was surprised to hear one of those "Don't press that key again" kind of sounds.

So, I hit the Enter key and to my further surprise, was logged in to the Guest account (which has no password).

I logged out and was greeted, again, with a login screen with no icons, no prompts, nothing.

This time, I pressed the first letter of my username, Enter, my password, Enter. To my relief, I was logged in to my account! Hooray!

So, it would appear that DisplayPad's graphics driver is screwing up the Login screen. I could be wrong, but that's my belief at the moment. I've worked around the problem temporarily by turning on Auto-Login again, so the Login screen is never required. And I've e-mailed Clean Cut Code for some advice; if not to solve the problem then how to successfully and completely uninstall the application.

I appreciate that these things happen, but I spent a lot of frustrated hours just getting my machine back to a usable state again. While I await a proper resolution, I'm going back to tame this Lion again...

UPDATE/FIX:

Well, I'm pretty convinced that it's a DisplayPad problem, that's for sure!

While mooching around Lion some more, I was finding all kinds of problems. Firefox 5.0.1 wouldn't display any content in its windows unless I start in Safe Mode. Google Earth didn't show me the Earth, that sort of thing.

So, a bit more investigation, and I think I've managed to work out how to uninstall DisplayPad.

Under

(HDD Root) /System/Library/Extensions

There are two files:

DisplayPadDriver.kext

DisplayPadFrameBuffer.kext

Move these to the Trash to delete them.

From the

/Applications

folder, delete DisplayPad.app by moving it to the Trash bin, too.

On restart, things seem to be back to normal. Specifically, Firefox is now working without resorting to Safe Mode and Google Earth is going its stuff, too.

And, YAY! The icons are back in the Logon screen!

Looks like DisplayPad needs a bit more work before it's ready for Lion.

Further Update:

I heard back from Clean Cut Code about the problem. Apparently, it's due to a bug in Lion that affects machines with NVidia 9400M graphics cards. My machine is a mid-2009 MacBook Pro and has dual-graphics, an NVidia 9400M for low-power use (typically on battery) and an NVidia 9600M GT which offers higher performance and is generally preferred when connected to the mains. Expect an update from Apple or Clean Cut Code sometime soon that resolves the issue.

They also advised removing

/System/Library/Extensions/SSFrameBuffer.kext

if it exists. It didn't exist on my machine, but could do on yours.

Mac Fix: iTunes was unable to load dataclass information from Sync Services.

Steve Morgan — Mon, 14 Mar 2011 14:57:00 GMT

Last week was a fairly big one for the iPhone stack, what with the release of iTunes 10.2.1 and subsequent release of iOS 4.3. Of course, I dutifully installed both. A fairly painless, if long-winded exercise.

I noticed a couple of days later a problem syncing my phone with my MacBook Pro. On connection of the phone, iTunes displayed the following error message:

iTunes was unable to load dataclass information from Sync Services. Reconnect or try again later.

Well, I did and the same message was displayed next time. So, I rebooted the MacBook Pro and, for good measure, rebooted the iPhone too. On connecting the phone (once iTunes had started up) - no change. Same message.

On closer inspection, it looked as though the phone was actually syncing but not backing up. Not a situation that I was happy with, so time to start investigating seriously. After much trawling of the usual haunts (starting with my search engine of choice, naturally), I uncovered both the cause and the fix, though the relationship between them isn't especially obvious. While I supposed that the issue lay with either iTunes 10.2.1 or iOS 4.3, there was another little detail that I hadn't noticed (and have thus far neglected to mention). Shortly after these upgrades, Apple posted the latest version of XCode to the Mac App Store. Despite not yet having built a real iOS application in anger, I couldn't not have it, so I clicked the button, entered my Apple password and let XCode install. It was XCode, it appears, that was the root of my synchronisation issues. The connection may seem tenuous, but I suspect it could well be related to the inclusion of the iPhone emulator replacing important files.

The solution was found on Apple's support site, here: iTunes: How to remove and reinstall the Apple Mobile Device Service on Mac OS X

Basically, it involves uninstalling iTunes, removing two Apple Mobile Device Service files and reinstalling iTunes. You'll be especially pleased to know, if you have to go through this process yourself, that all of your iTunes content and configuration remains unharmed.

Printing from iOS 4.2 (iPhone, iPad, iPod Touch) without an AirPrint compatible printer

Steve Morgan — Tue, 23 Nov 2010 12:39:00 GMT

Being an iPhone 4 user (I've yet to stump up for an iPad), the main feature of iOS4.2 that attracted my attention was the ability to print using AirPrint technology. Of course, HP would love me to go out and buy an AirPrint-compatible printer that the iPhone can talk to. However, I've already got three HP printers, a black-and-white Laserjet 3055 all-in-one, a colour Laserjet 2605DN and an A3+ Photosmart B9180 inkjet. HP have had enough money out of me in printers for a while.

Fortunately, those clever people at Collobos software have developed a utility for the Mac called Fingerprint. Fingerprint shares your Mac-connected printers as AirPrint devices, so as long as your Mac is on and Fingerprint is running (you can set it to run at startup), any iOS4.2 devices on the same wireless network can print to compatible printers. You have to remember to enable printer sharing in System Preferences, though. Collobos say that not all printers are compatible. I've only tested it with the Laserjet 3055 so far but it performed flawlessly printing from Safari and Mail.

Here's the FingerPrint application running on my MacBook Pro:

Back at the iPhone, selecting Print in Safari presents some printing options:

Selecting Printer in the print options, you get to choose a printer from the same list that Fingerprint is sharing...

There's a 7-day trial of Fingerprint available but after having printed the first page, I purchased a full license straight away. It only costs $7.95 (or £6.24 including VAT in the UK) and the purchasing experience is integrated nicely into the app.

So far, I've got nothing but praise for Collobos and their very timely release of Fingerprint. Hopefully, a similar facility will be available for Windows users very soon.

Soliciting comments of the Microsoft Azure Developer Portal

Steve Morgan — Fri, 15 Oct 2010 12:30:00 GMT

I'm interested in what Azure developers think of Microsoft's Developer Portal.

For instance, from an Enterprise delivery perspective, it greatly concerns me that developers can flip-flop Staging and Production with no formal change control.

What's your view? If there are things that you think are good or bad, post a comment and once I've reviewed it, I'll try to collate the results here into something meaningful.

Please note that comments are moderated, so you won't see anyone else's contributions until I've checked them out.

iPhone 4, iOS4.1 and Nokia 616 Bluetooth Compatibility Problems

Steve Morgan — Thu, 16 Sep 2010 09:56:00 GMT

I've posted previously about iPhone compatibility with the Nokia 616 Bluetooth car kit and made a few updates along the way. But having regressed in terms of compatibility with the iPhone 4, I thought I'd break out a separate post.

You'll find the previous post here.

Today, iTunes offered me the latest iOS version 4.1 for my iPhone 4. Among the release notes I was pleased to see "Bluetooth improvements". I was keen to see if Apple had addressed the issues that, having been resolved with the iPhone 3g and 3gs, were now plaguing the iPhone 4. So, I let iTunes do its thing and ventured out to the car to check.

On connecting to the phone, the car kit asked me the question it always asks (because it's never worked previously): "Download contacts?". I hit the green button and waited a few seconds. The display on the Nokia 616 displayed "Downloading" and a progress bar. Then, the Bluetooth connection was dropped and the display changed to "Download not supported". These are the same symptoms that I've had since upgrading to the iPhone 4.

To be on the safe side, I deleted the Bluetooth pairing from both the car kit and the phone, then set them up again from scratch. On attempting to download the contacts, I had exactly the same error.

The long and the short is that the problem still isn't fixed. You get prompted to download contacts every time the phone connects and you can only dial by using voice dialling (which works fine over Bluetooth) or using the keypad (which is dangerous on the move). I do wish they'd sort it out. Handsets come and go, but having a car kit fitted is a longer term commitment and having cracked the problem with the iPhone 3g/3gs it's a real disappointment to see it return with the iPhone 4.

As ever, I'll keep you updated if I learn more or until I change the car!

Private Cloud ≠ Private Public Cloud

Steve Morgan — Fri, 10 Sep 2010 11:55:00 GMT

Or when is a cloud not a cloud?

The economies afforded by using a public cloud arise from the utility pricing model where customers are charged only for the resources that they use. Maximum cost savings are made by minimising the resources that are consumed but with the benefit that, if additional resources are required, they are instantly available at a known cost. This is achievable for public cloud providers because a very large number of customers share a very large number of resources, each using a very small proportion. The cloud provider makes money through economies of scale as peaks average out and the amount of additional capacity required can be kept relatively small. Utilisation for a single customer is very low, but average utilisation for the cloud provider is relatively high.

Compare this with a customer employing a private cloud. The private cloud is a shared resource, but can only be shared by the subset of applications within the customer estate that are cloud-ready. Because the cloud is private, the entire cost of the cloud falls to the customer, regardless of whether they are using the resources or not. Applying a utility cost model to a private cloud doesn’t work (or is risky, at least) because somebody has to pay for the unused resources. So, in order to realise maximum cost benefit, utilisation of the private cloud must be as high as possible. But if utilisation is kept high, one of the major benefits of the cloud is lost; immediate elasticity. If the unthinkable happens (that the private cloud runs out of resource) the situation for customers is potentially worse than in a non-cloud environment as the lead time to procure and deploy additional nodes for a private cloud are potentially longer than procuring commodity servers. In order to mitigate the risk, there may be a tendency for service owners to want to ring-fence cloud resources, creating their own private cloud, and simply exacerbating the problem. Simply, the bigger the cloud, the more cost effective it is. As the size of a cloud reduces, it becomes nothing more than a standardised infrastructure platform offering a number of middleware services.

There are important differentiators between public and private clouds with advantages and disadvantages to each. Major candidates for private clouds include the public sector. But maybe greater benefit could be gained by promoting the use of a G-Cloud, an accredited cloud resource that is shared across the public sector but delivered by an accredited cloud service provider. I think I may know of such a service provider...

How much to run my app on Windows Azure? - Part 1

Steve Morgan — Sun, 22 Aug 2010 22:09:00 GMT

or how accurately do you need to estimate?

Windows Azure (previously codenamed Red Dog) is Microsoft’s foray into Platform-as-a-Service (PaaS). Rather than incurring large amounts of capital expenditure building, hosting and maintaining a mountain of infrastructure, Azure opens up the possibility of moving our applications to an environment of virtually limitless capacity (within reason) where organisations pay for only the resources that they use on an hour-by-hour basis.

Azure promises capacity to meet every conceivable demand for computing and storage (as long as it’s running on Windows) yet saving the consumers significant amounts of money. In these austere times, when all budgets are under considerable pressure, that’s a pretty appealing prospect.

The problem is that offering our customers a solution that reduces cost always leads to one tricky question:

“How much?”

Ooh, the killer question! There are, of course, a great many factors that influence the cost of bringing an application into service and running it for the entirety of it’s life expectancy. I’m going to touch on just the costs of delivering the infrastructure and running it for some arbitrary period of time.

There are two major influencers to the overall shape and size of any given infrastructure; the functions is must fulfil and the capacity it must provide.

Functionally, an infrastructure must include networks, routing, load balancing, computing resource, storage and backup capabilities. Many applications can operate perfectly effectively on a fairly standard or ‘commodity’ infrastructure. Others may have more exotic requirements; maybe some special hardware like a telephony system (PBX) or data gathering device. If this is the case for your application, you can forget hosting your application lock-stock-and-barrel in the cloud, at least for now. A hybrid cloud and on-premise solution may be a viable option, but I’m going to avoid discussion of such mongrels for now.

Computing in the cloud is about running your applications on an arbitrary slice of tons-and-tons of commodity infrastructure.

“Yeah, but how much?”

In my experience, customers need to know before committing to a project how much it’s going to cost them. Sizing infrastructure early in a project’s lifecycle is always a bit of a black art. Customers often don’t really know how many users their applications are going to support. They don’t know the blend of transactions that are going to be executed. We (the technical experts), don’t know how we’re going to implement what the customer’s asked for, either because they’ve not yet told us enough or we haven’t yet worked out how we’re going to build it.

Being the long-established experts that we are, we apply three key techniques to identifying the optimal infrastructure capacity for any given problem domain:

Experience
Guesswork
The Fudge-Factor

The amount and the nature of the experience you have is obviously going to have a massive impact on the effectiveness of point 1 in this list. If this solution is to all intent and purpose one that you’ve delivered time and time again, you’ll have a really strong baseline against which to assess the cost. In my experience, unless you’re delivering vertical solutions, you never really do the same thing twice. Experience will help you estimate those aspects that are similar to things you have delivered before. Experience will also give you some inkling as to how hard the bits that you don’t know are likely to be.

After experience comes guesswork. Hopefully, you’ll know enough to keep the guesswork to a minimum, because this is where the biggest risk to the accuracy of your estimate hangs out.

Finally, we introduce the fudge-factor. This might otherwise be known as contingency. If your experience and guesswork leads you to the conclusion that your database server needs two processors and 4GB RAM, let’s specify four processors and 8GB RAM – just to be on the safe side.

If you’re lucky, the scale of the infrastructure you come up with will cope with the actual load (at least until you’re safely ensconced elsewhere). If you were smart, you’ll have architected the solution to scale by simply adding more hardware at a later date (never mind the cost).

Now, what happens if you over-estimate the scale of the infrastructure that’s required to effectively carry the load? Typically, nobody will ever know! All the customer will know is that the application continues to be provide the performance they expected yet cope admirably with the ever increasing number of users. They’ll never realise that the reason they’re having to cut investment in future projects is because your application consumes no more than 10% of the hardware that they’ve paid (and continue to pay) for.

Now, think about what happens when they’re being billed my Microsoft for running their application in the cloud.

The first problem is that, for the moment at least, hardly anyone has experience of the costs associated with running applications on Azure. We can compensate by increasing our reliance on guesswork and the fudge-factor. We can still produce a cost. And if our project goes ahead, one of three things will happen:

Our estimate will be ‘in the right ballpark’ and everyone will be happy.
Our estimate will be way too low and someone is going to get a really nasty surprise when the bills from Microsoft start rolling in.
Our estimate will be way too high.

On the face of it, point 3 seems just fine. After all, we’ve set a level of expectation for the cost of running this application and it’s turned out to be much cheaper.

But let’s think about this more critically. When we put forward our estimate to the business sponsor, they (or maybe the CFO) had to assess the cost-benefit of proceeding with the project at all. Best case, you’ve made the business sponsor feel a fool for jumping through hoops to get financial approval for a project that may otherwise have slipped ‘under the RADAR’. Worst case, the project never got the green light anyway because the ROI wasn’t sufficient.

The other issue, the one that is more personal, is that you’ve made it blindingly obvious that you don’t know what you’re doing; your credibility and reputation are indelibly tarnished. I don't know about you, but even if I don't know what I'm doing, I don't really like to be found out!

When we’re designing for the cloud, we need to take a much more robust and scientific approach to estimate the operational costs for any solution that we assess. In part 2, I’ll explore some of the specific things I think we should be considering and wondering about just how we’re going to pull it off.

Getting to Know Windows Azure

Steve Morgan — Sun, 22 Aug 2010 20:06:00 GMT

I consider myself rather lucky at the moment. I was recently amongst forty-four ‘carefully selected’ specialists within my organisation to spend a week at Microsoft’s pleasure in Redmond to kick-start our Windows Azure partnership. We forty-four good men and true (and women) were apparently selected because we’re the ‘best of the best’. Though my suspicion is that in certain cases, it’s because we were the ‘most of the available’.

This was my first deep immersion into Windows Azure, though I’d been doing a little homework beforehand. There’s a lot to know. Sure, at a simplistic level, it’s a virtualized, massively scalable environment for running new or existing applications. But if you think that’s all there is to it, you’re going to be in for a nasty surprise. Us architects, especially, need to raise our game and get to know Windows Azure (and cloud computing in general) in intimate detail if our customers aren’t going to be in for some nasty surprises.

In upcoming posts, I shall be talking more about moving to Azure and what that means for Sales, Architecture, Development and Service Management, because Azure cuts across all of these disciplines.

Starbucks finally gets rid of T-Mobile. iPhone users rejoice!

Steve Morgan — Fri, 15 May 2009 07:35:00 GMT

I'd heard a mention that things were changing in Starbucks.

This morning, I had the great displeasure of dealing with T-Mobile's hotspot customer support when unable to connect from my local Starbucks. I use an iPass-enabled laptop for work, which makes working in Starbucks generally easy. But not today. Once again, I had to explain to the customer service representative that the IP address I had really was from their hotspot. "Oh, but our IP addresses always start 10.113, Mr. Morgan". Once they finally acknowledged that there was a problem, I then have to wait 24-48 hours while they think about fixing it. At which point, they'll undoubtedly get someone in Starbucks to recycle the power and the problem will go away.

This is a relatively new branch of Starbucks and the T-Mobile hotspot has been a regular problem since it opened. I've literally spent hours on the phone to their customer support trying to get them to sort it out. Of course, being "a member of the public", I can't possibly know what I'm talking about. You get used to being patronised after a while.

Thankfully, however, the curse of T-Mobile is shortly to disappear from Starbucks. Having finally brokered the relevant deals, Starbucks in the UK is moving to BT Openzone for it's hotspots. To be honest, I don't really know if BT Openzone is going to be any more reliable. Or if their customer service is better.

What I do know, though, is that it's good news for us iPhone users. At last, from May 25th, I can have complimentary iPhone Wi-Fi access with my Caramel Macchiato. Which will hopefully be much much responsive that O2's patchy 3G coverage.

In the meantime, while I wait patiently for T-Mobile to fix the problem (or BT Openzone to replace it altogether), I'm having to fall back to my Vodafone 3G card. Still, at least it's working.

Microsoft ships an iPhone app!

Steve Morgan — Wed, 17 Dec 2008 20:02:00 GMT

Well, who would have thought that Microsoft would get into iPhone development?

On 13th December 2008, Microsoft Live Labs announced the launch of a preview release of its Seadragon Mobile application for the iPhone.

Seadragon Mobile provides smooth image viewing and scaling, just like we're used to seeing on the iPhone with Google Maps.

It makes sense that the iPhone's the first mobile platform to get such an app. It's powerful UI is a natural for it. Blackberries and Nokias just don't have the graphics capability. What's interesting, though, is that there's no sign of a version for Windows Mobile yet.

I've been having a play with it and, while it's not going to set the world alight, it's nice to see that Microsoft are mature enough to recognise that the iPhone is a viable platform for their technology.

You can download Seadragon Mobile for free from the Apple iTunes App Store.

HowTo: Insert an image into a Word document and display it using OpenXML

Steve Morgan — Mon, 15 Sep 2008 11:01:00 GMT

One of the tasks I've had over the last few weeks has been to automatically generate documentation from an architectural model. It's been (and continues to be) a relatively painful, sluggish process, as I'm generating Microsoft Word 2007 documents using the Open XML SDK. Never more so than when I came to add support for including images.

I'm going to do a 10,000' pass through the way that I approached it, so it's not a beginners guide to OpenXML. You're going to need to be able to find your way around an OpenXML document using the Open XML SDK first.

The basic process I'm using is to generate from my architectural model an XML document that will ultimately become a Microsoft Word document, like this:

<?xml version="1.0" encoding="utf-8" ?>
<content xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main">
  <bookmark id="ContentToReplace">
    <para style="Heading1">Major Heading</para>
    <para style="Heading2">Secondary Heading</para>
    <table>
      <tr>
        <td>
          <para>Name</para>
        </td>
        <td>
          <para>Steve Morgan</para>
        </td>
      </tr>
      <tr>
        <td>
          <para>Address</para>
        </td>
        <td>
          <para>www.hastobe.net</para>
          <para>Somewhere in Hyperspace</para>
        </td>
      </tr>
    </table>
    <para style="Heading2">Another Secondary Heading</para>
    <image src="..\..\testfiles\mypic.png"/>
    <para>Good now, sit down, and tell me, he that knows,</para>
    <para>Why this same strict and most observant watch</para>
    <para>So nightly toils the subject of the land,</para>
    <para>And why such daily cast of brazen cannon,</para>
    <para>And foreign mart for implements of war;</para>
    <para>Why such impress of shipwrights, whose sore task</para>
    <para>Does not divide the Sunday from the week;</para>
    <para>What might be toward, that this sweaty haste</para>
    <para>Doth make the night joint-labourer with the day:</para>
    <para>Who is't that can inform me?</para>
  </bookmark>
</content>

I have some code that rips through a Word template document, looking for bookmarks and replacing the content of those bookmarks with Open XML fragments. The fragments themselves are derived from the above XML structure using nothing more than good old-fashioned XSLT.

Generating paragraphs of text and tables is pretty simple, as other than transforming the XML and inserting the resulting Open XML into the document, there's not much to it.

Images, on the other hand, are a little more complicated. There are two stages to adding an image to a Word document (or any Open XML document, for that matter). The first is that you must add an "Image Part" that holds the binary representation of the image, itself. Finding out how to do that was pretty easy:

WordProcessingDocument m_Document = ...the document you're building...
 
public void InsertImage(string imageFile, string contentType)
{
    ImagePart part = m_Document.MainDocumentPart.AddImagePart(contentType);
 
    using (FileStream stream = new FileStream(imageFile, FileMode.Open))
    {
        part.FeedData(stream);
    }
}

This function adds a new image part to the document and copies the content of the image file. However, getting the image displayed on the page is another matter entirely. It should be no surprise that we must add some markup to the main document to render the image. I created what I consider to be a minimal Open XML fragment that creates a single paragraph containing nothing but the requested image:

<w:p xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main"
    xmlns:wp="http://schemas.openxmlformats.org/drawingml/2006/
               wordprocessingDrawing"
    xmlns:a="http://schemas.openxmlformats.org/drawingml/2006/main" 
    xmlns:pic="http://schemas.openxmlformats.org/drawingml/2006/picture"
    xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships">
  <w:r>
    <w:drawing>
      <wp:inline>
        <wp:extent cx="{WIDTH-IN-EMUS}" cy="{HEIGHT-IN-EMUS}" />
        <wp:docPr id="{DRAWING-OBJECT-ID}" name="{FILENAME-OF-IMAGE}" 
                  descr="An automatically inserted image" />
        <wp:cNvGraphicFramePr>
          <a:graphicFrameLocks noChangeAspect="1" />
        </wp:cNvGraphicFramePr>
        <a:graphic>
          <a:graphicData uri="http://schemas.openxmlformats.org/drawingml/
                                2006/picture">
            <pic:pic>
              <pic:nvPicPr>
                <pic:cNvPr id="{PICTURE-ID}" name="{FILENAME-OF-IMAGE}" />
                <pic:cNvPicPr />
              </pic:nvPicPr>
              <pic:blipFill>
                <a:blip r:embed="{RELATIONSHIP-ID-OF-IMAGE-PART}" cstate="print"/>
                <a:stretch>
                  <a:fillRect />
                </a:stretch>
              </pic:blipFill>
              <pic:spPr>
                <a:xfrm>
                  <a:off x="0" y="0" />
                  <a:ext cx="{WIDTH-IN-EMUS}" cy="{HEIGHT-IN-EMUS}" />
                </a:xfrm>
                <a:prstGeom prst="rect"/>
              </pic:spPr>
            </pic:pic>
          </a:graphicData>
        </a:graphic>
      </wp:inline>
    </w:drawing>
  </w:r>
</w:p>

Now, do you see those upper-case names in parenthesis (such as {WIDTH-IN-EMUS})? They're placeholders for values that must be dynamically inserted and are derived as follows:

{WIDTH-IN-EMUS}

That's the width of the image. Not measured in terms of flightless birds, but English Metric Units. There are 914400 EMUs to the inch, so it's calculated from:

widthInEmus = widthInPixels / HorizontalResolutionInDPI * 914400;

{HEIGHT-IN-EMUS}

Similarly, this is the height of the image in English Metric Units. Calculated from:

heightInEmus = heightInPixels / VerticalResolutionInDPI * 914400;

{DRAWING-OBJECT-ID}

This is a unique identifier for the Drawing Object. I calculate the initial value by looping through all of the existing <wp:docPr> elements in the document to find the largest Id, then increment it for each new image.

{FILENAME-OF-IMAGE}

Just for the sake of convention, I put the terminal name of the image file in here.

{PICTURE-ID}

This is a unique identifier for the Picture (as opposed to the Drawing Object - hmm?) and is derived in the same way as {DRAWING-OBJECT-ID} but from <pic:cNvPr> elements, not <wp:docPr>.

{RELATIONSHIP-ID-OF-IMAGE-PART}

This is what binds the image placeholder in the document to the image resource that we've embedded in the new Image Part. To get it, we need to change our InsertImage function slightly:

public string InsertImage(string imageFile, string contentType)
{
    ImagePart part = m_Document.MainDocumentPart.AddImagePart(contentType);
 
    using (FileStream stream = new FileStream(imageFile, FileMode.Open))
    {
        part.FeedData(stream);
    }
    
    return m_Document.MainDocumentPart.GetIdOfPart(part);
}

The value returned by InsertImage is the Relationship Id that we insert into this attribute in the XML.

So there you have it (for now). Equipped with your new Image Part and appropriate Open XML fragment, you should now to able to insert images, ad-nauseum, into your Open XML documents. I apologise for having skimmed over the subject at such a high level. Hopefully, I'll get round to posting some more articles on Open XML that will fill in a few of the blanks. In the meantime, if you need some specific help, let me know.

It's called StackOverflow, it's good and it's coming soon!

Steve Morgan — Thu, 11 Sep 2008 15:26:00 GMT

I've been following with interest the development of StackOverflow.com, the brainchild of Jeff Attwood and Joel Spolsky, for a while now.

It the time of writing, it's still in beta. Hopefully, barring any further hiccups, it'll be released to the great unwashed on Monday 15th September, 2008.

StackOverflow is a Q&A site for developers. So what? Well, it's really slick. There's loads of activity, plenty of good questions and plenty more good answers. The way the site's been designed, it actively promotes high-quality content and discourages junk. I like it a lot.

If you can't wait until Monday (and I couldn't), there's information on how to access the beta on the blog.stackoverflow.com site.