Roho's Diary

Writing this HTML5 vs Silverlight post

2010-03-18T22:12:00.003+01:00

... vice versa could be easy

I think that the writer is on a wrong track here as he is comparing two completely different things.

A new and fast rising standard for HTML and a browser plugin. The first being implemented across multiple platforms in rapid pace. The second being upgraded every 9 months and available on not all platforms.

Mind you I think both of these have huge possibilities and can do things that the others may not yet can do. I think they both have their unique use cases, but in most cases they have much overlap in possibilities.

Which one to choose depends on the situation and the market one is aiming for.

The article could have been a lot more subtle and a lot more realistic.

Refers to: http://www.silverlighthack.com/post/2010/02/08/Top-Reasons-why-HTML-5-is-not-ready-to-replace-Silverlight.aspx (through Google Sidewiki)

Porting a framework to Silverlight

2010-01-22T10:49:00.011+01:00

.. a journey getting to the contents

The last time I have been getting familiar with Silverlight. As a .NET developer with a big focus on the web I am interested in the possibilities of this plugin. (Eventhough, I also keep a close eye on the interesting stuff around HTML5.)

Image via Wikipedia

I always wanted to add a little more to web applications and sites that are maybe possible using clever Javascript, but are probably easier to achieve using a plugin like Flash. Being quite fluent in .NET and C# and reasonably agile in Javascript I didn't want to learn yet another language and different IDE. So, when Silverlight came on stage I was immediatelly interested. I could be using the familiar Visual Studio, familiar C#, familiar (though slightly limited implementation of) .NET framework to do some exciting new stuff. XAML came in as a new aspect, but that was already part of the WPF stuff, so not completely new. Expression Blend to create all sorts of horrible interfaces and well, the sky is not even the limit anymore.

After some useless experimentation (extending "Hello world!") I decided to stop right there and drop everything and make a good start right from the beginning. Fundamentals first.

A framework

Within our company we use a clever framework that implements the MVC pattern. We use the same framework for Windows Forms application and ASP.NET applications. By completely separating the View logic from the Model and Controller (as it should be done) we can use the same Model and Controller logic in both types of applications.

The framework has a Mediator and all functionality is handled through that using Commands. All View controls are registered with the Mediator and are signalled to update the view when "things" change in the Model. The controls then update themselves and show or hide bits of themselves.

At first there was a bit of getting used to this way of programming, but now it strikes me again and again how little code is needed to implement logic.

All logic is handled in commands. When a button is clicked the Mediator is called to execute a command with some optional parameters. These Commands are implemented in various classes and are decorated with a Command attribute and implement an ICommand interface. The ICommand interface makes that the methods that a Command needs are implemented. The Command also has access the Model through ApplicationData object. I made a SilverlightApplicationData type with some small changes to the WebApplicationData type. For instance in the WebApplicationData we grab the buildnumber from the Web.Config and there is no such thing in Silverlight, so it is dropped for this first implementation.

Anyway, just add the attribute and we are good to go.

[Command("COMMANDNAME")]

The code the ICommand interface has among others a Do() method that can be called by the Mediator. A typical Button Click EventHandler is really simple.

Mediator.Execute("COMMANDNAME", "string parameter");

Or ..

Mediator.Execute("COMMANDNAME", "string parameter", intParameter, someOtherClassParameter);

Every parameter after the command name is passed to the actual Command as argument for its constructor.

The Mediator then executes the Command. However, the framework doesn't know which Commands will be implemented. Neither do I. What is needed is a Dictionary with all available Commands.

Dictionary<string, Command> commands

How to can the framework know which commands there are? Do we need to maintain a list in a config file? A bit nasty really. But the backbone of .NET Framework comes to the rescue: Relfection.

Reflection

Using reflection we can iterate all application assemblies and scan all types and make a Dictionary with all types that have the Command attribute. These can then be stored in the Dictionary.

Sounds easy, but is it?

Windows and web is simple

For Windows and Web applications iterating the assemblies is quite simple. We just look at the content of the Bin folder and load all *.dll (and *.exe) files. Then using reflection we scan all Types for the Command attribute. When found we add them to the Dictionary.

            foreach (string file in Directory.GetFiles( path, searchPattern, SearchOption.AllDirectories ))
            {
                Assembly assembly = Assembly.LoadFrom( file );
                Type[] types = assembly.GetTypes();
                foreach (Type type in types)
                {
                    object[] attr = type.GetCustomAttributes(typeof(CommandAttribute), false);
                    if (attr.Length == 1)
                    {
                        CommandAttribute attribute = (CommandAttribute)attr[0];
                        Command command = (Command)assembly.CreateInstance( type.FullName );
                        commands.Add( attribute.CommandName, command );
                    }
                }
            }

This is all done in a static CommandManager so we only need to scan the files once.

Silverlight is slightly different

With Silverlight the situation is a bit more complicated. There is no Bin folder that can be scanned. There is a ClientBin folder, but that's on the web server and even then the files are inside the Xap file!

Inside that renamed Zip file we find the assemblies and AppManifest.xaml file in which these files are listed. Well how do we get there?

Doing quite a bit of Googling I finally got on the right track. The above code sample needs quite a bit of rework. Whe need to do a number things:

Open the Xap file
Read the AppManifest.xaml.
Parse the names of the assemblies
Load the assemblies from the Xap file

Eventhough the Silverlight application is downloaded by the browser to the client computer we can not really get at the Xap file directly. That's the sandboxing security model working against us. So we need a clever workaround.

Luckily there is a way to get there. We can download the file to Local Storage. Since we are on the web this happens asynchronously. Using the OpenReadCompletedEventHandler we can continue the process.

Note that we also check for the existence of the Xap file in LocalStorage, just to prevent repeated downloads when you refresh the web page.

         Uri xapUri = Application.Current.Host.Source;
         string[] parts = xapUri.AbsolutePath.Split('/');
         xapFile = parts[parts.Length - 1];  // the filename is the last part of the path

         using (IsolatedStorageFile store = IsolatedStorageFile.GetUserStoreForApplication())
         {
            // check if the file is available
            if (!store.FileExists(xapFile))
            {
               // when not available download it
               WebClient webClient = new WebClient();
               webClient.OpenReadCompleted += new OpenReadCompletedEventHandler(WebClientOpenReadCompleted);
               webClient.OpenReadAsync(new Uri(xapFile, UriKind.Relative));
            }
            else
            {
               LoadAssembliesFromIsolatedStorage();
            }
         }

When WebClient has completed download the stream is copied to IsolatedStorage

      private static void WebClientOpenReadCompleted(object sender, OpenReadCompletedEventArgs e)
      {
         using (IsolatedStorageFile storageFile = IsolatedStorageFile.GetUserStoreForApplication())
         {
            IsolatedStorageFileStream fileStream = storageFile.CreateFile(xapFile);
            WriteStream(e.Result, fileStream);
            fileStream.Close();
         }
         LoadAssembliesFromIsolatedStorage();
      }

Now the assembly processing can begin.

      private static void LoadAssembliesFromIsolatedStorage()
      {
         using (IsolatedStorageFile storageFile = IsolatedStorageFile.GetUserStoreForApplication())
         {
            IsolatedStorageFileStream fileStream = storageFile.OpenFile(xapFile, FileMode.Open, FileAccess.Read);

            // parse the AppManifest.xaml file
            Stream manifestStream = Application.GetResourceStream(new StreamResourceInfo(fileStream, "application/binary"), new Uri("AppManifest.xaml", UriKind.Relative)).Stream;
            string appManifest = new StreamReader(manifestStream).ReadToEnd();

            XElement deploymentRoot = XDocument.Parse(appManifest).Root;
            List deploymentParts = (from assemblyParts in deploymentRoot.Elements().Elements()
                                              select assemblyParts).ToList();

            // now load assemblies one by one
            foreach (XElement xElement in deploymentParts)
            {
               string source = xElement.Attribute("Source").Value;
               AssemblyPart asmPart = new AssemblyPart();
               fileStream = storageFile.OpenFile(xapFile, FileMode.Open, FileAccess.Read);
               StreamResourceInfo streamInfo = Application.GetResourceStream(new StreamResourceInfo(fileStream, "application/binary"), new Uri(source, UriKind.Relative));

               Assembly assembly = asmPart.Load(streamInfo.Stream);
               // from here the code is unchanged
               Type[] types = assembly.GetTypes();
               foreach (Type type in types)
               {
                  object[] attr = type.GetCustomAttributes(typeof(CommandAttribute), false);
                  if (attr.Length == 1)
                  {
                     CommandAttribute attribute = (CommandAttribute)attr[0];
                     Command command = (Command)assembly.CreateInstance(type.FullName);
                     commands.Add(attribute.CommandName, command);
                  }
               }
            }
         }
      }

Well, it took quite some googling around and advice from a friend to get this working, but in the end the steps are quite easy to understand.

Almost there ..

So now we have everything we need. Almost that is.

As I mentioned earlier Silverlight has a slightly limited implementation of .NET framework in the sense that not all namespaces are completely implemented in the same way. And that threw up another hurdle to take.

To create an instance of Command we use an overload of the CreateInstance method in which we can pass the parameters for the command in the args parameter.

ICommand command = (ICommand)assembly.CreateInstance( commandType.FullName, false, BindingFlags.Default, null, args, CultureInfo.InvariantCulture, null );

Unfortunately Silverlight does not have this overload. We cannot pass any parameters to the instance using CreateInstance. In effect only the default constructor is called. Why the overload is omitted in Silverlight strikes me as odd.

Yet, more research was needed to solve this problem. More Reflection was needed to find the correct constructor for the type.

         Type[] parameterTypes = new Type[args.Length];
         // Initialize a ParameterModifier with the number of parameterTypes.
         ParameterModifier parameterModifiers = new ParameterModifier(args.Length);

         for (int i = 0; i < args.Length; i++)
         {
            parameterTypes[i] = args[i].GetType();
            parameterModifiers[i] = true;
         }

         // The ParameterModifier must be passed as the single element of an array.
         ParameterModifier[] paramMods = { parameterModifiers };

         // Get the constructor that takes an integer as a parameter.
         ConstructorInfo constructorInfo = commandType.GetConstructor(BindingFlags.Instance | BindingFlags.Public, 
                                       Type.DefaultBinder, parameterTypes, paramMods);
         ICommand command = (ICommand)constructorInfo.Invoke(args);

This made it all work.

Mediator.Execute("COMMANDNAME", "string parameter");

The Mediator can now make a deep copy of the Command referenced by the name "COMMANDNAME" (using the constructor that takes string parameter) and then call that Command's Do() method to actually execute it.

A really interesting journey it was and now I can start to make something useful using this framework.

Top one reason projects fail

2009-12-04T08:21:00.005+01:00

.. it's not about a person, but about the process

This morning I came across an article by Amit Sarkar "Top Ten Reasons Why A Project Fails And What You Can Do To Change" and it triggered me to write a long comment to that. I don't know if anyone will ever read it as nothing appeared on the site. Might need moderation. Still I copied what I had typed into the box to reproduce it here. And the added some more text here and there.

First I suggest to hop over read the article.

My comment

Having been involved in quite a number of projects myself I think that I can honestly say that you are clearly a project manager that works according to the book. The problem is that the book is not correct. Or it might be the not the correct book. The Bible is not the best source of knowledge for project managers, nor is the Koran. Nor Harry Potter.

Genral objections

I will not break down all your 10 points.

But the biggest mistake in managing IT projects is the misconception that everything can be known at the beginning of a project. Requirements are never completely clear and will change during the project because of changing insights and demands from other systems.

Same goes for risks. Some can be identified ahead, but most risks are only becoming clear when the projects moves ahead and bumps into related systems that have limited possibilities.

It all sounds too familiar

Image via Wikipedia

The solutions you propose are related to the 'good old' waterfall model and that is no longer a best practice in IT world, if it ever was. Nowadays we would like to work in iterations taking small steps to make things work a bit and have close commitment from all team members and the project owner, the stakeholder or at least someone who can and will take decisions and who is a member of the project team as well. That way progress can be made in short cycles and the stakeholder is always involved and knows every step of the way.

Typically in such a process it is not known what comes out at the end. You might think beforehand that you need a better mobile phone and end up with something like Twitter.

Then ... what

IMHO a project manager is a facilitator for the project team. The project manager makes it possible for the team to do their work. At best he goes around serving coffee.

I have seen project managers who spend a lot of time in endless meetings with all kinds of layers of management and keeping all kinds of lists, but were never around when you really needed them. I pity them for the time wasted and myself for the time waiting for them to come back from the trenches.

I think that the points you mention are known problems, but the solutions you mention are not the way to go. In IT so many times the stakeholder has no real knowledge of IT in general and has no real idea about what it really is that he needs or wants.

To me it is important that a project team together with the stakeholder iterate over the product that will be delivered. Not everything will be known at the start (and that's okay), but that also means that you can't really plan anything ahead. It's impossible to have all requirements and all risks assesed and have solutions for these as well.

A final great to Amit Sarkar:

Good luck.

Image by Phillie Casablanca via Flickr

Conclusion

I know, I know, I am all wrong. I am not a project manager. Happy for that in a way. But I have been involved in a lot of projects that failed not because of bad team members or project managers, but because of the process that was chosen to do the project. Not as much the project management methodology (Prince or Cadence or whatever) but the process that was used to get from the start (nothing) to the end (something that all involved people can live with).

I think project management has come a long way (to the moon and back) and that in IT projects people need to understand that we do not know at the start of a project to which moon we are going and whether we are going any further than the coffee shop around the corner. Most of the time it is not rocket science.

So once again: Good luck.

The Broken Nose principle

2008-12-23T13:56:00.001+01:00

.. beware

Image via Wikipedia

broken nose as punishment

Sometimes, it seems necessary to point someone out that what he did was not reallu correct. He stepped way out of line, made a bad remark or did some damage or almost did some damage. To the company or the company's reputation. Maybe because he was just plain stupid or just did not know what he was doing or causing.

A form of punishment or retribution seems to be in order here. But once that decision is made one need to establish if a simple converstation with a stern look will suffice. Or maybe some stronger method needs to be applied.

Most often when damage could potentially have been done (note that nothing has happened) managers tend to completely overreact and forget to thoroughly investigate what really happened. They usually tend break the nose of the person closest to the cookie jar.

As a figure of speech.

Image by sean dreilinger via Flickr

It sends the wrong kind of signal

They forget everything they ever learned in management training and they start addressing the subject in full view of everyone, start shouting or plain striking away with bad reviews or firing the poor soul.

Even though all these reactions have a time and place and also have just reasons. The problem is that the manager tends to forget to think first. And to investigate before that. First get your evidence complete and then draw your conclusions. Think before you act and investigate before you think.

The defendant usually ends up with a broken nose.

Here's what wrong with this kind of reaction:

even though punishment might be needed for the culprit, the reaction is just over the top, there's no connection between cause and effect;
others will fail to understand, because they can not see the reason why you react so Big Time;
others will dislike your action, because it is in no direct correlation with the error made;
others will dislike you and stay away from you, because you seem schizofrenic at best;
as a result they will sympatize with the bad guy;
all it shows you have no respect or understanding for your people.
it is just over reaction, even when it is a correct reaction, reacting too quickly and too loudly will give a bad first impression
the event will only help to spread bad news about you and not about the bad guy: it backfires completely.

A broken nose is a bad punishment

A broken nose is not a very good way to punish someone. There are at least five reasons for that

A broken nose is very painful.
A broken nose will keep coming back over the years through minor or major annoyances.
That way a broken nose will keep the grudge alive.
Also a broken nose will make the grudge grow with years.
A broken nose can cause reaction later on: Hey Manager, always look behind you!

Image by Getty Images via Daylife

Better think twice before you act

before you strike out at anyone, think again and again and always, always get the story from both sides. Simply because when you do punish in any way, you'll never know their reaction.

you may have them silent at first, but then what?
they may just strike back directly in an unexpected way
they may know karate and you may hit yourself
maybe they're a Ninja or worse a lawyer or their bigger brother is one

On top of that: it's almost Christmas!

Converting a WebSite project to a Web Application project

2008-09-23T10:06:00.001+02:00

.. an interesting journey

In the last few weeks we have been preparing a conversion from a WebSite project to a Web Application project in Visual Studio 2005 solution. To complicate things a bit there is also SharePoint in the equation.

The reason to do the conversion was that we could not automate the build process in TFS and subsequently could not automate the deployment. Also the building of the solution itself within Visual Studio was handwork and took about forever.

What did we do?

Well, quite a lot was done to do the conversion. Along the process we were wishing louder and louder that someone had used his brain before picking the WebSite project. We had about 180 User Controls and about the same number of WebParts, that used these user controls. I know, what you want to say, probably not the best architectual design decision. But then again, when you do not take the time to think about the architectual design no one can put the blame on you for making the wrong architectual design . Duh.

We needed to do a two fold conversion: once for the User Controls and once for the WebParts.

Image by Getty Images via Daylife

Converting the User Controls

We created a new Web Application project, deleted the Default.aspx and the Web.Config files and then copied all user controls from the WebSite project. Then we did a Convert to Web Application action through the context menu in the Source Explorer. This adds a.o. the designer files.

Using a specially written tool we processed all ascx and ascx.cs files to add namespaces, change and delete some of the attributes on the <%@ Control %> node. And we added a <%@ Assembly %> node.

Converting the WebParts

In the WebParts the UserControls were instatiated in the CreateChildPanel() method by calling the constructor. That needed to be replaced by a Page.LoadControl() command. Another tool was created to do that conversion.

Bumps in the road

We first figured the process out by creating a small and simple Proof Of Concept. That worked like a breeze. Then we started on a copy of the real code. Yep, that's when the manure hit the propellor. We came across multiple problems, mostly because of inconsistencies in the coding. Sometimes there were comments when did not expect them or a code file was not placed in the correct folder. So, some of our assumptions failed and we had some manual clean up after our conversion tools. But that was limited to roughly 20 files. Not too bad.

But we also saw that our Proof Of Concept was many times less complicated than our real world code. That kept us busy with debugging for quite a few days. That and the fact that we were not experienced SharePoint deployers did nt help speed things up.

Ready to roll

Last friday we were ready to roll. We had figured everything out and had fixes for all the small problems that we encountered. So, the monday was set as the big conversion day. We mailed our offshore team in India to stop any work on the code around noon and made a branch in TFS.

The Big Bang conversion

With the script in hand we took the steps one at the time and started the conversion at half past one. Ticking off each step we were ready to try and compile about an hour later. More manure hitting the fan. Some caused by missing references and some by the new code that was created by our friends in India. They did coding while we were testing out the conversion so we were now working with a new code base. So afcing some new interesting bumps.

We only had to do some things a second time, having forgotten to update one of our conversion tools. Gradually we saw the number of build errors go down from 300 and then up a bit again. And finally at half past seven we had a fully building situation we then checked in in TFS.

Today

Today, we hear some enthousiastic noise from fellow developers who find that the solution compiles a lot easier, without any manual building of separate projects and above all a lot faster than before.

Also we are now starting testing the deployment and ironing out any last bugs that are in there.

Lessons learned

The most important lessons learned is that especially in bigger application development projects it is absolutely necessary to think before you do, design before you start building as that will save you loads of time and trouble later on.

Further we have learned a lot about conversion and file manipulation through RegEx and changing files loaded into MemoryStreams which was an interesting excercise as well.

Blog traffic by Pippi Longstocking

2008-02-12T12:55:00.000+01:00

.. weird numbers

I haven't been too active on this blog in the last few months and am only picking things up a bit again. But I always kept some traffic. And I had a liitle more detailed look at that traffic and I just had to laugh.

I noticed that a lot of people were coming to the blog from Images searches on Google. In particular I found that a photo that I linked to last year before I went on holiday to Sweden for the first time in my life. I had posted a picture of Pippi Longstocking and apparently that attracts people to my blog.

Pippi Longstocking, once again!

So here she is once again: Pippi Longstocking! This time a picture I made myself in the Astrid Lindgren park near Vimmerby. See if traffic picks up on this as well.

Implementing a Google Mini

2006-07-11T12:48:00.000+02:00

.. if only everything in life was this simple

The Google Mini implementation together with a redesign (or rather re-alignment) of our website will be put in production on the 24th October. It took longer than this post suggests because of other higher priority projects that were implemented in the past few weeks. But hey! It is coming at last.

As briefly mentioned in my previous post I am working on the incorporation of a Google Mini appliance in the Univé website. I will go into this a little bit more detailed.

Side stepping

I am a sort of a DIY-person. I like to do things myself. That means most of the time that things take longer before it is finished, but when it is finished the feeling of pride and self esteem pays for that. Also along the way of any of these DIY projects I get to stages that I have to do something new with some new tools or materials. Most of these have words on the packages like "can be assembled in 5 minutes" and over time I have learned that that is the moment to be aware, to be very aware of pit falls. Usually I have picked something to use it in a slightly different way or want to use it inside a small space so it is impossible to use a screw driver to fasten it.

I think that these problems are directly related to my lack of skills, because usually (after some tears and sweat from me and cursing the people who ever designed the stuff) I can use it for the purpose that I had intended. I think of myself as a reasonably well DIY person.

The same applies in my IT skills. I sometimes wander about and pick the wrong route at first, but usually I get stuff working, again after some tears and sweat from me and cursing the people who ever designed the stuff.

But these marketing words: "up and running in NO time" still put me in a special mode. When something can not fail I know it will.

Stepping back

So, when we had wheeled in the Google Mini appliance I was ready for weeks of administering and tweaking to finally get some search results on my screen that vaguely would like what we would want to see.

I know I have been very enthusiastic about the stuff coming from Google in the past months, years and maybe it seems like I am a blind follower of the company at Mountain View, but I am not. But I am of the opinion that have created some very good stuff in recent years. And when they haven't created it themselves they give it a twist by presenting it for free, like Google Analytics.

Back to the topic

After these lengthy opening paragraphs I am now returning to the subject of this post: implementing a Google Mini.

During the build of our new website we had dropped search as one of the deliverables for Go-Live. It was postponed to the Point One Release. You can have a nice long discussion about that, but don't bother we did as well and management decided to drop it and so we did.

We had acquired a Google Mini to facilitate the site search and had it hanging in a server rack for a couple of months gathering dust, but once we were ready for launch I was given time to explore its potential. I must admit, that I was a bit afraid to have a stab at it. To me it was a black box (although it's happily blue) and I did not see an opening. And misplacing the manuals was another hurdle we had to take and after some Googling on Google we found a way to download the stuff and have a go at it. Then a colleague opened his drawer and said that he had secured the manuals. Sigh!

Initial setup

The first things you have to do is some basic setup like giving a IP address and an administrator password. This was done very simple using a laptop connected directly to the box. Then using the web interface it was really easy to configure the crawling and after the first crawl see the first results through the familiar Googlish interface. Tweaking a bit of XSLT brought out the company colors and logo and I was ready to be enthusiastic again. This machine would provide an easy site search for our website. And so we moved on.

Integration into the site

Well, we did not want to give visitors direct access to the Google Mini, but we wanted to really integrate search into the site context. We have added a search box to our menu bar and created a new search container or template. Communication with the Google Mini is managed through a Web Service.

Since, the Google Mini already came with an API it was relatively simple to set this all up. Cutting some corners we decided to stay with a plain search screen and leave out the advanced bits for the moment. This meant that we needed a simple Web Service method that would only take a string as the search query. Added to that we wanted have navigation from one page of search results to the other and we another method was needed for accepting the (more complex) query with page indication and the like. This second method was also used for the spelling suggestions and the keymatch results (aka Sponsored Links).

Communication with the Google Mini

The Google Mini API is very simple in nature. All one needs to do is build a long URL with query parameters and you are provided with XML string that holds the outcome of the search request.

The XML string is then returned to the site and there it is loaded into an XmlDocument and then parsed using XPath queries and presented to the visitor in simple XHTML.

Presentation to the user

First of all, we did not want to reinvent yet another better wheel. We feel that Google does a great job of presenting search results. But being a web developer I also know that the code used to produces the list of results is not exactly what we call modern or semantic web design.

So, we liked what it looks like on the outside and decided to change the inside.

Show in the screen shot is part of what the page will look like. The similarity with Google search result pages should be obvious. We even have space for AdSense ;-)

The search results are presented in a <dl>, with the linked title in a <dt> and the snippet from the page in a <dd>. This seems to me a reasonable microformat for this purpose. Keeping the pages lightweight was and is one of our primary goals.

Looking back

When I now look back at this project (just before it is moved into the final test cycle and put online) I think that we did a great job. And the Google Mini is a great way of providing site search. Implementation went smooth and the only challenges were again in getting the .NET Framework work as we want it to work.

The Google Mini once configured does not need any real maintenance. If the machine should lose it's complete settings and database then setting it up fresh and let it crawl our site can be done within an hour. The Google Mini really lives up to its expectations. It was great fun to do.

Looking forward

After putting it in production we have a very strong tool for our content writers to see how a search engines sees our site and how Google presents the results. I plan to work with them to make the site even better than it is already.

In that way the Google Mini will work as a benefit by providing search to the visitors, but also as a tool to improve the search engine performance of our site. Not bad for a little blue black box.

Tags:

A new Standard in screen resolution?

2006-06-13T13:54:00.000+02:00

.. smaller or bigger?

A sequel to this article has been made. In that I discuss the current standard screen width you could design for. Based on data from the first months of 2007.

In a post today Tony Patton over on TechRepublic discusses Determining a standard screen resolution for your application. He has some point to keep in mind when designing a web site.

There is a general trend towards bigger monitors with higher resolutions. Multiple screens are close to becoming a common household item as well.
Never ever use your own screen resolution as the standard.
Nor the monitors of your colleagues. Both will be probably more state of the art than the old CRT or cheapo flat screen from Joe Average.
Screen resolution does not equal the viewable area or the browser window size. Most users will not have their browsers maximized on screen.
It is good to look at visitor statistics, but be sure to sample the right audience. Taking the numbers from a site largely visited by developers will not do when the site you are designing is aiming at pensioners.
Also keep accessibility issues in mind. The disabled users can use screen readers with or low resolution monitors with odd metrics. You should test, test, test.
You can use JavaScript to establish the viewport size and serve the client a fitting design.

All in all, a not all too shocking article, but a good summing up of things to keep in mind.

Monitors are becoming smaller, not larger!

In a comment TechRepublic member Wombat Ron then adds the increasing use of handheld and smartphones that have much smaller screens. He predicts that the 640*480 resolution could be coming back into the charts. So, it is difficult to say whether we are moving towards bigger or smaller screens.

Separating content and styling

I think Ron is right, although I think these could be handled by using the mediatype'handheld' to serve special styling through a dedicated CSS file. This is where separation comes back in its full power. With a good design of the structure of your site (the html) you will keep that lean and you can then just as easily serve that to handhelds and adjust the styling so that it will fit on a small screen.

What is wrong in the article?

One thing struck me as quite odd in the article was the example code that was in the article. This code looks at the size of the window to determine what to present to the customer. See code.

<SCRIPT language="JavaScript">  </SCRIPT>

Here the principle of separation is missed completely. The users is redirected to another page and I think that it would be more then enough to serve another CSS file.

Separation of structure and styling gives the versatile option of changing the style while still leaving the structure intact. At our new website we have a 'normal' version and an 'impaired' version. The only difference is that other stylesheets are delivered to the visitor. The html stays the same. (Mind you: this does not make the site accessible, there are still plenty of issues that we will tackle soon.)

Right now, I am working on changing the layout of the site so that it strechted from an 800 pixel wide design to a 1024 pixel wide design. I this is all relatively painful as I only need to make changes to a couple of CSS files and stretch a few background images.

Afterburner

What is the most odd really in the 'resolution business' is that apparently designs are made for 1024*768 or 800*600. I hardly ever design the height of a web page. The width is a valid parameter but the height of the page is usually determined by the content and as a designer you have hardly any influence on that unless it is your own site.

I usually design for a certain screen width. Also I have a preference for fluid or liquid designs.

Google Sitemaps: overview of benefits

2006-06-06T12:30:00.001+02:00

.. looking back with pleasure

Sometime in September 2005 Google put out the Google Sitemaps product. To help website owners all over the world to inform Google of the pages of their websites. Being a website owner, designer and developer I immediately jumped on the wagon and tried it out. I started an experiment to try the effectiveness out and see if it did really do what I hoped it would do: make my new content appear in the Google index more quickly than before.

My experiment

In a couple of posts ( 1, 2 & 3 ) I outlined the experiment I did with Google Sitemaps on my site.

What was the setup?

I created a sitemap of my website. In this XML file all the interlinked pages appear. I added two pages to my website:

a page (A) that I added to my sitemap.xml but that was never linked from any the other pages on my website
another page (B) that I did not add to my sitemap.xml, but it was linked to from page A.

Both pages had legitimate content and also link back to other pages of my website. Actually, both were to be part of my website but I merely have omitted putting links on the other pages pointing to these new pages. They are in essence orphans. But could also be seen as landing pages.

The experiment was to see if the Google crawler would visit these two pages. The first directly from the sitemap.xml and the second through crawling the links on the first page.

So what happened?

After having submitted the sitemap.xml I was filled with joy when I discovered that Google scanned the file within 24 hours. Moreover it appeared to come by twice a day. That did not mean that the pages appeared in the Google search results immediately, but still it gave me hope that they would after the next crawl.

Image via Wikipedia

So, I kept checking my website logs to see if any of the secret pages (A or B) would pop up in my requested pages list. And checked again and checked again.

Now for the real purpose of the Sitemaps: getting your pages crawled. The Crawl process can take some time before it happens. The predominant belief is that a fresh crawl of the Web is done every couple of weeks or once a month.

So, after uploading my sitemap I expected it could take up to a month before my hidden pages were discovered. But after a couple of days the first secret page A was crawled. Bang! The sitemap.xml really showed its purpose. The file was indeed used by the Google crawler to crawl pages.

The second page did not show up immediately. So apparently, the links on the page were not yet crawled. They were probably queued to be crawled in a next crawl. Later they indeed did show up and so that part of the process worked as well.

The experiment was successful and has convinced me that Google Sitemaps really add something to a website.

Added value

A bonus of the Google Sitemap system is that a website master also has the possibility to view reports on the last crawl results. This can be achieved by placing an html file with a name provided by Google to verify that you can manage the site. Google then provides you with a list of failed pages. Pages that no longer exist or have another error.

These statistics have been extended over the last few months and now show also the most used search strings and also the search queries with the most clicks. There is more like a robot.txt checker and error reports. All very valuable for a web site owner. Read more about these handy sitemap statistics.

Conclusion of the experiment

By adding a Google Sitemaps file to your website (and keeping it up to date!) you can ensure that new pages are crawled at the next scheduled crawl. If you have a popular CMS there is probably already a plugin available to create and maintain a sitemap. I can recommend it to any site owner. It is worth the effort. For me it is a little bit of work because I still use static html pages on my site.

More goodies

Some time ago after the experiment more goodies were added. What extras are hidden in the Google Sitemaps console?

Crawl statistics: Pages successfully crawled, pages blocked by robots.txt, pages that generated HTTP errors or were unreachable.
The PageRank distribution within your site.
Various indexing stats (pages indexed, etc.)

If you, for a completely obscure reason, do not wish to create a Google Sitemap, you can rest assured. You do not need to have Google Sitemap to be able to use this functionality. All you have to do is create an empty HTML file and upload to your site to verify that you own the site. And then you start leveraging the benefits. For free.

Google Sitemaps turns itself into an absolutely awesome troubleshooting tool for all webmasters.

Experimentation is fun, but real life implementation is better

This interesting experiment shows that Google Sitemaps is doing what it is supposed to do: making pages noted by the Google crawlers.

As said we launched our new website on Tuesday 30 May and implemented Google Sitemaps with it. I had registered the sitemap with Google a couple of days before the launch so it was trying to download the then still not existing sitemap. On launch day around 6 AM our time the sitemap was downloaded by Google. That was the first possible moment less than two hours after the site went live.

Success #1: new pages are already in the index

Two days later we see already many pages in the Google Index. When I did a search on the first 100 pages changed in the last three months on Google I found that of these only 12 are still pointing to our old site. My guess was that these are there only because we had some problems setting the rewrite rules for the old asp files. That was not entirely correct as a week later we still see old pages, but those are different ones.

The Google index now already returns new pages only days after going live. Without the Google Sitemap this would taken much, much longer.

Success #2: we have a check on what goes wrong

Thanks to the statistics that we receive from Google we can now track the old, no longer existing pages that are being crawled. We can see errors popping up and if necessary we can further tweak our web application or web server to gracefully handle these.

Conclusion

Implementing Google Sitemaps has given us already huge advantages in the first week of going live. By having the Google Sitemaps ready at go-live we have been able to have Google find our new pages and content.

I would strongly recommend implementing Google Sitemaps with any web project so the pages will appear as quickly as possible in the index. Even though we had to implement the Google Sitemaps ourselves within our site framework it was not the greatest of challenges we had.

Many of the (open source) content management systems nowadays provide Google Sitemaps out of the box or through a plugin. So, there is hardly any reason not to implement them.

Update (4th September 2008)

The Google Sitemaps protocol has in the meantime been adopted by more search engines and is now als an integral part of the Robots.txt. So, it's nowadays even more important to use the protocol.

Better still: It's now an open standard.

Tags:

Ektron cms400.NET has an open back door

2006-03-03T14:05:00.001+01:00

.. and most admins don't know

Updated! See below.

In my daytime job I am busy with some colleagues to put together a new website for the company we work for. An interesting job involving many exciting subjects like:

XHTML & CSS
C#, ASP.NET and the .NET Framework
lots of OO and Patterns
and the exciting Ektron cms400.NET content management system

We have had our share of struggles with this system. Getting it to work with our framework which involves talking to the web service side of the cms and finding that not all functionality from the server controls were available in the web service. On top of that we have found problems with the performance. In the mean time we have worked around these problems and are now in the finishing phases of the long running project.

Now what's with that back door?

Well, it sounds pretty harsh like accusing Ektron of some sort of crime, but that's not my point. The back door is mentioned in the manual (at page 27) and there is a warning for administrators there so any admin who reads his manuals thoroughly ..

(space left blank to count the entire two of them)

any admin who reads his manuals thoroughly can close that back door.

A quick Google around the world by two of my colleagues showed quite a few instances of the exciting Ektron cms400.NET with their login pages exposed.

Then it was simply trying the admin account with a password I will not disclose (you go and guess it) and they were in. That was simply too stupid. Every admin should at least have the brains to change the admin password.

For most of these sites the admin password was changed, but the second built in user account (you try to guess it again) was usually still wide open!

So, it's just stupid admins?

No, it's not stupid admins, they usually are not stupid. Those who didn't change the admin password can be rated stupid, but those who left the second user exposed are not. I blame Ektron for that.

I think it is quite normal to have an admin user account in a system and maybe they should force changing the password the first time the admin logs in to the system. But I do not understand why there is a second user account that there that is completely hidden in the user list so most admins will not even know of the existence unless they have read page 27 of the manual.

So, most of the companies and organizations that use the exciting Ektron cms400.NET are completely unaware of the vulnerability of their web sites. After finding that we could login we immediately logged out. Less friendly people could cause a lot of harm.

And you?

We are completely safe with our website. We have simply left out all the admin stuff from our website. We manage the content from within our safe network.

Advise to Ektron

Inform your users and leave out this second user account in your future releases.

Update [3 March 2006]

Eventhough this article has been brought to the attention of Ektron and some site owners that had their website completely open to anyone clever enough to find the open door, no action has been taken by neither Ektron nor site owners.

The Internet is a jungle but they think it is OK to leave the door open with a welcome math in front of it.

Update [13 March 2006]

Ektron has taken notice and is taking action, read this.

Tags: