Almost every Web Content Management (WCM) solution I built on top of Microsoft Office SharePoint Server (MOSS) 2007 required using one or more Lookup fields. Comparing to other alternatives like a Choice field, a Lookup field allows users to maintain the list of value themselves in an intuitive way and all that without modifying any of the existing Site Columns.

Some of the Lookup fields, like Categories or Tags, must allow selecting multiple values. And this is where the problems begin: the Content Query Web Part doesn’t support MultiLookup fields at all. As soon as you include a reference to one in your query, you won’t get any results.

This problem isn’t new and has been mentioned on the Microsoft Enterprise Content Management (ECM) Team Blog nearly three years ago: “SPSiteDataQuery and by extension the Content Query Web Part doesn’t support multi-value lookup fields”.

Considering how often you might want to retrieve the value of a MultiLookup field in a content aggregation the lack of support for it in the CQWP becomes quite problematic: does it mean that we must develop our own alternative for aggregating content in order to use some of the basic functionality of Windows SharePoint Services v3? What about the performance: are you sure that it would perform as good as the standard Content Query Web Part?

If you do want to use the Content Query Web Part and you do want to make this work, there are a couple approaches you might consider.

Custom Multi Lookup Field

One of the first things that could come to your mind might be developing a custom Lookup field which would allow selecting multiple values. Because the LookupMulti field type isn’t supported by the CQWP you could create a totally custom field type deriving from the Text or Note field type. The value of a Lookup field with multiple values is a ;# delimited string, such as id1;#value1;#id2;#value2 which can easily be converted to the SPFieldLookupValueCollection type so that you can work with the stored values in a way comparable to the standard LookupMulti field. You could even go a step further and provide a similar User Interface to the one the LookupMulti field has using the GroupedItemPicker control.

While this is definitely a plausible approach, many people find working with custom field types rather complex. There are quite a few different components you have to consider while creating a custom field type if you want to make it fool-proof. The worst of it all in my opinion is, that a custom field type is available in all Web Applications across the whole farm. In other words: after you deploy your custom field type everyone else is able to use it in their Web Applications as well.

Custom XSLT function

Content Query Web Part uses XSLT for data presentation. The great thing about this combination is that it allows you to extend the standard transform process with your own XSLT parameters and functions.

For the purpose of retrieving the value of a MultiLookup field, you could create a custom function which would take three parameters: ID of the site, ID of the List and ID of the item. Using all these ID’s you would retrieve the item and return the value of the LookupMulti field.

CQWP automatically includes all these ID’s in the results, so you don’t have to do anything extra in order to retrieve them. The downside is however that such function would run in the scope of the item itself meaning: for every single item in the result set you would open a reference to a site, to a list and run a query to retrieve the particular item. And while there are some ways to optimize that process, it’s still quite poor from the scalability and performance point of view.

Depending on your scenario it might or might not be a suitable solution for your situation. The only way to find out about it is to build a proof-of-concept and test it in your solution/environment.

Custom XSLT function++

As I already mentioned the Content Query Web Part includes all different ID’s required to retrieve the SPListItem object for the given result. What the CQWP also does by default, is that it includes all results in the output XML. That gives you yet another approach to retrieve the values of MultiLookup fields.

What if we first gathered all the information about all the items from our result and then use that information to build one query (or a few depending on what kind of results your Content Query Web Part returns). Instead of running one query per item, you could run one query for the whole Web Part, store the results in a variable and retrieve them from the ItemTemplate for every item using the basic XPath selectors.

While this approach is definitely more complex than the previous one, it would definitely prevent you from putting some heavy load on your server. Although I haven’t tried this approach in real-life yet, I’m quite sure it would work and would help you get things done inside the Content Query Web Part without a too big performance penalty.

There is even more to XSLT functions

Using custom XSLT functions has yet one more advantage: you could leverage the same concept as described above to retrieve some related information for every item. Because CAML doesn’t support any JOIN mechanism, using custom XSLT functions might give you a solution for this shortcoming as well.

Summary

Content Query Web Part provided with Microsoft Office SharePoint Server 2007 is a great solution for aggregating content. In spite of its great power it has some serious shortcomings among which the lack of support for Lookup fields with multiple values. Using custom development you can create a solution for this challenge and leverage the power of the Content Query Web Part in your solutions.

It’s not bad to learn things through working with them. Theoretical knowledge is not as valuable and useful as real-life experience in my opinion. The downside is however, that if the only thing you’re doing is learning through practice, the odds are high that you will miss some things and not fully benefit of everything that’s already there.

For the last 2,5 years I’ve been working with Microsoft Office SharePoint Server (MOSS) 2007. Right from the beginning I started with development on the SharePoint platform. Because I had a few years (web)development and webdesign experience it wasn’t that difficult for me to adapt to the new development platform. And although I was doing my job and was getting things done, I was reinventing things without even knowing it. That is why just recently I decided to fill in all the gaps I had with the knowledge I should have had a long time ago.

The great thing about it was that about a half year ago Imtech ICT Velocity – the company I work for – started a new education program led by a colleague of mine and a great trainer Erik Schoenmakers. Within that program I got room to learn about the technologies I was working with every day.

I started off with the (70-536) Microsoft .NET Framework – Application Development Foundation exam. Developing on the SharePoint platform means developing in the .NET framework. It is therefore important that you know what the framework offers you and how things work. I prepared for the exam using the Self-Paced Training Kit and I have to admit it was a great experience. The kit covers all aspects of the .NET framework: from things I use daily like XSLT, Regular Expressions and Tracing to stuff I never work with as a SharePoint developer like drawing.

Next I took the (70-562) Microsoft .NET Framework 3.5, ASP.NET Application Development exam. This time I attended a two-day course delivered by Erik. And although the course didn’t go as far as the Self-Paced Training Kit I enjoyed discussing all the different things with other students.

Just when I thought that I had it all covered I stumbled upon yet another certification: (70-564) Pro: Designing and Developing ASP.NET Applications Using the Microsoft .NET Framework 3.5. From what I’ve seen and read about this certification on the Internet it seemed like a great match with all the things I’ve been working with for the last 2,5 years. There is one thing about that particular exam however: there are no official preparation materials for it! So does Microsoft expects us to pass it using nothing else than the basic knowledge and real-life experience? After all the 70-564 certification is targeted to developers with at least three years experience in working with ASP.NET.

As preparation I read the Pro ASP.NET 3.5 in C# 2008 book by APress. This book covers all core concepts and fundamentals of ASP.NET 3.5 as well as some more advanced topics. While this book isn’t really a deep-dive into real-life ASP.NET it gives you a lot of examples of what ASP.NET offers you out of the box.

After a couple weeks of studying and doing exams, I became Microsoft Certified Professional Developer in ASP.NET 3.5:

The first title element was the one defined by me in the Master Page. That title element correctly displayed the title of the page using a custom control:

For some reason SharePoint was automatically adding the second – empty title element during the rendering process of the page.

Having two title elements in an (X)HTML page is not allowed and results in an error during the markup validation process:

It is not the first time I have noticed such behavior. I saw it for the first time a while ago, while I was working on the Imtech SharePoint Compliancy Toolkit to improve accessibility of WCM solutions built on top of MOSS 2007. Because the toolkit uses HTML post processing to clean up the markup it allowed me to remove the second title element in the markup cleaning process.

In the project I’m working on at the moment there is no accessibility requirement which requires the use of the Imtech SharePoint Compliancy Toolkit. We were asked though to deliver an XHTML Transitional valid markup. To find out the reason of the doubled title element I did some research on the page rendering process.

One down one to go

During my research I’ve found that rendering two title elements is caused by not including the title element as a child element of the head section of the page. In the Master Page we were using, the title element was wrapped in a ContentPlaceHolder. It seemed like at some point SharePoint is checking whether the page has a title and if it doesn’t, it adds a new title element to the head section of the page.

Looking a bit further in the rendering process I noticed that it’s not SharePoint that is responsible for the check but ASP.NET! If I’m correct the HtmlHead control checks whether a title element has been defined. If not it creates a new title element. Because the title is wrapped in a ContentPlaceHolder it’s not being detected as a child control of the HtmlHead control and thus the check fails resulting in an additional title element.

The fix

The solution to this problem was straight-forward. After removing the ContentPlaceHolder from the head section and adding the title element just below the encoding definition, the second title element disappeared from the output:

Lessons Learned

Because I was working with MOSS 2007 I suspected it to be the reason of this faulty behavior. It turns out that although SharePoint enriches the rendering process, ASP.NET still has a lot of influence on how the output HTML looks like. Whenever something works not as you would expect it to, it might be helpful to not only look at SharePoint-specific components but those provided with ASP.NET as well.

At some point I compared the Master Page I was using with the Minimal Master Page sample provided on MSDN. Surprisingly the Minimal Master Page has exactly the same issue as it wraps the title element in a ContentPlaceHolder. Understanding the code you are using is a very important part of the troubleshooting process.

Expire immediately!

MOSS 2007 allows you to define the publication and expiration date and time for every Publishing Page. As soon as you finish editing the page and save your changes SharePoint will take it over from you and will publish the page on the given date.

You can provide the publication date and time using the so called Publishing Schedule Control:

A similar control is available to enter the expiration date:

Have you noticed that SharePoint asks you whether you want your page to expire Immediately? Something seems to be wrong here.

SharePoint allows you to set the schedule for a Publishing Page in at least three ways.

First of all you can go to Page > Page Settings and Schedule page. In the schedule section you will find date and time pickers to enter the publication schedule information:

Did you notice that the first option for the End Date says now Never instead of Immediate? Let’s check it in one more place.

Another spot where you can edit the schedule information is the Page Properties page:

Also here you are able to make your page never expire. So how come the users see the Immediately label while editing the scheduling information through a Page Layout?

Is it that simple?

MOSS 2007 ships with two controls responsible for rendering the schedule picker. There is the PublishingScheduleFieldControl which can be used on Page Layouts and the PublishingScheduleControl which SharePoint uses on the List Forms and on the Page Settings and Schedule page. The PublishingScheduleFieldControl control is nothing more than a wrapper for the PublishingScheduleControl. On page load it retrieves the value from the attached Column and passes it to the PublishingScheduleControl. So how come things get wrong on the Page Layout?

The PublishingScheduleControl has the UseAsEndDate public property (not documented in the MOSS 2007 SDK) which is responsible for rendering the proper label on the first radio button. If it’s a publishing start field it renders Immediately and if it’s a publishing end field it renders Never. As I mentioned the PublishingScheduleFieldControl uses the PublishingScheduleControl as the data input interface. Unfortunately it doesn’t provide any way to set the value for the UseAsEndDate property which therefore remains set to false.

The challenge

When things go wrong in SharePoint I usually try to fix it using the simplest solution possible with as little custom code as I can. The more you develop, the bigger the chance that you will introduce a brand new issue yourself.

So first of all I thought about creating a new Field Control, inheriting from the PublishingScheduleFieldControl, overriding the CreateChildControls method and getting the things right. Using a public property I would make it possible to set the value of the UseAsEndDate property of the underlying PublishingScheduleControl. Unfortunately, the PublishingScheduleFieldControl is sealed so you cannot inherit from it.

Another solution I had in mind was to create a new Field Control which would include all the functionality that the PublishingScheduleFieldControl has and then extend that with my custom UseAsEndDate property and some extra logic in the CreateChildControls method. Unfortunately for me it turned out, that the underlying PublishingScheduleControl is public but doesn’t provide a public constructor. It seemed like I would have to create a new control for this as well… So is it even possible to solve this issue without rebuilding a dozen of SharePoint controls?

The fix

Then I stumbled upon yet another idea: why not use a Control Adapter to set the property correctly? Initially I came up with the following piece of code:

public class PublishingScheduleFieldControlAdapter : ControlAdapter
{
  protected override void CreateChildControls()
  {
    base.CreateChildControls();

    PublishingScheduleControl scheduler = Control.Controls[0] as PublishingScheduleControl;
    if (scheduler != null)
    {
      scheduler.UseAsEndDate = true;
    }
  }
}

It worked but the problem was that it got applied to all PublishingScheduleFieldControls: both for the start and the end date. So how we could distinguish one field type from another? Adding another property is not an option since the PublishingScheduleFieldControl is sealed. We could wrap it in another control and then using the Parent property determine whether we should process it or not. I thought of an easier way though and yet quite elegant one if you ask me.

I extended the code of my Control Adapter as follows:

public class PublishingScheduleFieldControlAdapter : ControlAdapter
{
  protected override void CreateChildControls()
  {
    base.CreateChildControls();

    BaseFieldControl schedulerFieldControl = Control as BaseFieldControl;
    if (schedulerFieldControl.CssClass.Contains("UseAsEndDate"))
    {
      PublishingScheduleControl scheduler = schedulerFieldControl.Controls[0] as PublishingScheduleControl;
      if (scheduler != null)
      {
        scheduler.UseAsEndDate = true;
      }
    }
  }
}

And then changed the PublishingScheduleFieldControl markup in the Page Layout:

<PublishingWebControls:PublishingScheduleFieldControl FieldName="PublishingStartDate" InputFieldLabel="Scheduling Start Date" runat="server" />
<PublishingWebControls:PublishingScheduleFieldControl FieldName="PublishingExpirationDate" CssClass="UseAsEndDate" InputFieldLabel="Scheduling End Date" runat="server" />

And this is the result:

Using the CssClass property of the control I was able to distinguish the publishing start date control from the publishing end date control and apply the custom logic only when required.

To attach the Control Adapter to the PublishingScheduleFieldControl control I have used the following .browser file:

<browsers>
  <browser refID="Default">
    <controlAdapters>
      <adapter controlType="Microsoft.SharePoint.Publishing.WebControls.PublishingScheduleFieldControl, Microsoft.SharePoint.Publishing, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" adapterType="Imtech.SharePoint.Controls.PublishingScheduleFieldControlAdapter, Imtech.SharePoint.Controls" />
    </controlAdapters>
  </browser>
</browsers>

What I already knew

From the previous trials that I did I knew, that using PortalSiteMapProvider is the fastest way to perform data aggregation on MOSS 2007. Whether you will get the full benefit of it depends on your content and cache settings. If you optimize the cache settings considering the number and frequency of visitors the PortalSiteMapProvider can help you build a great performing site.

In my previous researches I focused on querying content stored in multiple lists/sites. The challenge was to minimize the penalty of retrieving content from multiple locations and merging it into one result. This time the challenge looked a little bit different: there was only one location I had to query, I knew exactly which list it was. All I needed to do was to retrieve a particular list item using its ID (SPListItem.ID).

Although I knew all the different approaches from the previous trials I noticed that the SPList class had one method designed to retrieve items using ID’s: SPList.GetItemById(int). And although I knew how fast aggregations based on the CrossListQueryCache and PortalSiteMapProvider classes were, I really hoped that this particular method would turn out to be a real gem. After all it was made to do nothing else than retrieve one list item using its ID – exactly what I needed.

The test

To test the various approaches I have created a small Console Application on my development VPC.

I have tested the following approaches:

• using SPQuery on the list instance
• using the GetItemById(int) method
• using SPSiteDataQuery on the parent Web
• using SPSiteDataQuery on the parent Web passing the List ID (using the SPSiteDataQuery.Lists Property)
• using CrossListQueryCache
• using PortalSiteMapProvider

These are the results I got:

The vertical axis represents the duration in milliseconds. The horizontal axis represents the run number. I have executed each query 10 times. Because initial runs are quite heavy due to all the objects that have to be instantiated I have removed them from the chart above.

As you can see PortalSiteMapProvider and CrossListQueryCache are the absolute winners. The PortalSiteMapProvider is almost 10 times faster than the SPSiteDataQuery approach and almost 100 times faster than using the dedicated GetItemById method!

The results

I was really disappointed with the poor performance of the GetItemById method which was the worst of all approaches I have tested. It was almost 3 times worse than running an SPQuery query on the list.

What surprised me is the difference between the initial and all subsequent queries using the regular SPSiteDataQuery object and an SPSiteDataQuery which uses the List ID to narrow the query scope. By initially retrieving the ID of the list and storing it in a static variable you can win 2,5 times performance on the initial call and more than 15 times on the subsequent calls!

Surprisingly or not once again the PortalSiteMapProvider turned out to be the fastest way of retrieving data stored in SharePoint. Once again the second place was taken by the CrossListQueryCache object. Please note, that both of them require passing the List ID to the Lists query property. Because both approaches use cross-site querying that’s the only way to narrow the scope to retrieve the item with a specific ID from a particular list.

One word on PortalSiteMapProvider

PortalSiteMapProvider is undoubtedly the winner when it comes to effectively retrieving content from a SharePoint site. There are two things however that you have to keep in mind when choosing for the PortalSiteMapProvider class.

First of all it’s available in MOSS 2007 only. If you’re developing for WSSv3 you won’t be able to use it. You would have to use the SPSiteDataQuery class instead.

Second of all, and probably the most important, calling the PortalSiteMapProvider.GetCachedSiteDataQuery method won’t give you the great results you would expect of it straight away. The PortalSiteMapProvider strongly relies on cached objects and it is therefore important that you optimize cache for your site, your content, your update frequency and your visitors. There are no magic numbers to this one. If you want to get the most of the PortalSiteMapProvider, you will have to measure the traffic on your website and tweak your settings. Not doing that will most likely work counterproductive: the PortalSiteMapProvider will query the content on every call instead of using the cached results.

What I learned

The first thing I learned from this research is that specific methods are not always better than the multi-purpose methods. GetItemById proves that: it’s almost 70 times slower than the PortalSiteMapProvider which allows you to do much more than only retrieving a single item using a CAML query.

If you really want to be sure if the method you’re using is the best for your scenario, the only thing you can do, is to create some kind of test and compare the results. It might be not perfect and you might miss some things, but at least it will give you a comparison which is better than a wild guess.

During this test I also found out how to use a PortalSiteMapProvider out of context of a Web Application. It turns out that you not only need the context information, but you have to include the siteMap element in App.config of your application.

Summary

SharePoint provides you with many different ways to retrieve a specific list item. Although it offers you some specific methods you should be careful while choosing them for a real-life solution and test them to see if they are really the best match for your scenario.

When ’null’ is null goes wrong

For the last couple of days I’ve been developing pieces for the brand new MOSS 2007 WCM solution for one of our customers. I’ve been working on custom Content Types, Site Definition, branding and controls. At some point I got enough resources to perform an initial build using my custom components. While editing the content of one of the pages I noticed a JavaScript error:

The first thing I thought of was the Rich Text Editor SP1 error. After looking closer at the issue it turned out to be something else.

The Master Page I was using was shipped with some custom JavaScript. Removing it didn’t change anything. Even removing CSS didn’t help: I was still getting the JavaScript error after clicking on the Rich Text Editor:

Surprisingly enough none of my colleagues using IE7 had the issue: it turned out to be something IE8 specific!

Internet Explorer – the (r)evolution

For years Internet Explorer was known for his non-standard rendering of HTML. If you’ve been working with webdesign/-development for the last few years: how many times have you had to implement a check if the browser was IE or not just to make things work? Finally after all these years Microsoft decided to change the way the things were going to turn IE into a proper browser.

While the IE team did a great job making Internet Explorer work according to the web standards, there is a big drawback to that change: legacy components “optimized” for previous versions of Internet Explorer don’t work correctly anymore or even worse: don’t work at all.

One of such components is the Rich Text Editor shipped with Windows SharePoint Services v3. While it works perfectly with IE6/7 it breaks in IE8. I haven’t found out what exactly isn’t working but only the fact that I’m getting a JavaScript error while editing text is alerting.

The back door to the past

The Internet Explorer team has followed the way things were going and has listened to webdevelopers all over the world. They knew that making IE working as it should, would cause trouble. Considering all the legacy web applications out there the IE team decided to build into IE8 a back door to the past: a way to make IE8 behave like IE7.

How it works? It’s really simple: all you have to do is to include the X-UA-Compatible meta element in your page and IE8 will automatically switch to the compatibility mode and render your page as its older brother:

<meta http-equiv="X-UA-Compatible" content="IE=7" />

The IE team wasn’t the only one knowing about the rendering issues. The SharePoint team has noticed the problems as well and at some point they released updated versions of all Master Pages: all of them containing the X-UA-Compatible meta element. Not surprisingly: if you’re SharePoint farm is up to date and you’re using IE8 you won’t see a change and things will keep working as they were before.

There is more to the challenge

While the solution to the SharePoint vs. IE8 rendering issues seems really simple, there is more to it. It is really simple if your complete solution uses legacy markup. Making IE8 behave like IE7 in all situations will fix things for you. But what if you work with branding developed according to all standards and the only issue is the editing experience in SharePoint? Using the compatibility mode in all situations might cause even more rendering issues than before, getting you even further from getting your things to work. So what should we do? Luckily there is a better way…

As the SharePoint editing mode is the only area causing trouble why not render the X-UA-Compatible meta element conditionally when in editing mode only? It would solve the rendering issues and wouldn’t mess up your branding while in display mode. The best of it all is that you can achieve that using the out-of-the-box available SharePoint controls:

<PublishingWebControls:EditModePanel runat="server">
  <meta http-equiv="X-UA-Compatible" content="IE=7" />
</PublishingWebControls:EditModePanel>

MOSS 2007 ships with the EditModePanel control, which has no User Interface and all it does is to render its content in editing mode only. It is extremely useful to render input fields to manage content and Page properties but also to output conditional content… like the X-UA-Compatible meta element.

SharePoint and menus

If you ever looked at the out-of-the-box available Master Pages provided with MOSS 2007 you know that they use the standard ASP.NET menu control to render the navigation. While it definitely does the job, have you ever looked at the rendered HTML?

Compare how much markup is being used for a couple of menu items which could be rendered like:

<ul>
  <li><a href="/PressReleases/Pages/default.aspx">Press Releases</a></li>
  <li><a href="/Search">Search</a></li>
</ul>

See the difference? A couple of issues should become clearly visible at this point.

ASP.NET menus are poorly accessible

Have you ever took a look at HTML as seen by screen readers or search bots? Because the links are buried under a bunch of tables it’s difficult for such applications to read them correctly. All the tables around the links make the visitors of your site loose the track of what they are “looking” at and just navigate away from your site. Additionally many search bots can’t properly deal with too complex markup. Making the HTML of your pages too complex can result in lower rating of your content.

How much markup do I have to download?

The markup of a standard ASP.NET menu I have showed you in the image above is 1308 bytes. Rendering the same menu using an unordered list is 185 bytes only! The standard ASP.NET menu produces 7 times more markup than you would want it to. Also consider the fact that the menu above consists of only two items. What if we were talking about a medium size website with 20 links? How much markup would you have to download only to see a couple of links?

Branding costs way too much time

Have you ever met a designer who would produce HTML that would match the markup of the ASP.NET menu in 100%? I’m glad I haven’t. Somehow designers have more experience with writing correct markup than most ASP.NET developers. Good designers seem to know what is going to work and what wouldn’t. If a designer gives you branding implemented as static HTML pages, CSS, JavaScript and the only thing you would have to do is to copy & paste all the different pieces in MOSS 2007, you would still spend hours on getting it all working with the CSS you received. There is no chance that branding a table would work with CSS designed for a list. You would have to change one or the other and that costs time – a lot of it.

ASP.NET menu anatomy

ASP.NET menus are not that bad. There are situations when you could use them and experience no problems at all. There are still many applications being developed for environments where performance, bandwidth and accessibility are not an issue. And although I would totally agree that it’s a bad practice to deliver poor markup, people do it, and it works.

ASP.NET menus have one great concept implemented in them: the templates. Using a template you can define placeholder for your data which is then filled using data binding. Although this concept could theoretically give you full control of the rendered HTML markup and separate the logic of the menu control from the rendered HTML, the ASP.NET menu fails to do the job correctly. Yes: you can define your own template but on the output everything you defined is still being rendered in a cells of a nested table.

We should know better

By know many people found their way around the issue of ASP.NET menus on MOSS 2007 Web Content Management solutions. So have I. In the last few years I have worked on a few Web Content Management solutions built on top of MOSS 2007: all of them being Internet-facing sites. In all that time me and my colleagues were trying to find a way to get rid of all these tables and get some more control over the rendered HTML to make it match the branding we had to implement.

At some point we stumbled upon the CSS-friendly Control Adapters. And while they definitely do the job I wasn’t particularly happy about writing HTML using .NET code. Mixing the logic of the Menu with a Control Adapter and the HTML rendering didn’t make it easier for us to maintain and modify the code when needed. Somehow it wasn’t the grail we were looking for.

In the meantime we tried some other approaches including inheriting from the standard ASP.NET menu controls, overriding stuff and creating our own controls from scratch. After all how difficult it can be to create a menu control that gets the data from a Site Map Provider and renders them as an unordered list? My answer? Much more difficult than it should be.

A solution (candidate)

Recently I decided to give it another chance. I still wanted to get the minimal amount of custom code, full control of the rendered HTML and preferably not use .NET to write HTML. I was still charmed by the idea of having these templates you could define in your Master Page/Page Layout.

Here are the results of my work:

Doesn’t look that awesome, does it? Now let’s have a look at the rendered HTML:

And this is how it did it:

To keep a long story short: I have developed a custom control that allows you to define custom templates and using data binding renders the menu items using your own templates!

Using the TemplatedMenu you can add a menu control to your page and use your own rendering without writing a line of code! Comparing to using a Control Adapter, the deployment of the TemplatedMenu is fully supported using the standard SharePoint WSP. If you want to use a Control Adapter in your solution, you have to deploy not only the assembly but also the .browser file what gets challenging in environments with multiple web front ends.

The TemplatedMenu is available on CodePlex. Feel free to experiment with it. I’m curious to hear if it works for you and if there is anything else you find challenging.

To help you understand what I’ve built and how it works I’ve included XML comments in the code. If you’re using DevExpress you can easily view it using the Documentor plugin.

The TemplatedMenu is provided as-is. Although it would be really cool if it could help you in your solution, I give you no guarantee it will. You use it at your own risk.

Download: Imtech SharePoint TemplatedMenu (ZIP, 12KB)

Hello World

To be honest I haven’t worked with WCF services yet. Somehow the new technology passed me and I didn’t even give it a chance. Recently I stumbled upon Daniel Larson’s book Developing Service-Oriented AJAX Applications on the Microsoft Platform. Not surprisingly Daniel heavily uses WCF services to show us what can be achieved using ASP.NET 3.5. Reason enough to try some things out myself.

So how do you start with WCF and SharePoint? Sahil Malik did a great job covering that particular subject. Sahil wrote a series of topics covering almost every aspect of the process of working with WCF in context of SharePoint: from creating the directory hosting the WCF services, through configuring the SharePoint Web Application to accept WCF requests, to deploying WCF end points as solutions.

One of the steps in configuring a SharePoint Web Application to handle WCF requests is to register a Virtual Path Provider. It turns out that SharePoint doesn’t handle requests to .svc files properly. Sahil did a great job and created a CodePlex project called Winsmarts SharePoint WCF Support which contains all the components required to make it work. The downside is that Sahil didn’t create a SharePoint Solution (.wsp) for it, so you’d have to make one yourself. It’s absolutely worth doing as it’s something you’ll be using in all your SharePoint+WCF projects. I chose the simple approach and used WSPBuilder to get the job done. I copied Sahil’s code, wrapped it in a project and used WSPBuilder to create a Solution for me. To spare you that 10 extra minutes of work I published my WSP on CodePlex @ http://imtech.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=28399.

Following Sahil’s instructions I was quite fast up and running: I had my custom WCF service running on a SharePoint Web Application:

Then I decided to take it a step further and check if it properly handles SharePoint context.

Why is context so important?

Every request you make in ASP.NET is wrapped in a context object. That object contains a lot of information including current location, authenticated user, etc. SharePoint uses that mechanism too. Every request made towards a SharePoint site is wrapped in an instance of the Microsoft.SharePoint.SPContext class. Comparing to the plain ASP.NET context object, the SPContext contains a lot of valuable SharePoint-specific information including references to the current Site Collection (SPSite), Site (SPWeb), List (SPList), Item (SPListItem), etc. Considering that almost every action you would perform using a WCF service on SharePoint requires one of these objects it is very useful to have them around instead of instantiating them yourself.

SharePoint context magic

How the context works? If you’ve been working with SharePoint you probably know of the _layouts Virtual Directory. In IIS Manager it looks like a regular IIS Virtual Directory. The cool thing is the SharePoint magic that happens behind the scenes: you can append _layouts to every existing path on your SharePoint site and while it still points to files located in 12\TEMPLATE\LAYOUTS on your web server, it instantiates context objects based on the URL, eg. http://moss/_layouts/settings.aspx opens settings for the Root Site but http://moss/PressReleases/_layouts/settings.aspx opens settings for the Press Releases Site. In both cases we’re requesting the same settings.aspx page located in 12\TEMPLATE\LAYOUTS but it does different things depending on the URL used to call it. See the magic?

Following Sahil’s advice to keep the things clean, I have decided to create a directory called _wcf under the root of my Web Application. Additionally I have created a Virtual Directory pointing to it:

Unfortunately the SharePoint magic wasn’t working for me:

It turns out that the SharePoint magic works for Virtual Directories owned by SharePoint not all Virtual Directories under a SharePoint Web Application. That takes us to two conclusions.

context != context

There is context and context that we’re talking about while referring to a WCF service running on a SharePoint Web Application. The first one is all about making a WCF service available through the URL of a Web Application using the SharePoint technologies. The request gets authenticated and you are able to get some information using the SPContext object. Let’s call it SharePoint context.

The second context is the Site Context. Remember how we were able to call one and the same settings.aspx page in context of different sites (Root Site and Press Releases Site) and make it behave differently depending on the URL? Site Context is all about being able to fully benefit of the context information provided by SharePoint and leverage it for your purposes. As far as I know such functionality is available to Virtual Directories owned by SharePoint only. While debugging the custom Virtual Path Provider I’ve found out that a request to http://moss/PressReleases/_wcf/SharePointTestService.svc is being passed to the Virtual Path Provider as ~/PressReleases/_wcf/SharePointTestService.svc A similar request to http://moss/PressReleases/_layouts/settings.aspx is being transformed to ~/_layouts/settings.aspx before it even hits the Virtual Path Provider! See the difference?

Once again it’s all about context

The second conclusion takes us to a question: do you really need Site Context in your WCF service to make it useful? It depends on what it is supposed to do and how you will leverage the functionality. Theoretically you would be able to get to the data in SharePoint even without the context object. As your service is running on a SharePoint Web Application you could instantiate a reference to Site Collection and all other objects you might need.

Whether your WCF service requires context or not determines your deployment process. WCF services that depend on Site Context must be deployed in one of the Virtual Directories owned by SharePoint: preferably _layouts or _vti_bin (where all SharePoint web services are located by the way).

Many people are concerned about deploying to locations owned by SharePoint. First of all there is a chance that a SharePoint update or a Service Pack, or a future version might deploy a file with the same name to the same location. Any existing file would be overwritten. You could mitigate this by creating a subfolder and store your files there. Choosing a non-general name (something else than wcf) might be a wise choice if you want to be sure that nobody will overwrite your files.

Another thing many people worry about is the security: everything you put in one of the SharePoint’s Virtual Directories is available across the whole Farm. You could partly suppress this by deploying the service’s assemblies to a specific Web Application. Everyone who would try to reference the service through a different Web Application would get a nasty error. It’s not a perfect solution but it could help you to solve some security issues.

On the other hand there might be situations when you won’t be needing the Site Context. In such scenario you can create a Virtual Directory under your SharePoint Web Application and everything will work just fine. One more question you should answer before you make your choice is: just because your service doesn’t need the Site Context today, are you sure it won’t be needing it tomorrow?

Summary

WCF services are really powerful and can improve the architecture of your solution. Deploying WCF services to a SharePoint Web Application requires you to configure some components which make calling WCF services possible. While WCF service give you great power of providing data stored in SharePoint to external clients, you have to consider a few things before you determine the architecture of your solution.

SharePoint Mythbusters: Debunking common SharePoint Farm Misconceptions – Spencer Harbar
This interactive session will dive into common SharePoint Farm Myths and discuss common misconceptions around Global Deployments, Farm Topologies, Shared Service Providers, High Availability, Security and more. Alongside best practices for each “myth”, the SharePoint “magic numbers” will be covered and there will be plenty of scope to discuss any particular queries you may have on farm deployment

Best Practices for Virtualization with Hyper V and SharePoint – Bob Fox
In this presentation I am going to give an overview of Microsoft Windows Server 2008 Hyper Virtualization Technology with a strong emphasis on how to leverage using SharePoint in a Virtual Environment. Some of the topics covered: – Hyper V What is it – Hyper V what are the benefits – Hyper V What you need to know – Hyper v How to implement (Demo) – Hyper v How do I deploy my SharePoint environments (Demo) – Hyper V Recommendations and take away info

Kerberos part 1: No ticket touting here. Does SharePoint add another head? – Spencer Harbar and Bob Fox
This session will provide an overview of Kerberos and the benefits it offers in a SharePoint 2007 deployment including: Discussion of whether your SharePoint deployment needs Kerberos. A tour of the required pre-requisites and configuration settings. Best Practices for automating configuration in a SharePoint Farm. Best Practices for diagnosing common configuration errors

Kerberos part 2: advanced scenarios and additional considerations – Spencer Harbar and Bob Fox
This second Kerberos session will provide an example, end to end scenario with MOSS, Excel Services and 3rd Party applications, and dive into additional troubleshooting and auditing best practices, known issues and workarounds, including Q&A/Discussion. This session is part two of a two part session. Part two assumes attendees are familiar with the concepts presented in part one.

Why SharePoint Application Pages? – Robin Meuré
When you are developing custom solutions on top of SharePoint you often need a page (or maybe even more) in Central Admin where you can configure your solution. And of course you want to this page to look like a SharePoint page and probably you also want to use the SharePoint controls because, if you are like me, you are a lazy developer and don’t want to reinvent the wheel over and over gain. Aside to settings the configuration you are also interested in how to store your configuration.

Additionally there will be a SharePint after the event so you will have the opportunity to meet a lot of great people involved with the Dutch SharePoint community and either talk geek or just hang around with some drinks. Before you attend: don’t forget to register at SDN.nl. See you there!

After spending a few days on it, trying almost every possible solution I thought I’d found a solution to this issue. Want to know what it is? Read Chris’ article where he discusses the whole thing: More on optimization, HTTP 304s etc. – a solution?