Chronicles of a Chronic E-Junkie

Book Review - Microsoft SharePoint 2010 Power User Cookbook

2011-12-29T12:40:00.004Z

The Microsoft SharePoint 2010 Power User Cookbook is the kind of book to give to the people in the business that use SharePoint to get their work done.

This book does not start with a background history of SharePoint, doesn’t fill out pages with sales figures, and doesn’t confuse the target audience with discussions on technical topics such as the hierarchical way SharePoint works.

Instead, this book goes straight into getting things done. The first chapter delves straight into using SharePoint to learn the very fundamentals in a couple of lessons, then continuing on to provide a total of 70 easy to follow ‘recipes’ which walk you through solving every day business problems.

Each recipe is well structured in the same format, starting with what environment or software the recipe has been tested on, followed by a step by step guide on performing that task, complete with a well balanced number of relevant screenshots. There is then a brief discussion explaining what is happening, or what SharePoint is doing as that recipe is carried out, written in the context of SharePoint terminology to familiarise yourself with the workings such that you can apply that learning to more scenarios. Finally there’s an expansion on what else SharePoint can do related to that recipe, and a selection of recipes that are related in some way to the current recipe.

This formula is repeated for every one of the 70 recipes, allowing the reader to simply skip to the recipe that they want to try out at any time.

The writing style throughout is simple to read, with a distinct absence of lengthy paragraphs. This allows the content to be quickly read and absorbed by the user, without being drawn in to technical and thoughtful discussions about a particular topic.

The book is complemented at the end with a chapter entitled ‘Joining the Dots’. This is an excellent addition, which explores the idea of building applications with SharePoint, and introduces three ‘composite’ application scenarios. These scenarios require a combination of select recipes from the book to build up a working application.

The chapter opens with a discussion on how to build a composite application in SharePoint, and the design consideration points that should be made every time. The three scenarios are then introduced, and again follow a set format, each describing a problem, an intended solution, and then step-by-step details on how to accomplish the solution. After each there is some detail on what skills and technologies have been looked at in building that solution, as well as a list of all the recipes that were used to create that application.

This book is an ideal introduction to the Microsoft SharePoint platform for the end user, written in the context of business problems in a way that gets things done. With a lack of technical detail usually intended for the system administrator or application programmer, the content is targeted directly to those who use SharePoint.

5 Common Mistakes Made By New SharePoint Developers

2011-12-20T21:22:00.004Z

After some time now supporting many others in the use and development of Microsoft SharePoint as a platform, I've come to notice some common traits of people who are new to developing solutions. I've decided to list the 5 most here, and I hope that these cover both those who are currently developers getting started with SharePoint, and those who've used SharePoint a little bit and have been graced with the task of developing a solution in the platform, wearing the 'developer' hat for the very first time.

Not Understanding The Capabilities of the Platform

Let's face it. SharePoint is big. Massive. Enormous. Possibly bloated, even. It can do one hell of a lot of things, and one of the things that trips up a developer when asked to do something in SharePoint, is that they might no realise that SharePoint already does part of what they need. So they end up reinventing the wheel. Sometimes reinventing the wheel means you end up with a bigger wheel, but also you will have to look after and support that wheel for when it breaks, and you may also have wasted a load of time making that bigger wheel. Sometimes though, the wheel that comes with SharePoint might be square shaped, and thus not do what is required, forcing one to reinvent it. Always find out if you can if SharePoint can already do something that you need it to, before embarking on development effort to build something.

Not Making Full Use Of The Tools

The web UI can help you do a fair bit, and solve quite a fair amount of common, simple business requirements. Then there's SharePoint Designer, which gives you a little more power. Finally, there's the all-powerful Visual Studio. That's a wide range of capability, though people use what they're comfortable with. New developers to SharePoint will probably always hit Visual Studio first, blowing in with an over-the-top solution to a dead simple requirement such as a simple list with a couple of views, and the substantial learning curve that comes with this simple requirement that could be satisfied simply in one of the other tools. Or experienced SharePoint folks might go no further than SharePoint Designer and end up with extremely complex, over the top solutions to satisfy something that could solved really easily with a coded solution in Visual Studio, should they ever have decided to brave it. Give all the tools consideration when deciding how to implement a solution in your environment.

Not Understanding Best Practices

There's more than a dozen ways to skin a cat and there's even more ways to accomplish tasks and satisfy requirements in SharePoint. Not all are the best idea. There's performance, security, scalability, portability and other factors to take into account with solutions. There are countless resources out there for best practices of almost every aspect of SharePoint usage and development out there.

Not Doing Any Researching

As aforementioned, there are countless resources out there for best practice in SharePoint Development. There are also countless resources about the capabilities of the platform. And the capabilities of the tools. But the only way you will learn about this is by doing research. Start with the MSDN Developer Centre. Then search the interwebs for the area of SharePoint your working with, coupled with keywords such as "development, programmatically, sharepoint designer".

Not Asking (The Right) Questions

We have all been there. If you're a seasoned ASP.NET dev new to SharePoint, you've been there. When you were first put in front of an MVC project and told to learn it. Yesterday. There are plenty of places to ask questions. There's SharePoint StackExchange, MSDN Forums, LinkedIn, and you can also ask the legion of Twitter users out there, by adding the hashtag #SPHelp to your question. You'll get the best answers by asking the questions of the ilk "Where can I get info on..." or "What's the best approach to...", rather than "Give me code to do...".

These five, and my little descriptions of each, are by no means in any way fully descriptive or exhaustive. There are many many more aspects that new developers need a hand with, and I believe it's up to us experienced developers in the community to ensure that new developers aren't left with a bad taste when it comes to developing in what is (when you get used to it), a pleasurable platform to develop on.

FBA Site - Unexpected Errors after App Pool Recycle

2011-12-15T20:22:00.002Z

Scenario - you have an FBA authenticated web application, and for some reason every time there's an application pool recycle, users are presented with error pages and cannot access any page in the site. The only way is to manually navigate to /_layouts/signout.aspx and to log back in to the site.

Solution - In my case, the incorrect defaultProvider was specified in web.config under the roleManager section. The default was set to the OOTB Claims auth role provider, it should have been set to the name of the AspNetSqlRoleProvider role that we had. Double check all your web.config modifications!

Here's ref with the web.config settings as they should be: http://msdn.microsoft.com/en-us/library/ff647401.aspx

SharePoint 2010 FBA and Logging out when closing the browser

2011-12-07T21:37:00.001Z

Upon revisiting this issue, I've discovered that this method is completely redundant thanks to the April 2011 CU. Read here for more details.

Had a requirement today to have our FBA based site log the user out automatically upon closing the browser.

Our site has 'username and password' controls on a visible portion of the web site, in a content area, not on a 'custom form' per se. Therefore we can't make use of the asp:Login control which you normally get.

A separate requirement for needing to expire sessions after a set time helped us out with this, really simply too.

Have a read at this first. SharePoint 2010: Set session timeout on a FBA enabled site

So, from that article, you have a custom SPHttpApplication object loaded via your Global.asax file, handling the method "SessionSecurityTokenReceived".

In order to log the user out when the browser is closed, we basically need to set the Session's token to be non-persistent. This is simple to do as we can get or set the SessionToken object via the event's arguments.

If you want a trimmed down version which does only the handling of expiring sessions on browser closing, then use this version of the event handler:

void SessionAuthenticationModule_SessionSecurityTokenReceived(object sender, SessionSecurityTokenReceivedEventArgs e)
{
    e.ReissueCookie = true;
    e.SessionToken.IsPersistent = false;
}

Wire this up to your custom Global.asax file as in the link above, and you should have FBA users logged out when they close their browser.

No ugly Javascript hacks needed for this! Try it, let me know how well it works! If this trimmed down version doesn't work, follow the blog linked above in it's entirety, and added the IsPersistent = false line just after the ReissueCookie one.

Quickly Load Sample data into SharePoint from Wikipedia

2011-11-26T19:52:00.001Z

Do remember that this application is designed solely to create sample content, not to replicate the service that Wikipedia provides. The content that appears is unformatted and without images. The content not intended for public (even for internal intranet) use, so please adhere to the Wikipedia Terms of Service, and do also consider making a donation to the Wikimedia Foundation to support the freely available content that we have all taken for granted in these modern times.

Need to load up a couple of hundred pages into your SharePoint publishing site for testing search? Bored of Lorem Ipsum, or needing to test out different structures of content?

Run this code in a console application which gets a random page from Wikipedia then saves to SharePoint. There is a slight pause (approximately 1 second) as it gets each document and saves it, so this shouldn't constitute as heavy traffic to Wikipedia. As aforementioned, the content is added unformatted, infact it adds the entire HTML document to the Page Content field (including all the 'edit' links, not best practice in the slightest, but all I was after was searchable content in my site).

Do note, however, that Wikipedia do offer archives of their content here: http://dumps.wikimedia.org/backup-index.html and if you're intending on populating hundreds of thousands to millions of documents to test out large scale search solutions, then use these archives combined with the bulk load tool (available from here: http://code.msdn.microsoft.com/windowsdesktop/Load-Bulk-Content-to-3f379974). This methods requires downloading the entire archive (7GB worth) in one go then convert & upload to SharePoint directly. Use the method in this blog post to create a couple of hundred sample documents that you need to test, without downloading the entire wikipedia archive.

In SharePoint Enterprise Search, there is the option to add an external web site as a content source and to add that to your search index. I opted with the method in this post over that, because a) I would feel that I'd be adding significantly more traffic to websites that I don't run if I indexed possibly hundreds of thousands of pages on other sites than running this code, and b) my environment doesn't have the disk space for that kind of index :] ).

Finally, thanks to Todd Klindt and Shaun O'Callaghan for a couple of pointers and considerations for this article.

Inspiration for this method came from an answer on this StackExchange question.

Create a SharePoint Publishing Portal, and change the URL in the code to point to your new site, and simply run. You could add logic to the GetPageLayout method to retrieve a random page layout each time for testing out different information architectures, such as for testing out the Refinements web part.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using System.IO;
using Microsoft.SharePoint;
using Microsoft.SharePoint.Publishing;

namespace SampleContentGetter
{
    class MyWebClient : WebClient
    {
        Uri _responseUri;

        public Uri ResponseUri
        {
            get { return _responseUri; }
        }

        protected override WebResponse GetWebResponse(WebRequest request)
        {
            WebResponse response = base.GetWebResponse(request);
            _responseUri = response.ResponseUri;
            return response;
        }
    }
 
    class Program
    {
        static void Main(string[] args)
        {
            string site = "http://demolab-sps2010/sites/content";
            int docsToGet = 50;

            Console.WriteLine("Opening SharePoint Site...");
            SPSite oSite = new SPSite(site);
            SPWeb oWeb = oSite.OpenWeb();
            PublishingWeb pWeb = PublishingWeb.GetPublishingWeb(oWeb);
            PageLayout[] pLayouts = pWeb.GetAvailablePageLayouts();
            PageLayout layout = GetPageLayout(pLayouts, "PageFromDocLayout.aspx");

            for (int i = 0; i < docsToGet; i++)
            {
                try
                {
                    Console.WriteLine(string.Format("Getting next page ({0} of {1})...", i, docsToGet));
                    string title;
                    string page1;
                    using (MyWebClient wc = new MyWebClient())
                    {
                        wc.Headers.Add("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)");
                        page1 = wc.DownloadString("http://en.wikipedia.org/wiki/Special:Random");
                        string url = wc.ResponseUri.AbsolutePath;
                        title = Uri.UnescapeDataString(url.Substring(6));
                        Console.WriteLine("Got document: " + title);
                    }

                    Console.WriteLine("Adding to SharePoint Site...");
                    PublishingPage newPage = pWeb.AddPublishingPage(title + ".aspx", layout);
                    newPage.ListItem["PublishingPageContent"] = page1;
                    newPage.Update();
                    newPage.CheckIn("Adding content from Wikipedia");
                    newPage.ListItem.File.Publish("Adding content from Wikipedia");

                    newPage.ListItem.File.Approve("Adding content from Wikipedia");

                }
                catch (Exception e)
                {
                    Console.WriteLine(e.Message + "\nSomething odd happened, trying again....");
                }

                i++;
            }
        }

        private static PageLayout GetPageLayout(PageLayout[] layouts, string name)
        {
            foreach (PageLayout p in layouts)
            {
                if (p.Name == name)
                {
                    return p;
                }
            }
            return null;
        }
    }
}

Video - Enterprise Search Connectors for Windows Explorer

2011-10-15T15:53:00.001+01:00

Here's a quick video about adding a search connector into Windows Explorer - really cool feature I've only just discovered. Probably because I spent too much time buried in Visual Studio writing CAML and C#.

Anyway, here it is - enjoy! :)

Update: I forgot to point out that the Search Connector in Windows is given the same display name as the title of the site - that's why I pointed out that the site is called 'Awesome Search' because that's what shows up in Windows :)

Further reading: Search the enterprise from Windows 7 (SharePoint Server 2010)

A Closer Look at a SharePoint SQL Query using the Developer Dashboard

2011-10-13T23:40:00.002+01:00

Out of sheer curiosity and boredom, I've decided to have a poke at the huge query that is provided in the developer dashboard and shows up for nearly every page you load in SharePoint that has something to do with List Items.

The query in particular is the one that starts 'DECLARE @DocParentRefForRF', and when you open it up it appears horrendously huge and complex. The reason it's larger than the others you'll see is because the majority of SharePoint functions performed on the database are done using Stored Procedures (which then call custom functions) in SQL Server, which are pre-compiled for performance reasons.

This reason this query is not in a compiled, optimised stored procedure is (I can only assume) because SharePoint Lists have fields. And these fields can be added by users. These fields are not added as new fields in the database (in the database term of a field or column). Rather these fields are added as references in a table in the database that already have prepared columns set up for a number of data types for every list item. Infact, all list items in all lists are stored in a single database table in a content database. This means that this table is incredibly wide (by normal standards, but not so by SQL Server support standards. This RDBMS can support very wide tables).

Let's first have a look at the full text, formatted by myself in a structure I hope is somewhat readable. This is pulled out when I opened a TeamSite homepage. All that's on the page is a web part list view of the Shared Documents library. The query is getting the items from that library.

DECLARE @DocParentIdForRF uniqueidentifier 

SELECT TOP 1 @DocParentIdForRF = Docs.Id 

FROM Docs WHERE Docs.SiteId = @SITEID 
 AND Docs.DirName = @FDN 
 AND Docs.LeafName = @FLN;  

SELECT TOP(@NUMROWS) 
 t1. AS c0, 
 CASE 
  WHEN DATALENGTH(t1.DirName) = 0 THEN t1.LeafName 
  WHEN DATALENGTH(t1.LeafName) = 0 THEN t1.DirName 
  ELSE t1.DirName + N'/' + t1.LeafName 
 END  AS c11, 
 UserData.[tp_ContentTypeId], 
 UserData.[nvarchar4], 
 UserData.[tp_CheckoutUserId], 
 t2.[nvarchar6] AS c31c6, 
 t3.[nvarchar1] AS c32c3, 
 UserData.[tp_ModerationStatus], 
 UserData.[tp_Level], 
 t1.[ScopeId] AS c17, 
 t1.[Type] AS c8, 
 t1.[LeafName] AS c1, 
 t2.[nvarchar1] AS c31c3, 
 t2.[nvarchar4] AS c31c5, 
 t1.[IsCheckoutToLocal] AS c12, 
 UserData.[tp_Editor], 
 t3.[tp_Created] AS c32c7, 
 t1.[SortBehavior] AS c14, 
 t2.[tp_Created] AS c31c7, 
 t3.[tp_ID] AS c32c4, 
 UserData.[tp_Modified], 
 UserData.[nvarchar3], 
 t2.[tp_ID] AS c31c4, 
 t3.[nvarchar6] AS c32c6, 
 UserData.[tp_ID], 
 t1.[LTCheckoutUserId] AS c21, 
 t3.[nvarchar4] AS c32c5 
FROM UserData 
INNER MERGE 
 JOIN Docs AS t1 WITH(NOLOCK) ON (UserData.[tp_RowOrdinal] = 0) 
  AND (t1.SiteId=UserData.tp_SiteId) 
  AND (t1.SiteId = @SITEID) 
  AND (t1.ParentId = UserData.tp_ParentId) 
  AND (t1.Id = UserData.tp_DocId) 
  AND ( 
   (UserData.tp_Level = 1 OR  UserData.tp_Level =255) 
  ) 
  AND (t1.Level = UserData.tp_Level) 
  AND (
   (UserData.tp_Level = 255 AND t1.LTCheckoutUserId =@IU OR 
   (UserData.tp_Level = 1 AND 
    (UserData.tp_DraftOwnerId IS NULL) 
    OR UserData.tp_Level = 2)
    AND (t1.LTCheckoutUserId IS NULL 
    OR t1.LTCheckoutUserId <> @IU )
   )
  ) 
LEFT OUTER 
 LOOP JOIN AllUserData AS t2 WITH(NOLOCK,INDEX=AllUserData_PK) ON (UserData.[tp_CheckoutUserId]=t2.[tp_ID]) 
   AND (UserData.[tp_RowOrdinal] = 0) 
   AND (t2.[tp_RowOrdinal] = 0) 
   AND ( (t2.tp_Level = 1) ) 
   AND (t2.[tp_IsCurrentVersion] = CONVERT(bit,1) ) 
   AND (t2.[tp_CalculatedVersion] = 0 ) 
   AND (t2.[tp_DeleteTransactionId] = 0x ) 
   AND (t2.tp_ListId = @L3) 
   AND (UserData.tp_ListId = @L4) 
LEFT OUTER 
 LOOP JOIN AllUserData AS t3 WITH(NOLOCK,INDEX=AllUserData_PK) ON (UserData.[tp_Editor]=t3.[tp_ID]) 
   AND (UserData.[tp_RowOrdinal] = 0) 
   AND (t3.[tp_RowOrdinal] = 0) 
   AND ( (t3.tp_Level = 1) ) 
   AND (t3.[tp_IsCurrentVersion] = CONVERT(bit,1) ) 
   AND (t3.[tp_CalculatedVersion] = 0 ) 
   AND (t3.[tp_DeleteTransactionId] = 0x ) 
   AND (t3.tp_ListId = @L3) 
   AND (UserData.tp_ListId = @L4) 
WHERE (
 (UserData.tp_Level = 1 OR  UserData.tp_Level =255)  
  AND ( 
   UserData.tp_Level= 255 
   AND UserData.tp_CheckoutUserId = @IU 
   OR  ( 
    UserData.tp_Level  = 2 
    AND UserData.tp_DraftOwnerId IS NOT NULL OR UserData.tp_Level  = 1 
    AND UserData.tp_DraftOwnerId IS  NULL 
   ) 
   AND 
   ( 
    UserData.tp_CheckoutUserId IS  NULL  
    OR UserData.tp_CheckoutUserId <> @IU
   )
  )
 ) 
 AND (
  UserData.tp_SiteId=@SITEID AND 
  (
   UserData.tp_ParentId=@DocParentIdForRF
  )
 ) 
 AND (
  UserData.tp_RowOrdinal=0
 ) 
 AND (
  t1.SiteId=@SITEID 
  AND (t1.ParentId=@DocParentIdForRF)
 ) 
ORDER BY 
 t1.[SortBehavior]  DESC ,
 t1.[LeafName]  ASC ,
 UserData.[tp_ID]  ASC  
OPTION (FORCE ORDER, MAXDOP 1)


   CommandType: Text CommandTimeout: 0
   Parameter: '@LFFP' Type: UniqueIdentifier Size: 0 Direction: Input Value: '00000000-0000-0000-0000-000000000000'
   Parameter: '@SITEID' Type: UniqueIdentifier Size: 0 Direction: Input Value: '5fd1bd21-1398-49ad-9b23-2dc31243a1ba'
   Parameter: '@IU' Type: Int Size: 0 Direction: Input Value: '1'
   Parameter: '@L3' Type: UniqueIdentifier Size: 0 Direction: Input Value: '92d21d45-2844-47df-8faf-744f7eca7434'
   Parameter: '@L4' Type: UniqueIdentifier Size: 0 Direction: Input Value: 'fb66871c-599e-464a-b67f-597f884e06ef'
   Parameter: '@FDN' Type: NVarChar Size: 4000 Direction: Input Value: ''
   Parameter: '@FLN' Type: NVarChar Size: 4000 Direction: Input Value: 'Shared Documents'
   Parameter: '@NUMROWS' Type: BigInt Size: 0 Direction: Input Value: '16'
   Parameter: '@RequestGuid' Type: UniqueIdentifier Size: 0 Direction: Input Value: 'fcaec8c6-a13e-407b-9983-87e0f23fd133'

That's a lotta code!

I'm not going to go through every single line, but to try to explain various groups, and infer what I think is going on with this query, and why it needs to be so complex.

The first line, which is what you see as the clickable link in the developer dashboard creates a SQL variable called DocParentRefForRF with a data type of uniqueidentifier, which is obviously a GUID.

There is then a query which sets this value to the GUID which identifies the particular list item we have open (in this case, default.aspx within the SitePages library). (As a bonus hint, compare this little query to when we look at this when we have a list full of unique item permissions! Also, see my other blog post on that topic).

The next query is pretty huge, and has a row limiter, shown by the TOP(@NUMROWS)part. This at first glance might make you think that this is the List View Threshold in action. It's not. If you look at the bottom of the query, you see the parameters which have been passed into this query, in this case is 16. We'll figure out what this means later.

Let's have a try at figuring out what data is attempted to be brought back in this query.

A timestamp for the file (c0), the site-relative path to the current file in question (c11), the content type ID, some custom field (nvarchar4), the user that the file is checked out to, a couple more user added fields (c31c6, c32c3), the approval state of the item (tp_ModerationStatus), the publishing status of the item (Level), a reference to the permission levels for this file (ScopeId as c17), a Type (this is a flag which just says if the item is a file or a folder, c8), the filename (c1), a couple more user added fields (c31c3, c31c5), a flag indicating whether the file is checked out to the local drafts folder (c12), the person who edited the file last, the Created field (c23c7), SortBehaviour(?) (c14), another Created field (c31c7), an Item ID (c32c4), the Modified date, yet another custom added field (nvarchar3), and a couple of other Item IDs and and custom fields.

You'll see that the table selectors for these fields within this Select statement come from 4 different tables - UserData, t1(Docs), t2 (AllUserData) and t3 (AllUserData). You'll notice there are two references to AllUserData, and one to UserData. Basically, UserData is a View provided for backwards compatibility, and just points to AllUserData. I haven't a clue if this view is cached for performance reasons (it probably isn't), and I can't confirm because my current CloudShare environmetn doesn't have SQL Server Management Studio installed.

What's important to note is that (All)UserData contains all the custom fields which relate to a list item. These are the fields that store anything which you can select in a view in the web interface (generally). (All)Docs stores all the metadata associated with a particular item. Stuff like Checked out status, version details, the actual URL to the item.

So now we know what kind of data is coming back from what tables. Let's try next to decipher the joins. Note that the joins are all merged into a single inner join.

The first join binds the Docs view (AllDocs table) to the UserData view, simply matching up the current item selected from both tables to produce a full picture of the item in question.

The next two outer loop joins act as secondary UserData (custom fields) bindings. They are almost identical except for the join clause which binds the person who last edited the file and who the file is currently checked out to (remember this is an outer join so it includes these rows in the result set if the result is null, which would be the case if the file is not checked out, making the CheckOutUserId field blank). The purpose of these joins appears to ensure that the most recent version of the item is retrieve.

The next part of the query is the Where section. This basically ensures that if you are the editor of the file, then you can see the draft version, otherwise you will only see the most recent published version. Here we can also see a couple of references to the DocParentRefForRF variable declared earlier. This makes sure you have permission to view the current publishing state of the item.

Finally we've got a couple of sorting operations to help things once the result set comes back to the COM object that made the call.

And that's it. The Item Limit, as it turns out, as actually what is defined for the default view for the web part. Go into the Edit Web Part pane, edit the current view and you'll see that the time limit is set to '15'. I guess it throws another one in for luck, or to account for null return values or something.

Hope you found this geek fest into what a SharePoint query looks like. Might do another one in future. Maybe for a query against a list that has many fields added, just to see what it looks like.

For reference, have a look at these early schema details of the AllUserData and AllDocs tables:

AllUserData - http://msdn.microsoft.com/en-us/library/dd358229(v=prot.13).aspx
AllDocs - http://msdn.microsoft.com/en-us/library/dd304112(v=prot.13).aspx

Here's also a glossary of terms used within the tables, for further reference:
http://msdn.microsoft.com/en-us/library/c6244ace-4199-450d-a215-d37fa591159d(v=PROT.13)

(Btw, I do like that prot.13 version number added to these files. Obviously early documentation and prototype documentation for SharePoint 2010 (at the time, version 13 before it was changed to 14 at Beta).

Get MySite Host with jQuery

2011-10-11T09:28:00.000+01:00

Here's a little snippet of code to grab the MySite host of the current environment using jQuery to pick it out from the user's Welcome Menu.

var str = $('#ID_MySiteMenu').attr('onMenuClick');
var myHost = str.substring(str.indexOf('http://'),str.length-2);
alert(myHost);

Nice and simple!

List Item Performance - Wide Lists and Lookups

2011-10-08T16:59:00.002+01:00

Disclaimer:
The examples shown in this post are extremely unrealistic in terms of real-life scenarios. The numbers and types of fields used in these experiments are designed purely to test the physical limitations of SharePoint. At no point is it suggested that a real-life use of such list architecture be used. If you do have lists this wide and are wondering why it is slow, remove the nutmeg from your coffee and learn how to design actually usable schemas. :)

As part of my ongoing adventures into how to make SharePoint fall over I decided to take a look at how wide Lists can be (in terms of number of fields) before performance really takes a nosedive. I set up a little test console application that uses a helper class that takes two numbers, one for the number of rows to create and a second number for the number of fields to create. The application then deletes the test List if it already exists, creates a new one, creates the specified number of fields then continues to create the specified number of items, populating all the fields for each one.

At first, I did this with simple Text fields, and the tests faired pretty well. After keeping folks updated with my proceedings, Neil Hodgkinson reminded me that it's lookup fields that really caned performance. So I modified the app ever so slightly to create lookup list columns instead. The lookup columns are set up to look at the Title field of the same list, and when populating the new items, I set the lookup value to the previous item in the list to the one just added. Confused? I'm just creating lots of fields and lots of items and timing how long it takes to do so.

I scripted the application so I could run it with increasing numbers of fields, then I was able to chart the performance hit as the number of fields grew. See the results below, and also the source code to the application I used. The code isn't exactly ideal at the moment to be a true, flexible performance stress testing application, but it sufficed to create the tests and time how long it took to populate them.

Whilst performing these test, I came across a similar blog post from a few years back, when SharePoint 2007 was still in beta. This post kinda brings these findings up to date, as well as testing with Lookup fields (the original post only tested Text Fields, and lots of them!). Read this post here: http://www.sharepoint-tips.com/2006/06/sharepoint-2007-column-limits.html

The Results

For each test I ran, I chose a fixed number of fields that would help in contribute to the stress test such that the number of fields created in each turn could show a perceived reduction in performance. The tables below show the number of items created and the fields created in each iteration. The timings shown firstly show the time for the entire operation in minutes, secondly the average time to create each list item in minutes, and finally a 'delta' which shows the difference in time (seconds) since the previous test iteration.

When graphing the results, I am taking the number of fields as the X axis along the bottom, and the time taken on average to create each item as the Y axis. This allows to show a progression in performance reduction as the number of fields increases, and hopefully points to a 'kill point' at which performance reduces exponentially.

Text Only Fields

I decided to create 1000 list items on each iteration, with each adding 10 more fields. The following table shows the results:

Operation	Item Count	Fields	Time (Minutes)	Time Per Item (Seconds)	Delta Overall Time
Create Items	1000	11	0.167	0.01002
Create Items	1000	21	0.22	0.0132	0.053
Create Items	1000	31	0.296	0.01776	0.076
Create Items	1000	41	0.253	0.01518	-0.043
Create Items	1000	51	0.27	0.0162	0.017
Create Items	1000	61	0.281	0.01686	0.011
Create Items	1000	71	0.335	0.0201	0.054
Create Items	1000	81	0.368	0.02208	0.033
Create Items	1000	91	0.378	0.02268	0.01
Create Items	1000	101	0.383	0.02298	0.005
Create Items	1000	111	0.419	0.02514	0.036
Create Items	1000	121	0.449	0.02694	0.03
Create Items	1000	131	0.498	0.02988	0.049
Create Items	1000	141	0.511	0.03066	0.013
Create Items	1000	151	0.524	0.03144	0.013
Create Items	1000	161	0.595	0.0357	0.071
Create Items	1000	171	0.58	0.0348	-0.015
Create Items	1000	181	0.588	0.03528	0.008
Create Items	1000	191	0.621	0.03726	0.033
Create Items	1000	201	0.7	0.042	0.079

And here is the graph:

This graph shows a steady increase the time to create each item as the number of fields increases. Nothing really out of the ordinary here, though if you have 200 text fields in a list I'd love to see your user interface...

Lookup Fields - Shorter Lists

After being pointed out to the fact that Lookups really degrade performance I decided to go all out. This first test tries out a relatively 'short' list with 100 items, but increasing in Lookup fields by 5 each time. Obvious, as the list grows, the performance of the lookup degrades as there are more items in the lookup field, so the average time per item shows as an average across the entire test run. At least that is the assumption. See the second test for a challenge to that.

Here's the table for the first test:

Operation	Item Count	Fields	Time (Minutes)	Time Per Item (Seconds)	Delta Overall Time
Create Items	100	5	0.031	0.0186
Create Items	100	10	0.034	0.0204	0.003
Create Items	100	15	0.035	0.021	0.001
Create Items	100	20	0.04	0.024	0.005
Create Items	100	25	0.636	0.3816	0.596
Create Items	100	30	1.7	1.02	1.064
Create Items	100	35	2.209	1.3254	0.509
Create Items	100	40	3.17	1.902	0.961
Create Items	100	45	3.737	2.2422	0.567
Create Items	100	50	4.835	2.901	1.098
Create Items	100	55	6.268	3.7608	1.433
Create Items	100	60	8.459	5.0754	2.191
Create Items	100	65	10.818	6.4908	2.359
Create Items	100	70	12.525	7.515	1.707

And here's the graph:

The first few test runs, up to 20 lookup fields didn't show much of a reduction at all, but performance soon started to take a hit from 25 fields, and getting exponentially worse from 50 fields and up, where it's taking nearly 3 seconds to create every single item!

Lookup Fields - Longer Lists

For the second test with Lookup Fields, I decided to see what would happen with 'longer' lists, that had 500 items. This is under the assumption that a lookup field's performance degrades as the list upon which the lookup operates gets longer.

Here's the results:

Operation	Item Count	Fields	Time (Minutes)	Time Per Item (Seconds)	Delta Overall Time
Create Items	500	1	0.397	0.04764
Create Items	500	2	0.469	0.05628	0.072
Create Items	500	3	0.536	0.06432	0.067
Create Items	500	4	0.624	0.07488	0.088
Create Items	500	5	0.745	0.0894	0.121
Create Items	500	6	0.823	0.09876	0.078
Create Items	500	7	0.919	0.11028	0.096
Create Items	500	8	1.013	0.12156	0.094
Create Items	500	9	1.138	0.13656	0.125
Create Items	500	10	1.369	0.16428	0.231
Create Items	500	11	1.455	0.1746	0.086
Create Items	500	12	1.586	0.19032	0.131
Create Items	500	13	2.442	0.29304	0.856
Create Items	500	14	1.981	0.23772	-0.461
Create Items	500	15	2.222	0.26664	0.241
Create Items	500	16	2.405	0.2886	0.183
Create Items	500	17	3.375	0.405	0.97
Create Items	500	18	3.127	0.37524	-0.248
Create Items	500	19	3.43	0.4116	0.303
Create Items	500	20	4.067	0.48804	0.637

And the graph:

This graph shows a steadily decreasing reduction in performance (disregarding the two anomalies). The thing to note here (apart form the trend towards an eventual exponential trend in performance reduction), is that 15 fields with 500 items is timed at 0.26664 seconds per item. The same width-list with 100 items came in at 0.021.

This mean a list with 15 lookup fields that looks up to 500 items is 10x slower than looking up to a list with only 100 items.

Let's compare to 20 lookup fields. Time of 0.48804 seconds per item with 500 items compared to 0.024 seconds per item when populating 100 items. This makes lookups to 500 items over 20x slower than with 100 items. Proving the fact that more items, coupled with more lookup fields, leads to far, far slower performance when creating and updating list items.

Stating the absolute obvious? Most definitely, but at least now you can see some numbers proving it :)

Addendum - Source Code

Here's the source code for my helper class and console application to create the lists and the list items. By the time I was scripting this application I had it optimised to create lookup fields, but at least you can see the process used.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.SharePoint;

namespace PerformanceTests
{
    public class ListUpdater
    {

        public static void CreateFields(SPList list, int fieldCount)
        {
            for (int i = 0; i < fieldCount; i++)
            {
                list.Fields.Add("Field" + i, SPFieldType.Text, false);
            }
            list.Update();
        }

        public static void CreateLookupFields(SPList list, int fieldCount)
        {
            for (int i = 0; i < fieldCount; i++)
            {
                list.Fields.AddLookup("Field" + i, list.ID, false);
                SPFieldLookup lkp = (SPFieldLookup)list.Fields["Field" + i];

                lkp.LookupField = "Title";
                lkp.Update();
            }
            list.Update();
        }
       
        public static TimeSpan CreateItems(SPList list, int fieldCount, int itemCount)
        {
            SPListItem item = list.Items.Add();
            item["Title"] = "Test0";
            item.Update();

            DateTime startTime = DateTime.Now;
            for (int a = 0; a < itemCount; a++)
            {
                item = list.Items.Add();
                item["Title"] = "Test" + a;
                item.Update();

                for (int i = 0; i < fieldCount; i++)
                {
                    item["Field" + i] = a + ";#Test" + a;
                }
                //Console.WriteLine("Updating List Item " + a);
                item.Update();
            }
            return DateTime.Now - startTime;
        }

        public static TimeSpan UpdateList(SPList list, int colCount)
        {
            DateTime startTime = DateTime.Now;
            SPListItemCollection items = list.Items;

            foreach (SPListItem item in items)
            {
                for (int j = 0; j < colCount; j++)
                {
                    item["Field" + j] = "Test item";
                }
                item.Update();
            }
            DateTime endTime = DateTime.Now;

            return endTime - startTime;
        }


    }
}

And here's the Console App:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.SharePoint;

namespace PerformanceTests
{
    class CreateAndPopulateList
    {
        static void Main(string[] args)
        {

            SPSite site = new SPSite("http://spf-dev/");
            SPWeb web = site.OpenWeb();

            SPList list = web.Lists["PerfTest"];

            list.Recycle();

            int itemCount = 0;
            int fieldCount = 0;

            Int32.TryParse(args[0], out itemCount);
            Int32.TryParse(args[1], out fieldCount);

            Console.WriteLine(string.Format("Starting Creating/Updating list {0} items with {1} fields...", itemCount, fieldCount));

            Guid listId = web.Lists.Add("PerfTest", "", SPListTemplateType.GenericList);
            list = web.Lists[listId];

           

            ListUpdater.CreateLookupFields(list, fieldCount);
            TimeSpan duration = ListUpdater.CreateItems(list, fieldCount, itemCount);

            Console.WriteLine("Done. Time Taken to create "+ fieldCount+" fields and "+itemCount+" items : " + duration.TotalMinutes);
            
        }
    }
}

Video - SharePoint Branding with CSS Customisations

2011-10-06T00:07:00.000+01:00

Here's a short presentation by myself in response to Mike Greene's post about CSS in SharePoint.

It's my first, it took all evening, so enjoy!

SharePoint Branding with CSS Customisations from James Love on Vimeo.
Talking through what to do and what not to do when customising SharePoint sites with CSS changes.

Making use of the Property Bag in the ECMAScript Client Object Model

2011-10-03T18:30:00.000+01:00

This article is a cross-post of the originally published article from NothingButSharePoint.com, available here: https://www.nothingbutsharepoint.com/sites/devwiki/articles/Pages/Making-use-of-the-Property-Bag-in-the-ECMAScript-Client-Object-Model.aspx

The “Property Bag” is a really quick and easy way to store key/value pairs that can be associated to a particular web site. You can use this to easily store settings for an application that you might want to scope to a single Website, or even an entire Site Collection (by referring to the RootWeb of the current Site Collection).

We can make use of this in the Javascript/ECMAScript Client Object Model of SharePoint, using the allProperties property of the web object.

Here is a quick example of both setting and getting a value from the property bag, using Javascript.

function getWebProperty() {
    var ctx = new SP.ClientContext.get_current();
    var web = ctx.get_site().get_rootWeb();
    this.props =  web.get_allProperties();
    this.props.set_item(“aProperty”, “aValue”);
    ctx.load(web);

    ctx.executeQueryAsync(Function.createDelegate(this, gotProperty), Function.createDelegate(this, failedGettingProperty));
}

function gotProperty() {
    alert(this.props.get_item(“aProperty”));
}

function failedGettingProperty() {
    alert("failed");
}

When we call getWebProperty, the usual stuff of setting up the Client Context is done to get a handle on the current site. I’m also getting the RootWeb of the current site collection, so this property could be retrieved in any site within the site collection. By creating a variable called ‘props’ and setting its scope to ‘this’, I can retrieve it in my success callback.

As you can see, it’s simply a case of calling ‘set_item’, passing in the name of a property and the value to set, and also calling ‘get_item’, passing the name of a property, to retrieve that value.

The main difference between this and the server object model is that because we use traditional ‘getter’ and ‘setter’ methods (rather than Property accessors) we don’t need to trap for possible ArgumentOutOfRange exceptions in our code. If the value doesn’t exist, the method call to ‘get_item’ will return null. In this sense, to me, this makes handling the Property bag far simpler, with less code needed to write to and get from the property bag.

Programmatically add Ribbon Custom Action to an Existing List

2011-09-27T21:41:00.002+01:00

This is a cross-post from the original article from SharePoint365, available at: http://sp365.co.uk/2011/09/programmatically-add-ribbon-custom-action-to-an-existing-list/

There is plenty of documentation on the web for creating Ribbon Customisations using the standard SharePoint CAML XML mark-up. This is a great dialect for creating complex customisations to the ribbon, but they only get applied on Feature Activation.

What if you have a List in the site that already exists and you need to apply some Ribbon actions? Sure, you can add Ribbon extensions to a Content Type (and this is probably recommended), however not all of our environments that we encounter use Content Types. Or, we have Lists being created from the “Custom List” list definition (or list type) programmatically in a Feature Receiver (which is executed after all the Feature CAML has been loaded & provisioned). Or we have lists created dynamically from Workflows, or other Event Receivers, or some other dynamic method at runtime.

This little hint will show you how to add Ribbon customisations programmatically at any time. There is still XML involved, to define the actual Ribbon buttons and the commands they execute, but this can easily be added to an existing instance of a List in a site.

In essence, you add this XML to the SPList.CustomUIExtension property.

Beware that this might overwrite any other Ribbon customisations applied directly to this List (if there are any added using the aforementioned Feature CAML code), and thus might remove customisations already existing. If this is the case in your environment, ensure that you don’t blindly add the code to the property without first ‘getting’ it, and carefully injecting the XML you require before ‘setting’ the property with your full XML.

Here is an example, which gets a List that already exists in my site, and creates a link to Site Settings for a selected item in that list. Bit of a useless example, maybe, but like I said earlier, there’re other great examples of more complex and uses of Ribbon customisation. For brevity, I haven’t included an image in this either, but the examples on the web will show you how to include 32px and 16px images for your Ribbon Actions.

using System;
using System.Linq;
using Microsoft.SharePoint;

namespace AddRibbonAction
{
    class Program
    {
        static void Main(string[] args)
        {
            SPSite site = new SPSite("http://demolab-sps2010/");
            SPWeb web = site.OpenWeb();
            SPList list = web.Lists.TryGetList("Department Performance");

            var action = list.UserCustomActions.Add();
            action.Location = "CommandUI.Ribbon.ListView";
            action.Sequence = 85;
            action.Title = "Test Action";

           action.CommandUIExtension =   @"<CommandUIExtension><CommandUIDefinitions>
              <CommandUIDefinition Location=""Ribbon.ListItem.Actions.Controls._children"">
              <Button Id=""TestAction.Button"" TemplateAlias=""o1"" Command=""TestCommand"" CommandType=""General"" LabelText=""Settings"" />
              </CommandUIDefinition>
              </CommandUIDefinitions>
              <CommandUIHandlers>
              <CommandUIHandler Command =""TestCommand"" CommandAction=""/_layouts/settings.aspx"" />
              </CommandUIHandlers></CommandUIExtension>";
            action.Update();

            Console.WriteLine("Done");
            Console.ReadLine();
        }
    }
}

Build a Simple Web Part for Office 365 that gets data from a List

2011-09-09T19:24:00.000+01:00

Cross-Post. Originally posted on http://sp365.co.uk/2011/07/build-a-simple-web-part-for-office-365-that-gets-data-from-a-list/

In this article we will create a simple web part in C# code that will access a task list and show a list of pending items. You will then be able to upload this web part to your SharePoint environment on Office 365!

This tutorial combines others on the internet that shows the basics of creating web parts, and the basics of interrogating lists using the SharePoint Object Model, with the final flourish of uploading our finished web part to the cloud for use.

We’ll use Visual Studio 2010 to create the web part, and we’ll assume that you have a development environment available, be it with SharePoint Foundation installed on Windows 7 on your PC, or on a Virtual Machine.
First of all, open up Visual Studio 2010, and go ahead and create an Empty SharePoint Project. For this tutorial, I will use the project name “SharePoint365.WebParts”.

Ensure that the URL to an existing web site on your development environment is selected in the next dialogue box, and that you are deploying as a Sandboxed Solution. The fact this web part will be deployed as a sandbox solution means you can upload the resulting package to your Office 365 environment.

Right click on the project file at the top of the Solution Explorer, select Add and click New Item.

In the Add New Item Dialog Box, ensure that SharePoint 2010 is selected from the left hand Template and select Web Part from the list of installed templates (making sure you don’t select Visual Web Part). Give it a name, for this tutorial I’ll use OutstandingTasks.

When you click Add, Visual Studio will set up the project with everything you need to get started, including adding .NET Assembly references that are needed, a Feature folder for your web part, and a Package folder which contains your feature. Your web part exists in what’s known as a “SharePoint Project Item” folder, and this also includes a .webpart declaration file (which makes your web part available when you add a new web part to a page), the “.cs” source code file for your web part and an Elements.xml file, which tells SharePoint where to copy the .webpart file.

Before we crack on and write some code, lets tidy up some names of things to make the visual aspect a bit more relevant in SharePoint itself.

From the Solution Explorer, click the “Feature1” folder (not the Feature1.feature file), and hit F2. The folder should be highlighted to let you rename it. For this tutorial, I’ll call this “Outstanding Tasks Web Part Feature”.

Once renamed, hit enter, and double click this file to open the Feature Designer. Set the “Title” to the same as the name of the folder (Outstanding Tasks Web Part Feature), and ensure the “Scope” is set to “Site”. Notice your Web Part Project Item is also automatically added to the Feature.

Finally, let’s make sure the web part has a relevant name when we’re adding it to a page.

Double click the OustandingTasks.webpart file in the solution explorer.

When it opens up, you’ll see it’s an XML file. This tells SharePoint what code to use for your web part, and also sets various properties – for here we only have Title and Description, so let’s set these to something more relevant.

Finally, we can now write some code!
Save and close the other windows that are open, and double click the OustandingTasks.cs file if it’s not already open.
Inside the CreateChildControls method, paste the following code:

SPList taskList = SPContext.Current.Web.Lists["Tasks"];

SPQuery query = new SPQuery();
query.Query = "<Where><Neq><FieldRef Name='Status' /><Value Type='Text'>Completed</Value></Neq></Where>";
query.ViewFields = "<FieldRef Name='Title' /><FieldRef Name='Status'/>";

SPListItemCollection tasks = taskList.GetItems(query);

string htmlTable = "<table border='1'>";
htmlTable += "<tr><td>Title</td><td>Status</td></tr>";

foreach (SPListItem aTask in tasks)
{
  htmlTable += string.Format("<tr><td>{0}</td><td>{1}</td></tr>", aTask["Title"], aTask["Status"]);
}

htmlTable += "</table>";

LiteralControl tableControl = new LiteralControl(htmlTable);
this.Controls.Add(tableControl);

Let’s go through the code and explain what’s going on.

SPList taskList = SPContext.Current.Web.Lists["Tasks"];

Here we get an object which represents our Task list. From the right hand side we’re looking at the current website in which the web part will be placed. Inside the square brackets we’re passing in the “Internal Name” of the List (which you can see if you look at the URL whilst you’re in the list):

SharePoint Training with Intelligent Decisioning Pty

2011-08-22T15:58:00.001+01:00

Thought I'd take some time here to plug my company's newly launched training delivery branch out in Brisbane, Australia (and it's been a wee while since my last post!).

Intelligent Decisioning Pty in Australia are now a Critical Path Training partner, have opened up several training workshops in October, delivering training materials from Critical Path Training, who are industry leaders in SharePoint Training. Critical Path Training is a joint venture by SharePoint MVP's Ted Pattinson and Andrew Connell, who deliver instructor-led training courses throughout the United States in Microsoft Training Centres.

Managing Director of Intelligent Decisioning Pty and SharePoint Expert instructor Dr Adrian Colquhoun, will be delivering the courses from the purpose built, fantastically located offices in Raby Baby, Brisbane.

There are 6 separate courses, for users of any aspect of using SharePoint, from 2 day End User sessions to full-on 4 day SharePoint Development and Administration courses, and everything in between!

Check out the ID Live Australia website at http://id-live.com.au/Training.aspx for more detailed information on the courses, and view the full schedule on EventBrite at: http://idlive.eventbrite.com/.

If you are an individual working with SharePoint requiring deeper level of skills, or part of a team needing more SharePoint knowledge in eastern Australia, you'll be guaranteed world-class SharePoint Training from a world-class expert.

Resetting Item Level Permissions Programmatically

2011-07-08T11:29:00.002+01:00

As part of my series of blog posts on item level permissions, I thought I'd post a quick note on "resetting" item-level permissions on a list.

This can be done using the method SPListItem.ResetRoleInheritance() on the list item in question. This will reset any item level permissions on this item to inherit from the List in which the item resides.

This method is part of the SPSecurableObject class, which is inherited by Webs, Lists and Items - so anything where you can have unique permissions will inherit this class and expose methods to handle the unique permissions for that item.

List Item Performance - Updating Large Lists

2011-06-09T23:07:00.001+01:00

As part of my series on performance, I thought I'd look at updating large (20,000 item) lists in bulk, and what are the fastest ways to do this.

I've gone through a few different ways of doing this, finding out where the slowness is coming from, and I present the results of the analysis in this blog. Hopefully someone will take away some value from this!

The Control Test

First of all, I decided to use the worst possible method as a control for the experiement: iterating over SPList.Items. Microsoft have stated quite clearly that one should avoid doing this whereever possible, and I've found out myself that isn't not even advisable to pass in an empty SPQuery to GetItems.

Here's the code I used to update the items with this method:

using System;
using System.Linq;
using Microsoft.SharePoint;

namespace ItemUpdatePerformance
{
    class Program
    {
        static void Main(string[] args)
        {
            SPSite site = new SPSite("http://demolab-sps2010/");
            SPWeb web = site.RootWeb;

            SPList list = web.Lists["PerfTest"];

            string startTime = DateTime.Now.ToLongTimeString();
            Console.WriteLine("Starting process... - " + startTime);

            foreach (SPListItem item in list.Items)
            {
                Console.WriteLine("Updating item {0}...", item.ID);
                item["Title"] = item["Title"].ToString() + "_1";
                item.Update();
            }

            string endTime = DateTime.Now.ToLongTimeString();
            Console.WriteLine("Finished process...\n Time taken: {0} - {1}", startTime, endTime);

            Console.ReadLine();
        }
    }
}

And here's a screenshot of the result from this test:

As we can see, 3 minutes 13 seconds for our control, using the "worst possible method" for both accessing and updating the list. One thing to note about this method: the delay between the output "Starting Process..." and the first "Updating Item..." was something in the order of 15 seconds. Imagine a page load delay of that long, even if you're not iterating over all the items in SPList.Items, but just setting a variable to that property! If you simply access SPList.Items, a trip is made to the content database to evaluate all the items in your query, so that when you do eventually come to iterate over the SPListItemCollection, it is somewhat more performant. Next up, the main hope is to reduce this delay, as well as trying to avoid updating each item one at a time.

Experiement 1: Batch Updating

For the next few tests, I decided to look at the MSDN article about Batch Updating list items, using the ProcessBatchCommand method of SPWeb.

I took the code samples on their site and structured it into my solution, here's how it looks:

using System;
using System.Linq;
using Microsoft.SharePoint;
using System.Text;
using Microsoft.SharePoint.Utilities;

namespace BulkUpdate
{
    class Program
    {
        static void Main(string[] args)
        {

            SPSite site = new SPSite("http://demolab-sps2010/");
            SPWeb web = site.RootWeb;

            SPList list = web.Lists["PerfTest"];

            string startTime = DateTime.Now.ToLongTimeString();
            Console.WriteLine("Starting process... - " + startTime);


            StringBuilder methodBuilder = new StringBuilder();
            string batch = string.Empty;
            DateTime currentDate = DateTime.Now;
            string formattedDate = SPUtility.CreateISO8601DateTimeFromSystemDateTime(currentDate);

            string batchFormat = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>" +
              "<ows:Batch OnError=\"Return\">{0}</ows:Batch>";

            string methodFormat = "<Method ID=\"{0}\">" +
             "<SetList>{1}</SetList>" +
             "<SetVar Name=\"Cmd\">Save</SetVar>" +
             "<SetVar Name=\"ID\">{2}</SetVar>" +
             "<SetVar Name=\"urn:schemas-microsoft-com:office:office#Title\">{3}</SetVar>" +
             "</Method>";

            SPQuery query = new SPQuery();
            query.RowLimit = 200;

            int i = 0;

            do
            {
                SPListItemCollection items = list.GetItems(query);

                methodBuilder.Remove(0, methodBuilder.ToString().Length);

                foreach (SPListItem item in items)
                {
                    methodBuilder.AppendFormat(methodFormat, item.ID, list.ID.ToString(), item.ID, item["Title"] + "_4");
                }

                batch = string.Format(batchFormat, methodBuilder.ToString());

                Console.WriteLine("Sending batch {0} for processing...", i);
                web.ProcessBatchData(batch);
                i++;
                query.ListItemCollectionPosition = items.ListItemCollectionPosition;

            } while (query.ListItemCollectionPosition != null);


            string endTime = DateTime.Now.ToLongTimeString();
            Console.WriteLine("Finished process...\n Time taken: {0} - {1}", startTime, endTime);

            Console.ReadLine();
        }
    }
}

And important line to look out for here is the query.RowLimit = 200; line. I change this for the next couple of experiments, to see how the size of a page I'm getting from the list varies with each update.

Here's the screenshot from the first run, with 200 items:

Here: 4 minutes, 39 seconds. Oh dear, even slower! Let's try a bigger batch. This time, 500 items at a time.

This time, 3 minutes 32 seconds! Much better, bit still slower than the forbidden way!

What happens with smaller batches? 50 items per batch this time:

Uhm, yeah. 10 minutes 53 seconds for that one. Scratch that idea.

So it seems bigger batches are going in the right direction. Let's try the default query throttling limit of 2000 items.

Whoa there, Betsy. 2 minutes 51 seconds! That's a record for us!

Experiment 3: Data Tables

Lastly, I want to see if I can speed up the "getting" of all the items in my list. I want to see if the GetDataTable() method of SPListItemCollection will get me the items any faster than just the Items property. At least, I want to see if I can iterate over the rows in a DataTable faster than I can the Items in the collection. I'm going to again use the Batch method with 2000 rows per time, as this got the best results yet.

Here's the code that uses GetDataTable()

using System;
using System.Linq;
using Microsoft.SharePoint;
using System.Text;
using Microsoft.SharePoint.Utilities;
using System.Data;

namespace BulkUpdate
{
    class Program
    {
        static void Main(string[] args)
        {

            SPSite site = new SPSite("http://demolab-sps2010/");
            SPWeb web = site.RootWeb;

            SPList list = web.Lists["PerfTest"];

            string startTime = DateTime.Now.ToLongTimeString();
            Console.WriteLine("Starting process... - " + startTime);


            StringBuilder methodBuilder = new StringBuilder();
            string batch = string.Empty;
            DateTime currentDate = DateTime.Now;
            string formattedDate = SPUtility.CreateISO8601DateTimeFromSystemDateTime(currentDate);

            string batchFormat = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>" +
              "<ows:Batch OnError=\"Return\">{0}</ows:Batch>";

            string methodFormat = "<Method ID=\"{0}\">" +
             "<SetList>{1}</SetList>" +
             "<SetVar Name=\"Cmd\">Save</SetVar>" +
             "<SetVar Name=\"ID\">{2}</SetVar>" +
             "<SetVar Name=\"urn:schemas-microsoft-com:office:office#Title\">{3}</SetVar>" +
             "</Method>";

            SPQuery query = new SPQuery();
            query.RowLimit = 2000;

            int i = 0;

            do
            {
                SPListItemCollection items = list.GetItems(query);
                DataTable table = list.Items.GetDataTable();

                methodBuilder.Remove(0, methodBuilder.ToString().Length);

                foreach (DataRow row in table.Rows)
                {
                    string moo = row["Title"].ToString();
                    methodBuilder.AppendFormat(methodFormat, row["ID"], list.ID.ToString(), row["ID"], row["Title"] + "_6");
                }


                batch = string.Format(batchFormat, methodBuilder.ToString());

                Console.WriteLine("Sending batch {0} for processing...", i);
                web.ProcessBatchData(batch);
                i++;
                query.ListItemCollectionPosition = items.ListItemCollectionPosition;

            } while (query.ListItemCollectionPosition != null);


            string endTime = DateTime.Now.ToLongTimeString();
            Console.WriteLine("Finished process...\n Time taken: {0} - {1}", startTime, endTime);

            Console.ReadLine();
        }
    }
}

And here's the results:

4 minutes to send the first batch... and there's 10 of them (2000 items for a 20,000 item list!). Screw that.

I had also been trying, as a final test to see if REST would be any quicker.

Unforunately REST has a couple of major limitations for large bulk operations. The first is that the "GetItems" equivalent only gets 1000 items at a time. The second is that the BulkUpdate command can only handle 100 items at a time. Not really adequate. If someone has been able to successfully do large operations with the REST API, please let me know how you get on! :)

Finally...

So, in summary, here's the table of results from these tests. If you have larger batches, Iterating over the ListItemCollections takes longer but the data is written to the server in one go, if you have smaller batches, Iterating over the ListItemCollections is indeed quicker, but there is the delay whilst many batches are written.

Method	Time
SPList.Items	3:13
Batches of 200 items	4:39
Batches of 500 items	3:32
Batches of 50 items	10:53
Batches of 2000 items	2:51
Batches of 2000, Iterating over DataTable	> 30 mins

Be sure to have a look at my other two posts on performance with Item-Level Permissions too (here and here), if you found this interesting/helpful.

Reordering Content Types Programmatically

2011-06-01T16:30:00.004+01:00

After this question was asked on SharePoint Stack Exchange, I thought I'd share my method of re-ordering content types in a list.

Note: this isn't the only method of doing this, see the thread on SPSE to see more (particularly the UniqueContentTypeOrder property).

private void ReorderContentTypes(SPWeb web, string listName, string firstContentTypeName)
{
 SPList list = web.Lists[listName];

 SPContentType cType = web.AvailableContentTypes[firstContentTypeName];

 List<SPContentType> oldCTypes = new List<SPContentType>();

 for (int i = list.ContentTypes.Count -1; i >= 0; i--)
 {
  if (!list.ContentTypes[i].Id.IsChildOf(cType.Id))
  {
   oldCTypes.Add(list.ContentTypes[i]);

   list.ContentTypes[i].Delete();
  }
 }

 foreach (SPContentType c in oldCTypes)
 {
  list.ContentTypes.Add(c);
 }

 list.Update();
}

SPList.Items versus the SPQuery

2011-04-18T21:57:00.001+01:00

So you've embarked on writing a bit of code that must enumerate a SharePoint List's Items one by one and do some stuff with them. You've (at some point) came across the Best Practices page on MSDN on Handling Large Folders and Lists, and seen that using SPList.Items is bad. It's bad because you could potentially have a very large (tens of thousands to hundreds of thousands of list items) list which could be queried and loaded into server memory potentially several times per second in a busy environment.

So you decide to use the much-loved (or loathed) SPQuery object to store a CAML query and use SPList.GetItems(), passing in your SPQuery. But you still want to iterate over all the items and use all the fields in the list, so you basically pass in the SPQuery with empty properties. All good right? Still follows the best practices document outlined above, right?

Wrong.

What Microsoft have slightly glossed over and neglected to highlight in their documentation is the necessity to use filters (the Query, RowLimit, ViewFields properties) in your SPQuery object. When you call SPList.Items, the object model creates an empty SPQuery object, populates with minimal properties (it only actually sets the Scope to Recursive, such that it gets all items in all folders) and calls GetItems upon itself. By creating yourself an empty SPQuery object, you're doing the same as SPList.Items.

The documentation only states to apply filers "if appropriate". This sentence should really emphasise the importance of using Filters, because if you don't, you're getting all items.

That page also has some good examples of paginating your results using a combination of RowLimit and ListItemCollectionPosition properties, to ensure you only query to return a small amount of data at a time. This will be particularly important when your web application has Query Throttling enabled.

Feature Stapling - The TL;DR Version

2011-04-06T19:02:00.000+01:00

In a Site-Scoped Feature, have this line in the Elements.xml:

The Id here is the GUID of a Web-Scoped Feature you want to activate every time a Sub-Web of template STS#0 (Team Site) is created.

Simples.

Programmatically Update a Content Type

2011-04-04T19:13:00.000+01:00

Here's a straight forward one. For some reason (don't ask) I had to update a content type in a Feature Receiver (I would have preferred to have created a new content type and inherit) by adding a new field to it.

Here's the code, purely for my personal reference but open in case other people find it helpful.

SPContentType myCT = web.ContentTypes[new SPContentTypeId("0x010100E271948D51B6564FB8D517B6347459D4")];

SPField myNewField = web.Fields.GetFieldByInternalName("AnotherSiteColumn");

SPFieldLink myNewFieldLink = new SPFieldLink(myNewField);

myCT.FieldLinks.Add(myNewFieldLink);

myCT.Update();

SharePoint 2010: Migrating solutions from SharePoint 2007 - A Caveat

2011-04-02T18:21:00.002+01:00

We're working on a large(ish) project right now in SharePoint 2007 that will quite soon be migrating to SharePoint 2010, so I decided to take the WSPs we've had so far (that work fine in 2007) and see how well they'd work in 2010, either as Farm solutions or User (Sandbox) solutions.

I came across an interesting play in behaviour.

The project in particular isn't uncontrollably big, but it's growing in size and there are 4 of us developers working on it. It turns out that there is a Field element (a site column) being declared twice in an Elements Manifest (elements.xml) file. Why hadn't we spotted this? Because it deploys fine, without error in SharePoint 2007.

Here's where the interesting bit comes in...

I decided to create a new project in Visual Studio 2008 destined for SharePoint 2010 consisting of a dead simple feature that deploys one site column - but declares it twice. Here's the solution's structure in Visual Studio, ready for packaging with WSPbuilder:

And here's the Feature and Elements.xml contents (excuse the loss of Case, think my syntax highlighter messes up formatting a bit too much):

And the Elements...

So what happens?

Well when you deploy this to SharePoint 2007, and activate the Feature at site collection level - it works without fault. Great!

When we take the WSP and upload it to the Solution Gallery of a SharePoint 2010 site, again it works fine (Site collection features from user solutions seem to be activated automatically on my system, not sure if this is normal).

But, add this solution as a Farm solution in Central Administration, deploy then activate from Site Collection Features and you'll get this error...

Exception type: SPException 
Exception message: The field with Id {3448d8bc-8497-4d67-8523-743e4e79b080} defined in feature {c01670b9-a98b-4fbd-9afd-ac6ef8d70c40} was found in the current site collection or in a subsite.

So Beware! If you intend your solutions developed for 2007 to work as Sandboxed solutions, give them a sanity check by testing the deployment on your development system as a Farm solution first - as it appears to have more stringent validation of Feature elements than the User Code environment.

SharePoint 2010 Performance with Item Level Permissions Part 2

2011-03-23T23:02:00.003Z

So why the hell's it so slow?

This post follows on from my last one when I looked at the developer dashboard to gain insight into where the performance issues came from when you have a large list with lots of item level permissions.

In this post I'll take a deeper look, using the SQL Server Profiler tool to see what data is being passed around, to confirm my assumptions that large lists with lots of unique item permissions slow down at the database level.

To analyse this better, I decided to use the code from my last post to create 2 identical lists. One was a custom list with 20k items in, all inheriting permissions as default. A second list would have every single item have unique permissions, as I did in the previous experiment. Doing this just lets me have both lists there so I can switch between the two to see what's going on.

The thing I noticed last time was a particular query which took an incredibly long time to execute, one that started DECLARE @DocParentIdForRF. The thing with the developer dashboard is that it hides a varbinary parameter which is passed in. This varbinary is used in the generate of a table variable in this query (look at the snippet from the end of my last post to see this).

You notice in the developer dashboard output that when we have item level permissions, a stored procedure is executed that is not when we don't have the item level permissions - one called proc_GetUniqueScopesInList.

It turns out that the word "scope" is used to describe what is visible from a certain context (well, duh!), and even in SharePoint, a Scope is the visible of an item from a security context!

So, fire up the SQL Server Profiler, start a new trace and filter it out by your application pool identity (this makes reading this A LOT easier).

When I refresh the page with the inherited permissions (NOT the item-level permissions), I notice that the DECLARE @DocParentIdForRF looks like this - with no varbinary parameter... (click for larger image). Stop the trace as soon as the page has loaded, makes the trace easier to read.

Let's clear the output, restart the trace and load the list with the item-level permissions, you'll notice the same query looks like this now:

See that huge black blob at the bottom? That's the varbinary parameter that the developer dashboard hides from you. Where'd the hell that come from? Look up. See the proc_GetUniqueScopesInList procedure? Have a look at the length of time to execute it...

... Next to nothing! It's a quick, high optimised (select a couple of columns from this table where these 3 columns match these values), using a table-valued function.

Let's go back to that huge black blob in the screenshot from before. If I get the entirety of that data (starting from the "@L10IMG=0x" somewhere in there, to the end, and dump it into Notepad++ as a quick way of seeing how long it is, it's a rather cumbersome 640084 characters long (or there abouts). As a hunch, divide it by 32 (the length of a GUID) and you'll get 20,002. Not far off the number of items in that list, is it?

An item-level permission in SharePoint is represented in the content database in a table called "Perms", where each item has the ACL stored as a mask.

When you load up a huge list with lots of item level permissions, a single operation gets every single GUID associated with the ACL for that item and passes that back to the data access layer of SharePoint. When the database retrieves the actual list item data, it will pass in all of the ACL Guids back in as one long string, all concatenated together. The query to get the data creates a table variable re-assembles the the item level ACL Guid associated with each item. How the rest of the query deals with this is anyone's guess at the moment - this table variable might just be passed back to the calling COM object (though I thought they couldn't be used this way....) for the COM object to then sort out which item should be visible to which "scope" (or ACL).

So, what can we take away form this? Passing 640k of data about the place, for a SQL Query to do some substring math and converting to Guids will soon bring your database server to its knees. This is one request and it takes 2000ms to work. Imagine if you have 5 requests per second or more hitting this list! If you think you're gonna need them, might be time to re-architect the solution.

SharePoint 2010 Performance with Item Level Permissions

2011-03-10T22:54:00.004Z

Tonight's little project was inspired by a conversation on Twitter with @WonderLaura and @Toddklindt. Laura had asked about the implications of lots of item-level permissions in a big list in SharePoint (here), to which Todd replied with this, and followed up with this and this.

So this got me thinking: where is the performance bottleneck when you have item-level permissions in place? Todd's comments leant towards the construction of the view at the Web Part level, but something was making me think that the checking of permissions would be done at the database-query level. Surely an SPRequest wouldn't poll the database for all items in a request if not all of them are viewable by the user starting the request?

Time, I thought, to do an experiment. I want to use the SharePoint Foundation 2010 Developer Dashboard to monitor the performance of a large (20,000 item) list before and after item-level permissions have been applied - to help us see if we can spot the bottleneck.

Using SharePoint Foundation 2010, I created a custom list and left it with just the one column for this experiment. I used Visual Studio 2010 (with the CKS:Dev pack) to populate the list with 20,000 items using a Console Application. (Sources for this project can be found at the end of this post).

I then fired up the developer dashboard to see what the performance looks like. I created a normal View in the GUI which returned 200 items from the list. Here's the results (click for larger version):

The figures I'm interested in looking at are the "DECLARE @DocParentIdForRF" Database Query and the PerfTest OnPreRender numbers. The latter is the time taking to execute the OnPreRender method of the List (which, in ListViewWebPart, prepares the Ribbon for use, and prepares client side ScriptLinks for DataFormWebPart). I have no idea what the first query is all about, it's not a stored procedure nor a function in the content database and it's definitely not constructed by any of the managed code of SharePoint. If anyone does know what this does, please let me know purely to satisfy my curiosity.

Next, I rolled up another console application to randomly apply one of the Role Definitions available in the site to every single item in the list, giving us a kind of "worse-case" scenario to look. Again, source code for this is at the end of this post.

Back to the list view, and here're the results from the Developer Dashboard. Notice huge jump in both the figures - the database query takes around 2 seconds as well as the OnPreRender for the web part! This was after refreshing the page a total of five times once every 10 seconds to let caching, etc. do its thing.

So, for some reason OnPreRender takes an absolute age with a massive list, even though the web part is rendering the same 200 items in the first page of the view, except now we have item level permissions for each one of those items. This method, however, doesn't deal with the items! It just registers ScriptLinks used by the DataFormWebPart.

As the database query takes less time in both before and after, this leads me to think that the query is slowing down the OnPreRender for some reason, thus leading me to suspect further that performance loss is indeed down to the querying of the database!

If anyone else has any thoughts on this, they'd be gratefully received.

And now for the source.

This first piece of code will create your 2000 items.

using System;
using System.Linq;
using Microsoft.SharePoint;

namespace ItemPerfTest
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.Out.Write("Preparing List...");
            using (SPSite site = new SPSite("http://demolab-sps2010"))
            using (SPWeb web = site.OpenWeb())
            {
                SPList list = web.Lists["PerfTest"];
                for (int i = 0; i < 20001; i++)
                {
                    Console.Out.WriteLine(string.Format("Adding item: {0}...", i));
                    SPListItem item = list.Items.Add();
                    item["Title"] = "Item" + i;
                    item.Update();
                }
            }
        }
    }
}

This second piece will assign each one a randomly selected Role Definition, after breaking the items Role Inheritance.

using System;
using System.Linq;
using Microsoft.SharePoint;
using System.Collections.Generic;

namespace UniquePermissions
{
    class Program
    {
        static void Main(string[] args)
        {
            List<SPRoleDefinition> roles = new List<SPRoleDefinition>();
            Random numbergenerator = new Random();
            using (SPSite site = new SPSite("http://demolab-sps2010"))
            using (SPWeb web = site.OpenWeb())
            {
                Console.Out.WriteLine("Preparing list...");
                SPRoleDefinition contributor = web.RoleDefinitions.GetByType(SPRoleType.Contributor);
                SPRoleDefinition guest = web.RoleDefinitions.GetByType(SPRoleType.Guest);
                roles.Add(contributor);
                roles.Add(guest);
       

                SPList list = web.Lists["PerfTest"];

                foreach(SPListItem item in list.Items)
                {
                    item.BreakRoleInheritance(true);
                    SPRoleAssignment randomRole = web.RoleAssignments[numbergenerator.Next(0, web.RoleAssignments.Count)];
                    item.RoleAssignments.Add(randomRole);
                    item.Update();
                    Console.Out.WriteLine("Applying role assment {0} to item {1}....", randomRole.Member.LoginName, item.Title);
                }
               
            }

            Console.WriteLine();
        }
    }
}

PS: Here's the detail on that query as called:

SqlCommand: 'DECLARE @DocParentIdForRF uniqueidentifier SELECT TOP 1 @DocParentIdForRF = Docs.Id FROM Docs WHERE Docs.SiteId = @SITEID AND Docs.DirName = @FDN AND Docs.LeafName = @FLN;        SET NOCOUNT ON; DECLARE @_scopeTbl TABLE(_listId uniqueidentifier NOT NULL,_id uniqueidentifier NOT NULL,_level bit NOT NULL,PRIMARY KEY CLUSTERED(_listId,_id,_level)); DECLARE @_count0_0 INT; SET @_count0_0 = 0; WHILE @_count0_0<@I9P BEGIN  INSERT INTO @_scopeTbl VALUES (@L2, CAST(SUBSTRING(@L10IMG,@_count0_0*16 + 1, 16) AS uniqueidentifier),0); SET @_count0_0=@_count0_0 + 1; END;SELECT TOP(@NUMROWS) t1.[TimeCreated] AS c0, UserData.[tp_Modified], t1.[SortBehavior] AS c2, UserData.[tp_ContentTypeId], UserData.[tp_ID], t1.[ScopeId] AS c4, t1.[Type] AS c1, UserData.[tp_ModerationStatus], UserData.[tp_Level], UserData.[nvarchar1], CASE WHEN DATALENGTH(t1.DirName) = 0 THEN t1.LeafName WHEN DATALENGTH(t1.LeafName) = 0 THEN t1.DirName ELSE t1.DirName + N'/' + t1.LeafName END  AS c3, UserData.[tp_HasAttachment], t1.[LeafName] AS c5, UserData.[nvarchar2] FROM AllUserData AS UserData WITH(INDEX=AllUserData_PK) LEFT OUTER LOOP JOIN Docs AS t1 WITH(NOLOCK) ON (UserData.[tp_CalculatedVersion] = 0 ) AND (UserData.[tp_IsCurrentVersion] = CONVERT(bit,1) ) AND (UserData.[tp_DeleteTransactionId] = 0x ) AND (UserData.[tp_RowOrdinal] = 0) AND (t1.SiteId=UserData.tp_SiteId) AND (t1.SiteId = @SITEID) AND (t1.ParentId = UserData.tp_ParentId) AND (t1.Id = UserData.tp_DocId) AND ( (UserData.tp_Level = 1) ) AND (t1.Level = UserData.tp_Level) AND (t1.IsCurrentVersion = 1) AND (t1.Level = 1 OR t1.Level =  2) WHERE ( (UserData.tp_Level = 1) ) AND (UserData.tp_SiteId=@SITEID AND (UserData.tp_ParentId=@DocParentIdForRF)) AND (UserData.tp_RowOrdinal=0) AND (t1.SiteId=@SITEID AND (t1.ParentId=@DocParentIdForRF)) AND (UserData.tp_ListID=@LISTID) AND ( (UserData.tp_Level = 1) ) AND (UserData.tp_SiteId=@SITEID AND (UserData.tp_ParentId=@DocParentIdForRF)) AND (UserData.tp_RowOrdinal=0) AND (t1.SiteId=@SITEID AND (t1.ParentId=@DocParentIdForRF)) ORDER BY UserData.[tp_ID]  ASC  OPTION (FORCE ORDER, MAXDOP 1)'
   CommandType: Text CommandTimeout: 0
   Parameter: '@LFFP' Type: UniqueIdentifier Size: 0 Direction: Input Value: '00000000-0000-0000-0000-000000000000'
   Parameter: '@SITEID' Type: UniqueIdentifier Size: 0 Direction: Input Value: '02ea4e54-c1ae-4b8c-92fc-45f0c91af393'
   Parameter: '@L2' Type: UniqueIdentifier Size: 0 Direction: Input Value: '4047d6c1-d522-4db6-91eb-ab257dabbac5'
   Parameter: '@FDN' Type: NVarChar Size: 4000 Direction: Input Value: 'Lists'
   Parameter: '@FLN' Type: NVarChar Size: 4000 Direction: Input Value: 'PerfTest'
   Parameter: '@RequestGuid' Type: UniqueIdentifier Size: 0 Direction: Input Value: '11bf00f2-9479-4ef7-a8c2-9c3b005f7b8e'
   Parameter: '@L6' Type: UniqueIdentifier Size: 0 Direction: Input Value: '17e4e245-cb54-4b57-bc17-ab4215ac66f2'
   Parameter: '@LISTID' Type: UniqueIdentifier Size: 0 Direction: Input Value: '4047d6c1-d522-4db6-91eb-ab257dabbac5'
   Parameter: '@NUMROWS' Type: BigInt Size: 0 Direction: Input Value: '11'
   Parameter: '@I9P' Type: Int Size: 0 Direction: Input Value: '20002'
   Parameter: '@L10IMG' Type: VarBinary Size: -1 Direction: Input

Text-only Description field in Google News RSS Feeds in XSLT

2011-02-21T18:13:00.006Z

One annoying thing about Google's awesome RSS feed from its news site is the Description field.

If you look at the XML generated by http://news.google.com/news?q=sharepoint&output=rss for example, you'll see the Description field is flooded with lots of crap, un-needed, encoded HTML, instead of the pure description in plain text as you'd expect (and probably what is defined in the schema for RSS).

What if you want to scrap all this crap from the Description field, leaving behind what should be in the text? Simple - take a bog-standard "strip HTML" XSLT template (can be found anywhere online after a quick Google/Bing), and pass it to a trimmed-down version of the description field.

Like this. Here's the "strip-HTML" template:

<xsl:template name="strip-tags">
  <xsl:param name="text"/>
  <xsl:choose>
    <xsl:when test="contains($text, '&lt;')">
      <xsl:value-of select="substring-before($text, '&lt;')" />
      <xsl:call-template name="strip-tags">
        <xsl:with-param name="text" select="substring-after($text, '&gt;')"/>
      </xsl:call-template>
    </xsl:when>
    <xsl:otherwise>
      <xsl:value-of select="$text" />
    </xsl:otherwise>
  </xsl:choose>
</xsl:template>

And this is the normal way to call it:

<xsl:variable name="SafeHtml">
  <xsl:call-template name="strip-tags">
    <xsl:with-param name="text" select="description"/>
  </xsl:call-template>
</xsl:variable>

So what we do is look for tags which are always present around the text we're interest in and strip them out. It might help if you open the feed in Internet Explorer so it renders the encoded HTML, and then copy & paste the contents of a "description" tag into something that can format & tabulate the text (like Visual Studio). (Here's the link for the RSS feed again: http://news.google.com/news?q=sharepoint&output=rss)

We can see that before the "real" description text there are the tags:

</font></b></font><br />

and toward the end there is

...</b></font><br />

So we just use a bit of string manipulation in the XSLT to make sure we pass in anything in between these two "tokens", like so:

<xsl:variable name="SafeHtml">
  <xsl:call-template name="strip-tags">
    <xsl:with-param name="text" select="substring-before(substring-after(description, '&lt;/font&gt;&lt;/b&gt;&lt;/font&gt;&lt;br /&gt;'), '&lt;b&gt;...&lt;/b&gt;&lt;/font&gt;&lt;br /&gt;')"/>
  </xsl:call-template>
</xsl:variable>

No Content Types or Site Columns when creating Site Collection from code

2011-02-11T15:10:00.001Z

Had a quick frustration today, that some code I had (bog standard SPWebApplication.Sites.Add) would create a site collection fine but had absolutely no content types or site columns. Both galleries were empty.

Lists and Libraries had content types (such as Pages) fine, but SFA in the gallery.

If you have this, you're probably doing what I'm trying to do: constantly tear down and rebuild stuff in an automated manner for testing.

What you need to do is to do an

iisreset

in between teardown and rebuild of the site collection.

Not sure why this is, something in the object model cache must be getting stuck, but I didn't see anything in the ULS logs as a clue.

Oh, and also: I'm keeping this blog theme. It's getting good feedback :P