InfoWorker Solutions

Exploring Search Results Step-by-Step in SharePoint 2010

2012-05-07T09:45:00.000+02:00

Using search to provide a news archive in SharePoint 2010 is a wellknown solution. Just add the core results web-part to a page and configure it to query for your news article content type and sort it in descending order. Then customize the result XSLT to tune the content and layout of the news excerpts to look like a new archive. Add also the search box, the search refiners and the results paging web-parts and you have a functional news archive in no time.

This post is about providing contextual navigation by adding "<< previous", "next >>" and "result" links to the article pages, to allow users to explore the result set in a step-by-step manner. Norwegians will reckognize this way of exploring results from finn.no.

For a user or visitor to be able to navigate the results, the result set must be cached per user. The search results are in XML format, and it contains a sequential id and the URL for each hit. This allows the navigation control to use XPath to locate the current result by id, and get the URLs for the previous and next results. The user query must also be cached so that clicking the "result" link will show the expected search results.

Override the CoreResultsWebPart as shown in my Getting Elevated Search Results in SharePoint 2010 post to add per-user caching of the search results. If your site allows for anonymous visitors, you need to decide on how to keep tab on them. In the code I've used the requestor IP address, which is not 100% foolproof, but this allows me to avoid using cookies for now.

namespace Puzzlepart.SharePoint.Presentation

    [ToolboxItemAttribute(false)]

    public class NewsArchiveCoreResultsWebPart : CoreResultsWebPart

        public static readonly string ScopeNewsArticles

            = "Scope=\"News Archive\"";

        private static readonly string CacheKeyResultsXmlDocument

            = "Puzzlepart_CoreResults_XmlDocument_User:";

        private static readonly string CacheKeyUserQueryString

            = "Puzzlepart_CoreResults_UserQuery_User:";

        private int _cacheUserQueryTimeMinutes = 720;

        private int _cacheUserResultsTimeMinutes = 30;

        protected override void CreateChildControls()

try

                base.CreateChildControls();

            catch (Exception ex)

                var error = SharePointUtilities.CreateErrorLabel(ex);

                Controls.Add(error);

        protected override XPathNavigator GetXPathNavigator(string viewPath)

            //return base.GetXPathNavigator(viewPath);

            SetCachedUserQuery();

            XmlDocument xmlDocument = GetXmlDocumentResults();

            SetCachedResults(xmlDocument);

            XPathNavigator xPathNavigator = xmlDocument.CreateNavigator();

            return xPathNavigator;

        private XmlDocument GetXmlDocumentResults()

            XmlDocument xmlDocument = null;

            QueryManager queryManager =

            SharedQueryManager.GetInstance(Page, QueryNumber).QueryManager;

            Location location = queryManager[0][0];

            string query = location.SupplementaryQueries;

            if (query.IndexOf(ScopeNewsArticles,

                StringComparison.CurrentCultureIgnoreCase) < 0)

                string userQuery =

                    queryManager.UserQuery + " " + ScopeNewsArticles;

                queryManager.UserQuery = userQuery.Trim();

            xmlDocument = queryManager.GetResults(queryManager[0]);

            return xmlDocument;

        private void SetCachedUserQuery()

            var qs = HttpUtility.ParseQueryString

                    (Page.Request.QueryString.ToString());

            if (qs["resultid"] != null)

                qs.Remove("resultid");

            HttpRuntime.Cache.Insert(UserQueryCacheKey(this.Page),

               qs.ToString(), null,

               Cache.NoAbsoluteExpiration,

               new TimeSpan(0, 0, _cacheUserQueryTimeMinutes, 0));

        private void SetCachedResults(XmlDocument xmlDocument)

            HttpRuntime.Cache.Insert(ResultsCacheKey(this.Page),

               xmlDocument, null,

               Cache.NoAbsoluteExpiration,

               new TimeSpan(0, 0, _cacheUserResultsTimeMinutes, 0));

        private static string UserQueryCacheKey(Page page)

            string visitorId = GetVisitorId(page);

            string queryCacheKey = String.Format("{0}{1}",

                CacheKeyUserQueryString, visitorId);

            return queryCacheKey;

        private static string ResultsCacheKey(Page page)

            string visitorId = GetVisitorId(page);

            string resultsCacheKey = String.Format("{0}{1}",

                CacheKeyResultsXmlDocument, visitorId);

            return resultsCacheKey;

        public static string GetCachedUserQuery(Page page)

            string userQuery =

                (string)HttpRuntime.Cache[UserQueryCacheKey(page)];

            return userQuery;

        public static XmlDocument GetCachedResults(Page page)

            XmlDocument results =

                (XmlDocument)HttpRuntime.Cache[ResultsCacheKey(page)];

            return results;

        private static string GetVisitorId(Page page)

            //TODO: use cookie for anonymous visitors

            string id = page.Request.ServerVariables["HTTP_X_FORWARDED_FOR"]

                ?? page.Request.ServerVariables["REMOTE_ADDR"];

            if(SPContext.Current.Web.CurrentUser != null)

                id = SPContext.Current.Web.CurrentUser.LoginName;

            return id;

I've used sliding expiration on the cache to allow for the user to spend some time exploring the results. The result set is cached for a short time by default, as this can be quite large. The user query text is, however, small and cached for a long time, allowing the users to at least get their results back after a period of inactivity.

As suggested by Mikael Svenson, an alternative to caching would be running the query again using the static QueryManager page object to get the result set. This would require using another result key element than the dynamic <id> number to ensure that the current result lookup is not scewed by new results being returned by the search. An example would be using a content type field such as "NewsArticlePermaId" if it exists.

Overriding the GetXPathNavigator method gets you the cached results that the navigation control needs. In addition, the navigator code needs to know which is the result set id of the current page. This is done by customizing the result XSLT and adding a "resultid" parameter to the $siteUrl variable for each hit.

. . .

 <xsl:template match="Result">

    <xsl:variable name="id" select="id"/>

    <xsl:variable name="currentId" select="concat($IdPrefix,$id)"/>

    <xsl:variable name="url" select="url"/>

    <xsl:variable name="resultid" select="concat('?resultid=', $id)" />

    <xsl:variable name="siteUrl" select="concat($url, $resultid)" />

. . .

The result set navigation control is quite simple, looking up the current result by id and getting the URLs for the previous and next results (if any) and adding the "resultid" to keep the navigation logic going forever.

namespace Puzzlepart.SharePoint.Presentation

    public class NewsArchiveResultsNavigator : Control

        public string NewsArchivePageUrl { get; set; }

        private string _resultId = null;

        private XmlDocument _results = null;

        protected override void CreateChildControls()

            base.CreateChildControls();

            _resultId = Page.Request.QueryString["resultid"];

            _results = NewsArchiveCoreResultsWebPart.GetCachedResults(this.Page);

            if(_results == null || _resultId == null)

                //render nothing

                return;

            AddResultsNavigationLinks();

        private void AddResultsNavigationLinks()

            string prevUrl = GetPreviousResultPageUrl();

            var linkPrev = new HyperLink()

                Text = "<< Previous",

                NavigateUrl = prevUrl

};

            linkPrev.Enabled = (prevUrl.Length > 0);

            Controls.Add(linkPrev);

            string resultsUrl = GetSearchResultsPageUrl();

            var linkResults = new HyperLink()

                Text = "Result",

                NavigateUrl = resultsUrl

};

            Controls.Add(linkResults);

            string nextUrl = GetNextResultPageUrl();

            var linkNext = new HyperLink()

                Text = "Next >>",

                NavigateUrl = nextUrl

};

            linkNext.Enabled = (nextUrl.Length > 0);

            Controls.Add(linkNext);

        private string GetPreviousResultPageUrl()

            return GetSpecificResultUrl(false);

        private string GetNextResultPageUrl()

            return GetSpecificResultUrl(true);

        private string GetSpecificResultUrl(bool useNextResult)

            string url = "";

            if (_results != null)

                string xpath =

                    String.Format("/All_Results/Result[id='{0}']", _resultId);

                XPathNavigator xNavigator = _results.CreateNavigator();

                XPathNavigator xCurrentNode = xNavigator.SelectSingleNode(xpath);

                if (xCurrentNode != null)

                    bool hasNode = false;

                    if (useNextResult)

                        hasNode = xCurrentNode.MoveToNext();

                    else

                        hasNode = xCurrentNode.MoveToPrevious();

                    if (hasNode &&

                        xCurrentNode.LocalName.Equals("Result"))

                        string resultId =

                        xCurrentNode.SelectSingleNode("id").Value;

                        string fileUrl =

                        xCurrentNode.SelectSingleNode("url").Value;

                        url = String.Format("{0}?resultid={1}",

                           fileUrl, resultId);

            return url;

        private string GetSearchResultsPageUrl()

            string url = NewsArchivePageUrl;

            string userQuery =

                NewsArchiveCoreResultsWebPart.GetCachedUserQuery(this.Page);

            if (String.IsNullOrEmpty(userQuery))

                url = String.Format("{0}?resultid={1}", url, _resultId);

            else

                url = String.Format("{0}?{1}&resultid={2}",

                    url, userQuery, _resultId);

            return url;

Note how I use the "resultid" URL parameter to discern between normal navigation to a page and result set navigation between pages. If the resultid parameter is not there, then the navigation controls are hidden. The same goes for when there are no cached results. The "result" link could always be visible for as long as the user's query text is cached.

You can also provide this result set exploration capability for all kinds of pages, not just for a specific page layout, by adding the result set navigation control to your master page(s). The result set <id> and <url> elements are there for all kind of pages stored in your SharePoint solution.

Almost Excluding Specific Search Results in SharePoint 2010

2012-04-30T10:11:00.003+02:00

Sometimes you want to hide certain content from being exposed through search in certain SharePoint web-applications, even if the user really has access to the information in the actual content source. A scenario is intranet search that is openly used, but in which you want to prevent accidental information exposure. Think of a group working together on reqruiting, where the HR manager use the search center looking for information - you wouldn't want even excerpts of confidential information to be exposed in the search results.

So you carefully plan your content sources and crawl rules to only index the least possible amount of information. Still, even with crawl rules you will often need to tweak the query scope rules to exclude content at a more fine-grained level, or even add new scopes for providing search-driven content to users. Such configuration typically involves using exclude rules on content types or content sources. This is a story of how SharePoint can throw you a search results curveball, leading to accidental information disclosure.

In this scenario, I had created a new content source JobVault for crawling the HR site-collection in another SharePoint web-application, to be exposed only through a custom shared scope. So I tweaked the rules of the existing scopes such as "All Sites" to exclude the Puzzlepart JobVault content source, and added a new JobReqruiting scope that required the JobVault content source and included the content type JobHired and excluded the content type JobFired.

So no shared scopes defined in the Search Service Application (SSA) included JobFired information, as all scopes either excluded the HR content source or excluded the confidential content type. To my surprise our SharePoint search center would find and expose such pages and documents when searching for "you're fired!!!".

Knowing that the search center by default uses the "All Sites" scope when no specific scope is configured or defined in the keyword query, it was back to the SSA to verify the scope. It was all in order, and doing a property search on Scope:"All Sites" got me the expected results with no confidential data in it. The same result for Scope:"JobReqruiting", no information exposure there either. It looked very much like a best bet, but there where no best bet keywords defined for the site-collection.

The search center culprit was the Top Federated Results web-part in our basic search site, by default showing results from the local search index very much like best bets. That was the same location as defined in the core results web-part, so why this difference?

Looking into the details of the "Local Search Results" federated location, the reason became clear: "This location provides unscoped results from the Local Search index". The keyword here is "unscoped".

The solution is to add the "All Sites" scope to the federated location to ensure that results that you want to hide are also excluded from the federated results web-part. Add it to the "Query Template" and optionally also to the "More Results Link Template" under the "Location Information" section in "Edit Federated Location".

Now the content is hidden when searching. Not through query security trimming, but through query filtering. Forgetting to add the filter somewhere can expose the information, but then only to users that have permission to see the content anyway. The results are still security trimmed, so this no actual information disclosure risk.

Note that this approach is no replacement for real information security; if that is what you need, don't crawl confidential information from an SSA that is exposed through openly available SharePoint search, even on your intranet.

Migrate SharePoint 2010 Term Sets between MMS Term Stores

2012-04-21T12:36:00.001+02:00

When using the SharePoint 2010 managed metadata fields connected to termsets stored in the Managed Metadata Service (MMS) term store in your solutions, you should have a designated master MMS that is reused across all your SharePoint environment such as the development, test, staging and production farms. Having a single master termstore across all farms gives you the same termsets and terms with the same identifiers all over, allowing you to move content and content types from staging to production without invalidating all the fields and data connected to the MMS term store.

You'll find a lot of termset tools on CodePlex, some that use the standard SharePoint 2010 CSV import file format (which is without identifiers), and some that on paper does what you need, but don't fully work. Some of the better tools are SolidQ Managed Metadata Exporter for export and import of termset (CSV-style), SharePoint Term Store Powershell Utilities for fixing orphaned terms, and finally SharePoint Taxonomy and TermStore Utilities for real migration.

There are, however, standard SP2010 PowerShell cmdlets that allow you to migrate the complete termstore with full fidelity between Managed Metadata Service applications across farms. The drawback is that you can't do selective migration of specific termsets, the whole term store will be overwritten by the migration.

This script exports the term store to a backup file:

# MMS Application ID has to be passed for -Identity parameter

Export-SPMetadataWebServicePartitionData -Identity "12810c05-1f06-4e35-a6c3-01fc485956a3" -ServiceProxy -ServiceProxy "Managed Metadata Service" -Path "\\Puzzlepart\termstore\pzl-staging.bak"

This script imports the backup by overwriting the term store:

# MMS Application ID has to be passed for -Identity parameter

# NOTE: overwrites all existing termsets from MMS

# NOTE: overwrites the MMS content type HUB URL - must be reconfigured on target MMS proxy after restoring

Import-SPMetadataWebServicePartitionData -Identity "53150c05-1f06-4e35-a6c3-01fc485956a3" -ServiceProxy "Managed Metadata Service" -path "\\Puzzlepart\termstore\pzl-staging.bak" -OverwriteExisting

Tajeshwar Singh has posted several posts on using these scripts, including how to solve typical issues:

Note that after the term store migration, the MMS content type HUB URL configuration will also have been overwritten. You may not notice for some time, but the content type HUB publishing and subscriber timer jobs will stop working. What you will notice, is that if you try to click republish on a content type in the HUB, you'll get an "No valid proxy can be found to do this operation" error. See How to change the Content Type Hub URL by Michal Pisarek for the steps to rectify this.

Set-SPMetadataServiceApplication -Identity "Managed Metadata Service" -HubURI "http://puzzlepart:8181/"

After resetting this MMS configuration, you should verify that the content type publishing works correctly by republishing and running the timer jobs. Use "Site Collection Administration > Content Type Publishing" as shown on page 2 in Chris Geier's article to verify that the correct HUB is set and that HUB content types are pushed to the subscribers.

Getting Elevated Search Results in SharePoint 2010

2012-04-16T11:19:00.004+02:00

I often use the SharePoint 2010 search CoreResultsWebPart in combination with scopes, content types and managed properties defined in the Search Service Application (SSA) for having dynamic search-driven content in pages. Sometimes the users might need to see some excerpt of content that they really do not have access to, and that you don't want to grant them access to either; e.g. to show a summary to anonymous visitors on your public web-site from selected content that is really stored in the extranet web-application in the SharePoint farm.

What is needed then is to execute the search query with elevated privileges using a custom core results web-part. As my colleague Mikael Svenson shows in Doing blended search results in SharePoint–Part 2: The Custom CoreResultsWebPart Way, it is quite easy to get at the search results code and use the SharedQueryManager object that actually runs the query. Create a web-part that inherits the ootb web-part and override the GetXPathNavigator method like this:

namespace Puzzlepart.SharePoint.Presentation

    [ToolboxItemAttribute(false)]

    public class JobPostingCoreResultsWebPart : CoreResultsWebPart

        protected override void CreateChildControls()

            base.CreateChildControls();

        protected override XPathNavigator GetXPathNavigator(string viewPath)

            XmlDocument xmlDocument = null;

            QueryManager queryManager =

              SharedQueryManager.GetInstance(Page, QueryNumber)

                .QueryManager;

            SPSecurity.RunWithElevatedPrivileges(delegate()

                xmlDocument = queryManager.GetResults(queryManager[0]);

});

            XPathNavigator xPathNavigator = xmlDocument.CreateNavigator();

            return xPathNavigator;

Running the query with elevated privileges means that it can return any content that the app-pool identity has access to. Thus, it is important that you grant that account read permissions only on content that you would want just any user to see. Remember that the security trimming is done at query time, not at crawl time, with standard SP2010 server search. It is the credentials passed to the location's SSA proxy that is used for the security trimming. Use WindowsIdentity.GetCurrent() from the System.Security.Principal namespace if you need to get at the app-pool account from your code.

You would want to add a scope and/or some fixed keywords to the query in the code before getting the results, in order to prevent malicious or accidental misuse of the elevated web-part to search for just anything in the crawled content of the associated SSA that the app-pool identity has access to. Another alternative is to run the query under another identity than the app-pool account by using real Windows impersonation in combination with the Secure Store Service (see this post for all the needed code) as this allows for using a specific content query account.

The nice thing about using the built-in query manager this way, rather than running your own KeywordQuery and providing your own result XML local to the custom web-part instance, is that the shared QueryManager's Location object will get its Result XML document populated. This is important for the correct behavior for the other search web-parts on the page using the same QueryNumber / UserQuery, such as the paging and refiners web-parts.

The result XmlDocument will also be in the correct format with lower case column names, correct hit highlighting data, correct date formatting, duplicate trimming, getting <path> to be <url> and <urlEncoded>, have the correct additional managed and crawled properties in the result such as <FileExtension> and <ows_MetadataFacetInfo>, etc, in addition to having the row <id> element and <imageUrl> added to each result. If you override by using a replacement KeywordQuery you must also implement code to apply appended query, fixed query, scope, result properties, sorting and paging yourself to gain full fidelity for your custom query web-part configuration.

The formatting and layout of the search results is as usual controlled by overriding the result XSLT. This includes the data such as any links in the results, as you don't want the users to click on links that just will give them access denied errors.

When using the search box web-part, use the contextual scope option for the scopes dropdown with care. The ContextualScopeUrl (u=) parameter will default to the current web-application, causing an empty result set when using the custom core results web-part against a content source from another SharePoint web-application.

SharePoint 'Approve on behalf of' for Publishing Pages

2012-03-08T09:21:00.000+01:00

Using require approval and the approval workflow in SharePoint 2010 publishing, or just approval on the pages library in the simple publishing configuration, is straight forwards when using the browser as you're only logged in as one person with a limited set of roles and thus set of rights. Typically a page author can edit a page and submit it for approval, but not actually approve the page to be published on the site. When you need to approve, you have to log on as a user with approval rights.

Sometimes you need to extend the user experience to allow an author to make simple changes to an already published page, such as extending the publishing end date, and republish it directly without having to do all the approval procedures all over again. So you create a custom "extend expiry date" ribbon button, elevate the privileges from code and call the Page ListItem File Approve method, only to get an access denied error.

In SharePoint 2010, it is not sufficient to be the app-pool identity (SHAREPOINT\System) or even a site-collection admin, you have to run the approval procedures as a user with approval privileges. So your code needs to impersonate a user in the "Approvers" group to be able to approve an item on behalf of the current user.

public static void ApproveOnBehalfOfUser(

SPListItem item, string approversGroupName, string userName, string comment)

    Guid siteId = item.ParentList.ParentWeb.Site.ID;

    Guid webId = item.ParentList.ParentWeb.ID;

    Guid listId = item.ParentList.ID;

    Guid itemId = item.UniqueId;

    SPUserToken approveUser = GetApproverUserToken(siteId, webId, approversGroupName);

    if (approveUser == null)

        throw new ApplicationException(String.Format(

"The group '{0}' has no members of type user, cannot approve item on behalf of '{1}'",

approversGroupName, userName));

    using (SPSite site = new SPSite(siteId, approveUser))

        using (SPWeb web = site.OpenWeb(webId))

            var approveItem = web.Lists[listId].Items[itemId];

            approveItem.File.Approve(comment);

private static SPUserToken GetApproverUserToken(

Guid siteId, Guid webId, string approversGroupName)

    SPUserToken token = null;

    SPSecurity.RunWithElevatedPrivileges(() =>

        using (SPSite site = new SPSite(siteId))

            using (SPWeb web = site.OpenWeb(webId))

                var group = web.SiteGroups[approversGroupName];

                if (group != null)

                    foreach (SPUser user in group.Users)

                        if (!user.IsDomainGroup && !user.IsSiteAdmin)

                            token = web.GetUserToken(user.LoginName);

                            break;

});

    return token;

The code picks a user from the approvers group, and then impersonates that user using a SPUserToken object. From within the impersonated user token scope, the given page list item is opened again with the permissions of an approver, and finally the page is approved on behalf of the given page author.

SharePoint ResolvePrincipal for ADFS users

2012-02-28T09:27:00.003+01:00

Granting permissions in SharePoint 2010 by code is done by assigning roles to user or group principals, or for claims-based application, to a claim type instance. As you will find out when implementing a claims-based applications against ADFS, the SPUtility ResolvePrincipal method that you can use against the Windows identity provider and also against forms-based authentication (FBA), don't work for ADFS users. That is no surprise when looking at the SPPrincipalSource enum that lists the site's user info list, the Windows provider, and then the FBA membership and role providers.

The solution is rather simple, pass the user identity claim value such as the user's e-mail address used in ADFS to the SPClaimProviderManager CreateUserClaim method using the TrustedProvider issuer type. Then pass the generated claim to EnsureUser to get a SPPrincipal wrapping the user identity claim:

public static SPUser ResolveAdfsPrincipal(SPWeb web, string email, string issuerIdentifier)

    SPUser user = null;

    if (!SPClaimProviderManager.IsEncodedClaim(email))

        SPClaim claim = SPClaimProviderManager.CreateUserClaim(email, SPOriginalIssuerType.TrustedProvider, issuerIdentifier);

        if (claim != null)

            string userClaim = claim.ToEncodedString();

            user = web.EnsureUser(userClaim);

    return user;

public static SPUser ResolvePrincipal(SPWeb web, string email)

    SPUser user = null;

    SPPrincipalInfo identity = SPUtility.ResolvePrincipal(web, email, SPPrincipalType.All, SPPrincipalSource.All, null, true);

    if (identity != null)

        user = web.EnsureUser(identity.LoginName);

    return user;

Use the name of your ADFS identity provider as the issuer identifer parameter. If you're unsure of what this string should be, add a user to a SharePoint group using the UI and look at the identity claim value generated by SharePoint.

The identity claim value follows a specific format, and with ADFS (trusted provider) the issuer identifier name is always the 2nd pipe-separated value. The 1st part starting "i:" is for "identity" type claim, while the ending ".t" is for "trusted" provider (".f" is FBA, ".w" is Windows). In between is an encoding of the claim type used as the IdentifierClaim specified when registering the ADFS trusted identity provider. See the full list of claim type and claim value type encoding characters here. The 3rd part of the claim is the actual user claim value. This was an oversimplified explanation, refer to the SP2010 claims whitepaper for complete details.

Side note: The claim type encoding logic is important when using a custom claims provider (CCP) to augment the claimset with your own custom claim types. If you have CCPs on multiple SharePoint farms, it’s very important that you deploy all the CCPs in the same sequence across farms to ensure that the claim type mapping and encoding character for a custom claim type becomes the same across all farms. The best approach for ensuring parity across farms might be using Powershell to add the claim mappings first.

The code really doesn't resolve the user against ADFS, it just creates a claim type instance that authorization later on can validate against the logged in user's token (claim set). If the user's token contains the correct value for the claim type assigned to the securable object, then the user is authorized according to the role assignment.

Reusable SPGridView with Multiple Filter and Sort Columns

2012-02-16T18:59:00.000+01:00

The venerable SPGridView still has its use in SharePoint 2010 when your data is not stored in a list or accessible as an external content type through BCS. A typical example is when using a KeywordQuery to build a search-driven web-part feeding on results as DataTable as show in How to: Use the SharePoint 2010 Enterprise Search KeywordQuery Class by Corey Roth. Another example is cross-site queries using SPSiteDataQuery.

The SPGridView can use a DataTable as its data source, but several things from sorting arrows to filtering don't work as expected when not using an ObjectDataSource. As Shawn Kirby shows in SPGridView WebPart with Multiple Filter and Sort Columns it is quite easy to implement support for such features.

In this post, I show how to generalize Shawn's web-part code into a SPGridView derived class and a data source class wrapping a DataTable, isolating this functionality from the web-part code itself, for both better reusability and separation of concerns.

First the simple abstract data source class that you must implement to populate your data set:

namespace Puzzlepart.SharePoint.Core

    public abstract class SPGridViewDataSource

        public abstract DataTable SelectData(string sortExpression);

        protected void Sort(DataTable dataSource, string sortExpression)

            //clean up the sort expression if needed - the sort descending

            //menu item causes the double in some cases

            if (sortExpression.ToLowerInvariant().EndsWith("desc desc"))

                sortExpression = sortExpression.Substring(0, sortExpression.Length - 5);

            //need to handle the actual sorting of the data

            if (!string.IsNullOrEmpty(sortExpression))

                DataView view = new DataView(dataSource);

                view.Sort = sortExpression;

                DataTable newTable = view.ToTable();

                dataSource.Clear();

                dataSource.Merge(newTable);

The SPGridViewDataSource class provides the SelectData method that you must override, and a completed Sort method that allows the SPGridView to sort your DataTable. Note that this class must be stateless as required by any class used as an ObjectDataSource. Its logic cannot be combined with the grid view class, as it will get instantiated new every time the ObjectDataSource calls SelectData.

Then the derived grid view with support for filtering and sorting, including the arrows and filter images:

namespace Puzzlepart.SharePoint.Core

    public class SPGridViewMultiSortFilter : SPGridView

        public SPGridViewMultiSortFilter()

            this.FilteredDataSourcePropertyName = "FilterExpression";

            this.FilteredDataSourcePropertyFormat = "{1} = '{0}'";

        private ObjectDataSource _gridDS;

        private char[] _sortingSeparator = { ',' };

        private string[] _filterSeparator = { "AND" };

        public ObjectDataSource GridDataSource

            get { return _gridDS; }

            private set

                _gridDS = value;

                this.DataSourceID = _gridDS.ID;

        public bool AllowMultiSorting { get; set; }

        public bool AllowMultiFiltering { get; set; }

        string FilterExpression

. . .

        string SortExpression

. . .

        protected override void CreateChildControls()

            base.CreateChildControls();

            this.Sorting += new GridViewSortEventHandler(GridView_Sorting);

            this.RowDataBound += new GridViewRowEventHandler(GridView_RowDataBound);

        protected void GridView_Sorting(object sender, GridViewSortEventArgs e)

            EnsureChildControls();

            string direction = e.SortDirection.ToString();

            direction = (direction == "Descending") ? " DESC" : "";

            SortExpression = e.SortExpression + direction;

            e.SortExpression = SortExpression;

            //keep the object dataset filter

            if (!string.IsNullOrEmpty(FilterExpression))

                _gridDS.FilterExpression = FilterExpression;

        protected void GridView_RowDataBound(object sender, GridViewRowEventArgs e)

            EnsureChildControls();

            if (sender == null || e.Row.RowType != DataControlRowType.Header)

                return;

            BuildFilterView(_gridDS.FilterExpression);

            SPGridView grid = sender as SPGridView;

            // Show icon on filtered and sorted columns

            for (int i = 0; i < grid.Columns.Count; i++)

. . .

        void BuildFilterView(string filterExp)

. . .

            //update the filter

            if (!string.IsNullOrEmpty(lastExp))

                FilterExpression = lastExp;

            //reset object dataset filter

            if (!string.IsNullOrEmpty(FilterExpression))

                _gridDS.FilterExpression = FilterExpression;

        public ObjectDataSource SetObjectDataSource(string dataSourceId, SPGridViewDataSource dataSource)

            ObjectDataSource gridDS = new ObjectDataSource();

            gridDS.ID = dataSourceId;

            gridDS.SelectMethod = "SelectData";

            gridDS.TypeName = dataSource.GetType().AssemblyQualifiedName;

            gridDS.EnableViewState = false;

            gridDS.SortParameterName = "SortExpression";

            gridDS.FilterExpression = FilterExpression;

            this.GridDataSource = gridDS;

            return gridDS;

Only parts of the SPGridViewMultiSortFilter code is shown here, see download link below for the complete code. Note that I have added two properties that controls whether multi-column sorting and multi-column filtering are allowed or not.

This is an excerpt from a web-part that shows search results using the grid:

namespace Puzzlepart.SharePoint.Presentation

    [ToolboxItemAttribute(false)]

    public class JobPostingRollupWebPart : WebPart

        protected SPGridViewMultiSortFilter GridView = null;

        protected override void CreateChildControls()

try

                CreateJobPostingGrid();

            catch (Exception ex)

                Label error = new Label();

                error.Text = String.Format("An unexpected error occurred: {0}", ex.Message);

                error.ToolTip = ex.StackTrace;

                Controls.Add(error);

        private void CreateJobPostingGrid()

            GridView = new SPGridViewMultiSortFilter();

  . . .

            GridView.AllowSorting = true;

            GridView.AllowMultiSorting = false;

            GridView.AllowFiltering = true;

            GridView.FilterDataFields = "Title,Author,Write,";

  . . .

            Panel panel = new Panel();

            panel.Controls.Add(GridView);

            Controls.Add(panel);

            PopulateGridDataSource();

            //must bind in OnPreRender

            //GridView.DataBind();

        protected override void OnPreRender(EventArgs e)

            GridView.DataBind();

        private void PopulateGridDataSource()

            var dataSource = new ApprovedJobPostingDataSource();

            var gridDS = GridView.SetObjectDataSource("gridDS", dataSource);

            //add the data source

            Controls.Add(gridDS);

Note how the data source is created and assigned to the grid view, but also added to the control set of the web-part itself. This is required for the grid's DataSourceId binding to find the ObjectDataSource at run-time. Also note that data binding cannot be triggered from CreateChildControls as it is too early in the control's life cycle. The DataBind method must be called from OnPreRender to allow for view state and child controls to load before the sorting and filtering postback events

Finally, this is an example of how to implement a search-driven SPGridViewDataSource:

namespace Puzzlepart.SharePoint.Presentation

    public class ApprovedJobPostingDataSource : SPGridViewDataSource

        private string _cacheKey = "Puzzlepart_Godkjente_Jobbannonser";

        public override DataTable SelectData(string sortExpression)

            DataTable dataTable = (DataTable)HttpRuntime.Cache[_cacheKey];

            if (dataTable == null)

                dataTable = GetJobPostingData();

                HttpRuntime.Cache.Insert(_cacheKey, dataTable, null,

DateTime.Now.AddMinutes(1), Cache.NoSlidingExpiration);

            this.Sort(dataTable, sortExpression);

            return dataTable;

        private DataTable GetJobPostingData()

            DataTable results = new DataTable();

            string jobPostingManager = "Puzzlepart Jobbannonse arbeidsleder";

            string jobPostingAssistant = "Puzzlepart Jobbannonse assistent";

            string approvedStatus = "0";

            SPSite site = SPContext.Current.Site;

            KeywordQuery query = new KeywordQuery(site);

            query.QueryText = String.Format(

"ContentType:\"{0}\" ContentType:\"{1}\" ModerationStatus:\"{2}\"",

jobPostingManager, jobPostingAssistant, approvedStatus);

            query.ResultsProvider = SearchProvider.Default;

            query.ResultTypes = ResultType.RelevantResults;

            ResultTableCollection resultTables = query.Execute();

            if (resultTables.Count > 0)

                ResultTable searchResults = resultTables[ResultType.RelevantResults];

                results.Load(searchResults, LoadOption.OverwriteChanges);

            return results;

That was not too hard, was it? Note that SearchProvider Default work for both FAST (FS4SP) and a standard SharePoint 2010 Search Service Application (SSA).

All the code can be downloaded from here.

Custom ADFS Login Form for SharePoint 2010 Claims

2012-01-19T18:34:00.000+01:00

This week I've been involved in creating a custom login page for SharePoint 2010 to bypass the standard "select a login method" page for multi-mode claims-enabled web-applications. What we wanted was similar to the Claims Login Web Part for SharePoint Server 2010 for Forms-Based Authentication (FBA) by Jeremy Jameson, but for a trusted ADFS 2.0 identity provider instead.

Having a custom login page allows you to stay in your site and avoid the passive STS authN redirect dance back and forth between SP and the ADFS STS for authentication. This requires you to use active mode (WS-Trust) rather than the passive mode used by SharePoint. Note that this active approach won't give you single sign-on, because you won't get the MSISAuth ADFS SSO cookies - it will simply authenticate you first and then give you the SharePoint FedAuth cookie.

The code you need to call ADFS to make it authenticate you, and thus issue a claims token for use with SharePoint, can be found at Using an Active Endpoint to sign into a Web Application by Dominick Baier or Making a web application use an active STS by Koen Willemse. The missing detail not shown in their code is the URL to the ADFS endpoint, which needs to match the chosen client credentials and security mode; when using UserNameWSTrustBinding and sending the username and password in the WCF message secured using SSL (i.e. mixed), the URL should be like "http://adfs.pzl/adfs/services/trust/13/usernamemixed/" including the important ending / slash to avoid "405 method not allowed" error from IIS.

protected void btnLogin_Click(object sender, EventArgs e)

    // authenticate with WS-Trust endpoint

    var factory = new WSTrustChannelFactory(

        new UserNameWSTrustBinding(SecurityMode.TransportWithMessageCredential),

        new EndpointAddress(

"https://adfs.puzzlepart.com/adfs/services/trust/13/usernamemixed/"));

    factory.Credentials.UserName.UserName = txtUserName.Text;

    factory.Credentials.UserName.Password = txtPassword.Text;

    var channel = factory.CreateChannel();

    var rst = new RequestSecurityToken

        RequestType = RequestTypes.Issue,

        AppliesTo = new EndpointAddress("urn:sharepoint:puzzlepart"),

        KeyType = KeyTypes.Bearer

};

    var genericToken = channel.Issue(rst) as GenericXmlSecurityToken;

    // parse token

    var handlers = FederatedAuthentication.ServiceConfiguration

.SecurityTokenHandlers;

    var token = handlers.ReadToken(new XmlTextReader(

       new StringReader(genericToken.TokenXml.OuterXml)));

    SPSecurity.RunWithElevatedPrivileges(delegate(){

        SPFederationAuthenticationModule.Current

.SetPrincipalAndWriteSessionToken(token);

});

    Response.Redirect("~/pages/default.aspx");

After getting authenticated against the ADFS STS when calling Issue on the WS-Trust channel with a RequestSecurityToken, the returned SAML security token must first be parsed and then written to a FedAuth cookie created from the SAML token. The SharePoint FAM wrapper will both set the thread principal and write the cookie, making the user a logged in SharePoint user.

Note how the writing of the cookie is wrapped with RunWithElevatedPrivileges to ensure that it runs as the app-pool identity and not as the impersonated SharePoint user. This is to avoid the dreaded "CryptographicException: The system cannot find the file specified" error in the internal ProtectedDataCookieTransform call.

When calling ValidateToken you will run into the SecurityTokenException: Issuer of the Token is not a Trusted Issuer error if your STS is not trusted by SharePoint. SharePoint is configured to use its own SPPassiveIssuerNameRegistry and that will either validate against the built-in SharePoint STS or the set of trusted STS token issuers. See how to add your STS certificate(s) at SharePoint 2010 Claims-Based Auth with ADFS v2 by Eric Kraus. The trusted providers are apparently only used if the login page is located under the /_trust/ folder that is part of the above "redirect dance" when authenticating against a trusted identity provider.

Over at stack overflow, Matt Whetton had run into the same exception as us and solved it by replacing the passive <issuerNameRegistry> with the Windows Identity Foundation (WIF) ConfigurationBasedIssuerNameRegistry instead. The Configuration of WIF post shows how to add the set of certificate names and thumbprints to the <trustedIssuers> list. This is how your web.config list of trusted STS token issuers may look like:

<issuerNameRegistry type="Microsoft.IdentityModel.Tokens.ConfigurationBasedIssuerNameRegistry,

Microsoft.IdentityModel, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">

  <trustedIssuers>

    <add thumbprint="1337133713371337" name="CN=adfs-puzzlepart" />

    <add thumbprint="0000000000000000" name="CN=SharePoint Security Token Service" />

  </trustedIssuers>

</issuerNameRegistry>

Remember to add the SharePoint self-issued certificates such as the "SharePoint Security Token Service" certificate to the list of trusted issuers, in addition to your own STS.

I strongly recommend putting your custom ADFS login page under /_trust/ to avoid having to change the SharePoint web.config files. We chose this approach to minimize risks.

Note that Fiddler seems to break the ADFS login process, at least when decrypting SSL. The customized rule provided in Fiddler and Channel Binding Tokens Revisited by Eric Lawrence alleviates this problem. Just make sure you click "remember my credentials" when logging in so that Fiddler can get it from the Windows Credential Manager.

Disclaimer: Note that even if things seems to work as normal after this configuration change, there is no guarantee that nothing was affected in the huge platform that SharePoint 2010 is. The combination of SP2010 claims and WIF is not very well documented, and any changes beyond supported configuration involves risks. Do not apply these changes if you are not sure that it will not break any of your SharePoint solutions or services.

Simple Feature Files Cleanup using Extension Methods

2012-01-17T09:22:00.000+01:00

As every seasoned SharePoint developer knows, deactivating a feature does not remove the files deployed by that feature. The deployed masterpages, web part pages, wiki pages, page layouts, web-part definitions, styling artifacts, etc files will stay in the target libraries - and they will not be overwritten on feature activation. Don't let Visual Studio 2010 trick you into believing otherwise.

You have to delete those deployed files yourself in the FeatureDeactivating event. The classic approach is to delete the files one-by-one, but this is tedious and error-prone. The following is a set of extension methods that allows you to simply do this:

public override void FeatureDeactivating(SPFeatureReceiverProperties properties)

     SPSite site = (SPSite) properties.Feature.Parent;

     properties.Definition.DeleteFeatureFiles("MasterPages", site.RootWeb);

     properties.Definition.DeleteFeatureWebPartFiles(site.RootWeb);

The code is an adaptation of Corey Roth's LINQ to XML and Deleting Files on Feature Deactivation, using extension methods and supporting cleanup of specific feature modules and all feature web-parts.

namespace Puzzlepart.SharePoint.Core.SPExtentions

    public static class SPFeatureDefinitionExtentions

        public class Module

            public string Name { get; set; }

            public string Path { get; set; }

            public List<string> Files { get; set; }

        public static void DeleteFeatureFiles

(this SPFeatureDefinition spFeatureDefinition, string moduleName, SPWeb web)

            List<Module> modules = GetModuleFiles(spFeatureDefinition, moduleName);

            foreach (Module module in modules)

                DeleteModuleFiles(module, web);

        public static void DeleteFeatureWebPartFiles

(this SPFeatureDefinition spFeatureDefinition, SPWeb web)

            List<Module> modules = GetAllModuleFiles(spFeatureDefinition);

            foreach (Module module in modules)

                if (string.Compare(module.Path, "_catalogs/wp",

StringComparison.CurrentCultureIgnoreCase) == 0)

                    DeleteModuleFiles(module, web);

        private static List<Module> GetModuleFiles

(SPFeatureDefinition spFeatureDefinition, string moduleName)

            string elementsPath = string.Format(@"{0}\FEATURES\{1}\{2}\Elements.xml",

SPUtility.GetGenericSetupPath("Template"),

spFeatureDefinition.DisplayName, moduleName);

            XDocument elementsXml = XDocument.Load(elementsPath);

            XNamespace sharePointNamespace = "http://schemas.microsoft.com/sharepoint/";

            // get each module name and the files in it

            var moduleList =

                from module in elementsXml.Root.Elements(sharePointNamespace + "Module")

                select new

                    Name = (module.Attributes("Name").Any())

? module.Attribute("Name").Value : null,

                    ModuleUrl = (module.Attributes("Url").Any())

? module.Attribute("Url").Value : null,

                    Files = module.Elements(sharePointNamespace + "File")

};

            List<Module> modules = new List<Module>();

            // iterate through each module with files

            foreach (var module in moduleList)

                Module m = new Module()

                                   Name = module.Name,

                                   Path = module.ModuleUrl

};

                List<string> files = new List<string>();

                foreach (var fileElement in module.Files)

                    string filename = (fileElement.Attributes("Name").Any())

? fileElement.Attribute("Name").Value : fileElement.Attribute("Url").Value;

                    files.Add(filename);

                m.Files = files;

                modules.Add(m);

            return modules;

        private static void DeleteModuleFiles(Module module, SPWeb web)

            foreach (string filename in module.Files)

                if (!string.IsNullOrEmpty(module.Path))

                    web.GetFile(string.Format("{0}/{1}", module.Path, filename)).Delete();

                else

                    web.Files.Delete(filename);

        private static List<Module> GetAllModuleFiles

(SPFeatureDefinition spFeatureDefinition)

            var moduleList = new List<Module>();

            string modulesPath = string.Format(@"{0}\FEATURES\{1}\",

SPUtility.GetGenericSetupPath("Template"),

spFeatureDefinition.DisplayName);

            DirectoryInfo folder = new DirectoryInfo(modulesPath);

            foreach (DirectoryInfo moduleFolder in folder.GetDirectories())

                moduleList.AddRange(GetModuleFiles(spFeatureDefinition, moduleFolder.Name));

            return moduleList;

Note that page layouts cannot simply be deleted if they are in use. Use code to revert the "GhostableInLibrary" files to the uncustomized (ghosted) feature files on disk in the SharePoint root [14].

SharePoint 2010 Localized Publishing Web Template

2012-01-03T14:11:00.000+01:00

When you try to create a new localized publishing site based on a minimal SharePoint 2010 publishing web template (or a similar minimal site definition), it might fail with a "CreateWelcomePage" error such as this:

System.Runtime.InteropServices.COMException (0x80070001): 0x80070001 at

Microsoft.SharePoint.Library.SPRequestInternalClass.GetMetadataForUrl

(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId,

Int32& plType, Object& pvarFileOrFolder) at

Microsoft.SharePoint.Library.SPRequest.GetMetadataForUrl

(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId,

Int32& plType, Object& pvarFileOrFolder) -

-- End of inner exception stack trace --- at

Microsoft.SharePoint.SPGlobal.HandleComException(COMException comEx)

at

Microsoft.SharePoint.Library.SPRequest.GetMetadataForUrl

(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId,

Int32& plType, Object& pvarFileOrFolder) at

Microsoft.SharePoint.SPWeb.GetListItem

(String strUrl, Boolean bFields, String[] fields) at

Microsoft.SharePoint.Publishing.PublishingWeb.GetPublishingPage(String strUrl) at

Microsoft.SharePoint.Publishing.Internal.AreaProvisioner.CreateWelcomePage

(PublishingWeb area, PageLayout pageLayout) at

Microsoft.SharePoint.Publishing.Internal.AreaProvisioner.SetDefaultPageProperties

(PublishingWeb area, Boolean& updateRequired) at

Microsoft.SharePoint.Publishing.Internal.AreaProvisioner.

InitializePublishingWebDefaults()

- -- End of inner exception stack trace --- at

Microsoft.SharePoint.Publishing.Internal.AreaProvisioner.

InitializePublishingWebDefaults() at

Microsoft.SharePoint.Publishing.Internal.AreaProvisioner.Provision()

at

Microsoft.SharePoint.Publishing.PublishingFeatureHandler.<>c_DisplayClass3.b_0() at

Microsoft.Office.Server.Utilities.CultureUtility.RunWithCultureScope

(CodeToRunWithCultureScope code) at

Microsoft.SharePoint.Publishing.CmsSecurityUtilities.RunWithWebCulture

(SPWeb web, CodeToRun webCultureDependentCode) at

Microsoft.SharePoint.Publishing.PublishingFeatureHandler.FeatureActivated

(SPFeatureReceiverProperties receiverProperties).

The typical cause is that your web template/site definition is a bit too minimal. The "Publishing" feature needs some initial configuration property data during feature activation. Don't strip these properties away completely. Also, activating the "Publishing" feature from code or a feature stapler will not work for localized sites if you don't pass in this configuration. It is not standard, but you can pass property XML data from code to feature activation as shown in Specifying Properties When Activating Features Through Code.

You must pass in the publishing feature property configuration for the "WelcomePageUrl" to ensure that is reference the localized pages library during activation, which is /sider/ for LCID 1044. The fallback for when this property is not set or is empty seems to be hardcoded to /pages/. Note that using "osrvcore" as the resource file is needed for some languages if you don't have SP1 of the language pack installed.

<!-- Feature: Publishing -->

<Feature ID="22A9EF51-737B-4ff2-9346-694633FE4416">

      <Properties xmlns="http://schemas.microsoft.com/sharepoint/">

            <Property Key="ChromeMasterUrl"

              Value="~SiteCollection/_catalogs/masterpage/puzzlepart.master" />

            <Property Key="WelcomePageUrl"

              Value="$Resources:cmscore,List_Pages_UrlName;/default.aspx"/>

It is important to reference an existing page in an existing library as the welcome page (home page). Deploy a page using a module if needed. Note that not all of these properties need to be specified as they have working default settings as fallback.

SharePoint Information Architecture from the Field

2011-11-01T11:19:00.002+01:00

Over the years, I've written quite a few articles on how to technically structure your SharePoint solutions into web-apps, sites, subsites, lists and document libraries. All based on having a defined Information Architecture (IA) as a basis for the solution design, or at least being able to reason about your content management using my "chest of drawers" and "news paper" analogies. The latter is based on my experiences from the field as most companies don't have a well-defined IA in place.

Common questions from customers are "what is Information Architecture?" and "what is the value of having an IA for SharePoint, can't we just create sites and doc-libs on the fly as needed?". Not to forget "how do we go about creating an Information Architecture for SharePoint?". So here are some IA advice from Puzzlepart projects:

In short, Information Architecture defines how to classify and structure your content so that it is easy for content consumers to find and explore relevant content, while making it simple for workers to contribute and manage content in an efficient manner.

The business goal of having an Information Architecture for your SharePoint solution is enabling workers to contribute, store and mange content in a manner that is simple and efficient, enabling more content sharing; and at the same time making it easy for workers to browse and find content they need, while also making it easy for workers to discover and explore relevant content they didn't know of. The outcome is more knowledgeable workers that are better informed about what’s going on in the company and about the range of intellectual property possessed by other employees, while also saving time wasted on finding information, and time wasted on incorrect or outdated information.

An outcome is a metric that customers use to define the successful realization of the objectives and goals. Outcomes happens after the project has delivered on its objectives and goals, and the customers must themselves work against securing the outcomes to achieve the desired business value.

The business value of having a working IA is capturing company knowledge from employees with better quality of shared content, which combined with good findability drive more knowledgeable workers that make better decisions and better faster processes. In addition, more and better content sharing helps user not only discover and explore content, but also people such as subject matter experts, allowing employees to build and expand their network throughout the company, helping the company to retain talented employees through social ties and communities. Access to discover more and better content and people expertise is central to enabling innovation and process improvement, as new knowledge is a trigger for new ideas and for identifying new opportunities.

The process of defining your IA for your SharePoint solution should focus on these objectives and goals:

Analyze and define the content classification and structure for the solution

goal: identify what content to manage and plan how to store it in SP, leading to sites, subsites and doc-lib structure organized into SP web-apps

Analyze and define how to browse and navigate the content

goal: make it simple and efficient for users to find and use known content that they need in they daily work to drive better faster processes

Analyze and define how to discover and explore the content

goal: make it easy for users to stumble upon novel shared knowledge based on "common focus" to trigger innovation and build social ties

Provide simple and efficient content contributor experience with liberal appliance of default metadata values, storing content close to the authors

goal: make workers contributors, not knowledge management grunts, and help them store content correctly with better metadata and tagging, driving findability and "common focus" content discovery; drive better sharing and collaboration

Analyze and define the starter content types with metadata and term set taxonomy based on the defined site and doc-lib architecture, with a strong focus on needed search experience capabilities

goal: enable content management and support both search-driven and "common focus" content; drive findability, sharing and innovation

Analyze and define the policies for social tagging and rating in the solution, also in relation to user profile interests, skills and responsibility tagging

goal: drive "common focus" content discovery, drive findability, drive social communities, drive innovation

Analyze and define the search experience, focusing on both search-driven content and on search center scopes and refiners

goal: drive findability and provide both search-driven and "common focus" content

Enable disposition of redundant and irrelevant content

goal: provide users with better, correct and up-to-date information, drive findability, save storage cost, save process cost

Including the search experience helps you avoid an initial version of your IA with a too narrow scope, which is easy to do for a starter IA when e.g. analyzing only project collaboration needs or just the document types of an attachment-based intranet. It makes you focus on what you get out, not only what you put in.

Note that navigation is not IA, its just one way to explore the content. Using navigation to structure your content is just reapplying the fileshare folder approach, which we all know doesn't work too good for findability and discovery. Navigation should not define the statical IA structure for the content, do a LATCH analysis to model the possible IA structures, and choose one of them to define the statical IA structure. The site map is closer to define statical IA structure than navigation, still it is only good for logical IA structure and cannot be expected to be used directly as the physical IA structure in SharePoint.

In an upcoming article I will give some practical advice from the field on how to define and realize the Information Architecture for your SharePoint solution in an agile fashion.

SharePoint is like a Chest of Drawers

2011-10-27T10:10:00.001+02:00

I often get asked "how many document libraries and sites will we need?" in SharePoint, followed by "how will we know whether we should use more doc-libs in a site or just throw it all in there?" and "where should content be stored? we need to show it on the intranet home page, but it is really edited and owned by HR in region Gokk". Well, SharePoint is like a chest of drawers.

To answer such questions, you need to know how to classify and structure your content; ideally you should have an Information Architecture for all your different kinds of data. If you are like most others, you don't. This is where the "chest of drawers" analogy might help you reason about your content.

Whether you need many doc-libs or subsites or not depends on your IA policies for information management. Still don't have an IA? Think of a chest of drawers for you clothes: it makes it easier to manage different types of clothes in different ways at different schedules, by e.g. separating t-shirts from trousers. Maybe even handle different kinds of t-shirts differently, such as your precious Maiden t-shirts. It also allows for delegating a few drawers to be managed by your wife; maybe you even want to have some locked drawers with more privacy :)

Your content are the clothes, the drawers are doc-libs or even subsites; and depending on the variety of clothes you have, you might need quite a sophisticated chest of drawers. Throw in all your other stuff, and you might need a bigger closet or a garage!

So now all your content is stored into nicely separated drawers, with delegated and secure handling where needed. But is is not so easy to see what is in the drawers without actually opening and browsing the content of each drawer. Until we get one of those science fiction closets that knows whats in the drawers and let us explore what trendy outfits we can wear today, its time for another analogy: the good old news paper, even in its modern online incarnation.

Think of a news paper with a front page and then multiple sections, such as domestic, foreign, sports, economics, etc. The front page and section front pages are used to show the (elsewhere) stored content to readers, helping them quicly browse the content at wellknow locations in the paper. The shown stories are typically rollup content stored elsewhere, typically where maintained, close to the content editors.

So a paper is built from dispersed storage of content that can be rolled up and targeted to readers multiple places. The home page and section pages rollup content "teasers" and allows the user to browse the content stored elsewhere in the paper and decide whether to explore it further.

The front page is the home page of your SharePoint site, the sections are subsites and the section pages are the subsite welcome pages in SharePoint parlance. As for the drawers, there might be different management policies and different people handling the different sections, and this helps you decide when subsites are needed. The rollup, or cross publishing if you like, is achieved using the content by query web-part or search-driven content based on content types, tagging and metadata.

Controlled and secure management of content according to different policies and schedules is much simpler when using subsites as compared to throwing it all into one site. Store the content close to the producers, show it everywhere the users expect to find it - and also where *you* want them to discover and explore knowledge new to them.

Issue with SP2010 Personal Site UserInfo Synchronization

2011-09-08T12:45:00.003+02:00

Today we discovered an issue with the SharePoint synchronization from the user profile database to the hidden UserInfoList in all site-collections. This sync is performed by two timer jobs (see profile sync details in this excellent article on the Bamboo Team Blog), which will update changes to your user profile in all the cached profile data in the hidden user info lists, except for the UserInfoList in your personal site under the profile site (My Site Host).

To verify the reported bug, I updated my mobile phone number in my user profile, and ran the two sync timer jobs. This is how my updated user information looks in a team-sites:

And this is how my non-updated user information looks in my personal site:

As you can see from the time stamps in the lower left corner, the profile data is still exactly as cached in the UserInfoList when I first created and visited my personal site. As of now I don't know any fix for this issue.

[UPDATE] A list of things to check, not all applies to SP2010 though: Troubleshooting User Profile Sync issues in Microsoft Office SharePoint Server 2007

As it turns out, all our personal sites get the "ProfileSynchronizationInternalException: ProfSynch: The site with ID <guid> cannot be synchronized due to an unprovisioned root web" error in the ULS. This seems to be a common problem in SharePoint 2010 according to this MSDN forum thread, which also provides an unsupported workaround that updates the Flags column of the Webs table in the My Site Host content database.

Some Gotchas when Customizing the "My Content" Personal Site

2011-08-10T09:58:00.004+02:00

Customizing an existing SharePoint 2010 site definition such as the personal site (SPSPERS) that provides the "My Content" section in the My Site Host web-application, is a bit different than customizing your own site definitions. As the supported way of customizing existing site definitions is to use feature stapling, you need to consider the provisioning order of elements in onet.xml and referenced and stapled 'SPSite' and 'SPWeb' features. Failing to do so might result in strange end results when creating a new site.

The MCS Norway team has done a good job of documenting the SharePoint element provisioning order, as part of their SiteConfigurator available at CodePlex:

"There are several steps in the creation process and SharePoint provisions in the following order:

Global onet.xml This file defines list templates for hidden lists, list base types, a default definition configuration, and modules that apply globally to the deployment.

SPSite scoped features defined in site definitions onet.xml, in the order they are defined in the file. The onet.xml file defined in the site definition can define navigational areas, list templates, document templates, configurations, modules, components, and server e-mail footers used in the site definition to which it corresponds.

SPSite scoped stapled features, in quasi random order

SPWeb scoped features defined in onet.xml, in the order they are defined in the file.

SPWeb scoped stapled features, in quasi random order

List instances defined in onet.xml

Modules defined in onet.xml

This is a fairly complex process and it can often be hard to know the method for customizing a site definition. A solution can be right in one scenario and completely wrong in another, making this somewhat confusing."

Here are some gotchas related to SPSPERS customization:

The doc-libs Shared Documents and Personal Documents do not exist yet during feature stapling; list instances in onet.xml are provisioned in step 6.
The my content home page default.aspx do not exist yet during feature stapling; files and pages in onet.xml are provisioned in step 7.
Do not create your own customized default.aspx file, it will get filled with the standard web-parts when the file's AllUsersWebParts are provisioned by the onet.xml module in step 7.
The quick launch heading node titles are not yet localized during site provisioning, look them up by their id rather than their title when adding links
The standard BlogView web-part only works when in a site page in the site root, it won't work in pages stored in lists, doc-libs or custom folders.
The Wiki Page Home Page feature is not activated by default for personal sites; do not provision your own /SitePages/ custom list, doc-lib or folder, as this will prevent enabling wiki pages later on.

To customize the standard personal site home page, you must provision a new home page with a different name and change the site's home page setting (SPFolder rootFolder.WelcomePage). Remember to restore the standard setting when deactiving your customization feature.

I strongly recommend using or learning from the SiteConfigurator, download the feature and the source code from CodePlex and join the community there.

Problem creating a FAST Content SSA in SharePoint 2010

2011-07-06T08:26:00.005+02:00

While installing Fast Search Server for SharePoint 2010 (FS4SP) on a dev farm today, I got a problem with the provisioning of a new FAST Content SSA (Search Service Application), it would hang forever at "0:01 Configuring the Search Service..." waiting for the TopologyConfigFinish.aspx page to complete.

The problem turned out to be that the SharePoint 2010 Administration service wasn't started after the mandatory server reboot after installing FS4SP. The FAST "nctrl status" cmdlet does not check this. Make sure that both the SP2010 Administration and Timer services are running:

If you still can't create new or delete search service application instances, or make topology changes at all, then you might need to delete the old SSA the hard way. See Deleteing the search service application and How to delete orphan configuration objects from SharePoint farm. Heed this warning: "Please be VERY careful when executing the deleteconfigurationobject command, if this command is not used in the correct way (if you end up deleting the wrong object) there is NO way to revert back the changes and it has the potential to render your Configuration Database useless, hence you may require to restore / rebuild your SharePoint farm".

Remember to configure SSL enabled communication again when recreating the FAST Content SSA, otherwise your next crawl will be stuck on starting while retrying every 60 seconds to connect to the document engine. Also remember to restart the FAST Search for SharePoint and the SharePoint Server Search 14 services before starting a new full crawl.

Delay Loading of Data in SharePoint 2010 Web Parts

2011-06-24T12:23:00.027+02:00

Sometimes your web-parts may take a long time to load their data, e.g. when connecting to external data through BCS, doing SPSiteDataQuery across a large number of sites, or when iterating over a user's site memberships to read some items from lists in different site-collections. Put a few of such web-parts on a dashboard page and wait for the combined load time of all those web-parts to complete before the page is shown. Not a nice user experience. These days users expect something as shown in this short screencast:

If you've used ASP.NET Ajax UpdatePanels, you might wish to utilize the asynchronous partial page update experience seen on postbacks also during page load. The simple thing seems to be calling __doPostBack for each UpdatePanel from the page load JavaScript event to trigger the Ajax async partial postback. That won't work, as only one concurrent postback is allowed by ASP.NET Ajax, so only one of your web-parts will work as expected, the other __doPostBack calls will get canceled by the ScriptManager.

A simple solution to this problem, is to put an asp:timer control inside the UpdatePanel and let it trigger a postback to your web-part code. Then load the data and update the content of the UpdatePanel during this async Ajax postback.

Here are the code to two base classes that implements this delayed load approach:

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using System.Web.UI;

using System.Web.UI.WebControls;

namespace Puzzlepart.SharePoint.WebParts

    public class AjaxPanelWebPart : System.Web.UI.WebControls.WebParts.WebPart

        protected UpdatePanel AjaxPanel;

        protected override void OnLoad(EventArgs e)

            base.OnLoad(e);

            this.EnsureChildControls();

        protected override void CreateChildControls()

            AjaxPanel = new UpdatePanel()

                ID = this.ID + "UpdatePanel1",

                UpdateMode = UpdatePanelUpdateMode.Conditional

};

            Controls.Add(AjaxPanel);

            UpdatePanelConfigurator.AddUpdatePanelProgress(AjaxPanel);

        protected virtual void ApplyUserActions()

            //to be overridden in derived classes

        protected void RebindControlsWhenNoViewState()

            if (Page.IsPostBack == true &&

            System.Web.UI.ScriptManager.GetCurrent(Page).IsInAsyncPostBack == false)

                ApplyUserActions();

    public class AjaxPanelDelayedLoadWebPart : AjaxPanelWebPart

        protected Timer LoadTimer;

        protected override void OnLoad(EventArgs e)

            base.OnLoad(e);

            this.EnsureChildControls();

        protected override void CreateChildControls()

            base.CreateChildControls();

            CreateLoadTimer();

        private void CreateLoadTimer()

            LoadTimer = new Timer()

                ID = this.ID + "LoadTimer1",

                Interval = 1 //millisecond

};

            LoadTimer.Tick += new EventHandler<EventArgs>(LoadTimer_Tick);

            AjaxPanel.ContentTemplateContainer.Controls.Add(LoadTimer);

        protected void LoadTimer_Tick(object sender, EventArgs e)

            LoadTimer.Enabled = false;

try

                ApplyUserActions();

            catch (Exception ex)

                Label msg = new Label()

                    Text = "An error occurred in delayed load: " + ex.Message,

                    ToolTip = ex.ToString()

};

                Controls.Add(msg);

The ApplyUserActions method is where you should fetch your data and update the content of the AjaxPanel member control. All the controls of your web-part should be created as usual, remember to call base.CreateChildControls in your derived web-parts to ensure that the Ajax controls get created.

Note that no slow data must be fetched and bound to your web-part controls during page load, e.g. in the CreateChildControls or OnPreRender methods, as this defeats the purpose of delay loading the data in the ApplyUserActions async postback method. A typical scenario is creating and configuring an SPGridView control in CreateChildControls and then fetch the data and set the grid's DataSource and call the grid's DataBind method in the overridden ApplyUserActions method in your derived web-part.

Note that all page event code get executed on partial postbacks for all web-parts on the page. This can cause problems that are unrelated to the web-part that triggers the postback, manifested as ScriptResource.axd JavaScript errors. Some problems are related to viewstate handling, such as the "Error=Value cannot be null. Parameter name: container" SPGridView exception. The simple solution is to turn off viewstate, and then call the RebindControlsWhenNoViewState method to load and bind the data when the postback is not an async Ajax postback. This must also be done for all controls that do not use viewstate, otherwise they will end up empty after e.g. modal dialogs that reload the page on close.

This ASP.NET Timer approach allows the page to load quickly, then each web-part will in turn get the timer tick postback and update itself using ASP.NET Ajax partial page updates. Note that this code won’t work as a sandboxed web-part. The UpdatePanel control requires the ScriptManager, which isn’t accessible from the sandboxed worker process.

The more professional way of getting real asynchronous loading for web-part content is to use PageAsyncTask as shown in Chapter 9 in Wictor Wilen's excellent SharePoint 2010 Web Parts in Action book. It does require a bit more code, but will allow parallell data fetching and thus faster page load time. It also works without using any UpdatePanels as all is done server-side.

New Sites and the SharePoint 2010 Content Type Hub

2011-05-19T20:18:00.018+02:00

The SharePoint 2010 content type hub does quite a good job of managing and publishing a centrally controlled set of content types. There are a few quirks and limitations, some of them documented in Content Type Hub FAQ and Limitations by Chaks' SharePoint Corner; not to forget the content type publishing timer jobs that actually push the content types to the subscribers.

One of the less documented areas of using a content type hub (HUB) is what happens to new site-collections that are provisioned? What if I have list definitions in my features, how can I be sure that their referenced content types have been provisioned at feature activation time? Can I deploy my enterprise content types feature at both the hub site-collection and also at new site-collections that users create?

First, when a new site-collection is created, it will immediately have all the published content types from the parent SharePoint web-application's connected Managed Metadata Service (MMS) application's defined HUB, automatically provisioned into its local content type gallery. Note that this applies only to the published content types as configured in the source hub. Content types that are not published, will not exist in your new site-collection. Note that hub content types are not by default published; this must be configured for every single content type in the source hub.

So if your list definitions depend on global content types that have not been published in the HUB, your feature activation will fail. You can of course solve this by publishing the applicable global content types in the source hub and run the timer jobs first, as this will ensure that new site-collections will have the enterprise content types auto-provisioned from the MMS HUB.

However, you can also deploy your enterprise content types feature to both the content type hub and to any other site-collection that you create. This works fine as the site content type definitions are identical, including the content type ID structure - after all it is the same content type CAML feature. This won't affect subscribing to the content type hub, and publishing new, updated or derived content types from the hub works just as expected.

Activate your site content type feature before activating your list definition feature, or any other feature that depends on the site content types being provisioned, to ensure that they exists locally in the new site-collection even if not yet published in the HUB.

As your taxonomy is subject to change, so are your enterprise content types. Thus, your deployment strategy for enterprise content types needs to handle change. I strongly recommend using the Open-Closed Principle for modifying and extending the enterprise content types. The Open-Closed Principle is based on using a set of immutable base content types that you derive from to make new specialized content types, inheriting fields from the base. The immutable base of the Open-Closed Principle coincides nicely with provisioning global content types through both a feature and the content type hub, as by policy any changes are made by extending the former through the latter.

Even trivial stuff such as providing standardized company templates for Word and other Office applications, is best done by publishing new derived content types. Use the content type hub to inherit your base PzlDocument into PzlDocumentMemo and attach a template, go to "Manage publishing for this content type" to publish the content type. Wait for, or run, the two HUB timer jobs, and then add the Word template to the applicable document libraries.

Now you're in for a surprise later on. The next time you try to create a new site-collection after publishing modifications in the HUB, you might get this "content type is read only" error:

The ULS log typically contains an exception like this:

SPContentTypeReadOnlyException
Error code: -2146232832
The content type is read only or updateChildren is true and one of the child objects of the content type is read only.

The root cause for this is that published content types are by default read-only in the subscribers. What typically leads to this error is the need to use code when provisioning content types, e.g. when renaming and reordering fields, or when adding the enterprise keywords field to your content type. Another typical scenario where code is required is managed metadata fields; see How to provision SharePoint 2010 Managed Metadata columns by Wictor Wilén.

Making changes to the site content type definition in the FeatureActivated code and then calling SPContentType Update with updateChildren=true will work fine, until someone creates a new derived content type in the source hub and publish it. Your carefully tested code will suddenly crash, as the published child content type is read-only! Alas, what better proof that the deployed and the published global content types are the same?

Luckily, the change is isolated to the new inherited content type, thus it can safely be ignored when deploying the base content types. Use this overloaded Update method when modifying the global content types:

public void Update(

         bool updateChildren := true,

         bool throwOnSealedOrReadOnly := false

The HUB change did not affect your global content type due to using the Open-Closed governance policy for enterprise content types. See my SharePoint 2010 Open-Closed Taxonomy post to learn more about this recommended policy.

The content type hub and the Managed Metadata Service are perhaps the best new features in SharePoint 2010, still there are some uncharted areas that make developers reluctant at using the MMS HUB. There are a lot of articles at Technet and MSDN on the architecture, but way too little about deployment scenarios and issues such as those in this post.

Site Lifecycle Management using Retention Policies

2011-05-07T19:53:00.023+02:00

The ootb governance tools for site lifecycle management (SLM) in SharePoint 2010 have not improved from the previous version. You're still stuck with the Site Use Confirmation and Deletion policies that will just periodically e-mail site owners and ask them to confirm that their site is still in use. There is no check for the site or its content actually being used, it is just a dumb timer job. If the site is not confirmed as still being active, the site will then be deleted - even if it is still in use. As deleting a site is not covered by any SharePoint recycle bin mechanism (coming in SP1), Microsoft also provides the site deletion capture tool on CodePlex.

Wouldn't it be nice if we could apply the information management policies for retention and disposition of content also for SharePoint 2010 sites? Yes we can :) By using a content type to identify and keep metadata for a site, the standard information management policies for content expiration can be configured to implement a recurring multistage retention policy for site disposition.

Create a site information content type and bind it to a list or library in your site definition, and ensure that this list contains one SiteInfo item with the metadata of the site. Typical metadata are site created date, site contact, site type, cost center, unit and department, is restricted site flag, last review date, next review date, and last update timestamp. Restrict edit permissions for this list to just site owners or admins.

Enable retention for the SiteInfo content type to configure your site lifecycle management policy as defined in your governance plan.

Add one or more retention stages for the SiteInfo content type as needed by your SLM policy. You will typically have a first stage that will start a workflow to notify the site owner of site expiration and ask for disposition confirmation. Make sure that the site owner knows about and enacts on your defined governance policies for manual information management, such as sending valuable documents to records management. Then there will be a second stage for performing the site disposition steps triggered by the confirmation.

You can also implement custom information management policy expiration formula or expiration action for use when configuring your retention policy. You typically do this when your policy requires retention events that are not based on date fields only. See Sahil Malik's Authoring custom expiration policies and actions in SharePoint 2007 which is still valid for SharePoint 2010.

Use a custom workflow or custom expiration action to implement the site disposition steps: user removal, automated content clean-up and archiving, and finally trigger deletion of the site. If the site is automatically deleted by a custom workflow, or marked for deletion to be processed by a custom timer job, or a custom action just sends an e-mail to the site-admin, is up to your SLM policy.

If you need to keep the site in a passive state for e.g. 6 months before deleting it, you can use a delegate control in your site master pages to prevent access to passive sites or you can move the site to an archive web-app that use a "deny write" / "deny all" access policy to prevent access. Note that the former is not real security, just content targeting for the site. The latter is real security, as "deny" web-app policies overrides site specific access rights granted to SharePoint groups and users. This allows for keeping the site users and groups "as-is" in case the site can be reactivated again according to your SLM policies. If site owners can do housekeeping on a site while passive, then grant them access by creating extra "steward" accounts that are not subject to being denied access.

I recommend removing all users from the default site members group before deleting the site, otherwise the site will not be deleted from the site memberships list in the user's my site.

The astute reader may wonder how the content type retention policy knows if the site is actually in use. The answer is quite simple; each SPWeb object provides a LastItemModifiedDate property. This timestamp is also stored in the SharePoint property bag. Use a delegate control in your site's master page to check and push the timestamp to a date-time field the SiteInfo item, so that the rentention policy can trigger on it. Remember to use SystemUpdate when updating the SiteInfo, otherwise you will change the site's LastItemModifiedDate to now. You can also use a custom expiration formula that inspects the last modified timestamp for the site when the information management policy timer job runs.

We also use the site information content type in our Puzzlepart projects to provide a search-driven site directory. It is quite simple to make a nicely categorized and searchable site catalog by simply using one or more customized the search results web-parts. This search-driven catalog can of course be sorted by the search result 'write' managed property, which must be mapped to the crawled property field that contains the LastItemModifiedDate of a site.

Using a search-driven approach makes it unnecessary to have a classic site directory list. The site metadata is simply stored directly in a list within each site, managed by the respective site owners. This is more likely to keep the site metadata up-to-date rather than going stale in a central site directory list that no one maintains.

I hope this post have given you some new ideas on how to store, manage and use site metadata both for site lifecycle management and for providing a relevant search-driven site directory.

Using SP2010 BCS Resource Files for BDC Model Settings

2011-04-29T09:05:00.011+02:00

You've probably seen way too many Business Connectivity Services (BCS) demos using SharePoint Designer (SPD), showing how simple it is to connect to a SQL Server 2008 database and automagically create external content types and operations, with external lists that can be used both to display and update external data.

Have you ever wondered how to manage those data source connection settings across multiple SharePoint 2010 farms? How do you change the SQL Server name and other login information when deploying to your test and staging farm, and then to your production farm without using SPD again? Even good BCS books such as Professional Business Connectivity Services in SharePoint 2010 have very little coverage of BCS external system settings in Central Admin and of how to actually use BDC resource files. There is a nice end-to-end overview in the Migrating Business Connectivity Services External Content Types in SharePoint 2010 article on MSDN, also lacking some of these details.

Data source connection settings are stored as External System properties that can be configured from Central Admin by managing the BDC service application. This can be a bit confusing, as you might run into the "there are no configurable properties" message when trying to manage Settings for an External System. The trick is to remember that these settings are not for the external system per se, but for a specific external system instance aka connection. Chose the External Systems view in the ribbon, click the link of the applicable external system to see the instances, then select the instance and finally click Settings in the ribbon. Change the connection properties and click OK.

In the example data connection I've used the Secure Store Service (SSS) application for the login information because the target database requires SQL Server authentication instead of passthrough integrated Windows authentication.

Instead of manually changing these External System settings whenever deploying a new version across your development, testing, staging and production farms, you can use BDC resource files to apply the settings. This is done by exporting and importing resource files, either using Central Admin, code or Powershell.

Prototype and test your BCS solution on your development farm first, then package your solution into a feature as explained in How to: Deploy a Declarative BDC Model with a Feature on MSDN. Remember to change the model name and external system name, including the entity namespace into durable names, or at least change the entity version, from your SPD prototype when packing your BDC model.

The declarative BDC model is really just some XML stored in a BDCM file. In addition to the BDC model file, SharePoint 2010 also supports using BDC resource files for specific metadata elements that commonly change, such as SQL Server connection configuration. A resource file is really just some XML stored in a BDCR file, that can be merged with the stored model without deleting the existing model and its configuration from the BCS metadata store.

The simplest way to get a BDCR file to start with, is to export the BDC resources using Central Admin. Chose the BDC Models view in the ribbon, then select the applicable model and finally click Export in the ribbon. Set the file type to Resource and select which resources to include in the exported BDCR file, typically properties, and click Export to save the selected set of resources.

Edit the the resource XML to change e.g. the name of the SQL Server (RdbConnection Data Source) and save your changes, using one file per BDC model and target farm. Only the XML for settings that should be updated when applying the resource file need be in the file.

The edited BDC resource file can now be applied to the applicable BDC model in one of your farms, typically when deploying a new version of a model to the staging or production farm. Chose the BDC Models view in the ribbon, then select the applicable model and finally click Import in the ribbon. Click Browse to select the applicable resource file and set the file type to Resource. Then select which resources to import from BDCR file, typically properties, and click Import to load the selected set of resources.

Validate that the correct settings were imported by reviewing the import log warnings, and by reviewing e.g. the settings for your external system instance or the permissions for the BDC model and its external content types.

See How to: Use a Resource File to Specify Localized Names, Properties, and Permissions on MSDN to include an edited BDC resource file in a feature in your Visual Studio 2010 package. Note that BDC models created with SPD cannot be exported from CA as they are not complete. Such declarative BDC models must be exported from SPD.

Using Dynamic Stored Procedures in BCS Finders

2011-04-28T09:24:00.006+02:00

We use quite a lot of Business Connectivity Services (BCS) at my current SharePoint 2010 project both for traditional integration of data from external systems into web-parts and lists, and also for crawling external systems for integrating those system using search and search-driven web-parts.

One of our integration partners prefers to provide their integration points as SQL Server 2008 stored procedures, which is very well supported by BCS. BCS supports both stored procedures and table valued functions, called "routines" in SharePoint Designer (SPD). SharePoint Designer is dependent on being able to extract metadata about the returned table data set when adding External Content Type (ECT) operations or when using the Operations Design View.

Alas, the provided integration sprocs used dynamic SQL statements, and for technical reasons this could not be rewritten to inline SQL select statements. This is as always a problem with tooling such as SPD, as no result set metadata can be discovered. When connecting SPD to the external system, I got no fields in the Data Source Elements panel in the Read List operation's Return Parameter Configuration. Rather I got three errors and a warning.

The workaround is quite simple and requires the use of a SQL Server table variable, which defines the result set and allows SPD to discover the table metadata. Rewrite the stored procedure by declaring a table variable, insert the result of the dynamic SQL statement into the variable, and finally return the result set by reading the table variable. The changes to the sproc is shown in blue in this example:

CREATE PROCEDURE [dbo].[GetFavorites]

AS

BEGIN

SET NOCOUNT ON;

DECLARE @DbName AS NVARCHAR(max) = 'ARISModellering1'

DECLARE @ObjDef AS NVARCHAR(max) = dbo.GetArisTableName(@DbName, 'ObjDef')

DECLARE @Model AS NVARCHAR(max) = dbo.GetArisTableName(@DbName, 'Model')

DECLARE @FavoriteSQL AS NVARCHAR(max) = N'

SELECT

ARISBPDATA.BOOKMARKS.ID AS FavoriteId

, ARISBPDATA.BOOKMARKS.DESCRIPTION AS FavoriteName

, ModelType.ModelTypeName AS FavoriteType

, LOWER(ARISBPDATA.BOOKMARKS.USERNAME) AS UserName

, ''http://puzzlepart/index.jsp?ExportName=ARISModellering&modelGUID=''

+ ARISBPDATA.BOOKMARKS.DATAKEY AS FavoriteUrl

FROM

ARISBPDATA.BOOKMARKS

INNER JOIN ' + @Model + ' m ON ARISBPDATA.BOOKMARKS.OBJID = m.Id

INNER JOIN ModelType ON m.TypeNum = ModelType.ModelTypeId

DECLARE @userFavs TABLE(FavoriteId int not null, FavoriteName nvarchar(max),

FavoriteType nvarchar(max), UserName nvarchar(max), FavoriteUrl nvarchar(max))

insert @userFavs EXEC sp_executeSQL @FavoriteSQL

select * from @userFavs

END

Refreshing the external system data connection and then creating the ECT read list operation now works fine, and all the return type errors and warnings are gone.

Note that the classic #temp table workaround won't work with SPD, you have to use a table variable in your stored procedure. The sproc will now use more memory, so the BCS best practice for keeping finder result sets small applies.

The table_variable declaration is also a good place to make sure that the identifier column is "not null" and that it is a supported BCS data type such as "int32". External lists cannot be created from an ECT whose identifier field is unsupported, such as SQL Server "bigint", and I strongly recommend using a supported identifier data type right from the start. Getting the ECT identifier wrong in the BCS model will give you problems later on when using SharePoint Designer.

BCS External Lists causes SharePoint 0x80131600 exception for SPSiteDataQuery

2011-04-20T15:09:00.000+02:00

Issue: Your SharePoint 2010 code use the SPSiteDataQuery or CrossListQuery and get an exception with code <nativehr> 0x80131600 and absolutely no other helpful details.

Cause: External list referencing an external content type that has been deleted from the BDC metadata store.

Solution: Delete the applicable external lists from your sites - or recover the deleted external content type.

A related error code is code 0x8102003 which is caused by missing list definitions in activated features.

Installing SharePoint SPSF on Visual Studio 2010 SP1

2011-04-01T09:34:00.006+02:00

In preparing for the 2011 Arctic SharePoint Challenge next week with my awesome Puzzlepart team, we're installing the SharePoint Software Factory (SPSF) tooling available at CodePlex. There is a nice prerequisite installer that helps you download and install the Guidance Automation Extensions and Toolkit, but it would not install the guidance packages properly on my Visual Studio 2010 SP1.

Installing GAT2010.vsix failed with missing "Visual Studio 2010 SDK", so I downloaded that and got the "you must have Visual Studio 2010 installed" error instead. As it turns out, you will of course need "Visual Studio 2010 SP1 SDK" (download).

Now I've got the SPSF project types in the Guidance Packages section of File > New Project and are ready for #ASC2011.

SharePoint News Feed Formatting of ActivityEvent

2011-03-15T15:28:00.006+01:00

It is quite easy the get the news feed for activities from your colleagues and for your interests and skills in SharePoint 2010. It is not, however, that simple to format each event to display them in your own web-part using the activity feed object model.

Sure, the data of the different activity types are all there in the ActivityEvent object, and you can get the ActivityTemplate based on the ActivityType of the event. But then you need to process the display template tags to merge in the event values or the event XML from TemplateVariable string property using the SimpleTemplateFormat and ActivityTemplateVariable classes. See the Fun and Games with the ActivityEvent post by Toby Statham to get you started.

Luckily, the activity feed is based on the web syndication model, so you can simply create a SyndicationItem object based on the activity event, and it will find and process the activity template for you:

private Panel CreateFeedEventPanel(ActivityEvent activity)

    Panel panel = new Panel()

CssClass = "MyProfileActivityFeedEventPanel"

};

    //access the LinkList property in order to populate the ActivityEvent

    List<Link> temp = activity.LinksList;

    string picture = activity.Publisher.Picture;

    picture = string.IsNullOrEmpty(picture) ? "/_layouts/images/O14_person_placeHolder_32.png" : picture;

    Image publisherImage = new Image()

ImageUrl = picture,

AlternateText = activity.Publisher.Name,

CssClass = "MyProfileActivityFeedEventPublisher"

};

    panel.Controls.Add(publisherImage);

    SyndicationItem syndicationItem = activity.CreateSyndicationItem(_activityManager.ActivityTypes, ContentType.Html);

    panel.Controls.Add(new LiteralControl() { Text = syndicationItem.Summary.Text });

    return panel;

private void PopulateNewsFeedActivityList(bool useTodayOnly)

    string url = SPContext.Current.Site.Url;

    using (SPSite site = new SPSite(url))

SPServiceContext context = SPServiceContext.GetContext(site);

UserProfileManager profileManager = new UserProfileManager(context);

SPUser user = SPContext.Current.Web.CurrentUser;

UserProfile userProfile = profileManager.GetUserProfile(user.LoginName);

_activityManager = new ActivityManager(userProfile, context);

if (useTodayOnly)

    DateTime todayFilter = DateTime.Now.Date;

    _activityList = _activityManager.GetActivitiesForMe(todayFilter);

else

    _activityList = _activityManager.GetActivitiesForMe(MaxItems);

The formatted HTML will be the same as rendered by the NewsFeedWebPartBase class, except for the profile picture size and some missing timestamps for some event types. Use Reflector on the news feed web-part base class to see the code for mitigating such details.

The code for getting activity events for a user and other SharePoint 2010 social computing "how-tos" can be found in the User Profiles and Social Data section at MSDN.

Help Content Cannot Be Displayed in SharePoint 2010

2011-02-23T14:03:00.004+01:00

Today we had a weird error on our SharePoint 2010 production farm: clicking on help got the "help content cannot be displayed" error for all normal sites, even though it worked perfectly well in Central Admin. The same applied to Site Settings>Help Settings for the site-collection, it worked in Central Admin, but not in any other site. In addition, the 'SharePoint Foundation Search' service was running on one WFE server.

First I checked all settings in KB939313 without that fixing the problem, then I checked the log files and found this access denied error for our site's app-pool account:

SqlError: 'The EXECUTE permission was denied on the object 'proc_EnumResourcesAtScope', database 'SharePoint_AdminContent_ABBAef34-7603-4da5-823a-43ee1327ABBA', schema 'dbo'.'

Before embarking on changing any database rights, we decided to test with an English site just in case, as all our custom site definitions are in Norwegian. Lo and behold - help worked for the new team-site; and what's more, suddenly help was working for all our existing Norwegian LCID 1044 sites also. Go figure...

[UPDATE] See the comments for tips on granting execute rights on the sprocs listed in the ULS to fix this problem once and for all - even beyond IISRESET.

Starting Term Store Management in SharePoint 2010

2011-02-21T10:28:00.005+01:00

If you can't get any edit or management popup menus such as add term store term group to show in the SharePoint 2010 Managed Metadata Service application Term Store Management Tool, check that:

Internet Explorer is started with "Run as administrator"
You have added your taxonomy managers to the "Term Store Administrators" for the MMS root node

This is required even if you are an administrator of the MMS application itself and you have full control MMS connection permissions.

So then you're ready to realize your ingenious taxonomy for classifying and organizing your knowledge with managed metadata and content types.