System.What?

Troubleshooting SharePoint 2010 Search

2012-06-17T12:08:00.001-07:00

You’ll Need a Logger For That

Although it’s a killer feature, Search is as awkward as ever to troubleshoot in SharePoint 2010. The Event Viewer chatter is pretty meek for troubleshooting, and the Search Center crawler reports have couple minute lag which makes them awkward to work with. The ULS logs can be great, but some troubleshoots produce so much data it’s hard to get a feel for everything that’s happening reading static flat files.

Case in point, other users on the web have gone so far as to set up fiddler as a reverse proxy, just to get a feel for what’s happening to with Search as it goes about its crawling business (it’s cool to watch btw).

Enter the aptly named real time ULS log viewer “SharePoint LogViewer”.

It’s the most sane conversation you’ll ever have with this service. You’ll be able to see everything from the callbacks from the administration site to the noise the crawler itself makes, all in real time.

You should recognize it all:

Listing Content Sources
Authenticating with the individual web applications
Web Service calls to actually get list items
And a lot of other minutia you might be interested in…

What’s more, you can set filters on the chatter to help spot pertinent info (see figure 1). It’s usually enough just to start the logger, begin real time logging (takes a couple seconds to connect to farm), and setting a filter (say “Search”) to get all the info you need to start discerning what’s going on with one of your search crawls.

Figure 1: SharePoint ULS mid crawl

It’s Great When it Works

Search is one of the coolest features that comes with SharePoint. Its ability to index a wide array of files, provide security trimmed results, and scale to huge 100 million item index sets (just to name a few) have help set it apart from other search alternatives (like Full Text Search, Lucene, Google Minis, etc…). If you have buckets of money, there’s also a great roadmap into FAST Search which has it’s own impressive additions to help you support even more robust search needs.

As the search space has matured, the SP 2010 Search itself has had to grow to keep up. For developer to continue to be able to troubleshoot/understand how these services are working, it’s also likely that your tool belt need to grow as well. A good real time log viewer is sure to make a great addition.

My Best,
Tyler

Extending the FedAuth / Claims Auth Ticket in SharePoint 2010

2012-04-16T04:03:00.001-07:00

It’s Not Long Enough

For those whom have serviced calls about authentication in SharePoint 2010, a portion of your last two years may have been spent learning the broad range of changes that have come with the Federated and Claims Based models that are now part of authentication in SharePoint 2010.
One common scenario, especially for Forms Based Authentication sites (implemented with CBA) is how to extend the login window.

The Default – 10 Hours?

Once you switch (or initially create) a web application over to CBA and implement a Forms Based Authentication provider you might start to notice that the cookie window has a peculiar lifetime…it’s 10 hours for both Windows and Forms Based Authentication.
It’s worth noting that until you implement the FBA the cookie’s still behave similar to those in Classic authentication (expire at end of session).
The cookie is relatively easy to find (seen here with the Firefox Add-in Cookies Manager+).

Figure 1: Authentication Ticket from FBA Login

How to Extend It

It’s very likely there’s someone in your organization that either doesn’t like logging in, or isn’t very good at it. For these genre of users the new FBA behavior isn’t a boon and it’s only a matter of time before the conversation of extending the login windows comes up.
Strangely enough there’s not a ton of straightforward documentation (partly the reason for this post) or options to remedy this.

One of the easiest ways to extend this is by altering the configuration of the Secure Token Service with PowerShell. To be fair, it has a pretty big impact (farm wide) and introduces some other concerns. One of these is the fact that the user won’t be tasked with authenticating for the duration of the new extended window. While this sounds obvious (that’s the point right?) the fact that users are authenticating less often means that other actions (say in Active Directory or at application levels) like invalidating users or hooking/processing events at authentication time won’t be firing.

If you extend the window to say 30 days, it’s possible that users that you’re marking invalid may have access to systems who don’t check said flag (and focus simply on authorization…I mean that’s partly the raison d'être for CBA) for quite a bit longer than if you’d left it alone.

For sites that we’ve been asked to do this, we usually implement an HttpModule to inspect the user for any properties that are normally screened at authentication time so that we know we’re dealing with a valid user. So far this has seemed to mitigate the effects.

Ok, enough talk. You can see the existing farm level settings for the Secure Token Service with the following PowerShell command:

Get-SPSecurityTokenServiceConfig

That should yield the following output, notice that the Forms and Windows Token Lifetime are individual settings (both initially 10 hours) and can be set independently.

Figure 2: Secure Token Service Settings

You can change these settings by running the following commands for the Windows and Forms token lifetimes respectively:

Set-SPSecurityTokenServiceConfig –WindowsTokenLifetime [minutes]

Set-SPSecurityTokenServiceConfig –FormsTokenLifetime [minutes]

This is of course done with PowerShell, I haven’t seen a way to do so with the web.config for the service.
Once making the changes, they begin immediately (no need to recycle app pools or web servers). Your cookie should look like below (set to 30 hours as an example).

Figure 3: Extended FedAuth Cookie

I know it’s not ideal (has huge impact and raises other considerations), but it IS the most straightforward way of extending the cookie, and if it’s available, you should at least know about it.
Hope that helps,
Tyler

No SharePoint 2007 Menu Flyouts/DropDowns in Chrome

2012-03-07T22:46:00.002-08:00

Broken Navigation Menus

As of Chrome 15 (we think) SharePoint 2007 sites started to behave a little funky. The stock SharePoint menu started to behave erratically, intermittently not showing navigation flyouts. To be clear, on every other page load the navigation would behave appropriately, but the other half of the time the menu wouldn’t do anything on hover, making it impossible to see sub navigation items under a given header.
This seems to affect all SP 2007 sites for Chrome version 15 and later. The error strangely enough seems to be more related to javascript than rendering behavior.

The Fix

I’ll save you the troubleshoot, we added a late loading script that tells the menu that flyouts are indeed kosher. For some reason this variable gets set incorrectly for newer versions of Chrome. There’s a good chance MS may not release a fix for this since Chrome isn’t a supported browser for SharePoint 2007.

<script type="text/javascript">
$(function(){
    //Updates the stock SharePoint menu settings to allow flyouts,
    //seems to get intermittently improperly set on Chrome >= 15.
    flyoutsAllowed = true;
});
</script>

That’s it, pretty simple (especially if you already have some jQuery running around). Otherwise you can get by having the script run really late in the document life cycle.

Hope that Helps Someone,
Tyler

How SharePoint Deals With Time And Time Zones

2012-02-29T23:22:00.000-08:00

What’s Going On Here?

It’s a little weird how hard it is to find some detailed descriptions on how SharePoint handles time, and time zones. So here’s a dump on a lot of things time and time zone related to better help you troubleshoot and predict behavior.

How Time Is Stored

Fields like created/modified are stored will all other list item properties in the dbo.AllUserData table as tp_Modified and tp_Created. Like all list items, events are stored in this table as well.

Time Calculation With Respect to Time Zones

Time gets stored using the clock of the machine, this is not influenced by the time zone of the machine. If the clock on the server is off, the time stored is off by the amount of the clock skew. Changing the time zone of the server (OS) doesn’t affect how time is stored. This is true for most applications.

WHY?

If a server is at 6pm PST (Pacific) and you change the time zone to EST (Eastern) sure the time looks like it changes from 6pm PST to 9pm EST, but those times are equivalent. 9pm EST is just another way of writing 6pm PST. Events taking place in California are really happening at the exact same time in New York, their citizens just happen to format time differently. Time zones are just another way of expressing (formatting) the current time. To be clear. There is only one time, there are many time formats…and there is no spoon.

The Clock IS Important Though

If you change the time on the server (OS) clock you’ll change how time will get stored (for new edits/additions). In this case you’ve actually had an impact on time. Namely because going from 6pm PST to 7pm PST means someone actually lost an hour they’ll never get back (not that fake ones you tell yourself about when you travel through different time zones). By changing the clock from 6pm PST to 7pm PST, you’ve effectively moved the server through time. As a result, this will effect the times that items are recorded to be stored at.

How Time Zones Get Set

By default all your users see the time as it is formatted at the root site (SPWeb) level. These settings get set when the site collection is provisioned by the web application default time zone (Figure 1 shown below). These settings get copied down when new sub sites are provisioned under the root site. Finally, if any user doesn’t like the way dates and times are displayed, they can override how time is presented to them using their personal regional settings (Figure 2 also below). But most of the time its the root site settings time zone that users are complaining about.

Default Site (SPWeb) Time Zones

This setting dictates the formatting/time zone for all data in the site. By default these get copied down from the parent web at time of creation. You can view/change these settings (if you’re a Site Administrator) by going to:

Site Actions->Site Settings
Click on Regional Settings

Figure 1. Site (SPWeb) Regional Settings

This setting is dictating how time is displayed to all users whom haven’t expressed a preference using their personal regional settings. At the root of the site collection (root site), you can also tell all sub sites to inherit the settings of the root web, resetting them to a single time zone.
Changing the Regional Settings time zone for a web programmatically looks like the code below. Time zone IDs come from a list of known time zones.

using (SPSite siteCollection = new SPSite("http://localhost"))

    using(SPWeb websiteRoot = siteCollection.RootWeb)

        SPRegionalSettings regionalSettings = websiteRoot.RegionalSettings;

        SPTimeZone timeZoneIndianaEast = regionalSettings.TimeZones[15];

        regionalSettings.TimeZone.ID = timeZoneIndianaEast.ID;

        websiteRoot.Update();

User Site Collection Time Zones

We say user site collection time zones settings because even though this is stored per user, these settings reset in across site collections. Just because you’ve set this in site collection A doesn’t have any impact on site collection B. It will follow you around across many sites (SPWeb) within a site collection though.

You can override the default site time zone by going to:

Click on your username in the header and click on My Settings.
Click on My Regional Settings.
Uncheck Always follow web settings and override the time zone with one of your preference for this site collection.

Figure 2. User Site Collection Time Zone settings
overriding the default time zone

After you click OK, time will be formatted for the time zone you dictate.
You can change this for a given user with the following code:

//Create a new regional settings based off of the current web.

SPRegionalSettings regionalSettings = new SPRegionalSettings(web, true);

//Set some settings

SPTimeZone timeZoneIndiaEast = regionalSettings.TimeZones[15];

regionalSettings.TimeZone.ID = timeZoneIndiaEast.ID;

//Assign to user and update.

user.RegionalSettings = regionalSettings;

user.Update();

Fallout

There are some interesting effects of the time zone settings. For instance, if you ship a site collection to another time zone, it will still behave as if it’s in your original time zone with no difference at all so long as the clock at the host is the correct time.

Different users can have very different time settings across different site collections. The good news is that they get the final say on how time is formatted.

Both these settings can be manipulated programmatically. When these settings are set appropriately they can help users from very different time zones collaborate on data with time formatted in a way that make sense to them. This especially applies to event/calendar items.

Hope that helps someone, regional settings can have a big impact on collaboration projects.

My Best,
Tyler

Troubleshooting The Windows SMTP Server

2012-02-12T19:07:00.000-08:00

Why Isn’t My E-mail Getting There?

No one gets a hi-five for implementing an email feature. I’ve tried it before, it’s impressively devoid of gratitude.
Email is one of those features that’s just supposed to work and very little thought is often given to how difficult it really is to get right. Even if users didn’t expect it to be instantaneous and scale to legendary sizes, they definitely wouldn’t hand out accolades for all the troubleshooting that email brings to your door.
Email is as complicated as ever, and requires a ton of different systems to be in sync just for a single message to get through. Components like:

Application Settings
SMTP Server auth/relay settings
DNS (appropriate MX, A, and potentially PTR and SPF) both internally and externally
Email client and Server server spam filtering (text processing, attachment blocking, black listing, etc…)

…just to name a few, often present issues and ensuring their correct often crosses multiple disciplines and sometimes organizations. The broad domain of knowledge required to diagnose email issues can often cause their troubleshoots to be prolonged.

Troubleshooting

Fortunately most troubleshoots usually fall into one of three categories:

Application/SMTP Server Issues – These are problems between the application and the SMTP server, usually there’s a stack trace or some kind of very visible error code thrown for the application or in the Event Viewer/SMTP Log files. These are for things like:
- Unable to authenticate
- Unable to relay
- Email is invalid (one of its fields breaks the RFC)
- etc…there’s a bunch of them, but they they’re usually pretty easy to troubleshoot.
Sending SMTP/The Internet/Destination SMTP – This is where things get a little bananas. A lot of SMTP server’s aren’t great at logging (ahem, all versions of IIS to date), and it’s hard to get feedback on where in all things are going sideways.

This is also where a lot of developers start to feel out of their element. Checking DNS records and relating them to email behavior has traditionally been reserved as a core IT task. This is where tools like SMTPDiag save the day. Weighing in at 79 KB this tool can be a life savor that stops you from needing to memorize telnet “ehlo” commands and other awkward SMTP command syntax.
SMTP Diag runs a variety of tests, like DNS lookups and then goes through the motions of sending (doesn’t actually send) an email for every server that has an MX record. This usually tests:

DNS – Both sending and receiving domains have MX records, have A records.
Reachability – Both the sending and receiving domains’ email servers are accessible.
Accessible - Connects to SMTP servers and goes through the motions of sending an email from/to the given addresses.

Here’s an example of it running in verbose mode:

SmtpDiag.exe “fromAddress@hotmail.com” “toAddress@someDomain.com” /v

Just today the utility helped me realize that some emails stuck in my Windows 2008 SMTP server queue were really stuck there not because my SMTP server was set up incorrectly, but because my ISP was stopping outgoing port 25 traffic. My only regret is not running the thing earlier in my troubleshoot (I started at my application and SMTP server).

3. Assuming both of the above check out, then you most likely have something on the user’s email client or on the destination email server (read spam filtering). Which also means it’s officially not your fault.

I hope the above helps someone, these troubleshoots can often be a nightmare and there’s nothing like a good tool to help you wake up.
Good Luck,
Tyler

401 SharePoint Designer Connectivity Issues

2012-01-11T21:53:00.001-08:00

How Many Times Must I Log In!?

SharePoint Designer 2010 is definitely a huge improvement over 2007. It’s free and has a ton of enhancements to make customizing a SharePoint install easier than ever.
It IS however, just as difficult to troubleshoot as the previous version when things go wrong. This is mainly because almost everything SharePoint Designer 2010 does happens over a web service. This means that a lot of the errors that you tend to get back (if you get any at all) are cryptic, and troubleshoots can be prolonged.
The other day one of our developers was trying to do some edits to a local SharePoint 2010 dev farm via SharePoint Designer 2010 and was having some issues logging in.
The user was a site collection administrator and every time they entered a correct credential, they were prompted to enter it in again (and again).

In addition when running Fiddler to inspect the login traffic, the log was full of authentication attempts to the login service (/_vti_bin/shtml.dll/_vti_rpc) but all returned 401 http status codes (unauthorized). There were no relevant events in the Event Log or in the SharePoint logs (14 hive\Logs directory).

Normally (if accessing it via a host header), we’d assume this has to do with IIS loopback checks, but this wasn’t the case. After a lengthily troubleshoot we found out that the behavior actually had to do with a change we’d made to the web.config.
As it turns out, when you enable ASP.NET tracing, you’ll no longer be able to login via SPD. We had the following line in our web.config.

After we removed this line, we were able to login. We’re still digging around trying to discern how the two are related, but in the interim I’m hoping this saves someone out there some time.
HTH Someone,
Tyler

Being Ignored By the SQL Membership Provider

2011-11-30T23:56:00.000-08:00

SQL Membership Provider in SharePoint 2010

Those of you who have worked on SharePoint 2010 sites that use Forms Based Authentication (under the umbrella of Claims Based Authentication), won’t act surprised when I start talking about the SQL Membership Provider. It’s one of the most common providers that gets used in SharePoint 2010 FBA sites. The fact that it’s a free, very flexible, and pretty straight forward way to set up FBA in SP 2010 have helped spur its popularity.
Those who have also worked with the SQL Membership Provider in environments where claims based authentication wasn’t in play, may very well have noticed some changes in its behavior. While the code is of course the same, the additional complexity of a claims environment, means that some properties may not behave as they normally would.

The Computer Isn’t Listening to Me

On two separate occasions now we’ve had developers set properties on the SQL Membership Provider in the web.config expecting it to change the behavior of their SP 2010 site. In both these cases a lack of understanding in how claims authentication works has left them scratching their heads when the properties didn’t take effect.
An example of this might be:

Developer wants to change one of the authentication options of the membership provider (say increase the MaxInvalidPasswordAttempts property), and does so in the web.config of the chosen web application.
After the change to the provider settings, the developer notices that the web application behaves no differently than before.
After a series of cuss words and IIS resets, developer starts aimlessly searching the internet (or an empty bottle) for the answer.

It’s important to remember that once claims based authentication is introduced, the web application doesn’t perform any of the authentication anymore. Even though those settings for the provider in the web.config speak to authentication behavior, in Sharepoint 2010 claims auth scenarios the actual authentication is most often performed by the SharePoint Secure Token Service.
Those web.config settings (in the web application) that don’t speak to authentication are still fair game and will behave as expected, but the properties that change auth behavior like MaxInvalidPasswordAttempts or PasswordAttemptWindow won’t have any effect (unless their made in the Secure Token Service web.config).
To be clear, you need to make those changes to the web.config of the STS, normally hosed at: C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\WebServices\SecurityToken\web.config.
In most scenarios, the authentication exchange looks like the picture below (with steps 3-8 being performed by the STS).

Once the relationship between the user, the STS, and the web application become clear the reasoning behind this behavior starts to make a lot more sense.
Since knowing is half the battle, it’s probably worth keeping an eye on claims based authentication technologies and how they behave. With how many systems are now using shared/claims auth both in and outside the enterprise, these are troubleshoots that are going to become increasingly common in future years.
My Best,
Tyler

SharePoint People Picker Filters Strike Back

2011-10-18T21:28:00.000-07:00

Yay, Free Integration

One of the benefits of rolling out SharePoint in intranet environments is that it marries pretty well with Active Directory (and other LDAPs). An example of this is the People Picker. It allows you resolve (or look up) any user in the LDAP that you’re authenticating against.

Since LDAPs often get pretty dated/polluted there’s often a need to filter out certain sections of the LDAP or a certain genre of results. These might be old employees who have left the company, or users from a section of the organization that just aren’t relevant to the web application at hand.

An example of a filter at the Organizational Unit is below. The following filter ensures that the people pickers in the given web application only pull users from the Employees organizational unit (OU).

stsadm -o setsiteuseraccountdirectorypath -url http://webApplicationUrl -path "OU=Employees,DC=fullly,DC=quallified,DC=domain,DC=com"

Should you change your mind later, the following removes the filter above. In general, no application pool recycle or IISReset is required.

stsadm -o setsiteuseraccountdirectorypath -url http://webApplicationUrl -path ""

It’s worth mentioning that these filters don’t affect existing accounts in the site collection that have already been added. They only stop future users additions that would collide with the give filter.

There are some pretty expressive filters available out there. Many can target specific LDAP properties like title, name, organization, etc… or other fields. Examples can be found at:

The Resulting Errors

The problem with these filters is that its easy to forget about them and remember that they’re still in play. Specifically when adding users to a group, administrators/site owners are sometimes greeted with:

The user does not exist or is not unique. at Microsoft.SharePoint.Library.SPRequestInternalClass.UpdateMembers(String bstrUrl, Guid& pguidScopeId, Int32 lGroupID, Int32 lGroupOwnerId, Object& pvarArrayAdd, Object& pvarArrayAddIds, Object& pvarArrayLoginsRemove, Object& pvarArrayIdsRemove, Boolean bSendEmail) at Microsoft.SharePoint.Library.SPRequest.UpdateMembers(String bstrUrl, Guid& pguidScopeId, Int32 lGroupID, Int32 lGroupOwnerId, Object& pvarArrayAdd, Object& pvarArrayAddIds, Object& pvarArrayLoginsRemove, Object& pvarArrayIdsRemove, Boolean bSendEmail)

Which is an awfully confusing error. When you get the above, make sure that there aren’t any filters in play that would normally preclude the given user from resolving as they can cause this error.

You can check the existence of a directory path filter by running the following stsadm command:

stsadm -o getsiteuseraccountdirectorypath -url https://webApplicationUrl

If there are, remove them first and then try again. This is a step that should often be done before getting too far into a troubleshoot.

Hope that helps.

My Best,
Tyler

Customizing SharePoint 2007 Welcome and Site Actions Menus

2011-10-07T15:16:00.001-07:00

Can We Trim That Down?

A good portion of a SharePoint branding implementation is discerning which out of the box features are working and those that need to be customized. Especially in branding engagements where we’re trying to write as little code as possible, being able to provide customizations w/out opening Visual Studio is a huge win.
Below are two very common modifications that can be done to both the Site Actions and the Welcome menu to restrict the options that appear.

Removing Options from the Site Actions Menu

There are two ways to declare a SiteActions menu, the very terse:

<SharePoint:SiteActions runat="server" />

Or the more verbose style like that found in the default.master:

<SharePoint:SiteActions runat="server"

AccessKey="<%$Resources:wss,tb_SiteActions_AK%>" id="SiteActionsMenuMain"
PrefixHtml="&lt;div&gt;&lt;div&gt;"
SuffixHtml="&lt;/div&gt;&lt;/div&gt;"
MenuNotVisibleHtml="&amp;nbsp;">
<CustomTemplate>
<SharePoint:FeatureMenuTemplate runat="server"
FeatureScope="Site"
Location="Microsoft.SharePoint.StandardMenu"
GroupId="SiteActions"
UseShortId="true">
<SharePoint:MenuItemTemplate runat="server" id="MenuItem_Create"                    
Text="<%$Resources:wss,viewlsts_pagetitle_create%>"
Description="<%$Resources:wss,siteactions_createdescription%>"
ImageUrl="/_layouts/images/Actionscreate.gif"
MenuGroupId="100"
Sequence="100"
UseShortId="true"
ClientOnClickNavigateUrl="~site/_layouts/create.aspx"
PermissionsString="ManageLists, ManageSubwebs"
PermissionMode="Any" />
<SharePoint:MenuItemTemplate runat="server" id="MenuItem_EditPage"
Text="<%$Resources:wss,siteactions_editpage%>"
Description="<%$Resources:wss,siteactions_editpagedescription%>"
ImageUrl="/_layouts/images/ActionsEditPage.gif"
MenuGroupId="100"
Sequence="200"
ClientOnClickNavigateUrl="javascript:MSOLayout_ChangeLayoutMode(false);"/>
<SharePoint:MenuItemTemplate runat="server" id="MenuItem_Settings"
Text="<%$Resources:wss,settings_pagetitle%>"
Description="<%$Resources:wss,siteactions_sitesettingsdescription%>"
ImageUrl="/_layouts/images/ActionsSettings.gif"
MenuGroupId="100"
Sequence="300"
UseShortId="true"
ClientOnClickNavigateUrl="~site/_layouts/settings.aspx"
PermissionsString="EnumeratePermissions,ManageWeb,ManageSubwebs,AddAndCustomizePages,ApplyThemeAndBorder,ManageAlerts,ManageLists,ViewUsageData"
PermissionMode="Any" />
</SharePoint:FeatureMenuTemplate>
</CustomTemplate>
</SharePoint:SiteActions>

Although there’s some testing that needs to be done, you can for the most part use either the PermissionsString/PermissionMode properties to conditionally hide menu items OR you can remove them in entire (by simply removing the menu item markup [above]).

Even when we start to work with publishing sites, it’s worth mentioning that the PublishingSiteAction menu (like that found in blueband.master) is really just a control template which pretty much does the above but adds additional controls to the menu. You can find said control at:

C:\Program Files\Common Files\microsoft shared\Web Server Extensions\12\TEMPLATE\CONTROLTEMPLATES\PublishingActionMenu.ascx.

See something you don’t like? Change it. Its worth mentioning that we typically don’t recommend altering any of the controls in the ControlTemplates folder themselves (since it has farm wide impact). If you can, either make your changes in the masterpage (using selective declarative markup like the above), or create a copy of an existing control, make your alterations in the copy and reference it from your master page.

Either way, keep your footprint as small as possible.

Removing Options from the Welcome Menu

The welcome menu works in pretty much the same way. Typically it’s declared looking like:

<%@ Register TagPrefix="wssuc" TagName="Welcome" src="~/_controltemplates/Welcome.ascx" %>

...
<wssuc:Welcome id="IdWelcome" runat="server">

Notice that similar to the PublishingSiteAction menu (above) it simply references a control template found in:

C:\Program Files\Common Files\microsoft shared\Web Server Extensions\12\TEMPLATE\CONTROLTEMPLATES\Welcome.ascx

The easiest way to work with a reduced version of the welcome menu is to replace your existing welcome menu control (above) with the actual control markup/template, and then selectively remove what you don’t want (and test). An example of such a trimmed control (which only shows the “Sign In as a Different User” and “Logout” options) would look like:

<SharePoint:PersonalActions AccessKey="<%$Resources:wss,personalactions_menu_ak%>" ToolTip="<%$Resources:wss,open_menu%>" runat="server" id="ExplicitLogout">

<CustomTemplate>
<SharePoint:FeatureMenuTemplate runat="server"
FeatureScope="Site"
Location="Microsoft.SharePoint.StandardMenu"
GroupId="PersonalActions"
id="ID_PersonalActionMenu"
UseShortId="true">
<SharePoint:MenuItemTemplate runat="server" id="ID_LoginAsDifferentUser"
Text="<%$Resources:wss,personalactions_loginasdifferentuser%>"
Description="<%$Resources:wss,personalactions_loginasdifferentuserdescription%>"
MenuGroupId="200"
Sequence="100"
UseShortId="true"/>
<SharePoint:MenuItemTemplate runat="server" id="ID_Logout"
Text="<%$Resources:wss,personalactions_logout%>"
Description="<%$Resources:wss,personalactions_logoutdescription%>"
MenuGroupId="200"
Sequence="300"
UseShortId="true"/>
</SharePoint:FeatureMenuTemplate>
</CustomTemplate>
</SharePoint:PersonalActions>

And voila, this:

Becomes this:

This can sometimes be super helpful, not only can you selectively pick links (and still hide the whole control with a <Sharepoint:SPSecurityTrimmedControl/> if need be), but you also don’t have to author a feature to change the menu or start messing with controls in the ControlTemplates folder (which can have a farm wide impact).

Hope this helps someone.

My Best,
Tyler

SharePoint 2007’s Most Common and Confusing Error

2011-09-19T20:48:00.001-07:00

That Seems Misleading

Today I ran into an error that I was positive I’d dealt with before. When we tried to visit any System pages (those that use the default.master) in a SharePoint 2007 application, we got the following error.

The DataSourceID of 'TopNavigationMenu' must be the ID of a control of type IHierarchicalDataSource. A control with ID 'topSiteMap' could not be found.

More formally, it looked like this:

When we tried to load other pages (where we were loading custom user controls from disk) we got another misleading error.

An error occurred during the parsing of a resource required to service this request. Please review the following specific parse error details and modify your source file appropriately.

Believe it or not, this is one of SharePoint 2007’s most well known red herrings. I post this in part because:

A lot of the existing content on the web doesn’t fully document this error and how to troubleshoot it (misleading).
I’ve run into it enough times (and forgotten the solution) that I feel a need to document it somewhere that I can easily look it up.

The Solution

The first thing to know about this error is that it gets thrown for a wide variety of reasons. While most often it has something to do with the web.config of a SharePoint 2007 web application, the root cause can get pretty tangential to a simple web.config parser error.

The first step to troubleshooting this, is to get an error message that actually speaks to the error at hand (the two above don’t, so don’t try to troubleshoot off of them). This genre of error seldom puts helpful details into the event log and so you should:

Open up the latest log file in the 12 hive (C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\Logs).
I usually go to the end of the file and search up for terms like “error” or “exception”.
You should be able to find some error. These can be quite varied from type/assembly resolution errors to parsing in the web.config.
From here troubleshoots typically go one of two routes.
- Either the error itself will inform you of what to do (e.g. a file is missing somewhere).
- You still have no idea and need to manually merge back in a working web.config.
If you can’t figure out what to do based on the error, 99% of the time you can repair the error by merging in (use a diff tool, don’t blindly stomp) a stock SharePoint web.config. To do this…
Backup your failing web.config.
Create a blank web application in some test farm.
Diff in (we use Beyond Compare) the newly provisioned stock web.config into your failing one (preferably line by line to determine root cause).
After you bring a line/section over feel free to save the file and refresh your browser. Every time you change the web.config (and save) the application pool should recycle, so don’t worry about needing to reset IIS.

This “baby steps” process has always yielded a quick troubleshoot down to root cause for us.

Troubleshooting SharePoint errors is often the beginning of a frustrating afternoon, so hopefully this helps someone out there.

My Best,
Tyler

Convenient, Yet Confusing Elements of Windows Integrated Authentication

2011-08-23T23:45:00.001-07:00

Windows Integrated Authentication

If you’ve written an application the intranet, you’ve likely used Windows Integrated Authentication at some point. When it works, it’s pretty slick. Not only is it flexible in the authentication protocols you can use, but users (if set up right) don’t even need to enter in their username/password.
Windows Integrated Authentication can use any of NTLM, Kerberos, Negotiate, or Digest. So if you’re picky about your security protocols (performance, client availability, hack-ability) there’s probably something in that list that will float your boat.
For those that haven’t had the pleasure yet, WIA is a way for users to authenticate (establish an identity) with an application in a Windows environment. The identity they’re usually establishing is one from from Active Directory or a local Windows credential.

Why’s It Useful

As it turns out, people don’t like logging in. Most users don’t even like typing in general, and the rest can’t be bothered to remember their password. When WIA works well, the user doesn’t do anything. They simply access a resource, and in the background a challenge/response ballet establishes their identity. All they see is a page that knows who they are treats them accordingly.
The equivalent of this happening to you in real life would be people just (seemingly) knowing who you are and treating you accordingly. If you were to arrive at a fancy restaurant (and you had reservations) you could just show up and be led to you table (no need to check in).
On the other side of things, if you showed up somewhere you weren’t supposed to be (no permissions) they’d immediately throw you into the street. The moral of the story here; regardless of how you get treated, you don’t need to type.

A Day In The Life

Here’s a look into what some WIA challenge/responses looks like in against a SharePoint 2007 site using NTLM and Internet Explorer 8 as a client. The important note here is that the user didn’t do anything beyond visit the site. All this happens in the background when WIA behaves appropriately.

Request/Response 1

This is the first of a series of requests and responses to a web server that requires the user to authenticate via WIA (in this case, NTLM).
The user types in the address of an end point they’re trying to visit (say http://server:81) and the browser sends the first request with the following request headers (request and response headers courtesy of Fiddler).
The server looks at the request, notices that there’s no cookie or any kind of identity sent with it, and kindly tells the browser (401) that not just anyone can visit this URL. In the response headers, the web server asks the browser to establish the users identity using the NTLM protocol.

Request Response 2

This is the second and final request/response in the WIA handshake. Its of note that the user still hasn’t done or seen anything, s/he’s still waiting on the screen to load…

The browser sends a 2nd request with the identity of the user according to the NTLM protocol. The server inspects the request, processes the NTLM response and decides that this user is indeed who s/he claims to be.
The server issues a response with a token that the client can use to identify themselves.

Request Response 3 (Not Shown)

Finally, the client makes the last request using the token just obtained, to identify themselves. The server authenticates them and hands back a cookie associating the user with a logged in session. The client is finally authenticated.
Its worth mentioning that this handshake takes place extremely quickly and the user is often none the wiser. All the user truly notices (if anything at all) is that the page they just requested recognizes who they are.
As mentioned earlier, when it works it offers a great user experience, but there’s a couple things that can get in the way.

Internet Explorer

Internet Explorer is the only browser that works with WIA “out of the box”. Any browser that supports the required authentication protocol (NTLM, Kerberos, Negotiate, Digest) can authenticate against the server, but there are some special conditions that need to exist.
For IE to send an encrypted identity over the wire, WIA has to be enabled (it is by default). You can check to see whether WIA is enabled under Tools-> Internet Options->Advanced Tab.
In addition to WIA being enabled, one of the following needs exist:

The site needs to be in the “Intranet Zone”. IE has a pretty simple test to verify this. Effectively it looks to see if there’s any periods/dots in the host name. For instance, http://someserver is in the intranet zone, but http://someserver.fullyqualified.com however is not (even thought they may both resolve to the same IP. As a result any IP address (e.g. 192.168.132.155) also won’t be in the intranet zone either.
If the site isn’t in the intranet zone, it needs to be a trusted site for IE to send the user’s identity over the wire w/out prompting. You can see/manage trusted sites by going to: Tools->Internet Options->Security->Trusted Sites

Once the above is handled (either access to the server takes place with a “intranet” URL, or its put in a trusted zone) WIA usually behaves itself pretty well.

FireFox

Firefox has no ideas of intranet zones like IE. If you want FireFox to trust a site and send over the user’s identity, you need to enlist the site as a trusted NTLM site (and ensure that NTLM is a usable protocol for WIA in IIS).
You do this by:

Opening FireFox
Type “about:config” into the address bar
Edit the Network.automatic-ntlm-auth.trusted-uris property and add the URL of the site you’d like to trust. You can enter multiple URLs delimited by commas.

This browser should now be able to authenticate via NTLM.

Chrome

At the time of writing, Chrome doesn’t have have a way to seamlessly integrate with WIA (the current browser does support NTLM though). Unfortunately for the time being, Chrome users will always need to enter in their credentials.

Summary

Windows Integrated Authentication can be pretty slick in certain environments. While some if its protocols aren’t rock solid security wise, and many are proprietary, it provides one of the cheapest solutions for authentication that balances both maintenance and ease of use for the user.
My Best,
Tyler

Troubleshooting Blue Screens

2011-06-29T21:01:00.001-07:00

How is that supposed to be helpful?

Anyone who’s used Windows long enough has eventually ran into the quintessential Microsoft error, the Blue Screen of Death (BSoD). Not that other operating systems don’t have their own versions, those that have played with their share Unix flavors know all to well about kernel panic.

Nevertheless, someone usually needs to troubleshoot these, especially if you own the machine. The tricky thing about BSoDs is that they:

Can be hard to reproduce
The error messages are seldom helpful
Can be driver related OR hardware related
Don’t always wait for user feedback before disappearing (sometimes the machine will restart right away depending on OS settings).

I recently did a new build at home. Latest gear. Full of promise. That is, until the machine (running Windows Server 2008 x64) started to BSoD when restoring sizeable databases (8 GB) on SQL Server 2008 R2 x64.

Even though the error messages themselves are usually useless, it still helps to see them (even the appearance of a useless error messages is evidence of failure). This isn’t always the default behavior with system faults in all versions of Windows. So for many, the first step is to acquire/witness the blue screen itself (instead of your system sitting at a login prompt, when 30 min ago it was restoring a database).

Disabling Automatic Restart

First ensure that you’ve disabled the Automatic Restart checkbox in Startup and Recovery. This is typically found in:

System Properties (Control Panel->System)
Click Advanced System Settings
Click on the Advanced tab.
Click on Settings under the Startup and Recovery section.
Ensure that the Automatically restart checkbox is unchecked and its usually not a bad idea to Write an event to the system log (checked).

Reproducing the Error and Narrowing the Field

If you’re like 99% of users, you can’t infer what the problem is from a BSoD error or analyze a memory dump. If that’s the case, the next steps are reproducing the error, and trying to figure out root cause.

If you can’t easily reproduce the error (or if it’s a new build), then all your components are suspect. It could be anything from hard drive, to your video card. If you’ve recently installed new hardware or drivers, those are likely the culprit.

For testing many components at once I’d suggest a product like PassMark’s Burn In Test, a tool we used to use back in my tech days. There are many similar products available (assuming you have a preference), essentially you just want to be able to selectively test many components at once and get either a:

BSoD (in which case you reduce the number of components you’re testing until you isolate the culprit)
A fail from the testing software which will usually tell you what component is causing the trouble

The next step (once the component is identified) is either:

Replacing drivers for that component, or failing that
Replacing the component all together (hopefully its not something that’s integrated into your motherboard)

Lather, rinse, repeat. That is, after you’ve identified (what you think is) the issue, and either swapped out a component or device driver, you need to run the tests again to see if you’ve actually improved anything. Once the errors start going away you can start to have some confidence in your fix.

Resolution

These take time. Running a full burn in test alone usually takes 20 min. In my case, my issue ended up being a faulty hot swap bay, which is unfortunate because its likely the last device you’d suspect. More often than not, its memory, a video card, or some other kind of removable component. Assuming you can’t narrow it down, power supplies can also be suspect.

Hardware’s great, especially when it works. But with increasingly complex computers these days, DOA’s and semi faulty components seem to be more common. Ideally this saves someone some out there some time, or even better, a tech bill.

My Best,
Tyler

SharePoint Health Rules Modifying Your Web.Config

2011-05-15T23:32:00.001-07:00

Is That a Web.Config Ninja?

One of the new feature in SharePoint 2010 is the Health Analyzer. At its core, it’s a set of rules that perform checks and/or repairs executed by timer jobs. When a rule executes, the output gets rolled up into a Health Analyzer Report that helps inform SharePoint Administrators about any potential issues with their farm configuration.

Even if you don’t have any interest in Health Rules/Reports, its something you should know about. The rules (including those that ship out of the box) don’t all just harass you into maintaining a cleaner farm, some of them actually “correct” possible issues without asking first. Having a series of timer jobs that can modify your farm adds complexity, and you should know what possible modifications can be made while you’re off the clock.

If you’re curious as to where these rules live, they can be found in the Central Administration site under Monitoring->Review Rule Definitions.

Case In Point

In a recent engagement we were using the stock ASP.NET Membership Provider for user management (for FBA accounts) in a Claims Based Authentication site. It became important to be able to decrypt users password for retrieval which meant that we had to modify the encryption/decryption settings. For the stock provider, these are housed in the <machineKey> setting in the web.config.

Then things got weird.

Across many of our machines, the settings kept getting rolled back. Every night at 12:00am (midnight) parts of the web.config would get reverted to the old machine key settings. It got to the point where we were considering making the web.config read only.

Then we did some snooping around and found out about a particular Health Rule that takes syncing web.config files pretty seriously. There’s one in particular called Web.config Files are not identical on all machines in the farm which out of the box will repair automatically. This can be a huge help if your web.config settings have unintentionally fallen out of sync, but if its rolling back intentional changes, then its downright troublesome.

There’s nothing wrong with this rule per se, but if you ever find yourself having similar issues, I’d recommend unchecking the Repair Automatically check box. This will hopefully help you both find closure, and move on to the next SharePoint woe in your life. Hope it helps someone.

My Best,
Tyler

Access Denied With Linq to SharePoint (SPMetal)

2011-04-17T16:49:00.001-07:00

Access Denied

SPMetal (Linq to SharePoint) is kind of a mixed bag. On one had it’s great addition to what used to be a very anemic toolset for SharePoint. On the other hand it has some pretty remarkable shortcomings that aren’t really advertised all that well in the documentation.
Two of the more notable ones:

No support for anonymous users (at least at the time of writing)
No support for RunWithElevatedPrivileges

These two shortcomings are actually very intimately related. If you’re using SPMetal (Linq to SharePoint) and you find that code in RunWithElevatedPrivileges still yields Access Denied errors (or doesn’t seem to run in any kind of elevated context) you’ve potentially encountered a very awkward shortcoming of SPMetal.
The good news is that there are some very functional workarounds to these issues but for you to troubleshoot these (and discern where they’re appropriate) it helps to know why these errors happen.

What’s Going On?

For those who have worked with RunWithElevatedPrivileges before you know that certain “shapes” of code don’t work. An example of this would be running some code with RunWithElevatedPrivileges context but then using an object you’d previously created outside the elevated context. Here are two examples of RunRunWithElevatedPrivileges, one which still throws an Access Denied and another that works.
The astute observer will recognize that the below will throw an Access Denied error

//Doesn't work, still yields access denied.
SPSecurity.RunWithElevatedPrivileges(delegate()
{
  using (SPSite site = SPContext.Current.Site)
  {
    using (SPWeb web = site.RootWeb)
    {
      web.AllowUnsafeUpdates = true;
      SPList list = web.Lists["ListName"];
      SPListItem item = list.AddItem();
      item["Property"] = "Property Value";
      item.Update();
    }
  }
});

The reason the above fails is that the SPContext.Current.Site gets created relatively early in the request lifecycle. By the time you call it from some page/webpart/etc… the SPSite object has long since been constructed, and at the time the SPSite WAS constructed, the application was NOT running in a RunWithElevatedPrivileges security context.

The below DOES work since a new SPSite (and all other objects) are created in the RunWithElevatedPrivileges security context.

//Works because the SPSite, SPWeb etc... are created in the new elevated security context.
SPSecurity.RunWithElevatedPrivileges(delegate()
{
  using (SPSite site = new SPSite("http://servername"))
  {
    using (SPWeb web = site.RootWeb)
    {
      web.AllowUnsafeUpdates = true;
      SPList list = web.Lists["ListName"];
      SPListItem item = list.AddItem();
      item["Property"] = "Property Value";
      item.Update();
    }
  }
});

The moral of the story is that objects you’re working with need to be recreated within your RunWithElevatedPrivileges code block.

The Problem with SPMetal

The two problems we kicked off this post with:

No support for anonymous users (at least at the time of writing)
No support for RunWithElevatedPrivileges

Actually stem from the same class of problem as the incorrect “shape” of code shown in the RunWithElevatedPrivileges example above. When the Linq to SharePoint Provider goes to create a connection to the given SPSite/SPWeb, it’ll take the current SPSite object out of the SPContext.Current context. The problem with this is that there’s no (easy) way to have that SPSite object constructed with an elevated security context.

Here’s a look at the reflected code.

The solution that most people have taken for the anonymous access issues also works for the RunWithElevatedPrivileges issues.

The trick is to set the HttpContext to null temporarily just prior to creating creating the DataContext in a RunWithElevatedPrivileges block. This bullies the Linq to SharePoint provider into creating a new SPSite, and this one is created with the correct context.

Below is a helper method which follows the RunWithElevatdPrivileges pattern which allows you to create a DataContext that will indeed run with elevated privileges.

/// <summary>
/// Runs with elevated priviledges after setting the HttpContext temporarily to null prior to doing so. This allows linq to sql 
/// to create a DataContextManager based on the RunWithElevatedCredential (farm account) instead of the current user.
/// Otherwise the linq2sql DataContextManager would take its SPSite off of the SPContext.Current.Site which is already cached
/// with the priviledges of the current user.
/// 
/// NOTE: This should ONLY BE CALLED by code creating Linq to SharePoint SharePointDataContexts. It's also of note that any
/// SharePointDataContext created using this code WILL BE ABLE TO DO ANYTHING the farm account can.
/// </summary>
protected static void RunWithElevatedPrivilegesAndContextSwitch(SPSecurity.CodeToRunElevated secureCode)
{
  HttpContext backupContext = HttpContext.Current;

  HttpContext.Current = null;
  SPSecurity.RunWithElevatedPrivileges(secureCode);

  HttpContext.Current = backupContext;
}

Usage might look something like the following:

SharePointDataContext context = null;
string url = SharePointHelper.WebApplicationRoot;

RunWithElevatedPrivilegesAndContextSwitch(delegate()
{
  context = new SharePointDataContext(url);
});

//Do something you normally wouldn’t be able to with this elevated linq to SharePoint context.
List<ListName> items = context.ListName.ToList();

It’s worth mentioning that if your data access is consolidated in to a formal data access layer you may need to centralize this kind of code so that it doesn’t get placed all over the site, but above is the central idea behind getting Linq to Sharepoint DataContexts to run with Farm Account privileged access.

I hope the above makes sense, and helps.

My Best,
Tyler

Awkward Usernames Courtesy of Claims Authentication (FBA)

2011-03-16T00:25:00.000-07:00

Who’s i:0#.f|membership_providerName|userName?

If you’re one of the many people who have set up Forms Based Authentication in SharePoint 2010, you’ve likely enjoyed dealing with the awkward usernames that can come with claims authentication and FBA (Forms Based Authentication).
In a recent engagement, we’d provisioned an ASP.NET Membership schema (aspnet_regsql.exe), populated it with a stock SQLMembershipProvider and then added the users to the site collection using the Site Settings->People and Groups screens.
What we ended up with, were a lot of SPUsers with "unfriendly" SPUser.Name properties set. Since the Name field wasn’t assigned it ended up getting set to the equivalent of the SPUser.LoginName property which looks a lot like:

i:0#.f|fba|administrator

The problem with leaving these cryptic display names in their native format is that they can seep all over a farm. Not only will they show up in stock SharePoint controls that display user names, but those same display names will also start to show up in other service applications like search.
There's also the fact that your users will likely ask you to set them to something more sensible.

Resting the SPUser.DisplayName Property

Assuming you have some other name that you’d prefer instead of the awkward native claims format, you can explicitly set these display names by using either PowerShell or the stock SharePoint API.
Below are two examples, both involve you iterating over everyone in the authentication store (think SQL, LDAP, AD, etc…) and updating the SPUser.Display/Name property after fetching the SPUser out of the site collection. See below:

PowerShell

$user = Get-SPUser -Web "http://wss2k10tholmes:81" -Identity "i:0#.f|providerName|userName"
$user.DisplayName = "FriendlyName";
$user.Update();

C# (API)

//We use ensure so that the given user will be added if they don't exist. They need to be resolved

//via their claims name.
SPUser spUser = web.EnsureUser(string.Format("i:0#.f|providerName|{0}", fbaUser.UserName));
spUser.Name = "FriendlyName";
spUser.Update();

Alternatives

You don’t necessarily need to go through the API or PowerShell, if you have a connection to an LDAP store or a BCS connection to your auth store. You can also map the properties yourself and leave it to the User Profile Synchronization service. That being said, if you’re dependent on BCS then you’ll also need to have SharePoint Enterprise Server license which isn’t available to all customers.
Once you’re done you should be
able to visit any of the users in your site collection and see their “Name” property set to something that is less likely to confuse your user base. Once the value is set, it helps to make sure that it doesn’t get stomped with any User Profile Synchronization (UPS) that may be in place in your farm.

Hope that Helps,

Tyler

Getting a Public Key Blob From 3rd Party Assemblies

2011-01-22T13:48:00.001-08:00

The Return of Code Access Security

Working with SharePoint sometimes takes you to some pretty weird places. There’s a lot of .NET’s surface area that I’m pretty sure I’d never have explored on my own if I hadn’t been tasked to do so via some SharePoint engagement.

One of these explorations has involved authoring custom CAS policies, the reasons for which I’ve blogged about in the past. Granted with the advent of SharePoint 2010 and sandboxed solutions it’s not as common a task these days, but it still comes up.

When authoring a custom CAS policy you’re taxed with obtaining the Public Key Blob from a Strong Name Key. This helps the .NET runtime identify assemblies (via their Strong Name) that you’d like to run with a particular code access security profile.

Now obtaining a public key blob is pretty straight forward if you have the .snk, but it gets a little more awkward if all you have is some 3rd party assembly. This is often the case when you want 3rd parties assembly (for us it was a bunch of Telerik Controls) to run in full trust. After all you don’t have their key and they’re super unlikely to give it to you.

Getting a Public Key Blob from a 3rd Party Assembly

1) First open up the Visual Studio Command Prompt (or optionally any command prompt, but have access to sn.exe which is typically found in Visual Studio’s SDK->BIN folder).

2) Run the following command:

sn -e [ThirdPartyAssembly.dll] publicKey.snk

This extracts the public key (and public key blob) into a file.

3) Then run:

sn -tp publicKey.snk

Which should display the public key blob and public key token (the blob is the long one).

At this point you can throw it in your own custom CAS file or resume whatever would make you seek out these awkward little strings in the first place.

Hope that helps. Had stare at the sn.exe docs for a good 10 minutes to find this and the web wasn’t all that helpful.

Happy CASing,
Tyler

Free Last Modified By/Date From SharePoint

2009-06-09T00:33:00.001-07:00

Who Responsible For This Edit!?

With the exception of politicians, most people are in favor of accountability. When it comes to web content management, I'd be the first to agree.

In this post we're going to go over how to get free Last Modified By and Last Modified Date fields on content pages by modifying either select page layouts or master pages. The end product will look like:

What We're Using

We'll be keying off two base fields (Modified, and Modified By) that you can expect to find on pretty much any list you'll ever work with. To prove to you I'm not making things up, feel free to look up the fields yourself (and a tonne of other useful fields) by using SharePoint Explorer for WSS v3.

Easy Win

Once you find out the names of the fields and their type, the markup to get at these fields is pretty straight forward. We use out of the box SharePoint SPField controls to pull and format the field values. It's of note that we put the ControlMode property to "Display" since we never want these fields to be editable.

Last modified at:
<%@ Register Tagprefix="SharePointWebControls" Namespace="Microsoft.SharePoint.WebControls" Assembly="Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" %>
...
<SharePointWebControls:DateTimeField FieldName="Modified" runat="server" ControlMode="Display"/>by
<SharePointWebControls:UserField FieldName="Modified By" runat="server" ControlMode="Display" />

That's pretty much it, throw the above markup into any page layout or master page and you're pretty much done.

But Wait There's More

I'd hate it if you left this post thinking all you'd learned about were two lousy fields and how to display them on a page. There's a tonne of fields that are worth exploring and could potentially add value to your next SharePoint related application.

Example 1: ETL developers could make use of the Created/Created By/Modified/Modified By fields to perform incremental loads when pulling data from the SharePoint object model.

Example 2: UI Developers could use the Comments and Version fields to display the current version and any comments associated with the current version.

The point is that there is yet another lever here to help you get the most functionality out of your next SharePoint related application. Best of all, it's dead simple and doesn't involve a single line of code.

Best,
Tyler

CSS Selector Collisions

2009-04-10T19:52:00.001-07:00

Thick Competition

For those brave souls who actively try to make stock SharePoint UIs look user friendly, there's a gauntlet that first needs to be run. The challenge has to do with competing CSS rules and styles. Stock SharePoint will emit many style sheets who will often also be trying to style the same element that you are.

When it comes to deciding which rule wins, you should keep the following rules in mind. They'll very likely help you troubleshoot, especially when combined with tools like the IE Developer Toolbar or the Firefox Web Developer Tools extension.

Specificity

In general more specific rules tend to win. Remember that CSS styles can come from four different places. Rules that are increasingly specific will trump those who come from more generalized sources. Below we list (from most general to most specific) the four most common sources of CSS for a given html document.

The browser's default styles. Believe it or not each browser has a stock set of styles that get applied to elements whether you've chosen to style them or not. You'll have to compete with these even if they're the easiest to best.
An external style sheet (referenced from within the <head> tag [below]).
<head>
<link rel="stylesheet" type="text/css" href="mystyle.css" />
</head>
Internal style sheet (rules described in <style> tags within the HTML document itself).
<HEAD>
<STYLE type="text/css">
H1 {border-width: 1;}
</STYLE>
</HEAD>
Inline styles (styles placed on individual elements themselves).
<P style="font-size: 12pt; color: fuchsia">Some Text</P>

More specific rules (those further down the list) will win out over more general ones.

Weight

Certain selectors are a little stronger than others. More specifically, ID has more weight than Class which has more weight than Tag name.

In fact weight of a selector is determined by taking the sum of (100 * [# IDs used] + 10 * [# Classes used] + [# Tags Used].

Here's an example:

#navigation div.container span a.footerLink

Has a weight of (123 since it has 1 ID, 2 classes, and 3 tags. It would be trumped by:

div#navigation div.container span.wrapper a.footerLink

Since the above has a weight of 134 (1 ID, 3 classes and 4 tags).

Order

If specificity and weight are the same then it comes down to who was emitted last. In general rules that were emitted later on in the document will win.

The Ace In The Hole

If you're too lazy to figure out the rest you can simply use the !important flag.

div
{
color:red !important;
}

This should trump all other rules but is considered to be pretty poor form and may get you dirty looks around the water cooler.

Hope that helps.

Best,
Tyler

Is It Really My Job To Fix That!?

2009-03-05T18:43:00.001-08:00

That's Not On My Resume

I have to tell you that working in a software services firm isn't always a picnic. Being a vendor (read outsider), and responsible for delivering working solutions on other peoples infrastructure, in a foreign LAN is nothing but a brazen undertaking. It's akin to trying to construct a building in a war torn, 3rd world country that happens to be in the middle of ongoing natural disasters. The drama sometimes matches that of most reality TV series, and while it's rarely dull in some regards, there's easier ways in life to get a win.

When the hardware isn't changing and internal IT isn't ignoring you, it's most likely that you'll get bad OS builds, under powered VMs or fail to take note of the hidden agendas in a politically charged atmosphere. This is all in addition to regular project management woes. I'm just talking about delivery here, requirements and software lifecycle are another boat in entier.

You need to be really sharp to see all the possible ways you could fail, and even then it'll probably be something small that never even crossed your worried mind that will end threatening you and your project. Game on.

An Instance of Success

Today we came up with this interesting problem when installing some ETL on a BI server. The machine was running SSIS RTM on a Win2k3 x86 Service Pack 1. The ETL was taking a bunch of records from some source SQL Server database (SP2) and migrating them to another destination SQL Server database (SP2). The ETL would run for about 20ish minutes before consistently throwing the following exceptions:

An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80004005 Description: "Protocol error in TDS stream".

An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80004005 Description: "Communication link failure".

An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80004005 Description: "TCP Provider: An existing connection was forcibly closed by the remote host. ".

Hmm...what to do. The ETL worked great in stage and in our own QA environment. After an exhaustive troubleshoot (on our client's machine) the fix ended up being...disabling TCP Chimney offloading. Not even on our BI server but on the destination SQL Server!

What!? You say? Your problems with delivery weren't even vaguely related to the technologies you were developing!? Exactly, something not even vaguely related to ETL became a huge friction point between us and a client. The worse part is that there's always a risk of this. Vendors come in and constantly take it on the chin (or sometime just look incompetent) because a dependency they depend on isn't working.

What makes it worse is you may have never even seen that machine in your life, and it's the one that just might bury you. Half the time you don't even have the necessary rights on the given machines to fix it let alone conduct a decent troubleshoot. All of this just might want to make a man take up farming.

This time we did have the necessary rights on the machine. We got lucky and there's even a chance that we might actually get paid for the troubleshoot...this time. Who knows what kind of circus fixes we'll be trying to pull off tomorrow. The good news is that after a while some partners actually start to trust you.

Hope that fix helps some vendor somewhere, we sure could have used it.

My Best,
Tyler

STSADM Properties

2009-02-18T17:11:00.001-08:00

Wait...There's Another Flag

If you've ever felt like the STSADM utility is taxed with doing more than Emacs, there's a good chance you're in the majority. I was recently trying to figure out what SharePoint settings affect the "New" icon when I stumbled upon the stsadm's getproperty and setproperty flags.

Essentially there's a bunch of SharePoint settings (some of which there's no admin UI screen for) that can be manipulated using this utility. The new icon is one such property.

It might be worth looking at some of these, they include both server and web application level properties.

Server Properties

Web Application Properties

As it turns out the default value for the new icon is 2 days. You can show and change this property by using the following commands:

stsadm -o setproperty -pn days-to-show-new-icon -pv 2 -url [WebAppUrl]

stsadm -o getproperty -pn days-to-show-new-icon -url [WebAppUrl]

This probably wont interest you in the least until you start to think about scripting and how this might help you ensure certain settings have been set when it comes time to deployment.

While many of these settings might be alterable using the SharePoint API, there are still many types of changes that deserve to be versioned in some kind of script rather than C#.

Best,
Tyler

Start Your April Fools Jokes Early

2009-02-15T23:08:00.001-08:00

Time Waits For No One

For some reason Valentines Day always reminds me that April Fools is coming up. This most likely draws from the fact that I'm the youngest of two siblings, and it's become a personality trait to express my affection by irritating the crap out of an individual.

Needless to say I have very few close friends.

Since nothing says "I care about you" like a good practical joke, I'd like to take the time to remind you that April Fools day is a very short 32 business days away, and depending on how deep your pranks run, you may need to get some infrastructure in place.

The Right Joke For The Right Bloke

A good practical joke is tailored to the individual. Knowledge of their technical depth, their patience, any history of violence, and past convictions are key to finding just the right level of crazy.

If you'd like to go shopping for ideas, I'd suggest starting here. My only advise is to be prepared for retaliation. Most pranks usually deserve another, and the minute your cover gets blow, it'll already be too late to start wearing a helmet.

Get Going

So get on it. Usually most April Fools jokes fall apart because people leave them to the last minute. DNS records may need to get updated, proxies may need to get provisioned, machines may need to get hacked. Don't risk leaving it to the last minute, start being devious and subversive today!

I'll share this years war story on April 2nd.

Best,
Tyler

How SharePoint Found Its Content Database

2009-02-09T00:02:00.001-08:00

Good Question

I was talking a client through a SharePoint web.config change a couple of weeks ago and he ended up asking a dynamite question.

How does the web application know where its content database is at?

You might say something like "it gets it from the config database", but that would beg another question, "how does it know where the config database is?". In reality, we're curious about how a web application orients itself to its environment when it comes time to respond to a request.

You'll notice that SharePoint web.configs don't have any connectionString elements in them that point to a particular content or configuration database, and SharePoint installs don't make any changes to parent web.configs or machine.configs. So how does this web application orient itself at runtime?

Lets Reflect, Literally

If you think about it, there's a few clues to where this code "ought to be". Pretend for a minute that you were tasked with authoring some piece of functionality that let a web application figure out where its database was at runtime. There's some typical places you'd hook in (besides the web.config). You'd know that this piece of functionality would need to be populated in time to serve requests who would need to know where the content db is at.

You might think about putting it in global.asax Application_OnStart or some http module event that's relatively early in the ASP.NET pipeline.

Lets look at how a typical SharePoint request gets resolved and see if we can't flush out some details. This is the path I took to piece this answer together.

A request comes in and gets handled by IIS. The request matches either a port number or a host header on a web site in the web server.
The matching SharePoint web site has wild card ISAPI mappings that pass all request to ASP.NET. Remember that in a typical ASP.NET web site there mappings for known ASP.NET file extensions (.aspx, .asmx, .ascx, etc...) to the ASP.NET ISAPI filter. When IIS gets a request that ends with the given file extension, it knows that this is an ASP.NET request and hands it off to the aspnet_isapi.dll ISAPI. In a SharePoint IIS web site, we pass everything to ASP.NET. This includes .gifs, .jpegs, .PSDs etc... SharePoint gets a chance to fulfill every requests that gets picked up by the site. Below is a screen cap the wild card mapping present in all SharePoint sites. This is created for you automatically when you provision a new web application. If you were to remove it, your SharePoint instance would no longer be able to serve non ASP.NET files out of the content database.
Once the request gets handed to ASP.NET, its fair game for the web application to start doing some magic. In fact, it had better figure it out soon cause odds are this request is going to want something from the content database before the page life cycle hits. Because we're talking about the ASP.NET request pipeline (and early in it), it's a reasonable guess to assume an http module could be in play. If we look in the web.config under the <httpModules> section we find that SharePoint has registered an HttpModule named "SPRequest", it's fulfilled by code found in the Microsoft.SharePoint.ApplicationRuntime namespace by a class called SPRequestModule.
If you reflect on SPRequestModule and look into BeginRequestHandler (one of the earlier events in the ASP.NET pipeline), it's obfuscated, but you can read it anyway. BeginRequestHandler makes a call to SPFarm.Local which ends up calling SPFarm.FindLocal in an effort to find the farm that this machine belongs to.
SPFindLocal finally calls into SPConfigurationDatabase.Local which runs to the registry looking for HKLM\SOFTWARE\Microsoft\Shared Tools\Web Server Extensions\12.0\Secure\ConfigDB. This is where your "connection string" for the current farm lives.
This is also how SharePoint finds the config database, from there it simply takes the current request and uses it to find the content database (if any) associated with the given request. After all this has been figured out, a lot of it gets cached.
Finally BeginRequestHandler starts through the complex task of mapping your request to a bunch of content in the web applications content database. The regular Page Life Cycle kicks off and at the end of the day we end up with an HttpResponse built up by SharePoint.

Thought that someone else might enjoy the answer. This is kind of a neat piece of functionality too, it allows the psconfig tool to move this machine from farm to farm without changing any web.config files, so long as you can tell it where it should remap the registry key to. And with that, there's a little less magic in the world.

Best,
Tyler

When the Reflector Tells You To Hit The Road

2009-02-06T01:18:00.001-08:00

Seeing Obfuscated Code

Like every other .NET developer out here, I'm a big fan of the .NET Reflector. It was originally written by Lutz Roeder but is now maintained by the kind gentleman at Red Gate. One of the elements that make the tool stand out (in addition to the ease at which it lets you navigate/disassembling code), is the ton of add ins on CodePlex that extend the base functionality. If you haven't already, you should explore them some time.

I mostly use the .NET Reflector when it comes to things that I don't quite understand and I can't seem to find an adequate answer on the web. Ninety percent of the time, this tool will usually make it easy for you to look underneath the hood and you can find your own answers.

Every now and then when you're reflecting on code that someone doesn't thing you should see, you'll get the following message from the tool.

This item is obfuscated and can not be translated.

If your stubborn like most rural mules or have never heard of "no means no", you can carry onward by going to View->Options->Click Dissassembler and set the language to IL.

If you're simply asking for the IL, Reflector will gladly show you the contents of the function (if you can make any sense out of it). Intermediate Language is even harder to read than disassembled MISL that's mapped to C# or VB.NET (which unreadable for another variety of reasons).

In fact if you're just going to be looking at IL, you can actually skip the .NET Reflector in the first place and use the MSIL Disassembler (ildasm.exe) that comes with the .NET framework. May not look as pretty, but you're guaranteed to have it on any machine which as the .NET SDK.

Here's a disclaimer.

It's also worth mentioning that by reverse engineering code you could be violating an EULA, and I'm hear to tell you that you should never do that. In fact, if you're ever given a piece of code that doesn't function correctly, you should never try to fix it or do anything to further your understand what's going wrong. Often the most legally sound action, is to turn off the computer and go home.

Hope that helps someone find an answer.

Best,
Tyler

Quick Deploys And Assembly Resolution

2009-01-30T00:14:00.001-08:00

The Error

We recently ran into the most bizarre error on a clients machine that hosts the central administration web site. Every 15 minutes on the dot we'd get the following error for many different assemblies. The error looked a lot like:

Event Type: Error
Event Source: Windows SharePoint Services 3
Event Category: Runtime
Event ID: 6611
Date: 1/15/2009
Time: 1:11:26 PM
User: N/A
Computer: [MACHINE NAME]
Description:
Error: Failure in loading assembly: [AssemblyName], Version=1.0.0.0, Culture=neutral, PublicKeyToken=e376b6bc65267f90

It's worth mentioning that these errors mentioned the names of assemblies that were used in multiple sites (even though they were deployed into bin folders). The sites that used these assemblies were customized, and their Master Pages and Page Layouts referenced the assemblies that supposedly couldn't be loaded (as far as the error was concerned. Some of these dependencies were strong named and others were not. None of them lived in the GAC (all bin folder deployed).

The Solution

We used the fact that the errors occurred every 15 minutes to link them to Quick Deploy jobs. When we changed the quick deployments to every 10 minutes, the errors followed suit reoccurring at the same interval.

At this point we were convinced that the SharePoint Timer (OWSTimer.exe) was trying to load these assemblies but couldn't find them. So how does assembly resolution happen in the .NET framework? We were saved by an an excerpt from Essential .NET, Volume 1: The Common Language Runtime, a book by Don Box and Chris Sells. Essentially it works like this.

When the assembly loader goes to load an assembly, it first looks to whether the assembly is strong named. If it IS, then the loader first looks in the GAC (not the bin folder).
If it can't find the assembly in the GAC, the loader will then look for <codeBase> hints in the applications configuration file. Remember that these settings inherit down from machine.config to app.config/web.config.
If there's no <codeBase> hints then the loader will resort to probing as a last ditch effort to find the assembly. This includes bin folders and any other location dictated by <probing> elements.

If the above seems confusing, consider the following flow chart.

We first tried putting hints in the local web.configs, but because the OWSTimer has nothing to do with the our applications, the settings were being ignored. We finally helped the timer out by making the following modifications to the machine.config. To ensure that we didn't remap ALL lookups for the given assembly to the specific location, we had any other application who used the same assembly name override these settings below with a similar one in their own web.config. The overridden setting listed a location that made more sense for the particular web application (ie. web applications should go hunting for .dll's in their own bin folder, not some other applications).

The machine.config was changed to read:

<runtime>  
 <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">   
   <dependentAssembly>   
     <assemblyIdentity name="[NameOfAssembly]" publicKeyToken="9871993fa258bc6" culture="neutral" />   
     <codeBase version="1.0.0.0" href="file://C:/folder/directory/bin/[NameOfAssembly].dll" />   
   </dependentAssembly>   
 </assemblyBinding>   
</runtime>

The web.config was changed to read:

<runtime>  
 <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">   
   <dependentAssembly>   
     <assemblyIdentity name="[NameOfAssembly]" publicKeyToken="9871993fa258bc6" culture="neutral" />   
     <codeBase version="1.0.0.0" href="file://C:/webapplicationLocation/bin/[NameOfAssembly].dll" />   
   </dependentAssembly>   
 </assemblyBinding>   
</runtime>

You can also provide <codeBase> hints for assemblies that aren't strongly named, you just need to omit the the optional publicKeyToken and culture attributes. Another tip is that you can also use the <probing> element if you want to probe folders other than the bin folder.

We then restarted IIS, followed by the timer service. Low and behold, our random error went away. This toast goes out to the highly configurable .NET framework, and the expressiveness of app/web/machine.config files.

Best,
Tyler

TroubleShooting The JQuery Accordion

2009-01-27T21:03:00.000-08:00

JQuery and the Accordion

I've been getting more and more into JQuery recently. It's light, free (as in freedom), and has a vibrant community. Sweetening the deal is the ever growing list of animated controls that effortlessly plug into the framework.

We've been using JQuery for quite a few of our clients who have expressed a desire for more interactive SharePoint sites. I've been so impressed with the framework of late that I've started to advocate it over heavier tools that try to achieve similar results.

The Problem

Just today I ran into a pretty interesting problem with the JQuery Accordion. I'm throwing up this fix because I couldn't find a solution on the web. We had the accordion binding off of a SiteMapDataSource in a SharePoint master page, everything looked great in Chrome, FireFox, and Safari...but IE 6 and 7 were giving us a really hard time.

What made this even more frustrating is that it worked fine in at least three other SharePoint sites with exceedingly similar master pages.

The accordion would flicker at the end of the animation every time the user would expand or collapse a sub menu. We tried removing all the CSS, but the page still yielded the same behavior. The menu would briefly flash the entire contents of a sub menu whenever a a new section was collapsed/expanded. The only way we could stop this was by turning the animation off in entire.

The Troubleshoot

At first I tried debugging the JavaScript for JQuery and the accordion, but the source quickly became difficult to follow. The guys who wrote this stuff have mad JavaScript skills, they're head over heels better JavaScript developers than I am.

So I ended up doing what I normally do when I don't understand what's going on. I started to make things simpler. I opened up SharePoint Designer and started removing sizeable sections of the page, block by block, seeing if I could get the problem to go away. I ended up with a ridiculously small html document that basically had the the accordion menu markup and a couple of other html tags.

It was only when the document was about forty lines long that the problem became apparent.

There was no doc type on the html document. I've written about Quirks Mode before, so I was a little embarrassed that I hadn't noticed it earlier. If you're not familiar, or are too lazy to click on the link above, in a nutshell quirks mode is a special rending mode that browsers go to when you either don't specify the doc type or use html that deviates from the declared doc type. The rules for what justifies quirks mode rendering changes from browser to browser. For IE, not declaring a doc type will cause it to render the page in quirks mode.

Quirks mode changes the way that the browser handles CSS rules. When we failed to declare a doc type, IE started rendering the menu in quirks mode. This changed the way that the browser handled the in line CSS that the JQuery accordion was tacking onto menu list markup. Hence the odd behavior.

I added my favorite and most relaxed doc type:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">

Low and behold, the animation issue went away. Quirks mode strikes again. I guess the lesson here is that before you get to involved in a markup troubleshoot, ensure the document isn't being rendered in quirks mode! You can figure out how to detect quirks mode by reading this.

Fool Me Twice,
Tyler