Explicit user types

Let’s first understand the use case. Some websites are structured in a way to allow visitors to self-select their interests, perhaps by self-selecting a user type. For example, an eCommerce website might categorize content differently between end consumers and businesses, so allowing the visitor to self select as a business user vs. an end consumer may be beneficial to getting the user what they want sooner. There are many ways we can achieve the ability to personalize content for explicit self-selection:

1. Leverage a custom cookie for “explicit user type” and create a custom rules engine cookie condition
2. Leverage another typical state management construct (e.g. session) for “explicit user type” and create a custom rules engine condition
3. Extend xDB contacts with a custom facet for “explicit user type” and leverage a built-in rules engine condition
4. Or, don’t reinvent the wheel, leverage personas/profiles! and leverage a built-in rules engine condition

The advantage of leveraging personas to handle this is the out-of-the-box analytics tracking in Experience Analytics will surface user types via these personas with no extra analytics code. While the persona/profile route is very similar to the xDB contact custom facet route, there may be more work involved in reporting analytics on custom facets.

Defining the Explicit User Type persona in Sitecore

So let’s get started with the definition work in Sitecore. We need to define the “Explicit User Type” persona itself with the possible user values, which would end up being dimensions or “profile keys” on the persona. For each possible user type, create a key under the persona. I’ll continue with the eCommerce user type example from before:

Since the trick of this solution is to leverage binary scenarios (a visitor should only be one type), set the max value for a profile key (think dimension, or value of their user type) to 1:

So the applied usage of this binary approach would be: Consumer user = 1, Business user = 0 which means they’re a consumer. More on this later…

Defining the pattern cards in Sitecore

Now that we have a profile (“persona”) defined, we need corresponding pattern cards in order to match against users in the rules engine via the built-in rule:

where the current [contact/visit] matches the specific pattern card in the specific profile

Since we plan to leverage the profile keys in a binary scenario (you can only be one user type), we want the pattern cards to match their possible 0 or 1 values. For each possible user type, create a corresponding pattern card and make each pattern match the corresponding 0 or 1 scenario:

Putting it together with some code

Now that we have the Sitecore constructs in place, let’s leverage the binary scenarios and some code to apply a 0 or 1 to the user based on their self-selected user type. NOTE: when you add a profile key value to a visitor, it gets incremented over time. So without code to handle it as a binary scenario, you could end up with incremented values over time, e.g. Consumer user = 4, Business user = 5 means the user self-selected as consumer four times and self-selected as a business user five times. So in our code we handle this by removing the profile and re-adding it every time a user self-selects.

Here’s the meat of the code (without all the upstream initialization and checks):

[sourcecode]

public static void SwitchProfileValues(string profileName, Dictionary<string, float> scores)
{
…

// if the user already had a value for
// the profile, delete it so we can
// record a new binary scenario
if (Tracker.Current.Interaction.Profiles.ContainsProfile(profileName))
{
Tracker.Current.Interaction.Profiles.Remove(profileName);
}

var profiles = new List<Sitecore.Analytics.Model.ProfileData>
{
new Sitecore.Analytics.Model.ProfileData(profileName)
};

Tracker.Current.Interaction.Profiles.Initialize(profiles);
profile = Tracker.Current.Interaction.Profiles[profileName];

profile.Score(scores);
profile.UpdatePattern();

…
}

[/sourcecode]

And example ways to call it:

[sourcecode]

// self-selecting as a consumer user:
SwitchProfileValues("Explicit User Type", new Dictionary<string, float>(){{ "Consumer user", 1.0}, { "Business user", 0.0}});

// self-selecting as a business user:
SwitchProfileValues("Explicit User Type", new Dictionary<string, float>(){{ "Consumer user", 0.0}, { "Business user", 1.0}});

[/sourcecode]

So, does this really work in the real world? Yes! In the last 2 months I’ve used this approach twice already. Let’s look at some real world analytics to see how it surfaces. Navigate to Experience Analytics > Audience > Pattern Matches. Below is recent analytics data with the real user types redacted since I don’t want to give away information about a current client.

And there you have it. If you have solved this problem in another way, please comment with your solution!

The process of rolling back content — or said another way — publishing a prior version of content, is the same as unpublishing content. You must leverage publishing restrictions to determine what should be published to the live site.

Publishing Restrictions

Let’s take a sample item with 4 versions as an example. Versions 1, 2, and 3 are approved; version 3 is live. Version 4 is a draft and in the process of being edited. Urgently, there is a need to get version 1 live right away!

In the publishing restrictions dialog, uncheck versions 2, 3, and 4 from being publishable. This will mean I will end up publishing version 1.

Publishing Warnings

When I stay on version 4, the warning will clearly tell me that version 1 will be published, because its the latest version that I am allowing to be published based on the above restrictions.

If I flip to version 1 you will not see that same warning because I am actually on the version that is allowed to be published.

The next step is to publish and version 1 is now live. You’ve just successfully rolled back to that version of content on the live site. The later versions (2-4) still exist for an audit trail if you need them in the future.

Approach

The general approach is to build four predefined conditions in Sitecore that we can use as our four region conditions. Think of Sitecore “predefined rules” as complicated composite rules that you might re-use over and over with a friendly name. By creating these once, we can apply them in any case where we have a rules field (e.g. personalization).

Create the Predefined Condition

1. Navigate to Marketing Control Panel > Personalization > Predefined Rules
2. Create a predefined rule for each of the four region scenarios, like in the image below:

In each predefined rule you can populate the rules field with all the inner composite details with the zip codes. In this case the regions have many zip codes, but this content population effort in the rule is a one-time pain we deal with up front. The image below shows just two zip codes but you can build the rule up as big as you need with all of the possible zips.
Note: Sitecore’s geolocation data captures zip code in the Postal Code attribute.

Use the Predefined Conditions for Personalization

So how do we make use of these composite rules we painstakingly populated with hundreds of zip codes?

When you personalize a rendering, there’s a condition to select where a specific predefined rule evaluates to true:

In the dialog you would select one of the four predefined rules you created for this personalization scenario:

Now you can set a unique data source based on this region and re-use these rules over and over without having to type in all of the zip codes on multiple pages.

Indexing Strategies

Part of the indexing configuration in Sitecore includes indexing strategies that you select from. These are C# classes that run at indexing time based on how you want the system to behave. The default strategy in a stock instance for the “web” database is onPublishEndAsync. This strategy uses the event queue to subscribe to the publish:end event. When raised, it triggers an incremental index update (i.e. not a full index rebuild but small deltas directly on the index). So how does this actually work to update the index? If you’re having indexing issues with this strategy then continue reading.

EQStamp

The event queue keeps a record of events fired on instances of Sitecore. Each instance polls the queue to check for changes. The onPublishEndAsync strategy first finds the EQStamp property for the Sitecore instance in the Core database’s Properties table. The format for the Key is EQStamp_<instance name> and the value is a sequentially updated decimal.

The strategy then finds all records in the event queue that are remote (i.e. fired by another instance such as a CM) and have occurred since the EQStamp. It does this by scanning the records and comparing the Stamp column (in EventQueue) to the EQStamp (in Properties). That said, the Stamp column is in hexadecimal format, so it needs to convert to decimal to compare these values.

Compare EQStamp and Stamp

How does this help you? To troubleshoot indexing issues, clear your event queue table contents and delete any EQStamp values in Properties. A simple query like this will do:

DELETE FROM Properties WHERE [Key] LIKE 'EQStamp%'

Make some change in the CMS and publish your content to web. This will fire events and seed the event queue with some data. Next, get the most recent events like so:

SELECT * FROM EventQueue ORDER BY Created DESC

Next, open up Google (you may have heard of it) and convert the latest value of the Stamp column to decimal (e.g. search for 0x3B64 to decimal). Now compare this value to the EQStamp. If its greater than EQStamp then this record in the queue has not been processed. If it equals the EQStamp then Sitecore has processed the event and should update the index.

Things to consider

If the queue isn’t being processed based on the data from above, make sure you don’t have any errors in the logs. You can also increase the logging verbosity to get more intel on possible issues:

<log4net>
  <logger name="Sitecore.Diagnostics.Crawling" additivity="false">
    <level value="DEBUG" />
    <appender-ref ref="CrawlingLogFileAppender" />
  </logger>
<log4net>

As well as <events timingLevel="high">

Additionally, make sure your versions of Sitecore match between instances. It sounds silly, but this happened to me before where a CM and CD were off by a minor revision and indexing failed to work properly.

Finally, another way to peek in on what’s going on is to use Microsoft’s Process Monitor and sniff activity from w3wp. Fire it up, go to Filter > Filter, and add Process Name contains w3wp.

Out of the box Sitecore creates an instance of the MongoDB C# driver as the client to allow the Sitecore application to interact and communicate with MongoDB. This occurs in Sitecore.Analytics.MongoDb.dll in the class Sitecore.Analytics.Data.DataAccess.MongoDb.MongoDbDriver. The constructor creates a MongoDB client from the URL settings via the connection string. That said, if you have a MongoDB enterprise client certificate, you need to ensure Sitecore creates the client in C# with that certificate. Luckily Sitecore has thought of this and exposed a pipeline to update the MongoDB client settings before it constructs the client object. Here’s the relevant code snippet directly from Sitecore’s DLL:

So to tap into this and update the settings we need to run a processor in that pipeline. The pipeline appears to have no processors out of the box according to its config file (App_Config\Include\Sitecore.Analytics.MongoDb.config):

Let’s tap in and add our client certificate settings via the updateMongoDriverSettings pipeline.

The MongoDB C# driver documentation to update the client setting with the certificate info indicates the following sample code snippet:

So we can write our corresponding pipeline processor like this:

Now we can obviously abstract the two hard-coded strings for “client.pfx” and “mySuperSecretPassword” and expose them in a config setting. Also note that the pipeline processor base class exposes an abstract method UpdateSettings(UpdateMongoDriverSettingsArgs args) that you will want to override in the processor (rather than implementing Process(...)).

And that’s about all you need. With minimal code and config changes you can ensure the MongoDB C# client object respects the necessary certificate that the server expects.

Let’s think of a quick example of the concern: you want to ignore the Sitecore pipeline for requests to /foo, /bar, and /baz. The way I would have done this before was to patch the Sitecore setting with a copy of the old one and added my new route(s) at the end, roughly like so:

So instead, I’ve created a new pipeline processor that comes after the built-in one (which will support the existing native IgnoreUrlPrefixes setting) and will allow you to add each path via its own XML config node. The advantage here is you can patch and continue to patch on without needing to copy the existing values. Here’s the final patch config to load my new processor with two sample paths to ignore:

So not only will the original list of paths be ignored, now we’ve added /foo and /bar to the list.

Here’ an example on another patch patching in a third value to ignore:

Here’s the source code for the processor:

Marketplace Module & Source

This is now available on the Sitecore Marketplace as a module.

Additionally, the source code is on GitHub.

Update: in the process of putting this on the Marketplace I found out that Ben Golden has a similar solution with the exact same name

• An global organization-level shared content repository: content here can be shared across all sites/brands within the Sitecore application
• A site-level shared content repository: content here can be shared across all pages of this site
• A page-level content folder: content here is specific to the page

The rendering now needs to know how to get these three locations when resolving the datasource locations for the pop-up dialog. Luckily, the GetRenderingDatasource pipeline has a nifty GetDatasourceLocation processor that makes this very easy. It has built-in support for queries and static paths with a pipe delimiter between locations. Let’s take them one at a time.

First, find the “Datasource Location” field on the rendering definition item:

Now let’s populate the global, site-scoped, and page-scoped paths.

The global query is pretty easy and can be a static path. Just make sure you associate the necessary access rights to disable deleting or renaming of these global folders:

The path is:
/sitecore/content/Global Shared Content/Global Carousels

The site-specific query requires an XPath query to find the closest ancestor of the “Site” type, and traverses down the hierarchy from there.

The query is:
query:./ancestor-or-self::*[@@templatename='Site']/Shared Content/Site Carousels

Don’t forget the pipe (“|”) delimiter!

Finally, the page-specific query requires a basic dot-notation query to traverse down the hierarchy of the current item/page.

The query is:
./Page Content/Page Carousels

The final value for all three locations is:
/sitecore/content/Global Shared Content/Global Carousels|query:./ancestor-or-self::*[@@templatename='Site']/Shared Content/Site Carousels|./Page Content/Page Carousels

Now when we launch the content dialog it provides access to all three locations, no code required!

The Redirect to Parent Page Component

Prior to reading Richard’s blog post I had always solved similar issues with a unique UI component called Redirect to Parent Page. The items in question to use it on would be data source folder items like the “Assets” folder in Richard’s post, item bucket folder items generated by Sitecore, or even manually generated folders under pages (e.g. year, month, and day folders with News Mover before item buckets came out). For any of the above items’ standard values I would add a layout and apply the Redirect to Parent Page component to any placeholder. The Redirect to Parent Page component would check to see if the current item is a page and if not, loop up the hierarchy until it finds a page. E.g.

[csharp]
Item item = Sitecore.Context.Item;

// add additional checks so there’s no infinite loop – e.g. look for the root item
while(String.IsNullOrEmpty(item.Fields[Sitecore.FieldIDs.LayoutField].Value)
{
item = item.Parent;
}

Response.Redirect(LinkManager.GetItemUrl(item));
[/csharp]

So if I have a page such as http://www.mysite.com/news/2015/12/01/some-page and I trim back the URL to http://www.mysite.com/news/2015/12/01/ the site would redirect to the closest parent page which may be the news landing page (http://www.mysite.com/news/).

Not only does this work for folders in the page hierarchy, but it can also be applied to data source content items in a folder under a page like in the Assets folder mentioned in Richard’s post.

Improving the None Page Pipeline Processor

Now taking the processor approach to this, it solves the problem of needing to assign a layout and the UI component to the standard values of any non-page items. That definitely saves time. But the next challenge is that pesky little Resolve to Url checkbox. Rather than rely on a checkbox to determine this and using standard values to set it, why not just see if the item is a page? Using the same code I have above, its possible to update the processor to check if the item has any presentation set and if not, redirect to the closest parent page. That said, the best idea is not to actually redirect, because in the end redirecting is telling the browser and search engines that this was a page and is temporarily or permanently being redirected. Instead, the best solution is to send a 404 status code to the browser. This can be achieved via the following simple code split across two processors:

[csharp]

// in the same processor in Richard’s post add a custom 404 flag to the current request
HttpContext.Current.Items["404"] = true;

[/csharp]

Then in the httpRequestProcessed pipeline, patch in the following code to respond to the request and set the 404 status code if need be:

[csharp]

var status = HttpContext.Current.Items["404"];

if (status == null || !(bool)status)
return;

HttpContext.Current.Response.StatusCode = 404;
HttpContext.Current.Response.TrySkipIisCustomErrors = true;

[/csharp]

Versioned Layouts as the Foundation

The primary driving factor that allows content authors to now implement multivariate testing and personalization changes via workflow is the addition of versioned layouts. Prior to Sitecore 8, the presentation settings for a given content page were stored in a shared built-in "__Renderings" field. As a shared field, there is only one value across languages and numbered versions. This means a presentation layer change (e.g. adding a new component) to version 80 in the Spanish language of an item would also reflect on version 20 in the English language of the same item. Sitecore 8 maintains the existing shared "__Renderings" field but introduces a new versioned "__Final Renderings" field.

Presentation changes that are global and apply to all languages and numbered versions are made to the existing shared field, and new language and numbered version specific changes are applied to the new versioned "__Final Renderings" field. This is where multivariate tests come in with workflow since versions can have different values.

Now that you have a background understanding of how this is possible, let’s create some variations of content that we can test with.

Create Content Variations

Before you create your test, create a few variations of content that we will test. Ensure your solution it setup to use data sources for passing content to the UI.

Now that we have variations of content, let’s assign a test and push the changes through workflow.

Configure a Test and Submit into Workflow

Let’s now go ahead and set a presentation component to be tested and assign our content variations. In the Layout Details, select a component and click Test, then provide the variations of content. In the screenshot below I’m showing this via the Content Editor however you can also configure tests via the Experience Editor.

Set the data source for the original version and create a test variation with another data source. Like personalization, you can also hide the component via the Hide Component checkbox or swap it with another via the Enable variation of component design checkbox.

Approve, Preview and Start the Test

As an approver, let’s approve the content item with the test. This launches the Preview and start test dialog to show what the variations may do to the experience.

NOTE: if you don’t have a MV test configured on your page prior to submitting the content item through workflow, the approval state will NOT provide Approve with Test and Approve without Test actions. Since no MV test is setup, it will simply provide an Approve action which is the equivalent of Approve without Test. So make sure you configure your MV test in the draft state (or a prior state to the “with or without test approvals”).

As you can see in the screenshot above, it will run preview mode and show an image of what each experience looks like to the visitors. If you swap or hide a component you will see those changes. If you adjust content in the component you will also see that. Sitecore will also tell you how many experiences will show on the site and the estimated number of visits required to select a winner.

When you are ready, click the Start test button to finally approve the content and start the test.

If you’re interested in reading more, Sitecore has the following references docs that may be helpful as well:

• Start a Basic Content Test
• Start an Advanced Content Test

Sitecore ships with its own implementation of a synonym analyzer: Sitecore.ContentSearch.LuceneProvider.Analyzers.SynonymAnalyzer. As you can see in Adam’s post referenced above, this analyzer uses the same standard tokenizer as the standard analyzer, and also leverages the same lowercase and stop filters to rule out case and stop words from searches. The key to the synonym analyzer is providing it a list of synonyms, which need to be set in your own custom XML file. The reason for this is that Sitecore includes its own synonym engine implementation that uses XML files to store the synonym mappings.

Configuring the Synonym Analyzer

In Sitecore.ContentSearch.Lucene.DefaultIndexConfiguration.config (or you can patch the patch), change the inner defaultAnalyzer parameter reference from the standard analyzer to the synonym analyzer:

[xml]
<analyzer type="Sitecore.ContentSearch.LuceneProvider.Analyzers.PerExecutionContextAnalyzer, Sitecore.ContentSearch.LuceneProvider">
<param desc="defaultAnalyzer" type="Sitecore.ContentSearch.LuceneProvider.Analyzers.DefaultPerFieldAnalyzer, Sitecore.ContentSearch.LuceneProvider">
<param desc="defaultAnalyzer" type="Sitecore.ContentSearch.LuceneProvider.Analyzers.SynonymAnalyzer, Sitecore.ContentSearch.LuceneProvider">
[/xml]

Now unlike the standard analyzer, the synonym analyzer requires an implementation of an ISynonymEngine as its parameter:

[xml]
<param hint="engine" type="Sitecore.ContentSearch.LuceneProvider.Analyzers.XmlSynonymEngine, Sitecore.ContentSearch.LuceneProvider">
[/xml]

Sitecore’s implementation of that engine is able to read from XML files, and its requires a path to the XML file as its only parameter:

[xml]
<param hint="xmlSynonymFilePath">C:\inetpub\wwwroot\yoursite\Data\synonyms.xml</param>
[/xml]

Putting it all together:

[xml]
<analyzer type="Sitecore.ContentSearch.LuceneProvider.Analyzers.PerExecutionContextAnalyzer, Sitecore.ContentSearch.LuceneProvider">
<param desc="defaultAnalyzer" type="Sitecore.ContentSearch.LuceneProvider.Analyzers.DefaultPerFieldAnalyzer, Sitecore.ContentSearch.LuceneProvider">
<param desc="defaultAnalyzer" type="Sitecore.ContentSearch.LuceneProvider.Analyzers.SynonymAnalyzer, Sitecore.ContentSearch.LuceneProvider">
<param hint="engine" type="Sitecore.ContentSearch.LuceneProvider.Analyzers.XmlSynonymEngine, Sitecore.ContentSearch.LuceneProvider">
<param hint="xmlSynonymFilePath">C:\inetpub\wwwroot\yoursite\Data\synonyms.xml</param>
</param>
</param>
</param>
[/xml]

Defining Synonyms in XML

Now that you have the synonym analyzer configured to use your custom XML file, you need to use the following basic XML structure:

[xml]
<?xml version="1.0" encoding="utf-8" ?>
<synonyms>
<group>
<syn>fast</syn>
<syn>quick</syn>
<syn>rapid</syn>
</group>
<group>
<syn>slow</syn>
<syn>decrease</syn>
</group>
<group>
<syn>google</syn>
<syn>search</syn>
</group>
<group>
<syn>check</syn>
<syn>lookup</syn>
<syn>look</syn>
</group>
</synonyms>
[/xml]

All terms listed in the same group are synonyms of each other. So for example, if a content item has the word “quick” in its CMS content but you search for the word “rapid” you will get that content item as a result.

Understanding the Full Context of the Analyzer Configuration

If you want to understand how this little piece fits into the overarching analyzer configuration, its actually quite impressive how Sitecore has abstracted so much to be configured:

1. The overarching <analyzer> is the PerExecutionContextAnalyzer which is described on the Sitecore 7 Dev blog. Its essentially a overarching “switcher analyzer” that allows a separate analyzer to be used per context.
2. That analyzer takes a default analyzer and a mapping which can map unique analyzers per culture (also described in the same post above). If the culture matches a mapped analyzer it will use it, otherwise it will fall back to the default analyzer.
3. The default analyzer is the DefaultPerFieldAnalyzer which looks at the same config file below for field-specific analyzers in the <fieldMap> section.
4. It too has a default analyzer in the case that an individual field doesn’t have a specific analyzer set. This default analyzer is now our new SynonymAnalyzer which is the final fall back scenario.