Soledad Pano's Blog

Customizing a Sharepoint 2010 Search Center

Fri, 20 Jul 2012 21:12:54 GMT

Almost all Sharepoint sites make use of Sharepoint Search Capabilities. Sharepoint also provides OOB site templates for setting up search centers. Today I would like to show how we customized the OOB Basic Search Center site to meet the overall site’s L&F and the customer’s needs. All the customization is done programmatically in a feature event receiver so it can be automatically applied when deploying to the different environments.

We are creating the Search Center as a subsite of the main site. The OOB Basic Search Center has three pages: Default, Advanced Search and SearchResults. The customization steps consists of:

Create Search Center subsite
Apply custom master page to search center
Configure the Web Parts on the Advanced and Search Results Page
Create custom Metadata Properties, Search Scopes and Search Rules

Search Center Creation

The search center creation is done programmatically on the feature activated event:

Code Snippet

     public override void FeatureActivated(SPFeatureReceiverProperties properties)           
{           
        SPSite site = properties.Feature.Parent as SPSite;           
          
        //create search center subsite           
        using (SPWeb web = site.OpenWeb())           
        {           
            try           
            {           
                web.AllowUnsafeUpdates = true;           
                using (SPWeb existingSearchCenter = site.AllWebs["Search"])           
                {           
                    if (existingSearchCenter != null && existingSearchCenter.Exists)           
                    {           
                        existingSearchCenter.Delete();           
                    }           
                }           
                using (SPWeb searchCenter = site.AllWebs.Add("Search", "Search", "Basic Search Center", 1033, "SRCHCENTERLITE#0", false, false))           
                {           
                    //customize search center           
                    cust.CustomizeEnterpriseSearchCenter(searchCenter);           
                    searchCenter.Update();           
                    //set search center in root web:           
                    web.AllProperties["SRCH_ENH_FTR_URL"] = searchCenter.ServerRelativeUrl;           
                    //set search drowpdown mode           
                    web.AllProperties["SRCH_SITE_DROPDOWN_MODE"] = "HideScopeDD";//Do not show scopes dropdown, and default to target results page           
                    web.Update();           
                }           
            }           
            finally           
            {           
                web.AllowUnsafeUpdates = false;           
            }           
        }           
}
   

In order to redirect search results to the Search Results Page on the Search Center sub site, some settings need to be configured on the main site. Under Site Settings - Site Collection Settings - Search Settings, the Search Center is connected to main site:

This is what the following lines in the feature receiver do:

//set search center in root web:
web.AllProperties["SRCH_ENH_FTR_URL"] = searchCenter.ServerRelativeUrl;
//set search drowpdown mode
web.AllProperties["SRCH_SITE_DROPDOWN_MODE"] = "HideScopeDD";//Do not show scopes dropdown, and default to target results page

Search Master Page

The Search Center has some particularities that prevent them to use the same master page as the main site. So a custom master page needed to be provisioned. In order to provide a master page compatible with the page layouts of the OOB Basic Search Center, we follow the instructions provided in this article: Converting a Custom SharePoint 2010 Master Page into a Search Center Master Page.

Advanced Search Page customization

The requirements for the advanced search web part, were to hide the language picker, display the scopes picker (with custom scopes, like Articles, News, Audio, Video, etc) and add some custom properties to the “Add property restrictions” filter.

This customization is performed by the following code:

Code Snippet
private void EditAdvancedSearchBoxWebPart(SPLimitedWebPartManager manager)       
        {       
            System.Web.UI.WebControls.WebParts.WebPart wp = GetWebPart(manager, "Advanced Search Box");       
            if (wp != null)       
            {       
                AdvancedSearchBox webpart = wp as AdvancedSearchBox;       
                if (webpart != null)       
                {       
                    webpart.ShowLanguageOptions = false;       
                    webpart.ShowScopes = true;       
                    webpart.Properties = GetFromResources("SearchCenter.WebPartsConfig.AdvancedSearchBoxProperties.xml");       
                    manager.SaveChanges(webpart);       
                }       
            }       
        }

Even if the code is not complete, you can see what we are doing, editing the page, getting the Advances Search Box web part by name and modifying its properties. We are loading the Properties from an xml file, which includes the custom properties we added for filtering.

Custom Managed Properties and Scopes

In order to add custom properties and scopes to the Advanced Search Web Part , we first need to create them. The matadata properties are mapped to crawled properties. This means that, when Sharepoint crawls the site content and finds data in a particular list and field, if will create a crawled property for that field. For example, in the image below we can see the 0ws_keywords crawled property mapped to the Keywords Managed Property on the Admin site.

The creation and mapping of the managed property is done programmatically thru the feature activation event receiver. The crawled property is created by Sharepoint within a crawl, and must exist before activating the feature.

Here is the code for creating a managed property:

Code Snippet
private static void CreateManagedProperty(Schema schema, string managedPropertyName, string crawledPropertyCategory, string crawledPropertyName)       
        {       
            if (!schema.AllManagedProperties.Contains(managedPropertyName))       
            {              
                Category category = schema.AllCategories[crawledPropertyCategory];       
                var crawledProps = category.QueryCrawledProperties(crawledPropertyName, 1, Guid.NewGuid(), String.Empty, true).Cast<CrawledProperty>();       
                var crawledProp = crawledProps.FirstOrDefault();       
                if (crawledProp != null)       
                {       
                    ManagedDataType managedPropertyType = GetManagedPropertyType(crawledProp);       
             
                    ManagedProperty managedProperty = schema.AllManagedProperties.Create(managedPropertyName, managedPropertyType);       
                    var mappings = managedProperty.GetMappings();       
                    mappings.Add(new Mapping(crawledProp.Propset, crawledProp.Name, crawledProp.VariantType, managedProperty.PID));       
                    managedProperty.SetMappings(mappings);       
                    managedProperty.Update();       
                }       
            }       
        }

For the custom scopes, we are mostly using Property Query rules using the contentclass for the restriction:

The example of the figure is the rule of the News scope. In this case the news are stored in a list with a custom list definition. Below is the code of the list definition. Look at the Type number: 10314

Code Snippet
<?xml version="1.0" encoding="utf-8"?>       
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">       
    <ListTemplate       
        Name="NewsItemList"       
        Type="10314"         
        BaseType="0"       
        OnQuickLaunch="TRUE"       
        SecurityBits="11"       
        Sequence="410"       
        DisplayName="NewsItemList"       
        Description="My List Definition"       
        Image="/_layouts/images/itgen.png"/>       
</ Elements > 

That is the same number that we are setting the contentclass property to match: STS_ListItem_10314. This means that only items from that list, the news list, will match that scope. This enables to restrict the search to the News scope on the Advanced Search Web Part. In order for that to happen, we also need to include this scope into the Advanced Search display group.

Conclusion

The code for the complete customization is too much to explain everything here, but the idea was to show how we can make use of the existing Search Center site templates and perform different customizations depending on the project’s requirements. So with little effort we can profit of the OOB advanced search and search results pages that Sharepoint provides.

Where is my Sharepoint 2010 Custom Timer Job running?

Sun, 10 Jun 2012 02:51:27 GMT

When building a custom timer job for Sharepoint 2010, special attention should be put on where do we need this job to run. When we have a farm environment, we can choose to run the job on all servers, in one particular server, only the front end servers, etc. Depending on our requirements the timer job implementation and installation approach will change, so we should decide where we want it to run at the first place.

All Sharepoint timer jobs ultimately inherit from the SPJobDefinition class. This class provides 3 constructors:

SPJobDefinition()	Default constructor needed for serialization purposes.
SPJobDefinition(String, SPService, SPServer, SPJobLockType)	Instantiates a timer job associated with the given SPService.
SPJobDefinition(String, SPWebApplication, SPServer, SPJobLockType)	Instantiates a timer job associated with the given SPWebApplication.

The first constructor is required for serialization and is for internal use only. One of the other two constructors will be invoked from our custom timer job constructor. The parameters passed to it will define where the timer job will run.

Here is a sample code of a custom timer job definition:

[Guid("{62FF3B87-654E-41B8-B997-A1EA6720B127}")]
class MyTimerJob1 : SPJobDefinition
{
    public MyTimerJob1()
        : base()
    { }

    public MyTimerJob1(string name, SPService service, SPServer server, 
        SPJobLockType lockType) : base(name, service, server, lockType)
    { }

    public MyTimerJob1(string name, SPWebApplication webApplication, SPServer server, 
        SPJobLockType lockType) : base(name, webApplication, server, lockType)
    { }

    public override void Execute(Guid targetInstanceId)
    {
        //Execute Timer Job Tasks
    }
}

Besides the required default constructor, we need to provide at least one of the other 2 constructors. Depending on which constructor we use, the timer job definition can be associated either with a service or a web application. It can also be associated with a particular server in the farm and a lock type. So, the first thing is that for a particular server to be eligible to run the job, it must be provisioned with the service or web app associated with the job. Then, if a particular server is passed to the constructor, the job will run only on that server (if it has the associated service or web app, otherwise the job won’t run at all). If no server is associated, then it will run on one or many servers depending on the lock type.

The SPJobLockType enumeration can take one of the following values:

None: Provides no locks. The timer job runs on every machine in the farm on which the parent service is provisioned, unless the job I associated with a specified server in which case it runs on only that server (and only if the parent service is provisioned on the server).
ContentDatabase: Locks the content database. A timer job runs one time for each content database associated with the Web application.
Job: Locks the timer job so that it runs only on one machine in the farm.

So, if we instantiate a timer job passing null as the associated server and None as the lock type, we will expect it to run on every machine in the farm on which the parent service is provisioned. If we passed an SPService to the constructor, we now which service we are talking about, and now on which servers it is provisioned. But, if we passed an SPWebApplication to the constructor, in which servers will the job run? The answer is on every web font-end server, that is the servers where the Web Application service is running on.

Remember that the different server roles that we can found on a Sharepoint farm are:

Database Server: the server that hosts the Microsoft SQL Server database for the farm. Since Sharepoint Foundation is not typically installed in this server, no jobs will run here.
Web Front End Server: server where the Microsoft SharePoint Foundation Web Application service is running on.
Application Server: Any other Sharepoint server.

Here are a couple of examples on where the jobs will run depending on the parameters passed to the constructor:

//Job associated with a web app, no server in particular and none lock:
//  will run on all fron end servers.
var jobRunningOnAllFrontEndServers = new MyTimerJob1("mytimerjob", 
    SPWebApplication.Lookup(webAppURI), null, SPJobLockType.None);

//Job associated with a web app, one front end server and job lock:
//  will run only in the frontEndServer1 server.
var jobRunningOnAParticularFronEndServer = new MyTimerJob1("mytimerjob", 
    SPWebApplication.Lookup(webAppURI), fronEndServer1, SPJobLockType.Job);

//Job associated with a webApp, and an app server and lock type job: 
//  it won't run on any server since the server specified is NOT running 
//  the Web Application Service
var jobRunningOnNoServer = new MyTimerJob1("mytimerjob", 
    SPWebApplication.Lookup(webAppURI), appServer1, SPJobLockType.Job);

//Job associated with the timer service, a particular app server and none lock:
//  will run on the appServer1 server only.
var jobRunningOnAppServer = new MyTimerJob1("mytimerjob", 
    SPFarm.Local.TimerService, appServer1, SPJobLockType.None);

Using Subclases

There are some other classes on the Sharepoint Object Model that inherit from the SPServiceJob definition and can be used to inherit our custom timer jobs from. For example:

SPContentDatabaseJobDefinition: This job is executed by all WFE servers in the farm. Each content database is processed by only one job so that work is distributed across all the running jobs.
SPFirstAvailableServiceJobDefinition:
An abstract base class for a timer job that will be run on the first available server where the specified service is provisioned.
SPServerJobDefinition:This job definition is executed on a specific server within the SharePoint farm.
SPServiceJobDefinition: A timer job that runs on every server in the farm where the service is provisioned.

So, for example, if you need a job to run on all servers (including the app servers) it would be better to derive directly from the SPServiceJobDefinition class and, if you need a job to run in one particular app server, to derive from SPServerJobDefinition.

Azure Flavor for the Sharepoint Media Component

Sun, 27 May 2012 13:35:41 GMT

Some time ago I wrote about a Media Processing Component for Sharepoint that I was working on. It is a Media Assets list for Sharepoint that lets you choose where to store the blob files. It provides also intelligence for encoding videos, generating thumbnail and poster images, obtaining media metadata, etc. On that first post the component was explained in detail, with the original 3 storage flavors: Sharepoint list, Virtual Directoy or FTP. The storage manager is extensible, so a new flavor was added later that enabled to store the media files on the cloud, in particular to AmazonS3. Today I’m going to talk about a new cloud flavor based on Azure Blob Storage.

Windows Azure Blob storage is a service for storing large amounts of unstructured data that can be accessed from anywhere in the world via HTTP or HTTPS. This kind of storage is special for storing large amounts of unstructured text or binary data such as video, audio and images. You can find documentation here on how to create a Windows Azure Storage Account and access it programmatically.

For the Media Processing Component I just needed to create a class (AzureStorageManager) that implemented the IAssetStorageManager interface. The interface has the methods to save and delete files from the external storage. It also needs four configuration parameters to be passed to its constructor:

AccountName: Name of the account that will be used to authenticate to the Blob Storage
AccountKey: Key that will be used to authenticate to the Blob Storage
BlobEndpoint: URL of the Blob endpoint, like http://someaccount.blob.core.windows.net
Container: Name of a container to store your application’s media files.

For example, this is the code required to upload a file to an Azure Blob Storage:

Code Snippet AzureStorageManager class - Save method

public string Save(System.IO.FileInfo file)
{
//Create service client for credentialed access to the Blob service.
CloudBlobClient blobClient = new CloudBlobClient(blobEndpoint, new StorageCredentialsAccountAndKey(accountName, accountKey));
//Get a reference to a container, which may or may not exist.
CloudBlobContainer container = blobClient.GetContainerReference(containerAddress);
//Create a new container, if it does not exist
container.CreateIfNotExist();
//Get a reference to a blob, which may or may not exist.
CloudBlob blob = container.GetBlobReference(file.Name);
//Upload content to the blob, which will create the blob if it does not already exist.
blob.UploadFile(file.FullName);
return blob.Uri.ToString();
}

You can download the complete code for the Media Component with all the storage options from gitgub.

Sharepoint Guidance Logger: usage, setup and extension

Tue, 20 Mar 2012 02:57:33 GMT

Introduction

Log records are essential to any application for troubleshooting problems. When beginning a new Sharepoint project, one of the first needs is to have a core logging component that can be used throughout the application code. In this post I will talk about the logging solution that we are using (based on the Patterns & Practices Sharepoint Logger), how to set it up, configure and read logs.

SharePoint 2010 includes enhanced functionality for logging and tracing. You can now throttle reporting to the Windows Event Log and the Office Server Unified Logging Service (ULS) trace logs by area and by category. Areas represent broad regions of SharePoint functionality, such as Access Services, Business Connectivity Services, and Document Management Server. Within each area, categories define more specific aspects of functionality. In our solution we use a single Area for the whole project’s custom code, and different categories inside it for the different components, like Caching, UserProfile, News, Workflows, etc…

In addition to areas and categories, SharePoint 2010 makes substantial improvements to event correlation. Every trace log entry now includes a correlation ID that identifies all the events that correspond to a particular action, such as a user uploading a document. Administrators can use tools such as the ULS Viewer to filter the trace logs by correlation ID. The Sharepoint Error Page shows the user the date and correlation Id. It is important to capture this information when an error occurs, to make it easier for administrators to track for a particular issue.

The Framework

In the project I’m working on we are using a common framework for logging, which is based on the Patterns & Practices Sharepoint Logger, included in the SharePoint Guidance Library. The P&P logger provides support to write to the Windows Event log and the ULS trace logs and support for creating your own areas and categories and use them when you write to the event log or the trace logs. Two external assemblies are needed for using it: Microsoft.Practices.ServiceLocation.dll and Microsoft.Practices.SharePoint.Common.dll.

Examples of how to call the logger thru code can be found on the SP logger documentation, but just as a sample, it is something like this:

P&P Logger usage
// Log an event with a message, an event ID, a severity level, and a category.
string area = "Custom Area";
string category = "Execution";
string areaCategory = string.Format("{0}/{1}", area, category);
logger.LogToOperations(msg, (int) EventLogEventId.MissingPartnerID, EventSeverity.Error, areaCategory);

Our logging component adds a small utility layer above the P&P logger to simplify this lines a little more. For example, as we have our custom area and categories predefined, we included them in an enumeration, and use the enums as parameters to the LogToOperations method. This way developers don’t need to type the area and categories names every time, which can lead to a lot of errors. It also provides the option to add parameters to the message and then does the String.Format to combine the message with the parameters inside the utility class, this saves some repeated code to the developers as well. Regarding the log event ids, they are also set automatically by the utility class, using a different number for each category, starting with 5000.

This leads to the logger utility having two overrides of the LogToOperations method, one for logging messages and the other for logging exceptions:

Custom logger utility
public class Logger 

{ 

    private ILogger logger = SharePointServiceLocator.GetCurrent().GetInstance<ILogger>(); 

    public void LogToOperations(MyCustomCategories category, EventSeverity severity, string message, params object[] args) 

    { 

        try 

        { 

            logger.LogToOperations(String.Format(message, args), GetEventId(category), 

                    severity, category.ToLoggerString()); 

        } 

        catch 

        { 

            //don't want the app to fail because of failures in logging 

        } 

    } 

    public void LogToOperations(Exception ex, MyCustomCategories category, EventSeverity severity, string message, params object[] args) 

    { 

        try 

        { 

            logger.LogToOperations(ex, String.Format(message, args), GetEventId(category), 

                    severity, category.ToLoggerString()); 

        } 

        catch 

        { 

            //don't want the app to fail because of failures in logging 

        } 

    } 

    private int GetEventId(MyCustomCategories category) 

    { 

        return 5000 + (int)category; 

    } 

}

And this is a simple example of usage:

Usage example
static void Main(string[] args) 

{ 

    string user = "Peter John"; 

    int id = 3; 

    Logger logger = new Logger(); 

    logger.LogToOperations(MyCustomCategories.Test, Microsoft.SharePoint.Administration.EventSeverity.Verbose, 

        "Testing logging with user '{0}' and id '{1}'", user, id); 

    try 

    { 

        string firstArg = args[0]; 

    } 

    catch (Exception ex) 

    { 

        logger.LogToOperations(ex,MyCustomCategories.Test, Microsoft.SharePoint.Administration.EventSeverity.Error, "Error ocurred in test program for user '{0}' and id '{1}'", 

            user, id); 

    } 

}

Setup and Configuration

In order for all this to work, we first need to create our custom diagnostic areas and categories. We are doing it inside a feature event receiver. The code adds the custom area and then iterates thru the Categories enum to add all the associated categories. The advantage is that when a developer needs to add a category, he just adds the value to the enum and the feature will automatically add it to Sharepoint. At the moment of crating the categories, the severity level must be set. We are setting all of them as verbose which is the most useful at a development stage, but this should be changed after deployment, especially in production. A level of Warning is recommended to save disk space and increase general performance of the application.

Installer helper called by the feature event receiver
class LoggingInstallerHelper

{

    public static void AddCustomAreaAndCategories()

    {

        DiagnosticsArea customArea = new DiagnosticsArea(Areas.MyCustomArea.ToString());

        var categories = Enum.GetNames(typeof(MyCustomCategories));

        foreach (var category in categories)

        {

            customArea.DiagnosticsCategories.Add(new DiagnosticsCategory(category, EventSeverity.Verbose, TraceSeverity.Verbose));

        }

        AddArea(customArea);

    }

    public static void RemoveCustomAreaAndCategories()

    {

        RemoveArea(Areas.MyCustomArea.ToString());

    }

    #region Private Methods

    private static void AddArea(DiagnosticsArea newArea)

    {

        DiagnosticsAreaCollection areas = GetCurrentAreas();

        var existingArea = areas.FirstOrDefault(area => area.Name == newArea.Name);

        if (existingArea == null)

        {

            areas.Add(newArea);

        }

        else

        {

            int index = areas.IndexOf(existingArea);

            foreach (DiagnosticsCategory item in newArea.DiagnosticsCategories)

            {

                var existingCateg = areas[index].DiagnosticsCategories.FirstOrDefault(categ => categ.Name == item.Name);

                if (existingCateg == null)

                {

                    areas[index].DiagnosticsCategories.Add(item);

                }

            }

        }

        areas.SaveConfiguration();

    }

    private static DiagnosticsAreaCollection GetCurrentAreas()

    {

        IServiceLocator serviceLocator = SharePointServiceLocator.GetCurrent();

        IConfigManager config = serviceLocator.GetInstance<IConfigManager>();

        DiagnosticsAreaCollection areaCollection = new DiagnosticsAreaCollection(config);

        return areaCollection;

    }

    private static void RemoveArea(string areaName)

    {

        DiagnosticsAreaCollection areas = GetCurrentAreas();

        if (ExistArea(areas, areaName))

        {

            while (areas[areaName].DiagnosticsCategories.Count != 0)

            {

                areas[areaName].DiagnosticsCategories.Clear();

            }

            areas.RemoveAt(areas.IndexOf(areas[areaName]));

            areas.SaveConfiguration();

        }

    }

    private static bool ExistArea(DiagnosticsAreaCollection collection, string areaName)

    {

        foreach (DiagnosticsArea item in collection)

        {

            if (item.Name.Trim().ToUpper() == areaName.Trim().ToUpper())

                return true;

        }

        return false;

    }

    #endregion

}

To verify the proper installation of the custom area and categories and modify their severity level, you can go to the Admin Site -> Monitoring -> (Reporting) Configure diagnostic logger. In the Event Throttling section select desired area or particular categories and change the event and trace levels. Then click ok.

You can also configure the logs folder path (the default is c:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\LOGS\), storage day and size limits on this page under the trace logs section.

Viewing logs

You can see the log messages on the Event Viewer and trace logs using the ULS Viewer.

In the Event Viewer you will find the custom events under Windows Logs/ Application. You can filter by Source = MyCustomArea, by severity and/or Event ID.

You can also find trace logs in the ULS log files, and use a tool, like the ULS Viewer for a friendlier interface and filtering capabilities. The ULS Viewer is very useful for filtering by correlation id.

This are the basic tools. There are also tools for viewing log information remotely, or for aggregating log entries in a farm environment.

Sharepoint Expiration Policy not working

Sat, 17 Mar 2012 20:36:09 GMT

I have a Sharepoint 2010 list with a custom content type with an associated retention policy. The policy consists of a custom formula and then the Send to Recycle Bin action. However, I realized that the items were not being deleted. I verified the list settings and the retention policy was configured:

I run the Expiration Policy job several times but no items were deleted and no errors found in the logs. I also added logging to the custom formula but no logging was found neither. Finally, I found one thing that I didn’t know: There is another job that needs to be run before the Expiration Policy job, it is the Information Management Policy job.

The two are Sharepoint OOB jobs:

Expiration Policy: This job processes items that are due for a retention action, such as deleting items passed their expiration date.
Information Management Policy: This job performs background processing for information policies, such as calculating updated expiration dates for items with a new retention policy.

So as in this case I have a new retention policy, I needed to run this job first. When I did it, I got an error in the logs:

Error processing policy updates for site http://mysite.com for list MyList.

Error: Object reference not set to an instance of an object.

Source: Document Management Server

Event ID: 7997

Task Category: Information Policy Management

The error didn’t tell much, but after digging a little more I realized that the problem was that the custom expiration formula was not installed, so the Information Management Policy job couldn’t calculate the expiration date.

Media from Sharepoint to the Cloud

Sun, 22 Jan 2012 23:32:00 GMT

On my last post I talked about a media processing component developed for Sharepoint 2010. Besides the processing features (video encoding, thumbnail generation, validations, etc…) there was an asset storage manager that enabled us to store the files on a configurable place, inside or outside Sharepoint. I said we initially started with three storage flavors: a Sharepoint library, an FTP server or a shared folder/virtual directory.

My co-worker Dwight Goins has extended the options by adding a cloud based storage manager. In this case he uses the Amazon Simple Storage Service (Amazon S3), which resulted in a great performing storage solution. In his post, he digs deeper in the issues of storing BLOBs (Binary Large Objects) in Sharepoint and the solution that Sharepoint provides, SQL’s Remote BLOB Storage (RBS). Then he talks about our solution and the details of the AmazonS3 client. Check this out!

Media Processing Component for Sharepoint

Thu, 12 Jan 2012 16:50:00 GMT

I have been working recently on the task of building a media processing component for the Sharepoint project I am working on. The requirements for the component are more or less the following:

The Site Content Creators must be able to upload media assets (images, audio and video) to a Sharepoint list, and these assets must accomplish certain validation rules.
The final destination where the assets are saved after upload must be configurable and extensible. To begin with, we are supporting saving to a Sharepoint library, a network share or an FTP server.
The videos must be encoded to MP4 format, and thumbnail and poster images must be generated. The encoding process must be run asynchronously and the user must be notified by email when it is finished.

Today I want to share the design of the component and the key pieces of code. I will focus on the video upload process which is the more complex one because of the encoding. Audio and image uploads are more straightforward.

The main parts that build the solution are:

Custom Upload Process: This is the front end of the solution. It consists of a custom list with a custom upload form. The list has the link to the media file and more metadata fields (title, author, date, keywords, etc). When you click on create a new item on the list the custom upload form is opened and you can browse for a file to upload. The form has the required validation logic and it serves to save the assets to the configured location, which can be a Sharepoint library or an external location, like File System or FTP server. When the upload finishes you are redirected to the list item edit form so you can enter the metadata. The experience is similar to uploading a file to a Sharepoint document library.
Media Processing Backend Process: This consists of a timer job that queries the Media Assets list for items to process. It encodes the videos, generates thumbnail and poster images and uploads everything to the final destination. Finally, it notifies the user of the result of the process by email. For the video encoding we used the Microsoft Expression Encoder SDK. As I will explain later, this SDK cannot be used inside a Sharepoint process, so it runs in a separated process that is invoked from the timer job.
Storage Manager: this is a flexible and extensible component that abstracts the logic of saving (and deleting) a file to the final location depending on the flavor chosen thru configuration (File System, Sharepoint library or FTP). This component is used both by the front end upload mechanism and the back end media processing job.

Here is a diagram of the overall design for the video processing:

Now I will explain in a little more detail each component:

1. Custom Upload Process

The Media Assets List

This is a Sharepoint list that stores the metadata of the media assets, but not the asset itself (the assets are stored in the definite storage, which can be a Sharepoint assets library, a network shared folder, or an FTP server). The list is based on three custom content types, WebVideo, WebAudio and WebImage, all three inheriting from a base MediaAsset content type. This content type has the required fields for saving the asset metadata. The more important ones for the processing being:

Location: the URL of the asset in its definite location (in the example on the picture it is a sharepoint library called MediaAssetsLib).
Temp Location: As videos needs asynchronous processing, they are saved in a temporary location on upload. It is the timer job that uploads them to the definite location after encoding. The temp location is a shared folder on the network.
Processing Status: It is Success for assets successfully uploaded to the definite storage, Pending for assets waiting for encoding in the back end process and Error in case of encoding fail.

The list has an event receiver attached in order for deleting the assets from the final destination or temporary folder when the items are deleted from the list.

To achieve the storage flexibility, a custom upload form was developed and hooked to the MediaAssets list. When you click on the “Add new item” link of the picture above, the custom upload form is launched. The form is attached to the base content type definition in the Elements file as this:

<?xml version="1.0" encoding="utf-8"?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <!-- Parent ContentType: Item (0x01) -->
  <ContentType ID="0x01004e4f21afc14c487892253cb129dd5001"
               Name="MyMediaAsset" Group="MyContent Types"
               Description="My Media Asset" Inherits="TRUE"
               Version="0">
    <FieldRefs>
    …    </FieldRefs>
    <XmlDocuments>
      <XmlDocument NamespaceURI="http://schemas.microsoft.com/sharepoint/v3/contenttype/forms/url">
        <FormUrls xmlns="http://schemas.microsoft.com/sharepoint/v3/contenttype/forms/url">
          <New>_layouts/MyMedia/Upload.aspx</New>
        </FormUrls>
      </XmlDocument>
    </XmlDocuments>
  </ContentType>
</Elements>

The Upload Form

The form was created as an Application Page (Upload.aspx) in the Layouts folder. It contains the browse control to upload the file.

A useful tip here is how to achieve the same look and feel as the Sharepoint OOB forms. The InputFormSeccion, InputFormControl and ButtonSection controls were used for that matter.

In order to use these controls, you need to register the namespace on the top of the page:

<%@ Register TagPrefix="wssuc" TagName="ButtonSection" Src="/_controltemplates/ButtonSection.ascx" %>
<%@ Register TagPrefix="wssuc" TagName="InputFormSection" Src="/_controltemplates/InputFormSection.ascx" %>
<%@ Register TagPrefix="wssuc" TagName="InputFormControl" Src="/_controltemplates/InputFormControl.ascx" %>

And then include them in the page like this:

<wssuc:InputFormSection ID="InputFormSection1" runat="server"
      Title="Upload Document" Description="Browse to the media asset you intend to upload." >
  <template_inputformcontrols>
    <wssuc:InputFormControl runat="server" LabelText="" >
      <Template_Control>                   
        <div class="ms-authoringcontrols">
          Name: <br />
          <input id="FileToUpload" class="ms-fileinput" size="35" type="file" runat="server">    
          <asp:RequiredFieldValidator id="RequiredFieldValidator" runat="server" ErrorMessage="You must specify a value for the required field." ControlToValidate="FileToUpload"></asp:RequiredFieldValidator>
          <br />
          <asp:RegularExpressionValidator id="FileExtensionValidator" runat="server" ErrorMessage="Invalid file name." ControlToValidate="FileToUpload"></asp:RegularExpressionValidator>
          …                                                                          
        </div>
      </Template_Control>
    </wssuc:InputFormControl>
  </template_inputformcontrols>
</wssuc:InputFormSection>

You can read more about how to use these controls here.

So, what happens in the code behind?

On the OK button submit handler, the file to upload is processed. The logic is different depending on the asset type. Images are copied to the final destination by using the Storage Manager. A thumbnail is also generated and uploaded for them. Duration is calculated for Audio and Video files (Microsoft.WindowsAPICodePack API is used for that). Audio files are also copied to the final destination. Videos instead are leaved in a temporary storage (a network shared folder), because they need to be processed later by the timer job.

In all three cases, a list item is created with the asset metadata, and inserted into the MediaAssets list. Then the user is redirected to the list item Edit form, so he can complete filling the rest of the metadata.

Since the upload process may take a long time, all this happens in the context of an SPLongOperation. This is a Sharepoint class provided to display the “Processing…” dialog with the rotating gear image in it.

So, here is part of the code:

protected void btnOk_Click(object sender, EventArgs e)
{
    if (FileToUpload.PostedFile == null || String.IsNullOrEmpty(FileToUpload.PostedFile.FileName))
        return;     //FileToUpload is the HtmlInputFile control
    var originFileInfo = new FileInfo(FileToUpload.PostedFile.FileName);
    SPWeb web = SPContext.Current.Web;
    try
    {
        //create a MediaAsset object to save all asset metadata
        MediaAsset asset = MediaAsset.FromFile(originFileInfo, web.Url, mediaConfig);
        //start long operation to show the user the "Processing..." message
        using (SPLongOperation longOperation = new SPLongOperation(this.Page))
        {
            longOperation.Begin();

            string newFileUniqueName = String.Concat(Guid.NewGuid().ToString(), originFileInfo.Extension);

            SPSecurity.RunWithElevatedPrivileges(delegate()
            {
                //save to file system. Need to elevate privileges for that
                var tempFileInfo = new FileInfo(Path.Combine(mediaConfig.TempLocationFolder, newFileUniqueName));
                FileToUpload.PostedFile.SaveAs(tempFileInfo.FullName);

                asset.TempLocation = tempFileInfo.FullName;
                asset.Duration = MediaLengthCalculator.GetMediaLength(tempFileInfo);
                ...
            });
            
            var list = web.Lists[mediaConfig.MediaAssetsListName];
            int id;
            string contentTypeId;
            //insert new item in the MediaAssets list
            mediaRepository.Insert(list, asset, out id, out contentTypeId);
            //build url of Edit Form to redirect 
            string url = String.Format("{0}?ID={1}&ContentTypeId={2}", list.DefaultEditFormUrl, id, contentTypeId);
            //long operation ends, redirecting to the Edit Form of the new list item
            longOperation.End(url);
        }
    }
    catch (ThreadAbortException) { /* Thrown when redirected */}
    catch (Exception ex)
    {
        logger.LogToOperations(ex, Categories.Media, EventSeverity.Error,
            "Error uploading file to MediaAssets list. FileName: '{0}'.", originFileInfo.Name);

        SPUtility.TransferToErrorPage(ex.Message);
    }
}

Ok, now let’s see how the asynchronous processing part works.

2. Media Processing Backend Process

The backend process consists of a Sharepoint Timer Job that orchestrates the video processing and a console application that performs the actual encoding and generate the images. The console application is invoked by the timer job.

Encoder Console Application

The tool chosen for encoding videos was Microsoft Expression Encoder 4. We used the Pro version (paid) which includes support for H.264 (this means can encode videos to mp4 format as required by our client).

The encoder comes with an SDK, so you can programmatically encode your videos. The thing is that this API depends upon .Net Framework 4.0, and it is also 32-bit only. This is incompatible with a Sharepoint process (either web or timer job), since Sharepoint relies upon .Net 3.5 and runs in 64 bits. Hence the need to build a separate process outside of the Sharepoint Timer Job. The console application was a simple solution, and it could be configured to target .Net Framework 4.0 and x86 Platform.

This application expects four input parameters: the path to the original video, the desired path for the thumbnail image to generate, the desired path of the poster image to generate and the desired path of the encoded video to generate.

The Encoder SDK provides full flexibility for setting the encoding parameters (like format, size, bitrate, etc). It also provides a set of presets, that let’s you implement the encoding very easily. For example, here is the code for encoding using the H264VimeoSD Preset:

public void EncodeVideo(FileInfo inputFile, FileInfo outputFile)
{
    Microsoft.Expression.Encoder.MediaItem mediaItem = new MediaItem(inputFile.FullName);
    int bitrate = GetBitrate(mediaItem);

    using (Microsoft.Expression.Encoder.Job job = new Job())
    {
        job.OutputDirectory = outputFile.Directory.FullName;
        job.CreateSubfolder = false;
        job.MediaItems.Add(mediaItem);
        //H264VimeoSD preset settings: Output Format: MP4. Container: MP4. Video Codec: H.264 - Main. 
        //Video size: 640, 480. Video Bitrate: 2500 Kbps. Video Encoding: CBR SinglePass. 
        //Audio Codec: AAC. Audio Channels: Stereo. Audio Bitrate: 128 Kbps. Audio Encoding: CBR Single Pass
        job.ApplyPreset(Microsoft.Expression.Encoder.Presets.H264VimeoSD);
        job.Encode();
    }
}

And here is the code for generating the thumbnail or poster images:

public void GenerateVideoImage(FileInfo mediaFile, string imageFilePath, int width, int height)
{
    var video = new MediaItem(mediaFile.FullName);
    using (var bitmap = video.MainMediaFile.GetThumbnail(
        new TimeSpan(0, 0, 5),
        new System.Drawing.Size(width, height)))
    {
        bitmap.Save(imageFilePath, ImageFormat.Jpeg);
    }
}

Media Processing Timer Job

Sharepoint supports asynchronous processing of data through Timer Jobs. These jobs run within the context of a windows service, and are easily managed and deployed using the same tools as any other Sharepoint solution.

As the requirement was to run the job only in one of the application servers, it inherits from SPServerJobDefinition. Here is the Timer Job code:

public class MediaProcessingTimerJob: SPServerJobDefinition
{
    privateLoggerlogger = newLogger();

    publicMediaProcessingTimerJob() : base()
    {
    }

    publicMediaProcessingTimerJob(stringname,SPServer server):base(name,server)
    {
        this.Title = "MediaProcessingTimerJob";
    }

    public override voidExecute(SPJobState jobState)
    {
        stringwebUrl = String.Empty;
        try
       {
            webUrl = this.Properties["webUrl"].ToString();
            var mediaProcessor = newMediaProcessor(webUrl, MediaConfig.FromConfigRepository(webUrl));
            mediaProcessor.ProcessMedia();
        }
        catch(Exceptionex)
        {
            logger.LogToOperations(ex,Categories.Media, EventSeverity.Error, "Error executing MediaProcessingTimerJob in web '{0}'", webUrl);
        }
    }
}

During the deployment process, the MediaProcessingTimerJob is installed on the required server. The URL of the website for processing the assets are passed thru the job properties.

Here is part of the code of the helper tool that installs the job and sets it to run every 15 minutes:

private static void CreateMediaJob(string webUrl,SPServer server)
{
    var job = new MediaProcessingTimerJob("my-job-media-processing", server);
            
    job.Properties.Add("webUrl", webUrl);
            
    var schedule = new SPMinuteSchedule();
    schedule.BeginSecond = 0;
    schedule.EndSecond = 59;
    schedule.Interval = 15;
    job.Schedule = schedule;
    job.Update();
}

The logic of the timer job resides in the MediaProcessor::ProcessMedia method. It essentially queries the Media Assets List for assets in the “Pending” status and for each of these items it invokes the Encoder process and then uploads the resulting mp4 video and the generated thumbnail and poster images to the final destination. Finally it notifies the user of the result by email.

This is the code that the job uses to call the console application. Since the job tells the console application the output parameters (the path to the images and encoded video files), it doesn’t need to read any output from the console application. It must only read the standard error in case the console application fails.

private void ExecuteMediaProcess(string inVideoPath,string outThumbnailPath, string outPosterPath,string outVideoPath)
{
    string args = String.Format("\"{0}\" \"{1}\" \"{2}\" \"{3}\"", inVideoPath, outThumbnailPath, outPosterPath, outVideoPath);

    ProcessStartInfo startInfo = new ProcessStartInfo(config.EncoderExePath);
    startInfo.Arguments = args;
    startInfo.CreateNoWindow = true;
    startInfo.UseShellExecute = false;
    startInfo.RedirectStandardError = true;
    startInfo.RedirectStandardOutput = true;

    Process process = new Process();
    process.StartInfo = startInfo;
    process.Start();
    string error = process.StandardError.ReadToEnd();
    process.WaitForExit(MaxWaitingProcessMillisecs);

    if (process.ExitCode != 0)
    {
        //the application failed, get error from message from standard error
        throw new MediaProcessingException(String.Format("Video encoder process returned with exit code '{0}'. Error was: '{1}'",
            process.ExitCode, error));
    }
}

3. Storage Manager

The Storage Manager is the piece of code used by both the upload media form and the backend process. It is just a file manager used for abstracting from the actual destination of the files, which is configurable. As I said, we started supporting saving assets to File System, to a Sharepoint library or to an FTP server, but this can be further extended to support other places, like some storage on the Cloud.

The need for a flexibility in the location to store the files may come from bandwidth or space limitations, for a need to share assets with other applications or a need to manage a centralized file store.

Anyway, the interface is very simple:

public interface IAssetStorageManager
{
    void Delete(string fileUrl);
    string Save(System.IO.FileInfo file);
    string Save(string fileName, System.IO.Stream fileStream);
}

There is a factory that will give you the particular storage manager framework depending on configuration (all configuration is saved in a Sharepoint list, and the Sharepoint Config Store is used for retrieving it). Here is part of the factory code:

public class AssetStorageFactory
{    …
    static public IAssetStorageManager GetStorageManager(string configCategory,string webUrl)
    {
        var configHelper = new ConfigHelper(webUrl);
        string storageMethod = configHelper.GetValue(configCategory, StorageMethodConfigKey);
        if ("SPLibrary".Equals(storageMethod, StringComparison.InvariantCultureIgnoreCase))
        {
            return new SPLibraryAssetStorageManager(webUrl, mediaLibraryName);
        }
        else if ("FileSystem".Equals(storageMethod, StringComparison.InvariantCultureIgnoreCase))
        {
            return new FileSystemAssetStorageManager(storageFolderPath,storageBaseAddress);
        }   
        else if ("FTP".Equals(storageMethod, StringComparison.InvariantCultureIgnoreCase))
        {
            return new FTPAssetStorageManager(ftpServerUrl,ftpServerPullAdress,ftpServerUsername,ftpServerPassword);
        }
        throw new ArgumentException(String.Format("Incorrect configuration Value '{0}' in ConfigStore for category '{1}' and key '{2}'. Supported options are: '{3}'",
            storageMethod, configCategory, StorageMethodConfigKey, "FileSystem|FTP|SPLibrary"));
    }       
}

The implementation for a particular flavor is simple. For example, this is how the FTPAssetStorageManager saves a file stream:

public string Save(string fileName, System.IO.Stream fileStream)
{
    string fileUrl = ftpServerUrl + fileName;
    FtpWebRequest request = (FtpWebRequest)WebRequest.Create(fileUrl);
    request.Method = WebRequestMethods.Ftp.UploadFile;
    request.Credentials = new NetworkCredential(username, password);
    using (Stream ftpStream = request.GetRequestStream())
    {
        FileUtils.CopyStream(fileStream, ftpStream);
    }
    return pullBaseAddress + fileName;
}

And this is how the storage manager is invoked from the Upload form or the Timer Job:

//save to final location
IAssetStorageManager storage = AssetStorageFactory.GetStorageManager("Media", web.Url);
asset.Location = storage.Save(newFileUniqueName, fileInputStream);

Conclusion

Having talked about the most important parts of the Media Processing Component for Sharepoint, I think I’m done here. The code is too much to show everything in a post, but I’ve chosen the most important parts. I might dig deeper in some other post. I will probably write about how to display videos in a web page, too.

The interesting part is that even if this whole component doesn’t apply to another project, maybe some of its pieces can be reused, like the video encoding thing, or the custom upload form and storage manager to save files outside Sharepoint. So I hope it results useful to someone else!

Sharepoint Upload File Error: ‘The remote server returned an error: (409) Conflict’

Thu, 15 Dec 2011 12:51:00 GMT

I’ve been using the following code to upload a file to a Sharepoint document library using the Client Object Model:

Code Snippet

 public string Save(string fileName, Stream fileStream)
        {
            string destinationUrl = String.Concat(serverRelativeLibraryUrl, "/", fileName);
            ClientContext clientContext = new ClientContext(webUrl);
            {
                Microsoft.SharePoint.Client.File.SaveBinaryDirect(clientContext, destinationUrl, fileStream, true);
            }
 
            return destinationUrl;
        }

where

serverRelativeLibraryUrl = String.Concat(webRelativeUrl, libraryName);

But I kept getting the 409 Conflict Error:

ExceptionType: 'WebException' ExceptionMessage: 'The remote server returned an error: (409) Conflict.'   StackTrace: '
at System.Net.HttpWebRequest.GetResponse()
at Microsoft.SharePoint.Client.SPWebRequestExecutor.Execute()
at Microsoft.SharePoint.Client.File.SaveBinary(ClientContext context, String serverRelativeUrl, Stream stream, String etag, Boolean overwriteIfExists, SaveBinaryCheckMode checkMode)
at Microsoft.SharePoint.Client.File.SaveBinaryDirect(ClientContext context, String serverRelativeUrl, Stream stream, Boolean overwriteIfExists)

After struggling for a while, I figured out that the problem was the library name. It contained a dash on it, like “My-LibraryName”. When I renamed it without the dash it started working.

The error in the code is how I built the URL. I was using the library name to build the library URL, but it seems that Sharepoint doesn’t like some characters in the URL, so when you create a library called “My-LibraryName”, the URL is actually http://<server>/MyLibraryName, without the dash.

Technorati Tags: Sharepoint

Sharepoint @ Buenos Aires Codecamp 2011

Wed, 21 Sep 2011 03:44:00 GMT

BA Codecamp 2011 is coming soon and Tellago is going to be part of it! More than 10 architects from our team will be presenting the most innovative technology sessions on next October 15. I’m very happy to be speaking at this event! Along with my colleague Mariano Escurra, we will be presenting Best Practices using Javascript and JQuery in Sharepoint 2010. Bellow is the abstract. Register!!

Mejores prácticas utilizando Javascript y JQuery en SharePoint 2010

El uso de Javascript es una parte importante en la construcción de cualquier sitio web interactivo, siendo JQuery una de las bibliotecas Javascript más utilizadas. Al crear este tipo de sitios sobre SharePoint 2010, los desarrolladores pueden combinar el uso esta biblioteca con el modelo de objetos cliente Javascript (ECMAScript) mejorando notablemente la experiencia del usuario final.
Durante esta charla abordaremos los aspectos más importantes del modelo de objetos cliente ECMAScript de SharePoint 2010 y su integración con JQuery siguiendo las mejores prácticas.

Fixing Sharepoint’s “Cannot connect to the configuration database” error

Fri, 17 Jun 2011 13:34:12 GMT

I have Sharepoint Server 2010 installed in my development environment. It happened a couple of times that the computer hangs and the Sharepoint Configuration dabatase remains in a corrupted state. When I try to access the Sharepoint local site from the browser it gives the following error:

Cannot connect to the configuration database.

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.
Exception Details: Microsoft.SharePoint.WebPartPages.WebPartPageUserException: Cannot connect to the configuration database.

So, how did I fix it?

I first used this addselftosqlsysadmin.cmd script to add my account to the SQLServer sysadmin group.

Then performed the following commands from SQL Server Management Studio:

select state_desc from master.sys.databases where name = 'SharePoint_Config_5ffca14b-db6f-4b30-9dd8-8317ee0f4f45'

The result was: SUSPECT, confirming we had a problem.

Then run:

ALTER DATABASE [SharePoint_Config_5ffca14b-db6f-4b30-9dd8-8317ee0f4f45] SET EMERGENCY;

ALTER DATABASE [SharePoint_Config_5ffca14b-db6f-4b30-9dd8-8317ee0f4f45] SET SINGLE_USER;

DBCC CHECKDB ([SharePoint_Config_5ffca14b-db6f-4b30-9dd8-8317ee0f4f45] , REPAIR_ALLOW_DATA_LOSS) WITH NO_INFOMSGS, ALL_ERRORMSGS;

ALTER DATABASE [SharePoint_Config_5ffca14b-db6f-4b30-9dd8-8317ee0f4f45] SET MULTI_USER;

That’s it! I run the state_desc command again and this time it says: ONLINE

Solved!

Azure Bootcamp @MS Argentina

Thu, 31 Mar 2011 21:52:00 GMT

I participated yesterday in the Azure Bootcamp that took place at Microsoft Argentina. Along with Vanesa Guccione, my colleague at Lagash, we talked about advanced roles. The contents of the presentation included the new features of the platform, such as full IIS mode, startup tasks, remote desktop plugin and local storage. We also talked about input and internal endpoints and inter-role communication, showing some demo code as well. You can download the material from here.

How to find Visual Studio command bars

Tue, 07 Oct 2008 14:07:00 GMT

When developing a Visual Studio Addin it is a common task to add custom commands to existing menus and toolbars. It is a common problem too, not to find the proper command bar (we have to traverse all command bars to see its names, the names are not unique, etc). In these two posts: Using IVsProfferCommands to retrieve a Visual Studio CommandBar and Using EnableVSIPLogging to identify menus and commands with VS 2005 + SP1 it is very well explained how to solve this problem.

The answer relies on the fact that every toolbar and menu is uniquely identified in Visual Studio by a GUID,Id pair. In order to see which is the GUID, Id pair for a given command, you must:

Add or change the registry key HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\9.0\General\EnableVSIPLogging to 1.

Click on the toolbar or menu you want identify while keeping CTRL+SHIFT pressed. This will show a dialog with the command bar properties. Take note of the Guid and the CmdID:

From the Addin, use the following code to add your command in the desired command bar:

private void AddMyCustomCommand()

{

    …

    object[] contextGuids = new object[0];

    myCommand = ((Commands2)this._applicationObject.Commands).AddNamedCommand2(this._addInInstance,

             partialCommandName,displayName, tooltip, true, null, ref contextGuids,

            (int)vsCommandStatus.vsCommandStatusSupported + (int)vsCommandStatus.vsCommandStatusEnabled,

            (int)vsCommandStyle.vsCommandStyleText,vsCommandControlType.vsCommandControlTypeButton);

    CommandBar ownerBar = FindCommandBar(new Guid("{9AEB9524-82C6-40B9-9285-8D85D3DBD4C4}"), 1280);

    myControl = workflowCommand.AddControl(ownerBar, 1) as CommandBarButton;

}

private CommandBar FindCommandBar(Guid guidCmdGroup, uint menuID)

{

   // Retrieve IVsProfferComands via DTE's IOleServiceProvider interface

   IOleServiceProvider sp = (IOleServiceProvider)_applicationObject;

   Guid guidSvc = typeof(IVsProfferCommands).GUID;

   Object objService;

   sp.QueryService(ref guidSvc, ref guidSvc, out objService);

   IVsProfferCommands vsProfferCmds = (IVsProfferCommands)objService;

   return vsProfferCmds.FindCommandBar(IntPtr.Zero, ref guidCmdGroup, menuID) as CommandBar;

}

[ComImport,Guid("6D5140C1-7436-11CE-8034-00AA006009FA"),

InterfaceTypeAttribute(ComInterfaceType.InterfaceIsIUnknown)]

internal interface IOleServiceProvider

{

    [PreserveSig]

    int QueryService([In]ref Guid guidService, [In]ref Guid riid,

       [MarshalAs(UnmanagedType.Interface)] out System.Object obj);

}

CodeCamp Buenos Aires 2008

Sun, 05 Oct 2008 14:13:18 GMT

I attended yesterday to the Microsoft Code Camp event at Buenos Aires. It was fun!

Here's a picture of the Lagash stand:

UI Automation Article Published on Level Extreme

Wed, 12 Mar 2008 17:38:00 GMT

On this month's edition of the Level Extreme .Net Magazine, my article about Building an UI Automation Client Application was published. The article walks through building a client application for automating a win 32 target application (the Windows Address Book). The idea is to show through simple code snippets how to use the UI Automation API for manipulating a target application programmatically through its UI. It shows how to find the UI elements, and how to work with the UI Automation patterns and handle events. The sample client application code contains further examples and can be downloaded from the article's page. Hope you read it!

Read more on the magazine from Martin Salias.

CCF: Listening to External Application Events

Tue, 15 Jan 2008 15:05:00 GMT

It is often necessary to listen to external application UI events when automating applications in CCF. For example, we may need to save a value in the Context if the user presses a button, or do something when an alert dialog appears. There are many ways we can handle events. In CCF 2008, the Hosted Application Toolkit (HAT) provides an easy way to listen to them. If not using HAT we can hook to events using the UIAutomation or MSAA (Microsoft Active Accessibility) APIs within our own application adapters.

Events in HAT

When we configure a hosted application for using an Automation Adapter, this adapter listens to the application events. We can hook and unhook to these events using the CCF's WF activities RegisterActionForEvent (see figure bellow) and UnregisterActionForEvent respectively.

It is the Data Driven Adapter (DDA) the one who throws the ControlChanged event (to which the activity subscribes). The arguments of the event consists of eventTypeName, controlName and controlValue.

For example, the WinDataDrivenAdapter that CCF provides supports the following event types:

SetControlValue: When the DDA set's a control's value.
ExecuteControlAction: When the DDA executes a control.
ContextChanged: When the context changes.
MenuShown: When a menu is shown
WindowShown: When a window is shown
WindowDisappeared: When a window is closed
LostFocus
SetFocus
CheckBoxSet
CheckBoxCleared
RadioButtonSet
RadioButtonCleared
ButtonPressed
ButtonReleased

There are some tricks about how to use this events, for example:

4-6: For menu and window events we must not register with the control's friendly name that we set in the databindings section of the AppInitString. We must specify the window or menu's caption instead.
7-14: For these events the related control must be previously found on the UI. You can use the FindControl activity to ensure that the control is successfully found. When the DDA finds the control it registers it in a KnownControls collection. Only for those registered controls the events are thrown.

Custom DDAs or Legacy Adapters

If we need to handle events from a custom DDA or from a legacy adapter we can use win32 or accessibility APIs. If we use a custom DDA we can throw a ControlChanged event in order to use the WF/Automation activities mentioned before.

WinEvents

A low level approach is to use the SetWinEventHook and UnhookWinEvent functions from user32.dll. When we hook for events we specify the callback function that will handle the events. You can see the list of supported events and its constants in the MSAA SDK documentation on msdn.

In order to show some sample code, here are the user32 functions imports:

[Flags]

internal enum SetWinEventHookFlags

{

WINEVENT_INCONTEXT = 4,

WINEVENT_OUTOFCONTEXT = 0,

WINEVENT_SKIPOWNPROCESS = 2,

WINEVENT_SKIPOWNTHREAD = 1

}

internal delegate void WinEventProc(IntPtr hWinEventHook, int iEvent, IntPtr hWnd, int idObject, int idChild, int dwEventThread, int dwmsEventTime);
[DllImport("user32.dll", SetLastError = true)]
internal static extern IntPtr SetWinEventHook(int eventMin, int eventMax, IntPtr hmodWinEventProc, WinEventProc lpfnWinEventProc, int idProcess, int idThread, SetWinEventHookFlags dwflags);
[DllImport("user32.dll", SetLastError = true)]
internal static extern int UnhookWinEvent(IntPtr hWinEventHook);

Here's how we can subscribe to win events from our custom adapter:

WinEventProc wep = new WinEventProc(this.EventCallback);

SetWinEventHook(1, 0x7fffffff, IntPtr.Zero, wep, 0, threadId, SetWinEventHookFlags.WINEVENT_OUTOFCONTEXT);

And finally how to react to events in the event handler method:

private void EventCallback(IntPtr hWinEventHook, int iEvent, IntPtr hWnd, int idObject, int idChild, int dwEventThread, int dwmsEventTime)

{

//get accObj from handler

//get accObj's name, role, etc...

...

switch (iEvent)

{

case 0x10: //EVENT_SYSTEM_DIALOGSTART

if (role.Equals(localize.ACC_ROLE_TEXT_WINDOW))

{

if (name.Length > 0)

{

//throw ControlChangedEvent

this.ControlChanged(accObj, new ControlChangedEventArgs("DialogStart", controlName, string.Empty));

}

break;

...

}

UIAutomation

Another option is to use the UIAutomation accessibility API, included on .Net framework 3.0 to hook to events from custom adapters. The Automation element provides the following static methods to hook to different event on different controls:

//Registers a method that handles UI Automation events.

public static void AddAutomationEventHandler(AutomationEvent eventId, AutomationElement element, TreeScope scope, AutomationEventHandler eventHandler);

//Registers a method that will handle focus-changed events.

public static void AddAutomationFocusChangedEventHandler(AutomationFocusChangedEventHandler eventHandler);

//Registers a method that will handle property-changed events.

public static void AddAutomationPropertyChangedEventHandler(AutomationElement element, TreeScope scope, AutomationPropertyChangedEventHandler eventHandler, params AutomationProperty[] properties);

//Registers the method that will handle structure-changed events.

public static void AddStructureChangedEventHandler(AutomationElement element, TreeScope scope, StructureChangedEventHandler eventHandler);

The UIAutomation API is much more friendly to use, but some times it takes too long for the event to reach the handler. You should be very precise when setting the element and scope to subscribe the event to. You can find more information about this API and sample code on my previous post about WPF Accessibility.

Hosting a Window Form Control in a SharePoint WebPart

Fri, 07 Dec 2007 01:48:00 GMT

Some months ago I posted about Hosting a Windows Form Control in a web page. I explained there how we can run a WinForm control from Internet Explorer by hosting it in a web page using the <object> tag. Now suppose you want to use the same solution in a custom web part that is included in a Sharepoint site. I will describe here where the control library should be placed and how it can be referenced from the web part.

When we develop a custom web part, we place its resources under the _wpresources web site, into a folder named <WebPartAssemblyName>/<WebPartAssemblyVersion>__<PublicKeyToken>. You can create there a subfolder (named "bin" por example) to place the win control assembly. This folder must have execute permissions set to Script Only (not Script and Executables as is the default under a SharePoint site). The picture bellow shows the resulting structure when placing the win form control assembly in a web part called MyCustomWebPart that is part of the MyCustomWebParts assembly:

Once the control assembly is placed in the correct place, the web part can render the <object> tag within the RenderWebPart method:

namespace MyCustomWebParts
{
  [XmlRoot(Namespace = "MyCustomWebParts")]
  public class MyCustomWebPart : WebPart
  {

protected override void RenderWebPart(HtmlTextWriter output)

{

EnsureChildControls();

string resourcePath = SPWebPartManager.GetClassResourcePath(SPContext.Current.Web, GetType());

output.Write( ReplaceTokens("<h1>Windows Form Control:</h1>" +

"<object id='MyWinControl' classid='" + resourcePath +

"/bin/WindowsControlLibrary1.dll#WindowsControlLibrary1.UserControl1'" +

"height='300' width='800' VIEWASTEXT>" +

" <PARAM name='Prop1Str' value='blabla' valuetype='data'>" +

" <PARAM name='Prop2Int' value='2' valuetype='data'>" +

" </object>"));

base.RenderWebPart(output);

}

CCF HAT - Making use of DDAs from Legacy Adapters

Wed, 31 Oct 2007 01:34:00 GMT

Although Data Driven Adapters (DDAs) are mainly designed for Automation Adapters in HAT, we can make use of them from legacy adapters, too. Here's an example of an external VB application, which is configured to use a legacy application adapter and a WinDataDrivenAdapter with the corresponding bindings:

<?xml version="1.0" encoding="utf-16"?>

<URL>C:\MyApp.exe</URL>

</interopAssembly>

<UseTopLevelWindow class="ThunderRT6Form" />

<URL>C:\MyApplicationAdapters.dll</URL>

<type>Microsoft.Ccf.QuickStarts.MyExtVBAppAdapter</type>

</adapter>

<Type>Microsoft.Ccf.HostedApplicationToolkit.DataDrivenAdapter.WinDataDrivenAdapter,

Microsoft.Ccf.HostedApplicationToolkit.DataDrivenAdapter</Type>

<Path>

</Path>

</AccControl>

</Controls>

</DataDrivenAdapterBindings>

</initstring>

One thing to take into account is that, if the application uses an alternative top level window, the DDA must be instantiated with that top level window instead of the process MainWindowHandle. The picture shows how the alternate top level window configuration is seen in the Admin Console:

And here is the code of the legacy adapter making use of the data driven adapter to automate the hosted application's UI:

using System;
using Microsoft.Ccf.Csr;
using Microsoft.Ccf.HostedApplicationToolkit.DataDrivenAdapter;
namespace Microsoft.Ccf.BancoGalicia.ApplicationAdapters
{
    public class MyExtVBAppAdapter : ApplicationAdapter
    {
        protected DataDrivenAdapterBase _dda;
        public override bool Initialize()
        {
            //IntPtr topLevelWnd = Process.MainWindowHandle;//this won't work for a VB app 
            IntPtr topLevelWnd = GetTopLevelWndHandle();
            //Initialize Data Driven Adapter with proper top level window handle
            _dda = DataDrivenAdapterBase.CreateInstance(ApplicationInitString, topLevelWnd);
            return _dda != null;
        }
        public override bool DoAction(Action action, RequestActionEventArgs args)
        {
            switch (args.Action)
            {
                case "DefaultAction":
                    break;
                case "PressButton":
                    PressButton();
                    break;
            }
            return base.DoAction(action, args);
        }
        private void PressButton()
        {
            if (_dda.FindControl("button1"))
            {
                _dda.ExecuteControlAction("button1");
            }
        }
        //Find the top level window of the VB application (for example of class "ThunderRT6Form")
        private IntPtr GetTopLevelWndHandle()
        {
            IntPtr agentDesktopPtr = Win32API.FindWindow(null, "Agent Desktio");
            if (agentDesktopPtr != IntPtr.Zero)
            {
                IntPtr tabWindowHandle = Win32API.FindWindowByText(agentDesktopPtr, "MyApp");
                if (tabWindowHandle != IntPtr.Zero)
                {
                    return Win32API.FindWindowByText(tabWindowHandle, "MyAppTopLevelWndCaption");
                }
            }
            return IntPtr.Zero;
        }
    }
}  

CCF 2008 Hosted Application Toolkit

Fri, 26 Oct 2007 03:40:00 GMT

The Hosted Application Toolkit (HAT) is a new component of CCF 2.6 (2008) that facilitates the task of automating the UI of the hosted applications. It consists of two main parts: the Data Driven Adapters (DDA) and the Automations (WF workflows) that automates the hosted application using the DDAs.

Data Driven Adapters

DDAs must inherit from DataDrivenAdapterBase in Microsoft.Ccf.HostedApplicationToolkit. CCF supports two implementations out of the box: WinDataDrivenAdapter and WebDataDrivenAdapter. We can inherit from any of the three to extend the DDA as necessary. The methods the DDAs expose to manipulate the hosted application UI are ExecuteControlAction, FindControl, GetControlValue and SetControlValue. These methods take a friendly name as parameter and lets the application adapters interact with the hosted application UI abstracting from the details of how to obtain the control from that friendly name. The mapping between the controls' friendly names and the directions for the DDA to finding them is called Data Driven Adapter Binding and is part of the hosted app initialization string:

<DataDrivenAdapterBindings>
    <Type>Microsoft.Ccf.HostedApplicationToolkit.DataDrivenAdapter.WinDataDrivenAdapter</Type>
    <Controls>
        <AccControl name="okButton">
            <Path>
                <FindWindow>
                    <ControlID>15</ControlID>
                </FindWindow>
            </Path>
        </AccControl>
        <AccControl name="cancelButton">
            <Path>
                <FindWindow>
                    <ControlID>15</ControlID>
                </FindWindow>
            </Path>
        </AccControl>
    </Controls>
</DataDrivenAdapterBindings>

Note: It is a known issue that the initialization string of a hosted application is limited to 3902 characters. This may result a problem if several controls must be listed.

Although DDAs can be used from any custom adapter (and even outside CCF), they were mainly thought to support Automations in HAT.

Automations

In previous versions of CCF we could choose to use an Application Adapter to implement the actions' logic. In version 2.6 we can choose between Automation Adapter, Legacy Adapter, or no adapter at all. The Legacy Adapter is the Application Adapter from the previous versions. We can implement our own Legacy Adapter inheriting from the [Web]ApplicationAdapter class. The Automation Adapter is the adapter that makes use of the new Workflow Foundation (WF) feature. The AutomationAdapter implements the IAutomationAdapter interface. However, if we choose AutomationAdapter in the hosted app configuration in the AdminConsole, we can't choose our own implementation of IAutomationAdapter.

If a hosted application is configured to use the AutomationAdapter, we can specify a WF workflow to implement an action's logic. Workflows in HAT are called Automations.

In order to orchestrate an action automation, a set of WF Activities are provided by CCF that interact with the DDAs to access the UI controls. There are activities for calling an action in a hosted application, setting or getting a control's value, accessing the context, register for an application event, etc.

All together

The connection between the Automations and the DDAs is achieved by the AutomationBridgeService, which is in turn connected to the AutomationAdapter. As the bridge service is added to the WF runtime, it is accessible from the CCF WF activities. We can extend the set of activities and make use of this server, too.

The following diagram depicts the relationship between the components mentioned above:

More detailed documentation can be found in chapter 9 of the CCF 2008 Development Guide.

Silverlight article published this month

Mon, 08 Oct 2007 02:23:00 GMT

My colleagues Ariel Neisen, Federico García, Rodolfo Finochietti and I wrote the cover article of the #42 edition of the .Code Magazine (in Spanish) about Silverlight. The article talks about this new technology's fundamentals, architecture, programming tools, advanced features, code samples and a lot more.

Choosing the right WCF binding

Tue, 02 Oct 2007 02:43:00 GMT

A WCF binding is the endpoint component that defines how the client needs to communicate with the service. It groups settings such as underlying transport protocol, security requirements, and message encoding.

WCF provides nine built-in bindings:

BasicHttpBinding: Basic web service communication. Exposes WCF services as legacy ASMX web services. Used for interoperability. No security by default.
WSHttpBinding: Web services with WS-* support. Supports transactions and reliable messaging.
WSDualHttpBinding: Web services with duplex contract and transaction support.
WSFederationHttpBinding: Web services with federated security. Supports transactions.
MsmqIntegrationBinding: Communication directly with MSMQ applications. Supports transactions.
NetMsmqBinding: Communication between WCF applications by using queuing. Supports transactions.
NetNamedPipeBinding: Communication between WCF applications on same computer. Supports duplex contracts and transactions.
NetPeerTcpBinding: Communication between computers across peer-to-peer services. Supports duplex contracts.
NetTcpBinding: Communication between WCF applications across computers. Supports duplex contracts and transactions.

Although most bindings will work on scenarios they are not designed for, it's a good practice to choose the right binding for a given endpoint. In chapter one of the "Programming WCF Services" book, Juval Lowy provides a very useful decision-activity diagram for choosing the right binding:

"The first question you should ask yourself is whether your service needs to interact with non-WCF clients. If the answer is yes, and if the client is a legacy MSMQ client, choose the MsmqIntegrationBinding that enables your service to interoperate over MSMQ with such a client. If you need to interoperate with a non-WCF client and that client expects basic web service protocol (ASMX web services), choose the BasicHttpBinding, which exposes your WCF service to the outside world as if it were an ASMX web service (that is, a WSI-basic profile). The downside is that you cannot take advantage of most of the modern WS-* protocols. However, if the non-WCF client can understand these standards, choose one of the WS bindings, such as WSHttpBinding, WSFederationHttpBinding, or WSDualHttpBinding. If you can assume that the client is a WCF client, yet it requires offline or disconnected interaction, choose the NetMsmqBinding that uses MSMQ for transporting the messages. If the client requires connected communication, but could be calling across machine boundaries, choose the NetTcpBinding that communicates over TCP. If the client is on the same machine as the service, choose the NetNamedPipeBinding that uses named pipes to maximize performance. You may fine-tune binding selections based on additional criteria such as the need for callbacks (WSDualHttpBinding) or federated security (WSFederationHttpBinding)."

You can download the book's sample chapter to read more on this subject. You would also find the diagram in Juval's article WCF Essentials - A Developer's Primer on the CoDe Magazine.