• Part 1 - Architecture and Logical Planning
• Part 2 - Installation
• Part 3 - Development Environment
• Part 4 - Backup and Recovery Strategy
• Part 5 - Virtualization
• Part 6 - Post Deployment (final)

Post Deployment

Manage all SharePoint ports

There are about 20 different ports SharePoint use and they need to be open across firewalls. Refer to “Planning an Extranet Environment for Office SharePoint Server” document for more details. It can be easier to have firewall disabled when configuring SharePoint, and then close all non-SharePoint ports when configuration is done.

Monitor how SharePoint operates

After finishing deployment the actual work for administrators and infrastructure guys starts. They need to monitor the health of SharePoint servers, status of web application and statistic of web sites usage, the growth of content database.

Microsoft recommends using “System Center Operations Manager 2007″ with SharePoint Monitoring Toolkit http://technet.microsoft.com/en-us/library/bb975149.aspx to monitor SharePoint farms. The only drawback is that it’s enterprise level tool and might not be an ideal solution for small SharePoint farms.

On the other hand “SharePoint Diagnostic Tool” is suitable for small farms that simplify the process of analyzing logs for troubleshooting, and helps significantly reduce the time to diagnose issues.

The commercial sector of 3rd party management tools for SharePoint is represented by Quest Manager, DocAve and others. Quest Manager is one of the best tools that provide detailed information about how SharePoint operates, visual statistic across web application and sites (see screenshot):

There are several free tools to manage large and small SharePoint farms with the review published on SharePointReview site: http://tinyurl.com/9p6j7e. One of the good tools is SharePoint Analyzer from Bamboo solution which provides flexibility to monitor status and diagnose errors.

To diagnose ULS logs the “SharePoint ULS Viewer” provides the most powerful features to sort, filter and search across ULS logs in different categories.

Add Central Administration to Trust/Local Zone

Windows 2008 enhances the security of the internet sites and in result Central Administration site does not play nice with the default internet settings - some links of Central Administration sections are hidden. The same behaviour exists when Central Administration site is opened under non-administrative account.
To fix this, navigate to the Internet Explorer Options -> Security Tab and “Central Administration” site to Trust zone (or Intranet Sites).

Manage size of Content DB

SharePoint performs the best on the content databases below 100GB. This is not a true limitation but rather a recommendation. SQL Server databases have been scaling far beyond 100GB for years now. Practically speaking, the recommendation is based primarily on two significant factors:
1) Service Level Agreement (SLA) requirements for a given organization may dictate that backup operations for the SharePoint databases must be executable in a limited amount of time. The size of the content databases will have a direct impact on how long it takes to execute that backup.
2) The storage subsystem must be robust enough to handle the disk I/O requirements of the SharePoint solution that it serves.
So, monitor how much database grows and create new database content after such threshold. The best way to achieve this is setting alerts via DBA.

Set SQL TempDB Data Files

It is a product team-recommended practice that the number and size of data files allocated for the SQL TempDB Data Files should be equal to the number and size of core CPUs presented on the SQL Server machine. Each data file should be equal in size. Optimal TempDB data file sizes can be calculated using the following formula:

[MAX DB SIZE (KB)] X [.25] / [# CORES] = DATA FILE SIZE (KB)

So, for the server with two quad-core CPUs and the 500 GB content DB there will be 8 files with 15 GB size per temp data file.

If possible, separate each data file to separate logical unit consisting of unique physical disk spindles

Maintain SQL Database

SharePoint databases are usually big, very big, so maintaining databases in primary task to provide the good performance (http://technet.microsoft.com/en-us/library/cc262731.aspx) Consider the following tasks:

• Checking database integrity.
• Defragmenting indexes by either reorganizing them or rebuilding them.
• Setting the fill factor for a server.
• Shrinking databases to recover unused disk space.

Create SQL Maintenance plan to schedule those operation being run automatically.

Manage security and permissions

For each site collection, there is a limited number of security principals (users and groups) that can be applied - 2000 per Web site (limitation of ACL). Use Active Directory security groups and SharePoint groups to manage access, instead of adding users individually. The more fine-grained permissions are applied, the more difficult to track who has access to what. Moreover, fine-grained permissions can affect performance because additional security checks must be performed for each item to which they are applied.

Remove unused and “ghost” users

During farm installation, configuring or migrating, SharePoint farm is usually end up with number of test users or permissions that were used for test reasons, or “zombifies” users - which survived migration and not assigned to nobody. It’s important to remove all these users and check  the permissions to lower the chance of security breach.
There are several tools that address current task - Access Checker, SharePoint SUSHI and “Deliver Point” which provide the best functionality to find “dead” users easily and check permissions hierarchy across all site collections and sites.

Plan windows updates

In a server farm environment software updates are not installed automatically, even if the Automatic Updates feature is enabled on Web servers. Windows Update Web site or the Microsoft Update Web site can’t be used for software update installation (http://technet.microsoft.com/en-us/library/cc263467.aspx). In this case planning of updates across SharePoint farm boxes is required. Recommendation is to test all updates properly on testing/staging environment with existed applications before propagating updates to production environment.

Adopt farm to number of users

If the environment serves approximately 5000 or more users, consider deploying a minimum of three front-end Web servers, or the number of front-end Web servers that are required to provide appropriate capacity plus one more. That way, when one front-end Web server is experiencing occasional high load, load balancers can route the traffic to other servers. In the case when there is only one remaining front-end Web server, this can cause a cascading outage in which the extra traffic that affected the first front-end Web server is transferred to the second front-end Web server, which brings it down too. Having an extra front-end Web server helps the farm to consume such loads and provide more performance and stability in the farm during such situations. Additionally, consider dedicating front-end Web servers to be configured for crawling reasons only. There front-end Web servers should be physical servers, not virtual.

Optimize index server

SharePoint crawls data incrementally, and index is provisioned the same way - by small portions. Usually new crawled data become available in 3-30 second. If new added file is not discoverable in such timeframe, it means that Index Server suffers from performance issues. Use USL logs and performance counters to investigate what is wrong. In virtualized environment use physical HDD or move Index Role to physical box.

Summary

Installing and configuring SharePoint farms never been a trivial task and usually characterized with number of different issues, which could lead to the reinstalling the farms from the scratch several times. Having proper plan and recommendation for the major milestones in the configuration provides benefits to have work done from the first attempt, and end up with the scalable and reliable environment prepared for the further extensibility.

Post Scriptum

I’d like to thank Darren Neimke, Jeremy Thake, Alex Meleta for their notes and recommendation regarding several chapters, and especially to Eric Shupps for noticing several errors and constructive criticism.  Special thanks to Neville Mehta for the inspiration to create and finish this whitepaper.

This article was posted on:

SharePoint Farm configuring and deployment Part 6 - Post Deployment

• Part 1 - Architecture and Logical Planning
• Part 2 - Installation
• Part 3 - Development Environment
• Part 4 - Backup and Recovery Strategy
• Part 5 - Virtualization
• Part 6 - Post Deployment (final)

Virtualization

Virtualization in SharePoint farm is one of the key design factors that simplify server availability by providing number of additional servers that might not be available over physical server models, or solution become very expensive. Microsoft officially supports SharePoint farm in virtualized environment since mid 2007. The following virtualizations technologies are supported: Hyper-V, Virtual PC, Virtual Server and 3rd party providers like VMware.

One of the key factors for virtualization is that performance of virtualized farm is competitive to the physical farm. Microsoft tests shows:
- 7.2% less throughput on virtual Web roles with 8GB of RAM than a physical Web role server with 32GB of RAM;
- 4.4% slower in the page response time on the Hyper-V Web front-end than the physical server;

Virtualize SharePoint Web Role

Web front-end servers are responsible for the content rendering and have comparatively lower memory requirements and disk activity than other roles, what makes them is an ideal candidate for virtualization.

Choose disk type for Query Role

The query role is responsible for a search performed by users is a good candidate for virtualization. The disk type choice for this role depends on the size of propagated index and the rate of index propagation.
The recommendation for the large indexes and the farm with the high rate of the updated information to use a physical disk volume that is dedicated for the individual query server, rather than a virtual disk file.

Consider using Index Role on physical server

The Index server role in a SharePoint farm is often the most memory-intensive role, what makes it less ideal candidate for virtualization. Virtualized Index server role might be appropriate for development environment, small farm or farm with small content usage.
Take into account, that index can vary from 10% to 30% of the total size of the documents being indexed. For the large indexes (above 200 GB) consider using physical disk volume that is dedicated to the individual query server, rather than virtual disk.
For large farms with big amount of crawled data use physical Index server role due to large memory requirements and high disk I/O activity.

Do not virtualize Database role

SharePoint database role is the least appropriate role for virtualization in production scenarios, mainly due to the highest amount of disk I/O activity and very high memory and processor requirements. However, it is very common to see the SQL Server virtualized in test farms, quality assurance (QA) farms, and smaller SharePoint environments.

Do I need to virtualize Application role?

The decision of virtualizations the application roles, such Excel Services and InfoPath Services, depends on the roles usage. Those roles can be easily virtualized, because they are similar to Web Roles and mostly CPU intensive. When necessary, those servers can be easily moved to dedicated physical servers.

Virtualized scenario sample

The following picture demonstrates the common virtualized scenario of SharePoint Farm.

Common deployment scenarios for the SQL role in a SharePoint farm may have multiple farms, both physical and virtual, use a single database server or database cluster, further increasing the amount of resources consumed by the role. For example, in the picture above, the sample SharePoint environment illustrated maintains a two-server SQL cluster that is used by several virtual farms and one production farm.

Use proper number of CPU

Do not use more virtual CPUs than physical CPUs on the virtual host computer - this will cause performance issues, because the hypervisor software has to swap out CPU contexts.
The best performance can be realized if the number of virtual processors allocated to running guests does not exceed the number of logical processors (physical processors multiplied by the number of cores) on the host. For example, a four processor quad-core server will be able to allocate up to 16 virtual processors across its running sessions without any significant performance impact. Note that this only applies to sessions that are physically running simultaneously.

Use proper amount of RAM

Plan to allocate the memory on virtual sessions according the next rule - divide the total amount of RAM in the server by the number of logical processors (physical processors multiplied by number of cores) in the host server. This will align allocated memory along with NUMA sessions. Otherwise it will provide performance issues.
In some testing, a virtual SharePoint Web server role with an allocation of 32GB of RAM actually performed worse than a virtual server with an allocation of 8GB of RAM.

Plan to use physical drives

In virtual scenarios front-end Web servers or Query servers disk performance is not as important as it would be physicals servers of the Index role or a SQL Server database. A fixed-size virtual disk typical provides better performance than a dynamically-sized disk.
If disk speed is a high priority, consider adding physical drives to the host computer. Add new virtual hard drive and map it to an unused physical drive on the host. This configuration, called a “pass-through disk”, is likely to give the best overall disk throughput.

Consider using hardware load balancing

Hardware load balancing provides the best performance, comparing with the software load balancing. It offloads CPU and I/O pressure from the WFE’s to hardware layer thereby improving availability of resources to SharePoint. Examples of Hardware: F5 BIG IP, Citrix Netscaler, Cisco Content Switch. Software load balancing examples are Windows Network load balancing, Round Robin load balancing with DNS. It is a trade-off between cost and performance. (Additional details are here)

Be careful with snapshot feature on virtual servers

Using snapshots for the backup might cause you troubles, because SharePoint timer service might be  unsynchronized during the snapshot process, and once the snapshot is finished, errors or inconsistencies can arise. So, consider backups over the snapshots for the production environments.

Measure virtualized environment performance

After you complete your virtualized environment installation and configuration measure how fast you environment operates and optimize it to the best performance. Measure the following parameters:
1) Processor performance - “\Hyper-V Hypervisor Logical Processor(_Total)\% Total Run Time” performance monitor counter.
Results: <60% Utilization is fine, 60%-89% - caution, > 90% significant performance degradation
2) Memory performance - “\Memory\Available Mbytes” of the physical memory available to processes running on the computer, as a percentage of physical memory installed on the computer.
Results: 50% is fine, 10% and below is critical
3) Disk performance - “\Logical Disk(*)\Avg. Disk sec/Read” or “\Logical Disk(*)\Avg. Disk sec/Write” disk latency on the Hyper-V host operating system.
Results: up to 15ms is fine, 15ms-25ms is warning, >26ms critical

Mode details about virtualized performance counters can be found in this post

Refer to the following document for more details regarding virtualization: http://technet.microsoft.com/en-us/library/cc816955.aspx

This article was posted on:

SharePoint Farm configuring and deployment. Part 5 - Virtualization

For your reference, these are our previous six articles:

1. Part 1, BDC Purpose and technical architecture
2. Part 2, Application Definition File (ADF) and its Development
3. Part 3, Developing an ADF to Connect to Web Services
4. Part 4, Consuming Business Data through Web Parts and SharePoint Lists
5. Part 5, Implementing Enterprise Search with Business Data
6. Part 6, Integrating User Profiles with Business Data

Until now, we have been working strictly with the out-of-the-box features and capabilities of BDC. MOSS Enterprise has also provided us another powerful approach for working with BDC, which is to employ the BDC Object Model and programmatically interact with a LOB system. This gives us as developers the power and control over BDC to further meet growing business needs.

You might be wondering why we use the BDC Object Model when you can just use ADO.NET. The reason is two-fold: 1) you already have BDC in place that has already abstracted the complexity of the business data and you want to maximize your investments, and 2) BDC takes care of authentication, authorization, and other security requirements, which is extremely helpful if you are writing a custom application that calls into a back-end system. So this makes the BDC Object Model a compelling API for us.

Using the BDC Object Model, we can accomplish many things such as the following examples:

1. Auto-populate a SharePoint list that has a Business Data Column
2. Directly edit a customer in SAP
3. Synchronize content between SharePoint and an LOB system
4. Update and manipulate the ADF schema

We now know what the Object Model can do so let’s first exam where the BDC Object Model fits in BDC architecturally.

The BDC Object Model Overview

To see where the BDC Object Model fits in the BDC world, let’s revisit the BDC architecture from Part 1 with Custom Applications highlighted in Figure 1.

Figure 1 The BDC Object Model Creates Custom Applications.

As you can see from the figure above, BDC allows developers to connect to disparate LOB systems and these systems are exposed as data sources to various MOSS features such as Web Parts, list columns, enterprise search, user profiles, and custom applications that can be developed to interface with SharePoint.

Custom applications can be also developed to customize and extend the existing capabilities of BDC. That’s where the BDC Object Model fits in the picture and becomes the focus of this article. The BDC Object Model provides us with new namespaces and classes (methods and properties) we can use in Visual Studio the same way we use a standard set of .NET classes and interfaces. The OOB Web Parts such as Business Data List Web Part goes through this same API that we are going to learn about. You can also create your custom Applications using the same API.

As we have learned already, BDC abstracts the business logic in an XML-based ADF file and stores it in a shared service provider (SSP) database. This file contains all the necessary configuration data that determines how the BDC will connect to the remote systems and which requested data to make available through the BDC Object Model. The following figure shows the interaction between the ADF and the Object Model.

Figure 2 The BDC Object Model and How It Works

As you can see, BDC Object Model consists of Runtime and Administration object models. Each of these performs different tasks:

1. Runtime object model

• Browse metadata, execute methods, retrieve instances, traverse relationships
• Used to build custom applications

2. Administration object model

• Create, read, update, and delete metadata and manage permissions
• Used to edit the metadata internally or to build metadata management tools

Using the BDC Object Model

In order to connect to the BDC from our code, we will use the root namespace within the BDC Object model Microsoft.Office.Server.ApplicationRegistry. If you’re curious as to the term Application Registry, just know that it was the original name for BDC. It’s defined in the assembly Microsoft.sharepoint.portal.dll. Through the root namespace we have access to the child namespaces and its classes that we are going to use later in the article. Here they are listed in this table:

Table 1: Namespaces and Classes

Tip: MetadataModel and Administration use some of the same class names such as LobSystem and Entity so one way to avoid conflict when using them together is to use an alias as follows:

using BDCAdmin = Microsoft.Office.Server.ApplicationRegistry.Administration;

MSDN already has a code sample linked here on how to use the Administration object model to create and import an ADF. Since it’s more practical to us, we will use the Runtime object model primarily throughout this article.

Now we have the basic understanding of the BDC Object Model. Let’s make use of it by applying it to the following three scenarios:

1. Auto-populate a SharePoint list that has a Business Data Column
2. Create a BDC Utility Class that can be used in custom Applications and Web services
3. Create a custom application to update the LOB systems

Since all three of these will use the Runtime object model, we can initialize our code examples by first referencing the assembly Microsoft.sharepoint.portal.dll and then import the namespaces accordingly depending on the tasks:

Now that namespaces have been imported, the next line of code designates the Shared Service Provider (SSP) that provides the ADF. This is accomplished using the following call to the SqlSessionProvider singleton Instance property:

using Microsoft.SharePoint;
using Microsoft.Office.Server.ApplicationRegistry.Runtime;
using Microsoft.Office.Server.ApplicationRegistry.Infrastructure;
using Microsoft.Office.Server.ApplicationRegistry.MetadataModel;

SqlSessionProvider.Instance().SetSharedResourceProviderToUse(sspName);

The next set of code is the actual connection to the appropriate BDC and system instance. The connection is established by indicating the LobSystemInstance and Entity objects with the following statements:

LobSystemInstance lobInstance = ApplicationRegistry.GetLobSystemInstanceByName(”SynergyData”);

//Connect to the customers entity

Entity customers = lobInstance.GetEntities()["Customers"];

Following are the top steps we perform to call into the BDC and retrieve the data:

1. Connect to the named SSP
2. Connect to LOB Instance
3. Connect to entity
4. Connect to method for the entity
5. Execute the method
6. Parse the results

Now that you understand how to initialize your projects that use the Runtime API, let’s now examine each scenario:

1. Auto-populate a SharePoint list with Business Data Column

This application automatically populates a SharePoint list that has a Business Data Column storing customer records. First we create a SharePoint teamsite named teamsite and a SharePoint list called Customers. Then we reference both the SharePoint assembly Microsoft.SharePoint.dll and BDC assembly Microsoft.SharePoint.Portal.dll. Then code the using clauses to import the namespaces and code against the BDC Object Model by connecting to BDC and execute the method instance which will return all customers.

As my colleague Randy Williams has already demoed in his blog, here’s the complete code:

//Get SPList handle

SPSite site = new SPSite(http://MOSS);
SPWeb web = site.OpenWeb(“teamsite”);
SPList customerList = web.Lists["Customers"];

//Connected named SSP using the following call to the SqlSessionProvider singleton instance property
SqlSessionProvider.Instance().SetSharedResourceProviderToUse(“SSP”);

//Connect to the LOBSystemInstance
LobSystemInstance lobInstance = ApplicationRegistry.GetLobSystemInstanceByName(“SynergyData”);

//Connect to the customers entity
Entity customers = lobInstance.GetEntities()["Customers"];

//Connect to our finder method for customers
MethodInstance getCustomers = customers.GetFinderMethodInstance();

//Execute finder method
IEntityInstanceEnumerator results = (IEntityInstanceEnumerator ) customers.Execute(getCustomers, lobInstance);

//iterate through each row returned and add a new item
while (results.MoveNext())
{
SPListItem item = customerList.Items.Add();
item["Customers_ID"] = EntityInstanceIdEncoder.EncodeEntityInstanceId(new object[] { results.Current["CustomerID"].ToString() });
item.Update();
}

2. Create a BDC Utility Class

We’re going to create a custom library that uses the BDC Object Model to connect to an ADF.  We treat the BDC source simply as a data reader and to build our create/update/delete logic in an assembly. The BDC object model provides the means to retrieve collections, list items and enumerators, and we do the rest. In this class, we are going to create the following three static methods:

1. ExecuteFinder()
2. ExecuteSpecificFinder()
3. ExecuteGenericInvoker()

Let’s go through the code one by one. Here’s the first static method named ExecuteFinder that returns all of the entity records.

namespace SharePointMag
{
public class BDCUtil
{
public static DataSet ExecuteFinder(string SSPName, string LobSystemInstance, string Entity)
{
DataSet ds = new DataSet(Entity);

//Connect to named SSP
if (SSPName != null)
SqlSessionProvider.Instance().SetSharedResourceProviderToUse(SSPName);

//Connect to LOB Instance
LobSystemInstance lobInstance = ApplicationRegistry.GetLobSystemInstanceByName(LobSystemInstance);

//Connect to customers entity
Entity entity = lobInstance.GetEntities()[Entity];

//Connect to Finder method for customers
MethodInstance finder = entity.GetFinderMethodInstance();

//Execute Finder method
DbEntityInstanceEnumerator records = (DbEntityInstanceEnumerator)entity.Execute(finder, lobInstance);

//Load matching entities into a DataTable
DataTable entitiesTable = new DataTable();
while (records.MoveNext())
{
//Load each entity and include the Action URL
DbEntityInstance record = (DbEntityInstance)records.Current;
DataTable entityTable = record.EntityAsDataTable;
entityTable.AcceptChanges();
entitiesTable.Merge(entityTable);
}

//DataTable ordersTable = customerOrders.Tables.Add(”Orders”);
ds.Tables.Add(entitiesTable);
return ds;
}
}
}

The following code calls ExecuteFinder of BDCUtil class to return the customers records:

public static DataSet GetEntity()
{
return BDCUtil. ExecuteFinder(“SSP”, “SynergyDataInstance”, “Customers”);
}

The following is a Web service example to call the BDCUtil class to return all the customer records:

[WebMethod]
public DataSet GetCustomers()
{
string SSPName=“SSPPS”;
string strLOBInstance=“SynergyDataInstance”;
string Entity=“Customers”;
DataSet ds = new DataSet(Entity);
ds = BDCUtil. ExecuteFinder(SSPName, strLOBInstance, Entity);
return ds;
}

Note: For simplicity, we explicitly pass in the name of LOB Instance and Entity in our examples. They could also be “discovered” if you desire. Please check out this MSDN article titled How to: Get Started with Using the Runtime Object Model.

The second method named ExecuteSpecificFinder returns a single row of record identified by its ID.

public static DataSet ExecuteSpecificFinder(string SSPName, string LobSystemInstance, string Entity, object[] parameters)
{
DataSet ds = new DataSet(Entity);

//Connect to named SSP
if (SSPName != null)
SqlSessionProvider.Instance().SetSharedResourceProviderToUse(SSPName);

try
{
//Connect to LOB Instance
NamedLobSystemInstanceDictionary lobInstances = ApplicationRegistry.GetLobSystemInstances();
LobSystemInstance lobInstance = lobInstances[LobSystemInstance];
Entity entity = lobInstance.GetEntities()[Entity];

//Get the specific mathcing record
DbEntityInstance record = (DbEntityInstance)entity.FindSpecific(parameters, lobInstance);

//Loading matching entity into DataTable
DataTable entityTable = record.EntityAsDataTable;

//Return the results as a dataset

//DataSet ds = new DataSet(Entity);
ds.Tables.Add(entityTable);
return ds;
}
}

The following code calls the ExecuteSpecificFinder of BDCUtil class to return one row of employee record.

public static DataSet ExecuteSpecificFinder()
{
object[] parameters = new object[1];
parameters[0] = 1; //Employee ID;
return BDCUtil.ExecuteSpecificFinder(“SSP”, “SynergyDataInstance”, “Employees”, parameters);
}

This is the returned employee record that’s displayed in XML format:

<Employees>
<Employees>
<EmployeeID>1</EmployeeID>
<LastName>Davolio</LastName>
<FirstName>Nancy</FirstName>
<Title>Sales Representative</Title>
<TitleOfCourtesy>Ms.</TitleOfCourtesy>
<BirthDate>12/8/1948 12:00:00 AM</BirthDate>
<HireDate>5/1/1992 12:00:00 AM</HireDate>
<Address>507 - 20th Ave. E.Apt. 2A</Address>
<City>Seattle</City>
<Region>WA</Region>
<PostalCode>98122</PostalCode>
<Country>USA</Country>
<HomePhone>(206) 555-9857</HomePhone>
<Extension>5467</Extension>
<Photo>FRwvAAIAAAA…truncated…</Photo>
<Notes>Education includes a BA in psychology from Colorado State University in 1970. She also completed “The Art of the Cold Call.” Nancy is a member of Toastmasters International.</Notes>
<ReportsTo>2</ReportsTo>
<PhotoPath>http://portal.synergy.com/hr/employeephotos/ndavolio.jpg</PhotoPath>
<LoginAccount>synergy\ndavolio</LoginAccount>
</Employees>

Note: The <Photo> element has been truncated to save space here.

Now let’s move on to GenericInvoker by adding ExecuteGenericInvoker method in our BDCUtili class. The ExecuteGenericInvoker method calls the GenericInvoker method instance to write back to LOB systems.

public static void ExecuteGenericInvoker(string SSPName, string lobSystemInstance, string entityName, string methodInstance, object[] parameters)

{ //Connect to named SSP
if (SSPName != null)
SqlSessionProvider.Instance().SetSharedResourceProviderToUse(SSPName);

NamedLobSystemInstanceDictionary instances = ApplicationRegistry.GetLobSystemInstances();
LobSystemInstance instance = instances[lobSystemInstance];
Entity entity = instance.GetEntities()[entityName];
MethodInstance methInst = entity.GetMethodInstances()[methodInstance];
entity.Execute(methInst, instance, ref parameters);
}

In our third example, we look at GenericInvoker and then call the ExecuteGenericInvoker method we’ve just added.

3. Create a Custom Application to Update the LOB systems

BDC is very well known as read-only tool. It’s read-only because the OOB functionality does not make use of GenericInvoker methods. However, the object model and ADF support this type of method instance that we invoke just as we would invoke any other method. If you think of this as a stored procedure call and if I can call a stored procedure that runs a select statement, I should be able to call a stored procedure that inserts or updates as long as the underlining data repository supports them. In other words, the GenericInvoker method allows us to call other SQL commands. For our situation, we’ll use it to write back to our back-end system.

So let’s first add the GenericInvoker method in our ADF file and then call the ExecuteGenericInvoker method in our BDCUtil class . (Note: I’ve used BDC Meta Man to help generate the ADF and create ExecuteGenericInvoker. Thanks Lightning Tools! If you use BDC Definition Editor, you cannot select byte array byte[] as the return type of the field. You can instead select byte and edit the ADF by hand by changing byte to byte[].)  The following is the ADF and for the purpose of this example, only a single entity called Employees has been defined.

<Entity EstimatedInstanceCount=“0“ Name=“Employees“>
<Identifiers>
<Identifier TypeName=“System.Int32“ Name=“EmployeeID“ />
</Identifiers>
<Methods>
</Methods>
</Entity>
<Method Name=“insertEmployees“>
<Properties>
<Property Name=“RdbCommandText“ Type=“System.String“>
Insert into Employees(LastName,FirstName,Title) Values(@LastName,@FirstName,@Title);select NewEmployeeID = SCOPE_IDENTITY()</Property>
<Property Name=“RdbCommandType“ Type=“System.String“>Text</Property>
</Properties>
<Parameters>
<Parameter Direction=“In“ Name=“@LastName“>
<TypeDescriptor TypeName=“System.String“ Name=“LastName“ />
</Parameter>
<Parameter Direction=“In“ Name=“@FirstName“>
<TypeDescriptor TypeName=“System.String“ Name=“FirstName“ />
</Parameter>
<Parameter Direction=“In“ Name=“@Title“>
<TypeDescriptor TypeName=“System.String“ Name=“Title“ />
</Parameter>
<Parameter Direction=“Return“ Name=“Employees“>
<TypeDescriptor TypeName=“System.Int32“ Name=“NewEmployeeID“ />
</Parameter>
</Parameters>
<MethodInstances>
<MethodInstance Name=“EmployeesInserter“ Type=“GenericInvoker“ ReturnParameterName=“Employees“ />
</MethodInstances>
</Method>
<Method Name=“updateEmployees“>
<Properties>
<Property Name=“RdbCommandText“ Type=“System.String“>Update Employees SET LastName=@LastName,FirstName=@FirstName,Title=@Title WHERE(EmployeeID=@EmployeeID)</Property>
<Property Name=“RdbCommandType“ Type=“System.String“>Text</Property>
</Properties>
<Parameters>
<Parameter Direction=“In“ Name=“@EmployeeID“>
<TypeDescriptor TypeName=“System.Int32“ Name=“EmployeeID“ />
</Parameter>
<Parameter Direction=“In“ Name=“@LastName“>
<TypeDescriptor TypeName=“System.String“ Name=“LastName“ />
</Parameter>
<Parameter Direction=“In“ Name=“@FirstName“>
<TypeDescriptor TypeName=“System.String“ Name=“FirstName“ />
</Parameter>
<Parameter Direction=“In“ Name=“@Title“>
<TypeDescriptor TypeName=“System.String“ Name=“Title“ />
</Parameter>
<Parameter Direction=“Return“ Name=“Employees“>
<TypeDescriptor TypeName=“System.Int32“ Name=“EmployeeID“ />
</Parameter>
</Parameters>
<MethodInstances>
<MethodInstance Name=“EmployeesUpdater“ Type=“GenericInvoker“ ReturnParameterName=“Employees“ />
</MethodInstances>

</Method>

<Method Name=“deleteEmployees“>
<Properties>
<Property Name=“RdbCommandText“ Type=“System.String“>Delete Employees WHERE(EmployeeID=@EmployeeID)</Property>
<Property Name=“RdbCommandType“ Type=“System.String“>Text</Property>
</Properties>
<Parameters>
<Parameter Direction=“In“ Name=“@EmployeeID“>
<TypeDescriptor TypeName=“System.Int32“ Name=“EmployeeID“ />
</Parameter>
<Parameter Direction=“Return“ Name=“Employees“>
<TypeDescriptor TypeName=“System.Int32“ Name=“EmployeeID“ />
</Parameter>
</Parameters>
<MethodInstances>
<MethodInstance Name=“EmployeesDeleter“ Type=“GenericInvoker“ ReturnParameterName=“Employees“ />
</MethodInstances>
</Method>

I’ve created three GenericInvoker MethodInstances and they are EmployeesInserter,EmployeesUpdater, and EmployeesDeleter. When a method instance is executed, it will eventually call the appropriate SQL command as shown here:

• Insert into Employees(LastName,FirstName,Title) Values(@LastName,@FirstName,@Title);select NewEmployeeID = SCOPE_IDENTITY()
• Update Employees SET LastName=@LastName,FirstName=@FirstName,Title=@Title WHERE(EmployeeID=@EmployeeID)
• Delete Employees WHERE(EmployeeID=@EmployeeID)

The following method ExecuteGenericInvoker is to call the GenericInvoker method:

public static void InsertEmployee()

{
object[] parameters = new object[3];
parameters[0] = “He”; //Last Name;
parameters[1] = “Wen”; //First Name;
parameters[2] = “SharePoint Architect”; //Title;

try
{
BDCUtil.ExecuteGenericInvoker(“SSP”, “SynergyDataInstance”, “Employees”, “EmployeesInserter”, parameters);
}

catch (Exception exception)
{
Console.WriteLine(exception.ToString());
}
}

The result is as you can see here highlighted:

In the same token, the following code will update the employee record:

In the same token, the following code will update the employee record:

public static void UpdateEmployee()
{
object[] parameters = new object[4];
//The parameters must match the parameters defined in ADF
parameters[0] = “10″; //Employee ID;
parameters[1] = “He”; //Last Name;
parameters[2] = “Wen”; //First Name;
parameters[3] = “MOSS Architect”; //Title;

try
{
BDCUtil.ExecuteGenericInvoker(“SSP”, “SynergyDataInstance”, “Employees”, “EmployeesUpdater”, parameters);
}

catch (Exception exception)
{
Console.WriteLine(exception.ToString());
}
}

Result is as follows:

Note: the order of the parameters must match that of the parameters defined in the updateEmployees method in the ADF. The ReturnParameterName (ReturnParameterName=”Employees”) is required in the EmployeeUpdater and EmployeeDeleter MethodInstance, even though they have no parameters to return.

As we have just learned, the GenericInvoker method is extremely useful because it takes care of authentication and authorization while enabling our custom applications to call in and execute business logic stored in the back-end to insert/update/delete LOB systems.

Write-back Best Practice

Even though BDC can update the LOB systems, it’s not suited for all scenarios. I strongly recommend you to take it into consideration governance and change management that your organization might require. Here are some additional best practices:

1. Understand the vendor support implications for changes that you make. For example, directly querying SharePoint databases is not Microsoft supported because the risk involved in damaging it.
2. Understand data integrity rules before using GenericInvoker method to make direct changes.
3. Have error handling in place.
4. When full inserts and updates are needed, consider using the LOB vendor’s own API that provides transactional write-back capabilities from the client to the underlying LOB application. Another way this can be done is using custom actions which will be covered in detail in Part 8.

Conclusion

BDC Object Model provides a new and exciting solution by allowing direct integration with LOB systems. In this article we have explored the BDC Object Model and exploited the capabilities of the BDC from code. BDC encapsulates the business logic of the back-end systems and developers can just work with the BDC objects to interact with those systems. We’ve demonstrated how to retrieve customer records to populate a SharePoint list that has a BDC data column, and then created a BDC utility class to act as a data layer, finally called the utility class to update the back-end system. With this you can now develop custom applications to interface with SharePoint using the BDC Object Model.  And for organizations that have invested in BDC, you are on your way to leveraging the real power of BDC!

This article was posted on:

Everything You Need to Know about BDC: Part 7 of 8

1. Introduction
2. Benefits of going green with SharePoint
3. Knowledge Management – Going green
4. Go Green – Green Benefits of Content Management

Introduction
Organizations are constantly looking for ways to cut costs, but don’t realize that those cost cutting measures may also have positive environmental effects that can increase goodwill and shareholder value.

Microsoft SharePoint solutions provide tremendous return on investment, but also make your organization more environmentally conscious or “green” by cutting the consumption of paper. Many organizations around the world are utilizing IT solutions to realize their green initiatives. One such technology is Microsoft Office SharePoint 2007

For example: There are government-sponsored green initiatives currently underway. In 2003, the United Nations Economic Commission for Europe enacted a strategy to form a “Roadmap for Paperless Trade”.
(http://www.unece.org/cefact/publica/ece_trd_371e.pdf)

The commission discovered that by moving from paper document processes to electronic technologies, they increased security and transparency in supply chains (Trading industry) and provided both the business sector with higher revenues and costs savings.

Benefits of going green with SharePoint
Information and Communications Technologies are transforming our world on a daily basis. SharePoint enables organizations to control the creation, routing, approval, and publishing of documents. Many organizations use documents in paper format such as invoices, patient records, insurance forms, and corporate policies. These documents are typically copied and dispersed to wherever they are needed—often too many disparate locations. By converting these paper based documents and processes to electronic procedures, a single copy can be accessed electronically from any location without it being printed, shipped and then later archived or discarded.

Here is how you can calculate how much of environmental impact you have contributed and thereby saving how many trees?
By using SharePoint Content management solution, the savings realized by the company can be translated into positive effects on the environment.

How do you calculate?
1 tree makes 16.67 reams of copy paper or 8,333.3 sheets
1 ream (500 sheets) uses 6% of a tree
http://www.conservatree.com/learn/EnviroIssues/TreeStats.shtml

For the a Company which uses 2,000,000 pages, the number of trees saved are calculated as below:
Calculating Paper Saving’s
Total Pages 2,000,000
Reams of Paper (500 sheets/ream) 4,000
Number of Trees Saved (17 reams/tree) 235

Other benefits Using SharePoint

Real Estate
Not storing paper has two benefits regarding real estate – paper storage space and office space. This is space that would require energy consumption for heating, lighting, and humidity control. Beyond that there is also the impact on the environment of using the physical space as a warehouse as opposed to, say, a reserve forest or wetland.

Another example of the real estate impact of storing physical documents is the fact that a paper document is only available at a single location. By storing documents electronically, they can be access in multiple locations, even simultaneously.

Lifestyle Benefits
Putting documents online reduces the number of employees who are commuting to an office. This has tremendous impact on the environment, but also on employee lifestyle. Some organizations are beginning to assign lifestyle scores to projects. While analyzing return on investment and other facets of a project, organizations can assess the impact on employee lifestyle. Raising lifestyle benefits makes a company a more attractive employer and helps organizations with recruiting.

Technology Benefits
Organizations which use SharePoint solutions are also well-positioned to meet their business challenges as they now have a scalable and manageable technology platform which is hot-pluggable and that it can be integrated with other standards-based applications and tools. Since .Net Framework is compatible with SharePoint, SharePoint also gives you an opportunity to customize application which will help the user community to accept the cultural change.

Knowledge Management – Going green
“Going green” has become a topic of increased attention, but it’s nothing new to knowledge management. By its nature, knowledge management promotes efficiency and optimal use of resources, which often reduces the amount of energy required to achieve a given goal.

Knowledge Management (KM) awareness is creating new interest in KM solutions that can improve business performance while reducing environmental effects. Knowledge management using SharePoint plays an important role improving their energy management to optimize the use of office equipment and energy in buildings. SharePoint knowledge management solutions, including records management (RM) and enterprise content management (ECM), reduce the use of paper. In addition, online collaboration and online meeting solutions can greatly cut down on the need for travel to meetings.

In the past, larger companies were well positioned to make use of knowledge management solutions that transformed paper-based information into digital information because they could justify and afford the investment. Now, the small and medium-sized business (SMB) market has more choices, with the lower entry costs for software as a service (SaaS) products. SharePoint provides an option for SharePoint online hosting KM solutions.

Go Green – Green Benefits of Content Management
SharePoint offers so many opportunities to go Green, Top six ways to go Green with SharePoint:
1. Save money on paper – and reduce your environmental footprint at the same time.

2. Increase the effectiveness of core processes.
Think about the benefits of SharePoint technologies in just a single core process – finance. Driving paper from financial processes allows an accounts payable group to pay invoices faster, taking advantage of discounts for early payment. It allows an accounts receivable group to improve cash flows and reduce the number of days that sales are outstanding. The cost of processing an invoice can typically be cut by a factor of 10, creating savings that drop directly to the bottom line. These kinds of savings can usually be replicated in other paper intensive processes, yielding BOTH bottom line AND Green benefits.

3. Truly integrate your field operations.
The application of distributed capture technologies – using a scanner in a field office to process locally created paper and electronically store it in a central location within SharePoint. These scanned documents can be accessed from any location over the internet for immediate review. Distributed capture solutions often pay for themselves just through savings in courier and shipping costs. But taking courier and overnight services off the road also translates into significant direct environmental benefits.

4. By using SharePoint we can reduce real estate costs. Less paper means fewer filing cabinets means less real estate – which reduces cost, but also significantly reduces your environmental footprint.

5. SharePoint helps in improving employee productivity.
As a content management tool, SharePoint proves to be the best in the industry for the following:

Collaboration
Portal
Enterprise Search
Enterprise Content Management
Forms Driven Business Process
Business Intelligence

6. Reduce off-site storage costs. Off-site storage of paper is expensive, often requiring rental and maintenance fees as well as significant transportation costs to the facility. SharePoint provides a SaaS option for hosting the portlet.

This article was posted on:

Go Green with SharePoint

Backup and Recovery Strategy

Use separate server for Disaster Recovery (DR)

Use additional server with installed SharePoint for DR. Do not connect this server to the existing SharePoint farm - it should be isolated environment with single SharePoint installation, or “3 servers farm”. Disaster Recovery servers must locate outside corporate networks, and ideally, outside corporation premises. Usually, it locates in different data centre. The reason for this to provide data availability in case of the network or data centre outage.

Configure mirroring

Configure SQL mirroring of content databases to provide data reliability. It should be done in the early stages, when content databases are very small, not for the hundreds gigs of content. The reason is that network bandwidth to DR is usually limited due to distant location of the servers, and setting mirroring for large databases became slow process.
Refer to the following document for more details about configuring SQL mirroring http://go.microsoft.com/fwlink/?LinkId=83725&clcid=0×409

Recovery data immediately

Highly available farms imply reliable strategy of data recovery. SQL Mirroring is only part of this strategy. Data restore in reasonable time is one of the factor of DR strategy

Consider several scenarios for backup/recovery:

1. OOTB backup/restore. SharePoint does not provide reliable mechanism of automatic backups. Backup/Restore feature available via Central Administration or via STSADM command. The only way to schedule this task is to use STSADM and windows scheduler to configure backups/restore.
Disadvantages:

• not reliable approach due to dependency of singe instance of scheduler;
• scheduler location on the single server, no multiple instances;
• no granularity of restored items;

2) SQL snapshots of content database (http://tinyurl.com/d2auns)
Disadvantage - lack of granularity in backup/restore

3) Data Protection Manager (DPM) - is one of the recommended solutions for data recovery strategy in large SharePoint farms.

a. DPM provides reliable and flexible functionality to backup, search and restore items.
b. Supports backup/restore of different servers in network - SQL Server, Exchange, file shares backups, and etc.
c. Adapts to the changes in Farm dynamically - add, delete or relocate servers.
d. Monitors and records all changes in SharePoint content DB by integrating into SharePoint VSS Writer.
e. Provides search functionality for any deleted items across content DB and restore data from content db.
f. Granularity in restoration of documents, sites, and applications.

One disadvantage of DPM is that it requires additional, clean, SharePoint instance to be used as temporary location of restored items to merge them with production content DB.

Some recommendation of DPM usage:

- Do not use existed SharePoint farm servers disconnected from the SharePoint farm - this approach has many bugs, and restoration fails in most cases.
- Iinstall SP1 for DPM - it provides significant performance increase for indexes and catalogue (http://technet.microsoft.com/en-us/library/dd347836.aspx)

4) Third party tools - Quest Recovery Manager, DocAve and Idera Backup. They provide the similar functionality of the DPM, but in some cases address SharePoint behaviour only, and might not provide enterprise level of backup strategy.

Quest Recovery Manager is the preferable tool for data restoration - lightweight application that does not require additional servers to restore deleted items.

This article was posted on:

SharePoint Farm configuring and deployment. Part 4 - Backup and Recovery Strategy

Development Environment

SharePoint development environment configuration depends on the processes, type of engagements and type of work. The most popular solution that addresses the development scenarios is using local SharePoint farm, separated from the production servers, with the single installations of the SharePoint server on the development boxes. This provides isolation for builds, tests, and debugging across different teams, projects and production environments. The local environment is mostly isolated by development box and is installed on the host server or on virtual server. The following procedure is an overview of the steps that are required to create a typical SharePoint development environment.

Development Box installation

• Use Windows Server, Visual Studio, and SQL Server. Windows Server 2008, VS 2008 and .NET 3.5, SQL 2008, with TFS 2008 is officially supported environment for SharePoint development. Advantage of Windows 2008 is that it is fast in virtualized environments.
• Install SharePoint on development boxes and prefer not to connect to existed farm instances used on other stages. Development environment should stay apart, to develop and tests in isolated environment.

Chose development tools

There are varieties of tools that can make development fast and easy - from commercial to open source and Microsoft products.

Microsoft recommends to use “Visual Studio extensions for Windows SharePoint Services” (VSeWSS), which simplify code-up solutions for SharePoint (e.g. Web Parts, List Definitions, Site Definitions, etc) via UI and allows reverse-engineering existed site to extract definitions for SharePoint entities.

The disadvantages of VSeWSS are:

1) not intuitive for beginners;
2) doesn’t provide usability to change all properties of the features, and other SharePoint items easily
3) Cannot use VSeWSS projects without VSeWSS extension.

Others 3-rd party tools, which simplify development, are:

1. Management Tools

• SharePoint Spy (http://www.echotechnology.com)
• SharePoint Manager (http://www.codeplex.com/spm)
• SharePoint Analyser (http://community.bamboosolutions.com)

These tools help to manage services of SharePoint farm and get detailed information about configuration settings.

For example, screenshots of SharePoint Analyzer (left) and SharePointSpy (right)

2. Visual Studio Tools

• STSDev to create project files (http://www.codeplex.com/stsdev)
• SPSource to get site sources (http://www.codeplex.com/spsource)

These tools create Visual Studio projects with the SharePoint 12-hive structure, and provide build settings to build/deploy/retract WSP packages directly from IDE. STSDev is the best tools to create a variety of projects - from simple features and Web Parts to Custom Workflow projects:

3. Development Tools

• TypeMock Isolator - Writing test for the SharePoint is a hard task for the developers, due to number of internal, sealed and private classes that are used across all SharePoint objects. ”TypeMock Isolator” is the only unit-testing and mocking framework for SharePoint, which provide ability to fake recursively, and simulate collection easily.
• SPDisposeCheck - tool to check assemblies that use the SharePoint API to build the better code. It provides assistance in correctly disposing of certain SharePoint objects to help following the best practice.

4. Other Tools

• U2U CAML Builder - editor for CAML queries (http://www.u2u.net)
• SharePoint Feature - set of different development and debugging plugins (http://www.codeplex.com/features)

Such tools give your additional flexibility in everyday development stuff, providing different editors, log viewers and analysers to make SharePoint development easier and faster.

Setup Deployment environment

Configure Continuous Integration (CI) system to build SharePoint solution, compile output to WSP packages, prepare deployment scripts and deploy WSP to Test Environment. Team Foundation Server (TFS) is one of the recommended tool for this task. There are several articles describing how to adopt SharePoint solution for CI, setup builds and configure TFS Deployer to deploy SharePoint packages across different environments:
http://www.azsharepointpros.com/ShareAndEnjoy/Lists/Posts/Post.aspx?ID=10

As to SharePoint Visual Studio Templates, CodePlex has a “SPTemplateLand” project that provides “12-hive” structure for SharePoint projects and deployments.
This solution was slightly modified by Microsoft SharePoint Consulting Services guys to have single deployment package for multiple projects and support packaging additional SharePoint Artefacts (site definitions, root files). This project is published there http://blogs.msdn.com/gderun/archive/2008/01/11/sharepoint-visual-studio-project-and-deployment-templates.aspx

Configure testing environment

The following diagram depicts the most common development environment, which is recommended by “SharePoint Guidance patterns & practices” team.

1. Stand-alone SharePoint environment for development, unit testing and debugging of SharePoint project. Runs continuous integration and builds verification tests before deploying the SharePoint solutions to the test environment.
2. Source Control/Build Server to build SharePoint packages (WSP) and to deploy solution to test environment.
3. The test environment performs user acceptance testing, manual functional testing, automated functional testing, system testing, security testing, and performance testing. After the solution meets production requirements, the SharePoint solutions are deployed to the staging environment.
4. The Staging server uses to test the “production-ready” solution in an environment that closely resembles the production environment. The purpose of this environment is to identify any potential deployment issues. Although the Staging environment is optional for smaller applications where deployment failures are not critical

The staging environment represents the target production environment as closely as possible from the perspective of topology (for example, the server farm and database structure) and components (for example, the inclusion of the Microsoft Active Directory service and load balancing, where applicable).

This article was posted on:

SharePoint Farm configuring and deployment Part 3 - Development Environment

Installation

The recommended Windows environment that offers the best performance for SharePoint is to have 64-bit servers. Such an environment provides significantly larger address space than 32-bit one; more room for SharePoint assemblies, CLR/Native APIs, Network Stack, IIS/ASP.NET and other components hosted in their respective tiers.

Windows Server 2008 and SQL Server

SharePoint install on the Windows Server 2003 and 2008. The recommendation is to use Windows 2008 because it has the outmost security. For the Windows 2008 you need to activate the following roles: Web Server role and the Microsoft .NET Framework 3.5.

Take into account that Windows 2008 Web Edition is not supported for farm roles, except WFE boxes, due to restrictions of non-Web editions of SQL Server on Windows 2008 Web Edition. The release of “SQL Server 2008 Web Edition” extends usage of Windows 2008 Web Edition for SharePoint. Refer to licence regulations and SQL Server 2008 Web Edition info for more details: http://tinyurl.com/b8nype

Do not install SharePoint on Domain Controller box in virtualized environment. DC role of Windows 2008 limits performance of hard drive by turning off caching to provide AD consistency. Any installations on virtualized DC might not work properly.

SharePoint installation supports  SQL 2005 and 2008 Servers (even SQL Server 2000 is supported, but there are not much advantages of its usage). SQL Exress edition is supported as well, but for basic MOSS installation, however basic install of WSS 3.0 will use Windows Internal Database (WID doesn’t have size limitation). SharePoint installs only on SQL Server, but you can use BDC use content from 3-rd party databases.

A few advantages of SQL 2008 over SQL 2005 are in performance, encryption, clustering, mirroring and etc. Moreover, it provides updated SharePoint Web Parts for Reporting Services and KPI. Detailed information about SQL 2008 and SharePoint can be found in the following post http://tinyurl.com/cdkcyw.

User SQL Aliases

When provisioning a new SharePoint farm, it is highly recommended to use an alias to connect to the Microsoft SQL Server, as this provides for greater flexibility to move the SharePoint databases to a new server. For example, using an alias during the installation will simplify the migration process of SQL database server from small environment to larger physical cluster during scaling out process.

Install Microsoft Office 2007 on farm premises (optional)

Office 2007 is not required on SharePoint server, but it might be  good to have it somewhere in you farm premises (not server box) for administrators, especially when you outsource your support or/and admins connect  remotely. In this case they might  need client apps installed somewhere to have access to

• Word and Excel for documents in Document Library
• PowerPoint for Slides in Document Library
• Access for “Edit in Datasheet” support in Document Library

Consider the same for Office SharePoint Designer(SPD), which is necessary for customisation purposes. (SPD is a  free product with is distributed as separate product that is not include into Office)

Install all Windows Updates

Make sure that all servers have the latest service packs and updates for Windows, SQL and Office prior installing SharePoint.

Choose the right edition of SharePoint

Be careful when use Enterprise edition of SharePoint, because Microsoft doesn’t provide the support if you decided to downgrade to Standard version, due to loss in features and functionalities. If you need Standard edition then consider a fresh installation.

Install SharePoint

Install SharePoint across all servers in farm. Start with WSS/MOSS slipstream package (with integrated latest Service Pack) rather then using basic WSS/MOSS installation and applying Service Pack later.

Follow the next order of installation:

• Application server where Central Administration site will be hosted
• All front-end Web servers
• The index server (if using a separate server for search queries and indexing)
• The query servers, if separate from the WFE servers
• Other application servers (optional)

Consider using scripts to automate SharePoint installation and configuration for the large farm deployment. SharePoint provides several configuration files and console commands that will do all deployment un-attendant. This will speed up installation and brings consistency of building and rebuilding servers in farm.

There are three commands to automate SharePoint installation:

• SharePoint Setup.exe + Config.xml - to script the setup questions
• PSconfig.exe - to script configuration wizard
• STSADM.exe - to script central admin UI for creating web apps and site collections

Alternative solution is to use already preconfigured Power Shell scripts. “SharePoint Deploy” tool on CodePlex provides the configured scripts for the standard installation, which can be adapted to any environments.

Refer to the following documentation for details about unattended installation http://go.microsoft.com/fwlink/?LinkID=135694&clcid=0×409, and info about installations scripting http://technet.microsoft.com/en-us/library/dd335964.aspx.

Take into account that SharePoint does not uninstall properly and additional steps are required to clean remaining files and remove database tables, which are kept for the security reasons (http://nehasinha.wordpress.com/2008/02/01/uninstalling-moss-2007-manually). It might be better to reinstall everything from the scratch if something went wrong, including Windows Server - because it will save a lot of time trying to fix potential issues that might be caused by files from the previous installation. In this scenario, using virtualization and snapshots feature saves a lot of time.

Detailed information about SharePoint installation guidelines are published on TechNet http://go.microsoft.com/fwlink/?LinkId=106632

Check Office Web Service availability

Sometimes, SSL protocol of Office SharePoint Web Service is broken.  To test it, go to IIS Management Console and open SharePoint Office Web Service in browser via https://. If it doesn’t work - don’t process further till fix it. This is very critical stuff, because SharePoint roles cannot be assigned on other services in farm. (I’ve seen such issues across several clients, when you can’t use other boxes in farm and only Application boxes are available for Index and Query roles, because SSL was broken).

No known fix at this moment, except reinstalling SharePoint.

Install SharePoint updates

Install all SharePoint updates (Infrastructure Updates and/or Cumulative Patches) after farm is deployed. Check the release notes of the latest cumulative patch if it includes all previous patches and updates. Sometimes you need to install previous updates manually. Cumulative patch releases each 2 months. Follow the official documentation “Deploy software updates for Office SharePoint Server 2007” for the processes of how to deploy infrastructure update (WSS Upgrade needs to be installed first and only then MOSS upgrade).

Be careful with SharePoint hot fixes

There are several hot fixes, for example “Coreserver.msp” package, which are released after Cumulative Patch.  However, be careful with these fixes, because they are temporary solution before the next official update, and they are not properly tested. Install hot fixes for specific problems only.

Configuration

Enable SharePoint Features

Navigate to the Central Administration site and enable Enterprise SharePoint features, if necessary. The default settings is Standard features.

Assign roles to servers

Navigate to the Central Administration site and assign SharePoint roles across all farms servers, according the infrastructure design and topology.

Configure administrative tasks

Configure administrative tasks across servers, like email settings, blocked type, logging and etc. Setup SharePoint Shared Services and configure all related services like Search, Query, Application Services, Profiles and etc.

Disable “Central Administration” role

Navigate to the Central Administration site and disable “Central Administration” role for all servers in farm, except application servers. This action will disable “Central Administration” site and IIS won’t use additional resources to host this application.

Configure Warm-Up scripts

Install site “warm-up” scripts. Those scrips will compile each page of SharePoint site collections when box restarts or after IIS poor restarts. This script improves the response time when users request pages first time.

Those warm-up scrips use STADM command to “warm-up” the administrative interfaces and hit each page in the portal to force their JIT. The collection “warm-up” scripts available there http://blogs.msdn.com/joelo/archive/2006/08/13/697044.aspx

Recycle IIS application pool at different time

Make sure that the application pools are set to recycle at different times on different Web servers, in case of multiple Web servers in the farm.

Recycle different IIS Web sites at different times to avoid peaks on the Web servers. When recycling more than one application pool on a specific Web server at the same time, temporarily remove that Web server from the load balancer to avoid bad user experience.

Following these steps helps to install SharePoint and configure basis settings with the minimum amount of time and avoid most common pitfalls, which usually happens when SharePoint installs in the wrong order

This article was posted on:

SharePoint Farm configuring and deployment Part 2 - Installation & Configuration

Overview

Planning and installing SharePoint Farm across enterprise network is not a trivial task. SharePoint is rarely installed in an isolated environment, and usually it interferes with the organization strategy and existing infrastructure. Many factors may affect farm design, performance, scalability and redundancy - from hardware devices in organization network, to network topology. As a result, leveraging and finding compromises among those factors helps to build consistent, reliable and flexible environment.
There are several whitepapers on the Microsoft TechNet portal describing requirements for SharePoint Farm, but most of them are either written without taking into account infrastructure scope or filled with irrelevant information that navigate the reader away from the problem scope.

In this document you will find the configuration recommendations regarding different SharePoint areas. All information is represented in the set of recommendations about different actions you need to undertake or pay additional attention when you install and configure your SharePoint environment. We tried to structure all section to follow the natural flow of SharePoint installation from the scratch - from pre-installing analysis requirements to post deployment actions.

We plan several whitepapers in our “Best Practices” series, and we are interested which topis you would like to see in our next SharePoint publications. Please send us your comments and suggestions via this form.

Introduction

Organizations adopting SharePoint face a variety of tasks - from planning, strategy, infrastructure and architecture design, UI Design, migration, and to development. All these tasks imply flexible infrastructural baseline before actual work starts. However, in reality we face the outdated environment and misconfigured farms that are not ready to implement new requirements. In such cases, baseline architecture becomes foundation stone of all SharePoint projects.
Why would we care about infrastructure and not about something else, for example development? Fixing infrastructure errors is very expensive task and leads into significant changes across SharePoint farm. For example, Index Role assigned to the wrong server and incorrectly configured Search can lead to performance and redundancy issues that might require up to 3 days fix. Development errors are not so expensive and can be fixed relatively quickly, but sometime such errors, eventually become infrastructure errors that lead to changes in infrastructure design.

“Architectural Planning”

Plan your farm and network communications before starting actual installation. The first thing to start is designing SharePoint architecture across corporate network. This includes understanding network structure, examining network devices and choosing the right SharePoint topology to fit the existing infrastructure and new requirements.

Examine corporate network

Start from description of the existing network design, location of all applications and system servers. Microsoft Visio 2007 and “Network Diagram” template is a good instrument for this task.

Record the location and information of corporate system servers, like Domain Controllers, File Servers, Mail Servers, Application and others. Dont’ forget about network services - firewalls, proxies, and etc. For example, locations of ISA Servers across corporate network - IP address, list of open ports and the administrative user.

The best way to maintain “Network Diagram” document is to update the single diagram that covers topology of all domains and how they are connected. The following diagram demonstrates the Visio document descibing the servers and devices across organization.

This diagram will give a holistic view of the existing topology and ensure quick access to information across different domains.

Examine network devices

All network devices in the topology play a vital role of how SharePoint performs and interacts among different servers. Information about locations and settings of all routers, switches, and accelerators become very important in planning server locations. For example, location of different WAN and XML accelerators across network affects SharePoint server organization and configuration.

Presence of different network devices affects the connection bandwidth and latency between farm’s servers, and thereby, affects the choice of appropriate SharePoint Farm topology. Network Load Balancers (NLB), routers and switches will affect how fast network response, therefore the farm should be designed with the least impact of these devices.

Refer to the following links for the detailed information about WAN accelerators, NLB and other network devices across SharePoint farms:

1. http://technet.microsoft.com/en-us/library/cc263099.aspx
2. http://blogs.msdn.com/joelo/archive/2008/01/17/global-sharepoint-deployment-partner-solutions.aspx
3. http://blogs.msdn.com/joelo/archive/2007/01/05/nlb-network-load-balancing-and-sharepoint-troubleshooting-and-configuration-tips.aspx

Network administrator is a friend

The IT administrator is the person who should participate in farm configuration from the very beginning. This person will be responsible for the configuration of all network servers and devices across corporate network.

Most of the SharePoint Farm topologies cross the bounds of domains and from the very beginning specific protocols and ports must be open. The best way to maintain current situation is to have a separate document, shared with administrator, with the description of protocols and ports to open across network services.

Detailed information about system accounts and list of ports is available in the following articles:

• Plan for administrative and service accounts (Office SharePoint Server) http://technet.microsoft.com/en-us/library/cc263445.aspx
• Office SharePoint Server security account requirements http://go.microsoft.com/fwlink/?LinkID=92883&clcid=0×409

Measure network latency

Network response time is one of the important factors that can affect SharePoint farm design. Ideally, you need to measure the latency between SharePoint servers and users in order to reorganize servers according the smallest response time.

Network latency is the key point to determine which of the proposed scenarios to implement in the current SharePoint deployment. (Latency is the time required for a packet to travel from one point on a network to another).

Use the Ping tool (ping.exe) to measure latency for:

• users - from the client computer to the Web server on the server farm;
• data centres that host servers of the same farm - from a Web server in the remote data centre to the database server in the primary data centre

Do not forget to divide the round-trip result by two, because all measures are one way only, not round-trip.

Compare results to the data below, and adopt environment to have latency lower those values.

Number of users	Concurrent users (10%)	Central Solution	Distributed solution
100-5,000	10-500	Bandwidth:   3+ Mbps (dual T1)Latency:   < 100 ms	Bandwidth:   1.5 Mbps (T1)Latency:   <100 ms
10,000	1,000	Bandwidth:     3+ Mbps (dual T1)Latency:   <250 ms	Bandwidth:   1.5 Mbps (T1)Latency:   <500 ms
100,000	10,000	Bandwidth:   3+ Mbps (dual T1)Latency:   < 250 ms	Bandwidth:   1.5 Mbps (T1)Latency:   <500 ms

The critical bandwidth for any SharePoint farms is 1.5 Mbps (T1) with 500ms latency. Overstepping these values will increase the page-load times dramatically, in 4 times at least. Refer to the diagrams in the “Plan for bandwidth Requirements” document, for more details about the bandwidth and latency results under different conditions.

Available network bandwidth and latency influences geographic deployments significantly. Data transfers across WAN links that span multiple cities, states, provinces, countries, or continents requires really fast lines to provide adequate response time, so design such topologies thoroughly.

More details for bandwidth requirements available in the following article http://technet.microsoft.com/en-us/library/cc262952.aspx

Become familiar with SharePoint farm communications

Before discussing servers’ redundancy and farm topologies let us review farm servers and how they communicate with each other. The following picture from from “Planning an Extranet Environment for Office SharePoint Server” TechNet article illustrates the communication channels within a server farm and which servers handle client’s request.

When a user issues a query, the query is sent to a Web server. The Web server communicates with the query server to build a list of results, and then communicates with the computer running Microsoft SQL Server to extend the list of results with summarization text, URLs, and security trimming. In parallel, the Web Server gets page data from SQL Server and renders them on fly. This diagram will help in understanding which roles to use on farm servers.

Plan a baseline topology

Analyse the existing infrastructure and plan a SharePoint topology for redundancy. The term redundancy is often misinterpreted to be synonymous with availability.

Redundancy refers to the use of multiple servers in a load-balanced environment for any of several purposes, such as to improve farm performance, to scale out to accommodate additional users, and to improve availability.
Availability is a more specialized concept that refers to a multiple-server environment that is designed to accept connections and operate normally even when one or more of the servers in the farm are not operational. Therefore, availability implies redundancy.

There are several different topologies - from three to six servers in farm, which can be used as a baseline. Which one to choose depends on the level of redundancy and available hardware. Not all clients can afford topology with six or ten servers in farm due to budget limitation or data centre capabilities. Finding the compromise between numbers of servers, type of hosting and servers’ roles become critical task, because this choice will affect performance and extensibility of the SharePoint farm for several years ahead.

Three Servers Farm

The minimum availability for the farm with few servers can be achieved with “3-servers farm” topology. In the current topology Web and Application Servers locate together on the one box and the database is on another box. The remaining, third, server gives a choice of which server role make redundant - Web server role or the database server role.

The farm with the two Web Servers provides redundancy of the Web and Application roles, improving the overall performance. A drawback of this design is that your data is not redundant (left farm). In other case, farm with two Database Servers (cluster) provides data redundancy, increasing availability of critical data, but users might suffer from temporary loss of access, when Web Server unavailable (right farm).

The “3-servers farm” is one of the most questionable farms in terms of redundancy and performance. This limitation in the number of servers cannot provide redundancy of Query Server and high performance at the same time.

Redundancy can be achieved with Query Roles on both Web App servers. In this case, Database Server is the only place for Index Role, but this will hinder the overall performance. The Index Role is very CPU and HDD consuming role and that is why database servers are not very optimal place for this role. Alternative solution is to assign Index Role to the Web Server with the Query Role, but this will not work effectively, because in this case, index will not be propagated to another Query Server in farm.

If performance is one of the priorities then consider using Query Server and Index Server Roles on different Web Application Servers. This is flexible design in terms of extensibility, because with the new servers in farm changing roles of Index and Query servers is not required.

Interestingly, a TechNet article (http://is.gd/8QbS, page 26) explains, that a Query Server can’t be used with Web Applications server for 3-servers farm. The reality is that, Web App and Query Role together are super common, more common than not (one of the reasons is that Query Server doesn’t use Network-Load Balancer - it uses its own algorithm).  What they actually mean in the TechNet article is that having the  Index on database server is not at all a recommended solution.

Four Servers Farm

Additional, forth server will add redundancy either for Data Server or for Web Server. However, it does not help much with performance. Current topology suffers from the same “3-servers farm” drawbacks - no place for Index Server with Query Role redundancy.

Five+ Servers Farm

The most common and highly available server farm topology is “5+ servers farm”, the farm with the middle tier server.

This middle tier server solves all issues of three and four servers topology by providing the dedicated tier for Index and Application roles. Additional servers in farm will extend middle tier, by assigning new roles to those servers - Excel Calculation Services Role, and Microsoft Office Project Server 2007 Role.

The following table summarize farm topology:

To optimize the overall performance of five and more servers SharePoint Farm, configure a dedicated Web Server for crawling content, especially when crawling a server farm that contains more than 500 gigabytes (GB) of content or crawling content over the WAN. To ensure that user requests are not affected by content crawling, remove the dedicated Web server from the network load balancing rotation. This is especially important in global environments in which the off-peak hours of a regional farm (when crawl jobs are likely to be schedule) coincide with the peak hours of the central farm.

Plan extranet topology

Choose the topology based on requirements for external users. This topology will provide a basis of network extensibility for applications servers and communications between them.

The simplest topology is “Edge firewall topology”, which is represented by following diagram, from TechNet article.

This topology applicable for the small farms, when there is no need to separate internal services from corporate network and secure communications between server farms. All remote users are separated from farm by ISA server which plays a role of remote proxy.

For the big farms, when security of communications is a priority, the recommended topology is “Back-to-back perimeter topology”. This is very flexible and adaptable topology for network changes.

The main advantage of this topology is that it isolates the server farm in a separate perimeter network. Layers logically separate all servers and communications are under control - any security damages affect only specific layer, not the entire farm. External user access is isolated to the perimeter network and users can be isolated in different AD for remote and corporate access.

There are some other extranet topology variations, but mostly all of them are based on “Back-to-back perimeter topology” with some modification.

Detailed information about farm topologies can be found in the following documents:

1. Best practices for My Sites: http://technet.microsoft.com/en-us/library/cc262706.aspx
2. Best practices for team collaboration sites: http://technet.microsoft.com/en-us/library/cc850694.aspx
3. Planning an Extranet Environment for Office SharePoint Server: http://technet.microsoft.com/en-us/library/cc262400.aspx

“Logical Planning”

Plan site collections

Plan number of site collections and sub sites in advance - content, location, security. Start with the single site collections and several sub sites rather then creating several site collections, and try to avoid new site collection if there are no requirements for this. The reason of such structure is that each new site collection works as a new application, with isolated scope to features, templates and search. Maintaining such structure is much easier than several site collections.

Organize site collection across several content databases

Do not end up with one big content database, because data optimisation will cause troubles in this case. For the small and development environments, single content database might be a preferable choice. However, for the large farms create several content databases and organize site collections among them. Having several content databases with sites helps to address the following:

• Keep content database size <100 GB, otherwise it could hinder performance (MS recommendation)
• Data usage optimization.
• Simplify farm backup and restoration.
• Flexibility for Disaster Recovery (DR) strategies.

More details about site collections in several content databases available in the following blog post: http://msmvps.com/blogs/laflour/archive/2008/10/14/tips-to-create-a-site-collection-in-new-content-database.aspx

Script actions

Prefer to script installation and SharePoint Farm configuration actions: setting roles, creating web sites and site collections, etc. Configuring successful farm from the first attempt has a change to fail due to complexity of SharePoint. Running scripts to repeat all actions will save time when something went wrong and new server installation is required.

This article was posted on:

SharePoint Farm configuring and deployment. Part 1 - Architectural and Logical Planning

In this article, I’m am going to introduce you to some of the features of Nintex Workflow 2007 (NW), and show you why I think this is a really cool product. I’ll also tell you about some of the annoyances I found.

Full Disclosure: Now, I’m trying to be as unbiased as I can and give you my honest opinion, but you should know that I was paid by Nintex for creating the USP Journal issue. No, they didn’t get any editorial control, but I was still paid. As such, feel free to hold a salt shaker ready and take what I say with an appropriate grain of salt.

Nintex Workflow in a Nutshell

Basically, Nintex is an easy to use workflow add-on to SharePoint. If you have worked with workflow in SharePoint before, chances are you have either worked with SharePoint Designer or Visual Studio. Both of these options are well suited for their intended use, but there is a big gap between SPD and VS that NW fills really well.

On one side, SharePoint Designer offers an incredibly easy and free way of adding simple workflows to a SharePoint site. Using a bit of creativity you can even create fairly complex solutions, using only the built-in actions and features. However, SharePoint Designer lacks scalability and flexibility, and re-use is at best very cumbersome.

On the other side is Visual Studio, which is like having your own thermo nuclear device. It’s incredibly powerful and you can more or less control at least parts of the world, but you had better know exactly what you are doing, or you risk your solution blowing up in your face. Where Visual Studio offers immense power, its learning curve is so steep, it can scare the most die-hard mountaineer.

Nintex Workflow bridges the gap between these two solutions, by giving end users some of the power of a Visual Studio solution while maintaining the ease of use and gentle learning curve of SharePoint Designer. In addition, NW includes some truly unique features, such as the ability to combine sequential and state-machine workflows and user interaction with tasks through email in a feature that deserves mention for its name alone: LazyApproval.

Authoring Workflows in Nintex

What initially sparked my interest for NW was the interface for designing workflows. The interface resembles the workflow designer in Visual Studio in that you get a visual overview of the ‘flow’ part of your workflow. Branching, loops, and state machines are presented in what I think is a really intuitive interface that makes it easy for end users to understand what is going on.

What makes this workflow designer especially friendly to end users is that it resides inside SharePoint itself. Using scripting and more fancy scripting than I’d dare to undertake, you are actually designing your entire workflow right inside SharePoint with no client software to install or learn at all. You simply drag-and-drop actions from the action menu onto the designer surface and configure each action using dialog boxes.

True, the interface of the Visual Studio workflow designer offers more options and allows for a greater degree of flexibility, but Visual Studio is still an all-or-nothing package; if you want the added flexibility, you also need to deal with the added complexity. That complexity is probably far more than any end users are prepared to handle.

Templates and Snippets

Another nice feature is the ability to easily create templates of existing workflows. You can create the generic templates for common tasks such as approval or customer follow-up and then re-use the templates when creating new workflows. This saves a lot of time if you are creating similar workflows or have common tasks used by several units in an organization.

Another option for workflow re-use is snippets. A snippet is basically a set of workflow actions stored as a single action. You create these snippets in the workflow designer, and they become part of your available actions for use in any workflow. For example, you might create a snippet to notify an item or document owner about changes to that item or document. Whenever you want to include that functionality in your workflow, you simply drag the entire snippet onto the workflow designer, and you get that functionality added without having to re-create all the individual actions.

Go with the Flow

If there is one thing I really miss, especially in SharePoint Designer workflows, it is better debugging and tracking of what is going on. A common approach is to add a lot of history logging to track what the workflow is doing. In Visual Studio, you can attach the debugger and step through the entire workflow, which is really nice and gives developers extreme control.

While NW lacks the stepping option, it does offer a very nice workflow information page far superior to the default workflow status page of SharePoint. On that workflow information page, you get a visual representation of your workflow, including which actions and branches are currently executing.

This workflow information page can also give you a detailed view where you can get a whole boatload of more information about the workflow execution, including how long each action takes, the outcome of conditions, who approves or disapproves which actions, and a range of other information.

The workflow information provided by NW goes a long way towards providing the debugging information you need to figure out workflow bottlenecks, logical errors, and other problems you may encounter.

State of the Machine

The final thing I want to point out as truly unique in NW is the support for state machines. You may wonder what is so special about supporting state machines, after all, most workflow products support state machines in some fashion.

What is really nice in NW, though, is that a state machine is simply an action, just like any other multi-branch action. This means you can combine state machines with sequential workflows inside the same workflow. For example, you may have a long running customer follow-up process, in which one part of the workflow consists of contract negotiations. That contract negotiation may move back and forth between different states, but is still only part of a longer workflow in which the ultimate goal is to make the customer happy.

In NW, you can simply add a state machine action and configure that action as you would if you had developed a stand-alone state machine workflow. Then, after that part of the workflow completes, you can continue in a sequential workflow, or even transition into a new state machine inside the same workflow.

OK, So What’s the Catch?

Alright, I’ve said enough about the cool things, so it makes sense to point out some of the weaknesses as well. Note that most of these issues are annoyances at worst.

First, the interface for controlling configuration scopes leave a lot to be desired. Basically, you can configure settings for sites and site collections, and the scopes control inheritance and all that, but the way that Nintex has solved the interface is not good. You need to pay attention to small pieces of texts on the page to know whether you are configuring for a site or for the entire site collection.

Second, NW supports setting up holidays to avoid workflow activities while no one is available to respond to tasks or interact with the workflow. However, the way the holidays are configured could have been done so much better. You can only add one day at a time, so if you are closing the office for two weeks during summer you need to add each day of those weeks individually. Really sad for schools with months of vacation at a time.

Third, the workflow designer can become cluttered when you are creating complex workflows, especially when using multiple branches. The branches keep expanding horizontally and will eventually fill any screen size if you have enough branches. And frankly, ‘enough’ branches is too low a number for example in state machines with many states. The same applies to the workflow information; to get an overview, you need to do a lot of scrolling. A “zoom out” function would be useful.

None of these issues are deal-breakers, though. It feels to me like they have gone 95% of the way towards making a perfect workflow solution, but with a product like this, the last 5% can be annoying.

And still, their 95% are still light-years ahead of anything else I have seen in this niche.

Want to Learn More?

The state machine image above is an example from the USP Journal I mentioned in the beginning of this article. In that issue, you’ll learn how to develop workflows in Nintex Workflow. The image shows part of a “get well card” that you’ll develop during the exercises in the issue. I’ll show you how to install and configure NW, familiarize you with the Workflow Designer, teach you at least some of the actions available, as well as explain the logic of how various tasks are done, such as the above example of inline state machines.

You can get the 128-page issue from the issue download page, and as I said, it’s free, as in beer.

.b

This article was posted on:

Review: Workflows with Nintex Workflow 2007

Or would it? It may be that we allow for 10% leeway before we say that our variance was bad. So in our example, we would actually have a green light, or maybe orange. We would only get a red light if our variance was greater than -$10,000. Let’s face it; we’re still making money, just not as much as we had hoped.

Did you notice that I didn’t say “Revenue” was the KPI? In fact, I said that the performance of Actual Revenue to a target is the KPI i.e. the variance. This is important. KPIs measure PER-FOR-MANCE.

So, a KPI measures how well we performed against a given target.

That makes sense right? So what’s the fuss about scorecards and KPIs and all that? Well… balanced scorecards are designed to show overall performance of the organisation. Not just easily quantifiable things like Profit Margin. And how do you compare, equate and aggregate information such as Employee Satisfaction, Customer Satisfaction, Operational Effectiveness, Environmental Sustainability and Profit Margin altogether?

 

Where do I Come From?
	Now we get down to the meat of it. And what this has to do with SharePoint. Someone, somewhere has to actually physically build our KPIs, store them, access them and use them.  What are My Options? PerformancePoint Monitoring & Analysing PerformancePoint delivers excellent reusability and sharing of KPIs, customisable indictors. Coming from the Office suite of tools, it is a UI* which doesn’t require IT involvement – it is aimed at BAs and super-users. Integrates with existing technologies, SQL and SharePoint. SharePoint PerformancePoint still uses SharePoint as it’s UI but here I’m talking about building KPIs themselves. As of Office 14, PerformancePoint M&A will be entirely bundled into MOSS. Which I guess means that SharePoint’s existing rudimentary KPI building functionality will disappear. Yes, it’s rudimentary and simple - it’s ok for really small stuff but if you’re from an IT department, my advice is, don’t waste your time. SSAS Get your MDX skills going because you’re gonna be writing a lot of MDX queries. That’s not a bad thing. I’m just saying that’s what you will be doing. Also, all information relating to a single KPI must be contained within that cube where the KPI sits. And you’re gonna have to create the interface using Excel or suchlike yourself. Numerous Other Software Vendors It all comes down to cost doesn’t it? And integration. And user familiarity with the UI*. I know I’m biased. I like PerformancePoint. It integrates nicely with technologies that I’m already using (SQL and SharePoint). And I like that it’s gonna be bundled with MOSS. So sue me (no, don’t really)… * user interface
Who Am I?
	Profit - What Else is There?   I could give you a massive list of KPIs, but I ain’t gonna. Check out the EPM Review. They have an EXHAUSTIVE list of common KPIs. It’ll knock your socks off… or provide just the impetus you’ve been needing to clean out your garage. Who am I to judge? Just remember this… you want to measure performance in all areas that are important to you. While there is some commonality across organisations, there is by no means uniformity. This means, the very first thing you’ve got to do, is consider your organisations values, mission and strategic plan. This will guide you on what’s important FOR YOUR ORGANISATION. Once you know what you want to achieve, then you need to identify what data you already have that can be used to quantify those areas. Then consider, what data you could capture but don’t already have. Finally, don’t forget to think through and identify what data you can’t capture. It may be just as important to know what’s missing. How Long is a Piece of String Defining targets is not as simple as it seems initially. Again, this is something you need to think through at the outset. Targets could be manually entered values, but based on what? It could be as simple as Revenue Last Year + 4%. But someone somewhere has to decide how many percent, and whether that base revenue number should include Revenue from discontinuing entities, or interest or a number of other factors. The fact is, you need to think this through separately for each and every KPI.
Getting on the Merry-Go-Round
	Common issues you will face when designing your KPIs…  Defining the KPI What’s the formula? Each segment of the organisation defines things in slightly different ways. In order to measure them effectively, a corporate view needs to be taken. This can be tricky. If you want the data used to derive the results to be meaningful then you either have to (a) ensure it’s 100% automated and requires no business unit level input or (b) find a way to get everyone to agree on a common definition. Good luck with that! Weighting the KPI Some KPIs are more important than others. Well, maybe not more important but certainly the impact on the organisation of poor performance, or conversely good performance, is more greatly felt. So we need to identify what matters. Again, getting everyone to agree can be challenging. But this is an important thought process and should be well considered before jumping in the deep end. It helps to look at your strategic plan and that may point you in an appropriate direction. Too Many KPIs I see this time and time again. How do you hit the bullseye when you’re aiming everywhere? It’s got to be achievable, not the KPI targets but the project itself. Getting a scorecard up and running is challenge, no doubt about it. But getting people to use it, now that’s the real challenge. So start small. I’ll say it again. Start SMALL. Did you miss it? START SMALL!! Get users comfortable with a few key areas before you throw a thousand or a hundred or even 50 KPIs at them. They just won’t read it. They won’t understand what they are looking at. My suggestion, seriously, start with 10 at the most. I know there are BAs and scorecard developers out there looking at me in stunned silence (well, reading at me in stunned silence… or perhaps not, perhaps you are yelling at the screen (if so, ask yourself why because I can’t actually hear you). Of course, you can revisit it later… just don’t start with information overload. That’s all I’m saying.
	Part 1 – Getting to Know You - Intro to SharePoint BI
	Part 2 – Sell! Sell! Sell! Why Build a Dashboard Anyway?
	Part 3 – KPIs – Where do I Come From?
	Part 4 – Dissecting Dashboards
	Part 5 – We Really Need to Talk – Scorecards & Reports
	Part 6 – Beware the Bogeyman – Securing Dashboards

Siyonara amigos! Till next time.

Kristen Hodges, MCTS, MCITP
http://www.bi-tch.blogspot.com
http://b-iq.org

References

http://www.epmreview.com/KPI-Library.html
http://nickbarclay.blogspot.com/search?updated-max=2008-01-09T15%3A44%3A00-05%3A00&max-results=20

This article was posted on:

KPIs – Who Am I and Where do I Come From? (Part 3 of 6)

As a CFO and senior business executive would life be better if all your commercial information, everything from files and documents to LOB systems, were all accountable and controllable from one platform or portal?

Imagine definitive decision support, comprehensive implementable governance, anything and everything at your finger tips via a dashboard and ‘best bet’ query.

In recent times, and with its ubiquitous commercial acceptance showing no sign of abating, SharePoint has been moving to take this kind of ‘center stage’ in the information management universe.

But in many cases SharePoint is filling this role by default, due to its momentum within IT and other business areas, without a comprehensive assessment as to its suitability for this pivotal role.

Could SharePoint be a black hole for your business, drawing in information and content without providing an adequate foundation for information management?

Let’s look at the typical benefits SharePoint delivers then consider some of the strategic implications these benefits portend.

Typical SharePoint Benefits
There is no doubt that SharePoint delivers extensive business benefit in the information worker and general office productivity area:

• Document Libraries - Business content such as documents, spreadsheets, graphics, presentations, media files, even emails and their attachments, can be housed and ‘managed’ in SharePoint libraries easily accessible by anyone, anywhere, anytime, via intranet, Extranet or Internet

• Roles & Controls - Library controls, and permissions across SharePoint, enforce author, editor, publisher roles and structured responsibilities along with fail-safe versioning, garbage, archival repository and other content controls

• Search & Surfacing - Search promises that nothing will ever be ‘lost’ again. Anything in your Enterprise in SharePoint, and beyond, can be found or ’surfaced’

• Security - Security is leveraged from your existing Active Directory investment or other authentication system integrating seamlessly with content roles and controls. Single sign-on can make any LOB system, such as financials, ERP or CRM, directly accessible or integrated with your portal

• Information Management - Metadata can be tailored to your business and assigned to assist with search, categorization, and information processing

• Customisation -SharePoint is extensible with features and functionality that can be exposed and customized, or created for example as Web Parts and seamlessly integrated

Implications
But what are the implications behind these SharePoint benefits and how do they impact your business?

Let’s consider three basic SharePoint issues that span the business technology ’sophistication’ spectrum from: the basics of content library storage; to custom development; to business intelligence

SharePoint’s Storage Paradigm Shift
While the concept of computer storage might be considered somewhat boring and ’something for the Techs’, it is important to note that SharePoint involves a massive paradigm shift in this area.

In the past, Fileshares have been the most common method of general storage. But their time has passed and it is now almost universally considered a good idea to have all your content in record management systems, document libraries and workspaces.

But where does all that content actually live once it is in SharePoint in a document library?

Fileshares in Windows, and folders in SharePoint, while functionally similar are very different entities from a storage perspective

Doesn’t SharePoint store content in its own folders and ‘fileshares’, the way it appears on screen?

No. SharePoint stores all its content in a Microsoft SQL Server database.

SQL Server is not a new technology and your business likely already has SQL Server running, but consider the implications of this change:

• Licensing Cost - SQL Server involves its own licensing costs both for the purchase of the software and CALS (client access licenses). This is in addition to the SharePoint costs. By contrast storing and sharing files in folders is essentially ‘free’

• Infrastructure & Performance - SQL Server is a sophisticated platform that necessitates specialised infrastructure including servers, storage devices, and staff, in addition to SharePoint. The cost for the use of this infrastructure may be spread over other applications that are already using SQL, but if your business is looking at putting all its content into SharePoint+SQL then this infrastructure will have to scale both in terms of its size and mission critical support.

• If you require the SharePoint content to be available on an operational basis special attention will need to be given to SQL I/O performance along with addressing problematic performance issues with SharePoint itself, such as the performance penalty when returning more than 2,000 records in a list. Performance limitations might make certain types of enterprise-wide solutions untenable. Capacity planning is essential. Here is a useful link on SharePoint capacity planning.

• Migration - While there are tools that allow the migration of content into SharePoint from earlier versions of SharePoint and other information sources such as Lotus Notes, this is typically not an easy or ‘lossless’ exercise. Looking to the future, will there be tools to migrate your content out of SharePoint+SQL if needed? What about all the feature rich content attributes you might build into your SharePoint information management solution - will you be able to take them with you?

Custom Development Dilemma
SharePoint is a great OOTB, ‘out of the box’, productivity tool. But OOTB functionality is never enough! Customization is always demanded to support specific business processes and applications.

However, while SharePoint has a wealth of inbuilt functionality waiting to be tapped, commensurate with the richness of features there is a high degree of complexity that is largely underrated. Development in SharePoint brings its own challenges:

• Business Coverage- SharePoint covers many different business areas including content management, business intelligence, document and records management, workflow, portals, etc. Ideal SharePoint solutions should leverage as much of the existing platform functionality as possible without reinventing the wheel.

• For best results a developer should have a holistic functional understanding of SharePoint across all its business applications

• Complex Technology - SharePoint is a complex technology in its own right that spans from web front ends to SQL Server database processes. The right approach, expertise and understanding are required to fully leverage it. Good .Net developers are not automatically good SharePoint developers. Experience, lots of experience, is needed to bring about holistic development understanding. Experience also assists in coping with SharePoint’s many ‘undocumented features’.

• Because many SharePoint options can be configured without development, and initial development appears easy, overconfidence is often engendered in in-house developers, and overly ambitious development projects can be undertaken with disastrous results

• Duplicitous Paths -With Microsoft technologies there are often several development paths to achieve a solution and SharePoint is no exception. But despite the seeming logical and extensible nature of a path, a satisfactory result may not be achieved. As this stage of SharePoint’s maturity, many likely development paths often come to a dead-end through no fault of the logic of the developer or their approach, but because of a SharePoint bug, anomaly or undocumented feature.

• SharePoint is only early into SPK release lifecycle and the many CUs (cumulative updates) are a must

• Development Environment - a suitable development environment is required which ideally should replicate Production conditions which may involve topologies of web farms, index servers and SQL clusters, Active Directory and Exchange servers. Few environments provide that level of development platform support, although virtualization is closing the gap. This means that SharePoint development projects do not always travel well to the Production environment, even with Solutions and Packages, delivering unforeseen results

SharePoint Production environment topologies can be complex, and are not often replicated in development, with the result that SharePoint solutions do not always travel well when deployed

Metadata Intelligence
The leveraging of metadata is one of the most underrated areas in SharePoint yet it is invaluable to your business and information management strategy.

For example the implementation of an information management system, incorporating governance and compliance built on SharePoint’s metadata and workflows functionalality, is a logical application for the platform.

Metadata support allows you to attach keywords that can be used to provide meaning about the content that is loaded, assisting in categorization, search and aggregation. It is also a key to ensuring the alignment of content and business processes to your governance, compliance and information management plans, by being linked back to your associated business goals and objectives.

SharePoint OOTB can leverage the Properties page in all Office documents where metadata for author, title, subject, status, keywords, etc., can be entered. In addition SharePoint has a whole metadata infrastructure that allows you to create, customize and manage your own metadata infrastructure.

This SharePoint metadata, called content types, have particular value in that they can also be used to trigger actions and workflows. For example when a document is loaded into a library, special processes can be triggered based on its metadata.

Let’s consider some of the implications of even a basic implementation of SharePoint’s metadata on a large scale. (For ease of discussion I will use the term metadata rather than content type.)

Note the message ‘You MUST fill out any required properties’

• Mandated Metadata - It seems like a good idea to allocate metadata to all your SharePoint content because of the associated benefits. So the next obvious step is to mandate metadata i.e. ensure every piece of content that is loaded into SharePoint must be assigned relevant metadata.

• To add metadata to a single document may only take a minute but scale that out across your business to every staff member and to every piece of content, and a sizeable amount of time will be taken up assigning metadata that will erode overall business productivity

• Dirty Metadata - One way to overcome this ‘mandated metadata productivity hit’ is to make the metadata faster and more intuitive to allocate. This can be done by providing metadata in the form of dynamic lists, as opposed to free text fields.

• But even when presented with an easy or ’smart’ pick list, staff may not know what metadata value to assign, or be confronted with too many choices. For example combinations of departments, business units, and projects could produce an overwhelming list of options.

• ‘Time poor’ information workers don’t want to stop their task to find out what metadata should be allocated or go through the process to create a new category. This can lead to erroneous values being assigned just to expedite content loading.

• In some cases there may be purposeful erroneous metadata provided, to obscure business activities such as fraud

• Legacy Metadata - Most discussions of SharePoint benefits focus on ‘greenfields’ SharePoint implementations but all businesses have legacy fileshares, content and systems that can be essential to migrate into SharePoint.

• While it is possible to programmatically assign metadata during migration, in reality it is not worthwhile to do so on any scale unless a metadata or similar policy has previously been in place.

• Consider trying to allocate even simple metadata such as author, document title and version if these are not already associated with the content: fileshare locations by nature provide multi-author storage so authors are not differentiated from PAs, etc.; duplicates and drafts with the same or entirely different titles are common confounding categorization and titling; latest date may not reflect a file master version. These are just some of the issues.

This much used image helps demonstrate the typical nature of the information and content in most businesses fileshares. Determining, on any scale, the author, title, keywords of this information is in reality an impossible task.Thus much legacy information comes into SharePoint sans the metadata that could be vital to an understanding of the material, and assist with the information management of the business

• Metadata Meaning -Metadata seeks to give meaning to content beyond that given by a file name.

• How many metadata properties might be needed to adequately indicate the meaning of a document that spans many diverse areas and topics? Free text metadata would allow more comprehensive explanation but we are back to the ‘productivity hit’ issue of the time taken to enter all the metadata.

• Meaning may also differ from different perspectives. How would the allocation of ‘definitive meaning’ to files and content be achieved? How would different perspectives and synonyms be handled where different metadata words and terms are used for similar concepts?

• Consider the issue of trying to assign comprehensive metadata to all the images, video, audio and VoIP content in your business. How much resource would be required? Would the metadata be accurate and definitive using any current methods?

• In terms of scale, how would the process of allocating metadata relating to every email be handled?

• What about metadata relationships between content, such as between documents and documents, images, emails, projects, departments, people, roles? Would this type of meaning and understanding gained provide any useful insight into your business and be of value?

These are issues that may currently seem most poignant to the legal profession and e-Discovery but are increasing in import for all businesses.

Metadata that provides meaning as described above, including that for unstructured content, can provide valuable insight into business operations providing a basis for real business intelligence, way beyond that of the traditional BI with just numbers and charts in a spreadsheet.

Conclusion
SharePoint is a good information management and general office productivity platform. However it has its issues that are best to be aware of and managed, rather than just ‘let loose’ in your business, preventing it from becoming a ‘black hole’.

In the context of the issues discussed above:

The Good is that SharePoint will, in line with the Pareto principle, OOTB meet roughly 80% of your general information management needs and in the process provide a stable scalable platform.

Good also is that assistance with SharePoint development, both in terms of available tools and developer experience, is increasing at a rapid rate. At the time of writing this article, a quick search of www.CodePlex.com returned nearly 700 ‘open source’ projects providing explanation and assistance on SharePoint development. Countless blog articles by SharePoint gurus likewise provide commentary on numerous SharePoint solutions.

The Bad is that if you want SharePoint, then SQL Server and its associated overheads are unavoidable.

Bad also is that with regard to going beyond the metadata basics, there is no solution available within SharePoint that address the issues discussed. However this is not so bad as it seems as there is no solution available within any of the current generation of similar platforms.

How to get more meaning from business information and unstructured content is the next major conundrum facing business managers today. However I believe this type of metadata support and analysis is one of the most exciting challenges, and will be the basis for the next level of real business intelligence and competitive business advantage.

Next generation tools, providing definitive meaning to structured and unstructured information and a full understanding of information relationships, are just starting to enter the mainstream and gain more attention. These tools can work with SharePoint as well as hundreds of other information sources and hold the promise of providing an incredibly comprehensive understanding and powerful information management tool for your business.

I will be looking at this next-generation solution in more detail shortly.

This article was posted on:

SharePoint –Black Hole or Star of Your Business Universe?

There are quite a lot of people in the SharePoint community flying the SharePoint Development flag high and proudly. The SharePointDevWiki.com is growing steadily each week and I’m extremely pleased that there are more and more community members submitting content on there.

There are plenty of people trying to take the SharePoint Development approach and using Solution Packages to deploy to the farm. But more and more I am seeing this scenarios below.

Scenario One

• The development team sit down with the client and get the requirements for their new “Intranet”
• Start off with a Solution Package and buildingSite Fields, Content Types, Lists, Pages, Web Parts androlling this into a UAT environment (or worse Production)
• The “Intranet” in UAT then gets a review from the client
• Changes are requested by the client, but the client also wants to have a “play” with the UAT environment
• Developers start off on the right foot and add a few extra Site Fields, amend the List Templates and have tested it in their Development environment
• The Dev team are then ready to deploy this to UAT…but the client has done more than “play” in this environment and wants this content in there when the Developers release the changes.

FIRST TIP: ensure they are aware that anything they do in UAT is not making it to production! UAT is very different from content STAGING environments.

• So the Dev team release the new .wsp solution package to the farm and then run a script to delete the current site collection and create a new one using the new templates and scripts…this obviously deletes any content that was in there.
• This gets a tick in the box and the site goes into Production
• Content is authored in the new production site and they want to add a new Site Field to a list that is commonly used across multiple department sites within the Intranet
• Changing the List Template in the Solution Package and redeploying to the farm will not amend the instances of the List already created. So the Dev manually makes these changes and so the spiral of pain starts…

Take this scenario 6 months down the line, the Dev no longer bothers updating the solution package as he is doing it all manually in production anyway and no need to go back and change it in source control…”that’s not used anymore anyway”.

Scenario Two

Scenario two steps in where Devs originally deployed style sheets to the Style Sheets library using Solution Packages and then someone gets hold of SharePoint Designer and ignores the warnings and modifies the style sheet here. Lots of changes are made…but not put back into the source style sheet in the solution package.
Later that month a web part C# file changes and a new style is added to the style sheet in the Solution package. The new package is redeployed…guess what, it’s going to redeploy the style sheet but not overwrite the one in the site because it’s broken its ties with the original source! So the web part will not find that style and not render properly.

TIP 2: lock down files deployed with solution packages so that they can only be changed via deployment. Do this via ’sealing’ elements and also via permissions on items.

TIP 3: don’t let developers near the production and UAT boxes. Don’t let their dirty fingers anywhere near the 12 hive directly! Get administrators to deploy to UAT and Production using WSPs and change scripts.

Why does all this happen?

The biggest problem is that developers want to be able to automate the entire setup process especially in a build server scenario and this is why everything initially ends up being deployed via a Solution Package.

TIP 4: draw a line between what is development and what is customisation.

Either do everything in Development or do everything in customisation for an element e.g. don’t build your event receivers in a dll and package with a WSP, and have a List Template in the wsp, but then manually use a tool like Event Receiver Manager to add the event receiver! On creating the list via a Site Definition or via a Feature, use a Feature Receiver to attach the event receiver there and then! It’s self documenting that way!

Whatever you do, document where each type of element sits and where it will be found: in a WSP, update script, command line up, manual steps etc.

Below is an example table that shows the boundaries between elements. Note this is by no means complete, but just to give you an idea…plenty of things in Development may sit in Customisation for a lot of teams. You can also give this to external integrators your team may use for implementations to make sure EVERYONE works the same way.

TIP 6: Remember that changing elements won’t affect their instances e.g. List Templates, Site Templates, Workflow Instances. All the other elements are referenced and linked and will be automatically updated once pushed out into the Farm.

To modify existing instances you will need to provide update scripts using the object model either as custom window console application or powershell script.

TIP 7: Get your dev team to first off create a script that deletes the site collection and recreates the site collection (using custom Site Defintion if approach taken) and then run the scripts to populate content required. Use this script when in own Dev Virtual Machine AND Shared Development Environment AND Build Server.

TIP 8: Always keep UAT and Production one change step away from each other. Always deploy to UAT and Prod using WSPs, but also use update scripts rather than deleting Site collection and creating it again! Have the ability to take the approach of Updating in Shared Dev environment (by taking snapshot of UAT/Prod) to test update scripts before going to UAT.

Conclusion

Well this is the end of this 6 part series on Leveraging the Platform. Hopefully it’s been useful to you. Very shortly I will be placing some more ideas up on the SharePointDevWiki.com around these change control processes to help SharePoint Server teams with these tasks!

This article was posted on:

Part 6 - Lessons learnt from Leveraging the SharePoint Platform

In case you’re just joining us in the series, here are our previous five articles:

Part 1.  BDC Purpose and technical architecture

Part 2.  Detailed overview of the Application Definition File (ADF) and how to develop one to connect to a back-end database

Part 3.  Developing an ADF to connect to Web Services

Part 4.  Consuming Business Data through Web Parts and SharePoint lists

Part 5.  Implementing Enterprise Search with Business Data

Overview

User Profiles within MOSS are a great way to store key information about SharePoint users.  This information, stored within individual profile properties, contains useful content such as first name, last name, date of hire, the user’s manager, and numerous personal details such as interests and skills.  This user metadata can be used when executing workflow, searching for people, utilizing social networking aspects such as my sites, defining audiences, any many other SharePoint capabilities.  SharePoint contains dozens of built-in properties, and you can add any custom properties as needed.

While all of this a great benefit to enhancing SharePoint’s knowledge of these people, there may be a drawback to this.  Here is yet another database of information that needs to be populated!  In most organizations some of this information lives within Active Directory (AD) and perhaps other details can be found in repositories such as HR systems, so you can say that SharePoint is really just duplicating what already exists elsewhere.  The primary goal of this article is to show you how you can eliminate this redundant data entry by pulling this authoritative information from your external systems through the BDC.

User Profiles and the profile properties are all maintained within your Shared Services Provider (SSP).  User profiles can be created manually, or you can create import connections which allows you to automatically synchronize these values from external systems.  Supported external systems are AD, LDAP (such as Lotus Notes), and BDC.

An important detail you must understand is that only AD or LDAP can be set up as a primary (or master) connection.  A BDC connection cannot be.  A primary connection means that a SharePoint profile with an account name (e.g. domain\username) can be created from this source.  BDC can only be a secondary connection, meaning that it is only able to fill in additional properties for an existing profile—in other words, it cannot create a profile.

To better explain, see Figure 1 below which shows the profile being created (based on the Account name) from AD and additional properties coming from an HR System (click image to enlarge):

Figure 1: Mapping profile properties between SharePoint and external systems

The arrows reflect the mapping between SharePoint profile properties and columns in external systems.  You can see that some information (such as first and last name) may be duplicated in external systems.  When defining mapping for a profile property, you can only specify a single source for the data.  Based on the arrows, the decision was made to pull the first and last name from AD.

For many organizations, this is a common scenario.  AD, maintained by the IT group, typically contains only a minimal amount of accurate user meta data.  Other systems, often maintained by the HR group, contains additional values.  SharePoint allows you to set up multiple primary and secondary connections to ensure all your profiles can be created and populated as needed.

Now that you have a basic understanding, the rest of the article will focus on how to create and configure these secondary BDC import connections.

Creating BDC Import Connections

Prior to creating a BDC Import Connection, I recommend you configure and ensure that your AD or LDAP primary connection is functioning.  Once this is working, you can then focus on your secondary BDC connection.  For my upcoming demos that I’ll walk you through, I am pulling accounts from a single domain inside AD.

To create a BDC Import connection, first access your User Profile and Properties screen within your SSP (Central Administration –> SSP –> User profiles and properties).  On this screen, click on the View Import Connections link.  Then click on the Create New Connection toolbar button.  Finally, choose Business Data Catalog type in the drop down list.  This will yield the following screen:

Figure 2: Creating a BDC Import Connection

In Figure 2 above, you’ll notice that you have two mapping options (depicted by the radio buttons),  1:1 and 1:many.  Let’s look at each option in detail:

1:1 Import Connection Type

A 1:1 connection is the most common type of connection that is used.  This means that a single profile in SharePoint will map to a single entity record that is returned from BDC.  For example, if we have a user profile for synergy\afuller as shown in Figure 1 above, this user will also exist in an external system.  For a 1:1 mapping, a Specific Finder method will be called based on the selected entity.  This shouldn’t surprise you as a Specific Finder method returns a single record from the external system.  A good question you might be thinking is how does it know which record to pull?  I’ll answer this question as I show you how to configure the connection.

To configure the 1:1 mapping as shown in Figure 2, we need to first start off by creating or identifying an existing profile property that can be used to uniquely identify the user in the external system.  This property must also match the identifier that is defined for your entity in your ADF.  In my example, I will be using EmployeeID which is the primary key for an Employees table in a SQL Server database.  Employees represents a table usually found in HR or ERP systems.  Here is the Identifier as reflected in the ADF:

Figure 3: EmployeeID as the Identifier for Employees Entity

And here is the Specific Finder:

Figure 4: Specific Finder for Employees Entity

As you can see, I have selected EmployeeID to be the unique identifier.  Since I don’t have a built-in profile property in SharePoint that stores EmployeeID, I’ll create one.  This is necessary for the 1:1 mapping to work.  Here is how it looks after I’ve created it:

Figure 5: Newly Created EmployeeID Profile Property

I chose the integer type because the property’s type must be compatible with the type defined in the Identifier (which was defined as System.Int32 as shown in Figure 3).

Now that this is done, I can create my BDC Import connection.  (Note: Even though you are only creating a 1:1 mapping that will use a Specific Finder, this screen also expects a Finder method.  Even though the Finder method is only used when creating 1:Many connections as discussed below, you must still have one or you will get an error when you select the entity from the catalog in this screen.)

Going back from Figure 2 above, after selecting Business Data Catalog as the type, you must enter a connection name.  I recommend a friendly name that you’ll easily recognize. For my demo, since I’m pulling employee information from an HR system, I’ll call the connection HREmployee.  Here is how it looks just prior to saving:

Figure 6: Creating a 1:1 Import Connection

You’ll notice that I selected the EmployeeID profile property that was just created.  For this to work, I now need to go into my current SharePoint profiles and manually enter in a values for EmployeeID.  This is necessary in my example since SharePoint must know what value to pass to the Specific Finder.  If you’re using a built-in profile property that is already populated (such as the Account name), you don’t need to do this.

Now that the 1:1 relationship between the User Profile and the external system is set up, it’s time to figure out which profile properties you want to import.  If you refer back to Figure 1 above, you’ll see that I illustrate the importing of Title and About me properties from the external system.

For the demo here, I’ll map two existing properties, Hire Date and About me.  You do this by editing the profile property (SSP –> User profile and properties –> View profile properties –> Choose a property and edit).  There’s no reason why I can’t create new properties and map these; the process is the same.  Here is how to map the Hire date profile property to the HireDate column from the HREmployees import connection:

Figure 7: Mapping Hire Date

Figure 7 above shows the upper and lower portion of the same screen.  When choosing a column in the Data source field to map drop down list, you’ll be presented with a list of all the columns returned from the Specific Finder.  So, the rule of thumb is the ensure that you are returning all columns you want to map from your Specific Finder.  As with creating the import connection, when mapping properties this way, you must ensure the data types are compatible.  If they are not, you’ll be given an error message like this: Cannot map to this data source field. This property is of type ‘date’ and the data source field you are mapping to is of type ‘System.String’.

I have also mapped the About me property to the Notes column defined in the Specific Finder.  At this point, the mapping configuration is in place.  We’re not quite done yet, however.

Another important step is to ensure that your Default Content Access Account has permissions to update these profile properties through the BDC.  The Default Content Access Account is configured when setting up Enterprise Search (SSP –> Search settings –> Default content access account).  If you prefer, you can also set a specific account when configuring the User Profile Import.  Just go to SSP –> User profile and properties –> Default access account.  Now that you know what your access account is, you must grant this account two sets of permissions:  1) Permissions to update your profile properties and 2) permissions to call into these BDC methods.

To grant permissions to update profile properties, go to SSP –> Personalization services permissions.  On this screen, grant your access account the Manage User Profiles permission as shown here:

Figure 8: Granting Content Access Account Manage User Profiles Permissions

If you’ve worked much with profile properties, you’ve probably noticed that each property can be individually secured by adjusting its privacy setting.  The Manage User Profiles permission grants my access account (SYNERGY\MOSS.Search) write permissions to all profile properties.  Without this, you’ll need to set the privacy level of each mapped property to Everyone, which may not be the best security level.

The second permission is to grant this access account permissions to the BDC entity.  If you’ve been following our series, we covered BDC permissions in Part 2.  The minimum permission you need to grant is Execute as shown below.  You access this screen from SSP –> View Applications –> <Select ADF> –> <Select entity> –> Manage Permissions.  In this screen shot, I grant MOSS.Search execute permissions to the Employees entity:

Figure 9: Granting Access Account Entity Permissions in BDC

Finally, we are now done and ready to import.  Let’s quickly recap to make sure we have these steps down.  Again, we’re talking about a 1:1 mapping.

1.  Ensure ADF is prepared with a Specific Finder returning the columns you want to map.

2.  Create a new or choose an existing profile property that contains values to match the entity identifier.  In my example above, I used EmployeeID.

3.  Create a 1:1 import connection.

4.  Map your desired profile properties to columns returned from Specific Finder.

5.  Grant your content access account permissions

To start the import, return to the main User Profiles screen and click Start full import.  This will first do a full import from your primary import connection.  The status here is reflected next to Profile import status shown below.  Once that is done (and sometimes with a brief minute or two delay after it finishes), the secondary imports from BDC will begin.  This is reflected next to Membership & BDC import status.  Both are shown highlighted here:

Figure 10: User Profile import status

Once the profile imports complete (and it may take from several minutes to possibly an hour for thousands of users), you’ll want to check the import log to get a sense on whether it succeeded.  To do so, click the View import log link.  When reading the log, you should know that a PEOPLE_IMPORT content source refers to individual person crawls through your primary import connection, whereas PEOPLE_DL_IMPORT refers to group (i.e. distribution list) crawls.  PEOPLE_DL_IMPORT is also where your secondary import connections through BDC are logged, so it’s relevant to us in the scope of this article.  Here is a part of the log showing both content sources for one user, Andrew Fuller.

Figure 11: User Profile Import Log

Even though the logs may look fine, you’ll certainly want to verify some of the mapped values come over.  To view one, simply click on View user profiles and select a user.  In Figure 11 below, here are the two mapped properties for synergy\afuller:

Figure 12: About me and Hire date values pulled in by profile import

Since I mapped the About me property, you can also see this within the user’s My Profile found in their My site.  Here is Andrew Fuller’s:

Figure 13: Andrew Fuller’s My Profile page

Now that you have a solid overview of 1:1 mappings, let’s turn our attention to 1:many.

1:Many Import Connection Type

While less common than 1:1 connections, a 1:many is handy in a few situations.  Let me give you one example.  As mentioned in the overview section above, the IT group typically manages basic user attributes in AD.  In all my years of consulting, I’ve encountered very few organizations that actually keep these attributes up to date.  The problem is that IT is usually busy handing a myriad of critical problems, and they don’t usually bother updating less important attributes such as someone’s manager.  As long as the permissions are right, that’s what is important to them.

This can cause problems, however.  One of the useful SharePoint profile properties is Manager which gives SharePoint knowledge of the organizational hierarchy.  This was shown in Figure 13 above when looking at Andrew Fuller’s My Profile page.  By default, this property is mapped to the manager setting for the user in AD.  However, if this value is not set or out of date, SharePoint won’t have an accurate picture of the organization.

To correct this problem, we will pull the manager from a Departments table that is stored in our external HR system.  Here is where the 1:many comes in.  Recognize that many users will have the same department.  The idea is that we want to use the department profile property in SharePoint, which is correctly set, and use that to look up into the Departments table (in the external system) to pull the manager for that department.  So, you can think of the 1:many to be for each department there will be many user profiles with that value.

Let me show you how I configure the mapping to pull the Manager property from our HR System.  I start off by adding another Import connection.  Yes, you can have multiple import connections coming from BDC.  I call this one HREmployeesGetManager and here is how it is configured:

Figure 14: Creating a 1:many import connection

When choosing the 1:many mapping radio button, you have two drop downs to fill in.  You first need to select the filter.  The choices you have will be the filter descriptors that have been made available for the entity’s Finder method.  Hence, to use 1:many mapping, you must have a finder method and at least one filter descriptor.  (Note: if you cannot select 1:many mapping, it is probably because you don’t have filter descriptors defined in your ADF.)

For my demo, here is the Finder method and filter descriptor for the Departments entity:

Figure 15: Finder method for Departments entity

In the SQL select query, you can see that I am selecting the DepartmentID, DepartmentName and Manager columns based on a certain department name.  The next step is to change the default mapping for the Managers property.  Here is how I have changed it:

Figure 16: New mapping for Managers profile property

For the Data source field to map, I have pointed this to the Manager column.  Other column choices are the other columns returned from the Finder method.  For the last text box (Manager field string format in AD), you can leave this unchanged.

That’s it.  Run another import, and you should now have your manager property coming from your external HR system.  For this to work, you will need to ensure your Department property is populated for your user profiles.  In my case, this value was still coming from AD.  That’s right, you can use a value that is imported from AD and pass that as a parameter to your BDC connection.  Pretty cool.

Let me make one last point on 1:Many import connections:  Be careful if your Finder method returns more than one record, and your mapped property does not allow multi-values.  For example, perhaps the HR system had two department records with the same name.  And for each record, a different manager is set.  When the import runs and it tries to pull the manager based on the name, two records will be returned. When this happens, the import process will take the first record it finds, and this may not be the one you want.  If you are using a multi-valued property, however, it works nicely and will store the values for all records that are found.

Troubleshooting Tips

When troubleshooting import problems, you’ll definitely want to go to the SharePoint logs (by default C:\Program Files\Common Files\microsoft shared\Web Server Extensions\12\LOGS).

When mapping, a profile property can be mapped to only one source (primary or secondary).  Similarly, a column from the Specific Finder (1:1 connection) or Finder (1:many connection), can only be mapped once.  For example, if you map the HireDate column to one profile property, you cannot also map it to another.

Remember that you can only map compatible data types.  Fortunately, you will get this error when you try to save your profile property.

When mapping strings, be careful about the column length.  If your profile property only supports 25 characters, and you try to store 50 characters that is the returned value from BDC, you will get an import error for that profile. You would only know this is a problem by seeing this error in the SharePoint logs.  (Note: you cannot change the length of any of the out-of-the-box profile properties.  In this case, you would need to create a new property.)

When creating BDC import connections using the UI, the entity selected from the catalog must expose both Finder and Specific Finder methods.  This is necessary, even if you plan on using only 1:1 or 1:Many connection types.

Conclusion

In this article, we covered everything you need to know about importing User Profiles from BDC.  And you now know as much about this as I do.

Coming up in Article 7, we will cover the BDC API.  This will allow you to call into the BDC programmatically from your custom applications, web parts, or any other .NET code you need to write.

This article was posted on:

Everything You Need to Know about BDC: Part 6 of 8

I have worked with SharePoint as a framework since its first full scale release in 2001. Since then I have witnessed the steady growth in third party tools to provide either enhancements to weaker functionality provided ‘out of the box’ (OOTB) by SharePoint, or new features to complement existing core functionality.

Most third party utilities or web parts started to come out with the previous release of SharePoint (SharePoint 2003 & WSS V2). But with the release of Microsoft Office SharePoint Server (MOSS) 2007 (and WSS V3), there has been an exponential growth in a whole variety of end user or administrative focused tools.

Entire businesses have in fact emerged over the last few years with their sole business model aimed at meeting the clear demand for list aggregators, backup & migration utilities or reporting enhancements, to name but a few. Not forgetting the MVPs, Open Source efforts over at Codeplex and the work by specific individuals, often referred to in posts and blogs within the SharePoint community.

Growing pains

The growth in third party tools and expanding SharePoint community, whom also provide a vast array of often ‘free’ downloadable utilities, means businesses deploying SharePoint have never been better served and can arguably gain even more out of their investment in by using the latest Microsoft SharePoint technology.

However, my experience over the last few years with third party utilities, in particular with the very nature of new and often immature utilities and add-ons, has led me to believe that these tools can be fraught with issues for the unwary.

Issues encountered include:

• Code developed and tested in a MOSS environment, not specifically WSS (but released for both types of platform anyway, causing key functions purported to be available, that fail to work as they are not supported or available in WSS)
• Tools and utilities often unsupported and provided for download on an ‘as is’ basis
• Poor documentation to allow for effective and safe configuration and setup
• Little or no testing prior to release, hence tools are ‘buggy’ and unstable
• Unproven usability and functionality across load balanced environments
• Poor or non-existent end user support.

Even some of the better known third party tools on the market have, in my experience, been quite poorly developed, therefore the development companies are unable to respond to even basic support related queries or product bugs. The tools and utilities often have been clearly developed on and for MOSS environments, but advertised for WSS as well, with the originating developers knowing very well there will be problems with its lesser feature set. I suspect these are just growing pains of small businesses and hopefully their development processes and support services will improve over time.

Is your selected third party product really the right choice?

It is clear many of the tools available are extremely useful, otherwise they wouldn’t have been created to fill a gap in the first place, or sold so well. I do think however that businesses, or rather inexperienced individuals, often forget (or just don’t know) to consider fully the issues involved in deploying 3rd party products that have been developed outside of certified Microsoft and other core platform environments.

What was once your relatively clean, core and stable environment has now been ‘dirtied’ by dlls’, web .config changes and registry settings. This provides you with the risk of a potential level of instability that is unacceptable, resulting in endless hours trying to troubleshoot and resolve issues that could have been avoided in the first place. Hence embark on deploying such tools without the proper due diligence at your peril!

Questions regarding third party tools I would suggest you consider are as follows:

• Supportability - Does the organisation provide timely updates and bug fixes to the components? Will it be supported in 64 bit platform? Does it need to and actually work across different browsers? What do others say about the product in the community? Who will pay for supporting it internally? What if the person whom installed it leaves?
• Code updates - Do you need and hence have access to the source control for your developers to make and support changes in-house? What is the roadmap for product updates following service pack updates issued from Microsoft? Will the product be affected by service pack updates from Microsoft?
• Robustness/Stability - What type of testing has been carried out prior to release? Was the code developed on both MOSS and WSS platforms? Has the product been tested on load balanced farms? Does it work across multiple farms? Are there any performance issues on lists or indexing? Does it impact on existing OOTB features or other third party tools deployed? Will it affect existing pages and data? How is it installed, WSP, STP and or DLLs?
• Scalability/Security - What level and type of security access does it need in your farm? Will it scale as your deployment grows from single to multiple servers?
• Cost - What are the overall costs for deploying third party tools on your live, pre-production and development environments? What is the true annual cost of support?

Ultimately then, organisations can and do gain a tremendous amount of value for money from their investments as long as they invest wisely. However, I think a sizeable amount of businesses will have had numerous issues to do with performance, functionality, reliability or stability when using third party tools.

Due diligence

To reduce your exposure to such issues when purchasing third party SharePoint tools and utilities, you should carry out an appropriate review and justification process with these products.

I recommend the following as a guide:

• Understand in detail what it will take to provide an evaluation & test of the third party tool(s) assuming you will set up an separate environment to do it
• Document a list of business and or technical requirements you need to fulfil, matched by a list of product features stated. Compare & evaluate
• Review if possible on ‘pre-production’ evaluation environments in as close to a ‘like for like’ scenario as you can afford (UAT preferred)
• Read the specialist SharePoint forums and appropriate feedback from others who have used a particular product before deploying and essentially gauge a view on others experience of using the product
• Ensure you have a backup/restore approach that works (and has actually been tested), if you need to rollback following a tool/product deployment due to issues or other constraints
• Ensure you have considered your pre-production and disaster recovery environments in your planning, testing and budgets
• Consider documentation needed to not only install, but to provide support to your helpdesk team responsible for support and overall governance
• Before you embark on the process of introducing 3rd party tools really do look at the feature set provided out of the box and understand if minor changes to existing requirements can be made to avoid introducing such products or bespoke changes
• Where a third party tool is to be made available to end users, ensure the appropriate business testing and training is planned and made available in advance of any trial or pilot deployment.

Conclusion

The simple reason for this post is to make you aware of the dangers of introducing third party tools into your SharePoint environments and to recommend a series of steps to take to help you understand and decide if a third party tool is right for you. Third party tools can and do add real value, but be sure that these tools do not interrupt your core SharePoint environment.

In summary, you must ensure that you have some factual assurances that deploying any third party component is truly going to save you money or provide some other worthy and tangible benefit.

Measures must also be taken by you to ensure that uncertified tools are not going to damage your existing core SharePoint environment and that they are manageable and acceptable costs in terms of support longer term.

Finally, as I posted in the SharePoint Magazine recently, you potentially ‘pay’ for your bespoke changes and arguably 3rd party tools, to some degree, several times over. Consequently, do your homework before you download that evaluation web part and press setup.exe!

Regards,

Andrew

This article was posted on:

Using third party tools in SharePoint

Ready? Let’s get started!

The terms “Bug” and “Debugging” are facts of a programmer’s life. Debugging in general is a lengthy and tiresome task.   In my mind, for you to be a good SharePoint developer, you must know how to troubleshoot your applications. The trick is SharePoint debugging can be quite complex for those who are new to the platform, that’s why I decided to kick off my series of articles with how to simplify Debugging SharePoint code before we dive deep into the world of SharePoint programming. At the end of the article we’ll create the “SharePoint troubleshooting toolbox” that we’ll frequently use in our series “Getting Started to SharePoint Programming“.

Simplifying SharePoint Debugging By Creating The Troubleshooting Toolbox

Debugging in WSS or MOSS is similar to debugging an ASP.NET application but unfortunately debugging custom code in SharePoint isn’t as easy as just pressing F5 from inside Visual studio. While SharePoint provides an excellent platform for developing Web applications, debugging them can be a bit of a pain. In this article I will give you some debugging tips that could make your life easier and will help you deal with the silly “An unexpected error has occurred” screens. Also I will shed the light on some tools provided by the SharePoint community that facilitate the troubleshooting of SharePoint applications and error isolation and accordingly increasing your productivity.

Five main resources are available for you when you experience a problem:

1. Detailed Error Messages
2. Attaching the Visual Studio Debugger to W3WP.EXE
3. SharePoint Trace Logs
4. Windows Event Logs
5. Debug And Trace

1. Detailed Error Messages Vs. The Annoying Page!

Usually, an error message will be your first indication that something went wrong. Unfortunately, SharePoint uses a user friendly (Yellow & Blue) error page to show that a problem occurred. Yeah, they call it the user friendly page, but we, as developers, call it the Annoying Page. The first time I wrote SharePoint code, I received a screen like the one shown in the figure below:

This error page points out that an assembly has thrown an unhandled System.Exception. Sometimes, the user-friendly error page will specify the nature of the error if the code that threw the exception used something more specific than System.Exception or if it included a message in the Exception class constructor.

We need to get a more detailed Error Message than the annoying “Unexpected error has occurred “. This can be achieved by doing three modifications to the Web.Config file from the virtual directory containing your SharePoint application. These modifications are listed in the next table.

No Custom Errors shows the full error to every client, every time. This is typically used Development environment, since there are no clients using it.

I recommend using the Debug Config feature to automate this process. This is just a feature that when you activate on a web application, it automatically tweaks the Web.Config of the specified web application across the farm.

2. Attaching the Visual Studio Debugger to W3WP.EXE

Now you have the standard ASP.NET error page with a stack trace , OK this is very helpful but you don’t want to guess what the problem might be and randomly change your code to fix it.

First thing you need to do to avoid the old fashioned trial and error technique is to enable debugging. This could be achieved by setting the compilation element debug attribute to true in Web.Config from the virtual directory containing your SharePoint application, otherwise, breakpoints inside Visual Studio will be shown but will never be used.

Once you’ve done that, there is one more thing you need to do in order to be able to debug your SharePoint code in Visual studio which is attaching a debugger to W3WP.EXE, Attaching a debugger in Visual Studio will allow you to step through the code and find exactly where the error occurs.

It’s really handy but there are three questions that I always receive from my colleagues when they get started to SharePoint programming, so I decided to share the answers with you.

The first one is: Sometimes, I set breakpoints and attach to the process but Visual Studio skips loading the symbols if I’m debugging a deployed assembly to the GAC and I fail to debug the code

Easy, Open Tools -> Options ->  Debugging.

You’ll find an option labeled Enable Just My Code (Managed Only) as shown in the figure which is checked by default. Uncheck this option to be able to debug the assemblies located in the GAC. There is a common myth among .NET developers in general and especially SharePoint ones, that in order to debug assemblies that have been deployed to the GAC, you need to copy the debug symbols (PDB File) to the GAC as well. This was true in the early days of .NET but this is no longer true.

The second one is: When I try to attach the debugger to the W3WP.exe process to debug, I always see multiple instances of it. Which one should I attach to?

Ok, this is easy too. Just follow the following steps.

• Open the command prompt window, run IISAPP to get a list of the current instances of W3WP.exe.

• Note the PID of the instance that corresponds to your web application.
• Now return to VS, select Debug -> Attach to process and attach to the W3wp.exe instance with an ID equivalent to the PID you got in step 2 -> click Attach.
• Now you can trace through the code and find the error causes easily.

But the question still remains, why sometimes are they more than one w3wp.exe instance? This is because the SharePoint Administration Site Collection and the SSP Administration site always have their own Application pools for error isolation purposes.

And the third question is : Is it possible to debug Inline code blocks ?

Actually, I hate Inline Code Blocks and it’s really a good practice to avoid it due to the performance issues when the JIT compiler compiles them but YES, you can do that.

So now you knew the process of debugging? Very boring and repetitive huh? Hmm, Can’t that be automated?
Fortunately Jonathan Dibble, a brilliant developer, created a very handy “Debugger Feature” that can be installed and activated on a SharePoint site to automate this process. Once activated, the Debugger Feature adds an “Attach Debugger” menu item to the Site Actions menu.  Extremely helpful and faster than doing that from visual studio.

3. Unified Logging Services ( SharePoint Trace Logs )

Sometimes you receive error screens while activating a feature or creating a Site Collection or during deploying packages.  For instance, a couple of months ago I got an annoying error “File not found” when creating a site collection from a custom template!

SharePoint trace logs are at your disposal when you need to troubleshoot errors occurring before and after custom code; they are very useful in the deployment and provisioning operations as they are the main sources of information about everything happening inside SharePoint.

Most of SharePoint components use the Unified Logging Services (ULS), those logs are normal text files and you can find them in the 12 hive at 12\Logs. You can tweak the ULS settings via the Diagnostic logging link under Logging and Reporting in the operations section of SharePoint central administration to control the verbosity of events captured in the Log files, you can modify the settings per category or for all the categories as shown in the figure; thus modifying the number of events captured. The list entries are sorted in order from most-critical to least-critical. I always just set all of the categories to verbose because it’s sometimes difficult to specify the categories needed to be changed when a problem occurs.

If you want to tweak the verbosity setting per category, make sure to leave the default value of the General category for trace logging as it is (Verbose) because entries about the provisioning and deployment processes are logged under the General category. Also be aware that updating all categories will make you lose the changes you made to individual ones.

When you choose to increase the number of events captured by the ULS, you will find the log files very cluttered and hard to use, and SharePoint provides no built-in viewer. To fill the gap and facilitate troubleshooting, free third-party tools exist like LogViewer, SPTraceView and WSS/MOSS Log File Reader .Some utilities including SPTraceView will aggregate log data from more than one server in the farm.

Next figure shows LogViewer in action.

I’d also like to point out that you can write code that incorporates logging to MOSS logs. Writing to the same trace log alleviates the need for developers to log their development information in other places such as the Windows Event Log, which is more commonly used by system administrators.

The only limitation to using the code snippet above is that you cannot set the log event level (e.g. low, medium, and critical) as it’ll always display the error level in the trace logs as High. The logger is located in 12\ISAPI \Microsoft.Office.Server.dll and therefore it’s only available with the MOSS install, not WSS 3.0.

What if you need to write into SharePoint trace logs while having full control on the error level? This is a pretty easy thing to do, because MSDN has a wonderful sample of code that you can reuse. I always use this code in my features.

Simply add the class to your project and then you can write to the logs using something like the following code.

TraceProvider.RegisterTraceProvider();
TraceProvider.WriteTrace(TraceProvider.TagFromString(”XXXX (must be 4 letter tag)”), TraceProvider.StringToSeverity(”Exception or Information”), Guid.NewGuid(), “Method Name”, “Assembly Name”, “Project Name”, “Message”);
TraceProvider.UnregisterTraceProvider();

4. Windows Event Logs

You should include Windows Event Logs as part of your normal troubleshooting because it sometimes contains useful entries that SharePoint is unable to log via ULS. You can increase and decrease logging by severity level in the same way you do for ULS.

It’s possible to log errors to the system event log from your custom code but you need to promote the privileges as shown.

5. Debug And Trace

After you take your SharePoint site live, you are not the only one who is using it anymore (hopefully), so you need  effective plans to track down errors.

System.Diagnostics.Debug and Trace statements are very useful in this case. Used in conjunction with DebugView you can determine what has gone wrong! Debug Calls are removed from the release builds, so you should use them in your development environment but Trace Calls remain in release code and so will add an overhead to the code execution yet they are very helpful when you cannot attach a debugger  (for instance on the staging or the production environments) . Since manually adding trace calls might be time consuming, I would strongly recommend using a tool such as ReSharper to drop in these statements quickly rather than relying on typing or copy/paste. The next figure shows the trace output using DebugView.

SharePoint Troubleshooting Toolbox!

Well, let’s create our debugging toolbox, the following utilities should be included in our SharePoint debugging toolbox; we’ll certainly need them in our long journey with SharePoint development.

• Debug Config Feature
• Debugger Feature
• LogViewer
• SharePoint Manager
• SPTraceView
• WSS/MOSS Log File Reader
• DebugView
• SPDisposeCheck
• ReSharper
• Fiddler (HTTP Debugging Proxy which logs all HTTP traffic between your computer and the Internet)
• LogParser ( for IIS log analysis )
• Araxis ( File comparison tool )
• CLR Profiler (The CLR Profiler allows developers to see the allocation profile of their managed applications)

Summary

This was the first article in a series that will help you getting started with SharePoint programming and I decided to introduce you to some of the troubleshooting utilities and tools that can really make your life easier before getting our hands dirty with SharePoint code since troubleshooting can really be a nightmare for those who are new to the platform.
Remote-Only Custom Errors allow you to display custom errors only to remote clients. This means that if you are browsing your SharePoint site locally, you will get the fully detailed error messages. However, anybody else will receive the standard SharePoint page.

This article was posted on:

Simplifying SharePoint debugging by creating the troubleshooting toolbox

• Part 1 - Introduction

In our everyday work we all use several Line-of-Business systems as well as all of us know well the Office client applications. Although the integration between them is not too common yet, I’d like to show you the advantaged and disadvantages, pros and cons of the integration, and give some useful tips and tricks if you decide to make some OBAs.

In the first part I was introducing you, what the OBA (Office Business Application) means, what parts and components can you build them from, and what are the basics of an OBA building. In the second part we’ll see the scenario about the full demo what is going to built in the following parts.

Background

Nowadays the expression “Software + Services (S+S)” is very-very fashionable as well as important and promising. Few weeks ago the basic topic of the PDC 20008 was the cloud computing, Azure and related things. My current articles are not about them (although they are very-very interesting and exciting things and we should be very happy to be the part of that story), but I can’t avoid this.

So, of course you can have your own, on-premise softwares as you need them. Nowadays you can buy them as services yet (Software as a Service - SaaS). These services are hosted somewhere in the cloud, in huge data centers - but doesn’t matter where and how, you simply get capacity, scalability, maintenance, etc. as you need, and you don’t have to have resources on your side anymore.

If you would like Software + Services, all together, you have to plan carefully what you should have on-premise (for example client applications), and what you can (or should) place to the cloud.Basically you have a third option too: you can give your applications and services to a hoster, which is a way between on-premise and cloud services. From my view logically this is almost the same as you host them (outsides of the walls of your office), so I’ll talk about a real, very clear S+S architecture in the followings.

Of course, my focus is based on Office Business Applications. First of all, let’s see what does S+S mean in the OBA world. The Software side is very simple: you have your Office clients on the client computers in the offices, and/or you have SharePoint on the server-side. But the SharePoint not only have User Interface and client services to the users: it has some Services as well, so it can be on the both sides in the same time. For example, these services can be the Enterprise Search, LOB system integration, Content Management, etc. which are accessible by the SharePoint UI, Office clients and other (even custom) applications as well.

The Story

In the followings I’m building a very simple application based on the idea that we have a company what has a typical architecture with a lot of separated LOB (Line of Business) systems, huge amount of connections and interfaces between them (like a spider web), as you can see on the following picture. The end-users access these systems mostly via dedicated clients.

Because of several reasons, this company decides to make a very new structure to their architecture, and divide their LOB Systems (with data and processes) to two groups: the first group is located on-premise afterwards, but the others going to the cloud (as services). The second important issue is the question integration: what systems should be integrated with each other, and which ones have to be separated?

Basically, the integration decision might seem to be technical issue, but actually this is a business one. During the planning of this architecture, you have to answer a ton of questions:

• Do you want to build architecture with collaborative systems, or need independent applications?
• What kind of collaborations do you need in your company?
• Do you have/ need solution-oriented or task-oriented systems?
• What is your primary goal: to have set of systems with well-defined, global responsibilities, or separated applications with separated responsibilities?
• How important are scalability, maintainability, robustness etc. to you?
• etc.

Well, our company has decided GO! - so let’s see what functionalities do we need, and what the first steps are during implementation.

Functionality

Our company needs a complex document management system, with SharePoint-contents, custom Office extensions, and some data coming from the cloud. The following picture shows the basic architecture of the system:

So, from the client side you can see two kind of User Interfaces: your Office 2007 applications (Word) with OBA addins, and MOSS sites with custom contents. These contents come from not only MOSS itself, but from SQL Data Services and SQL Server Reporting Services as well. On the following pictures I show you, what the users can see in their Word 2007, and on a MOSS page:

Of course, to make your system do things like these, you have to configure and develop a lot. But don’t worry, it’s much more easier, and much more beautiful that you’re expecting, believe me! In the following parts I’ll show you the following steps to make your own Service-based OBA:

Part 3. Client side - Office 2007 add-ins
Part 4. SharePoint - The server
Part 5. SharePoint - Business Data (LOB Systems)
Part 6. SQL Data Services - My favourite LOB System
Part 7. Conclusions

Stay tuned, next parts are coming soon!

This article was posted on:

The World of Office Business Applications (OBAs) - Part 2

We are coming near the end of this 6 part series, but don’t worry, it’s not the last you’ll hear of me! This part focuses on ‘Why develop leveraging the SharePoint Platform?’. This is really targeted at ASP.NET developers specifically, but also other web application developers from other walks of life also. Or anyone currently working with Vanilla SharePoint and wanting to dig a little deeper!

Development Languages

As discussed in the Part 1, SharePoint leverages ASP.NET and NET WF among other languages such as XHTML, CSS, XSLT, XML and JavaScript. Most ASP.NET developers will already be proficient with these languages making the learning curve pretty much around the Framework itself. The Collaborative Application Markup Language (CAML) is about the only new language that is new to SharePoint Developers which is used to build queries in XML.

The SharePoint Framework

One of the first things to remember is that SharePoint is a Platform and NOT a Product. SharePoint is there to be leveraged. It is pretty much useless vanilla out of the box from a fresh installation as an end user. It needs customisation and development implementations for a Solution to be usable and successful.

Functionality already there

As discussed in Part 1 and Part 2, there is a lot of functionality out of the box and it is worth being aware what this is to see if you can get any wins out of reusing this in your solution implementations.

Plumbing already there

SharePoint provides lots of plumbing out of the box that you’d typically be writing yourself or plugging together a bunch of components. Common bits you have to put together are: User interface Framework, Windows Services, SQL Databases, Persistence Layers, CRUD UI screens, Security, Workflow engines, File Upload, Deployment Framework etc. All this is there at you disposal with a few lines of code or clicks in the User Interface. This saves a significant amount of time up front.

Learn the Framework

Like the .NET Framework, some things are hidden away in various namespaces and often you’ll find developers writing functions that already exist. There’s plenty of content out there to ensure you don’t make this mistake. I personally find MSDN currently a bit dry as the structure is based on each object in the object model hierarchy and it’s methods and properties. Often these don’t fit into a scenario where you’d use them in isolation. The scenario based examples are littered all over the blogosphere and more recently being structured on the SharePointDevWiki.com. MSDN do have these scenarios, but they are just written into one long list of articles with no structure which makes it hard to discover what you’re looking for. SharePointDevWiki.com gives you the best of both worlds.

More than one way to do things

As I discussed in Part 4, there is more than one way to skin a cat in SharePoint and it is important to agree on a standard approach within each SharePoint environment for consistency. Being aware of each approach and it’s strengths and weaknesses is just the start of the implementation process!

Infrastructure Handled for You

SharePoint is becoming more and more common across IT infrastructure and you can bet your bottom dollar it’ll only get more common as time marches on. The platform allows you to build solutions and not have to focus on things such as: Scalability, Disaster Recovery, Performance, Interface Design Frameworks, Security and Deployment. All this stuff is part of the plumbing of SharePoint Infrastructure and managed by your Infrastructure team and propeller head Architects. More often than not, they’ll be a few guidelines to follow when building your solution and that’ll be the last you hear of them! Please don’t get into the habit of doing this stuff as well as development, see my recent post on ‘The SharePoint Implementation Market needs to grow up!’ to find out more on this.

Frameworks Evolve

One thing to remember with Frameworks is that they evolve over time. SharePoint has already gone through 3 major versions with another just around the corner in Q1 2010 (SharePoint 14). When you implement your solutions on top of SharePoint, without any extra effort (if you’ve done things correctly) you get a whole heap of extra functionality when you upgrade to the next major version.

Extend the Framework

You’ll find people rewriting functionality because it doesn’t work “quite right”, but realistically people should be extending the framework. Often this is purely down to the skill level as a .NET Developer or in Power Users case…no skill level at all. If you rewrite or hack the current functionality, you’ll find that updates will overwrite your changes, extending the framework ensures they still exist. You’ll probably have to do some integration testing to ensure your extensions still work on top of the new version. This is happening more and more as Power Users find ways to work around the Development approach with tools such as jQuery, see my post on ‘jQuery: the SharePoint Band Aid’ for more information on this.

Maturity of the SharePoint Dev Community

As I mentioned recently the SharePoint Development community has got a long way to grow to be respected. I would recommend reading that post to get a deeper understanding of what I mean. I’ll cover off a few things here that are relevant to the objective of this post:

Rapid Application Development

Like any platform, SharePoint is great a rapidly building applications and is therefore great for Prototyping. Be extremely careful to set the expectations to the End User that these are Prototypes and not the real thing. You will need to build the end solution with a lot more rigour than throwing components together to get an end to end solution running to pilot.

Agile Methodology

Because you can rapidly build solutions you will find development teams having very short iterations producing extremely rich functionality. This lends itself well to an Agile approach to development, which is where the Development Market should really be, moving away from the traditional Software Development Lifecycle (SDLC).

Test Driven Development (TDD)

Andrew Woodward is leading the charge with SharePoint and TDD. There is also ongoing documentation on the community happenings around this on the SharePointDevWiki.com. Eric Shupps has just posted an anti-TDD SharePoint post here which has caused a lot of discussion around the whole paradigm. I take the stance that it is suitable in certain scenarios, those of which I have not had time to define yet, but will do on the SharePointDevWiki.com shortly. If you are currently in th e.NET space and using TDD, the community would welcome your feedback as it is still in limbo!

Source Control

A lot of implementations can be done without writing a single line of code and some can be written with a few. Often it is just all too hard to keep a repository of these “little” changes..but we’ve all seen an avalanche right? Don’t be fooled by teams that say you can’t use source control with SharePoint Implementations or by those who say SharePoint does it for you! That is not totally true! You should place as much in source control as possible outside of the SharePoint environment and it is completely feasible!

Development Tools

Unfortunately the Development tools aren’t what they should be for such a used Platform, so much so that the community has had to lend a hand! I advise you to get familiar with the tools that are available and their limitations. I also advise you to take a look at what could be possible in the future with a community project to build the Ultimate SharePoint Development tool!

There has been some sneak peeks into VS2010 (and VS2010 UI look and feel) as well as the introduction of VSeWSS 1.3 CTP which has come along way from VSeWSS 1.2. But the community still seems to be backing WSPBuilder very strongly due to the limitations of the VSeWSS product paradigm where it doesn’t conform to the 12 Hive structure like WSPBuilder and STSDev.

Community Content

As discussed in a recent blog post, the SharePoint community is simply unprecedented against any other platform out there! I have never seen a community so eager to share their knowledge. Go to Google and do a search for OpenText ‘RedDot CMS’ or ‘SAP’ or ‘Business Objects’ and see how much information is online to help the community on implementing that platform. There are 500+ blogs, forums, the SharePointDevWiki.com, podcasts and webcasts, I listed a few below but keep up to date with this list here.

Blogs

At a minimum you should read these blogs to keep up with SharePoint Customisation and Development:

• Microsoft SharePoint Team Blog
• SharePoint Designer Team Blog
• Andrew Connell (WCM)
• Heather Soloman (CSS/UI)
• Joel Oleson (Architecture)
• Mike Gannotti
• SharePointDevWiki.com
• SharePointMagazine.net
• EndUserSharePoint.com

Podcasts

• SharePointPodShow.com – click here listen to my interview
• MOSS Gone Wild

Webcasts

• Channel 9 - SharePoint
• Mike Gannotti (Webcasts)

Forums

• MSDN SharePoint Forums

Twitter

• Twitter Grade | SharePoint – note to SharePoint Twitterers – make sure you mention SharePoint in your profile!

Training Opportunities

There is an enormous amount of options with regards to training in SharePoint Customisation/Development.

Classroom Based

Classroom based training is spreading like wildfire with a variety of options in most countries across the World. These courses tend to focus on the Framework fundamentals currently rather then leveraging the tools to give Developers a grounding…without this, and knowledge just of the tool would mean that debugging code would be incomprehensible.

Online Based

Microsoft along with other Vendors are providing lots of rich online training, including Virtual Labs. As a trainer myself, I find that online training is not suitable for everyone and that class room training supported by a trainer who knows the platform inside out is a lot more valuable.

As I’ve mentioned before there are some great places to start learning:

• Do Less, Get More, Develop on SharePoint
• Ramp Up
• Patterns & Practices: SharePoint Guidance
• SharePointDevWiki.com

There are paid labs too:

• Microsoft e-Learning: SharePoint
• Ted Pattison Group

Books

It seems every time I check Amazon there is another SharePoint book being released. They are getting more and more specific to the particular capability areas of SharePoint such as BDC, Workflow, InfoPath which allows Developers to deep dive into the areas they are trying to implement.

Check out Amazon for SharePoint or the SharePointDevWiki.com book list.

The Cool Stuff to play with in the Toy Box

There is plenty of cool activity in the development space to keep us Developers happy and challenged…which will obviously need keep passionate about the Platform. It is also a great way to differentiate yourself in the SharePoint Community as mentioned when listing the bloggers out there.

Silverlight

Silverlight is making inroads into the Internet with its powerful Video Streaming technology but also because of the implementation process that separates Design from Development. There has been some posts already surfacing on integrating Silverlight into SharePoint and also rumours of SharePoint 14 improving the UI using Silverlight.

TDD

As mentioned above, TDD is a hot topic, and would also be a great area to focus your existing skills on and apply them to SharePoint Implementations. I encourage you to check out the posts around on TDD.

WCF

WCF is still relatively new and with those long horrible xml configuration files going to take a while for the majority of Developers to move away from standard .NET Web Services. SharePoint doesn’t support WCF out of the box but there are posts out there on this.

Azure

A post for Developers wouldn’t be ‘cool’ if I didn’t mention Cloud Computing SharePoint Online is SharePoint in the Cloud. There are plenty of post on it and what you can/can’t do if the platform is hosted up in the cloud.

jQuery

So I caused a bit of a storm in a teacup with my ‘jQuery: the SharePoint Band Aid’ post. jQuery + SharePoint is not going to go away and there are some great examples of how powerful it can be. Microsoft have picked up jQuery and are running with it for SharePoint 14 also.

PowerShell

PowerShell (PoSH) is another scripting language that seems to have taken the SharePoint world by storm. If you are a PoSH wizard, then you should start showing off your stuff by manipulating the SharePoint Object model. Check out these PoSH posts to see what others are already doing. SharePoint 14 will also have a lot of PoSH stuff going on for you to leverage.

Conclusion

So hopefully this gives you a feel for what SharePoint can do for you as a Customiser/Developer and also where to find more information to help you.

I encourage everyone who is going down this path to help collaborate on the SharePointDevWiki.com to make experiences for new people moving in this direction easier in the future.

This article was posted on:

Leveraging the SharePoint Platform (Part 5) - Why use SharePoint as a Development Platform?

If you’ve been following our BDC series, we’ve previously gone through the following topics:

1. Part 1, BDC Technical Architecture
2. Part 2, Application Definition File (ADF) and its Development
3. Part 3, Developing an ADF to Connect to Web Services
4. Part 4, Using BDC Web Parts to Consume BDC Data and Embedding BDC Data in Lists

• How can I find the customer information for Great Lake Good Market in CRM?
• How can I find the projects and resources details in Project Server 2007?
• How can I find an employee’s legal name in HR system?

With the help of the BDC Search, these all become possible. The Enterprise Search in MOSS can gather, index, and offer full-text search on all business data (entity instances) from applications registered in the BDC. Not only does it enable us to find answers in MOSS, but also it allows us to do so without having to leave SharePoint to go to these systems individually. It makes integrating search into existing line of business (LOB) applications like a CRM solution easier.

For example, if we register the Northwind sample database in the BDC, Enterprise Search can crawl it and return customer data in search results. In addition, the Enterprise Search feature’s extensibility enables us to create customized search experiences. We can create a custom search tab in the Search Center where our users can search for customers in the database, and we can also create a custom UI for the search results based on which we can also take action. So let’s dive right into it.

BDC Search Overview

As we’ve covered in prior articles, surfacing the BDC data to SharePoint using BDC Web Parts and lists is just first part of the BDC saga, it can also enable you to index and search on the BDC data. The following figure is an example of searching for information about a customer and then displaying both structured data from the database and unstructured documents and content stored within SharePoint.

Figure 1: Enterprise Search with BDC

Here’s another example of leveraging BDC to index Project Server data and enable us to search the different objects (projects, tasks, resources, custom fields, lookup tables, and so on). This is of great value because out of the box the Project Server data is not indexed. Only the objects in project workspace are indexed such as Issues, Risks, Documents, etc. For instance, if we are looking for project detail information for the Enterprise resource planning (ERP) project, we can search our Project Server and find the answers as shown in Figure 2:

Figure 2: Search Project Server with BDC

The BDC is truly a catalog of business applications that helps bridge the gap between MOSS and business applications by bringing in key data from various applications to SharePoint sites, lists, searches, and user profiles. Let’s revisit the BDC architecture from Part 1 with both XML Metadata and Search components highlighted in Figure 3.

Figure 3: BDC architecture with emphasis on the Search

Firstly, we’ve highlighted the XML Metadata because it is the key element in the Application Definition that makes the BDC Search possible. BDC operates off of an XML-based Application Definition File (ADF). The metadata in ADF enables MOSS to crawl the BDC data source and add them to the search index.

Secondly, we highlight the Search component because the search engine uses a protocol handler to crawl and a security trimmer to filter results. BDC uses these just like third party implementations when supporting BDC searching.

BDC Search enables us to quickly find content in business data through search queries. It also integrates with search scopes and search center to become an integral part of MOSS Enterprise Search. Here I’ve created a custom search tab and a search scope for the BDC data to create a unique search experience in which the BDC search results are segregated from the rest of SharePoint search results. See Figure 4.

Figure 4: Customized BDC Search Experience in Search Center

By enabling the search of BDC data, BDC Search extends the SharePoint search capabilities. This extends the reach of the back-end LOB systems. Today most LOB systems don’t have Full-Text search capability and BDC Search can serve for that purpose.

However, out of the box the BDC Search does not support indexing unstructured data like documents in a document management system. This is because the protocol handler for BDC is meant to connect to external structured data only. With custom protocol handlers, it’s possible to make MOSS index unstructured data . For example, using Web services to directly support BDC we can enable SharePoint to search, list, and retrieve documents from an external document management system via Web services.

Now let’s look into how to enable BDC Search.

Enabling BDC Search

One might think once we import an ADF, we can start to search. BDC Search doesn’t come automatically, and we must enable it. To enable BDC Search, we must first make sure these two methods exist in the ADF for the entity you want to crawl:

1. SpecificFinder
2. IdEnumerator

These are the only required metadata for Enterprise Search to use BDC to index the business application’s data.

In Part 2, we’ve covered the ADF and its vocabulary: System (data source), Entity (table), Method (operation on entity), and Association (relationship). There are three primary types of methods: Finder, SpecificFinder and IdEnumerator. An IdEnumerator method returns a searchable list of unique IDs for an entity. This list enables indexing of the entities and SpecificFinder is then used to find details for the entity instance based on the specific ID. Therefore, IdEnumerator and SpecificFinder work together to enable Enterprise Search to use BDC to crawl the business data.

Now since we have a good understanding of what it takes, let’s put it in action. We’re going to use Business Data Catalog Definition Editor, a free tool from the MOSS SDK, to create our ADF (shown in Figure 5 below). Though one of the limitations of this tool is that it doesn’t create a Finder method, BDC Search doesn’t need use this anyway, so it works very well for creating an ADF that’s just used for searching. You can learn how to use this tool here.

Incidentally, both MOSS BDC Design Studio and BDCMetaMan will create these methods necessary to implement BDC searching.

We begin by adding a LOB system for our sample database, a modified version of Northwind database. We’re going to use the Customers entity that resembles one in a CRM system (or the like).

Figure 5: Create Application Definition

To index and search for Customers entity in the database, we’re performing the following tasks:

1. Define an IdEnumerator method in the BDC definition (Figure 6)
2. Define a SpecificFinder method in the BDC definition (Figure 7)

Figure 6: Define a special type of MethodInstance method IdEnumerator.

The IdEnumerator method returns a list of CustomerID records, which will be indexed by MOSS Search. So now we know that the sole purpose of IdEnumerator here is to enable Search. Here’s the SQL query:

Figure 7: Define a SpecificFinder

This article was posted on:

Everything You Need to Know about BDC: Part 5 of 8

Before we get going, however, I’d like to give you a small gift for your attention through this week. Exclusive to the first 25 SharePoint Magazine readers you can use the discount code SPMUX to get a $5 discount on the first issue of Understanding SharePoint Journal. This message will be removed once the first 25 people have got their issue.

What is a DelegateControl?

A DelegateControl is a SharePoint ASP.NET control that acts as a placeholder for content or other controls. Think regular ASP.NET content placeholders but with a SharePoint Feature deployment method, meaning you can activate content on a page as you would activate any other feature in SharePoint. You might have seen these already is you’ve looked into the default.master file of a standard SharePoint installation.

A lot of SharePoint default functionality uses DelegateControls. For example, the search box is really a DelegateControl, and when you install or activate MOSS or Search Server search functionality these products uses the DelegateControl to add their own search boxes to any existing site.

Your First DelegateControl

Let’s get our hands dirty right away and see how this gets done. Also, this is the example used by MSDN to show off the Delegate Control, so you might have seen this before.

1. Start by creating a new feature. If you use WSPBuilder, as explained in the “Fast Track to Feature Generation” article, use a Blank Feature item type, and set the scope to Web.
2. In your elements.xml file, add the following code:

<?xml version=”1.0″ encoding=”utf-8″ ?>
<Elements xmlns=”http://schemas.microsoft.com/sharepoint/”>
<Control ControlSrc=”~/_ControlTemplates/MySearchControl.ascx”
Id=”SmallSearchInputBox”
Sequence=”1″/>
</Elements>

1. Create a new file in the [12]\TEMPLATE\CONTROLTEMPLATES folder called MySearchControl.ascx. Open the file in Visual Studio

You may also deploy the .ascx user control using WSPBuilder. If so, add a folder in the Solution Explorer under 12\TEMPLATE called CONTROLTEMPLATES and add the .ascx file there.

1. In the MySearchControl.ascx file, add some sample ASP.NET controls, for example:

<%@ Control Language=”C#”%>
<asp:Label ID=”Label1″ runat=”server” Text=”No search here…”></asp:Label>

1. Deploy your feature and activate it before retuning to the front page. You should see something along the lines of:

The cool thing here is that once you grow tired of the support calls asking where the search box went you can simply deactivate the feature and your old search box returns.

Ok, this was a really simple, and likely completely useless example, but I just wanted to show how DelegateControls work. Let’s see if we can explore something a bit more useful.

The Structure of a SharePoint:DelegateControl

While the example above demonstrates how to use a DelegateControl and override its contents it is a simple and not very useful example. However, it serves as a starting point for the rest of this article, so I’ll explain what actually goes on.

Control types and Id

In the elements file we created earlier we added two attributes, the ControlSrc and the Id. These attributes determine which content (ControlSrc) should be placed in which DelegateControl (Id). Every DelegateControl must have a ControlId property when added to an ASP.NET page, and your Id attribute in the elements file should match this ControlId property.

The actual control is either a user control, in which case we specify the ControlSrc attribute and point it to our control, or it can be a server control, in which case we need to exchange the ControlSrc with a ControlAssembly and a ControlClass attribute:

<?xml version=”1.0″ encoding=”utf-8″ ?>
<Elements xmlns=”http://schemas.microsoft.com/sharepoint/”>
<Control ControlAssembly=”[STRONG ASSEMBLY NAME]”
ControlClass=”[CLASS NAMESPACE AND NAME]”
Id=”SmallSearchInputBox”
Sequence=”1″/>
</Elements>

You would of course exchange the [STRONG ASSEMBLY NAME] and [CLASS NAMESPACE AND NAME] with their respective values for your assembly and class. If you need help finding the strong name, just check out the post on ”How do I find the strong name of an assembly” at JustAsk or read on for a tip on how to do this if you have WSPBuilder installed.

Sequence

However, once we have defined which control to add to which DelegateControl we must understand a bit more about how the DelegateControl works to understand the remaining attribute Sequence.

The Sequence attribute basically tells SharePoint the order in which to process multiple controls. To understand how this works, consider having two features that both add content to the same DelegateControl. Now, a DelegateControl in a SharePoint page can set the AllowMultipleControls to True, in which case both the features’ content gets added, with the feature with the lowest Sequence attribute added first. The AdditionalPageHead DelegateControl is an example of a control that allows multiple controls to be added, and it is the only DelegateControl that allows this in the default.master that ships with SharePoint.

If the DelegateControl does not set the AllowMultipleControls to true, the only the feature with the lowest Sequence attribute gets added at all. The other features will not have their content added. This means that when you add a control to a DelegateControl you need to know which features have deployed content to the same DelegateControl, if any.

Adding a bit of flair

The AdditionalPageHead is a really useful DelegateControl. It is located inside the HEAD section of the default.master and it has the AllowMultipleControls set to True. This means we can add all sorts of goodies to the head using a feature.

Style sheets through DelegateControls

If you have read fellow SharePoint Magazine author Paul Culmsee’s series on SharePoint Branding you will know the difficulty facing designers looking to get their CSS added at the right time. Oh, and if you haven’t read the series it is well worth spending a few minutes reading through the first three parts at least.

Since the AdditionalPageHead DelegateControl is located right where we need to add any custom style sheets we can create our own control to add that style sheet and activate the sheet with a feature. As an added bonus, the AdditionalPageHead DelegateControl is also present in the application.master file, meaning we can add style sheets to the application pages as well, without ever touching a single Microsoft provided file.

Let’s see if we can solve Paul’s problem with a DelegateControl control.

Start with a new Visual Studio project, type WSPBuilder Project.

Add a new Feature with Receiver element. Next, add the following to your elements.xml file:

<?xml version=”1.0″ encoding=”utf-8″ ?>
<Elements xmlns=”http://schemas.microsoft.com/sharepoint/”>
<Control ControlAssembly=”[STRONG ASSEMBLY NAME]”
ControlClass=”[CLASS NAMESPACE AND NAME]”
Id=”AdditionalPageHead”
Sequence=”1″/>
</Elements>

Why use a Feature with Receiver instead of a Blank feature? Here’s a little trick, WSPBuilder does some real magic for you when building anything that requires you to input assembly information anywhere. Not only will WSPBuilder create a new class file for you, but it will enter the information you need for your Control element into the ReceiverAssembly and ReceiverClass of the feature.xml file in your new feature folder. Just copy the ReceiverAssembly attribute into the ControlAssembly and ReceiverClass into the ControlClass.

Delete the FeatureReceiver and FeatureClass from the feature.xml file, you wont need it, and it will break your feature activation.

Open the class file created by WSPBuilder for you, in the FeatureCode folder. Currently this is set to be a FeatureReceiver, so modify the file until you are left with this:

using System;
using System.Collections.Generic;
using System.Text;
using Microsoft.SharePoint;
using System.Web.UI.WebControls;

namespace StyleSheetDelegateControl
{
public class StyleSheetControl : WebControl
{
}
}

This is just the outline for our control, the rest of the code we will add to the RenderControl method in a few steps. First, let’s add a simple CSS file to add to our pages.

Add a new folder called LAYOUTS to the TEMPLATE folder in your solution explorer.

WSPBuilder will map this folder to the LAYOUTS folder in [12]\TEMPLATE. The LAYOUTS folder in [12]\TEMPLATE is again linked to the virtual path _layouts in any SharePoint site, so it makes sense to put our style sheet there.

Add a new item to the LAYOUTS folder, type StyleSheet and name it “MyCustomSheet.css” or something nice. Just don’t call it anything that is already in the LAYOUTS folder.

Your Solution Explorer should now look something like this:

Add something to the CSS file to test that your changes are actually happening. You might for example put

body
{
background: yellow;
}

into the file.

Open the class file again, and output the link to the new style sheet in the RenderControl method as such:

public override void RenderControl(System.Web.UI.HtmlTextWriter writer)
{
writer.Write(”<link rel=\”stylesheet\” type=\”text/css\” href=\”/_layouts/MyCustomSheet.css\”/>”);
}

Of course, if you named your file something else you would change the file name, but you know that. Also, I’m just linking to the root of the site collection (/_layouts/) and you might want to change that the use the SPWeb.Url property or something like that. this is just for illustrative purposes, right?

To test your new control, build your WSP file by right-cliking the project name in the Solution Explorer and choose WSPBuilder->Build WSP. To deploy, simply re-enter the WSPBuilder menu and select Deploy.

Finally, go to your favorite SharePoint site and activate the feature on the Site Settings->Site Features page. Depending on how fancy you made the CSS file you should see something like this:

Sweet! Not only have we created a method to dynamically add a CSS file to a page using a feature, but what is really cool is that this feature also works on application pages.

Nice trick, eh?

Further ideas

Now that you have seen how your can create a DelegateControl and have it run regular .NET code I hope you get inspired to take this idea further. For example you could create a solution that displays different style sheets for different users. how about displaying a style sheet only if the current user is site admin? simply replace the writer.Write line with:

if (SPContext.Current.Web.UserIsSiteAdmin)
{
writer.Write(”<link rel=\”stylesheet\” type=\”text/css\” href=\”/_layouts/MyCustomSheet.css\”/>”);
}

Now site administrators get and extra reminder that they are indeed running as site administrators and are thus capable of havoc, while regular users are get the standard interface.

Or, you could evolve this into a full-blown solution that allows users to select from a pre-defined set of styles, and have the control pick the right style sheet on an individual basis.

Before I end this article, I just want to share a few tips with you regarding utilizing DelegateControls.

Creating your own DelegateControls

DelegateControls support default content, which you can see demonstrated in the default.master in the TopNavigationDataSource control, around line 132. If no control has been added using a feature then the default content is used. When creating your own master pages, include DelegateControls for any part of the site that might potentially need to be changed, such as style sheets, navigational elements, images, etc. It is a lot easier to not use a DelegateControl when you have the option than it is to implement it later. Since the default content can contain any other ASP.NET control, all you have to do, to enable the option to override content using DelegateControls, is to wrap your content with a SharePoint:DelegateControl as such:

<SharePoint:DelegateControl runat=”server” ControlId=”[MyControlId]“>
<Template_Controls>
[YOUR CONTENT]
</Template_Controls>
</SharePoint:DelegateControl>

Then, when you need to change the content on a site, simply add a new feature that uses the Control element with the Id attribute you chose.

Second, when creating your own DelegateControls, think carefully about wheter you want to set AllowMultipleControls. If you add it, remember that _all_ features with Control elements matching your DelegateControl will be added. However, if you have good feature control it is easier to de-activate an unwanted feature than it is to add AllowMultipleControls afterwards.

The End of SharePoint User Experience Week

I do hope you have enjoyed this little tour of DelegateControls, and seen how cool it is to be able to insert content into any SharePoint page using a feature.

And, of course, I really, really hope you have enjoyed the SharePoint Magazine User Experience week, and that you now have a better understanding of the possibilities you have for creating or modifying the SharePoint User Experience. I can tell you that I have really enjoyed writing for you.

If you have ideas for future articles, comments, or questions, don’t be afraid to ask. You can contact me at furuknap<[at]>gmail.com. If your idea or question is really good you might just be featured in an article or even in the Understanding SharePoint Journal.

Now, if you’ll all excuse me, I have a development machine that needs some cleaning.

.b

This article was posted on:

User Experience Week: Using DelegateControls to Customize the User Experience

The Customizing the User Experience of SharePoint series aims to explain how the user experience works, from how the interface is built down to details on how columns in lists get created.

The series is also an exclusive preview of the topics from my upcoming book, “Building the SharePoint User Experience”, which deals with SharePoint user experience for 350 pages. Actually, as with this series, the book explains the SharePoint architecture from a user experience perspective.

Here is an article outline for the six parts:

Part 1: Overview of the default SharePoint interface from a technical point of view

In the first article we will look at how the default SharePoint interface is built. We will look at a site, going from top-down, explore some of the the default lists, the fields used to create the basic field types, which content types are available, and how list forms are rendered.

Part 2: Modifying the default experience

This article will show you which options are available for you to modify and improve the default setup. Learn how to override the default rendering of fields or forms without voiding your supported state.

Part 3: Lists and custom list forms

The third article will cover the basics of customizing lists using different views, custom list forms, and fields.

Part 4: Content types user interface

The next article will explore how you can utilize content types to display different input forms and display forms.

Part 5: Custom fields deep dive

Ever wanted to create a new field type? SharePoint enables you to do this and it is a very powerful tool for customizing the user experience.

Part 6: Fast track to feature generation

Writing custom lists with content types by hand can take a massive amount of time. In the final installment I will share with you some tools and techniques that makes list, field, and content type generation very fast.

And that is exactly what we’re going to cover today. We’ll explore how you can speed up development time using some simple tools and techniques that are freely available. So let’s get down to business, shall we?

Tools of the trade

Being an efficient developer is not about having the most tools, but about having the right tools. More importantly, it is about learning how to apply those tools in the right fashion. I’ll give you a few tips here, tips that enable me to create features, having complex columns, content types, and list views within minutes rather than hours.

I am going to assume that you have a copy of Visual Studio that you like. VS 2005 or 2008 will do, I’ll use 2005 here to make sure we support everyone. If you don’t have Visual Studio you should get that now, the Express versions will do.

Here’s what you’ll need in addition:

WSPBuilder

WSPBuilder is the coolest little tool for SharePoint developers. The main function of WSPBuilder, as the name implies, is to build WSP file. If you don’t know, WSP files are installable solution packages for SharePoint. What you should also know is that creating these WSP files is a **** ** *** ***.

WSPBuilder makes building WSP files a lot easier for us, but its true beauty lies in an extension that you can install to Visual Studio. Doing so gives you a set of tools which we will explore in a moment. For now, get the WSPBuilder and its Visual Studio extensions from the CodePlex link above.

SharePoint Manager 2007

SharePoint Manager 2007, or SPM for short, not to be confused with SharePoint Magazine, which is also SPM for short, is a SharePoint object model browser that allows you to inspect you SharePoint installation. Not just that, but SPM allows you to perform basic manipulation of your SharePoint sites, lists, content types, and columns.

Oh, and to make SPM even more perfect, it is absolutely free. Which is true for both usages of the abbreviation.

Let’s start learning how to utilize these tools to become a SharePoint development superstar.

WSPBuilder

Once installed you should get a few new project types in Visual Studio under the WSPBuilder node. Select to create a new WSPBuilder Project and name it something you like. The name here will be the name of the solution so pick something nice

Once your project is created you will see that you get a few items in your solution explorer. Most notably, notice that you have a 12 folder. This 12 folder will map to the [12] hive when you deploy your solution later. This in itself will save you a lot of time when deploying components that are not features, such as custom application pages, images, site definitions, and custom field types.

However, for our demonstrations here we will focus on creating features. After all, this is a fast track to feature generation, right?

Your first demonstration is to create a feature with an event handler attached. Of course, feature receivers requires you to deploy an assembly to the GAC, meaning you need to strong name your assembly, which entails setting the project to be signed, creating a new signing key, and building the assembly. Then, to tie the assembly to the feature receiver you need to build the feature.xml file, reflect on that assembly to get the strong name, and get the right class name before you past all that stuff into the feature.xml file. Then you need to build your solution, deploy the assembly to the GAC before you copy all your feature files into the 12 hive. Or you could build a DDF file and create a WSP solution file, which is about as fun as watching grass grow in real-time. An this is before you create your class file and remember which assemblies to reference and which class from which to inherit.

Or, you could simply right-click your WSPBuilder project, add new item, and select Feature with Receiver from the WSPBuilder node as your item type. Select which scope your feature shuold have, and optionally give it a nice name and description, and hit Ok. In one single and short operation you have created a new feature, configured your project to be signed, which, by the way, WSPBuilder did even before you added the feature, extracted the strong name from the assembly, written the feature.xml file, created the class file which inherits from the correct class, overridden all the right methods, and created an empty, ready-to-use, just-add-water elements.xml file.

Neat, eh? Oh, you haven’t seen anything yet.

Note that, by default, WSPBuilder will create a class file that just throws an exception whenever you try to do anything with the feature in SharePoint, so your first order of business is usually to open the class file and remove the exception throwing from all four methods.

Of course, creating the feature is only the first part of the job. Next you need to actually deploy the feature. This, it is rumored, is a very complex operation, requiring you to master the art of DDF file building. With WSPBuilder, however, all you need to do is right-click your project, choose WSPBuilder->Build WSP and you suddenly have a WSP file ready to deploy. And if you can’t be bothered to actually install and deploy the solution yourself, WSPBuilder handles that for you as well, by simply giving you a Deploy menu item inside the Same WSPBuilder menu you used to build the WSP file.

You’ll also notice that you have several other options on the WSPBuilder menu as well. Upgrading and uninstalling your solution is quite a time-saver, but you can also very easily copy files to [12] or deploy your assembly to the GAC if you need to do these steps manually for some reason. Also, what really saves me time when debugging SharePoint code is the ability to quickly attach the debugger to the IIS Worker processes. I know, this is quite possible quite fast using the regular menu in Visual Studio, but I just like to have this option a bit more conveniently located.

The only thing I’m missing from this menu is IIS Reset. Some changes are not picked up by AppPool recycling alone, so Carsten, if you’re reading this, please add IIS Reset to the WSPBuilder context menu.

By the way, Carsten Keutmann is the fool that has made both WSPBuilder and SharePoint Manager. I’m saying he’s a fool, because he would rival Bill Gates in wealth if he charged you just a percent of the time you save using his products. But no, he goes around giving this away for free. Which is very honorable, but quite follish. He is a genius in writing his tools, though, so I guess he is the most foolish genius there is.

But wait! We were talking about developing features. Let’s move on.

Extracting feature CAML.

You may not know this, but most SharePoint objects, such as columns, content types, and lists, can export the CAML used the create them. To get this XML you would grab the contents of the SchemaXml property of the object using code. So, for instance, if you want to extract the XML used to define a certain column you would call SPField.SchemaXml, and for a content type SPContentType.SchemaXml.

This is very useful because it allows you to create an object, such as a column, in the web interface of SharePoint, and then extract the CAML and put it in your feature for later deployment.

And this is fine, but a little awkward because you need to write code to extract the code. Or, you can have someone create a program that makes this a lot easier.

Enter our overly generous Dane again, Carsten Keutmann. Now, know that I really love you man, in fact, I plan on naming my first two kids WSPBuilder and SharePoint Manager, but you have got to start at least taking donations for your programs. Think of the Danish economy, you would support a nation from the taxes alone!

For the rest of us, Carsten has created another nice program, the SharePoint Manager, which you should have downloaded by now. SPM allows you to browse your entire SharePoint installation and view all the properties of all the objects, which is exactly what we need. I’ll walk you through creating and deploying a custom site columns in three easy steps:

1. Start by creating a site column in any of your sites. Make it something a bit complex, such as a multi-choice column (checkboxes).
2. Using SharePoint manager, browse to your site where you created the column and open the Fields note to locate your column.
3. Copy whatever XML is in the Schema Xml tab of SPM, less the XML declaration, and paste into your elements.xml which you created earlier. Deploy and you’re done.

How much easier can you create a new column? I absolutely love SPM, and when combined with WSPBuilder, you got a killer duo that will save you hours of development time on every project.

You can do the same thing for other objects as well. However, some of the XML that you get is not deployable right off the bat. For example, for content types you need to change the Fields element to FieldRefs, as content types only contain references to fields. Doing so you also need to change all the Field elements into FieldRef elements, which also means removing a lot of illegal attributes. But still, this is incredibly much faster than writing one from scratch.

Now, another really useful object with which you can attempt this is for Views. Creating custom views in CAML can be a difficult task, as we saw in part 3 of this series. However, if you create a view on a list and you want that view added to a custom list template you can, still in SPM, go to the list where you created the view and get mostly usable XML in the Schema Xml tab. I say mostly, because there still are some things you need to do yourself. For example you will want to change the View Url to something that is list relative and not an absolute path, which is the default when you export a list’s XML, and you would also need to set the WebPartZoneId and the SetupPath, but all in all this beats writing 2500 lines of code any day.

So long and thanks for all the fish!

It’s been a long week, and a lot of hopefully useful information for you. I’m going to stop for now and just recap what we have seen in this series.

We’ve explored a SharePoint site down to a fairly low level, leaning how the architecture of SharePoint really works and how you can use that knowledge to create a user experience that is much more in tune with your needs than the default out-of-the-box experience. You’ve seen how sites are built, how lists and list templates are created, how View CAML is not really so bad after all, you’ve seen how to create custom site columns as well as learned at least that you can create your own custom field types, although we didn’t get to explore this nearly as deep as we could.

Finally, I hope this article has shown you some tips on how you can create all these things very rapidly, utilizing the SharePoint web interface and some nice free tools to speed up the development of new features.

Thanks for reading all this, and please, if you like it tell your friends about SharePoint Magazine. If you don’t like it, well, you don’t need to tell anyone.

.b

Hey, what about the bonus article?

Oh, yeah, I nearly forgot. The topic of tomorrow’s article will be DelegateControls. I mentioned these briefly earlier, but since they are so powerful I felt I needed to elaborate a bit more on their use.

Also there will be a special surprise available only to the first readers of that article. Subscribe to the SharePoint Magazine RSS Feed and you will be notified as soon as the article is out. The early bird catches the worm, you know…

This article was posted on:

Customizing the User Experience of SharePoint: Fast track to Feature Generation (Part 6 of 6)