The Simple Programmer

Integration between Word 2003 and SharePoint 2010 using Managed Meta-data

2011-11-25T01:09:00.001Z

You cannot check out/edit/check in directly from Word 2003 if you are using managed meta-data in the list. Word 2003 does not seem to integrate properly with managed meta-data columns, and will refuse to check in the document.

This leaves teh document in an unsaved, checked out state if a user attempts to edit it directly from Sharepoint.

My only solution to this has been to create a new content type and remove the managed meta-data from it.

C# function to get the Content Type GUID from a Sharepoint list using web services

2011-11-04T21:02:00.000Z

Simple helper function to get the GUID of a list from Sharepoint web services.

Note that in the code below I have a class level object pointing to the Lists.asmx web service called m_listService already established , but I have left in the code to establish this connection in the comments as an example.

string GetListContentTypeGuid(string listName, string listID, string contentTypeName)
        {
            string defaultGuid = "";
            
            try
            {
                
                //ListsService.Lists listService = new ListsService.Lists();
                //listService.Credentials = System.Net.CredentialCache.DefaultCredentials;

                //XmlNode ndList = listService.GetList(fileInfo.m_listInfo.m_listName);
                //XmlNode ndVersion = ndList.Attributes["Version"];

                XmlNode contentTypes = m_listService.GetListContentTypes(listID, "anything");

                // Loop through the fields
                foreach (XmlNode node in contentTypes)
                {
                    System.Diagnostics.Debug.Print(node.Attributes["Name"].Value.ToString());
                    if (node.Attributes["Name"].Value.ToString() == contentTypeName)
                    {
                        defaultGuid = node.Attributes["ID"].Value.ToString();
                        break;
                    }
                }
                
            }
            catch (Exception ex)
            {

                throw new Exception("ERROR: Reading content types from target site.\r\n"
                    + ex.Message + "\r\nDetails: " + ex.InnerException + "\r\n" +
                    "Check the settings file to ensure that the list settings match the target site.", ex);

            }
            return defaultGuid;
        }

Sharepoint 2010 managed meta-data taxonomy field data using web services

2011-07-20T00:00:00.004+01:00

Sharepoint 2010 includes a very useful new concept: managed meta-data. This allows you to build taxonomic hierarchies, and provide document routing based on the meta-data.

Unfortunately, it seems that using this field type using a web service call is extremely complicated, and hardly seems to be documented at all (this is so typical of Sharepoint stuff...). Microsoft have given us a new web service class ("taxonomyclientservice") but there hardly any documentation on it all (http://msdn.microsoft.com/en-us/library/microsoft.sharepoint.taxonomy.webservices.taxonomyclientservice.aspx).

Here is an example that I have managed to get working (after MUCH trial and error) that will hopefully help you to do the following:

opens a list using the Lists web service,
iterates through the fields in the default content type for the list (you can remove this bit and just iterate through the fields in the list if you like),
if it finds a "TaxonomyFieldType" field, it then gets the "Shared Service ID" (GUID of managed meta-data service) and Term Set ID (GUID of term set defined on this field) required to run the Taxonomy web service call
sets up the CAML to run the Taxonomy web service call,
retrieves the taxonomy data for that field in XML.

I do not try to analyse what the taxonomy XML means - that is for another day!

Note that all my web service proxies are called "ListService" and "Taxonomys".

//get list data from webservice
ListService.Lists listService = new ListService.Lists();
// Use your own list name
XmlNode list = listService.GetList("My List Name");

// Let's have a look at it
System.Diagnostics.Debug.Print(list.OuterXml);

// Get the content types for this list
//Get our defined content type (see previous post on how to do this - you need your own values here!)
string contentTypeGuid = GetListContentTypeGuid("My List Name", "{GUID}", "My Content Type");

// Now get content type definition 
XmlNode contentType = m_listService.GetListContentType(mylistinfo.m_listName, contentTypeGuid);

//Get the fields from the content type
XmlNodeList nodes = RunXPathQuery(contentType, "//sp:Field");

// Loop through the fields in the content type
foreach (XmlNode node in nodes)
{
 // This stuff just sets up some variables based on the content type and skips over hidden fields
 string Name = node.Attributes["Name"].Value;
 if (Name == "ContentType") continue;
 
 string ID = node.Attributes["ID"].Value;

 if (string.IsNullOrEmpty(ID) || string.IsNullOrEmpty(Name))
     continue;

 bool hidden = false;
 bool ReadOnly = false;
 try { hidden = Convert.ToBoolean(node.Attributes["Hidden"].Value); }
 catch { }
 try { ReadOnly = Convert.ToBoolean(node.Attributes["ReadOnly"].Value); }
 catch { }

 if (hidden || ReadOnly)
     continue;

 string ShowInFileDlg = "";
 try { ShowInFileDlg = node.Attributes["ShowInFileDlg"].Value; }
 catch { }
 if (ShowInFileDlg == "FALSE") continue;


 string StaticName = node.Attributes["StaticName"].Value;
 string DisplayName = node.Attributes["DisplayName"].Value;

 
 // Now we can check the "Type" attribute
 string FieldType = node.Attributes["Type"].Value;

 if (FieldType == "TaxonomyFieldType")
 {
     // WE HAVE A TAXONOMY FIELD!!!

     // Lets get the shared service ID and the termset ID from teh List schema
     string sharedServiceIds = "";
     string termSetId = ""; 
     // jump a few nodes to get the correct bit of the schema (included for clarity)
     XmlNode childNode =  node.ChildNodes[1];
     XmlNode termNodes = childNode.ChildNodes[0];

     //Loop through these nodes until we find the information we need
     foreach (XmlNode term in termNodes.ChildNodes)
     {
         System.Diagnostics.Debug.Print("term = " + term.ChildNodes[0].InnerText.ToString());
         if (term.ChildNodes.Count > 1)
         {
             System.Diagnostics.Debug.Print("value = " + term.ChildNodes[1].InnerText.ToString());
             if (term.ChildNodes[0].InnerText.ToString() == "SspId")
             {
                 // Get shared services ID from list
                 sharedServiceIds = term.ChildNodes[1].InnerText.ToString();
             }
             if (term.ChildNodes[0].InnerText.ToString() == "TermSetId")
             {
                 // Get Term Set ID from list
                 termSetId = term.ChildNodes[1].InnerText.ToString();
             }
         }
  
     }
     
  
     int lcid = System.Globalization.CultureInfo.CurrentUICulture.LCID;
     string serverTermSetTimeStampXml = "";


     string termStoreIds = "<termStoreIds><termStoreId>" + sharedServiceIds + "</termStoreId></termStoreIds>";
     string termSetIds = "<termSetIds><termSetId>" + termSetId + "</termSetId></termSetIds>";
     //Always set timestamp to this
     string oldtimestamp = "<timeStamps><timeStamp>633992461437070000</timeStamp></timeStamps>";
     //Always set version to 1
     string clientVersion = "<versions><version>1</version></versions>";
     string timeStamp = "";

     // Taxonomy web service!!
     Taxonomys.Taxonomywebservice taxonomy = new Taxonomys.Taxonomywebservice();
     taxonomy.Credentials = System.Net.CredentialCache.DefaultCredentials;
     string resultXML = taxonomy.GetTermSets(termStoreIds, termSetIds, lcid, oldtimestamp, clientVersion, out timeStamp);



     //Loop through the XML
     string termValue = "MHA";
     string termGUID = "";
     string parentID = "";
     XmlDocument termSetXML = new XmlDocument();
     termSetXML.LoadXml(resultXML);
     XmlNodeList terms = termSetXML.GetElementsByTagName("T");
     foreach (XmlNode term in terms)
     {
         string termName = term.FirstChild.FirstChild.Attributes["a32"].Value;
         termGUID = term.Attributes["a9"].Value;
         try
         {
             parentID = term.Attributes["a25"].Value;
         }
         catch (Exception)
         {
             parentID = "";
     
         }
  
         System.Diagnostics.Debug.Print("termName = " + termName + ", termGUID = " + termGUID + ", parentID = " + parentID);   
     }                          
     
 }
}

If you would like to know how to analyse the attribute codes returned in the XML, then take a look at this blog entry: http://blogs.solidq.com/sharepoint/Post.aspx?ID=37&title=SharePoint+2010%2C+Managed+Metadata%2C+TaxonomyClientService+in+depth

Mark ContentType as default in Sharepoint Designer 2010

2011-06-02T23:34:00.003+01:00

This is pretty simple stuff, but I just spent ten minutes looking at Sharepoint Designer trying to figure out where I clicked to change the default content type on a list! I guess I'm just not used to the ribbon :)

Google didn't help (probably too easy a problem for it) so I thought I would right this up in case anyone else was suffering.

I've created a content type, here it is associated with a list in Sharepoint Designer:

But how do I make it default? I finally spotted the option after clicking through endless options in SP designer - the trick is to highlight the content type (and not click on its name to see the content type editor!) and the look at the ribbon:

Hurray! There it is.
Click on this and we have a new default:

I hope this helps some other Sharepoint 2010 newbie.

VBA code to check in a document to Sharepoint and set meta data

2011-03-10T21:51:00.001Z

The following VBA code will upload a file to Sharepoint 2007 using a web service, and then sest some meta data (the title) for that file in the document library.
Remember, you need to have installed the VBA Microsoft Office 2003 Web Services Toolkit and created a reference to the web service in your project before this will work. See these previous posts for more information on calling Sharepoint web services from VBA:

' Change these values to your own
sSourceFile = "C:\mtest.doc"  ' File to upload
sTargetFile = "HTTP://my sharepoint site/Shared%20Documents/mytest.doc" ' Target site, document library and file name


' First set up DOM document containing fields
Dim xmlDoc As New MSXML2.DOMDocument30
xmlDoc.async = False

xmlText = "<root>" & _
"<Batch OnError='Continue' ListVersion='" & iVersionNumber & "' PreCalc='TRUE' xmlns=''>" & _
"<Method ID='1' Cmd='Update'>" & _
"<Field Name='ID' />" & _
"<Field Name='FileRef'>" & sTargetFile & "</Field>" & _
"<Field Name='Title'>Uploaded from VBA</Field>" & _
"</Method>" & _
"</Batch>" & _
"</root>"


xmlDoc.LoadXml (xmlText)
Debug.Print xmlText

' This bit is just for testing
If xmlDoc.parseError.errorCode <> 0 Then

Set myErr = xmlDoc.parseError
MsgBox (myErr.reason)
GoTo fnUpload_Error
Else
MsgBox xmlDoc.XML
End If

' Set up IXMLDOMNodeList
Dim myXMLNodeList As MSXML2.IXMLDOMNodeList
Dim root As MSXML2.IXMLDOMElement


Set root = xmlDoc.documentElement
Set myXMLNodeList = root.ChildNodes

' Create an array of IXMLDOMNodeList
Dim ar_Fields(1) As IXMLDOMNodeList
Set ar_Fields(0) = myXMLNodeList
'Debug.Print "ar_Fields(0) = " & ar_Fields(0)


' Now set up an array of strings to hold the URL
Dim ar_URL(1) As String
ar_URL(0) = sTargetFile

Debug.Print "ar_URL(0) = " & ar_URL(0)
' Set up the results object
Dim myresults() As struct_CopyResult

' Set up the byte array and read the source file into it
Dim ar_Stream() As Byte
ar_Stream = ReadFile(sSourceFile)


' NOW CALL WEB SERVICE
' The follwoing comes from the "Microsoft Office 2003 Web Services Toolkit":
'"ar_DestinationUrls" is an array with elements defined as String
'"ar_Fields" is an array with elements defined as IXMLDOMNodeList
'"ar_Stream" is an array with elements defined as Byte
'"ar_Results" is an array with elements defined as struct_CopyResult
'See Complex Types: Arrays in Microsoft Office 2003 Web Services Toolkit Help
'for details on implementing arrays.
documentId = copyws.wsm_CopyIntoItems(sDocumentPath, ar_URL, ar_Fields, ar_Stream, myresults)
Debug.Print "DocumentID = " & documentId

Dim updateReturn As IXMLDOMNodeList

Set updateReturn = listws.wsm_UpdateListItems(sListID, myXMLNodeList)

Dim xmlReturnDoc As New MSXML2.DOMDocument30
If (updateReturn.Length > 0) Then    
xmlReturnDoc.LoadXml (updateReturn.Item(0).XML)

Dim errorText As String
errorText = xmlReturnDoc.Text
If (errorText <> "0x00000000") Then
MsgBox ("Error: Cannot upload load file to Sharepoint." & vbCrLf & _
"     : " & errorText & vbCrLf & Err.Description & vbCrLf )

End If
End If

' Uncomment for debug information.
'MsgBox ("Return XML = " & xmlReturnDoc.XML)

I use the following VBA to read the target file in as a byte array:

Private Function ReadFile(ByVal strFileName As String, Optional ByVal lngStartPos As Long = 1, Optional ByVal lngFileSize As Long = -1) As Byte()
    Dim FilNum As Integer
    FilNum = FreeFile
    Open strFileName For Binary As #FilNum
    If lngFileSize = -1 Then
        ReDim ReadFile(LOF(FilNum) - lngStartPos)  
    Else
        ReDim ReadFile(lngFileSize - 1)
    End If
    Get #FilNum, lngStartPos, ReadFile
    Close #FilNum
End Function

Deploy Office 2003 VSTO add in to All Users using Visual Studio 2008

2011-02-27T23:34:00.007Z

Deploying VSTO add-ins can be a real pain, at least if you are not using 2010. Having just successfully deployed my first add-in, I thought it would be useful to explain how I did and the resources I used.

Firstly, it is important that you follow the instructions for the appropriate version of Office and Visual Studio that you are using. My example is for a legacy solution using Office 2003, created with VSTO Second Edition (Visual Studio 2008). The general principles are the same for other editions, but the specifics may be subtly different.

To get started, look at the walkthrough here:

http://msdn.microsoft.com/en-us/library/bb332052.aspx
This is another useful collection of resources:
http://xldennis.wordpress.com/2007/03/04/creating-and-deploying-managed-com-add-ins-with-vsto-2005-se-part-vi/

This explains how to configure a deployment project in Visual Studio 2008 and set up the additional security package required to grant the add-in full trust. I would recommend you follow the steps in here to create a brand new simple package and get that working first. I created a simple Word 2003 add-in package to test, rather than the suggested Outlook.

The security package can be downloaded from MSDN here:
http://code.msdn.microsoft.com/VSTO3MSI
Unpackage the download and you should find a .SLN file that has everything pre-configured. Copy the "SetSecurity" package from here and add it to your solution as explained in the walkthrough. Configure the Setup project's outputs as described, and you should end up with a package that looks something like this:

Once you have built your setup package, you can find the .MSI and .EXE files in solution bin/debug (or bin/release) directory. Run the .exe and you should have installed your add-in onto your development machine. Run the appropriate Office application to check that the pop up message appears to confirm this. Your development computer should have all the pre-requisites installed, so if this doesn't work you have probably missed something in the walkthrough, or have perhaps used an incorrect version of the setup project, so go back and double check everything. It is vital that you get a simple version working at this stage, or it will cause you all sorts of problems down the line.

Once you have a simple install project working you can go about making it available to ALL USERS on the machine. This is not covered in the walkthrough, although there is mention of it for newer versions of Office in this excellent blog here:
http://blogs.msdn.com/b/mshneer/archive/2008/04/24/deploying-your-vsto-add-in-to-all-users-part-iii.aspx

In Office 2003, deploying to all users is actually really simple: all we have to do is copy the registry keys that have automatically been generated for us by Visual Studio in the Setup project from HKCU to HKLM. In practice, this mean highlighting your setup project (called Word2003AddInSetup in my example), right click and choose "View" and "Registry":

This will open up the registry editor. Now, expand all, and renambe the "Software" key in HKLM to some temporary name (say "SoftwareTESTTESTTEST").

You can now drag the "Software" key from the HKCU leaf to the HKLM leaf. Once done, drag your renamed "SoftwareTESTTESTTEST" key from HKLM to HKCU, and rename this back to "Software". You should now have something that looks like this:

One thing that is not really very well documented at all, and kept me scratching my head for many days, is that you must set the "InstallAllUsers" property on the setup package to "True". This is obvious, but not mentioned anywhere!

That's it! Build your setup solution and re-install it (you will have to un-install your original version first). You should now find that any user that logs into the machine will have the add in enabled.

Please note that if you are using Office 2007, this technique does not work - you will have to refer to the blog post linked above to learn how to set up the appropriate registry keys.

Sharepoint 2007 Batch Update example using web services and CAML

2011-01-26T00:03:00.000Z

Here is an example of running a batch update process on Sharepoint 2007 using web services. It was written in C# in Visual Studio 2008.

In this example, I take the value of a text field "Date_x0020_of_x0020_Birth", turn this into a DateTime value, and then use this to update the the value of a DateTime field "Date_x0020_of_x0020_Birth0". Obviously this section of the code will not be particularly generalisable, however I have left in all the code in order to show you the principles involved, and hopefully you will find it useful in building your own batch routines.

In order to get it running, you must create a reference to the Sharepoint Lists web service and call it "ListsService". The SDK shows you how to do this, and is available here: http://www.microsoft.com/downloads/en/details.aspx?FamilyId=6D94E307-67D9-41AC-B2D6-0074D6286FA9&displaylang=en

The GetListItems() method of the Lists web service is used to query the data, and the UpdateListItems() method is used to update items.

I used an XPathNavigator object to filter only rows that contain data. You could just loop through the nodes if you prefer.

CAML is used to query the Sharepoint list and look for items where the "Date_x0020_of_x0020_Birth" is null. Tailor this to your own needs.
For CAML reference, look at the MSDN article here: http://msdn.microsoft.com/en-us/library/lists.lists.getlistitems%28v=office.12%29.aspx

The CAML update command uses owshiddenversion, and so is useful in situations where versioning is enabled. Lists without versioning do not need this field.

If the UpdateListItems command returns an error, in my experience this is usually caused by using an incorrect internal name for a field. You may need to add the URI prefix "urn:schemas-microsoft-com:office:office#" to the field name (e.g. change line 42 to something like: "<Field Name=\"urn:schemas-microsoft-com:office:office#{4}\">{5}</Field>").

Full code:

using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.SharePoint;
using Microsoft.SharePoint.Utilities;
using System.IO;
using System.Xml;
using System.Xml.XPath;
using System.Net;

namespace BatchUpdateDOB
{
    class Program
    {
        static string LogFile = "D:\\BatchUpdateLog.txt";  // Log file location

        static void Main(string[] args)
        {
            try
            {
                StatusMessage("Writing log file to: " + LogFile);
           
                // Set up the variables to be used.
                StringBuilder methodBuilder = new StringBuilder();
                string batch = string.Empty;
                DateTime currentDate = DateTime.Now;
                string formattedDate = SPUtility.CreateISO8601DateTimeFromSystemDateTime(currentDate);
                string fieldToReference = "Date_x0020_of_x0020_Birth";
                string fieldToUpdate = "Date_x0020_of_x0020_Birth0";

                string batchFormat = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>" +
                  "<ows:Batch OnError=\"Return\">{0}</ows:Batch>";

                
                // CAML update command for web service
                string methodFormat = "<Method ID=\"1\" Cmd=\"Update\">" +
                 "<Field Name=\"ID\">{1}</Field>" +
                 "<Field Name=\"FileRef\">{2}</Field>" +
                 "<Field Name=\"owshiddenversion\">{3}</Field>" +
                 "<Field Name=\"{4}\">{5}</Field>" +
                 "</Method>";

                //UPDATE THESE TWO LINES TO SITE SPECIFIC VALUES
                string siteName = "http://<your site root uri here>";
                string listName = "<your list name here>";

                //string viewName = "";  // No need for viewname
                int rowLimit = 10;       //The number of items returned per batch (not used)

                StatusMessage("Processing " + siteName + ", List: " + listName + "...");

                //Web service setup
                //ListService should be the name of your web service reference
                ListService.Lists listService = new ListService.Lists();
                listService.Credentials = System.Net.CredentialCache.DefaultCredentials;
                listService.Url = siteName + "/_vti_bin/Lists.asmx";
                

                //Build CAML query to search for records
                XmlDocument camlDocument = new XmlDocument();
                XmlNode queryNode = camlDocument.CreateElement("Query");
                //queryNode.InnerXml =  "<Where><IsNull><FieldRef Name='Date_x0020_of_x0020_Birth'/></IsNull></Where>";
                //queryNode.InnerXml = "<OrderBy><FieldRef Name='Country' /></OrderBy>";

                XmlNode viewFieldsNode = camlDocument.CreateElement("ViewFields");
                viewFieldsNode.InnerXml = "<FieldRef Name='Title' />" +
                  "<FieldRef Name='owshiddenversion' />" +
                  "<FieldRef Name='" + fieldToReference + "' />" +
                  "<FieldRef Name='" + fieldToUpdate + "' />" +
                  "<FieldRef Name='FileRef' />";

                XmlNode queryOptionsNode = camlDocument.CreateElement("QueryOptions");
                queryOptionsNode.InnerXml = "<ViewAttributes Scope=\"Recursive\" /><IncludeMandatoryColumns>True</IncludeMandatoryColumns>";

                // Get the GUID of the list
                XmlNode ndListView = listService.GetListAndView(listName, "");
                string listGuid = ndListView.ChildNodes[0].Attributes["Name"].Value;
                string listVersion = ndListView.ChildNodes[0].Attributes["Version"].Value;

                // Run the CAML query
                XmlNode resultNode = listService.GetListItems(listGuid, null, queryNode, viewFieldsNode, rowLimit.ToString(), queryOptionsNode, null);
                // If you want to put a rowlimit on, then use this command:
                //XmlNode resultNode = listService.GetListItems(listGUID, null, queryNode, viewFieldsNode, rowLimit.ToString(), queryOptionsNode, null);

                long totalRecords = 0;
               
                //string itemDOB;
                string dob;
                DateTime dtDOB;
                string owsHiddenVersion;  
                string fullFileRef;
                string owsFileRef;
               

                // Create xpath navigator   
                XPathNavigator navigator = resultNode.CreateNavigator();
                // Set up namespace manager for XPath   
                XmlNamespaceManager ns = new XmlNamespaceManager(navigator.NameTable);
                ns.AddNamespace("rs", "urn:schemas-microsoft-com:rowset");
                ns.AddNamespace("z", "#RowSetSchema");

                // Select only the z:row nodes
                XmlNodeList nodes = resultNode.SelectNodes("//*[name()='z:row'] ", ns);

                // Iterate through the returned nodes
                foreach (XmlNode node in nodes)
                {
                    string itemID = node.Attributes["ows_ID"].InnerText;
                    try
                    {
                        dob = node.Attributes["ows_" + fieldToReference].InnerText;
                        owsHiddenVersion = node.Attributes["ows_" + "owshiddenversion"].InnerText;
                        owsFileRef = node.Attributes["ows_" + "FileRef"].InnerText;
                        // For some reason my data is returned in the format: 2#/listname/path, 
                        // where 2 seems to be an id or version number, and listname/path points to the file in relation to the root site.
                        // The following strips that out and gets the full path (required for the CAML update)
                        fullFileRef = siteName + "/" + owsFileRef.Remove(0, owsFileRef.IndexOf("#") + 1);
                    }
                    catch(Exception ex)
                    {
                        // Ignore this row
                        //StatusMessage("ERROR: Cannot process: " + node.InnerXml.ToString() + ": " + ex.Message); // If you want to see why, uncomment
                        continue;
                    }

                    if (String.IsNullOrEmpty(dob)) continue;  // Ignore list items with no value

                    dtDOB = Convert.ToDateTime("01/01/1900"); // Set default value for dates before 1900

                    // Convert date text to DateTime
                    try
                    {
                        dtDOB = Convert.ToDateTime(dob);
                    }
                    catch (Exception ex)
                    {
                        StatusMessage("Cannot process: " + siteName + "/" + fullFileRef + " DOB: " + dob + ".  Setting to 01/01/1900. " + ex.Message);
                    }

                    MessageToFile("Processing: " + fullFileRef + ", DOB = " + dob);

                    methodBuilder.AppendFormat(methodFormat, listGuid, itemID, fullFileRef, owsHiddenVersion, fieldToUpdate, SPUtility.CreateISO8601DateTimeFromSystemDateTime(dtDOB));

                    // Put the pieces together.
                    batch = string.Format(batchFormat, methodBuilder.ToString());
                    //MessageToFile(batch);

                    //Build CAML command to update using Web service
                    XmlDocument xmlDoc = new XmlDocument();
                    XmlElement elBatch = xmlDoc.CreateElement("Batch");
                    elBatch.SetAttribute("OnError", "Continue");
                    elBatch.SetAttribute("ListVersion", listVersion.ToString());
                    elBatch.SetAttribute("ViewName", "");
                    elBatch.InnerXml = methodBuilder.ToString();

                    // This line runs the CAML update command
                    XmlNode updatesResponse = listService.UpdateListItems(listGuid, elBatch);
                    if (updatesResponse.FirstChild.FirstChild.InnerText != "0x00000000") throw new Exception("Could not update properties.\n\n" + updatesResponse.InnerText + "\n" + updatesResponse.FirstChild.FirstChild.InnerText);
                    StatusMessage(fullFileRef + " processed...");

                    totalRecords++;
                }
                StatusMessage("Completed - " + totalRecords + " processed.");
            }
            catch (Exception exp)
            {
                StatusMessage("ERROR: " + exp.Message);
                
            }
            
        }

        
        /// <summary>
        /// Display a  message
        /// </summary>
        /// <param name="myMessage"></param>
        static void StatusMessage(string myMessage)
        {
            Console.WriteLine(myMessage.ToString()); 
            MessageToFile(myMessage);
        }

        /// <summary>
        /// Write message to the log file
        /// </summary>
        /// <param name="myMessage"></param>
        static void MessageToFile(string myMessage)
        {
            using (StreamWriter sw = new StreamWriter(LogFile, true))
            {
                DateTime now = DateTime.Now;
                sw.WriteLine(now.ToString() + ": " + myMessage.ToString());
            }
        }
    }
}

Binding a CheckBoxList object to an ordered Sharepoint list of reference values in a web part (c#)

2011-01-11T22:42:00.003Z

It's common when creating database applications to use reference tables to provide lookup values. These tables ofter contain a flag that is used to mark a record as no longer available, and an order column that provides a method of putting the values in an order of importantance, perhaps when using them in a drop down list or list of checkboxes. Sharepoint is NOT a database, however there is no reason why good practice such as this cannot be applied to Sharepoint's list model.

In the following example I will show you how to set up a reference list and then use the Sharepoint object model to programmatically access this list and bind it to an control (in this case a Sharepoint CheckBoxList control). Once you understand this, you can use the same method to connect to any reference list and bind it to other objects and controls, such as drop down lists. This example has been developed for Sharepoint 2007, but should also work for Sharepoint 2010.

Step 1: Set up a reference list
Create a new list on your Sharepoint site with teh following structure:

ID - integer (this is a default field) - this will be used as the unique key (lookup value) used to link this reference filed to other lists
Title - string (this is a default field) - this is used to hold the description of the reference field, and will be the value shown in lookups
SortOrder - integer, default 1000 - this can be used to sort the list values in lookups and programmatically using CAML
Enabled - Yes/No field, default Yes - setting to NO will enable you to filter out these values in views and in programmatic lookups in CAML

You can now enter you reference values into the list, for example here is a list of types of medical record:

Step 2: Add method to webpart that use CAML to read and order the list
Now for the code that reads the Sharepoint list in a structured manner so we can use it in web parts. First, add the following method to your web part:

/// <summary>
/// Returns a list collection, ordered using a field in the list called "SortOrder"
/// </summary>
/// <param name="listName">The literal string name of the list to query</param>
/// <returns>SPListItemCollection</returns>
private SPListItemCollection ReadOrderedList(string listName)
{
    try
    {
        using (SPWeb configWeb = SPContext.Current.Site.OpenWeb())
        {
            //Get the list
            SPList lookupList = configWeb.Lists[listName];
            // Use CAML to order the list
            SPQuery query = new SPQuery();
            // This CAML where clause searches for only items where the Enabled field is set to "Yes"
            query.Query = "<Where><Eq><FieldRef Name=\"Enabled\" /><Value Type=\"Integer\">1</Value></Eq></Where> ";
            // This CAML order by claus uses an integer field of "SortOrder"
            query.Query += "<OrderBy><FieldRef Name=\"SortOrder\" /></OrderBy>";
            SPListItemCollection lookupItems = lookupList.GetItems(query);
            return lookupItems;
        }
    }
    catch (Exception ex)
    {
        ErrorTrap("ReadOrderedList: " + ex.Message + Environment.NewLine + ex.StackTrace.ToString());
        throw;
    }
}

This method will return the content of the list in a SPListItemCollection object. CAML is use to order the list by the "SortOrder" column, and to ignore values that do not have the boolean field "Enabled" set to true.

Step 3: Bind the list to a control
At the class level of your webpart, create the control that you wish to bind to (in this case a CheckBoxList):

CheckBoxList myCheckList = new CheckBoxList();

To bind this control to your Sharpoint list, put this code in the CreateChildControls() method. Simply replace "My List" with the real name of your list:

// Set up the Record Section checkboxes list
    SPListItemCollection dt = ReadOrderedList("My List");
    myCheckList.DataSource = dt;
    myCheckList.DataTextField = "Title";
    myCheckList.DataValueField = "ID";
    myCheckList.DataBind();

That's it! You now have a Sharepoint object bound to your Sharepoint list of reference values. When rendered, the output will look something like this:

Yyou can use the following method to render the CheckBoxList to the web part. This example puts the CheckBoxList in a DIV that fixes the maximum size of the list with a scrollbar.

/// <summary>
/// render a list of checkboxes  to the page
/// </summary>
void RenderCheckBoxes()
{
    try
    {
        // Set up list of checkboxes
        myCheckList.EnableViewState = true;
        myCheckList.CellSpacing = 0;
        myCheckList.CellPadding = 0;

        for (int i = 0; i < myCheckList.Items.Count; i++)
        {
            ListItem li = new ListItem(myCheckList.Items[i].Text, myCheckList.Items[i].Value);
           
            // Set all boxes to selected if not a postback
            if (!Page.IsPostBack)
            {
                myCheckList.Items[i].Selected = true;
            }

        }
        // Give the checklist an ID (can use this in javascript etc)
        myCheckList.ID = "RecordSections";

        //Add controls to web part
        Controls.Add(new LiteralControl(@"<TABLE border='0' bordercolor='red' cellpadding='0' cellspacing='0' valign='top'>"));
        Controls.Add(new LiteralControl("<TR>"));
        Controls.Add(new LiteralControl("<TD>"));
        Controls.Add(new LiteralControl(@"<div class=""ms-frombody""><b>Record Sections</b></div>"));
        Controls.Add(new LiteralControl("</TD>"));
        Controls.Add(new LiteralControl("</TR>"));
        Controls.Add(new LiteralControl("<TR>"));

        Controls.Add(new LiteralControl("<TD>\r\n"));
        Controls.Add(new LiteralControl(@"<input type=""hidden"" id=""ScrollPos"" runat=""server""/>"));

        Controls.Add(new LiteralControl(@"<div id='RecordSectionScroll' style=""OVERFLOW: auto; WIDTH: 247px;  TOP: 0px; HEIGHT: 300px"" onscroll=""javascript:document.getElementById('ScrollPos').value = this.scrollTop""> "));

        Controls.Add(myCheckList);
        Controls.Add(new LiteralControl("</div></TD>"));
        Controls.Add(new LiteralControl("</TR>"));     
        Controls.Add(new LiteralControl("</TABLE>"));
    }
    catch (Exception ex)
    {
        ErrorTrap("RenderCheckBoxes" + ex.Message + Environment.NewLine + ex.StackTrace.ToString());
    }
}

Note that I am using literal controls to render the check boxes in a HTML table. This is pretty horrible and I don't recommend it, but it does work.

Here is the ErrorTrap method. Note that this is just a simple error handling function and is not required to render a checklist bound to a Sharepoint list, but since it appears in the examples I though I would show you what I do.

/// <summary>
/// Give an error message
/// </summary>
/// <param name="myMessage"></param>
void ErrorTrap(string myMessage)
{
    Controls.Add(new LiteralControl("<strong>ERROR: " + myMessage.ToString() + "<BR/></strong>"));
    ErrorToFile(myMessage);          
}

/// <summary>
/// Write errors to the log file
/// </summary>
/// <param name="myMessage"></param>
void ErrorToFile(string myMessage)
{
    using (StreamWriter sw = new StreamWriter("D:\\SearchErrorLog.txt"))
    {
        DateTime now = new DateTime();
        sw.Write(now.ToString() + ": " + myMessage.ToString());
    }
}

Render Content Type fields as a form in Sharepoint Web Part using c#

2010-11-16T13:56:00.004Z

In this blog entry I will give you some example code that will render a simple form in a Sharepoint 2007 web part. The form fields are taken from a pre-defined Sharepoint Content Type. I struggled to find any examples of this on the net, despite it seeming to me to be a fairly ordinary thing to want to do (after all, if you go to all the trouble of defined your own Content Types to organise your data, surely you would want to use them in your own web parts?).

In order to develop this, I used C# / Visual Studio 2008 and Sharpoint 2007. This should also work in Visual Studio 2005, although you will have to install the web part manually - Visual Studio 2008 has far tighter links to Sharepoint, meaning debugging and deploying is much easier. It should also work in VS2010 / Sharepoint 2010.

The following method will return a SPContentType object, you only need to pass it a list name and content type name. The SPContentType object can be used later to iterate through its fields in oder to render controls based on the content type's fields.

/// <summary>
/// Returns the content type object
/// </summary>
/// <param name="listName"></param>
/// <param name="contentTypeName"></param>
/// <returns></returns>
SPContentType ReadContentType(string listName, string contentTypeName)
{

    //get the web
    this._web = Microsoft.SharePoint.SPContext.Current.Web;

    SPList dirList = this._web.Lists[listName];
    SPContentType contentType = dirList.ContentTypes[contentTypeName];

    //You could simply iterate here like this if you wish, like this:
    //foreach (SPField field in _contentType.Fields)
    //{
    //    string dispName = field.Title;
    //    SPFieldType type = field.Type;
    //    object defaultValue = field.DefaultValue;

    //}

    return contentType;
}

The following code overrides the "CreateChildControls" Sharepoint method to create our own controls based on the content type. In this example, only text fields and date-times are rendered, but hopefully it straight forward enough for you can see how you could extend this. Simply change:

"Your_List_Name" and "Your_Content_Type_Name"

to your actual values from your Sharepoint site.
I use the "LiteralControl" method to render raw HTML on the page to format the form, which feels really nasty but is efective.

protected override void CreateChildControls()
{
    //Controls.Clear();
    //base.CreateChildControls();

    // Read in the content type for the target list
    SPContentType contentType = ReadContentType("Your_List_Name", "Your_Content_Type_Name");
    
    Controls.Add(new LiteralControl("<TABLE>"));

    foreach (SPField field in contentType.Fields)
    {
        // Ignore hidden or readonly fields
        if (field.Hidden) continue;
        if (field.ReadOnlyField) continue;     

        if (field.Type == SPFieldType.Text)
        {
            string dispName = field.Title;
            string staticName = field.StaticName;
            SPFieldType type = field.Type;
            object defaultValue = field.DefaultValue;

            Controls.Add(new LiteralControl("<TR>"));
            Controls.Add(new LiteralControl("<TD>"));
            Label lbl = new Label();
            lbl.Text = dispName;

            Controls.Add(lbl);

            Controls.Add(new LiteralControl("</TD>"));

            // Textbox
            TextBox tb = new TextBox();
            // I give my controls a unique ID so I can reference them later
            tb.ID = "xx_ctl_xx" + dispName;
            
            if(defaultValue != null)
                tb.Text = defaultValue.ToString();

            Controls.Add(new LiteralControl("<TD>"));
            Controls.Add(tb);
            Controls.Add(new LiteralControl("</TD>"));
            Controls.Add(new LiteralControl("</TR>"));

        }
        else if (field.Type == SPFieldType.DateTime)
        {
            
            string dispName = field.Title;
            string staticName = field.StaticName;
            SPFieldType type = field.Type;
            object defaultValue = field.DefaultValue;

            Label lbl = new Label();
            lbl.Text = dispName;

            Controls.Add(new LiteralControl("<TR>"));
            Controls.Add(new LiteralControl("<TD>"));
            Controls.Add(lbl);
            Controls.Add(new LiteralControl("</TD>"));

            // Datetime picker
            SPDatePickerControl dtp = new SPDatePickerControl();
            dtp.ID = "xx_ctl_xx" + staticName;
           
            if (defaultValue != null)
                dtp.SelectedDate = defaultValue.ToString();

            Controls.Add(new LiteralControl("<TD>"));
            Controls.Add(dtp);
            
            Controls.Add(new LiteralControl("</TD>"));
            Controls.Add(new LiteralControl("</TR>"));
        }       
    }
    Controls.Add(new LiteralControl("</TABLE>"));

    //In my code, I use the controls to perform a custom search.  The following code plugs in
    //the event handler to do this, based on the values the user puts into the controls rendered above.
    //This blog post will not go into the detail of this so I have commented the following lines out.
    //cmdSearch = new Button();
    //cmdSearch.Text = "Start Search";
    //cmdSearch.Click += new EventHandler(cmdSearch_Click);
    //this.Controls.Add(cmdSearch);

    //lblQueryResult = new Label();                     
    //this.Controls.Add(lblQueryResult);
    
    base.CreateChildControls();
}

Once deployed, you can add this web part to your site in the normal way using the web front end. Depending on the fields in your content type, it should render something like this (web part highlighted in pink):

Colour code a RAG column in Sharepoint 2007 list view using Jquery

2010-07-08T00:29:00.004+01:00

I'm a big fan of Microsoft Sharepoint, even though it really annoys me sometimes. It's amazing that no matter what question a client asks of it, it never can quite answer the question straight out of the box. However, this is not a problem, because Sharepoint is virtually infinately customisable by a competent programmer with enough time to become emersed in the "Sharepoint Way" and no fear of learning new techniques.
An example of this came up for me recently when a client requested that a column on a list would be colour coded depending on the value of the data in that column. The data would be a RAG rating (i.e. a "choice" field containing a set of values that relate to red, amber or green), and therefore it would be nice to colour code the column in eiter red, amber or green to match the rating. Sharepoint does not give you any easy way of doing this unfortunately.

This is where Jquery comes in. This terrific library of javascript offers developers a new and consise way of dealing with the internet document object model (DOM). With it developers can quickly and easily scan the object model of the page returned in the browser, and change the page before it is rendered. There is far more to it than this of course, but this is the trick I used to colour code the columns.

To colour (color) code a coloumn, follow these steps:

Download the jquery.js library from teh link above, and upload it to your Sharepoint site in a convenient location (I use the Site Documents Collection in my top level publishing site, but anywhere will do)
Edit the Sharepoint page that the holds the list, and add a Content Editor Web Part (CEWP).
Edit the CEWP, choose the "edit source" option, and paste in something like the following code

<script type="text/javascript" src="http://<root site>/SiteCollectionDocuments/jquery-1.4.2.js"></script>

<script type="text/javascript">

$(document).ready(function(){
$(".ms-vb2:contains('1: No Assurance')").each(function(){
$(this).css("background-color", "#FF0000");
//$(this).css("filter", "alpha(opacity=50)"); 
});

$(".ms-vb2:contains('2: Limited Assurance')").each(function(){
$(this).css("background-color", "#FF9900");
});

$(".ms-vb2:contains('3: Significant Assurance')").each(function(){
$(this).css("background-color", "#FFFF00");
});

$(".ms-vb2:contains('4: Full Assurance')").each(function(){
$(this).css("background-color", "#00CC00");
});

});
<script>

Note that in this example, the Jquery library is located at: http://<root site>/SiteCollectionDocuments/jquery-1.4.2.js. Change this to the URL of your own Jquery library.

In my example, the RAG rating column can contain the following values:

1: No Assurance (RGB #FF0000)
2: Limited Assurance (RBG #FF9900)
3: Significant Assurance (RGB #FFFF00)
4: Full Assurance (RGB #00CC00)

Now to explain what this is actually doing: "ms-vb2" is the standard Sharepoint site template CSS class that is used to identify columns in a list view web part.
What this Jquery script is effectively doing is looking for a DOM item with the class "ms-vb2", and in each case that it this
DOM item contains a specific piece of text ("1: No Assurance", "2: Limited Assurance" etc.), it modifies the CSS of this
particular section of the DOM to add a "background-color" CSS tag.

Of course your list will have different values, so remember to change the strings in bold to match your own list values!
The colour codes are standard RGB, so you may find this RGB table useful to decide your colours.

Your results will look something like this:

Best of luck!

Write lines to a RichTextBox control in different colours (VB .net example)

2010-05-11T23:27:00.007+01:00

I create a lot of back end "processing" software, for example migration or upgrade tools that process documents as they find them on a server. As a standard, I tend to write their progress out to a text file, but I also like the program to show me what is happening as it processes files by writing the log file to the screen. This is particularly useful when debugging the application, but it's also far more satisfying to see something happening on the screen.

In this situation it is nice to write out warnings or non critical errors in different colours in order to attract the eye.

To achieve this, create a RichTextBox control on your main form (in my example it is called "fInfoBox"), and write to it using the code in the sub routine below.

Sub ShowLog(ByVal qFore as Drawing.Color, ByVal sMesg as string)
   Dim sDispMess As String
   sDispMess = Date.Now()& ": " & sMesg & vbCrLf 

   With fInfoBox 'This is the name of you RichTextBox control on your form
       .SelectionStart = Len(.Text)
       .SelectionBullet = True  ' This shows the line in bulleted form, comment out if not required

       .SelectionColor = qFore
       .SelectedText = sDispMess.ToString()
    End With

    fInfoBox.ScrollToCaret()  ' This scrols to the last line of the text box
End Sub

To display a message with a font colour of blue, call the routine like this:

ShowLog(Drawing.Color.Blue, "This message will appear in a font color of blue")

For a red message, try this:

ShowLog(Drawing.Color.Red, "ERROR: This message will display in a font color of red!!")

This is a very simple example, but I hope it helps someone get started with this sort of thing and gives you some ideas.

Remember that if you use threading or the backgroundworker class to remove your time consuming processes from the UI (which you should, especially since this is so easy after .Net 2.0), then you will need to use a callback technique for your worker thread to be able to talk to your UI thread.

VBA code to iterate through the results of GetListCollection web service from Sharepoint 2007

2010-04-21T22:06:00.016+01:00

In a previous post I discussed how to set meta-data in Sharepoint 2007 directly from VBA. I mentioned that the Microsoft documentation for the Sharepoint web services does not appear to be correct, insofar as you cannot address a Sharepoint "list" in the web service using its name, but rather you are forced to use the GUID.

I have now had to write a function to retrieve the GUID for the list from teh Sharepoint site. This proved to be much harder in practice than I had expected it to be, partly at least because of the problems in working in the VBA IDE with its poor debugging functionality, partly because VBA is a much more limited language than .Net and I have got used to using all the nice constructs such as generics that make life so much easier, and partly because I still find MSXML very "brain" hard to work with. I don't know why I find it so hard, it just always seems that everything ends up being far more complicated than I think it should be. I find the .Net XML model much easier to deal with.

Anyway, after playing about with XSLT and XPATH in order to extract the ID and VersionNumber attributes from the wsm_GetListCollection web service (and failing miserable), I finally decided to just iterate through the nodes and attributes till I found what I wanted. The code below is a simple function that you can call with three parameters: the name of the list that you need the GUID for (sListName), a string variable in which the GUID will be placed (sListID), and an integer that the list version number will be placed (iListVersion).

Note: You may also be interested in this post: VBA code to check in a document to Sharepoint and set meta data.

Function GetListCollection(ByVal sListName As String, ByRef sListID As String, ByRef iListVersion As Integer)
    ' Class created by toolkit to connect to the Web service
    Dim ws As New clsws_Lists
    ' The collection is returned as an XML node list
    Dim ListCollectionNodeList As MSXML2.IXMLDOMNodeList
    'Root node of the returned list
    Dim nod As MSXML2.IXMLDOMNode
    ' Output string for the XML transformation
    Dim strOutput As String
 
    On Error GoTo GetListCollection_OnError
 
    ' Retrieve the collection of lists
    Set ListCollectionNodeList = ws.wsm_GetListCollection
 
    ' Get the root node from the list
    Set nod = ListCollectionNodeList.Item(0)
 
 
    ' Iterate nodelist to find details for our named list
    Dim bGotNode
    bGotNode = False
 
 
    Dim procNode As MSXML2.IXMLDOMNode
    Dim TitleNodeList As MSXML2.IXMLDOMNodeList
    Dim IDNodeList As MSXML2.IXMLDOMNodeList
    Set TitleNodeList = nod.SelectSingleNode("//Title")
    Set IDNodeList = nod.SelectSingleNode("//ID")
 
    Dim ListNode As MSXML2.IXMLDOMNode
    Dim ListChildNodes As MSXML2.IXMLDOMNode
    Dim oAttr As MSXML2.IXMLDOMAttribute
 
    Dim retID As String
    Dim listVersion As Integer
    Dim ListName As String
 
    ' Iterate through list nodes returned (should only be one)
    For Each ListNode In ListCollectionNodeList
        'Iterate through the child nodes (each one represents a single Sharepoint list)
        For Each ListChildNodes In ListNode.ChildNodes
            'See if we have any attributes (we should have!)
            If ListChildNodes.Attributes.Length > 0 Then
                ' For each list, reset our variables
                retID = ""
                listVersion = 0
 
                ' Iterate through the attributes
                For Each oAttr In ListChildNodes.Attributes
                    Debug.Print "Name= " & oAttr.Name
                    Debug.Print "Value= " & oAttr.nodeTypedValue
 
                    'Look for the ID attribute
                    If oAttr.Name = "ID" Then
                        ' FOund it, so remember it in case this is our list
                        retID = oAttr.nodeTypedValue
                    End If
 
                    ' Look for the "Title" attribute
                    If oAttr.Name = "Title" Then
                        If oAttr.nodeTypedValue = sListName Then
                            ' Found our list, so remember it
                            ListName = oAttr.nodeTypedValue
                            bGotNode = True
                        End If
                    End If
 
                    ' Look for the "Version" attribute
                    If oAttr.Name = "Version" Then
                            ' Found our list, so remember it
                            listVersion = oAttr.nodeTypedValue
                    End If
 
                    ' Once we have found a title, ID and version, break
                    If retID <> "" And listVersion <> 0 And bGotNode = True Then
                       Exit For
                    End If
 
                Next oAttr
            End If
            ' Once we have found the data, break out of iteration
            If bGotNode = True Then
                Exit For
            End If
        Next
    Next
 
    If bGotNode = False Then
        Err.Raise 65000, "", "ERROR: Cannot find list named <" & sListName & "> in Sharepoint site.", "", ""
    End If
 
    sListName = ListName
    sListID = retID
    iListVersion = listVersion
 
    Exit Function
 
GetListCollection_OnError:
    MsgBox ("ERROR: Retrieving list collection from Sharepoint." + vbCrLf + _
           Err.Number + " - " + Err.Description)
End Function

Remember, you will have to have installed the Microsoft Web Services Toolkit before this will work, and add a web reference to your Sharepoint site to your VBA project. This has been tested with Office 2003 linking to Sharepoint 2007 and works :).

Set Sharepoint meta-data from VBA using updatelist web service

2010-03-10T23:24:00.009Z

I've been working on a problem trying to integrate a legacy Microsoft Office 2003 VBA application with Sharepoint 2007. In theory, I should be able to use the SOAP toolkit provided my Microsoft to talk to the Sharepoint (WSS) web services, but in practice there was very little documentation of how to do this, and I could find no resources on the internet aside from the MSDN definitions of the web services, and only a little attached to the SOAP toolkit (available here: Microsoft Office 2003 Web Services Toolkit ).

The basic problem I was trying to solve was: how do I set meta-data held on a document in Sharepoint from Office 2003 using VBA?

After much head scratching I finally solved this by making some assumptions based on the MSDN reference documentation, then using the Fiddler tool until I had worked out what I was doing wrong. Fiddler allows you to inspect the HTTP calls and responses that you are issuing/receiving. This is very helpful when debugging web services, and well worth looking into if you are doing anything in this area.

Anyway, through Fiddler I could see the response from the web service, including the error message. This meant that my initial best guess on how to structure the XML was at least nearly correct, because the web service was giving me a proper error message. Eventually I had a properly structured call and everything worked!

Remember to install the SOAP toolkit before you try this. You will then be able to add this to your VBA project and add a web reference to your project to the WSS Lists.asmx service as you would if your using .Net. This is all explained in the MSDN link above but drop a comment if you would like me to go into more detail.

The actual VBA code used structure the call to the web service looks like this:

'Define a new list service web service class
Dim listws As New clsws_Lists

' Set up the batch command used to set the meta data. 
' Note that FileRef is set to URL of the file you want to change
'ListVersion is from my test site, you may not need that
' First set up DOM document containing fields
Dim xmlDoc As New MSXML2.DOMDocument30
xmlDoc.async = False
Dim xmlText As String
xmlText = "<root>" + _
  "<Batch OnError='Continue' ListVersion='59' PreCalc='TRUE' xmlns=''>" + _
      "<Method ID='1' Cmd='Update'>" + _
        "<Field Name='ID' />" + _
        "<Field Name='FileRef'>http://myservername:1080/sites/mysite/shared documents/TestWebService.doc</Field>" + _
        "<Field Name='Title'>Uploaded from VBA</Field>" + _
        "<Field Name='Surname'>" + sLastName + "</Field>" + _
        "<Field Name='Forename'>" + sFirstName + "</Field>" + _
        "<Field Name='Date_x0020_of_x0020_Birth'>" + sDOB + "</Field>" + _
      "</Method>" + _
    "</Batch>" + _
  "</root>"
xmlDoc.LoadXml (xmlText)

' This is a bit of debug that checks the XML is well formed and show you it if it is. 
If xmlDoc.parseError.ErrorCode <> 0 Then
    Dim myErr
    Set myErr = xmlDoc.parseError
    MsgBox (myErr.reason)
Else
    MsgBox xmlDoc.XML
End If

' Set up IXMLDOMNodeList object
Dim myXMLNodeList As MSXML2.IXMLDOMNodeList
Dim root As MSXML2.IXMLDOMElement
' Now the the XML node list to the batch command we have just constructed above
Set root = xmlDoc.documentElement
Set myXMLNodeList = root.ChildNodes

' Now run the web service to update the meta-data.
' Note that the first parameter is the GUID of the list that holds the data you want to change.  I can't get this to take a list name - it only seems to wrok with a GUID.
Dim updateReturn As IXMLDOMNodeList
Set updateReturn = listws.wsm_UpdateListItems("{B768B024-8B98-4918-990A-ECE34691DBBC}", myXMLNodeList)

If you need to find out the GUID or ListVersion values for your list, use the "GetListCollection" web service and iterate through the XML it returns:

Dim listCollection As IXMLDOMNode
Set listCollection = listws.wsm_GetListCollection
' You can iterate this to find the GUID of our list

I can post further examples if anybody wants any.

*Edit* - I have written another blog post on how to find the GUID of lists in VBA here: http://the-simple-programmer.blogspot.com/2010/04/vba-code-to-iterate-through-results-of.html
This simple function will return the version number and the GUID of the Sharepoint list from its name.

You may also be interested in teh following post: : VBA code to check in a document to Sharepoint and set meta data.

SQL Server - MySQL 4 integration - openquery does not work for tables with datetime values of 0000-00-00

2010-02-21T16:32:00.005Z

I have been trying copy the contents of an old MySQL 4.0.12 database to SQL Server 2008 in order to archive its contents.

To archive this, I followed these steps:

Create an database within SQL Server 2008 to hold the data called ARCHIVE_DB.
Download and install ODBC driver for MySQL (available here). Note that due to the version of the MySQL database I was unable to use the newer driver (version 5.1.xx), since this requires version 4.1 of MySQL.
Set up a linked server in SQL Server 2008 using this driver to point to the data source. I called this data source MYSQL_ls.
I can now select data from this data source using the "openquery" TSQL syntax, eg:
```
SELECT *
FROM openquery([NADT-SQLHOST], 'select * from webdb.episode')
```
I can extend this query using the TSQL "INTO" clause, which will build the SQL table from the source data, eg:
```
SELECT * INTO ARCHIVE_DB
FROM openquery([MYSQL_ls], 'select * from mydatabase.mytable')
```
In theory the above will copy all the data from the MySQL database table called (in the MySQL database called ) into the SQL Server table (SQL Server database ). I do not have to create the table schema, this will be created automatically by the INTO clause.
I can now create an SSIS package that loops through the tables in the MySQL database and runs the SQL statement show in number 5 against each table. This should in theory copy each table across.

This technique works very well until you come to tables that include datetime values.

MySQL allows datetime values in the format of "0000-00-00", which is the equivalent of a NULL date in SQL server. Unfortunately the "Select * Into From Openquery..." trick above does not deal with this situation, and you will recieve a version of the following error:

Msg 7342, Level 16, State 1, Line 4
An unexpected NULL value was returned for column "[MSDASQL].enddate" from OLE DB provider "MSDASQL" for linked server "MYSQL_ls". This column cannot be NULL.

This error is caused by using "openquery". Fortunately there is a very easy work around for this: where tables have datetimes in "0000-00-00" format, you must use an SSIS data flow task to copy the information, rather than the "openquery" described above:

Set the data pump source to the MySQL table pointed to in the Linked server and click on "preview" and you will notice that it can return data. This is not possible by running a query in the SQL Management studio, so clearly we are getting somewhere.
Now set the data pump target to "" and click on "Mappings" and you will notice that each column in the source table will be mapped to an identically named table in the destination table.
Save the data pump and run.

If you run this data flow, the data will be copied across without reporing the datetime mismatch error.

Use a variable as a table name in an SSIS Execute SQL Task

2010-02-17T21:40:00.001Z

I spent quite a while today trying to build an SSIS package that iterated through a dataset that contained a set of table names usign a for-each container, then run a piece of SQL against each table.

Setting up the for-each loop was relatively simple, but I couldn't get the Execute SQL task to run a simple query of the type:

Select count(*) from @table

Where @table is the name of the parameter generated from the for each loop.

It appears that this is not possible in SSIS. Eventually I found a solution: use another variable to store the SQL statement and set up the SQL task to use this variable as its input source.

Assuming that the input parameter is already set up and called [User::TableName], then the steps to achive this are:

Set up a new variable (say called "SQLStatement"), make this a string type.
In the properties box of the paremeter, Set EvaluateAsExpression to True
In the properties box of the paremeter, click on the elipses (...) of the "Expression" property to build an expression. Type in the SQL you need and drag the variable you want to use from the "variables" tree.
Escape you SQL in this expression wwithin quotation marks. e.g. the count statement above would look like:

"Select count(*) from " + @[User:TableName]

You can use the "Evaluate expression" button to make sure that this evaluates as real SQL. If you are using a "for-each" container then you may want to give your [User::TableName] variable a sensible default value in order to see the results of the evaluated expression in a format you could use for testing.
Drag an "Execute SQL Task" object into your control flow and open its properties page
Set the "SQL Source Type" property to "Variable" to tell it that the SQL will come from a variable, rather from a typed in SQL statement.
Set the "SourceVariable" property to the name of the variable you created in step 1 (in this example it would be "SQLStatement".

That's it!

Blogger.com code syntax highlighting, or always remember to save your code before you start playing around with it!

2010-02-08T21:41:00.006Z

I've decided to re-vamp the look of this blog to give it more width and make the code fragments easier to read. I hope you like the new look!

When doing this I made one of the oldest mistakes in the programmer's book: I didn't save my work!

This meant I had to add all the site specific code to render the code in syntax-highlighted format, and add Google analytics tracking code back into the HTML, which whilst not the worst job in the world, did take me some time to remember what I had done. REMEMBER TO SAVE YOUR WORK!

This does give me the opportunity to thank the author of the fabulous syntax highlighter for Blogger.com that I have used, which can be found here: http://alexgorbatchev.com/wiki/SyntaxHighlighter. Thank you very much Alex, it's a superb bit of code!

The code I used to enable the nifty highlighter for C# should be pasted into the HEAD section of your blog:

<link href='http://alexgorbatchev.com/pub/sh/current/styles/shCore.css' rel='stylesheet' type='text/css'/>
<link href='http://alexgorbatchev.com/pub/sh/current/styles/shThemeDefault.css' rel='stylesheet' type='text/css'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shCore.js' type='text/javascript'></script>
<!-- add brushes here -->
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushJScript.js' type='text/javascript'></script>

<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushBash.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushSql.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushXml.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushCpp.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushCSharp.js' type='text/javascript'></script>
<script type='text/javascript'>
  SyntaxHighlighter.config.bloggerMode = true;
  SyntaxHighlighter.all();
</script>

To use it, make sure that the code you want to display has been HTML escaped (which can be done using the utility available here: http://www.string-functions.com/htmlencode.aspx.

Now, to use this, just paste your escaped code into your blog and wrap it in the following HTML tags:

<pre class="brush:csharp">
   ...your code here...
</pre>

Note that this will work for C Sharp code. For other types of code highlighting, simply change the brush type in the <pre> tag, and make sure the appropriate brush is included in the list of brish types defined in the code above in the "!-- add brushes here --" section.

Manage c# threads easily using an array of BackgroundWorker class objects

2010-01-22T00:29:00.013Z

I recently built a piece of software that processed files on a server. There are hundreds of thousands of files, so this took a long time to run through, so I decided to use threads in order to process multiple files at once.

I wanted to be able to manage these threads, and be able to control how many threads I used by a programmatic variable so I could use more or less threads depending upon the power of the machine the process would be run on, and I could try different numbers of threads and benchmark the results to find the optimum number of threads.

The solution I came up with uses an array of .Net 2.0 BackgroundWorker class objects, initialised at run time to the size of the integer variable "maxThreads".

The following code sets up the variables and initialises the array:

static int maxThreads = 20;  //Make bigger or smaller, it's up to you!
private BackgroundWorker[] threadArray = new BackgroundWorker[maxThreads];
static int _numberBackGroundThreads --;  //Just for fun

// Set up the BackgroundWorker object by 
// attaching event handlers. 
private void InitializeBackgoundWorkers()
{
    for (int f = 0; f < maxThreads; f++)
    {
        threadArray[f] = new BackgroundWorker();
        threadArray[f].DoWork +=
            new DoWorkEventHandler(backgroundWorkerFiles_DoWork);
        threadArray[f].RunWorkerCompleted +=
            new RunWorkerCompletedEventHandler(backgroundWorkerFiles_RunWorkerCompleted);
        threadArray[f].ProgressChanged +=
            new ProgressChangedEventHandler(backgroundWorkerFiles_ProgressChanged);
        threadArray[f].WorkerReportsProgress = true;
        threadArray[f].WorkerSupportsCancellation = true;

    }
}

Each BackgroundWorker class has three event handlers assigned to it:

backgroundWorkerFiles_DoWork - This delegate method is used to run the process
backgroundWorkerFiles_RunWorkerCompleted - this delegate method is called once the "DoWork" method has completed
backgroundWorkerFiles_ProgressChanged - this delegate method is used to pass information back to the calling thread, for example to report progress to the GUI thread.

This delegate methods are discussed at length on the MSDN site so I wont go into them in detail here other than to show you this simple code outline:

private void backgroundWorkerFiles_DoWork(object sender, DoWorkEventArgs e)
{
    //Just for fun - increment the count of the number of threads we are currently using.  Can show this number in the GUI.
    _numberBackGroundThreads --;
    
    // Get argument from DoWorkEventArgs argument.  Can use any type here with cast
    int myProcessArguments = (int)e.Argument;

    // "ProcessFile" is the name of my method that does the main work.  Replace with your own method!  
    // Can return reulsts from this method, i.e. a status (OK, FAIL etc)
    e.Result = ProcessFile(myProcessArgument);
}

private void backgroundWorkerFiles_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
    // Use this method to report progress to GUI
}

private void backgroundWorkerFiles_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
    // First, handle the case where an exception was thrown.
    if (e.Error != null)
    {
        MessageBox.Show(e.Error.Message);
    }

    // For fun - print out the result of the ProcessFile() method.
    debug.print = e.Result.ToString();

    // Just for fun - decrement the count of threads
    _numberBackGroundThreads --;
}

OK now comes the fun part. The following code shows a loop thorugh a large number of items. Rather than run ProcessFile() for each item in the loop in turn, we instead choose an unused thread to run it in. This allows the loop to step onto the next item, which also is allocated an empty thread.

// Some process with many iterations
for(int f = 0; f < 100000; f++)
{

    //Use the thread array to process ech iteration
    //choose the first unused thread.
    bool fileProcessed = false;
    while (!fileProcessed)
    {
        for (int threadNum = 0; threadNum < maxThreads; threadNum++)
        {
            if (!threadArray[threadNum].IsBusy)
            {   // This thread is available
                Debug.Print("Starting thread: " + threadNum);
        
                //Call the "RunWorkerAsync()" method of the thread.  
                //This will call the delegate method "backgroundWorkerFiles_DoWork()" method defined above.  
                //The parameter passed (the loop counter "f") will be available through the delegate's argument "e" through the ".Argument" property.
                threadArray[threadNum].RunWorkerAsync(f);
                fileProcessed = true;
                break;
            }
        }
        //If all threads are being used, sleep awhile before checking again
        if (!fileProcessed)
        {
            Thread.Sleep(50);
        }
    }
}

Using this technique, it's eay to create 2, 10, or even 100 threads to process each loop iteration asychronously, which speeds up execution time enormously. Simply change the value of "maxThreads" to whatever you need!

What is the difference between a URI and a URL?

2010-01-06T00:04:00.000Z

If you get confused between URLs (Uniform Resource Locators) and URIs (Uniform Resource Identifiers), then simply think of a URL as a special type of URI used to point to resources on the WWW.

The term "URL" is now deprecated (a term meaning marked for removal in computing circles), and the term "URI" is now the accepted term in computer science. However with the rise of the internet "URL" has become entrenched within the English language and is used and understood widely among non computer scientists. It's not possible for any group of individuals to deprecate words (although plenty have tried...) so "URL" will probably long outlive its technical definition.

At the end of the day, I expect that if you should use the term "URL" when you really mean "URI" everybody will know what you mean!

Strip and manipulate a URL by breaking it into segments (.NET 2.0)

2010-01-05T22:00:00.001Z

I recently required a method that would take a string that contain a URL (href), and another string that contained a root section of this URL (root), and return a string that contained the remained of the URL (i.e. the section of href that remained once the root had been removed).

To make matters more complicated, the href parameter could have some unusual features. Because this URL was pointing to content created by users in a Content Management System (CMS), some segments of the URL contained trailing or leading white space (segments being the bits of the URL between the slashes). This whitespace is fine in the CMS system, but my method must strip this whitespace to return a canonical URL.

Fortunately .Net 2.0 onwards provides us with the URI class. This has lots of fabulous methods and properties, but in this example I shall use it to:

turn the parameter strings "root" and "href" URLs into canonical URIs
break down the "href" parameter into segments,
ignore the segments that exist in the "root" parameter,
strip leading and trailing whitespace from the remaining segments
return the canonicalised section of the URL

string GetRootStrippedURI(string root, string href)
{
    Uri fileUri = new Uri(Uri.UnescapeDataString(href));
    Uri rootUri = new Uri(Uri.UnescapeDataString(root));

    // Create the return string from the root
    string strippedExtension = "";

    // Loop through segments not in the root and clean them up
    for (int i = rootUri.Segments.Length; i < fileUri.Segments.Length - 1; i++)
    {
        strippedExtension += fileUri.Segments[i].TrimEnd().TrimStart();
    }
    return strippedExtension;      
}

A c# timer stopwatch class to help analyse lengthy program execution

2009-12-15T22:12:00.002Z

I have recently been developing a piece of software that uses the old Microsoft COM Interop technology to walk through a Sharepoint 2001 folder structure. I found that this was working pretty slowly, but by the time I came around to do anything about it the code had got fairly complicated and it wasn't clear where the bottleneck was occuring.

To help understand the problem I created this simple static timer class:

class Timer
    {
        static public DateTime timeStart;
        static public DateTime timeEnd;
        static public bool debugTimer = true;  //Switch to false to turn off timing messages
 
        public  Timer()
        {
        }
 
        static public void TimerStart(string msg)
        {
            if (! debugTimer) return;
            timeStart = DateTime.Now;
            Console.WriteLine("TIMER: " + msg + " BEGUN: " + timeStart.ToString(), true);
        }
 
        static public void TimerEnd()
        {
            if (!debugTimer) return;
            timeEnd = DateTime.Now;
            TimeSpan totalTime = new TimeSpan();
            totalTime = timeEnd.Subtract(timeStart);
 
            Console.WriteLine("TIMER END: " + timeEnd.ToString() + " Total time: " + totalTime.Milliseconds, true);
        }
    }

In order to use this in your code, simply copy the class and put the following lines around whatever process you wish to monitor:

Timer.TimerStart("Beginning timer");
    //...
    //do lenghtly process
    //...
    Timer.TimerEnd();

This will give a nice message, with the time elapsed in milliseconds.

Creating Sharepoint list fields on the fly with web services

2009-11-27T19:06:00.007Z

I've recently been trying to create Sharepoint 2007 list fields on the fly using the Lists.UpdateLists() method described here:
http://msdn.microsoft.com/en-us/library/lists.lists.updatelist.aspx

Unfortunately, after modifying the sample code to my needs, a SOAP exception kept getting thrown: "NodeType 'Element' is not an valid type":

<errorstring xmlns="http://schemas.microsoft.com/sharepoint/soap/">'Element' is an invalid XmlNodeType. Line 1, position 10.</errorstring>

Using Visual Studio 2005 to debug, I have been unable to work out what the problem was. The CAML query built in the webservice to create the field must be incorrect, but the innerXML of the CAML query (I was only using the "updateFields" node) looked fine. Clearly the problem was elsewhere in the call, but is not clear to me how the call is constructed once the parameters are passed to the web service.

SOAP does not seem to me to give a huge amount of debugging information when a call fails. When I create my own web services, I can use VS to step through them and get lots of debug information, but I haven't been able to work out how to do this with the MS DLLs yet.

Anyway, I eventually solved this problem by going back to the example and creating another copy of the example Microsoft code with minimal changes in order to point the call at my webservice. This proved to work OK, although it did create two sample fields (Field1 and Field2) that I cannot remove using the Sharepoint IE front end!. I suspect this is becuase they are created as required fields.

By making minimal iterative changes, I finally managed to create the following c# method which successfully creates the fields on the fly, and even gives them an appropriate Sharepoint type.

(Please note this code looks better in IE than Firefox which is not wrapping the lines - click on "View Plain" to view or copy in Firefox)

public bool CreateNewField(ListInfo listInfo, string fieldName, object fieldValue)
{
    bool canCreateField = true;
    try
    {

                //ListsService.Lists listService = new ListsService.Lists();
                //listService.Credentials= System.Net.CredentialCache.DefaultCredentials;
                //NOTE: I have already instatiated my connection (listService) and the GUID of the list (listInfo.listName)

                XmlNode ndList = listService.GetList(listInfo.listName);
                XmlNode ndVersion = ndList.Attributes["Version"];

                XmlDocument xmlDoc = new System.Xml.XmlDocument();

               

                XmlNode ndNewFields = xmlDoc.CreateNode(XmlNodeType.Element,
                    "Fields", "");
                XmlNode ndUpdateFields = xmlDoc.CreateNode(XmlNodeType.Element,
                    "Fields", "");

               
                
                // Field types
                if(fieldValue.GetType() == Type.GetType("System.String"))
                {
                    ndNewFields.InnerXml = @"<Method ID='1'>" +
                   "<Field Type='Text' DisplayName='" + fieldName + "' Required='FALSE' FromBaseType='FALSE' Description='Generated Field'/>" +
                   "</Method>";
                }
                else if (fieldValue.GetType() == Type.GetType("System.Int") || 
                         fieldValue.GetType() == Type.GetType("System.UInt16") ||
                    fieldValue.GetType() == Type.GetType("System.UInt32") ||
                    fieldValue.GetType() == Type.GetType("System.UInt64")
                    )
                {
                    ndNewFields.InnerXml = @"<Method ID='1'>" +
                  "<Field Type='Integer' DisplayName='" + fieldName + "' Required='FALSE' FromBaseType='FALSE' Description='Generated Field'/>" +
                  "</Method>";
                }
                else if (fieldValue.GetType() == Type.GetType("System.Double"))
                {
                    ndNewFields.InnerXml = @"<Method ID='1'>" +
                  "<Field Type='Number' DisplayName='" + fieldName + "' Required='FALSE' FromBaseType='FALSE' Description='Generated Field'/>" +
                  "</Method>";
                }
                else if (fieldValue.GetType() == Type.GetType("System.Date"))
                {
                    ndNewFields.InnerXml = @"<Method ID='1'>" +
                  "<Field Type='DateTime' DisplayName='" + fieldName + "' Required='FALSE' FromBaseType='FALSE' Description='Generated Field'/>" +
                  "</Method>";
                }
                else if (fieldValue.GetType() == Type.GetType("System.Array"))
                {
                    ndNewFields.InnerXml = @"<Method ID='1'>" +
                  "<Field Type='Text' DisplayName='" + fieldName + "' Required='FALSE' FromBaseType='FALSE' Description='Generated Field'/>" +
                  "</Method>";
                }
                else
                {   // Data type not handled - skip field
                    //Logger is my helper class - Remove from your code
                    Logger.LogMessageToFile("WARNING: Undefined field <" + fieldName + "> has an unhandled data type of: " + fieldValue.GetType().ToString() + "\r\n" +
                        "This field cannot be created");
                    canMigrateField = false;

                }
                Type myType = fieldValue.GetType();
                
                if (canCreateField)
                {
                    //NOTE: You need to use the GUID in the listname field.  I have this pre-populated in the listInfo.ListName class
                    XmlNode ndReturn =
                     listService.UpdateList(listInfo.listName,
                     null, ndNewFields, null, null,
                     ndVersion.Value);


             Logger.LogMessageToFile("NEW FIELD <" + fieldName + "> Created successfully.\r\n");
        }

    }

    catch (Exception ex)
    {
        Logger.LogMessageToFile("Message:\n" + ex.Message + "\nStackTrace:\n" + 
            ex.StackTrace);
        Logger.PopupMessage();
    }

    if (canCreateField) return true;
    else return false;
}

Blog Archive

2009-09-12T13:24:00.057+01:00

Deploy Office 2003 VSTO add in to All Users using Visual Studio 2008

Sharepoint 2007 Batch Update example using web services and CAML

Binding a CheckBoxList object to an ordered Sharepoint list of reference values in a web part (c#)

Render Content Type fields as a form in Sharepoint Web Part using c#

Colour code a single column in Sharepoint 2007 list view using Jquery

Write lines to a RichTextBox control in different colours (VB .net example)

VBA code to iterate through the results of GetListCollection web service from Sharepoint 2007

Set Sharepoint meta-data from VBA using updatelist web service

SQL Server - MySQL 4 integration - openquery does not work for tables with datetime values of 0000-00-00

Use a variable as a table name in an SSIS Execute SQL Task

Blogger.com code syntax highlighting, or always remember to save your code before you start playing around with it!

Manage c# threads easily using an array of BackgroundWorker class objects

What is the difference between a URI and a URL?

Strip and manipulate a URL by breaking it into segments (.NET 2.0)

A c# timer class to help analyse lengthy program execution

Creating Sharepoint list fields on the fly with web services