Anton Gochev's Weblog

EXECUTE permission denied on object ’sp_sdidebug’, database ‘master’, owner ‘dbo’.

ganton — Fri, 13 Nov 2009 15:54:28 +0000

XmlConverter performance improvement

ganton — Thu, 06 Aug 2009 09:16:33 +0000

Last year I wrote a post about how to serialize and desterilize objects to/from XML using standard .Net Framework classes (see this post). Yesterday, I need to use the class but I need to improve the performance. When one creates XmlSerializer class many time is spend to configure it behind the scene. After its initialization serialization and deserialization is much faster. I’ve decided to store already configured XmlSerializer and to reuse it. This will improve the performance significantly. For this purpose I extended the XmlConverter class from my old post. XmlConverter stores in a static hash table each created XmlSerializer and then reuse it. If it is not required to store it it will not store it. Now XmlSerializer contains two new methods FromXml and ToXml that accept a parameter named ‘useStaticSerializer’ which tells the XmlConverter to store and reuse XmlSerializer or not. Below is the new code of the XmlConverter.

using System; 
using System.Collections.Generic; 
using System.Linq; 
using System.Text; 
using System.IO; 
using System.Xml.Serialization; 
using System.Xml; 
  
public static class XmlConverter 
{
    private static Dictionary serializers = new Dictionary();
  
    private static XmlSerializer GetSerializer(bool useStaticSerializer, Type serializerType)
    {
        XmlSerializer serializer = null;
        if (useStaticSerializer)
        {
            lock (serializers)
            {
                if (!serializers.TryGetValue(serializerType, out serializer))
                {
                    serializer = new XmlSerializer(serializerType);
                    serializers.Add(serializerType, serializer);
                } 
            }
        }
        else
        {
            serializer = new XmlSerializer(serializerType);
        }
        return serializer;
    }
  
    public static string ToXml(T value) where T : new() 
    { 
       return ToXml(value, Encoding.UTF8, false); 
    } 
  
    public static string ToXml(T value, Encoding encoding) where T : new() 
    { 
       return ToXml(value, Encoding.UTF8, false);
    }
  
    public static string ToXml(T value, Encoding encoding, bool useStaticSerializer) where T : new()
    {
        try
        {
            XmlSerializer serializer = GetSerializer(useStaticSerializer, typeof(T));
            using (MemoryStream stream = new MemoryStream())
            {
                using (TextWriter writer = new StreamWriter(stream, encoding))
                {
                    serializer.Serialize(writer, value);
                    int cnt = (int)stream.Length;
                    byte[] arr = new byte[cnt];
                    stream.Seek(0, SeekOrigin.Begin);
                    stream.Read(arr, 0, cnt);
                    return encoding.GetString(arr, 0, arr.Length).Trim();
                }
            }
        }
        catch { }
        return null;
    }
  
    public static T FromXml(string xml) where T : new() 
    {            
       return FromXml(xml, false); 
    }
  
    public static T FromXml(string xml, bool useStaticSerializer) where T : new()
    {
        try
        {
            XmlSerializer serializer = GetSerializer(useStaticSerializer, typeof(T));
            using (StringReader stream = new StringReader(xml))
            {
                using (XmlTextReader reader = new XmlTextReader(stream))
                {
                    return (T)serializer.Deserialize(reader);
                }
            }
        }
        catch { }
        return default(T);
    }
  
    public static void ClearStaticSerializers()
    {
        lock (serializers)
        {
            serializers.Clear(); 
        }
    }
  
}

Think twice before to implement some idea – # 1

ganton — Thu, 09 Jul 2009 17:24:35 +0000

I’ve already written some posts that show some source code that can be rewritten in a different way and this way to achieve better performance. Now I will open a series with a topic of this post with a number and I’ll post different samples of implementations that I’ll run into and I’ll do some simple tests in order to find which is better. I’m not an expert of the performance improvement and for sure not all my conclusions are correct but let me share my opinion.

Sometime ago I had to improve the performance of a part of the source code in one of our applications and during this process I decided to cache in-memory some data frequently used during the operation that I was optimizing. I needed all the collected information in small arrays by parent id in a hash table. Initially, I used a list to prepare the array and after that each time to call List.ToArray() in order to use the information (it is not important why it is a matter of other implementations in the application). Then I’ve decided to prepare the arrays at the beginning and to save time in their future use. I wrote a code similar to those below.

Dictionary<int, Item[]> dictionary = new Dictionary<int, Item[]>();
for (int i = 0; i < items.Length; i++)
{
    Item item = items[i];
    Item[] currentItems;
    if (dictionary.TryGetValue(item.ParentId, out currentItems))
    {
        Item[] newCurrentItems = new Item[currentItems.Length + 1];
        Array.Copy(currentItems, newCurrentItems, currentItems.Length);
        newCurrentItems[newCurrentItems.Length - 1] = item;
        dictionary[item.ParentId] = newCurrentItems;
    }
    else
    {
        dictionary.Add(item.ParentId, new Item[] { item });
    }
}

Yesterday, a colleague of mine Saso Jordanoski ask me why I didn’t use a List in order to prepare the arrays. We’ve started a conversation and I’ve explained why but the point of Saso was that I re-allocate so much memory creating a lot of unusual arrays and that it will be better to use a List. Yes, in this case we think it is true but during the discussion we came up with an idea to just use the array of items with sorted by parent id array of items and to copy from it. And do you know? The result of my simple test shows that the last idea is much better, almost 8 times :). Think twice before to implement an idea! Below is the implementation of the last idea.

int count = 0;
for (int i = 0; i < items.Length; i += count)
{
    Item item = items[i];
    count = Count(items, i, item.ParentId);
    Item[] currentItems = new Item[count];
  
    Array.Copy(items, i, currentItems, 0, count);
    dictionary.Add(item.ParentId, currentItems);
}

It even looks simpler and easy to be understood. Below is the fool code of the test application.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;
  
namespace CopyToTest
{
    class Program
    {
        static void Main(string[] args)
        {
            Item[] items = new Item[100000];
  
            for (int i = 0; i < 20000; i++)
            {
                int startIndex = i * 5;
                for (int j = startIndex; j < startIndex + 5; j++)
                {
                    items[j] = new Item { ParentId = i, Value = string.Empty };    
                }                
            }
  
            Stopwatch sw = new Stopwatch();
            sw.Start();
            Dictionary<int, Item[]> dictionary = new Dictionary<int, Item[]>();
            for (int i = 0; i < items.Length; i++)
            {
                Item item = items[i];
                Item[] currentItems;
                if (dictionary.TryGetValue(item.ParentId, out currentItems))
                {
                    Item[] newCurrentItems = new Item[currentItems.Length + 1];
                    Array.Copy(currentItems, newCurrentItems, currentItems.Length);
                    newCurrentItems[newCurrentItems.Length - 1] = item;
                    dictionary[item.ParentId] = newCurrentItems;
                }
                else
                {
                    dictionary.Add(item.ParentId, new Item[] { item });
                }
            }
            sw.Stop();
  
            Console.WriteLine(sw.ElapsedMilliseconds);
  
            sw.Reset();
            sw.Start();
            dictionary = new Dictionary<int, Item[]>();
            int count = 0;
            for (int i = 0; i < items.Length; i += count)
            {
                Item item = items[i];
                count = Count(items, i, item.ParentId);
                Item[] currentItems = new Item[count];
  
                Array.Copy(items, i, currentItems, 0, count);
                dictionary.Add(item.ParentId, currentItems);
            }
            sw.Stop();
  
            Console.WriteLine(sw.ElapsedMilliseconds);
            Console.Read();
        }
  
        private static int Count(Item[] array, int startindex, int parentId)
        {
            int count = 0;
            for (int i = startindex; i < array.Length; i++)
            {
                if (array[i].ParentId == parentId)
                {
                    count++;
                }
                else
                {
                    break;
                }
            }
            return count;
        }
    }
  
    public class Item
    {
        public int ParentId;
        public string Value;
    }
}

And you can see the results on my machine below.

is operator + object casting or as operator + null comparison?

ganton — Sat, 27 Jun 2009 20:44:46 +0000

This week I found in the source code of one of our projects that people use “is” operator + object casting in places that they can use “as” operator + null comparison.

// "is" operator + object casting
if (myObj is MyObject)
{
    MyObject obj = (MyObject)myObj;
}
  
// "as" operator + null comparison
MyObject obj = myObj as MyObject;
if (obj != null)
{
    // do some work with obj
}

What is the problem? The problem is that in the first case you cast the object two times which is slower than if you cast the object only one time as in the second case. Why it happens? Because the “is” operator casts the object behind the scene and if the cast is successful it will return true, otherwise false. Consequently, the first case is slower than the second case. You may suggest that nowadays the hardware is very fast and you will be correct but the problem is that most of the developers that use the first case even don’t understand why is better to use the second one. Below is the test sample for the performance that I did. The count is a very high number that will allow us to see the difference.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;
  
namespace AsIsTest
{
    class Program
    {
  
        public class MyObject
        {
            public MyObject Parent { get; set; }
        }
  
        static void Main(string[] args)
        {
            int count = 100000000;
            object myObj = new MyObject { Parent = new MyObject() };
            Stopwatch watch = new Stopwatch();
            watch.Start();
            for (int i = 0; i < count; i++)
            {
                MyObject obj;
                if (myObj is MyObject)
                {
                    obj = (MyObject)myObj;
                }
            }
            watch.Stop();
            Console.WriteLine("is and cast time if true: " + watch.ElapsedMilliseconds);
  
            watch.Reset();
            watch.Start();
            for (int i = 0; i < count; i++)
            {
                MyObject obj = myObj as MyObject;
                if (obj != null)
                {
  
                }
            }
            watch.Stop();
            Console.WriteLine("as and null comparison time: " + watch.ElapsedMilliseconds);
  
            Console.Read();
  
        }
    }
}

The result is below.

Note that in the sample above both operators are able to cast the project. In this case there is a performance advantage of the second sample but if the object cannot be casted you will get the similar results for both cases because only one casting takes place in the first sample. Below is a sample when the object can not be casted.

watch.Start();
for (int i = 0; i < count; i++)
{
    string obj;
    if (myObj is string)
    {
        obj = (string)myObj;
    }
}
watch.Stop();
Console.WriteLine("is and cast time if true: " + watch.ElapsedMilliseconds);
  
watch.Reset();
watch.Start();
for (int i = 0; i < count; i++)
{
    string obj = myObj as string;
    if (obj != null)
    {
  
    }
}
watch.Stop();
Console.WriteLine("as and null comparison time: " + watch.ElapsedMilliseconds);

And the result is below.

It’s funny but on my machine the results are completely the same. I’ve started the application several time and they are the same.

SQL Server Setup failed to obtain system account information for the ASPNET Account….

ganton — Fri, 19 Jun 2009 14:05:18 +0000

Today, during installation of MS SQL Server 2005 DE on my new work machine i ran into an installation error that stated “SQL Server Setup failed to obtain system account information for the ASPNET Account.To proceed, reinstall the .Net Framework, and then run SQL Server Setup again.”. And it even offer me to reinstall the .Net Framework and then to retry to install the MS SQl Server.

Please do not follow this message directly if you have installed the .Net Framework. The problem in my case was that I have installed the framework on the machine but ASP.Net wasn’t installed. In other words you just need to run aspnet_regiis.exe with a switch –i. Then you can run iisreset in order to be sure that all changes have taken an effect. It will install the runtime and all other things and you can continue with the installation.

Enjoy your installation of MS SQL Server 2005 DE.

Configure linked servers for Excel and MS SQL Server in MS SQL Server 2005

ganton — Wed, 22 Apr 2009 08:47:33 +0000

Linked servers allows

A linked server configuration allows Microsoft® SQL Server™ to execute commands against OLE DB data sources on different servers

Quote from MSDN

This way we have an abstract layer to other data sources (different from MS SQL Server) and we can connect them using T-SQL of MS SQL Server and only one instance of it. In addition, we can join tables from different resources which is completely transparent for our stored procedures or ad hoc queries.

In this post I’ll show you how you can configure a linked server for Excel file and a linked server for a MS SQL Server database. Assume that we have an Excel file named people.xls as those below

We also have a MS SQL Server database named test with one table named test. Well, the names are not too descriptive but for the sample are good enough.

Assume that we’d like to use the information from the excel file and also to extend it with some information like address and city of the person. I added external_id column into the table in order to be able to join the table with the Excel file. This field contains the same values as id column of the excel file.

Before to configure linked services you should ensure that DTC (Distributed Transaction Coordinator) Service is started.

You can configure both using the Microsoft SQL Server management Studio but I’ll use a query to do so. SQL Server contains a stored procedure named sp_addlinkedserver. It allows you to create linked servers.

   1: exec master.dbo.sp_addlinkedserver    2:     @server = N‘Excel’,    3:     @srvproduct=N‘Excel’,    4:     @provider=N‘Microsoft.Jet.OLEDB.4.0′,    5:     @datasrc=N‘C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Data\people.xls’,    6:     @provstr=N‘Excel 5.0′;

The @server parameter is the name that we will use to connect the server. The @srvproduct is not required here but left it with a value of Excel. The @provider parameter shows which OLE DB provider we gonna use. In our case it is Microsoft.Jet.OLEDB.4.0 provider. The @datasrc is the full path to the Excel file and @provstr should be Excel 5.0. After executing the procedure we can access the linked server. I’ll use sp_tables_ex stored procedure to retrieve all sheets (tables) that current linked server contains.

   1: exec sp_tables_ex @table_server=N‘Excel’

The result is below.

Note that you should not use the Excel file with any other program when you are using it from the SQL Server. If you do so you will get an exception like those below.

Msg 7399, Level 16, State 1, Procedure sp_tables_ex, Line 41

The OLE DB provider "Microsoft.Jet.OLEDB.4.0" for linked server "Excel" reported an error. The provider did not give any information about the error.

Msg 7303, Level 16, State 1, Procedure sp_tables_ex, Line 41

Cannot initialize the data source object of OLE DB provider "Microsoft.Jet.OLEDB.4.0" for linked server "Excel".

Having the linked server named Excel we can query the file using a script like those below.

   1: select * from Excel…Sheet1$

The result is below.

Now let’s connect our SQL Server database as a linked server. To do so I’ll use the same stored procedure as before but with a different OLE DB provider.

   1: exec master.dbo.sp_addlinkedserver    2:     @server = N‘SQL’,    3:     @srvproduct=N”,    4:     @provider=N‘SQLNCLI’,    5:     @datasrc=N‘GOCHEVA\SQLEXPRESS’;

We are still not ready to connect to the SQL linked server. Before trying to access it we should allow remote connections to the target SQL Server Instance. You can complete this action via SQL Server 2005 Surface Area Configuration as shown below.

Now I can connect it and get all databases that are the SQL Server instance contains and I also can get their table information. Using sp_catalogs stored procedure I’ll get all DBs.

   1: exec sp_catalogs @server_name=N‘SQL’

And the result.

I’ll use sp_tables_ex to get all tables of a specific catalog (database) of the SQL linked server.

   1: exec sp_tables_ex    2:     @table_server=N‘SQL’,    3:     @table_schema=N‘dbo’,    4:     @table_catalog=N‘test’;

And the result.

Now, we can use an as hoc query using T-SQL and to retrieve a joined information from both files.

   1: select s.*, t.address, t.city    2: from Excel…Sheet1$ as s, SQL.test.dbo.test as t   3: where s.id = t.external_id

And the result is.

That’s all for now. have fun using SQL Linked Server option.