Kevin Miller

Introducing Dovetail DataMap

kmiller — Mon, 27 Apr 2009 15:46:29 GMT

I want to introduce a handy library called Dovetail DataMap created during the development of Dovetail Mobile. DataMap makes it easy to populate model objects from a Clarify/Dovetail CRM database. It is a sort of one-way Clarify specific object relational mapper tool.

Your CRM data to Objects

It is very common to need a way to pull data out of your CRM for creation of a user interface. What if you could define a plain old C# object to receive data and a map which defines how data in your Clarify database should populate that .Net class? Does this sound better than hand rolling your own code every time? I hope it does. Let’s take a look at a scenario.

A Plain Old C# Object

Let’s say you want to create a user interface that displays solutions. First thing you do is create a class with a public property for each piece of data about the solution you wish to display.

public class Solution
{
	public string ID { get; set; }
	public DateTime Created { get; set; }
	public string Title { get; set; }
	public string Description { get; set; }
	public bool IsPublic { get; set; }
}

A Data Map

Next up you define a map for the Solution object that describes where the data for each Solution property comes from.

public class SolutionMap : DovetailMap<Solution>
{
	protected override void MapDefinition()
	{
		FromTable("probdesc")
			.Assign(d => d.ID).FromIdentifyingField("id_number")
			.Assign(d => d.Title).FromField("title")
			.Assign(d => d.Description).FromField("description")
			.Assign(d => d.Created).FromField("creation_time")
			.Assign(d => d.IsPublic).BasedOnField("public_ind").Do(isPublic => isPublic == "1")
	}
}

In the map above you are seeing what is called a Fluent Interface. The idea is to make this map very readable while being easy to edit and extend. Hopefully after taking a few seconds to look between the Solution class and this map you’ll understand what is happening.

What table is the solution map getting its data from? The Solution object maps to the probdesc database table.
Which field is the identifying field? The id_number field maps to the ID property and is the identifying field.
When is the IsPublic property true? When the public_ind field is “1”.

The map above acts as the glue between the Solution class and another object called an assembler.

Assembling Solutions

The assembler does all the hard repetitive ookie work for you. It does stuff like querying and retrieving data from the database, and with that data, creates Solution objects. The best part is all you have to do is create one and tell it which solutions you want.

Getting a solution by identifier

//Use an assembler to retrieve a solution by id. 
string solutionID = "124";
Solution solution = solutionAssembler.GetOne(solutionID));

Remember the identifying field in the map above? When the assembler is getting one Solution filtering by the id_number field. Here is a view of the Solution object from the debugger.

Solutions created in the last week

Solution[] solutions = solutionAssembler.Get(FilterType.WithinDays("creation_time", 7));

Getting solutions from the assembler in an ad hoc manner requires that you give the assembler criteria about which solutions you want returned. Take a look at the documentation for all the types of filters you can use and at Dovetail SDK advanced filtering if you want to learn how to chain these filters together.

DataMaps In Production

How would you use something like this? The Solution class we are using here is very similar to what Dovetail Mobile is using.

Our Dovetail Mobile product is using the new ASP.Net Model View Controller (MVC) framework. We use the DataMaps library to create the Model part of this equation. Here is an example of all the work our Mobile solutions controller needs to create the screen you see above.

public ActionResult Show(string id)
{
	var solutionViewModel = _solutionAssembler.GetOne(id);

	return View("ShowSolution", solutionViewModel);
}

DataMaps makes doing the data retrieval for this controller action quite simple.

Finally – What do you think?

Are you a Dovetail SDK customer? Can you see how Dovetail DataMap might be handy in your application? Do you want to know more? I’ve just hit the surface with this post. We want to see if there is more interest before dedicating more time and resources to making something like this available. Post a reply or send me a Tweet with what you think.

Syntax Highligher

kmiller — Fri, 24 Apr 2009 13:40:51 GMT

This is a sample post using Alex’s very nice looking Syntax Highlighter for code snippets. Let’s see if I can get Community Server to behave.

Teaser Code

public class SolutionMap : DovetailMap
{
	protected override void MapDefinition()
	{
		FromTable("probdesc")
			.Assign(d => d.SolutionID).FromIdentifyingField("id_number")
			.MapMany().To(d => d.Resolutions).ViaRelation("probdesc2workaround", workaround => workaround
				.Assign(d => d.DatabaseIdentifier).FromField("objid")
			);
	}
}

Update: Looks like I got it working. Sorry about the guinea pig post. The code above is actually something I have been wanting to post about for some time. Maybe now with these super sweet code blocks I’ll get’er done.

Installing Visual SVN Server

kmiller — Wed, 15 Apr 2009 18:59:48 GMT

After being forced to re-install a Apache hosted Subversion service I decided to use Visual SVN Server and found the process of migrating from a existing multi-repository Apache Subversion service to be quite straight forward. I did learn a few things along the way that I wanted to share.

Why I Had to do it. (a.k.a Whoops!)

I did a dumb thing. I attempted to upgrade the version of our own existing Apache Subversion service in the middle of the day. You likely know how the rest goes, terribly. I Backed up all the files I was going to touch and dropped in the new .dlls in the right places and viola upgrade! right?. Not so much. The server didn’t start. So I reverted to my backups and the server still didn’t start. Oh noes!!!111one. Apache puked and people started to complain that they couldn’t commit.

The repositories were backed up and intact so I was not worried about losing data just getting the service back up. I had wanted to move to Visual SVN Server since they announced so now I had my chance.

Installation

The installer was quite easy to use and defaults to installing a secure self-signed Apache hosted Subversion web server. There is a wonderful MMC Snap-in that you use to manage the installation. Don’t worry. You can figure it out. They made it easy.

User and Groups Migration

We had pre-existing groups and users files in our repository root folder. All I had to do was rename the group file to authz and the user file to htpasswd. Unfortunately the hash used in the htpasswd file was different than our previous one so our users had to reset their passwords. This was only a minor pain.

SSL Certificate

We have a SSL certificate we use to secure our Subversion server. To get our pre-existing SSL certificate to be used by Visual SVN Server I had to combine our .key and .cer files into one file called server.pem found in C:\Program Files\VisualSVN Server\conf

Hosting Custom Web Content

Your Visual SVN Server has a nice front end to your repository that authenticates and allows users to navigate source code. It is easy to customize the look and feel of this code browser by editing the images or css that is in your Visual SVN Server configuration directory. Found in: C:\Program Files\VisualSVN Server\htdocs

We have a post commit hook which generates an RSS with details about each commit. This feed file needs to be served up by the web server. Easy, just copy the file somewhere under the htdocs directory and all is good.

Speaking of Hooks

The management snap-in has a nice UI for editing your SVN hooks batch file: Repository –> All Tasks -> Manage Hooks.

Conclusion

My hat is off to he Visual SVN team for giving the community this excellent free tool for setting up and managing a Subversion server. Thank You.

I am sure their strategy in this is to grow a community of users who will pay for their excellent Visual SVN tool. I’ve been a user of Visual SVN for many years and have nothing bad to say about it. It just makes using Subversion flow naturally. They keep it up-to-date, and they keep adding features What else do you need? Apparently, you’ll need 49$ for a license but I tell you it is very much worth it.

Can your CRM place a phone call?

kmiller — Mon, 13 Apr 2009 17:30:59 GMT

When new cases come in we have many ways of having our support team notified so we can provide speedy assistance to our customers.

Recently we needed a better way to notify on-call support agents of new cases. Our existing notification mechanisms work great during office hours but at night a spam email getting past my filters better not wake me up at 2am just because it might be a new support case. We needed something better. The simplest thing we could think of was to have our CRM make a phone call.

Phone calls are cross platform. No special smart phone client application beats a phone call. Better yet spammers don’t place calls at 2am pitching “Enlargement”. At night I can confidently leave my phone volume set to 11 with a screeching Ride of the Valkyries ring tone and I sleep in peace knowing that if my phone rings at 2am it is truly important. Or else I should get friends that don’t drunk dial me. Stop it guys. You know who you are.

What I Want To Happen

We need a rule in our CRM system that places a phone call to the current on-call support agent when an Urgent or High priority case comes in.

We already have Rulemanager which we have talked about many times before. The new thing here is having a rule action that places the phone call. Kudos and bonus points if it could say something catchy like “There is a new Urgent or High priority support case.” The Agent receiving the call at this point can use our Dovetail Mobile web application to work the case.

What Didn't Work

We don’t have a fancy PBX or the Telephony skills and hardware to setup our own Asterisk server.

Skype seemed like a good inexpensive solution as they have Skype Out and a Skype4Com API that seems pretty nice. Sadly my attempts to get the Skype API working were only intermittently successful and I have learned to avoid COM whenever possible so I moved on.

Jajah looked interesting. Ribbit I just don’t understand. PhoneNotify seemed perfect but expensive and the API looks a little klunky.

Then I Found Web + Telephony Simplicity

Twilio is a telephony web service that is simple to use, quite powerful, and priced very reasonably.

Our make a phone call requirement appears to be Twilio’s most basic capability. To do this essentially all you need to do is call the Calls Rest API giving it a number to call and the public URL of a script which will control the conversation.

Start A Call

To have our CRM place a call we need to have Rulemanager run a console application giving it a number of the current on-call agent. To do this we created a rule whose action calls a console application passing it a phone number as the first argument.

This Support-Dialer console application will invoke the Twilio Calls Rest API. Twilio makes it easy for .Net developers by having a thin C# helper library with good examples. Here are the important bits of the Support-Dialer console application.

private static void Main(string[] args)

    var logger = GetLogger();

    if(args.Length < 1 )

        const string message = "Could not dial number as no arguments were supplied";

        logger.ErrorFormat(message);

        return;

    var phoneNumberToCall = args[0];

    logger.InfoFormat("Calling {0}.", phoneNumberToCall);

    // Create Twilio REST account object using Twilio account ID and token

    var account = new Account(AccountSid, AccountToken);

    // Initiate a new outbound call

    var restCallUrl = String.Format("/{0}/Accounts/{1}/Calls", ApiVersion, AccountSid);

    var parameters = new Hashtable

            {"Caller", DovetailSupportCallerId},

            {"Called", phoneNumberToCall},

            {"Url", SupportDialerConversationUrl},

            {"Method", "GET"}

};

    var requestResult = account.request(restCallUrl, "POST", parameters);

    logger.Debug(requestResult);

All this really does is make a web request to Twilio’s Calls Rest API giving it the number to call, the callerId of the number the call should appear to come from, and the URL of the conversation script XML.

Scripting the Conversation

Once a call is in progress Twilio needs to know what to do. Twilio has an XML markup for controlling what happens during a call. In our case all I want to happen is that the support agent is made aware of a new urgent case so this static XML does the trick.

<?xml version="1.0" encoding="UTF-8" ?>

<Response>

    <Say loop="3">There is a new urgent or high priority support case. Please check Dovetail mobile.</Say>

</Response>

If we wanted to we could do more. Have the script Say details of the case to or allow the agent to Dial the customer contact immediately.

Wrap Up

I am very impressed with Twilio. Given my use case was very simple I went from knowing nothing about Twilio to having an integration with our enterprise CRM completed in about 3 hours doing everything from getting an account setup and charged to leveraging their example code and integrating it with our business processes. Their website is a great resource for learning how to use the Twilio service with plenty of examples to get you going.

Looking at the 5 primary verbs Say, Play, Gather, Record, and Dial has my brain spinning with ideas for where we could be using Twilio to create great customer experiences when the web is just not good enough.

Minor Complaint and or Suggestion

It would be nice if Twilio provided a way when invoking their Calls Rest API to specify a static conversation script. It was weird to have to host a static file on a web site just to have the call say something to the receiver.

Searching For Motivation

kmiller — Fri, 20 Feb 2009 17:26:28 GMT

People love search. I love search. I have to admit though that I am guilty of not using my own tools to the best of their ability. Dovetail Seeker is a search product we created to find important knowledge stuck in your Clarify/Dovetail CRM. At Dovetail we use our own software to support our customers and Seeker is always indispensible, when working new cases, effectively mining our solutions and previous cases for knowledge.

That said, I am stuck in my old ways. I’ve been supporting our product line since before we had an effective search tool and whenever I get a support question I am quick to rack my brain and not our knowledge base.

To combat this affliction I created this motivational poster in effort to remind me to hit the search button before I think too hard.

When a Guid is not a Guid?

kmiller — Wed, 18 Feb 2009 15:17:08 GMT

Databases are fun things. Assigning identifiers to things in your database can be important. Sometimes it is handy to have an Uber alternate key that is guaranteed globally unique. This can be useful in many ways one of which is database replication.

Globally Unique Identifiers

I just ran into a spot in a legacy database where it seems they wanted to future proof globally identifying database rows. They have quite a few tables with a field called guid. That might ring a bell. In case it doesn’t GUIDs are a Microsoft implementation of globally unique identifiers. Easy to create. 16 bytes wide and lovely to look at.

string guid = Guid.NewGuid().ToString();

value: ba72c396-4b49-4d95-b9d7-03d42d5d5141

Funny thing about this “guid” database field is that it’s a Unicode string 31 characters wide. You might notice the Guid value above is 36 characters. Ok, Get rid of the dashes and it is 32 characters.

Crap still too big. Wait, I need to shove this hex encoded value into a 31 character Unicode field?

Sigh, There is a lot wrong with that statement. Something smells wrong. Good thing you can base64 encode the raw bytes of a Guid into 24 characters.

string base64Guid = Convert.ToBase64String(Guid.NewGuid().ToByteArray());

value: JD/V9w5370OzsVK0RQ4u3A==

Wait a second

But still that’s fishy why do I have so much spare room left over. What are they getting at? I started looking at what values were actually in these guid fields and found a surprise.

12345678zzABCTowingbbbbb00000123

What the heck?

Oh look, 12345678 is part of the database identifier for the row.
Oh look, 0123 is the id for the database table (long story.)
Is that a company name in there?

Why are they manually creating “globally unique identifiers” with what is technically a compound key. Sure they are technically globally unique in this database but collisions could easily occur with other databases (for the same company).

So much wrong with this.

To set the “guid” field you have to know the database row’s identifier. Which means you need to insert the row. Get the identifier back create the faux guid and then update the row again. Yuck, that is a pretty chatty create mechanism.

Worse yet. What if someone couples to the information encoded into the faux guids?

Do I need to replicate their broken technique? Nah, I’ll just shove Base64 encoded Guids in there.

Setup ASP.Net caching using a super simple fluent interface

kmiller — Tue, 10 Feb 2009 18:13:11 GMT

While reviewing what I did to Create an RSS Feed Using ASP.Net. We got worried that overzealous customers or feed readers might hit this feed kind of often, and since the data served up by the feed does not change too a lot we could easily do some caching. Luckily this is easy to do using ASP.Net’s built in caching support. Unfortunately the code to, programmatically, setup the caching is a little ugly. To make things easier to read I added a couple extension methods to create a very poor man’s DSL.

Following the Microsoft code example is ugly

Seeing the code below in my HttpHandler made me cringe. Hopefully it makes you cringe too.

TimeSpan freshness = Timespan.FromHours(1);

DateTime now = DateTime.Now;

context.Response.Cache.SetExpires(now.Add(freshness));

context.Response.Cache.SetMaxAge(freshness);

context.Response.Cache.SetCacheability(HttpCacheability.Server);

context.Response.Cache.SetValidUntilExpires(true);

context.Response.Cache.SetValidUntilExpires(true);

context.Response.Cache.VaryByParams["days"] = true;

The essence of what I want to do is to have ASP.Net cache the content of the feed for a few minutes. It should also cache requests having the “days” parameter the handler accepts. I added that nifty “feature” without blogging about it. Please forgive me.

A Better Way

Let’s wrap the essence of what is being done into a couple of extension methods making the icky code above a bit easier to read and understand.

context.Response.CacheFor(30.Minutes()).OnParameters("days");

Fluenty Goodness

The, hopefully, easier to ready version of the cache setup code is accomplished with extension methods that use the fluent interface “compliant” Method Chaining technique.

public static class WebCachingExtension

    public static HttpResponse CacheFor(this HttpResponse response, TimeSpan timeSpan)

        response.Cache.SetExpires(DateTime.Now.Add(timeSpan));

        response.Cache.SetMaxAge(timeSpan);

        response.Cache.SetCacheability(HttpCacheability.Server);

        response.Cache.SetValidUntilExpires(true);

        return response;

    public static HttpResponse OnParameters(this HttpResponse response, params string[] parameters)

        foreach (string parameter in parameters)

            response.Cache.VaryByParams[parameter] = true;

        return response;

    public static TimeSpan Minutes(this int numberOfMinutes)

        return TimeSpan.FromMinutes(numberOfMinutes);

Ok, so there is also a little integer extension method in there to make a pretty TimeSpan. I use extensions like this all the time where they make sense and improve the readability of the code.

The key here is that the extension method returns the object being extended allowing the method calls to be chained together. The extended object in this case is the HttpResponse. The context being modified by the fluent interface in this case is the Cache object. Say you need some extra behavior? Just add another extension method. One downside to this trick is the lack of readability when you switch around the ordering of the calls.

context.Response.OnParameters(“days”).CacheFor(30.minutes())

Yuck. While, legal and correct it is at all easy to understand. I’ll punt on solving that problem in this blog post but I will say the solution lies in going to the next level and using an Expression Builder and potentially controlling the ordering of the method chain using interfaces.

Oops, I’ve said too much and slipped into more advanced waters that have been visited by smarter people that myself. I will bid you adieu.

Creating RSS Feeds Using ASP.Net

kmiller — Fri, 06 Feb 2009 23:33:55 GMT

Warning: This is one of those late to the party posts where I show off some cool thing that everyone already likely knows about. I just feel special right now because I was able to push out something we’ve wanted to do for quite some time in an afternoon.

We wanted to have an RSS feed so customers could keep up to date with new or recently updated Dovetail Knowledge Base articles. My personal mental gate was that I really didn’t want to try to figure out how to properly create an RSS or Atom feed and I didn’t want to bother with complex heavy handed frameworks like Argotic. Then one day Josh introduced me to the System.ServiceModel.Syndication namespace (introduced in .Net 3.5) and my life was never the same.

Simple stuff really. Create a SyndicatedFeed filled with feed items and hand that to a Feed Formatter.

Generating an RSS feed

public void ProcessRequest(HttpContext context)

    SyndicationFeed feed = CreateRecentSolutionsFeed();

    var output = new StringWriter();

    var writer = new XmlTextWriter(output);

    new Rss20FeedFormatter(feed).WriteTo(writer);

    context.Response.ContentType = "application/rss+xml";

    context.Response.Write(output.ToString());

But I get ahead of myself. We have to serve this feed from somewhere. Not really being an ASP.Net expert Josh got me started with a Generic Handler.

Which seems a thin construct for simply spiting content out of a URL. Perfect.

A Syndication Feed

To build the syndication I created this little helper method.

private static SyndicationFeed CreateRecentSolutionsFeed()

    var syndicationItems = GetRecentOrModifiedSolutionSyndicationItems(TimeSpan.FromDays(30));

    return new SyndicationFeed(syndicationItems)

                   Title = new TextSyndicationContent("Dovetail Software Knowledge Base"),

                   Description = new TextSyndicationContent("Recently created or modified solutions regarding products offered by Dovetail Software."),

                   ImageUrl = new Uri("http://www.dovetailsoftware.com/images/header_logo.gif")

};

Assembling a Syndication Item

I’ll skip the gory Dovetail SDK data access code that goes out to the Clarify database and materializes *poof* any solutions that have been created or modified in the last 30 days. Unless, of course, one of my 2 readers asks nicely for it. The interesting bit in any case is that it basically loops over all the solutions found and assembles a Syndication Item for each one.

private static SyndicationItem AssembleSolutionSyndicationItem(ClarifyDataRow solution)

    var id = solution["id_number"].ToString();

    var title = String.Format("[{0}] {1}", id, solution["title"]);

    var content = solution["description"].ToString();

    var url = new Uri(String.Format("http://www.dovetailsoftware.com/resources/solutions/{0}.aspx", id));

    return new SyndicationItem(title, null, url) { Summary = new TextSyndicationContent(content) };

Results

The end result is a totally hard coded HTTP Handler generating an hopefully useful RSS feed for Dovetail’s customers.

Next up was to add a meta link to the feed in the html header of our knowledgebase pages. This makes the little RSS icon shows in the address bar of honest and decent hard working web browsers.

<link rel="alternate" type="application/rss+xml" href="/resources/knowledgeBaseFed.ashx" title="Dovetail Knowledgebase Feed"/>

Mobile Agent 1.1 - Now with better access to your knowledge

kmiller — Fri, 23 Jan 2009 21:33:59 GMT

Today we are releasing a better Dovetail Mobile Agent. In this release we focused on exposing the knowledge contained in your Dovetail CRM to your mobile agents and public users. For more details please take a look at the documentation but to sum up I’ve composed for you a sort of visual what’s new in this release.

Agents can now…

Search for and use solutions while working cases. Solutions found to apply to a case being worked can easily be sent and linked to the case.

Searching now returns both solution and cases.

Once a solution is found details about it can be easily sent to a case.

Because sending emails from a mobile device can be time consuming, Mobile Agent fills in most details of the email for the agent. The solution is also linked to the case giving support managers feedback about hot support topics.

Agents can easily browse and navigate to all cases linking to a given solution.

Likewise agents viewing a case can now see which solution a case is linked to and optionally unlink the solution from the case.

Public users can now…

New for this release is the ability for public, non-authenticated, users to search for and view public solutions found in the knowledgebase.

The home page now welcomes public users.

Searching for public solutions.

Viewing a public solution.

Avoid Mickey Mouse Email Loops

kmiller — Mon, 12 Jan 2009 17:16:22 GMT

Say you have an application (i.e. bot, script, automaton, service) polling an email account (POP3 or IMAP) for emails. And say that your application sends an automated response back to the sender of the email. Be careful. You can easily get into situations that will create an email loop or worse yet the Sorcerer’s Apprentice Syndrome.

After sending email from an automaton it is quite common for the receiving mail servers to bounce emails back or accounts to temporarily respond with an “Out of Office” response. If your service is not smart enough to ignore automated responses to it’s own emails? Well. The streams will cross and poorly written email services will implode under the weight of your unending email loop. Kittens die. System administrators call you. This is bad.

RFC 3834 comes to the rescue with this sage advice:

An automatic responder MUST NOT blindly send a response for every message received.

Thank you Captain Obvious. Thankfully this RFC sets up a standard way to detect and inform your email service in order to avoid the dreaded email loop.

Being a good email citizen

The RFC provides guidance on informing your senders that you are indeed an automation. The crux of the guidance is that your email response should include the Auto-Submitted email header.

Return-Path: <admin@kevin>
Received: from Kevin ([192.168.0.106])
by KEVIN
with hMailServer ; Mon, 12 Jan 2009 10:21:47 -0600
Auto-Submitted: auto-replied
X-FC-SentBySECNET: Dovetail-Email-Agent-Service-Instance-001
From: Dovetail Software Email Agent <admin@kevin>
To: barney <barney@kevin>
Date: Mon, 12 Jan 2009 16:21:47 GMT
Message-ID: <e75d9c16520b461e8d10a1f63c029639@Kevin>
Subject: RE: send me an automated response? - About case 12

To accomplish this I created a little extension method used when creating emails. Note, the MailMessage type is part of an extinct .Net Email API we are currently saddled with:

   1:  public static readonly string AutoSubmittedHeaderTitle = "Auto-Submitted";

2:

   3:  public static void ShouldBeAutoReplied(this MailMessage message)

   4:  {

   5:    message.Headers.Add(AutoSubmittedHeaderTitle, "auto-replied");

   6:  }

Being a good email consumer

On the flip side your email application should have a plan for what to do when it gets an automated message. For our usage we’ve updated our Email Agent service to detect and ignore auto submitted emails. Again I use an extension method to test each email.

   1:  public static bool IsAutoSubmitted(this MailMessage message)

   2:  {

   3:    return (message.Headers.Contains(AutoSubmittedHeaderTitle) &&

   4:    message.Headers[AutoSubmittedHeaderTitle].Value != "no");

   5:  }