Whether you realize it or not, you need a tool that finds duplicate source code in your applications. In fact, if you’ve never used one before, you probably don’t realize how much you need an automated solution to this problem. It’s nearly impossible to manually locate the types of duplicate code that such a tool can easily bring to the surface. Even if you think you’re intimately aware of an application’s code base, every line of code you write contains the potential to awaken the duplicate code dragon.

To combat the problem, we have Atomiq – what I consider to be the best solution for finding duplicate/similar code in C#, VB.Net, ASPX, Ruby, Python, Java, C, C++, ActionScript, and XAML.

UPDATE 11/9/2010 10:46:26 AM by AD: The promotional code that used to appear on this post has been removed.

How can I make such bold claims? Well, for one, because I know I always write awesome code and yet it’s astonishing how frequently Atomic says, “No, Alex, you do not always write awesome code.”

Oh, and it happens to be the only code similarity finder I could find that was easy to use for my purposes. You might say nothing else duplicates the Atomiq experience! *SNORT*

Atomiq doesn’t really need this much of an introduction. If you haven’t done so already, it’s an easy, safe, and incredibly eye opening experience to run Atomiq against any of your projects. Here’s how to get some immediate gratification:

1. Download Atomiq from http://getatomiq.com (There’s no real need to spend much time on their website if you just want to get some immediate gratification)
2. Get some immediate gratification.

That’s pretty much it as long as we ignore the part where Atomiq points and laughs at all the duplicate code it found!

I was able to quickly gain an extensive understanding of where my duplicate code existed with minimal knowledge of how to use Atomiq. After I finished wiping up the tears, I was able to begin the gratifying process of fixing things.

A personal example

If you’ve read my posts about using delegates to eliminate duplicate code and using IDisposable responsibly, then this example is going to look familiar to you. To be honest, it was Atomiq that led me to the delegate-based design in those posts!

Below you will find two methods that do two different things with an OdbcDataReader. Don’t spend a lot of time trying to figure out what they do.

The first bit of code returns a list of first names from a database.

The second bit of code sends an email notification to a list of email addresses in the same database.

It’s pretty typical code you might find in any application that connects to a database. You can tell the author had good intentions, but it’s not hard to think of a few simple things we could do to make the code a better.

But it’s probably not a good idea to just dive in and start refactoring!!!

If you looked at the code above, you might have noticed that the two methods followed very similar patterns. Though the methods do two different things, they’re not that different from each other.  In other words, there’s a lot of duplicate code. There might be a hundred instances of that pattern in your application! It would be very difficult to find those instances manually.

Before you attempt to refactor, you might want to use a tool like Atomiq to help you find all of the duplicate code patterns in your code. Finding and eliminating those patterns will help you make better refactoring decisions.

For example, if we look at the NotifyPeople method from above in Atomiq, we can see from the red lines that there are two other places that have the same pattern as lines 103-115 and 120-131.

Closer inspection shows that, indeed, one of the places in our code that duplicates that pattern is the GetFirstNames method from above. Again, my two previous blog posts, using delegates to eliminate duplicate code and using IDisposable responsibly, go into detail explaining how I chose to solve this particular problem in one of my projects.

Digging deeper

I’m not going to go into great detail on how to use Atomiq, as its smart developers have already graciously done this on getatomiq.com. It’s not massively complicated software which makes getting to know its full extents a quick exercise. I do suggest watching the introductory videos on the homepage if you’re sitting on the toilet with nothing else to do.

The entire Atomiq user interface isn’t exactly what I would call “typical”, so spending some time on their website will help you get the most out of the tool.  And because of that unique interface, Atomiq has a few features that you might not discover without the aid of the website.

Constructive criticism

This wouldn’t be a proper review if I didn’t throw my opinion in the air and wave it like I just don’t care, now would it?

Here’s my bullet list of “other” notes I took during my review of Atomiq.

• I’m not sure why they don’t include the option to download and run a standard installer for Atomiq, but I don’t necessarily find this to be a problem. I chose to xcopy the Atomiq exe out to c:\Utilities\Atomiq.
• If you don’t configure the analyzer’s settings appropriately, the Atomiq user interface might not show anything useful. When this happens, it’s easy to assume you didn’t do something right.
  
• Minor detail, but I like to look at change logs. At the time of this writing, there isn’t one on the website or included with the Atomiq application.
• The entire user interface isn’t exactly what I would call “standard”. That doesn’t mean it’s a bad/unusable interface, but I’m typically the kind of guy who likes things to look/work the way they do by default. I mean, I never even changed my myspace theme from the default skin for crying out loud! It just catches me off guard when I can’t ALT+F, N to start a new project for example.
• When you create a new project in Atomiq, the first thing you are required to do is pick a directory that contains your source code. The Pick button shows a pretty standard directory picker, but I really wish I could locate the directory I want more quickly by pasting the directory from my clipboard. I’m quite ninja-like when it comes to wrangling a computer, so it’s sometimes easier for me to get the directory I want in the clipboard from another application than it is for me to locate the directory with this user interface. But now that I think about it, I don’t think wrangling is something ninjas do.
  
• It would be great if Atomiq had an MRU list to help me open the files I use often.
• It doesn’t appear that Atomiq is able to find duplicate code patterns that vary only by magic numbers, variable/class names, or syntactic variations. Using a tool like Atomiq most effectively sometimes requires gently massaging your code beforehand. For example, the screenshot below shows that Atomiq doesn’t find any similarities between NotifyPeople above and NotifyPeople2. Yet they are, for all practical purposes, identical.
  
  If everyone on your team uses roughly the same coding and naming guidelines, you might not have to worry about that problem much.Here’s another example that I think Atomiq might be able to shed some light on one day. There’s clearly a very important similarity between lines 166 and 177 below.
  
  Maybe the developers of Atomiq could provide some kind of clue that there’s an opportunity to perform the following “extract method” refactoring:
  

Still a winner

Prior to finding Atomiq, I’d never used a code similarity finder. These days, it’s something I use often and can’t imagine living without! It’s such a simple, useful tool, I don’t know why anyone wouldn’t want to use it on their own projects.

Most of the time, you’ll likely use Atomiq as more of a detective tool. It usually finds little pieces of a pattern that, upon closer inspection, are much bigger, more important patterns.  So even though Atomiq can’t perform miracles, the tremendous satisfaction that comes along with deleting tons of duplicate code from your application is worth many times its $30 price tag.

UPDATED 11/9/2010 10:46:26 AM by AD: The promotional code that used to appear on this post has been removed.

Here’s a simple technique you can use to easily eliminate duplicate code. Try not to let the word “delegates” scare you away from learning the technique and I’ll try to refrain from using the word “delegate” as much as possible.  I’ll start small and then work my way towards a more complex example that really demonstrates its power.

Before I go any further, I’d like to point out that my post about Using IDisposable objects responsibly contains a more extravagant example of how this technique is useful. This post goes further to explain HOW we use the technique.

A simple example

Let’s look at two overly simplistic methods that do different things with a connection object. Whereas the yellow code is identical across the two method, the blue and green code illustrates the differences between the two methods.

There may not be much duplicate code here, but that’s not important right now. When thinking about how we can make this code better, consider these things:

• What if we decide one day to use try/catch statements instead of using statements? Doing so would allow us to catch and process specific exception types.
• What if the code required to create the connection changes one day?
• What if there were dozens of places throughout our application where this type of code is used?!?!?!?!

In order to make any of those changes, you’ve got to touch the code in every single place where the pattern exist. That’s the problem we want to fix. We want to be able to make those change, or any other changes, in one place, and have it affect the entire application.

Looking the blue and green code above, you can see that the only dependency those methods have is in the connection object. If you think about it, the blue and green methods don’t care where the connection object comes from or what happens to it afterwards – they just need a connection object. Our code would be much better if we could prevent from showing how the connection object is created or disposed.

So let’s change our two methods to see how we can better accommodate change in our application.

I can just hear my boss complaining about that strange => syntax, but bare with me. For the most part, this new code looks the same as our original code. Look back at the first code example to verify the differences between the two. As you’ll soon see, they definitely do the same thing, while, at the same time, providing a great deal more flexibility.

Creating the UsingConnection method

Here’s a different example that, whether you realize it now or not, illustrates the concept. Look at it closely, and then look at the CallIndirectly method even closer. If you can grasp this, the rest is relatively easy.

Here’s the same thing with a bit of color coding to try and help illustrate what’s happening:

Hopefully, you’ve determined that this should be the output of such a program:

In the example above, CallIndirectly takes a method (Action) as a parameter and stores that parameter in a variable called methodToCall. Just as we write WriteHelloWorld() to execute WriteHelloWorld, we write methodToCall() when we want to call the method that was passed to CallIndirectly. You can immediate see from the output of the program that using CallIndirectly provides us with an opportunity to do things both “Before” and “After” our desired method gets called. Essentially, CallIndirectily is a method that calls another method. The trick is, we have to pass a method as a parameter to CallIndirectly so it will know which method to call.

Again, if you can grasp that much, the rest is pretty simple.

UsingConnection is pretty much the exact same thing as Callindirectly. If we wanted to, we could use UsingConnection in the exact same way that we used CallIndirectly above! Here’s proof:

And again with the color coding (let me know if it helps – I’m curious).

What’s that Action stuff?

Notice how the method signatures for DoSomething and DoMoreStuff are identical with the exception of their names.

For our UsingConnection method to work, the first thing we need to do is figure out a way to generalize those signatures.  In other words, we need a way of defining a method signature that could have any name (or even no name, as we’ll eventually see), and that takes a single OleDbConnection parameter. We can represent that last statement with Action<OleDbConnection>. You can pretend “Action” represents the name of the method and OleDbConnection represents the first parameter. If our methods took three parameters – An OleDbConnection, OleDbCommand, and an OleDbDataReader, for example, we could represent the method by using Action<OleDbConnection, OleDbCommand, OleDbDataReader>.

I find the easiest way to do this is to imagine placing the words DoSomething and DoMoreStuff with the word Action. What’ you’d then have is:

While that’s close to the desired result, there’s still a bit more we have to do. It turns out, the name of our OleDbConnection parameter isn’t important either, so let’s get rid of that:

Also, the void return type isn’t important to us…

Finally, to really use the Action keyword , we have to replace the ( and ) with < and >.  The end result looks like this:

Because UsingConnection takes a parameter of type Action<OleDbConnection>, that tells us we can pass any method to UsingConnection, so long as it takes a single parameter of type OleDbConnection. In other words, just as void DoSomething(OleDbConnection connection) tells us that we can pass an OleDbConnection object to DoSomething, we can see that void UsingConnection(Action<OleDbConnection> action) tells us that we can pass any action (read: method) to UsingConnection so long as the method we’re passing accepts an OleDbConnection parameter.

What about that pesky lambda syntax?

The Test1 and Test2 methods below do the exact same thing, yet it’s apparent that Test2 uses a syntax that isn’t exactly intuitive at first.

To understand that => syntax, let’s expand on our understanding of Action. As you should recall, the Action<OleDbConnection> parameter tells us that we can pass any method to UsingConnection so long as it accepts an OleDbConnection parameter. Okay, so let’s copy/paste the entire contents of our DoSomething method into our call to UsingConnection and see what happens.

Well, that’s basically what we want to do, but it’s obvious that code isn’t going to compile. To make it compile, we only have to make TWO changes.

Remember, Action<OleDbConnection> means that the name of the method and its return type do not matter. So the first thing we can do is remove the void DoSomething part.

The ONLY other thing we have to do now is separate the parameters from the method body with the two characters =>.

That’s technically it. The code above compiles and works fine, but there are a couple of things we can do to clean the code up a bit. The main thing is that we don’t have to explicitly declare the type of our connection object. Let’s get rid of that declaration and tidy things up.

Look how clean that is!

Just to make sure you understand how to make use of this technique, here’s a more concrete example of using the connection:

Conclusion

Again,  you’ll probably want to look at my last post on Using IDisposable objects responsibly to get a better understanding of why we want to use this technique in the first place.

I’ve been on a mission lately to eliminate duplicate code, so there’s a good chance my next two or three posts will be on the same subject.

How many times have you complained about how much repetitive code is required to properly dispose of objects in your code? How many times have you found yourself in a situation where you’re unsure whether or not you’re even supposed to dispose of an object in the first place?

I want to show you a technique which completely (or, at least, temporarily) avoids the need to bother yourself with those very important concerns. This is an extremely powerful technique which can be used for much more than dealing with IDisposable related issues, but that seems to be the scenario I use it on most often.

What not to do (AKA, what we’re used to doing)

Let’s take some scenarios we’re probably all very familiar with – ADO.NET’s vast array of IDisposable bad boys.

The first example below illustrates a typical usage of OleDbConnection, OleDbCommand, and OleDbDataReader objects.

Things to notice:

• There are ~15 lines of code in this method.
• Only ~5 lines of that code is specific to the act of returning a list of first names from the method.
• The surrounding ADO.NET related code could definitely be made more robust.

In case you haven’t pointed them out on your own yet, I’ve highlighted the important lines of code below:

ere’s a similar example. You can see we’re doing something different, but the same problems exist here that exist in the example above. The thing to notice here is that most of the code in this method, unfortunately, is related to using ADO.NET, and most of the code is exactly the same as the previous example.

Now think about that for a minute. Every single time we go to the database, there are ~10 extra lines of code we’re writing just to ensure that our application works smoothly. How do we know that we’ve written those ~10 lines correctly? And how do we ensure that we’ve written those ~10 lines in the same way throughout our entire application where a connection to the database is involved?

The solution is to completely delegate those responsibilities to something else. In other words, we’re not (yet) going to worry about those things.

How do we fix this?

If you look at the two examples above, you’ll probably be able to detect a common theme. We’re basically just going through the process of creating an OleDbDataReader (the reader object) so we can get values from each row in the table. All of the other code related to the connection, the command, the try/finally, disposing, null checking, etc…. is practically boilerplate code that we don’t want to have to deal with inside our GetFirstNames and NotifyPeople methods. Again, in these examples, it’s all about the OleDbDataReader and that’s very important to keep in mind.

Here’s the trick: What we want is a new method that allows us to instantiate a new OleDbDataReader with whatever information is required, and which will provide us with the OleDbDataReader only at the point where we need it. In this case, we need to pass a SQL query to instantiate the OleDbDataReader, and we want to use the OleDbDataReader every time we successfully call its Read() method.

Let’s look at some examples of what we’re trying to achieve. The examples below do the exact same things as their respective counterparts, yet the important work is immediately visible and easily readable.

Here’s how we can retrieve the first names….

And here’s how we can notify our people…

i

As you can see, pretty much all of the ADO.NET related code has been removed from GetFirstNames2, and NotifyPeople2. Pretty snazzy, eh?

It’s may be a little confusing to you, but the body of the method (highlighted below) actually gets called every time we successfully call the reader.Read() method on the while loop.

But what happened to all the other code, you ask? It’s in that method you see above called ForEachRowInReader. ForEachRowInReader fully handles the responsibility of creating the connection, wrapping everything in a try/catch, calling reader.Read(), disposing of any IDisposable objects, etc….  We’ve moved this complicated, error prone code in ForEachRowInReader so that it only exists in one place within our application.

Here’s what that method looks like:

As you can see, ForEachRowInReader pretty much only contains ADO.NET related code. There’s nothing in this method even remotely related to retrieving names from the database.

The heart and soul of ForEachRowInReader is the Action<OleDbDataReader> parameter:

What is Action<OleDbDataReader>? Well, it’s called a “delegate”, ironically. I know delegates are confusing if you’re not used to working with them, but hopefully this post will inspire you to take a closer look. What we’ve essentially done is wrap a method in another method, and even if you don’t completely understand what’s happening here, surely you will admit it’s pretty awesome.

While I fully intended on going into greater detail in this post, I think its length has reached a point where adding much more might confuse the reader. And since this is my first real post on my site, I think I’m going to save the rest for part two.

8/2/2010 1:14:39 PM Update by AD: To learn how to use this technique, be sure to check out my post on Using Delegates To Eliminate Duplicate Code.

So, happy coding, interwebz.

I’m being forced to hold off a bit on writing anything really meaningful due to all of the other things involved with starting a new website like this. For example:

• I’m not very happy with the registration process that is required for commenting on my posts. I think I want to leave registration as a requirement, but the process of commenting on a post needs to be streamlined.
• I haven’t yet found an OpenId plug-in for WordPress that functions the way I want. OpenId is enabled on the site, but only nerds like myself would have any clue how to use it. What I really want is something like this:
  
• The major search engines still don’t see my new site the way they should. If you search for “Alex Dresko”, I want AlexDresko.com to be the first result. At least Bing shows my site first when you search for “AlexDresko.com”.  I predict this is going to take a while, but there are still a lot of things I can do to make my site the most attractive result when people search for me.
• For crying out loud, I don’t even have an “About Me” page yet! What about my resume and links to my social network profiles?  I bet you don’t even know what my purpose is for this site. And have you seen that default faveicon in the address bar? Gag me with a spoon! Yes, I just wrote that. The list of small stuff like this grows and shrinks daily and I’m going to keep hammering away at it until there’s nothing left to sweat.

So that’s a rough break down of what’s happening here right now.

I. Am. Not. Going. To. Give. Up. On. This.

On occasion, I get to perform such profound tasks as shifting columns to the right so the labels don’t wrap in our Employee Self Service application. Oh, and I think  I get to remove the “12” from the right column of the Box 12 data.

I just kinda figured the world would want to know that.

In 2002, I purchased AlexDresko.com with the sole purpose of creating the most badass website to ever grace this beautiful planet. However, much like a mechanic who never has time to work on his own car, none of the dozen or so drastically different versions of AlexDresko.com I started on over the years ever got a chance to please the public.

But recently, I got tired of not having a voice on the internet. I have all these ideas for things to write about, but I’d always tell myself, “I don’t want to post anything until I can post it on my website, and I can’t post it on my website because my website isn’t ready, and my website isn’t ready because… SQUIRREL!”  Well enough is enough, man. I want  a place to call home and if some PHP *eeeeeek* blog system called WordPress is the way it’s got to happen, then so be it.

I’m going to stop beating myself up about not having everything exactly as I want it and move on. It’s a bitter pill to swallow, but I honestly feel like this is a hugely important step in the right direction. In my mind, there’s still a lot to do here, but something is better than nothing.

Wow. My first, real, suck-it-if-you-don’t-like-it blog post.