His heart was pounding, racing at a pace that surely would make any competent doctor raise an immediate concern for the safety of his patient. Beads of sweat rolled slowly down his forehead, wiped away by a hurried swipe of a forearm, only to be replaced by fresh ones. He was hunched over his keyboard, fingers flicking over its surface in a frantic blur of activity. His eyes gleamed with fury; two dark spots whose glare was entirely focused on the screen before him.

It was anything but an elegant fight that they waged, the operator and his silicon-based opponent. They darted back and forth, each one probing the other for any sign of weakness. Even as one seemed to gain the upper hand, the other would counter with a move both swift and unexpected. It was a war of attrition waged by two stubborn parties: one driven by his need for dominance, the other because it did not know how to do anything but respond in kind.

Words and numbers trailed endlessly across the screen, a seemingly meaningless stream of nonsense, uttered by a mind bereft of all reason. Yet to the operator it all made perfect sense; perfect nonsense, that is. As his nemesis endlessly spewed chunks of data, his eyes sprinted back and forth, trying as best he could to collect and analyze it all. But try as he might, it seemed always that his enemy held the upper hand. Just as he gleaned a tiny bit of understanding, some small nugget of progress, a new wrinkle would appear, more confounding than the last.

As his frustration grew he felt his temples begin to throb; gently at first, but increasingly as time passed. It was as if his body was trying to tell him “Back off, you’re in dangerous territory.” But he was past all reason, so completely consumed he was with his task at hand. On and on he pushed, till the throbbing became outright agony. He flung open the drawer to his right, digging feverishly through its contents until a small bottle was found. Pawing it open, he quickly extracted two small white pills and swallowed them hurriedly, without water. They offered little hope of relief, but perhaps they would give him the edge he so desperately desired, no, needed.

So, I’ll give you three guesses around what the above narrative is about. A piece from my first attempt at an epic sci-fi perhaps (yes, that is very much on my someday list)?

I’ll give you a hint: been following the saga of Urban Meyer, the embattled yet highly acclaimed football coach of the Florida Gators? If you have, you know that he’s basically been told by his doctors that the very thing that makes him one of the most successful coaches in history may also cause him to prematurely leave this Earth: his singular focus and intensity.

When I first read this story, my immediate thought was “Wow, that could be describing me to a T, minus the arachnoid cyst pressuring his skull and all.” I am probably one of the more driven individuals you’ll ever meet; when my mind is set on solving some problem, it’s a nearly useless task to try and pull me away. Just ask my wife, she’ll be happy to oblige you with all sorts of evidence supporting that statement.

On the one hand, this trait is an asset: it lets me persevere when others would stop, long discouraged by setbacks and seemingly insurmountable challenges. It’s probably the major reason why I’ve been so successful in a field that, up until around three years ago, I knew nothing about. But there is a dark side to this unending focus. It manifests as the (not so at times) occasional splitting headache, the missed personal responsibility, the midnight awakenings when nothing short of a blow to the skull will seemingly quiet my mind.

Faced with the prospect of enduring this escalating chain of physical and emotional malaise, I’ve made the decision, much as Meyer did, to step back. No, I’m not taking a leave of absence, but I am engaging in a concerted effort to become more aware of my emotions and the physical manifestations thereof. Journaling, reminder alarms to “check in” with my state of mind, even asking others to notice when my symptoms begin to show themselves.

I love my work, but I love my family and my health above all else.

By default this is set to zero, which tells SSIS to attempt to contain the entire bulk insert within one transaction. In my case, that meant over ten million rows! Needless to say, my 2GB transaction log filled up in no time. By changing this value to a more manageable 1,000 rows, the package completed without issue.

Note that in some cases containing your operations within a transaction is a good thing. In this case, it was not necessary, as the package is performing a simple bulk transfer to a reporting database, no atomicity is not a concern.

In SQL Server, it is rather handy to retrieve data from a different SQL Server instance and use the result locally in another SQL statement for further processing. In theory and in the set purists’ fantasy land, it shouldn’t matter where you get your data or even how you get the data as long as you can use them to further compose a set-based solution.

That is all fine. After all, in a distributed environment the data you want may not be on the same instance where the processing takes place. Well, that is fine until it comes to performance and troubleshooting. If you use linked servers indiscriminately, sooner or later you’ll run into these issues, and they are not pretty.

We have used linked server functionality at various points at work. As Linchi points out, it can be quite useful when you have data distributed across multiple systems. But I have seen cases where it’s caused issues, such as excess network traffic, authentication problems (anyone actually been able to get Kerberos auth to work for a two-hop Windows auth’d linked server?), and general performance malaise.

So what are some alternatives?

• Replication – true, it may not solve the network issue, and depending upon the type used, it might cause a delay in data transfer. But it would allow for queries to execute against a single server.
• Static data transfer using SSIS – I would say this would be fine for offline use of data such as reporting. There have been very few cases in my experience where true real-time data has been a hard business requirement, though at times the point has been, shall we say, vigorously argued.

In part I of this two-part series we talked about the first form of what I call “Racing Brain Syndrome”, the “Stress Induced” variety. To briefly recite the main points:

• It is usually caused by underlying anxiety about some aspect of your life, be it work, home life, or just plain keeping stuff in your head about things you’ve committed to doing.
• This “stuff” can be either real or imagined (i.e. you’re going to get fired for writing a personal e-mail at work, once)
• It is best remedied by either:
  • writing things down and clarifying all the details of what you need to do to take care of whatever is bothering you (best for dealing with the former “real” source of anxiety)
  • practicing a little psychological technique called cognitive behavior therapy, which teaches you to examine and refute irrational thoughts using logical techniques.

Now we’ll talk about the second form of RBS, called “Excitement Based”.

It’s, well, due to excitement (duh)

Yeah, that does seem pretty obvious, doesn’t it?

In some ways this form of RBS can almost be seen as a positive thing, in that it means you have some good positive feelings about your life. Think about it: if you were bored and depressed, would you be waking up at three in the morning with some fascinating new take on that problem at work? At the time you may find yourself more annoyed by your brain’s utter refusal to quiet down, but if properly harnessed, these midnight revelations can actually greatly add to your productivity and creativity.

The trick to properly capturing these sudden bursts is to do just that: capture them

“Huh?”, you may say. Well, think about it this way: the very reason why your mind is refusing to let you drift off is because it’s saying “Hey, I have this great idea, but if you go back to sleep, you’ll forget me and I’ll be lost forever!” In a way it’s almost a built in self defense mechanism, designed to keep your late night brainstorms from slipping through the cracks.

To convince your brain not to concern itself with retaining this newfound wisdom, I suggest you follow the same basic tenet of GTD we used to combat the stress-induced variety of RBS: get it out of your head, now. Write it in the notebook you keep by your bedside table (you do keep one there, right?), send yourself a quick e-mail on the CrackBerry (yes, I’ve been known to do that in the wee hours of the morning), even scribble it on a spare Kleenex or other handy medium if you must.

You might think this behavior will just exacerbate the issue, since it will likely make you more fully awake. While that’s true, I would suggest that until the thoughts rolling around in your brain are properly captured and accounted for, you’ll likely find yourself tossing and turning restlessly anyway. This way, you’ll at least get things recorded so that you’re not battling that inner voice, nagging you about unfinished business. Trust me, that’s a battle you’ll come out on the losing end of every time.

There are two main forms of this nasty little bug, which we’ll call “Stress Induced” and “Excitement Based”. In this post, we’ll look at the first variety in more detail.

Stress Induced

As the name implies, this version is caused by an excess of built up stress that has yet to be dealt with. Common symptoms (not inclusive of the other variant of this syndrome) include racing pulse, pounding heartbeat, cold sweats, and possibly (in extreme cases) delusions of persecution or general paranoia.

Now stress, as you well know, can come from many sources, including the practice of keeping things in your head, nagging concerns over projects left un-planned, fear of upcoming regulatory audits (a favorite of us IT folks), and of course the ever present conflict between the Ego and the Id caused by an underlying need for affection, complicated by an Oedipus complex.

Whoops, I must apologize for that last one. This post has me reverting to my old psychobabble style of writing. Ignore that one, will you please?

When dealing with this variety of RBS, one’s best course of action is tri-fold:

1. Determine if the cause of the stress is a rational one. That is, are you feeling stressed because you’ve fallen off your good practice of keeping things out of your head, or are you suddenly having a sinking feeling that you’ve left your torrent bot up and running at work, and the folks from InfoSec are, at this very moment, hot on your trail? Ok, that’s an extreme example, but you get my drift.
2. If the former, your best bet is to take a few minutes and put some thoughts down on paper around what is bothering you. You don’t have to answer every question out there; just make sure every question is written down so you are confident it won’t get lost in the shuffle.
3. If the latter, you would be advised to fall back on a technique I used to teach to the children at the mental health clinic I worked at out of college, called (in it’s most complex form) Cognitive-Behavioral Therapy. The basic idea is that you write down the thoughts that are causing you discomfort, such as “I’m afraid the ninjas from security are after me”, then examine them in a critical, analytical fashion. For instance, what evidence is there that you’re really about to be attacked in your sleep? Do you even have a torrent bot on your work computer? These techniques are usually used by patients with more severe mental health issues such as depression or anxiety disorder, but they serve RBS sufferers equally well.

On a sidenote, if you’re actually interested in a more clinical view of CBT, I’d heartily recommend the book The Feeling Good Handbook by a fellow named David Burns.

Next time we’ll examine the milder, and perhaps more pleasant variety of RBS, “Excitement Based”.

One of the really nice features of RTM is its flexible search system. For example, let’s say I had about 20 minutes to spare, and wanted to see if there was any low hanging fruit available. I simply type in “timeEstime:’<20min’ and voila, a list of tasks that I could easily get out of the way. I’ve found it to be quite a useful trick when faced with 15 – 30 minutes of useful time on my hands. It doesn’t happen that often, but I’ve found that it definately has decreased the number of simple one-off tasks malingering on my lists.

So what do you think? Are time estimates worth their effort, and helpful in planning your daily work? Or do you find them inaccurate and a waste of time, especially for small, low-level tasks?

When reviewing the build for a new SSIS / SSAS deployment, I noticed that because of the way their build was structured, a failure could occur in some critical underlying processes, while on the surface, everything appears normal. <geekery> This was due to 1) a bug in how SQL Agent jobs work in SQL 2005 (see here), and 2) the way their process was calling the SQL Agent job to process an Analysis Services cube. </geekery>

Now, immediately in my mind, this was a clear showstopper. I mean, if I can’t tell for sure if an automated process failed or not, how on Earth am I supposed to support it? I luughed at suggestions such as manually checking on a daily basis or just telling business “it’s broken, but if you don’t care we won’t fix it”. Ultimately, the developer’s final comeback was basically that “I followed conventions of how this was supposed to be set up, so you can’t require me to make this change before we go live.”

Huh? Since when does “it’s broken in other places, so I can leave it broken here” work as a reason not to fix something, especially before it’s actually running for real? How about showing the business we’re looking out for their best interests and ensuring a quality client experience?

Anyway, I thought I would post my thoughts on two new skills I’ve been working on: Analysis Services design and Data Warehousing. If trends at work are indicative of anything elsewhere, BI and data analysis are on a lot of people’s lists. Being a total newbie, I think I’m coming in with a bit of a fresh perspective.

This time I’ll be talking about Data Warehousing. In no particular order, here are my thoughts:

• Doing DW well is really tough. You have to have both a really good picture at a 30,000 foot level of where all your data is coming from, as well as a runway level view to understand the structure and format of individual fields.
• Depending on your sources, you may spend a good amount of effort just cleaning up invalid data. In my case, I was aggregating metadata from several document management systems, and when forced into a system with real referential integrity, I was amazed as how many holes there were. Assume your data has gaps and design ways to handle them, whether it’s ignoring them or adding them to some “unknown” category.
• If you’re aggregating data from different instances of the same software, think from the start about how your going to do so. For instance, are you going to want to separate the data for analysis, or will you want to see it all at once (or both)? Will you load it all in one shot, or separately?
• This is blurring the lines a bit between this post and a future one (Analysis Services), but determining how granular you want your data to be in terms of time aggregation should be an early task. I had to basically redesign my extract process about halfway through after figuring out one of my data points would be better seen at a minute by minute basis.
• Once you make that decision, put specific steps in your ETL process to ensure your data fits within those boundaries. You can’t assume because you see the first thousand rows in neat 30 minute intervals that all the data meets that standard. Again, spoken from late game experience.

Overall I’ve gained a great deal of respect for the folks who do good work in data warehousing, and lost respect for those that try and do it with a shoehorn mentality.

• Thanks to the generousity of a local gentlemen I know from Twitter I acquired one of these devices for free. Don’t ever say Twitter can’t get you stuff!
• It’s low power and unobtrusive. While I love my big ol’ quad core monster, I didn’t really want to take up space at my parents house.
• With DD-WRT, it functions great as an OpenVPN client.

Now, the setup was to look something like this:

(Please excuse the rather awful drawing, I’m afraid I’m used to Visio)

Once I had everything setup, everything appeared to work, except I couldn’t ping anything at my parent’s end of the tunnel from my home network (vice versa worked fine). Well, normally I might try and sniff the traffic at the two endpoints, but this was a little more complicated than you might think, since the router at my parents doesn’t support such an operation.

In then end, I was able to collect some data after hacking a solution (<geekery>I used an already compiled version of tcpdump and manually moved it to the router</geekry>), and figured out that for some reason the VPN router at my paren’t house was performing what’s known as “masquerading”, or “NATing”. What this means is that any traffic that passed out of it was translated such as to appear that it came directly from the router itself, and not some machine behind it. This novel concept is actually the basis for how pretty much every home router functions; but in this case it was bad, in that it made the tunnel between our networks essentially one way.

So what did I do? Well get royally annoyed of course, proceed to spend the next several hours pouring over the setup of both ends, trying various hacks, googling like crazy, and perhaps questioning my intellect (or that of the programmers of OpenVPN, DD-WRT, and Linux in general) a few times. Finally after giving up for a day, I decided I’d look at a tutorial on the OpenVPN site itself I remembered. It took awhile to find, but there was the answer, right in front of me.

Apparently OpenVPN needs a little special tweaking to allow for traffic from both the “client” (my parent’s house) and “server” (my house) networks to fully talk to each other:

<geekery>

Next, we will deal with the necessary configuration changes on the server side. If the server configuration file does not currently reference a client configuration directory, add one now:

client-config-dir ccd

In the above directive, ccd should be the name of a directory which has been pre-created in the default directory where the OpenVPN server daemon runs. On Linux this tends to be /etc/openvpn and on Windows it is usually \Program Files\OpenVPN\config. When a new client connects to the OpenVPN server, the daemon will check this directory for a file which matches the common name of the connecting client. If a matching file is found, it will be read and processed for additional configuration file directives to be applied to the named client.

The next step is to create a file called client2 in the ccd directory. This file should contain the line:

iroute 192.168.4.0 255.255.255.0

This will tell the OpenVPN server that the 192.168.4.0/24 subnet should be routed to client2.

Next, add the following line to the main server config file (not the ccd/client2 file):

route 192.168.4.0 255.255.255.0

Why the redundant route and iroute statements, you might ask? The reason is that route controls the routing from the kernel to the OpenVPN server (via the TUN interface) while iroute controls the routing from the OpenVPN server to the remote clients. Both are necessary.

Yep, you read correctly. “Both are necessary.” Long story short, I only had one of the two necessary configurations in place. Basically, while the normal

route 192.168.a.b 255.255.255.0

command got the packets as far as the OpenVPN tunnel, the software required that extra

iroute 192.168.a.b 255.255.255.0

configuration line to correctly send the traffic on its merry way from there. Without it, the traffic would just appear to go out the proper interface, but actually go to that awful place where packets disappear and are never seen again.

</geekery>

Hi everyone, my name is Josh, and yes, I admit it, I should have read the… oh hell you know what the rest means.

Ever since the birth of my son Taylor, things at home have been, well, busy to say the least. It’s been a struggle just to keep up with the day to day work, such as keeping the house clean, the laundry done, and the trash empty. With such little time and energy left over for personal projects, strict adherence to priorities and ruthless cutting of scope has been the rule of thumb. So much for things like learning Perl (though I’m sneaking this in at the gym on the treadmill), building a new Snort server, or even non-geeky work like painting the exterior windows on the house.

Work, while slightly less crazy, has been quite a whirlwind as well. After some purging of wartime troop levels, the remaining force has been tasked with a “lights on” mantra. That’s all fine and good, except you’d be ludicrous to call what we do “keeping the lights on”. Software still needs to be updated, security maintained, systems administered. Yes, some of the excess fluff has been removed: no more long troubleshooting of user issues (is it replicated on a clean system? If not, guess what, you get to re-image your computer) or extra out-of-scope work. But still, there is no shortage of necessary tasks to be had. Combine that with taking on a new product and expanding my role to include some levels of data-guru, and you’ve got a packed agenda.

On the one hand, it is more than a little frustrating to see the mounting list of “not yets” and “maybe somedays”. But at least I can be secure that everything is safely tucked away, waiting for the day when changing diapers isn’t an hourly occurence, or treading water at work less the norm.