This week I’m going to share a few observations from the “almost one year” point of my life as an independent SQL Server Consultant. I figured I should come clean in this first post this week and tell you all that I’m cheating but I figured I’d let you copy off of my paper while I’m at it.

Unemployment Is High – But Not In All Fields

The first cheat? For good resources with skills (heck, even for me) – there doesn’t seem to be a shortage of database related jobs. Even the ones that aren’t so glorious, that don’t pay so well and have long drives and other annoyances associated. I can’t speak for where you are but I bet you’ll find a similar situation in your job market – It seems hard to find good people. I’ve helped clients interview and phone screen through a LOT of folks to find quality people (even ignoring most of my “6 reasons I won’t hire you” it is tough to find lately). Jumping off of a perfectly good cruise ship to build a boat of your own sounds risky, but not that bad when you are moored 125 yards from a boat factory.

What is “Permanent” ???

Not only am I a cheat – I’m a liar…  I just said going off on your own means jumping off of a perfectly good cruise ship. Well that cruise ship is only as good as long as it stays good. What I mean is your full time job is only full time as long as it makes sense for your employer to spend their budget on salary and benefits for you. They may be the nicest employer in the world but if it doesn’t make business sense, you are being kicked off of the cruise ship –  no matter where it is. Sure you’ll hopefully get a life raft (severance pay) of some size, but you are still at risk of getting kicked off. To steal a line from the Princess Bride – “Job Security” you keep using that word but I don’t think it means what you think it means…

I Know I Can’t

With a self defeating attitude, who needs enemies? It is tough being independent. I’ll talk about that later this week. There are things you have to take care of that are different. There are non-billable tasks. There are taxes to pay. There are contracts to sign. It isn’t really that scary though. Let me put it this way – it isn’t as scary as you’ve convinced yourself it is. Like I said in that pool fear analogy post linked above – don’t be paralyzed by “I think I can’t” – that’s one prediction that always comes true, because it is self defeating. If I can? You can. I KNOW you can.

Have a Plan

If that cruise ship was in the middle of the pacific – and you still jumped – I’d call you a complete moron (in love, of course…) Just because you can and you have a lifeline (the ability to get even a 3-6 month contract position with long hours, a long drive in an industry you don’t like – but pays fine), it would be really irresponsible to leave without a plan. Later this week I’ll share some ideas about how you can slingshot, parachute or partner your way into independent consulting. I chose the parachute approach with a little slingshot added. It worked. There are other options, I’m sure, but the point is – work out a plan, create some savings and test out your relationships to see if work can find you first. I can’t think of a bolded sentence to end this paragraph with, just make a plan!

What’s the worst that can happen?

I’m a man of faith – this paragraph is about that – quick warning… I’m a Christian. I believe the Bible is the inspired and inerrant Word of God. I believe that God is a loving (among other attributes perfectly balanced) God. I believe I am Heaven bound when my time in this messed up body is done (I’ve blogged about this a lot already. The really long version of my testimony is here if you care to read.. In short – A loser w/o a high school diploma, living for himself avoiding God and “His followers” at all costs met the real Jesus and my life hasn’t looked the same since, in spite of my attitude at the time). That gave me a peace about this decision and was probably my biggest cheat of them all. What’s the worst that can happen to me? Even if I somehow managed to really screw this up and could never get a job doing anything ever again, my pitiful savings dried up and my church and family abandoned me and no safety net existed whatsoever – the worst that would happen probably involves death. Maybe even a slow or painful one. So? My God has a plan for eternity, I’m included in it and my eternity (I’ve seen some horribly slow queries in my life, but none were that long) is secure. I’m going to live forever in the presence of God. The one who fashioned this world, the one who created me, the one who holds the keys of life and death – my friend, my father. How risky is it to go off on your “own” when you’ve got an adoptive father who runs the show and already promised an ending that never really ends and is filled with goodness?

The Point

Coming Soon!

So the rest of this week the posts will look like

• Things I’ve learned (and wish I knew) – Sick days are different, taxes are a bit different, etc.
• Relationships – We all knew they mattered, but they make a world of difference in business
• Slingshots, Parachutes and Partners – a few ways to go it alone without being so alone
• Benefits and Drawbacks (for me, for clients, for you)

When this post goes live it will be Easter Sunday – resurrection sunday. The day that Christians throughout the world celebrate the literal and actual resurrection of Christ Jesus. As I’ve done for the past 3 Easters (doesn’t seem like I’ve been blogging for 4 years) I wanted to share a post on some aspect of Easter. First I posted about the meaning of the word “Tetelestai” that Christ shouted from the cross. Then I posted about what was good about Good Friday. Last year I posted about living in the shadow of an empty tomb.

This year, I want to visit Good Friday again and talk about the torn veil in the temple. You see, like with most things in the life, death and resurrection of Christ there is specific Old Testament explanations that go beyond just reading the words in the New Testament here. In the past I have talked of the proofs and historical evidences in favor of the resurrection of Christ as being great reasons to believe Christianity. The harmony of the Bible is another reason. There is no doubt from a scholarly perspective (from believing non-believing/even hostile scholars alike) that the various books of the Bible were written at different times by different authors in different professions (tax collectors, fishermen, Kings, prophets, farmers, shepherds, a doctor, a pharisee who persecuted the Christians originally). Yet… This collection of 66 separate books written over a stretch of 1500 years – is in harmony.  Let’s talk about example – the significance behind the tearing of the temple veil in light of Christ’s mission and purpose of his life, death and resurrection.

What was significant about this temple veil?

The veil of the temple was no ordinary curtain. It separated the “Holy Place” from the “Most Holy Place”. You see in the temple and the tabernacle that came before it, the Priests would receive various offerings described in the Old Testament. The Holy Place was where most of this work was done. It was where most of the offerings were taken. It was a holy place and there were examples in the Old Testament of people not approaching it with the appropriate honor and spirit of Worship and their end was death. The Holy Place was still a set apart place, but the Most Holy place (the Holy of Holies as it is also referred to)??? That was set apart further. There was a large 60′ high veil separating the Holy Place from the Most Holy Place. It was inside this Most Holy Place that only the High Priest could enter, and then only but once a year for the national day of atonement (Yom Kippur) to make a sacrifice for the sins of the nation. The High Priest himself had to make a sacrifice for himself and prepare himself before he went in there.. For in the Holy of Holies the physical presence of God dwelled. We are sinful, we are not able to enter the presence of God on our own. This veil represented that – on one side was God, on the other side was us in our sins, unable to directly approach a perfect and Holy God on our own. The veil was significant because it was a great sign of the relationship between us and God in Old Testament times… There were believers in those times, many people looked forward to the redeeming work of Christ and trusted that God would do what He said and put their faith in that work, many examples throughout the Old Testament, the Holy Spirit did fill people and work in people He chose to work in but it was not a permanent dwelling.

The Significance of the Tearing

Christ came to earth as a love offering of God. Jesus Christ was God wrapped in human flesh coming to pay the full satisfaction of our sin debt. Every wrong we’ve committed. Every small or large sin separates us from God and we are behind that veil of sin. On Good Friday some 2,000 years ago Jesus Christ – the judge of the world – stepped down from His heavenly bench, took off His heavenly judge’s robe and allowed Himself to be nailed to a cross to pay the price of our sins. When He did this He gave us a new way of access to God, if we accept his gift. The tearing of the veil brings a few lessons to mind…

It was torn from top to bottom – This was an act of God – not of man. God made the step of sending Christ for us. God opens our hearts to allow us to accept (or reject if we so choose) Christ. When Christ’s redeeming work was complete on the cross, the veil in the temple was no longer necessary. We didn’t need to go through a High Priest and religious ritual to get to God. Not because we changed, not because we were any better.. But because God Himself bore the weight of our sin and opened that path in the person of Jesus Christ.

Significance for believers today – We should look to the veil that was once separating the temple as a reminder of the separation our sin creates from God. We should remember how we were when we were dead in our sins and what life is like now. It is also important to remember that the veil was torn from top to bottom..  God didn’t just cause the veil to be torn, God also paid for our sins. Our salvation is nothing we could earn (even if we wanted to!), we didn’t make the first move – God did. This should help us to remember to love others with the love we are called to, it should help us to stay grounded – we aren’t any better than anyone – we are sinners who happen to have accepted the grace of God.

We should remember what the veil signified also – separation. Even after believing, our sin can separate us – in a sense – from God. No, we aren’t cast away, we aren’t out of a relationship but our fellowship can be broken. God wants what is best for us. He wants us walking as Christ walked. He wants us intentionally choosing to live lives of Worship and let our direction be guided. When we don’t do that. When we let our flesh get in the way – that relationship is busted… We are still His, we are still Heaven bound saints but our relationship isn’t what it could or should be. Our fruit won’t be what it could be, our lives aren’t the examples of Christ likeness they could be. We’ve put a veil up and we need to let God tear it down by getting back to Him. We need to be in prayer, in the Word, in fellowship with other believers and seek daily to die to ourselves and live for Christ (A verse from a Bible Study I’m doing with some brothers from the SQL community comes out here – Luke 9:23 – “If anyone wants to come after Me, he must deny himself, and take up his cross daily and follow Me.” If we aren’t willing to do those things, that second veil won’t be lifted and our lives aren’t going to be what God intends for them to be.

Significance for non-believers – Quite simply, there is still a veil between you and God. If you have never trusted Christ but are leaning on your own goodness, leaning on a set of rules you try to follow, leaning on letting your good outweigh your bad, or a set of works – the veil to a relationship with God is still there. The Bible tells us that no one is good enough to cross that barrier for we have all sinned (I do daily, Billy Graham does daily – even your grandmother did/does). So there is a veil between you and God. On one side stands God – perfect holiness, absolute justice but also perfect love and mercy. On the other side is you – an imperfect person who has committed sins as small as a white lie – but just as much a sin to a holy and just God as any other. That veil is there because we can’t enter into a Holy God’s presence like this. He can’t accept us like that and still be just, Holy, Righteous and perfect Truth. The veil is thick and real. When Christ died and that physical temple curtain was rent in two, he provided a gap in this veil of your heart. That’s the heart of Christianity. That’s the theme of the Bible – God is perfect, we ‘aint – yet He provided a way that only He could provide. You can let Christ show you the gap through that veil, you can let Christ tear this veil in your heart from top to bottom. That’s the message of resurrection Sunday – that Christ was victorious over death – and because He lives – we can too.

That’s it…

I hope this Easter season we can reflect on the physical veil and contemplate the spiritual veils in our hearts. I can tell you that I’ve built and accumulated a lot of threads that have built a veil in my heart. I am God’s but I can’t say that I’ve lived Luke 9:23 every day or even many days. How foolish of me. I’m an adopted son of the living God’s and yet I try to lean on me, try to go it alone instead of in complete reliance and utter dependence on the one who wrote the book of life and already wrote it’s ending. Let’s work on our diligence in chasing after God and let Him finally remove the remnants of any veil over our hearts. Drop me a note if I can pray with you for anything.

God Bless and Happy Easter. He is risen!

[Insert Movie/TV-Show Sound Effect to Signla Time Travel Here]

It’s 2012. I have a literal bald spot now that I can touch. I have two more kids, I’m 6 months away from my 10th wedding anniversary. I’m sitting in the airport waiting to board a flight home from a business meeting with my new business partners. I’ve been working on high end and fun SQL Server challenges for all sorts of customers as an independent consultant. I’ve gone into business with one of the very people I was having dinner with that night – one of the original founders of that company, in fact – Brian Moran. I’m going into business with two guys who are SQL Server MVPs and incredibly well respected members of the SQL Server community. I’ve actually taken flights just to go speak at events – in public – in front of people – with Microphones! I’ve spoken for two years at PASS – on those raised platform stages. I blog, I’ve had a couple people even say “Oh! So you’re Mike Walsh – I’ve read your blog!”

So… What the heck happened? How did I go from a high school dropout whose life consisted of partying to a technologist? How did I go from a tech support rep who couldn’t understand what the folks in training mean when they said something about an application having a database (“What?! An application is like an .exe file… Why would it need a database? What do you mean application ‘information’ is stored in the database?!?!”) to someone who enjoys teaching other people how SQL Server works? From a guy too afraid to speak up in a meeting to someone eager and pumped up to deliver a presentation to a room of strangers?

Well – I’ve blogged about that a bit for various #memes over the years (on my testimony, on mentors, on perspective) but I wanted to use that long introduction to setup three big parts of where I am. I don’t share any of this in a spirit of bragging (I firmly believe I am where I am because of God’s influence in my life and the three things I mention here. I like to say I am where I am in spite of me). Instead I share it to inspire. No… I’m not so proud to say “even you can be me!” (I wouldn’t want you to be me) but I mean you can do more than you think you can. You can overcome more than you think you can overcome. I am not smarter than you – I guarantee you that. In fact I could rattle of at least 40 names without applying any effort of SQL people who can tune better than I can tune, who carry more knowledge than I carry and can talk internals better than I can – and that’s without any effort, I am sure there are hundreds I could name! I didn’t have a better upbringing than you – I was on free lunch my whole life, had a hard working single mom, never went to college and didn’t even get my high school diploma until I was 20 something. I make up for these shortcomings by trying to work a little bit harder, I’ve focused more on knowing where to find the right answers and how to test and use searches to help me. I listen (my wife may disagree a bit here…) and work well with clients. I want to solve their problems with them. All of these things combine with an intense passion for SQL Server and its community to put me where I am.

Those three things that I wanted to thank?

Inside SQL Server 2000

Sounds silly, but I owe a huge part of where I am to Kalen Delaney and her work on this book. I started with SQL Server 6.5 and 7.0. but I really got going with SQL Server 2000. This hardcover book lived with me. It went everywhere I went. Right.. Everywhere. I brought it on the train ride to DC when my wife and I were going on a first trip together. I consulted it at work and read and re-read the chapters. The way Kalen treated the internals made it an exciting book to read. I wanted to know how SQL worked, and what the building blocks of the building blocks were. This book was an amazing thing to read for someone who wanted to learn it only cemented my desire to learn more. It only made me eager to read more articles and try more experiments.

Andy Kelly

I’ve mentioned him before in posts here. How blessed was I to have my very first DBA job manager be Andy Kelly?! He was in one of the earlier batches of SQL Server MVPs, he had a passion for internals in SQL and was working closely with the product team of SQL in a lot of ways already. I’ll never forget the first couple weeks working with him – half of my day was sitting in his office with him whiteboarding everything about SQL Server. He didn’t have to do this! There were experienced people around. He could have just gave me tasks, showed me how and been done. Instead he chose to mentor me . He chose to invest his time in teaching me. He didn’t just “teach” me, but he got me excited about SQL Server. He talked about all the new things in this SQL Server 2000 that was just coming out. He talked about why it rocked, how it worked and went as deep as I wanted with the knowledge. I learned a ton from him – but even more importantly, I caught his passion for technology. That’s what I want Linchpin People to be – I want the Andy Kellys of the world, even the folks who are where I am – even the folks who’ve been doing this for only 3 years – I want them to be able to invest their energy into mentoring others and see others grow both in knowledge and passion for their career.

SQL Server Community

I’ve been a broken record and cheerleader for the SQL community for a few years now but looking to list all the folks who inspired me to blog and taught me through their blogs would turn this already long post into a lot longer. The people like Paul Randal and his early days of blogging over at the Storage Engine blog. The people like Brent Ozar encouraging me to try it out and syndicate. The people like Adam Machanic and his SQLBlog site. These people further inspired me to be passionate about technology. Posts like Linchi’s showed me how to test things myself and quantify information before making a stand. The folks at PASS – especially now with the initiatives that board members like Tom LaRock thought up with the PASS first timers welcomes (wish they had that in 2006!) The PASS HQ folks who encouraged me to help organize the Birds of a Feather lunch. The people like Andy Leonard who encouraged me to run a SQL chapter. I could keep going but, as I’ve said over and over and over again, we have a community of givers in the SQL Server world. We have a community of people who throw themselves into other people. That has inspired me to a desire to do the same and I’ve learned in the process through receiving what knowledge they freely share.

The SQL community inspired me to give. I started blogging a few years ago, I started answering online questions 5 or so years ago. I didn’t know it all then (in fact I probably thought I knew more then than I do now… The more I’ve grown, the more I realize how little I know relative to what is out there to know – if that makes sense – or do I sound like Yogi Berra?) I couldn’t answer every question, I didn’t understand every nuance of every topic I intended to blog on. You know what happened, though? I started learning more. I started increasing my knowledge – I grew. So I’ve grown by taking from the SQL community and I’ve grown by giving to the SQL Community. Interestingly enough that is also one of the models that Linchpin is looking to build out in our philosophy, or culture. Like I said in the announcement post – we all grow, or no one grows… And there will be times where we are on the giving and receiving end of mentoring and teaching  – but there is learning no matter which side you are on. I’m a great example of that – 12 years ago I didn’t know how to say SQL Server (well to be fair, some say I still say it wrong, but you knew what I meant…)

You add these three things together and it’s tough to not be inspired to strive to be better, to want to know more and to want to apply it. I am no different or better than you – I guarantee it. You’ll grow when you get involved. Whether you take advantage of the learning at SQL Saturdays, SQLServerCentral, the bloggers, the 24 hours of PASS, the conferences or you take advantage of learning through investing yourself in helping others grow (Or BOTH!!!) – you will learn. You will grow. It may not feel like it right now but you will.

What Is Linchpin People?

Friends and SQL Server legends Andy Leonard and Brian Moran have been working hard starting a SQL Server professional services company with a twist. Around the time I was contemplating going off on my own I had a few conversations with them about what they were starting. I told them it sounded interesting and we talked about maybe someday somehow working together. Well that day has arrived, we’ve been  talking, praying and chatting about the decision for since August – carefully making the decision. Along with Robert Pearl, I’m one of the first two partners in Linchpin People.

The chief concept of Linchpin People is great – It’s a philosophy that actually finds it birth in a series of Bible verses (Matthew 22:37-40) – that portray the concept of our relationship with the Lord but also our relationship with each other. That we are to love our neighbors as we love ourselves. We call this principle applied at Linchpin – SERVICE 22:40… This was one of the motivators of me going into business on my own to begin with so this resonates with me. You can look at it a few ways but the ways I think this makes this professional services organization different:

• We realize there is more to our careers than billing high rates for as many hours as possible – which has many side effects, but two I like:
• Everyone grows or no one grows – As we build to the model we are creating – our staff will be committed to a culture of growth. The senior people (call them Master Craftsmen, if you like) will have a large part of their time focusing on investing in a team. Investing in folks just starting with us (call them Apprentices, or even a bit further up – Journeymen) When you talk to Andy, Brian, Robert or I (or really anyone who has been doing technical work for awhile and is successful and advanced), you can see that we grew because people chose to invest in us. We had mentors along the way. It’s time to give back. A Linchpin People person invests in mentoring and developing skills. We all learn together.
• Clients win – We want to put our money and our approach where our mouths are. We can say that we put people first, that we love our neighbors as ourselves – but that has to look different. Our clients should (and already have – I’ve done some work with Linchpin for a bit now before announcing and making it official) feel like they are dealing with someone different. We are not here to put our foot in the door, up-sell an entire bench of people waiting for work and nickel and dime the client. That model works great for a lot of companies but it isn’t where we want to go. Sure we want to make money, and this model allows for that, but we want to solve problems. We want to be there for a customer and mentor them, share with them and deliver what is needed, not what we can “sell” them. Our value is going to be felt as we help equip our customers to solve problems, not stuck in meeting rooms planning more meetings for more consultants.
• Knowledge will be a shared resource, not a weapon – We won’t have the “my client” or “my department” attitude. We will openly discuss and share our knowledge and work within the company to help solve a client’s problem. When a client engages with us, they may have one or two specific points of contact but they are really paying their rate for access to the collective knowledge at Linchpin. They pay for a resource but get access to the hive mind and experiences that our teams bring to bear.
• We want our customers to help each other and even bypass us – Not sure how this looks in implementation yet, but we don’t want to hold our expertise over a client’s head. If it takes us less time to point out a good resource than engage on a specific issue, we’d rather get our client help and not worry about billability. We are talking about private social media opportunities for anyone currently engaged with us to access answers to their questions while respecting their privacy. Potentially even opening up cross client communication channels. If we can put a customer in one vertical in touch with another having similar questions and they can help each other, why do we have to stand in the middle and bill for that information sharing? We want to add value – whether or not we directly bill for it each time.

There are many other appealing principles and business opportunities that are born from Service 22:40 but these are the things that come to mind right off.

Partner? What Does That Even Mean?

So I’ll be working with the other partners to help setup and maintain a couple specific practices where I have expertise. Performance Tuning, DBA related services (HA/DR consultation, Health Checks, DBA Mentoring, etc.). I’ll currently be heading up the principle Linchpin People New England region presence and will help facilitate client and business partner relations and conversations in that region. I’ll be helping to build up the right staff both in the region and in the practice areas I’m focused in- looking for apprentice & journeymen resources looking to expand their skills, try out consulting (either full time as we grow, or part time in addition to work for their own anchor clients or full time jobs) or other master craftsmen looking to work with us in some manner of relationship.

Being a partner doesn’t mean I “own” the business or have a direct claim to its equity, but I share in the profits and business decision making as we go. I’m emotionally invested in the growth, I’ll be spending my own time growing the business model and working with Brian, Andy and Robert to shape our future. I’ll be an individual contributor on many projects but over time, I’d like to focus even more time on business development growth and mentoring others and advising/consulting to our own internal teams to help them help our clients. The goal is to give back more and to eventually help encourage and equip other master craftsmen to spend less time directly billing and more time growing the culture and bring up the next “batch” of technologists.

What About Straight Path?

To sound like a bank after a merger, “The same great Straight Path Solutions that you’ve grown to know and love will still be here.” It really will. Most of what I do for Linchpin will probably be through the umbrella of Straight Path. I’ll even continue to work with, and possibly still grow my own direct client base at Straight Path IT Solutions. I love working for myself and I love the flexibility of helping direct. Partnering with Linchpin People will give me access to more resources easily and a group of partners for advice and tips on business development. I think over time, I’ll push more new clients (if it makes sense for them) towards Linchpin because of the ability to better serve them with a team approach instead of having me as a single contact, but even my partnership with Andy, Brian and Robert is an entrepreneurial endeavor – I still get to exercise that creativity and love of making business decisions but now I have an “executive team” helping me out and advising me as I go! Through Linchpin, we will be exploring other areas where we can provide value and have passion – areas like Leadership, Employee Growth, Professional Development and helping companies solve the team struggles that sometimes end up masking as technical problems.

So Straight Path will still be there, I’ll still be working on working to live instead of living to work and I’m really hopeful that this model of “people first” professional services does for our partners, staff and clients what we are hoping and expecting it to do!

It's T-SQL Tuesday Time! Another month well underway

What Do They Mean by “Big Data” Anyway?

I don’t know (see I’m not afraid to say it ) – But seriously, I don’t know. It gets thrown about a lot lately. I am not sure if there is an official definition out there that all can agree on. I found the below quote as the first search result in my recent search for “what is big data?” It comes from Edd Dumbill at O’Reilly and I kind of like the definition (emphasis mine):

Big data is data that exceeds the processing capacity of conventional database systems. The data is too big, moves too fast, or doesn’t fit the strictures of your database architectures. To gain value from this data, you must choose an alternative way to process it.

So for the rest of this post, that’s what I mean when I say big data. Well let me back up – the bolded sentence anyway. I think of Big Data as the data that comes in so fast, the data that humans don’t directly make – or don’t necessarily think about making. Financial transactions – even at Amazon – isn’t really what comes into my head as Big Data (I’m sorry but I think I’m going to have to say that phrase a few more times in this post, feels weird typing it over and over). I think of it as data that doesn’t necessarily need to have a schema defined first. Data that doesn’t necessarily need to conform to a certain order right away. Today you may want to do x,y and z with the data and tomorrow you may want to do a,b and c – so you can’t format away data elements, you can’t structure it in a way that prohibits a mid-stream change in approach to it. It isn’t just data that we need to be a bit looser around the structure of and hoarders of – it is also “big”.. So it probably doesn’t come from you typing something into a POS or client application. Think scientific data; sensors; traffic patterns; tweets from everyone – even when someone famous dies or an earthquake happens; web visits and all the associated data – lots of data. mmmmm data.

What Am I Doing With It?

I’m currently helping a well known travel industry website. They use Hadoop and Hive – a query engine that sits on top of Hadoop – turns queries that look a lot like SQL into map reduce jobs.. So folks who know SQL can more quickly ask Hadoop questions without having to be Java programmers writing their own map/reduce jobs. But they also use SQL Server Reporting Service and SQL Server Analysis Services. The model we are moving towards with them seems to be a great model for companies who want their unstructured data and want to query it fast too. The main data they care about are primarily log files. Two kinds – 1.) the log files from the web servers – IP of visitor, length of visit, referral, etc. and 2.) Internal xml logs that all of their servlets write to as a user does stuff on the site. This amounts to over 100 million new rows of raw log data per day from the millions of visitors per day which turns into billions of rows when joined to various data sources. We then query this data in a few ways based on what the goal is:

Brand New Question

Let’s say an analyst wants to ask some question of the raw data that we’ve never thought of asking and never really developed a cube or report to ask that question with. Well instead of sitting in a series of meetings, ramping up a BI project, etc. – the analyst just asks that question. They would write a Hive query in the Ad-Hoc Hadoop cluster that joins, aggregates, etc. the billions of rows they are looking at (typically the data is partitioned by date created in Hadoop with this client and the analyst may ask a sample question of a day of data or perhaps a month at a time). This will take some time depending on their query and the amount of data. That’s fine, the cluster can handle it. The jobs spin up on all of the nodes that the pieces of the data live on and the cluster spins through the map and reduce jobs and eventually spits out an answer. If this was really just a one time question, or one asked infrequently that’s the end of it. They ask it this way and they are fine with the delay because they understand how much data they are crushing through to get the answer.

Same Question More Often

So when the analysts realize there are questions that are good indicators of business and critical decisions can be made to optimize search engine traffic, optimize click throughs, make a more positive user experience, etc. they want to ask these questions more often. This sounds like a more traditional data warehouse experiment. There are key indicators of how things are going, there are reports that indicate how partners are doing, etc. These shouldn’t have to be run by submitting jobs to the Hadoop jive engine or through Hive queries. So now we can approach this in a few ways:

Create base aggregations in Hadoop and export to SQL – Rather than export the full set of raw data into SQL Server and have to scale up to be able to join and aggregate the data – we aggregate the data down into the set the business unit cares about (for instance the B2B teams don’t care about all of the vacation rental clicks, the consumer teams don’t care about business properties and neither team cares about all of the search engine crawler traffic). We aggregate the data down as much as possible – letting Hadoop and all of it’s nodes do the leg work for us overnight and we export the resulting aggregates to SQL. These aggregates can then be queried directly or with Reporting Services for some basic reports. If that isn’t enough we can

Create SQL Server Analysis Services Cubes – So if users need to slice and dice more, if they need to get access to further pre-aggregation and the benefits of analytical processing, the teams can create cubes. Either load them directly from the initial base aggregations in Hive (and other reference data stored in Hadoop and on some other processing systems – lookups for location names and other reference information like that) or they can create a dimensional warehouse in SQL which is populated from the aggs in Hadoop and then populates the cube.

So what we’ve done and are doing is identifying the best tool for the job. In our case we’ve determined that it makes more sense to buy commodity compute clusters (though it’s hard to call them commodity, they are some nice servers that this client chooses to buy) with local storage for the big data, and then go with smaller SQL and SSAS environments that only contain the data necessary to answer those questions repeatedly asked. This is constantly changing – as new insights are gleaned from the raw data, new cubes and new reports in SSRS are created but we never have to worry about throwing away data and we never have to worry about having billions of useless rows stored in the relational databases slowing things down (or pushing us to more expensive SANs and servers) just because we were too afraid to throw the data away or put it “offline” or on cheaper storage someplace. – All of the data is available and the business can be more agile to market conditions and changes.

So yeah, I guess I kind of would say I love the concept of Big Data – but I still love my relational databases and I don’t think either minds my love for the other. When it comes to data, I guess I’m polyamorous. I’m glad that Microsoft also made the decision to start to love Big Data – I wrote this Big Data blog post for the SQL PASS Summit where Microsoft talked about some of their strategy around Big data.

This post is my attempt to start back up with my “Lessons From Disasters” series I promised I would start awhile back. I’m intrigued by disaster preparedness and I have an interest in learning from history. We’ll take a look at some real disasters and find the things that we can learn as IT professionals in them. We’ll explore fields that deal with disasters and learn how to apply their training principles to our day jobs and we’ll hopefully search out some examples of “everything done right” to serve as examples to go after. I’m going to work on having one of these posts out a week  – on Thursday.

Splinted to Death, Really?

Yeah. I still remember the EMT course I was in when I first head the concept – it made sense immediately. In fact it is beat into your head in the Emergency Medical Services (EMS)  trough trainings and refresher courses in the form of the ABCs (well now the CABs) and the proper order of patient assessment. Most of the training and education standards in EMS come from data – really rich data collected and analyzed by states and the National Highway Traffic Safety Administration (the folks who maintain EMS standards in the US) They see what works and what doesn’t. They analyze the documentation done in the pre-hospital setting and then track the progress of patients in the system. One of the things they note is that folks in EMS had splinted patients to death. It still happens but less with the focus on proper assessment.

I keep using this phrase, what do I mean? Imagine this scene – you are on an ambulance crew and you arrive to the site of a car accident. You see that there is one patient and they are walking around screaming, “My arm! My arm! Help!!! Please, help, my arm!!!” you look down to the arm they are holding and instantly realize it isn’t setup the way it’s supposed to be. Obvious deformities, clearly broken – probably in multiple places. Every move the patient makes sends them into bone chilling, nausea inducing, scream worthy pain. It needs to be stabilized. The patient was up and walking – a great indication they have a pulse. They were able to scream  – a good indication they can breathe (contrary to what a panic attack patient believes as they tell you over and over again “I can’t breathe” while they hyperventilate at 50 respirations/minute.) You are there to help and you can almost imagine their pain, so it makes sense to start splinting that arm and treating the symptoms… Have that patient lie down on the stretcher and get to work on the splinting. It’s going to be a tricky one to stabilize just right and it hurts every time you touch it, so be careful. Eventually, you’ll have a well splinted arm on this patient who has really calmed down as a result.

But you forgot to check vitals… You forgot to listen to lung sounds… The patient is calm because the patient is on the downward slope of shock – decompensated shock. Their body gave up trying to make up for the issues affecting circulation or cellular respiration. They are just about out of it, you can’t even feel a pulse at the wrists anymore because the blood pressure can’t get it there, you can barely feel it around the neck… You focused on that obvious, ugly, painful arm and ignored everything important. You missed the internal bleeding in the chest crushing against a lung. You missed a cruddy blood pressure and a rapid weak heartbeat. You missed they were already trending towards bad on the “good/bad scale”… Now maybe you noticed all this in time after the fact and maybe you didn’t. Maybe the patient survives another couple weeks and then dies of organ failure, maybe they walk away fine, maybe they don’t leave the emergency room alive. You’ve splinted your patient to death.

But We’re IT Professionals

Right. Right. I was getting carried away. We “splint” our database environments to “death” all the time! With no NHTSA governing how we are trained and how we operate, with no databases tracking outcomes, we probably do it a lot more than you see in EMS with human patients. I know I’ve blogged quite a lot about troubleshooting and many of these points even sound like a SQL Server Central article I wrote a few years back but I’m still bumping into this phenomenon. So this does apply to us and I think the EMS training and tips for dealing with the risk applies to us, too.

The Solution

The same solution in EMS actually works here:

• ABCs – Spend time on the basics necessary for life before moving on. Power on? Services up? Machine pingable? The point is – the ABCs have to be the things that you can’t do without. Minor bleeding will stop on it’s own if you don’t fix the airway. You can restart the app server and reboot the client as many times as you like – if the SQL Server service won’t start you’ll never connect to it.
• Rapid (Primary) Assessment – Once you check for and fix any ABC deficits (Clear airway, adequate ventilation, beating heart and immediately life threatening bleeding), do a rough assessment of the patient. Note anything found and fix anything serious. Anything critical in the error logs? Blocking chains? Nightly job running crazy? Someone change the app’s config file? Can you log in yourself?  The point is - find and treat any remaining threats to life or limb, don’t get caught up buddy taping fingers – do take the time to splint and unstable pelvis that could damage critical arteries. Find and fix the things related to or likely related to the outage. Note the things you find that may not be optimal but aren’t causing this issue. The CIO wants those other things fixed, but they want the server up now.
• Make Transport Decision  – You’ve done the basic checks and you should know by now what priority this patient is. Need a helicopter? Closest hospital or trauma center? Do you need advanced licenses to meet you en route? Are we looking at an extended downtime? Can we try a few more quick things based on the info we have? Should we go live in the DR site because we need more troubleshooting time? Do I call in vendor support? Do I wake up the dev team? The point is – You’ve done some quick checks and quick fixes – what are you going to do next? Chat sports and wash cuts, hand out ice packs or get hauling with a critical patient – do you need more resources? Don’t get lost in this weird limbo state of “I don’t know what is going on and what I’m going to do next” – Move the situation towards resolution. If you need help, no one cares – they just want the system up. If you have to failover – do it and move on.
• Secondary Assessment – So you’re heading to the hospital, you have a bit more time – go back and look in more detail at the findings you noted during the rapid. Fix those broken fingers, clean wounds, look for other hidden injuries.Fix that max memory setting so this issue doesn’t happen again, create the task to take Domain\Staff out of thy sysadmin role, Look for the tertiary issues related to the outage and resolve them. The point is – you want to double check your findings, look for additional issues and continue to make improvements/prepare the hospital to make improvements. Look for other issues, go deeper where you felt “off” about something but didn’t see any obvious and quick issues -Remember – the secondary assessment only begins after the first three steps – you have your system coming back to life, users can start to get in to prod or DR – if not then you are still fixing ABCs and primary items – there are some ambulance calls where the entire call and drive never leaves the ABCs because you are fighting an airway the whole way in.
• Reassess/Monitor Vitals – How are your treatments working? Is the patient’s vital signs trending stable or going downhill? Are connections remaining? Can that first batch of users you let in get their job done? Are processes running and staying running? The point is – you need to monitor your interventions. You want to make sure that things aren’t getting worse again and if they are you need to act.
• Cleanup, Documentation, Etc – The next call doesn’t work if the ambulance isn’t restocked, organized and ready. The EMS system never gets better if the calls aren’t documented. That patient’s medical team misses an important fact about the call if you didn’t document it. Lessons learned meetings (not blamestorm meetings) help us avoid the problem and improve the troubleshooting approach for next time.

Related Posts

• If You See Something, Say Something – If something is wrong and you know it – say something! Don’t assume everyone else already knows.
• Avoid Using Those Troubleshooting Skills – Acquiring troubleshooting skills is an important endeavor for folks. But what if you handled your environments in such a way you needed them less and less?
• Best Practices: Explain and Understand Them! - I hate it when folks say things like “this is best because I feel…” or “I’m not sure why, but just always do this!” No one will follow your best practices if you don’t explain them!
• Are You Planting Asparagus? – Asparagus can’t be picked for the first couple years after you plant it. It still takes preparation and hard work. Are you making decisions with the long term in mind? You’ll be less likely to find a situation where you have to splint your database.

1 – “I Don’t Know” isn’t a phrase you know…

I ask different kinds of questions. Some are questions you’ll find in books or internet searches – like, “how many clustered indexes can you have on a table?” Some are more “describe or explain”  in nature with no absolute right or wrong answer – I want to see you can string some thoughts together and add common sense, logic and knowledge. Some questions I don’t expect you to know! In fact, I learned from a great manager awhile ago to go out of my way to ask a question you won’t know the answer to. I’m looking for, “I don’t know” as a response here. I’m fine with an “I’m not sure, but is it” type of answer. I don’t normally get those responses though. Even a “simple question” -  I would rather hear “I don’t know” to an easy question than some far fetched stab in the dark that was  confusing and off. (ex. “What are fixed server roles?” “well in those old versions of SQL Server the roles didn’t do what people told them to do, so now they went and fixed them, so the roles are right”) If I am recommending someone give you the keys to their prized production environment, I don’t want you to get all “what does this button do?” in it. “I don’t know” is not a bad answer*

2 – You have no defense…

I’m not mean in an interview. I’m not dishonest in an interview. I do like a little disagreement from time to time, however. I want to hear you defend a position. I want to hear you stand up for your answer when it is a case of lots of ways to do the same thing. So I may disagree with you mildly. Or play the opposing view of one of those “religious” wars in SQL circles. I don’t approach it in a rude way, but I want to prod you further along. Defend yourself. When I am hiring someone to join a team I am on or a team I help – I’m not looking at it like I’m a dictator trying to fill cabinet positions. We are dealing with important environments and significant investments. If I decide to do something that burned you at your last employer, I want to know you’ll raise a flag and say “Actually, the last time I saw this here is what happened… Here is why I don’t think we should do this and have you considered this instead?” not “Okay! Go for it, Mike!” Even in positions that have “junior” in the title, I want to see folks willing and comfortable raising objections… As I talk about in presentations on learning from real life disasters – agreement in spite of concerns is a disaster causing attitude… I don’t want that anywhere near database servers I care about.

3 – You don’t need to defend yourself…

Along those lines, I also really don’t want you if your only defense is, “I’ve been doing this a long time, that’s the right answer, how long have you been doing this?” Arrogance is also a disaster causing attitude. Just because you personally detest anyone ever using an identity column as a primary key doesn’t mean you are right and any exception to this rule proves the rule breaker is an idiot. So if I ask you to defend your position, just remember it has to be a valid defense (Or at least heading towards validity.. I’ll even take “the glove didn’t fit” over “I said I’m not guilty, what more do you want?”) Tell me why identities are bad as primary keys. Answer my counter points when we have that discussion. Then tell me how you’d work with them anyway if that’s the scenario we paint. You may be an industry expert, but if you can’t get over yourself and outrage that someone would ask you to explain a little further, I’d be hard pressed to suggest you join a team that ever has the potential of being staffed by more than one person.

4 – You do, know and learn what is expected…

Only. Only what is expected. It is a competitive marketplace out there!  So you need to stand out, you have to be different. I will always ask about what blogs you read (if you say my blog, by the way, I know you just googled my name before the interview), what SQL events you go to, what SQL books you’ve read lately, what you do to learn new features, etc.  If you are coming to me for a SQL related job, I will take points away if you can’t show me you actively work to improve your skill set. I don’t mean I expect you to spend every single waking hour living SQL Server – Family comes first, kids  come first, weekends enjoying the backyard come first… There is still time to increase your knowledge and wear your more important hats. Stand out from the crowd by showing you care enough about what you do to know more than just what the scope of your last role was! If you feel you’ve already learned enough and don’t need to grow anymore, then you aren’t the right person.

5 – It’s not you, it’s me (okay, it’s you…)

This one may seem odd to write and maybe odder to read. We have to be able to work together – especially if it is a team I am on or a team I help out frequently. I don’t mean that we have to have the same political leanings, faith, hair stylist (in fact, my wife does my hair, she better not be doing yours too) or like the same bands. I want to see -you- not just a talking resume but a human being. I’m looking for that in the later stages of interview processes definitely but even to a degree in the beginning screening calls. Basic things like eye contact, smiling from time to time, engaging in some idle conversation while walking someplace, you asking questions about the role, the company or even the team or talking about your life. We may have to work on some tough issues together from time to time, it helps to have that feeling of a team atmosphere. Again, I’m not looking for you to hug everyone and be fake, or to bring cupcakes to your interview (I won’t refuse them, though..) I just want to know we’ll get along. Be confident and outgoing, those little things add to the complete picture.

6 – You don’t have the right skills for the role…

Finally, yeah – you do have to have the right skills for the job. If you were great in every other aspect for a report developer role but you didn’t know what SSRS was, it would be tough to give you that position. If you were a little off on some of the technical questions but showed me you had a passion to learn, could explain your thoughts, use logic and apply common sense to troubleshooting I may consider you over someone with a bit more skill but less of these traits. In fact, I’ll do that a lot, I believe you can teach skills better than you can teach “traits” but at the end of the day the skills have to at least me someplace on the same map. Make sure you read about the position you are interviewing for and honestly assess yourself before committing. It shows when you just read a bunch of interview questions the second we go a level deeper.

I’m sure I missed a bunch, what do you look for when hiring for that next position? What are you most nervous about when looking for that next SQL job?

Which kind of brings me to adding yet another post on the latest #SQLFamily dysfunction/feuding. I’m not going to say anything revolutionary and I’m going to probably contradict the way I said some things in comments on Brent’s post – at least the first thing I said there…

First – why share? Because I care. I just want to put another thought/side out there to the decision for the board appointment (If you don’t know what I am talking about, Brent’s post is probably the one that summarizes all of the others the best, you may disagree with some of the commentary – but he summarizes the events pretty well)

I only have a couple some related main thoughts to share -

I Like Representative Democracy better than Direct Democracy

Especially in professional organizations. It’s not that I don’t trust all my fellow citizens, it’s that I trust the folks we send to represent us. I trust the ability to change them if they screw up. When looking at the congress and senate in the United States, I’m happy that they vote on the bills and discuss them. They make the time to understand the nuances, to poll the people but then to vote on what they think is right. Now I may (and I often do – from both sides of the aisle!) disagree with what was decided and what bills made it into laws, but I don’t want the solution to be me and a few hundred million others voting it out. We elect a representative because we feel they can handle business on our behalf.

With PASS we send folks to the board who are willing to take on the time commitment, who are willing to put up with criticism and make decisions on an organization they care about. We vett these candidates (sometimes better than others) then have access to ask them questions, learn more about them and decide to vote on them, or not. I like this system.

I’m Okay With The Board  Appointing Board Members

Because I’m okay with indirect democracy (we don’t like a decision we vote ‘em out, but in the meantime, we get their decisions), I’m fine with the board making these kind of appointments. There are rules to prevent them from stacking the board with folks from their company. There are rules in place on who is and who isn’t qualified. There are standards to be met. Then these people that represent a cross-section of our SQL community have to still approve the recommendation. This still represents checks and balances. Just like a legislator in a committee (although I say even more so with our Board – they aren’t doing this for money, prestige or lucrative speaking engagements while collecting a pension when they are done), they are acting with the best interests of the organization in mind. Even if the decision is a tough one, one that will catch flak and draw ire and silly comments like “Bill should resign”, they make it for the good of the organization. Can you get some bad apples? Sure, but you have to get a majority of them to have a bad decision. I don’t see that with our board. In fact, three of the people who just voted on the recent appointments were the top three vote earners in the same election we’re discussing! The same clamor for “go with the popular vote!” could also consider those who won the popular vote agreed on the decision. I don’t know Denise well but I know Rob and Adam and I know they aren’t pushovers who would be forced into voting against their conscience from peer pressure. I know Tom cares so deeply about the PASS organization. I know Allen has poured so much of his time into the group. We are talking about people who really do seem to care about PASS. I trust them to appoint members. If I didn’t trust them with that, I probably wouldn’t trust them to manage the world’s largest SQL Server event, a budget and the various portfolios they manage.

Further, as Aaron Bertrand has pointed out in numerous direct and indirect ways in response to folks, so many professional organizations already do appointments this way.

The Timing…

Brent raises an interesting point that stands in a vacuum. I don’t know what and why the board did what they did, but perhaps we should stop to consider that maybe they had reason. They discuss people issues in closed session. This is standard in just about every organization I’ve been a voting member of, it’s standard in town or city politics. Because I like a representative democracy and because I trust those we elect to represent us to do what is right and best – I trust their thought process. The only frustrating part here is the timing, it was right near an election, in fact the vacancies essentially coincided with the election process. The by-laws allow what was done to be done and I support the board and exec team in their decision making process, whatever their reasons are. (In fact I respect them even more for doing it! I think the easy decision would be to say, “we are gonna feel this one! let’s just go with the popular vote, we are fine by by-laws either way, if we say no to anyone folks voted on you know we are going to have to explain and if we don’t we get called out, insulted and annoy folks.. If we do explain ourselves then we potentially risk hurting reputations of folks passed over”.. They went with the hard choice, I think).

Maybe A Change Like…

So I signed the petition that Andy Warren put out. Early on but now I wish I hadn’t. I like the spirit of the change but there are issues with it.. What happens if a vacancy is 6 months after the election? A lot can change in 6 months. I wouldn’t mind seeing something like:

- If the vacancy is before, during or within 1 month of a general board election, the Board shall appoint the next highest vote getter

- If the board learns anything that disqualifies that individual (and has clear and convincing rationale – maybe even discussion with the NomCom about that info) then they can pass them over to the next on the list.

- If no one is next on the list, the board can proceed with a recommendation from the President of PASS as they do today. Same thing if those appointed decline.

- If the vacancy is greater than 1 month after the election, but less than 1 month before the next election, the PASS president shall recommend candidates in a manner they see fit. These candidates are then interviewed, discussed and voted on by the voting members of the board.

- If the vacancy is less than one month prior to the next general election, that position is not filled until the general election

- All Board appointments made outside of the general Board elections shall be valid for the remainder of the calendar year, and any appointed candidate who wishes to run for a seat shall go through the nomination committee process as though they were a new applicant.
Or something like all of that… Now we still run the risk of the NomCom allowing someone to go through, having them get position four and then learning something about them in the 2 weeks between election and nomination. The board would be able to reject that person and to protect privacy they wouldn’t be able to state why. That does lead to speculation, that does lead to “reputation concerns” for the individual rejected. I don’t think there is a way around that though. If something about you coming out would hurt you and your reputation, I don’t suggest you run for President of the United States – it will come out. If you are honest with yourself and feel there is something that could do damage to your reputation or cause you to be rejected by the NomCom or the board during an appointment, it probably isn’t a good idea to submit an application. That sounds harsh, but kid gloves only work so far.

Wrap It Up, Walsh…

This is another tough moment for PASS. We can handle it in a few ways that these statements summarize:

• “Fire them all! JRJ should resign immediately, Bill should be fired and this whole process is junk.. They are all liars and corrupt!”
• “Forget it… I’m done with PASS.. It isn’t going to change”
• “Let’s tackle this new challenge. Let’s analyze the by-laws, let’s see if we can be a bit more direct in explaining our reasons without causing issues, let’s figure it out and clean it up”

I’ve never uttered statement 1. I started off with the sentiment in number 2, when I got fired up in chain reaction mode and let other comments influence me. I’m at sentiment number 3 now… No heads need to roll here. No confidence has been shaken. We found a potential weakness in one of our processes. I don’t think it is a weakness that needs radical change like “anytime someone vacates a position, fire up an election” – that hamstrings an organization that is actually trying to adapt to change and make itself better. But maybe something heading towards the c- towards the c-c-c..compromise described above (or someone else’s compromise idea, I don’t care)

Some of the comments (thankfully there aren’t tons of them) on some of the blogs are just brutal and mean spirited and without all of the information. It seems like we are learning too much from the political discourse coming out of both major political parties in the USA and the media supporters on either side. We can all just get along, so let’s do it.

The noise is loud right now and I want to see it quiet down, I want to see folks support the board (the board that we elected) and then I want to see the board tackle some of the sentiment I had in my series of posts linked to above (and more). After all, our ability to work past our differences and talk about touchy subjects is one of the reasons I really love this #SQLfamily and PASS is one of the orphanages for all us SQL People in the family.

SQL Server Community Involvement

Like I said, I hadn’t blogged as much as I would have liked (about 50 posts) but I did get to speak at the SQL PASS Conference (With Erin Stellato – we gave a fun presentation basically asking the all-important vendor and DBA question, “Can’t we all just get along?!” I’m inclined to say no still – it’s that “all” word in there…) had a good ending to the User Group and a bit of a bumpier start this year with it (we’ve still met, just not as many times as I would have liked). All in all – not a bad year -

• Spoke at SQL PASS Conference
• Spoke at SQL Rally Orlando
• Spoke at 7 or 8 local or regional SQL/Developer Events in addition to the two conferences
• Helped Run a SQL Saturday in Boston
• Was recognized as a SQL Server MVP in April
• Still had people reading the blog (just over 33k unique visitors per Analytics and about 21k views/clicks on the feed by feedburner’s reckoning- not a ton and more than half found me through google searches, not because their friends and family said “you have to check out this blog!” but still surprised me like last year)
• Helped some folks out with some of the StackOverflow sites towards the end of the year (mostly with the dba property – great resource, even if their DBA is a turtle expert who dresses his dog up and takes it out in public like that)
• Volunteered on the Speaker Selection Team for SQL PASS’ Program Committee
• Helped out the Deep Dives 2 book as a technical editor
• Helped organize the Birds of a Feather lunch at PASS again

I am already signed up for speaking at SQL Connections Spring ’12 and hope to help get a SQL Saturday together in Boston as well as one in NH. I am sure I’ll be speaking at some SQL Saturdays and User Groups this year as well.  I want to maintain my level of community involvement and try and minimize the travel time at the same time. We’ll see how that works out in 2012!

Straight Path Solutions

Last year, one of my “goals” for this year was to be a great employee. I started off with the year having that defined as for the consultancy I was a SQL Server Practice Lead for. It ended with me being the “CEO”, Book Keeper, Principal Consultant, Sales/Marketing Team and Janitor for my own Consultancy. What a ride it has been! I am so happy I made the jump off to my own and think I’ve done well in the role of Principal Consultant. I need help in the other areas (especially book keeper and janitor) and I need to make sure I give myself the free time I need and make sure I remember one of the goals of going off on my own was to spend more time with the family and giving time back to God. I’ll be working on that as a 2012 goal. Some highlights for this hat, though:

• Continued helping with Winxnet with several of their SQL Server needs (performance tuning, health assessments, helping out the Remote DBA team with escalations, etc.)
• Stayed busy with one customer in particular – a well known travel industry website – primarily as a SQL Server performance and database administration resource but also helping out with their Hadoop/Hive environment and loading data to and from this environment – I’ve learned more about Linux shell commands and writing Bash scripts in the 6 months I’ve been with this client than I ever have. Definitely wouldn’t mind doing some more Hadoop work in the future.
• Helped out a handful of companies with SQL Server Performance and Health Assessments – tuned a lot of queries, made systems run faster and helped DBA teams be more efficient.
• Was selected to be a remote DBA resource to help out a busy DBA team at the 4th largest credit union in the US. Started working with them towards the end of the year and hoping to see that pick up in 2012 and take a more prominent role in the schedule.
• Continued helping as the sole DBA resource for the IT department for one of the largest cities in New England via Winxnet.
• Helped Microsoft Learning develop some new exams for the next release of SQL Server in a few different capacities and on a few different projects.
• Started helping and talking to a couple SQL Server gurus in a venture they are working on. I’ve done some work for one of their clients and talking about ways of aligning with them a bit more in 2012. More on that hopefully in the coming months.

I am still glad I chased that dream down in July and I’m hoping that 2012 continues to be a success but I’m hoping I don’t just define that success by staying busy with clients but that I do give myself more family time and that I am able to give abundantly to charities and ministries like Options for Women and Living Water International.

More Popular Posts of 2011

Some of the more popular posts in terms of traffic or comments/tweets/mentions last year (as well as some of my favorite ones thrown in):

• Your Dreams? They Don’t Come Looking For You! – A little background – I wrote those while in the final stages of mulling over my decision to go off on my own or not. Ultimately? I decided to get off the ladder and try the water.
• Can’t Troubleshoot? Don’t Apply – Reflections on changing my truck’s starter and the kind of person I look for when interviewing.
• Are You Planting Asparagus? It’s work, It’s a picky plant, You can’t even enjoy it for a few years… Success in the workplace is a lot like Asparagus.
• You Can Restore It! (right?) – Your backup is verified, the job isn’t failing. You’re all set, aren’t you? No? Maybe?
• If You See Something, Say Something - Normally, I don’t have a ton of praises for government advertising campaigns. I think there is something for DBAs in this one, though.

Here’s To 2012!

I really thank you for checking out this blog. I thank you for the contributions you’ve made or are planning on making to the SQL Server community in 2012. Let’s make 2012 a prosperous year. Let’s make it a year where we all learn and grow together in amazing ways. Let’s make this the year of the #sqlfamily… But… Let’s do what we can to make 2012 the year where everyone can drink water that won’t kill them.

I don’t want much. Bill Gates is already taking care of the world, so no beauty pageant wish there. As a SQL Server professional I just want a few simple gifts – stocking stuffers really. In fact a tune comes to mind…

The defaults in SQL Server anger me
Auto Shrink and Full Recovery mode by default especially
I do know just who to blame for this catastrophe!
So my one wish on Meme Monday is as plain as it can be!

All I want for Christmas
is the auto-shrink option dead
the auto-shrink option dead
see! that fragmented database!

Gee, if I could only
have the option dead
then I could wish you
“Merry Christmas”
It seems so long since I could scan
all those fragmented tables quickly
Gosh oh gee, how happy I’d be
If it were only dead

All I want for Christmas
is the auto-shrink option dead
the auto-shrink option dead
see! that fragmented database!

That’s It -

I want SQL Server to make it a little tougher for well meaning, but ill-trained people to shoot themselves in their feet. I don’t want people to have to hire high priced consultants to fix problems that are caused by defaults, by maintenance plan options that allow them to fragment their database, by a simple check box to enable auto-fragment mode of their database. It isn’t that consultants are bad – it’s just that my posts on transacation log shrinking and management (and why you shouldn’t need to do it) shouldn’t be the most popular google search traffic to my blog

Reminds me of the series of posts about the top 5 things I wanted SQL Server to drop during an earlier meme. That’s what I want.

P.S. – That means if you are auto shrinking your database or you have to do gymnastics to clear your huge transaction log every time you run out of disk space – you’re doing it wrong. See these posts.

Merry Christmas – I’ll be leaving cookies and milk out for Microsoft in a few weeks. I’ll let you know if they came.

… Ok, you can put your socks back on – there won’t be any more counting. According to most of the studies on world health, somewhere around 3 kids under the age of five just died in the time it took you to count because they didn’t have access to clean drinking water or safe sanitation. That’s right – depending on whose statistics you use you can come up with a ballpark of about 5,000 kids under the age of five die every day from this. About 3.75 Million people die of water related disease a year (per this study done by the World Health Organization.. I’ll let you look for the other statistics yourself)

Seriously? Water?

That was my reaction when I first saw this quick video (that states the cost to bring clean drinking water to the world is about $10B while us Americans spend somewhere around $450B on Christmas presents each year). I knew people were dying of hunger. I knew kids were dying in war ravaged countries. I knew there were AIDS orphans throughout Africa. It never really hit me that that many people die because of what they drink. It never occurred to me that so many people are kept down because of their daily trek to a distant water source to bring contaminated water back to their thirsty family.

As I type this my home office is a little bit of a mess. Looking around I see three 20 ounce empty water bottles, a couple empty 12 ounce cans of Coke Zero and an empty 1 Liter empty seltzer water bottle. Look around you right now. See any drink containers? Yeah. 3.75 million people die because they share their drinking water source with a bathroom, a disease infested river and all that lives within it or some muddy puddle that someone just washed their hands and face in.

It never even occurred to me that kids aren’t getting educated right now because they are journeying miles to bring water back home that we wouldn’t even touch – let alone drink…  It never even occurred to me that people are poor in many places simply because of water. I never imagined that some women spend 15-20 hours per week on the task of bringing water back to their family. Forget about Women in Technology chapters there in those villages…

Now I’m not saying this to challenge you – or me – to say we have to live in guilt. I’m not sharing that video link with you (here it is again) to tell you you have to give up every last possession, you have to give up every trip to Starbucks, etc. No, I’m just sharing it with you because maybe you are like me – maybe you never stopped to consider this crisis that is happening in our world. That’s okay, we can make a difference – even in small ways.

What Can We Do?

There are many groups out there helping in many ways with this water crisis. First – let me tell you they are making a difference. Villages are thriving because of clean water being brought to them. People are alive because of these groups. The grim statistics are improving but they still have far to go. Until everyone in this world has the ability to drink clean water without spending their day journeying for it we aren’t done. But it is important to remember – a difference is being made and help only increases that difference. Some organizations are giving the tools and tricks to filter water in innovative and inexpensive ways. Some groups are drilling wells so they can have the water where they are. What do they need? Awareness – maybe you aren’t at a point where you can help out one of these groups directly but you can let people know they are there, right? They also need money.

They aren’t asking for a lot to solve this problem. We are talking about a $10B problem from one source – This summer the iPhone topped $28B in revenue for Apple. Black Friday 2009 (just that weekend) saw around $41.2B in sales… This is the kind of problem that a small sacrifice (though I wonder if that is even the right word?) from everyone could see a world of difference.

I realize that there are SO MANY other needs out there. I just think that drinking water – clean drinking water – is potentially the most basic of those needs. Fixing it leads to fixing other aspects of life. Fixing it in our generation is a powerful message. Fixing it now means we can tackle that next problem next.

A Challenge For December (and beyond)

It’s 12/1/2011 when I am typing this. I am going to go a month drinking water tap water in most places – unless on a flight or finishing off a gallon in the fridge already. When I feel tempted to go grab an iced tea or a cappuccino while driving to a client site – I am going to figure out the cost that I would have spent and set the figure aside. Same thing at the restaurant, etc. Even “just” living on tap water – I’ll still be living better than the billions without safe or clean water. I don’t get many presents for Christmas or my Birthday any more but if folks ask – I’ll ask them to make a donation instead. I’ve also decided to give an additional portion of my consulting gross revenues to the Living Water charity starting now – and I’m going to keep doing that until this isn’t a problem anymore.

I don’t share this to brag. I don’t share this to say I’m better than you – I guarantee you I am not   I share this to encourage you to see what you might do. Maybe you can share the story of the water crisis in the world with someone. Certainly you can pray. Perhaps you, too, can do the Christmas gift giving frenzy just a tiny bit lighter this year or skip the latte and pitch in to bring fresh, cool and clean water someplace it has never been before. Talk about a Christmas gift! Imagine drinking mud your whole life and someone offering you a sparkling glass of fresh water? Maybe you are wiling to see some percentage of your company’s revenue going to making sure we can tell our kids we live in a world where everyone has clean water.

I encourage you to check out the source of the video: Advent Conspiracy. Or the organization I’ve felt led to support: Living Water.  Or do a quick Google search for water charities and find one that works for you.  Living water has placed about 10,000 wells so far and they’ve seen lives changed from their presence. They are a recognized and audited charity.

Who else is in?

#sqlwater works as a hashtag here, I think. You don’t have to be a SQL person to join. Blog about what you are going to do, leave a comment here if you want. Tweet about a small (or any size) “sacrifice” you are willing to make this month and tell us which charity you’ve picked and why. I’ll keep my updates going on that twitter hash tag and here. Remember – the goal in sharing here isn’t for us to brag about the little bit we are doing. It is to encourage each other along and to raise awareness. I know we are a giving community donating to waks, marathons, food banks, cancer treatment or awareness, etc. I think it is great and I happen to think we still have a little room individually to give but as a community of small donations? We can bring water to those in need – right now. Who else wants to hep out?

This post is all about my Savior and my faith. If that isn’t your kind of reading, check out one of the other posts from this series-

• Perspective - Why looking back at past challenges can make current ones seem, well, not that big.
• Relationships - I am where I am in large part because of the people who have put time into my life. Relationships matter.
• Self Employment – I made the decision to try it this year, and I’m glad I did. I think you can make that same decision.
  
• Community – The SQL Server community is an amazing community of technologists that gets stuff done.

What is Grace and Why be Thankful?

Some have turned the word grace into an acronym – God’s Riches At Christ’s Expense – and I like it. That sums grace up quite well, I think. Another definition that works is, Unmerited Favor.  These drive at the same point – Grace is a gift. Grace is a gift from God. Grace is a gift from God that we don’t deserve. Grace is a gift of God through the sacrifice of His Son, Jesus Christ, that we don’t deserve. But he still offers it.

I never realized all this before I started talking to Christians. I had my own notions of Christianity before. I thought Christians were mean, self-righteous people who worked at good works to earn favor with God. I thought they were all hypocrites. I wasn’t sure what I believed about the eternal state of my soul but I certainly felt that I hadn’t done anything that bad. I hadn’t killed anyone, I helped other people, I shared. I figured I’d be fine and didn’t need to turn into one of them.

It wasn’t until I started a Bible study with a Pastor who was trying to answer all of my objections to Christianity that I realized that the Christianity I saw on TV or even in the way some faiths try to live it out wasn’t what the Bible said at all. Condensed way down -

God is perfect. God is Holy. God is Righteous. God is Truth. Our sin separates us from that type of being. We can’t ever enter into His presence in a state of sin – nor would we ever want to. If you think of God’s standard as a “heavenly yardstick” with holiness at the full yard, we barely measure a quarter of an inch with our “good” or with our  attempts at “righteousness”. God’s standard is Himself – Romans 3:23 describes this standard – “For all have sinned and fall short of the glory of God.” One lie is enough. One lustful look is enough. One selfish, pride-filled moment is enough. One moment of greed is enough. You get the point. We daily fall short of the glory of God. Whether you are Christopher Hitchens, Mike Walsh, Billy Graham or your dear grandmother this holds true.

So where does that leave us? Condemned in our sins – heading towards the just punishment for those sins – an eternity apart from Christ (Romans 6:23a – For the wages of sin is death). In fact it isn’t even a punishment only, it is what we want. If we don’t want to chase after God in this life, if we don’t want to love Him then why would He force us into His presence forever? Whether we want to admit it or not (and I didn’t for the longest time) – our sin has a price. Our sin fixes a gulf between us and God.

But Grace created a bridge. God’s grace provided a path – the only path – to salvation. Out of a motivation of Love (John 3:16), God sent Christ to take our place. The only One who could legally pay the price of sin – for He was man – and the only One who could pay the price for all that ever have or will live – for He was God. This Christ came to earth, as history and secular scholars agree, and took on our punishment on that cross. He laid down His life, but more – he faced the punishment of our sin on that cross. His fellowship with His Father was broken, He bore the full weight of our sin. Romans 5:6-10, one of my favorite passages in the Bible, describes this act:

For while we were still helpless, at the right time Christ died for the ungodly. For one will hardly die for a righteous man; though perhaps for the good man someone would dare even to die. But God demonstrates His own love toward us, in that while we were yet sinners, Christ died for us. Much more then, having now been justified by His blood, we shall be saved from the wrath of God through Him. For if while we were enemies we were reconciled to God through the death of His Son, much more, having been reconciled, we shall be saved by His life. (Romans 5:6-10 NASB – emphasis mine)

That’s grace. Grace is a gift. It is a gift offered to all people in all times but having a gift offered doesn’t mean it’s ours. If we need a lifesaving blood transfusion and someone is willing to offer it to us – that isn’t enough to save our life. We have to accept that gift and apply it to our lives. We have to receive the blood. We have to open ourselves to it. God’s grace -  God’s unmerited favor – is the same. We have to accept it. We have to receive it and believe it. At that moment we get to see the joy in the second part of Romans 6:23 – It started with “For the wages of sin sin is death”  but I thank God that it doesn’t end there. Instead it ends “But the gift of God is eternal life through Christ Jesus” That’s grace. I’m thankful for Grace. I’m thankful for God entering history to make a difference for me. I’m thankful that 10 years ago I was convicted of where I was on that yardstick and realized that my “good acts” mattered not for eternity in light of all the ways I fall short. I’m thankful for all the people along that path.

To this day, I still hold that this is a decision I would have NEVER MADE on my own. I didn’t want it, I didn’t think I needed it and I thought that I was smarter than “those” Christians. I was relying on me and thought the Bible to be foolish (The Bible even says it is foolishness to those who are perishing, as I was! – 1 Cor 1:18). God, through His Grace hasn’t left me in that state. I have banked my eternity on His grace and after ten years of answered prayers, walking with Him, Bible study, apologetics study, research, etc. I can say with confidence that my faith is well founded. That the grace it is built on is real.

Not Holier Than Thou

As I see the Christians I spend time with – the committed, Bible believing, Grace reliant Christians – I see that the perceptions I had of holier than thou attitudes were not typical of Christians at all. The perception of people working hard at being better than everyone else was not Christianity! I’ll even say that those churches that rely on works for salvation are not founded on Biblical principles. Salvation is a gift of grace. That means that I’m not any better or worse than Billy Graham. That means that I don’t deserve this grace any more or less than Ron Bronski (His story is an interesting one). You see these Christians that I started getting to know are all people, just like me. They have hurts. They have pains. They mess up and still make mistakes after accepting Christ but they are still different. Their hearts are burdened for Jesus. Their hearts ache when they mess up, they have a love for others and a spirit of giving that puts the United Way to shame.  They don’t judge, because they realize that they are the same. They don’t judge because they know they, too, need grace. I’m telling you, you haven’t seen love like the love you see when a group of Christians comes around another battling sin or battling something personal and perhaps even embarrassing. The outpouring of love isn’t natural, it isn’t something that the world normally resorts to. It is the result of that Grace that they’ve received. I am so thankful for the loving church I go to. The compassionate believers I worship our Creator with. Thankful for the love they show new believers, visitors and even believers who fall down. We are called to be Christ to people and the Christians who work at this instead of seeing their mission as condemning and distancing actually make a difference in lives every day. I’m thankful for that.

A Community

Last, but not least, I’m thankful for the #PassPrayers group and discussion list. I’m thankful that Christians are part of one body – Christ’s body. That we are united by something that lasts for eternity. Wherever I go, when I meet a Christian – there is an instant friendship, there is a closeness that I can’t describe. I got to witness it first hand at this years SQLPASS and saw the impact the #PassPrayers group had. Brothers and sisters from around the world gathered together to share prayer requests, to share burdens, to Worship our God in song and thanksgiving. It was great. It was actually the highlight of the entire conference. There were deep hurts shared openly at prayer request time. There were unashamed, unembarrassed voices lifted loudly to bring praise to God in public while we sang along with Rob Farley and his guitar. Even now, the #PassPrayers e-mail group is active. Sharing encouragements, stories, prayer requests and looking to have more meetings across the country. I am so incredibly thankful for the technology that allows us to stay connected, for the hearts that are willing to share with others, for the hearts that pray for others. I’m thankful for God’s grace that allows me to be a member of the body of Christ.

• Perspective - Why looking back at past challenges can make current ones seem, well, not that big.
• Relationships - I am where I am in large part because of the people who have put time into my life. Relationships matter.
• Self Employment – I made the decision to try it this year, and I’m glad I did. I think you can make that same decision.
  
• Community – To quote a mall or theme park map, “You are here”
  
• Grace – I touched on this in the relationships section but I’m going to talk a bit more about grace and perhaps Pass Prayers on Friday.
  

This Thing We Call “Community”

Or, as Thomas LaRock put it – #sqlfamily…

I’ve blogged about the SQL Server community before a couple of times (“Community, Community, Community” comes to mind first) but I still think it’s a mighty neat group of technologists. I am not even sure how it came to be this way and I’d love to study the family tree someday to figure out why this community is different than some other technology communities. To clarify when I talk about the SQL Server community, I’m talking mainly about the folks who actively blog, read blogs, tweet, speak at or attend events, answer questions on SQL Server Central, actively vote in the SQL PASS elections, etc.

So what is there to be thankful for about this community? Well in no particular order, let me count the ways – I’m thankful that…

We don’t always agree but we just about always get along.

To me the classical examples are the things we aren’t supposed to talk about in public. Faith. Politics. Dynamic SQL vs. Stored Procedures (Stored Procedures, btw) Surrogate Keys vs. Natural Keys, etc.

There have been many a sidetracked twitter conversation that escalated into a multiple person conversation with folks on very different sides. We’ve talked about heated points. There have been animated discussions over political policies and the direction a nation should go. Tough discussions/debates on principles of faith. Yet.. Yet.. It seems like anytime I see (or take part in) one of those discussions, everyone ends up knowing each other a bit better, finding points of agreement and even making new friendships with people whose views completely clash with your own. I’ve seen families torn apart by these divisions yet we find unity in our differences. That doesn’t mean I am endorsing views I disagree with – it means I see that a human being holds them and I want to learn about myself and my world through their views or why they came to them. That is a common attitude. I am thankful that we all can just get along.

The “leaders” seem to pour themselves into other people.

Have you seen the movie Pay It Forward? Well it’s almost like that kid was person one in the SQL Server community. With rare exception, take a look at the people with the “must visit” blogs or “must read” books. Take a look at the long time SQL Server MVPs, the sought after speakers, etc. They almost all share a trait that you don’t always see everywhere – They can’t help but help bring others up. Everyone who is active in this community of technologists seems to want to see others grow. Even the well experienced and much desired consultants at the top of their game seem only happy to give referrals to other consultants. They seem happy to give tips to others. They all realize there are a lot of potential customers and rather than get stingy and hoard knowledge, they share it and hope others do well. That’s different. Everyone learns something every day if they stay active in the SQL Server Community. SQL Saturday leaders have to say no to speakers because so many people want to learn by teaching. We are a community of learners and teachers.

We take time to get to know each other.

Meaningful relationships. When there is a SQL Server event (conference, SQL Saturday, etc.) I always bump into at least one person (and usually a lot more) that I know fairly well. Someone who wants to know how I’m really doing. Someone I care about and want to know more. Plenty of razzing and joking to go around but when someone is stuck… When someone is down on their luck… When someone needs a job… When someone needs prayer… When a family member dies… The community rallies around that person. The only real superiority contest I ever see is who can outgive or outcare someone else. That’s different than any other technology community I’ve been a part of.

 We could fix this country.

Seriously. Some of those political twitter conversations I mentioned? We talk about hard issues. We come at them from different points of view but through our logical and rational thought processes, we usually end up exploring the heart of the matter. We end up finding the points that unite us and let go of partisan bickering or allegiances. We talk about some interesting real solutions together. You fire Congress (all of them) and replace them with members of the SQL Server community and I think this nation is a bit closer to solving those issues that plague us perennially. Common sense wins. Logic wins. Doing what’s right for all wins.

We love new members of the SQLFamily.

No jealousy. No frustration with someone learning. We welcome people just starting out and put on big events for them at our Summits, we go out of our way to find the “new ones” and ask “how’s this summit treating you? Are you learning everything you wanted to learn? Have you met  so and so yet?” I was just chatting about this with Brent Ozar the other day and he drew a parallel to High School. I think it works…. If the SQL Server community were a High School or Middle School – there would be less traumatic memories for some… If the SQL Server community were a lot of workplaces, new employees would be more productive. Again – the folks in this community pour themselves into others and want to see new folks learn as much as possible and they love helping them out there.

So yeah, I’m thankful for this SQL Server community I fell into when I got deeper into SQL Server. I have some friendships that will last well beyond my time working with any one technology. I’ve seen projects touch lives (Like the Deep Dives books, Project Phoenix, etc.) and I’ve seen resources put together to get stuff done. This SQL Server community is pretty amazing.