So for Dating DNA & Clipish I put together the “Basic Server Administration Document” which detailed everything a person would need to know to perform basic sys admin tasks. It was nicked named the “Oh Crap” document. Fortunately, while I was gone, there were no problems, and the document was not needed.

For my other main project, CEVO, that was not the case. CEVO has 4 times the servers that Dating DNA has, and is a bit more complex. But there are two other programmers who also monitor the servers, so I wasn’t worried. Then, while I was gone, half the physical servers for a Virtual Server “Grid” went down, which was my pet project. The CTO for CEVO, Eric, had to try to figure out what was wrong and co-ordinate with the Data Center to figure what happened and how to fix everything. Since we host our Databases on this “Grid”, if it goes down, everything practically goes down. At the time, I was in Glacier Bay, watching Glaciers crumble and fall into the ocean, with zero internet connectivity.

So thank you Eric for putting up with everything while I was gone. It was all fixed by the time I came back.

But if there is anyone wondering if an Alaskan Cruise is worth it, it definitely is. My father-in-law took us, which was very nice of him. I’ll probably post some pictures in the near future of the trip.

Related posts:

1. Speaking: Utah Open Source Conference 2009 I will be speaking this year at the Utah Open...

Related posts brought to you by Yet Another Related Posts Plugin.

How I’ve gone this long without setting up Nagios to monitor servers is beyond me. However, I was tired of my clients and so finding out a server was offline before me. So I threw up a 256MB virtual server on Slicehost, since I don’t have any servers with them at the moment, in case a data center went offline.

I put Ubuntu 10.04 LTS on the VM, and I followed the instructions found here on Linode:

http://library.linode.com/server-monitoring/nagios/ubuntu-10.04-lucid

It worked pretty well, but I think in the future I will just install the packages that come with the server distro. The reason being is there were several checking tools, like check_mysql and check_nrpe that were not installed because I didn’t have some dependencies installed.

Now after a few hours I have 9 servers being monitored with about two dozen services being monitored. I also found out that one server had a 96% full HDD, so it already is paying off.

On another note, for those who know me and are curious, I’m in full swing in finding sponsors for this years Utah Open Source Conference. I’ve talked to quite a few, just need to finalize and close their deals. If you know of a company interested in sponsoring this year’s conference, let me know (check the contact me link up above).

Related posts:

1. The Cloud to Bare Metal A Little Background It was the summer of 2008 when...

Related posts brought to you by Yet Another Related Posts Plugin.

If you have ever been interested in creating, improving or just participating in an open source project, here’s your chance!  Do you already have an free and open source software project you are working on?

Come On Down!

At #hackUTOS, there will be developers, designers, documenters, testers and more looking for something to get involved in. Come down and learn how to get involved in an open source project!  Even if you aren’t a developer, there are needs for documentation, testing, promotion and more.

hackUTOS – A Code Festival (RSVP Requested)

April 24, 2010 :: 9am – 6pm

Open Source Technology Center at Novell (map)

#hackUTOS on irc.freenode.net

free admission, free software, free SWAG and free food sponsored XMission and Stackable.com!

More details at the UTOS Wiki.  See our previous posts as well.

Sponsored By:

See you all there!  It’s going to be a blast!

Cheers,

Clint

I don't believe I will be able to attend, but if you are are going to be around the Provo area, and are interested in getting involved in an Open Source project, check it out.

Related posts:

1. Dear 2010 – My Plans for Next Year Update: I started to write this post on the 31st,...

Related posts brought to you by Yet Another Related Posts Plugin.

The Problem

When creating virtual machines, especially virtual linux servers, I want them to be able to do the following things:

1. Connect to the Internet
2. Connect over SSH from my Host Machine to the Guest Machine
3. Have the servers connect to each other on a private internal network

Out of the box, a VM is configured for #1, however it cannot do #2 and #3. There are some solutions that do some archaic port forwarding by issuing special commands from the Terminal. This is just painful, and if you just want one port to work, that's manageable, but what if you want ssh, ftp, myql, http, https, etc? Any on multiple VMs? What a nightmare! What is painful is when googling for a solution, almost every "answer" is this crazy port forwarding method. There is a much cleaner, better way!

Solution - Host to Guest Connectivity

Go into your VM's settings and go to "Network", it will list options for 4 different adapters. Instead of hacking one adapter to do all the work, just setup more adapters for your configuration. For adapter #2, set it to Host-only. This enables an IP range for communicating from the Host to the Guest and vice versa. Start up your VM, and if you are using something like Ubuntu, edit your /etc/network/interfaces file so it looks like this:

After adding these lines, run the command:

Now VirtualBox is configured to host this traffic over the 192.168.56.x IP range (I'm not sure if/where you can change this). So do an ifconfig on your guest machine, and it should show you the IP it has been assigned. If you want, you can change your /etc/network/interfaces so it can be statically assigned. You should now be able to SSH into your box on it's IP:

Solution - Guest to Guest Communication

Now, what if I have a Cassandra server, a Web Server, and a MySQL server each on their own VMs. What you do is create a third adapter set to "Internal Network". Then in your /etc/network/interfaces file you set eth2 to a static IP of your choosing. I typically use the 10.0.0.x range. So each server is respectively 10.0.0.10, 10.0.0.20, and 10.0.0.30. However, over the 10.0.0.x range there is no DHCP, no Gateway, no nothing, just direct communications over these IPs.

Conclusion

So please, for the love of everything good, don't resort to crazy port forwarding, complicated XML file editing, solutions for VirtualBox. Just use each type of interface configuration for what you need. Thank you!

Related posts:

1. The Cloud to Bare Metal A Little Background It was the summer of 2008 when...

Related posts brought to you by Yet Another Related Posts Plugin.

It was the summer of 2008 when I attended the LT Pact conference that I first drank the "Cloud Kool-Aid." As you can see from my notes, I was pretty excited about this whole concept. Excited enough, one client of mine signed up for Layered Tech's grid layer (which is what they were demoing at the conference).

One interesting thing I heard at the LT Pact conference was one of their keynotes was as IT Researcher for Gartner. He outlined what phases the current industry would go through with "cloud computing":

• First, several early adopters would have great success implementing the technology.
• Second, mainstream hype would "over-hype" cloud computing.
• Due to the hype, many people would start "using the cloud" as a silver bullet, thinking the cloud would be perfect for anything.
• A "cloud crash" would occur where disillusioned IT managers realized the cloud had its drawbacks, and wasn't the perfect solution for any problem. More over-hyped people will start to move away from the cloud.
• After the dust had settled and the hype faded away, the actual cloud technology would find a common place in today's IT.

Currently I think different people and groups are at different stages of the last three listed above. Some are still raving and hyping the cloud for everything. Some are bitter about their disillusionment. Many are now looking beyond the hype at the real concepts behind "cloud computing" and finding "the services are wildly different. While many parts of Web hosting are pretty standard, the definition of 'cloud computing' varies widely." (InfoWorld)

After a few hours, the fog of hype starts to lift and it becomes apparent that the clouds are pretty much shared servers just as the Greek gods are filled with the same flaws as earthbound humans. Yes, these services let you pull more CPU cycles from thin air whenever demand appears, but they can't solve the deepest problems that make it hard for applications to scale gracefully. Many of the real challenges lie at the architectural level, and simply pouring more server cycles on the fire won't solve fundamental mistakes in design.

In a rebuttal blog post, Don MacAskill defending accusations that EC2 was over 50% slower than stated.

... let me explain what I think is happening: Amazon’s done a poor job at setting user expectations around how much compute power an instance has. And, to be fair, this really isn’t their fault – both AMD and Intel have been having a hard time conveying that very concept for a few years now...

Bottom line? EC2 is right on the money. Ted’s 2.0GHz Pentium 4 performed the benchmark almost exactly as fast as the Small (aka 1.7GHz old Xeon) instance. My 866MHz Pentium 3 was significantly slower, and my modern Opteron was significantly faster.

So what about that guy with the Ruby benchmark? Can you see what I missed, now? See, he’s using a Core 2 Duo. The Core line of processors has completely revolutionized Intel’s performance envelope, and thus, the Core processors preform much better for each clock cycle than the older Pentium line of CPUs. This is akin to AMD, which long ago gave up the GHz race, instead choosing to focus on raw performance (or, more accurately, performance per watt).

My Experiences

Virtualized Servers can be awesome. They have some great advantages, while also having clear disadvantages (if you look beyond the hype). While I know I've quoted a lot about Amazon EC2, I have only experimented with it. However, I have a good deal of experience with 3Tera's AppLogic, and have recently moved a website to Rackspace Cloud.

Learning Curve

While Rackspace Cloud was more straight forward, learning 3Tera's AppLogic is a large task. I spent several months learning a lot (and tearing my hair out some days) and then another year of learning to deal with it's quirks. It is pretty cool once you get it down, but in all honesty its features didn't line up with what the company using it needed. Every now and then we spin up a custom server for a website or something else. However, AppLogic is built more for people who are doing a large volume of similar instances. So while it works, it probably isn't the best fit for us.

Also, with AppLogic's components, they are engineered for very very specific tasks. Doing things outside of their scope *is* possible, however there was a vast amount of tinkering, reading, blood, sweat, and tears trying to get it to work. There are still things, like with the way AppLogic handle network routing, that are just unique at best.

Bare Metal to The Cloud

Each instance I've gone to deploy on the Cloud, either Rackspace Cloud, Amazon EC2, or AppLogic, the project itself had started on a dedicated server that we then moved to a virtualized server. Since when virtualized servers started to pop up, even in the earlier days of simple VPS solutions, you would be moving from dedicated bare metal to a virtual server. The inherit problem is that virtual servers, especially on shared hardware, have *less* resources available to them, especially CPU power. On Bare Metal Dedicated Servers, I rarely run into CPU issues. When the CPU power is being shared with several other "virtual servers", you suddenly only have a quarter of what was available.

Then you have your neighbors. While moving Clipish to the Rackspace Cloud the little 1GB RAM server felt peppy. I was confident in our choice. Clipish isn't complicated, and it's web services are pretty easy going. The only intense part is when you clear the image cache. Then when a web server request comes in and the image isn't there, it pulls the image from the Database, performs the ImageMagick operations to resize, watermark, create a thumbnail, etc on demand. During the early morning, it isn't a problem at all. However, one night I "cleared the cache" during peak times, and within minutes the server had locked up.

Turns out on our old server, the web services could easily burst to all 8 cores for a few minutes while regenerating hundreds of image caches. Even the Rackspace server could burst during off-peak hours and repopulate. However, during peak times, when the dozen or so "neighbors" we're using their allotted CPU, and our server couldn't "burst" like before. So what did we have to do? We wrote a script that would loop through our image library one at a time and regenerate it. This works, but after the move and we were experiencing these problems, we thought "did we make the right move to go to the cloud?"

Cloud to Bare Metal

I think the approach with the "cloud" needs to be this. You have your baby project, so throw it up on the cloud. Let it cost $15-$20 bucks a month. Take advantage of the daily backup snapshots, etc, and just worry about development. Let it grow and at some point you're going to hit a fork in the road:

• Bare Metal Benefits outweigh Cloud Benefits - Bare Metal is typically cheaper than Cloud power. If you are not spinning up and down servers on a daily biases, but is a more fixed size, then quickly scaling isn't an issue. Cost can and should be a huge priority for hosting your application. If your application needs more higher CPU, Memory, or Bandwidth that doesn't fit the Cloud model then going Bare Metal is a good idea.
• Cloud Benefits outweigh Bare Metal Benefits - If your application, however, can fit on the cloud and the budget at the same time, then it can be a beneficial to stay on the cloud. Just remember you can't expect a Cloud server to outperform a Bare Metal server of the same specs. But you can take advantage of things like spinning up new servers in minutes, take quick backups of virtual machines, etc.

Conclusion

I guess my biggest advice when it comes to deciding on bare metal vs cloud is to understand the pros & cons. Remember that while you can burst to more CPU, you shouldn't rely on that always being the case. The Cloud does afford some cool flexibility, but it isn't perfect, and it can be a lot more expensive than going bare metal. Also, if you're on bare metal, and moving to the cloud, the cost of raw power is a great deal more expensive on the Cloud.

Related posts:

1. Setting up Nagios for Servers Ok, so I need to confess, I haven't been a...
2. VirtualBox Networking Setup Every time I go to set up VirtualBox and several...

Related posts brought to you by Yet Another Related Posts Plugin.

So I went ahead and logged into my BestBuy account website. Sure enough, I only had 4,386 points worth about $87.00. Its amazing what a two digit difference can make. I wonder how many complaint phone calls BestBuy will field due to this error. I wonder which developer is to blame, and if he is going to get an ear full for this mistake. Personally, I always get very nervous when sending out mass emails. Especially when putting dollar amounts on them.

Maybe I should call up and see if I can get them to honor their email.

Related posts:

1. My iPhone Has Hindered My Email What!? What is this that I speak of? My very...

Related posts brought to you by Yet Another Related Posts Plugin.

Then I started to think about how people make changes in their lives. When I just focused on not drinking coke, it wasn't easy, but totally possible. They say it takes about 21 days to form a habit. So instead of listing out a dozen goals at the beginning of the year, why not tackle a new one each month? Focus on one habit you want to create or change, spend the first 21 days creating it, and the remainder of the month keeping it. By the time the new month rolls around, your new habit is set and ready to go, and time to tackle a new habit.

One reason I believe setting one goal per month is typically one goal requires multiple changes. With giving up coke, I had to figure out what to replace it with. That means I had to keep our Brita pitcher full of water, which I had to make sure we had enough filters. When night time server emergencies occurred, or I had to drive late in the night, how could I get some caffeine?

So February I've made the goal to work out at least four times a week, hopefully five. That required me to get up earlier in the morning, which in turn means I had to go to bed earlier. I made sure I knew where my workout DVDs were at, that I pulled the weights out, and I had my workout clothes ready. That way, when my alarm went off, I could get up and go work out.

I can say, this week, I've worked out 3 days and tomorrow will make a fourth. Its not easy, but taking one habit at a time makes it totally possible. So from now on, each month, I'm going to tackle one habit. I have a theory, that the ability to make and change habits is an actual skill. Maybe down the road I'll be able to make two, or even three habits a month. I'll keep posting updates with how things are going.

For March, I think I'm going make it a habit to bring my lunch to work each day, which will help me eat healthier and save some money. My suggestion to anyone who wants to make some changes in their life, take it one habit at a time each month.

Related posts:

1. Dear 2010 – My Plans for Next Year Update: I started to write this post on the 31st,...

Related posts brought to you by Yet Another Related Posts Plugin.

Well, Facebook has made their announcement, and it is called HipHop. Here is an excerpt:

Today I'm excited to share the project a small team of amazing people and I have been working on for the past two years; HipHop for PHP. With HipHop we've reduced the CPU usage on our Web servers on average by about fifty percent, depending on the page. Less CPU means fewer servers, which means less overhead. This project has had a tremendous impact on Facebook. We feel the Web at large can benefit from HipHop, so we are releasing it as open source this evening in hope that it brings a new focus toward scaling large complex websites with PHP. While HipHop has shown us incredible results, it's certainly not complete and you should be comfortable with beta software before trying it out.

HipHop for PHP isn't technically a compiler itself. Rather it is a source code transformer. HipHop programmatically transforms your PHP source code into highly optimized C++ and then uses g++ to compile it. HipHop executes the source code in a semantically equivalent manner and sacrifices some rarely used features — such as eval() — in exchange for improved performance. HipHop includes a code transformer, a reimplementation of PHP's runtime system, and a rewrite of many common PHP Extensions to take advantage of these performance optimizations.

Marco Tabini has a nice summary on what PHP need to know about HipHop. However, people I've talked to have had several opinions on HipHop, ranging from pure delight to doubts to indifference. Personally I think this is a great thing for PHP, and I'll tell you why.

HipHop will enabled business to pick a web scripting language and use it from start-up to internet-giant. While HipHop isn't the first tool used to speed up PHP, such as APC, Memcached, and others; it is more of a current reminder: PHP is serious about the web. For people already using PHP, this is just a nice feature. For those who aren't using PHP, it shows that it is a very strong candidate.

I've worked in a non LAMP shop before. To non-PHP people, it has the stigma, regardless of actual performance, of being hard to scale. It is a "scripted language", and in the Java / .NET world where I was working, it was looked at as being an absurd choice. Now, PHP can make a strong case of being able to mature with your application and scale appropriately.

One fear that I have as a Project Manager, and anyone has who decided on the technology that they will implement, is "picking the wrong team." During my .NET days, Microsoft's team announced a new technology for .NET applications. It showed a lot of promise and simplified some things that were a pain in .NET. Our team adopted the new technology and ran with it. It wasn't perfect, but we liked it. Less than a year later, Microsoft announced that it would be moving the project to a drastically different team, which basically meant "there wasn't enough adoption, so we're putting this project on the back burner." The new team was basically to maintain the code forward, but not develop new features.

What happened is we invested on this new technology, and we saw the potential it would have in the future. We lost our investment, since that future potential would never be realized. Now our projects had this new technology embedded in many spots. Re-writing those places would take a great deal of time. From a Project Manager perspective, it was a poor choice.

For PHP, the emergence of HipHop, a concept that is working across thousands of production servers, it sends a strong message to the web development community as a whole. Not only is PHP widely adopted, flexible yet powerful, and easy to implement. PHP is competitive, and is continually evolving to adapt to web developer's needs. It is a safe bet for many projects, and it isn't going anywhere but up.

Will I be implementing HipHop right now? Probably not. Ilia Alshanetsky has a very healthy reality check on what HipHop is for. But the bottom line is I don't need what HipHop is offering yet. I can see very great uses in the future for Dating DNA's compatibility generation, as well as certain pieces of other PHP projects that could use a serious performance boost. But for now, my Blog will not be HipHoping along. I'm excited for not only what HipHop does, but what it's greater underlining meaning as a whole with PHP.

Related posts:

1. So You Want To Be A Web Developer? <tangent> This last week I've been talking with my brother...
2. Speaking: Utah Open Source Conference 2009 I will be speaking this year at the Utah Open...
3. #hackUTOS – April 24, 2010 I've been busy with several blog post floating around in...

Related posts brought to you by Yet Another Related Posts Plugin.

Dear Google Apps admin,​

In order to continue to improve our products and deliver more sophisticated features and performance, we are harnessing some of the latest improvements in web browser technology. This includes faster JavaScript processing and new standards like HTML5. As a result, over the course of 2010, we will be phasing out support for Microsoft Internet Explorer 6.0 ​as well as other older browsers that are not supported by their own manufacturers.

We plan to begin phasing out support of these older browsers on the Google Docs suite and the Google Sites editor on March 1, 2010. After that point, certain functionality within these applications may have higher latency and may not work correctly in these older browsers. Later in 2010, we will start to phase out support for these browsers for Google Mail and Google Calendar.

Google Apps will continue to support Internet Explorer 7.0 and above, Firefox 3.0 and above, Google Chrome 4.0 and above, and Safari 3.0 and above.

Starting this week, users on these older browsers will see a message in Google Docs and the Google Sites editor explaining this change and asking them to upgrade their browser. We will also alert you again closer to March 1 to remind you of this change.

In 2009, the Google Apps team delivered more than 100 improvements to enhance your product experience. We are aiming to beat that in 2010 and continue to deliver the best and most innovative collaboration products for businesses.

Thank you for your continued support!

Sincerely,

The Google Apps team

For all Web Developers around the world, we should all be rejoicing! Having a large internet giant telling its consumers to "drop IE 6" and other older browsers will make all our lives easy. I remember while working on a very large site, we used PNG images sparely for some background images. The deadline was extremely tight, but we were (barely) on schedule. Out of the blue I get an email forwarded to me from the marketing director for the client, and it went something like this:

Why are all the backgrounds looking nothing like the mock? Its all a funny blue.

Of course, the Marketing Director had IE 6. Whats worse, is that he was working for a very large computer technology business. We had to drop everything and apply ugly IE 6 hacks to enable PNG support. At the time, I tried explaining that the browser he was using was released while I was still in highschool, but he didn't understand. Hopefully now the big internet giants will start telling people "Get off IE 6." Once people adapt better browsers, the more we can do with the web.

Related posts:

1. My iPhone Has Hindered My Email What!? What is this that I speak of? My very...
2. jQuery Tip: Better Toggle For many web developers, jQuery is the most awesome JavaScript...

Related posts brought to you by Yet Another Related Posts Plugin.

First, everyone does everything. Check your ego at the door. In the first startup I was with, I was in charge of adding reporting to the eCommerce application and taking the garbage out. Was it an effective use of my time? Not at all. Did it have to be done? Yes. Did we have the money to hire someone else? Nope. If your startup is renting office space, you probably won't have to worry about this... but who signs for packages? Who re-orders coffee? On the flip side, you could be the one taking technical questions at the next investor pitch or invited to the big customer Christmas party.

I thought he hit the nail on the head with the article. Over the last few years I've worked with three startups, one of which was successfully bought by a large corporation. The other two have successfully gained some good revenue, but are still in the "Looking for More Money" phase, and I continue to do work for them on a contractual basis.

My only concern was that at the end of the article, he only mentions that he is still okay with working for a startup. He didn't mention some of the biggest "perks" I consider working for a startup. So here are some of the items that, if you can handle the "draw backs", are some great things. These are not "guaranteed", as each startup is different, but typically you'll find some of these.

• Accelerated Learning - I've felt that I've learned at such a faster rate while working in a startup. In large development teams, you typically work on very specific things. Your team only works on one part of a larger project or product. However, with the startup, you typically have to do a lot more varied work. At Ambient, the company that was sold, I was a junior developer of a company of three developers. Here is the list of some of stuff I did in roughly one year:
  • Company Website
  • All Web Services
  • All Web Designs
  • Website Demos & Prototypes
  • Data Mining Services
  • Executive Dashboard
  • Sales & Inventory Reporting Website
  • Customizable Online DVD Rental System

  Not only did I work almost exclusively on these projects, they integrated with the pieces the other two developers worked on. For three developers, we cranked out a lot of great software. Since I was the junior developer, when ever we had a tech support issue that was a little too complicated for our two tech support guys, I had to diagnose and solve a lot of complicated issues. I got experience with VPNs, LDAP and Active Directory, very large databases (100,000,000+ rows), VNC & RDP, and automated deployments across thousands of remote locations, to name a few.

  If you enjoy getting your hands on a lot of different technologies, and do a lot more than in a normal company, you can really learn a lot at a startup.

• Less Politics, More Work - While I know this doesn't apply to all startups, the ones I've worked in seem to have a lot less politics than other companies. I believe its because, first of all, there are less people to have politics with. Second, there is way to much work to get done to spend time on playing politics. Everyone seems to just be focused on getting work done.
• More Input, More Direction - While working in a startup, I've found that I have a lot more say in how we do things compared to other companies. Creativity isn't just a bonus, it is a requirement. Many times you are working on an idea, not a detailed specification. Your creativity is required to make the idea fully functional and work properly. I've found many times my ideas are implemented, and while looking at the final product, I can see my influences all over the place.
• Closer Team, Fewer Annoying Co-workers - Some may think this is a myth, however what I have experienced is startups don't have time to put up with teams that don't work well together. For the exception of one guy who was really nice, but a major pain to work with, I've enjoyed working with all my co-workers and (I hope they can say the same about me). Typically people who don't enjoy the fast pace leave after a short while. If there is an employee who doesn't get along with the others, typically they will go find work elsewhere because the stress of the pace and not getting along well with others compound out of control. Even if they don't leave on their own, management will usually find some solution, because a team that doesn't work well together won't produce the needed products to make a success. Unlike a large corporation, there isn't anywhere to hide bad workers. If you're not carrying your own weight, you're gone.
• No Boredom, High Adrenaline - The pace can be stressful, but I found it most of the time exhilarating. There is never, ever a lack of work. Currently for the two startups I'm doing work for, there are years and years worth of work I can do. Sometimes the deadlines can be a killer, but meeting them is very rewarding. Its an awesome feeling when your team of two developers and one project manager create a product that out performs & demolishes competing products from large, well established companies.
• High Levels of Satisfaction - I personally feel I get more out of working for a successful startup. Since developers will produce more code and applications than in other companies, there is more to be proud of. If you're team is really tight, and a few of you create an amazing product, it is an amazing feeling.

So while startups are not for everyone, and they have their drawbacks, I believe certain types of developers can find rewarding work in them. I know I've been lucky to work on some excellent products, and I haven't had an experience (yet) where I've gotten burned. Maybe the things I've mentioned aren't as common as I think, but I truly believe startups offer opportunities for developers to really accelerate their learning and excel at rewards jobs.

Related posts:

1. Google Apps Dropping IE 6 This morning I found this email in my inbox. I...
2. PHP HipHop – What It Means There has been a lot of speculation about PHP, Facebook,...
3. Article: Why Developers Get Fired Here is an article that showed up on my RSS...

Related posts brought to you by Yet Another Related Posts Plugin.