Realjenius.com » Journal

My Experience with a MediaTemple (ve)

R.J. — Thu, 05 Aug 2010 02:51:01 +0000

I just switched RealJenius.com (and a whole crap-ton of other sites I host) from a MediaTemple (dv) 3.5 to one of their new MediaTemple (ve) servers. This was kind of an ideal move for me, as I’ve been using Ubuntu as my primary development desktop for months, and with my recent career change, I’ve been doing a lot more system administration work (on Ubuntu VMs none-the-less).

If you’re as lured by the (ve) as I was, let me give you some details from my experience:

Provisioning is lightning-quick. I ordered the server, and it was available within 5 minutes.
It is truly as they say: just a Linux box with SSH. For my install, I was given a stripped down Ubuntu 10.04 server instance and told “Go Play!”.
For that reason I would recommend, if you are migrating from somewhere else, that you practice the migration on a VM somewhere, like with VirtualBox. Since you’re responsible for the “whole shootin’ match”, it’s in your best interest to have everything inline to make your transition as seamless as possible.
Post-install configurations are everything! I have been using pre-packaged VPS solutions for so long, I forgot that they spend a lot of time tweaking and tuning software to fit in a box the size they choose. Software like PHP, Apache, MySQL, Java, Tomcat, etc – all comes with default values and selections that, I guarantee you, are different than most VPS’s are running.
Aptitude is your friend! I chose Ubuntu because I knew how blindingly simple the package manager was to use. The installation of Apache, PHP and MySQL support (while not tuned) was literally only bound by the time to download the packages. My job was to sit and watch it work.

I chose to do a number of things to tune my installation after I got it up and running. There are a lot of things to consider when you are your own sys-admin!

Tuning Apache Memory

First – tell Apache to calm down on memory usage. The default Apache install will use massive amounts of room in each thread stack (mostly because Linux tells it to). In reality, unless you’re expecting to handle some pretty impressive load while shipping some pretty impressive HTML, you probably don’t need 8MB per stack. To detune Apache in Ubuntu 10.04 to use less memory, you can edit ‘/etc/init.d/apache2′ and put a stack-ulimit adjustment at the top of the file:

1	ulimit -s 256

Lowering my stack for Apache to 256k (which coincidentally is Java’s default stack size), lowered my memory usage for Apache and whatever else was on the box at the time (running standard pre-fork) from 750MB, to 280MB. Now, that includes virtual memory and everything else – but when you’ve only got 1GB to work with, that’s a pretty significant savings.

Switching Apache’s Multi-Processing Module

The next thing I did to trim the Apache fat was to switch to mpm-worker threads instead of mpm-prefork. Now, most folks may comment here that mpm-worker actually uses more memory at idle, and while that’s technically true, it is much more predictable and scales much better from a memory perspective (think threads vs. processes if you are a Java or Ruby developer). It also generally happens to be faster, which is a nice benefit.

Unfortunately, when you move away from pre-fork, you also move away from easy PHP installations. To get PHP to work with mpm-workers, you have to use something like FastCGI, which acts as a PHP worker process pool that runs outside of the Apache process. Now – I made this a lot harder on myself by trying to host sites outside of /var/www – I wanted all of the folks sites I host to be under their respective home folder: /home/guy-who-mooches/guys-site.com/yadaydayada (just kidding people who I host!). I’ll point you to several articles on the net about it:

All of these sites have different (albeit similar) approaches. The key things to double check in this effort are:

Install apache-suexec-custom (the ‘custom’ is key!). Some sites suggest that you have to compile your own apache-suexec install; this is no longer true (that’s what the custom is for).
Leave Apache running (or make Apache run) as ‘www-data’ – I made the mistake of switching it to the user apache/apache, and getting suexec-custom to work with that user was a fool’s errand for me.
With 10.04, you have to edit the file in /etc/init.d/apache2/suexec called ‘www-data’ and change the first line from ‘/var/www’ to wherever you want suexec to be allowed to run from (for me it had to be ‘/home’).
You’ll need to create a wrapper script for each user. Don’t use symbolic links. This wrapper script must have that user as the owner/group (whatever user is going in the suexec directive, anyway).
Pay attention to the properties you set in the wrapper script! They have drastic impacts on both performance and resource utilization. You need to have a feel for your site’s intended usage to be able to tune these.
The suxec directive and FCGI wrapper directives in the Virtual Host file vary in every article I read about this (there is more than one way to skin a cat) – I tried both the Action/AddHandler combo approach, and the FCGIWrapper approach. FCGIWrapper is simpler, but Action/AddHandler is more flexible. I’d start with FCGIWrapper to get it working, and when you are feeling more confident, try action/addhandler if you think you need it.

Anyway – rather than rehash a full walk-through, I felt it would be better to provide some bullet-points. Until I embraced understanding how FastCGI actually worked, I found I was struggling to properly implement it. So take your time, read the articles, try to understand the scripts, and come back here and read my tips. I think you’ll be better off for it!

Best Apache Tuning Tip

Many folks say the best Apache tuning tip is to use Nginx. That may be true – I wanted to move fast, and I know very little about administering Nginx. I know that I will have to do some .htaccess file conversion… so maybe in a couple weeks.

User Management

I highly recommend creating a non-root user for yourself. Giving that non-root user sudo access gives you enough of a barrier to be careful, but still gives you the flexibility you need to administer. In Ubuntu, sudo-ing requires that your user be in the “sudo” group. That’s as simple as:

1	usermod -G sudo [username]

Use Identity Files

This is a huge boon in my opinion. Identity files are much more secure than passwords, and can make your login process faster (if you’re willing to trade-off the security of a passphrase). Here is a great article about it: http://www.csua.berkeley.edu/~ranga/notes/ssh_nopass.html

Learn to SSH Tunnel

SSH tunneling is secure. If you know how to tunnel, there is no reason to want to install PHP My Admin for example. It’s a security risk (don’t believe me? scan the Apache access logs of any decently popular domain, and you will see hack-attempts against dozens of phpmyadmin common URLs), and it’s not a very awesome administration tool. Meanwhile, MySQL administrator and MySQL Query Browser are awesome and just need an HTTP port. Here is one way to do it:

1	ssh -L 1234:127.0.0.1:3306 myuser@myfakeserver.com

This forwards the port 1234 on your machine to port 3306 on your server (which is the default port for MySQL). Then, when you open MySQL Administrator/Query Browser, just try to connect to 127.0.0.1 on port 1234. Now you have full administrative access to MySQL (bells and whistles abounding) and you can be comfortable that the traffic is encrypted.

Tunneling with -D (dynamic SOCKS proxy) is also an excellent way to monitor a Java instance as well, using VisualVM and JMX (http://stackoverflow.com/questions/1609961/visualvm-over-ssh). Again, totally secure monitoring of a running JVM – hard to beat.

Conclusion

Media-Temple’s (ve) offering is excellent in my estimation. The servers are fast, the memory #s are phenomenal (especially considering you can run whatever you want), and you have total unadulterated control. Two-thumbs-up MT!

JRuby and Sinatra in 2 Minutes

R.J. — Sat, 17 Jul 2010 16:53:22 +0000

While at RubyMidwest I decided to explore Sinatra in more detail. I’ve spent a lot of time with Rails, and while I love it, there is something alluring about the simplicity of Sinatra (and, well… ooh shiny). Being a recovering Java developer (Hi, I’m R.J., and I haven’t developed in Java for 18 hours) I have a server that runs Java, and would like to be able to use Sinatra to build my fancy-awesome web-apps. On those lines, I want all of the shiny benefits of JRuby’s multi-threading awesome-ness, as opposed to just trying to use WEBrick, which does not a powerful server make. So here is a 2 minute tutorial (well, depending on the performance of your computer, and how fast you type) startup with Sinatra, JRuby, Bundler, and Glassfish.

I’m cheating already by assuming you already have JRuby installed as your default Ruby installation. No? Go get it!

Next step is to get bundler:

1	gem install bundler

Now we need to make a home for our application, and prep it for Bundler:

1
2
3

mkdir testapp
cd testapp
edit Gemfile

Here I’m creating a new file in testapp called ‘Gemfile’ in your favorite editor. This is where we will sketch out our dependencies for Bundler to do all the hard work for us – here are the contents for this example:

1
2
3

source :rubygems
gem "sinatra"
gem "glassfish"

Frankly, that’s it. We tell Bundler to look for gems in RubyGems core repo, and then we ask it to make sure we have Sinatra and Glassfish. Now we can create the program – create the file ‘hello.rb’, and use these contents:

1
2
3
4
5
6
7
8
9

require "rubygems"
require "bundler"
Bundler.setup

require "sinatra"

get '/hi' do
"Hello World!"
end

So what’s special for JRuby? Absolutely nothing. We do have special sauce for Bundler, (by calling Bundler.setup prior to the require for ‘sinatra’) but trust me – you’ll be happy you used it. You’ll also make @wycats happy.

And – that’s it! Now, if you were to start this file the standard (well, bundler-standard) way, we’ll see this:

1
2
3
4
5

realjenius$ bundle exec hello.rb
== Sinatra/1.0 has taken the stage on 4567 for development with backup from WEBrick
[2010-07-17 11:24:46] INFO WEBrick 1.3.1
[2010-07-17 11:24:46] INFO ruby 1.8.7 (2010-06-06) [java]
[2010-07-17 11:24:46] INFO WEBrick::HTTPServer#start: pid=44490 port=4567

…and we can visit this URL: http://localhost:4567/hi. But, recall that our goal was to work with Glassfish, not WEBrick. All that has to change (and for folks who has done Glassfish/Rails before, this won’t be a surprise) is to run this startup instead

1
2
3
4
5
6
7

realjenius$ bundle exec glassfish
Log file /Users/realjenius/Projects/testapp/log/development.log does not exist. Creating a new one...
Starting GlassFish server at: 0.0.0.0:3000 in development environment...
Writing log messages to: /Users/realjenius/Projects/testapp/log/development.log.
Press Ctrl+C to stop.

Running sinatra

This time, we’ll visit this URL: http://localhost:3000/hi, and if all worked as desired, Sinatra will be crooning away. Boom goes the dynamite.

Upgraded to WordPress 3.0

R.J. — Fri, 25 Jun 2010 02:37:23 +0000

I just completed the upgrade to WordPress 3.0. So far, the upgrade appeared to go seamlessly. Everything is running, no plugins are complaining, yadayadayada. If you have a problem on the site please be sure to let me know.

If you are a WordPress user, and want to see what has changed since 2.9, here is a lovely screencast that covers the particulars:

The Ides of March

R.J. — Sat, 13 Mar 2010 06:05:41 +0000

So Monday is my birthday: March 15th. For years I remember two things about the way adults reacted to me as a child: (1) Oh, your name is R.J.! That’s just like Dallas! (and for the record, no, no it is not). Or (2) Oh my! You were born on the Ides of March!

It’s funny, because when I was a kid, I had no idea what the ides of march was; I just knew it carried with it a certain degree of playful dread, given the reaction of adults. As I’ve gotten older, I’ve learned a lot more about what the day actually is, and not what people seem to attribute to it.

If you have no idea what I’m talking about, I’m not surprised. It seems like the superstition around the date has faded in recent years. Historically speaking, the term ‘ides’ meant the middle of the month for either Martius (March), Maius (May), Quintillis (July), or October on the Roman calendar. On the Ides of March, the Romans actively celebrated for Mars, the god of war (who, by the way, is awesome).

But the reason that we don’t go around talking about the ides of October is not because the God of the Month was less impressive, but simply because of the historical significance of 3/15. Julius Caesar (a rather important Roman) was murdered by the Brutus’, Longinus, and a bunch of other political turds.

So over the years, the day has carried a negative connotation. It has Dark Ominous Tones.

Now, the truth is, I am no Julius Caesar. Additionally, I find it highly unlikely that anyone will stab me to death. However, I’ve had enough people come up and chatter at me about the ides doom and gloom that I can think of a few to poke with the pointy end of a stick, at least threateningly.

Incidentally – you’ll likely notice that Quintillis is the only month that doesn’t have any aural relationship to it’s English counterpart. Well, it turns out that one of Julius Caesar’s many faults (for which he got the stabby stabby) was reforming the national calendar. The Roman calendar became the Julian calendar as he worked to resolve the listing that occurred in the traditional model set up by Romulus, which had 304 days, and was then later refined by Numa (Numa) which had 355 days. The Julian calendar is identical to our modern Gregorian calendar in terms of length and usage of leap years, but was simply not shifted to match the same days of the year.

When Julius Caesar was murdered, they renamed Quintillis to Julius, and then bippity-boppity-bacon you hava July.

So anyway, that’s how I began to call myself the Roman god of war. Don’t make me smite you.

Check Out the Play Framework

R.J. — Mon, 01 Mar 2010 16:54:10 +0000

I’ve spent a lot of time recently investigating a variety of languages other than Java, such as JRuby and Scala, and truly believe from these experiences that traditional Java MVC web frameworks are inherently flawed in design and implementation. The effort involved in implementing on a framework like Struts or Spring MVC is astronomical, especially if you are going to implement things “the right way”.

It’s amazing to me how much these platforms push “hello, world” examples that are simply not realistic web applications. After trying these short examples, developers turn around and start trying to implement a complete application, and this simple example balloons into a mess of code, and that’s without any real functionality yet in the application. A co-worker of mine is a fan of saying “[These frameworks] make the simple things trivial, and the hard things impossible”.

Historically, I’ve been known to say “If you are doing web-development in Java, use Wicket“; this was based on the fact that to my experience Wicket took the most advantages from the strongly-typed, and strongly IDE-supported, Java language, as opposed to trying to hide them behind anemic and broken templating languages that have horrid editors and basically trade one problem for another.

Recently, however, I spent some time doing some significant development with the Play Framework. I have to say that I think the Play Framework has eclipsed my Wicket fever. That’s not to say that I don’t still think Wicket is very powerful, but I have been particularly impressed with the feedback loop provided by Play. It has, without a doubt, the most direct code-test-cycle I have seen in any platform for Java (it approaches the instant feedback of Rails), and also has the distinct advantage of being stateless out-of-the-box (something Wicket is definitely not).

Play manages this feedback loop problem in a rather novel way – embedded in the framework is the Eclipse compiler for Java (ECJ). This means that when you’re coding for the play framework, you’re not sending it your class files, but rather your source files. This allows Play to recompile code in a running instance on the fly – I literally only restarted my application a handful of times while I was coding over the course of several days. It also integrates seamlessly with IDEs, and ships with an embedded HTTP runtime (no deployment is necessary during development).

There are a number of other benefits Play can provide by working with source files instead of class files. Much like Rails ability to add functionality to your application at runtime, Play can (and does) pre-process certain Java classes to add functionality.

I was further heartened to see that the next release of Play is meant to fully support Scala, which would allow for other modern language features to be used with this highly interactive framework.

It’s hard to describe all of the neat features Play provides in a few hundred words, but I would highly recommend you check it out – they have a 10 minute screencast they sells it better than I can. While I’m still convinced Java (as a language) will be surpassed for an overwhelming majority of the web-development as the language continues to stagnate, this is a compelling framework for the Java platform as a whole, even if Java isn’t your language of choice.

Fistful of Awesome: IntelliJ Open-Sourced

R.J. — Thu, 15 Oct 2009 20:39:00 +0000

In a surprising move, the JetBrains team has decided to open-source the JavaSE portions of IntelliJ IDEA 9.0 and beyond under an Apache 2.0 license. They will begin offering two products, a community edition, and an ultimate edition:

Starting with the upcoming version 9.0, IntelliJ IDEA will be offered in two editions: Community Edition and Ultimate Edition. The Community Edition focuses on Java SE technologies, Groovy and Scala development. It’s free of charge and open-sourced under the Apache 2.0 license. The Ultimate edition with full Java EE technology stack remains our standard commercial offering. See the feature comparison matrix for the differences.

Very cool news! The impact in competition for other IDEs (namely Eclipse and NetBeans) remains to be seen, but this certainly brings another aggressive (and already well-liked competitor) to a broader market.

WordPress 2.9 Beta Testing Call to Arms

R.J. — Wed, 14 Oct 2009 01:44:23 +0000

“For a limited time only – you can beta-test WordPress 2.9 from the comfort of your own installation!”

In the “another reason I like WordPress better for blogging” category, the folks over at WordPress.com have published a detailed article on how WordPress users can get involved in the beta-testing of WordPress 2.9 – the next major release. Included in the options for aspiring beta-testers is a plug-in that will automatically upgrade your site to the new version, and will keep you on the up-and-up.

To make is as easy as possible for you to get a beta testing install up and running we have put together a small WordPress plugin which makes it really easy to convert a test install of the latest release version of WordPress into a beta test install of the next up and coming release. The plugin is called WordPress Beta Tester and is available to download from WordPress Extend or can be installed using the built-in plugin installer. Please make sure you to only install this plugin on a test site, as we don’t recommend running beta versions on your normal live sites in case anything goes wrong. You can read more about the plugin in “Making it easy to be a WordPress Tester”

I don’t want to suggest that Drupal doesn’t work for transparency in their development, or that they don’t encourage community participation; but certainly, they don’t package it up as nicely as the WordPress folks have right here. While I won’t be upgrading RealJenius.com until final drops (I did consider pulling a Picard maneuver and upgrading now), it’s nice to know that the option is there. The release schedule for 2.9 has been revisited in this entry as well.

We are aiming to release the first beta version of 2.9 around the end of October, once we have put the finishing touches on the new features, and then we switch to full on beta testing mode and your help and feedback will be very much appreciated. During the beta program will push out new builds for automated upgrades regularly and once we feel that a suitable level of stability has been achieves we will release a release candidate, and we hope to be able to make the final release 2.9 build available in either late November or early December.

What Am I Downloading Today, Eclipse?

R.J. — Fri, 09 Oct 2009 00:41:43 +0000

With the release of Eclipse 3.5, the plugin installation and update manager was completely revisited once again; the update process was reorganized, and the strengths of the p2 provisioning framework were surfaced. It’s nice to be able to hop in, download updates, and go.

However, I think Eclipse, as a product, still has a way to go.There are a number of products I use everyday that frequently require or recommend updates and upgrades. Some good examples include:

…and, of course, the list goes on. All of these apps have different ways to handle notifying and installing these upgrades.

Of course, both Windows and Mac OS X have built in update mechanisms, the former being the ever-controversial “Windows Update” tool, and the latter being Mac’s Apple Software Update (which has also recently been pissing people off). Firefox and Songbird use their own update mechanism that’s part of the Gecko platform. Most Mac users are also probably aware of Sparkle (or at least have seen it in action), as many Mac software distributions (like TextWrangler) use it as their update mechanism.

Your mileage may vary with the different implementations; some are more informative than others, and some are more reliable than others. But one thing can generally be said: each of these mechanisms tie in with documentation explaining what was updated, corrected, or otherwise tweaked. Some tools, like Sparkle, embed the update information in the notification dialog. Other tools, like the Firefox notifier, provide a link to the content that takes you to a rich and user-friendly webpage.

Firefox has thorough documentation online

Today when I came in to the office I decided to perform an update check in Eclipse, and was presented with a number of plug-ins that had newer replacements. At the time, I didn’t grab a screenshot, but here is an example from a brand-new download of 3.5, which has been since supplanted by 3.5.1:

So, is this a good thing?

Unfortunately, this doesn’t show off the obscure list of plug-ins my installation did earlier today, but there is still plenty I can pick on. So, what’s wrong with this picture?

I have no information on what is being changed or improved in this update.
The ‘Details’ section is suspiciously empty.
The version number is unnecessarily cryptic for end users.
The ‘More’ button simply gives me the description, copyright, and license for the plugin/feature being shown, which in this case is basically empty boxes.
The version number is cryptic for an end user.

Now, as an Eclipse-enthusiast, I have no problem deciphering what is going on here (particularly not since I had to contrive this picture for the blog entry), however as an end-user product, this could use some work. More information could be provided, condensing what was actually improved or fixed in this release (any information at all would be nice).

I understand that Eclipse is a platform first, and a product second, however I think this is one area that could use improvement.

Danny Carey at Drum Explorers Clinic

R.J. — Fri, 02 Oct 2009 20:44:56 +0000

As most of my close friends know, while having only recently picked up any form of drum stick, I’m passionate about drumming (though I’m personally still a beginner in my ability to play). One of my favorite modern drummers is Danny Carey; I’m sure in no small part due to Tool being one of my personal favorite bands. Even if I attempt to exclude my bias, however, I can see that Carey is a well-admired drummer, generally somewhere in the top 15 of top-100 drummer lists of all time; sharing space with the likes of Neil Peart, John Bonham, Keith Moon and Mike Portnoy.

Danny Carey has a rare technical accuracy in his playing, but he is able to combine it with his singular artistic flair to create remarkable tapestries of sound. Tool songs are initially accessible to listeners in search of a driving rock sound, but continue to breathe and grow as the complex rhythms unravel over time.

I’m always amazed to pick up an album like Undertow (from 1993) and to this day still hear new poly-rhythms in the beat and other minor adjustments for the first time. And it only takes listening to a song like Jambi, where the guitar, vocals, and drums are all on entirely different time signatures to be able to understand how capable he really is.

Recently, Danny Carey came into Kansas City for the 25th anniversary of KC Drum Explorers. He gave some tutorials, played along to some particularly drum-intensive Tool songs, and also had some great company: Terry Bozzio (another brilliant drumer) and Aloke Dutta (a wizard of the tabla, and mentor and teacher to both Carey and Bozzio).

I have no idea how I missed hearing of this event; it was mere minutes from my house (in a high-school auditorium in suburban Overland Park of all places), but thankfully YouTube has come to the rescue to make me feel like something other than a total failure.

I’ve dug up some videos of the event, and included them here (thank you evo462, jdub816, and andrewl85). Enjoy!

JRuby 1.4 Plan Details Discussed

R.J. — Mon, 21 Sep 2009 19:52:29 +0000

The JRuby team has posted some key JRuby 1.4 details over at the Engine Yard blog.

Of high-level note:

Ruby 1.8.7 is the new Baseline – This promises a number of library changes and backports from 1.9. To see the 1.8.7 changes you can look here.
Ruby 1.9 Compatibility Improvements - As mentioned in the blog entry, Ruby 1.9 is a moving target, however they are moving a lot closer to having major libraries working as desired. Some high-visibility features (like Fibers) are still on the pending list, however.
New YAML Parser - Ola Bini re-visited the YAML parser in Ruby recently (he has blogged about the creation of the new parser and how he ported it). The new parser, Yecht, is said to be completely compatible with Syck (the Ruby parser), warts and all.
Java Method Dispatch Improvements - One of the major promised features of 1.4 for quite a while has been improvements to the Java integration; they have been wanting to formalize this portion of the language for sometime. This involves calling Java methods, calling overloaded Java methods, the coersion of types (Java string <–> Ruby string), etc. Charles Nutter has posted a very detailed, developer-centric breakdown of the plans in this integration on the Developer mailing list (which goes beyond what’s available in the blog entry). Certainly an interesting read.
Java Class Generation via JRuby - JRuby will finally be able to construct runtime-available Java classes on the Java classpath. This will be available on-demand only, however that could be quite handy in a number of difficult integration scenarios.

This set of features is exciting to me, particularly for the Java integration features. The current Java integration functions, but can be somewhat tempermental, and can end with surprising results at times (I have learned this the hard way via hand-coding Swing and SWT apps).

Additionally, one of the killer features of Scala, in my opinion, is its ability to interact seamlessly with Java libraries at compile-time (create a class in Scala, and code against it in Java). While the semantics defined for JRuby are more runtime-centric for now, it does provide an ability to construct something that can be fed into Java APIs at runtime that expect certain “inferred” contracts to exist (such as the Jersey example they used in the original post).