kovshenin.com

Amazon Web Services: Moving to a New Region

kovshenin — Thu, 24 Dec 2009 14:49:11 +0000

I wrote about Optimizing Your Amazon Web Services Costs back in November, where I mentioned some of the upsides of Reserved Instances at Amazon, but haven’t mentioned any downsides, and here we are. Two weeks later Amazon announced the Northern California Region opening. I thought it wouldn’t differ from the Virginia data center, but still decided to give it a shot for a few hours.

Happy Holidays from Amazon Web Services

I didn’t do much benchmarking but hey, I’m running a Twitter app.. Foller.me, remember? This means that access times to the Twitter API are crucial, so I started off with some basic pinging, and the pings from California seemed to be a few times faster than the ones from Virginia. Next, I ran Xdebug and analyzed the cache grind sheets for a few requests to different profile pages. Sweet to know that 95% of the time taken to load a page is curl accessing the Twitter API ;) this means that my code is well optimized. The overall results in the California region was ~40% better than Virginia, so I thought of moving there. The problem was that I already had a 1 year contract with Amazon for an instance in the Virginia region.

I wrote to Amazon via their contact form and asked about reservation transfers from one region to another, of course with additional charges (the California region is slightly more expensive) and soon got a negative reply. They mentioned that reserved instances are not transferrable from one region to another but I can always cancel my reservation in one region and open up a new one in the other. They didn’t mention any refunds so I decided to ask, but soon, scrolling through their FAQ I found this:

Q: Can I move a Reserved Instance from one Region or Availability Zone to another?
A: No. Each Reserved Instance is associated with a specific Region and Availability Zone, which is fixed for the lifetime of the Reserved Instance and cannot be changed.

Q: Can I cancel a Reserved Instance?
A: The one-time payment for a Reserved Instances is not refundable. However, you can choose not to run or entirely stop using your Reserved Instance at any time, at which point you will not incur any further usage charges.

So I asked myself, why the heck would anybody want to cancel a reserved instance if they don’t get refunded? The conversation kept going on Twitter. Friends mentioned that I could purchase an additional reserved instance in the California region and then sell computing time on the one I have in Virginia, but that sounded too sarcastic. I felt unlucky and sad, and thought I thought should stick to the instance I had in Virginia. If only I had waited a few more weeks before making the purchase…

This morning I received another email from Amazon, stating that although they don’t usually do this sort of stuff, they got approval to process the cancellation with a refund just this one time, so now I’m free to reserve an instance in Northern California, happy holidays! Well, on Christmas Eve, this feels like a gift and I’m very excited about launching all my stuff in the new region, hopefully in January. So, thank you Amazon and Happy Holidays to all of you.

WordPress 2.9: From a Developer’s Point of View

kovshenin — Tue, 22 Dec 2009 10:11:43 +0000

Yup, WordPress 2.9 is finally out and I’ve seen a lot of people upgrade immediately – sweet thing to do, I did so too and hadn’t had any problems here (I’m not using too much plugins), then I switched over to the 3.0 branch, all good! The semi-square buttons in the admin interface are fixed by the way ;) Anyways, this post is not about my blog, but about something I’m working on locally in the intranet of a company. And here’s a quick reminder: do not upgrade the running instance before you checked everything on a duplicate testing one, especially if a bunch of plugins there are handwritten ;)

Upgrade to WordPress 2.9: Plugins Gone Bad

A Handwritten WordPress Plugin is a huge set of files, with hundreds of duplicate lines of code and tonnes of commented code. The MVC/MVP models are usually transformed into the AIO (All-In-One) model. The only version control systems used are the “Copy & Paste” and “Undo & Redo”. The Core Upgrade process does not involve reading the changelog, backing up, etc.

Okay this is not a very smart thing to do, but sometimes you just need to, especially when you’re in a hurry (duh!). I’m not going to go in detail about the plugins, because you wouldn’t understand it anyway (no offence, I don’t understand most of it either), but let’s just say that I was using the Post Tags interface that you see in your “Edit Posts” admin interface. The sweet little box where you can type some tags or choose from the most used ones. All I did was copy the HTML contents of that little box onto my page, and it used to work (in 2.7 and 2.8), but as soon as I upgraded to 2.9, bang! It suddenly stopped. The javascript bindings were gone!

I figured out that the bindings were all made in the post.js so I took a look at the diff between 2.8 and 2.9 – completely different. The major change was the jQuery selector from span#ajaxtag to div.ajaxtag (line 99 in post.dev.js) which prevented all the other bindings. A few more ids have changed to include the tax name, and some other minor changes which I didn’t figure out yet. Although what’s the point in figuring it out? Wait until 3.0 is released and rewrite the plugin again, meh!

I did a lot of thinking here. This was not the only interface that broke, and rewriting everything on every core upgrade would be pretty difficult. People talk a lot about WordPress upgrades, plugins compatibility, etc. But most of the time they refer to plugins developed by other people, which sooner or later will get an upgrade. So waiting for a few weeks before launching the core upgrade generally does the job. But what if it’s a completely different page with a completely customized admin interface?

I worked a lot with WordPress metaboxes a few weeks before the upgrade and today I figured out that the best and easiest way of building WordPress admin interfaces would be, that’s right, metaboxing. The standard components used in the edit posts/pages interface (tags, publishing, categories, custom fields, etc) are simple metaboxes. So why copy HTML if you could simply use the predefined ones? And when a change is on its way, you’d sleep tight. Unless of course you’re interacting with the standard components from your customized ones, which involves using certain selectors for ids and classes. In that case you’ll just have to read the changelog, tickets (over 500 for WordPress 2.9) and of course code diffs.

Creating your own custom metaboxes is not as difficult as it is at first sight (yet I still haven’t figured out how to save their positions via AJAX, but I will). You will get the hang of it after your third or fourth metabox ;)

P.S. Have you noticed the new post thumbnails here? They used to be wide images, now they’re h2 text and small thumbnail. I love WordPress shortcodes ;)

The Twitter API v2 Transition

kovshenin — Fri, 18 Dec 2009 13:38:26 +0000

It’s a mess around the current working copy of the Twitter API, there are more issues than functionality and the whole naming and renaming is a total disaster. Today for instance I tried a simple search query to the API and kept receiving “400 Bad Request” errors without any further explenation. As soon as I changed the address from search.twitter.com to api.twitter.com/1 (I got this from Abraham Williams’ php code), which is not clearly mentioned anywhere in the Twitter docs, everything started working fine.

Twitter API: Solving Compatibility Issues

But then I realised that the from_user_id field that’s being returned is far from the correct user ID. People in the Twitter Development Talk Google group stated this problem a few times (since March 2009 I believe). It seems that the “wrong” user IDs are meant for the second version of the Twitter API, thus cannot be used before it’s released. But wait! What the hack should I do with my app now? It’s not working y’know! Here you go:

$response = $oauth->get('search', array('q' => $search_query));
foreach($response->results as $result)
{
	$id = $result->id;
	$text = $result->text; 
	$user_name = $result->from_user;
 
	$user = $oauth->get("users/show", array("screen_name" => $user_name));
	$user_id = $user->id; // Get the old-style user ID
}

Yeah, that’s one extra API call, but it solves things temporarily ;)

I guess there’s nothing that we could really do right now, and it’s probably true that we’ll have to rewrite some parts of our code as soon as Twitter API v2 is released, but then again, what about the apps which stopped development? Will they stop working? Tonnes of Twitter clients and web apps still use basic authentication. Twitter mentioned that everybody must use OAuth these days, and that basic auth will be closed sooner or later.

Oh well, software comes, software goes. The best thing to do right now would be sign up to Twitter API Announce Google group and follow @twitterapi. By the way, @web2feed can now use the new features of the API to retweet messages based on hashtags and build user lists ;)

Cloud Tips: Amazon EC2 & Rejected Email

kovshenin — Wed, 16 Dec 2009 11:43:06 +0000

A few weeks ago I’ve setup my email in the /etc/aliases for user root (and the others) and started to actually read my root email from time to time (I wonder why I never did that before). Anyways, what bugged me straight away is that I had some rejected emails that were not being delivered, yielding the following errors (I removed some numbers):

Cloud Computing with Amazon Web Services

Deferred: 450 4.7.1 <domU.compute-1.internal>: Helo command rejected: Host not found
421 invalid sender domain 'domU.compute-1.internal' (misconfigured dns?)

And some others that looked alike. Tonnes of them, every four hours! The emails to other addresses were delivered fine though. I had WordPress notification messages delivered to my email, never lost a message. I also tried sending out a few using the mail command via SSH, everything okay. For a second I thought that maybe those addresses were simply invalid, but wouldn’t the server reply with an “Invalid recepient” error? Probably.. Here’s what I got from the Amazon Web Services support forums:

It seems that some remote mail servers complain about your server identifying itself in the SMTP dialogue as domU.compute-1.internal, while its external name is ec2.compute-1.amazonaws.com

Makes total sense. Perhaps some servers do try to see where the e-mail is coming from and of course the .internal domain is unresolvable (thus the “dns” misconfiguration error). I had to identify myself with an external, resolvable name. So I copied the external name into the /etc/mailname file and hmm.. Well, it’s been a week now and I haven’t received anymore delivery errors, so that must have worked.

The Twitter OAuth PHP Class Gets Even Better

kovshenin — Mon, 14 Dec 2009 11:55:54 +0000

Or perhaps simpler?.. Together with the Twitter API itself, the TwitterOAuth PHP class (the one by Abraham Williams) is being updated too! According to GitHub the latest changeset was commited on December 3rd so yeah, I tried to take a look at what’s going on there a few days ago and was quite disapointed. Disappointed with the fact that all my previous code was broken without giving any reason.

Just like everybody else, I never read the readme or other documentation files so I dug straight into the class code and examples. Soon after I realized that the new changes were not that bad, so instead of the usual 5 lines of code, I shortened it up to only one. I stopped worrying about parsing XML or JSON, converting them to objects, and I stopped typing in the full address for Twitter API calls. Abraham did all that for us, so all we have left is:

1
2
3

$credentials = $oauth->get("account/verify_credentials");
if ($oauth->http_code == 200)
	echo "Hey there, {$credentials->screen_name}!";

I’m not going to publish all the new features and stuff (read about them at GitHub), but hey, this is quite sweet isn’t it? The only drawback was having to rewrite some parts of the code I wrote for the past few months (the Twitter Robots stuff), but I guess that’s partly my bad as it’s not as organized as it should be. That’s the main reason why I’m not publishing the whole code here yet, have a lot of cleaning up to do ;)

Meanwhile you may take a look at this buddy: @web2feed. I turned off the auto-replies because they were getting quite annoying, and I’ve added a couple of feeds to the big list, oh and it’s DM controlled too!

Inspired: Javascript & jQuery Love

kovshenin — Tue, 08 Dec 2009 11:07:55 +0000

Remember the “inspired” blog posts series I started back in November? Well, let’s keep playing that game and today’s topic is jQuery and Javascript. The things you can do with javascript today are pretty impressive and sometimes unbelievable. Starting from simple AJAX calls and ending up with complete rich user interfaces with sweet animation (poke Facebook for instance).

Who Needs Flash When You Have Javascript?

You know I browse the web a lot – StumbleUpon, Digg, Twitter, etc. I go through tonnes of content, so today I made a short list of a few interesting jQuery and Javascript links that made my day! For rookies and for pros, check them out (ordered randomly):

Now you tell me – who needs Flash when you have Javascript? ;) Oh and by the way, if you’ve anything to add to the list (preferably articles published in late 2009) feel free to suggest via comments. If you’re looking for even more jQuery love, follow the real-time search results for the #jQuery tag on Twitter. Cheers!

W3 Total Cache with Amazon S3 and CloudFront

kovshenin — Mon, 07 Dec 2009 10:54:48 +0000

A few days ago Frederick Townes, author of the W3 Total Cache for WordPress has released an update to this wonderful plugin, and yes, it now fully supports Amazon S3 and CloudFront as the Content Delivery Network! This is a major one for me as I manually upload most of the static assets to my CloudFront account which may take quite a lot of time. The W3 Total Cache plugin does that for you in seconds! Post attachments, images, javascript, css.. All those could go to CloudFront in just 4 clicks. Frederick also mentioned that the upcoming update will also be surprising, which keeps me wondering.

WordPress: Now with Amazon S3 & CloudFront!

I also tried out the other options for page and database caching. A few tests showed up that memcache is faster than APC, so that’s where I stopped at database caching. Page caching was switched to enhanced, which I believe is a new option. The site performance graph at Google Webmaster Tools shows pretty good performance for Novermber and December (very close to 1.5 seconds) although the overall average is still up at 3.5 seconds, which in terms of Google is slower than 59% of sites. This is probably caused by the force majeures in September and October. Page load time peaked at over 7 seconds there.

One more funny fact about Google’s Site performance and Page Speed tools is the “Minimize DNS lookups” section, which most of the time shows up a single entry:

The domains of the following URLs only serve one resource each. If possible, avoid the extra DNS lookups by serving these resources from existing domains: http://www.google-analytics.com/ga.js

Interesting. Perhaps I should copy that javascript file and serve it from my CDN, I wonder if that will work. Oh and then I’ll be missing all the nifty updates to Google Analytics, like the most recent one called Asynchronous Tracking – very neat by the way!

Creating Mockups with Mockingbird

kovshenin — Thu, 03 Dec 2009 10:47:03 +0000

Everyone these days knows that creating any user interface should start with a mockup. Whether it’s something drawn on a piece of paper or almost designed in Photoshop. When speaking about application UI, software like Borland Delphi or Microsoft Visual Basic would be just fine, especially if you need to create some simple click events. The web though is slightly different, and creating mockups with online web applications is way cooler (and faster too) than VB.

Take a look at Mockflow. It’s driven by Adobe Flash, the interface is sweet, but meh.. You have to pay if you’d like to use it for more than 4 slides. And here’s an alternative that comes to rescue – Mockingbird! Totally awesome, free of charge, javascript driven, no limitations. Here’s something I managed to do based on my previous post. It took me about 1 minute and 50 seconds, sweet huh?

By the way there’s also an option to share you mockup. Both in Mockflow and Mockingbird, so you wouldn’t have to print screen your slides like I did ;)

No related posts.

WordPress: The wp_update_post Dates in Drafts

kovshenin — Tue, 01 Dec 2009 06:00:49 +0000

This one’s pretty tricky. If you’ve ever tried to update a post using the wp_update_post function with drafts or pending posts, you might have noticed that the post_date argument is ignored, instead the current date/time comes up. The post_date argument works only for published posts or during the publishing process. I looked into the WordPress core code, the wp_update_post function has an additional argument called edit_date, which is not mentioned in the codex, so if you’re trying to update an existing post which is a draft or a pending one, use the following method:

$post_data = array(
	"ID" => 123,
	"post_date" => $post_date,
	"edit_date" => true
);
 
$post_id = wp_update_post($post_data);

This will update the post 123 and set it’s time to $post_date (which is in format Y-d-m H:i:s if you know what I mean ;). The other possible parameters for wp_update_post are described in the docs for wp_insert_post in the Codex.

Optimizing Your Amazon Web Services Costs

kovshenin — Mon, 30 Nov 2009 13:38:39 +0000

I’ve been with Amazon for quite a long time now and you must have heard that their web hosting services aren’t very cheap. The average total of one instance per month (including EBS, S3 and all the others) was around $120 at the start. That was back in July 2009 when I had no idea about how all this stuff works. With a lot of experimenting I managed to drop my instance per month costs down by around 40%. Below are a few tips that can help you lower your Amazon Web Services charges:

Cloud Tips: Spare Those Extra Dollars

Use reserved EC2 Instances where possible. Amazon charges $0.085 per hour for an m1.small Linux instance in the US, that’s around $61 per month and $734 per year. A reserved instance costs me $227 for one year, plus $0.03 per running hour, that makes it around $490 per year for an m1.small instance. Use reserved instances only if you’re sure that you’ll be using it for a whole year. You can save even more if you purchase a reserved instance for three years.
Storage: EBS vs EC2. Pick EC2! That’s right, EC2! EBS charges you for provisioned storage, IO requests and snapshots. These may rise pretty quickly if you’re running MySQL on an EBS block – very risky! Run your MySQL on EC2. The php files and everything else should preferably be on EC2 aswell. You can use your EBS block for tiny backups of core PHP files if you’re running more than one EC2 instance.
EBS is cheaper than S3. S3 should only be used in cases where you have to serve your static content from different servers (perhaps through CloudFront), and maybe store some backups there too (don’t forget to remove the old ones!), but EBS is cheaper, even with snapshots.
CloudFront is okay. It does speed up your website, but you have to know that it’s more expensive for requests to Japan and Hong Kong

There you go. With these tips you should be able to get the Amazon hosting services for around $90/month, unless of course you have a 3 million visitors per day website ;) Also, for those of you wondering.. I haven’t used RackSpace, but I did compare their prices to Amazon’s and they’re more expensive.