Tales of an IT Nobody

Disable PHP 5.4's built-in web server, while keeping CLI ...

noreply@blogger.com (Justin Rovang) — Mon, 06 Feb 2012 07:27:00 +0000

Administrators: Don't get blind-sided by PHP 5.4's CLI web server!

I've gone over a similar issue like this before regarding the likes of git/hg. While those are developer tools and are less likely to be present on a production machine.

PHP 5.4 is jumping on the bandwagon to include a 'cute' little internal server - which is enabled by default.
The 'everything needs a standalone server' thing is starting to ~~get on my security nerves~~ feel silly.

It has limited use, and most developers will have limited use for it due to it's lack of mod_rewrite (and equiv.) behavior ... The worse part is: You can't disable it if you want to keep cli (e.g.: no pear!)

Wish I spoke up on the list!

Anywho, here's a hob-knobbed patch (for PHP 5.4.0RC6) that will change that for you.
(GNU/*nix only!) The patch adds a new configure option '--disable-cli-server'.

Download the patch here: patch-php5.4.0RC6-no-cli-server.diff
Place it in the PHP source base directory.

tar -zxvf PHP-5.4.0.RC6.tar.gz
cd PHP-5.4.0.RC6/ 
patch -p1 < foo.patch
./buildconf --force
./configure [config options, etc] --enable-cli --disable-cli-server
make # (-jX if you've got spares ;) )

In the future I'll plan on formalizing this patch and propose it in php.internals when I get a chance to make the windows part of the patch.

References:
https://wiki.php.net/rfc/builtinwebserver
http://svn.php.net/viewvc/php/php-src/branches/PHP_5_4/sapi/cli/
https://gist.github.com/835698

Observations: Google's new Terms of Service

noreply@blogger.com (Justin Rovang) — Fri, 27 Jan 2012 15:56:00 +0000

The new TOS and Privacy Policy documents from Google are a welcome change, reducing 60 individual ones into a standard, global set is a much better idea for understanding's sake.

Observation 1:

We may review content to determine whether it is illegal or violates our policies, and we may remove or refuse to display content that we reasonably believe violates our policies or the law. But that does not necessarily mean that we review content, so please don’t assume that we do.

Usng Our Services http://www.google.com/policies/terms/#toc-services

I get what they're saying, but the wording seems a little humorous if you don't hone in on 'necessarily'.

Observation 2:

We provide information to help copyright holders manage their intellectual property online.

Privacy and Copyright Protection http://www.google.com/policies/terms/#toc-protection

Odd, this leaves some open questions to what information they provide ... are they helping police things "SOPA style"?

Observation 3:

Some of our Services allow you to submit content. You retain ownership of any intellectual property rights that you hold in that content. In short, what belongs to you stays yours.

That's neat... but wait, the next paragraph:

When you upload or otherwise submit content to our Services, you give Google (and those we work with) a worldwide license to use, host, store, reproduce, modify, create derivative works (such as those resulting from translations, adaptations or other changes we make so that your content works better with our Services), communicate, publish, publicly perform, publicly display and distribute such content.

Your Content in our Services http://www.google.com/policies/terms/#toc-content

Uh... ok...

I don't have any intent on copyright infringement, so the first two don't bother me, but the third one leaves me with some questions ...

PHP Vulnerability - DJBX33A - Hash table collisions

noreply@blogger.com (Justin Rovang) — Sat, 14 Jan 2012 08:22:00 +0000

Trickling through my RSS feeds this morning was an article with quite the topic "PHP Vulnerability May Halt Millions of Servers".

In a nutshell: A modest size POST to almost all PHP versions in the wild (Sans 5.3.9+) are in danger of an extremely simple DoS.

The vulnerability exploits the PHP internal hash table function (responsible for managing data structures) - more specifically: the technique used to 'hash' (generate a key for the hash table) the key for a key=>value relationship.

Here's the informative part regarding PHP's problem in the security advisory for this:

On an i7 core, the 60 seconds take a string of multi-collisions of about
500k. 30 seconds of CPU time can be generated using a string of about
300k. This means that an attacker needs about 70-100kbit/s to keep one
i7 core constantly busy. An attacker with a Gigabit connection can keep
about 10.000 i7 cores busy.

Apache has a built in limit of 8K max request length (that is, maximum size in request URL) by default.
Can the damage from an 8k request (this affects GET) - really cause the mentioned DDoS attack on reasonable hardware?

Additionally - PHP has a limiter on POST data too: max_post_size.
It's this configuration variable in particular I think should be put in the limelight.

max_post_size is a run-time/htaccess configurable directive that maybe we don't respect like we should.
Often, administrators (myself included) just tell php.ini to accept a large POST size to allow form based file uploads - It's not uncommon to see:

upload_max_filesize = 20M
post_max_size = 21M # or multitudes more!

- in almost any respectable setup.

Perhaps we should evaluate the underlying effects of this setting; maybe it should be something stupidly low by default (enough to allow a large WYSIWYG CMS article's HTML and a bit more? 32K?) - and then delegate a higher limit using Apache configuration.

Caveat: these settings are PER DIR meaning:

.htaccess use is limited, you can't set the php_value in a .htaccess with a URL match - you're stuck using a context sensitive .htaccess (within a dir) or use the directive - this won't work for people using front controllers through a single file on their websites/apps.
Modifying the actual vhost/host configuration is a sound bet - you can do Location/File matching and set these at will; for situated web apps, this may be a feasible decision to take whitelist or blacklist approach on uploader destinations.

More resources:

Here's the video that thoroughly covers the vulnerability - I've shortcut it to their recommended mitigation (outside of polymorphic hashing):
A full blown rundown, including proof of concept (USE AT YOUR OWN RISK!)
A string of hash collisions targeting DJBX33A for vuln testing (PS: Firefox seems to struggle with this in a GET format, Chrome doesn't, odd!)

Why I won't (can't) adopt Google Chrome yet...

noreply@blogger.com (Justin Rovang) — Wed, 11 Jan 2012 01:22:00 +0000

Privacy aside, simply put: in my role, I do my fair share of design work, AJAX debugging, CSS, you name it - I need tools at my fingertips to quickly do more than just rip apart the DOM of a page, these are my deal breaker extensions/capabilities that aren't in chrome:

Dealbreakers:

1. Web Developer Toolbar - Session toggle, disable/enable ~~cache~~
~~Chrome has no way to turn on/off cache at the click of a button. The closest thing I have found is a to create an icon that has a switch in the launch parameters.~~ Another biggie for me is to clear a specific set of session cookies for a domain instead of all of them. The chrome version of Web Developer Toolbar completely lacks these options.

2. Selenium IDE
Only firefox has the Selenium IDE plugin; for those of us who perform automation or frequent checking on forms for SQL injection or other; there's a few alternatives out there for chrome, but none as extensive as Selenium (you can also reuse the IDE tests for Selenium RC)

3. S3Fox (or equiv.)

4. View image info ('nuff said; even IE has it!)

Google Chrome IS the browser of the future; it's still not quite there yet for me...

If you're not off of Godaddy yet ...

noreply@blogger.com (Justin Rovang) — Fri, 23 Dec 2011 07:30:00 +0000

You should be. The Godaddy girls are stupid. The commercials are worse. Bob Parsons is kinda creepy (not just the elephant thing). The ads are terrible. The site is terrible.

Do you need another excuse to move your registrar needs to another company such as Gandi or Namecheap?

You need another excuse? here it is.

You should know what SOPA * is about between the lines. (Job growth? Puh-leese, the job growth from the .com boom didn't need SOPA thank you very much!)

Over the last few months I've moved a dozen domains off of Godaddy on to others (client's discretion).
If you're still on the fence, there's a pretty good run down of good alternative registrars on this blog post.

ICANN also has a full (but impersonal) list of accredited registrars as well.

PS: Namecheap has a coupon code for a little bit off "SOPASUCKS".

/bin/false - Report false bugs to ...

noreply@blogger.com (Justin Rovang) — Wed, 21 Dec 2011 00:50:00 +0000

While sifting through the manpages for /bin/false (looking for crafty uses for this oddball command) - I just had to share a funny line from within the manpage:

REPORTING BUGS
       Report false bugs to bug-coreutils@gnu.org

Maybe it's been a long day =|

ab - Apache Bench, understanding and getting tangible results.

noreply@blogger.com (Justin Rovang) — Sun, 04 Dec 2011 04:51:00 +0000

ab is a very powerful tool when used right. I use it as a guideline for how to set up my apache2/httpd.conf files.
All too often I see people boasting that they can get an outrageous number of RPS in ab (the Apace Bench tool).

"OMG, I totally get 3,000 rps in an AWS micro instance!!!" (I've seen this on the likes of Serverfault)

Debunking misunderstandings:
Concurrency != users on site at same time
Requests per second != users on site at same time

ab is meant to give a 'feel of pants' diagnostic for the page/s you point it to.

Every page on a website is different; and may require a different number of requests to load (resources: css, js, images, html, etc).

Aspects of effective Apache benchmarking:

Concurrent users
Response time
Requests

"Concurrent users" - Have you ever stopped to ask yourself: What the hell is a user? (in the Apache sense) We don't stop to think about them individually, we just think about them as a 'request' or the 'concurrent' part of benchmarking.

When a user loads a page, their browser may start X connections at the same time to your server to load resources. This is a complex dynamic, because browsers have a built in limit of how many concurrent requests to make to a host at a given time (a hackable aspect).

So at any given time, let's say a user = 6 concurrent connections/requests.

"Response time" - What is an acceptable response time? What is a response? In the context of this article, it's the round trip process involved with the transfer of a resource. This summarizes the intricacies of database queries and programming logic, etc into a measurable aspect for consideration in your benchmarking.

Is 300ms to load the HTML output of a Drupal website acceptable? 400? 500? 600? 700?

How fast does a static asset load? What is the typical size of an asset for your webpages? 10KiB? 20?

"Requests" - Requests happen at the sub-second level, measured in milliseconds.
Let's say the average page has 15 resources (images, js, css files, html data, etc) - aka: 15 requests per page

This means if a single user comes to load a page, there's a good chance his/her browser will make 15 requests total.

Another part of this aspect is to be aware that the browser will perform these requests in a 'trickle' fashion, meaning one request to get the HTML, then an instant 6 requests (browser concurrency) but the next 8 will happen one at a time once concurrent connections free up.

Putting aspects together:
We have to draw an understanding of how these aspects all tie together to determine the start-to-finish load of a typical page.

Let's say a page is 15 requests (14 images/js/css files and HTML) with an average payload of 10KB of data each. 150KB of data.

A user/browser makes 6 simultaneous requests (All completing at slightly different times, ideally, at the same time).

Response time is the metric we're interested in.

The questions we ask of ab are - given the current configuration and environmental conditions:

What's the highest level of concurrency I can support loading a given asset in less than X milliseconds
How many requests per second can I support at that given level of concurrency

By attempting to answer these questions, we'll derive the tipping point of the server - and what it's bottlenecks are. (This article will not cover determining bottlenecks - just how to get meaning from ab)

Caveats
Naturally these seat of pants numbers are generated on a machine on the same network with plenty of elbow room - making them best case scenarios - in today's world it's close enough with how widespread HSI is. It also assumes the network can handle the throughput of the transfers involved with a page. It also assumes all assets are hosted on the same host, nothing cross domain, e.g.: Google Analytics, Web Fonts, etc.

How to test
First, we need to classify the load. As mentioned a few times, there's two types of site data: static files, and generated files.

Static files have extremely fast response times, generated files take longer because they're performing logic, database work and other things.

A browser doesn't know what to load until it retrieves the HTML file containing the references to other resources - this changes how we look at timings, and most cases, the HTML document is generated.

First
Let's simulate a single user/browser loading a page from an idle server.
First, we must get the HTML data...

So you can see, this took 29ms, at 299KB/s to generate the HTML from Drupal and send it across the pipe.

Now, let's simulate a browser at 6 concurrent connections loading 15 assets at 10KiB each.

So here you can see, each request completed at 35ms or less - The whole test took about 560 ms.

This means that under pristine conditions, the user would have the entire webpage loaded in 589ms.

Finding the limits...
So, looking at our numbers - it's clear that 1 user is a piece of cake.
It's also clear that there's two big considerations into acceptable timings:

Time to get HTML (our 1 generated request)
Time to get references from that HTML (our 15 static requests)

We're going to multiply our numbers and start pushing the server harder - let's say by 30 times:

So once again, let's emulate 30 users making 30 requests to our Drupal page:

Every user got the HTML file in ~298ms or less.

Next up, static files - 30 users = 180 concurrent connections, 450 requests.

So, here's where things start getting interesting - 2% of the requests were slowed down to 600+ms per request. The exact cause of that is out of the scope of this article - could be IO - regardless, the numbers are still good and it's clear that this is indicating the start of a tipping point for the host.

Take the HTML load time - 298ms
Do some fuzzy math here: 265ms* 2.5 = 662ms (mean total)
957ms average for entire page to load.

This is 'fuzzy math' because I'm not simulating the procedural/trickle loading effect of browsers mentioned above (initial burst of 6 connections, and making them busy as they become available at different times to complete the 15 requests). But instead treating it as 6 requests in sets. Someone more mathematically inspired might calculate better numbers - I use this method because it's my "buffer factor" that I use for variables (bursts of activity, latency changes, etc).

So from this data, we can say the server can sustain 30 constant users given our assumptions.

Phase two:
So now we have a ballpark figure of where our server will tip over. We're going to perform two simultaneous ab sessions to put this to the test, this will simulate both worlds of content: generating content and loading the assets. This is the 'honing' phase where we dial down/up our configuration by smaller increments.

5 minutes of load at our proposed 30 users
30 concurrent connections for our generated content page.
180 concurrent connections for static data.

Ding ding ding! We've got a tipper!
Ok, so as you can see, for the most part, everything was fairly responsive. If the system could not keep up, most of the percentiles will have the awful numbers like the ones shown in the highest 2%. However, overall these numbers should be improved upon to deal with sporadic spikes above our estimates, and other environmental factors that may have a swaying factor in our throughput.

How to interpret:
Discard the 100% - longest request; at these volumes it's irrelevant.
However, the 98-99% matter - your goal should be to make the 98% fall under your target response time for the request. The 99% shows us that we've found our happy limit.

(Remember, at these levels, theres many many variables involved - and the server should never reach this limit, that's why we're finding it out!)

Let's tone our bench down to supplement 25 users and see what happens ...

Wrap up
25 simultaneous users may not sound like a lot, but imagine a classroom of 25 students - and they all click to your webpage at the exact same moment - this is the kind of load we're talking about; to have every one of those machines (under correct conditions) load up the website within 1 second.

To turn that into a real requests per second: 375. (25 users @ 15 requests).

The configuration - workload (website code, images) and hardware (2 1Ghz CPU's...) are capable of performing this at a constant rate - these benchmarks indicate that the hardware (or configuration) should be changed before it gets to this point to supplement growth. These benchmarks indicate that ~430 pageloads out of 21,492 will take longer than a second to load. In reality, the ebbs and flows of request peaks and valleys make these less likely to happen.

As you can see, the static files are served very fast in comparison to the generated content from Drupal.
If this Apache instance was backed by the likes of Varnish, the host would be revitalized to handle quite a bit more (depending on cache retention).

Testing hardware

1x AWS EC2 Large instance on EBS - Apache host

2 virtual cores with 2 EC2 Compute Units each

AKA: (2) 2Ghz Xeon equiv

7GB RAM

1x AWS EC2 4X Large CPU - AB runner

8 virtual cores with 3.25 EC2 Compute Units each

AKA: (8) 3.25Ghz Xeon Equiv

68GB RAM

Grepping extremely large files

noreply@blogger.com (Justin Rovang) — Mon, 28 Nov 2011 18:40:00 +0000

So you forgot to set up logrotate on an active log eh? You've got a many gigabyte file to weed through and you need to extract a chunk of time from it?

Here's a quick cheat sheet to help you get by, quickly and sanely.

It's about byte offsets!

Get the byte offset in the file where your time range starts
Get the byte offset in the file where your time range ends
dd the data out!

# Get first byte offset, leftmost number is the offset, "offset_start"
grep -m 1 -b "2011-11-15 11:3" error_log
 # Get last byte offset, "offset_end"
grep -m 1 -b "2011-11-16 01:3" error_log

 #Subtract offset_start from offset_end to get byte length, then do:
dd if=error_log of=filtered bs=c skip=[offset_start] count=[byte_length]

Caveats

You should tack on extra bytes to the byte length, because the offset_end number is actually the beginning byte of your boundary log entry
Figuring out the boundary is a bit tricky because a log entry -has- to be present in order to match, so if you're looking for what happened at 20:00 hours on X date, you may have to round up to the date level depending on how busy your log is
This is just a trick to extract a chunk of entries to speed up further filtering.

Full example

dev:/var/log/apache2# grep -m 1 -b  "23/Nov/2011:00" access.log.1
6633711:3694 127.0.0.1:53784 - - 176us [23/Nov/2011:00:00:10 -0600]

dev:/var/log/apache2# grep -m 1 -b  "24/Nov/2011:19" access.log.1
9674209:10872 127.0.0.1:46097 - - 173us [24/Nov/2011:19:00:10 -0600]

# byte length: 9674209 - 6633711 = 3040498 bytes (Plus 256 to cover the caveat) = 
dd if=access.log.1 of=filtered bs=c skip=6633711 count=3040754

Worthy of distribution: Cloud analogy

noreply@blogger.com (Justin Rovang) — Sat, 05 Nov 2011 05:10:00 +0000

This post on Beyond Bandwidth seems to summarize some of my feelings about cloud computing - it's best thought of as an outsourcing task for the most part; Although the benefits of something like an extra DNS server are a bit more than an 'outsource benefit'; but you get the idea:

Cloudy analogies with a chance of illusion

Is there a hacking campaign against open source?

noreply@blogger.com (Justin Rovang) — Mon, 26 Sep 2011 19:55:00 +0000

Linux.com, kernel.org, mysql(twice this year), wordpress and php have all reported breaches of some sort this year. Is there some sort of campaign against these 'high profile' open source projects? It's starting to feel like it, to me.

The more hands you get in the pot, the more nervous you should get as an administrator. System issues stem from more than password change frequency and difficulty - stale keys and giving access to folks that shouldn't have access happens.

I also feel isolation or 'separation of concerns' is a tactic that is pushed aside in the name of maxing out a system, more often than not this stop gap would save a lot of trouble. Apache's ability to mitigate concern from last year's breaches is a good example of isolation, they had a fairly sophisticated break in and the repercussions weren't as vocal as the ones from this year.

There doesn't seem to be sufficient coverage of this MySQL hack right now - how sure are we this isn't a sample set from a compromised browser as opposed to the site?

I hope there will be continued disclosure so everyone can learn something extra to safeguarding themselves.

While it doesn't feel right to ream MySQL (at all, or at this point of the news) I have some initial thoughts I just can't shake:

If MySQL was 'hacked': Infiltrated earlier this year; you made no extra measures on a wider scope? really?
Why the hell is your web/any cluster accessible without a VPN? It sounds like they're selling shell access directly to the host/s..

C'est la vie

The inherent risks of 'daemonize' features in developer tools - Git, Mercurial (hg)

noreply@blogger.com (Justin Rovang) — Sat, 24 Sep 2011 23:47:00 +0000

A handful of tools such as mercurial, git, (soon PHP - which chances are will be it's own binary) have their own 'daemonize' functionality.

Whatever your reasons - if you want to disable these; there's little to no help in figuring out how... til now...

If you want to disable Mercurial's `hg serve`:
Open the file (Your python install path may differ, but this should give you an idea of what to search for)

/usr/local/lib/python2.x/dist-packages/mercurial/hgweb/server.py:

Find the function 'create-server' and add 'sys.exit()' in the first line:

def create_server(ui, app):
    sys.exit()
    if ui.config('web', 'certificate'):
        if sys.version_info >= (2, 6):
            handler = _httprequesthandlerssl
        else:
            handler = _httprequesthandleropenssl
    else:
        handler = _httprequesthandler
(etc, etc)

How to verify this works:

1. Before patching - run 'hg serve' from a mercurial repository. It will report the port number and remain active in console.
2. After patching - 'hg serve' from a mercurial repository will simply exit and say nothing.
3. netstat, ps -A ux |grep 'hg serve'

If you want to disable git's `git daemon`:
This one is probably the easiest of the two: find and 'chmod a-x' (remove execute permissions) from the 'git-daemon' binary on your system - mine is in /usr/libexec/git-core. You can also relocate it somewhere in-accessible.

How to verify this works:

1. Before relocating/removing/chmodding - run 'git daemon' - your console will remain active as if it's listening. (You can try a base dir for a proper daemon setup if you want ...)
2. After relocating/removing - run 'git daemon', you'll get an error saying there are insufficient privileges, or in the case of relocating/removing you'll see "not a git command".
3. netstat, ps -A ux |grep 'git daemon'

Day of the Googmonster - from ... a google blog...

noreply@blogger.com (Justin Rovang) — Mon, 12 Sep 2011 06:05:00 +0000

This is a must read for anyone who feels Google can do no evil, putting them on a pedestal.
If you embrace every little 'tech' knick knack they throw out to the world, or If you're in the percentile with a who's seeing Google turn into a cashgrabber like everyone else - you should read it!

It is by far the most concise rundown of why I have a love-hate relationship with Google. I'm not against a company making some coin; anyone who knows me knows I'm a reasonable capitalist, but I do -not- agree with the direction Google seems to keep poking at.

The pace of change from Google over the past year has been alarming. I'm not talking about the new pretty UI stuff - I'm talking about their business and technological tact.

Google business observations:

- Apps angine, dirt cheap - now expensive and complicated for saving money.
- Labs is being retired (I view this as a strong indicator of their new business stance).
- "Music beta" - seeing this first hand makes me wonder "what's the catch" - it doesn't feel like Google, it -WILL- change dramatically! (I predict this will either be pulled, or quickly move to a "paid" service - another 'get em hooked' tactic).
- More aggressive advertisement in every facet - especially GMail.
- Self driving cars. What don't they want their fingers in?
- Drop of Android app inventor (Platform training / consultation anyone?)

The above things are all OK with me - they can do what they want with their company!

My problem lies in an old fashioned tactic used by the likes of Microsoft, Netscape, etc to round up users and get them stuck on an exclusive technology (ranging from mundane protocols to programs) - now it's Google bringing Dart and the likes of WebP to the fray.

I don't hate Google - a lot of engineering feats give them their credibility and "trust" from the masses - the world has benefited for sure! However, I trust them much less than I did 2+ years ago... and most certainly don't think that there's no strings attached to these attempts to re-invent (add to) old problems.

Maybe my contention for all of this is just a sign of being winded in "web development"... I'd rather setup key gen + git access on "dev", or work on making MegaCli stomachable than tread water in the emotions of browser and it's dependent technology...

Kernel.org, linux.com down, still... also, Git! - Updated!

noreply@blogger.com (Justin Rovang) — Mon, 12 Sep 2011 02:22:00 +0000

With news breaking about the compromised systems for kernel.org, linux.com, which are sites are "down for maintenance". Completely - and it's been this way for many days now. (Kernel.org since the 28th)

I think it's safe to say the range and scope of the issues are pretty disappointing - the longer these systems stay down the more obvious it is that the damages are probably higher than perceived before; I'm having a hard time saying the administrators of these groups are just this slow at being cautious. (Especially the 'forward facing ' hosts)

(Segway to Git)

It's interesting to note that a LOT of bullets with the security breach are dodged with the parity cryptography used by Git. Pretty cool! (Linus seems to be flirting with the idea of using Github for latest kernel developement.)

After keeping an eye on the "choosing a DVCS" discussions for PHP, a lot of people are in favor of leveraging Git purely because of Github - whereas Mercurial, something we use at M State has been around for a bit less - has a stronger, more mature toolset (Albeit, a bumpy ride for sure!); and, from my standpoint - better cross platform implementation. The speed differences are somewhat minimal.

The allure of Github is the social endeavour; this has fueled a much more active community (compared to Bitbucket, less 'social-ly'). It looks like Git has finished clobbering competitors like Bazaar and Perforce and finally, I'm willing to throw in the towel for my support of Mercurial and say there's little room for traction for Hg.

The programming world is a lot of work to follow =\

(Also: It's September 11th - take a moment to reflect - cast some sympathy and reverence for the lives lost! )

UPDATE: 2011-09-21-
linux.com has been updated and now states that they will be "restoring service shortly" - different from the original FAQ page they had up about the breach.

I'd expect kernel.org to follow.

After over half a month (almost a month for kernel.org) of being down with poorly communicated "maintenance" pages. I hope there's a fallout for the culprits - and I hope the maintainers of those domains take a more serious approach to how they handle this situation next time...

Google App Engine - Pricing changes and another prediction

noreply@blogger.com (Justin Rovang) — Fri, 09 Sep 2011 22:34:00 +0000

I'm not sure what the Goog'monster is thinking with so many dramatic changes over the last year. Most of them have been good - but the loss of labs, app inventor and the pricing hikes for the Google App Engine platform is really rising some eyebrows for me - It seems they're tightening their belt and distancing themselves from individuals and more toward bigger dollars.

Anywho, anyone unhappy with the pricing changes will enjoy reading what I will call the "GAE reaming" thread.

Is it a matter of time before they trickle pricing for Music and Docs? Starting to wonder ...

System admin 'helper': Zebra stripe log / console output

noreply@blogger.com (Justin Rovang) — Thu, 25 Aug 2011 05:25:00 +0000

Looking at an ASCII data table can be difficult - so to start a small trip into Perl programming - I tossed together a simple Perl script, with no module requirements - zebra.pl as I call it; adds a nice touch to say `vmstat` or viewing something like the interrupts on a multicore box. It's super simple and done in the nature of Aspersa.

It doesn't work 100% like I want - I would have liked it to take an $ARGV; to do that it seems like I'd have to create a dependency with a module (something like GetOpts) - so I decided one can simply modify the script to change how many X rows are striped.

You can fetch it here.

On coining terms: Kiloreq, Megareq, Gigareq, Terareq

noreply@blogger.com (Justin Rovang) — Tue, 09 Aug 2011 16:49:00 +0000

I'm inventing these terms. You heard them here first!

Ok so the idea goes like this:
We use kilo(bit/byte, etc) as measurements of rate, and size - even weight (kinda).

I thought it'd be fun to come up with another terminology that's right in line with the nature of these units of measurement geared toward server load: "R" for request - prexed accordingly: Kiloreq, Megareq, Gigareq, etc.

So for example, if you get 1000 requests a second, you can say "I get 1KR/sec", if you have 500 request per second, instead of '500 rps', use the standardized "KR" (Kiloreq) suffix: 0.5KR/sec

How many requests did foo Apache server handle this month?
About 3MR. 3 mega reqs. (3MR * 1000KR * 1000R = 3,000,000 requests).

Or how about for the year? (Assuming a flat rate of 300MR over 12 months)
3.6GR. Gigareqs!

Far fetched? Yes. But I plan on using them in my day-to-day language to try and make them stick :)

Netflix is run by monkeys!

noreply@blogger.com (Justin Rovang) — Fri, 05 Aug 2011 04:28:00 +0000

An entertaining read for the HA operations for netflix - a good sense of humor and a very cool, hardcore philosophy for testing!

http://techblog.netflix.com/2011/07/netflix-simian-army.html

It's nice to see Netflix stepping up their involvement in the technical community even more; with the Netflix prize and their blog and API feedback - I hope they become even more successful because of these investments.

MySQL - max_allowed_packet - what is going on?

noreply@blogger.com (Justin Rovang) — Fri, 05 Aug 2011 03:32:00 +0000

So there's enough noise in the MySQL community about what's covered well here (https://www.facebook.com/note.php?note_id=10150236650815933)

Unfortuantely the bug is private for the time being; in my conversation with others, the general premise seems to be what good does max_allowed_packet really do?

First off, I'd like to point out what seems to be what I hope is heading for deprecation - otherwise it just feels a bit sloppy; the default max_allowed_packet for the MySQL client is 1GB. (AKA: Maximum).

As the FB post recognizes, there's some ambiguity to how this setting is even enforced in the first place, especially when considering a master->slave configuration (Why does replication even have to follow that rule? Maybe replication clients can have a hard-coded packet to the maximum to get over this?)

I'd propose one of the two:

1. Enforce max_allowed_packet at the server - negotiate a loose communication with the client, where the client will obtain the server's value and take it for it's own.

2. Better yet, allow it to be set on a per user basis, following #1.

mk-heartbeat - sample conf file

noreply@blogger.com (Justin Rovang) — Tue, 26 Jul 2011 18:32:00 +0000

There's not a whole lot (if any) documentation on what the file for the --config parameter for Maatkit's mk-heartbeat should look like.

If you quickly pilfer through the source code, what it's looking for becomes pretty straight forward (It'd be nice to have this in the documentation, however).

The config files are newline delimited and resemble a my.cnf file using 'param=value' notation.

E.g.:

Sample config

dev:~/scripts# cat heartbeat-xxx-mysql.conf
host=my.host.name
pass=password
user=username
database=maatkit

So this would be launched using something like:

dev:~/scripts# mk-heartbeat --config /path/to/heartbeat-xxx-mysql.conf --update --daemonize

In my book, this is essential behavior to avoid having your password hang out on the output from ps

dev:~/scripts# ps -A ux |grep mk
root     11612  0.0  0.1  12756  6384 ?        Ss   13:27   0:00 perl /usr/bin/mk-heartbeat --config heartbeat-xxx-mysql.conf --daemonize --update

All of your PHP code is going to look silly! PHP 5.4, here we come!

noreply@blogger.com (Justin Rovang) — Fri, 22 Jul 2011 16:19:00 +0000

No, not as in 5.4 will make you write stupid code. As in it will make all pre-5.4 attempts to maximize re-use through crazy inheritance chains look ridiculous compared to what traits will deliver.

I already have a sense of dread of being a year into PHP traits looking at not-too-old code; how painful it will be to look back at what kind of crap we put up with!

This is going to be awesome, and I for one, welcome our new trait supporting overlords.

Now .. WHERE ARE MY RETURN TYPE HINTS!

I recently had a discussion with a peer and we pretty much summarized what we wish the stance on types for PHP will become something like this:

PHP is a loosely typed language, as it matures in OOP, it must support more strict typing features (including generics: string, int, float, bool, etc). Emphasis on support, not enforce.

Those who want to leverage the dynamics of loosely typed programming can go for it, there's a place for everyone here to do what they want.

It's the hippy thing to do!

P.S.: Here's a fun conjecture about the php.net domain name and a get-rich-quick scheme!

Nay say for ext2/ext3, seemingly ext4 for MySQL servers

noreply@blogger.com (Justin Rovang) — Tue, 19 Jul 2011 14:46:00 +0000

Basically I felt compelled to make a note regarding what filesystem to evaluate when you are performing a MySQL install. There seems to be a lot of reasons NOT to use the ext filesystems, and instead use XFS..

This is a straight out quote from a MySQL at Facebook blog entry:

ext-2 and ext-3 lock a per-inode mutex for the duration of a write. This means that ext-2 and ext-3 do not allow concurrent writes to a file and that can prevent you from getting the write throughput you expect when you stripe a file over several disks with RAID. XFS does not do this which is one of the reasons many people prefer XFS for InnoDB.

More on IO concurrency from MySQL big name Domas Mituzas

O_DIRECT serializes writes to a file on ext2, ext3, jfs, so I got at most 200-250w/s.

xfs allows parallel (and out-of-order, if that matters) DIO, so I got 1500-2700w/s (depending on file size – seek time changes.. :) of random I/O without write-behind caching. There are few outstanding bugs that lock this down back to 250w/s

A patch for ext4 was created, but it doesn't appear that it made it in; it seems to yield minimal benefit.

And some other performance and risk observations involved with the most wildly used ext3.

If you're looking to install or upgrade a MySQL server, it may very well be worth the time investment to research the depths of what filesystem you select, since it has just as much to do with the database performance as the MySQL configuration itself!

Worthy of distribution: Reset root MySQL password

noreply@blogger.com (Justin Rovang) — Mon, 18 Jul 2011 21:03:00 +0000

Oh snap! Need to reset your mysql root/admin (or any?) MySQL password? Well, you'll need root and control over MySQLd to some extent, but this is worthy of a rainy-day bookmark indeed: http://mysqlpreacher.com/wordpress/2011/03/recovering-a-mysql-root-password-three-solutions/

Upgrading to PHPUnit 3.5 on Zend Studio

noreply@blogger.com (Justin Rovang) — Thu, 30 Jun 2011 03:39:00 +0000

Here's a quick clip I made to show those who hankering to have an updated PHPUnit plugin for Zend Studio.

I hope this video is obsoleted soon so others don't have to endure the annoyance.

View in HD!

Worthy of distribution: PHPUnit's dbunit testing rundown

noreply@blogger.com (Justin Rovang) — Mon, 27 Jun 2011 14:51:00 +0000

This is by far the most complete and best example of a rundown of database testing using PHPUnit's "dbunit" extension. It seems it's difficult to track down a whole rundown on the more technical aspects of 'getting into it'.

The Ultimate Guide to Database-Testing with PHPUnit
Author: Benjamin Eberlei (kontakt@beberlei.de)
http://www.beberlei.de/dbunit.html

Interfaces in PHP are worthless. Blasphemy? yes.

noreply@blogger.com (Justin Rovang) — Tue, 07 Jun 2011 04:38:00 +0000

What is the purpose of an interface? One would simply say "they're a contract, etc.", and you'd be right.

Long story short, all PHP interfaces do - is enforce:

- What methods are implemented
- What their signatures look like

However there's one HUGE (in my book) fundamental lack: RETURN TYPES.

Thus, I will call interfaces 'half a contract'.

The PHP developers and contributors often examine other 'big brother' languages to determine some direction for the PHP OO offerings. I think this is where this half-assed implementation came from - an aspiration of the language trying too hard to implement features of strongly typed langauges.

Need examples?

http://en.wikipedia.org/wiki/Interface_(Java)
http://msdn.microsoft.com/en-us/library/64syzecx.aspx

PHP misses a very important component of the 'contract', return types. In Java / C# you can define your methods as such:

public string foo() {}

Now, if you're willing to shell out for the honor system of implementation or to add some spice to your inductive reasoning - go for it. For what little an interface really does for you, it's not worth the added complexity and lack of piece of mind interfaces in PHP bring to the table.

I believe that in the current state of PHP, developers will end up implementing additional checks and balances to ensure interface implementors are still behaving correctly.