profeshunl newbie

MongoDB as in huMONGOus, not retarded

william — Mon, 12 Apr 2010 11:01:43 +0000

Unlike the rest of the known world, I won’t write anything on the iPad. Yet. Instead, I’m still in the process of writing the upcoming parts of my Popurls Clone series and as per usual, it takes a lot more time than expected. In combination with a massive amount of bills, life is good! As it turns out, you actually need to “work” in order to get “money”. How, when and why did we agree upon that? That’s just as stupid as, say, doing a fully animated Star Wars prequel/sequel to take place during the Clone Wars. Oh, wait… While pondering life’s great mysteries, such as how you can evade paying your bills, I felt it was time to write something about databases. You see the connection, right? Good, because I don’t. A quick Google search for MySQL rendered roughly 117 million results so I figured that another article about it would be kinda superfluous. A search for MongoDB on the other hand resulted in “only” 1,2 million results. Much better. That means that if and when this article gets indexed by the Google, it will constitute no less than ~ 8.33 × 10^-7 % of the relevant search results, in stark contrast to the ~ 5.88 × 10^-9 %, should I’ve written some stuff about MySQL. Goodie! In short, this article will cover MongoDB and how you can use it with PHP. Good times ahead so if you’re a killjoy, you’re not welcome.

A brief introduction to MongoDB

First of all, MongoDB is not a relational database management system (RDBMS) such as MySQL. A relational DBMS is based on a relational model, based on first-order predicate logic as proposed by Edgar Frank Codd in 1969. Data and its internal relationships are more often than not stored as tables. These tables are what constitute the relations. The tables are organized using a kind of model of vertical columns and rows where there are a specified number of columns but the number of rows are, in theory, infinite. The data is fetched via queries using SQL. But enough about that – we’re here to talk about MongoDB which is a part of the NoSQL movement. In NoSQL there are basically three kinds of databases; column oriented, key-value pairs and document oriented. MongoDB employs the latter kind. A document in this context is a structure of data with a given number of properties. These properties can be strings, numbers, arrays or objects, etc. If you’ve dabbled with associative arrays or objects in PHP, you’ll know what this means. You can also group documents in collections. Also, there are something called sub-documents which are pretty much what you’d think they’d be.

In the case of MongoDB, we have no schema to play with but the power lies in the fact that we get to play with JSON-like structures. And JSON is good, right? MongoDB is written in C++, it just hit version 1.4.0 and it stores data using BSON, which is a “binary-encoded serialization of JSON-like documents”. Thankfully, there’s a native PHP extension. So the first thing you’d want to do is install it. Download the appropriate package and perhaps read the Quickstart. Assuming you use PECL, which you should, you can install it easily with the following command;

pecl install mongo

or you can do it manually by downloading the source

phpize
./configure --enable-mongo
make install

The install should have created an extension for you and put it in the extension directory. You may need to edit your PHP.ini and add

extension=mongo.so

where appropriate (hint: maybe where the other extensions are defined). If you get an error in the lines of “phpize: comand not found”, this page is for you. If you’re on linux, a quick fix is to install php5-dev with apt-get or php-devel with yum. Restart your webserver. Do it. If you’re interested in MongoDB but loathe PHP (god knows you have your reasons), there are drivers for other languages, such as C# and .NET (why are you reading this?), Clojure (you pervert), ColdFusion (oh no, you dideeent), Python (you’re all right) or Ruby (pff).

I feel the vibe

Let’s do this. Connect to your brand spanking new Mongo database at localhost (default port is 27017)

$the_mongo_connection = new Mongo();

Or if you’d like to connect to a remote host

$the_mongo_connection = new Mongo("10.0.0.2[:optional_port]");

Pro tip: use a persistent connection. New connections to the database can be very slow. Let this quote borrowed from php.net serve as a cautionary, um, well, example;

for ($i=0; $i<1000; $i++) {
    $m = new Mongo();
}

This will take roughly 18 seconds to execute. Yes, 18. That’s like an eternity on the Internets. If we use a persistent connection like so;

for ($i=0; $i<1000; $i++) {
    $m = new Mongo("localhost:27017", array("persist" => "x"));
}

…we’re down to 0.02 seconds. So, yeah, persistent may be the way to go. And by “may” I mean “do it, ass hat”. If you’re wondering about the “x” in the last piece of code, it’s because persistent connections need a unique identifier string.

The name of the database can use almost any character in the ASCII range, except a blank space, a period or just an empty string. You can actually name a database “null” if you’d like. Might be fun just to piss people off. Selecting a database can be done in two ways and isn’t harder than

$db = $the_mongo_connection->name_of_database;
$db = $the_mongo_connection->selectDB('name_of_database');

The latter is to be preferred if you’re using wonky characters in your database name, such as a comma, a slash or something like that. Valid, but it’ll break your code into millions and millions of pieces if you don’t use the quoted selectDB(). Now we need to select collection to work with. You might say that this is similar to selecting a table in a RDBMS. And you’d be correct. Gold star for you.

$sweet_collection = $db->name_of_collection;
$sweet_collection = $db->selectCollection('name_of_collection');

Sweet. Let’s do some tangible, hands-on exercises, mm’kay? Mm’kay.

The insert [note to self: penis joke here]

For this part, I’ll go with something stupid, like pretend that we’re creating a database of DVD’s. Cheezy, but it’ll do. First let’s set up some dummy data.

$dvd = array(
    'title' => 'A Nightmare On Elm Street',
    'slug' => 'anoes',
    'year' => '1983',
    'format' => 'Blu-Ray',
    'screen' => '16:10',
    'media' => array(
        'audio' => array(
            'English - DTS 5.2 HD',
            'Master Audio',
            'English - DD 1.0',
            'French - DD 1.0'
        ),
        'subtitles' => array(
            'english',
            'french',
            'spanish',
            'spanglish'
        ),
        'extras' => array(
            'Commentary Tracks',
            'Featurettes',
            'Alternate Endings',
            'Trivia Track'
        )
    ),
    'starring' => array(
        'Robert Englund',
        'John Saxon',
        'Heather Langenkamp',
        'Johnny Jepp'
    )
);

Yes, Nightmare on Elm Street. Johhny Depp’s debut movie. As you can see we have a pretty basic multidimensional associative array. The actual code to insert the data I want, you say with a Yoda type of grammar. Sure.

$sweet_collection->insert($dvd);

Boom. There is an optional second argument that can be used if you want to check if the insert succeeded or not. If set to true, it will return an array with the status of the insert. That is, the the array will be returned with the _id assigned by Mongo. Think about the lack of need to call insert_id() or similar. Sweet, huh? If the insert was not successful, it will return a boolean which represents if the array provided was not empty. Empty arrays will not be inserted. Let’s try that out. Assuming we just executed the insert above, that insert gets a unique id (the key is “_id” – try to echo out your last insert id with echo $dvd['_id'];) and then fire this little baby;

try {
    $sweet_collection->insert($dvd, true);
}
catch(MongoCursorException $e) {
    echo "Why you wanna play me like dat? That movie ID already exists in the DB!";
}

Pretty nice, huh? One more thing about the id – MongoDB creates a unique id if non is supplied manually. The MongoId is an object and not a string. It’s the MongoDB equivalent to an autoincrement in RDBMS. Each id is 12 bytes – 4+3+2+3. The first four bytes are a timestamp. The following three are a hash of the client machine’s hostname, the next two are “the lease significant bytes of the process id running the script” and the last three bytes are an incrementing value. Food for thought.

The update

As you may or may not have noticed, there are some minor mistakes in the data above. Oops. Guess we’ll have to update. Funny how these things work out! There are a couple of handy dandy modifier operations to make our Mongo life a little bit easier. The syntax is as follows;

update( [criteria], [objNew], [upsert], [multi] );

criteria – query which selects the record to update
objNew – updated object or $ operators (e.g., $inc) which manipulate the object
upsert – if this should be an “upsert”; that is, if the record does not exist, insert it
multi – if all documents matching criteria should be updated (the default is to only update the first document found)

Our criteria would, in this example, be a value we can search for. Let’s search for the slug, “anoes”. We need to update a couple of fields here; the year should be 1984, screen is 16:9, not 16:10, the english audio is 5.1 surround, not 5.2, obviously, and there is no ‘spanglish’ subtitle. Before we do the actual update, we should discuss the modifier operations that are available. Using these modifiers, we’re able to do atomic updates! Yes, that last sentence deserved an exclamation mark. Shut up.

$inc – increments the value by the factor you provide. If the value is not present, the key is set to that value.
$set – sets the value for a specified key.
$unset – deletes given key and value.
$push – append the specified value to the key if an array is present. If not, an array is set to the key. If the key is present but not an array, an error condition is raised.
$pushAll – append each value in the value array to the key. Works similar to $push.
$addToSet – adds value to the array only if it’s not in the array already.
$pop – removes the first or last element of the array. Use “1″ to remove the last, and “-1″ to remove the first.
$pull – removes all occurrences of the specified value from the key if the key is an array. If the key’s there but is not an array, an error is raised.
$pullAll – removes all occurrences of each value in the array given from the key. If the key is there but isn’t an array – you guessed it – an error is raised.

Still here? no?, I’ll wait some more then.

Okay, so how do we update our movie info? Like so;

$criteria = array(
    'slug' => 'anoes'
);

$objNew = array(
    '$inc' => array(
        'year' => 1
    ),
    '$set' => array(
        'screen' => '16:9',
        'media.audio.0' => 'English - DTS 5.2 HD'
    );
    '$unset' => array(
        'media.subtitles.3' => 1
    );
);

$sweet_collection->update(
    $criteria,
    $objNew
);

The last two arguments are not set since we’re not updating multiple records, nor do we want it to upsert. As you may have noticed, you can target a certain index in the array – see media.audio.0 and media.subtitles.3. Pretty nice. But what if you don’t know the index? Then you’re screwed. Go home.

No, not really. There is something called the positional operator and it’s represented by the $ sign. Use it to find an array entry and manipulate it. For example, let’s update the audio this way instead.

$sweet_collection->update(
    array(
        'media.audio.$' => 'English - DTS 5.2 HD'
    ),
    array(
        '$set' => 'English - DTS 5.1 HD'
    )
);

Note that the $ only applies to the first matched item in the query. This may be changed in the future, but as of now, no.

Fetch, Bobo, fetch! Good boy.

Okay, then. How do we actually fetch our data? In many ways, as it turns out. Queries in MongoDB support both conditional operators (less than, greater than, etc, no equals, in (much like SQL’s “in”), mod, and much more), regular expressions and has a terminology, or, rather, methods, that work similar to SQL syntax. Here’s a quick reference – the MongoDB method to the left and its MySQL equivalent to the right.

sort() - ORDER BY
limit() – LIMIT (chocking, I know)
group() – GROUP BY

Sorry, I thought that list would be longer. You also have the following to toy around with;

skip() – allows you to specify at which object the database should begin returning results. Cool. Think paging results.
count() – returns the number of objects. It counts at server level instead of client level.
snapshot() – this makes sure that no duplicates are returned or objects were missed which were present at both the start and at the end of the query’s execution – even if the objects were updated. Short query responses – less than 1 MB are aways snapshotted.

Having said that, now’s the time to get to know find() and findOne(). These functions lets you query a specified collection. Basically, you tell it which collection to query and which fields to return. Say we have a number of entries in our DVD database and we want to find titles released from 1980 up to 1990, we could write something like;

$years = array('x' => array( '$gte' => 1980, '$lt' => 1990 ));
$best_movies_evar = $sweet_collection->find( $years );

Conditional operators FTW. findOne() works the same way but it returns a single element instead. It returns the record or null if it fails. Let’s say you need to paginate your results as the last query return a massive amount of titles. And right now, you need ten results, starting from title number 30. No biggie.

$num_movies = $best_movies_evar->total();
$best_movies_evar->limit(10)->skip(30);
foreach ( $best_movies_evar $sweet_movie ) {
    var_dump( $sweet_movie );
}

Note that queries are actually not executed until the result is requested. That means you can refine your results before actually fetching them, like we did above. Got it? Nice. Niiccceeeeee.

Aww, that’s nice. C-click. Deleted!

Sooner or later, you’ll want to delete a record or two hundred. Easy enough.

$field_to_remove = array(
    'year' => 1984
);
$sweet_collection->remove( $field_to_remove );

This would remove any document with the year 1983. If you want to remove the first document found, pass true as the second parameter and you’re good to go, sparky.

Now you can create an index, too!

This should be filed under optimisation, bot whatever. You can create indexes in MongoDB which is just gravy. Let’s create an index based on year of release in our sweet sweet DVD collection database. It’s as simple as calling ensureIndex(). Pass an array with the keys you want o have as an index and either “1″ or “-1″ 1 for ascending and -1 descending.


$sweet_collection->ensureIndex(
    array(
        'year' => 1
    )
);

… and that’s it. There are however some options you may pass in an associative array in the second parameter.

unique – creates a unique index.
dropDups – If a duplicate value exists while creating an index, drop all but one value.
background – create indexes in the background while other operations are taking place. It’s synchronous by default, so set this to true to make it asynchronous.
safe – check if thecreation of an index succeeded. If it failed, a MongoCursosException is thrown
name – specify an index name, should Mongo complains about the index name being too long while you’re indexing many keys ot once.

So let’s say we want to create two indexes instead of just one, make it unique, have it work in th ebackground and see if it worked.

try {
    $sweet_collection->ensureIndex(
        array(
            'slug' => 1,
            'title => -1
        ),
        array(
            'unique' => 1,
            'background => 1,
            'safe' => 1
        )
    );
}
catch(MongoCursorException $e) {
    echo "Oh, hells noes!";
}

I think you get the general idea here. You can’t create unique indexes on non-unique values, of course. Shoudl we have two or more identical titles or slugs, the unique index will fail, whch actually is quite probable in this example. Fun, huh? That’ll teach you to copy paste code without checking what it does.

I think that’ll suffice for now. As a closing hint, I’d like to point you to a sweet package on phpclasses.org by Cesar D. Rodas. It’s called MongoFS and allows you to store and get data in MongoDB GridFS like files. See, MongoDB stores large objects as chunks of data as well as its metadata. It stores it in chunks are BSON objects in MongoDB are limited to 4 MB in size. The chunk approach allows you to fetch a range of data, which could come in handy. Of course, there are articles covering GridFS already, so I’ll just point you to them instead (by “them” I mean “this specific article” – deal with it). Definitely worth checking out.

No related posts.

A Popurls Clone with PHP, jQuery, Awesomeness

william — Tue, 23 Mar 2010 13:21:57 +0000

Good people of the Internets, I know a good deal of you frequently visit some sort of social website and/or aggregator such as Popurls, Reddit, Digg, Developer Zone, or what have you. After a deep and thorough analysis of my visitor statistics (read: pulling data out of my ass), many of you seem to like articles about PHP. This is not surprising, as PHP is a very popular (scripting) language. Then again, why anybody would want to read my incoherent ramblings is still a mystery to me. If you’re reading this, kudos! You officially have nothing better to do. Encouraging, isn’t it? Btw, Reddit, thanks for /r/nsfw/.

Since I have a crazy workload right now, I feel this is the perfect time to write a quick n’ dirty tutorial on how to build your very own Popurls. Impress your friends and/or boss with a nifty, hand made news aggregator. Yes, very buzz word friendly. This is part one of a series of articles where we start from this article’s code and later on expand it to employ a simple admin interface for the feeds as well as pulling our content and playing with it in Flash using ActionScript 3 and from there build a sweet little Flash widget. I am weirdly excited about this to the point of scaring myself, so let’s STFU and get started. If you want to see a demo, well, you’re in luck. Go there!

Abstract

What we’re going to do is use SimplePie to fetch an array of feeds of different formats, cache them and output the whole shebang. Then we’re gonna style it with some CSS and use some simple jQuery interactivity to sweeten the deal.

SimplePie is simple, not related to pi, tastes like it

We’ll start, obviously, with the back end. I should start with confessing that I’m an avid fan of the KISS and DRY principles. Instead of building our very own feed parser, we’ll use the excellent SimplePie class together with the IDNA Convert class. Don’t worry, the last one is bundled with the SimplePie 1.2 download. The IDNA Convert class helps us, amongst other things, to handle feeds where the domain name is internationalized according to RFC 3490, 3491, 3492 and 3454. IDNA is optional but as long as we already have it, why the fuck not?

First things first. Are you ready, willing and able to harness the awesomeness that is SimplePie? Assuming you have downloaded the package, unzipped/untarred it and uploaded to your web server/LAMP/WAMP, direct yourself to

http://[your_server]/[simplepie_path]/compatibility_test/sp_compatibility_test.php

Great success? Good, then let’s get started. As a side note, if you want to explore all the goodies that you get to play with , go to the SimplePie Documentation, which is a comprehensive API documentation.

Getting dirty

Before continuing, I should point out that SimplePie has automagical caching. It’s quite sweet. Just create a folder in the same directory as the SimplePie class called cache and chmod it so that the script may write to it. Let’s start by including the SimplePie library and the aforementioned IDNA Class as well ass initializing SimplePie and tell it which feeds to use. For the remainder of this article, I’ll assume that SimplePie is in the root of your folder and IDNA is in the folder idn.

require_once('simplepie.inc');
require_once('idn/idna_convert.class.php');

$popurls = new SimplePie();
$popurls->set_feed_url(
    array(
        'http://feeds.digg.com/digg/popular.rss',
        'http://www.reddit.com/.rss',
        'http://feeds.delicious.com/v2/rss/',
        'http://www.dzone.com/links/feed/frontpage/rss.xml',
        'http://feeds.wired.com/wired/index',
        'http://www.tuaw.com/rss.xml',
        'http://www.engadget.com/rss.xml'
    )
);

I twelve or so lines o’ code, we’ve done the dirty work for fetching the feeds that we want o play with. And who said dealing with code is hard? Pff. Thankfully, the SimplePie method set_feed_url accepts an array which makes our lives so much easier. Before getting down to the nitty gritty of displaying the feeds, you should start out with setting a parameter called set_item_limit. It does what you think, namely, limiting the number of items to return from each feed. If you don’t, chances are that you’ll have to wait for a long, long, looong time before actually seeing anything on your screen. That’ll make you come back here and add some snarky comment, claiming that the c0dez dun work lulz, n00b. Fair enough, the blog is called profeshunl newbie, but still. It’s about kharma. For our first proof-of-concept, a limit of three items per feed should suffice. While we’re at it, we’ll initialize the feeds and make sure that it’s served with UTF-8.

$popurls->set_item_limit(3);
$popurls->init();
$popurls->handle_content_type();

Not that hard, was it? Told you so. Now let’s set up a very simple XHTML structure below the PHP block in order to display the goods. For this part of the article, well settle for displaying the content in tables. Don’t try validating the code, though. It may seem a little bit slow the first time you runt this script, which is expected. When your browser’s done, check the cache directory. There should be seven .spc files there.




    p0wnurls
        


    p0wnurls

    

    get_items() as $feed_item ) {
        // Create a reference to the current item's parent feed
        $feed_parent = $feed_item->get_feed();
    ?>

        
            
                Favicon: get_favicon(); ?> (get_favicon(); ?>">)
            
            
                Permalink: get_permalink(); ?>
            
            
                Title: get_title(); ?>
            
            
                Date: get_date(); ?>
            
            
                Content: get_description(); ?>
            
            
                Description: get_description(); ?>
            
            
                Parent Permalink: get_permalink(); ?>
            
            
                Parent Title: get_title(); ?>

Still here? Nice. As you can see, more often than not the description is the same as the content. Also, we need to strip away the tags and crap that we don’t need. Basically, we only want the raw text. Let’s add a row to each table and see what we can come up with, mm’kay? Let’s try something like


    Description Stripped:get_description()), ENT_QUOTES, 'UTF-8'); ?>

Boom. Much better. html_entity_decode will convert html entities to their should-be characters. SInce we don’t need any images or, really, any inline elements, we’ll use strip_tags. Note that strip_tags does not in any way validate the HTML it parses. If the HTML is borked, it may lead to unexpected results, which is fancy talk for “it’ll blow up in your face and have you sleeping in the shower”. If you want to, you can always roll your own stripping mechanism with a regular expression. I wouldn’t recommend it, but if you’re the adventurous type or just don’t trust strip_tags, go ahead! The last link provides you with the following regex;

$regex = "/<\/?\w+((\s+(\w|\w[\w-]*\w)(\s*=\s*(?:\".*?\"|'.*?'|[^'\">\s]+))?)+\s*|\s*)\/?>/i";

…which should suffice as a starting point for you. And if you want to just strip whitespace, how about

$regex = '/(?:(?<=\>)|(?<=\/\>))(\s+)(?=\<\/?)/';

which was found over here.

Now that we know how to fetch the information we need, there is one thing that need to be sorted out. The default action for SimplePie is to aggregate all the feeds. This means that they’re all in one big list o’ items, sorted by date. Normally this would be fine, but we want a per feed sorting mechanism, right? Right. If you don’t you can safely skip this part. Then again, if you skip it, the rest of this article will be reeeally confusing.

Are we there yet?

I like playing with arrays, so let’s expand our initial code a bit. Let’s make an associative array of it and add a key for each feed. Let’s go with a nice name of the source domain. Manually for now. If you don’t like nested foreach loops, now’s your time to look away.

// Include SP
require_once('simplepie.inc');
require_once('idn/idna_convert.class.php');

// Instantiate a source array with domain short name as key and feed URL as value
$_source = array(
    'digg'=>'http://feeds.digg.com/digg/popular.rss',
    'reddit'=>'http://www.reddit.com/.rss',
    'delicious'=>'http://feeds.delicious.com/v2/rss/',
    'dzone'=>'http://www.dzone.com/links/feed/frontpage/rss.xml',
    'wired'=>'http://feeds.wired.com/wired/index',
    'tuaw'=>'http://www.tuaw.com/rss.xml',
    'engadget'=>'http://www.engadget.com/rss.xml'
);

// Instantiate empty array
$_sorted = array();

// Loop through the source array
foreach ( $_source as $_feed_key=>$_feed_value ) {

    // Instantiate new SP
    $_feed = new SimplePie();
    $_feed->set_feed_url($_feed_value);
    $_feed->init();

    // Loop through the items of the current feed
    foreach ( $_feed->get_items(0, 3) as $__item ) {

        // Add title, favicon and permalink to feed and items to the sorted array
        $_sorted[$_feed_key]['title'] = $_feed->get_title();
        $_sorted[$_feed_key]['favicon'] = $_feed->get_favicon();
        $_sorted[$_feed_key]['permalink'] = $_feed->get_permalink();
        $_sorted[$_feed_key]['feed_items'][] = array(
            'title' => $__item->get_title(),
            'date' => $__item->get_date(),
            'permalink' => $__item->get_permalink(),
            'content' => html_entity_decode(strip_tags($__item->get_description()), ENT_QUOTES, 'UTF-8')
        );
    } 

}

Short and sweet. Try a print_r( $_sorted ); and view source. Looks neat, doesn’t it? Note that set_item_limit has been removed in favor of specifying an offset and a limit to get_items() instead. set_item_limit wouldn’t work at all. As you can see from the date values, the items are sorted in ascending order. Jut the way we want it. The content part may be way too long though. We only need something like the first 100 characters of it to function as an excerpt. Let’s truncate the content. Yes, let’s. Add some sort of truncation function such as

// Truncate string
function truncate( $string, $length=100 )
{

    // Is it shorter than out string? If so, return it as is.
    if( strlen( $string ) <= $length )
    {
        return $string;
    }

    // No? Well, regex and substr the fuck out of that string and append an ellipsis at the end.
    return preg_replace('/\s+?(\S+)?$/', '', substr($string, 0, $length)).'...';
}

After that it’s only a matter of wrapping the value for the content key with the truncate function in the inner foreach loop

$_sorted[$_feed_key]['feed_items'][] = array(
    'title' => $__item->get_title(),
    'date' => $__item->get_date(),
    'permalink' => $__item->get_permalink(),
    'content' => truncate(html_entity_decode(strip_tags( $__item->get_content() ), ENT_QUOTES, 'UTF-8'))
);

Looking good, right? As you may notice, some feeds actually don’t have any content or just have a “submitted by…” kind of string, which no fun. Let’s add an ugly conditional that checks if the content is empty or, in this case, the feed is from Reddit. In that case, we’ll use the title as excerpt instead.

// Get the content
$_content = $__item->get_content();

// If it's empty or if it's Reddit...
if ( empty( $_content ) || $_feed_key == 'reddit')
{
    // Jut use the title
    $_content = $__item->get_title();
} 

$_sorted[$_feed_key]['feed_items'][] = array(
    'title' => $__item->get_title(),
    'date' => $__item->get_date(),
    'permalink' => $__item->get_permalink(),
    'content' => truncate(html_entity_decode(strip_tags( $_content ), ENT_QUOTES, 'UTF-8'))
);

Much better. However, we need to rearrange and restyle our code a bit. And lets get rid of those horrible tables and go with unordered lists instead.




    p0wnurls
    


    p0wnurls

    $feed )
    {
    ?>

        
        
            
                Favicon: (">)
            
            
                Permalink:
            
            
                     

                    
                        
                            Title:
                        
                        
                            Permalink:
                        
                        
                            Date:
                        
                        
                            Content:

So, are we there yet?

Starting to see the resemblance to Popurls? No? You douche. Well, actually, me neither. So lets fix that. First off, let’s add a container that we center on the screen. Then we’ll divide the feeds into columns of three. Then do some hacky crap to add a div after every third column. After that, we’ll divide each feed’s items in two – the top half is shown and the other part is hidden until we press a link. After that, we should style it like Popurls and include jQuery. Let’s begin with the body. This may seem a little messy. And that’s because it is.



    
    p0wnurls
    $feed )
    {
    ?>
        
            " alt="" />

Don’t go copy/pasting that code – as it turns out, my code highlighting script of choice in combination with WordPress strips away any plus signs, rendering the loops somewhat… lacking… Hm. Just check that $i and $j get incremented in the end of each loop and you should be fine. I added two very stealthy clues as to where to add the plus signs. Look for the clues, eagle eye, and report back later. Notice the rearrangement of code. We use the title for the permalink anchor as, well, the title, and put the content in a span. As a courtesy for all those of you who prefer to echo the content instead of multiple opening and closing php tags, here’s the inner loop for you to play with;

$j = 0;
foreach ( $feed['feed_items'] as $feed_item )
{
    if ( $j == $half_feed_items )
    {
        echo '
    
        
        
        ';
    }
    echo '
    
            ' . $feed_item['title'] . '
            ' . $feed_item['content'] . '
    
    ';
    $j  ;

    // Have we printed the last feed item? If so, close the ul up
    if ( $j == $total_feed_items )
    {
        echo '';
    }
}

Next up – CSS. This is nothing special and shouldn’t be hard to follow.

body {
        font-family: 'Lucida Grande', Helvetica, Arial, sans-serif;
        font-size:12px;
        text-align:center;
        background :#000000;
        color: #f3f3f3;
    }
    a {
        text-decoration:none;
        color: #83bacf;
    }
    #container {
        width:960px;
        margin: 0 auto;
        text-align:left;
    }
    .feed_container {
        float:left;
        width:320px;
        display:inline;
    }
    ul {
        padding: 5px 5px 5px 0;
        margin:0;
    }
    ul > li {
        list-style-type: none;
        margin:0;
        padding:0;
    }
    ul li span {
        display:none;
    }
    .feed_items li {
        padding: 5px 0;
        border-top: 1px solid #505050;
    }
    .more {
        display:block;
        background:#505050;
        height:10px;
    }
    .feeds_extra {
        display:none;
    }
    .clear {
        clear:both;
        padding: 0 0 10px 0;
    }
    #tooltip {
        position:absolute;
        border:2px solid #afa06b;
        background:#e4dd9c;
        padding: 5px;
        color:#333;
        display:none;
        width: 320px;
        text-align:left;
    }

Nothing fancy. The container has a width of 960 pixels and each feed gets a third each to play with. We added a dark background and colored the anchors blue and the tooltip yellowish. It matches Popurls color scheme. I think. We’ve added the necessary code for the tooltip as well. Speaking of tooltip, let’s include jQuery and the tooltip script. I’m going with jQuery 1.4.2 and a neat little tooltip that I found on css globe. The only problem is the way that script works. We have to alter it a bit in order to have it display our content in the tooltip. No biggie. We only need to change one line of code, namely line 21 from

this.t = this.title;

this.t = $(this).siblings().html();

so that it looks for content in our sibling span instead of the anchor’s title. Life’s good, right? Assuming that you put your javascript files in the root of your project, add some javascript in the head of the page

Simple. live() adds a handler to the event for all elements that match the selector no matte rif they’re in the DOM now or are added later. bind(‘click’, function(){ … }); would also do if that’s your preference. With me so far? I thought so. Let’s add the finishing touches to our main PHP code vlock so that it actually works. Who knew, right? We need to add two variables.

$total_feed_items = 10;
$half_feed_items = round( $total_feed_items/2 );

They’re used in the inner loop in the document body. Really, they are. We also need to change the hard coded value in the inner foreach loop to use our variable. Also note that we changed the number of feed items form 3 to 10.

foreach ( $_feed->get_items(0, $total_feed_items) as $__item )

To sum up, our PHP block should look something like this (comments removed)

function truncate( $string, $length=100 )
{
    if( strlen( $string ) <= $length )
    {
        return $string;
    }
    return preg_replace('/\s ?(\S )?$/', '', substr($string, 0, $length)).' ...';
}

require_once('simplepie.inc');
require_once('idn/idna_convert.class.php');

$_source = array(
    'digg'=>'http://feeds.digg.com/digg/popular.rss',
    'reddit'=>'http://www.reddit.com/.rss',
    'delicious'=>'http://feeds.delicious.com/v2/rss/',
    'dzone'=>'http://www.dzone.com/links/feed/frontpage/rss.xml',
    'wired'=>'http://feeds.wired.com/wired/index',
    'tuaw'=>'http://www.tuaw.com/rss.xml',
    'engadget'=>'http://www.engadget.com/rss.xml'
);

$_sorted = array();

$total_feed_items = 10;
$half_feed_items = round( $total_feed_items/2 );

foreach ( $_source as $_feed_key=>$_feed_value )
{
    $_feed = new SimplePie();
    $_feed->set_feed_url($_feed_value);
    $_feed->init();

    foreach ( $_feed->get_items(0, $total_feed_items) as $__item )
    {
        $_content = $__item->get_content();

        if ( empty( $_content ) || $_feed_key == 'reddit')
        {
            $_content = $__item->get_title();
        } 

        $_sorted[$_feed_key]['title'] = $_feed->get_title();
        $_sorted[$_feed_key]['favicon'] = $_feed->get_favicon();
        $_sorted[$_feed_key]['permalink'] = $_feed->get_permalink();
        $_sorted[$_feed_key]['feed_items'][] = array(
            'title' => truncate($__item->get_title()),
            'date' => $__item->get_date(),
            'permalink' => $__item->get_permalink(),
            'content' => truncate(html_entity_decode(strip_tags( $_content ), ENT_QUOTES, 'UTF-8'))
        );
    } 

}

YMMV. And the entire HTML page should look similar to this;




    
    p0wnurls

    
        
        
        



    
    p0wnurls
    $feed )
    {
    ?>
        
            " alt="" />

See, that wasn’t so hard, was it? The complete page is available here for your viewing pleasure. Again, note the lack of two plus signs in the inner loop. Instead of a nifty “More” button, we have a clunky grey bar. I leave it as an exercise for the reader to do something fun with it. That’s about it for this time. Next time we’ll see if we can’t make this code OO and add an admin interface to handle the feeds. Calm down, spanky, we’ll get there. Now I need to get some sleep. I’ve been watching Dexter a whole lot lately and he’s an inspiration, to say the least. The down side is that the show takes time from quality doing-jack-shit-on-the-Internet time. I have to see what I can do about that.

Hands-on tips for PHP security

Week 10 in review

william — Mon, 15 Mar 2010 22:50:02 +0000

Well, it’s time to sum up the week. Since this is my first sum-o-the-week, I really have no point of reference as to what to write here. I know, I know, today’s Monday and shouldn’t the week be summed up late Sunday? Well. Yes. But hear me out anyway. Please. I need my ego stroked and caressed.

My basis for analysis is my very own Twitter feed, so, yes, it’s biased as hell. Deal with it.

March 8

- PHPillow – OO wrapper for CouchDB http://bit.ly/cy5pix
The week started off quite slowly, to be honest, with PHPPillow. First a small introduction to CouchDB. What is CouchDB? In short, it’s a document database server built on Erlang which is accessible through a RESTful JSON API. Free form schema with a flat address space. The sweet part is that features a table oriented reporting engine that uses, get this, JavaScript as a query language. Did I mention it’s indexable? Please note, though, that it’s really not a replacement for a RDMS. Now, CouchDB employs what they call documents. A CouchDB document is an object that is basically constituted of named fields, which can be strings, numbers, dates or even associative maps or lists. If you’ve dabbled with anything that recebles JSON, you’ll feel right at home. A simple document may look like

"Subject": "I like Plankton"
"Author": "Rusty"
"PostedDate": "5/23/2006"
"Tags": ["plankton", "baseball", "decisions"]
"Body": "I decided today that I don't like baseball. I like plankton."

Example borrowed from here . This means that a CouchDB database is just a flat collection of documents where each document has a unique ID. How do you access these results, you may ask. As it turns out, CouchDB uses views in order to bring some sort of structure to the pseudo-structured data. Views are the way in which documents are aggregated and reported. THey are served ad hoc and don’t affect the underlying document. You may also have several views that represent the same data.

CouchDB is also a peer distributed database system, meaning that any number of hosts, i.e. servers and offline clients, can hade independent “replica copies” of the same database, where applications have full database interactivity. Any changes in the database are replicated in a bi-directional manner when the user is back online.

PHPillow proveds an object oriented wrapper for CouchDB. To connect, all you need is to instanciate it with something like

phpillowConnection::createInstance('localhost', 5984, 'user', 'password');

As soon as the connection has been established, the connection will be used in your document and view classes automagically. All documents extend the abstract base class phpillowDocument and a complete model that, say, defines a blog entry may look like the following

class myBlogDocument extends phpillowDocument
{
    protected static $type = 'blog_entry';

    protected $requiredProperties = array(
        'title',
        'text',
    );

    public function __construct()
    {
        $this->properties = array(
            'title'     => new phpillowStringValidator(),
            'text'      => new phpillowTextValidator(),
            'comments'  => new phpillowDocumentArrayValidator(
                'myBlogComments'
            ),
        );

        parent::__construct();
    }

    protected function generateId()
    {
        return $this->stringToId( $this->storage->title );
    }

    protected function getType()
    {
        return self::$type;
    }
}

Exampled borrowed from this place. This means that you can use the document in your code like so;

$doc = new myBlogDocument();
$doc->title = 'New blog post';
$doc->text  = 'Hello world.';
$doc->save();

Pretty nice, huh? Now, this is maybe not for your next Facebook scoped project, but I’m sure you’ll find a good use for it for something where a full fledged RDMS is a bit overkill. Go ahead and play with it. I dare ya.

March 9

- jQuery alternative documentation for 1.4.2. Downloadable http://bit.ly/c9AuA3

Everybody’s (well, maybe not everybody’s) favorite JavaScript library jQuery quite recently hit version 1.4.2. As with every new release of jQuery, it promises massive speed enhancements (here compared to other libraries), which makes you wonder just how awfully slow it must have been earlier. Moving on. There are already a number of decent-to-really-sweet jQuery documentations, but they mostly suffer form lack of updates. This one is brand spanking new, targetting version 1.4.2. Not only is it updated, it has a search function that actually doesn’t suck – fast and more accurate than its competition. It’s even available for download for those offline moments. Yeah, I know, you don’t have any offline moments, but bear with me here, it’s a nice feature.

- Get Twitter status via PHP and some jQuery http://bit.ly/b8zUyG

This is just a small tip that show you how to fetch a given user’s Twitter status via PHP and jQuery. Note that is uses a WordPress function called parse_feed so it isn’t al that useful if you’re not using WordPress. Weel, thankfully, there’s an example in the comments that uses file_get_contents instead. Happy days.

- How to crack RSA 1024 bit encryption by clusterf*ck http://bit.ly/c869tf

Well, this is more of a shameless self-promotion sort of link, so I’ll just link to my own article. Go on and play, now, y’hear?

March 10

I apparently decided to practice my coma skills as I did’t tweet for a whole day. I’m half the man I used to be. Which I guess would make me a quarter of a man, since, let’s face it, it was an up hill battle to begin with.

March 11

- PHP Security, some hands-on tips http://bit.ly/ddqJdj

Again a self-promotion piece of crap. There are some nice pointers there, so, yeah, check it out! It deals with validation, filtering and CSRF/XSS.

- 18 high quality fonts for your next design http://bit.ly/adGLeY

Fonts, fonts, fonts. Love ‘em or hate ‘em, you need ‘em. This is a pretty decent list of 18 quality fonts, several of which are free. It sports previews as well.

- HTML5 Web Sockets – quantum leap in scalability http://bit.ly/bwWa4c

This article actually had me quite excited. IT start off by explaining how transfer of information over the HTTP protocol works and just how much overhead there is in each transaction. Or as the men themselves put it

Simply put, HTTP wasn’t designed for real-time, full-duplex communication [with, for example, a] web application that displays real-time data from a back-end data source using a publish/subscribe model over half-duplex HTTP.

If you look at the Communications section, that’s section 9, for you lazy asses out there, of the HTML5 specification (PDF), you’ll get a comprehensive definition of Web Sockets. I’m guessing you won’t read it, so I’ll just continue by quoting the authors again;

Once established, WebSocket data frames can be sent back and forth between the client and the server in full-duplex mode. Both text and binary frames can be sent full-duplex, in either direction at the same time. The data is minimally framed with just two bytes. In the case of text frames, each frame starts with a 0×00 byte, ends with a 0xFF byte, and contains UTF-8 data in between. WebSocket text frames use a terminator, while binary frames use a length prefix.

For you Java heads, the article compares Comet with HTML5 Web Sockets. A good read, even if you’re not into Java. The results speak for themselves. The authors claim a 500:1 or even 1000:1 reduction in unnecessary HTPP traffic and a 3:! reduction in latency. You can read more on Web Sockets over at w3.org.

- Flash vs HTML5 comparison – who’s the worst CPU hog http://bit.ly/b4Bg2a

For some perverse reason, I kinda like articles that compare Flash with HTMl5 *cough* like this one – sort of *cough*. Jan Ozer did a really good job comparing watching video with Flash (10.0.45.2 first adn then 10.1.51.95) and HTML5 on both Windows and Macintosh computers with a range of browsers – Apple Safari (4.0.4), Mozilla Firefox (3.6), Google Chrome (4.0.239.89 on Windows, 5.0.307.9 on Mac) and Internet Explorer 8.0. A good read for both young and old. He concludes by stating that

[w]hen it comes to efficient video playback, the ability to access hardware acceleration is the single most important factor in the overall CPU load. On Windows, where Flash can access hardware acceleration, the CPU requirements drop to negligible levels. It seems reasonable to assume that if the Flash Player could access GPU-based hardware acceleration on the Mac (or iPod/iPhone/iPad), the difference between the CPU required for HTML5 playback and Flash playback would be very much narrowed, if not eliminated. … We also learned that not all HTML5 browsers/H.264 decoders are created equal. Significantly, with Flash 10.1 deployed, Google’s HTML5 implementation required the most CPU horsepower of all playback scenarios — by far — on the Windows platform. On the Mac, Firefox and Safari with Flash required less CPU horsepower than Chrome’s HTML5 implementation. At least from a CPU utilization perspective, Flash isn’t BAD and HTML5 isn’t GOOD. It all depends upon the platform and implementation.

Words of wisdom, right there.

March 12

- How-to: monitor your Facebook Page traffic http://bit.ly/9hpAeO

Given the massive amount of referring traffic that Faceboook can produce, it might be a good idea to measure the amount of traffic your Facebook presence generates. Nick O’Neill gives tips on using Webtrends Tool, Facebook Insights, Core Metrics and the grand daddy of analytics, Google Analytics with the help of a London based web development company.

- A day in the life of a programmer http://bit.ly/9CBvwa

A graph that is simply too true to be funny. But it’s funny, so check it out, laugh and slowly come to realize that it’s about you. Feel better? Me neither.

- Best yo-moma joke evar! http://bit.ly/cdYCl7

Don’t know why, but I giggled like a school girl, high on a hallucinogenic frog. I’m retadred like that. Spelling intentional.

March 13

- Detect Flash Blockers with JavaScript http://bit.ly/cNuny6
This piqued my interest. It claism to be able to detect FlashBlock 1 and for Chrome, FlashBlock for Firefox and ClickToFlash 1.5.x. I’m guessing this is of importance if your site has a lot of banners. What it does is inserting some dummy flash objects into the page and basicallywait to see if they get blocked. It requires a callback function which gets called almost instantaneously if a blocker is detected and a wee bit later if it times out. I should point out that it removes any dummy flash objects it creates. I hope.

- CodeIgniter 2 in the works, drops PHP4 http://bit.ly/bPzlKF

CodeIgniter 2 is in the works and, bug whoop, they’re dropping support for PHP4. Finally. Anybody still running on PHP4? I thought not. EllisLab is also moving away from Subversion in favor of Mercurial (http://mercurial.selenic.com/) and Assembla (http://assembla.com/). Also, Plugins are gone. What you have now are instead Helpers – one or more functions – and Libraries – lots of functions that share properties (i.e. classes). Good times.

- Top ten Linux distros compared http://bit.ly/dzZjyV

Distrowatch compiled a list of ten popular distributions. Pretty self explanatory stuff. The add pros and cons to their review of each distro as well as alternatives and available editions. That’s nice of them. If you visit this link, chances are that you’re already using one of the listed distros, in which case you are not likely to swap distro. So, really, the links is to no use to you. Unless you have no prior knowledge or preference of these sort of things. So the both of you visiting, you’ll like this link.

- 10 sweet free WordPress frameworks for designers http://bit.ly/97rtq4

I’m not really sure what I think about using WordPress frameworks, but I guess it’s truly awesome if you have limited scripting experience. Many of these also employ child themes, which efficiently turns your WordPress installation into a five to two hundred tier system. Yay redundancy! We like layers upon layers.

March 14

– Sweet SED one-liners http://bit.ly/bdczjZ

Good stuff for the CLI nerd in you. You know what I’m talking about, you hacker, you.

- Sony opens Apple St… er, Sony Store in Nagoya, Japan. http://bit.ly/daeofJ

I kinda feel like I’ve seen this sort of store somewhere before. I just… can’t… think of where…

- New Digg revealed http://bit.ly/9JW2OW – interview with CEO http://bit.ly/axo9Rt

New functions and layout, bla bla bla. Also the ability to submit stories anonymously. Yes. Gone are the days of 1000 diggs on a cracked.com link. Enter the era if three million diggs on a cracked.com link and more spam. Goody!

- Sweet PHP+jQuery Buzz stream reader http://bit.ly/b2mz4O
This was a really nice write up by Brian Wisti (http://coolnamehere.com/brian/index.html) on how to incorporate Google Buzz in your PHP project. It even caches the results for you. Good stuff.

There you have it. The week in hindsight was full of crap, as per usual. I’ll do better next time. I promise (kinda).

Hands-on tips for PHP security

william — Thu, 11 Mar 2010 12:00:29 +0000

I got asked to review a fairly large piece of PHP code recently and, whoooo boy, was I in for a treat (treat as in clawing my eyes out with a rusty spoon while listening to Nickelback, as interpreted by Dr Zoidberg. In reverse). This magnificent piece of code was an eye opener in many ways and it made me feel a little bit better about myself, to be honest. It employed not only what would be called “bad practice”, but also a lot of plain ol’ stupidity. I know you are just dying for examples and I am not one to deny you the satisfaction. Also note that if this resembles your own code in any way, just leave because we don’t take kindly to you kinds of people, y’hear? And bring your cousin dad too. The beast contained gems such as

$connect_db = mysql_connect("localhost","username","password");
if (!$connect_db) {
    die('Could not connect becaus of ' . mysql_error());
} else {
    mysql_select_db("some_random_db", $connect_db);
    $command="INSERT INTO table_name ('username', /* some other crap */)
    VALUES ('".$_GET['user']."' /* more crappy crap crap */)";
    if (!mysql_query($command,$connect_db)){
        die('Could not add user because ' . mysql_error());
    }
    echo "Successfuly added user ".$_GET['user']."";
}

Spelling errors not omitted. No cup of heavenly blessed Kopi Luwak coffee, poured into a cup of golden banana leaves by naked triplets could counter the massive damage done to my corneas or, for that matter, my soul. Or what was left of it. The kicker was that the developer claimed it was OO and showed me the following “controller”. This is paraphrasing a bit, but you’ll get the general idea.

class MySite
{
    var $page;
    function MySite($page) {
        $this->page = $page;
        $content = "page.php?page=".$this->page;
        /*
        This is where page.php returned the entire document
        */
        echo $content;
    }
}
$init =  new MySite($_GET['page']);

Yum.

Needless to say, everything was passed around in the URL. Nothing was escaped as it “would hurt performance”. The e-mail function only checked that there was an @ in the string. You get the picture. Of course, a refactor, and I use the term loosely as it really was a steaming pile of vomit that would need a complete rewrite, was mission critical and the investors didn’t like the idea that the site would be down for maintenance for a while. As it turns out, this started as a hobby project that got the attention of somebody with money and decided that it would make a nice platform for a thriving business.

So this little article will deal with some security related matters such as input filtering. I expect there will be a second part to this as there is a plethora of things to consider, but this’ll do for now.

First off, I’d like to introduce you to a wonderful library without external dependencies called Inspekt. Mr. Ed Finkler of Funkatron built Inspekt upon the now deprecated Zend_Filter_Input, by Chris Shiflett.

[Insert Inspector Gadget joke here]

Inspekt acts as a firewall API between user input and the rest of the application. It takes PHP superglobal arrays, encapsulates their data in an “cage” object, and destroys the original superglobal. Data can then be retrieved from the input data object using a variety of accessor methods that apply filtering, or the data can be checked against validation methods.

Everybody wins! In order to take advantage of this black magic, all you need to do is something like this;

require "Inspekt.php";
$input = Inspekt::makeSuperCage();

Okay, what have we done here? When the library’s been required, we create an Inspekt_Supercage – that’s a single object that contains each input superglobal as an Inspekt_Cage. Inspekt_Cage objects encapsulate arrays of data. The original array is destroyed so you always have to access the data through the Input_Cage object’s methods. Say you need to check if $_POST['important_id'] really is an integer. Note that we are extending the introductory example above.

$input->post->testInt('important_id');

If it is not an integer, it will return false. More on that in a bit. What’s nice about Inspekt is that you can specify the kind of cage to create, i.e. a cage for just $_GET or $_POST, etc. You have the following tools to play with;

Inspekt::makeGetCage(); // Returns an Inspekt_Cage for the $_GET array
Inspekt::makePostCage(); // Returns an Inspekt_Cage for the $_POST array
Inspekt::makeCookieCage(); // Returns an Inspekt_Cage for the $_COOKIE array
Inspekt::makeServerCage(); // Returns an Inspekt_Cage for the $_SERVER array
Inspekt::makeFilesCage() // Returns an Inspekt_Cage for the $_FILES array
Inspekt::makeEnvCage() // Returns an Inspekt_Cage for the $_ENV array

Isn’t that nifty? The Inspekt_Supercage has no less than six public properties, such as get for $_GET, post for $_POST, etc. It correlates to the list just above. I promise. You access them by something like the following (again, extended from our first makeSuperCage example);

$input->get->testEmail('email');

If it’s valid, it will return the value. So, what else do we have in our toolbox? Well, the cage is all good and well, but we need some sort of properties to actually do the filtering or examining values. Let’s start with some tools for testing, shall we? The testers are neat methods to check the value on a given key. They return the value of the key if it’s good and false if it fails.

testAlnum (mixed $key)
// Returns value if every character is alphabetic or a digit

testAlpha (mixed $key)
// Returns value if every character is alphabetic

testBetween (mixed $key, mixed $min, mixed $max, [boolean $inc = TRUE])
// Returns value if it is greater than or equal to $min and less than or equal to $max.
// If $inc is set to FALSE, then the value must be strictly greater than $min and strictly less than $max.

testCcnum (mixed $key, [mixed $type = NULL])
// Returns value if it is a valid credit card number format.
// The optional second argument allows developers to indicate the type.

testDate (mixed $key)
// Returns $value if it is a valid date. The date is required to be in ISO 8601 format.

testDigits (mixed $key)
// Returns value if every character is a digit

testEmail (mixed $key)
// Returns value if it is a valid email format

testFloat (mixed $key)
// Returns value if it is a valid float value

testGreaterThan (mixed $key, [mixed $min = NULL])
// Returns value if it is greater than $min

testHex (mixed $key)
// Returns value if it is a valid hexadecimal format

testHostname (mixed $key, [integer $allow = ISPK_HOST_ALLOW_ALL])
// Returns value if it is a valid hostname

testInt (mixed $key)
// Returns value if it is a valid integer value

testIp (mixed $key)
// Returns value if it is a valid IP format

testLessThan (mixed $key, [mixed $max = NULL])
// Returns value if it is less than $max

testOneOf (mixed $key, [ $allowed = NULL])
// Returns value if it is one of $allowed

testPhone (mixed $key, [ $country = 'US'])
// Returns value if it is a valid phone number format. The optional second argument indicates the country.

testRegex (mixed $key, [mixed $pattern = NULL])
// Returns value if it matches $pattern. Uses preg_match() for the matching.

testUri (string $key)
// Returns value if it is a valid URI as defined in http://www.ietf.org/rfc/rfc2396.txt

testZip (mixed $key)
// Returns value if it is a valid US ZIP

Sexy, isn’t it? And in order to access your values after they’ve been filtered, you may use the following methods;

getAlnum (mixed $key)
// Returns only the alphabetic characters and digits in value.

getAlpha (mixed $key)
// Returns only the alphabetic characters in value.

getDigits (mixed $key)
// Returns only the digits in value. This differs from getInt().

getDir (mixed $key)
// Returns dirname(value).

getInt (mixed $key)
// Returns (int) value.

getPath (mixed $key)
// Returns realpath(value).

getPurifiedHTML (string $key)
// This returns the value of the given key passed through the HTMLPurifer object,
// if it is instantiated with Inspekt_Cage::loadHTMLPurifer

getROT13 (string $key)
// Returns ROT13-encoded version

getValue (string $key)
// Retrieves a value from the cage

getRaw (string $key)
// Returns value, unfiltered.

noPath (mixed $key)
// Returns basename(value).

noTags (mixed $key)
// Returns value with all tags removed.

noTagsOrSpecial (mixed $key)
// Returns value with tags stripped and the chars '"&<> and all ascii chars
// under 32 encoded as html entities

And of course, there is

keyExists (mixed $key)
// Checks if a key exists

It wouldn’t be complete if there where no methods for escaping strings prior to database calls. Well, aren’t we in luck?

escMySQL (mixed $value, [resource $conn = null])
// Escapes the value given with mysql_real_escape_string

escPgSQL (mixed $value, [resource $conn = null])
// Escapes the value given with pg_escape_string

escPgSQLBytea (mixed $value, [resource $conn = null])
// Escapes the value given with pg_escape_bytea

I should point out that you actually can wrap any array in Inspekt_Cage and not just superglobals, making the library a handy dandy tool for all your filtering and testing needs.

$dirty = array( /* imagine some random, unfiltered values here */ );
$johnny_cage = Inspekt_Cage::Factory($dirty);

to test some value in the array, just use

$johnny_cage->testInt('key');

To get even more nitty gritty, you can use static methods for those one off moments.

Inspekt::isAlnum (mixed $value)
Inspekt::isAlpha (mixed $value)
Inspekt::isBetween (mixed $value, mixed $min, mixed $max, [ $inc = TRUE])
Inspekt::isCcnum (mixed $value, [mixed $type = NULL])
Inspekt::isDate (mixed $value)
Inspekt::isDigits (mixed $value)
Inspekt::isEmail (string $value)
Inspekt::isFloat (string $value)
Inspekt::isGreaterThan (mixed $value, mixed $min)
Inspekt::isHex (mixed $value)
Inspekt::isHostname (mixed $value, [integer $allow = ISPK_HOST_ALLOW_ALL])
Inspekt::isInt (mixed $value)
Inspekt::isIp (mixed $value)
Inspekt::isLessThan (mixed $value, mixed $max)
Inspekt::isOneOf (mixed $value, [ $allowed = NULL])
Inspekt::isPhone (mixed $value, [ $country = 'US'])
Inspekt::isRegex (mixed $value, [mixed $pattern = NULL])
Inspekt::isUri (string $value, [integer $mode = ISPK_URI_ALLOW_COMMON])
Inspekt::isZip (mixed $value)
Inspekt::getAlnum (mixed $value)
Inspekt::getAlpha (mixed $value)
Inspekt::getDigits (mixed $value)
Inspekt::getDir (mixed $value)
Inspekt::getInt (mixed $value)
Inspekt::getPath (mixed $value)
Inspekt::noPath (mixed $value)
Inspekt::noTags (mixed $value)

Built in goodies

If you feel that using a library isn’t cool enough, or if I haven’t made the benefits clear, there are some things you ca do for filtering your values. I’m talking about the often overlooked PHP Filter functions. These are available if you are using at least PHP 5.2. Before continuing, I want to point you to the types of filters available. They’re validation filters, sanitizing filters, miscellaneous filters and, well, the predefined constants. First off is filter_var.

filter_var ( mixed $variable [, int $filter = FILTER_DEFAULT [, mixed $options ]] )

filter_var does what you think it does – it filters a variable with the specified filter. Go tautology! the $filter is specified in the list in the previous paragraph. The $options are an array of options or bitwise disjunction of flags. The function returns the filtered data or false if it fails. A quick example of e-mail validation could be

$is_it_okay = filter_var("some_dude\X00= _@ne@example..com", FILTER_VALIDATE_EMAIL);

Then there’s filter_var_array, which allows you to check an array of values.

filter_var_array ( array $data [, mixed $definition ] )

An example could be (borrowed from this place)

$data = array('bold', '', 'P*}i@893746%%%p*.i.*}}|.dw');
$myinputs = filter_var_array($data,FILTER_SANITIZE_STRING);
var_dump($myinputs);

//OUTPUT:
//formarray(3) { [0]=> string(4) "bold" [1]=> string(10) "javascript"
[2]=> string(26) "P*}i@893746%%%p*.i.*}}|.dw" }

Then we have filter_input which gets an external variable by name and optionally filters it.

filter_input ( int $type , string $variable_name [, int $filter = FILTER_DEFAULT [, mixed $options ]] )

$type may be either INPUT_GET, INPUT_POST, INPUT_COOKIE, INPUT_SERVER or INPUT_ENV ( INPUT_SESSION and INPUT_REQUEST are not implemented yet). Returns the requested variable on success, false if not or null if the $variable_name is not set (unless you use the flag FILTER_NULL_ON_FAILURE – in that case it returns false if the variable is not set and null if it fails). Say you want to sanitize $_GET['search'] for special characters. Try something like

$search = filter_input(INPUT_GET, 'search', FILTER_SANITIZE_SPECIAL_CHARS);

There is also filter_input_array, which does what you think it does – It filters an array instead of just a single variable.

$example_filters = array (
    "url" => FILTER_VALIDATE_URL,
    "ip" => FILTER_VALIDATE_IP,
    "temeraturep" => array(
        "filter"=>FILTER_VALIDATE_INT,
        "options"=>array(
            "min_range"=>1,
            "max_range"=>666
         )
    )
);

$sweetness = filter_input_array(INPUT_POST, $example_filters);

A quickie is filter_has_var, which checks if a variable of a specified type exists or not.

filter_has_var ( int $type , string $variable_name )

$type can be either INPUT_GET, INPUT_POST, INPUT_COOKIE, INPUT_SERVER or INPUT_ENV.

CRSFXSSFRSFCSFCSS X XSS SS XX C FFCCCE

A note on CRSF and XSS as well, while we’re at it. Inspekt allows you to run your data through the getPurifiedHTML method which basically runs it through HTMLPurifier and should provide a reasonable amount of XSS protection. If you have any tips on other XSS prevention projects or functions, please let me know. I stumbled upon a related project, csrf-magic. It uses PHP’s output buffering to “dynamically rewrite forms and scripts” and also check tokens in POST requests. You call it with

include_once 'csrf-magic.php';

It will even dynamically rewrite AJAX requests that uses XMLHttpRequest, if you want it to. Pretty sweet. It’s supposed to play along quite nicely with jQuery, MooTools, Prototype, Ext, Dojo, Yahoo UI Library, script.aculo.us and Prototype. There is also an option to set a configuration in the PHP script. Check out the following (borrowed form the bundled readme);

// This gets called if a csrf check fails
function my_csrf_callback() {
    echo "You're doing bad things young man!";
}

function csrf_startup() {

// While csrf-magic has a handy little heuristic for determining whether
// or not the content in the buffer is HTML or not, you should really
// give it a nudge and turn rewriting *off* when the content is
// not HTML. Implementation details will vary.
if (isset($_POST['ajax'])) csrf_conf('rewrite', false);

// This is a secret value that must be set in order to enable username
// and IP based checks. Don't show this to anyone. A secret id will
// automatically be generated for you if the directory csrf-magic.php
// is placed in is writable.
csrf_conf('secret', 'ABCDEFG123456');

// This enables JavaScript rewriting and will ensure your AJAX calls
// don't stop working.
csrf_conf('rewrite-js', '/csrf-magic.js');

// This makes csrf-magic call my_csrf_callback() before exiting when
// there is a bad csrf token. This lets me customize the error page.
csrf_conf('callback', 'my_csrf_callback');

// While this is enabled by default to boost backwards compatibility,
// for security purposes it should ideally be off. Some users can be
// NATted or have dialup addresses which rotate frequently. Cookies
// are much more reliable.
csrf_conf('allow-ip', false);

}

// Finally, include the library
include_once '/path/to/csrf-magic.php';

If you would like to roll your own CSRF solution, a simple one can be scripted on a coffee break. When dealing with forms, add a random token to your $_SESSION after calling session_start() such as;

$_SESSION['my_token'] = md5(uniqid(rand(), TRUE));

And then in the form itself, add a hidden input like so;

After submit, you can check if the token is valid with something simple as;

if ($_POST['my_token'] == $_SESSION['my_token']){
    /* Fun stuff */
}

You may ask, why don’t we limit the amount of time a token is valid? Yes, well, let’s do that then.

// Add token and time
$_SESSION['my_token'] = md5(uniqid(rand(), TRUE));
$_SESSION['my_token_time'] = time();

// Assume we now submitted the form, yadda yadda

$are_we_screwed = time() - $_SESSION['my_token_time'];

if ($_POST['my_token'] == $_SESSION['my_token']){
    /* It's all good! Or is it? */
    if ( $are_we_screwed < 60 ) {
        /* Less than a minute to play with */
    } else {
        /* Fail */
    }
}

There you have it. Some quick tips and/or things to think about. I’m not sure how I can end this little tidbit, so I’ll just paste a picture to keep you busy. Mm’kay?

Update: Reddit user pashj gave me a tip on adding a link to XSS examples so you can test you own project for, well, attack vectors. A classic cheat sheet full of examples is this one over at ha.ckers.org. Try it out! Let me know how it goes.

A Popurls Clone with PHP, jQuery, Awesomeness

How-to: crack 1024 bit RSA encryption by clusterf*ck

william — Tue, 09 Mar 2010 15:58:25 +0000

Granted, it’s a how-to on a theoretical level, but still. The RSA algorithm is used for for public-key cryptography and was publicly described in 1978 by messieurs Ron Rivest, Adi Shamir and Leonard Adleman (in case you read this before you’ve had your first cup of coffee of the day – the initials of their surnames spell RSA. You’re welcome). It was released to the public domain on the 6th of September, 2000. The abstract of the patent explains that what we’re talking about here, is basically

[a] communications channel coupled to at least one terminal having an encoding device and to at least one terminal having a decoding device.

That’s all well and good, but what does that mean?

A message-to-be-transferred is enciphered to ciphertext at the encoding terminal by first encoding the message as a number M in at predetermined set, and then raising that number to a first predeterminated power … and finally computing the remainder, C, when the exponentiated number is divided by the product of two predetermined prime numbers … [where] [t]he residue C is the ciphertext. The ciphertext is deciphered to the original message … by raising the ciphertext to a second predetermined power … and then computing the residue, M’, when the exponentiated ciphertext is divided by the product of the two predetermined prime numbers associated with the intended receiver. The residue M’ corresponds to the original encoded message M.

There you have it. Now, three computer scientists at the University of Michigan, Andrea Pellegrini, Valeria Bertacco and Todd Austin, claim to have found a way (PDF) to exploit a weakness and they start off by explaining that computer security boils down to having both secure hardware ans well as secure software. Compromise to the hardware layer may cause information in the software layer to be exposed, while being very hard to detect. They explain, in detail, “… an end-to-end attack on a microprocessor system and practically demonstrate how hardware vulnerabilities can be exploited to target secure systems”. The attack is an injection of “transient faults” in the target machine by, get this, “regulating the voltage supply of the system”. This means that the attack doesn’t require direct access to the target’s internal components but proximity to it.

There are three major blocks to consider in the paper; first off, they develop a “systematic fault-based attack on the modular exponentiation algorithm for RSA”. Then they exploit a flaw in OpenSSL‘s implementation of the RSA signature algorithm. Lastly, a physical demonstration of a fault-based security attack. The attack targeted the OpenSSL authentication library running on a SPARC Linux system and extracted the system’s 1024-bit RSA private key in 104 hours, using a 81 2.4 GHz Intel Pentium4-based Linux cluster with the OpenMPI libraries where one computer acted as the server and the remaining 80 a slaves. “The master distributed around 110 messages to each slave for checking. Individual slaves could check a message against a single potential window value and all fault locations and squaring iterations in about 2.5 seconds. During the analysis, the master directed all slaves to check their own messages for a particular single-bit fault in a particular window of the FWE computation. To reduce the time for synchronizing slaves, they divided their messages into 4 equal-size groups, and processed these groups serially until the value of the key window was found”.

This is pretty wild for a couple of reasons. First off, this sort of attack doesn’t damage the device, thus eliminating any evidence. OpenSSL is basically everywhere and even though the private keys contain more than 1 000 digits of binary code, which would take longer than the age of the universe to guess, voltage tampering cut that number down to roughly 100 hours. The hardware needed to do the actual tampering is quite inexpensive and simple and the varying voltage stresses the computer, causing it to produce small mistakes in its communication. These mistakes reveal ever so small pieces of the private key. When enough faults had been caused, the key could be reconstructed offline. Thankfully, the researchers hinted at a solution as simple as salting, which randomizes the order of the digits every time the key is requested. Still, not that shabby for a 30+ year old piece of technology. Or math. Whatever. The press release can be found right about here.

I’d like to leave you with some words of wisdom from an Engadget user.

If humans make it, Humans [sic] can destroy it.

Word.

No related posts.

Random thoughts on JavaScript. Part one of who knows. The basics.

william — Sun, 07 Mar 2010 15:42:48 +0000

This is going to be some sort of article on JavaScript. I’m still not sure exactly what will come of this, which means that now is your time to turn back. I won’t give you any more chances! Here’s something to keep you entertained while you think it through. And you may have to increase the volume while you watch it as it is just that mind numbingly awesome.

Still here? Cool. I wouldn’t be, but you know, to each his own.

People much smarter and more patient than me has written a lot about JavaScript. I guess that link should suffice as an article, but that would be a bit of a cop out on my part so I think I’ll give writing some random thoughts a try. It may just be a couple of pointers or tips. Or not. Hold my hand as we take the leap into the unknown.

First off, you know that you can try out your JavaScript online, right? Good. Now, let’s start with some JavaScript basics. And I mean really basic stuff.

Comment your code, but don’t go crazy with it.

var i = 0; // Declare i and set it to zero

A bag full of hurt. Use the good ol’ double slash for single line comments as per above. Use block comments for documentation or commenting out pieces of code.

/*
This is commented out
Isn't this pretty?
*/

Notation, notation, notation

In JavaScript, you can access properties of objects in two ways; square bracket notation and dot notation.

mySweetObject["property"] // Square bracket notation
mySweetObject.property // Dot notation

It may be tempting to use the dot notation as it’s a bit easier to type – because all coders are lazy, right? The downside of dot notation is that the property name is hard coded and thus can’t be altered at run time. Bracket notation, however, means that the property name is a string that gets evaluated. This means that you can use a variable, a hard coded string or a function call. Also, should the property be generated at run time, a bracket notation is required. Let’s start with a silly little example and say that you have a number of properties named “myValue1″, “myValue2″, “myValue3″ and so on. For some reason you need to loop through these properties using a for loop. If you try to access these properties like this

mySweetObject.myValue+i

you’ll fail. What you need is

mySweetObject["myValue"+i]

Boom, you’re in. So for the sake of consistency, use square bracket notation. Consistency is good. Repeat as a mantra.

Typecasting

The wonder that is JavaScript is weakly typed which means that you do not need to cast, or define, the type of data you are using. JavaScript, or rather the runtime system, performs automatic adjustments so you do not need to worry about it. This is either a good thing or a bad thing, depending on your background. Good for new programmers, bad for people used to C or Java (any language with strong typing). There are however times when it can be useful to typecast, such as when you are working with both strings and numbers. Since the plus sign is used both for string concatenation as well as regular addition, you might get some unexpected results. You can cast to three different types – Boolean, Number and String.

myBoolean1 = Boolean(666); // true
myBoolean2 = !! 666; // true
myString1 = String(666); // "666"
myString2 = "" + 666; // "666"
myNumber1 = Number("666"); // 666
myNumber2 = 1 * 666; // 666

You can try it out and make sure that it really gets typecasted by alerting with the help of typeof.

alert( typeof(myBoolean2) );

If we’ve behaved, that should alert “boolean”. The Boolean type returns true when the value passed to it is a string with at least one character, a number other than 0 or an object. E contrario it returns false when passed an empty string, the number 0, undefined or null. Number converts the entire value to a number, as compared to parseInt() and parseFloat(). The latter methods will only convert up to the first invalid character they encounter in a string. “6.6.6″ with parseInt() becomes “6″ and becomes “6.6″ with parseFloat(). Make a good habit of defining the radix with parseInt(). The second argument that you can pass to it and allows you to convert strings in binary, octal, hexadecimal or any other base into an integer. This may also be a good time to read up on floating point values. With Number() you get a “Not a Number” error, or the dreaded NaN. String should be a no-brainer. It’ll convert the value you want to a string, simple as that. What it does is call the toString() method. Note that String() will not result in an error for a null or undefined value – instead you get the string “null”.

Reserved words

This is something that can cause some confusion. I don’t know why, but I’ve seen my fair share of code where a developer uses a reserved word as a property name. It makes me kind of sad. The list of reserved words is a rather long one and pasting it here would just take up a lot of space, so instead I’ll just link to a page that has already done the dirty work. Please, please, please note the words that are reserved for future use, in the brand spanking new JavaScript 2.0 *cough* ECMA Script 4.0 *cough*. This will also leads us to…

Naming

See, there are limitations on the way can name things. You may use upper and lower case letters (a-z and A-Z), digits (0-9), and the underscore character (_). Although you can use other characters, it may break in certain cases. Better safe than sorry, right? Avoid starting names with the underscore character as it’s not rarely used to indicate privacy (more on that later on). Note that it doesn’t in itself create privacy.

Declare your variables

We know that JavaScript doesn’t require them to be declared, but, really, is it that hard? No. There is no excuse not to. You declare a variable with the use of the var keyword. Side note: If you come from C, you may need to know that JavaScript does not have block scope so be a good guy, and declare all your variables at the very top of your functions.

var myVar;
var myOtherVar = myRandomValue;

It makes it so much easier to identify undeclared variables that may, brace yourself, be implied globals. Implied globals can cause mayhem (mayhem as in a-minor-inconvenience-I-need-to-shape-up-my-code, not as in genocide). You see, the global object is the one namespace to rule them all. It holds, as you may have gathered by now, global variables and top level functions. It doesn’t take a mastermind to realize that collisions may occur and those collisions are part of the mayhem mentioned earlier. It will break your code. Basically, all JavaScript files you use run in the same scope which means that if you use globals, implied or not, and a script that loads after yours use the same variable or function names, the latter will replace your variables and functions. And that’s not good, is it? On a related note, you can declare multiple variables using a single var keyword and a comma separated list or variables.

var myVar1 = "Random",
    myVar2 = "Ramblings",
    myVar3 = "Fun stuff", myVar4, myVar5, myVar6;

Going global

Let’s back up for a moment and remember that in JavaScript, everything is an object. Or rather, a property of an object. In turn, an object is basically a collection of name-value pairs. The names are strings and values can be objects, numbers, booleans and strings. And a quick note on terminology – if the value is a function, strictly speaking, it’s called a method. But what about those functions, you ask? I’ve seen those around the code, you say. Well, those are actually properties to the window object.

function myFunction(myString) {
    alert(myString);
}

This little nifty function that alerts the variable string, actually means that we assigned a Function instance to the myFunction property. Functions do not have names. Actually, myFunction is a property of the window object. Does that make sense? It may be a bit more clear if we type it like so:

window.myFunction = new Function('myString', 'alert(myString)');

Feel a bit better now? Standalone functions are bound to the window object. When you would like to execute myFunction, you would normally type

myFunction('alert this string');

Nothing fancy. This can also be written as

window.myFunction('alert this string');

Remember, we are now dealing with global objects, which is in general a no-no, but nice for a simple explanation. Do you remember how we declared variables? When we type something like

var myVar = 'My string'

we are using a shorthand syntax (object literal – more on that a few lines down) for

var myVar = String('My string');

And these are also in the window object, i.e.

window.myVar = 'My string';
window.myOtherVar = String('My string');

Literal this, literal that

Thankfully, there are ways around the problem of collisions. One way is to employ object literals (see, I told you we’d get there) which basically means that you can define your own namespace (read: object or, going with OOP terms, a Constructor. Please note that it is Good Practice™ to capitalize the Constructor. I won’t in this case, because that’s just how i roll) and play with it. Or, as specified by the good people at Mozilla:

An object literal is a list of zero or more pairs of property names and associated values of an object, enclosed in curly braces ({})

Take this super simple example;

var myNameSpace = {
    myFunction:function(data) {
        alert(data);
    }
}

If you now try to call

myFunction('Alert this, you douche');

it will fail, as there is no property called myFunction in the window object. Instead, you need to call it using

myNameSpace.myFunction('Alert this, you douche');

That works just fine and if a script later on also has a myFunction() (preferrably in its own namespace), there will be no collisions. Goodie! The drawback of this technique is that you always have to go through the name of your object, or namespace. The solution would be to wrap your object in an anonymous function (or lambda). This will protect the scope. Our little snippet above would be typed like this:

var myNameSpace = function() {
    function myFunction(data) {
        alert(data);
    }
}();

Note the addition of the parenthesis on the last line and the ending semicolon. Also, you don’t need to stray away from the use of function myFunction() if that feels more comfortable. The parenthesis at the end makes the anonymous function execute right away. This is called the Module Pattern. This has the downside – or upside depending on your goals – of making myFunction() only available from within myNamespace. Any outside calls will fail (except maybe…). To fix this, you need to make a publicly available return statement.

var myNameSpace = function() {
    return {
        myFunction:function(data) {
            alert(data);
        }
    }
}();

Isn’t this fun? myNameSpace.myFunction(‘hey’); gets you going aaaand you’re pretty much back where we left off. Let’s alternate the example;

var myNameSpace = function() {
    function myFunction(data) {
        alert(data);
    }
    return {
        pointer:myFunction
    }
}();

See what we did there? We switched back to the other syntax for the private method and returned a pointer to that method instead. In this case, we call our function with

myNameSpace.pointer('hey');

Pretty sweet, right? The pointer can be as short as you want it and the private function can have any name you want, if that’s your preference. It makes me cringe a bit to mention public and private methods when dealing with JavaScript, but what the hell. Private methods or members are those that are created within the constructor. In the example above, myFunction is only accessible within the constructor myNameSpace and is returned only with the pointer or alias.

The reason that this works is due to the fact that JavaScript has closures. In essence, this means that a function defined within a function always has access to the vars and what have you of its outer function. Think about that for a while. That’s power, right there. Note that private and privileged methods can only be made when the object in questions is constructed. Public methods on the other hand, can be added ad hoc.

Friends with privileges

Although it might not be obvious, you also have the ability to create privileged methods. Privileged methods are accessible from outside the scope and can access private methods and variables. While you can delete or replace a privileged method, you can’t alter it. This is where you use the keyword this. This points to the method calling object.

function Constructor(data) {
    this.name = data;
}

This means that if we now construct a new object like so

var myConstructor = new Constructor('Gandhi');

myConstructor.name is “Gandhi”. Let’s expand.

function Constructor(data) {
    this.name = data;
    this.fetch = function() {
        return this.name;
    }
}

Okay, what can we do with this little gem of a function? Test it out with something basic like

var myConstructor = new Constructor('Gandhi');
alert( myConstructor.fetch() );

Did it work? Nice. It didn’t? Fail. Try again. Also try these quick examples:

var myConstructor1 = new Constructor('Gandhi');
var myConstructor2 = new Constructor('Shiva');
alert( "myConstructor1's name is "+myConstructor1.fetch() );
alert( "myConstructor2's name is "+myConstructor2.fetch() );

Did it work? Sweet. That’s about all I have to write for now. Fret not, there will be many more random thoughts coming quite soon, and not just on JavaScript. I’d like to leave you with an awesome talk from the JavaScript guru Douglas Crockford. Embedded below is a 45 minutes talk on Quality. It’s a good watch and, trust me, you’ll learn something new. It’ll make your code look better.

Douglas Crockford: “Quality” @ Yahoo! Video

No related posts.

Flash, King of the Impossible

william — Tue, 02 Mar 2010 22:36:55 +0000

If you have been using a web browser to surf the Internet at any point in time since 1996 (thank you, Tim Berners-Lee), you’ve probably encountered Adobe Flash in one way or another. In short, you have. Be it a CPU hogging banner on your favorite news site or a special CPU hogging movie at your favorite web 2.0 porn site (you know it, I know it, the neighbour who steals your DSL knows it. Livejasmin.com is roughly #30 in Alexa’s ranking and PornHub’s at #55. Knowledge is power), the Flash Player is an integral part of the Internet as we know it. It has come a long way since FutureSplash Animator and it’s been a success story, to say the least. Flash Player 9 boasts a penetration rate [insert joke] of roughly 98%. With the latest iteration, version 10.1,initially announced on October 5th, 2009, now in beta 3 and a part of the Open Screen Project, Adobe has taken the step to expand the full Flash experience to more platforms, i.e. mobile platforms, instead resorting to of half-breeds such as Flash Lite. One of the several vantage points of 10.1 is, or rather will be, the usage of GPU for accelerated video and graphics, which is said to create an 87% improvement in software rendering speeds and a mobile phone memory consumption reduction of 55%. Also, there is talk of native support for multi touch, gestures, accelerometer, screen orientation, to name a few features. Do i have your attention? Good, because there are some video demos of how it works in several devices such as the NVIDIA Tegra-powered tablet, HTC HD2, Dell Mini 5 tablet, Palm Pre, Motorola Droid, Google Nexus One and HP TouchSmart, and you can watch them all right about here. Off you go.

Another interesting aspect is that as of 10.1, private browsing, or “porn mode” for the rest of us, is available as a feature. That is, the Flash plugin respects the web browser if it is in porn mode and does the same. or as Jimson Xu and Tom Nguyen at Adobe put it;

Starting with Flash Player 10.1, Flash Player actively supports the browser’s private browsing mode, managing data in local storage so that it is consistent with private browsing. So when a private browsing session ends, Flash Player will automatically clear any corresponding data in local storage.
Additionally, Flash Player separates the local storage used in normal browsing from the local storage used during private browsing. So when you enter private browsing mode, sites that you previously visited will not be able to see information they saved on your computer during normal browsing. For example, if you saved your login and password in a web application powered by Flash during normal browsing, the site won’t remember that information when you visit the site under private browsing, keeping your identity private.

All in all, good times. One caveat – apparently it doesn’t work with Safari. Time to go Chrome?

As I haven’t had the opportunity to play around with the new features of 10.1 myself, I am forced to scour the Internet for clues, tips and ideas on how to use them. Personally, I’m very keen on the multi touch thingy. Luckily, I found this, written by none other than Lee Brimelow, Platform Evangelist at Adobe. This gem of an article tells me, among other things, that the Flash Player can respond to an infinite amount of touch points, only limited to the device’s touch points of course. You can even create your own gestures and capture the raw data. Sweet. Queue Japanese multi touch sex game.

Now, all isn’t cherry pies, golden showers and leprechaun smiles. Devices such as the iPhone and the iPad do not, and will probably never, natively support Flash. I’m deliberately skipping stuff such as Adobe Acrobat Connect Pro, or the Packager for iPhone, no matter how cool they seem. The latter actually allows you to package Flash applications as native iPhone apps and plublish them on the App Store. The same is the case with the iPad and Flash based applications. But I digress. Another big wtf is the exclusion of support for Windows Mobile 6.5. Adobe’s Antonio R Flores has stated that

[w]e have made the tough decision to defer support for that platform until WinMo7. This is due to the fact that WinMo6.5 does not support some of the critical APIs that we need

Yeah. Wow. Thankfully, WinMo has a limited user base *cough*. At least Android users are safe, right?

The HTC Hero will not be supported b/c it does not have the correct Anroid OS version and it’s chipset is not powerful enough. We require a device with an ARM v7 (Cortex) processor. Examples include the Qualcomm Snapdragon chipsets and TI OMAP3 series.

ARM V7 you say. Well, gosh darnit, cousin dad, don’t you actually require something like, I don’t know, Cortex A8? Soon enough they’ll require a Core 2 Duo – for that truly mobile experience. On a lighter note, 10.1 beta 3 actually has support for Intel’s GMA500 graphics chip, which means that at least four, maybe five, users in the world are more than thrilled about this piece of news.

Although Adobe is such a prominent player, there has been much talk of HTML5, CSS3 and other emerging technologies. In combination with JavaScript, either as raw, sexy code or with libraries such as jQuery, MooTools and Dojo Toolkit, you can accomplish much of the functionality that Flash applications can offer. Or as Steve Jobs put it (as published by Mashable);

Apple does not support Flash because it is so buggy. Whenever a Mac crashes more often than not it’s because of Flash. No one will be using Flash. The world is moving to HTML5.

In related news, you should check out the video of Ben Parr interviewing Aaron Filner, the group product manager for the Flash Platform. They discuss, inter alia, HTML5, Apple, the hover effect problem on touch screens (you know, with a mouse pointer, you can easily check for mouse events such as if the pointer is hovering a given link. With a touch screen you generally don’t have that option) and much more. Mashable recently polled their readers on which technology is better – HTML5 or Flash. In this epic web faceoff, HTML5 won by a broad margin; a massive 61% out of 6331 votes. Mr Steven Richter has published some valid thoughts on the matter.

Personally, I welcome HTML5+CSS3+JavaScript+Canvas+whatever and related technologies. Whole heartedly. See, I really don’t think that Flash and HTML5 (i’m using HTML5 as a common name for, well, a lot of things. Sorry.) need to compete. At all. If anything, the competition is healthy. It’ll hopefully lead to Flash being more open (one can dream) and focus on efficiency (especially on the Mac. I mean wtf? Also, focus on security *coug again*) while HTML5 needs some time and attention in order for smarter people than myself to build more impressive applications, extensions and features around it. Flash and HTML5 serve different purposes in many ways and let’s face it – you can just as well watch your favorite cat-doing-something-cute-or-stupid-clip on YouTube with HTML5. In fact, it’d probably not tax your CPU as much. For simpler websites, relatively speaking, HTML5 will suffice and still give the user a more than adequate experience with all the bells and whistles of many Flash sites. Not using Flash also has the benefit of more precise bookmarking and using the browser’s history in a less hacky way than with Flash’s deep linking. On the other hand, Flash has the ability to provide developers with features that are simply not possible with HTML5, such as 3D (like Swift 3D, Papervision 3D and Away 3D – compared to examples using Canvas such as this. To be fair, you need to check out WebGL. Looks really sweet so far), advanced audio controls and much more. I can’t wait to do a project with HTML5. And I can’t wait for my next Flash project – if the Flash IDE didn’t suck so much.

I’d like to leave you with pictures of a man who spent two years building a replica of Minas Tirith out of matchsticks, once again proving that a) there are more important things in life than bickering over stuff like I just have, b) there are people out there that are more awesome than you, and finally c) that there are more people other than me that need to get laid.

Oh, and I hope you got my Queen reference.

Turn your arm into a touchscreen, surfing for porn may get confusing, sticky

william — Tue, 02 Mar 2010 14:44:58 +0000

So a couple of bona fide geniuses have come up with a way to project, say, a keyboard or dial pad directly to your skin via a device that fits snugly on your upper arm. They call it the Skinput. Yes, the Skinput. Accompanied with technology to acutely detect ultra low frequency sounds and subseqent vibrations that are produced when you tap your skin (that’s what she said), you may be able to use yourself in ways you never dreamt possible. Or have you? You dirty monkey.

Read the PDF over here or you can view Skinput – the movie. I’m guessing this will make it to the Wii soon enough. You can read more over at NewScientist. Oh, since the projection seems to be more of a courtesy (i.e. make it easier for you to remember where to touch yourself – giggety), you could use this technology while, say running. You know, touch your input point 1 to skip track on your iPod, etc. Should be quite fun. Now go find hidden input point 11. I keed, I keeed.

No related posts.

CSS3 pseudo-class selector emulation for IE 5.5 – 8 and html5 with video

william — Mon, 01 Mar 2010 22:01:28 +0000

While gathering information as a part of my ongoing plan to be omniscient, if came across something that makes IE behave like a capable browser. Although to be fair, IE 8 is actually not that bad a browser – in the way that Star Wars Episodes 1-3 are not that bad. So, yeah, you do the math. I mean, Jar Jar. Pisses me off just thinking about it.

Mr Keith Clark has been conjuring up some serious magic and created a way to emulate CSS3 pseudo-class selectors alongside the javascript library of your choice. This means that you can use goodies such as :nth-child, :first-of-type, :first-child, :last-child, etc. It’s a simple as adding

… in your head tag. But how does it work, I hear you scream. Well, allow me to quote the man himself;

“First off, ie-css3.js checks the page for known JavaScript libraries and picks the one with the best CSS3 selector support.
Next, the style sheets on the page are downloaded and each of their selectors are parsed. Any CSS3 pseduo-classes are replaced by a standard CSS class of a similar name. For example: div:nth-child(2) will become div.iecss-nth-child-2. The detected JavaScript library is then used to find the DOM nodes matching the original CSS3 selector and the same CSS class is applied to them.
Finally, the original stylesheet is replaced with the new version and any elements targeted with CSS3 pseudo-classes will be styled.”

As of this writing, the :not selector isn’t supported but is on its way. There are some caveats, which, to be honest, are not that serious. Except that last bullet point. So don’t go all ajaxy on me now, y’hear?

You have to use a link tag (@import is OK)
page level style sheets will be ignored
Style sheets have to be on the same domain as the page itself and can not use file://
There is no dynamic emulation, which means that once the page has been rendered and the styles have been applied, changes in the DOM will not be updated.

While on the topic or, rather, something only slightly related, 52 framework promises a way to enable you to use the clean, semantic markup of html5 – today. It employs Eric Meyer’s CSS reset, which is nice in itself, and a grid system that is “loosely based on the 1kb grid and 960.gs”, and… Actually, why are you still here? Just get over there, and check out the 52 framework docs – with video as well as the demo. Don’t forget to check out the source code. can’t wait to play around with this in a project soon.

Come to think of it, I should mention a project on Google Code – HTML5 Video (or is it HTML5 Media?). This sweet litte package will enable you to embed MPEG-4 files using the H.264 video codec. Or, if you’re using Firefox 3.6, Ogv files encoded with Theora. If you use them both, you could sexy-up your code with the likings of

You just try to pretend that isn’t sexier than the horrible mess that is embed and object tags. Wrapping up, I might as well give you a tip about an awesome HTML 5 Canvas cheat sheet from the people at Nihilogic. You’re welcome.

CSS typography is fun. Really. If you’re in to S&M

william — Mon, 01 Mar 2010 13:58:02 +0000

So I found this article over at Smashing Magazine dealing with CSS typography. It’s a nice write up of both current and upcoming features – depending on your browser of choice – that will allow you to do some text formatting that I remember doing with Aldus PageMaker. Good times. Nice to see that technology is progressing at a steady pace. Competent writer Inayaili de León does a fine job going through some common and some not-so-common features such as hyphenation, indentation and hanging punctuation. I’m quite sure that you’ll find some new things in there.

She also covers the wonders of font-face, although I feel there is more to be written on that topic. Paul Irish, Esq, is a handsome fellow that makes jQuery even more fun. He has concocted a “bulletproof @font-face syntax” which works really, really well. Even in IE. Yes, in IE. Read all about it over here. Using these methods, you’ll want to use a font-face generator, such as this one @ fontsquirrel.com. It will make your online presence a bit more tolerable. But if you use Comic Sans, I will hunt you down. No, really. I’ll find a way.

Yeah, I tent do visit Smashing Magazine quite frequently even though I’m not on their payroll. Of course, they have written about typography before, and they have done so very well. One of my favorite articles is 50 Useful Design Tools For Beautiful Web Typography, covering everything from free fonts to which fonts are (usually) available on a given user’s computer. There’s just too much to mention here. Just go already! oh, and if you need some more CSS centric list o’ links, check this out instead.

If you’re into the fine art of controlling word wrapping (yeah, good luck with that), hyphenation and spaces, Alex Dunae has written an article that should make you feel all warm and fuzzy inside. What’s so special about his article, you ask? Well, my impatient, link loving visitor, he has actually made some screen shots with how different browsers interpret non-breaking space, soft hyphens and zero-width space. I know, crazy, right?

Speaking of CSS, CSS3 is shaping up quite nicely and it is a joy to play with. If you want some sort of example in the form of a sleek, sexy menu, you’re in luck! It also degrades quite nicely. As another example, you can actually create e sweet bokeh effect – using only CSS3 (and some jQuery). Also, here’s a cool effect that mimics a photo shoot, using jQuery and CSS. Should be quite nice for image cropping. And you really shouldn’t miss this article regarding multiple backgrounds and gradients with CSS! The important thing is to play and try out different techniques until you’re satisfied, look at it in IE 6, cry, ponder the meaning of life and then start over. That, my friend, is the circle of CSS.

Update: I forgot to mention a really sweet page with no less than 50 animations, using CSS3 (and some JS here and there). Definitely worth watching!

On a random note, here’s a newly hatched baby turtle. It resembles how I felt this morning.

I’m the profeshunl newbie

william — Sun, 28 Feb 2010 16:03:10 +0000

Hi, anonymous visitor!

I have no idea how you found this place, but I’m kinda glad you did. On this site I intend to post things, stuff and arbitrary crap I find amusing or related to my work as a developer and designer. I do a lot of online things for a living and have done so for what feels like a long, long time. The sheer amount of time spent on developing and creating, I assume, would label me as a professional, alongside the copious amount of satisfied clients. Copious, I say! Funny thing is, I never feel as if I fully know what I’m doing even though I get things done. In many ways, I still consider me a newbie. Hence the name profeshunl newbie. And please, don’t post a snarky comment telling me how I misspelled “professional”. Or “newbie”. Yes, that would be the joke.

Anyhoo, if you like graphic design, developing with PHP, MySQL, jQuery, Flash and other goodies on a neat LAMP stack, I’d like to say that you’ve come to the right place.

Moving on, I hope you find something to read here and tell your friends about it. If not, that’s cool. I mean, I’ll put you on my list, block your IP and haunt you in your sleep, but other than that – we’re cool.

Oh, and the awesome pictures of Zack Galifianakis and Bradley Cooper on the index page are © Frank Masi. Relevance of the photos? None. If you haven’t seen the movie The Hangover yet, you’re no friend of mine.

No related posts.

Favicon:	get_favicon(); ?> (get_favicon(); ?>">)
Permalink:	get_permalink(); ?>
Title:	get_title(); ?>
Date:	get_date(); ?>
Content:	get_description(); ?>
Description:	get_description(); ?>
Parent Permalink:	get_permalink(); ?>
Parent Title:	get_title(); ?>