bjori doesn't blog

The Frills

noreply@blogger.com (bjori) — Thu, 23 Apr 2015 17:12:00 +0000

I don't like ORMs. I really don't. It's not just because all of the ORM frameworks I've seen make mockery out of performance needs, but they also tend to be extremely over-engineered, which makes me want to cry. I'd wager that most people using ORMs are actually solving the wrong problem with the wrong solution, simply because there wasn't a better alternative for their data-modeling needs and the benefits of adopting an ORM outweighed its inherent problems.

It's like driving a gasoline-hungry car. You say that you care about its MPG rating, but you are still wasting a lot of gas without ever looking back. It is just a cost of living. Or insurance: You know you can save 15% or more... but you don't. You've accepted that bi-annual rip-off as cost of living.

Just like using an ORM, you have accepted the pain as cost of living. Well, I don't :)

ODS - Object<->Document Serializer

The new MongoDB PHP driver includes an experimental ODS interface (BSON\Persistable) that allows you to automatically store your object as documents in MongoDB the way you see fit. And the best part? When you retrieve the document from MongoDB, the driver will know which class it represented and can reconstruct the object (again, the way you see fit).

All you have to do is implement the two methods of the BSON\Persistable interface:

bsonSerialize() which returns an array to store as a BSON document
bsonUnserialize($array) which receives a BSON document just like the one returned by the serialize method

That's it! There is no special way of querying by type, parent object, or any of those weird things that ORMs have introduced to work around storing objects or documents in databases. Your document is your object. Your relations are your relations.

If you would like to build on top of this functionality, it's trivial to implement a trait that implements both methods and provides basic change tracking. At that point, we are entering ORM and ODM territory and are going beyond what we should ask from an extension. All we want from an extension is performance, and simplicity.

Next Generation MongoDB Driver for PHP!

noreply@blogger.com (bjori) — Tue, 14 Apr 2015 13:00:00 +0000

For the past few months I've been working on a "next-gen" MongoDB driver for PHP -- codename "phongo".
The aim was to build a new PHP extension ontop of the mongoc and libbson libraries to reduce maintenance of the extension itself and focus more on providing the ecosystem with improved support and libraries.

The new driver is available on PECL (called "mongodb", surprisingly enough). It doesn't include any of the bells and whistles found in the previous "mongo" driver. It doesn't include any `group` or `count` command helpers, and you won't find any Collection or Database objects; however, it really doesn't need any of these things.

All it has is simplicity. You can execute a command of your choosing. You can execute a query. You can execute a write. That's really it (as with everything, there are certain exceptions). Oh, it is also very, very fast.

What's the Point?

Most developers interact with databases these days through their preferred framework abstraction layer. That layer often has its own way of providing the bells and whistles irrespective of the driver's functionality -- it seldom matters what helper methods the driver offers, since these libraries can call MongoDB::command() directly. Besides, upgrading the driver just to get a new command helper, which is easily implemented in userland PHP, is just weird.

For our new "mongodb" extension, the driver only implements the most essential features for interacting with a MongoDB server:

Server Discovery and Monitoring
Automatic Server Selection (with manual selection possible) using Read Preferences
BSON [en|de]coding (to/from extended JSON, too)
Implementing the MongoDB wire protocol

Again, What's the Point?

Well, now we can provide you with a stable driver that really shouldn't change all that much, which means less extension upgrades.

Furthermore, it should be faster. The legacy driver, which dates back five years, had unfortunate design quirks that couldn't be fully resolved without a costly rewrite (e.g. the way MongoGridFS invokes MongoCollection methods internally). Creating a brand new, no-frills^[1], and simple to use driver gives us a fresh starting point for the next five years.

MongoDB PHP Libraries

Of course, we aren't planning on leaving our users out to dry. Whether or not your framework of choice offers an amazing MongoDB abstraction layer, we do want to make it easy, simple, and natural for you to develop applications with MongoDB.

To that end, we are writing a PHP library on top of this new extension, which will have all of the frills, bells, and whistles you might expect. It implements the "Standard MongoDB Driver CRUD API" (among others) and we'll continue roll in new features into this library as needed. And because this library will be implemented in PHP, we expect to iterate on new features much more quickly than we were able to do with the legacy driver.

This won't be the only library we will be writing. There are also plans to develop a library to deal with MongoDB administrative tasks (e.g. creating users, reconfiguring wiredTiger nodes, tailing oplogs) and develop tools to introspect MongoDB clusters.

What do you think?

The biggest question at this point is: "what else?" What is missing? What are the other pain points you've experienced as a MongoDB developer, that the driver or a library can help with?
We would love to hear your feedback! Check out the projects on GitHub (driver, library), and let us know what you think.

^[1] Well.. Almost. There is one frilled feature I'm excited about -- we'll cover that later.

Going tableless

noreply@blogger.com (bjori) — Mon, 13 Apr 2015 18:02:00 +0000

Tableless html was all the rage a decade ago -- for a good reason.
One of the advantages was maintainability of the markup -- especially in a deeply nested table layout.

It also turned out, the table layout didn't actually fit the way we wanted to present the data. The rigid structure it enforced wasn't what we were after - and it also limited the freedom designers had to push forward and create next generation website and application layouts.

In the beginning though, it really was the only way to get stuff done -- and the best tool for the job. It was fun while it lasted -- but we are over it now.

Today -- we are entering a similar situation -- in databases, of all places.
Tableless databases are becoming all the rage today -- for a good reason.
One of the advantages of going tableless db is maintainability of the data -- especially with deeply nested data.

It turns out, the table schema doesn't fit the way we want to work with or present data. The rigid structure it enforces isn't what we are after either - and it also limits the freedom application designers have to push forward and create next generation websites and applications.

In the beginning though, it really was the only way to get stuff done -- and the best tool for the job. It was fun while it lasted -- but we are over it now.

I have a dream

noreply@blogger.com (bjori) — Fri, 23 May 2014 18:05:00 +0000

I am happy to join with you today in what will go down in history as the greatest demonstration for freedom in the history of our ~~nation~~ project.

Today we will revolutionize PHP.
- We will make it easier to upgrade the things you care about[1].
- We will make it easier to not upgrade things you don't want to upgrade[2].
- We will make it easier to distribute your extensions[3].
- We will make it easier to release according to your own schedule[4].
- We will make it easier to add functionality[5].
- We will make it easier to work.

Ok, today is a white lie here maybe... I haven't actually implemented this, but bear with me here for a second.

- Wouldn't it be great if PHP would just be language/syntax things?
- Wouldn't it be great if you could upgrade an extension you use independently from anything else?
- Wouldn't it be great if you could get your hands on the new intl/mysqlnd/MongoDB/curl/whatever features without needing to upgrade PHP and everything else?
- Wouldn't it be great if you could get your hands on new PHP syntax without needing to upgrade intl/mysqlnd/MongoDB/curl/whatever?

What the hell are you talking about?

The PEAR package manager is dying. It has been dying for a while.
Thats not a bad thing, or even sad, it is actually a very good thing.
It has served its purpose and its time to die and let the next thing take over, like dependency managers.

This however leaves PHP extensions in an unfortunate situation. They rely on the PEAR packaging manager, and there currently isn't any alternative.

Recently, during the process of making Windows binaries for PECL extensions available again (e.g. pecl4win, which died in 2008) the issue of licensing came up - some licenses (including the PHP license which many/most PECL extension are licensed under) requires certain paragraphs to be reproduced (generally simply including the LICENSE file).. making distributing binaries a problem.
Detecting what is a license file became a problem since the packaging manager cannot mark a file(s) as the license.. Adding that functionality is trivial, but there is a problem.
There hasn't been a release of the packaging manager since 2011, and a similar change (adding unix manual pages detection) took 2 years to be merged (and still isn't released!) - but there most likely won't be any release in the near future anyway... And we are stuck.

Now we have the chance to make it right!

I would like to create the new "pecl install" tool, a new extension manager -- a PHP manager.
A way for you to decide what you want to upgrade - do you want bugfixes or new features at the language level, or is there just one specific bugfix or a feature you need from a specific functionality (i.e. extension)?

This tool would effectively only install PHP - the language - not any "default extensions". Then its up to you to decide what you want or need for your project.
You'll get the language version you decide, and you can upgrade the extension set independently of what language version you want to use.

Currently, if you download PHP from source and compile it with its "default extension set" you'll get all sorts of extensions that may or may not be useful to you. If you then install PHP on Windows you get a different set of enabled by default extensions. God bless if you install PHP on a Linux distribution from their packaging repositories, you'll get whatever they decide is best for you.

What about my hosting situation?

Whatever extensions are available won't change (random set of extensions on shared hosting, some management panel to enable/disable on dedicated hosting), but what version (features/bugfixes) these extensions are will most likely be improved in the dedicated hosting scenario as you'd be able to pick "new features" vs "old and stable, bugfixes".

In the case you actually have control, you'll be able to "pecl/apt-get/yum install/upgrade" based on your preferences - latest features, or bugfixes only.

When will this be available?

There seems to be a great momentum on internals@ these days for making a lot of internal changes which will require a new major version of PHP. This is a perfect time to introduce a new PHP extension manager into the mix. Let PEAR and 5.x stay as-is, while the next major release of PHP that will dominate the next decade or so get an improved tool to deal with this.
The focus needs to be for the future, and once we have something solid.. We could think about adding support for the current extension system, although that should be a secondary goal - we need something for tomorrow, not yesterday.

-Hannes

[1] New functionality or security update
[2] Backwards compatibility breaking releases
[3] No centralized management
[4] No need to wait for PHP releases, or work overtime to make it in time for release
[5] No PHP version restrictions

Query logging and profiling in MongoDB

noreply@blogger.com (bjori) — Wed, 18 Sep 2013 11:30:00 +0000

In my previous blog post I mentioned that the 1.5.0 release of the MongoDB driver for PHP has improved notification support using the PHP Notification API, and showed off a fairly useless (but fun) progress bar.
Now its time for a little bit more practical examples using this system: Query logging and profiling.

Those of you who clicked through to the github Pull Request may have noticed the long long description of the API and noticed all the different "MONGO_STREAM_NOTIFY_LOG_xxx" constants it included..

When instantiating MongoClient you can pass in "Driver Options" using the 3rd argument. Currently there is only one available Driver Option: "context".

The context key takes a Stream Context, and is assigned to all of the underlaying connections the driver creates - which means you can set any of the Socket Context and SSL Context options and it will Just Work (maaan.. that saved me a ton of work.. Not needing to implement any of the SSL things myself!).
Since its just a normal Stream Context, the lesser known Context Parameters will also Just Work, but what does that mean?

Well, PHP has a trivial notifications system when operating on streams. The out-of-the-box support is (incremental) progress updates when reading and writing to streams, which I covered in my previous blog post. The MongoDB driver however extends this support with a lot of other things.

Keep in mind I haven't documented any of this just yet as its all still experimental.

OK. Lets check this out.

(The entire script is available as a Gist on Github).

<?php
class MyStreamNotifications {
    public $reqs = array();


    /* Our callback function
     * The prototype is define by the PHP Stream Notification system:
     * http://php.net/stream_notification_callback
     */
    function update($notification_code, $severity, $message, $message_code, $bytes_transferred, $bytes_max) {
        switch($notification_code) {
            /* For now, lets focus on the TYPE_LOG, and ignore the others */
            case MONGO_STREAM_NOTIFY_TYPE_LOG:
                /* This notification only uses two arguments, so we can ignore the rest */
                return $this->logMessage($message_code, $message);

            default:
        }
    }

    /* Our logging function
     * The $code describes the operation being executed,
     * and $message is a json serialized array, packed with information
     * Note: If ext/json is not available, $message with be PHP serialize()d instead
     */
    function logMessage($code, $message) {
        $data = json_decode($message, true);

        switch($code) {
            /* Write operations are a one way street in MongoDB, so these don't end with a
             * MONGO_STREAM_NOTIFY_LOG_RESPONSE_HEADER.
             * When using WriteConcern (defaults to 1), and additional query (command) is
             * sent to check for errors, called getLastError*/
            case MONGO_STREAM_NOTIFY_LOG_INSERT:
            case MONGO_STREAM_NOTIFY_LOG_UPDATE:
            case MONGO_STREAM_NOTIFY_LOG_DELETE:
            case MONGO_STREAM_NOTIFY_LOG_GETMORE:
            case MONGO_STREAM_NOTIFY_LOG_KILLCURSOR:
            case MONGO_STREAM_NOTIFY_LOG_BATCHINSERT:
                /* We only care about the query and response headers, for now */
                break;

            /* We´ve got a query */
            case MONGO_STREAM_NOTIFY_LOG_QUERY:
                /* Retrieve the Wire protocol Request ID, so we can match it with the
                 * Response headers once the query as executed */
                $reqid = $data["server"]["request_id"];

                /* Log all the metadata we have about the query, and add the start time */
                $this->reqs[$reqid] = $data;
                $this->reqs[$reqid]["start"] = microtime(true);
            break;

            /* The query has executed on the server, and now we are reading the
             * response headers */
            case MONGO_STREAM_NOTIFY_LOG_RESPONSE_HEADER:
                // Get the Request ID this respose is for
                $id = $data["server"]["request_id"];

                /* The driver runs some commands internally, such as isMaster() to
                 * sanitycheck the connection to the server, these commands currently
                 * don't trigger the notification system - that may/will change in the
                 * future though */
                if (!isset($this->reqs[$id])) {
                    return;
                }

                // Fetch the request info
                $req = $this->reqs[$id];
                // End the timer
                $end = microtime(true);

                // Add the ending time, and how long it took
                $req["ended"] = $end;
                $req["total"] = $end - $req["start"];

                $this->reqs[$id] = $req;
            break;

            default:
                // Unknown log message code, ignore
        }
    }

    function getRequests() {
        return $this->reqs;
    }
}

Thats a good chunk of code, just to profile queries.. But bare with me for a moment.

There is a lot of things happening right there, but hopefully the inline comments make sense of it all.
The update() method is the actual callback function (which we will be registering a little later).
It has a lot of arguments, and the $bytes_transferred and $bytes_max aren't even relevant in most scenarios, but ohhwell. Nothing we can do about that as this is the interface PHP provides us with.

The $notification_code is the "main argument" in the callback function, it tells us what sort of notification it is, and there are a handful of different notifications that can happen.
Each notification then has its own set of "$message_code", in this case, one of the MONGO_STREAM_NOTIFY_LOG_xxx constants.
Since we are only interested in logging and profiling the queries we ignore all the different MONGO_STREAM_NOTIFY_LOG_xxx (they are only there for completeness - in case you want to extend the example and experiment with them later), and only catch the MONGO_STREAM_NOTIFY_LOG_QUERY and MONGO_STREAM_NOTIFY_LOG_RESPONSE_HEADER.

Moving on..

/* Some Helper Functions */

/* Split a Server Hash to extract the server name from it */
function getServerName($hash) {
    list($server, $replicaset, $dbname, $pid) = explode(";", $hash);

    return $server;
}

/* Resolve the server type to a readable string */
function getServerType($type) {
    switch($type) {
        // FIXME: Do we not export these as constants?
        case 0x01: return "STANDALONE";
        case 0x02: return "PRIMARY";
        case 0x04: return "SECONDARY";
        case 0x10: return "MONGOS";
    }
}

/* Resolve Cursor (Query) Options to something meaningful */
function getCursorOptions($opts) {
    // FIXME: Do we not export these as constants?
    if (($opts & 0x04) == 0x04) {
        return "slaveOkay";
    }
    return "Primary Only";
}

Here we define some helper functions, and actually few FIXMEs for myself so you don't have to remember these magic numbers :)
Other then that, nothing really important.
Next up, boilerplate setting to connect to a ReplicaSet.

/* Boilerplate stuff to setup a ReplicaSet connection */
// My replicaset seed list
$seeds = "toddler.local:30200,toddler.local:30201";

// MongoDB Connection options
$moptions = array(
    // Don't wait to long on servers that are down
    "connectTimeoutMS" => 300,
    // Read from secondaries, if available
    "readPreference" => MongoClient::RP_SECONDARY_PREFERRED,
    // My local ReplicaSet name
    "replicaSet" => "REPLICASET",
    // Don't wait more then a second to confirm Replication
    "wTimeoutMS" => 1000,
);

Again, that wasn't fun... Lets take a look at some fun things next!

/* Registering a stream context! This is the fun part! */

// Create an instance of our callback object
$mn = new MyStreamNotifications;

// No Stream Context Options, for now
$coptions = array();
// Configure the Notification Stream Context Parameter to call MyStreamNotifications->update()
$cparams  = array("notification" => array($mn, "update"));

// Create the Stream Context
$ctx = stream_context_create($coptions, $cparams);

$doptions = array("context" => $ctx);


// Instanciate a MongoClient with all our options
$mc = new MongoClient($seeds, $moptions, $doptions);

If you have never played around with PHP Stream Contexts, this may look a bit alien..
The stream_context_create() function takes two arguments, context options and parameters.
There is a very subtle difference between those two, and we do support both (yes yes, not documented.. :) still highly experimental), but for now, we are using the notification parameter.

Now that we have setup everything we need and have connected to MongoDB, configured just the way we like it, its time to run few commands and queries to check this thing out

// Query the example collection in the test database
// you should probably change this to something meaningful
$collection = $mc->selectCollection("test", "example");


// Random search criteria, you should change this to something meaningful
$criteria = array("search" => "criteria");
// Add some sorting & limit for the fun of it
$cursor = $collection->find($criteria)->skip(13)->limit(5)->sort(array("_id" => 1));

// Count on a secondary (we constructed the MongoClient with SECONDARY_PREFERRED)
$count = $cursor->count();
echo "We have $count results in total\n";

// Fetch the data from a primary
$cursor->setReadPreference(MongoClient::RP_PRIMARY);

/* The query isn't execute until we start iterating over it.. */
foreach($cursor as $doc) {
}

When you are trying this out for yourself, make sure you have establish a connection to your MongoDB instance (modifying the $seeds and $moptions["replicaSet"] variables up there), and modify the database+collection name, along with the query criteria.

Now that we have executed some things, our $mn->reqs array has now some interesting info its time to print out

// Fetch our requests from our logger
$reqs = $mn->getRequests();
foreach($reqs as $request) {
    printf("Hit %s (which is a %s, I asked for %s) with:\n%s\n",
        getServerName($request["server"]["hash"]),
        getServerType($request["server"]["type"]),
        getCursorOptions($request["info"]["options"]),
        json_encode($request["query"], JSON_PRETTY_PRINT)
    );
    printf("Skipping the first %d matches, limiting the results to %d documents..\n",
        $request["info"]["skip"],
        $request["info"]["limit"]
    );
    printf("Server execution time+network latency: %.8f sec\n\n", $request["total"]);
}

Putting it all together, and running the script should then print out something like:

We have 205 results in total
Hit localhost:30201 (which is a SECONDARY, I asked for slaveOkay) with:
{
    "count": "example",
    "query": {
        "search": "criteria"
    }
}
Skipping the first 0 matches, limiting the results to -1 documents..
Server execution time+network latency: 0.00042319 sec

Hit localhost:30200 (which is a PRIMARY, I asked for Primary Only) with:
{
    "$query": {
        "search": "criteria"
    },
    "$orderby": {
        "_id": 1
    }
}
Skipping the first 13 matches, limiting the results to 5 documents..
Server execution time+network latency: 0.00045013 sec

You may want to var_dump() the $request variable to see what other fun things it contains to play with, hopefully I'll have it all properly documented by the time we release 1.5.0 version of the driver.
As you can see, the driver needs to massage the query a little bit to add things like orderby clauses, where as "skip" and "limit" are part of the wire protocol and therefore not part of the actual query syntax.
If you have the MongoDB profiler enabled, you could then lookup how long the query execution took on the server, subtract the time we got and you have the time it took us to read and write over the network.

Another related feature is the $comment operator that "tags" a query, so its easier to spot in the profiler. You could use this to for example group your queries ("person model", "retrieving comments", "product query", ...).

Although this is a good chunk of code, I hope it will be useful, especially for applications that require thorough auditing of its queries. This should also be easily integrable into most "Profiler/Developer toolbar" that the various frameworks provide.

Since this is all still highly experimental, there is still time to change some things, so I wonder.. what do you think?

-Hannes

PHP Stream Notifications ❤ MongoDB

noreply@blogger.com (bjori) — Tue, 17 Sep 2013 10:00:00 +0000

PHP Streams have several pretty nifty features that most people don't really know about; Filters, Wrappers, Context, and Notifications. Documenting these is a bit difficult, and getting the user to discover these features is even more problematic, as these things usually live outside of the normal path (function reference).
Maybe I'll blog about these things in the future, but for now I want to talk about the Stream (context) Notifications - or more specifically; Stream Notifications in the MongoDB extension for PHP.

The Stream Notifications are essentially pretty simple: when a stream does something, it notifies something (a callback function) that it did something (notification codes, importance, message code, ..).
Since this feature is a little neglected and people don't seem to know about it, this "something happened" isn't really a lot of things. When PHP reads from the http stream it will tell you if it came across a "redirect header", "mime-type", and, when available, the content size. Simple things.
This is actually pretty powerful and allows you to do some work, even when file_get_contents() a large large file, so it doesn't have to be a blocking operation anymore.
An example of a simple, yet pretty cool, script would be a wget/curl style download application with a progress bar (we actually use that example to download pear things when you install PHP and don't have wget or fetch).

Enter MongoDB for PHP.
As of pecl/mongo 1.4.0 we have switched out our homegrown sockets library in favor of using the native PHP streams. This gives us several advantages, such as greater portability, automatic support for SSL, and memory reporting is now more inline with reality, as everything is mapped using the PHP allocation functions.

Now, since we are using the PHP streams.. a pretty cool thing happens: We can now play around with the notification API and Stream Contexts.
A highly experimental (undocumented) API was included in the 1.4.0 release, and now as we are prepping for 1.5.0, I've spent more time on it, adding things that are "useless, until you play with it".. Such as progress reporting for read/write.. Wait what?

Yeah. OMG LOL For realz.
I created a little example, called spindle.php, which queries the enron dataset, and prints out a progress bar for all socket read and writes, along with meta information like the actual command and query being sent - and even some information about the actual wireprotocol operation, in case you are interested. Writing the queries and reading the data is too fast on my laptop so I had to add usleep() into the code to see the progressbar spinning.. but its still fun.

I can't really think of a real practical use case for reporting socket read/write progress updates.. But its there. Hopefully someone can come up with some cool things using it :)

Note: The Pull Request hasn't been merged yet, and the API is still considered as experimental, so you'll have to use my improved-notify-support branch for now if you want to play with it.

Progress reporting obviously isn't the only thing the new Notification API does, I'll blog about the more practical notifications later :)

-Hannes

New PHP.net designs floating around

noreply@blogger.com (bjori) — Fri, 06 Sep 2013 11:00:00 +0000

Since 2008 there have been numerous efforts to create a new design for www.php.net, all of which have failed - so far.
We've never come as close as two years ago, when the "beta mode" option was added to our website, but we never really got around to finish it.
The "beta design" has even received a lot of makeover compared to what is "beta mode" now.
To make things a little bit more awesome, there is also a new branch called "responsive" which has a lot of changes in it too, especially for manual pages.

Hopefully, one day, we'll actually finish one of these and flip the switch forever.

In the meanwhile, I'm working on retiring my little baby, PhD, as I want to encourage more people to get involved with the overall project, especially the neglected parts like documentations.
It doesn't help anyone to be greeted with a "ohh, btw, you have to have a doctorate to be able to work on the docs" like "almost" is the case with our usage of Docbook - not to mention the fact DocBook isn't exactly the easiest markup format to learn.

I've been a huge advocate of highly semantic markup, and never dreamed that I would want to get rid of Docbook for our documentation.. But now I finally understand.
It must be *simple* to contribute.

Although I have optimised our tooling from taking over a week down to roughly 3minutes, it still isn't good enough. It needs to instant. And we can't reach that point with out current markup. We need to change. Make it simpler. Attract new people. Make it fun to contribute and help out.

To that end I've started working on a tool I call "WTFM", which is a Markdownish renderer..
The goal is to keep some of our smart trickery and simplifications we do in Docbook, just without that massive amount of markup and confusion. There shouldn't be any need to "learn a new format". Just write the docs, click save and you see it rendered instantly. No weirdo wiki markup noone understands, or hundreds of rules to obey.

I haven't exactly finalised the tool, or the actual "non-format format" yet, but the current (pre-alpha has-a-lot-of-bugs) proof-of-concept is running - and to stay true to our "lets make a new design" goals.. It has completely different markup rendering, so I had to create a new CSS for :)

What is your favorite layout so far? You think you can help us finish any of these?
Shamefully enough, getting the website running locally is a huge pain in the ***, but these instructions should take away most of the pain...

Hope to hear from you!

-Hannes

A lot of things happen in two years..

noreply@blogger.com (bjori) — Fri, 06 Sep 2013 04:12:00 +0000

Turns out; The title of the blog is completely true.. I really really don't blog :)
Since my last blog entry, just over 2 years ago, I..

Left my job at Redpill Linpro
Moved from Oslo to London
Joined 10gen (now MongoDB)
Moved to San Francisco
Got married
Moved to Menlo Park

..and now I'm expecting a female ninja early next year..

That is a good chunk of life changing events crammed into only ~2 years :)
Thats it for now though.. Mainly just wanted to see if this thing still worked.

-Hannes

There is an app^Wppa for that

noreply@blogger.com (bjori) — Thu, 28 Apr 2011 11:23:00 +0000

Want to try out PHP5.next (PHP5.4?) on Ubuntu?
After the death of PHP6.0 a while ago, development of PHP.next (PHP5.4 probably?) has been going on.
There are a lot of cool features there try out - like traits, scalar type hint, and sh**loads of smaller features.

There is still no public "development preview" or alpha release, but that doesn't mean we can't play around with it, report bugs, ensuring our apps still properly work with it etc etc etc.
It is however a bit annoying needing to "go old-school" and fetch a snapshot and build it yourself though.

When I was playing with launchpad the other day I figured.. why not provide a daily build of PHP trunk/ (PHP.next, PHP5.4 or whatever you want to call it)?

Launchpad isn't all to happy with git, and doesn't support mirroring git branches, so a fork of the php5-vanilla-ubuntu repo to php5-next-vanilla-ubuntu was needed as some of the patches there don't apply to trunk/. But that was about it.

So, want to try out PHP5.4 daily builds? Checkout the PHP5.4-daily PPA on launchpad!

Play around with the new features, provide feedback to the PHP development mailinglist, and make sure your application if forward compatible with it today.

up2date PHP5.3 packages for Ubuntu

noreply@blogger.com (bjori) — Sun, 17 Apr 2011 11:22:00 +0000

Most Linux distributions have a policy of not being to up to date with upstream releases of software, mostly for good reasons. This however is extremely painful for developers, as it means they often need to use really outdated version, containing all sorts of bugs - and even missing features.

This has been pissing me off for quite some time, so I have been running my own PHP builds for a while - but when it comes to deploying the apps... the sysadmins obviously start complaining that they have to invest a lot more work into maintaining the servers then they otherwise would have to.
So.. What can we do about that? Launchpad to the rescue!

Launchpad makes it really easy to provide your own custom packages, and even has a vast build farm to build packages automatically for different architectures and different Ubuntu releases. The only down side is it doesn't build rpm packages.. Thats fine by me, but that would be really useful for those wishing to deploy on a RedHat based distro.

After hunting down the debian PHP packaging repo, I forked it onto github (as php5-vanilla-ubuntu) and started ripping out some of their weird patches and enabled mysqlnd, but otherwise keeping their package splitting and the things you would expect from a debian package.

What does this mean?
- Well, if you are looking for up2date PHP packages (currently PHP5.3.6) to work with, checkout the PHP5.3 PPA on launchpad :)
This PPA works just fine as a drop-in-replacement for the default Ubuntu packages, and provides builds for Lucid and Maverick (there are some changes in Natty I need to look into..).

I have been running these packages for some time now, but if you notice any issues - please let me know :)

The PHP project and Code Review

noreply@blogger.com (bjori) — Fri, 24 Dec 2010 09:35:00 +0000

Reading code is not only fun, its also a great way to exercise your brain - not to mention a fantastic way to discover new ways to solve problems. At work (we are hiring btw!), for example, I read pretty much every single commit (and merge requests, for that matter) - and I'm subscribed to several different OSS commit lists. I can't say I read every commit to PHP, I focus on the areas I care about, but I do skim over the rest - if only just to see when new features are added.

The PHP project has a good chunk of mailinglists, everything from support lists, developer discussions, QA, and so on. Every commit to our SVN is automatically posted to the relevant commit mailinglist(s).
The main commit lists are;
- PHP (php-cvs@lists.php.net, http://news.php.net/php.cvs)
- PHP Documentation (doc-cvs@lists.php.net, http://news.php.net/php.doc.cvs)
- PHP-GTK (phpgtk-cvs@lists.php.net, http://news.php.net/php.gtk.cvs)
- PEAR (pear-cvs@lists.php.net, http://news.php.net/php.pear.cvs)
- PECL (pecl-cvs@lists.php.net, http://news.php.net/php.pecl.cvs)

Then we have specific lists for each translation of the docs, all the websites, PEAR docs and so on.
The great thing about these commit lists is that *anyone* can subscribe to them, and quite a lot of people actually are.
For example, there are over 200 people registered to the PHP commit list, and over 100 people on the documentation commit list alone. That is ca 25% of the people that are subscribed to their discussion counterparts.
Granted that most of the subscribers do not actively review the commits, I guesstimate there is still a sizable percentage of them that do.

Just the simple fact that I know people will be reading through my commits makes me think about what I am doing a bit more; "Is this really needed?", "Is there be a better way solving this?", "Could it potentially break other things?", "Is this actually correct?"..
The people who review the commits often don't seem like the friendliest people in the world.. If there are issues with the commit; You will be told. No doubt about it. And you will feel like an idiot, for few minutes, for not having caught it yourself - and promise yourself to review your commits better in the future. Learning from your mistakes. I love it. Mission accomplished.

Last year I updated the credit list, printed out by phpinfo() & phpcredits(), for the people maintaining our websites, with the commit message "Throw some credit around" (yes, the typo and incorrect colspan in the commit was caught and fixed).

Fast forward 15months, to last Friday. Another commit from me to PHP trunk with the commit message "Throw some credit around", and then to PHP 5.3 a minute later with the same commit message.
Less then 10minutes after the commit I had received 2 emails asking me "WTF?" - and a poke on IRC.
I had no idea what those guys were talking about. None what soever. I had not committed anything to any PHP project for several days. Nothing. Not even a typo fix.

Turns out that someone in Asia had somehow managed to get his/hers hands on my PHP.net account credentials.
Interestingly enough the commit doesn't introduce any security holes, bugs, or anything at all really. It just adds "Wolegequ Gelivable" to the credit list (whatever that means).

Its not a great feeling to have your account hacked into, but I do wonder what the intentions were.. Maybe just an credentials check, which was supposed to be followed by evil commits if noone had spotted the first one? The Chinese government trying to introduce security holes so they can break into PHP websites?

In any case. It took less then 10minutes for 3 people to catch it, that is pretty cool.

-Hannes

p.s. Last year, one of my new year's resolutions was to "blog once a month". My last blogpost was in January.

Unix manual pages for PHP functions

noreply@blogger.com (bjori) — Tue, 05 Jan 2010 08:45:00 +0000

Did you know that unix manual pages for PHP functions and methods existed?

For a while I had vim configured to run reflection when I hit "K", but after the PHP documentation team released unix manual pages for PHP I now get the manual page in all its glory; function description, parameter descriptions, return values, examples, notes, see also and everything you are used to see from the online manual.
Its awesome.

As many PHP functions use "standard names" the documentation team decided to not install the manual in the standard man directory.
When you install the unix manual package you get a little shell script (called "pman") which runs the normal "man" command for you, and sets the manual directory to wherever you installed the package.

To install the PHP manual as unix manual pages:



$ pear install doc.php.net/pman

If you are running very old pear version you need to "discover" the doc.php.net channel first:



$ pear channel-discover doc.php.net

And then run the command above.

The manual will install into whichever directory you have specified as the "doc_dir" in the pear configuration, and the "pman" script into "bin_dir".
By default I believe pear sets the "bin_dir" to something that is already in your $PATH so you should be able to execute "pman strpos" now.

In VIM you need to update the "keywordprg" config (or add it, if it doesn't exists already) to "pman":



set keywordprg=pman

Now open up a .php file, place your cursor over some PHP function and hit "K", you should get the manual page for that function \o/.

Unfortunately Symfony, Zend Framework, Doctrine, and other frameworks/libraries/.. do not distribute unix manual pages yet so I use the Taglist plugin to jump to the code.

Looking forward to when these frameworks start producing unix manual pages, so much easier to read :D

-Hannes

Most PHP releases in August!

noreply@blogger.com (bjori) — Mon, 10 Aug 2009 18:15:00 +0000

Since PHP 3.0.18 (released October 2000) (no, php.net doesn't keep release records before that for some reason) 60 PHP releases have been made. August and May are the most common months for PHP releases, with 9 releases each.

PHP5.3.1 and PHP5.2.11 are in the works these days.. Judging by the August release "frenzy" I bet both will be out by the end of the month!

Other useless stats?

26 PHP releases have been on Thursdays
1 On Saturday
There have only been 2 releases in 2009 so far
2004 had 10 releases
January, April and October have only had 3 releases (each)
There has never been a release on Sunday
The 89th, 121st, 122nd, 194th, 228th, 241st and the 349th day of the year have had 2 releases each
The 3rd day of the month is most likely to be a release day, with total of 6 releases
The 9th, 16th, 18th and 21st (of any month) have never been release days...
The 18th week of the year is most likely to have an release, whole 8 releases have been made in that week!

Want more?


php -r '
function fetch($arg) {
  static $mirror = "http://no2.php.net/releases/index.php";
  return unserialize(file_get_contents("$mirror?$arg"));
}
foreach(fetch("serialize=1") as $major => $unused) {
  foreach(fetch("serialize=1&max=-1&version=$major") as $version => $data) {
      echo $version, " - ", $data["date"], "\n";
             $foo[date("l", strtotime($data["date"]))]++;
  }
}
arsort($foo); print_r($foo);
'

Google breaking up with me?

noreply@blogger.com (bjori) — Fri, 01 Aug 2008 23:17:00 +0000

Dear Google.

There is a reason why my browser does not send out request for Norwegian content. That reason is the same reason why I change all my language profiles to English. Even though I live in Norway please respect my wishes and stop serving me Norwegian content.

Earlier today I got this usually weird message from Google Reader saying "There is an update available for Google Reader click _here_ to refresh" (or something along those lines). I have no clue what that means, but I get that message regularly. I always click "here" and never notice any changes.
I suspect this is Google Gears (yes, it will always be called "Google Gears") that is updating the content, but what is the "Go offline" button then for? Clicking that button fetches my unread items and "disconnects" so I can read my feeds on the train.

Related note: I am pretty sure Google Gears swallows some of my still-unread-items when going online again. Very annoying as I explicitly check the "Keep unread" box so I can read the posts later on in a more friendly environment.

Back to the point. After clicking "here" earlier today Google Reader decided to change my profile settings to say "Norwegian" and displays everything in Norwegian, removing the offline feature and "Friends shared items".

After changing my profile settings again to English only the settings dialog is in English. The rest of Google Reader is still in Norwegian with missing features I use regularly.
To make things worse, clicking "Logg ut" does not work. Simply throws "Beklager, det oppstod en feil. Prøv igjen om noen sekunder." back at me. No matter how long I wait or how often I try.

Going into my profile settings again, and change the language to Norwegian gives me back the offline feature and "Friends shared items" - and some random things in English. Apparently I can log off now, which is a appreciated.

While writing this entry on blogger.com "Autosave failed" has been blinking for most of the time, and now a big fat red line showing "ERROR" below the "Title" textbox is showing.

Dear Google, do you not like me anymore? I thought we were friends and shared something special. If you are breaking up with me I'd like to hear it straight up. Not randomly throwing errors at me for several months. I do understand your feelings though. After growing up together for so long we have finally reached the the tipping point, and started to grow apart the past months. You not caring anymore about the products I used to love, and me moving on to greener pastures.

We had a good run you and I. But I guess all good things come to an end at some point.

Love,
Hannes

8 reasons why you should not write for the php.net manual

noreply@blogger.com (bjori) — Sun, 14 Oct 2007 15:40:00 +0000

8 reasons why you should *not* write for the php.net manual:

You get more page hits by publishing your work on your blog
You get a free pass to conferences by converting your work into presentation
You get paid to submit your work to magazines & commercial websites
You get a "trading card" and considered a star
The community will know your name
You can flame the documentations without needing to do anything about it
You can copy&paste parts from the manual and sell it (see point#3)
You can license your work to forbid any commercial use or further improvements

PhD: The [PH]P based [D]ocbook renderer RC1 released

noreply@blogger.com (bjori) — Tue, 02 Oct 2007 09:46:00 +0000

Quick note; We released PhD0.1RC1 today \o/

Building the php.net documentations has never been as easy or as fast.

Note: You'll need 200M free diskspace (90M for the phpdoc XML sources and 110M for the generated html/php files).

Fetching the phpdoc sources:
bjori@lindsay:~$ cvs -d:pserver:cvsread@cvs.php.net/repository co phpdoc cvs checkout: Updating phpdoc U phpdoc/.cvsignore U phpdoc/LICENSE U phpdoc/Makefile.in [snip]

Configure and test the XML
bjori@lindsay:~$ cd phpdoc/ bjori@lindsay:~/phpdoc$ php configure.php configure.php: $Id: configure.php,v 1.7 2007/10/02 12:03:16 bjori Exp $ PHP version: 5.3.0-dev [snip] * No missing ids found All good. All you have to do now is run 'phd /home/bjori/phpdoc'

Installing and rendering the php.net documentation
bjori@lindsay:~/phpdoc$ cd .. bjori@lindsay:~$ pear install http://doc.php.net/phd/PhD-0.1RC1.tgz downloading PhD-0.1RC1.tgz ... Starting to download PhD-0.1RC1.tgz (19,683 bytes) .......done: 19,683 bytes install ok: channel://__uri/PhD-0.1RC1 bjori@lindsay:~$ mkdir build bjori@lindsay:~$ cd build/ bjori@lindsay:~/build$ time phd /home/bjori/phpdoc Creating php/toc/manual.inc... Creating php/toc/introduction.inc... Creating php/toc/getting-started.inc... [snip] Creating php/toc/userlandnaming.inc... Creating php/toc/about.inc... Creating php/toc/opl.license.inc... real 1m40.448s user 1m20.309s sys 0m6.420s bjori@lindsay:~/build$ ls -l total 26M -rw-r--r-- 1 bjori users 26M 10-02 12:28 bightml.html drwxr-xr-x 2 bjori users 364K 10-02 12:28 html/ drwxr-xr-x 3 bjori users 336K 10-02 12:28 php/ bjori@lindsay:~/build$ ls php | wc -l 6288

Thats right. It takes less than 2 minutes (on my two years old Precision M70 laptop) to render the entire php.net documentations in three formats.

To celebrate the release we have registered a new "special" mirror, docs.php.net,
which is running these experimental builds of PhD, and would like to request
that people take a quick look at it before we start pushing these builds out
to the rest of the mirrors.

There is currently no known bug in these builds so if you find one then
please file a bug report or at least let us know about it.
I don't care how small or major the bug is, a missing whitespace or a
whole page missing, let us know if you find anything.

-Hannes

"Latest releases" box and "conference teaser"

noreply@blogger.com (bjori) — Sun, 18 Feb 2007 18:52:00 +0000

Update (February 19th): Thanks to Steph this should look fine in our favorite browser, IE6, now.

I committed the conference teaser patch few hours ago, and as usual it only took about an hour for it to go live. I really think its a nice compromise, looks nice there above the news entrys and isn't that intrusive.

Anyway. I also committed "latest releases" box on the right, above the "event section".
The idea is to list the latest, stable, releases there - and when we are in RC phase then list them in a box below it. The links will be to qa.php.net where a "big fat warning box" will welcome the user and explain what a "release candidate" really is.
Release Candidates will now get more exposure and, hopefully, get more testing - benefiting all of us.

Speaking of 'latest releases', I've seen lot of code parsing the frontpage, the download page, the php-announce mailing list, heck, even cvs log in order to get the latest release info.
Since I moved the 'current releases' info into "a giant" includable array, living in include/version.inc, we can now provide a simple XML feed (or whatever) for you to parse to get the latest release info simpler and more accurately...

What do you think? Is it worth it? What kind of format would you want it in?...
Send me a mail (private or to the webmaster@ mailing list), add a comment, blog about it, scream loudly (hoping we hear you) or whatever. If the idea interest you, its your chance to get heard!

PHP.net frontpage changes

noreply@blogger.com (bjori) — Mon, 12 Feb 2007 16:55:00 +0000

As many of you know I changed the frontpage of php.net recently by moving conference announcements and call for papers to its own dedicated page in a desperate attempt to regain the control of our frontpage.

WTF?! came up among conference planners which didn't like the changes at all. Heck, why should they? They lost an important, free, ads on a very popular page on the web. The flip side: they have now their own page free of any distractions and irritations around their advertisements.

Regain control? Yes. Conference planers have treated php.net frontpage as their private commercial ground for long enough. It was time for the php.net "staff" to take control of the frontpage back and publish php.net relevant news.

This is a community effort, you are no king. Good point. I am no king. But enough is enough. Flooding the frontpage with conference announcements and call for papers had to stop. It was so overwhelming that there was no point for us to post real news about what was going on with php.net. 24 hours after an important PHP release a call for papers was committed above it, making the release "old news". Within a week of phpdoc major update there was a new conference announcement posted. This is ridiculous. I kicked around on IRC the idea of dedicated conference page and immediately got praised for it. Everyone (online and awake) agreed. Few hours later in a frustration caused by all those announcements I cleaned the frontpage up, moving old stuff into the archives and then added the conference page. Few days later I then added a dedicated feed for the announcements.

Now that things are starting to cool off I posted a patch for a conference teaser on the frontpage. I've learned my lesson. There is no way in hell I will commit without a notice.

Just to be clear. I do think PHP conferences are important, but they are not as important as what the php.net team is doing. They are however important enough to be on php.net, just not cluttering and making fun of php.net efforts.

Seeing post like the one from Cal Evans, Extra! Extra! Get Your Conference News Here!, is kinda funny. I had a hard time finding those feeds, in fact I can only see them in that blog entry.. and his php.net has had a change of policy link points to a post by conference planer.. Heh? Even funnier is the the fact I can't see any indication on importance of conferences on Zends Developer Zone at all. *sigh* That's the spirit Cal! Kick my nuts and send out fancy statements without anything to back it up.

Cal: I'll avenge my nuts in London... ;)

bjori doesn't blog

The Frills

ODS - Object<->Document Serializer

Next Generation MongoDB Driver for PHP!

What's the Point?

Again, What's the Point?

MongoDB PHP Libraries

What do you think?

Going tableless

I have a dream

What the hell are you talking about?

Now we have the chance to make it right!

What about my hosting situation?

When will this be available?

-Hannes

Query logging and profiling in MongoDB

PHP Stream Notifications ❤ MongoDB

New PHP.net designs floating around

A lot of things happen in two years..

There is an app^Wppa for that

up2date PHP5.3 packages for Ubuntu

The PHP project and Code Review

Unix manual pages for PHP functions

Most PHP releases in August!

Google breaking up with me?

8 reasons why you should *not* write for the php.net manual

PhD: The [PH]P based [D]ocbook renderer RC1 released

"Latest releases" box and "conference teaser"

PHP.net frontpage changes

8 reasons why you should not write for the php.net manual