Debuggable.com - Blog

Sales Almost Closing for CakeFest#3 in Berlin!

tim@debuggable.com — Mon, 06 Jul 2009 21:59:18 +0000

Hey folks,

There are only around 10 hours left until sales close for the up until now biggest CakeFest. It will take place from Jul 9 until Jul 12 in Debuggable's hometown Berlin, Germany. It is in fact two events in one: A CakePHP workshop and the main conference with talks presented by core developers and community members.

The CakePHP Workshop

The CakeFest CakePHP Workshop (July 9-10) provides an opportunity for beginner and intermediate developers to jump-start their experience with CakePHP by learning directly from the CakePHP core team. Participants are strongly encouraged to bring their laptops, as they will be learning hands-on how to set up and build CakePHP applications.

Nate and Mariano will talk about topics including basic application design, expert project workflow techniques, interacting with databases and web services, JavaScript and Ajax, and much more. The presentations are classroom-style allowing participants to follow along and ask questions. Check out the workshop schedule.

Felix, others and myself will be on-hand to provide one-on-one help. If you are a designer and the workshop pace is quite fast for you or if you are experienced, but have a difficult architectural problem that you need help with: The workshop and the one-on-ones will get you up and running and will provide all necessary help.

The Main Conference

The CakeFest Conference (July 11-12) is the primere event for CakePHP developers of all levels to meet, socialize, and learn collaboratively. Attendees will be immersed in a collective learning environment where some of the coolest ideas of the CakePHP community will be discussed and presented.

Topics include - but aren't limited to - Demystifying Webservices in CakePHP, Building Custom APIs and JavaScript for PHP Developers. Check out the full schedule and head on over and get your ticket.

What do we do in the evenings?

The evenings will be packed with fun dinners, bar visits and club visits. If you thought discussions would end then you are wrong. The CakeFests from the past proved to have very enjoyable evenings with fun action going on.

How many folks will be there?

Just to give you an idea about how many folks will be attending. As of now there are:

9 people from the CakePHP Development Team
35 people
62 people

.. attending.

So you can see, there are tons of opportunities to network, discuss and learn about new and old things. You will be able to meet core team members and discuss your applications, experience and wishes with them. In the evenings we will hang out, drink one or two beers, and just have fun! And all of that at affordable prices!

So, what are you waiting for? Hop on over and grab your ticket. :)

See you all there!

-- Tim Koschuetzki aka DarkAngelBGE

CouchDB Insert Benchmarks

felix@debuggable.com — Thu, 25 Jun 2009 13:30:41 +0000

Hey folks,

I am currently working on replacing Amazon S3 as the key value storage service for Debuggable's new startup. The main reason for that choice is that we want the ability to license the technology for in-house usage, which means the S3 dependency has to go.

Over the past year or so I have repeatedly heard good things about CouchDB (not to mention that Damien Katz became a personal hero of mine after seeing this video). But the main reason for preferring CouchDB over all the alternative key value stores is CouchDB's simplicity. Using HTTP + JSON as the protocol and embracing complete RESTfulness makes getting started with CouchDB ridiculously easy. It also makes it very awesome as you can build your architecture with other HTTP tools such as reverse proxies, http load balancers, etc..

Anyway, having done absolutely nothing with CouchDB before, I feel I should avoid shooting myself in the foot by making poor performance / scaling assumptions that may or may not be true for our particular use cases. So I did what every new user of an open source project ~~would~~ should do and setup some benchmarks.

However, the whole point of these is not to find out how fast CouchDB really is - I don't care as long as it scales horizontally. No, I am mostly interested in finding out 2 things:

a) How much slacking time do I have left before we seriously need to scale out to multiple CouchDB nodes
b) When that day comes, what can we expect in terms of replication delay (eventual consistency) in a multi-master setup

So far I have been mostly studying question a). Since our application is mostly storage heavy, but does not necessarily have lots of read hits (see, this is the point where you should realize my tests might not help answer your questions at all - sorry), I was wondering about the disk space consumption relative to the number of documents stored in the database.

After reading a bit about the B+ tree CouchDB uses for storage, I assumed that the required disk space would grow linear with the number of documents stored. However, my initial tests indicated that the disk space / document was growing with each document I was adding. Assuming a bad setup and after discussing this a bit in #couchdb, I decided to create a more serious setup on Amazon Ec2 inserting anywhere from 0 - 1 million records.

I put all the code for setting up the environment and running the tests on GitHub, see my couchdb-benchmarks project.

Anyway you waited long enough. Time for my initial results:

doc count (before compact): 0
doc count (after compact): 0
insert time: 0 sec
insert time / doc: n/a ms
compact time: 1.0064 sec
compact time / doc: n/a ms
disk size (before compact): 79 bytes
disk size (after compact): 79 bytes
.couch size (before compact): 79 bytes
.couch size (after compact): 79 bytes
.couch size / doc (before compact): n/a bytes
.couch size / doc (after compact): n/a bytes

doc count (before compact): 1
doc count (after compact): 1
insert time: 0.0015 sec
insert time / doc: 1.46 ms
compact time: 1.0051 sec
compact time / doc: 1005.14 ms
disk size (before compact): 315 bytes
disk size (after compact): 4179 bytes
.couch size (before compact): 315 bytes
.couch size (after compact): 4179 bytes
.couch size / doc (before compact): 315 bytes
.couch size / doc (after compact): 4179 bytes

doc count (before compact): 2
doc count (after compact): 2
insert time: 0.0015 sec
insert time / doc: 0.75 ms
compact time: 1.0057 sec
compact time / doc: 502.87 ms
disk size (before compact): 503 bytes
disk size (after compact): 8281 bytes
.couch size (before compact): 503 bytes
.couch size (after compact): 8281 bytes
.couch size / doc (before compact): 251.5 bytes
.couch size / doc (after compact): 4140.5 bytes

doc count (before compact): 3
doc count (after compact): 3
insert time: 0.0017 sec
insert time / doc: 0.56 ms
compact time: 1.0053 sec
compact time / doc: 335.1 ms
disk size (before compact): 693 bytes
disk size (after compact): 8281 bytes
.couch size (before compact): 693 bytes
.couch size (after compact): 8281 bytes
.couch size / doc (before compact): 231 bytes
.couch size / doc (after compact): 2760.33 bytes

doc count (before compact): 4
doc count (after compact): 4
insert time: 0.0017 sec
insert time / doc: 0.44 ms
compact time: 1.0053 sec
compact time / doc: 251.33 ms
disk size (before compact): 883 bytes
disk size (after compact): 8281 bytes
.couch size (before compact): 883 bytes
.couch size (after compact): 8281 bytes
.couch size / doc (before compact): 220.75 bytes
.couch size / doc (after compact): 2070.25 bytes

doc count (before compact): 5
doc count (after compact): 5
insert time: 0.0018 sec
insert time / doc: 0.36 ms
compact time: 1.0054 sec
compact time / doc: 201.08 ms
disk size (before compact): 1071 bytes
disk size (after compact): 8281 bytes
.couch size (before compact): 1071 bytes
.couch size (after compact): 8281 bytes
.couch size / doc (before compact): 214.2 bytes
.couch size / doc (after compact): 1656.2 bytes

doc count (before compact): 6
doc count (after compact): 6
insert time: 0.0019 sec
insert time / doc: 0.32 ms
compact time: 1.0051 sec
compact time / doc: 167.52 ms
disk size (before compact): 1261 bytes
disk size (after compact): 8281 bytes
.couch size (before compact): 1261 bytes
.couch size (after compact): 8281 bytes
.couch size / doc (before compact): 210.17 bytes
.couch size / doc (after compact): 1380.17 bytes

doc count (before compact): 7
doc count (after compact): 7
insert time: 0.0021 sec
insert time / doc: 0.3 ms
compact time: 1.005 sec
compact time / doc: 143.57 ms
disk size (before compact): 1459 bytes
disk size (after compact): 8281 bytes
.couch size (before compact): 1459 bytes
.couch size (after compact): 8281 bytes
.couch size / doc (before compact): 208.43 bytes
.couch size / doc (after compact): 1183 bytes

doc count (before compact): 8
doc count (after compact): 8
insert time: 0.0022 sec
insert time / doc: 0.27 ms
compact time: 1.0049 sec
compact time / doc: 125.61 ms
disk size (before compact): 1655 bytes
disk size (after compact): 8281 bytes
.couch size (before compact): 1655 bytes
.couch size (after compact): 8281 bytes
.couch size / doc (before compact): 206.88 bytes
.couch size / doc (after compact): 1035.13 bytes

doc count (before compact): 9
doc count (after compact): 9
insert time: 0.0023 sec
insert time / doc: 0.25 ms
compact time: 1.0053 sec
compact time / doc: 111.7 ms
disk size (before compact): 1849 bytes
disk size (after compact): 8281 bytes
.couch size (before compact): 1849 bytes
.couch size (after compact): 8281 bytes
.couch size / doc (before compact): 205.44 bytes
.couch size / doc (after compact): 920.11 bytes

doc count (before compact): 10
doc count (after compact): 10
insert time: 0.0025 sec
insert time / doc: 0.25 ms
compact time: 1.0042 sec
compact time / doc: 100.42 ms
disk size (before compact): 2043 bytes
disk size (after compact): 8281 bytes
.couch size (before compact): 2043 bytes
.couch size (after compact): 8281 bytes
.couch size / doc (before compact): 204.3 bytes
.couch size / doc (after compact): 828.1 bytes

doc count (before compact): 50
doc count (after compact): 50
insert time: 0.0072 sec
insert time / doc: 0.14 ms
compact time: 1.0038 sec
compact time / doc: 20.08 ms
disk size (before compact): 10319 bytes
disk size (after compact): 16473 bytes
.couch size (before compact): 10319 bytes
.couch size (after compact): 16473 bytes
.couch size / doc (before compact): 206.38 bytes
.couch size / doc (after compact): 329.46 bytes

doc count (before compact): 100
doc count (after compact): 100
insert time: 0.0136 sec
insert time / doc: 0.14 ms
compact time: 1.0054 sec
compact time / doc: 10.05 ms
disk size (before compact): 20430 bytes
disk size (after compact): 24665 bytes
.couch size (before compact): 20430 bytes
.couch size (after compact): 24665 bytes
.couch size / doc (before compact): 204.3 bytes
.couch size / doc (after compact): 246.65 bytes

doc count (before compact): 500
doc count (after compact): 500
insert time: 0.0687 sec
insert time / doc: 0.14 ms
compact time: 1.0062 sec
compact time / doc: 2.01 ms
disk size (before compact): 104616 bytes
disk size (after compact): 110690 bytes
.couch size (before compact): 104616 bytes
.couch size (after compact): 110690 bytes
.couch size / doc (before compact): 209.23 bytes
.couch size / doc (after compact): 221.38 bytes

doc count (before compact): 1000
doc count (after compact): 1000
insert time: 0.1361 sec
insert time / doc: 0.14 ms
compact time: 1.003 sec
compact time / doc: 1 ms
disk size (before compact): 212260 bytes
disk size (after compact): 217186 bytes
.couch size (before compact): 212260 bytes
.couch size (after compact): 217186 bytes
.couch size / doc (before compact): 212.26 bytes
.couch size / doc (after compact): 217.19 bytes

doc count (before compact): 2500
doc count (after compact): 2500
insert time: 0.4686 sec
insert time / doc: 0.19 ms
compact time: 1.006 sec
compact time / doc: 0.4 ms
disk size (before compact): 814957 bytes
disk size (after compact): 819298 bytes
.couch size (before compact): 814957 bytes
.couch size (after compact): 819298 bytes
.couch size / doc (before compact): 325.98 bytes
.couch size / doc (after compact): 327.72 bytes

doc count (before compact): 5000
doc count (after compact): 5000
insert time: 0.9165 sec
insert time / doc: 0.18 ms
compact time: 1.0065 sec
compact time / doc: 0.2 ms
disk size (before compact): 2012394 bytes
disk size (after compact): 2015330 bytes
.couch size (before compact): 2012394 bytes
.couch size (after compact): 2015330 bytes
.couch size / doc (before compact): 402.48 bytes
.couch size / doc (after compact): 403.07 bytes

doc count (before compact): 7500
doc count (after compact): 7500
insert time: 1.5116 sec
insert time / doc: 0.2 ms
compact time: 2.0112 sec
compact time / doc: 0.27 ms
disk size (before compact): 3778774 bytes
disk size (after compact): 3797090 bytes
.couch size (before compact): 3778774 bytes
.couch size (after compact): 3797090 bytes
.couch size / doc (before compact): 503.84 bytes
.couch size / doc (after compact): 506.28 bytes

doc count (before compact): 10000
doc count (after compact): 10000
insert time: 2.3111 sec
insert time / doc: 0.23 ms
compact time: 3.015 sec
compact time / doc: 0.3 ms
disk size (before compact): 5653905 bytes
disk size (after compact): 5652578 bytes
.couch size (before compact): 5653905 bytes
.couch size (after compact): 5652578 bytes
.couch size / doc (before compact): 565.39 bytes
.couch size / doc (after compact): 565.26 bytes

doc count (before compact): 25000
doc count (after compact): 25000
insert time: 6.8684 sec
insert time / doc: 0.27 ms
compact time: 7.0746 sec
compact time / doc: 0.28 ms
disk size (before compact): 20595235 bytes
disk size (after compact): 20635746 bytes
.couch size (before compact): 20595235 bytes
.couch size (after compact): 20635746 bytes
.couch size / doc (before compact): 823.81 bytes
.couch size / doc (after compact): 825.43 bytes

doc count (before compact): 50000
doc count (after compact): 50000
insert time: 15.8227 sec
insert time / doc: 0.32 ms
compact time: 14.1612 sec
compact time / doc: 0.28 ms
disk size (before compact): 51808040 bytes
disk size (after compact): 51724386 bytes
.couch size (before compact): 51808040 bytes
.couch size (after compact): 51724386 bytes
.couch size / doc (before compact): 1036.16 bytes
.couch size / doc (after compact): 1034.49 bytes

doc count (before compact): 100000
doc count (after compact): 100000
insert time: 35.3071 sec
insert time / doc: 0.35 ms
compact time: 33.4723 sec
compact time / doc: 0.33 ms
disk size (before compact): 125497442 bytes
disk size (after compact): 125419618 bytes
.couch size (before compact): 125497442 bytes
.couch size (after compact): 125419618 bytes
.couch size / doc (before compact): 1254.97 bytes
.couch size / doc (after compact): 1254.2 bytes

doc count (before compact): 250000
doc count (after compact): 250000
insert time: 104.0009 sec
insert time / doc: 0.42 ms
compact time: 97.3738 sec
compact time / doc: 0.39 ms
disk size (before compact): 394489375 bytes
disk size (after compact): 394457190 bytes
.couch size (before compact): 394489375 bytes
.couch size (after compact): 394457190 bytes
.couch size / doc (before compact): 1577.96 bytes
.couch size / doc (after compact): 1577.83 bytes

doc count (before compact): 500000
doc count (after compact): 500000
insert time: 230.6021 sec
insert time / doc: 0.46 ms
compact time: 209.0139 sec
compact time / doc: 0.42 ms
disk size (before compact): 900271866 bytes
disk size (after compact): 900280422 bytes
.couch size (before compact): 900271866 bytes
.couch size (after compact): 900280422 bytes
.couch size / doc (before compact): 1800.54 bytes
.couch size / doc (after compact): 1800.56 bytes

doc count (before compact): 750000
doc count (after compact): 750000
insert time: 354.7959 sec
insert time / doc: 0.47 ms
compact time: 380.9895 sec
compact time / doc: 0.51 ms
disk size (before compact): 1446452532 bytes
disk size (after compact): 1445376102 bytes
.couch size (before compact): 1446452532 bytes
.couch size (after compact): 1445376102 bytes
.couch size / doc (before compact): 1928.6 bytes
.couch size / doc (after compact): 1927.17 bytes

doc count (before compact): 1000000
doc count (after compact): 1000000
insert time: 487.3284 sec
insert time / doc: 0.49 ms
compact time: 570.2633 sec
compact time / doc: 0.57 ms
disk size (before compact): 2023280441 bytes
disk size (after compact): 2022334566 bytes
.couch size (before compact): 2023280441 bytes
.couch size (after compact): 2022334566 bytes
.couch size / doc (before compact): 2023.28 bytes
.couch size / doc (after compact): 2022.33 bytes

From this data, a few assumptions can be made:

CouchDB inserts ~2-3k documents / second in a >100k documents database (for this particular hardware / benchmark setup)
CouchDB inserts get slower on bigger databases
CouchDB seems to use more bytes / document the larger the database gets (this is scary, but might explain the previous 2 observations)
The time it takes for compacting a database with identical, unmodified documents seems to be almost equal to the time it took to insert the initial documents. Makes lots of sense assuming the writes are I/O bound.

Now since I am new to CouchDB there is a very large chance for problems with my setup and the logic behind my assumptions. However, I hope that by sharing them I can get feedback to make the benchmarks better and provide explanations for the observed characteristics.

I also hope that some of you might feel compelled to fork the project on GitHub and provide more benchmarks. Personally I am going to work on analyzing replication next. If I find time I'll also add some CSV exports and pretty rendering facilities with some google charts.

Comment if you have any thoughts, want to see more tests or share some religious propaganda related to your key-value storage system of choice : ).

-- Felix Geisendörfer aka the_undefined

PS: A few questions I can already see coming up:

Why does compact always take at least 1 second?
Because I use a while() loop with sleep(1) to determine when compact is done. I could check more frequently, but its not really a variable I'm interested in.

Why does compact increase the file size for < 50000 documents?
Good question, I have no idea. Anybody?

What insert method is used?
Have a look at the benchmark source. Basically it's bulk-inserts of 1000 items at a time with pre-generated UUIDs.

I ran the benchmark and got some additional output
Yeah, I removed some debugging / activity indicators to make the results more readable for this article.

Final CakeFest Schedule Announced!

tim@debuggable.com — Wed, 24 Jun 2009 11:23:54 +0000

Hey folks,

the final schedule for the upcoming CakePHP Conference - our beloved "CakeFest" - is announced! For those of you who don't know about the event yet, here is a little wrapup.

The next and third CakeFest will be held in Berlin, hometown of Debuggable! From July 9-12, people from all over Europe and the rest of the world will travel to Germany in order to celebrate and learn about the best PHP framework there is. Throughout the day you will learn a lot about:

.. how you can tweek your application to run faster
.. how to write cleaner code
.. how to make your code ready for the real world
.. how you can just become a better CakePHP developer

The evenings are packed with fun events .. dinner with the Core Team, going to bars, chatting about this and that and programming. And more.

You don't want to miss this event! :)

If you have not attended a CakeFest so far, here are some cool memories from the past:

The event consists of two parts:

July 9-10: CakePHP Workshop - (Lead by the CakePHP core team)
July 11-12: CakePHP Conference - (Presented by the core team + community)

In order to check out the full schedule with the talks and their speakers, head on over to the official CakeFest site.

Hope to see you all there!

-- Tim Koschützki aka DarkAngelBGE

[JOB] Senior CakePHP position with english360.com

felix@debuggable.com — Mon, 22 Jun 2009 13:50:28 +0000

Hey there,

if you are looking for a fantastic full-time opportunity to write CakePHP code, check out this position with english360.com:

We’re a unique combination: an e-learning startup looking to shake things up through innovation, recently partnered with one of the oldest, most prestigious educational publishers in the world, Cambridge University Press.

Our flagship web application is now at version 1, and we are looking for a senior PHP developer with the desire and attitude to take the technology lead within English360. Have no fear of a poor code base: the initial application has been developed by the Cake Development Corporation, the frameworks largest contributors.

Now we are looking for you:

You’re a world-class PHP developer who has previous experience with CakePHP, or the ambition to pick it up from the folks who wrote it. You are a testing / Q&A fanatic, proficient in jQuery/HTML/CSS, and a SysAdmin wizard.

After a 6-month, full-time contractor role, you would take full responsibility for the technology side of the company, manage contractors, build your department, and collaborate with senior management on the future direction of our products. Ideally you would also be located in, or willing to relocate to either our New York City or Cambridge offices, after successful completion of your contract.

Interested? Contact e360jobs@gmail.com and impress them with a fantastic cover letter and a list of previous work!

-- Felix Geisendörfer aka the_undefined

What is a Testcase - especially in CakePHP?

tim@debuggable.com — Tue, 09 Jun 2009 16:16:39 +0000

Hey folks,

it's amazing how many people on the public CakePHP irc channel don't know what a test case is. This has to stop. To clear up some confusion, let's have a look at it.

What is a Test Case? A Possible Definition

This is what wikipedia has to say about a testcase:

A test case in software engineering is a set of conditions or variables under which a tester will determine whether an application or software system is working correctly or not. The mechanism for determining whether a software program or system has passed or failed such a test is known as a test oracle. In some settings, an oracle could be a requirement or use case, while in others it could be a heuristic. It may take many test cases to determine that a software program or system is functioning correctly. Test cases are often referred to as test scripts, particularly when written. Written test cases are usually collected into test suites.

So is this me testing stuff in my browser?

Yes and no. The definition does not necessarily imply that a testcase has anything to do with Unit Testing, or other testing approaches.

You testing your menu bar's javascript in several browsers in different resolutions can also be considered to be a testcase (or many for that matter) according to the definition. However, most of the time a testcase is meant to be a written test. When it comes to CakePHP, it's mainly a Unit Test.

Where can I learn about test cases?

There are a ton of resources on the web for tests. If you are looking for material for CakePHP, try out these:

The Article "Introduction to Unit Testing Part 1" written by yours truly
The CakePHP CookBook Chapter on Testing
Your First Testcase
Mark Story's Unit Testing Talk Slide for CakeFest Argentinia December 2009
My slides for the Unit Testing Talk for CakeFest Orlando February 2008

Lets get a discussion started to clear up confusion.

-- Tim Koschuetzki aka DarkAngelBGE

Migrating a Database Table for use with the CakePHP Sluggable Behavior

tim@debuggable.com — Mon, 08 Jun 2009 11:30:31 +0000

Hey folks,

if you are not familiar with Mariano's Sluggable Behavior, you should definitely check it out. It's a nice tool to generate SEO-friendly urls in your application. If you are anything like debuggable.com with our long urls, you might want to look at it. ;)

If you don't know what a url slug is:

A slug is a few words that describe a post or a page. Slugs are usually a URL friendly version of the post title, but a slug can be anything you like. Slugs are meant to be used with your site's urls as they help describe what the content at the URL is. They might or might not help your SEO ranking as well for the keywords in the slug/content.

Example post url: http://flashfun247.com/games/play/racing/tg-motocross-2.

The slug for that game content is "tg-motocross-2".

This blogpost deals with migrating an existing production table into using Mariano's sluggable behavior and thereby increasing SEO friendliness of your site.

Step 1: Back up the database table

Back up the table for which you want to add slugs. We cannot be blamed for any database damage this might cause.

Step 2: Modify your database

Add a "slug" field to the table. The default length that the behavior uses is 100 chars which should be enough in most cases. So VARCHAR(100) is what you need or the equivalent for your db driver.

Step 3: Add your $actsAs declaration

Add the proper call to $actsAs to the model that you want to migrate. Put in any existing behaviors that you need as well:

php

var $actsAs = array(
'Containable',
'Lookupable',
'Sluggable' => array('overwrite' => true)
);

Make sure to set the overwrite option, as otherwise the behavior will refuse to overwrite the slug field the way we will do it. Also make sure to set the 'label' option if the table field that your slugs will depend on is not called 'title'.

For the full host of options, check the sluggable behavior.

Important: The field your slugs depend on (title, name or combinations) MUST NOT be empty for any row in your table. The shell takes this into account and provides you with an error log. To save you some headaches, make sure the fields are properly filled before you run the sluggish shell.

Step 4: Modify The Sluggable Behavior Code

You need to change the Sluggable behavior code on two occurances to make it work with the recent Cake releases.

1. Change Line 121 to:

php

$conditions = array($Model->alias . '.' . $this->__settings[$Model->alias]['slug'] . ' LIKE' => $slug . '%');

2. Change Line 125 to:

php

$conditions[$Model->alias . '.' . $Model->primaryKey . ' <>'] = $Model->id;

This is simply changing the syntax for NOT and LIKE conditions.

Step 5: Download Debuggable's Sluggish Shell

Copy the following code to a 'sluggish.php' file in your vendors/shells folder. Just mouseover the code to see it in raw text.

php

<?php
/**
* Sluggish Shell
*
* Set overwrite => true before running this in your $actsAs declaration!
* This shell allows you to generate unique slugs for a database table ready for use
* with the sluggable behavior by Mariano Iglesias
*
*
* Sluggish Shell : Make your table sluggable
* Copyright 2009, Debuggable, Ltd. (http://debuggable.com)
*
* Licensed under The MIT License
* Redistributions of files must retain the above copyright notice.
*
* @filesource
* @copyright Copyright 2009, Debuggable, Ltd. (http://debuggable.com)
* @link http://www.cakefoundation.org/projects/info/cakephp CakePHP(tm) Project
* @license http://www.opensource.org/licenses/mit-license.php The MIT License
*/
class SluggishShell extends Shell {
/**
* undocumented function
*
* @return void
* @access public
*/
function main() {
if (empty($this->args)) {
return $this->out('You need to specify a modelname');
}
$model = $this->args[0];
$force = isset($this->args[1]) ? $this->args[1] : false;
$Model = ClassRegistry::init($model);
$behavior = 'Sluggable';
if (!is_object($Model)) {
return $this->out('This model does not exist.');
}
if (!in_array($behavior, $Model->actsAs) && !array_key_exists($behavior, $Model->actsAs)) {
return $this->out('The Sluggable Behavior is not yet linked to the model.');
}
$label = 'title';
if (isset($Model->actsAs['Sluggable']['label'])) {
$label = $Model->actsAs['Sluggable']['label'];
}
$Model->recursive = -1;
$conditions = $force ? false : array('slug' => '');
$rows = $Model->find('all', compact('conditions'));
$count = count($rows);
$i = 0;
foreach ($rows as $row) {
$Model->set(array(
$Model->primaryKey => $row[$model][$Model->primaryKey],
$label => $row[$model][$label]
));
$Model->save();
$row = $Model->find('first', array(
'conditions' => array(
$Model->primaryKey => $row[$model][$Model->primaryKey],
'slug' => ''
),
'contain' => false
));
if (empty($row)) {
$i++;
} else {
$this->out('Problem saving the slug for ' . $row[$model][$label]);
$this->out($Model->validationErrors);
}
}
$this->out('Added ' . $i . ' slugs for ' . $count . ' ' . Inflector::pluralize($model));
}
/**
* undocumented function
*
* @return void
* @access public
*/
function help() {
$this->out('Debuggable Ltd. Sluggish Shell - http://debuggable.com');
$this->hr();
$this->out('Important: Configure your paths in the shell\'s initialize() function.');
$this->hr();
$this->out('This shell allows you to migrate a database table to use Mariano Iglesias\' Sluggable Behavior.');
$this->out('Add a slug field to the table, download the sluggable behavior, add your $actsAs declaration and run this shell.');
$this->out('');
$this->hr();
$this->out("Usage: cake sluggish ModelNameInCamelCase");
$this->out('');
}
}
?>

The code is very simple. The shell takes a modelname and checks if your $actsAs declaration is properly set up. If it is, it finds all rows from the table and saves them right away, with the label field in the Model::set() call.
Since we have specified overwrite => true in the $actsAs declaration, the sluggable behavior now overwrites all slug fields in the table, giving you nice slugs.

Step 6: Run The Sluggish Shell

Run "cake sluggish ModelNameInCamelCase" in your terminal.

Step 7: Change your urls

The Sluggable behavior creates unique slugs. If two of your blogposts for example have the same title, the first one will have 'my-blogpost-title' as its slug and the second one will have 'my-blogpost-title-1'. Mariano's behavior attaches an integer to the slug depending on how often the slug was used already.

Now that you have unique slugs, you can change your controller code and view code to take account of this. If you are heavily indexed in google already, you might want to provide 301 (permanent) redirects for the old urls, or just offer both urls to access the same blogpost, but only use slugs throughout the app.

Enjoy! Post any feedback or help requests in the comments below.

-- Tim Koschuetzki aka DarkAngelBGE

CakePHP RemoveCache Shell - Remove Your Cache Files Easily

tim@debuggable.com — Thu, 04 Jun 2009 09:38:32 +0000

Hey folks,

it's been a while since the last post here on Debuggable. However, this ends now and apart from this very post here we have something bigger brewing for you as well. So please stay tuned a little longer. :)

Anyways, on to some Caching fun.

The Motivation

1. When you work with large projects you can end up with a ton of cache files (models, db cache, etc.) as well as view caching files. When you try removing them with:

cd /app/tmp/cache/models && rm -f cake_*

and

cd /app/tmp/cache/views && rm -f *

.. you can end up with with the "argument list too long" error easily.

2. Also, executing two commands sucks - we are lazy after all. Now you could pull out some fancy bash fun to pipe file names. Have a look at this:

find . -type f | awk '!/empty/ {print "rm", $0}' | bash

The problem is, when you run this in /app/tmp it will not only remove cache files, but also files in /tmp/sessions, /tmp/logs and so on. If you ask me, the command is complex enough, so no need to add more funny stuff there to take this into account.

(For you peeps who want to see this, I bugged Felix to tell me: find . -type f | awk '!/empty$|^.\/logs|^.\/sessions/ {print "rm", $0}' | bash)

3. Once you are on windows, you do not have a powerful bash to your side.

I thought a simple call to a CakePHP shell can do the trick as well and doesn't force you to waste half a minute to remember and type in the proper bash command.

The Solution

The RemoveCache shell allows you to remove your cae cache files easily. It takes two parameter:

A boolean to control if you want to remove standard cache files (models, db cache, etc.)
A regex pattern to control which view cache files to remove

Here are some common usage scenarios:

Usage: cake remove_cache <std_cache_boolean> <pattern_to_match_viewcache_files>
Usage: cake remove_cache // removes all cache files
Usage: cake remove_cache 0 // removes only view cache files
Usage: cake remove_cache 0 home // removes only the view cache file for your homepage
Usage: cake remove_cache 0 articles_ // removes all view cache files for your articles controller
Usage: cake remove_cache 1 /letter_z$/ // removes all std cache files and view cache files ending with 'letter_z'

I did not put in a pattern for standard cache files, because most of the time you cannot remember your cache keys anyways and most of the time it doesn't harm if a cache file is invalidated to rebuilt the cache. If someone wants a pattern for that too, because they have long-taking queries, just comment and I will add it.

By default, the shell looks in your standard /app/tmp directory (plus subfolders) to find the cache files. If you have a shared Cake installation or any other fancy setup, please adjust the cache paths in the inititalize() method.

The Code

php

<?php
/**
* Remove Cache Shell
*
* This shell allows you to remove cache files easily and provides you with a couple configuration options.
* If run with no command line arguments, RemoveCache removes all your standard cache files (db cache, model cache, etc.)
* as well as your view caching files.
*
*
* RemoveCache Shell : Removing your Cache
*
* Licensed under The MIT License
* Redistributions of files must retain the above copyright notice.
*
* @filesource
* @link http://www.cakefoundation.org/projects/info/cakephp CakePHP(tm) Project
* @license http://www.opensource.org/licenses/mit-license.php The MIT License
*/
class RemoveCacheShell extends Shell {
/**
* undocumented function
*
* @return void
* @access public
*/
function initialize() {
parent::initialize();
$this->settings = array(
'view_cache_path' => APP . 'tmp' . DS . 'cache' . DS . 'views',
'std_cache_paths' => array(
APP . 'tmp',
APP . 'tmp' . DS . 'cache',
APP . 'tmp' . DS . 'models',
APP . 'tmp' . DS . 'persistent'
)
);
}
/**
* undocumented function
*
* @return void
* @access public
*/
function main() {
$args = $this->args;
$stdCache = !isset($args[0]) || $args[0];
$viewCachePattern = isset($args[1]) ? $args[1] : '.*';
if ($stdCache) {
$this->_cleanStdCache();
}
$this->_cleanViewCache($viewCachePattern);
}
/**
* Cleans the standard cache, ie all model caches, db caches, persistent caches
* Files need to be prefixed with cake_ to be removed
*
* @return void
* @access public
*/
function _cleanStdCache() {
$paths = $this->settings['std_cache_paths'];
foreach ($paths as $path) {
$folder = new Folder($path);
$contents = $folder->read();
$files = $contents[1];
foreach ($files as $file) {
if (!preg_match('/^cake_/', $file)) {
continue;
}
$this->out($path . DS . $file);
@unlink($path . DS . $file);
}
}
}
/**
* Cleans all view caching files. Takes a pattern to match files against.
*
* @param string $pattern
* @return void
* @access public
*/
function _cleanViewCache($pattern) {
$path = $this->settings['view_cache_path'];
if ($pattern{0} != '/') {
$pattern = '/' . $pattern . '/i';
}
$folder = new Folder($path);
$contents = $folder->read();
$files = $contents[1];
foreach ($files as $file) {
if (!preg_match($pattern, $file)) {
continue;
}
$this->out($path . DS . $file);
@unlink($path . DS . $file);
}
}
/**
* undocumented function
*
* @return void
* @access public
*/
function help() {
$this->out('Debuggable Ltd. Remove Cache Shell - http://debuggable.com');
$this->hr();
$this->out('Important: Configure your paths in the shell\'s initialize() function.');
$this->hr();
$this->out('This shell allows you to remove cache files easily and provides you with a couple configuration options.');
$this->out('If run with no command line arguments, RemoveCache removes all your standard cache files (db cache, model cache, etc.) ');
$this->out('as well as your view caching files.');
$this->out('');
$this->out('Set the first parameter to 0 (zero), to not remove standard cache files.');
$this->out('Set a regex pattern for the second argument, to match viewcache files to delete.');
$this->hr();
$this->out("Usage: cake remove_cache <std_cache_boolean> <pattern_to_match_viewcache_files>");
$this->out("Usage: cake remove_cache \t\t// removes all cache files");
$this->out("Usage: cake remove_cache 0 \t\t// removes only view cache files");
$this->out("Usage: cake remove_cache 0 home \t// removes only the view cache file for your homepage");
$this->out("Usage: cake remove_cache 0 articles_ \t// removes all view cache files for your articles controller");
$this->out("Usage: cake remove_cache 1 /letter_z$/ \t// removes all std cache files and view cache files ending with 'letter_z'");
$this->out('');
}
}
?>

Enjoy! Feedback welcome.

-- Tim Koschuetzki aka DarkAngelBGE

The biggest CakeFest to be held in Berlin

felix@debuggable.com — Wed, 25 Mar 2009 16:08:44 +0000

I will make this short. The 3rd and so far biggest CakeFest will be held in Berlin, home of Debuggable! From July 9-12, people from all over Europe and the rest of the world will travel to Germany in order to celebrate and learn about the best PHP framework there is.

If you have not attended a CakeFest so far, here are some good ideas of what to expect:

The event consists of two parts:

July 9-10: CakePHP Workshop - (Lead by the CakePHP core team)
July 11-12: CakePHP Conference - (Presented by the core team + community)

The 2-day workshop is packed with a series of tutorials designed to give developers, both new and seasoned, a solid understanding in building reliable CakePHP applications. Veterans and experts can skip over stuff they already know and use this time for 1 on 1 sessions with the non-presenting developers. The Workshop + Conference ticket is 599 EUR (499 EUR if you do not attend the conference).

The conference itself is going to be packed with talks, delivered by both the CakePHP core team as well as interested community members. At just 199 EUR (student discounts to be announced soon) there is no excuse for not attending.

I will talk on JavaScript for (Cake)PHP developers as well as share my experience in project management using GitHub + Lighthouse. Tim is still making up his mind, but will probably talk about Advanced Debugging.

The location for the conference is a few streets down in my neighborhood, so be assured that we will have more than enough opportunity to gather & celebrate in the evenings.

If you have any questions, please feel free to post them here or email them to me at felix@debuggable.com.

Otherwise go ahead and sign up, you will not regret it as the conference will also feature some major announcements exclusively made there.

- -Felix Geisendörfer aka the_undefined

Git alias for displaying the GitHub commit url

felix@debuggable.com — Wed, 18 Mar 2009 13:09:01 +0000

If you often find yourself pointing your team members to commit urls in GitHub, this might be fun for you.

I created git alias called 'hub' that automatically guesses the github repository url for the repository you are currently in:

$ git hub
https://github.com/felixge/my-project

Based on that I created a second alias called 'url', which gives you the url to HEAD commit:

$ git url
https://github.com/felixge/my-project/commit/0bdc57323a1ffec7ffe10bf83147cab5d6838d45

You can however also provide another sha1 you want to link to:

$ git url 22db8914220b717b0954b84365030ae3c9602a17
https://github.com/felixge/my-project/commit/22db8914220b717b0954b84365030ae3c9602a17

If you find those aliases useful, here are my ~/.gitconfig alias definitions for them:

[alias]
  hub =! echo "https://github.com/"`git config remote.origin.url` | sed -E s/[a-z]+@github\.com:// | sed s/\.git$//
  url =!sh -c 'HEAD=`git rev-parse HEAD` && SHA1=`[ "$0" = "sh" ] && echo $HEAD || echo $0` && echo `git hub`"/commit/"${SHA1i}'

(Bash gurus: I am sure you can do the above much more elegantly, wanna give it a try?)

Further I also have this little bash script in ~/bin/tiny:

#!/bin/bash
curl "http://tinyurl.com/api-create.php?url=$1"
echo ""

This allows me to make tiny urls for my links if I need to paste them on long-url unfriendly territory:

$ git url | xargs tiny
http://tinyurl.com/cva44t

Or if you are on OSX, you can use this to open the git url in your browser:

$ git url | xargs open

Now all of the above could probably be done much smarter, but as far as I am concerned it works great ; ).

-- Felix Geisendörfer aka the_undefined

PS: What command line tricks are part of your daily workflow?

Muscles on demand - Clean a large git repository the cloud way

felix@debuggable.com — Fri, 13 Mar 2009 17:34:28 +0000

Hey folks,

don't you hate it when you sometimes have to stop your work because your dev machine is ultra-busy doing some CPU or I/O heavy operations that will take hours?

Even so it doesn't happen to me a lot, I actually ran into such a case last night while trying to fix the Git repository of a project we are working on. The repository itself was not corrupted, but it became so fat that git-index-pack would explode on many of the team members. How did that happen? Well it turns out that over time some of the image directories of the project were committed into the repository by accident. This ended up being an insane 1.7 GB of '.git' objects.

With SVN, this is when you realize you made a poor choice in versioning control software and it is time to start the repository over - loosing all history.

Not so much wit Git. Git has an excellent tool called git-filter-branch that you can use to rewrite an entire repositories history.

In our case we wanted to pretend app/webroot/files had never existed:

git filter-branch --index-filter 'git rm -r --cached app/webroot/files' -- --all

However, our repository is full of commits (3.5k) and as I mentioned has some incredibly huge and ugly blobs in it. This means the operation above is slow like crazy.

So instead of having my poor laptop tortured with it for an hour, I decided to hire some muscle. I knew the operation I planned to do was fairly CPU and I/O heavy. For that reason I fired up an c1.xlarge Ec2 instance featuring 8 cores and 7 GB RAM.

Launching one of Eric Hammond's excellent Ubuntu EC2 AMI's and installing git took < 5 minutes. Add a few minutes to transfer the 1.7GB repository over and I was ready to go.

Having recently read Kevin's excellent article on tmpfs, I just put the repository in /dev/shm. This simply meant that the repository was now fully stored in memory - 30x faster then HDD!

Even with all this power, the whole process still took 15 minutes to complete, but the result was impressive. Instead of 1.7GB the repository was shrunk down to 80MB and little angels were dancing & singing around it. It was beautiful : ).

I pushed the lean and mean clone up to github using 'git push -f' and then switching over the local clones of each team member was just a matter of:

git checkout -b backup-master
git branch -D master
git fetch
git checkout -b master origin/master

Of course the new master branch and its backup wouldn't be very nice to each other as far as merges are concerned, but cherry picking the most recent commits worked great.

As you can see cloud computing is not only for the application level, but it can also be a great tool for your development process as a whole. After all it is incredible what can be accomplished for $0.80 this way ; ).

-- Felix Geisendörfer aka the_undefined