Queries are case-sensitive

Fields and queries in MongoDB are case-sensitive:

1
2
3

var test1 = db.test.find({'tags': 'jquery'}).count();
var test2 = db.test.find({'tags': 'jQuery'}).count();
test1 == test2; // Output is false - they do not query for the same information

This can cause some headaches if you don’t normalize user input ahead of time. Imagine you already have several posts tagged with ‘NoSQL’ and users enter ‘nosql’ in a tag search box. If you don’t normalize how the data is stored internally (like lowercase all tags), your user will see a much smaller set of posts than they are expecting to see. This is something you don’t have to worry or even think about with MySQL and most other relational databases.

If you can’t normalize the stored data, but still want a case-insensitive query, then you must perform a slower regular expression query:

1

db.test.find({'tags': /jquery/i}); // Note the 'i' flag for case-insensitive

Data is type-sensitive

Data stored within MongoDB knows it’s type. There is a small but significant difference between these two records:

1
2

{'count': 102}; // 'count' is stored as an int
{'count': "102"}; // 'count' is stored as a string

This is obvious to any programmer when presented like this, but what may not be obvious is that this also affects how you can query for these records:

1
2

// This returns 1 instead of 2, because it only matches the integer value
db.test.find({'count': 102}).count();

This is due to how MongoDB stores documents internally with BSON (Binary JSON), using various MongoDB data types. This means – like the point above – that you must pay attention to how you are saving data into MongoDB, because it will affect how you can query for it later.

Documents sizes are capped at 4MB each

This isn’t a big issue for most people, but it’s something to be aware of if you plan on storing large chunks of text or nesting a bunch of objects inside a single document. Nesting comments inside article documents is a particular approach that may give you pause knowing there is an upper threshold on document size.

Note that this limitation is not an issue for storing files in MongoDB because GridFS transparently divides the contents of files larger than 4MB across multiple documents.

Only one index is used per query

Simply adding more indexes doesn’t always help queries run faster. MongoDB can’t use multiple indexes together like MySQL and other RDBMS can (see “Index Merge Optimization“). This means that if a query is selecting or sorting based on multiple fields, a compound index should be created with those fields for the query to run most efficiently.

MongoDB is for high-memory and 64-bit systems only

Although MongoDB can be downloaded, compiled, and installed on 32-bit systems, it should never be run in production on them. This is because MongoDB stores all indexes in memory as well as memory-mapped files for all disk I/O for increased speed and throughput. While these two facts aren’t “gotchas” themselves – they are clearly explained on the website – it does mean that MongoDB can quickly use a lot of memory, especially as the size of your database grows and becomes significant. Since 32-bit systems have an effective 3GB memory limit of addressable memory space, they prevent you from being able to add more memory as the size of your database grows. This is also something to think about if you plan to run MongoDB on a small VPS with limited memory, even if it is 64-bit. You may be forced into a memory upgrade sooner than you are prepared for if your database is large or rapidly growing.

Subtle Differences

When starting out with MongoDB, it’s easy to draw a lot of parallels to MySQL and make a lot of assumptions based on those similarities. Collections are kind of like tables, fields are kind of like columns, a document is kind of like a row. You may find yourself going on to make more assumptions about how it’s storing data, how you can query it, what you can do with it, etc. without even knowing it. So while MongoDB has a lot of similar features and is one of the easiest NoSQL databases to transition to from a relational database, it has some very distinct differences “under the hood” that you have to be aware of and plan for ahead of time. The MongoDB documentation is well-written, and is pretty good at explaining any potential differences or “gotchas”, so make sure to read it thoroughly before making the jump – and watch that first step.

Once you fully get the NoSQL viewpoint and the implications it has on the future of data storage, you have one of those “why didn’t anyone think of this sooner?” moments. And yes, I know key/value stores like memcached existed long before this whole new “NoSQL” movement, but it’s not quite the same thing.

Object Databases

Object databases are an interesting case. They are in the “NoSQL” category and solve the same problem other NoSQL databases do – creating a storage mechanism that fits your data instead of making your data fit a predefined rigid storage mechanism. So why didn’t they catch on? What gives? It’s anyone’s guess why object databases didn’t catch on and achieve widespread adoption, but one thing is for sure – object databases really missed the boat while other NoSQL database technologies are running rings around them.

The primary problem is that most object databases are designed for a specific language, like Java or .NET. Some have more language support than others, but the problem is still the same – by integrating in directly with a language’s object model they depend on specific language implementations, which severely limits adoption. They solve the impedance mismatch problem that RDBMSes present, but fail to provide language-agnostic data persistence and access, effectively tying their use to specific technology stacks and platforms.

The Best of Both Worlds

NoSQL databases are gaining traction like no other database technology has in over 30 years. The reason is that good NoSQL databases solve both key data storage problems: impedance mismatches AND language-agnosticism. By storing the data in an intermediary format like JSON (or BSON) and allowing custom file content types, the focus is on the actual data itself instead of specific technologies or data models. The technology stack becomes irrelevant, and the walls to gaining user adoption crumble.

The bottom line is that NoSQL databases are not just the latest fad. They represent a fundamental paradigm shift in data storage and are an entirely new way of thinking about how to store and access your data in a way that makes sense for your actual usage. NoSQL databases are absolutely something you should be paying attention to and following closely – they are a strong contender to the RDBMS model and will likely become the de-facto data storage choice for most new web applications within a decade.

Return all the rows with multi-byte characters in table posts on field title:

1
2
3

SELECT *
FROM posts
WHERE LENGTH(title) <> CHAR_LENGTH(title)

This does pretty much what it looks like: returns all the rows in the posts table where the title’s character length does not match the title’s length. This works because the LENGTH function returns the number of bytes in the string, while the CHAR_LENGTH function returns the number of characters in the string. If the string contains multi-byte characters (individual characters that are made up of more than one byte) like international UTF-8 characters, the two functions will return different numbers and will not be equal, thus including that row in your results.

1
2
3
4
5
6
7
8
9
10
11
12

class BobUser
{
	public $name = 'bob';
	public $publicFlag = true;
	protected $internalFlag = true;
 
	public function getFields()
	{
		$getFields = create_function('$obj', 'return get_object_vars($obj);');
		return $getFields($this); // Returns only 'name' and 'publicFlag'
	}
}

We can also do this the PHP 5.3 way with a much nicer-looking closure:

1
2
3
4
5
6
7
8
9

class BobUser
{
	// ...
	public function getFields()
	{
		$getFields = function($obj) { return get_object_vars($obj); };
		return $getFields($this);
	}
}

“… So a blogging platform won the content management system award? How sad is that?”

My knee-jerk “how sad is that?” reaction comes not because I don’t think WordPress is worthy, but because of what it implies about the state of other open source CMS projects. The reaction comes from the fact that a blogging platform is kicking your CMS’s ass in its own category.

WordPress bridges both the blogging and CMS categories due to the ‘Pages’ feature, and is extremely useful for managing a blog-focused website. Mostly. That is, until you want to do something that a CMS should be good at, like have an event calendar, custom form, photo gallery, etc. – which is why WordPress is not focused on being a CMS in the first place. Yet it does such a better job at the basic things like creating new pages, tagging, categorizing, comments, and having custom SEO-friendly URLs out of the box that it edged out other software projects like Drupal MODx and Joomla! SilverStripe whose sole focus is content management. Sad indeed.

The fact that I can go through literally hundreds of open source content management systems and still end up settling on WordPress because I know it’s the only one that won’t totally confuse my client is what’s sad. Usability and ease of use matter. They are the number one feature to the end user. If you’re involved in a CMS project, you need to do better. A lot better. Right now.

UPDATE: Some commenters have pointed out that the awards website has a specific rule:

Previous winners of the Overall category are not eligible for the Overall category in 2009. Previous winners compete amongst one another in a separate Hall of Fame category designed specifically for them.

Since both Drupal and Joomla! have won the award previously and thus were automatically excluded, I replaced their names with the CMS projects that were 2nd and 3rd behind WordPress (MODx and SilverStripe) instead. The premise of the post still holds true.

Using the MySQL Control Flow Functions, I quickly crafted an IF() statement in the SELECT portion of the query that looked like this:

1

IF(m.date_released IS NULL OR m.date_released = '0000-00-00', 1, 0) AS in_production

So if the release date has a NULL value or has been inserted as a blank string and formatted to ‘0000-00-00′, it is given a value of 1, and 0 otherwise. We then sort descending by our new ‘in_production’ alias in the ORDER BY clause to get the NULL or empty values on top (assigned a 1 value), and then by release date and other criteria second.

The whole query looks something like this:

1
2
3

SELECT m.*, IF(m.date_released IS NULL OR m.date_released = '0000-00-00', 1, 0) AS in_production
FROM movies AS m
ORDER BY in_production DESC, m.date_released DESC

It’s an easy way to manipulate the result set in MySQL without slowing down the query or issuing multiple queries.

Object Oriented Apologetics

In defense of object-oriented programming – How and why you should use object oriented programming for your next project.This talk is for PHP programmers who are just learning about object oriented code, who cling to old excuses(“object oriented code is slower”), or who are otherwise unconvinced of its usefulness. Concrete real-world examples of commonscenarios and challenges that programmers face will be presented, and how taking an object oriented approach is better than a proceduralone in most cases. Copious code examples in both object oriented and procedural approaches will be provided throughout, and thedifferences and benefits of the object oriented approach will be explained.

Download Object Oriented Apologetics on SlideShare

If you were in the talk, please rate it on Joind.in

The CodeWorks Experience

All in all, the CodeWorks roadshow Dallas stop was much smaller than I expected. There were about 20 people in the talk I gave. I suppose it was both a good a bad thing. On one hand, I had a lot of fun connecting with the other speakers and attendees on a more personal level than I would have had the opportunity to do otherwise. I met a lot of new people in the PHP community that I will probably stay connected with on some level, even if it is just Twitter and IRC. We had a lot of fun the night before the presentation day eating together and hanging out.

On the other hand, I know that as a business, the relatively low attendance levels coupled with the high travel expenses could mean that something like this can’t happen again, which is a shame. This was one of the best efforts I have seen in a while to really lower the price of the conference for attendees by bringing the speakers directly to their cities or at least ones that are close by. Going to ZendCon, for instance, could easily cost up to $2,500 for the whole trip with airfare, hotel stays, and food, if not more.

The bottom line here is that CodeWorks was a good conference that offered a great value for your money. I had a great time meeting and connecting with new people and finally meeting people in real life that I had been communicating with for years. I learned some new things and finally got the kick in the pants I needed to jump into Test Driven Development, thanks to Jason Sweat’s TDD tutorial and hand-holding.

Now I’m just looking forward to php|tek in May 2010.

I was able to arrive at this solution by combining the one-line comment with the comment block in a way that takes advantage of the rules the different types of comments have to follow.

The One-Line Comment

One-line comment rules dictate that everything after the comment characters must be ignored for the rest of that line. They look like this:

1
2
3
4
5

// My code below
$someVar = array("foo", "bar", "blah");
echo "<code>";
print_r($someVar);
echo "</code>";

The Block Comment

Block comment rules dictate that once the beginning characters are started, everything up until the ending characters is ignored. They look like this:

1
2
3
4
5
6

/* My code below
$someVar = array("foo", "bar", "blah");
echo "<code>";
print_r($someVar);
echo "</code>";
*/

Combining The Two: The Best of Both Worlds

Using both those comment style rules, we can combine the two comment styles in a way that will allow us to comment and un-comment a block of code with by adding or deleting a single character. The trick is to make both the beginning and ending lines both single-line AND block style comments, like so:

1
2
3
4
5
6

//* My code below
$someVar = array("foo", "bar", "blah");
echo "<code>";
print_r($someVar);
echo "</code>";
// */

In the above example, you notice that the code will still run – it’s not commented out because the first line is following the rules of the single-line comment – ignoring the block comment declaration on the same line. By removing the first backslash at the beginning of the block, we can comment out the entire code block by enabling the block comment style instead of the single-line style. The last single-comment line is ignored because the block comment rules ignore everything up until the end block comment declaration at the end of the line:

1
2
3
4
5
6

/* My code below
$someVar = array("foo", "bar", "blah");
echo "<code>";
print_r($someVar);
echo "</code>";
// */

So a single character – a backslash at the very beginning of the block declaration can now comment or un-comment the entire block by switching the comment rules back and forth between single-line and block style. The code examples provided are in PHP, but this trick will work with any language that supports both single-line and block style comments.

I will be presenting my talk on Building a Data Mapper with PHP5 and the Standard PHP Library, followed by a discussion on ORMs and whatever else comes up. The presentation will cover all the thought processes, goals, theories, and actual code that goes into building an ORM (or really any other larger project that requires more advance planning). The project that was the basis of this presentation is phpDataMapper – an open-source PHP5 data mapper ORM layer that I started in the fall of 2008. It now powers the model layer of InvoiceMore, a live web application I launched in March 2009.

This is a presentation I have given before at Tulsa TechFest.

Watch the video of this presentation online

1
2
3
4
5
6
7

<script type="text/javascript">
$(function(){
	$('input.calendarSelectDate').live('click', function() {
		$(this).datepicker({showOn:'focus'}).focus();
	});
});
</script>

You would think just a simple “$(this).datepicker()” call wrapped inside the live() event would work, but it doesn’t. Turns out that in order to get it working consistently, you have to add the ’showOn: focus’ config option as well as manually focusing on the element with the focus() event. Charming.