boringguys.com » PHP

Development Environment layout using Linux, Apache, PHP, and Subversion

2009-07-08T10:13:49Z

Some of the age old questions I face lately are:

What’s for dinner?

Should I accept that friend request on Facebook for the friend of a friend of a friend that I knew 15 years ago?
What’s the best development and test environment layouts for PHP using Apache as a web server with Subversion for version control for multiple developers?

Some of you may be asking yourselves the same questions. The choice of dinner is a personal one. I won’t go into that except to say that everyone loves a good burrito. Spicy! And you probably don’t care about my take on Facebook etiquette since your friends list probably dwarfs mine.

But I do have some definite thoughts on the layout of development environments. And I find that there’s a huge lack of information about this on the interweb, so here you go.

We use Linux, Apache, PHP, and subversion in our development environment and so these instructions will be biased towards these topics but I think you can apply this method using various other technologies.

I like to give each developer their own development web site and development database. I find it’s easier for everyone to have their own individual sandbox to play in. We give them each their own domain using their initials, something like rzdev.domain.com for me and vbdev.domain.com for another developer, Vinny Bag-o-Donuts. We set the Apache directories up on the Linux dev box in a similar fashion:

/var/www/rzdev.domain.com/
/var/www/vbdev.domain.com/

This has a few benefits. If I need to show Vinny something with my site development, I can just send him the link to http://rzdev.domain.com/broken-page. I can make changes to code, even major infrastructure code and not break anything for the other developers. We do the same thing with the databases, prefacing them with our initials.

Now, since our dev boxes use Linux, we set up Samba for sharing on these web directories. This means that all the devs can edit files and use source code management on the Linux server itself or on their Windows machines (we use either Eclipse or Zend Studio and create projects on the shares, that’s a whole different posting!).

This dev site layout is closely linked to the way we use Subversion for version control. When we start a new site or application, if we can split out the development evenly enough, we’ll just have everyone work from the trunk version of the code, with each developer working on their own little section. Each developer puts the trunk in their Apache dir and we edit the Apache configs to reflect this:

/var/www/rzdev.domain.com/trunk/
/var/www/vbdev.domain.com/trunk/

The root of the dev sites typically look like this:

/var/www/rzdev.domain.com/trunk/docs ( your Apache document root )
/var/www/rzdev.domain.com/trunk/lib ( non-public PHP library code )

When we commit code changes in Subversion, we have a hook that updates our main development site here:

/var/www/dev.domain.com/trunk

Again, that site can be seen on the web at http://dev.domain.com/ This way, we can do integration testing on our code to make sure our new code doesn’t break code from someone else within the dev site.

Now, the important thing here is that the Quality Assurance (QA) and Testing people ( if you’re lucky enough to have them ), don’t use any of these previously mentioned sites for their testing. Why not? Well, because if they’re doing a good job and are therefore sufficiently anal, they’re going to complain when code is changing on the site they’re looking at.

So we give them their own test site and database that’s viewed on the web at http://test.domain.com/ and setup in apache at:

/var/www/test.domain.com/trunk/docs
/var/www/test.domain.com/trunk/lib

The developers will meet and create the list of files and database changes that get moved over to the test site. How the actual moving is done doesn’t really matter. If you have the time and energy to set up some Ant or Phing tasks, that works great. But copying/rsyncing files and running some SQL on the test database works just as well. The most important part is that the developers meet to decide which part can go to test. Otherwise, you could have code going to test and eventually production that might not be fully vetted.

When QA finds bugs in test.domain.com, they can send them to the developers. The developers can instantly start working on fixing the bugs in their own dev space at rzdev.domain.com and not affect the other developers or the ongoing testing of the application. Pretty nice right?

Advantages of this approach

Uses source code management
Developers can unit test their own code
Developers can do integration testing between each other’s code
Developer A typically doesn’t destroy code or data that developer B is using
Developers don’t destroy code or data that QA/testing is looking at
Developers can both edit files and use source code management in either Linux or Windows environment
Very scalable. Adding new developers into the mix is as simple as adding their respective sub domains and databases (of course, this can also be viewed as a disadvantage, see below)
Less bugs make it to production

Disadvantages

Lots of sysadmin overhead initially and with each subsequent domain added. You have to set up all those developer sites, rzdev, vbdev, etc. Same overhead when using branching within subversion. Plus, you have to setup all those databases and setup the config code to connect to the appropriate database for each developer domain.
Lots of file space for all the sites and databases.
Confusing for lone wolf and gunslinger developers who are used to overwriting production or each other’s development code (too bad for them!)

So what do you think? How do you setup YOUR PHP development environment?

Share this via del.icio.us, digg, email, etc.

Mysql and PHP help Obama become President of the United States

2009-07-08T10:14:17Z

We have our answer to my earlier question about technology used by Obama and McCain. One small step for LAMP, one giant leap for LAMP-kind.

Share this via del.icio.us, digg, email, etc.

Obama and McCain website technology

2008-08-02T12:53:59Z

Hey all, I just realized that johnmccain.com is running MS Windows and .NET while barackobama.com is running LAMP with PHP. May the best tech, errr, candidate win.

Also, Obama needs a PHP contractor in Boston, Ma.

Share this via del.icio.us, digg, email, etc.

Beware the Lone Wolf PHP Developer

2008-08-02T12:54:08Z

With all the posts on interviewing PHP candidates popping up lately, I thought I’d post this draft that I’ve been sitting on for awhile that’s related to new jobs and interviewing candidates.

I’ve run into the Lone Wolf PHP Developer at several places I have done work. Sometimes, I’ve had to work side-by-side with the Lone Wolf. Other times, I’ve replaced the Lone Wolf who had moved on to different hunting grounds. Still other times I’ve had to hire people and had to choose between a Lone Wolf and several other candidates.

Just who is this Lone Wolf and why should we fear them so much? Here are some telltale signs of the Lone Wolf PHP Developer:

The Lone Wolf doesn’t understand how to work in a team of developers. They typically don’t even understand what benefits that would create. They do all development on their own, listening to very little input from qualified sources.
The Lone Wolf got to page 141 on Enter-A-Beginner-PHP-Book-Title-Here and no further. Objects? They’ve never heard of them. They must not need them.
The Lone Wolf re-invents the wheel for every project and doesn’t use standard tools and practices.
The Lone Wolf eats their young. OK, I made that one up. Frankly, how could a Lone Wolf have young anyway?
The Lone Wolf is perfectly satisfied with doing programming work on production servers and using FTP to deploy their code. It never occurs to them that they should strive to create development and test environments. It never occurs to them that deployment via FTP doesn’t scale higher then one developer.
The Lone Wolf doesn’t know what the letters SCM, CVS, or SVN are, or how to use them in their daily work.
The Lone Wolf never reads my blog or any other blogs on programming. The Lone Wolf may not know what a blog is.
The Lone Wolf says crazy things like “MySQL can’t do transactions” and somehow gets management to believe them.
The Lone Wolf was initially adored by management because they launched a lot of code live during their short stay. Too bad all that code is buggy and completely un-maintainable moving forward. Management doesn’t like that.
The Lone Wolf whips up incredibly stupid and unnecessarily complex solutions like template systems in which the templates are stored in a database instead of the file system/memory/cache. They shun using tried and true templating methods like PHP files, XSLT, or at the very least, Smarty. (see also: reinventing the wheel)
The Lone Wolf names variables after themselves that mean nothing to anyone else (ex, $lonewolfFlag )
And worst of all… The Lone Wolf PHP Developer fails to realize that there are other developers out there in the business world, trying to earn a living just like they are. By failing to conform to development standards that have been proven and tested, they make everyone’s job more difficult.

Now, I personally have run into this Lone Wolf scenario with other programming languages as well. But I think because it is so simple to work with PHP without much formal training that it lends itself to this problem much more readily then other languages. The blessing and curse of PHP is that it is the new VB 6.

Share this via del.icio.us, digg, email, etc.

PHP Developer Jobs are the Hottest EVER!!!

2008-08-02T12:54:18Z

** I’ll preface this by saying that I’m not specifically looking for a new position but I think it is important to be aware of the market that you serve. Even in happy times during my employment, I routinely scan job ads to see the health of the market and to gauge its direction. **

If anecdotal evidence means anything, and sometimes it does… and at the risk of sounding like a teenage girl writing on a myspace page, the PHP job market is literally blasting off! I’ve never seen the market for PHP programmers as strong as it right now in the NYC/Philadelphia region.

While there are lots of PHP jobs in the usual suspects like small start-ups, there also seem to be a number of jobs at established companies and developer shops. I’m also seeing a definite split in PHP jobs where there are both entry level jobs as well as jobs for more experienced people with titles like “Senior PHP Developer” and “PHP Tech Lead.” I can’t remember seeing so many senior PHP positions. The salaries I’m seeing are also at an all time high. Many ads have salary ranges that extend well into 6 figures.

As far as requirements I have seen, there definitely seems to be a shift towards frameworks like Symfony and Zend Framework as well as items specifically mentioning IDEs like Eclipse and Zend Studio. However, I rarely see mention of source code management and unit testing. Hopefully those items were cut from the job ads due to space requirements.

If you are a PHP developer in the area and are highly experienced, get ready to reap the rewards for your hard work. If you are a hiring manager in the area, prepare to shell out more for top talent.

So what is everyone else seeing in their area with regards to PHP developer jobs?

Share this via del.icio.us, digg, email, etc.

First impressions on Google App Engine

2008-08-02T12:54:35Z

From the Google site:

Google App Engine lets you run your web applications on Google’s infrastructure. App Engine applications are easy to build, easy to maintain, and easy to scale as your traffic and data storage needs grow. With App Engine, there are no servers to maintain: You just upload your application, and it’s ready to serve your users.

You can serve your app using a free domain name on the appspot.com domain, or use Google Apps to serve it from your own domain. You can share your application with the world, or limit access to members of your organization.

Without actually firing up the SDK yet, here are my initial thoughts:

Do we trust Google? With our data? With our users? I love the idea of the Google App Engine (GAE), a scalable web app system where you only have to worry about coding your business logic and structuring your models. This is kind of like Amazon’s EC2 on steroids. No sysadmin stuff like with EC2, just coding. However I do not love that Google is already in the web app space themselves and is now marketing a web app space hosting platform. You have to trust them a little bit more then I’m comfortable with. They’ve already got your users and your data and your code. Don’t they conceivably control your business at that point? Maybe I’m being paranoid here.Plus, there’s the lock-in factor. If I develop my app using MySQL as a backend, I know I could with only some minor pain change that backend to Oracle. It’s all SQL at some point. How would you swap out of using the Google DataStore API? It’s a proprietary system with no published standards.
GAE is currently only in Python. No PHP love? I realize a lot of Google runs on Python and C++ and that’s what it is in their wheelhouse. But I still consider Python to be a fringe language. If GAE were opened up to support PHP and C#, it would blast off in popularity. Then again, a Python only crowd is a good beta test before the crush of PHP devs comes in.
The DataStore API (ie, BigTable) is difficult to wrap my head around. After spending so many years carefully crafting db tables and relationships to get the most bang for my buck, I now have to throw out a lot of that hard-earned knowledge. I need to think in terms of objects (or columns) instead of rows.There are still relationships and keys to keep track of but not in the traditional ways of SQL. It seems like much more of your data fetching is done in code rather then in SQL. The shift in thinking reminds me of my own move to Object-Oriented Programming. I whined like a baby during that phase of my programming development (well, at least I did on the inside).

Hopefully I’ll get some more time to play with this stuff soon and report back.

Share this via del.icio.us, digg, email, etc.

wordpress 2.5 and subversion

2008-08-02T12:54:43Z

For some reason, performing an svn update on the external repository http://svn.automattic.com/wordpress/trunk/ does absolutely nothing. This is pretty baffling behavior. Same wordpress and subversion issue also mentioned here.

I’ve also tried /tags/2.5 and /branches/2.5 and nothing. No changes come across. Very weird. Anyone else getting this?

Share this via del.icio.us, digg, email, etc.

Symfony sfException Call to undefined method BaseModel::__toString

2008-08-02T12:55:10Z

Working on some projects in Symfony recently. I ran into a problem when I used the symfony admin generator to create a "backend" module for a table that has a foreign key relationship.

Symfony was looking for me to have a __toString() function in my model so it could create the drop down menu. So for instance, let's say you have tables like countries and states in your schema.xml. Country_id in the states table is a foreign key to the Countries table like so:

PLAIN TEXT

XML:

name="states" idmethod="native"> name="id" type="SMALLINT" required="true" autoincrement="true" primarykey="true">
> name="country_id" type="SMALLINT" required="true">
foreigntable="countries">
local="country_id" foreign="id">
>
>
> name="name" type="VARCHAR" size="40">
> name="abbrev" type="VARCHAR" size="2">
>>

When you do:

PLAIN TEXT

CODE:

symfony propel-init-admin backend states States

Symfony is going to create the edit and create actions of the States screens with a drop-down menu of countries that the state is a part of. The value of the country drop down is the country_id. the value needs to be set in a __toString() function in your Country propel model. The easiest thing is to just return a simple string name (if that's in your db model):

PLAIN TEXT

PHP:

function __toString()
{
return $this->name;
}

That should solve the problem. It's odd that this comes up. And I couldn't find anything on it when googling.

Share this via del.icio.us, digg, email, etc.

Drupal 6 and new Apress Drupal book

2008-08-02T12:55:18Z

I have been working on a couple of projects using Drupal lately that required creating some custom modules. So again, I flipped thru my well-worn copy of Pro Drupal Development. Here is my earlier review of Pro Drupal Development.

Drupal has released version 6 now. So I click to the Apress site and lo and behold, they're already working on Pro Drupal Development, 2nd edition which will feature updates for Drupal 6. If it is as good as the first book, it will be a must have for any Drupal developer.

Share this via del.icio.us, digg, email, etc.

Using Subversion externals property for WordPress upgrades

2008-08-02T12:55:28Z

I find that upgrading apps like WordPress, Drupal, Symfony and open source PHP apps is simple for less complicated environments, but once you start adding in things like new directories, custom themes/modules, source code control as well as separate development, test, and production systems, the upgrades start to get pretty hairy.

Take WordPress for example. A typical upgrade of WordPress involves copying the new WordPress files over your existing files. Then you have to copy back safe versions of things like wp-config.php, .htaccess (if you're using it), as well as any custom themes/modules from the wp-content/ directory. Not to mention any of your own directories that should exist alongside your wp-includes/ and wp-content/ directories. After that you can run the upgrade.php file.

These upgrade steps aren't terrible. They're quite a bit better then most open source apps out there but they still suffer from a few problems:

If you have your code, including your WordPress install, in Subversion or another source code control system, you have to commit all the files that change with each WordPress version. There may be files added, deleted, etc. You'll have to keep combing thru "svn status" messages to figure out everything you need to do to get all the WordPress files into your repository. This can be painful. And take a long time.
WordPress is specifically written so that you don't ever have to muck with the guts of it. You create themes and plugins for added functionality. So, since you're not maintaining the code that powers WordPress, do you really need all those deltas in your Subversion repository? I think not.
What do you do with your own code in directories that sits alongside wp-includes/? What if it’s in a Subversion repo?

The WordPress site also has instructions for using Subversion with your site. Here, they advocate the use of “svn switch” to update your site. This is much more manageable and solves a few of the above problems. Most svn users can probably can get away with this method. But unfortunately not me.

I have additional directories on some of my sites that I need to add into my WordPress install. So I have to copy/move them into the WordPress dirs which gets tough. And then my “svn status” will get all wonky because my WordPress dir is under one repo and my code is under another. This was endlessly confusing for me.

So I found myself looking for a way to completely wall off my WordPress install from the rest of my files. I was reminded recently of the use of the Subversion externals property and my mind started buzzing with possibilities. With "externals," I can say:

Pull the stable WordPress code from WordPress.org and put it into this directory named /docs/wp/

Then my other directories, which are under my own local subversion repo can exist at /docs/dir1, /docs/dir2, etc. Of course, some Apache Alias magic is needed to make all this work.

Here's the way I set it up for my some of my projects. So far so good. This is a bit hairy to set up but subsequent upgrades are a breeze. I use this across development, testing, and production systems (how to get those environments to work with WordPress will be another entry)

First off, the previous Apache document root for domain1 was at /www/domain1/docs, so the WordPress files wound up like this:

/www/domain1/docs/wp-content/
/www/domain1/docs/wp-includes/

But I also have a lot of dirs that sit alongside of wordpress like this:

/www/domain1/docs/dir1
/www/domain1/docs/dir2

We're going to wind up changing that.

Create an subversion external property in /www/domain1/docs for WordPress

PLAIN TEXT

CODE:

$$ cd /www/domain1/docs
$$ export SVN_EDITOR=vi ( or your editor of choice )
$$ svn propedit svn:externals .

vi starts up and you can add the following line:

PLAIN TEXT

CODE:

wp http://svn.automattic.com/WordPress/tags/2.3.3

Save and exit

PLAIN TEXT

CODE:

$$ svn commit
$$ svn update

( This downloads the WordPress code from the above address into your wp/ directory. Now we are cooking. )

Now, /www/domain1/docs/wp is where all your WordPress code lives.
Copy wp-config.php to /www/domain1/docs/wp/
Copy .htaccess to /www/domain1/docs/wp/
Create a link from the stock wp-content/ dir to your personal wp-content dir like this

PLAIN TEXT

CODE:

$$ cd /www/domain1/docs/wp
$$ rm -Rf wp-content/ ( use -Rf with care please! )
$$ ln -s /www/domain1/docs/wp-content wp-content

In Apache config, set document root for this domain to /www/domain1/docs/wp
Put wp-content/ dir and any other non-WordPress dirs/files into /www/domain1/docs
Create an alias for wp-content/ and any other non-WordPress dirs/files in Apache config

PLAIN TEXT

CODE:

Alias /wp-content /www/domain1/docs/wp-content
Alias /dir1 /www/domain1/docs/dir1
Alias /dir2 /www/domain1/docs/dir2

This looks like a lot of work, but it's really only a lot the first time around. Next time WordPress has an upgrade:

PLAIN TEXT

CODE:

$$ cd /www/domain1/docs
$$ svn propedit svn:externals . ( change the tag to new version of WordPress )
$$ cd wp/
$$ rm wp-content (to remove the link)
$$ svn update (to update to new version of WordPress)
$$ rm -Rf wp-content/
$$ ln -s /www/domain1/docs/wp-content wp-content

Everything after the propedit in this group can and should be scripted which will basically give you a 2 step process for upgrading WordPress, while keeping you wp-content/ dir under local source code control, as well as leaving room for any other directories or files your site might require.

This technique will probably also work with Symfony although I haven’t tried it yet.

Share this via del.icio.us, digg, email, etc.