Andrew Bredow

Happy New Year!

2012-01-08T00:00:00+00:00

Happy (slightly late) New Year! I’m finally getting back into things after taking a little more than a week to spend with family and relax, followed by an awesome retreat with the Table Project team as well as the rest of the staff of other organizations that are part of our parent company. It was a good time to unplug and re-focus. As we do every year, Carrie and I had took time at the end of the year to evaluate our goals and set new goals for the new year. I’ve also spent time over the last week thinking through a few more detailed professional goals for myself this year, and I thought I’d take a minute to share them.

Read less technology news - I have always spent a large amount of time keeping up with what is happening in my field. I read Hacker News regularly, and on a less regular basis I keep up with about 30 or so technology blogs that are relevant to the web. Historically, I think this has been a good practice to get an idea of what other developers find useful, and also to benefit from their collective wisdom. However, I think I need to take a step back.

It feels like there has been a huge increase in negativity lately. I love it when developers and other technology folks share their wisdom in a constructive and helpful manner, but it becomes a huge distraction when we waste so much energy attacking other perspectives over minute differences (e.g. CoffeeScript vs. Javascript, SCSS vs. SASS). Generally, there are many roughly equivalent tools to complete the job, and it doesn’t matter which one you are using so long as you are productive, responsible, and enjoy what you are doing.

I don’t plan to unplug from tech news altogether, but I will not consume it more than every couple of days. There is still much to learn, but I’d rather spend less time in the drama, and more time creating cool things.
Support awesome open source projects - I benefit from so much awesome open source software on a daily basis. Much of it is created from out of the generosity of others with their time and talents. Unfortunately, I have been able to cut out enough time to contribute to existing open source projects. However, I’d like to make this the year that I start giving back. I’m not sure what form this will take yet, but I do have a few things in mind. I’d love to contribute patches for bugs, even if they aren’t directly affecting me. I’d also love to take the time to seek out and thank individuals who spend so much time working on awesome projects.
Add new tools to my arsenal - While the previous point may be a little vague for a New Year’s resolution, I do have a few very specific targets for learning this year.

First, I’m working my way though Seven Languages in Seven Weeks. My goal is not necessarily to develop a strong command of every language in the book, but I would like to develop a better understanding of some new coding paradigms, and become familiar some lass mainstream languages that could be useful in specific applications.

Next, I plan to spend a lot more time in node.js this year, both professionally, and on my own time. I really like Javascript, and I think it has huge potential as a sever side language in the future.

Finally, I plan to develop my understanding of non-relational data stores. I’m particularly interested in mongoDB and Redis. I think I have about enough knowledge to be dangerous with these tools at this point! However, I need to take the time learn the last little bit (particularly around backups and long term scaling) before I can confidently use them in a live application. I’ll definitely be exploring this topic more in a future blog post.
Release More Code - With the exception of my marginally successful Library Detector browser extension, I have never been a huge generator of open source software. This is rather hypocritical given my previously stated appreciation of open source software. Over the past years I have written quite a bit of code that could be useful to others if I can take a bit of time to polish the documentation and clean up the APIs a bit. My goal is to actually make this happen over the next year!

Well that pretty much sums it up. I hope everyone has a safe year with lots of productivity and growth!

Will You Help With My Wife's Dissertation?

2011-12-09T00:00:00+00:00

My wife, Carrie, is currently working on her Ph.D. at the University of Texas at Austin. As part of her data collection, she is running an online survey. I’d really appreciate it if you would be willing to help by either taking the survey, or by passing it on to your friends.

What is it?

The survey is about people’s dating behaviors and relationships. In particular, she is looking for people who are currently unmarried (e.g. single, divorced, engaged - just not married).

How can you help

There are 2 ways you can help with this.

1) Pass the survey on to your single friends or family who may be willing to help out with a research project (or would at least be motivated by the prospect of a free gift card!). Feel free to either link them to this post, or invite people via email, Twitter, or Facebook. Here is some ready-to-copy text that would work well on Twitter or Facebook:

Not married? Take a survey to help out a grad student, and you could win one of 15 $40 Amazon Gift Cards! http://datingsurvey.bredowresearch.com

2) Take the survey yourself! If you match the demographics she is looking for, please take the survey. I’d really appreciate it.

What’s in it for you?

If you are planning to take the survey yourself, you could win one of 15 $40 Amazon gift cards. There is a form to fill out at the end of the survey to enter yourself in the drawing.

If you are willing to pass this along to your friends, I’ll totally buy you a beer the next time I see you! Moreover, you would win my undying appreciation!

To take the survey, just go here: http://datingsurvey.bredowresearch.com.

Thanks so much!

A new beginning at the Table Project

2011-10-18T00:00:00+00:00

I am now a full time employee at the Table Project. This month has been a gradual transition from my previous job at Live Oak, and since I only have a few things left to wrap up, I think it is time to finally make this public. Interestingly, it feels like I have been working on this project for a long time since I actually made some of the first commits to the repository back in early 2009 when they were a client! Having the opportunity to join back up with the awesome people behind this effort has been at the back of my mind for a long time, and I am happy that this day has come!

The Table Project fits in well with my beliefs of how Christian community should function, and is also a hugely interesting on a technical level. I’d love to take a few moments to dive into this a little more!

The great reformers of the 16th century worked tirelessly to move the church to a decentralized model: one that empowered individuals to learn and grow in relationship with God and one another without any intermediaries. However, as we can see even today, this is very hard to live out in practice. We tend to create bottlenecks that make it difficult to maintain good communication and learn who needs help in our communities. I believe that technology like the Table can help us overcome these barriers to better support and serve another by flattening our interactions. It is is exactly the kind of change that I am humbled to have the opportunity to foster with the code I write.

Technologically, the Table application and its supporting infrastructure is a vast landscape of challenging issues - and it’s only going to get more interesting as we continue to grow! Up to this point in my career, I have mostly partnered with clients to help them get their initial, 1.0 product out the door. I have learned a ton, but I am ready to apply my skills in some new areas and to pick up some new skills along the way.

I hope this is just the beginning of a long journey that leads to lots of interesting information and inspiring stories to share.

Back in the Saddle!

2011-10-02T00:00:00+00:00

OK, things have been a bit dead around here lately (where lately = “for about the last 2 years”). A lot has changed in that time. In short, I have moved, traveled quite a bit, completed several sizable projects, become a father (twice), adjusted to parenthood, and I am about to start a new job. Phew.

Things are tiring at times, and I sometimes feel like my life is a delicate balancing act. That being said, I miss blogging (even though my previous tenure was rather short). Twitter has been great in the meantime, but there are so many things that just can’t be expressed in 140 characters. Not to mention, it is very difficult to feed my habit of overusing punctuation (especially parentheses) in that space! So, I have decided to “relaunch” my website, finally leaving Wordpress behind for greener pastures.

If you used to keep up with my old blog, you should still be subscribed, thanks to Feedburner. If we have gotten to know one another in the last couple of years, please consider subscribing! I’ll do my best to keep interesting things rolling in.

Compiling PHP on Snow Leopard

2009-08-29T18:13:40+00:00

UPDATE (Oct, 2010): These issues were resolved. You can now compile PHP 5.2 OR PHP 5.3 without following these steps

I have been wanting to do a clean install on my laptop for the last couple of months and decided that I would just wait for 10.6 to avoid having too much downtime from work. I like to compile my dev stack from source, and as expected this turned out to be the only problematic area for my upgrade. As a whole the transition was very smooth - both MySQL and Apache compiled without a hitch. However, I had a couple of issues with PHP (5.2) that took a little more work (nothing some Google fu couldn’t solve).

Both of these problems are related to shared libraries that PHP relies on:

libresolv

Running make resulted in the following error:

Undefined symbols:
  "_res_9_dn_expand", referenced from:
      _zif_dns_get_mx in dns.o
  "_res_9_search", referenced from:
      _zif_dns_get_mx in dns.o
      _zif_dns_check_record in dns.o
  "_res_9_dn_skipname", referenced from:
      _zif_dns_get_mx in dns.o
      _zif_dns_get_mx in dns.o
ld: symbol(s) not found
collect2: ld returned 1 exit status
make: *** [libs/libphp5.bundle] Error 1

Thanks to a great StackOverflow thread I was able to get around this fairly easily by editing 2 lines in the Makefile after running the configure script:

EXTRA_LIBS = -lresolv [... all of the other flags that are already here]
EXTRA_LDFLAGS = -lresolv [... all of the other flags that are already here]

libiconv

An error was also thrown in reference to libiconv. Apparently there is something strange with Apple’s included libiconv libraries. I was pretty easily able to get around this by compiling a local copy of libiconv from the source at gnu.org:

# download source, extract, enter directory
./configure --prefix=/usr/local
make
sudo make install

Then, just add –with-iconv=/usr/local to your configure options so PHP will link to your copy instead of the Leopard one.

Final configure command:

./configure --prefix=/usr/local/php52 --with-apxs2=/usr/local/apache2/bin/apxs\
 --enable-shared=all --with-zlib --with-curl --with-freetype-dir=/usr/local\
 --with-jpeg-dir=/usr/local/lib --with-png-dir=/usr/local --with-gd\
 --with-mcrypt --with-mysql=/usr/local/mysql\
 --with-mysqli=/usr/local/mysql/bin/mysql_config\
 --with-pdo-mysql=/usr/local/mysql/bin/mysql_config --with-iconv=/usr/local

Note: I did find that these issues have been resolved in the PHP 5.3.1 branch and will most likely be backported to 5.2 at some point, so make sure to check future versions before taking this route.

Hopefully this helps someone else out until these issues are resolved by either the PHP team or Apple. Do let me know if I have missed something stupid here or if there is an easier way to resolve these issues.

Pipe text from terminal to clipboard

2009-05-14T00:00:00+00:00

Here is a cool little OS X tip that I stumbled upon a couple of weeks ago. I was writing documentation for the setup of one of our applications and I found myself copying the contents of configuration files into the wiki manually. If you have done this before, you know that it can be pretty annoying when you are working from the terminal and you end up cutting off the ends of lines because you don’t realize that they are chopped off in nano.

However, thanks to Stack Overflow, I was able to find a very elegant solution for this situation when you are working from a Mac client: pbcopy (and the yen to it’s yang, pbpaste). These 2 little programs allow you to pipe text to and from your clipboard into your terminal. For example, pretend I have a configuration file that I need to copy into a blog post or wiki:

$ cat localhost.conf | pbcopy

Now I can just use a standard paste command to insert the contents of the file into any input field. No need to open the file in an editor: easy. The inverse is also possible. Pretend you have copied a code snippet from an online tutorial to your clipboard and you need to create a file with the copied text as the content.

$ pbpaste > somefile.txt

I am sure that there are some other cool uses for these commands, so put your ideas into the comments!

A couple of site changes

2009-05-07T00:00:00+00:00

There are many reasons to have a personal blog. The most obvious (and frequent) reason is to have a place to keep your friends up-to-date with what you are doing. However, blogs are also a great tool for allowing potential employers or clients to know what you are passionate about and how you think. While I am not currently looking for a change in either of those areas, I decided that it is important to lay the groundwork now.

To begin with, I have finally moved this blog to a permanent home at andrewbredow.com instead of a subdomain to a site that I have no intention of developing any time soon. Any old links to site will be redirected here with 301 (permanent) redirect for the time being, and I will probably get rid of the subdomain altogether in a couple of months.

Additionally, I have made the switch over to Feedburner (which is now owned by Google) for my RSS feed. This will provide me with some better metrics and additional exposure and it also allows the possibility of switching out blogging software while keeping the same feed address. If you are subscribed to the old feed it will continue to work as long as the redirect is in place. However, it would be great if you re-subscribed using the Feedburner feed!

That’s all for now; there are more exciting things coming soon!

Using multiple Apache virtual hosts on a development machine

2009-05-04T00:00:00+00:00

The problem

When I first developing web applications I was only responsible for one site. My typical workflow was to have a checkout of the application in my document root and to just use http://localhost to debug it before I checked in my changes. As I started working on other projects, I would just go into my Apache configuration and change my document root to another directory. This got rather annoying after a while since I had to edit files and restart Apache every time I switched gears.

Then the light bulb clicked on: I knew how to configure virtual hosts to run multiple sites on web servers, why didn’t I just do the same thing on my local machine? Once this is accomplished, it becomes very painless to switch contexts between multiple applications. As an added bonus it also makes it very easy to install open source web applications on your local machine, do some hacking, and learn how they work. While this same thing is technically possible just by adding new folders to your document root, this can really confuse some web applications that are expecting to handle all of their URLs as directly relative to your base path.

Prerequisites

The only prerequisite to this tutorial is that you have a relatively clean install of Apache on your local machine. You can use the default OS X install, or also also a vanilla copy installed via the Windows Apache installer. I am going to give instructions for a *NIX OS, but this setup also works fine in Windows.

Set up

First, you will want to locate your Apache configuration directory (containing http.conf). On OS X or most other flavors of UNIX it is /etc/apache2/, or Windows: C:/apache2/conf. Next, open your terminal and cd into the directory: $ cd /usr/local/apache2/conf
Add a “sites” subdirectory to your Apache configuration directory. This is the directory that will hold all of your virtual host configuration files. $ mkdir sites
Locate your httpd.conf file, open it up for editing $ nano httpd.conf
Now, we want to edit a couple of configuration settings. You may find variations of these already in the file that you can either delete or modify.
```
NameVirtualHost *:80
Include conf/sites
```
The first line tells Apache to use name-based virtual hosts. This means that it will match virtual hosts based on the domain name of the request. The second line causes all files in the sites directory to be included when Apache reads in it’s configuration files. Note that this folder location is relative to your ServerRoot location which you will find elsewhere in httpd.conf if you search for it.
Next, you are going to go into your “sites” folder and create your first virtual host configuration file. It is helpful to name the files based on the domain that they reference (I am going to create http://local.andrew).
```
$ cd sites
$ nano local.andrew.conf
```
The basic file contents are

<VirtualHost *:80>
  ServerName local.andrew
  DocumentRoot /Users/andrew/Sites/somecoolsite
  <Directory /Users/andrew/Sites/somecoolsite>
    Options +FollowSymLinks
    AllowOverride All
  </Directory>
</VirtualHost>

This is fairly self explanatory. In the first line you set the domain that you want to use with the ServerName directive. The document root of your files (they can be anywhere on your computer) goes after the DocumentRoot directive. Finally, the Directory block is where you can put any settings that are specific to this host (e.g. Rewrite rules).

Next, you will need to create an entry in your host file to direct all traffic sent to your custom domain to your local machine. On any *NIX system the file is /etc/hosts, and on Windows it is C:\WINDOWS\system32\drivers\etc\hosts. The format is the same on all systems, just add a line like: 127.0.0.1 local.andrew
You are all done setting up your first virtual host. Restart Apache (you will probably need to use a hard restart - apachectl restart - for your vhost settings to be applied), and direct your browser to whatever hostname you created. You should see your site right away!

Now that you have 1 host set up it is super easy to add more. Just copy your existing host file to a new name and edit the settings accordingly! This should greatly improve your workflow and help you be a more productive ninja developer.

What I've been up to

2009-04-29T00:00:00+00:00

Things have been a little crazy for the last couple of months. When I first started blogging in January, I had a big list of topics that I wanted to write about. However, in reality, I have been working on too many things to have time to develop and write any of these longer posts, so I think I am going to try to step back a bit and focus a little more on what I am working on and also try to bring a little more of a personal presence into this blog by sharing some of my musings and interests outside of technology.

My day job at Live Oak is has been great lately. I am continually reminded of how blessed I am to have a great job where I am surrounded by such smart people! On the project front, I have spent the last few months planning and beginning development on a large social networking application. I am not sure how much I can actually talk it about on here, but I hope to fill you all in as soon as possible. However, I will say that this is a unique project for me not only because it is technologically awesome and challenging, but also because I am excited about the impact that it will make when it launches. For me, this is the height of career satisfaction: being able to use my abilities and intellect for a cause that I believe in.

Outside of work, I am continuing to work on SailboatConnection.com, a project that has been in the works since last summer. While it got off to a bit of a slow start, the pace has really picked up over the last couple of months, and we are nearing the first milestone that will accessed by select public users. This project has been really fun mostly because it is the first project that I started using Zend Framework (including the MVC stack) from the beginning. I will share more details about this project as soon as it is available to the public.

On the personal front, things at home have been good. Carrie is nearing the end of her last semester with classes! I am really looking forward to spending more time with her this summer, even though she has to start work on her comps for her dissertation (boo). We don’t have tangible plans yet, but I am guessing that we will make it up to Michigan to see family and friends at some point, in addition to branching out and seeing more of Texas than just Austin and the hill country (please leave any suggestions on the comments!)

Scripting successful migrations

2009-03-03T00:00:00+00:00

One of the first projects I inherited at Live Oak was a rather large custom CMS implementation with several modules for managing users and organizations in the nonprofit sector. The first iteration had some issues (I won’t get into them in this post, but they will most certainly be the topic of future posts on this blog), and for the last few months, I have been performing a massive overhaul of the data model and several of the frontend processes. It has been great to rip out some of the more obscure code and refactor the rough parts of the application.

However, there were some complications as well. New users have been registering and entering all kinds of valuable information for the last 7 months, and much of this data needed to be moved around to new tables and models. This presented some interesting challenges during development because the changes had to all be implemented in a one time application upgrade, eliminating the possibility of gradually moving the information into the new data model.

Thankfully, the upgrade went very smoothly, so I thought I would take a bit and summarize some of the principles that I utilized to make this happen. This should generalize pretty well to any web application that is heavily data-centric and needs to undergo significant model/schema changes with minimal downtime.

Automate as much as possible - For small changes to an application, it is acceptable to just keep a running list of steps you have completed in your development environment and then perform these in your live environment when you roll out your changes. However, this is hugely prone to human error, and cannot be tested with great reliability. Maintain a sql file with all of your schema changes, and begin generating a script to perform all data migration operations as early as possible in your development cycle. Ideally, these will encompass all of your data migration (with the possible exception of filesystem changes). You will be most in-tune with the changes that need to happen at the time of implementation - not right before you are ready to go live!
Use your data models in your migration script - When I originally started working on my migration script for this project I thought I would write an old-school procedural “quick and dirty” script to just shuffle the data around. But as I thought about it, I realized that I was ignoring the dependencies that existed within the database. For example, whenever a new user is created, it is not as simple as just inserting a row in the users table. I needed to create a user group, the associated group memberships, assign permissions, etc… Of course, all of this already happens inside of the data models in the application. So instead of going in the back door, have your migration script utilize your application’s data models so you can rely on code that you have already verified (hopefully!) to follow your business rules.
Simulate your migration, and then do it again! - Creating a comprehensive migration script during the development phase of your application affords you the awesome opportunity to be able to do some simulations before you actually run your script in the live environment. This is pretty simple. Do a dump of the database from your live environment and import it into your testing database (if your database is too large for this, just choose a representative sample). Now you can run your migration script until you feel comfortable that you are covering all of the different scenarios that exist in your live database. Once you have fixed an issue in the script, just re-import the data from your database dump and try again. I had tested my migration script like crazy before I started simulations, and I still found quite a few issues that only turned up when I encountered the live data that had been entered by real users and not programmers. Once you have run the simulation several times with success, you are ready to think about deploying your application.
Stage your application in the live environment - I know that it is not possible for everybody to do this. However, I have encountered several ugly MySQL bugs that affected either my development server or my live server (but rarely both). It is not possible to anticipate environment variables such as these - so if it is at all possible, get your application running on a new virtual host in your live environment as soon as possible. This is the perfect place for your co-workers, testers, or even your clients to interact with the application. After a while, you can be sure that you will not encounter issues with your live environment when you go live with your changes, allowing you to focus on more important things.

That should give you a good start on lowering the stress associated with large application changes in a production environment. Of course, I still ran into a couple of issues with the migration process - so don’t get down on yourself. The bottom line is that migrations are really ugly and almost always come accompanied with unexpected issues. The first goal of developers should be to minimize the chances that the affects of the resulting bugs will significantly affect the end user. Automation is a significant step in that direction.

So what have you learned about upgrades and migrations? Please share any stories and lessons that you have learned in the comments.