Joseph Piché

5 years of achievements

jpiche — Wed, 18 May 2011 03:42:40 +0000

Friday, May 13th marked an important milestone in my life: 5 years of being married to my beautiful wife. And while those five years were filled with both excitement and hardship, they have gone too fast. Here's a snippet of what happened in the last 5 years: I got married, graduated with a BA after only 3 years of college, got promoted to Director of IT at that college, witnessed the birth of my son, got credentialed as a minister in the Assemblies of God, bought a car, moved from North Dakota to Minnesota for my first job as a full-time programmer, gave up my ministerial credentials, peaked at $92k of debt, got a new job, paid off almost $20k of debt in one year, and witnessed the birth of my daughter.

After reminiscing on all that, I feel a bit nostalgic and remember that I will probably never go back into the classroom for another degree and that I have already worked in Technology much longer than the average. However, I know I have so much to look forward to in the next five. Watching my son grow up brings me more joy than anything else, and I can't wait to see both my kids play and grow together.

As for future achievements, I am proud to say that after years of talking about writing a book—and not doing anything serious about it, I finally found a good topic with enough material, and officially started outlining. A couple weeks ago, I had a bit of a “lightbulb” moment when I really understood that I will not be able to pursue anything more in academics—probably ever. But who says I have to? The mountain of writing I did in college should have taught me at least that I am capable of writing a book if I simply sit down and do it; and if I'm able to write a book already, why would I need a master's degree? And you know what? Biblical exegesis, translating ancient Greek, and reading entire philosophy books in a day is much more fun than I remember.

Shutdown from the system menu in Ubuntu

jpiche — Mon, 21 Feb 2011 16:58:03 +0000

My introduction to GNU/Linux came when GNOME 2.10 was still in development and Gentoo still seemed like a good idea. I still get nostalgic thinking about the hours I wasted compiling X on a 256MB RAM laptop with a 1.2GHz Celeron, so it is no wonder that tiny changes to the GNOME user experience can bother me immensely.

Most recently, changes in Ubuntu like defaulting the border button layout to the left or not allowing rhythmbox to have a tray icon have just about made me scream, but both are easily solved: gconf-editor and banshee. However, the one issue which has been a thorn in my side for over a year is moving the shutdown and logout functions from the system menu to the indicator applet. Shutting down, suspending, logging out—those are all essential tasks which have been located in the same place since long before I started using GNOME. As soon as I noticed this change, two questions arose: "why?" and "how do I put them back?"

As annoying as the change was, I eventually got used to it, as neither of my questions were answered. However, that changed this week when I was reminded of this dilemma and decided to take a second look. To my surprise, the functions are easily moved back by simply removing Indicator Applet Session from the panel. I don't know why I never thought of doing that before instead of digging through endless control panels and configuration files. Now I wonder if I will keep it this way since I've been using the session applet for so long.

Single-threaded pancake flipping

jpiche — Sat, 22 Jan 2011 01:23:24 +0000

Having an educational background in Philosophy, I tend to think about trivial tasks more than I should. Sometimes this gets me into trouble, like when I waste endless hours studying the Python GIL or study code used in The Computer Language Benchmarks Game.

This week while looking at code, I ran across the Fannkuch-redux benchmark, and was instantly intrigued. The problem being solved by the benchmark can be described as brute-forcing a conjecture of the limit of “flips” of a certain set of numbers. The task is relatively simple (taken directly from link above):

Take a permutation of {1,...,n}, for example: {4,2,1,5,3}.
Take the first element, here 4, and reverse the order of the first 4 elements: {5,1,2,4,3}.
Repeat this until the first element is a 1, so flipping won't change anything more: {3,4,2,1,5}, {2,4,3,1,5}, {4,2,3,1,5}, {1,3,2,4,5}.
Count the number of flips, here 5.
Keep a checksum.
Do this for all n! permutations, and record the maximum number of flips needed for any permutation.

The name of the benchmark comes from the German word for pancakes, and is used because the analogy of pancake flipping sort-of fits (and sort-of doesn't).

After studying both single-threaded and multi-threaded implementations of the benchmark in a number of languages, I found that the Java, Python, and PHP versions told the most interesting stories. When running for input 12, these programs took 68.99 seconds, 43 minutes, and 1 hour 26 minutes respectively on a single-core x86 machine.

While an extremely small portion of web programming relies on pure number-crunching, these numbers are a bit unsettling nevertheless. Each version of the benchmark runs almost identical code and yet the time taken to run the code varies by over an hour. I've always known that PHP is a joke and should not be considered a “real” language, but this is a bit ridiculous.

Of course since the C implementation was missing, I translated the Java version and submitted it. The result is rather intriguing: it only ran for 57.79 seconds. Then I noticed the memory usage. My C version only used 256 KB of RAM, where the Java version used 12MB, Python used 3MB, and PHP used 5MB.

Drawing conclusions from benchmarks like these is probably a bad idea; different programming languages are built for different purposes. For example, I am very glad PHP does not have a volatile keyword or similar nasty things Java and C have, because PHP is built as a simple language that anyone can pick up and start programming in with no experience. However, I am sad that Python—in its zen of language and “batteries included” philosophy—does not have a simple method for compilation to machine code; interpreted code can be harmful at times.

The risk of third-party code

jpiche — Tue, 11 Jan 2011 07:34:24 +0000

In business, the purpose of programming is to solve problems. Some problems are simple, and even if time consuming, have solutions which are easily discernible. These types of problems encompass the majority of what makes up web development, like if one or more programmers are given the task of implementing a design into a fully-functioning website.

Large and complex problems for a web developer are few and far between. Examples might include persistently tracking user clicks on a page without the use of cookies, or performing data processing on hundreds of millions of records in a database and sending the results across the continent in under 0.1 seconds. These are the types of problems I thrive on; give me something seemingly impossible, and I will obsess over it until I find a solution.

In real life though, the dollar drives the workload. And if someone wants just a plain old website like everyone else on the internet, I then have a simple problem with an easily discernible solution. So easy in fact, that my job is no longer writing code, but re-purposing already existing code. And due to the plethora of code already written by other programmers—both open source and proprietary—chances are that a simple cost-benefit analysis of the available options reveal that using code someone else has already written will yield better results. This is codified in the principle “Don't repeat yourself.”

And here is the warning: if you use someone else’s code, always research the risks involved. If the project is to implement a cookie-cutter brochure-like website, using Drupal as a Content Management tool probably has a small amount of risk. However, if the line is starting to be blurred between the simple problem and the complex, make sure to analyze the situation correctly. The risk of using third-party code may be larger than you realize.

This may not be as obvious as it sounds. The quick case-study is inheriting a project from another developer in your company where the project has a small budget—or no budget. Every minute spent on the project is time lost, so taking 10 minutes or an hour to review the codebase may not seem like a good idea. The project launches. Then 3 months go by and everything breaks at a crucial moment and your company loses $100,000. You spend hours on end frantically searching the code and eventually discover a bug in an untested third-party module in which someone used “YYYY” instead of “yyyy”. Not good.

TinyFeed, rewritten

jpiche — Sat, 08 Jan 2011 03:59:35 +0000

Today marks the release of TinyFeed 2.0 (and subsequently 2.0.1 because of svn issues). Version 2 is a rewrite of the plugin, and while the features are the same as the original, the way the plugin functions has been greatly improved.

TinyFeed was the first Wordpress plugin I ever wrote and was initially released September 2009. Wordpress has changed a bit since then, however not enough that anything impacted how TinyFeed works. I was new to Wordpress, and didn't take enough time to figure out how tactics like inline loading of jQuery might effect other plugins. My Wordpress-ing has matured a bit and I have now taken the time to research how TinyFeed might interact with other plugins, and have rebuilt how javascript is loaded, and all the scripting itself.

For future enhancements, I plan on adding StatusNet support, customizable timestamps from the Wordpress admin, and linking within status text.

File Descriptors vs Threads

jpiche — Thu, 06 Jan 2011 05:15:17 +0000

Being an obsessive computer programmer, I spend the majority of my time on a computer. Not all of it is work—in fact, in an average week I spend about 10-15 hours researching and teaching myself new things. A few weeks ago, I spent some time reading code from a few HTTP servers, and researching they handle upwards of 10000 simultaneous connections (the c10k problem).

Almost all of the sites I put out run on Apache. Most PHP hosting providers use Apache as the web server of choice, Magento requires Apache, and Django documentation recommends Apache + mod_wsgi. I don't run any sites that get near to c10k, but after attending a seminar on Javascript which talked about events vs threads, I started to think I was setting myself up for failure by using a web server which is restricted to a static number of connections.

The two web servers I spent the most time analyzing were Apache and Nginx. After hours of pouring over code, learning the difference between epoll and /dev/poll, and finding new benchmark tools (ab is amazing), I came to the conclusion that the web server is not the bottleneck: ~~CGI~~ dynamic content through scripting languages is the bottleneck. If Apache is configured correctly, c10k is achievable, but only as long as you keep web scripting away from it. If fact, because of the way Debian builds mod_php, Apache is forced to run by forking and is unable to use threads or event-based connection handling.

On the other hand, while Nginx uses the event model to solve the c10k problem, it does not directly allow scripting at all. Dynamic content can only be served by proxying content from solutions like PHP-FPM, WSGI modules, or another web server like Apache. So if your webserver only has 256MB of RAM, serving thousands of simultaneous connections is possible if your site is completely static, but as soon as you make something dynamic, your machine just might explode.

So is there a solution to this mess? If you are using PHP, opcode caching solutions like APC will help you keep your server from swapping. It is a bit like cheating, but if you have a solution to keep your cache fresh, it might just help you sleep at night.

Free Software, not Open Source

jpiche — Tue, 14 Sep 2010 13:48:23 +0000

Being an advocate of Free Software of all kinds, I often resort to using colloquial terms. “Open Source” is much easier to say, and I thought is more understood, than “Free Software”, “Free/Libre Software”, or “Freedom Software”. I understand that “Open Source” misses the point, but I recently discovered that some people are instead using the phrase to mean “you can view the source code even though you have no rights to do anything with it.” Two Magento extension development companies, AITOC and AheadWorks, both advertise their products as such: “100% Open Source.” This is smart advertising for them, as it is true: the source code can be easily viewed (unlike other Magento extensions with obfuscated or compiled source). But as a promoter of Free Software, I now have the added responsibility of explaining to people how those companies are false advertising.

“Over half of the web is flash”

jpiche — Tue, 20 Jul 2010 04:04:51 +0000

Out of nowhere I hear someone at the other end of the office (we share with two other companies) blurt out, “over half of the web is flash.” This person always leaves his door open and talks loudly, so I've learned to tune him out, but this line grabbed me. First, I feel this number is wrong. Second, there is no way to know with a good amount of certainty what the percentage is. From the continuing context, I learned that he was talking about the iPad and its lack of Flash support, criticizing Apple for becoming “the next Microsoft”—whatever that means. He concludes with saying an increasing amount of developers are starting to program for android.

This whole conversation triggered feelings of anger and frustration at this person. I hate Flash; I hate Flash a lot. But this is not because Flash is not supported on the iPad, or that it is full of security holes, or that distributing unchecked byte-code is an inherently bad idea. I hate Flash because it is closed source, anti-freedom software controlled by a single entity interested only in financial gain.

Do I have Flash installed? Yes, and according to certain surveys, up to 99% of internet users have Flash installed. And it disgusts me. I try to not install it, or run it only in controlled virtual environments, but it is ubiquitous. Many “social marketing” companies promote using sites like YouTube, which almost exclusively uses Flash for its videos, so when clients ask for YouTube, I reluctantly grant.

But while I concede to its use when necessary, I never create new material using Flash and whenever possible evangelize open technologies and Free/Libre alternatives. And I beseech you, reader, to also stop developing for or using Flash.

Getting Paid to Write AI

jpiche — Sat, 05 Jun 2010 01:01:17 +0000

One of the best parts of my previous job—and also the part that kept me going for so long—is that I had the chance to work on a variety of projects, in many cases for completely different business ventures. One week I had been privileged with the task of writing a small program which ended up being semi-intelligent (of course artificially, thus AI). As it so happens I also seemed to have stretched PHP to its limits.

When originally typing up this post, I kept the details to a minimum, but now that I've moved on I can freely discuss it. This program—code-named “twitterbot”—was tasked with searching Twitter for tweets with certain keywords within a radius of a location, like for tweets containing "hungry" within 30 miles of Minneapolis, and @reply to that person within an hour of their tweet about ... a website I probably shouldn't name. However this becomes tricky since Twitter is really good at detecting spam, and a lot of Twitter users really hate spam (like me). To get around it, I made a bunch of twitter accounts using anonymous email addresses, created bios, set profile pictures, and set color schemes, all to make them look like real people.

However, just because a Twitter account has a profile picture, it does not mean it will be treated like a human: the account has to have realistic-looking posts. No problem. First I set a limit to the number of posts an account can make per day, make sure all the posts are within normal waking hours (like 6am to 10pm), have 30-50% of the tweets be the spamming tweets, and the rest be copies of tweets from a different search that none of the accounts will be replying to. Then since the spam tweets themselves are also randomized, the sum of it all equals twitter accounts that even people are sometimes not able to recognize that the account is a bot.

For the technical, I wrote this program in PHP both because we had thousands and thousands of lines of legacy code in other programs that we didn't want to touch, and second because it is extremely simple and quick to code and deploy. With a semi-AI bot though, the scene changes and suddenly both opportunities and concerns that weren't there before suddenly are. Example opportunity: pcntl-fork becomes available. Example concern: the PHP program will probably be running as a daemon on a server, use up all your RAM and explode.

In the end, the bot ran strong for about 2 weeks before management changed their minds and had it shut down—spamming people does not make for good press. I felt terrible having to write such a program; I really don't like spam, let alone being the one doing the spamming. But since then I've moved on to a position at a different company and write ethical code now.

Tabs or Spaces

jpiche — Wed, 14 Apr 2010 18:34:52 +0000

Any programming environment with more than one person working on the same project requires communication; it is an essential part of getting a job done efficiently, and it is common sense. But while normal business communication includes writing, speaking and body language, programming communication also includes readability of code along with good documentation.

Having experience with coding in a large project with other people, I clearly see that indentation is a large part of readability of code. It is extremely important to be consistent with spacing regardless of the language. However, whether to use tabs or spaces for indentation is up for debate. In fact, the issue of tabs versus spaces is a highly debated issue; a simple search will show that. Even so, it is important to have a clearly defined rule when engaging in a project with other people.

To show the importance of this many different open source projects define strict guidelines for indentation. To list a few:

Most of these say 4 spaces for per indentation level, and all say to never mix tabs and spaces. I do 4 spaces; my programming team at work has agreed on 4 spaces—it works really well in pretty much all cases. All good editors have ways to make this easy to do using the tab key too.

More than likely the language you code in already has some guide already. Please follow it, and remember what the Zen of Python states: “Readability counts.”

Joseph Piché

5 years of achievements

Tags:

Shutdown from the system menu in Ubuntu

Tags:

Single-threaded pancake flipping

The risk of third-party code

TinyFeed, rewritten

File Descriptors vs Threads

Tags:

Free Software, not Open Source

Tags:

“Over half of the web is flash”

Getting Paid to Write AI

Tags:

Tabs or Spaces