Next in line was PopularData.com who offers a free database of 765,345 postal codes and their geographic data (city, province, latitude, longitude). Such data seems to be something recurring in various projects, so I figured I'd post here once and for all to tell you (and make sure I remember) how to get that data and inject it into a MySQL database on a remote server. Here goes.

Log into the server and fetch the data

$ wget -qO- www.populardata.com/downloads/postal_codes.zip | funzip > /tmp/postal_codes.csv | sed -i 1d /tmp/postal_codes.csv

Create the MySQL database/table to inject the data in

$ wget -qO  /tmp/postal_codes.sql  www.fusi0n.org/data/postal_codes.sql | mysql -u USER -p DATABASE /tmp/postal_codes.sql

Then run the import:

$ mysqlimport --fields-terminated-by="|" --lines-terminated-by="\r\n" --user=USER --password DATABASE /tmp/postal_codes.csv

Voila. Records: 765345  Deleted: 0  Skipped: 0  Warnings: 5 ]]> http://fusi0n.org/coding/canadian-postal-codes-database/feed 0 http://fusi0n.org/coding/canadian-postal-codes-database http://feedproxy.google.com/~r/fusi0nv2/~3/LrBQrVQShII/newnewtwitter-keyboard-shortcuts http://fusi0n.org/internets/newnewtwitter-keyboard-shortcuts#comments Sun, 15 Jan 2012 08:28:49 +0000 pluc http://fusi0n.org/?p=2004 Twitter's new design, the second one since the mass adoption of the platform, lovingly called #newnewtwitter was just enabled on my account. You can get a nicer output than what's below by pressing '?' in #newnewtwitter. They're pretty useful keyboard shortcuts - if you, like me, find yourself using the Web version more often than anything else.

Someone who is tired of technology being difficult and knows there is something better; someone that loves to get the job done instead of working on their machine; a person that isn’t afraid of breaking the status quo; someone that appreciates quality design and workmanship; a person that realizes cheapest isn’t always best. Apple fanboys are commonly confused with the sensible people of society.

I don't care much about his argument for quality design and workmanship, his affirmation that cheapest isn't always best or his stab at societal common sense. What transpires from the rest of his 'definition' is what I don't want to see happen with the world. Telling people to get an Apple product if they think technology is too difficult is how we'll end up enslaved to the fruit-people. It's supremely important in this day and age not to let technology become overwhelming. Technology, in the state it's in, is meant to be understood and played with, not merely consumed. If you have no understanding of what you're using, well first you're going to be in a world of hurt, then you'll have to rely on somebody who actually understands to help your sorry ass out. Technology isn't kind to neophytes. For every Apple fanboy there is, there's a geek who knows better.

'there is something better'

If you buy a Mac because the alternatives (namely Windows and Linux) are too complicated for you, you have no clue what you're talking about and you should go back to studying technology for a bit. Windows is just as easy as Mac, but has numerous downsides due to its popularity so I won't delve into that. Linux? Well, Linux isn't for everyone. I'd still recommend to try (yes, you can just try Linux without installing it) an easily-accesible distro such as Ubuntu on your old machine before I suggest dishing out over a thousand dollars for a new machine.  Calling Apple products better than the alternatives because they're more automated is like calling automatic cars better than standard. If you want technology that will do it for you, come back in a decade or two. Right now, we're trying to understand it and see what we can do with it. Once we have that figured out, we'll let you fanboys know so you can line-up 3 days in advance.

'love to get the job done instead of working on their machines'

Jim forgets that everything from the keyboard he's typing on to the software he uses to post his rants were made by people who preferred to 'work on their machines'. The world needs hackers, people who refuse to settle for what they're given and want to do and learn more. Jim is saying to buy Apple instead, as it's a lot easier to get around while being completely oblivious to what's actually going on. What do you do when your beloved glowing Apple doesn't heed your command? What if you want to achieve something the engineers didn't expect or design? Settling for automation is the death of innovation and right now, that's the worst thing you could possibly do.

'break the status quo'

That's just silly. The only thing that's breaking the status quo when it comes to computer OSes is using Linux. Go in any conference/tech meetup and after you're done counting all the Macs and iPhones, tell me how many people you notice using Linux. The iPod came out 11 years ago. Ever since, Apple has been the status quo. Just look at the units sold, and the math is fairly simple.

Don't settle for a shiny gadget. Technology is beautiful and opens up a near-unlimited number of opportunities. Don't let fanboys scare you from embracing it, and don't let them advise you on what you buy. Evaluatuat the possibilities in front of you and make a choice for yourself. That's the first step to learning technology.

Back when people were still walking around with Treos and BlackBerrys that looked like they were fresh out of a Tamagotchi factory, Jobs and his pirates were taking every manufacturer to school with Apple's  iPhone line, leaving the rest of us dizzy over the mere implications of the technology. And he did it again with Apple's tablets, the iPads, a few years later, always opening the trail.

We have a lot to be grateful for from Steve Jobs. He was a truly brilliant man - the kind that surface two or three times in  your lifetime. A ferocious and determined businessman - a true a visionary who shepherded society itself into a concrete bigger and better future for everyone. Steve's legacy most certainly lives on, with over 100 million iPhones, 300 million iPods, nearly 10 million iPads and 10 million Mac computers sold every month, it's safe to say his influence will still cause ripples for exciting years to come...

Thanks Steve.

Here it is:

#!/bin/bash
# To change wallpaper automatically after N minutes, add this to your CRON (where N is minutes): N 0 * * * /path/to/wp-show
base="/home/pluc/.walls/"
photo=$(find $base -regex ".*\([jJ][pP][gG]\|[jJ][pP][eE][gG]\|[gG][iI][fF]\|[pP][nN][gG]\|[bB][mM][pP]\)$"  -type f | while read x; do echo "`expr $RANDOM % 1000`:$x"; done | sort -n | sed 's/[0-9]*://' | head -1)
uri="file://"$photo
gsettings set org.gnome.desktop.background picture-uri "$uri"

define('BIG_COMMERCE_DEBUG', false);
define('BIG_COMMERCE_SUCCESS', 'SUCCESS');
define('BIG_COMMERCE_FAILURE', 'FAILED');
define('BIG_COMMERCE_USERNAME', 'YOUR_USERNAME');
define('BIG_COMMERCE_TOKEN', 'YOUR_TOKEN');
define('BIG_COMMERCE_URL', 'https://YOUR_STORE_URL/xml.php');
function bc($type, $criteria) {
  $xml = new SimpleXMLElement('');
  $xml->addChild('username', BIG_COMMERCE_USERNAME);
  $xml->addChild('usertoken', BIG_COMMERCE_TOKEN);
  switch (strtolower($type)) {
    case 'product':
    case 'products':
      $xml->addChild('requesttype', 'products');
      $xml->addChild('requestmethod', 'Get'.ucfirst($type));
      $fields =  is_array($criteria)? $criteria : array('productId' => (int)$criteria);
      break;
    case 'customer':
    case 'customers':
      $xml->addChild('requesttype', 'customers');
      $xml->addChild('requestmethod', 'Get'.ucfirst($type));
      $fields =  is_array($criteria)? $criteria : array('customerId' => (int)$criteria);
      break;
    case 'order':
    case 'orders':
      $xml->addChild('requesttype', 'orders');
      $xml->addChild('requestmethod', 'Get'.ucfirst($type));
      $fields = is_array($criteria)? $criteria : array('orderId' => (int)$criteria);
      break;
  }
  $details = $xml->addChild('details');
  foreach ($fields as $k=>$v) $details->addChild($k, $v);
  $stream = stream_context_create(array('http' => array('method' => 'POST','header' => 'Content-Type: text/xml', 'content' => $xml->saveXML())));
  $response = new SimpleXMLElement(file_get_contents(BIG_COMMERCE_URL, false, $stream));
  switch ($response->status) {
    case BIG_COMMERCE_SUCCESS:
      return $response->data;
      break;
    case BIG_COMMERCE_FAILURE:
    default:
      return BIG_COMMERCE_DEBUG? $response->errormessage : false;
      break;
  }
}

So... how do we remotely create users on a platform that doesn't let you create users with their API? After a bit of looking around for solution, I stumbled upon my client's store registration form (which is hosted, like everything related to Big Commerce, on a server you don't control). Different domains, different servers, no API methods... but I have a form. Have you ever heard of XSS? In the world of developers, it's a real annoyance. Essentially, if you fail to secure your forms properly, anyone can grab your form, put it somewhere else on the internet and submit bogus data (because the form still sends to you, get it?). Anyway, it's usually really only a security concern, but for this particular project, it became a feature.

Big Commerce does not validate their registration form, so you can submit it from anywhere... like right after a WordPress (or in my case, BuddyPress) user is created for example! A simple cURL request with the proper data set and we've got ourselves a relatively easy bridge. Now, there are cons to this method... for example, if Big Commerce ever realizes that their lack of respect for good practices is used as a feature, I'm not sure they'll keep it up for long. Notice how this post doesn't have a single link...? Yeah.

If this reaches their ears, can we please make a compromise? You pull the non-feature in exchange for a user creation method in your API. Deal?

On to other things..

That wasn't the only thing I was tasked with doing. My client also wanted the ability to use the header from his WordPress theme on his Big Commerce site. Again, Big Commerce doesn't let you put any code on their servers (except static files like HTML and CSS) even though that's where your store resides. Not hating, it's a business model... it's just not a very convenient one. So I have a WordPress header (including a carousel and a drop-down menu of WordPress-centric items) and a lousy API. I initially tried to use BC's %%Include%% "feature", but as you can guess, it fails for remote resources.

Long story short, I created a new file at the root of my client's WordPress site and tossed this in:

define('WP_USE_THEMES', false);
require('./wp-blog-header.php');
get_header('store');

With that, I can essentially be inside WordPress while remaining outside of its scope. Notice the get_header() parameter that allows you to use specific files (such as, in my case, header-store.php). Alright, so I have a decent looking header (that's a few lines of random HTML, JavaScript and CSS) but I still can't fetch it from the store. So I had to get creative. After a fair amount of digging around, I found that I could upload files to Big Commerce - but only static (HTML, CSS, JS, etc) and only in a very specific location (that's /content by the way). With that, I can build something:

#!/usr/bin/php
$ch = curl_init('http://secret.url/headerForBC.php');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, false);
$header = curl_exec($ch);
$file = tempnam('/tmp', 'bc2wp');
file_put_contents($file, $header);

$context = stream_context_create(array('ftp' => array('overwrite' => true)));
// Big Commerce requires a secure connection
$ftp = ftp_ssl_connect('server2600.bigcommerce.com');
$login_result = ftp_login($ftp, 'duh', 'ohai');
echo ftp_put($ftp, '/content/wpheader.html', $file, FTP_ASCII)? 'success' : 'fail';
ftp_close($ftp);

Then toss it in a cron that'll run it every 24 hours and use %%Include.http://my.store-url.com/content/wpheader.html%% to render it. That wasn't so bad, was it?

See, the OAuth protocol uses timestamps and nonces to validate requests. Should the timestamp of the requestee be negative or inferior to the timestamp of the requested, the request will fail and the whole OAuth workflow will fail, and chances are you'll have no goddamn clue why. Fear not! I experienced it, so I had to understand it.

Here's what happened: first, I need to properly illustrate the data I was working with:

• OAuth request sent to Context.IO (in the  "America/Montreal" (EST)  timezone);
• OAuth request sent from my application's server  (in the "America/Chicago" (CST) timezone);
• Server from which the request originates  is  a VPS from WebbyNode running Xen under Debian 6;

Now, there's a chance that, if you're in the same timezone or your timezones and date settings are properly set, all OAuth API calls will work just fine... but if the time would never change, there would be little reasons to make so many utilities to update it. So, it changes. If that happens, and it changes just enough to fail the OAuth check.. well that's where the problem is.

Having a Xen-powered VPS means that the system clock is controlled by the host (or the "parent" machine if you will). Whenever I would set the date to match the one of the server I was sending requests to, it would revert right back. It sucked. For a long time. The fix is pretty simple, you just have to know it:

echo 1 > /proc/sys/xen/independent_wallclock && ntpdate pool.ntp.org

If independent_wallclock exists, permissions won't let you pipe output to overwrite it, so use:

echo 1 >! /proc/sys/xen/independent_wallclock && ntpdate pool.ntp.org

After that, your timestamp should be good and the update you performed with ntpdate will finally stick.

In order to ensure this works, however, there are a few dependencies:

• You need to have a domain registered with Namecheap;
• That domain needs to use Namecheap's nameservers (https://manage.www.namecheap.com/myaccount/modsingle.asp?domain=domain&type=backtous&rkey=NC);
• You must've enabled "Dynamic DNS" in your domain settings (https://manage.www.namecheap.com/myaccount/modsingle.asp?domain=domain&type=ddns&rkey=NC)
• This script is CLI-oriented, meaning it's meant to be executed via the command line. Regardless of your OS, you will need:
  • php5-cli php5-curl;
  • A working Internet connection;

That being said, it's relatively simple to understand. Most of the job is done server-side, so all you really need to do is hit a specific URL and let Namecheap do the rest:

https://dynamicdns.park-your-domain.com/update?host=&domain=&password=

Once you know that URL, it's pretty easy to implement something around it. Here's my little facilitator built in PHP:

Download it or browse the source.

Until I got really fucking tired of all the direct messages. Here's a reboot of Damon Cortesi's DM Whacker: DMDelete-v2.0. It's fast, uses Twitter-specific resources (such as their jQuery and dialogs), and is merciless. Give it a try, all you have to do is drag the link blow to your bookmark toolbar (or wherever you want really), go to your Twitter inbox and click it!

This is beta software! Feel free to relay any feedback you might have after using - it's rather challenging to thoroughly test this piece of code on my own, so there may be random hiccups.

DMDeleter-v2.0

Browse the source