e;blog

pretending to still be a geek

e; — Mon, 04 Jan 2010 00:19:04 -0800

Photo by Eric Richardson.

If you look back through the archives here, you'll find a lot of intense geekery. I ran Linux as a desktop OS for nearly a decade, administered servers, etc. I enjoyed it.

These days, though, I just tend to do less of it. Needs and situations change, and I don't end up doing as much geeky stuff as I once did.

Today, though, was an exception. Today I had to rebuild a home Linux server, migrating 350gb of data over to new drives before adding those same old drives into a logical volume.

Starting point: Linux server with three 250GB drives concocted into a weird 375GB RAID1 array. Two 400GB drives that have been sitting on my desk for a year.

Step 1: Install new OS onto 400GB Drives

Figured why not start fresh, so I hooked the two 400GB drives up along with a CD-ROM drive and installed a fresh Ubuntu server installation. Each drive got a 1GB partition and a 399GB partition, and each set was made into a RAID1 array. 1GB /dev/md0 became /boot/, and the 399GB /dev/md1 became the first part of an LVM volume mounted at /.

Step 2: Hook up old drives to copy data

It's an old server, and they're IDE drives, so I only had four slots to hook up five drives. It's RAID, though, so I simply used two of the three 250GB drives and pretended the third had failed.

Since the data was RAID / LVM, life was a little more complicated than just mounting the disk.

Rinse and repeat for md's 2, 3 and 4:

sudo /sbin/mdadm -A /dev/md2 /dev/sdb3
sudo /sbin/mdadm -R /dev/md2

Once the md devices were up, ask LVM to recognize the volume group:

sudo vgchange -a y

Then just mount it up:

mkdir /tmp/bit
sudo mount /dev/mapper/group1-main /tmp/bit/

Perfect. Now copy data at will.

Step 3: Unmount old drives and add them to /

Once all the data I needed was off, I wanted to turn the two 250GB drives into a RAID1 array and add that to the volume mounted as /, giving me roughly 650GB there.

First, unmount and deactivate the volume / arrays:

sudo umount /tmp/bit
sudo vgchange -a n group1
sudo vgremove group1
sudo /sbin/mdadm --stop /dev/md2
sudo /sbin/mdadm --stop /dev/md3
sudo /sbin/mdadm --stop /dev/md4

Then just use fdisk to re-partition the drives and create one big raid partition. Create a new md device with those two partitions:

sudo /sbin/mdadm -C /dev/md2 --level=1 \
  --raid-devices=2 /dev/sdb1 /dev/sdd1

Add that new device to the logical volume group:

sudo pvcreate /dev/md2
sudo vgextend root /dev/md2

And extend the volume to use the new space:

sudo lvextend -L +232.88G /dev/mapper/root-main

Finally, resize the filesystem to use the new space:

sudo resize2fs /dev/mapper/root-main

Bingo.

Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/root-main 594G  304G  260G  54% /

Here's a question, though: 594G total, with 304G used, equals 260G available? Where's my extra 30G?

The irony of all of this is that 600GB is a fairly pointless amount of storage, and already far less than I need. I plan to repeat this exercise in six months or so, except with the aim of putting a couple terabytes in instead (though either in a different server or potentially just in a device like a Drobo instead).

Back in the Cold

e; — Mon, 28 Dec 2009 12:10:59 -0800

Looking across Bear Lake from Kathy's house in North Muskegon, MI. Not pictured: 40mph wind gusts. Photo by Eric Richardson.

Kathy and I are back in Michigan for the holidays, which means we're getting our annual dose of cold and snowy weather. Christmas Eve brought freezing rain, which turned the roads into sheets of ice. Saturday and yesterday it was snow, and today's weather is characterized by 40 mile per hour wind gusts that are whistling around the house.

People complain about not having seasons in Los Angeles, but I just don't get it. I'm perfectly happy with 70 and sunny.

Joe Purdy at the Hotel Cafe

e; — Tue, 15 Dec 2009 22:02:43 -0800

A friend and I took a trip up to Hollywood last night to see Joe Purdy at the Hotel Cafe and, for once, no one fell asleep.

It's a long story.

The night marked at least the fifth time I've seen Joe play, four times now at the Hotel Cafe and once at an event at the old Conga Room on Wilshire. Combine that with the fact that I own three of his CDs, and you can get a general sense of my impression of his music.

It was my first time out to the Hotel Cafe in a long while, and it got me thinking about the venue that really defined my taste in L.A. music. Way back when the Hotel Cafe was half its current size and still just an all-ages coffee shop, I used to say that if the venue offered a monthly pass, I would have been there two or three times a week.

Looking through the artists on my iTunes, Eric Hutchinson, Erik Penny, Jay Nash (though I had met him before, actually at that Joe Purdy Conga Room show), Jim Bianco, Joe Purdy, Pedestrian, Quincy Coleman and Steve Reynolds were all acts I first heard at the Hotel Cafe. That's a nice little chunk of my music collection to have come out of a then-40-seat venue.

Importing Old Music

e; — Sat, 12 Dec 2009 21:12:53 -0800

If you dig back through the 11 years of archives here (11 years!), you'll find a lot of posts where I wrote about music: what I was listening to, what shows I had seen, what I had rediscovered...

That last one seems to be a recurring trend for me. I have a song or an album and somehow it gets misplaced, but then I pull it out a few years later to realize I still think it's great.

I'm doing that right now, converting some albums I had sitting around as .ogg files. At the moment I'm being reminded how much I liked Massive Attack's Mezzanine, and a few minutes ago I was doing the same with Blonde Redhead's Melody of Certain Damaged Lemons.

Next up (and currently being converted to mp3 via Switch) is Tricky's Pre-Millennium Tension.

Maybe this isn't odd at all, but it surprises me how little my tastes have changed in the last ten years or so. I made a distinct switch in style right around 1998 / 1999, but it's been all the same stuff since then.

My music collection has also been primarily stored on the computer for ten years or so now, which at times has proved to be an all too fragile home. I still remember the surprise of firing up what should have been a dead drive in September of 2004 and finding a stash of songs I had thought were lost.

Misplacing albums does have its upside -- the recovery has all the fun of a iTunes shopping spree with none of the bill.

The Phone is Ringing

e; — Sat, 12 Dec 2009 00:25:33 -0800

Breaking with today's trends, Kathy and I have a home phone. It's not something that we really ever use and it's probably a waste of money to have it, but it's just a few dollars a month and was an easier option than deciding that the building call box would ring to my cell phone.

Seeing how much I love over-designed and half-implemented solutions, it shouldn't surprise that this phone is a VoIP handset that connects to an Asterisk server and an Adhearsion dial plan app.

What is a surprise is that over the last 24 hours, the phone's been ringing at the oddest hours.

In fact, it rang at 2:21am, 4:57am, 6:01am, 7:15am and 8:36pm. Instead of a caller ID number, the handset shows "sip" and "asterisk."

The last bit is the key that these aren't just wrong numbers or robo-calls. Instead, they are calls coming in via direct SIP connections to the Asterisk server.

Frankly, I hadn't even considered that a possibility. Who randomly dials IP addresses looking for phone systems?

The first connection arrived via 66.117.50.225, an IP address in Virginia. Three calls were placed, trying to ring "extensions" of 00442086371406, 0442086371406 and 000442086371406.

The second came via 113.105.152.104 and 113.105.152.103, both IPs in China. They attempted to dial 003318288029, 00033182880295 and 033182880295.

The third came via 89.37.11.12, an IP in Romania. Again, three calls, this time to 00442073942500, 0442073942500 and 000442073942500.

What to make of these? Apparently, there are people / scripts out there war-dialing IPs looking for asterisk installs that are mis-configured to allow arbitrary connections to dial out. They're hoping to use these installs to place international calls.

My setup isn't one of those, though it probably shouldn't be allowing random IPs to connect, either. Perhaps it's time to button that up.

For now, though, I'm just curious what would be there if I actually answered one of those calls.

Three Years in Macland

e; — Mon, 07 Dec 2009 11:52:24 -0800

I noticed this morning that three years ago today, I bought my Macbook. The laptop remains my daily computer -- and is in fact what I'm typing this post on -- though it has had the memory, hard drive and battery replaced in that span.

I made the switch after having run Linux as a desktop OS since 1997 or so. I sort of miss all the tinkering that involved, but at the same time I've enjoyed things just working much more.

Squeezing Those Bytes

e; — Thu, 03 Dec 2009 10:25:04 -0800

Photo by Eric Richardson.

Overnight I managed to reduce the load size of blogdowntown's home page by 400kb without changing a single line of HTML. We went from a bloated 750kb to a much more svelte 340kb.

What could make that much of a difference? The size of our JPEGs. 660kb of that load was images, a number that's way, way too high. We do load 58 images, but I was still pretty floored to see the size of some of the smallish images we were loading up.

So what was the deal? Was I just being sloppy with my image compression? No, Flickr was.

blogdowntown's bdv4 codebase uses Flickr as a primary mechanism for photo handling. We upload shots there, and then drop the URL into the bdv4 Newsroom. From there it grabs the needed information directly from Flickr.

Turns out, Flickr images are anything but small. They're stuffed with EXIF data and compressed for best quality, not for quick loading. That makes sense for a photography site, but doesn't make sense when we're trying to serve up a dense home page.

For instance, consider the nice model railroad photo attached to this post. The 240x160 small version of the image tips the scale at 52kb. If I download it, open it in Photoshop and use Save for Web, default settings drop that to just 16kb.

Obviously I'm not going to do that, though. November was a light month, but we still published 195 images on 73 stories.

That doesn't mean I'm out of luck, though. Earlier this year I got frustrated with occasional Flickr downtime and decided that we needed to start caching copies of all the photos we use. I implemented that via a piece of code that grabs the Flickr images and uploads them to Amazon S3.

In that code, I use MiniMagick to create a 24x24 version of the Flickr square thumbnail. Outside of that, though, I was caching the images from Flickr untouched.

It was only a half-hour or so of work to figure out how to streamline the process, stripping EXIF and compressing the images before caching them to S3.

The result isn't as good as Photoshop, but it's meaningful. That 240x160 train? Just 20kb. For the 75px square thumbnail (a size we use all over the place), size drops from 36kb to 8kb.

For reference, here's the code involved, part of bdv4's ImgCache::Image class:

QUALITY = 90

def cache
  if self.photo.cached?
    return nil
  end

  uri = URI.parse(self.photo.photoBase)

  Net::HTTP.start(uri.host) {|http|
    # -- get square thumbnail
    sraw = http.get(uri.path + "_s.jpg")
    simg = MiniMagick::Image.from_blob(sraw.body)
    simg.quality QUALITY
    simg.strip
    ImgCache::Image.store(
        [self.photo.photoid,self.photo.secret,"s.jpg"].join('_'),
        simg.to_blob,
        :access => :public_read,:foo => true
    )

    # -- generate small square thumb
    simg.resize "24x24"
    ImgCache::Image.store(
        [self.photo.photoid,self.photo.secret,"x.jpg"].join('_'),
        simg.to_blob,
        :access => :public_read
    )

    # -- get medium size
    raw = http.get(uri.path + ".jpg")
    img = MiniMagick::Image.from_blob(raw.body)

    img.quality QUALITY
    img.strip

    # save this at its current size
    ImgCache::Image.store(
        [self.photo.photoid,self.photo.secret].join('_')+".jpg",
        img.to_blob,
        :access => :public_read
    )

    # resize for _m (max side 240px) and save
    img.resize "240x240"
    ImgCache::Image.store(
        [self.photo.photoid,self.photo.secret,"m.jpg"].join('_'),
        img.to_blob,
        :access => :public_read
    )

    # resize for _t (max side 100px) and save
    img.resize "100x100"
    ImgCache::Image.store(
        [self.photo.photoid,self.photo.secret,"t.jpg"].join('_'),
        img.to_blob,
        :access => :public_read
    )        
  }

  self.photo.imgcache = true
  self.photo.save

  return true
end

Fizziness Quotient

e; — Wed, 25 Nov 2009 13:47:04 -0800

Subconsciously, I think we've all gotten used to a certain range for how much a standard soda fizzes when dispensed from a fountain. You leave a certain amount of room in the cup, the fizz rises, settles, and you adjust that final bit of fill accordingly.

But have you ever found a certain drink that far exceeds that normal fizziness range? It's hard to adjust.

One of my favorite downtown eateries, Mendocino Farms, just opened a second location. As part of that, they added a fountain with sodas from Boylan Bottling. They're great sodas, and they use cane sugar instead of high fructose corn syrup.

I'm particularly fond of the root beer, but have found myself consistently surprised by the amount of fizz. Even when I purposely under-fill, the fizz still sometimes wins. Perhaps that's the cane sugar at work.