Duane's Brain

Array Sampling Without Repeating [Ruby]

noreply@blogger.com (Duane Morin) — Sat, 23 Feb 2013 02:49:00 +0000

I've often come across the problem of random sampling in situations where you don't want repeats. For instance say you've got a list of the user's MP3 files and you are playing them randomly. You've just played a song and you choose another random one. It's easy to say "Don't pick the same one that you just picked." And with a little work you could pick an N and say, "Don't repeat a song in the last N songs."

But you also don't want to say "Don't repeat any songs until all the songs are played", because if the user's got 500 songs in their library it'll be a long time before they hear their favorite song again. What you really want is just something that says "Make it less likely to pick this song too too frequently."

So, here's the idea I finally came up with. Say that you've got an array of files to work with, regardless of what is in them. In Ruby you can just ask for file = files.sample to get a random element out of the array.

Now what? We go and do what we want with file, but now we want to assure that this is less likely to be called in the near future (and, preferably, absolutely guarantee that it is not called within the next N samples).

Start by snipping file out of filenames and moving it to the end of the array:

files <<= files.delete(file)

Try it:

(main)> files=["foo", "bar", "baz", "quux"]
=> ["foo", "bar", "baz", "quux"]
(main)> file = files.sample
=> "bar"
(main)> files <<= files.delete(file)
=> ["foo", "baz", "quux", "bar"]

So what? The sample method is equally likely to pick any item in the array, right?

What if we tell it not to? What if we tell it to only sample a portion of the array?

file = files[0..files.size>>2].sample

Here, we're saying to only get samples out of half the array. So when a sample is chosen and moved to the back, we've established that there's no way this sample can be repeated until at least N/2 samples have been taken.

Depending on your N you might want to do something like N*0.90 or something (if N is a thousand maybe only 100 samples have to go by before it's ok to repeat).

It's a little trick, but I think it's important. I've got an app I'm working on that chooses random quotes, placing them on random images. It bothers me when I know that I've got over 500 quotes in the database, and the user sees a repeated quote after 4 refreshes of the page. So now they won't, now it'll take hundreds of refreshes before they see a repeat. Truthfully I'll probably switch that over to cap at 100.

RubyMotion : Gems Are Not Being Compiled / Built / Included

noreply@blogger.com (Duane Morin) — Fri, 31 Aug 2012 13:51:00 +0000

I was working on the most basic of apps, really nothing complicated at all, and trying to include BubbleWrap. But the first time I tried to use anything (in this case, Device.screen.width) I was getting a name error - as if BW was not included.

I had require 'bubble-wrap' in my Rakefile. I'd done a gem install bubble-wrap. That wasn't it. What appeared to be happening was that over in build/iPhoneSimulator-5.1-Development/objs there should have been a /Users/dmorin/.rvm/... directory where my required gems were getting copied, and that directory didn't exist.

I double checked everything, I ran "rake clean", I could not for the life of me make it include that directory. I even created a brand new empty motion directory and included bubble-wrap in it so that this was literally the single line change I made from default -- and it compiled. Go figure.

Finally, I found it. I had been modifying some sample code that I found online, and it was this line:

app.files = Dir.glob(File.join(app.project_dir,'app/lib/**/*.rb'))|

Dir.glob(File.join(app.project_dir, 'app/**/*.rb'))

I'd seen something like that trick before, and it seemed like something I understood - basically saying that you're going to put some common code into a /lib subdirectory, so compile that first in order to make it available to your primary code.

But that's not quite accurate. I think there was a mistake in the code that I copied, because what this does is to make app.files *only* point the files in your app/ and app/lib directory - completing removing any gems from the filepath. You can check this with rake config - performing this command on a brand new directory will return all of your gem paths (whether it's .rvm or not), but doing so in the directory where the above line is included in the Rakefile? Shows all my app/ code, but no gems.

In my case it turns out I don't need that line at all, so I've removed it. Soon as I did that, my gems were included. If you do need to manipulate app.files and the way things are included, you'll want to append or otherwise manipulate the existing list, not just replace it with a plain old equals!

Rails MVC vs iOS MVC

noreply@blogger.com (Duane Morin) — Fri, 10 Aug 2012 18:59:00 +0000

If your primary experience with Model View Controller (MVC) has been via Ruby on Rails, and then you switch over to iOS (iPhone / iPad) programming, see that it also supports MVC and think, "Oh, cool, I already know that!" Not so fast.

I am not expert at either of these technologies, but let me put it in the best terms I can. In the Rails world, the controller is primarily associated with manipulating the model. You have a Person object, you have a PersonController. That PersonController has actions like index (show a list of Person objects), show (a single Person), edit/update, create/save, destroy and so on. Things that you can do with that object. Each action, in general, has a view. To the point where the default behavior of an action called "foo" on the PersonController is to assume that there is a file called foo in the /views/person directory.

In iOS world, each controller handles a single view. I'm still trying to get my head around it. I keep thinking of the View as an entirely separate object that I instantiate and "connect" to the controller in some way, such that the controller's only real job is to take in some user input, manipulate some stuff, and then let the next view do its thing.

Not really. In fact, the controller can go ahead and just instantiate a UIView object and start populating it. There' a method called viewDidLoad, that gets called when the view is all loaded. Fair enough. Where is that method? Is there a didLoad() method on UIView? Nope! It's on the controller. So it's practically hard coded in the framework that a controller deals with "the view" and you don't get to trick it into managing many views.

I don't yet fully get it.

UPDATE - As I do more googling on stuff like "switch UIView" I'm learning some techniques for how to manage multiple views inside a controller. It's not quite the same thing as having a specific view associated with a specific action like Rails does, but it's inaccurate to say that a controller can only handle a single view at a time. It's more like, "The controller has a pointer to a view. You are free to swap out and/or otherwise regenerate that view according to whatever rules you like." So I don't think I have to have a PeopleIndexController and a PeopleShowController and a PeopleEditController...

Speeding Up the Stanford iPhone / iPad / iOS Development Course

noreply@blogger.com (Duane Morin) — Mon, 16 Jul 2012 17:35:00 +0000

I've known for years that I should learn iOS and Objective-C programming. I've just never found the time and/or patience to do it. I'm a full time Ruby on Rails developer with a bunch of years of Java before that, so I feel relatively confident in my ability to pick up a new platform/language, but I found that there were a number of key differences between that world and the world of Objective-C that merely picking up a book on the subject and banging on it for a few hours a night wasn't working for me.

What is the greatest resource for learning iOS programming? Everybody points to the Stanford iPad and iPhone Development Course. Tens of thousands of people have gone through the course, and most rave about it. I like the idea, a lot. My biggest problems with it are two-fold. First, it's a video long video course - almost 20 hour-long episodes. I don't have the attention span for that, honestly. Second and perhaps related to the first, it is impossible for all learners to get the same amount out of a single course. Someone who has never touched Smalltalk, for example, will take longer to grap the idea of message passing than someone who knows Smalltalk. That means that a lesson on messages will be the kind of thing that some folks want to just fast forward past. But then if you FF too far you may miss important things.

This is why I've never done the Stanford courses, and instead tried to wing it with heavy Googling and lots of time on Stack Overflow.

Today, I found a better answer. Playback the Stanford courses at double speed. How? Harder than it looks, actually, if you've got the latest of everything on your machine. Turns out you have to go back in time a bit. [ Note that I am running a Mac with Lion installed. Your mileage on Windows may vary.]

Download Quicktime Player 7. This advice comes straight from discussions on the Apple.com website, by the way.
Find your video in iTunes, select Show in Finder.
Right-click (or ctrl-click or whatever) on your video, go to Open With... and pick QuickTime Player 7. Note that you did not upgrade QuickTime in the first step, and it will not become the default player. You deliberately installed a previous version. You may have to find it under "Other" the first time you do an Open With, but in my experience once you've done that the first time it will show up as an option the next time.
Got it open in QTPlayer 7? Good. Now go to Window -> Show A/V Controls. This is the part that you needed QT7 for, because they've taken it out in later versions.
In the lower right corner there's a slider for Playback Speed. Pick a speed. I'm running them at 2x, and I find it the right combination of understandable while causing me to yell "Faster, talk faster!" at the screen like I do at 1x.

Now I feel like I'm getting somewhere. If you've got two monitors, that's even better - put the video up on a second screen, and then go about your business doing whatever other tasks are important while you half-listen to the lecture (just like most of us developers do in staff meetings when we bring the laptop, am I right?) When it gets to something that seems interesting, pay more attention to the video.

I learned this trick this morning, and blew through the first lesson (the one that I would have been most likely to scream "Talk faster!" at) while doing other things. Now I'm halfway through lesson 2 and into stuff that's more interesting to me, but still only at that "Confirm and clarify stuff that I'd already experienced" sort of phase (remember, I've been banging around on all this stuff by myself for a long time). Eventually I'll get to the "Entirely new to me" stuff and, if necessary, I can bring the speed back down to 1x.

But probably not 1x. ;)

How To Hard Reboot / Factory Reset Your Verizon Droid Incredible 2

noreply@blogger.com (Duane Morin) — Tue, 19 Jun 2012 17:07:00 +0000

I have a Droid Incredible 2 from Verizon. Had it coming up on 2 years. This past Father's Day it stopped powering on. Very weird circumstance, it rebooted itself while in camera, then acted flaky, then when I put it on the charger....nothing. Stopped responding in any way to any sort of stimulus. No lights, no nothing. When I'd first plug it in there'd be a little red charging light, but that would go off in 5 seconds. Left it on over night, nothing. Popped the battery, still nothing.

Having tried everything and now bracing myself for the worst (a dead phone), I call Verizon tech support, who quickly moves me along past all the above steps to "Ok let's do a factory reset on the phone."

Hold down the Volume Down key.
Press and hold the Power key for 3-5 seconds.
You should now get a boot screen. (The hacker/geek in me squealed with glee when I saw this come up).
One option is Factory Reset, which if you've never done one I'll warn you wipes your entire phone as if it's brand new, and you'll be tasked with setting up your Google accounts and such al over again, not to mention the havoc it will wreak on your various installed apps.
Another option, however, is Fast Boot.

I asked the tech person, "Since I know what a factory reset will do, can I try Fast Boot? Worst case it does nothing, and I repeat this process and select the reset."

"Sure, sounds good," she tells me. "All we really have is directions for how to reset it."

Wonderful. I hit the Reboot and thank her for her time.

A few minutes later (the I2 has a very long boot time, you ever notice that?) I have my phone back! Yay.

Documenting here because I did go googling for how to hard reboot my phone, and could not find easy instructions to do so. Maybe the next guy to come along looking for it will land here. Hold down Volume Down + Power, get boot screen, pick Fast Reboot.

Zazzle on Rails

noreply@blogger.com (Duane Morin) — Tue, 19 Jun 2012 15:15:00 +0000

I have merchandise on the Zazzle store. I also have sites that I manage, and I would like to advertise my merchandise on my sites under my own control. Zazzle offers a handful of banner options, but I wasn't really finding the level of control that I wanted. So I've set about writing my own.

Zazzle does offer a PHP front end if you want to host your own store. However I'm a Rails guy and would much prefer to work in that language. From what I can tell, there is no Zazzle gem. Yet.

What they do offer is a simple enough RSS feed that will allow you to easily pull the relevant info on your products, in a variety of ways. You can customize the call to grab newest items, or most popular, via search terms or by searching for specific product types only. In my case, and since I have less than 100 items in the store, I can just go ahead and grab all of them:

@feed = Feedzirra::Feed.fetch_and_parse("http://feed.zazzle.com/ShakespeareGeek/feed?st=date_created&pg=1&ps=100")

You'll get back the item URL, description, and even a pre-made

complete with image that you can just go ahead and drop right on your page, if you like.

Now it's really up to you what you do next. I actually use this call to go ahead and seed a local database of Product objects, which I'm then tagging and categorizing as I see fit. Since I know when I create new products, I can manually go in and update my database to stay in sync.

I wrote up a simple banner script for now that just goes in and says "Get me 3 random products," and then I display the supplied HTML that came with the RSS feed. Later, depending on the context of the page, I'll be able to say "Get me 3 random Hamlet products" or "Get me 3 t-shirts" or possibly even change what and how I'm displaying the items.

Note that the URL takes you back to the product page on Zazzle, where your customer would then still have to hit a Buy button. Not the greatest user experience, but I'm not trying to rebuild the store from scratch, either. I just wanted a way to display advertisements for my merchandise in my own way. Your mileage may vary.

When I have time I hope to expand this with more source code and examples about exactly what I'm doing. At the moment, though, I have no idea if Google will pick up this post and if anybody will ever see it. So if you did land here because you do want to connect Zazzle and Rails, leave a comment so I'll know whether to keep posting on the subject.

Rails Migrations Break Over TIme?

noreply@blogger.com (Duane Morin) — Wed, 21 Mar 2012 01:01:00 +0000

Here's a fun little problem. I'm sure I'm not the only one to have run into it, but I'll document it for kicks even though I'm not quite sure yet how I'll solve it.

I just inherited a legacy codebase for a Rails 2.3.x project. So I point to a clean database, run a rake db:create followed by a rake db:migrate, and suddenly I'm getting an error - in this case about the absence of a certain "ambassador" column in my AdminUser object.

Two things are odd. First, there is a migration to set up that column on that object, we just haven't gotten to it yet.

Second, this migration has nothing to do with that column.

...or does it?

Here's where the time warp comes in. The original owner of this code set up the AdminUser object, then at some point later decided he needed an ambassador column, so he wrote a migration to add it. Fair enough. But then after that he went in and enhanced the create method for AdminUser to say "If your ambassador flag is set, do something different."

Remember that migration that was failing? That migration tries to insert a new AdminUser (as a mock object). Before the column is added.

So one migration created the AdminUser object (without the ambassador column). Next in sequence comes this innocent migration that tries to create a mock AdminUser. However, the AdminUser can't be created without looking at the ambassador flag - which hasn't been added yet.

Not quite sure how I'm going to solve this one -- probably comment out the ambassador related code long enough to mock the object and let the migrations run.

I just found it interesting. You inherit a Rails project, you try to run migrations, they break in the middle. Your first thought is quite possibly, "Oh great, the last guy didn't keep his migrations up, now I have to do everything in SQL." Maybe not the case! This appears to be a completely innocent problem that crept in over time, and he would never have noticed it because it's not like he's constantly going back and building his database from scratch like I just did.

noreply@blogger.com (Duane Morin) — Wed, 25 Jan 2012 01:49:00 +0000

I want to share with you something that I wrote in an email back in 12/2004:

Imagine your favorite book. The sort of book that you always like to
have a copy of, because there's always something in there worth
revisiting. Maybe it's the Bible, maybe Shakespeare, maybe the Tao of
Physics. Or, maybe it just happens to be the text book for a course
you're currently taking.

Now imagine you have a thought on that book. Maybe it's an idea you
want to scribble down so you don't forget. Maybe it's a question.
Maybe you just want to talk to somebody about the book, but none of
your friends are into it. How do you find a community of people who
are interested in it?

What if that book was really an e-book, and that e-book had a wireless
internet connection? And, whenever you had such a thought or a
question, with a simple tap of the keys you could connect immediately
to a community of people exactly like you. I like the notion of a
"deep" book, it feels very alice in wonderland to me. Because you're
leaving the original intact, and that's key. You're telling somebody
"Look, we're not gonna get in the way of your enjoyment with this
book. But when you want us, just look a little deeper." The book
itself is merely a reader onto the larger, centralized service that
tracks all the comments, questions, and so on.

Right now, the idea would have to take the form of a PC-based reader.
It's really the only technology that can handle it. But hey, that
could easily mean laptops. What about PDAs? Pretty soon. Need a
better UI. And before you know it, ebooks really will take off.

How does this fit our model? Imagine the kind of content that can be
plugged in to that sort of interface. Who says it has to be a
straight blog/message board? Maybe you work up a quiz on a certain
subject, and you link it directly into the book. And charge people
for it. Not everyone will take it, of course, but some will. Or
maybe you're the author of another book on the subject. Maybe you're
teaching a class that uses this text, and you want to leave a message
for your students. The book itself not only becomes a walking
course on itself, it becomes an infinite number of courses on itself.
By grabbing a "deep" view of Shakespeare you open up the door to
everything you could possibly learn about the subject, all depending
on how you choose to navigate through it. On your terms and your time
line. Want to pay for some stuff? Ok. But there'll be free stuff as
well. You pick.

The question, as always, is revenue. Who pays? Will readers pay to
be a part of this service? Probably not 100%. But they might pay for
certain premium services. Imagine if J.K. Rowling offered sneak peeks
into her books to premium subscribers? And what about content
authors? Do they expect to be paid, or will they pay us to be a part
of the service? The latter is not so very far fetched. Because
basically what you're talking about is a built in community support
forum. All an author needs to do is enable his book to work with this
system and presto, a community might/can/will spring up, and he's
done. He can even make some money back by offering premium
services, as above.

We've finally reached the point where this is possible. Check out Subtext for iPad. I hate when this happens.

Rails : no such file to load -- devise/orm

noreply@blogger.com (Duane Morin) — Tue, 14 Jun 2011 15:04:00 +0000

If you're like me, you're getting this error while trying to get Devise set up. You google around and everything talks about mongoid or mongo_mapper, which you're not using.

Turns out the solution to this one is easy. Go into config/initializers/devise.rb and not this section:

# ==> ORM configuratio

# Load and configure the ORM. Supports :active_record (default) an

# :mongoid (bson_ext recommended) by default. Other ORMs may be

# available as additional gems

require 'devise/orm/'

See it? When I read that I assume it meant that I could leave this alone and get active_record support. Apparently not. Change the line to read : require 'devise/orm/active_record' and see if your problem doesn't go away. Mine did! :)

How'd You Get Your Job?

noreply@blogger.com (Duane Morin) — Fri, 27 May 2011 02:00:00 +0000

In my day job I'm tasked with building a job board. What that means is still pretty open. Never one to just copy what the other guys are doing, I've been on a research kick.

So tell me. How'd you get your job? Job board? If so, which one (if you remember)? Or was it more about the connections, the who-you-know? Right place, right time? Newspaper ad? Something else altogether?

I'm interesting in all kinds of input from all angles, regardless of what sort of job you have. If people are using job boards I want to understand what the most important features are. If they're not, I want to figure out if there's something I can build into one that will make them useful again.

I'll start - my last two jobs came when a former coworker said "Come work for/with me." In both cases I knew and trusted the coworker and made the move. Prior to that I had a trusted recruiter that I'd used on multiple occasions.

Who's next?

In Praise of Single Use Devices

noreply@blogger.com (Duane Morin) — Mon, 03 Jan 2011 16:32:00 +0000

There's an old marketing joke about the guy who went into the corporate office of the shampoo company and said "I can double your business with one word." That word? "Repeat."

Very related to this story has to be the first guy who decided that any device with a computer chip in it has to come with a game. Cell phones, calculators, MP3 players, what device of sufficient computing power these days does not have some sort of game? Even the early ipods had some sort of solitaire in them if I recall, as well as this "Name that tune" game that was actually pretty innovative since if use your own music.

Stop to think about it for a second. Why? Why does your phone need to play Blackjack?

The answer is simple - it's so that when you don't need to use your phone, you will still want to use your phone. "I'm bored. Hey, I know, my phone plays Solitaire, I'll do that."

Never forget that this is an invention intended to benefit the producer of the device, not the consumer. If there are two phones on the market, one of them does 3 things and one of them does 4 things, which one will most customers go for? Why, the one that does 4 things of course.

Well as I sit here with my new Kindle in hand, I'm having something of a Luddite moment. There's value in single use devices. Let's go back to that very first example, your stupid phone that still managed to play Blackjack on that tiny keypad. That moment when you said to yourself "Nobody's calling me, I don't have anyone to call...but my phone plays Blackjack, I'll do that." Imagine that the phone didn't play blackjack. What would you do? How about SOMETHING THAT DOESN'T INVOLVE YOUR PHONE? Read a book. Exercise. Go visit someone. Clip your fingernails. There's got to be 1000 things you could do besides play that game, but because someone put that game in your hand, you will play it.

I have a Kindle, not an iPad. Most people agree right now, at least from a distance, that multi-use devices like iPad will destroy single-use devices like Kindle. Why not, after all? Why get a device that does one thing when you can get a device that does 100 things? Well, because that device does 100 things, really.

In the week I've had my Kindle I've already finished one book (The Princess Bride), am half way through another (A Wrinkle In Time, which I am scanning to see whether my 8yr old is ready for it), and have made a dent in 4 others. I carry it with me, and whenever I have a moment I read.

If I had an iPad, what would I have done in that time? Yes, iPad has an ebook reader. Would I have used it? Of course not, and we all know it. I would have played Angry Birds every chance I got. New levels! New levels!! There are new Angry Bird Levels!!!

When a device only does one thing, you come to appreciate the quality of doing that thing. The Kindle could be a better reader, on a number of fronts. I'd like a light, but not at the expense of that crisp contrast it has. I'd like better text-to-speech, and better page turning. But will either of those make me say "Oh, well, see, the iPad is better, then." Nope, because I know that the iPad is too much in the wrong direction. The Kindle wants to do one thing better than anybody else. The iPad wants to do everything good enough. Sometimes everything is too much.

When Does “You” Become “We”?

noreply@blogger.com (Duane Morin) — Mon, 16 Aug 2010 18:46:00 +0000

I’ve noticed this every time I start a new job. How long does it take before you stop referring to the company as “you” (as in, “What did you guys mean to do over here…”) and when do you start using “we”? Twice now my new bosses have caught me and said “Feel free to start using ‘we’ any time, you’re part of the company now too.”

I think the distinction I make is that when I’m speaking of the efforts that came before me, I say “you”. After all, it’s not like I want to take credit for their work. I can’t say “Here’s what we meant to accomplish in version 6.1” if it came out before I was around, right? Once I’ve got new features added to the product I’ll feel more comfortable saying We.

Test Driven Learning

noreply@blogger.com (Duane Morin) — Wed, 28 Jul 2010 15:13:00 +0000

So, I’ve been out of work for about a month. During my time back in the hunt I learned just how big test-driven development, or TDD, has become. Quite literally every place I interviewed (for Java and Rails jobs alike) asked me what my experience with it was, my philosophy, and on and on.

And I had to answer, in each case, truthfully. I get the idea of TDD. I’m a fan. But never have I been in a place that’s truly adopted it. Usually the typical legacy excuses all get in the way – nobody wants to write tests for code that already exists, so there’s no good framework from which to build new tests, and it’ll take too much effort to make that happen when it’s easier to just continue writing code like they’ve always done. Not a good excuse, just a popular one.

Well, I’ve got a new job now and all the above is still true. However I think I’ve found a bit of a loophole in that logic that’s going to allow me to force feed TDD into the mix. I’m a new hire. There’s an existing code base. I can read the code all day long, and I can think I understand it, but how do I really know? Should my first real interaction with the code be when I’m making changes that could potentially introduce bugs? Into production? Not such a great idea.

So instead, as my first project, I integrate JUnit (it’s a Java shop). Then, I start writing unit tests. I read an existing function of an existing object, I think I understand what it’s supposed to do, so I write a test for it. One of two things is going to happen. Either the test will work, in which case my understanding of the code was correct, or else it’s going to fail, in which case either my understanding of the code was wrong, or else the code itself was wrong.

The end result is that a) I know the code does what I think it does, b) I potentially uncover bugs that would have otherwise been uncovered the hard way, and c) now unit tests exist for the next guy. Win win win.

I call it “test driven learning”, and I dub it TDL. Pass it on.

What Do Drivers Do In Computers?

noreply@blogger.com (Duane Morin) — Mon, 24 May 2010 14:32:00 +0000

I've been working on a project recently where I get to look at the kinds of questions people ask of Google, when phrased as a question. For any given subject I get a glimpse of what people want to know, based on how they ask it.

Today's question is a very popular one on the subject of Windows device drivers :

What do drivers do in computers?

To really understand what drivers do in computers, you need to stop for a second and consider what's really going on under the covers. You have "hardware". That's the physical stuff - your memory, your CPU, your hard drive, your DVD player, your microphone. Then you have "software". That's the programs and files that live as bits (ones and zeroes) in the memory and on the hard drive. The magic of how computers work could be summed up very briefly as, "The software tells the hardware what to do."

When you load a DVD, there are some physical steps. You open the tray, usually by pushing a button. You put the DVD in, and then close the tray either by pushing the button again or giving the tray a little shove. And then your movie starts playing. But that's where software comes in. How exactly did the computer know that you put a new disk in the drive? The DVD player itself has just enough smarts to know when that tray door opens and closes. So when you close it, a signal is sent down the cable that connects player to motherboard. The CPU, which listens to everything going on around it, says "Ok, I just got an XYZ signal from the PDQ cable, what am I supposed to do with that?"

And that's where your "device driver" comes in. The driver is a tiny little program whose sole purpose in life is to raise its little digital hand and say "Me! Ooo! I know how to handle XYZ signals from the PDQ cable!" Ok, I'm being a little silly, but that's really what it comes down to. Signals come through that cable, they go over to this little driver program, and the driver speaks a language that the rest of the software (Windows, typically) can understand. So that XYZ signal is might translate into something like, "A new DVD has been inserted, pop up that dialog box that asks the user what she wants to do with it."

Before Windows, you actually would have had to do this all yourself. When you bought a new printer or even a mouse, it would come with a disk that contained these drivers. You'd copy them over to your hard drive, and then have to modify the startup programs to let the computer know these drivers were available. One of the reasons that Windows took over the PC world was because, in the words of one famous industry analysis, it was just "a big bag of drivers." That was the whole point -- now you don't have to deal with drivers anymore, they're just there.

Except when...they're not. New devices come out all the time, it's impossible for Windows to have every driver for every device. Worse than that, sometimes these driver programs have bugs in them and need to be updated. So it's important to recognize the role these drivers play, and keep them updated. Modern versions of Windows have an Update Service that will let you know when it's time to download new drivers. Or you could look into a service like DriverAgent.com which will do all the work for you, scanning your computer to determine which drivers are out of date and then offering you an easy way to download them all in one visit.

Verizon DVR Hack : Deleting Multiple Episodes

noreply@blogger.com (Duane Morin) — Sat, 22 May 2010 02:20:00 +0000

If you've got Verizon DVR and you're like me, particularly if you've got the home media option for sharing between boxes, you'll often get 5 or so episodes of a tv show that you've watched but haven't deleted (because you watch them on other tv's where you can't delete). And perhaps you've noticed a rather annoying issue with this where, once you're down to one episode, what happens is that it'll suddenly kick you back to the list of programs (because now there's no longer a folder for that show) and, because that episode is a few days/weeks old, it's buried at the bottom and you have to go find it.

Know what I'm talking about? You go into the Grey's Anatomy folder, and delete one. This is already complicated enough because for Verizon you need to hit like 4 buttons - Right arrow to get into folder, then Enter to select episode, then Down two times (or three, if you're half way through the program and have "Resume Play" option) to get to Delete, and then down for Yes, then enter. That's alot of keystrokes. So you have to do that for N shows. And when there's that last show left, suddenly you have to do even more because now you have to scroll through your shows to go find where in time that last one is hiding.

Ok, enough of that, want to know the trick? Skip the most recent episode. Instead of deleting them top down, move to the second most recent one, and then delete them all. What will happen when you only have the one most recent one left, it'll kick you back to the top level -- but look at that, your cursor is still sitting on the show you want. Delete.

What We Need Is Psychic Search

noreply@blogger.com (Duane Morin) — Mon, 17 May 2010 13:37:00 +0000

Sunday afternoon, out having lunch at a noisy restaurant. I hear part of a song I recognize, enough to say to my wife “Good song,” but then we can not hear the rest. For the life of me I can not think of the song, and it drives me nuts all the way home. Easy enough to Google lyrics, right? Wrong, the sequence I heard was nothing but harmonizing, just background singers singing “Ahh ahh AHHHH….ahh ahh AHHHHH….ahh ahhh AHHHHHHH” where the third one has gotten higher, and then presumably it moves into the next verse which is what I can’t remember.

Try googling “song opening ahhh ahh ahhh” and see what happens. I got lots of hits for Led Zeppelin’s Immigrant Song, which I think is funny because it is both an accurate answer to the question, and also not even close to what I meant. There were also lots of hits on modern dance club songs, but I’m old and don’t really know those enough to say one way or the other.

So I post on Facebook, and my friends begin guessing. One says “At the Hop”, that old 50’s tune that starts “bahh bahhh bahhh bahhh, bahhh bahhh bahh bahhhh At the hop! Well you can rock and you can roll and you can stomp and you can stroll at the hop….” (or whatever the words are). Again, an accurate answer to the question pretty much, but not at all what I was looking for.

Undaunted I started googling for “Hum a few bars” hoping to turn up one of those search engines where you can sing into the microphone to guess your song. I find nothing but hits from the 2006-08 era, none still active, so I’m wondering if I searched wrong. I do try the iPhone app “Shazam”, but it can not recognize the tune so chances are I’m singing it too poorly.

I do strongly think that it is a 60’s tune, what I described to my friends as “hippie era” :). Running out of ideas I google “summer of love music” and find a list of hits from that year (1967, I believe?)

And there it is.

Groovin, On A Sunday Afternoon by the Young Rascals. My “ahh ahh AHHHH'” sequence is right there in the middle.

Now what I need is for somebody make the connection from my original query to the query that ultimately found me the answer. We’ll make millions. :)

UPDATE : Adding to my own post because the more I think about this, I think it’s not that far off. I knew that it was a 60’s tune, right? Imagine if for my music search engine I was able to start there. Just like the other folks I mentioned knew that there’s was a dance club song, and I knew mine was not. From there maybe the engine could offer up some guiding questions like the tempo (this one is pretty slow), single or group (group), background harmony (definitely)…and so on. I know that recommendation engines like Pandora break songs up into N different parameters and then match, saying “If you liked this song with these values then you’ll probably like this other song with similar values.” But what if instead of doing that you offered up an interface where the user could just select those values on sliders? “I want strong lead vocals, meaningful lyrics and a fast drum beat.”

A slow 1960’s song with a harmonizing group certainly wouldn’t guarantee finding my song, but it would at least help narrow the search, and that’s huge.

Variety in User Interfaces

noreply@blogger.com (Duane Morin) — Wed, 12 May 2010 19:59:00 +0000

Once upon a time Robert Heinlein wrote a book about a car with a voice synthesizer. You talked to it, it talked back. When it started speaking in phrases that the driver had never added, he chalked it up to one of his friends messing around with the database. Eventually it got to the point where he wasn’t sure that the car wasn’t simply sentient.

I think that there’s lots of room for voice applications in today’s user interfaces. Take the GPS in your car. I expect that it talks. You’re not supposed to read the screen while you’re trying to drive, right? So let’s think about how it talks.

One way is to have a good solid database of what it can say, including names of all the streets. That’s very handy, because instead of just “Turn left” it can say “Turn left on Quantum Street.” But the downside is that you’ll be limited in other ways, and there’ll probably be very few ways that the computer phrases certain things. It’s strength lies in being able to read you what is effectively a bunch of proper names.

The other approach to take is more generic, and not speak the street names. “Turn left” and you rely on the layout of the road to know that you’re taking the left it wanted you to take. Of course there is still an image on the screen, which can have the street name, so it takes half a second to look down and see whether you’re going where you are supposed to.

BUT! Once you substantially drop the size of the database by focusing only on a handful of “Turn left” / “Turn right” types of phrases, now you’ve opened the door to downloading your choice of voice. You could have a celebrity read the 100 or so phrases, zip it up, and there you go.

Even better, you could be given instructions for how to do it yourself. My TomTom has this, but I’ve never taken advantage of it. I do have John Cleese from Monty Python doing my directions, but I’ve never sat down to record the kids and wife doing it (“Turn left here, turn left, here, turn left! You’ll kill us all!”). The instructions were fairly complicated, explaining how you had to get each soundclip exactly the right length for it not to sound broken. Forget that.

What I’d love to see, though, and this gets be back to my original story, is more variety. The TomTom worked by saying “Ok, provide a folder consisting of 59 sound files labelled as follows.” There’s a single sound clip for “end of trip”, so no matter how many times I use it, John Cleese is always there saying “You have reached your destination. You may get out now, but I’m not going to carry your bags.”

What if instead of 59 sound files I was told to provide 59 directories, and in each directory there could be as many files as I wanted? And whenever the device needed to play something it would instead just grab a random file from the appropriate directory?

That would be AWESOME. For the most common ones you could sit down and record a good couple of dozen different ways to say it. Maybe set something up that is user contributed so you just plain don’t know how many different sounds there are? Or set a priority on them so that most of the time you get “Turn right” but once in a hundred you get “Turn right, moron.”

This approach, of course, would thus work for any device that talks back to you. Your answering machine, maybe?

Type Type Revolution?

noreply@blogger.com (Duane Morin) — Wed, 12 May 2010 18:52:00 +0000

http://blog.shakespearegeek.com/2010/05/shakespeares-thighs.html

So, anybody know how to hack a “dance mat”? I’ve seen a bunch of games with this type of input device where you jump up and down on it – my kids even had a simple little “Strawberry Shortcake And Friends Dance Party” one if I recall.

Take one of those and build a game up around it where instead of dancing to music, you’re typing like you would on a cell phone.

Improve your spelling, your accuracy…your appreciation of classic literature. A game could be made out of it, where you have to answer questions but instead of jumping to A-D you have to spell out the answer.

I would totally buy that. Somebody, if they haven’t already, needs to make a generic “dance mat” controller for Wii or something (balance board does not count!) that games can be written to. Or maybe even a USB one so you can use it on the PC?

MongoDB Ruby Driver : nToReturn=100?

noreply@blogger.com (Duane Morin) — Tue, 11 May 2010 13:45:00 +0000

The other day I noticed in my logs that I kept getting my very large (6 million rows) result set back 100 rows at a time. The log kept saying getmore, getmore, getmore… with nToReturn=100 set as a parameter. So it was doing the right thing, but where was that 100 coming from?

A post on the user group gave me the answer – the default “batch_size” option for find is set at 100 in my version of the Ruby driver, when it should be set at o (which is the signal for the driver to just figure out the optimal size). So you can override this by forcing batch_size:

@results = @objects.find(@conditions, {:batch_size=>0})

You should notice an improvement in performance.

Mongo Good.

noreply@blogger.com (Duane Morin) — Mon, 03 May 2010 20:31:00 +0000

Been working on an interesting project lately where I was handed some 6 million XML files with no meaningful schema, and told to make a warehouse out of them. Each XML file also happens to have several hundred elements in it. With no schema it would be a nightmare to try and make one, never fully sure whether you'd allocated enough space, whether just because your sample of this field yielded integers means that it always has to be an integer .. stuff like that.

I tried a very simple "Elements and Attributes" schema that is capable of loading any XML file. Basically each Element and each Attribute get a row in the database. This works fine for smallish data sets, but mine quickly blew up into billions of rows and was no longer manageable.

Then I stumbled over the "No SQL" movement, where we quite literally throw out decades of knowledge about how to set up a SQL schema of tables and rows, joins and where clauses .. and replace it with something more document-driven, no schema. This sounded perfect, since that's exactly what I had.

Enter Mongo DB. With a little help from Ruby, my loader script for 6 million XML files now looked like this:

Foreach XML file:
  Load it via REST web service
  Parse it into a JSON object
  mongo.insert(object) Done!

Done! Now if I'm hiding the name of a software vendor somewhere in the middle of one of those objects, like say {"machine"=>{"software=>[ "instance"=>{"name"=>"Microsoft something" ... }]}} and I want to find them? All I need to say is find("machine.software.instance.name" => /Microsoft/) and presto, I've got back my filtered list. I can add a count() to get the total number of matches, or an each(...) or map(...) to process the list, or basically anything else I might want to do. Of course I can also add an index on it to keep the performance high, too.

I'm finding this NoSQL stuff fascinating. Takes a little while to mentally get around everything you've learned about normalizing a schema. Instead of a dozen tables all crosslinked with each other, I've got one "collection" that is nested in a variety of ways. But I quite literally spent 0 effort on schema design.

I haven't even begun to touch on things like the built in sharding, which enables you to scale across multiple instances without having to worry about which server hold which portion of your data. Another player in this space, Cassandra (from the Facebook people), is most well known for being great at that. We looked at Cassandra but it had some schema setup required that I was not able to complete. With Mongo I just inserted everything and now I'm free to query at will, no matter how complex the original key structure was.

I hope to write more here as I come up to speed on what is obviously the next big thing in the world of database design.

How Geeks Pick Battles

noreply@blogger.com (Duane Morin) — Fri, 12 Mar 2010 19:21:00 +0000

My son’s got a birthday coming up, and we’re having it at one of those local gymnastic places. On their website is a “print your invitations” link, which turns out to be a 2 page PDF. More specifically it is 4 rectangles – two front, two back – intended to be printed out double sided on card stock, then cut out and mailed. Cool enough.

I do not have a double sided printer, though, so I print out both pages and take it down to the local copy place, where I realize that…they don’t line up. Damnit. It’s clear what’s gone wrong, the people who cobbled this thing together put a 1” border from the left on both pages – but since page 2 is on the back they should have measured 1” from the right.

The wife tells me no worries, she will just buy some invitations at the store. I assure her that this is not a problem, I just need to muck with the margins on page 2.

After discovering that your typical PDF reader does not let you play with the margins on a per-page level like that, I switch over to a PDF editor – InkScape, to be specific. Technically, though, InkScape is a graphic editor with import/export PDF, so I can only edit one page at a time. Fair enough.

I will save you lots of cussing, but fast forward a half dozen sample printouts when I realize that it’s not that they are not lined up properly (they are not), but that one of the front rectangles is a different size than the others! This is throwing everything off. That’s seriously annoying.

But! I’m in an editor after all. Select all. Ungroup. Select background rectangle. Resize. Done. :)

I spent maybe an hour and a half on that. At any time I could have just skipped the whole thing and let the wife buy some new ones, but part of being a geek is taking the battle for supremacy very seriously, and not letting the machines win. If the only thing standing between a problem and a solution is something involving technology, I’m not about to give up easily. And it has been known to drive me occasionally crazy. But I wouldn’t change it. >:)

Thoughts on iPad

noreply@blogger.com (Duane Morin) — Wed, 27 Jan 2010 21:32:00 +0000

On the one hand, Apple’s new gadget is a big fat iTouch. After all, what’s the real difference between an iPhone and an iTouch to begin with? Voice calls via 3G, for one. Take those out and the only real difference is the video camera, which is neglible since they can put one in at any time.

Here is why the best ebook reader in the world will still not get me to buy one, and it’s not because I’m a book nut who loves the smell of paper. Ready? You can’t rip a book.

Go back to when iTunes was new, and you could buy music from Apple. Not all music, sure. But what were your options at that point? If you already had the music in a different digital format then maybe you could port it. Fine. But the big killer idea was that you could get the CD and you could rip it. Convert physical into digital. And then you win.

What’s the comparable example for books? I don’t read a great deal of brand new bestseller types. When I pick up a book it’s either a non-fiction thingie out of the Shakespeare section, or its a classic scifi that I missed the first time around. Sometimes, to be fair, a new scifi tome or Shakespeare novel will come out and I do pick those up off the “New Releases” shelf, but those are only a portion of my purchases.

So, what am I to do? The book I want is not available in iPad format. Is it available in some other digital format because somebody else did the work for me? Maybe. Sometimes. Not often.

So then what? I can’t meaningfully rip a book. I am well aware that there is technology that can do it, and Google’s pioneering some work in that area. I’m talking about a single home user trying to turn his physical book into a digital version. Can’t be done. And that kills me. If you told me that you were installing a book ripping machine in every Apple retail store, and that I could walk in and get a genius to rip my books into iPad format for me? I’d be all over it. I’d systematically rip my whole collection (depending on much Apple charged me for the service :)).

Where I do see a future for these devices is a few years down the line when my kids go to college. I see them carrying a single device, instead of a backpack full of text books. All their text books are right there in a tablet. All the updates, too. And homework and study guides, and question and answer forums. It’ll have a keyboard for speedy word note taking, but you can also draw on it for diagrams. It’ll have net so you’ve always got Google and Wikipedia, but it’ll also have instant message and SMS because you can’t live without those.

The big question will be how you keep the difference between “study machine” and “cheating machine”. If you’re allowed to have it, you’ll be able to cheat with it. But if you’re not allowed to have it, you’ll lose all your reference materials. I’m not sure how that will play out yet. I’d love to imagine some sort of signal blocking device so that once you’re in the classroom your teacher could assure that all your devices switched to local mode, but I don’t see that happening.

Rails : Passenger Is Deleting my Sub / RailsBaseURI Prefix

noreply@blogger.com (Duane Morin) — Mon, 11 Jan 2010 16:12:00 +0000

Ok, this was a weird one. I’m not going to jump into a lesson in how to make Phusion Passenger work to deploy Rails applications, but suffice to say that I had a situation working where we have multiple controller entry points all in to the same application, so I had this:

RailsBaseURI /foo
RailsBaseURI /bar
…

/foo and /bar, then, are soft links from the docroot over to the Rails application’s public folder. The dispatcher is smart enough to say “Ok, based on the public folder I know where the Rails app lives, and once I know that I can go invoke the appropriate controller.” So a request for /foo would expect there to be a foo_controller.rb file.

That’s been working fine for me in production for months.

But this week I got the most annoying problem – I added /baz, and kept everything else the same – the soft link, the RailsBaseURI, the baz_controller file. But now all of a sudden I’m getting :

No route matches "" with {:method=>:get}

And, let me tell you, it is very hard to Google for empty strings. I got plenty of examples of people having trouble with No route matches “/foo” or “/approot”, but in each case it was at least telling them their route – mine was being eaten!

On a hunch I would try /baz/baz – and it worked fine. I tried /baz/ (note the trailing slash) and suddenly my “No route matches” error would turn into “/”. Aha, I had it – somehow my prefix, which I’m relying on to handoff to my controller, is being eaten. So by the time Passenger spots “/baz” and hands off to Rails, it just gives “” – and Rails in this case doesn’t know what to do with “” because I don’t have a default route set up.

The problem appears related to an update to Rails, in conjunction with Passenger. Many people pointed me to this fix:

config.action_controller.relative_url_root=”/baz”

I dismissed this out of hand, though, because of my multiple controllers. What value would I put in there? I did not want my app to default to one particular entry point.

Turns out I dismissed it too soon. What I did was add this:

config.action_controller.relative_url_root=’’

And guess what? Success! The best way I can take to interpret this is that it’s telling Phusion “Hey, the value in this variable? I need this, don’t kill it.” For instance when I made it a “/” value I suddenly started getting errors saying that I had no “baz” route (see how no slash?) So by saying that I have no relative url root, I’m saying “It’s not safe to delete anything that came in as if it were extra baggage, just leave it all in place please.”

That may be a painful explanation, particularly to anybody who understands the guts of the Passenger/Rails handoff, but it’s working for me now and I’m abiding by my “If I couldn’t find the answer by googling that means other people have the same problem” rule and posting for the next guy.

Verizon FIOS Disappoints … Again.

noreply@blogger.com (Duane Morin) — Tue, 03 Nov 2009 03:52:00 +0000

When I heard FIOS was coming to my town, I got in line. I couldn’t wait to get what I felt was the best technology in the market. True, I’m no fan of Comcast. Competition is good.

The core technology that gets me my signal? Fine, can’t complain. As far as I know it’s even faster than I had before. I have no complaints.

It’s the extras. The user experience. For instance? Take the DVR option. I have several TVs, and a common scenario is to start watching some primetime show, like a Grey’s Anatomy or something, downstairs in the main family room with HD. But often my wife will say “Can we finish watching this upstairs?” Verizon has a Home Sharing option that allows exactly this. Very handy.

Except for the limitations. Like, for instance, it won’t “downscale”. So with a standard tv upstairs I have to record all my programs in standard (even though the box itself is a HD box) if I ever want to stream them. I’ve learned to live with that. It also can’t be programmed from the remote boxes, so if you finish watching most of your programs upstairs on the bedroom tv you can’t just hit Delete – you have to remember, next time you’re downstairs, to purge all the old shows you’re done with. An annoyance, but you can live with it.

The latest disappointment though comes with the new Media Manager for Mac. I’ve got all the family photos on a Mac Mini (which, also, is connected up to the HD tv in my family room). That’s also where I rip the kids’ DVDs, and manage all my iTunes stuff. So when they said “Run this software and your iTunes media will show up on your FIOS boxes” I thought, “Awesome.”

Except for one small problem – it only works on the DVR box. In a “Home Sharing” setup, you only have 1 DVR box in the house, and the rest are just dumb satellite player boxes.

See the problem? In order to share recorded television shows, you need – are required to have – exactly one Home Sharing DVR. If you had a DVR box in every room, you could get Media Manager content on every DVR box. But then you couldn’t share recorded tv programming!

That’s what bugs me. I appreciate having to work the kinks out of the system. But here you’ve got a clear cut case of Verizon setting up mutually exclusive features, and then leaving it up to the customer to realize that. I’m not even misinterpreting it, either, as I had a lengthy conversation with Verizon Support over Twitter when I discovered this problem and they confirmed that yes, the options are mutually exclusive. Get multiple DVRs to share media manager content, or get one DVR to share recorded tv shows, but not both.

I hope they get some of these major problems worked out. I doubt it, given the underlying architectural problem (namely, the remote boxes are without a hard drive and thus doomed to be brain dead). But, still. Hope remains.

Who Is Blocking / Locking My Windows Clipboard ?

noreply@blogger.com (Duane Morin) — Tue, 27 Oct 2009 17:29:00 +0000

Ever had this problem? Suddenly you realize that cut-and-paste doesn’t work. You try it a couple of times to make sure you’re not being an idiot, but nope, something’s wrong. You try it in several apps, and it’s just plain stopped working.

Turns out there’s a reason for this, and it has lots of blah blahs. Eventually you should end up over here where someone was nice enough to create an unlocking utility. I just tried it, works great.