Heedspin

Unconvincing Opinions

2012-12-24T00:00:00+00:00

The Sandy Hook Massacre has inspired an abundance of opinion pieces on gun control. A majority of these publications are what I’ll call “unconvincing opinions:” articles on controversial subjects that make no attempt to engage the opposition. While readers who already agree may feel justification, those with opposing views will likely just become angry. I don’t see the purpose of writing an unconvincing opinion, unless the goal is to stir opposition.

This unconvincing post on NorthJersey.com is what inspired the present post. At one point the author wished “the NRA would present sober and thoughtful solutions.” But in the same post he calls for the reinstatement of every past gun control law as well as revising the second amendment of the U.S. Constitution. Gun control proponents may love this type of talk, but those against will likely be moved to fight. I hope the author reads the post comments and recognizes the irony.

Responsibility To Persuade

I can see why unconvincing opinion pieces exists. Articles like the NorthJersey post appear to be emotionally inspired. This is as it should be, since purpose of emotions is to move us to action. Furthermore, inspiration is perishable, so wise writers act while the blood is still boiling.

But I’d like to suggest that, in addition to the other great responsibilities inherent to journalism, writers have a responsibility to persuade. On topics of controversy, opinion authors have a responsibility to pursue peace. Even if an author’s opinion is 100% correct, what good is it when presented such that the opposition can’t hear? Indeed if you hold the truth, you have a responsibility to share it, and to do so effectively!

Though I never saw the 2006 movie The Last King of Scotland, I never forgot this scene from one of the previews. Just knowing the truth is worthless! It’s your job to be persuasive!

How To Persuade

To persuade adversaries, a good first step should be to understand their premise. Bryan Caplan says: “If someone can correctly explain a position but continue to disagree with it, that position is less likely to be correct.” He also quotes John Stuart Mill who says: “he must know [the opinions of adversaries] in their most plausible and persuasive form.”

In this excellent post on gun control by Ross Douthat (pre Sandy Hook), he starts out by recognizing the validity of an argument for gun control. Then he finishes with an argument against gun control written almost as a suggestion for consideration. Though the article has a pro-gun conclusion, it’s accessible from both sides of the controversy. In facilitating mutual understanding, this is a model example of steady persuasion on a subject that typically only stirs tempers.

Demand To Be Persuaded

My hope is that more writers would engage in persuasive discourse. Perhaps they exist, but are not promoted to the top of the news sources I frequent. If news sites prioritize opinion posts based on popularity, does the blame return to us readers? Convincing and persuasive writings are not promoted because we do not demand them (and do not share them on Facebook and Twitter)?

Let’s be like Forest Whitaker in The Last King of Scotland and shock our writers with their responsibility to persuade. In fact, I think I’ll use this response next time someone offends me with an unconvincing opinion: “You may be right, but you have failed to persuade me.”

What I learned from Paul Ryan

2012-08-24T00:00:00+00:00

Look at all those people!

I recently had the privilege of helping host Vice Presidential Candidate, Rep. Paul Ryan for a rally at SMT Inc. in Raleigh, North Carolina. Politics aside, anyone witnessing such an event would be impressed. Here are some things I learned from Paul Ryan’s visit:

You Can Do A Lot in 2 Days

I’m not sure how long the Republican party was considering SMT as venue, but we got official word on Sunday night for a Wednesday event. SMT and Creative Visions turned a regular factory floor into a political rally venue in two days. Stages, bleachers, rafter lights, a full sound system, and everything. The whole event went perfectly. I didn’t see a single hiccup.

Cleaned and ready

Setting up

Just add people!

By the way, while these guys were creating a venue from scratch, all of the Republican staffers were working past midnight planning every detail of Paul Ryan’s visit: Which hallway does he walk down, who does he talk to, for how long, and on and on. One of the staff put it to me this way: “We’re the frantic duck feet underwater; the event is the calm duck”. So… that would make me the slimy pond water? I like it!

Political Candidates Work Hard

I understand not everyone will agree with Rep. Paul Ryan’s politics. But I’m here to tell you: the man works hard. The Secret Service cleared me for access into the same areas as Congressman Ryan, so I witnessed his life for about 5 hours.

Paul Ryan started the day in Virginia. He arrived at SMT in Raleigh, NC and immediately did some photos with VIPs and a couple on-camera press interviews. Then he gave the big 30-minute speech to thousands of supporters. Immediately following the rally, he spent about 2 hours meeting with several teams. At one point he emerged from a room and joked that they had successfully stuffed more information into his head. I spoke with one of his briefing staff and learned that congressman’s head had just been filled with the inner workings of the United Nations. Enthralling, I’m sure.

After all the meetings, he took a few more VIP photos while the entourage began moving towards the exit. Almost out the door, he stopped and asked an aid who he could thank before leaving. Only a coworker and I were around, so he came over and thanked us. I said “you’re welcome” and then peed my pants.

And in a flash he was gone. And all the Secret Service and police disappeared with him. Two hours later the stage was gone too, and I think I saw a tumbleweed blow across the factory floor. I was in bed and asleep an hour after that.

But Rep. Paul Ryan wasn’t done. They stopped at a lemonade stand on the way to his big fund raiser that evening. One of his aids told me he had a 2-hour phone call scheduled for the drive after the fund raiser.

When does he sleep?

The Secret Service Is Awesome

Probably my favorite part of the whole event was hanging out with the Secret Service. These guys are cool. And thorough.

They have a plan for everything from a power outage to a chemical attack. They bomb-swept this giant building several times. Every nook and cranny was inspected. As they explained to the building manager: “If we find a locked door, we’re breaking it down.” Once all the rooms were swept, if you had to enter a room that would be used by the Representative, you were escorted.

While doing a tour of the building, one of the agents pointed at this sign and laughed at me:

Unless you're the SS!

I also learned that the Secret Service’s “primary investigative mission is to safeguard the payment and financial systems of the United States”. One agent said his office had a record number of fraud arrests this year. I had no idea!

Maybe it’s a coincidence, but we started having intermittent network issues two days before the event. All the problems stopped after the event and have not recurred. Maybe those dudes have some serious computer hacking skills.

Men In Black

Bye, See You Later…

Now maybe I’ll know better next time I see a campaigning politico and think to myself: “I can do that!” I most assuredly can not. Good news: I want not!

Thanks for reading.

Susan Rothecker from SMT introducing Paul Ryan

Using Any File Format for the Made2Manage CAD Images

2011-11-10T00:00:00+00:00

The Made2Manage (M2M) ERP system lets you attach 3 CAD Images to your item master. Like so:

These image locations get copied to jobs in production. Like so:

The problem is that M2M will try to open these CAD Images using a single application that you configure in your user management. The assumption is that we will only put CAD files there and so we just need to tell M2M where our CAD program is.

We want to put PDFs and images and CAD files there. If you’ve read this far, you probably do too. So here’s what we put in our User Management:

We’ve created a little start.bat script that will launch whatever we want. You ready for the brilliance of start.bat? Brace yourself. Seriously…

Yup. It’s that awesome. The entire file contents are “%1” without the quotes. I’m no windows expert, but I think it’s the equivalent of just double-clicking the file. Let windows figure out what app to launch to view your file.

That’s all. I hope this helps.

M2MHub: Open Source Reports For Made2Manage

2011-09-27T00:00:00+00:00

I work with a few companies that use the Made2Manage ERP system. In short, building custom reports in Made2Manage (M2M) is difficult. As a software engineer with a lot of web development experience, I find it discouraging how difficult custom reports are for me. So, after struggling to master M2M, I’ve finally given up and started writing simple web-based interfaces to the M2M database. In the hopes of finding others who might be going down this road, I’m providing m2mhub as open source. Though it is not a turn-key application, I would be interested in collaborating with others to make it more useful.

The Made2Manage Group on LinkedIn

I had earlier created an m2mhub google group to provide a forum for anyone to collaborate. However, I've learned that the Made2Manage group on LinkedIn has more traffic than anywhere else. With less that 2,000 companies using M2M, the world is not big enough for more than one forum. So please, join the Made2Manage group on linked in.

http://www.linkedin.com/groups/Made2Manage-1793953

M2MHub Overview

At present I have relatively little built. I am not approaching any report in a comprehensive manner. Instead I only write specific interactions that I know will be useful. So, for example, there is no customer search yet. That hasn’t been as important as searching the item master.

Strangely, the most powerful part of this app is that I made almost everything clickable. That makes dashboard of recent sales orders (or quotes) a great launching point. You can click on the sales order, or the customer. And from either of those pages, you can click on the item. The initial goal is to make all sales information very easily accessible. This is surprisingly difficult in the M2M app itself.

So here’s what I have so far:

Quotes - This is simple crud around quotes. The index page is sorted in reverse chronological order. Quotes are more useful on the home page (dashboard) and on the customer view.
Sales Orders - This is also simple crud around sales orders. Same as quotes, these are more useful in the context of a dashboard or viewing a customer.
Customers - The customer show page is intended to display as much reference information as possible about a single customer. This page should be useful to sales and sales engineers when speaking with a customer. It contains every previous sales order and quote. In particular, having a quick table mapping from customer part number to company-specific part number has been useful to sales engineers.
Items - At this point items page is just a search. It searches by company and vendor part numbers. This has been been useful as a reference for engineers to look up vendor part numbers. It has also been useful for the shipping receiving department to find a company part number when a vendor only includes their own part number in a shipment.
Home Page Dashboard - Right now the home page shows recent sales orders and quotes. This is just the quickest and easiest dashboard I could think of.

Connecting to SQLServer

The hardest part about any of this is just connecting your Rails app to a SQLServer database. If you get discouraged, just remember this: “You can DO IT! Victory will be sweet.” I have a few more details in the m2mhub github readme. But a good starting point is here: tiny_tds and activerecord-sqlserver-adapter.

Basic Architecture

Our M2M databases run in our building. I believe this is a typical setup for M2M. So I setup a small Ubuntu server behind our firewall and give it a public IP address. Though I’m a big believer in the cloud, I don’t see another way to do this. I feel ok opening up HTTPS access to my Ubuntu box. I would be much more nervous opening up access to our M2M database to some web app in the cloud.

Future Plans

I’m planning on doing something with RMA’s and integrating with Lighthouseapp. I’m also planning on doing a custom shipping report. It would be great to have other people writing some custom reports so we could share code. Join me!

Shareable and Secure Notes and Passwords

2011-09-05T00:00:00+00:00

My wife and I share files via dropbox. I’ve just set us up using encfs, Notational Velocity, and Dropbox to keep important files (with passwords etc) both encrypted and easy browse on our macs. I don’t think it will last, and I don’t recommend others try setting this up. But if you must, here’s what I have done.

Install encfs

We’re on Snow Leopard and Lion. Installing encfs was not easy. I prefer DMG installs. When I can’t find those, I first try homebrew. Homebrew has a recipe for encfs, but it wasn’t playin. Something about “dyld: Library not loaded: /usr/local/lib/libintl.8.dylib”. I had to fall back to macports.

sudo port install encfs

The underlying magic that lets developers play with the macos filesystem is in flux. You’ve got the MacFUSE incumbant, and new Fuse4X and OSXFuse on the scene. I think MacFUSE is end of life. (There are also some alternate builds of macfuse out there). They’re all in various stages of support of OS X Lion. It’s confusing.

I think the macport of encfs uses Fuse4X. I have some stability issues (below) that I suspect are from Fuse4X. But I’m happy it mostly works.

Once installed I recommend playing around with encrypted folders by following encfs’ extended intro. In the end you’re going to do something like this:

encfs /Users/myusername/Dropbox/.encrypted /Users/myusername/unencrypted

If you don’t have a Dropbox folder you’re doomed. You can’t be helped.

We’re going to tell Notational Velocity to store it’s notes in the unencrypted folder. If you’re confused about how all the encryption works, my suggestion is to just play with it. It’ll make sense when you see it.

Notational Velocity

Notational Velocity is a note taking app. Go to the website and read all about it. Like many others, I use the nValt fork. No reason, really. I just follow the cool kids.

Go install it and play around. Make sure to configure it to store separate txt files that are NOT encrypted. You’ll see what it’s doing pretty quick. When you’re ready to party, configure Notation Velocity to store it’s text files in your unencrypted folder.

By the by, it’s not documented well, but you can import tab separated files into Notation Velocity. The first column is the title. The remaining columns are the body on a new lines. So if you have a bunch of existing notes you want to import, you can create a .tsv file and do them in one swoop. Or you could just create a bunch of text files and put them in your unencrypted folder.

The Bootstrap Challenge

One of the challenges with this whole thing is that you need encfs to mount your unencrypted folder before Notational Velocity starts up. If NV gets its grubby hands in your unencrypted folder before encfs has started, then NV will create default files and just be confused. And encfs needs its password, so this step may or may not need to be interactive.

Some folks have tried using a mac os login hook. I played around with launchd LaunchAgents, which are pretty cool. In the end, those approaches scare me. If something is awry with your setup, your login is hosed. You have to take extreme measures to halt the normal login process.

Instead, I chose to use the greatest technology ever developed by mankind: AppleScript. That’s right. Fear me. I really just wanted to learn a little bit about AppleScript. It’s a horrible language. But it’s working, so whatever.

All it does is prompt for a password, mount your encrypted folder using encfs, and then launch Notation Velocity (or nValt in my case). All the other code in there is basic error checking to make sure it doesn’t try to mount the folder if it’s already there. I tried to make it fancy with KeyChain, but failed on Lion.

on run
  set home_directory to (system attribute "HOME")
  set encrypted_folder to home_directory & "/Dropbox/.encrypted"
  set unencrypted_folder to home_directory & "/unencrypted"
  if (fileExistsAtPath(home_directory & "/unencrypted/.mount_verification")) then
    # Already Mounted
  else
    if not (directoryExistsAtPath(encrypted_folder)) then
      display dialog "Can not find Dropbox folder"
    else
      if not (directoryExistsAtPath(unencrypted_folder)) then
        display dialog "Can not find unencrypted folder"
      else
        set user_password to getPassword("Encrypted Folder")
        do shell script "echo " & user_password & " | /opt/local/bin/encfs -S " & encrypted_folder & " " & unencrypted_folder
      end if
    end if
  end if
  tell application "nvALT" to activate
end run

# I originally wanted this to look into keychain before prompting. However, I was
# only able to get it working on Snow Leopard. Keychain Scripting is gone on Lion.
# The interwebs say to use the security command line. That works, but throws noisy 
# errors if the password isn't stored there. So in the end, I'm removing support for 
# keychain. Thanks apple.
on getPassword(key_name)
  set password_result to text returned of (display dialog key_name & " Password" default answer "" with hidden answer)
  return password_result
	
  #local password_result
  #try
  # This only works on Mac OS Lion
  # tell application "Keychain Scripting"
  #	tell keychain "login.keychain"
  #		get {password} of (first key whose name is key_name)
  #		set password_result to result
  #	end tell
  # end tell
  # This works but throws error if not in keychain.
  # set password_result to (do shell script "security find-generic-password -ga 'Magic Box' 2>&1 > /dev/null | cut -d'\"' -f2")
  #on error theErroer
  #	set password_result to missing value
  #end try

  #if (password_result is missing value) then
  #	set password_result to text returned of (display dialog key_name & " Password" default answer "" with hidden answer)
  #end if

  #return password_result
end getKeychainPassword

on directoryExistsAtPath(path)
  local directoryExists -- boolean	
  try
    set result to (do shell script "if [ -d " & quoted form of path & " ] ; then exit 0 ; else exit 1 ; fi")
    set directoryExists to true
  on error theError
    set directoryExists to false
  end try
  return directoryExists
end directoryExistsAtPath

# Credit: Mikey-San from http://hintsforums.macworld.com/archive/index.php/t-90226.html.
on fileExistsAtPath(path)
  local fileExists -- boolean	
  try
    set result to (do shell script "if [ -f " & quoted form of path & " ] ; then exit 0 ; else exit 1 ; fi")
    set fileExists to true
  on error theError
    set fileExists to false
  end try
  return fileExists
end fileExistsAtPath

I compile this into an Application that I can install on all relevant computers. The first time it’s launched, it prompts for a password and mounts encfs. If you never need it, you don’t waste time mounting it.

Why We Probably Will Not Use It

After all this work I'm still not sure if we will use it long term. There are a couple of drawbacks:

encfs can hang my mac like it just don't care. If I mount an encrypted drive, unmount it, and then try to mount again? Good night. Press and hold the power button. You're d. u. n. done. Maybe it's the underlying fuse library. Who knows. But if we find we can not avoid that sequence then I'll unuinstall it.
Multiple machines editing the same notes does not work with Notational Velocity. While running, NV does not deal well with notes changing out from underneath it. If someone else adds a new note, that's ok. But even just viewing a note will save it. So if I'm viewing a note while my wife is editing it, it's a race to the finish. Changes will be lost. Maybe you can recover them from a dropbox version. Or maybe we have a "convention" of never editing secure notes. We can only add new versions and delete the old ones. Did you just feel icky? Yeah, me too.
It's sad to only use Notational Velocity for secure notes. I like the app so much, I wish I could use it on unencrypted notes. I may eventually point NV at an unencrypted dropbox folder and just use a text editor to manage the secure notes.

Questions?

I know I left out a bunch of details. How to configure Notational Velocity, what’s the .mount_verified file, etc. This post is mostly to commiserate with others feeling the encfs + dropbox pain. If you really want to feel the thunder and you think I can help, feel free to ask questions.

Archive Email Before Deleting a Google Apps User

2011-08-13T00:00:00+00:00

We (at lxd) are using google apps for business. When someone leaves the company, we'd like to archive all their emails so that we can access them later without paying google $50/year per user. It's pretty easy with Thunderbird. In short, we just download all the user's emails and zip up the Thunderbird profile directory. In long...

Change password to archive_user@mycompany.com

This should probably be part of your normal steps after an employee leaves. Since we’re going to archive the emails, we want to just change the password instead of deactivating the account. Google instructions for changing a user’s password.

Rename and Forward user@mycompany.com

As a google domain administrator you need to rename user@mycompany.com to something like archive_user@mycompany.com. Google Instructions For Renaming A User.

That’s going to create a nickname/alias for user@mycompany.com. So scroll down and delete that nickname. Then go to another user (e.g., new employee) and add an alias for user@mycompany.com. That way you don’t lose any emails intended for the account you are going to delete.

Make sure POP/IMAP is enabled

You need to login to the archive_user@mycompany.com account, go to mail settings, “Forwarding and POP/IMAP”, and make sure that “Enable POP for all mail (even mail that’s already been downloaded)” is set. These Google Instructions are pretty close.

Export all the user’s emails

Install a fresh copy of Thunderbird and connect it to archive_user@mycompany.com with your new password. Thunderbird is pretty easy to setup. I have confidence that you can do it!

It’s best to do this in an empty Thunderbird profile. If you already use Thunderbird, you can Setup Multiple Profiles in Thunderbird.

Once you kick off Thunderbird, it will take a long time to download everything. Just let Thunderbird sit there for many hours. I leave it overnight. I don’t really know how to tell when it’s done other than when it just seems to stop downloading more. :-)

Zip up the profile directory

Exit Thunderbird and find your Thunderbird profile directory. You can just zip that profile directory and put the zip file on a DVD or where you want to store it permanently. On the same DVD, it’s a good idea to include installers for the version of Thunderbird you used to export the emails. I include a windows and mac installer.

Accessing archived emails

Should the day come when you need to go find an old email, it’s pretty easy. I just install a fresh copy of Thunderbird (from the DVD). I’ve done this on both Windows and Mac. When it asks to enter an email address, just cancel. Depending on your operating system, it may complain about a failed install because you cancelled. Ignore that. What you just did was create an empty profile.

Exit Thunderbird, and find that empty profile directory. Unzip your DVD copy the Thunderbird profile and copy the contents into the new empty profile directory. Then just start Thunderbird. All the archived emails should be there in their searchable glory. I’ve even exported from Windows and opened them from Mac. Thunderbird’s profile directory seems to be nicely portable.

BTW, there’s a mozbackup program out there that makes backing up a profile super easy. But that program does not run on mac, so I vote no. Just zip up the profile directory. Sorry mozbackup.

Delete the old accounts

Before deleting a user I recommend verifying that you have access to the archived emails. Also, you’ll want to transfer ownership of any google docs and share calendars. Once you delete the archive_user@mycompany.com account, any Thunderbirds will start complaining about failed authentication. Just go into Thunderbird’s server settings and uncheck everything. That’ll keep it from trying to connect to an account that doesn’t exist.

Will My Archive Work 10 Years From Now?

No. Whatever OS you’re exporting on now will be end of life eventually. Your archived installers will not work. And the latest version of Thunderbird will have no idea how to load that profile directory. This approach is good for a few years. If you need to guarantee access for many years you should really just keep the google accounts and deactivate them. Pay the $50/year and thank your lucky clouds.

Feedback welcome

Let me know if you approach this differently…

Verizon Network Extender Doesn't Work Until You Activate It With Verizon

2011-08-04T00:00:00+00:00

Updated August 18th, 2011 after Giametti edumacated me

Thanks to Giametti's comment below, I called Verizon and was able to get this thing working. The process is awesome.

You need a Verizon account. I have a wireless account. Don't know if another service would do. Perhaps.
Call Customer Support (*611 on your mobile). I don't know what option to press. I just hit 0 to get an operator.
Wait on hold and try to keep the on hold music from making you crave brains.
Tell them you want to activate a Verizon Network Extender.
Guess what happens next? Yup, they transfer you. More on hold music. More brains...
You will eventually be connected to someone who has delicious brains. He/she will actually know what's up. You'll need to read them the mac id on the bottom of the device. He'll also want it plugged in and ready to go at the time. The GPS actually went blue while I was on the phone with them.
It might be a few minutes before the sys light goes blue. Mr Delicious Brains said my box was old and would automatically download a firmware update and reboot.
Victory

What's the lesson here? A red GPS light might just mean you're not activated yet. I'm amused at the whole experience, but I'm super happy to be able to call Mom from home again.

Original whiny post below...

I purchased the Samsung SCS-26UC4 in an attempt to get a Verizon signal at home. But drats, I just can’t get it to work. So with this post I cast my vote into the great google expanse. Boo Verizon. Boo Samsung. Your product doesn’t work for me. Can you hear me now? No…

No GPS signal

The problem is that my device can’t get a GPS signal. In short, these network extenders need a GPS signal for 911 calls. We live in Raleigh, NC. My droid, iphone, and ipad all know exactly where I am. The Samsung SCS-26UC4 does not. I used the external antenna. Put that thing out every window. Put it on the roof. Wrapped it around a tree. Whispered sweet nothings to it. No luck. I even took that blasted box to work (near the airport) and tried there. Nope.

So really? No GPS signal? I kind of feel like it’s not really trying. So I vote no. The Samsung SCS-26UC4 is a P.O.S..

Any ideas welcome

If you have any ideas on how to get a GPS signal, please let me know…

Configuring delayed_job Logging

2011-01-17T00:00:00+00:00

I use delayed_job to run things when you least expect it! I want my web request to be logged to RAILS_ROOT/log/production.log and my delayed_job work to be logged to RAILS_ROOT/log/production_delayed_job.log. Here’s how I configure delayed_job to log to a separate file.

RAILS_ROOT/config/delayed_job_config.rb

I drop this into my RAILS_ROOT/config/delayed_job_config.rb.

Delayed::Worker.logger = 
  ActiveSupport::BufferedLogger.new("log/#{Rails.env}_delayed_jobs.log", Rails.logger.level)
Delayed::Worker.logger.auto_flushing = 1
if caller.last =~ /.*\/script\/delayed_job:\d+$/
  ActiveRecord::Base.logger = Delayed::Worker.logger
end

The auto_flushing=1 addresses some issue with delayed_job assuming it’s getting a buffered logger. You can read about it here.

The caller.last parsing is the hack I put in to get all my application logging into the delayed job log. delayed_job_config.rb is loaded by your web app as well as by the delayed job processes. This conditionally hijacks the ActiveRecord::Base.logger if we’re running in the delayed_job daemon.

An important part of getting this to work is to use ActiveRecord::Base.logger correctly in your models.

class MyModel < ActiveRecord::Base
  def mymethod
    logger.info <<-TEXT
      This will get written to log/production.log
      when called by a web request, but
      it will be written to log/production_delayed_job.log
      when called in a delayed job.
    TEXT
    Rails.logger.info "Always goes to log/production.log"
  end
end

Questions

Is there a better way to detect that I’m running in the delayed_job daemon?
Is there a better logger than the BufferedLogger and auto_flushing=1?

Setting up monit's environment to start and stop delayed_job

2011-01-08T00:00:00+00:00

Update March 18, 2012:

Using Rails 3?

I'm using Rails 3.1.3. To get DJ working I've had to stick with delayed_job 2.1.4. The 3.0.1 build results in "undefined method `before_fork' for nil:NilClass". If you google around enough you'll find the reasons why. If you're like me and have other things to do, just stick with 2.1.4 for now.

Thanks jonathan dean.

I use Monit to keep delayed_job running. Getting monit to start and stop delayed_job is a giant pain, mostly because of environment problems. In this post I explain how I make it work.

Monit

First off, install monit. It’s pretty easy. But remember to set startup=1 in /etc/default/monit. Also, I recommend adding your email as an alert recipient in /etc/monit/monitrc.

To tell monit how to start and stop delayed_job, I create /etc/monit.d/check_process_delayed_job with the contents below. Check monit docs for details around my alert and restart settings.

check process myapp_delayed_job with pidfile /path/to/myapp/tmp/pids/myapp_delayed_job.pid
  start program = "/bin/bash /path/to/myapp/script/monit_delayed_job.sh start"
    as uid wei and gid wei
  stop program =  "/bin/bash /path/to/myapp/script/monit_delayed_job.sh stop"
    as uid wei and gid wei
  if cpu > 60% for 2 cycles then alert
  if cpu > 80% for 5 cycles then restart
  if totalmem > 200.0 MB for 5 cycles then restart
  if 3 restarts within 5 cycles then timeout
  group background_tasks

monit_delayed_job.sh

Notice that it’s starting and stopping delayed_job through an sh script. My rails_root/script/monit_delayed_job.sh is below. It does two important things: loading /etc/profile, and logging. But the greatest of these is logging!

All the output of monit_delayed_job.sh is logged to log/monit_delayed_job.log, so you can see if you have any environmental errors. Otherwise monit will swallow the errors and you’ll go mad. We don’t want that.

BTW, you can see I’m using bundle exec. You can easily change the last line of the sh script to start delayed_job however you like.

#!/usr/bin/env bash                                                                                                                                                  

if [ $# -lt 1 ] ; then
    echo "Usage:   " $0 " <start | stop> "
    exit 1
fi

action=$1

script_location=$(cd ${0%/*} && pwd -P)
cd $script_location/..
rails_root=`pwd`

if [ -f "/etc/profile" ]; then
  . /etc/profile
fi

logfile=$rails_root/log/monit_delayed_job.log
echo "-----------------------------------------------" >> $logfile 2>&1
echo "Running bundle exec ./script/delayed_job $action" >> $logfile 2>&1
echo `date` >> $logfile 2>&1
echo `env` >> $logfile 2>&1

bundle exec ./script/delayed_job $action >> $logfile 2>&1

/etc/profile

In /etc/profile somewhere I set PATH and RAILS_ENV. Obviously, you should change these accordingly. Keeping the path and environment settings in /etc/profile allows me to setup hosts differently.

export PATH="/usr/local/sbin:/usr/sbin:/sbin:/usr/local/ruby-enterprise/bin:$PATH"
export RAILS_ENV=production

Capistrano

I use capistrano to deploy. My delayed_job.rb recipe is below. It’s pretty straight forward.

Capistrano::Configuration.instance(:must_exist).load do
  namespace :delayed_job do
    desc "Start delayed_job daemon"
    task :start do
      alert_user("Starting delayed job using #{use_monit_for_delayed_job ? 'monit' : 'script/worker'}")
      
      if use_monit_for_delayed_job
        sudo "monit start #{monit_service_name}"
      else
        run "cd #{current_path} && RAILS_ENV=#{rails_env} script/worker start"
      end
    end
    
    desc "Stop delayed_job daemon"
    task :stop do
      alert_user("Stopping delayed job using #{use_monit_for_delayed_job ? 'monit' : 'script/worker'}")
      
      if use_monit_for_delayed_job
        sudo "monit stop #{monit_service_name}"
      else  
        run "cd #{current_path} && RAILS_ENV=#{rails_env} script/worker stop"
      end
    end
    
    desc "Restart delayed_job daemon"
    task :restart do
      alert_user("Restarting delayed job using #{use_monit_for_delayed_job ? 'monit' : 'script/worker'}")
      
      if use_monit_for_delayed_job
        sudo "monit restart #{monit_service_name}"
      else
        run "cd #{current_path} && RAILS_ENV=#{rails_env} script/worker restart"
      end
    end
  end  
end

To enable monit in the recipe above, I set these two things in my deploy.rb:

namespace :delayed_job do
  set :use_monit_for_delayed_job, true
  set :monit_service_name, 'myapp_delayed_job'
end

Logrotate

Rotating logs is fun. Do it. Here’s my /etc/logrotate.d/myapp file.

/path/to/myapp/log/*.log {
  weekly
  missingok
  rotate 100
  compress
  copytruncate
  delaycompress
  sharedscripts
}

Victory

Simple enough, eh? I hope this helps you get it working.

ActiveHash and SuperEnum!

2010-12-26T00:00:00+00:00

Jeff Dean’s ActiveHash is a great gem for status-type associations. In this post I show how I extend ActiveHash to improve readability.

A NewsItem model might belong to Status to indicate values like Published, Draft, and Deleted. ActiveHash lets you do ActiveRecord type associations, but without having to hit the database.

class Status < ActiveHash::Base
  include ActiveHash::Enum
  self.data = [
    {:id => 1, :name => 'Draft'},
    {:id => 2, :name => 'Published'},
    {:id => 3, :name => 'Deleted'}
  ]
  enum_accessor :name
end

# == Schema Information
#
# Table name: news_items
#
#  id         :integer(4)      not null, primary key
#  title      :string(255)
#  body       :text
#  status_id  :integer(4)
#  created_at :datetime
#  updated_at :datetime
#
class NewsItem < ActiveRecord::Base
  belongs_to :status
  scope :not_deleted, 
        :conditions => [ 'news_items.status_id != ?', 
                         Status::DELETED.id ]
end

NewsItem is a standard ActiveRecord model, but notice that Status inherits from ActiveHash::Base. Status will not touch the database, but you can use it like a regular ActiveRecord model. E.g., in your controllers and models:

news_item = NewsItem.create(:status => Status::DRAFT)
if news_item.status == Status::PUBLISHED
  # ...
end

And in your formtastics, you can treat Status like you would an ActiveRecord model:

  f.input :status, :collection => Status.all

In the above examples I’m using the ActiveHash::Enum module to magically add the DRAFT, PUBLISHED, and DELETED enumerations. Super handy. However, I have to leave my mark on everything, so I extended ActiveHash a little. Instead of Status::PUBLISHED, I like to write Status.published. And I’d also like to be able to easily test the value of a Status instance with shorthand such as @status.published? and @status.deleted?:

  @news_item = NewsItem.create(:status => NewsItem.draft)
  # ...
  if @news_item.status.draft?
    # ...
  elsif @news_item.status.published?
    # ...
  elsif @news_item.status.deleted?
    # ...
  end

To do the above I create a little helper module in RAILS_ROOT/lib/active_hash/super_enum:

module ActiveHash
  module SuperEnum
    def self.included(base)
      base.send(:include, Enum)
      base.extend(Methods)
    end

    module Methods
      def insert(record)
        super
        loud = constant_for(record.attributes[@enum_accessor])
        return nil if loud.blank?
        quiet = loud.downcase
        self.class_eval <<-RUBY
        def #{quiet}?
          self.id == #{loud}.id
        end
        class << self
          def #{quiet}
            #{loud}
          end
        end
        RUBY
      end
    end
  end
end

Now in my Status module I do this:

require 'active_hash/super_enum'
class Status < ActiveHash::Base
  include ActiveHash::SuperEnum
  self.data = [
    {:id => 1, :name => 'Draft'},
    {:id => 2, :name => 'Published'},
    {:id => 3, :name => 'Deleted'}
  ]
  enum_accessor :name
end

Thus in your controllers and models:

news_item = NewsItem.create(:status => Status.draft)
if news_item.status.published?
  # ...
end

Functionally, SuperEnum is the same as ActiveHash::Enum. SuperEnum just shouts a little less for those of us with sensitive eyes. In any case, you should all be using ActiveHash. It saves you from writing useless database migrations, db initialization tasks, and it probably saves trees.

What are other people using for Status-type associations?

Transactional Factories

2010-03-22T00:00:00+00:00

Transactional-factories is a ruby gem that I created to help speed up my tests. It uses nested database transactions to allow efficient, programmatic initialization of test data. In short, it gives each test case a class-level setup callback that is invoked only once.

You can see the source code at github.com/heedspin/transactional-factories and download the gem from gemcutter.org/gems/transactional-factories.

Why?

Many of test scenarios require creation of complex test data and are rather difficult to set up. Here are some requirements for a testing framework:

Complex test data - The framework should allow creation of complex test data.
Cleanliness - Tests should clean up after themselves and not leave behind garbage data. (hint: transactions!)
Efficiency - Tests should not waste time needlessly recreating test data. (hint: nested transactions!)

How complex is the test data?

One of my projects generates state-mandated reports. For a glimpse into the level of complexity of state reporting requirements, the reader is invited to visit the California Department Of Education site for Child Development Data Reporting. This level of detail drives some of our most complicated test cases.

Transactional-factories address complexity, cleanliness, and efficiency using nested transactions and adding class level setup/teardown callbacks. Here’s how it works: A top level transaction wraps the entire TestCase. A class-level setup method is called once to allow programmatic creation of test data shared across all tests. Then each test method is called within a nested transaction that is rolled back to protect the test methods from each other. The top level transaction is also rolled back to protect the TestCases from each other. Consider the following test code:

require 'transactional_factories'

class MyModelTest < Test::Unit::TestCase
  # Class method called only once to create test data.
  def self.setup
    100.times { MyModel.create }
  end

  # Instance method called before each test method.
  def setup
    MyModel.create
  end
  
  def test_1
    assert_equal 101, MyModel.count
    MyModel.delete_all
    assert_equal 0, MyModel.count
  end

  def test_2
    assert_equal 101, MyModel.count
  end
end

The sequence of events would be as follows:

Top level transaction begin
Class level setup()
Nested transaction begin
Instance method setup()
test_1
Nested transaction rollback
Nested transaction begin
Instance method setup()
test_2
Nested transaction rollback
Top level transaction rollback

And that’s all there is to it. You get one class level setup to create test data that’s shared across all your tests. It’s programmatic, so you can use your existing model code to create all your complicated dependencies (addressing complexity). The transactions keep the test methods and test cases from interfering with each other (addressing cleanliness). It’s only run once, so you don’t waste time recreating complex data (addressing efficiency).

Check it out! github.com/heedspin/transactional-factories and gemcutter.org/gems/transactional-factories.

What about ActiveRecord Fixtures?

ActiveRecord fixtures are a clean way to explicitly describe your test data. And the infrastructure around fixtures in tests is pretty flexible. With use_transactional_fixtures = false, loading a fixture deletes all data in the respective table, so you get cleanliness that way. With use_transactional_fixtures = true, you only have to load your fixtures once, and you use transactions to keep each test method from introducing garbage data. That gives you efficiency and cleanliness. However, fixtures don’t just address the complexity requirement in my opinion.

By the way, transactional fixtures are not currently implemented to support nested transactions. I believe this is just a legacy issue since transactional fixtures came before widespread support for nested transactions (through savepoints). In any case, transactional fixtures must be disabled in order to use transactional-factories. I think if transactional fixtures are updated to allow nested transactions, the two can coexist.

Why class-level setup?

I decided to make the transactional-factory callback a class-level method to force the programmer to recognize the non-intuitive object semantics of TestCases.

The way ruby’s Test::Unit::TestCase works is to create a new instance of your test case for each test method. It’s just a strange thing that probably makes sense to people smarter than me. So in the MyModelTest example above, two instances of MyModelTest would be created; one for test_1, and another for test_2. (Extra credit to the person who can explain why there are actually more than 2 instances of MyModelTest created).

Once consequence of this approach is that test methods can not share instance variables. You might never notice this if you’ve been using the instance method setup, since it’s called on each instance. However, since we will only call our transactional-factory setup method once, we have to decide what the scoping should be. I decided to make it class-level so I wouldn’t be tempted to use instance variables. Class variables will work just fine.

Questions

Is anyone else approaching this differently? Do any other test frameworks address this? E.g, spec?

What are the implications for testing application transactions?