Terra Firma Design & Consulting Blog Posts

Chimpified = Mailchimp + Shopify

Brian Getting — Wed, 04 Nov 2009 05:45:52 GMT

Tonight we have launched a new service called Chimpified for beta testing. Chimpified connects a Shopify store to a Mailchimp account, so that when an order is placed in the store (and the customer has agreed to receive marketing materials), their email address will automatically be added to the mailing list(s) that the store owner chooses at Mailchimp. There is no coding involved, and it takes less than a minute to set up.

Installing Chimpified

To connect your Shopify store with your Mailchimp account, first visit chimpified.com and enter the URL of your Shopify store to sign in. The first time you sign in, you will be asked if you want to install Chimpified to your store. After agreeing, you will be prompted to enter in your Mailchimp API key for the account you want to connect your store to. Once your Mailchimp API key is accepted, you will be able to select the list(s) in your Mailchimp account that you want your customers added to. Finally, you’ll need to set up an order notification in your Shopify store admin to notify Chimpified of new orders. That’s it.

There are instructional movies available to help you with installing Chimpified and finding your Mailchimp API key at chimpified.com.

Beta Release

For now, the service is free since we are testing it out and getting through any bugs that we find. For the official release, which will probably be sometime in early 2010, Chimpified will be available in the Shopify App Store. There will be some additional features added as well, though nothing is set in stone, but some ideas are:

- Update Shopify if a customer unsubscribes from the emails. (Targeted for beta release)
- Add “Email with Mailchimp” link to product pages in the Shopify admin.
- Add “Email with Mailchimp” link to collections pages in the Shopify admin.
- SSL security.

Since the app requires resources, there will probably be low monthly fee ($5-10) once the official release launches.

Your Feedback Is Appreciated

We welcome any ideas that our users have to offer, since they using Mailchimp and Shopify, and it would be great to hear their opinions.

Hominid Gem Version 2.0

Brian Getting — Thu, 22 Oct 2009 18:24:14 GMT

We’ve finally gotten around to doing some updating on the Ruby gem that we maintain called Homind. Hominid provides a wrapper for the Mailchimp API, allowing developers to easily integrate with Mailchimp’s email marketing service. Along with updating the gem, we have also moved it to Gemcutter since Github is no longer hosting gems.

Installing Hominid

To install the Hominid gem, be sure that you have installed the Gemcutter gem, or that Gemcutter is available as one of your gem sources. The easiest way is to simply install the Gemcutter gem:

sudo gem install gemcutter
gem tumble

This will add Gemcutter to your list of gem sources, and sets up to install gems from Gemcutter. To install the Hominid gem, simply enter:

sudo gem install hominid

Configuration

The upgrade to version 2.0 is essentially a complete overhaul, thanks to some fantastic help from Micheal Struder, and is no longer available as a Ruby on Rails plugin. From here on out, Hominid will just a pure Ruby gem. Detailed documentation is available, but getting set up with your Rails application requires that you set the gem dependency in your environment.rb file.

config.gem "hominid", :version => '>= 2.0.0', :source => "http://gemcutter.org"

It is also convenient to set up a configuration file with your Mailchimp account information at config/hominid.yml. Refer to the Hominid documentation for more information on setting up that YAML file.

Simple Usage

There has been a lot of refactoring with the gem, which makes usage a slight bit different than with previous versions. The gem has been changed from a single class to a module, and we have implemented the Hominid::List class (for working with lists) and the Hominid::Campaign class (for working with campaigns).

To find a particular list that you want to work with, simply use one of the finder methods for the List class:

list = Hominid::List.find_by_name('mailing list name')

Once you’ve found your list, let’s say that you want to subscribe someone to your mailing list. Use the subscribe method in this instance:

list.subscribe('my@emailaddress.com')

Or, perhaps you want to unsubscribe someone from that email list, in which case you would use the unsubscribe method:

list.unsubscribe('my@emailaddress.com'

Finally, you may want to updated someone’s information in your mailing list, which is done using the @update_member) method:

list.update_member('my@emailaddress.com', {:EMAIL => 'another@emailaddress.com'}, 'html')

You can send other attributes in the hash, such as :FNAME to change other attributes of the persons subscription, aside from just their email address.

Documentation

We are doing the best that we can do document the usage of the Hominid gem. Don’t forget to visit the gem documentation to get a better idea of how to use the new gem. We are hoping that more and more people will get involved with helping us to improve this gem. There are quite a few API methods that have not been implemented yet, primarily since we don’t use them much. However, the gem is primed and ready to become a complete wrapper for the Mailchimp API, and any contributions are greatly appreciated.

PIV Oregon Goes Live

Brian Getting — Thu, 23 Apr 2009 19:04:53 GMT

Over the weekend, in addition to enjoying the great weather here at the beach, we launched a new website for Public Information Verification (PIV) Oregon. PIV provides background screening, pre-employment checks and other screening services. Some of the notable features of their new website are:

Secure Screening Submissions – PIV deals almost exclusively with sensitive personal information, so security was on our minds during each phase of development. Every effort was made to ensure that all sensitive data is transmitted and stored securely. To communicate to their clients that their website can be trusted, we installed an Extended Validation SSL certificate, which provides 1024-bit encryption as well as triggering the “green bar of trust” in newer browsers.

Paypal Integration – Looking for a background screening on yourself for a job or volunteer opportunity? Simply submit the request, review it for accuracy, and check out via Paypal and your request will be returned soon. It’s simple. It’s secure. It’s being used a lot already.

Client Oriented – By researching and understanding how PIV’s corporate clients are using their website, we were able to design an incredibly intuitive interface to facilitate their needs. To accommodate these corporate clients, who submit many screening requests and are billed by invoice, we created an account management system that let’s corporate clients upload spreadsheets of requests. Where in the past submitting 100 screening request meant filling out an online form 100 times, we’ve made it so that submitting 100 or 1000 screening requests is as simple as submitting one request. So far, the response has been positive and PIV has gained a competitive edge over their competition.

Streamlined Administration – Not only were we thinking about PIV’s clients during development, but we thought it was important to also consider how we can make PIV’s internal processes more streamlined. A simple interface allows PIV to easily find and organize screening requests, track the progress of requests, and conveniently export data for use in spreadsheets and reporting.

Analytics & Goal Tracking – As with every website we build, PIV’s new site includes an enterprise-level analytics solution that can help them understand how their clients are using their website, and make improvements. By measuring how clients are traversing through the site, and specifically tracking conversion goals, we are able to work with PIV on an on-going basis to evaluate and improve upon their website.

We are excited to see the new PIV site launched, and have been extremely impressed with the positive feedback we have received and the number of people that have been using it on the first week. We are looking forward to a long relationship in the future as we help PIV to streamline and improve their business.

Installing an EV SSL Certificate on Nginx

Brian Getting — Mon, 20 Apr 2009 21:12:38 GMT

This is probably one of those things that everyone else knows, and I just didn’t show up for class that day. I had some trouble installing an Extended Validation SSL certificate on a clients site this weekend, mostly due to my own lack of knowledge.

Generate the CSR

The goal of my task was to install an EV SSL certificate on an nginx web server. I had done it before with self-signed and normal (is that the right word?) certificates before. As with any SSL, the process started by generating a Certificate Signing Request (CSR) and a private key on the server. There is tons of documentation about how to do that, in my case it was accomplished using OpenSSL.

Get a Certificate from a Certifying Authority

The next step, which is common to all SSL certificates, was to send the CSR to a company that will issue a trusted SSL certificate. In my case, we chose Comodo since they had the best pricing. The process is pretty simple, and just involves verifying company information and uploading the CSR to them. However, be sure that your company information is in order before ordering an EV SSL certificate- the process of getting approved is much more stringent for extended validation.

I have had a few clients run into problems because their business registration didn’t match their domain registration, or because their physical address differed from their registered address. All of this can be resolved with a letter from a lawyer or an accountant (be sure they are a registered CPA).

Concatenating Multiple CRT Files

What came back from Comodo, in my case, was a ZIP file containing about 5 CRT files. The next step is to put them all together into one file so that nginx can use them. This is easy enough, although it turns out that you should do it in a particular order. Comodo had sent me the following files:

AddTrustExternalCARoot.crt
ComodoEVSGCCA.crt
ComodoUTNSGCCA.crt
UTNAddTrustSGCCA.crt
sample-domain-name_com.crt

To put them all together into one file, you simply type the following into a terminal:

cat  AddTrustExternalCARoot.crt ComodoEVSGCCA.crt ComodoUTNSGCCA.crt UTNAddTrustSGCCA.crt sample-domain-name_com.crt >> ssl-bundle.crt

Of course, you will have a different name for your domain CRT file, and will probably want to name the output file for your domain. So if your domain is mysite.com, you would call the output file (the file name after the “>>”) mysite_com.crt. It probably doesn’t matter, but it seems like a good idea.

The output file will basically just be a text file with all the certificates put together, so you could just copy and paste all the files (in the right order) into another file and save it with the right name. Whatever method you use, you’ll want to open this file up and copy it’s contents into a new file on the web server, probably at:

/etc/ssl/certs/ssl-bundle.crt

You could also scp the file into place, but for me it’s just easier to copy and paste it. Save this file on the web server and we are almost ready.

Configuring Your Nginx Virtual Host

The last thing that we needed to do was to make some changes in our virtual host file. Basically, just like any other SSL on nginx, you want to point to the file that contains all the certificates that we received from Comodo:

server {
  listen  443;

  ssl  on;
  ssl_certificate  /etc/ssl/certs/mysite_com.crt;
  ssl_certificate_key  /etc/ssl/private/mysite.key;

  server_name mysite.com;
}

Obviously, you want to be sure that you put your key (named mysite.key in the above example) into the correct private directory. From here, restart nginx and the certificate should be fine.

Real Estate on Rails (Working with RETS Data)

Brian Getting — Wed, 15 Apr 2009 16:30:23 GMT

In the past, I have worked on a lot of real estate websites. After having taken a few year break from doing any realty-related web development, which was a welcome break, I have recently started working on another real estate site. My hope going into this was that working with Rails would make the job much easier, and that things had improved somewhat since my last experiences. I was right, and the combination of Rails and RETS data has actually made the process almost fun, relative to the past.

IDX or RETS

The first order of business was to figure out the best way to get at the MLS (Multiple Listing Service) data that I wanted. In the past I had used IDX feeds, which translates to FTP access to a whole bunch of property data and images. Each day, someone or something would need to access this FTP server, download all the new files and update the website. It always seemed like there was a better way, and it turns out that there is.

Rather than an IDX feed, I have been working with RETS data, or Real Estate Transaction Standard data, which basically means that (finally) the MLS data is available as XML data via a web service. To me this is much more attractive, and seems to be a much more efficient way of updating MLS data.

RETS for Ruby Gem

Of course, the first thing I did was check to see if anyone had created a Ruby gem or Rails plugin for accessing and handling RETS data, and it turns out that someone has. The rets4r gem is what I have been experimenting with, and so far it has been extremely useful. I would be lying if I said I wasn’t tempted to rewrite the thing, and probably will in the future sometime, however it provides me with the functionality that I need to access the RETS data that we have access to and deal with that data as Ruby objects, which is nice. Of course, a large part of working with RETS is understanding how the data is formated and developing a strategy for keeping your site updated. We’ll start by requiring the rets4r gem in our Rails application:

config.gem "rets4r", :version => '>=0.8.5'

This will go in your config/environment.rb file.

The Daily Download

Even though RETS provides a web service that could probably be queried directly, it is still a much better idea to pull the MLS data down into a local database and go from there. That being said, I needed to come up with a strategy where all the properties that need to be displayed are pulled down originally (only once) and then each day check the RETS server for updates and deletions. These are essentially the same operations, in that the first time we pull down data we will be asking the RETS server for all properties that we have access to, and the daily updates will just ask for properties that have been updated since our last request. Let’s take a look at how we do that using the rets4r gem:

client = RETS4R::Client.new('http://retsprovider.com/rets/')
client.login('username', 'password') do |success|
  if success
    client.search('Property', 'A', "(FIELD=#{24.hours.ago.strftime('%Y-%m-%dT%T')}+)", {'Format' => 'COMPACT-DECODED'}) do |result|
      result.data.each do |row|
         property = Property.find_or_create_by_mls(row["MLS"])
         property.price = row["PRICE"]
         property.sqft = row["SQFT"]
         ...
         property.save
      end
   end
  client.logout
end

In this (very simplified) example, we are logging in to the RETS server, and running a search for all properties that have been modified in the last 24 hours. You will need to check your own MLS data provider to find out exactly what the various fields are, but I am using examples here. One thing to note is that I am searching the Property resource on the RETS server, and within that resource I am specifically searching within the A class of properties. In my case this happens to be residential properties, but you will want to do this for each resource class that you want to update (perfect place for a loop). Additionally, you will want to figure out what to replace FIELD with so that you are querying the last modified date field for your particular RETS server. Also, notice that I am adding the Format option to my search, and asking for the results to be returned in COMPACT-DECODED format. For my particular provider, this means that all lookup values for particular fields will be shown, rather than the obscure lookup codes that might otherwise be there. This is nice since dealing with lookup values is not really central to what I am trying to do, so that is handy. The rest is just about assigning the correct values to the correct attributes in my Property model, and we are off.

One thing that is worth mentioning is that not all the resource classes have the same fields. For example the field denoting the street surface type for a residential property might be called FEAT00001. However, the same street surface type field for a commercial property might be FEAT00003. It’s important to track down each field that you want for each resource class. It is done this way to account for different values and flexibility between property types, but can be a pain. In my case, there are about 15 fields that I want data for in my Property model that have to be mapped differently depending on the resource class. It’s a fact of life.

Getting at Images

So the script above, if run daily, will check the RETS server for any properties that have been modified in the last 24 hours and pull them down to our local database. Not a bad start. The next thing we need this script to do is to download the images for each of the properties. Typically there will be a field for photos_count or something along those lines so that you can tell for each property if it has associated photos. Using that field, we can add the following to our script, just before the property.save line:

if row["PHOTOS_COUNT"].to_i > 0
  photo_index = 1
  row["PHOTOS_COUNT"].to_i.times do
    client.get_object('Property', 'Photo', "#{row["ID"]}:#{row_index}, 0) do |photo|
      handle_object(photo)
    end
    photo_index = photo_index + 1
  end
end

This one is pretty basic as well, and you will need to write your own handle_object method depending on what you want to do with the images once they are downloaded. There is a way of pulling down all the images in a single request, although I could not coax the rets4r gem to play nicely with that one.

Deleting properties

One thing that was interesting with the RETS feed I have access to is that the property status is not really used. In some cases it is there, but it cannot be used to see if a property has been deleted. Since our feed only spits out properties that we have access to (which does not include deleted records), we have to come up with some way to delete properties that have been removed.

Interestingly enough, the company that provides my RETS feed has a solution, although I can’t help but wonder if there isn’t a cleaner way. They suggest that each day as part of the daily update, we query the RETS server for all the properties that we should be displaying, grabbing only one field (like the id or mls number) to make the query more efficient. We then compare that to the contents of our local database, and any records that we have but do not appear in the daily list need to be deleted. Backwards, huh? Here’s what it looks like:

client.search('Property', 'A', "(FIELD=1950-01-01T00:00:00+)", {"Select" => "MLS_NO"}) do |result|
  local_properties = Property.find(:all, :select => 'mls')
  local_properties.each do |property|
    property.destroy unless result.data.index(property.mls)
    # Don't forget to delete all images too.
  end
end

Again, depending on how you are storing or managing your images, you will want to be sure and delete them when you delete an associated property. However, our loop simply grabs a list of all the MLS numbers that we should have in our local database. Then we loop through all of our property records, deleting any that have MLS numbers that aren’t in our list. Easy enough, although something about it seems ugly to me.

Development As Usual

Now that the challenging part is over, we can run our script and watch as properties fill our database and images come down as well. From here, developing a real estate site becomes like developing any other data-driven site. Rather than spending our time worrying about how to deal with MLS data, we can spend our time developing a great application.

From my standpoint, the hassle of dealing with MLS providers, and the intimidation of daily MLS updates are now removed from the equation, which means that real estate sites are no longer something to be feared. Although, that is from a technical point of view. You still need to deal with realtors to build their sites, and I’m not sure I’ve overcome that one yet.

Chuck Victory Goes Live!

Brian Getting — Mon, 13 Apr 2009 16:22:45 GMT

We’ve just launched a new e-commerce website for Newport, Oregon photographic artist Chuck Victory. Chuck has a background in advertising and photography, and a serious interest in wine and art. While a simple website, there are a few features that we feel should be mentioned:

Content Management – Chuck’s website has one of our custom content management systems that allow Chuck to control the aspects of his website that he needs to, without jeopardizing the integrity of his site. Chuck can easily create new galleries and manage his photos. In addition, the website offers Chuck the ability to add any number of available sizes that he wants to make available. In fact, I intend to do a blog post here soon about how that was accomplished, as other developers might find such a system handy since so many products can be available in multiple sizes, or colors…

Paypal Integration – As a newcomer to e-commerce, we worked with Chuck to determine that Paypal integration was the most economical way for him to start selling his photos online. In our case, Chuck’s site uses Paypal for it’s checkout process, and is securely integrated. Rather than simple posting variables to Paypal, the website encrypts all shopping cart data prior to being sent to Paypal, which prevents anyone from tampering with the order data.

Image Protection – While there is no way to fully protect images on the web, we wanted to at least make it harder for someone to try and get at the images. Of course, nothing can stop a screenshot, or even someone familiar with HTML from grabbing images, as that is the nature of the web. However, we composed a solution that removes the ability to ‘right-click’ on an image, or drag-and-drop it to your desktop to play with. Nothing groundbreaking here, but a feature worth noting.

We are excited that Chuck’s site is completed and launched, and as the bugs are worked out will be excited to see the response. For our part, working with Chuck has been an absolute pleasure, as he is not only a creative fellow but one that understands what he wants. Visit his website, and check out his photographic art…

Ruby Gem and Rails Plugin for Mailchimp

Brian Getting — Wed, 25 Mar 2009 20:02:32 GMT

Recently, the limitations of the acts_as_mailchimp plugin that I threw together a while back have been staring me in the face. The more that I work with Mailchimp, and the more that they improve their API, the more I realized that it was time to do something about it.

The Hominid Gem

Hominid is a gem that wraps the Mailchimp API so that you can easily work with it in Ruby or Rails. You can install Hominid as a plugin by typing:

script/plugin install git://github.com/bgetting/hominid.git

Or, you can install it as a gem by including the following in your environment.rb file:

config.gem "bgetting-hominid", :lib => 'hominid', :source => "http://gems.github.com"

If you use Hominid as a gem, be sure to create a YAML file at config/hominid.yml to hold your Mailchimp login information for each environment. From there usage is pretty simple. For example, to add someone to a mailing list you might use:


h = Hominid.new
list = h.lists.find {|list| list["name"] == "My Mailing List" }
h.subscribe(list["id"], 'bob@smith.com')

We first create a new Hominid object, which will log in to Mailchimp using your account information from the hominid.yml file. From there, we use the lists method to return all the mailing lists at Mailchimp for this account, and pull out the one with the name that we are looking for. We use that list’s id to subscribe someone to the list. Pretty simple.

There are also methods for dealing with campaigns and other things. One particular method that is worth noting is the convert_css_to_inline method, which uses the Mailchimp API to convert any CSS declarations in an HTML document into inline styles for use in emails. That thing comes in handy!

I haven’t yet had time to write tests for this gem. I would love any input that anyone has with regard to the best way to test this, and how to write the tests. Hopefully the lack of adequate tests doesn’t wreck anyone’s day.

Acts_As_Mailchimp Plugin

The Acts_As_Maichimp plugin then needed to be updated. I decided that since Hominid wraps nearly all the functionality of the plugin, I would just make it dependent on Hominid. So one you have Hominid installed, you can install acts_as_mailchimp by typing:

script/plugin install git://github.com/bgetting/acts_as_mailchimp.git

Now add the following to your model so that you can add, remove and update a member of a Mailchimp mailing list:

acts_as_mailchimp

For the most part, the plugin now just wraps the process of finding the list ID of the list you want to work with, and the connecting to Mailchimp using Hominid.

Someone also pointed out the lack of tests on this plugin. Again, I apologize for not taking the time, and would welcome any suggestions for what would make effective tests for such a plugin.