Waseem Ahmad

Integrating Zencoder API with Carrierwave in Rails

2013-04-29T00:00:00+00:00

h1. Integrating Zencoder API with Carrierwave in Rails Zencoder is a cloud based service that lets integrate video and audio encoding in your application. I'll show you how to integrate Zencoder API to encode videos in your Rails application that uses Carrierwave gem to upload files. First of all we will create a Video model that will be responsible for holding information related to uploaded videos. Lets see migration for corresponding videos table.

class CreateVideos < ActiveRecord::Migration
  def change
    create_table :videos do |t|
      t.string  :video
      t.text    :meta_info

      t.timestamps
    end
  end
end

The video column in the table is where we will mount our carrierwave uploader and meta_info to save meta information like duration, dimensions and bitrate about the original and encoded video files. Lets write our Carrierwave video uploader.

# app/uploaders/video_uploader.rb
class VideoUploader < CarrierWave::Uploader::Base
  after :store, :zencode
  storage :fog

  def store_dir
    "videos/#{model.id}"
  end
  def extension_white_list
    %w(avi mov mkv mpg mpeg mp4 m4v flv)
  end

  private
  def zencode(*args)
    params = {
      :input         => @model.video.url,
      :test          => true, #  https://app.zencoder.com/docs/guides/getting-started/test-jobs-and-integration-mode
      :outputs => [
        {
          :public      => true,
          :base_url    => base_url,
          :filename    => 'mp4_' + filename_without_ext + '.mp4',
          :label       => 'webmp4',
          :format      => 'mp4',
          :audio_codec => 'aac',
          :video_codec => 'h264'
        },
        {
          :public      => true,
          :base_url    => base_url,
          :filename    => 'webm_' + filename_without_ext + '.webm',
          :label       => 'webwebm',
          :format      => 'webm',
          :audio_codec => 'vorbis',
          :video_codec => 'vp8'
        },
        {
          :public      => true,
          :base_url    => base_url,
          :filename    => 'ogv_' + filename_without_ext + '.ogv',
          :label       => 'webogv',
          :format      => 'ogv',
          :audio_codec => 'vorbis',
          :video_codec => 'theora'
        },
        {
         :thumbnails => {
           :public      => true,
           :base_url    => base_url,
           :filename    => "thumbnail_" + filename_without_ext,
           :times       => [4],
           :aspect_mode => 'preserve',
           :width       => '100',
           :height      => '100'
         }
       }
     ]
    }

    z_response = Zencoder::Job.create(params)
    @model.meta_info[:request] = z_response.body
    @model.save(:validate => false)
  end
  def filename_without_ext
    @filename_without_ext ||= File.basename(@model.video.url, File.extname(@model.video.url))
  end
  def base_url
    @base_url ||= File.dirname(@model.video.url)
  end
end

We tell Carrierwave to zencode uploaded video after it has finished storing. The after :store, :zencode directive tells Carrierwave to call method zencode after the video file has finished uploading. storage :fog tells the storage type that should be used by Carrierwave. store_dir and extension_white_list tell it which directory to upload the file on our storage provider and what file extension should be allowed to be stored. Method zencode prepares parameters that specify the formats in which we want our original videos to be encoded and some other information. ":input":https://app.zencoder.com/docs/api/encoding/job/input points to the url from where Zencoder will download the original file to encode. ":test":https://app.zencoder.com/docs/api/encoding/job/test specifies if encoding should be done in "Integration Mode"; for testing purposes it's true. ":outputs":https://app.zencoder.com/docs/api/encoding/job/outputs has settings for videos that will be output of encoding process. Our original video will be encoded according to each hash in the outputs hash. Each hash has some "general output settings":https://app.zencoder.com/docs/api/encoding/general-output-settings ans some "specific to the formats":https://app.zencoder.com/docs/api/encoding/format-and-codecs in which we want our output videos. ":public":https://app.zencoder.com/docs/api/encoding/s3-settings/public is specific to S3 in my case to make the uploaded videos publicly readable. ":base_url":https://app.zencoder.com/docs/api/encoding/general-output-settings/base-url is the full url of the directory where Zencoder will put the encoded video. ":filename":https://app.zencoder.com/docs/api/encoding/general-output-settings/filename tells it what filename to use when uploading the encoded file to that directory. Here, we're following a pattern in naming the files which will be useful for us generating the urls of the output files which is explained a little bit further. We also specify an optional label, format, audio_codec and video_codec. Zencoder not only encodes our videos, it also lets us generate thumbnails for those videos. The last hash in outputs hash specifies the options for the "thumbnails":https://app.zencoder.com/docs/api/encoding/thumbnails of the video. It should be noted that the filename for generated thumbnails has an extension attached to it according to its format - by default png - by Zencoder itself; We don't need to specify it. ":times":https://app.zencoder.com/docs/api/encoding/thumbnails/times is an array of times in seconds at which to grab a thumbnail. In our case it has only one entry because we want only one thumbnail at four second. We 'preserve' the aspect ratio and also specify 'width' and 'height' of the thumbnail. Next we send all these options to Zencoder API. To talk with the api, we use the "zencoder":http://rubygems.org/gems/zencoder gem. Zencoder::Job.create(params) makes a request to the API and returns the response. We save the response in meta_info attribute of the Video record. We also save the video. Once Zencoder receives our request, it'll encode our videos. The question arises how we'll know when encoding process completes. The answer is that Zencoder "notifies":https://app.zencoder.com/docs/api/encoding/notifications our application about completion of the process by making an HTTP POST request. We need to tell it where to make that request in "notifications":https://app.zencoder.com/docs/api/encoding/notifications/notifications option. We can also ask Zencoder to send us some data back when it makes this request to our servers. We can use ":pass_through":https://app.zencoder.com/docs/api/encoding/job/pass-through option to specify such data. In our case we'd use the id of the video that is being encoded. We change the uploader code to include those information.

params = {
  :input         => @model.video.url,
  :test          => true,
  :notifications => [zencoder_url],
  :pass_through  => @model.id,
  :outputs => [
    ...
  ]
}

We also include modules that are necessary to use zencoder_url url helper in the uploader code and also define necessary route.

class VideoUploader < CarrierWave::Uploader::Base
  include Rails.application.routes.url_helpers
  Rails.application.routes.default_url_options = ActionMailer::Base.default_url_options
  ...
end

# config/routes.rb
resource :zencoder, :controller => :zencoder, :only => :create

We now write our ZencoderController that will listen to requests from Zencoder API.

class ZencoderController < ApplicationController
  skip_before_filter :verify_authenticity_token

  def create
    video = Video.find_by_id(params[:job][:pass_through])
    if video && params[:job][:state] == 'finished'
      video.meta_info[:response] = params[:zencoder]
      video.save
    end

    render :nothing => true
  end
end

We skip the verification of authenticity token for requests coming to this controller. POST from the API will trigger create action of our controller. In the action, we first find the corresponding video for which the encoding process finished. The id of the video is given to us in the :pass_through parameter of the request. We check if the video exists and encoding state of process was 'finished' successfully. We save all the parameters coming in the request to meta_info of the video. We don't give any response to the request because the request does not expect any. Now the video is completely encoded and we can use it in our application. We can use @video.video.url to get the url of original video. Carrierwave does not know URLs of encoded videos. We'll have to write a little bit more code so that we are able to access encoded videos and thumbnail.

class VideoUploader < CarrierWave::Uploader::Base
  ...
  def thumbnail_url
    @thubnail_url ||= url_for_format('thumbnail', 'png')
  end

  def mp4_url
    @mp4_url ||= url_for_format('mp4')
  end

  def webm_url
    @webm_url ||= url_for_format('webm')
  end

  def ogv_url
    @ogv_url ||= url_for_format('ogv')
  end
  ...

  private
  ...
  def url_for_format(prefix, extension = nil)
    extension ||= prefix 
    base_url + '/' + prefix + '_' + filename_without_ext + '.' + extension
  end
end

Above methods will give us URL to encoded videos and thumbnail. We can access them like @video.video.mp4_url. Now we write some helpers in our Video model that will let us calculate the duration and find out the encoding status of the video.

class Video < ActiveRecord::Base
  mount_uploader :video, VideoUploader
  serialize :meta_info, Hash

  # To locally test the videos, after Zencoder has done
  # transcoding, do following in rails console
  #
  # >> v = Video.find(id) # id of the video model
  # >> v.meta_info = v.meta_info.merge(:response => { :input => { :duration_in_ms => 3000 }, :job => { :state => 'finished' }})
  # >> v.save!
  def duration
    ((meta_info[:response].try(:[], :input).
      try(:[], :duration_in_ms) || 0) / 1000.0).ceil
  end

  def transcode_complete?
    meta_info[:response].try(:[], :job).
      try(:[], :state) == 'finished'
  end
end

You can find code listed in this post in "this gist":https://gist.github.com/waseem/5479964

Testing Super Calls in Modules

2012-08-07T00:00:00+00:00

h1. Testing Super Calls in Modules The DRY principle says that we should take up common functionality between related classes and move them to a common module. While doing so, every once in a while you will come across writing a method in the module that calls super. Let me show you an example.

class Product
  def delivered?
    # Complex logic to see if product was delivered
  end
end

module ProductDeliveryHelper
  def delivered?
    supports_multiple_delivery? || super
  end
end

class GiftCard < Product
  include ProductDeliveryHelper
  def supports_multiple_delivery?
    true
  end
end

class Spoon < Product
  include ProductDeliveryHelper
  def supports_multiple_delivery?
    false
  end
end

class Shirt < Product
  def within_delivery_duration?
    # Complex logic to calculate delivery duration
  end
  def delivered?
    within_delivery_duration? && super
  end
end

Here Product implements a basic delivered? method that checks if a product was delivered. The subclasses of Product have their own logic to see if those were delivered. GiftCard and Spoon are two products that have same behavior of checking for their delivery status. We see if those support multiple delivery and if they do, we also check if they follow the basic delivered? check in the Product class. We have followed the DRY principle here and moved the logic for Spoon and GiftCard classes in ProductDeliveryHelper module. The Shirt class stands out because it does not have a notion of multiple delivery. Instead it has a concept of a delivery duration. Lets say we deliver shirts only during some specific months of the year. Like other good developers, we will test our code. We can write unit tests for different classes fairly easily. We will also have to test ProductDeliveryHelper in isolation. When testing ProductDeliveryHelper#delivered? we want to test its behavior of calling super. But we do not want to couple the test with a GiftCard or Spoon. This way, we will be sure that its behavior is tested in isolation. To test it in isolation, we must have a test object available to us. For this, we will have to create a test class in the test itself.

require 'test/unit'
class ProductDeliveryHelperTest < Test::Unit::TestCase
  class TestParent
    def delivered?
      'super called'
    end
  end
  class TestDelivered < TestParent
    include ProductDeliveryHelper
  end
  def test_delivered_with_no_multiple_delivery
    test_object = TestDelivered.new
    def test_object.supports_multiple_delivery?; false; end
    assert_equal 'super called', ob.delivered?
  end
  def test_delivered_with_multiple_delivery
    test_object = TestDelivered.new
    def test_object.supports_multiple_delivery?; true; end
    assert ob.delivered?
  end
end

Above makes our tests complete. Thanks to workmad3 for answering and discussing my original question in #RubyOnRails.

Multiple Route Files in Rails 4

2012-05-27T00:00:00+00:00

h1. Multiple Route Files in Rails 4 __UPDATE:__ This feture was reverted in favor of "fa736e69a197...2d9dbf416b14":https://github.com/rails/rails/compare/fa736e69a197...2d9dbf416b14 . There is also a "related discussion":https://groups.google.com/forum/?fromgroups=#!topic/rubyonrails-core/EFiL2BHAevI Often in our Rails applications, we end up with a very long config/routes.rb. The application that I am currently working on has a routes file with over 470 lines. As the number of lines grow, the routes file becomes difficult to maintain. Searching where a particular route is defined also becomes quite difficult. With commit "6acebb38":https://github.com/rails/rails/commit/6acebb38bc0637bc05c19d87f8767f16ce79189b made by Yehuda Katz, we can specify multiple route files under config/routes/ directory in our application. Lets say originally our config/routes.rb looks like:

# config/routes.rb
Tesapp::Application.routes.draw do

  resource :session

  resources :comments
  resources :posts
  resource  :profile

  namespace :admin do
    resources :comments
    resources :users
    resources :posts
  end

  root :to => "welcome#index"
end

We can move _admin_ related routes into its own route file to config/routes/admin.rb.

# config/routes/admin.rb

namespace :admin do
  resources :comments
  resources :users
  resources :posts
end

And we tell Rails router to pick _admin_ routes from that file using ActionDispatch::Routing::Mapper::Resources#draw in config/routes.rb.

# config/routes.rb

Tesapp::Application.routes.draw do

  resource :session

  resources :comments
  resources :posts
  resource  :profile

  draw :admin

  root :to => "welcome#index"
end

#draw will try to find a file with the same name as the symbol passed to it and simply call the route generator methods present in it. This way we can make our routes file more manageable and concerned only with one aspect of our application. Routes will be drawn in the order #draw is called. The routes that are drawn earlier will take precedence over the routes drawn later.

Test Post

2012-05-26T00:00:00+00:00

h1. Test Post

class Foo
  def bar
    'bar'
  end
end