29 Steps

Testing API Web Calls in Elixir using meck

Sat, 20 Dec 2014 21:09:00 +0000

Elixir is a powerful functional programming language with an equally powerful test suite call ExUnit. it is possible to write unit and high level integration specs while developing.

As a Rubyist, one of the things that intrigued me was how do you test API calls as one could do in Ruby. I use WebMock a lot in Ruby tests - so is there something similar in Elixir / Erlang?

In a recent elixir project, I needed to do something similar and while reading through the test code for HTTPoison, I noticed a library called meck being used. Its a mocking library developed in Erlang. The following is a gist of an example of mocking api calls in Elixir:

The module under test makes an API call to the Github api to fetch the latest issues for a given project.

After adding meck as a dependency, I import it into the test using import :meck.

In the setup block, I created a new mock object using

:meck.new(Issues.GithubIssues)

The on_exit call unloads :meck and restores the module its original functionality.

Within the test block, I stubbed out the actual method call using

:meck.expect(Issues.GithubIssues, :fetch, fn("elixir-lang", "elixir") -> mock_response end)

. Every call to Issues.Github.fetch(“elixir-lang”,“elixir”) will return the mock response and will not make the actual api call.

Like in WebMock, we can force the module to make the actual API call using

:meck.passthrough(["elixir-lang", "elixir"])

Mixpanel JS in an enterprise application - some wee notes

Tue, 04 Feb 2014 21:57:27 +0000

I currently work in a corporate which is progressive in its thinking on how they would like to monitor user behavior and activity in order to provide the best level of service for their customers. One of the biggest projects I was involved with since inception was the integration of Mixpanel into their main website.

Mixpanel is an event tracking application which allows you to specify which events you would like to track within an application. This is not the same as google analytics say as mixpanel is more powerful than just page or link tracking although that is also built into the library. For more information please visit the mixpanel website for a better explanation.

We developed a custom piece of JS which is incorporated across their current suite of websites and applications but we hit a barrier, which is the amount of data that could be stored on a client’s browser. A typical browser supports only 4k of cookie data from the same domain. If we have multiple applications with subdomains all running mixpanel, that is when disaster strikes as mixpanel stores user data through its own cookies resulting in an overflow issue.

To overcome this, I came up with the idea of using localstorage to store larger volumes of user activity which are then retrieved from storage at the point it is about to be sent to mixpanel. JStorage is an excellent integration choice as it has support all the way to IE 7 and has a simple API for retrieving and setting data. By doing that we managed to eliminate the first issue of storing too much data in the user’s browser.

The second issue was with the way mixpanel stores cookies. If you have multiple applications all running with the same domain even subdomains, mixpanel sets a separate cookie for each. Which means if a user visits your main site ‘mysite.com’ and another application such as 'myapp.mysite.com’, mixpanel sets 2 cookies. Coupled that with storing super properties, it would cause an issue.

This behaviour can be disabled using the following snippet:

mixpanel.init(<API KEY>, {cross_subdomain_cookie: false})

The above would only set a single cookie for each subdomain the user visits on your site.

I hope to provide more concrete examples of the inner workings of mixpanel and some of the pitfalls to avoid in your own applications.

Mongoid currency price search

Tue, 04 Feb 2014 21:40:29 +0000

In an ongoing project I am working on, I am using the excellent Money gem with money-rails integration within a Rails 3.2 application and running on top of MongoDB.

If you are familiar with the gem, it stores the price as a type of Money object within mongodb which means typical full text search with greater than or less than will not as expected.

In order to do a search on price, you would need to create a Money object within the search criteria as follows (assuming my main model is a Product document):

Product.where(:price => Money.new(899,‘EUR’)).first

The above example looks for a product priced at 8.99 euros.

I’m still in the process of working out how to search for a given price range using the money-rails gem.

Dynamically output HTML within AngularJS

Tue, 10 Sep 2013 21:27:00 +0100

I recently started to integrate AngularJS into a Rails application which has an external API to speed up the view rendering process and also as an exercise into using another JS framework apart from Backbone.js

The AngularJS app essentially renders the json output from the existing API and renders it in the view. The only issue is that some of the JSON string is actually raw HTML. If you render this in the view it gets returned as pure string and not actual HTML. Using either ‘raw’ or ’.html_safe’ will not make any difference as the rendering is with Angularjs framework.

To be able to output dynamic HTML , we would need to make use of the ngSanitize module by including it within the application. These are the steps I took to get the html output to show:

include angular-sanitize within application.js


//= require angular
//= require angular-resource
//= require angular-sanitize

Include ngSanitize into your angular module:

app = angular.module("MyApp", ["ngResource", "ngSanitize"])

Within the main template use the ng-bind-html directive within the appropriate html tag to display the content:
```
<div ng-bind-html='myhtmlstring'></div>
```

Further information and more examples of this usage can be seen on the angularjs site: http://docs.angularjs.org/api/ngSanitize.$sanitize

Rebinding UJS actions in Rails 3.2 application

Wed, 04 Sep 2013 12:20:00 +0100

In a recent rails 3.2 application I am refactoring, some of the js actions had to be redefined to use the ‘on’ event rather than 'live’ as that is deprecated since jquery 1.7.

By doing so some of the events stopped working immediately. This is due to the fact that the 'on’ event has to be defined on the document body rather than the individual elements in question.

The example I have provided is a simple example of liking a blog post. The user would see a 'like’ link within a blog post and upon clicking on the button it would call a UJS action within the rails application which has a corresponding JS template containing certain actions that must be triggered to update the view. If we change the implementation to define 'on’ event for the like link element once the view gets updated the event is lost until a page refresh.

If we define it in the context of the document object it persists and no other changes need to be done to the remaining js or UJS template code.

For further information on how to rewrite 'live’ actions to 'on’ please refer to the 'live’ documentation on the jquery website:

http://api.jquery.com/live/

Below is a gist of the code in question mentioned above.

https://gist.github.com/cheeyeo/6435628

Refinery CMS is_match() nil after asset compression using closure compiler

Sat, 29 Jun 2013 22:51:33 +0100

In a recent project using RefineryCMS the page speed analytics results keep indicating that the main application.js after compression is still showing trailing whitespaces and hence triggering ‘js file requires compression’ warnings when tested within google page speed

In an attempt to fix this I switched over to using the closure compiler during assets compilation. However after deploy the entire compiled javascript for refinery failed with the following error inside console:

is_match() is undefined

This issue is mentioned here and has been fixed on the main refinerycms master branch on github.

Heroku + Assets + Dragnfly + Optimization + CDN

Sat, 29 Jun 2013 22:43:00 +0100

In a recent client project, in order to speed up the loading of assets and to improve the overall page speed score, I decided to store the assets outwith of the app and to be able to use the dynamic asset hosts functionality within rails.

This works really well out of the box apart from one caveat: the app uses Dragonfly to serve dynamic images which gets resized based on certain conditions.

I originally used asset_sync to store the precompiled assets into Amazon S3 but of course this fails miserably as S3 cannot find images with urls beginning with ’/system/’ etc as that is supposed to be handled by the Dragonfly engine whereas asset_sync is looking for a folder or directory path inside S3 !!

Add to that the complexity of adding in a cloudfront CDN

The solution I came up in the end was not to use asset_sync but to precompile all the assets before deployment. Then I added a CNAME entry such as ‘assets.mysite.com’ to point to the actual domain 'mysite.com’ This is what you would do anyway if you are using asset hosts. Make sure you wait and test the CNAME to make sure it points to the domain 'mysite.com’ before proceeding further.

Then within the CDN, set up an origin for 'assets.mysite.com’ and wait for it to finish setup and turn green. if you start testing before it turns green you will get some weird 503 unavailable errors or your assets will 404 with 'Image returned with type of text/html’ etc

Within the config/production.rb add the following line:

config.action_controller.asset_host = "//test.cloudfront.net"

This will make sure that all the assets requests pass through to the CDN rather than the app itself. It also solves the problem of using a dynamic image engine such as Dragonfly while keeping your compiled assets intact on heroku.

Another issue to note is that on Heroku cedar stack there is no reverse proxy cache which means your html will not be minified. The following links from heroku mentioned this: Link 1 and Link 2. This can be fixed by using the heroku defalter gem which will automatically gzip both images and html responses.

In initial tests, the page speed for various pages drop from an intial 4-5ms to under 1ms when the CDN has cached all the assets properly.

If you still wish to use asset_sync to serve assets from S3 with Dragonfly, there are two possible options:

1. Generate a static image for each version required e.g. thumbnail version; actual version
2. Specify the host within the dragonfly initializer. In my case the dragonfly is within a gem which preloads it so i did not even attempt to spend hours overwriting the initializer.

Rails 3 convert superclass to subclass

Wed, 20 Mar 2013 12:24:00 +0000

In a recent rails 3.0 project I was working on I had an interesting dilemma. I have two models, User and Member. Member is a subclass of User.

Now when a user logs into the system the system checks what roles and permissions each user has and then redirects them accordingly. e.g. if the user record has a role of ‘Member’ then we redirect them to the member section.

Now since Member is a subclass of User, we are using STI ( Single table inheritance ) within rails whereby all the member attributes are stored as columns within the single user table. Member has an additional method called active? which determines if the membership level is active or not. However, in the logic which checks where to redirect signins because only User objects are returned, each call to active? is made on the superclass of User and not Member. To fix this I could redefine active? within the User model but it means duplication of the active? method inside Members class.

Another alternative would be to convert the User model into a Member object. ActiveRecord instances have a method called becomes(klass) which takes the name of the class to convert to. The definition of becomes from the API docs as follows:

Returns an instance of the specified klass with the attributes of the current record. This is mostly useful in relation to single-table inheritance structures where you want a subclass to appear as the superclass.

Full link of the api can be found here: http://api.rubyonrails.org/classes/ActiveRecord/Persistence.html#method-i-becomes

Now we are going the other way round from superclass to subclass which is unorthodox but works in my case here. An example of what I ended up with:

Example of refinerycms nested model links with image dialog

Tue, 19 Mar 2013 16:23:51 +0000

Example of refinerycms nested model links with image dialog

Nested model partials in Refinery CMS

Tue, 19 Mar 2013 15:56:00 +0000

In a recent refinerycms project I noticed a strange bug / error after upgrading the system to one of rails’s security patches.

Within a custom engine model, there is a nested form which uses the cocoon gem to show and remove itself. Within the nested form itself, is a link to call the js image picker within refinerycms to select an image for upload. Problem is this: because the nested field is dynamic which means that the image selector link is not generated until the ‘Add This’ link is clicked and the nested form is on the screen, the image selector link does not get instantiated within a new nested model form.

This can be seen with this piece of js code from refinerycms-core ver 1.0.11, in public/javascripts/refinery/admin.js


  $(document).ready(function(){
    // other initialization code
    init_modal_dialogs();
  })

init_modal_dialogs() is a function which opens up a modal window in an iframe for selecting images or attachments. Since my dialog selectors are not going to be present until the nested fields are added, I bind the click event to the live action once the link is clicked. e.g.


  $('a.add_nested_image_link').live('click', function(e)){
     init_modal_dialogs()
   })

This fixes the dialog open and also shows the currently selected image within the nested form.

To remove the image or to replace it is a bit tricky.

Refinerycms by default uses a class of ’.remove_picked_image’ to define the remove image link within the dialog. Here we have another problem. Since the dialog is not created until the nested model forms are added, we need to bind the 'remove_picked_image’ links to the live event too else it won’t work:

Please note that the above only works if you are using a nested model form within your own engine in refinerycms with the cocoon gem.

Some screenshots are attached in the post preceding this.

Money gem + Mongoid

Tue, 19 Mar 2013 15:19:50 +0000

I have just switched over to using the money and money-rails gems in a recent rails 3.2 app recently after having issues with the formatting and storing of currency values in a Mongoid db. Plus, the money-rails gems comes with some nifty view helpers to transform the values into suitable representations on the front end.

Below are some notes to remind myself how I managed to do this within a Rails 3.2 appli<codebe foRedefine the field within the mongoid document to be of type Money. e.g. Assuming we have a Product document with a field of price, we can define it as follows:

```
nufield :price, type: Money
```
Once defined the currency values will be stored as a hash in the document with the following structure:
```
price: {"cents" => "100", "currency_iso" => "GBP"}
```
The price value is converted into cents automatically by the gem e.g. 1 USD gets converted into 100 cents. The currency_iso sets the country code of the currency according to the iso 4217 standard which maps the country to a specific currency code.

The price value is converted into cents automatically by the gem e.g. 1 USD gets converted into 100 cents. This is important to remember else you will end up with strange values.

Since I will be accessing this value quite frequently i also added an index for it.
Use the money-rails gem to help format the currency values inside views. The gem contains useful helper methods such as humanized_money_with_symbol and humanized_money to convert the currency value into a useful string representation for display.
To help validate against the price value since it is set in a form, I still need to add some validations to make sure that the price value is present and valid.
```
validates :price, :numericality => {greater_than_or_equal_to: 1.0}, :presence => true
```
Money gem provides a numerical validator to catch non-integer values such as strings and this can be found inside the money-rails gem lib folder ‘lib/money-rails/active_model/validator.rb’
Passing in a string value to a money object field through a form does not save it properly for the simple reason that the field is no longer of type string or decimal but of object Money hence any value will need to be converted first before it can be saved.

To cate for this I created a before_save callback which formats the form value into a money object before saving it:
```
before_save :format_price

def format_price
  self.price = Money.new((BigDecimal.new(self.price.to_s).round(2) * 100).to_i, 'GBP')
end
```
The price value needs to be converted into its cents equivalent and its currency code set in the callback.
To test that the attribute returns a Money object on successful save I have the following structure inside my rspec2 model specs:

Note that within the tests, the numeric validator within money-rails is testing for the presence of strings and malformed decimal places while the other validations are added by myself.

I hope the above helps someone to get to grips with using Money and money-rails gems in their own projects. Please comment on better ways of doing the above if you have come across similar setup in your own projects.

likeafieldmouse: Sayaka Ganz - Emergence (2011) - Discarded...

Tue, 19 Mar 2013 13:24:07 +0000

likeafieldmouse:

Sayaka Ganz - Emergence (2011) - Discarded plastic

"I’m not gonna sit around and waste my precious divine energy trying to explain and be ashamed of..."

Tue, 19 Mar 2013 13:23:50 +0000

“I’m not gonna sit around and waste my precious divine energy trying to explain and be ashamed of things you think are wrong with me.”

- Esperanza Spalding (via likeafieldmouse)

likeafieldmouse: Lynda Benglis

Tue, 19 Mar 2013 13:18:11 +0000

likeafieldmouse:

Lynda Benglis

robotcosmonaut: 宇宙怪獣ガメラ via gurafiku Back to the eighties...

Tue, 19 Mar 2013 13:17:27 +0000

robotcosmonaut:

宇宙怪獣ガメラ

via gurafiku

Back to the eighties again

Just completed phase 1 of safagrow.com to make it mobile...

Tue, 12 Mar 2013 20:19:55 +0000

Just completed phase 1 of safagrow.com to make it mobile friendly or responsive. The home and blog sections are working and am working on the other sections of the site. Done using Zurb Foundation. Check it out on your mobile. safagrow.com

Photo

Thu, 07 Mar 2013 21:51:39 +0000

More on SSL - some notes Feb 2013

Sat, 23 Feb 2013 17:43:02 +0000

My previous post was platform specific and mentions some of the problems I encountered while using the Heroku PAAS using SSL. I aim to be more generic if possible and point out some of the pitfalls I made and to avoid.

Applying for your first SSL certificate

Depending on which hosting provider you use, some providers can apply for a SSL certificate on your behalf while some require you to do the manual work.

If you are using dedicated hosting such as Parallels with a plesk panel, they provide the private key and CSR request when you click on ‘Add SSL Certificate’ when you go to the specific domain. PLEASE USE THESE KEYS TO APPLY FOR YOUR CERTIFICATE. I ignored it and had 'hours of turmoil’ trying to work out what the incompatibilities issues are! Some images are included below for reference.

If you are using your own self generated key and CSR, it is also important to note that some CAs require your CSR to be formatted in a certain way. For example, on RapidSSL, they require you NOT TO add 'email’, 'password’ and 'optional business’ within your CSR request. Other CAs will have similar requirements so make sure you read the DOCS before going any further.

Some CAs also require an additional level of confirmation before proceeding such as domain verification. This would normally involve sending the ssl email request to a special email address that begins with 'admin@mysite.com’ or 'administrator@mysite.com’. Again make sure these email accounts exist before applying for the SSL certificate.

Heroku SSL Endpoint + Rails 3.2

Sat, 23 Feb 2013 17:17:00 +0000

I had the opportunity to deploy and use SSL on heroku and it has been a while since I had to do this. Heroku’s SSL has now been moved to a dedicated endpoint which means all https traffic to your heroku app will be re-routed from the endpoint to <yourapp.herokuapp.com>

After setting up and deploying the SSL certificates to heroku successfully, I realised i had to set a DNS CNAME entry for the SSL endpoint.

This raises an interesting problem: the Heroku application I am working on already has a CNAME record pointing to the heroku domain. If you change the CNAME setting to just the SSL endpoint then the entire app cannot be found.

The way I found to resolve this is as follows:

Within your DNS panel set an ALIAS to <yourapp.herokuapp.com>
Change the CNAME entries for <yourapp.herokuapp.com> to point to the ssl endpoint e.g. <ssl.herokuapp.com>. This can be found using the command heroku certs from the application directory. If you have setup the certificates properly this will show you a table with the matching ssl endpoint and your app url.
Within your app make sure that config.force_ssl = true if you are using SSL for the entire app or create your own custom ssl filters inside application_controller.rb or otherwise.

If it all goes well you should see your site with a lock icon and your certificate details should be shown:

I bought the SSL certificate from Rapid SSL and it was processed within minutes and another half an hour to figure out how to setup and to implement SSL. There are some pitfalls to avoid when registering your SSL certificate for the first time and this will be highlighted in the next article from this.

I used to think that SSL is an ‘addon’ and only required if you are doing ecommerce or storing some form of personal information. With the levels and frequency of hacking attempts made I think the initial investment is worthwhile for peace of mind for your data security and integrity. Even though SSL does not prevent or stop any hacking or hijacking attempts at your app or site at least it is a level of defense or barrier between you and the dangers out there rather than having none at all.

For a more detailed discussion or article on why SSL is important check out this article on html5rocks:

http://www.html5rocks.com/en/tutorials/security/transport-layer-security/

Below are links for setting up SSL on heroku:

https://devcenter.heroku.com/articles/ssl-endpoint

https://devcenter.heroku.com/articles/ssl-certificate

https://devcenter.heroku.com/articles/ssl#customdomain-ssl

Please pay attention to 'Preparing your SSL certificate’ section as it requires merging your CA (Certificate Authority) intermediate cert with your issued certificate.

Postgresql Shared Memory error

Tue, 08 Jan 2013 13:51:53 +0000

Ran into this interesting issue this morning while playing around with Rails 4.0. Normally postgresql would just run after running pg_ctl but this time it did not appear inside the process list. Upon checking the log file this is the error that was shown:

FATAL: could not create shared memory segment: Cannot allocate memory

DETAIL: Failed system call was shmget(key=5432001, size=3809280, 03600).

HINT: This error usually means that PostgreSQL’s request for a shared memory segment exceeded available memory or swap space, or exceeded your kernel’s SHMALL parameter. You can either reduce the request size or reconfigure the kernel with larger SHMALL. To reduce the request size (currently 3809280 bytes), reduce PostgreSQL’s shared memory usage, perhaps by reducing shared_buffers or max_connections.

What this means is that there is insufficient shared memory to start the postgresql process. in my case it was caused by running virtualbox in the background.

What I did to solve the above was this:

sudo sysctl -w kern.sysv.shmall=65536

The command increases the shared memory usage system wide and postgresql was able to run again. To make this appear after reboot you also need to do add this to /etc/sysctl.conf:

kern.sysv.shmall=65536

Works for me on OS X Lion