Dev logger

Is the Future Functional? by Rob Harrop

noreply@blogger.com (Martin Carel) — Tue, 25 Sep 2012 00:10:00 +0000

Here are my notes on Rob Harrop's presentation about functional programming.

Functional is important because

1) Simplicity

it's declarative
you write what you want to achieve, rather than give instructions how to achieve it
express solutions, not worry about individual steps

2) Agility

functional fits well with Agile methodology
testing is much easier (because of purity, no side-effects)

3) Contentment

self-content while you are working

4) Concurrency

e.g. parallelization
often the reason why people switch to functional

Imperative vs functional languages

Functional:

function application (calculations to data), not state change
immutable state (makes testing easier)

Imperative:

focus on decisions (instructions), and state change
mutable state

Functional:

recursion not looping
descriptions (of solutions), not statements

Functional features

higher-order functions
pure functions (same args in always lead to same results)
recursion
type systems (Haskell: strong typing; Erlang, LISP: loosely typed)
strict/lazy evaluation

DDD: primary focus should be on the domain and domain logic

DDD concepts

ubiquitous language
strategic design (identify which bits are harder to model and put time on it)
models
context

Domain-driven with imperative programming leads to anxiety. Always wondering if you got your model right, because there are a myriad of ways to implement a solution.

Functional DDD

core concepts still apply
everything is just a function or data structure

Why is functional better?

fewer concepts (no factories, no patterns)
easier concepts
lower model overhead

Modelling

UML doesn't really work for functional
fortunately, we all know how to model in functional languages

Transferring large files: easy and free!

noreply@blogger.com (Martin Carel) — Thu, 13 Sep 2012 15:10:00 +0000

I needed to transfer a 4GB file the other day. Unfortunately, there is no easy and free way to do it. So I went the tech way.

How to

Upload the file to my_server (e.g. your server slice) via scp:
scp my_large_file my_server:.

In case your connection breaks up, you can resume the upload like this:
rsync --partial --progress my_large_file my_server:.

Log to your slice and move the file to a directory of its own:
ssh my_server
mkdir pub_data
mv my_large_file pub_data/

Start a simple HTTP server on the port of your choice (in this case 8080), by running:
python -mSimpleHTTPServer 8080 &

That's it! Your file is now available for download at:
http://my_server_hostname:8080/my_large_file

If you want to have that HTTP server process running even after logging out, find out its PID by running the jobs command, and then you can disown that process from the shell like so (assuming the PID was 1):
disown %1

Advantages

no need to go through the hassle of installing a FTP server on your own machine.
don't care about leaving your machine turned on because you don't know exactly when the other party will be fetching the file.
the other party is not slowed down by your ISP's upload limits while trying to fetch the file from you.

How to remove every n-th line from a file

noreply@blogger.com (Martin Carel) — Tue, 29 May 2012 21:31:00 +0000

A bit of googling made me read this article, which was about sampling a file. What I really wanted was to remove every fifth line from a file.

Here is the input file:

1

00:00:06,244 --> 00:00:09,004

The public has a very romantic idea of what wine's about.

Le public a une idée très romantique du vin. 

2

00:00:09,984 --> 00:00:14,373

They think it's some little guy, like me, working in the cellar,

On pense que c'est un petit producteur, comme moi, qui travaille à la cave,

3

00:00:14,473 --> 00:00:16,741

working in the vineyard, bringing the grapes in.

qui travaille dans le vignoble, qui vendange ses raisins.

And I wanted to have that kind of output:

1

00:00:06,244 --> 00:00:09,004

Le public a une idée très romantique du vin. 

2

00:00:09,984 --> 00:00:14,373

On pense que c'est un petit producteur, comme moi, qui travaille à la cave,

3

00:00:14,473 --> 00:00:16,741

qui travaille dans le vignoble, qui vendange ses raisins.

So this made the trick:

awk '{if ((count++ +2)%5!=0) print $0;}' input_file.srt > output_file.srt

JavaScript: The Good Parts by Douglas Crockford

noreply@blogger.com (Martin Carel) — Fri, 30 Mar 2012 23:38:00 +0000

I have really enjoyed reading JavaScript: The Good parts. It has been recommended to me by many friends, and I'm certainly not disappointed. Douglas Crockford did an excellent job.

The book is highly opinionated, and that's a good thing, given that the author is seen as a prolific and talented Javascript expert by his peers. Here is an excerpt that gives you an idea of the book's tone:

The best thing about JavaScript is its implementation of functions. It got almost everything right. But, as you should expect with JavaScript, it didn't get everything right.

Things to remember

The prototype link is used only in retrieval. If you set a new property on an object (not on the prototype), it sets it on the receiver only.
The hasOwnProperty method does not look at the prototype chain. It looks in the receiver only.
Objects are passed around by reference. They are never copied.
It is important to understand that the inner function has access to the actual variables of the outer functions and not copies.
Objects produced from object literals are linked to Object.prototype.
Function objects are linked to Function.prototype (which is itself linked to Object.prototype).
JavaScript does not have block scope. JavaScript does have function scope.
The general pattern of a module is a function that defines private variables and functions; creates privileged functions which, through closure, will have access to the private variables and functions; and that returns the privileged functions or stores them in an accessible place.
If all of the state of an object is private, then the object is tamper-proof. Properties of the object can be replaced or deleted, but the integrity of the object is not compromised. If we create an object in the functional style, and if all of the methods of the object make no use of this or that, then the object is durable. A durable object is simply a collection of functions that act as capabilities.

Advice

It is best to declare all of the variables used in a function at the top of the function body.
One way to minimize the use of global variables is to create a single global variable for your application: var MYAPP = {};

Is this book for you?

If you are wondering if you should read that book, you could test yourself. If you fully understand the code snippet below, you can skip it. If not, then you might want to invest in some quality reading. It's an excellent book to beef up your Javascript understanding.

Function.method('bind', function (that) {

  var method = this,

  slice = Array.prototype.slice,

  args = slice.apply(arguments, [1]);

  return function (  ) {

    return method.apply(that,
      args.concat(slice.apply(arguments, [0])));

  };

});

var x = function (  ) {

  return this.value;

}.bind({value: 666});

alert(x(  )); // 666

An Introduction to Procs, by Peter Cooper

noreply@blogger.com (Martin Carel) — Thu, 02 Feb 2012 02:10:00 +0000

I have just watched the 27-min introduction on Procs by Peter Cooper, which is really a short section of his RubyReloaded course.

Here are my notes:

blocs are essentially nameless functions
the block_given? method can be used to find out if a block was passed to the current scope
blocks are anonymous functions which can be passed around and called at will
one can only pass a single block to a method, but one can pass multiple procs around (as arguments of a method)
Proc.new will create a Proc object if there was a block passed in the current context (otherwise it will raise an error)
In Ruby 1.9, there are 4 ways to call (or run) proc objects:

my_proc = Proc.new do |e|

    puts "this is #{e}"

end

my_proc.call(2)

my_proc.(2)

my_proc[2]

my_proc === 2

a lambda is a proc object which respects argument arity, much like an anonymous method
in 1.8, "proc" (surprisingly) created a lambda (and not a Proc object)
in 1.9, "proc" is the same as doing Proc.new (as we would expect)
a return in a proc will try to return from the context where the proc was defined (and not from where it is executed)
in a closure, ruby keeps references to variables. You can thus modify a variable's value inside a proc even after the proc was defined. Thus this code will output "Marc":

def run_proc(p)

  p.call

end

name = "Fred"
my_proc = proc { puts name }
name = "Marc"
run_proc my_proc

Architecture the Lost Years, by Robert Martin

noreply@blogger.com (Martin Carel) — Wed, 11 Jan 2012 03:20:00 +0000

I watched a keynote from Robert Martin during the Ruby Midwest 2011 conference. It's always fun to watch veterans from the field.

Here are some thoughts:

Decoupling

the whole talk is about decoupling
the web is just a "delivery mechanism"
Ivar Jacobson's use case driven approach:

make the intent of your system obvious
architecture: boundary, interactor (app specific), entity (app agnostic)

we should keep in mind that:

the delivery mechanism (e.g. the web) is "just a detail"
the database is "just a detail"

Architecture

art of architecture: drawing lines
architecture is about:

drawing lines
when you have a line crossing another, it's a dependency
making sure all dependency lines goes into the same direction (when crossing a line)
once you have that you have a gem/plugin

a good architecture allows major decisions to be deferred
if the first thing you type is "rails new_app", you've lost it. Rather, you must think about your entities, interactors and boundaries
a good architecture maximizes the number of decisions NOT made
in software engineering, at the end of the 80s, we were on the verge to understand that software architecture needed a use case-driven approach. But then the web got us away from it. And now we seem to be in the process of remembering it.

Ruby Object Model and Metaprogramming screencasts by Dave Thomas

noreply@blogger.com (Martin Carel) — Sun, 18 Dec 2011 23:24:00 +0000

I bought 6 screencast episodes of Ruby metaprogramming about 3 years ago and only recently have I watched them all. And I must say: it was quite worth it! Just a reminder that it's never too late :) See below for my notes (which have been approved by Dave Thomas himself).

Episode 1 - Objects and Classes

Think in terms of object-oriented programming instead of class-oriented programming
2 ways to change self:

by a method call with an explicit receiver
by a class or module definition

a class does not have a name until you assign it to a constant
classes are first-class objects
class methods don't exist: they are actually singleton methods on class objects
method calling always works the same way. The ruby interpreter:

identifies the class of the receiver
looks up the method in that receiver's class
if it does not find it, it goes up the hierarchy, and looks up the method in the parent of that class
retries the last step until it finds it

Episode II - Sharing Behavior

Object#clone() copies the singleton method too; Object#dup() does not
with a prototype, you can clone both the behavior and the state of an object (which is not the case with class-based inheritance)
one can subclass from any expression, e.g. class A < (rand > 0.5 ? B : C)
do something in the singleton class like so: class << self ; # ... ; end
modules have 3 distinct usages:

namespace
to create module methods
to create instance methods

module methods can still be modified even if they have been included in a class, by reopening the module
a method of an included module will come before a method defined in a superclass, during Ruby's method lookup
extend(module_name) actually opens the receiver's singleton class and sticks the content of module_name in it
reason because Person < ActiveRecord::Base is conceptually wrong: the intention is not to build a hierarchy (a "is a" relationship), but rather to share behavior

Episode III - Dynamic Code

several ways to create blocks. Worth mentioning:

lambda cares about arity (just like a method), proc does not
a "return" inside a proc exits surrounding context
a "return" in a lambda exits the lambda
a "return" inside a do/end block will exist the surrounding context
lambda : like an anonomymous method
Proc.new : like inline code

a Kernel method is available everywhere in our program
eval() takes a string and evaluates it as ruby code
a binding encapsulates:

self,
local variables (including method params)
any associated block
return stack

a proc object always has an associated binding
nothing prevents you from defining a method within a method
define_method() is only available in modules and classes

Episode IV - instance_eval() and class_eval()

instance_eval() can be called on any object
class_eval() is an alias to module_eval
class_eval() can only be called on classes and modules
a method definition in class_eval() evaluates in the receiver's class (thus creating instance methods)
a method definition in instance_eval() evaluates in the receiver's singleton class (thus creating class methods)
"self" in a block of class_eval() or instance_eval() is set to the receiver's
include() is a private method
you can change the "self" of a block with instance_eval(&block), e.g.

class Robot

  def move(&block)

    instance_eval( &block )

  end

  def up;  puts "u"; end

  def down; puts "d"; end

end

robot = Robot.new

robot.move do

  up   # no need for robot.up()

  down # no need for robot.down()

end

Episode V - Nine examples

memoization: @memory[skus] ||= expensive_calculation()
interesting examples making use of several metaprogramming techniques seen in previous episodes. If there's one episode to watch, make it that one!

Episode VI - Some hook methods

Struct.new returns a subclass of Struct
method-related hooks:

method_missing
method_added
singleton_method_added
method_removed
singleton_method_removed
method_undefined
singleton_method_undefined

Class and Module hooks:

inherited
append_features
included
extend_object
extended
initialize_copy
const_missing

Marshalling hooks:

marshal_dump
marshal_load

Coercion hooks:

coerce
induced_from
to_xxx , e.g. to_s, to_proc

advice: don't forget to call the original method when writin code in the hook method to preserve behavior

Ssh credentials not forwarded when using Capistrano on Mac OS X Snow Leopard

noreply@blogger.com (Martin Carel) — Tue, 06 Dec 2011 04:45:00 +0000

I'm running Snow Leopard and it seems like a fresh Capistrano install did not forward my ssh credentials when I was deploying my Rails 3 app on a remote server, having my git repo on beanstalk (similar to the popular github).

After a lot of googling, I understood the problem and went on to fix it. For me, it was a matter of adding my private ssh key to my ssh-agent, like so:

ssh-add ~/.ssh/id_rsa

Then, when running:

ssh-add -L

I could finally see my public ssh key listed (before, no keys were shown). I re-fired a "cap deploy", and it finally worked. With 3 hours of my life down the drain.

Ruby quiz (intermediate) #6

noreply@blogger.com (Martin Carel) — Tue, 22 Nov 2011 22:32:00 +0000

Question: What would be the output of this code snippet?

 1 class A

 2   attr_accessor :var

 3   def method

 4     var = 45

 5   end

 6 end

 7 

 8 a = A.new

 9 a.method

10 puts a.var

Solution: The output is "nil". Here is the answer, taken from Programming Ruby 2nd edition (p. 86):

[W]ritable attributes have a hidden gotcha. Normally, methods within a class can invoke other methods in the same class and its superclasses in functional form (that is, with an implicit receiver of self). However, this doesn’t work with attribute writers. Ruby sees the assignment and decides that the name on the left must be a local variable, not a method call to an attribute writer.

As such, the attribute :var has never been assigned any value, and thus at line 10, its value is still "nil". More appropriately, self.var = 45 would have assigned the value 45 to the attribute :var.

Ruby 1.9 Walkthrough by Peter Cooper

noreply@blogger.com (Martin Carel) — Sun, 20 Nov 2011 14:17:00 +0000

I have been away from hacking some Ruby for about a year by now, and I wanted to get up to speed with Ruby 1.9.3, so I bought Peter Cooper's Ruby 1.9 Walkthrough. No doubt: my $24 was excellent value for my money.

That's a 3-hour session where he talks about what's new in Ruby 1.9.2 and 1.9.3. I certainly recommend it because it's fun, pedagogical, and rather comprehensive.

Here are my notes (which have been approved by Peter Cooper himself):

Strings

Parse lines in a String object with #each_line or #lines, instead of #each
String#ord returns the UTF-8 index
"x".ord is the new way of doing ?x
?a == "a" (and no more "97")
"A".ord == 65, "ABC".ord == 65
String#codepoints to get an enumerator of each codepoint (UTF-8 index)

> "Aab".codepoints.each {|x| print "#{x} "}
65 97 98  

 => "Aab"

String#clear clears to an empty string in place
String are now in terms of characters (UTF-8 by default) rather than bytes
String#length now returns length in chars, not bytes
String#[] now returns entire characters, not individual bytes
String#encoding returns the relevant Encoding object for the string
Encoding.list.length = 95 encodings
tip #1: explicitly declare encodings on any IO objects you're opening
tip #2: add the magic comment on top of all source files (# encoding: utf-8)

Hashes

new hash syntax: {name: "value"}
a Hash instance is now an ordered Hash
Hash#select now returns a Hash instead of an Array
Hash#select! is an in-place version of Hash#select
new methods: Hash#default_proc and Hash#default_proc=
Hash#flatten turns the Hash into an interleaved Array
new methods: Hash#assoc and Hash#rassoc
Hash#keep_if amends a Hash in place

Arrays

Array#to_s no longer joins, but returns an #inspect-style representation
Array#choice goes, Array#sample arrives (and takes an optional quantity arg)
Array#shuffle is still around
Array#uniq, Array#uniq! and Array#product now take blocks
not new but useful: Array#inject can take a symbol representing a method to run each time, e.g. inject(:+)

Procs and Lambdas

proc {} now creates a Proc, not a lambda like in 1.8
check if Proc objects are lambdas with #lambda?
recap: lambdas enforce arity (argument count), regular procs do not
"stabby" lambda syntax: ->x{x * 2 } or ->(x,y,z){x * y * z}
Proc#[] still works for calling a proc/lambda (as does #call)
recap: Ruby 1.8 lambdas didn't enforce arity when no params were listed, and it's been fixed in 1.9
recap: Ruby 1.8 lambdas with 1 param only gave a warning when called with many args, and it's now fixed in 1.9
.() calls the #call method on proc/lambda, but also on other types of object

> a = {}; def a.call; 20; end; a.()

 => 20

Proc#=== will call the proc/lambda (useful in case statements in a "when" block)

 > ->x{ x.odd? } === 3

  => true

Proc#curry takes a multiple arg proc and turns it into a chain of single arg procs

>
 > ->(a,b,c){ a + b + c }.curry[1][2][3]

  => 6

Proc#source_location returns [filename, line_number]

Blocks

Block parameters are now always local to their block
non block parameters are not seen in a different scope when in a block
in a block parameter list, you can list block-local variables, after a semi-colon
you can now pass a block when calling a proc:

> proc {|&b| b.call("hello") }.call {|c| puts c}
hello
 => nil

Enumerator was back-ported to Ruby 1.8.7
Enumerator#with_index accepts an optional starting index
Enumerator#with_object accepts an object, e.g. [1.2.3].each.with_object({}) {|i,a| a[i] = i}
Enumerator#peek checks out next item w/o advancing pointer

Regular Expressions

"abc123" =~ /[[:digit]]/
"a++" returns 1 or more "a" but 'possessive' (no backtracking allowed by parser)
(?= … ) is a zero-width positive lookahead
(?<= … ) is a zero-width positive lookbehind
(?! … ) are zero-width negative lookahead
(?<! … ) is a zero-width negative lookbehind
(?<name>pattern) is a "named match" or "named group" called "name"
MatchData#[] lets you use matched names using symbols, e.g. md[:name]
(?<name>…){0} doesn't match immediately but stores regex for later
\g<name> makes use of the named match "name" at the current point
Unicode properties: \p{property name} (\P{property name} for opposite)

Threads

Threads are real system-level threads
GIL (Global Interpreter Lock) still applies
enforce that only one thread can run at any one time in the Ruby VM
IO is async thread (another thread can run while IO is occurring)
C extensions have the option to "release" the GIL
Thread#set_trace_func and Thread#add_trace_func

Fibers

lighter way of doing concurrency
fiber is a subroutine with multiple entry points
Ruby VM uses fibers to implement enumerators internally
Fiber#new to create a new Fiber object
Fiber#resume will resume execution of the fiber
Fiber.yield yields control (and sometimes data) back to the caller

Time

Time#parse was using the American format; it'snow using the British format, e.g. Time.parse("30/12/2001")
We now have: Time#monday?, Time#tuesday? , etc
Time.now.usec gives the microseconds
(Time.now == Time.now) will return false
Time.now.subsec returns a fraction of 1 000 000 usec

New in the Standard Library

JSON support: #to_json and JSON.load(js)
j and jj
Syck and Psych
MiniTest runs test methods in a random order to test their fragility
Need to add "minitest/autorun" to make MiniTest tests run automatically
assert_not_[something] becomes refute_[something]
must_equal() becomes wont_equal() in the negative
FasterCSV becomes CSV
Foreign Function Interface library
Fiddle library : a nice abstraction to make using FFI easier
rake is now included
the Tk Windowing Toolkit is now included
Ripper library to see how Ruby 1.9 parses Ruby to an AST
Prime is a prime library
Coverage library gives basic coverage stats for your code (must load file)

Coverage.start
require_relative 'file'
puts Coverage.result

Gone from the Standard Library

date2 (was a derivative of date)
ftools library (FileUtils does what ftools used to do)
The Generator class (we've got Enumerator now)
getopts (we should use optparse)
jcode library (used to handle Japanese EUC/SJIS strings)
mailread library (use TMail instead)
parsedate
ping (used for TCP echoing)
readbytes library
RubyUnit (old way of doing unit testing)

New Syntax and misc elements

RUBY_VERSION
RUBY_ENGINE

>  def my_method; p __callee__; end; my_method
:my_method
 => :my_method

File::Stat#world_readable? and #world_writable?
Many File methods can use objects that implements a to_path method

a = []; def a.to_path; '/etc/passwd'; end; File.open(a)

Object#id was deprecated and it's now removed. Use #object_id instead
Object#tap is a "passthrough" method for inspecting things
Negative operators can now be defined/overridden (i.e. !, !=, !~)
Splat operator enhanced (can be applied to non-last parameters, or have multiple splats on the right-hand side)
Optional arguments can now appear before mandatory ones

('a'..'z').include?('car')
 => false
('a'..'z').member?('car')
 => false
('a'..'z').cover?('car')
 => false

Kernel#methods (and friends) now returns an array of symbols, not strings anymore
Block args can no longer be instance variables, e.g. {|@x|}
You can now spread methods chains across multiple lines
You can also spread a ternary operation across multiple lines
Object.superclass = BasicObject
Method#source_location is much like Proc#source_location
useful to know if a method has been monkey patched
Symbol supports #=~, #!~ and #match (but does not return a MatchData)
Symbol#to_i is no more
Symbols get #< , #> , #<= , #>= for comparisons
No more when: in case blocks (use ; or newlines instead)
No more colon on "if" (use ;, then, or newlines instead)
Module#const_defined? now looks for the parameter in the ancestors too (use false to prevent that)
Same applies for const_get and method_defined?
Class#class_variable_set and #class_variable_get are now public
Object#define_singleton_method
public_send and public_method respect private definitions
Class X; end; X.singleton_class == (class << X; self; end)
base64 is still there
Process#daemon daemonizes the current process
Process#spawn
Complex(3,4) == (3 + 4.i)
Binding#eval(code_string) has been added, but eval("name, binding) still works
Float::INFINITY

sprintf("%A", 1.234) # hex floating point format
 => "0X1.3BE76C8B43958P+0"

no more retry in loops
new method: respond_to_missing?

Garbage Collection

GC.count

GC::Profiler

ObjectSpace module

garbage collection tweaking options (e.g. RUBY_GC_MALLOC_LIMIT)

Ruby 1.9.3 Specifics

"load.c" patch to speed up requiring/loading files
Garbage collection parameters now can be set in the environment
Pathname and Date libraries re-implemented in C for more performance
Random.new.rand(1..10)
Random.rand(1..10)
rand(1..10)
Time.now.strftime("%b %d", %Y %z %:z %::z)
String#prepend prepends a string in place
String#byteslice(range) or String#byteslice(quantity, index)
io/console
joint GPL2-Ruby to a joint 2-clause BSD and Ruby license
OpenSSL has new maintainers
New encodings: cp950, cp951, UTF-16, UTF-32

 > File::NULL
  => "/dev/null"

matrix library being improved
net/http now supports "100 Continue" status

Node.js from Ryan Dahl (JS Conf 2009)

noreply@blogger.com (Martin Carel) — Thu, 17 Nov 2011 18:10:00 +0000

I'm a bit late with that one, but I wanted to have an overview of Node.js from the man behind it. So I watched the presentation from Ryan Dahl at JS Conf Berlin 2009.

Here are my notes:

Apache vs nginx (as regards to request concurrency)

Apache uses OS threads
nginx uses an event loop

Threads

context switching is not free
it costs some CPU time
each thread takes mem
threads is not the right way to do concurrency

The solution for efficient concurrency is thus to use the event model and use callbacks. And javascript has that event model architecture.

Node.js design goals

no function should perform direct IO
low level (stream everything, never forced to buffer)
have built-in support for the most important protocols (TCP, DNS, HTTP)
support many HTTP features (e.g. chunked requests and responses, keep-alive)
friendly/intuitive API
platform-independent

So the node.js project promotes a purely evented, non-blocking infrastructure to script highly concurrent programs. Its only dependency: python.

Here is some sample code taken from the node.js website:

1 var http = require('http');
2 http.createServer(function (req, res) {
3 res.writeHead(200, {'Content-Type': 'text/plain'});
4 res.end('Hello World\n');
5 }).listen(1337, "127.0.0.1");
6 console.log('Server running at http://127.0.0.1:1337/');

Write access to a HFS+ drive from Ubuntu

noreply@blogger.com (Martin Carel) — Sun, 25 Jul 2010 02:10:00 +0000

I wanted to have write access to my HFS+ external hard drive. It took me (frustrating) hours to find out, since I kept on having it mounted automagically (i.e. seconds after I inserted the USB plug in my labtop) as read-only. So I could read from it, but not write to it.

This post did the trick for me. In other words, I had to install the hfsprogs Ubuntu package and run:

fsck.hfsplus [block device]

so for me it was:

fsck.hfsplus /dev/sdc6

Nothing wrong was found after running that check, but it apparently needed to be run. Because right after, I unmounted it, and let it mount automagically, and it worked.

Usecases using git log

noreply@blogger.com (Martin Carel) — Wed, 20 May 2009 23:19:00 +0000

I did a lightning talk (< 10 min) about 'git log' @ Montreal.rb yesterday. My sources were Scott Chacon's git log screencast, Nick Quaranto's gitready post, as well as git-log manpages. Here are my notes:

Usecase: I want to be up to date with the latest activity

git log

git log -4

git log --since="1 day"

git log --since="2 hour"

git log -2 --stat

git log -2 --name-status

git log --since='2009-05-05' --until='3 days'

git log i-2 --author='Martin C'

git log 9485fdd2..7b766227 # optionally specify a file

Usecase: I want the latest changes on a specific file

git log [file] # gives you all commits for that specific file

git log -3 [file] # gives you the 3 last commits

git log -p [file] # gives you all diffs for every commit of that file

git log -2 -p db/schema.rb

git blame [file] # gives SHA1, author, line changed, app/controllers/questions_controller.rb

Usecase: I want the changes of a commit you partly remember the comment

git log --pretty=oneline | grep [comment 

git log --grep='^user icon' 

git log --grep='user icon' -i

git show [SHA1]

git show [SHA1]:[file 

Usecase: I want stats on commits

git log --pretty=oneline | wc -l

git log --pretty=oneline --no-merges | wc -l  # no-merges: do not show commits that have more than 1 parent

git log --pretty=oneline --author='Martin C' | wc -l

git log --pretty=format:'%an' | sort | uniq -c | sort -n

git log --pretty=format:'%h by %an'

git log -4 --pretty=format:'%h (%H) by %an %ar (%ad) %s' 

git log -4 --pretty=format:'%h by %an %ar (%ad) %s' db/schema.rb 

RailsConf 2009: what to remember

noreply@blogger.com (Martin Carel) — Sat, 09 May 2009 18:10:00 +0000

I attended RailsConf 2009 and I'm quite happy about it. With close to 2 years into the Rails world, I wish some of the presentations were for a more advanced audience. The most interesting part was in the panels and keynotes.

Note: a few presentations are available through blip.tv. All in all, and somewhat briefly, what to remember from RailsConf 2009? If I had to choose 2 words to summarize it, I'd pick productivity and Rack. If you need more details, keep on reading!

Keynote by DHH

Rails has been there for 5 years (2004-2009)
Rails 3 will have:

new routing
XSS
unobstrusive and agnostic JS
More agnosticism (e.g. ORM-agnostic)

The real secret to high productivity comes from "renegotiating the requirements", i.e. make sure the said requirements fulfill the need in its simplest form (which is very often different from the way it was originally expressed).

Keynote with DHH and Tim Ferris

Although a lot of people didn't enjoy Tim Ferris' keynote (which was anything but terse, and seemingly too far beyond software engineering), there were nevertheless some points which were interesting.

Set rules for one's self

Tim Ferris explained that having very little time to do something forced him to set rules so that he could focus on the most important things. This resonates with Eric Evans' conviction on identifying and subsequently focusing on the core business value. Or the idea that 80% of your time should be spent where your business has most value, and not only 20% of your time (as it is often the case).

He also talked about 'lifestyle design' as opposed to 'deferred lifeplan' (having retirement as one of its assumption). I thought that this resonated with the concept of 'lifestyle business'.

convention over configuration

And here is a parallel with a core concept in Rails:

The idea of convention over configuration is a very appealing one because you want to have rules and rituals for everything that does not involve your core competencies, so that you can use your creative resources where they are most valuable.

Information diet

The idea is to develop 'selected ignorance'. In our digital world, there is more accessible information that it is possible to digest. So we need to create barriers or filters in order to select.

"If you look at how media/news is generated, the less you believe it's necessary to read it, partially because it's biased or inaccurate".

Path to success?

Through all conversations he had with successful people in technology or in other domains, he consistently heard one piece of advice:

There is no one path to success or productivity. It's very personal. But there is one path to be completely miserable [or failure], and that is trying to please everybody"

Keynote by Chris Wanstrath

Productivity

This idea of friction, it's a lot like the inverse of productivity. Being productive means getting things done efficiently and effectively. Friction keeps you from doing those things, slows you down, conspires against you, wastes energy.

Examples of productivity

Here are two of the most important investments you can make when starting a new business: having professionals file the paperwork and handle the numbers.

We all love Rails because it makes most of the tedious stuff go away.

And testing is famous because, well, bugs and bad design are the worst.

Focus on the community

So let's follow these examples. Let's create more projects that scratch an itch or ease some pain. Let's stop obsessing about which test framework to use and start obsessing about building sites that solve problems. Let's stop arguing about languages and continue improving our favorite ones. Let's stop blogging lengthy tutorials to get RSS subscribers and start contributing to official documentation efforts.

Keynote message: Let's focus more on code and less on talk. More on the community and less on ourselves.

Keynote by Robert Martin

If Tim Ferris' keynote was judged as the least appreciated keynote, uncle Bob's keynote was certainly the most appreciated. It was hailed as both entertaining and interesting. His keynote title was: "What killed SmallTalk and might kill Ruby?" Here are his answers:

lack of discipline (specifically: using proper TDD)
arrogance, lack of humility ('us' vs 'them')
lack of acceptance to solve the dirty problems (those of the enterprise)

In addition to that, some catchy phrases:

SmallTalk epitomized what OO was about
SmallTalk was the most powerful language in the 70s and 80s
what is clean [code] today might not be clean [code] tomorrow
Tests eliminate fear
professionalism is the disciplined wielding of power

Rack is the new cool kid

Several talks during RailsConf were about Rack (which really is a very simple way to plug functionality before or/and after the handling of a web request), people were talking about it. There certainly was a buzz about Rack.

Personally, I think Rack middleware is very similar to a ServletFilter in the filter chain, in the Java world. If you haven't played with Rack yet, the best introduction to Rack I could recommend is Dan Webb's 8-min presentation.

Rails innovations

Greg Pollack and Jason Seifer (RailsEnvy guys) gave a presentation about the Rails innovations in the last year. Here are the pieces of software worth mentioning:

RubyMine IDE
Rack
Rails metal
Rails templates
metric-fu gem
Cucumber for BDD
FakeWeb
Spike (log analyser)
Ultrasphinx (full-text search)

Agile development tips, by David W. Frank

noreply@blogger.com (Martin Carel) — Thu, 07 May 2009 17:24:00 +0000

I watched the talk from David W. Frank (from Pivotal Labs) at RailsConf. It was interesting to listen to advice on agile software development, although on some I would disagree. Here are my highlights and some thoughts afterwards.

Stick to conventions

follow the ground rules
be a lord of discipline
pop goes the story stack (pick the very first task on top)

Stay in sync

cubicules kill
go to lunch as a team
keep up the jibber jabber (talking while working)

Tools matter

rtfw (or be quick to google it)
get stories done
this is a whiteboard
don't forget the hardware

Pair

live together, die alone
let the rookie type
pairs stink after 2 days
two pairs are better than one

Test drive

beat the blews
tests, not comments
ping pong isn't just table tennis

Check your dashboard

every failing test is sacred
fix first, ask questions later
slow suites are project heart disease

Work simply

architecture is a 4-letter word ('test')
make it green, then clean it
check-in early and often
kill dead code dead

My thoughts

pop goes the story stack

Code ownership is generally a bad idea, but in reality you have deadlines, i.e time is a constraint. If you need to be effective, some tasks will be done much quickly if done by certain team members (it might not be yourself). You shouldn't refuse a task because it's a boring chore (pleonasm intended), you should refuse a task if you feel it would take you substantially more time than if it would be done by another team member.

go to lunch as a team

But not all the time. Maybe 50% of the time? Lunch is a very important break during the day, and it can be relaxing (sometimes needed) to have lunch with some people outside of your team.

keep up the jibber-jabber

Nah. Totally not. Go for open spaces, but no to the jibber-jabber. You need to interrupt another team member when you really can't deal with the issue yourself. That's why a chatting channel (e.g. campfire) is useful. People will go read it whenever they feel like it, instead of being interrupted in the middle of something.

tests, not comments

Absolutely, yeah. Comments are code pollution, very often. If you feel you need to write a comment, it probably means you need to refactor or rename method/class names.

Ruby quiz (easy) #10

noreply@blogger.com (Martin Carel) — Fri, 27 Mar 2009 21:50:00 +0000

Question: How do you make this line a one-liner?

value = compute_value

value = (value > CONSTANT) ? value : CONSTANT

Solution:

value = [compute_value, CONSTANT].max

Unobstrusive metaprogramming, by Sean O'Halpin

noreply@blogger.com (Martin Carel) — Wed, 25 Mar 2009 19:59:00 +0000

I watched Sean O'Halpin 30-minute Ruby Manor presentation (Nov 2008) called "Unobstrusive metaprogramming". He gave a few tricks about monkeypatching.

Here are my highlights.

Problems with monkey patching "the Rails way"

namespace pollution
documentation pollution
cognitive pollution

Case in point:

>> [Kernel, Object, Class, Module].inject({}){|hash, klass| hash[klass] = klass.methods.size; hash}
=> {Object=>98, Kernel=>159, Module=>99, Class=>99}
>> require 'active_support'
=> true
>> [Kernel, Object, Class, Module].inject({}){|hash, klass| hash[klass] = klass.methods.size; hash}
=> {Object=>187, Kernel=>225, Module=>188, Class=>188}

Recommendations

indirect access (using functions)
namespaces
hide your mess

So for instance, instead of monkey patching the Array object, we can do something like that:

a = [[1,2,3,[4],5]]
Doodle::Utils.flatten_with_1_level(a) # => [1,2,3,[4],5]

The plea

Don't monkeypatch: Object, Kernel, Module, Class, Hash, Array, etc.

Everytime you monkeypatch Object, a kitten dies.

Hashtweeps scraper: mixed feelings about Scrubyt

noreply@blogger.com (Martin Carel) — Thu, 12 Mar 2009 22:36:00 +0000

Motivation

Hashtweeps is a simple site where you can search all tweets with a specific hash term. I have decided to give Scrubyt a go and write a little Hashtweeps scraper to get a feel of it. The CEO of the company where I am at was at the leadscon conference, so I was challenged to gather all tweets with the leadscon hash (#leadscon).

The code

require 'rubygems'

require 'scrubyt'

HASH_TERM = "leadscon"

data = Scrubyt::Extractor.define do

  fetch 'http://www.hashtweeps.com/'

  fill_textfield "term", HASH_TERM

  submit

  # build XML structure

  item "//li[@class='result']" do

    msg "//div[@class='msg']"

    link "//div[@class='info']/a[1]/@href"

    user "//div[@class='info']/a[1]", :generalize => false do

      # this will follow link because it ends with "_detail"

      page_detail do

        profile_info '//ul[@class="about vcard entry-author"]' do

          full_name "//li//span[@class='fn']"

          location "//li//span[@class='adr']"

          website "//li//a[@class='url']/@href"

          bio "//li//span[@class='bio']"

        end

      end

    end

  end

end

# dump XML to file

dump = File.new("output.xml", "w")

dump.puts data.to_xml

Screenshot of my output

I added a very simple XSLT to my XML output and here it is:

Scrubyt: Pros

All in one: navigator, extractor, output builder. With very few lines of code, you can write a simple scraper which can navigate pages, scrape and build/output a custom XML structure.

Scrubyt: Cons

Lack of a good API reference. I had issues with the official one. How am I supposed to know that if you end your method with "_detail" it will navigate to that page? It seems hard to ge beyond the "scrape Google results"-type of scenario. Indeed, there's lots of TODO's in the reference. Hopefully this reference will get more structure and coverage.
No debug info. Even though the code above is fairly simple, when it breaks, you have no idea what the error was: Scrubyt just exits.
Impossible to test. How do you make sure the second page you navigate still exists? How do you make sure the HTML elements are still where you thought they were? How do you make sure your code constructs the XML structure as you want it to be?
The dependencies are not harnessed. I am a fan of having tight control on dependencies' versions. I wasted time to figure out that Scrubyt could not run with the (latest) version of the Mechanize gem I had installed.
The unofficial doc is outdated. Tutorials are nice when you want to get a first feel of a new tool. Unfortunately, probably because the Scrubyt source code has been changing a lot in the last year, Scrubyt tutorials out there are no more accurate. Remedy: the first stop to kick start your first scraper should be the tutorials on Scrubyt's wiki.

Conclusion

If you're serious about scraping, scrubyt is not a viable option. As soon as you're beyond the trivial scraper (like the one I did), you're in for some waste of your time.

Ruby VM's, by Jason Seifer

noreply@blogger.com (Martin Carel) — Thu, 05 Mar 2009 23:49:00 +0000

I watched Jason Seifer's QCon 30-min presentation on Ruby VM's. He describes the main characteristics of most Ruby VM's out there. A little note: his criterium to decide whether a VM is production-ready or not is based on its support (or lack of support) for Rails, since that's a Ruby framework which touches on a lot of aspects of the Ruby implementation.

Here are my highlights.

Ruby versions

Ruby 1.8.6 is the most widespread implementation
Ruby 1.8.7 is the current stable version of Ruby (middle ground between 1.8.6 and 1.9)
all VM's focus on Ruby 1.8.6 (with the exception of JRuby, which has support for both Ruby 1.8.6 and Ruby 1.9)

MRI

the de facto standard
the only concise Ruby spec out there
supported by all major platforms
production-ready

YARV

Ruby 1.9.1 will have YARV as the default VM
not production-ready

MacRuby

Ruby 1.9 VM on OS X core technologies
mostly uses Objective-C and YARV C code
open source
early in development (< 1.0 release)
not production-ready

XRuby

first Ruby to Java compiler
last release: 0.3 in Nov 2007
not production-ready

MagLev

not released yet
made by GemStone
uses OODB (allow Ruby objects persistence)
not production-ready

Rubinius

"Ruby in Ruby" (since part of it is written in Ruby)
the VM proper is written in C++
the standard lib written in Ruby
Ruby objects mapped to a C++ object
migration from C to C++ was for lowering the barrier of entry for project contribution
slow
best of the MRI contenders
Ruby spec project came out of Rubinius
not production-ready

IronRuby

Microsoft's implementation of Ruby
Ruby on .Net
released under MPL (Microsoft Public Library)
runs on the DLR (Dynamic Language Runtime), on top of the CLR
DLR supported languages: IronPython, IronRuby, Javascript, Dynamic
not production-ready

JRuby

big advantage: can use existing Java code
most performant Ruby implementation
multi-threaded (native ones, i.e. it threads in different processes)
introduced a compatibiltiy land (can start the VM with a Ruby 1.8 or 1.9 "compatibility flag")
The solution for the enterprise-class Ruby applications
production-ready (!)

My journey towards CouchDB, by Damien Katz

noreply@blogger.com (Martin Carel) — Thu, 12 Feb 2009 15:53:00 +0000

I watched the presentation that Damien Katz (the one behind CouchDB) gave at RubyFringe in 2008. The presentation is a non-technical and inspirational description of his journey to be able to live off "doing cool stuff", which for him is to write open source software.

Here are my highlights.

Reasons for doing "cool stuff"

educational
more family time
see what I can do
makes an interesting story

Java vs Erlang

One of the rare technical bits was that comparative slide:

A very tough journey towards happiness

It was really hard to take off and risk to jeopardize his family for "doing crazy stuff". It took him a long time to get there; he went through highs and lows; he had to make sacrifices. That might be why so few people are willing to do it.

But today he can proudly say:

I'm that guy who gets paid to work on cool stuff from home.

There are no predetermined steps to reach this state of mind, this kind of living style. It is an intrinsically personal process which is discovered as one goes through it.

A requestioning of the status quo

Ruby quiz (easy) #9

noreply@blogger.com (Martin Carel) — Tue, 03 Feb 2009 18:28:00 +0000

Question: What will be the output of the following snippet:

require 'rubygems'

require 'fileutils'

puts File.expand_path(File.dirname(__FILE__))

FileUtils.mkdir 'test'

FileUtils.chdir 'test'

puts File.expand_path(File.dirname(__FILE__))

Solution:
If run in user's account, the output will be:



/home/user

/home/user/test

Surprised? So was I.

What Makes Code Beautiful?, by Marcel Molina, Jr.

noreply@blogger.com (Martin Carel) — Wed, 28 Jan 2009 23:16:00 +0000

I watched What Makes Code Beautiful?, a presentation Marcel Molina Jr. gave at RubyConf '07.

Here are my highlights.

A definition of beauty

From all the definitions of beauty he explored, he chose the one from Thomas Aquinas, where beauty is defined with 3 criteria:

proportion (economy of size: smallest size needed to do the job)
integrity (faculty of being well suited for its purpose)
clarity (simple and clear)

Each of those 3 criteria is necessary, and none are sufficient.

The idea is to apply those criteria to our code, so that it becomes 'beautiful'.

A definition of expert

He dropped in an interesting quote from a famous physicist:

An expert is a person who has made all the mistakes that can be made in a very narrow field. -Niels Bohr

In other words, by accumulating mistakes, you gain expertise.

Ruby == beautiful

At the end, Marcel Molina enthusiastically encourages the crowd to give a a warm applause to Matz, sitting in the front row, for the beautiful programming language he has created.

Ric-rac-roe in a soup of technologies

noreply@blogger.com (Martin Carel) — Tue, 27 Jan 2009 17:30:00 +0000

It all started with a desire to experiment a few things: Sinatra, Shoes, and the context testing framework. So I decided to write a simple application - the classic tic-tac-toe game. I ended up with 3 interfaces:

command line interface (cli)
web app (using Sinatra)
GUI desktop app (using Shoes)

I thought it'd be nice to give a little feedback on my experience.

Ric-rac-roe: the app

I tried to write 'nice Ruby code', that is, readable/intuitive code along side tests (as a result of TDD, but complemented so that it serves as API doc). You can check the code out on github.

Design

1 Grid model for the 9 squares of the tic-tac-toe as well as its rules (e.g. check for winning position, only play in an empty square, etc.). 1 Game model so that a gameplay can interact with the tic-tac-toe grid (via the Game#play method). The contract for the gameplay is to define 3 methods, which will be called at each turn (from the Game#play method):

position(position) #reads the player's move
status_message=(msg) #sets the status message
squares=(squares) #sets the grid's squares

There currently exists 3 gameplays: CLI, web and desktop.

The classes diagram could look like this:

Stats

(non-tests, non-views) LOC: 345
tests LOC: 308
C0 code coverage: 98%

Sinatra

I must say it was a joy to write Sinatra code. Quick, easy and fun. I'm aware my app is quite simple, but the light nature and the hands-off approach of Sinatra fit my tic-tac-toe project perfectly.

Support from the Sinatra community

Sinatra's Rdoc got me started real quick. No waste of time. On their #sinatra IRC channel, Ryan Tomayko, one of Sinatra lead developers, as well as some others, were there to assist on a few occasions. Lots of nice people in there, good vibe.

Shoes

Booting up my first Shoes app was also easy, but it took me more time.

The development is intuitive because the layout management for your components is done web-style, with margin, padding, colors, etc. for components that you toss into either stacks (akin to usual block div's) or flows (akin to inline div's). You define your callbacks (like a click handler on a button) with a Ruby block appended to the button's click method. So you get the drift. It starts to feel natural, after a little getting used to.

Support from the Shoes community

The IRC channel #shoes is as quiet as a cemetery when it's -35 C. Seth Thomas Rasmussen seems to be the only one active in there, and he is the one who helped me out (and some others) with my newbe questions. I thought I'd see _why hanging in there, but according to Seth, he hasn't hanged in there for months. No doubt, busy elsewhere. But he's actively working on Shoes (and has been for a long time), when you look at the github's project history. As for the Shoes mailing-list, there's medium activity (circa 1 mail per day).

The neat built-in manual (which comes with the Shoes installer) is your best way to find your answers; you don't need much else. Finally, perusing through a few sample apps is a great way to get some source code inspiration.

Context

I read a few interesting posts on testing frameworks, with a focus on context. I wanted to try it, and enjoyed it. All it requires is test/unit and you can have contexts in your tests, and play it a la BDD, RSpec-style (describe/it), shoulda-style (context/should), or context-style (context/test) it doesn't matter. With context, you can mix all that, and have the context/it flavor if you will, without even including RSpec.

I ended up using pure test/unit in some places (e.g. testing simple input parsing, or my Array's monkeypatch), RSpec in some others (e.g. testing Sinatra's routes). And in either situations sometimes I found I needed contexts on top of test/unit. So it all depends on what you want to do, it just felt good to use the most appropriate tool of my tools belt, depending on the context (no pun intended).

Final words

Overall, I had a good time, and now I'm happy to know a bit more about quality tools. Any feedback on the code is welcome (design-wise, Ruby-wise, testing-wise, etc.).

Ruby quiz (easy) #8

noreply@blogger.com (Martin Carel) — Sat, 24 Jan 2009 04:34:00 +0000

Question: What would be the output of this code snippet (note the type of the key in the hash and the type of the key when accessing the hash):

hash = {:language => 'ruby', :author => 'martin'}

p hash['language']

Solution:

The output is nil because the hash is defined using symbols, and the access to it was done using a string.

Rails' active_support introduced a class called HashWithIndifferentAccess (a subclass of Hash), which, according to the class' doc:

[...] we only have it so that people can write params[:key] instead of params[‘key’] and they get the same value for both keys.

Referring back to the example above, we could get what we want (indifference between symbol or string hash access) with:

require 'rubygems'

require 'active_support'

hash = HashWithIndifferentAccess.new({:language => 'ruby', :author => 'martin'})

p hash['language']

which would print "ruby".

Ruby quiz (easy) #7

noreply@blogger.com (Martin Carel) — Thu, 22 Jan 2009 20:45:00 +0000

Question: What would be the output of this code snippet?

ary = [1,2,3]

ary.each do |element|

  element += 1

end

p ary

Solution:

The output would be [1, 2, 3]. The entire array is not modified by the Array#each method. This is because the iterator, while traversing the collection, passes a copy of each element in turn, to the block.

If you want to dig a bit, if you look in the C code from Ruby 1.8.4 in the array.c file, the parameter yielded to the block is passed "by value", as opposed to "by address" (which would then allow modification of its value).

VALUE

rb_ary_each(ary)

  VALUE ary;

{

  long i;

  for (i=0; i<RARRAY(ary)->len; i++) {

    rb_yield(RARRAY(ary)->ptr[i]);

  }

  return ary;

}