Poem of Bragi

The Little Schemer

2014-02-05T00:00:00Z

My next project has a little bit of Clojure on it, so I decided that is finally time to embrace the parenthesis. I have some experience of functional languages, mainly Haskell and Scala, but I've never programmed in anything lisp related.

Instead of diving into Clojure right out of the bat, I am going to be a hipster and start with Scheme. More specificaly, I am following the exercises laid out on the amazing The Little Schemer book

Why not a Clojure book?

Lisp syntax is very simple and for most versions, the standard set of functions is really small. The whole thing fits in the back of a napkin, and maybe that simplicity is what inspired many authors to write amazing lisp books. Or rampant LSD consumption, who knows...

And boy, the books are amazing! Legends like Structure and Interpretation of Computer Programs, The Land of the Lisp and the Little Schemer/Lisper itself, make any other language books look dry and boring in comparison.

Clojure is a proper modern language, with a rich set of standard libraries and all the Java baggage, which makes it probably amazing for real development work, but can be a hindrance when you want to learn the essence and philosophy of the Lisp family.

Anyway, the books are fun, what else do you need to know?

Installing Scheme

The Lisp language family is ridiculously huge, it's one of the oldest languages after all, with active language development happening nonstop since the sixties. Most of the implementations are from a time where even the computers themselves where not standardized. Finding the right version is actually quite challenging for the Lisp uninitiated

To follow the book you need a Scheme implementation and a nice REPL. The easiest one to install for me was the Petite Chez Scheme. Just select the proper version for your system and it should just work.

A gotcha to look for, at least on the 64bit OSX version, is that the runner has very little memory management capabilities. Any infinite recursion, something that happens very frequent to lisp newbies, can freeze your whole system

A very special book

The Little Schemer is entirely written as a series of questions and answers, something like this:

Is this an s-expression?
xyz

Yes, all atoms are s-expressions

This is what is called the Socratic Method, the questions are used to stimulate you to think critically about a subject and arrive at your own conclusions.

In a programming book this style feels very weird at the beginning, but after a while the questions put you in a very nice and easy reading pace.

The magic of this book is that, in only 200 pages, it goes from totally basic definitions to some really complex topics, all in a single narrative flow. The use of the questions has a "slowly boiling frog" effect, you don't feel the increase in complexity until you notice yourself deriving a Y-combinator.

The book presents a set of five laws and ten commandments, that start very simple but grow over the book to encompass a very complete guide on writing recursive programs in scheme.

The Seventh Commandment
Simplify only after the function is correct

The Eight Commandment
Use help functions to abstract from representations

The Ninth Commandment
Abstract common patterns with a new function

Also included are some solid explanations for many topics we usually struggle to understand, like recursion and the halting problem. The book ends with the crowning jewel of every scheme book, a implementation of Scheme itself. Can you imagine if a Java or even Ruby book tried to do that?

Conclusion

The Little Schemer surely lives up to it's reputation, a proof that programming books can be short and fun. Totally recommended for anyone looking to expand his horizons on functional thinking and recursion. Go read it!

discuss

Refactoring by the book

2013-01-08T00:00:00Z

Refactoring is one of the techniques that allows us to be agile and apply an evolutionary approach to our design. A core XP practice and one of TDD's pillars, constant refactoring keeps the evil design upfront at bay and maintain our codebases in a healthy state. One of the few things I think most developers agree on is that refactorings are helpful and should be done at some point in any project, preferably in a continuous manner.

The problem is that, like every other developer term, the definition of refactoring has become muddled over time. Refactoring is now commonly conflated with it's more dangerous cousin, the Rewriting. The term is used whenever we want to improve the design of some part of a codebase, either a single class or entire subsystems.

Day or week long refactoring are becoming common place in agile projects, they even have their own stories and board lanes. While its all done with good intentions, these long sessions of madly rewriting every bad part of a codebase carry a lot of risks and are a far cry from what refactorings were supposed to be.

And since everybody knows refactoring is core agile thing, the way we do it is seldom questioned. After all, getting to a better design is supposed to be a good thing right? Let's go back to how refactorings were introduced to the developer community and see what we missed along the way.

The original refactorings

Although they existed a long time before that, Refactorings were introduced to the large world by Martin Fowler's book Refactoring: Improving the Design of Existing Code, where Fowler writes about coding practices from legendary SmallTalkers like Kent Beck and Ward Cunningham A definition from the same book:

Refactoring is a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior.

Its heart is a series of small behavior preserving transformations. Each transformation (called a “refactoring”) does little, but a sequence of transformations can produce a significant restructuring. Since each refactoring is small, it's less likely to go wrong. The system is kept fully working after each small refactoring, reducing the chances that a system can get seriously broken during the restructuring.

The original refactorings were presented using a pattern format. They have names, like Extract Method or Replace Temp with Query, a explanation of when to use the refactor and a list of steps you have to follow in order to apply them. You can browse the refactorings catalog on refactoring.com

They are meant to be applied in a conscious manner, one after the other, until your design is good enough for you to move forward. Words like disciplined and small are present on the very definition of the term.

When you wildly rewrite large parts of your system, you are not refactoring, even if you are just trying to achieve a better design. The system needs to be kept working the whole time. That means the tests need to stay green! The TDD flow is supposed to be Red -> Green -> Refactor, and not Red -> Green -> Red -> Red -> Shit -> Rollback.

Some ways to refactor on the green

Let's see how we can refactor some code in a more disciplined manner, keeping the tests always on the green.

Instead of working on some contrived User class we are going to use a real example, the class Request from the Pacto project. You don't need to understand what the class does, just pay attention to the code structure and how we apply the refactorings iteratively.

module Pacto
  class Response
    attr_reader :status, :headers, :schema

    def initialize(definition)
      @definition = definition
      @status = @definition['status']
      @headers = @definition['headers']
      @schema = @definition['body']
    end

    def instantiate
      Faraday::Response.new(default_env)
    end

    private

    def default_env
      {}.tap do | env |
        env[:status] = @definition['status']
        env[:response_headers] = @definition['headers']
        env[:body] = JSON::Generator.generate(@schema) if @schema && !@schema.empty?
      end
    end
  end
end

This class has many small issues, but the main thing I want to do is to remove the instatiate method. This method is forcing the Pacto codebase to deal with instances of a foreign class, Faraday::Response. The thing is that Faraday::Response is a very simple class, and we can easily implement the interface we need on the Pacto::Response class itself, eliminating the dependency on external code.

Step 0 - Check the current tests

Before we start to refactor, a very important step is to check if the code in question is actually covered by tests. A quick way to do it is to just do the refactoring, in our case simply remove the instantiate method and see if anything breaks.

If you get zero red tests, that's not a good thing! That means you have no way to be sure that your refactorings didn't impacted the codebase, no safety net to catch your mistakes. There's a famous phrase that says "refactorings without tests is just changing shit". If that happens, just rollback your change and start implementing the tests for the behavior you want to change.

Luckily for us Pacto has test suite, so we get a precise red spec if we remove the instantiate method.

Step 1 - Move Method

The failing spec tell us that we need a method body on whatever instantiate returns. We can see the body definition passed on the Faraday::Response initialization. Let's apply a slightly altered Move Method refactoring to move it from the default_env definition to the Response class.

module Pacto
  class Response
    attr_reader :status, :headers, :schema

    def initialize(definition)
      # snip
    end

    def instantiate
      Faraday::Response.new(default_env)
    end

    def body
      JSON::Generator.generate(@schema) if @schema && !@schema.empty?
    end

    private

    def default_env
      {}.tap do |env|
        env[:status] = @definition['status']
        env[:response_headers] = @definition['headers']
        env[:body] = body
      end
    end
  end
end

We run the tests to make sure we didn't break anything. Since we only changed private methods and we kept the original structure, everything runs fine.

Step 2 - Change Faraday::Response to self

It turns out that step one is the only thing we need to do in order to be able to replace Faraday::Response with self on the *instantiate method, so we change the method.

# Inside Pacto::Response class

def instantiate
  self
end

That's probably the most useless method ever, but remember, we want to keep the tests green. By allowing this travesty to live a little longer on the codebase we can be sure that nothing will break and move forward with a few other refactorings before paying attention to other parts of the Pacto codebase.

Step 3 - Remove the unused default_env method

So we have a private method that's not used anymore. Safest refactoring you can have, we don't even need to run the tests for this one, right? Well, remember that in Ruby private methods can be called with send (a very bad practice btw), so you are never totally sure that's something is not important just by eyeballing a class. Always run your tests.

Thankfully Pacto is not a metaprogramming happy codebase, so our tests still run green after removing the default_env.

module Pacto
  class Response
    attr_reader :status, :headers, :schema

    def initialize(definition)
      @definition = definition
      @status = @definition['status']
      @headers = @definition['headers']
      @schema = @definition['body']
    end

    def instantiate
      self
    end

    def body
      JSON::Generator.generate(@schema) if @schema && !@schema.empty?
    end
  end
end

Step 4 - Remove unused @definition field

Another piece of private unused code, same rules as the previous step.

module Pacto
  class Response
    attr_reader :status, :headers, :schema

    def initialize(definition)
      @status = definition['status']
      @headers = definition['headers']
      @schema = definition['body']
    end

    # snip
  end
end

Step 5 - Remove excessive conditionals

On the Response#body method we have some conditionals that could be easily replaced by a default value for the schema attribute. Doing a "let's see if something break" trick we discover that this conditionals are not under test, so first we add the relevant specs:

module Pacto
  describe Response do
    # some other specs

    describe 'the response body' do
      context 'when the definition has an nil body' do
        let(:response) { Response.new(definition.merge('body' => nil)) }

        it 'is nil' do
          expect(response.schema).to eq(Hash.new)
        end
      end
    end

    # more specs
  end
end

Then we replace the conditionals with a default value for the schema field:

module Pacto
  class Response
    attr_reader :status, :headers, :schema

    def initialize(definition)
      # other attrs
      @schema = definition['body'] || {}
    end

    def instantiate
      self
    end

    def body
      JSON::Generator.generate(schema)
    end
  end
end

Step 6 - Remove instantiate method

Finally the class is looking good enough and we can go forward and remove the instantiate method. Whoever is using that method can just use a Response instance instead.

For this step we simply delete the method and fix all the broken specs. Thankfully in this case, the Response class was used only by another class, so it was a very straightforward process. For cases were you have many dependencies, you might let the offending method live for a little longer and replace each use separately on work on each case separately.

module Pacto
  class Response
    attr_reader :status, :headers, :schema


    def initialize(definition)
      @status = definition['status']
      @headers = definition['headers']
      @schema = definition['body'] || {}
    end

    def body
      @body ||= JSON::Generator.generate(schema)
    end
  end
end

There's still some things we could do on this class, like turn it into a Struct, but it's good enough. Knowing when to stop refactoring is as important as knowing how to do it, the world is full of yaks and rabbit holes.

Conclusion

You might be thinking this whole thing was a bunch of dev melodrama, that a good developer could have done all the changes in a fell swoop. Well, if you look at the simple example, that's true. But we face much nastier situations in our daily projects.

By using a more methodical approach to refactoring I've had a fun coding session, that went quite fast. All the commits were short and precise and the coolest thing is that I actually pushed the individual steps to the master branch in the Pacto code base, since I kept the whole project working all the time.

The next time you see an ugly class that's begging for some love, fight the urge to just hack it away and try to be more disciplined about your refactorings. Argue with your teammates whenever you see some someone creating a refactor branch, or diving in a month long refactor epic.

A good advice is to read Martin Fowler's book, there is even a more recent ruby version, although the java code in the original one is very easy to read. Another cool place to learn more about the refactoring practice is the c2 wiki, where you can see people arguing about the practice, including some famous names, at the time it was introduced.

Why rubyists ignore concurrency?

2012-09-16T00:00:00Z

Concurrency (threaded concurrency in the context of this article) is a very interesting subject in the ruby community. On one hand, you have language implementators and gem writers going through a lot of hurdles to implement it correctly. On the other hand, you have the community at large doing its best effort to ignore the whole thing and still writing single threaded apps. Why?

It is a hard problem

Concurrency is a very hard thing to do. You can get yourself bitten by the slightest mistake and nasty bugs that are impossible to debug are a constant reality. But then, so is every other software problem you approach without conscious thought and good techniques.

The multicore overlords did not arrive

Functional programming zealots have been making prophecies about the end of OO programming due to our 'multicore' future for quite some time. They have a very valid point, it is getting ridiculous to ignore concurrency while even our smartphones have multiple cores, but the doomsayers didn't take into account the force of the cloud.

Instead of worrying about how to make the best use of the 10 cores of a server you can just boot 10 AWS instances. Who needs concurrency when you can just throw money at the problem?

The language itself does not care

Ruby 1.8 and 1.9 have the infamous GIL. If you don't know what that is please go read Matt Aimonetti's very good post about it. The GIL was a conscious decision of Matz and the early ruby developers to just ignore concurrency issues while they were building an awesome dynamic language.

They probably had very good reasons for that, and the lock is not as inefficient as people think, but still, it created the ruby culture of simply ignore thread safety and the whole threaded concurrency area.

While other languages are seeing great progress on writing safe and maintainable concurrent code, with things like Go routines and channels and Clojure's use of STM, the average ruby guy still does not know what a Mutex is all about.

You can live without it

Ok, we suck and don't use concurrency very well. But still, Ruby is being used to serve apps to a bazzilion users and generating tons of money, performing well under load. That's undeniable proof that you can ignore threads and live happily ever after.

After all, threaded concurrency is not the only way of scaling, you can spawn or fork processes, do asynchronous computations with background workers, use an evented reactor and lots of other options.

These options work very well, but they have a somewhat hidden cost of leaving ruby with a big memory footprint. You can always add more machines to the problem, but that doesn't change the fact that all of our apps could a have a very big efficient boost if only we cared about concurrency.

The Sidekiq gem is a very good example of that. It is a Resque 'clone', but built with threaded and concurrency issues in mind. The result is a gem that's getting very famous for saving people's money on AWS accounts.

Ignoring a problem does not solve anything

Yes, concurrency is hard. Yes, every single statement I made above is full of caveats. But, even so, concurrent code is not equal to bad code. There are a lot of good models that can help you write awesome concurrent code without tripping in mutexes on every second line of code.

Big companies can solve this problem with money, but can your awesome but still not known startup do the same? Why don't we just use computer science instead? Ignoring threads is not doing our community any favors. The next time you add yet another node to your web farm, stop and think if adding some concurrency would not solve your problem.

So please, go read something about threads, actors, or channels and bring that knowledge back into our community.

Ruby File IO Primer - Part 4 - Building a Clone of the Watchr Gem

2011-06-29T00:00:00Z

To put the techniques described on earlier parts of the Ruby File IO series in practice, we are going to build a small clone of the Watchr Gem.

The Watchr Gem

The Watchr gem is a nifty continuous-testing tool. It watches for changes on your project's files and executes a code block when that happens. Watchr offers a simple, regexp based, DSL to let the user specify what code gets executed when which file changes, ilustrated on the following sample:

  watch("spec/.*\.rb") { |md| system "rake spec #{md[0]}" }
  # run rspec when a .rb file inside the spec folder changes

  watch("app/model/.*\.rb") { |md| system "rake spec #{md[0]}"  }
  # run tests a model class changes

Watchr is "sold" as a testing tool, but it can do much more, like calling the coffescript compiler when a .coffee file changes, compiling some assets when you change a .css file and any other filesystem change based task you can imagine.

Stalker, Our Little Clone

Our purpose with this clone is to practice the Ruby File IO API, consolidating our knowledge about it, so we are going to do things a little differently then the original gem.

*Watchr" relies on other libs to fire events when a filesystem change occurs. To not lose focus on our demo, we are going to use a simpler (and probably much less performant) solution of running a loop and constantly checking the watched files for changes, based on its filesize.

We are also goin to change the DSL, Watchr uses regexp to specify the watched files, we are going to use the Dir glob syntax, because it's more relevant to our study and also more suitable to describe filesytem paths.

  watch("spec/.*\.rb") { |md| system "rake spec #{md[0]}" } # Watchr
  stalk("spec/*.rb") { |f| system "rake spec #{f}" } # Stalker

The first class in our script is the Stalker class. It's responsible for providing the DSL to our clients and also for the storage and execution of all file watching rules. (Yay for the SRP!)

class Stalker
  attr_reader :paths

  def initialize(script)
    @handlers = {}
    @paths = []
    instance_eval(script.read)  
  end

  def stalk(glob_path, &block)
    Dir[glob_path].each do |p|
      path = Pathname.new(p)
      add_handler(path, block) if path.file?
    end
  end

  def add_handler(path, block)
    if @handlers.key? path
      @handlers[path] << block  
    else
      @handlers[path] = [ block ]
      @paths << path
    end
  end 

  def handle(path)
    @handlers[path].each { |handler| handler.call(path) }
  end
end

The constructor receives a script object (either a File or a Pathname) and instance_eval its whole content, accessed by the read method, providing the DSLy powers of our script.

The stalk method receives a glob path and a block, and stores the block as a handler,for all the files matched by the glob_path. This feat is achieved by the usage of the Dir.[] method, that allows us to offer the power of the dir globbing to our clients and eliminates any worries about bad or empty rules, since Dir.[] just returns an empty array on those cases. We also call Pathname.new and Pathname.file? on all results returned from the Dir call, to ensure we only store rules for files and not directories.

The add_handler method is standard ruby stuff, adding the block as an handler to an array stored on hash keyed by the Pathname instances. The handle method receives a file path and calls all the stored handlers for that path.

The second class is the Engine class, responsible for running the loop and detecting any changes on the stalked files.

class Engine
  def initialize(script_path)
    script = Pathname.new(script_path)
    if script.file?
      @stalker = Stalker.new(Pathname.new(script_path))
      init_file_sizes
    else
      puts "Attempt to load the script file from #{script_path}. File not found."
    end
  end

  def run
    loop do
      @sizes.each do |file, size|
        if file.size != size 
          @sizes[file] = file.size
          @stalker.handle(file) 
        end
      end
      sleep 1
    end
  end

  def init_file_sizes
    @sizes = Hash.new(0)  
    @stalker.paths.each do |path|  
      @sizes[path] = path.size
    end
  end
end

The constructor receives the path to the script containing the DSL calls, checks if it's a real file, and then creates a Stalker instance with it. After, it initializes a hash containing the size of all the stalked files.

The run method executes indefinitely on a loop, testing each stalked file for a change on its size, calling the Stalker.stalk method when it finds one. We could also use the last modification date, calling the mdate method, for the file change comparison.

The rest of the script is just the initialization of the file stalking process

puts "Start stalking..."
Engine.new(ARGV.first || "watchlist").run

And that's it! With just 66 lines of Ruby we built a clone of a very useful Gem, showcasing the eloquence of the Ruby File IO API.

The whole code for this sample is on github

Ruby File IO Primer - Part 3 - The Standard Library

2011-06-25T00:00:00Z

Following our series on Ruby File IO, we will now delve on the two Standard Library components that complements the core ruby IO API, the FileUtils module and the Pathname class.

The FileUtils Module

The FileUtils module brings an interesting approach to file manipulation, by emulating a ton of file related Unix commands, and its most commonly used flags. The effects of commands like rm -rf and ln -s are done by calling the FileUtils.rb_rf and FileUtils.ln_s methods.

Since they follow a syntax known by most ruby developers, the FileUtils methods are very straightforward to understand, making a list of FileUtils method calls really feels like a bash session. The following sample illustrates that similarity.

  require "fileutils"

  FileUtils.touch(["some_file.rb", "another_file.rb"])
  FileUtils.mkdir("code")
  FileUtils.mv(["another_file.rb", "../other_file.rb"], "code")

  Dir["code/*"]
  >> ["code/some_file.rb", "code/another_file.rb"]

  FileUtils.cp_r("code", "bkp")
  FileUtils.rm_r("code") 

  Dir["code/*"]
  >> []
  Dir["bkp/*"]
  >> ["bkp/some_file.rb", "bkp/another_file.rb"]

Like Unix commands, most FileUtils methods knows how to deal with multiple files by receiving arrays as parameters,like the FileUtils.cp method. They also accepts flags to change its behavior.

  require "fileutils"

  FileUtils.rm("a_file.rb") # removes this file
  FileUtils.rm(Dir["bkp_*"]) # remove all files that start with bkp
  FileUtils.rm(Dir["bkp_*"], :verbose => true) #print the equivalent stmt and remove the bkp files

The Pathname Class

The Pathname class represents a pathname, the location of a file on the underlying filesystem, and provides facilities for querying and manipulating filepath data.

Although not universally useful as the FileUtils module, the Pathname class can bring in a lot of niceties when you need to do heavy filesystem transversing, and it is used on gems like Sprockets and Carrierwave.

You create a Pathname object by passing a string with the filesystem path to the class constructor. You can also get Pathnames by joining two paths with Pathname.join or Pathname.+. Also, most Pathname methods return Pathname objects.

  require "pathname"

  path = Pathname.new("/home/marcos/projects")
  >> Pathname:"/home/marcos/projects"

  other = path.join("bragi")
  >> Pathname:"/home/marcos/projects/bragi"

A Pathname object can provide a lot of information about its underlying filepath, like its parent path, dirname, whether it is a absolute or relative path, whether it represents a file or not, and so on.

  require "pathname"

  path = Pathname.new("/home/marcos/projects")
  path.dirname   # Pathname: "/home/marcos/"
  path.basename  # Pathname: "/home/marcos/projects"
  path.parent    # Pathname: "/home/marcos/"
  path.file?     # false
  path.absolute? # true
  path.relative? # false

Pathname.ascend iterates and yields a Pathname object for each element in a given path on an ascending order. Pathname.children returns an array with all the siblings of a given path, and Pathname.each_child iterates over then.

  require "pathname"

  path = Pathname.new("/home/marcos/projects")
  path.ascend { |x| puts x }
  >> Pathname: "/home/marcos/projects"
  >> Pathname: "/home/marcos/"
  >> Pathname: "/home/"
  >> Pathname: "/"

  path.each_child { |x| puts x }
  >> Pathname: "/home/marcos/projects/bragi"
  >> Pathname: "/home/marcos/projects/mimir"
  >> Pathname: "/home/marcos/projects/guard-clone"

Pathname also offers facades for many methods found on the File and Dir classes, allowing you to write cleaner code in some cases.

One gotcha is that most Pathname methods, including its constructor, are just wrappers around string manipulations, and will happily accept any string you throw at it, but some methods, like Pathname.children and Pathname.realpath access the filesystem and will raise errors if the Pathname object does not represent an actual filesystem path.

Ruby File IO Primer - Part 2 - The Dir Class

2011-06-19T00:00:00Z

Continuing our series on Ruby File IO API, we will now take a deeper look at the Dir class

The Dir Class

The Dir class main purpose is to offer facilities for quering, iterating and filtering over filesystem directories entries. It also offers some basic methods for creating and removing directories. Let's go over those first.

Creating a Directory

You create a new directory on the filesystem by passing it's path (full or relative) to the Dir.mkdir method. This method will throw an Errno::EEXIST error if something with that path already exists, or an SystemCallError if the dir cannot be created due to OS permissions.

All the relative paths (including the ones passed to the File class methods) are derived from the current working directory path, accessed through the Dir.pwd method. You can change the pwd using Dir.chdir. To help with the pwd hopping, you can pass a block to the chdir method and the pwd will be reset after its execution.

  Dir.pwd
  >> "/home/marcos/code"
  Dir.mkdir("test")
  Dir.chdir("test") do
    Dir.pwd
    >> "/home/marcos/code/test"
    File.new("file.rb", "w")
  end
  Dir.pwd
  >> "/home/marcos/code"

Removing a Directory

To remove a dir you pass its path to the Dir.rmdir method (also called remove and unlink). This method can only remove empty directories and will raise a SystemCallError otherwise. You can get more powerful directory manipulating methods on the FileUtils class, that will be discussed on a later part of this series.

Iterating over a Directory content

The Dir class offers two options for iterating over the contents of a dir, the Dir.entries method and the Dir.glob method.

Dir.entries returns an array with the name of every single entry inside the directory, including the current path ("."), the parent directory ("..") and all hidden files (those with '.' before their names). The same rules apply to the Dir.each instance method, which returns an Enumerator with all the entries and acts as the base for the Enumerable module methods.

The Dir.entries method can be very annoying if you want to work with just the files, like on the next sample, where we try (in a very contrived way) to remove all the files from a certain directory.

  d = Dir.new("/home/marcos/code")
  d.entries.each do |e|
    next if e =~ /^\./
    file = File.join(d.path, e)
    File.delete(file) if File.file?(file) 
  end

A much better way to list the files in a directory is to glob then, using either the Dir.glob method, or the very similar Dir.[]. These methods accept a pattern and return an array with the path of all the visible files that match the given pattern.

The patterns are based on the terse shell globing syntax, where '' represents any number of wildcard characters, '*' represents all the children directories (matched recursively) and '?' represents just one wildcard.

  "*"         #All Files (Current Dir)
  "help.*"    #All Files with the name help
  "*/**/*.rb" #All Ruby Files (Current and Children dirs)

With the globing technique, our little demo gets much more concise.

  Dir["/home/marcos/code/*"].each { |f| File.delete(f) if File.file?(f) }

The Dir.glob method also accepts flags (combined with bitwise or operators) to customize its behavior like the File::FNM_DOTMATCH which includes hidden files on the result.

  Dir.glob("rakefile")
  >> []
  Dir.glob("rakefile", File::FNM_CASEFOLD)
  >> ["/home/marcos/code/Rakefile"]

Ruby File IO Primer - Part 1 - The File Class

2011-06-06T00:00:00Z

Ruby offers a very powerful API for File IO, allowing for very complex operations with very little code, but the myriad of methods and classes with similar names can be a little confusing for the Ruby uneducated like me.

The File Class

The File class, built in the language (you don't have to require it), offers the most commonly needed facilities for manipulating files in Ruby. Like all other IO things in Ruby, File is an subclass of the IO, which mixes in the Enumerable module. Let's go over the basics.

Reading from a File

The simplest way to get a file is just calling File.new with the correct path (you get an exception otherwise). You can then call File.read to get the entire content of the file as a string.

  f = File.new("lib/file.rb")
  content = f.read
  f.close

That may suffice for some simple uses, but most of the times we want a little more control. You can read the file line by line with the File.readline method or the File.gets method. The difference is that File.readline throws an exceptions if the file is over, while File.gets just returns nil.

  f = File.new("lib/file.rb")
  while line = f.gets
    puts line
  end
  f.close

You can also read the file char by char, with the File.readchar, or byte by byte with the File.readbyte. Both have their get* counterpart, with similar behaviour to those described above. The amount of bytes in a file depends on the encoding.

Since File is an Enumerable, you can also use File.each or File.each_line to loop through the file and access each line with a block.

  f = File.new("lib/file.rb")
  f.each do |line|
    #do something with the line
  end
  f.close

You can have even more read precision by moving the File "inner pointer" with more low level methods like File.seek.

Writing into a File

To write into a file you must first open it on write or append mode. Write mode creates the file if it does not exist or overwrites the old version, and is indicated by a second argument 'w' passed to the File.open method. Append mode creates a file if it doesn't exist or appends to the existing content, and is indicated by the 'a' flag.

The methods used to write on a File instance are puts, which places a newline after the content, or print, that does not append the newline. You can also use File.write, witch returns the number of bytes written into the file.

  f = File.new('lib/file.rb', 'w')
  f.puts("a new line will be appended")
  f.print("no new line")
  f.print(" at all")
  puts f.read
  >> "a new line will be appended\nno new line at all"
  f.close

Using blocks

When you use the File.new method, you have to manually close the file with the File.close method. That is quite boring and also prone to mistakes. A better approach is the File.open method, that pass the file to a code block and then closes the file for you. The open method accepts the same filemode flags as new. If you provide no code blocks, File.open is identical to File.new.

  File.open("lib/file.rb") do |f|
    f.each do
      #do something
    end  
  end

The Enumerability of Files

The fact that File is an Enumerable gives the class a lot of nifty abilities. Its a little strange at first, thinking about files as an array of lines, but you get used to it very quickly. Here are some cool samples.

  #counting commented lines 
  File.open("lib/file.rb").count{ |line| line.starts_with?("#") } 

  #get all the file require lines as an array
  File.open("lib/file.rb").grep(/^require/)

  #the avg of words from each line
  File.open("lib/file.rb").inject(0) do |total, line|
    total += line.split.size
  end