English - AkitaOnRails.com

Chatting with Adam Jacob

Wed, 18 Nov 2009 07:46:21 PST

Configuration Management is a tricky subject. For non-starters, when you’re a developer and you have few boxes to take care of, you can usually get away with just managing them manually. People are probably just used to pop in a CD, double-click the “install” program and click “next”, “next” until the end, then you manually log in to backup (when you remember it), and sometimes you do apply some security updates when you remember about them.

But then you have more than a dozen machines, things start to get uglier, you end up making more mistakes, forgetting important steps, and all of a sudden managing machines become a nightmare. You end up being woken up in the middle of the night because you forgot to install some crucial component, and so on and so forth.

The same way you need testing, continuous integration tools when you’re a developer, you also need automated, reliable and flexible tools for the system administrator role. That’s where tools such as Chef kick in to help.

From Opscode Inc., we have Adam Jacob (CTO) and Jesse Robbins (CEO) to talk about the new contender in the automated system administration field, Chef, already in use by many companies which are striving with the cutting edge to maintain their datacenters.

AkitaOnRails: To kick start this interview, it would be great to have more background info about you guys. So, how did you end up in the configuration management space?

Adam: I’ve been a systems administrator for 13 years, and for a majority of that time I was working for companies who did a lot of mergers and acquisition work. Every couple of months we would acquire a new company, and it was my job to help figure out how to absorb those companies into the whole. I got really good at looking at an application I had nothing to do with creating, and figuring out what needed to be done to make it scale (or at least make it run.)

One thing we couldn’t do was tell the people who had built the applications that they needed to be radically altered (in many cases, they weren’t even around anymore.) What that meant in practice was we needed to have a very flexible, modular underlying architecture – everything we did had to be in service of the application, not the other way around. By necessity that meant becoming a tools developer – if we didn’t have the tools we needed, we would never be able to do the job in front of us.

Eventually I co-founded a consulting firm called HJK Solutions. We built fully automated infrastructure for startups – everything from OS installation to application deployment, all fully automated (including Identity Management, Monitoring and Trending, etc.) Over the course of the next two years we built infrastructure for 12 different startups, whose products ranged from electrical car fleet management to online dating.

So I come to configuration management through the trenches – as a line-level systems administrator trying to make my life easier, as a consultant helping others to reap the benefits, and now as a tool builder trying to move the state of the art forward.

Jesse: Jesse Robbins is CEO of Opscode (makers of Chef) and a recognized expert in Infrastructure, Web Operations, and Emergency Management. He serves as co-chair of the Velocity Web Performance & Operations Conference and contributes to the O’Reilly Radar. Prior to co-founding Opscode, he worked at Amazon.com with a title of “Master of Disaster” where he was responsible for Website Availability for every property bearing the Amazon brand. Jesse is a volunteer Firefighter/EMT and Emergency Manager, and led a task force deployed in Operation Hurricane Katrina. His experiences in the fire service profoundly influence his efforts in technology, and he strives to distill his knowledge from these two worlds and apply it in service of both.

AkitaOnRails: What’s the story behind Opscode, what’s its mission, and what’s the story for Chef’s creation?

Opscode: The story behind Opscode really starts when we met Jesse. He had written an article for O’Reilly Radar about Operations being the new secret sauce for startups. He and I met for coffee, became friends, and stayed in touch. Jesse understands operations culture at a visceral level, and he’s very well connected to a huge community of like-minded people that I didn’t even know existed.

As our consulting company grew, we reached a crossroads – we clearly had a nice business going, and it wasn’t difficult for us to find more work. What was difficult was finding people who had the skills to actually deploy a new infrastructure, and to adapt the stack we had developed for a new application. We couldn’t avoid the fact that there was a couple of weeks of very high-touch work that was required to get the entire infrastructure up and running for a new client.

We tried to recruit Jesse during this period, since he was one of those rare people who could do that initial high-touch engagement. He turned us down – primarily because he saw what we did: we could build a consulting company that was huge, but it would still be down to us in the trenches every day. Unless we could get over that hump, there were probably less stressful ways to make a living. :)

So we started looking at what was stopping us from being able to get that initial part of the engagement done as quickly as possible. What was stopping us from literally having a customer fill out a questionnaire, and letting that data drive 95% of the decisions about the infrastructure?

It turned out the answer was, in a word, “everything”. The entire stack of open source tools we were using had been built in a different era, and they saw the world through a very different lens. We needed everything to have an API, we needed everything to be more open with it’s data, and we needed everything to be flexible enough to handle the next evolution of application architectures (whatever that may be.)

Once we had that revelation, the next step was figuring out what a ‘new stack’ would really look like. If we could start from scratch, what would we take with us, and what would we leave behind? Amazon had done such a great job showing us what an API over the bootstrapping process could look like, and the kind of benefits that could be had from the approach. So assuming something like that existed for bootstrapping, what about the next layer of the stack (configuration)?

We started experimenting with Chef during this period of questioning. We started working on Chef with the goal of putting ourselves out of business – making the barrier to entry to having a fully automated infrastructure so low that any developer or systems administrator could just do it. We built a prototype, showed it to Jesse, and he agreed to come on board.

Opscode was born then, and our mission came out of those experiments: we are bringing Infrastructure Automation to the Masses. We want to tear down the barriers to entry that stop people from having really great, repeatable, automated infrastructure. Our role is to bring developers and systems administrators the best tools possible, so that they can build the systems they have always wanted.

We raised 2.5 million from DFJ in January of 2009 on that vision, and have been at it ever since.

AkitaOnRails: I’d like to say that only amateur sysadmins do everything manually, but I think most small to medium corporations at least still do everything manually or with random scripts spread all over the place. The notion of “configuration management” is still new to a lot of people. Could you briefly explain what it is, and why it is important?

Opscode: To me, “Configuration Management”, at it’s core, is all the stuff you have to do to take a system from “running an operating system” to “doing it’s job”. When a systems administrator configures a system by hand, and posts her notes on a Wiki, she’s practicing the most primitive form of Configuration Management.

Now, having those notes is better than nothing – when she needs to do that task again, she can at least go back and read them to remember what she did last time. She still has to do it over again, though – and the repetition gets tiresome. So she starts writing scripts that encapsulate that knowledge in code, so that now she only has to run a series of smaller commands.

Time passes, though, and entropy sets in. The systems begin to drift from where they were when the systems administrator wrote the scripts. Next thing you know, the scripts don’t run anymore, or if they did, the configuration they build is wrong. Our intrepid admin then starts editing the scripts, or making new ones, to deal with the system when it’s in this new state. This is the stage we call “tool sprawl” – you have a tool for each different phase of a systems observable life-cycle.

Modern configuration management tools solve this problem by providing a framework for describing the final state a system should be in, rather than the discrete steps we should take to get there at any given time. Rather than writing a script that lists the commands to install Apache, write the configuration file, and start the service, you would describe that “apache should be installed”, the configuration file “should look like this”, and the service “should be running”.

When the configuration management system runs it looks at each of these descriptions individually, and makes sure that they are in the proper state. We no longer care what the initial state of the system is – the configuration management system will only take action at each step in the process if the system is not in the state you described. If you run it again, and nothing has changed, the system takes no action. (Configuration management geeks call this property idempotence)

This means that you can describe your systems once in code, and as they change over time, simply update that code to reflect what you want the state of those systems to be. The impact of this model on the daily life a systems administrator cannot be overstated – it makes everything easier.

It also has huge impacts outside the systems administrators world. You now have a living document that describes how all your servers are configured – you can share it, you can put it in revision control, you can print it out for an auditor. Business processes that previously required manual intervention frequently can be boiled down to discrete changes. The more you work in this way, the more the impacts spread throughout the organization.

AkitaOnRails: Even though the Ruby community probably compares Chef to Puppet, I think one of the most widely used system is CFEngine2. How does Chef compare with CFEngine2?

Opscode: Cfengine 2 is where almost everyone in the configuration management world cut their teeth. Mark Burgess is responsible for the academic papers that outlined the idea that each part of the system should be idempotent, and his work in studying how real-world systems can be managed at scale has done more to impact the evolution of configuration management than any other individual.

In addition to the idea of idempotent resources, Burgess introduced the larger concept of “convergence”. The basic idea here is that if you have the description of how every finite resource should be configured, given enough time, those resources will bring the system into a compliant state. The order in which the resources run fundamentally does not matter – eventually, they will all wind up in the right place.

While this model works at a fundamental level, it has some pretty dramatic inefficiencies. Cfengine applies resources based on their type – all the files are managed at once, then all the services, then all the packages. You can control the order they are run with the ‘actionsequence’, but each system gets only one order. So if you need to have a file copied, a package installed, and a service restarted, it’s easy enough to model. When you start getting into more complex configurations, however, it becomes more and more difficult to get an actionsequence assembled that allows you to configure your entire system in a single run.

This is not a bug in Cfengine – it’s by design. Convergence is the answer – it’s okay that it has to run more than once, it will get there eventually. If you are thinking too much about the order things should happen in, you’re probably not thinking about idempotent descriptions.

In my experience, though, this was frustrating at scale. It meant that the amount of time it took to configure a system increased as the configuration became more complicated, and your ability to model complicated interactions in the system became increasingly opaque.

For the era in which it was written, this problem mattered a lot less. If it takes you 6 to 8 weeks to get a server even ordered, then another week to get it racked, stacked, and the OS installed, the fact that Cfengine needs to run 3 times to configure it doesn’t matter very much. (You have lots of bigger problems!) In a world where many of us can go from bare-metal to running operating system in 5 minutes, either via our own install systems or an API call to AWS, it starts to matter a whole lot.

Chef takes a different approach. We start with the idea of an idempotent “resource” – a package, a service, etc. We add the idea of “actions” – these are all the different states you might want to request a resource to be in. Then you put these resources into “recipes”, which are evaluated in the order they are written. You can then have recipes rely on other recipes having completed before they are run, giving you the ability to say “make sure Apache is installed before you configure my Web Application”.

The goal is that, with Chef, you can always bring the system into the proper state with a single Chef run. It sees convergence as a response to a bug – a system that is 1/2 configured is, in fact, broken. If the cause of the bug is environmental – a network service is not available, for example – then running Chef again will likely fix things. If the issue is that you haven’t specified the order that you want resources to be configured in, though, then it’s a bug in your recipes. If you can’t write a recipe to configure your system in a single Chef run, it’s a bug in Chef.

This also has side-effects for reasoning about the system at scale. Given the same configuration and attributes, Chef will always behave the same way, on every system. As you add more and more things to the system, it’s still easy to reason about when, and how, they will be configured.

Interestingly enough, Cfengine 3 implements a system that is very similar to Chef. Rather than a single global actionsequence, it uses a ‘bundlesequence’, where bundles are roughly analogous to Chef’s concept of a recipe.

Another major difference is that Chef allows you to ‘look up’ at your entire infrastructure when configuring a system. This ability comes in handy when you want to do things like configure a monitoring system, or a load balancer. You can ask Chef questions like “what are all the servers running my application in production”, and use the response to configure your system. (You can see an example of this on our blog)

From a systems architecture point of view, Chef and Cfengine are actually fairly similar. They both push the majority of the hard work of configuring a system out to the systems themselves. This is highly advantageous at scale – the Chef and Cfengine servers are really glorified file transfer systems.

We differ pretty deeply on our approach to what a language for configuration management should look like, but I’ll talk more about that in a later question.

Cfengine is at work in some of the largest data centers in the industry, and it fundamentally altered the landscape of systems management. While it’s not the tool I want anymore, you can’t understate the impact it’s had on the design of every configuration management tool that came after it – including Chef.

AkitaOnRails: Chef has a lot of components to it. Could you briefly describe all the main components that work together? The client side, the server side, cookbooks?

Opscode: Sure! The most important part of Chef is the cookbooks – they are where you actually describe your configuration. They collect recipes, and all the assets needed for the recipe to run (files, templates, etc.). Cookbooks are very often share-able, and lots of cookbooks already exist.

Chef can run your cookbooks in two modes – a client/server mode (chef-client), or a stand-alone mode (chef-solo). When you run chef-solo, you pass a URL to a tarball full of the cookbooks you want to have run. There is no more infrastructure than that – put your tarballs someplace you can download them, and go nuts.

In client/server mode, each Chef client is configured to talk to a Chef Server. The Server stores information about each client (a whole lot of it – things like IP addresses, loaded kernel modules, and more), and distributes the cookbooks they need to configure themselves. It also provides a REST API and an interactive Web UI, so you can easily alter the configuration of your systems centrally. Finally, all the data the server collects is indexed and searchable – you can then use this in your recipes to configure services that require complex, dynamic configuration. (Some examples would be dynamically discovering a master MySQL server, or finding all the memcached servers in a cluster)

AkitaOnRails: What would you say about Chef’s maturity? CFEngine has more than a decade of usage, which is difficult to beat. Would you say that it’s “mature enough”? Meaning, it’s already in production in companies of many sizes, its APIs don’t change too much and my cookbooks will probably work if I upgrade to newer version of Chef?

Opscode: Chef is a little over a year old – it was first released to the public on January 15th, 2009. Since then, 42 different developers have committed to the project, around 5 of whom work for Opscode. It’s been in production use in the Engine Yard Cloud from the day it was released. Since then, it’s seen adoption by companies and universities of all different sizes, from small startups to huge enterprises with very heavy compliance requirements.

So it’s definitely “mature enough” for real world use – lots of people are using it, and relying on it, every day. Balancing that knowledge with the reality that the project is fast moving and evolving is important, and we try and do it in a number of ways:

The recipe syntax is considered basically “complete”. If we do make changes, they are backwards compatible to previous Chef releases. Short of something truly amazing coming along that radically alters the shape of the universe, this will remain true.
We test a lot. Chef has over 2000 unit tests, and functional tests that cover the entirety of the REST API, and much of the individual resources (we’re aiming for 100% feature test coverage.) There are more lines of code testing chef than there are lines of code in Chef proper.
If we do break backwards compatibility, we try and make it happen only when we bump the minor revision. If you have an 0.7.x version of Chef installed, it should always work with other versions of Chef in the same release cycle.

Most of the work in Chef currently is around adding more functionality, not in changing the way existing functionality works. It’s safe to use Chef today – in the future, you’ll just keep getting more good sauce to add to the mix, rather than having to deeply re-factor the way you do things.

Finally, the Chef Community is truly epically great – we have lots of people who are spending significant amounts of their time with Chef. Even if they aren’t contributing code, they are answering questions, they are writing documentation, they are hanging out on IRC and offering you cupcakes. It’s a group of people that are focused on solving real world problems, and helping each other to do the same. It’s by far what I’m proudest of, and I think it has a significant impact on whether Chef is ready for prime time. The bench is really, really deep.

AkitaOnRails: Chef uses Ruby directly, which some would say it’s both a blessing and a curse. It’s probably perfect for Rubists, but I feel that most sysadmins are used to Bash, Python and are not very flexible on change. Why did you choose to use Ruby instead of a simpler language?

Opscode: The answer to the question of why we use Ruby directly for the configuration language comes in two parts: why we extended a 3GL rather than build a declarative DSL or a complete modeling language, and why we chose Ruby as that 3GL.

First, why we extended a 3GL. Tools like Cfengine and Puppet build a declarative DSL for configuration management – a custom language, which provides a model within which systems administrators or software developers can work to automate their system. Other tools like Elastra’s EDML and ECML, OpsWare’s DCML, or Bcfg2 give you an XML schema to describe how the system should behave.

The issue with these approaches is that, by definition, they must build an abstraction for every task the end-user may want to perform: an impossible feat. The level of complexity inherent in automation, coupled with the inability to break out to 3GL language constructs when necessary, result in a system that can only target a subset of the total complexity, rather than enable users to find innovative solutions to their specific problems. By leveraging Ruby, adding support for other use cases is a matter of adding new sets of base classes while maintaining consistency and approachability in their interface design, because the full scope of the language is available.

Our goal with Chef was to keep the simplicity that comes from having the focus be on idempotent resource declaration, while giving you the flexibility of a full 3GL. In practical terms, anything you can do with Ruby you can do with Chef – and since Ruby is a 3GL, that amounts to essentially anything.

Larry Wall, the creator of Perl, has a quote that I love:

“For most people the perceived usefulness of a computer language is inversely proportional to the number of theoretical axes the language intends to grind.”

Since we wanted Chef to have the maximum amount of usefulness, we have actively tried to remove as many theoretical axes as possible – a belief that we can imagine the total breadth of the problem space being one of them. There is more than one way to do it with Chef, and the only valid criteria for rating the success of your automation project is whether it solves your problems in a reliable way.

In practice, you need to know very little Ruby to use Chef. Here is an example of installing the program “screen”:

123	package "screen" do action :installend

The same thing in Puppet:

123	package { "screen": ensure => present }

And in Cfengine 2:

12	packages: screen action=install

While all of these systems require learning the syntax, at a base level, there isn’t much difference between them in terms of raw learning required. The difference is that when you hit a limitation in Chef, you have the ability to innovate easily, and when you hit those same limitations in other tools, you do not.

We chose Ruby because of it’s fairly unique ability to create new syntax easily. Tools like Rspec are fine examples of ways you can manipulate Ruby for fun and profit that are very difficult to duplicate in other tools. We wanted to make sure that, even though you were “in” a 3GL, you didn’t have to go through any extra hoops to make the simple things work. Ruby was the language that I was comfortable enough in, that I knew had the ability, to make that a reality.

That said, one of our goals is to extend the ability to write Chef recipes into other 3GL’s. We have an example of doing this already with Perl – and we made no changes to the Chef source to accomplish it. You can see the demo here on CPAN. It works by using Chef’s JSON API to ship the compiled resources over to the Ruby library for execution, and over time we’ll be extending those interfaces. You’ll be able to have recipes written in Python interoperating with recipes written in Ruby and Perl.

AkitaOnRails: Does Chef have (or for that matter, need) something like Augeas, which Puppet is trying to support?

Opscode: Augeas is neat. Chef doesn’t have Augeas support today, and the reason is that nobody has needed it badly enough to write the integration. One reason is that, with Chef, it is quite easy to dynamically add to a resource (like, say, a template) or search for particular systems that match a criteria. This means that the use-case for Augeas (which edits files in place) is less necessary – you can often get the data you need to render a template, rather than needing to build it up over time with incremental edits.

We think this is a better practice in general, as it ensures that all the systems you are managing can always be restored to a working state from nothing but the cookbook repository. If you use Augeas to allow idempotent changes of individual lines of a configuration file, it encourages the behavior of individual administrators editing files in place, which is a configuration management anti-pattern.

AkitaOnRails: Sysadmins used to CFEngine complain about Ruby’s dependencies and overall weight. Because for Chef to run you need Ruby installed. Not all distros have Ruby in the same version (although most already migrated to 1.8.7). Then you have the problem of weight. I am not familiar with Chef, but Puppet can grow to hundreds of megabytes. What they don’t want is to have clusters of Chef machines (which, by themselves, also need maintenance, adding to the overall complexity). How do you deal with datacenters with thousands of servers? I know it’s difficult to measure precisely, but what would be a reasonable ratio between Chef servers x managed servers?

Opscode: When you are evaluating the scalability of configuration management systems, you want to look at two different axis. The horizontal one, which is the number of systems that can be managed by a single configuration management server at a particular rate of change, and the vertical, which is how much of your infrastructure can be automated by the tool.

On the vertical axis, I think Chef is the clear winner, for reasons I think are pretty well summed up by my answer to question 7. I would put Puppet second, and Cfengine last.

On the horizontal axis, Cfengine is the clear winner. It’s written in C, and it has the thinnest possible server component – it does nothing but authenticate clients and serve files, essentially. I know first hand of data-centers that are running huge numbers of servers off a single cfengine server.

One important metric to keep in mind when discussing the horizontal scalability of a configuration management solution is that the most important metric is the rate of change. All the tools we’ve been talking about are ‘pull’ based – the clients check in at an ‘interval’ with the server, and apply a ‘splay’ to ensure that not all the systems check in at once. A common out of the box configuration is an interval of 30 minutes, with a splay of 15 minutes. (This means a server checks in every 30-45 minutes, depending.) If you are comfortable increasing that interval, you will get more scalability out of fewer resources (by lowering the amount of concurrency.)

So when anyone says “I have 10 thousand servers on a single configuration management server”, ask them “at what interval?”.

Chef scales like a web application. The server itself is quite thin, and is responsible for authenticating clients, transferring files, storing node state, and providing a search interface. It scales horizontally by adding new Chef Servers as necessary. The API is RESTful, and there is no session state between the clients and the server. (At least not in Chef 0.8+) When you encounter scalability problems with Chef, the tools you apply are the same ones you apply to any well designed web application.

You asked specifically about memory utilization – Chef does quite well in this regard. Individual server processes usually are between 14-50MB resident. The client itself, running in daemon mode, is usually around 28MB resident.

In our testing, the current bottleneck in a Chef server is CPU. Chef is smart about how it handles file transfers – we only transfer files that have changed on the server. To support this we calculate a checksum for each file requested, and we currently don’t cache the results. We’re planning on fixing this for the next major release of Chef (0.8.0) which should shift the bottleneck over to RAM.

With Chef 0.7, you should be able to support thousands of clients at a half-hour interval and fifteen minute splay on reasonable commodity hardware. The changes in Chef 0.8 should bring that number up dramatically – I’ll get back to you with some benchmarks once the patches are in. :)

AkitaOnRails: Probably related to the previous question, seems like specially after Sarbanes Oxley there’s been an increasing interest in stuff such as ITIL, CoBit. Have you ever seen successful implementations of those in the Web-style infrastructure? I mean, I can see them succeeding in Banks, Aerospace and Defense, etc but I fail to see them working as advertised in a very dynamic environment such as Web services hosting. What are your experiences regarding this issue?

Opscode: Well, I think you can think of ITIL in the same way you think about the classic ‘Waterfall’ model of software development. For some kinds of projects and companies, it is essential – it’s hard to imagine working a different way. Most often these are companies with huge manufacturing or quality control concerns – medical health, aerospace, banking and finance, etc.

The same thing applies to ITIL – the larger the concern, and the more stringent the requirements, and the longer the lead time, the more the processes they describe start to make sense. Like all large process, though, they tend to de-emphasize the human element – people have roles to play, and forms to file. :)

In the Web Ops culture, things are different. I’ve never seen a successful marriage of ITIL and Web Ops, and the reason is that the domains are so very different. If there is a bug on the website, it’s better to ship the fix now than wait for a release management process to ensure that the site won’t have any more issues based on your fix, for example. It’s also a bad cultural fit – in the best Web Ops teams, the focus is heavy on communication, agility, and respect, rather than process, formalism, and tooling. The shift you start too see in the really great Web Ops companies is that their operations personel become enablers of the organization, rather than end-line blockers of change (to keep stability high.)

In general, I think the 4 steps outlined in visible ops for emergency management are not bad ones, but the devil is always in the details. The guy you should really ask about this is Jesse – he’s largely responsible for Amazon’s operational culture, and knows what it means to start hacking on that sort of thing from within huge organizations.

AkitaOnRails: Awesome, I think this is a wrap. Thank you very much for this interview!

Chatting with Luke Kanies

Wed, 18 Nov 2009 07:45:32 PST

This time I’ve interviewed Luke Kanies, from Reductive Labs, former contributor to the famous CFEngine tool and creator of Puppet, one of the most acclaimed configuration management tool for 21st century datacenters.

AkitaOnRails: To kick start this interview, it would be great to have more background info about you. So, how did you end up in the configuration management field? I understand that you have a long history with CFEngine development, right?

Luke: I was a Unix admin going back to 1997, always writing scripts and tools to save myself time, and around 2001 I realized that I shouldn’t have to write everything myself – that someone somewhere should be able to save me time. After a lot of research and experimentation, I settled on Cfengine, and had enough success with it that I started consulting, publishing, and contributing to the project.

AkitaOnRails: What’s the story behind Reductive Labs, what’s its mission, and what’s the story for Puppet’s creation?

Luke: After a couple of years with Cfengine, I had a lot more insight but was frustrated because it still seemed to be too hard – no one was sharing Cfengine code, and there were some problems it made you work really hard to solve. The biggest issue, though, was that its development was very closed – it was difficult to contribute much more than just bug fixes.

I got frustrated enough that I stopped consulting and looked for other options. I worked briefly at BladeLogic, a commercial software company in this space, but in the end I decided that the insight I had and the lack of a great solution were a good enough start for a business that I decided to morph my consulting company into a software company and write a new tool.

Luke: It’s surprisingly difficult to describe it succintly, but for me, there are two key rules: You shouldn’t need to connect directly to a given machine to change its configuration, and you should be able to redeploy any machine on your network very quickly.

These two rules combine to require comprehensive automation and/or centralization for everything that goes into making a machine work. Annoyingly, they also immediately introduce dependency cycles, because your automation server needs to be able to build itself, which is always a bit of a challenge.

AkitaOnRails: I think one of the most widely used system is CFEngine2. How does Puppet compare with it? Meaning, what do I have as value added when switching to Puppet, and what are the known caveats?

Luke: There are multiple important functional differences. The biggest is Puppet’s Resource Abstraction Layer, which allows Puppet users to avoid a lot of the detail they don’t really care about, like how rpm, adduser, or init scripts work – they talk about users, groups, and packages, and Puppet figures out the details.

We also have explicit dependency support, which makes a huge difference – it’s easy to order related resources and restart services when their configuration files change, for instance.

The language is also a bit more powerful. Like Cfengine, we have a simple custom language, but Puppet’s language provides better support for heterogeneity, along with a composite resource construct that allows you to easily build in-language resource types like Apache virtual hosts that model more complex resources consisting of multiple simple resources.

AkitaOnRails: Puppet has a lot of components. Could you briefly describe some of the main ones that work together? The client side, the server side, recipes?

Luke: Most people use Puppet in client/server mode, where the central server is the only machine that has access to all of the code, and it runs a process capable of compiling that code into host-specific configurations. Then each machine runs a client (including the server), which then retrieves and applies that host-specific configuration. This has nice security implications because you’ve not shipped your code to every machine on your network.

If this model doesn’t work for you, though, it’s also easy to run Puppet stand-alone, where each machine has all of the code and compiles it separately. Multiple Puppet users do this for various reasons. This stand-alone ‘puppet’ executable is a standard interpreter – it can be used to run 1 line scripts or thousands of lines in a complete configuration.

Beyond this, we’ve got a few other interesting executables, such as to access our certificate authority functionality, and an interesting executable called ‘ralsh’ that provides a simple way to directly manage resources from the Unix shell, without having to write a separate script.

AkitaOnRails: What would you say about Puppet’s maturity? CFEngine has more than a decade of usage, which is difficult to beat. Would you say that it’s “mature enough”? Meaning, it’s already in production in companies of many sizes, its APIs don’t change too much and my recipes will probably work if I upgrade to a newer version of Puppet? I think the 0.x version makes some people nervous :-)

Luke: Really we should have called a version from 2007 1.0, but it’s hard to know how stable a release is going to be until it’s been out for a while. :)

It’s obviously tough to match Cfengine’s long life, although they’re somewhat forcibly migrating to Cfengine 3, which is a complete rewrite, so that maturity isn’t worth quite as much right now.

However, Puppet’s been in production usage around the world since 2006, and it’s currently used by more large companies than I could reasonably name – Twitter, Digg, Google, Sun, Red Hat, and lots more – and our community and customer base consider it to be mature. For the line came some time early last year, when I found that the vast majority of issues people had were user issues on their part, rather than some flaw or shortcoming in Puppet.

In general the APIs are quite stable, and we’ve done quite well, I think, at maintaining backward compatibility when the APIs have had to change. The point about API stability in a 1.0 release isn’t so much to differentiate it from previous efforts as to make a promise for the future. This especially matters for companies like Canonical, who want a release that they can support on Ubuntu for five years.

AkitaOnRails: Puppet has its own language and you can use Ruby for the advanced cases. It’s probably perfect for Rubists, but I feel that most sysadmins are used to Bash, Python and are not very flexible on change. Why did you choose to use Ruby instead of a more widespread language? What do sysadmins need to realize to shift paradigms?

Luke: Part of it is that most people don’t really need to know any ruby to be effective with Puppet. Sure, you can get some more power if you do, but if you’re not a language person, you’re perfectively functional with just Puppet.

Another nice thing is that we have a pretty smooth scale in terms of Ruby knowledge – you can start out writing ERB templates or five line extensions to Facter, which is our client-side querying system, and grow smoothly through to writing custom resource types.

In the end, I chose Ruby because I was most productive in it. I likely should have chosen Python, given its speed benefit and popularity at Red Hat and other places, but I found I just couldn’t write code in it. I started thinking in Ruby after only a few hours of usage, so it was impossible for me to turn away from it.

AkitaOnRails: Sysadmins used to CFEngine complain about Ruby’s dependencies and overall weight. Because for Puppet to run you need Ruby installed. Not all distros have Ruby in the same version (although most already migrated to 1.8.7). Then you have the problem of weight. Puppet can grow to hundreds of megabytes. What they don’t want is to have clusters of Puppet machines (which, by themselves, also need maintenance, adding to the overall complexity). How do you deal with datacenters with thousands of servers? I know it’s difficult to measure precisely, but what would be a reasonable ratio between Puppet servers x managed servers?

Luke: It’s as impossible to tell you how many clients a Puppet server can handle as it is to tell you how many clients a Rails server can handle – it all depends on the complexity of the tasks. Google scaled to 4500 client machines on a single server, but most people tend to add another server at around 500-1000 clients.

It’s true that it’s hard to keep memory usage down in a Ruby process, but we’ve made great strides in our recent releases by doing things like deduplicating strings in memory and being more efficient in our code paths. Really, though, we’ve spent a lot more time on features and bug fixing and less time on optimizing – until recently, we’ve been a small development team, and we just didn’t have the bandwidth for it.

Now that my company, Reductive Labs, has some investment, we’ve been able to add three full time developers, which is going to really help in this area.

As to dependencies, this is one area we break strongly from the Ruby community – we don’t require a single gem, other than our own Facter tool (and it’s usually not shipped as a gem). Rubyists tend not to worry too much about package dependencies – they just put it in vendor, as I’m fond of saying – but that doesn’t work when you have to deploy thousands of copies. So yes, you might have to install Ruby, but there won’t be any other dependencies you have to deal with, which greatly simplifies it.

It’s generally as tough to know how you’ll need to size your puppetmaster as it would be to size a web server – it depends on how complicated the workload is. In general, somewhere between 500 and 5000 clients, you’ll need to have a second server, but most people probably find it closer to 500. Really, though, if you’ve got 3000 clients hitting a service, you probably want to make it horizontally scalable for stability in additional to performance.

AkitaOnRails: Security is a big concern nowadays, Puppet was worried from the beginning on the handshake procedure between clients and server, can you describe it a little bit? Also, is there any built-in recipes for hardening machines, for example? Or at least any desires to add such tools in the future?

Luke: Puppet uses standard SSL certificates for authentication and encryption, including a certificate signing phase. By default, the client generates a key and a certificate request (CSR) and then uploads the CSR to its server. This upload, along with the later certificate download, are the only unauthenticated connections that are allowed by default.

From there, a human normally has to trigger the client’s certificate to be signed, but many organizations, including Google, automatically sign client certificates because they trust their internal network.

As to automatic hardening, there aren’t any recipes that I’m aware of right now, but it’s something that I’m definitely interested in. Years ago I was a big fan of TITAN, which is a hardening package for various *NIX platforms, and it was part of the inspiration to write Puppet – I’ve always wanted a portable, executable security policy.

AkitaOnRails: The puppetmaster uses Webrick by default, but the documentation also describes using Mongrel or Passenger. Are there any real gains in using those? Is it more for convenience or do we have performance/robustness improvements?

Luke: Holy cow Webrick is slow. It’s really fantastic for proof of concepts – get up and running in minutes. Once you get beyond that proof of concept, though, you really need to switch to Mongrel or Passenger. If you get more than one concurrent connection in webrick, your clients start to suffer, but you can scale to far more with the other solutions out there.

AkitaOnRails: Are there any clients case you are allowed to talk about? Meaning, more details on the kind of infrastructure, difficulties, caveats, best practices?

Luke: The possibilities here are pretty open ended. Google uses Puppet to maintain their corporate IT, which means they’re running it on thousands of laptops and desktops which is pretty different. MessageOne, a division of Dell, is really interesting in that their developers have to ship Puppet code to manage the applications that they ship, so if an app isn’t trimming its logs or backing itself up, it’s a bug that the app developer has to fix rather than the sysadmin. This really helps to bridge the divide between dev and ops, which has worked out really well for them.

Otherwise, there are lots of stores and best practices, but I’m afraid that would be a whole second article. :)

AkitaOnRails: I’ve seen Andrew Shaffer talk about Agile Infrastructure for a couple of years now, but I still think most IT organizations are unaware of this concept. Can you elaborate on what does it mean to be Agile outside of the development field?

Luke: I think Agile Infrastructure has even less adoption than Agile Development. The vast majority of IT shops haven’t changed practices significantly in years and are largely unprepared for the growth in server count that they’re experiencing. They mostly try to scale by adding more people, which we call the meatcloud, rather than scaling their tools and practice.

Luke: In general, I think these kinds of high-level policies are great for filings but aren’t so great for actually solving problems. The bigger a company is and the more public they are, the more likely they are to care about ITIL et al, but it doesn’t really help them solve problems outside of PR in my experience. You can be ITIL compliant and dysfunctional, or completely out of compliance but in fantastic shape. Considering that the best standards are derived from implementation and best practice, which few of these are, I don’t have a lot of hope for these being adopted by the best shops out there.

My personal experience is that very few companies ask or care about these standards, and the ones that do usually do so in a kind of checkbox way, in that they want to make sure they can check off things like CMDB but they aren’t really that concerned with the specifics.

AkitaOnRails: I think this is it! Thank you very much for this conversation!

Rubistas.com.br

Wed, 11 Nov 2009 11:22:15 PST

Last night I made available www.rubistas.com.br, a translation fork of the original Rubyists.eu. I thought it’s a great website because of its simplicity on the approach to group together communities and make them easier to find for everyone.

But, I was careless. According to the Creative Commons License I failed to add proper attribution to Rock & Code and Rubyists.EU. So, Rock & Code contacted me. We talked and I agreed to cease and desist both my fork at github and the website.

I was told that the way I wrote the original blog post in Portuguese (already removed) did upset some of the European collaborators. Specially the part that would translate as “… it is nothing more than a Google Maps mashup.” Again, the intention was never to be negative, quite contrary, it was to pinpoint the fact that it would not be difficult for people to collaborate.

So, rubistas.com.br will remain down. I have no plans for it for the time being.

[Rails Summit 2010] Commence Planning

Thu, 29 Oct 2009 21:39:14 PDT

Railers all around the globe, I am very happy to report that the Ruby on Rails community in Brazil is growing strong. Last 13, 14th of October we had Rails Summit Latin America, with 550+ attendees we had the 2nd edition of the second largest Rails conference in the world after the official RailsConf.

This conference is possible thanks to Locaweb, the largest hosting company in South America. Together, we were able to organize two very successful events. When I started being a Rails activist back in 2006, with virtually no rubyists in Brazil, I never imagined how far we would be able to go. Now we have a healthy ecosystem, with some great recognized open source programmers, some small consultings starting to pop up, a few startups starting to show up, even some ‘enterprisey’ companies coming to adopt it. The Brazilian Rails community is a force to be reckoned.

There’s a lot more to do though, and I am again counting on the international rubyists support to keep on going. The last 2 Rails Summits were real challenges, and we will strive to continue to deliver the best conference in South America. Up to this year’s edition I had to have closed speakers list, but for 2010 I will finally be able to have an open Call for Participation, and I hope Railers all around the world show interest to come visit Brazil, get to know our beautiful country and meet our friendly people. I will let you know the details next year, as soon as I can.

Rails Summit gets inspiration from the official RailsConf. The format of the conference comprises 2 full days, with 2 parallel tracks, a Lightning Talk session, real-time translation from English to Portuguese and Portuguese to English, rooms with power plugs and ubiquitous wireless internet access. Up to this year we used the Elis Regina auditorium, inside the Anhembi Convention Center area, in São Paulo City. For next year we will move it to Frei Caneca Convention Center, which is also in São Paulo City, nearer the famous Paulista Ave. area, much more convenient to find hotels and entertainment areas. I think you will enjoy it. Prices will be around USD 140 for the same 2 full days of event. Above, you can see 2 photos from one of the other conferences that Locaweb does in the Frei Caneca Convention Center.

We recorded all the talks from this year’s edition and I will release them as soon as possible, so you can see how it was. Follow me on twitter or subscribe to this blog’s feed. I’ll keep you posted.

[Rails Summit 2009] ThoughtWorks pousa no Brasil

Thu, 08 Oct 2009 14:55:34 PDT

English readers: click here

Como eu twitei ontem, estamos muito orgulhos de anunciar que a ThoughtWorks se juntou como patrocinador da conferência na última hora. Inclusive eles resolveram dar um presente à comunidade: patrocinar metade de 30 ingressos para o Rails Summit, promoção que começou algumas horas atrás e ficará no ar até acabarem os ingressos patrocinados. Não percam!

Mais ainda, você deve ter ouvido rumores sobre isso. Roy Singham, fundador da ThoughtWorks, soltou o anúncio na conferência Agiles 2009 acontecendo em Florianópolis, SC alguns minutos atrás. Estou muito contente de confirmar que sim, os rumores eram verdade: a ThoughtWorks decidiu aterrissar no Brasil. Eles estabelecerão seu quartel-general em Porto Alegre, RS, começando este ano.

Agora, eles tem intenções de começar a contratar desenvolvedores de software brilhantes imediatamente. Eles estarão no Rails Summit para falar com você, procurando grandes pessoas para embarcarem nessa empreitada.

A ThoughtWorks é muito conhecida como uma força no mundo de desenvolvimento de software, com diversos escritórios pelo mundo. Eles tem grandes e conhecidos engenheiros, arquitetos trabalhando lá, redefinindo a maneira como fazemos software, como Martin Fowler.

Não percam essa oportunidade de vir conhecê-los, pessoalmente. Estaremos esperando por você no Rails Summit 2009, semana que vem!

ThoughtWorks lands in Brazil

As I twitted yesterday, we are very proud to announce that ThoughtWorks joined us as sponsor for the conference at the last moment.

Moreover, you may have heard rumors about this. Roy Singham, founder of ThoughtWorks, just released the announcement at the Agiles 2009 conference in Florianópolis, SC a few minutes ago. I am very happy to confirm that yes, the rumors are true: ThoughtWorks decided to land in Brazil. They will establish their head-quarters in Porto Alegre – RS, starting this year.

Now, they have intentions to start hiring bright and insightful software developers right away. They will be at Rails Summit to talk to you, searching for matches to sign up for their enterprise.

ThoughtWorks is a well known software development powerhouse, with offices around the globe. There are several great and well known engineers, architects working there, redefining the way we do software, such as Martin Fowler.

Don’t miss this opportunity to come and get to know them, in person. We will be awaiting you at Rails Summit 2009, next week!

[Rails Summit 2009] Palestrante Arthur Zapparoli

Tue, 06 Oct 2009 08:03:42 PDT

English translation: click here

Arthur é Desenvolvedor de Softwares, trabalha com Ruby e Rails desde 2006. Atualmente trabalha para a startup brasileira Spix. É conhecido como @ArthurGeek na comunidade e publica artigos no seu blog no qual fala sobre assuntos relacionados à desenvolvimento.

Ele foi um dos que começou a evangelizar mais o Merb na comunidade brasileira, assim como o Git como versionador de controle.

AkitaOnRails: Como você encontrou Ruby/Rails, o que foi que mais te atraiu?

Conheci Rails na época da versão 0.13. Eu já trabalhava com Python, e adorava os recursos de uma linguagem dinâmica. Quando conheci o Rails, não me apaixonei por ele de cara, ainda gostava mais do Python. Nessa época, me envolvi com alguns “clones” do Rails para o Python (ironicamente, nenhum deles deu certo). Porém, como precisava de algo pronto para ser usado, acabei usando o Rails nos projetos da empresa onde trabalhava, e, com isso, precisei estudar Ruby da maneira certa. Depois disso, me apaixonei por Ruby e estou nessa até hoje. Claro que não abandonei o Python, mas hoje posso dizer que Ruby é minha linguagem favorita.

AkitaOnRails: Quais assuntos mais te interessam atualmente e porque?

Tenho um grande interesse pelo Rails 3, já que sempre gostei do Merb. Ruby 1.9 é algo que utilizo nos dias de hoje e estou sempre de olho nas novidades com relação à isto. Fora do mundo Ruby/Rails, me interesso bastante pelos “Document based databases” como CouchDB, e, principalmente o MongoDB. Tanto o Ruby 1.9 quanto o MongoDB é algo que utilizamos no dia-a-dia da empresa onde trabalho.

AkitaOnRails: Sobre o que trata a sua palestra e para que tipos de desenvolvedores é mais indicado?

A palestra se chama: “Git: Controle de versões do jeito certo” e abordará temas básicos, diferentes workflows e algumas dicas mais avançadas. É indicada para desenvolvedores que não usam nenhum controle de versões ou para aqueles que querem saber mais sobre como o Git pode ajudar no dia-a-dia de seu trabalho.

Speaker Arthur Zapparoli

Arthur is a Software Developer, works with Ruby and Rails since 2006. He currently works for a Brazilian startup Spix. He is known as @ArthurGeek in the community and posts articles in his blog, talking about topics related to software development.

He also began helping to evangelize more Merb in the Brazilian community as well as the Git version control system.

AkitaOnRails: How did you find Ruby / Rails, what attracted you the most?

I got to know Rails at the time of version 0.13. I’ve worked with Python, and loved the features of a dynamic language. When I met Rails, I didn’t fall in love with it right away, I still liked Python more. At that time, I got involved with some “clones” of Rails for Python (ironically, none of them worked for me). However, as I needed something ready to be used, I ended up using Rails in projects for the company where I worked, and, therefore, I had to study Ruby the right way. After that, I fell in love with Ruby and I’m with it today. Of course not I’ve not abandoned Python, but today I can say that Ruby is my favorite language.

AkitaOnRails: What topics interest you the most today and why?

I have great interest in Rails 3, as I always liked Merb. Ruby 1.9 is something that I use today and I’m always on the lookout for news regarding this. Outside the Ruby / Rails world, I’m very interested by “Document based databases” such as CouchDB, and especially MongoDB. Both Ruby 1.9 and MongoDB are things we use in a daily basis at the company where I work.

AkitaOnRails: What is your talk about and what kind of developers will benefit the most?

The talk is called “Git: Version control the correct way” and will address key topics, different workflows and some more advanced tips. It is recommended for developers who do not use any version control or for those who want to learn more about how Git can help in your daily workflow.

[Rails Summit 2009] Palestrante Nando Vieira

Thu, 01 Oct 2009 08:32:35 PDT

English translation: click here

Nando Vieira é um dos rubistas mais antigos da nossa comunidade. Ele bloga no Simples Ideias, um dos blogs mais conhecidos sobre Ruby on Rails do Brasil. Trabalha atualmente como desenvolvedor web na Abril Digital, utilizando Rails. É também o fundador do Spesa, além de manter dezenas de projetos Open-source.

Recentemente lançou um e-Book da sua série HOWTO entitulado O que mudou no Ruby 1.9. Da mesma série você pode baixar o e-Book gratuito Rails Application Templates que ele também escreveu.

Venha conhecê-lo no Rails Summit Latin America, que acontece nos dias 13 e 14 de Outubro. Estamos chegando perto. Você já se inscreveu?

AkitaOnRails: Como você encontrou Ruby/Rails, o que foi que mais o atraiu?

Conheci o Rails ainda na versão 0.13. Na época, não tinha quase nenhum blog em português sobre o tema e decidi criar o meu! Tinha visto algum artigo falando sobre ele e sobre como era fácil criar aplicativos e fui conferir. Por coincidência, estava querendo conhecer uma nova linguagem e quando vi a sintaxe do Ruby, decidi que era ela que eu iria aprender! E nessa estou até hoje!

AkitaOnRails: O que você tem estudado ou desenvolvido recentemente, seja em projetos particulares ou open source?

Tenho visto bastante coisa sobre o Ruby 1.9, principalmente para poder adequar meus projetos a esta versão. Além disso, tenho me interessado bastante sobre testes em geral, principalmente Javascript (JSpec e Screw Unit) e Aceitação (Cucumber).

AkitaOnRails: Sobre o que trata a sua palestra e para quais tipos de desenvolvedores é mais indicado?

Irei abordar as mudanças do Ruby 1.9 em relação à versão 1.8, com exemplos mostrando as novas funcionalidades e como corrigir comportamentos que deixaram de funcionar. É um assunto indicado para todos os programadores, pois cedo ou tarde, esta será a versão recomendada!

Speaker Nando Vieira

Nando Vieira is one of the luminaries of the Brazilian Ruby Community. He blogs at Simples Idéias, one of the most popular blogs about Ruby on Rails in Brazil. He currently works as web developer for April Digital, using Rails. He’s also the founder of Spesa, and maintains dozens of Open-source projects.

He recently released an e-book of his HOWTO series titled What has changed in Ruby 1.9. In the same series you can download the free e-Book Rails Application Templates which he also wrote.

Come meet him at Rails Summit Latin America, on October 13th and 14th. We’re getting close. Have you registered already?

AkitaOnRails: How did you find Ruby/Rails, what attracted you the most?

I found Rails when it was still in version 0.13. At the time, there was nearly zero blogs in Portuguese about the subject and I decided to create my own! I had seen an article talking about how easy it was to create applications with it and I decided to give it a shot. Coincidentally, I was thinking of learning a new language and when I saw the syntax of Ruby, it was decided that I would learn it! And I continued up to this day.

AkitaOnRails: What have you studied or developed recently, for private projects or open source?

I’ve seen quite a lot about Ruby 1.9, mainly in order to tailor my projects to this version. Also, I have been concerned enough about testing in general, especially Javascript (JSpec and Screw Unit) and Acceptance Testing (Cucumber).

AkitaOnRails: What is your talk about and what kinds of developers are going to benefit the most?

I will address the changes in Ruby 1.9 from version 1.8, with examples showcasing the new features and how to fix the behaviors that no longer work. It is a subject recommended for all developers, because sooner or later, this will become the mainstream version!

[Rails Summit 2009] Palestrante Ilya Grigorik

Sun, 27 Sep 2009 19:18:40 PDT

English translation: click here

Ilya Grigorik é um desenvolvedor Ruby bem conhecido, especialmente por causa do seu blog Igvita.com onde encontramos alguns dos artigos tecnicamente mais ricos e ainda assim de fácil compreensão, não só em torno de aplicações Rails, mas tudo que se relaciona com ele, incluindo drivers de banco de dados, análise de desempenho, integração de serviços e muito mais. Por causa disso ele foi premiado como Ruby Hero na RailsConf 2008 em Portland.

Ele também é o fundador do PostRank, um sistema de monitoramento em tempo real de engajamento social de mídia e plataforma de análise. Ele acabou de lançar um outro produto complementar chamado Analytics, outra ótima maneira de controlar os seus relacionamentos.

Você terá uma grande oportunidade de conhecê-lo no Rails Summit Latin America este ano. Não esqueça de se inscrever logo! Estamos apenas a mais alguns dias de distância!

AkitaOnRails: O que você mais gosta sobre a linguagem Ruby, o recurso ou conceito que “clica” para você?

Eu não consigo escolher apenas um atributo, pois como a maioria das coisas, geralmente é a soma das partes que compõe a experiência. E, na verdade, é o que eu mais gosto sobre Ruby: todas as peças “clicam” juntas. As decisões de design por trás da linguagem Ruby dão uma natureza incrivelmente maleável: é fácil estendê-lo, é fácil modificá-lo, é fácil fazê-lo funcionar da maneira que você precisa para trabalhar na sua aplicação. Rails é o principal exemplo disso, mas as centenas de outras estruturas construídas em cima de Ruby são todos exemplos dessa propriedade.

AkitaOnRails: Como um desenvolvedor Rails, o que é a coisa que você realmente gosta quando está codificando aplicações web?

A capacidade de ir do nada para um protótipo totalmente operacional, em uma xícara de café. Se você está olhando para uma aplicação Rails, ou uma rápida em Sinatra / Rails Metal, a enorme quantidade de recursos e plugins torna o processo divertido e imediatamente gratificante.

AkitaOnRails: Qual é a sua mais recente pesquisa ou interesse em desenvolvimento?

Muito do meu tempo é dedicado à investigação e criação de protótipos de instrumentos e abordagens de arquiteturas de software distribuído – desde mensagens, para balanceamento de carga, a bancos de dados. Combinado com o fato de que a maioria dos serviços que implantamos no PostRank são escritos em Ruby, significa também se aprofundar mais na compreensão das partes internas do Ruby, tentando otimizar a VM para uma arquitetura distribuída.

AkitaOnRails: Sobre o que é a sua palestra para o Rails Summit e quem são os tipos de desenvolvedores que se beneficiarão mais?

Nesta palestra vou cobrir as tecnologias emergentes por trás da “Real-Time Web” (XMPP, AMQP, PSHB, Webhooks). A adoção do Rails ao modelo RESTful liberou os dados dos aplicativos e nos permitiu definir serviços que poderiam facilmente falar uns com os outros, a Real-Time Web está indo um passo além: em vez de diferentes web-services se consultando o tempo todo, conectá-los através de interfaces push. Esta palestra terá início a partir do início e fornecerá uma visão geral de alto nível das tecnologias subjacentes. Saindo dela, você deve ser capaz de diferenciar e decidir sobre qual tecnologia se adaptará melhor às suas necessidades (não são todos iguais!).

Speaker Ilya Grigorik

Ilya Grigorik is a very well known Ruby developer, specially from his blog Igvita.com where one can find some of the most technically rich yet easily understandable articles surrounding not only Rails apps but everything that goes with it, including database drivers, performance analysis, services integration and much more. Because of that he was awarded Ruby Hero at RailsConf 2008 in Portland.

He is also the founder of PostRank, a real-time social media engagement monitoring and analytics platform. He just released another companion product called Analytics, another great way to keep track of your online engagements.

You will have a great opportunity to get to know him at Rails Summit Latin America this year. Don’t forget to Register! We’re just a few more days away!

AkitaOnRails: What do you like the most about the Ruby language, what feature or concept really “clicks” for you?

I can never pick any one attribute because as with most things, it is usually the sum of parts that makes up the experience. And in fact, that’s what I enjoy most about Ruby: all the parts click together. The design decisions behind the language give Ruby an incredibly malleable nature: it is easy to extend it, it is easy to modify it, it is easy to make it work in just the way you need it to work for your application. Rails is the premier example of this, but the hundreds of other frameworks built on top of Ruby are all examples of this property.

AkitaOnRails: As a Rails developer, what is the thing you really enjoy when you’re coding web apps?

The ability to go from nothing to a fully operational prototype with one cup of coffee. Whether you’re looking at a fully fledged Rails application, or a quick Sinatra / Rails Metal app, the sheer amount of resources, and plugins makes the process both fun and immediately rewarding.

AkitaOnRails: What are your most recent research or development interests?

A lot of my time is always dedicated to research and prototyping of tools and approaches for distributed software architectures – anything from messaging, to load balancing, to databases. Combined with the fact that most of the services we deploy at PostRank are written in Ruby, it also means delving deeper into understanding the internals of Ruby, and trying to optimize the VM for the distributed architecture.

AkitaOnRails: What is your Rails Summit talk about and who are the kind of developers that will benefit the most?

In this talk I will cover the emerging technologies behind the ’Real-Time Web" (XMPP, AMQP, PSHB, Webhooks). Rails adoption of the RESTful model liberated the data from the app and allowed us to define services that could easily talk to each other, the Real-Time Web is looking to go one step beyond that: instead of polling different web-services, it aims to connect them via push interfaces. This talk will start from the beginning and will provide a high-level overview of the underlying technologies. Coming out of it, you should be able to differentiate and decide on which technology suits your needs best (they are not all the same!).

[Rails Summit 2009] Palestrante Matt Aimonetti

Wed, 23 Sep 2009 17:13:12 PDT

English translation: click here

Matt Aimonetti é um ativista Rails, ele começou como evangelista de Merb e se envolveu na já famosa junção do Rails + Merb. Ele também é ativo na área de MacRuby e HotCocoa. Ele mora em San Diego, onde dirige sua própria consultoria, fazendo projetos e treinamentos relacionados com Ruby, e ele também é um palestrante conhecido em eventos. Nós nos encontramos pela primeira vez no ano passado, na QCon de São Francisco. Você pode ouvir a entrevista que gravamos naquela época aqui

Ele estará conosco no Rails Summit Latin America 2009. Não esqueça de se inscrever!

AkitaOnRails: O que você mais gosta na linguagem Ruby, qual recurso ou conceito te atrai?

Acho que o Ruby é a melhor linguagem de programação para expressar o que eu quero no meu código. É tão natural que parece mesmo “certo”. As pessoas são, como o falecido Guy Decoux era, capaz de se comunicar apenas em código Ruby e isso significa muito para mim. Mas há mais do que a fantástica sintaxe. Ruby tem uma maravilhosa comunidade e valores fortes, tais como testes, refatoramento, melhores práticas, open source etc.

Eu estou realmente empolgado vendo empresas como a Apple e a Microsoft apostando fortemente na linguagem Ruby e sua comunidade.

AkitaOnRails: Qual é a coisa que você mais gosta sobre Rails quando codifica aplicações web?

O que eu mais gosto quando escrevo uma aplicação Rails está em usar Ruby. Eu tentei usar outros clones de Rails em outras línguas e nenhuma chegou perto da mesma experiência, devido à linguagem de programação subjacente. Mas além do Ruby, minha segunda coisa favorita sobre o Rails é a seqüência de uso de convenções inteligentes. Eu posso ver qualquer projeto Rails e compreender imediatamente o que está acontecendo.

AkitaOnRails: Qual é o seu interesse mais recente de pesquisa ou desenvolvimento?

Eu aprendi muito sobre Ruby, trabalhando em MacRuby. Como se vê, há um monte de coisas no Ruby e suas bibliotecas-padrão que muito raramente uso. Eu também estou experimentando com pequenos jogos 2D escrito em Ruby. É uma mudança agradável de desenvolvimento web. Finalmente, eu ainda estou trabalhando com CouchDB e armazenamento de dados alternativos.

AkitaOnRails: Sobre o que é sua palestra para o Rails Summit e quem são os tipos de desenvolvedores que irão se beneficiar mais?

Minha palestra é sobre o futuro do Ruby on Rails e concretamente o que isso significa para os desenvolvedores. Não será uma palestra sobre o encanamento interno, mas mais sobre uma perspectiva de alto nível com exemplos de nível mais elevado de código e exemplos concretos de coisas novas impressionantes que estarão disponíveis para os desenvolvedores Ruby.

As pessoas que estão interessadas no que está chegando, os céticos e pessoas considerando usar Ruby/Rails pela primeira vez provavelmente acharão isso interessante.

Além disso, se você gosta de “futebol”, deveria vir assistir minha palestra ;)

Speaker Matt Aimonetti

Matt Aimonetti is a Rails Activist, he started as a Merb evangelist and was involved in the now famous Rails + Merb merge. He is also active in the MacRuby arena with HotCocoa. He lives in San Diego where he runs his own consultancy, doing Ruby related projects and training, and he is also a well known speaker in events. We met for the first time at last year’s San Francisco QCon. You can listen to an interview we recorded back then here

He will be with us at Rails Summit Latin America 2009. Don’t forget to register soon and make reservations for hotel and city tour (it will be a great opportunity to also get to know São Paulo City).

AkitaOnRails: What do you like the most about the Ruby language, what feature or concept really “clicks” for you?

I found Ruby to be the best programming language to express what I want my code to do. It is so natural that it feels “right”. People are, like the late Guy Decoux was, able communicate only in Ruby code and that means a lot to me. But there is more to it than the awesome syntax. Ruby has an awesome community and strong values such as testing, refactoring, best practices, OSS etc..

I’m really psyched to see companies like Apple and Microsoft bet heavily on the Ruby language and community.

AkitaOnRails: What is the thing you really enjoy about Rails when you’re coding web apps?

What I enjoy the most when writing a Rails app is using Ruby. I tried using other Rails clones in other languages and none came close to the user experience due to the underlying programming language. But apart from Ruby, my second favorite thing about Rails is the string use of smart conventions. I can take almost any Rails projects and understand right away what’s going on.

AkitaOnRails: What are your most recent research or development interests?

I learned so much about Ruby by working on MacRuby. As it turns out, there are a lot of goodies in Ruby and its standard libraries that we very rarely use. I’m also experimenting with small 2D video games written in Ruby. It is a nice change from web development. Finally, I’m still working with CouchDB and alternative data storage.

AkitaOnRails: What is your Rails Summit talk about and who are the kind of developers that will benefit the most?

My talk is about the future of Ruby and Rails and what it concretely means for developers. It’s not going to be a talk about the inner plumbing but more about the big picture with higher level code examples and concrete examples of awesome new things that are going to be available to Ruby developers.

People who are interested in what’s coming up, skeptics and people considering using Ruby/Rails for the first time will hopefully find the talk interesting.

Also, if you like “futebol” you should definitely come check out my talk ;)

[Rails Summit 2009] Palestrante Carlos Brando

Mon, 21 Sep 2009 06:18:29 PDT

English translation: click here

O Carlos Brando acho que dispensa apresentações :-) Todos já o conhecem pelo blog Nome do Jogo, pelo e-book Rails 2.1 e por projetos open-source como o Remarkable. Dentre suas colaborações mais recentes, está a tradução do excelente livro do _why, o Poignant Guide to Ruby; e o lançamento da versão nacional do site britânico Ruby Inside

Na sua própria descrição, ele é viciado em programação e atualmente é diretor de tecnologia da Amanaiê, uma empresa que cria, produz e distribui aplicativos sociais. Atua na área de desenvolvimento de softwares há mais de doze anos, tendo trabalhado com Visual Basic, C, C++, Java, C# e agora Ruby. Já trabalhou em várias companhias brasileiras e estrangeiras como Portugal Telecom Group, AT&T Latin America, DirecTV, SKY, VISA e Surgeworks Inc.

Já se apresentou em vários eventos brasileiros relacionados com Ruby, incluindo a edição do ano passado do Rails Summit Latin America.

Além de suas atividades profissionais, Carlos e sua esposa trabalham como voluntários dando aulas para deficientes auditivos. Ele regularmente ensina língua de sinais e atua como interprete.

AkitaOnRails: Como você encontrou Ruby/Rails, o que foi que mais te atraiu?

Eu já trabalhava há 12 anos como programador e pelo menos 7 principalmente com tecnologia Microsoft e estava um pouco cansado de fazer as mesmas coisas todos os dias, literalmente tinha perdido o prazer no meu trabalho. Foi quando ouvi falar do Rails pela primeira vez, mas sem entender do que se tratava, simplesmente o ignorei. Alguns meses depois li um artigo no blog do Chad Fowler onde ele mencionava o Ruby e como estava feliz em trabalhar com esta linguagem e isto me fez voltar a olhar para o Rails.

Comecei estudando por conta própria e comprei o livro “Repensando a web com Rails” do Fábio Akita e o The Ruby Way de Hal Fulton. Em duas semanas eu já tinha montado meu primeiro site em Rails, o extinto CarreiraTI, um site de empregos para profissionais da área de tecnologia. Seis meses depois fui convidado para fazer parte da equipe da Surgeworks, onde trabalhei por quase 2 anos.

AkitaOnRails: Quais assuntos mais te interessam atualmente e por quê?

Tenho me interessado muito pelas entranhas da linguagem, entender como o Ruby funciona por dentro. Aos poucos estou deixando o Rails cada vez mais de lado, e me envolvendo mais com o Ruby.

AkitaOnRails: Sobre o que trata sua palestra e para que tipos de desenvolvedores é mais indicado?

Atualmente fui contratado por uma startup brasileira chamada Amanaiê, que desenvolve aplicativos para redes sociais. Depois de passar um tempo fazendo software para estas plataformas, estou trabalhando especificamente no desenvolvimento de um framework para acelerar a criação de projetos deste tipo. Durante muito tempo acompanhei de perto o desenvolvimento do Rails, o que me rendeu dois livros. Este conhecimento é o que tem me ajudado a repensar algumas coisas e criar um novo framework baseado no Ruby.

Minha palestra no Rails Summit tratará exatamente deste tema: o processo de construção de um framework utilizando Ruby. Se você realmente quer entender como o Rails funciona, então crie o seu próprio framework. Nesta palestra vou comentar sobre algumas das dificuldades que encontrei durante este projeto e quais as soluções encontradas. O processo de renderização, geradores de códigos, helpers e banco de dados estão entre os assuntos que serão tratados. Acredito que ao analisar este tópicos de um ponto de vista mais simplista fará com que você entenda melhor como algumas coisas funcionam internamente no Rails.

Speaker Carlos Brando

I think Carlos Brando needs no introduction :-) Everyone knows his blog Nome do Jogo, the e-book Rails 2.1, his open-source projects such as Remarkable. Among his most recent collaborations, there’s the translation of the excellent book from _why, the Poignant Guide to Ruby; and the launch of the Brazilian version of the British website Ruby Inside.

In his own words, he is addicted to programming and is currently chief technology officer for Amanaiê, a Brazilian company that creates, produces and distributes social applications. He is in the software development field for over twelve years, having worked with Visual Basic, C, C++, Java, C# and now Ruby. He has worked in several Brazilian and foreign companies like Portugal Telecom Group, AT&T Latin America, DirecTV, SKY, VISA and Surgeworks Inc.

He has performed in several events related to Ruby in Brazil, including last year’s edition of Rails Summit Latin America.

In addition to his professional activities, Carlos and his wife volunteer teaching to deaf people. He regularly teaches sign language and acts as interpreter.

AkitaOnRails: How did you find Ruby / Rails, what most attracted you?

I’ve been working 12 years as a programmer and at least 7 of them primarily with Microsoft technologies; and I was a bit tired of doing the same things over and over. I’ve literally lost the pleasure in my work. When I heard about Rails for the first time – without understanding what it was – I simply ignored it. A few months later I read an article in Chad Fowler’s blog where he mentioned how happy he was to work with Ruby, and this made me take a second look at Rails.

I started studying on my own and bought “Rethinking the web with Rails” book by Fabio Akita and The Ruby Way by Hal Fulton. In two weeks I had set up my first site in Rails, the now extinct CarreiraTI, a job site for professionals in technology. Six months later I was invited to join the Surgeworks team, where I worked for almost 2 years.

AkitaOnRails: What subjects interest you the most nowadays and why?

I’ve been very interested in the inner plumbings of the language, to understand how Ruby works from the inside. I am gradually leaving Rails aside, and getting more involved with Ruby.

AkitaOnRails: What is your talk at Rails Summit about, and what kind of developers will benefit the most?

I am currently hired by a Brazilian startup called Amanaiê, which develops social network applications. After spending time making software for these platforms, I’m working specifically on developing a framework to speed up the creation of such projects. For a long time I closely followed the development of Rails, which resulted in two books. This knowledge is what helped me to rethink some things and create a new framework based on Ruby.

My talk at Rails Summit will address exactly this issue: the process of building a framework using Ruby. If you really want to understand how Rails works, then create your own framework. In this talk I comment on some of the difficulties I found during this project and what were solutions I used. The process of rendering, code generators, helpers and database are among the subjects to be treated. I believe that in examining this topic from a more simplistic standpoint will make you better understand how things work internally in Rails.

Rails Summit will have Portuguese to English translator, so you can watch Carlos talk too! Register now!

[Rails Summit 2009] Palestrante Bryan Liles

Fri, 18 Sep 2009 09:29:47 PDT

English translation: click here

Bryan Liles ficou bastante conhecido depois do Ruby Hoedown de 2008 com sua palestra Test All the Fucking Time cunhando o termo “TATFT”.

RailsConf 2009 – Mensagem de Bryan Liles para Brasileiros from Fabio Akita on Vimeo.

Ele virá falar sobre boas práticas de programação, baseado numa séria que começou recentemente no seu blog Smartic.us, chamado #yerdoinitwrong (literalmente, “voceestafazendoerrado”). Veja a apresentação da série:

#yerdoinitwrong intro from Bryan Liles on Vimeo.

O primeiro episódio fala sobre enviar os logs do Rails para um servidor remoto usando syslog, e recomenda o excelente Splunk, que eu também gosto. Não perca os episódios.

Não deixe de trocar figurinhas com ele durante o Rails Summit! Inscreva-se já!

Speaker Bryan Liles

Bryan Liles became very well known after his talk Test All the Fucking Time at Ruby Hoedown 2008, coining the term “TATFT”.

RailsConf 2009 – Bryan Liles message to Brazilians from Fabio Akita on Vimeo.

He’ll come to talk about best practices in software development, based on the series he recentely started at his blog Smartic.us called #yerdoinitwrong. Watch his introduction to the series:

#yerdoinitwrong intro from Bryan Liles on Vimeo.

The first episode talks about sending your Rails logs to a remote server using syslog, and he even recommends the excellent Splunk, which I like as well. Don’t miss his episodes.

And come to the Rails Summit to talk and exchange information with him! Register now!

[Rails Summit 2009] Palestrante Leonardo Borges

Thu, 17 Sep 2009 15:51:48 PDT

English translation: click here

A partir de hoje vou colocar alguns posts sobre alguns dos palestrantes do Rails Summit Latin America 2009. Não deixe de se inscrever! Dias 13 e 14 de Outubro, em São Paulo.

Leonardo Borges é desenvolvedor com 9 anos de experiência, já trabalhou em empresas de portes variados criando aplicações corporativas especialmente em Java. Atualmente vive em Madrid e trabalha com Ruby On Rails na Mirai España onde, dentre outros projetos, participa da iniciativa JRuby de integrar código legado com Rails. Escreve no seu blog sobre desenvolvimento de software e temas relacionados.

AkitaOnRails: Como você encontrou Ruby/Rails, o que foi que mais te atraiu?

Foi no início de 2007. Lendo blogs e notícias na internet cheguei no site oficial do Rails e vi o famoso vídeo Creating a weblog in 15 minutes. Fiquei impressionado! Aquele vídeo foi capaz de vender nao só a produtividade que o Rails proporciona, mas o nível de expressividade ao qual podemos chegar em Ruby.

AkitaOnRails: Quais assuntos mais te interessa atualmente e por quê?

Devido à natureza dos projetos aqui na empresa – um mix de Ruby e Java – tenho me interessado bastante em JRuby e a JVM de uma maneira geral, estudando também alternativas como Scala.

AkitaOnRails: Sobre o que trata sua palestra e para que tipos de desenvolvedores é mais indicado?

Eu acredito que JRuby veio pra ficar e que uma de suas maiores forças está em atrair empresas/desenvolvedores já acostumados ao mundo Java. Para isso uma integração precisa e simples entre Ruby e Java deve ser possível.

Na minha palestra “JRuby no mundo corporativo” vou mostrar como essa integração pode ser feita para dar uma nova roupagem a um sistema legado usando JRuby on Rails.

Essa sessão é para qualquer desenvolvedor Ruby interessado em rodar suas applicaçoes na JVM. No entanto, se você está trabalhando com Java no momento e está pensando como pode introduzir Ruby na sua empresa, essa palestra pode ser particularmente útil.

AVISO: Você verá fontes Java e XML nessa sessão! :P

Speaker Leonardo Borges

Starting off today, I’ll post about some of the speakers for Rails Summit Latin America 2009. Register today! It’s gonna be at São Paulo, Brazil, October 13th and 14th.

Leonardo Borges is a developer with 9 years of experience. He’ve been working for several companies creating enterprise applications specially with Java. He currently lives in Madrid, where he works with Ruby on Rails at Mirai España where, among other projects, colaborates in the JRuby initiative to integrate legacy code with Rails. He writes in his blog about software development and related subjects.

AkitaOnRails: How did you find Ruby/Rails, what attracted you the most?

It was in the beginning of 2007. By reading blogs and news on the internet I arrived at the Rails official website where I saw the famous Creating a weblog in 15 minutes screencast. I was impressed! That video was able to sell not only the productivity that Rails delivers, but also the level of expressiveness where we can achieve in Ruby.

AkitaOnRails: What subjects most interest you and why?

Due to the nature of the projects I have in my company – a mixture of Ruby and Java – I’ve very interested in JRuby and the JVM in general, studying alternatives such as Scala as well.

AkitaOnRails: What is the subject of your talk and what kind of developers will benefit the most?

I believe that JRuby will stay for a long time and one of its major strengths is to attract companies/developers already used to the Java world. For that end, a precise and simple integration between Ruby and Java is necessary.

In my talk “JRuby in the enterprise” I’ll show how that integration can be achieved to bring new life to a legacy system using JRuby on Rails.

This session is recommended to any Ruby developer interested in running his applications on the JVM. On the other hand, if you’re currently working with Java and trying to figure out how to introduce Ruby in your company, this talk can be particularly useful.

Warking: You’ll see lots of Java and XML source code in this talk! :P

Playing with Dry Scaffold and Inherited Resources

Wed, 09 Sep 2009 11:40:37 PDT

There are several ways to start a new Rails project nowadays, specially since Rails brought project templates to the mix. On the other hand there are still lot’s of generators. So, today I want to just pinpoint some tips on this regard.

First of all, make sure you have all the usual suspects installed:

1	sudo gem install rspec rspec-rails josevalim-inherited_resources mislav-will_paginate justinfrench-formtastic thoughtbot-factory_girl haml ryanb-nifty_generators

One other is dry_scaffold from grimen. I’ve made a fork and added rspec support today. I think the 0.3.3 version should be generated soon from Github, then you will be able to install it like this:

1	sudo gem install akitaonrails-dry_scaffold

But you can also install it manually from the source:

1234	git clone git://github.com/akitaonrails/dry_scaffold.gitcd dry_scaffoldgem build dry_scaffold.gemspecsudo gem install dry_scaffold-0.3.3.gem

Now, create a normal Rails 2.3.x project and configure your config/environment.rb with:

config.gem 'haml'
config.gem 'will_paginate'
config.gem 'justinfrench-formtastic', :lib => 'formtastic', 
  :source => 'http://gems.github.com'
config.gem 'josevalim-inherited_resources', :lib => 'inherited_resources', 
  :source => 'http://gems.github.com'

And also add the following to your config/environments/test.rb:

12	config.gem 'rspec'config.gem 'rspec-rails'

In order to add Rspec support, create the initial Formtastic stylesheets and add a little bit of sugar to your layouts do the following:

123	./script/generate rspec./script/gererate formtastic_stylesheets./script/generate nifty_layout --haml

Notice that I am referring to Ryan Bates’ Nifty Generators. I really like it’s initial CSS and layout to start a new project instead of the default ‘blank’ theme. But that’s just me :-)

The excellent Formtastic will create your scaffolded views with a bit more of web semantics, so it’s highly recommended to use. To make it look pretty, add the following to your ‘app/views/layouts/application.html.haml’:

123	= stylesheet_link_tag 'application' = stylesheet_link_tag 'formtastic' = stylesheet_link_tag 'formtastic_changes'

Finally, you can start scaffolding some resources to get started, for example:

1	./script/generate dry_scaffold Post title:string body:text --rspec

If you don’t like fixtures you’d rather use Factory Girl instead doing this:

1	./script/generate dry_scaffold Post title:string body:text --rspec --fgirl

By default, it’s going to create thin controller with Inherited Resources and paginated with Will Paginate. The view templates will use HAML and Formtastic for a semantically rich form. It will create tests using the standard test/unit and fixtures, but I’d rather choose Rspec and Factory Girl.

It’s also going to generate a separated “_form” partial that both “new” and “edit” views will use, which is one of those things that you do all the time after a normal scaffold. The usage of HAML is not very common, but I would recommend everybody to at least try it for a while, it will grow on you quickly. And having well formatted HTML outputs is always nice.

The Dry Scaffold also supports generating just models (“./script/generate dry_model”). Read the documentation on the github project site to understand all the options.

With this, you should be good to go and start to rapidly prototype your next web application.

Default Scopes and Hash Posers

Mon, 07 Sep 2009 22:07:58 PDT

While I was editing my last article I was thinking about multi-site. Usually, Railers prefer to have a single app per Rails project. But what if I do want to have a CMS, an e-Commerce, something like Wordpress MU and so on?

I thought to myself: “dead simple: just add default_scope to the relevant Active Record models”. Ended up not being quite so simple because it seems like ‘default_scope’ doesn’t support blocks just yet as you can follow on this open ticket at Lighthouse.

The problem is that ‘default_scope’ will be different per user, so I can’t have an static conditions hash pre-defined in the model. To solve that – at least until a more official solution comes along – Brian Mitchell has a great suggestion: use a Hash Poser.

Now, a Hash Poser is a class that acts and behaves exactly like a normal Hash. Let’s see his code:

Dead Simple, you can copy and paste this code to somewhere like ‘config/initializers/hash_poser.rb’. You can easily see how this can be used in your models:

class Post < ActiveRecord::Base
  default_scope HashPoser.new do
    { :conditions => { :user_id => UserSession.find } } 
  end
end

This is supposing you’re using something like Authlogic, but actually you will want something more like ‘Site.find’ or whatever. The point is, now whatever finder you were using throughout your application will have this conditions added to it. So, a simple ‘Post.all’, in this example, will automagically run:

1	SELECT * FROM "users" WHERE ("users"."id" = 1)

Considering that you’re not using something like ‘find_by_sql’ with hand crafted SQL queries, the ‘default_scope’ will override all your finders. Of course, it’s easier said than done, so have your test suite ready when you do this change. As I said, you will have to add a column like ‘site_id’ to every table you need to be multi-site.

I would like to see what other people think of this way of thinking and what caveats there are for this use case. Most of the time Rails apps will be stand-alone, but in a few exception cases, we really do want to have a single app serving multiple customers. What’s your take on this concept?

User Editable Liquid Templates in the Database

Mon, 07 Sep 2009 20:49:52 PDT

For quite some time I wanted to create an experiment: having user-editable view templates stored in the database instead of the filesystem.

The Use Case is quite simple: it’s either a theme system or a website where each user can edit his own set of templates to have a customized usage of the application. In the case of a simple theming system, just adding additional view load paths would be enough. But the second case needs scalability and having a potential set of thousands of view sub-folders around never felt well.

One solution is to have every view in a database table. If performance becomes critical, we can always add Memcached to the equation, so it shouldn’t be a problem. Now, there is another problem: we can’t just store the ERB view files directly into the database and make them user-editable. That’s because ERB executes arbitrary Ruby code, which means that the user would have access to the entire machine. Haml and other templating engines have the same feature. One of the few that implements a user-centric, restrictive templating engine is Tobias Lütke’s Liquid, created precisely for the same purpose for his Shopify.com website.

Having that in mind, I just published the dynamic_liquid_templates plugin as an attempt to have just that. In order to get started, you can create a new Rails project and add it as a normal plugin:

1234	rails democd demo./script/generate dynamic_templatesrake db:migrate

You will have to require Liquid from config/environment.rb

1	config.gem "tobi-liquid", :lib => "liquid", :source => "http://gems.github.com"

And, of course, sudo rake gems:install to have it installed if you still don’t have it. Now, create a normal Rails resource, such as ‘posts’ or whatever. You will have to modify your scaffolded controller so the ‘format.html’ calls look like this:

123	respond_to do \|format\| format.html { render_with_dynamic_liquid }end

If you’re inside a nested controller (such as ‘Comment belongs_to Post’), declare a method named ‘parent’, for example:

class CommentsController < ApplicationController
  ...
  def parent
    @post ||= Post.find(params[:post_id])
  end
end

The plugin will use this method to refer to the proper named routes, such as _post_comments_path(parent)_ or _post_comment_path(parent, @comment)_ and so on. Namespaces will work too, for example:

class Admin::PostsController < ApplicationController
  def index
     @posts = Post.all
     respond_to do |format|
       format.html { render_with_dynamic_liquid(:namespace => 'admin') } 
     end
end

There’s probably a better syntax for that, but it works like that too. The other thing is that your models need to be ‘liquified’ in order to be usable from inside a Liquid template. You have to declare a _to_liquid_ method explicitly saying which columns you want exposed:

class Post < ActiveRecord::Base
  ...
  def to_liquid
    { 'title' => self.title, 'body' => self.body }
  end
end

Notice that the hash keys have to be strings, and not symbols. The other thing is that the plugin will append other attributes to this hash automagically, so you can do the following from within your Liquid templates:

1	{{ 'Show' \| link_to: post.show_path }}

It adds ‘show_path’ and ‘edit_path’ to your model instances, so you can use them to create links or form actions. There are other globally assigned variables as well, such as:

collection – refers to the controller collection (for index action) such as @posts
object – referes to the controller single object (for non-index actions) such as @post
collection_path – the named route for the index action, including nested and namespaced versions
object_path – the named route for the non-index action, including nested and namespaced versions
parent – if you have the ‘parent’ method in your controller, it’s exposed within Liquid
parent_path – if you have the ‘parent’ method in your controller, it’s used for named routes

And, of course, your models are properly assigned to Liquid too so, for example, ‘@posts’ is exposed as ‘posts’ to Liquid.

Next Steps

Now, this is just the beginning. You can create a scaffold for the DynamicTemplate model, so you have an editing interface. You can refer to the ‘spec/fixtures’ folder within the plugin for examples on how a Liquid template looks like (it’s less convenient than ERB, I can tell).

Then you can add multi-site support to your website by adding a foreign-key to all your models, a new table such as ‘Site’ and a ‘site_id’ column in all your other models. Then, each Active Record model can have the ‘default_scope’ declared, such as:

1234	class Post < ActiveRecord::Base default_scope proc { { :conditions => ["site_id", Site.current_site] } } ...end

But, this is not quite possible today because ‘default_scope’ seems to not be accepting a proc just yet. Let’s hope this feature comes sooner, or we can just create yet another plugin anyway, so no big deal.

Another thing, we can still improve it’s performance by serializing the pre-compiled template instead of the raw text version as explained in this article. I will do it this week, probably.

Then, we need to add caching support. As I am using plain Active Record, simple config.cachestore_ configuration should do. And you will also want to add other features such as versioning the dynamic_templates table so the user can change his mind later on. For this one, it’s as easy as watching Ryan Bates’ recent vestal_versions screencast.

This is just an experiment, I hope it can be useful to someone.

[Screencast] Real Thin Restful Controllers with Inherited Resources

Tue, 01 Sep 2009 04:24:14 PDT

Update 10/01: José just released version 0.9.1 while I was uploading the video :-)

José Valim is doing a great work in the soon to the released Rails 3.0. He surprised everybody with his Google Summer of Code project, rewriting the entire Generator code to use Thor and be more agnostic. Not only that he enhanced the ActionController with ways to make it leaner and cleaner.

Real Thin Rails Restul Controllers with Inherited Resources from Fabio Akita on Vimeo.

Much of this later work comes from an old Rails gem of his called Inherited Resources. It can be used right now in your Rails 2.3 projects and he just backported some of the Rails 3.0 features such as ActionController Responders and other goodies.

In essence it feels a lot like James Golick resource_controller or Hampton Caitlin’s make_resourceful, Yet Another Thin Controller Plugin if you like. But José’s is definitely interesting.

So, I decided to make a screencast showcasing some of its basic features. Bear in mind that this gem offers a lot more and I highly encourage you to read his Wiki page about it. There are dozens of options that you can leverage to make your projects even more maintainable and lean.

Para utilizar os códigos que eu menciono no screencast, apenas comece assim:

12	git clone git://github.com/akitaonrails/ryanb-15min-blog.gitcd ryanb-15min-blog

This video is divided in two parts, the first is a simple conversion to use inherited_resources. To access this code do like this:

1	git checkout -b inherited_resources --track origin/inherited_resources

This will bring the branch with the first part. To see the second part, with more goodies, do like this:

1	git checkout -b inherited_resources_complete --track origin/inherited_resources_complete

Now you should have 3 branches: master with the original 15 min. blog, ‘inherited_resources’ with the first part and ‘inherited_resources_complete’ with the second part. Use the ‘git checkout [name of the branch]’ to switch back and forth between branches.

José is also a speaker for Rails Summit Latin America 2009, the largest Rails conference in South America, and it’s gonna happen on October 13th and 14th, at São Paulo, Brazil. Help us promote it and be sure to attend! :-)

Rails Summit Latin America 2009

Wed, 29 Jul 2009 23:59:56 PDT

Brasileiros: Cliquem aqui

Don’t miss this year’s Rails Summit Latin America. This is a huge opportunity for Railers throughout of South America to gather and meet. It will take place in Sao Paulo, Brazil once again bringing together a great roster of known Railers such as Matt Aimonetti, Rich Kilmer, Chad Fowler and many others for 2 full days, with 2 parallels tracks of Ruby and Rails goodness.

Rails Summit 2008 was a huge success with a great positive repercussion on the community. We do intend to even surpass the level of quality of last year’s event.

This year, we’ll have a larger area, with 2 rooms, both with Portuguese-English and English-Portuguese real-time translators so everybody can enjoy every single talk. We will have a great lobby for social networking complete with wi-fi network and power plugs so you can code during the event and share code and techniques with your new developer mates.

Help us out promote the event by wearing our banners in your website as well.

Rails Summit Latin America

Não percam o Rails Summit Latin America. Esta é uma enorme oportunidade para os Railers se encontrarem e se conhecerem. Acontecerá novamente em São Paulo trazendo uma lista estelar de Railers conhecidos como Matt Aimonetti, Rich Kilmer, Chad Fowler e muitos outros para 2 dias inteiros com 2 sessões paralelas de palestras de Ruby e Rails.

O Rails Summit 2008 foi um enorme sucesso com uma grande repercussão positiva pela comunidade. E nós pretendemos superar o nível de qualidade do ano passado.

Este ano, teremos uma área maior, com 2 salas, ambas com tradutores em tempo real tanto de inglês-português e português-inglês para que todos possam aproveitar todas as palestras. Teremos um grande lobby para networking, completo com rede wi-fi e tomadas para que você possa codificar durante o evento e dividir código e técnicas com seus novos colegas desenvolvedores.

E nos ajude a promover o evento vestindo nossos banners em seu site também.

Registre-se agora!, as vagas são limitadas! Estaremos esperando por vocês nos dias 13 e 14 de Outubro!

Rails developers in Japan

Tue, 14 Jul 2009 23:19:24 PDT

これは僕のはじめの日本語で核ブログポスト。減った糞な日本語で書くのもすみません、でも今日は@satokoさんていうプログラマーと話して何となく日本語で書ムードになりました。僕はブラジル生まれなので、子供のころからアニメや漫画読みながら少しだけ日本語学んだ、でも経験不足です。

残園ながらまだ日本まで旅行した事がまだない。僕はブラジル生まれで Ruby on Railsの初めてのエバンジェリストです。２００６年に初めてのRails本がポルトガル語で書きました。ルビーも日本生まれって事は面白い、マッツさんが本当に英雄と思います。

で、ほかの日本人が、もし、このブログを読んだら、日本にRailsがどうやっているか書いてください、すごく興味あります。

どうもよろしくお願いします。

RailsConf 09 - Ninh Grosenbach Bui

Sun, 10 May 2009 23:12:03 PDT

RailsConf is not your “normal” tech conference. You will have people ranting, doing real rocket-science and lot’s of people having real and genuine fun. We are very lucky to have people such as Geoffrey Grosenbach, Jason Seifer, Peter Cooper, Ninh Bui and so much more to remind us that there is an upper layer to technology: geeks love to have fun.

RailsConf 2009 – Ninh Grosenbach Bui from Fabio Akita on Vimeo.

And don’t miss the awesome Rubystein, Phusion’s reimplementation of Wolsfenstein 3D in Ruby. They wanted to prove that Ruby is fast enough even to write games on it, and they succeeded in spades!

A RailsConf não é sua conferência “normal” de tecnologia. Você verá pessoas reclamando, fazendo coisas realmente avançadas e muitas pessoas genuinamente se divertindo. Somos muito afortunados em ter pessoas como Geoffrey Grosenbach, Jason Seifer, Peter Cooper, Ninh Bui e tantos outros para nos lembrar que existe uma camada ainda acima da tecnologia: geeks adoram se divertir.

RailsConf 09 - Message from Bryan Liles (TATFT) to Brazil

Fri, 08 May 2009 04:26:35 PDT

Bryan Liles’ Test All The Fucking Time video really struck a cord and people have been repeating his motto ever since. He was so kind to send a message to the Brazilian programmers as well, check it out:

RailsConf 2009 – Mensagem de Bryan Liles para Brasileiros from Fabio Akita on Vimeo.

O vídeo do Bryan Liles, Test All The Fucking Time realmente tocou num ponto importante e as pessoas vem repetindo seu lema desde então. Ele foi muito legal em enviar uma mensagem aos programadores Brasileiros, dêem uma olhada ;-)

RailsConf 09 - Exclusive Audio Interviews

Fri, 08 May 2009 04:07:01 PDT

I’ve had a great time interviewing several interesting Rubysts and Railers on their new projects. I think you will like to hear what they have to say.

My first guest was Joshua Timberman. He is a fervorous evangelist for the Chef project. Chef could be seen as a more modern Puppet, which by itself, already is a modern systems configuration manager. Chef is composed of several pieces, cookbooks and details that Joshua explains in his interview.

Download (22:24)

One project that I am particularly very interested is Spree. Sean Schofield started this Rails based e-commerce system to support developers that had to reinvent the wheel all the time, and e-commerce systems are notoriously not easy to do. Spree is a fairly complete project, with many nice features, including integration with ActiveMerchant, shipping support, tax categories and so on. I helped a bit doing the Brazilian Portuguese internationalization of the project as well. Highly recommended.

Download (22:15)

By now I think we all know New Relic, Fiveruns and Rails monitoring systems. But there are more competition coming up, from the guys of Highgroove Studios we have Scout a non-nonsense approach to Rails monitoring and data analysis. They are willing to go an step further: instead of just presenting raw data as reports, they are analyzing this data and giving you relevant recommendations so you can further optimize your application. And more than that: they are highly competitive in price. And the client agent that gathers data and send to their servers is open source and extensible through plugins, so you can add even more to what they already offer. Definitely worth checking out:

Download (13:43)

At RailsConf 2008, last year, I interviewed James Lindenbaum about Heroku. They were still in beta at that time. Now they finally released a commercial version with lots of new features. I was particularly surprised to find Ryan Tomayko at their booth, working for Heroku. I think Heroku really nailed easy deployment for Ruby applications over Amazon EC2. If you don’t want to worry about infrastructure, Heroku may be the answer:

Download (33:49)

Again, last year, everybody was blown away by the announcement of Gemstone – traditional Smalltalk software-house – showing a very preview version of Ruby actually running over a very mature Smalltalk VM. This is the Maglev project. Since then the development has been quite secretive. But they are finally disclosing more and more information on how the project is going. This year, they were able to show a small Sinatra application already running – albeit, with some tweaks. I think they are evolving very fast. Ruby is notoriously not an easy language to implement and Maglev will be incredible when released. In this interview we have Monty Williams, Peter McLain and Michael Latta discussing about the current development and future roadmap.

Download (36:42)

Finally, I think everybody knows Ilya Grigorik by now, from igvita.com. He received last year’s Ruby Hero Awards, and it was well deserved. He is one of the few developers that can tackle very advanced subjects in a way that anyone can understand. He started a new company recently and they have a very very interesting product called PostRank. The overall idea is fairly simple: they went a step further over Google’s own PageRank system. Instead of just considering link tracebacks, they now weigh in social network behavior. For example, a single Digg page traces back to a website with just one link. But this same page at Digg can have something like hundreds of comments, or “engagements” as Ilya calls it. This gives a totally different weigh to this traceback instead of just a simple link. So companies are starting to pay attention to social networks such as Digg, Reddit, Twitter and others and now Ilya comes up with the tool to deliver them the necessary metrics. I highly recommend you to test-drive this site, I think you will be impressed.

Download (21:43)

And this wraps up my series of interviews at RailsConf 2009. I wish I had more time to interview more people. There were very insightful and smart developers there, and I would have to git clone myself many times to be able to talk to all of them. I hope you enjoy this set of interviews.

RailsConf 09 - Message from Peter Cooper to Ruby Inside Brazil

Fri, 08 May 2009 03:11:19 PDT

Maybe not everybody knows it but the well known Ruby Inside website now has a new branch in Brazil. It was recently released and we have a bunch of highly motivated Railers that are doing a great work posting everything that is news in the Ruby and Rails world both in Brazil and in other countries. Check it out Peter Cooper’s take on this:

RailsConf 2009 – Peter Cooper from Fabio Akita on Vimeo.

Talvez nem todos saibam, but o bem conhecido site Ruby Inside agora tem uma nova filial no Brasil. Eles iniciaram recentemente e tem uma equipe de Railers muito motivados que estão fazendo um grande trabalho publicando notícias do mundo Ruby e Rails tanto do Brasil quanto de outros países. Dêem uma olhada na opinião do Peter Cooper sobre isso.

RailsConf 09 - DHH - The Secret of Productivity

Fri, 08 May 2009 03:07:33 PDT

David Hansson delivered a nice keynote, showing how Rails evolved throughout the years, the criticisms, the discussions. He introduced the future Rails 3 feature set and, finally, discussed about the secret for productivity. There was nothing particularly new if you are already an Agilist, but it is always good to reinforce the basic concepts:

DHH Keynote na RailsConf 2009 – O Segredo from Fabio Akita on Vimeo.

And don’t miss the complete keynote at railsconf.blip.tv

O David Hansson deu um excelente keynote, dentre outras coisas mostrando como o Rails evoluiu nos últimos anos, as críticas, discussões. Deu uma introdução às novas features do Rails 3 e finalmente, falou sobre qual o segredo da produtividade. Na realidade, para agilistas, não há nenhuma novidade. Mas vale a pena reforçar o conceito de qualquer maneira.

RailsConf 09 - Uncle Bob - Professionalism

Thu, 07 May 2009 03:34:25 PDT

Brasileiros: tradução mais abaixo

Update 08/05: Bob Martins’ full keynote is now available at RailsConf’s Blip.TV page. Definitely go check it out!

It seems that O’Reilly promised to deliver – at the very least – the main keynote recordings, so unfortunately we will have Tim Ferris, but much fortunately we will have Uncle Bob. Definitely the best talk I’ve seen in eons. This particular snippet comes from the Q/A session, where Bob answer the question: “can you be too professional?” Where many would be discussing for hours, he was able to distill it in a very simple and elegant way. Check it out:

Uncle Bob na RailsConf 09 – Profissionalismo from Fabio Akita on Vimeo.

And the official, complete video of the keynote is already available! Kudos to the RailsConf staff for releasing it so quickly.

Parece que a O’Reilly prometeu entregar – pelo menos – as gravações dos keynotes principais, então infelizmente teremos Tim Ferris, mas muito felizmente teremos Tio Bob. Definitivamente a melhor palestra que vejo em muito tempo. Este trecho em particular é da sessão de perguntas e respostas, onde Bob responde à pergunta “podemos acabar sendo profissionais demais?” Onde muitos discutiriam por hora, ele foi capaz de destilar tudo de maneira simples e elegante. Dêem uma olhada.

Aproveitando, vejam a descrição da palestra neste blog também.

RailsConf 09 - Uncle Bob Martin

Wed, 06 May 2009 21:26:06 PDT

Brasileiros: tradução mais abaixo

Today we were very fortunate to have the ending keynote by Robert Martin himself (a.k.a. Uncle Bob), from Object mentor.

I recorded most of the talk and I will publish them soon (and I still have lots of material that I was unable to compile just yet), but at the end of the talk I asked Bob to quickly give an advice to my fellow Brazilian programmers that were still unable to wrap their heads around TDD.

Uncle Bob Martin na RailsConf 2009 from Fabio Akita on Vimeo.

Hoje tivemos a sorte de ter o keynote de encerramento do dia apresentado por ninguém menos que Robert Martin (também conhecido como Uncle Bob), da Object Mentor.

Eu gravei boa parte e depois vou publicar (aliás, tenho muito material que ainda não tive tempo de compilar), mas no final eu pedi que ele rapidamente desse um conselho aos programadores Brasileiros que ainda não entenderam TDD.

The Best Environment for Rails on Windows - Part 2

Mon, 27 Apr 2009 18:58:20 PDT

If you’re a Windows user new to this article, I recommend you start reading my original The Best Environment for Rails on Windows article from a few months ago, so you know what I am talking about.

Assuming you already have the Exuberant CTags and msysGit installed as described in my original article, then, once you get to the “Configuring gVim for Rails” section, you can avoid all those steps and just do this (on Windows):

cd %HOME%
git clone git://github.com/akitaonrails/vimfiles.git
cd vimfiles
git submodule init
git submodule update
copy vimrc ..\_vimrc

If you’re on Mac OS X or Linux you can do:

cd ~
git clone git://github.com/akitaonrails/vimfiles.git .vim
cd .vim
git submodule init
git submodule update
ln -s ~/.vim/vimrc ~/.vimrc

I don’t quite remember all the shortcuts, but the ones that you will immediatelly need are:

12	Ctrl-F Open up FuzzyFinder (similar to Textmate Command-T)\-P Open up Nerd Tree (similar to the Project Pane in Textmate)

If you’re a developer that had previously forked from my vimfiles repository, keep reading.

Git Conflicts

A few months ago I had forked from scrooloose’s repository and added several modifications, including a new set of snippets for the NERDSnippet plugin and configurations to respect Windows file path differences.

Scrooloose accepted my changes but instead of properly merging them into his repository, he copied the changes and made major changes to his repository. He decided that it would be better for maintenance to have the ‘snippets’ sub-directory be a separated git project, which is now here in Github. Then he replaced the original ‘snippets’ folder for a git submodule, linking both projects together.

Having no common ancestor no more, I was unable to easily merge back from his changes. I was getting this error whenever I tried to git pull—rebase or git pull -s resolve:

123	fatal: cannot read object c1a25b84627516da46b6c375f4dc874deedbb597 'snippets~a4c30e94d52232e958d4f53c6a633ed438c54bcc': It is a submodule!

The actual commit refs are different, but this is the error message I was getting. Unfortunately I am no Git-expert yet so I didn’t know if there’s a better solution for this problem. So, what I ended up doing was hard resetting back to commit 7e2366c20fb19b5709369e77aa3bc095a869e1ce, which was the last known common ancestor for our repos. Then I git pulled from scrooloose, which brought me back on track with his, but also made me lose all my custom changes.

But as most of my work was on the snippets and he already merged them into the separated project, I decided I could lose my commits. Finally I manually added minor tweaks into just 3 files to make them compatible with Windows again and adding a little bit more of eye-candy for GUI-mode on Windows, Mac and Linux.

If someone knows how to resolve this kind of situation in git without having to resort to those drastic measures, please let me know, I still don’t know how to handle submodule situations with git. I hope this doesn’t disrupt too much of your work if you were forking from me.

E-TextEditor, or, how NOT to do business

Sun, 12 Apr 2009 14:04:12 PDT

There was a very exciting news recently regarding the Windows software E-TextEditor becoming open source. The E is a very good editor, still in need of more features, reliability, stability.

But, the news is not as good as it seemed. You can certainly build E for Linux for “free” (as in beer), but you cannot do that for Windows. They are using a “modified” BSD style license, which is very restrictive (and therefore CANNOT be called BSD anymore). Calling it “essentially BSD with one extra clause” is extremely misleading. You can contribute if you like to work for free, but you cannot fork the project, you cannot remove the licensing functionality and you don’t get anything for contributing: you will still have to pay for it. So, what they want is just free labor. The “extra clause” goes like this:

Any redistribution, in whole or in part, must retain full licensing functionality, without any attempt to change, obscure or in other ways circumvent its intent.

I am certainly not against commercial applications, as I paid for TextMate myself for the Mac and I am very happy paying for high quality software. I am against paying for low quality software and I am very very against trying to misuse the open source values in a misleading way.

You either ARE Free Open Source Software or you are not. There is no such thing as “almost like FOSS, but with an extra restrictive clause”. If it is restrictive, it is, by definition, not FOSS anymore. Bummer.

Locos x Rails Wrap up

Tue, 07 Apr 2009 12:55:46 PDT

Brasileiros: tradução abaixo.

This last weekend I had a wonderful time in Buenos Aires, Argentina, at the Locos x Rails event which I think is the first big Ruby and Rails community gathering there. They did an extraordinary job making a very compelling and exciting event that I am sure people liked a lot. I don’t have any official numbers but I think there were more than a 100 attendees and a stellar international speaker roster including Obie Fernandez, Desi McAdam, Yehuda Katz, Evan Phoenix, Evan Henshaw Plath.

Locos x Rails from Fabio Akita on Vimeo.

(see it on YouTube! as well)

In the first day we had Obie Fernandez opening keynote where he presented the same talk he did at Rails Summit in Brazil, about the 4 Agile principles and the “Hashrocket-way” on implementing them. As always, a very inspiring talk. Then Fabián Ramírez, from Chile, talked about Ruby on Rails on real projects with his real life experiences and why companies should be looking seriously toward adopting Rails. Evan Henshaw Plath (aka Rabble), from ENTP, followed with an explanation on privacy issues on the Web and the OAuth solution toward web services interoperability and authorization.

After lunch, Carla Ares and Claudio Zamoszczyk demonstrated how to control Arduino hardware using Ruby – and for those who are interested, Randal Schwartz recently interviewed Massimo Banzi, co-founder of Arduino in the FLOSS Weekly podcast. Evan Phoenix, from Engine Yard, didn’t talk about Rubinius this time, but his talk was nonetheless important because he was pointing out code smells in Ruby (such as using ‘rescue nil’), I think it was very informative. And the first day closed with a video conference with David Hansson himself, answering the audience’s questions. Unfortunately this session had some internet connection drawbacks but all in all I think the audience was able to have a good chat with DHH.

The second day opened with Luis Lavena talking about Ruby cross compatibility, the One-Click Ruby Installer challenges and his Rake compiler project, which allows even Linux developers to cross compile Windows compatible binaries. After that Emilio Tagua, who is also a Rails Core Contributor, talked about ruby-debug and techniques on debugging Rails applications. Nicolás Sanguinetti gave an introduction about Sinatra and non-Rails light web frameworks. Desi McAdam, also from Hashrocket and leader for devchix.com, gave a tutorial on how to create web applications with Rails for Facebook, using the Facebooker gem.

After lunch we had Adrian Mugnolo, one of the organizers, replacing Ben Scofield – who unfortunatelly was unable to fly to Argentina. Adrian talked about Sequel as an alternative to ActiveRecord to talk to databases. Chad DePue followed talking about ActiveResource and Restful web services in Rails. Finally, we had the final talk with Yehuda Katz, also from Engine Yard, who talked about the Rails and Merb merge, the challenges and what we should expect. His main point: merging Rails and Merb does not end healthy competition in the community: on the other hand, it should increase it, but instead of focusing on big chunks as an entire web framework, people can compete on smaller pieces such as ActiveRecord vs DataMapper vs Sequel. He did a great job on explaining the evolution of the merge.

To finish the event, Tom Aadland conducted a raffle of 10 books from O’Reilly. All in all, I think this event was very well organized, the talks were compelling, the audience was full of Railers, we had real-time translators from English to Spanish as well so I think no one missed anything. They also recorded all the talks and they will release it very soon. Follow them at Twitter as @locosxrails and their website. Congratulations to all the organizers and staff for a well done event, and we hope to see you all, first at Rails Summit in Brazil this year, and again in Argentina next year!

Locos x Rails

Este último fim de semana tive uma excelente estadia em Buenos Aires, Argentina, no evento Locos x Rails que eu acho que foi o primeiro grande encontro da comunidade Ruby e Rails de lá. Eles fizeram um trabalho extraordinário criando um evento interessante e empolgante que tenho certeza que as pessoas gostaram. Não tenho números oficiais mas acho que foram mais de 100 pessoas além de uma seleção internacional estelar incluindo Obie Fernandez, Desi McAdam, Yehuda Katz, Evan Phoenix, Evan Henshaw Plath.

Locos x Rails

View more presentations from Fabio Akita.

No primeiro dia tivemos Obie Fernandez abrindo com a mesma palestra que ele fez no Rails Summit no Brasil, sobre os 4 princípios Ágeis e a “maneira Hashrocket” de implementá-las. Como sempre, uma palestra inspiradora. Então Fabián Ramírez, do Chile, falou sobre projetos reais de Ruby on Rails com suas experiências reais e porque empresas devem olhar seriamente em adotar Rails. Evan Henshaw Plath (Rabble), da ENTP, seguiu com uma explicação sobre problemas de privacidade na Web e a solução OAuth interoperabilidade de serviços web.

Depois do almoço, Carla Ares e Claudio Zamoszczyk demonstraram como controlar hardware Arduino usando Ruby – e para os interessados, Randal Schwartz recentemente entrevistou Massimo Banzi, co-fundador do Arduino no podcast FLOSS Weekly. Evan Phoenix, da Engine Yard, não falou sobre Rubinius desta vez, mas sua palestra foi muito importante porque ele explicou sobre código ruim em Ruby (como usar ‘rescue nil’), eu acho que foi muito informativo. E o primeiro dia fechou com uma vídeo conferência com o David Hansson, respondendo perguntas da platéia. Infelizmente essa sessão teve alguns problemas de conexão mas no final acho que a platéia teve chance de ter uma boa conversa com o DHH.

O segundo dia abriu com Luis Lavena falando sobre compatibilidade de Ruby em múltiplas plataformas, os desafios do One-Click Ruby Installer e seu projeto Rake Compiler, que permite até a desenvolvedores em Linux de compilar binários compatíveis com Windows. Depois disso Emilio Tagua, que também é Rails Core Contributor, falou sobre ruby-debug e técnicas de debugging em aplicações Rails. Nicolás Sanguinetti deu uma introdução sobre Sinatra e web frameworks leves não-Rails. Desi McAdam, também da Hashrocket e líder do devChix.com, deu um tutorial sobre como criar aplicações web com Rails, para Facebook, usando a gem Facebooker.

Depois do almoço tivemos Adrian Mugnolo, um dos organizadores, substituindo Ben Scofield – que infelizmente não conseguiu chegar a tempo por problemas aéreos. Adrian falou sobre Sequel como uma alternativa para ActiveRecord para conversar com bancos de dados. Chad DePue seguiu palestrando sobre ActiveResource e serviços web Restful em Rails. Finalmente, tivemos a palestra final com Yehuda Katz, também da Engine Yard, que falou sobre o merge do Rails com Merb, os desafios e o que deveríamos esperar. Seu principal ponto: mesclar Rails e Merb não acaba com a competição saudável na comunidade: do contrário, deve aumentá-la, mas em vez de focar em pedaços grandes como um framework web inteiro, as pessoas devem competir em pedaços menores como ActiveRecord vs DataMapper vs Sequel. Ele fez um bom trabalho explicando a evolução do merge.

Para finalizar o evento, Tom Aadland conduziu um sorteio de 10 livros da O’Reilly. No geral, acho que o evento foi muito bem organizado, as palestras foram interessantes, a platéia estava cheia de Railers, tivemos tradutoras em tempo real de Inglês para Espanhol então acho que ninguém perdeu nada. Eles também gravaram todas as palestras e devem lançá-las em breve. Siga-os no Twitter como @locosxrails e seu website. Parabéns aos organizadores e equipe por um evento bem feito, e esperamos vê-los todos, primeiro na Rails Summit no Brasil este ano, e novamente na Argentina ano que vem!

Rails (Im)maturity Model?

Tue, 17 Feb 2009 01:53:41 PST

As usual in the Ruby on Rails community, another drama took place :-) It all started by a very well intended Obie Fernandez in the Rails-business Google Group. He jotted down a proposal for a Rails Maturity Model (RMM), which would somewhat resemble the CMMi in its core but geared towards helping companies and clients to get higher quality Rails projects.

I can totally understand where it went bad: the association with “CMMi” and the assumption that it is all about “certifications”. Good developers hate certifications, this is a given. We all know that you can totally suck in programming and still get certified at something fancy. When you’re ‘certified’ you’re only asserting that you can memorize random stuff out of a book, not that you can really apply the theories in real life. Hence, real developers are always terrified of working close to ‘certified’ developers with no real working hours.

The bottom line about certification is that good developers will be good without any certification process, period. It is also a given that certification has the sole purpose of being a marketing stunt for people with mediocre real past success cases in real life. HR people – who are total noobies about technology – rely on certifications to filter out candidates for an interview. They would be lost without it. I know, I am being too general, but you get the idea. Don’t argue on this point just yet and keep on reading.

So, this is a vicious market, sorry. The rules of engagement are almost craved in stone and it is very difficult to change them. Agilists know what it is like to fight old habits in companies. Convincing an old corporate geezer to take a look at the Agile philosophy hurts, badly.

Since the inicial discussion, Obie wrote another article trying to be more specific on his ideas of a maturity model for Rails. And just yesterday he was really pissed off and wrote yet again that RMM has nothing to do with certifications. Understandably, people can’t spell CMM without thinking of certifications right away. Please, take your time to read Obie’s articles before continue reading my points here. But rest assured: if you don’t agree with Obie, chances are that you have no idea of what CMMi stands for.

People don’t understand CMMi

Ok, for starters, I am not AT ALL a fan of CMMi. But let me explain about CMMi so I can go back to Obie’s point on the RMM. And also note that Obie himself stated that he doesn’t fully understand the CMMi either.

I am not an expert in CMMi by far. I’ve researched a lot about it and for a short period of time I lived and breathed CMMi. Actually it was SE-CMM, prior to it becoming CMMi. I think it was around 2002. I’ve studied the texts, participated in conferences, talked to consultancies and so on. I was very much into it – and believed it, go figure! 3 years later, I also certified myself as a PMP (a Project Management Institute certificate for Professional Managers) and I studied a lot of the OPM3 model. Before all that I’ve been into the Unified Process and RUP stuff. So, yes, I’ve been into lots of ‘enterprisey’ stuff in the early days. I am out of the drugs by now, thanks for asking. None of it matters because I am very rusty in all this.

So, first and foremost: the CMMi is not a “standard” and, therefore, you can’t be “certified” in CMMi – at least not in the legal definition of “certification”. There is no certification for CMMi. It is a body of knowledge that outlines a model of process improvement, thus ‘maturity’. Well, at least that’s the theory.

You can go through an appraisal though. The SEI, the lead organization that manages CMMi, is still an academic research center in Carnegie Mellon University. People can go through a lengthy process to become a Lead Appraiser and work for Partners to appraisal a company. I think prior to 2007, when there was an update to the appraisal policy, it was a little bit easier to get appraised.

Now, don’t quote me on that because I will be saying just what I have heard in the industry some 6~7 years ago. Back then we were very interested in the outstanding growth of the indian IT market, specially the big guns such as Tata, Infosys, Wipro, Satyam, etc. There were unfounded rumors that they were all appraised CMM Level 5 because they had their own people certified as Lead Appraisers. If you just look at the SEI partner’s list, you will see them listed there, though it is not evidence for anything and this is not an accusation. Rumors, just rumors. The appraisal model has been criticized before, and if I am not mistaken, one of the reasons being that once you’re asserted to be in one Level of maturity you don’t go down because there was not a post-appraisal follow up.

CMM was first created as a kind of an insurance for the military software projects to assert that the winning companies could demonstrate at least a shred of evidence that at the very least they had some best practices and the processes in place to back them up. Anyone that worked for Tata will tell you how bureacratic they are. The appraisal process itself (SCAMPI) is based on evidence collection – though this can be very subjective at times and it is not an investigative process.

Well, to be fair, I hope that big companies do at least have that. Really big government projects have to have some level of manageability. A maturity model is by no means a hard evidence for guaranteed success, but it helps for big guns to have that. Companies such as Lockheed-Martin, for instance. If you’ve never worked for really really massive enterprise projects, you have no idea of the nightmare.

But again, I have the personal feeling that the SEI’s original goals were corrupted long ago. The way the market works is pretty straight forward: big companies and governments “requires” you to be CMMi at some Level. So, ‘serious’ companies will invest some time (a couple of years) and money (a lot) to put their processes in place and call in an appraiser. This is very expensive and traumatic to do, so only companies that can take the punch will do it. This is very elitist indeed, but it works. This is outside of the market that I personally want to be in, that’s why I walked away.

If you have no idea of what CMMi stands for, it is a model divided in 5 levels of maturity, from 1 to 5, and can be summarized this way:

Level 1: Initial (Ad hoc, chaotic, no processes, success out of luck, where most companies really are)
Level 2: Repeatable (some project management, some discipline, a rough process that can be used repeatedly)
Level 3: Defined (the organization has institutionalized the processes)
Level 4: Managed (management means that is can be quantified, there are metrics in place to actually measure a project)
Level 5: Optimizing (with the proper processes in place, the process can give feedback to itself, helping to refine the process)

To assert that you’re in a certain level, you must implement (in a verifiable manner) a series of Key Process Areas (KPAs). Each level has its own set of KPAs which define a Goal, Commitments, Abilities, Activities, Methods of monitoring, Methods for verification. As I said, this is not a small thing.

“Implementing” the CMMi up until Level 5 can really take some good years. Implement a KPA properly means really to understand the gut of it, sometimes knowing how to consciously bend it over and still have it validated. It is a real struggle to go through the whole process, specially when you already have lots of projets going on in an ad hoc manner (Level 1)

There is a whole lot more to be said about the CMMi and I recommend you read this small FAQ about it that can be quite enlightening. Also read some of the criticisms, specially to its appraisal system.

This is a very very short summary of what CMMi is. There is a vast literature about this subject and if you’re interested I suggest you go research about it. You will learn a lot, but don’t take it too seriously. This is the kind of thing that every good developer/manager should’ve read at least once in their lives.

Some people think that it is possible to blend Agile with CMMi. I wouldn’t say it is impossible, but I would certainly ask “What is the point?” I really think it is kind of pointless to attempt such a thing, but I digress.

Obie

Now, going back to Obie’s idea, the way I understood it, it is just an idea of maybe implementing something akin of CMMi’s 5 Levels of maturity, where in each level you comply to a set of KPAs, such as “we do pair-programming 100% of the time” or “we have extensive and almost total test coverage”. Not going into the merit of the KPAs themselves, one could assert a certain level of maturity with some well known criteria, if they can be clearly defined.

The idea in itself is not bad. Having a reliable, updated body of knowledge is a good idea. The CMMi is based on past history and past data of successful projects. A possible “RMM” would be similar, with the best practices and techniques used by the most successful companies of the time, such as Hashrocket or Pivotal.

There are some problems though: this is an appraisal for the company, not for an individual, so the problem of lack of good quality developers would still be an issue as this doesn’t attempt to solve this problem. The second problem is just that there is still not that many Rails projects, consultancies and a true record of past projects to even start thinking of a maturity model. We are, by definition, not mature enough yet. And don’t take this as a criticism to anyone. Thinking about it, Rails is just 4 ~ 5 years old now. This is just too short.

Finally, Rails is a technology. Even though the idea may sound compelling for us, fans, Rails is a technology, not an specific professional area such as “Software Engineering”. You can certainly apply Rails into CMMi companies, but having Rails implemented properly doesn’t lend anyone a Level 5 stamp.

I think the CMMi is not a very good starting point as it is just too monumental, with a very very big overhead, and very very bureaucratic processes. It is just like RUP vs Agile again. The core idea is good if we keep the CMMi analogies out of the question.

Now, I do think the idea of a body of knowledge is appealing. This could be an organic repository where people would be able to organize their metrics, their own discovered best practices and outlines for successful projects. Over time new material would stack up until we could finally have enough data to actually start thinking of maturity: when the whole community goes into the next level.

Neither a possible RMM nor CMMi actually answers the questions most clients have. None of them can guarantee a software life-cycle with real healthy, without sloppy programming. None of them will make it easier for customers to find good freelance Railers. And if you, as a savvy customer, ever need a Railer you would never trust a consultancy with an “RMM badge”, I bet.

In the end of the day, processes are important, but never to become too overwhelming. There is a gray area between monumental models such as CMMi and agile methodologies (I know, I know, a capability model is not an engineering methodology, I am just comparing complexity). I think the community is still too young to make decisions on maturity. But, I am open to ideas. If someone come up with a detailed RMM specification proposal, I would be very interested in reading it.

But again, I do think the Agile philosophy, the techniques from XP, DDD are enough by now. Before we start debating on bigger processes, some people don’t even know the basics yet, such as algorithms, data structures, proper OO programming, etc. Heck, there are web developers that don’t even understand the HTTP protocol or the TCP stack yet. This article is by no means an attempt to answer the question and it is still open for debate, but it is very valid to debate it. Bashing and complaining will not bring good ideas to the table.

Off-Topic: (Probable) Leopard fix to connect to Windows VPN

Wed, 14 Jan 2009 23:00:05 PST

This has been frustrating me for the longest time. Mac OS X has both L2TP and PPTP support for VPNs but for some reason it has a very difficult time dealing with Windows based PPTP VPNs. Speculation goes on along the lines that OS X uses a newer implementation of PPTP, and Windows doesn’t (or most probable, it has a buggy implementation that only works on Windows, same problem as SMB, LDAP3, etc).

Whatever the reason, the behavior that I’ve been seeing is that Leopard is able to connect to a Windows VPN, it is able to authenticate, it is able to route correctly but then it won’t drop the connection but simply stops routing packages into the VPN tunnel after around 5 min of usage. So, I decided to Google around and I tried a few tricks.

Up until now I am connected to my company’s Windows VPN for 1 hr – which might be a world record! Let me explain a few things I tried and that might have helped in this.

First, I tried a trick I read in a forum:

Open System Preferences → Network
Click the “Location” drop down menu and choose “Edit Locations …”
Create a brand new Location
Configure your Wifi or Ethernet connection so you’re online
On the lower left corner, click on the ”+” button, then on “Interface” choose “VPN” and on “VPN Type”, choose “PPTP”.
Configure your VPN using your company’s setting (IP address, “DOMAIN\username”, type your password clicking on the “Authentication Settings …” button.
Again on the lower left corner, click on the 3rd icon, next to the ”-” icon and choose “Set Service Order …”
Drag and drop your VPN service to the top of the list

If you have VMWare Fusion installed, I read that the virtual network adapters could be one of the reasons so you need to stop them typing this in the Terminal:

sudo /Library/Application\ Support/VMware\ Fusion/boot.sh --stop

After that, again in the Network Preferences Pane, I connected into my VPN and surprisingly the connection stayed on for more than 1 hr whereas I couldn’t stay connected for more than 5 min before that. None of the steps above make much sense, but it seems to have worked for me.

I also left a Terminal open constantly pinging some server inside the company network to make sure I am keeping the connection alive. I have no idea if those steps actually solve the problem or I just got lucky all of a sudden. Please drop a line in the comments if it solved it for you or if you have other insights around this issue.

Bottom line: if you can, don’t build a Windows-based network.

English - AkitaOnRails.com

Chatting with Adam Jacob

Chatting with Luke Kanies

Rubistas.com.br

[Rails Summit 2010] Commence Planning

[Rails Summit 2009] ThoughtWorks pousa no Brasil

ThoughtWorks lands in Brazil

[Rails Summit 2009] Palestrante Arthur Zapparoli

Speaker Arthur Zapparoli

[Rails Summit 2009] Palestrante Nando Vieira

Speaker Nando Vieira

[Rails Summit 2009] Palestrante Ilya Grigorik

Speaker Ilya Grigorik

[Rails Summit 2009] Palestrante Matt Aimonetti

Speaker Matt Aimonetti

[Rails Summit 2009] Palestrante Carlos Brando

Speaker Carlos Brando

[Rails Summit 2009] Palestrante Bryan Liles

Speaker Bryan Liles

[Rails Summit 2009] Palestrante Leonardo Borges

Links

Speaker Leonardo Borges

Links

Playing with Dry Scaffold and Inherited Resources

Default Scopes and Hash Posers

User Editable Liquid Templates in the Database

Next Steps

[Screencast] Real Thin Restful Controllers with Inherited Resources

Rails Summit Latin America 2009

Rails Summit Latin America

Rails developers in Japan

RailsConf 09 - Ninh Grosenbach Bui

RailsConf 09 - Message from Bryan Liles (TATFT) to Brazil

RailsConf 09 - Exclusive Audio Interviews

RailsConf 09 - Message from Peter Cooper to Ruby Inside Brazil

RailsConf 09 - DHH - The Secret of Productivity

RailsConf 09 - Uncle Bob - Professionalism

RailsConf 09 - Uncle Bob Martin

The Best Environment for Rails on Windows - Part 2

Git Conflicts

E-TextEditor, or, how NOT to do business

Locos x Rails Wrap up

Locos x Rails

Rails (Im)maturity Model?

People don’t understand CMMi

Obie

Off-Topic: (Probable) Leopard fix to connect to Windows VPN