Marco Valtas

CFIT in Software Development

2013-02-10 00:00:00 +0000

Controlled Flight into Terrain or CFIT is a type of accident where an aircraft with no apparent mechanical problems is flown into some obstacle like a mountain. Couple of days ago I used this acronym to explain some situations that happen during software development, but first let’s understand better CFITs.

These can be caused by a large amount of reasons; poor visibility, confusion in the cockpit, wrong navigation maps, poor capability to operate the aircraft and so on. Usually what happens is that given these factors the aircraft is put in collision course and when alarms go off there’s little time for the crew to react or the crew is so confused that doesn’t understand what is happening and take the wrong measures. Running the risk of adding one more acronym to the large set we have in IT I propose the usage of CFIT for some situations.

In Software Development sometimes we have situations similar to these, when information is so bad that we end up coding the wrong solution and this is a factor for a CFIT. We might also have poor visibility and make the wrong calls i.e. poor performance, because the real load was never shared with the development team. Infrequently but still of note is poor directions given by someone responsible for the solution, like a senior developer or an architect, that instead of sharing the inputs (business problems) only share the outputs (decisions made) leading the team steer the software into a collision course. And by collision course I mean once gets release it fails in some way.

We can deal with such situations adopting some practices, like fast and better feedback and good visibility. A GPWS (Ground Proximity Warning System) for software development is nothing more than good monitoring, if metrics go off chart we can correct the course before is too late. And a good CRM (Crew Resource Management) is good and honest feedback, of the product, of the code and where the team is headed, poor CRM is a Command Control environment which is a great way to provoke a CFIT. I believe we could draw some more parallels but in fact what I’m looking for is lessons learned and how we can adopt them to Software Development.

Continuous Delivery of this blog

2012-10-17 00:00:00 +0000

For some time I’m using Jekyll which gave the opportunity to automate some things for this blog, I went as further I could in creating a Continuous Delivery environment for it.

Tooling, stages and more

In order to make this Continuous Delivery environment I had to make sure that I had some tools and stages defined, here’s a short description of these.

Version Control

I have a private repository on github for all the code and posts, so whatever automation that I will put in place it has to integrate with it.

Build and Deploy tool

Rake is my tool of choice for compiling and deploying the blog, actually rake today is my tool of choice for building almost anything, the tasks and the fact you can write plain Ruby in it is enough for most of automation that I’ve been doing.

Compiling

Jekyll is a static blog generator which pretty much is the same to say that it reads your files and create a static website. That’s my definition of compilation for this blog, whenever I use compilation I’m referring to the execution of:

	jekyll --pygments

Testing

When I’ve mentioned that I was doing this kind of automation to my colleagues one of the first questions was “What are you testing it for?” Definitely a fair question. All this automation without something to test would feel like doing a Goldberg machine for a simple copy of static files.

What I’ve found later was rather interesting, first I didn’t have tests but as soon as the deployment was working I found it that I had the perfect environment for doing all kind of tests.

Currently I test the blog for two things:

All internal links point to a valid resource.
No spurious CSS classes are left around cluttering without being used.

Packaging

How do we package a static website? My answer to the packaging was as simple as you might guess, tar.gz. This is the simplest packaging you can have around and works pretty well on Unix based systems and with a small website it doesn’t hurt to do a replacement of all files every time I deploy. There’s a catch though; you have to clean up the destination, otherwise files that got removed will hang on the directory indefinitely.

Staging environment

Before going to production I setted up a staging environment, basically this environment plays two roles on the pipeline. First it’s where I can run some tests with the website up. Second, it allows me to do a proof reading of what is about to get published.

Continuous Integration / Agile Release Management

I’m using ThoughtWorks GO to coordinate all steps of testing and deploying.

The pipeline

Here’s a screen shot of couple last runs of my blog.

There’s nothing fancy about it, it compiles and that has a couple of steps including packaging, publishes to UAT (User Acceptance Tests) environment, the internal links tests happens and than there’s the last stage which is manually triggered by me to publish to production.

Please note that not every build got published to production, often times when I’m proof reading or working on the site plumbing I find a problem that I didn’t catch on my local environment.

Another thing that might caught your attention is a broken build. That specific one happened when I’ve added the jquery library I didn’t use a absolute path on the src of the script tag, the issue was discovered by the checklinks on pages that were in directories and unable to find jquery.js.

Build / Compile stage

More than just run jekyll happens on build stage, here’s all the tasks currently executed by Go:

bundle install --path vendor/bundle
bundle exec rake build
bundle exec rake archive
bundle exec rake send_pkg

The first task is just installing jekyll and rake and it’s dependencies, using bundler gives me the option of updating jekyll without having to worry if the CI’s jekyll is updated too, plus, is the update brakes the website and I don’t catch the problem locally it will be caught by the CI.

When we invoke rake build here’s what happens:

task :build => [:compile, :check_css_redundancy]

task :compile => [:pygments_is_installed] do 
  run_or_show "jekyll --pygments"
end

task :check_css_redundancy => [:compile] do
  run_or_show "bash -c \"_bin/css-redundancy-checker.rb _site/css/style.css <(find _site -name '*.html' -type f)\" " 
end

css-redundancy-checker.rb will do the work of checking is there’s any CSS leftover on the site. I’ve tweaked a little bit to fit the automation specially to give an exit different of 0 in case it finds something, that causes Go to fail the task.

If all goes well we arrive at the archive task, which is this one.

task :archive => [:version_info] do
  FileUtils.mkdir ARCHIVE_DIR if !Dir.exists?(ARCHIVE_DIR)
  run_or_show "tar -zcf #{ARCHIVE_DIR}/#{pkg_name} -C _site ."
end

def pkg_name
  label=ENV['GO_PIPELINE_LABEL']
  fail "Can't do much without environment variable GO_PIPELINE_LABEL" if label.nil?
  "blog-#{label}.tar.gz"
end

As I mentioned earlier packaging of the site is nothing more than a tar.gz of the files compiled by jekyll. The GO_PIPELINE_LABEL environment variable is used so that I have different packages for each build. The result of this task is files like these:

blog-63.tar.gz
blog-64.tar.gz
blog-65.tar.gz

And after the file is packaged it is send to the server with scp.

task :send_pkg do
  run_or_show "scp -v #{ARCHIVE_DIR}/#{pkg_name} #{AT_USER + HOST}:tmp/"
end

Here AT_USER is my artifacts user which has an account on my server just to host the packages. You may think about it as a repository similar to Yum or Apt for the my artifacts just a much simpler one. This is the end of the build / compile stage.

Publish to User Acceptance Test environment (UAT)

The next stage is publishing the site to the UAT environment, this is accomplished by this rake task:

task :publish, :env do |t, args|
  env = valid_env_parameter args
  run_or_show "ssh -v #{user env} ' [ -f ~#{AT_USER}/tmp/#{pkg_name} ] " \
       " && rm -Rf #{target_dir env}/* " \
       " && tar -zxf ~#{AT_USER}/tmp/#{pkg_name} -C #{target_dir env}' "
end

Which is a fancy way to just execute a shell command on the server. It first checks if the package for the version of the blog we are about to deploy is in the repository (remember if just a directory on the server). Than it follows by removing all the files and uncompressing the package. This is the same task used for UAT and production, it just differ by a parameter given to rake, which changes the host and the user to be used.

UAT Testing

In this stage currently we run just the checklink tests, here’s the task that triggers it:

task :checklinks, [:env] => [:checklinks_is_installed]  do |t, args|
  domain = target_dir valid_env_parameter args
  run_or_show "_bin/checklink_wrapper.sh #{domain}"
end

In order to make checklink useful on the pipeline I coded a simple shell wrapper that does the job of failing the task if checklink finds a problem, also in order to test only internal links and changing the domain to which checklinks runs I had to add one regular expression filter on it. Checklink is good but it’s not so easy to understand how to use the parameters it has. Here’s the wrapper:

#!/bin/bash
set -e

i=0; 
while read line
do 
  [[ $line =~ 'Code:' ]] && i=$(($i+1)); 
  echo $line; 
done < <(checklink -r -b -X "^((?!$1).)*$" http://$1)
exit $i

This script will exit with the number of links that were broken, again, if the number is different than zero it will break the task and thus the build.

Publishing to production

This is the same as publishing to UAT, it’s the same deployment script and it changes only the user and host where the site will be published. This is a manual triggered stage in Go, when I’m confident that I want to release I push the button and it goes.

Notes and Observations

Let’s discuss some principles of Continuous Delivery and how these relate with what I did.

One package, one binary

You will note that the packing/archiving of jekyll compiled files is done only once. As mentioned in the Continuous Delivery book, you want to compile only once, test and deploy this “binary” as much as will feel like but not recompiling it. Why? Because if you recompile it who guarantees you didn’t change something from one compilation to another? When we create a binary or a package we want that this package prove that is production worthy, it has to face all challenges we can think of. If it survives, it’s production worthy.

Rolling back / Rolling forward

Another important factor on a Continuous Delivery environment is being able to revert your changes. With the current pipeline and deployment scripts I can release whatever version of the blog I want just with a click of a button. In fact I did it just now for the fun of it and reverted the blog to a month old version, it worked.

In this particular environment there’s not actual rollback, it’s just roll forward, the only thing that is needed is the deployment of an older version.

Extend your pipeline to production, do it as earlier as possible

When I decided to move my blog to CD I had another publishing routing, it was simpler but still bases on the triggers of version control. In short, it ran jekyll as soon as I pushed to the repository.

Moving to a proper pipeline and automation changed a lot of things; first I stopped running jekyll on the version control server, which as I already guessed was the same as the web server. All of this stack was pilled up on one server, now there’s a better organization of the roles of each server.

I had to go through a lot back and forwards to design an architecture of deployment to suit my scenario. A big challenge and one that I would see only when automating to production was security. Currently I use DreamHost to host my blog and for all my users I use enhanced security, maintaining security and still being able to perform automated deployments required some thought and some different strategies that I first thought.

This is why extending your pipeline to production is not something to be left behind. It will pose to you different challenges from running your application on your local machine, it will pose security questions, third party integration, cleaning up the file system, making sure that if a deployment fails it will not leave your application in an invalid state; and the list goes on.

Keeping a version number

Another thing that I did just for the fun of it was add a version number to the blog. If you access this url you can check what is the current build number of the website, the version control revision and the date of compilation. I don’t have any practical usages for this information, not by now anyways. Here’s the task that generates the version.txt file:

task :version_info do
  File.open('_site/version.txt','w') do |f|
    f.write "package: #{pkg_name}\n"
    f.write "date: #{Time.now}\n" 
    f.write "revision: #{ENV['GO_REVISION']}\n"
  end
end

That’s it.

Scripting Token Retrieval on OSX

2012-10-09 00:00:00 +0000

It’s being sometime that I’m working from home and because of that I have to use my RSA software token way more often. After a couple of: start SecurID, type your pin, copy the token number, close the SecurID and paste on whatever intranet site I want to access; I went and automated the whole thing with AppleScript.

Now I just fire up the script using and done, the token is added to my clipboard and I can paste it anywhere I need.

Note on security

There’s a caveat. If you’re willing to automate the token retrieval, where did you plan to write down your pin? If you do it on the script itself it’s a bad idea since is easy to get your pin just by executing grep on it.

A better and more secure solution is to have an encrypted storage on your OSX from where you can retrieve the pin, and guess what, you have it and it’s called Keychain. That’s what I did, just added a new key to “login.keychain” (where all your web passwords get saved) called rsatoken.

The implementation

I needed to search the internet for some information (since AppleScript is not my strongest skill) and I ended up with this script:

set appName to "SecurID"
set thePin to RsaTokenPin()

activate application appName

tell application appName
	activate
	tell application "System Events"
		keystroke thePin -- type the pin number
		key code 36 -- return key
		delay 0.3 -- wait for token appear
		key code 48 -- press tab
		key code 49 -- space (to hit the copy button)
	end tell
end tell

quit application appName

on RsaTokenPin()
	return (do shell script "security -q find-generic-password -gl rsatoken 2>&1  | egrep '^password' | awk -F\\\" '{print $2}'")
end RsaTokenPin

What the script does is pretty much the way you have to do it manually, it fires up the SecurID application, enters your pin, type return than a tab followed by a space (which will press the “copy” button) and closes the application.

The catch is on retrieving your pin number, it doesn’t use the AppleScript API to do it. Several sources stated that AppleScript and Keychain are not a good combination and dreadful slow. So I followed their advise and used the security shell command.

The security command doesn’t give a usable output with just the password so I had to use a little more piping around to get what I wanted.

That’s it, quick recap. Add a key named “rsatoken” to your Keychain with your pin as password. Paste this script in your Apple Script Editor and save it as an Application. This should save some minutes during the day if you need to enter your token quite often as I do.

Cheers, Marco.

Developer Checklist

2012-08-15 00:00:00 +0000

Couple weeks ago I finished reading the Checklist Manifesto and guess what? I did my own checklist.

I was familiar with checklists because my curiosity about aviation and human factors in high reliable organizations, this book was just the little push I needed to making one myself.

Check the code in GitHub

A little more about checklists

A checklist is not a step by step procedure and it shouldn’t be comprehensive, it’s not a manual. The point is to help experts to not forget simple steps that have great importance. Checklists are only good if they are used and prevent mistakes. That’s why they have to be simple and focus on important steps.

Another characteristic is pause points. Some checklists will have a big bold title (like the one above) telling you when to use it and that’s a “pause point”. Some tasks will have several pause points, here’s some examples of pause points in aviation:

Before push back
Before take off
After take off
Before landing
After landing

Checklists are divided in two types, one known as READ-DO and another DO-CONFIRM. In short the READ-DO type you read item by item while executing each item. For the DO-CONFIRM you do all steps by memory and confirm these steps against the checklist.

More information about checklists you can find in the book and if you feel like trying do your own checklist also check Project Check website.

What the left terms mean

CI: Continuous Integration - your build system, i.e. ThoughtWorks Go, Jenkis, Team City…
Upstream: The code repository you’re working on, i.e. Github, SVN, Bitbucket…
Tests: Your tests, basically the unit tests.
Files: The files you are working or are part of your change.
Commit mgs: The message you write on your commits, also known as commit log.

Understanding this checklist

Before I started this checklist I asked myself “where did I and my colleagues made more mistakes?” Where we usually have to follow certain steps and we end up making a mistake? My conclusion was: when we check-in code into repository and that was my pause point.

BEFORE PUSH in capital letters sets the time when this checklist fit. This is a little biased to distributed version control systems, but checklists are encouraged to be changed, so if you use SVN you can change the pause point to BEFORE COMMIT.

This is a DO-CONFIRM checklist. The items are simple and well known for software developers, so just saying “CI green.” is very similar to “Gear down, three greens.” (When pilots say “Gear down.”, a lever should be moved and there’s three lights on it. “Three greens” means that all three landing gears are down and locked.)

Note that this checklist has the style of aviation checklists, the user will be familiar with the acronyms and the actions/states they should be in.

The order and the purpose of each item

1. CI…GREEN

Before sending the code you will update, this step is to avoid updating/pulling from a broken build (unless you intend to fix it). If you pull from a broken build you might break your current work. Instead, you might choose to keep working until the build is fixed.

2. Upstream…PULL/UPDATE

Most version control systems will not check-in code if conflicts show up, but that’s not the thing you are focused on. Updating your code before checking-in is considered a good practice, gives you the opportunity to see if other team members changes will play well with yours and sometimes you need to make further changes to accommodate the others.

3. Tests…FULL SET - GREEN

During your work usually you run just a small set of tests to keep things fast. Before sending your changes and specially because you just pulled changes from others, it’s good to check if all the tests are green.

4. Files…ADDED/STASHED

This step is to check is the files you about to be check-in are the ones part of your change. It’s very common check-in files that are not related to your change.

5. Commit msg…REVIEWED

Most companies have hooks on version control to relate the commit with a ticket/story. People tend to check-in wrong identifiers, wrong messages and similar mistakes.

6. CI…GREEN

After all the checks above the build might be broken (other team member checked-in code) or potentially brake if is still running. This step is for you double check if sending the files is safe.

Last notes

I plan to expand this work with other checklists, I believe there’s space for it and enough pause points in software development where we can benefit from checklists. If you have ideas or want check out the code and the PDF go to GitHub.

Schrödinger's release

2012-07-20 00:00:00 +0000

Schrödinger’s release is just a thought experiment as the original one for teams pushing Continuous Delivery methodology. If your team got into to a good Continuous Delivery process that means that at any time a release into production can be made. Business feel empowered to release the current code into production just with a push of a button.

Imagine now that the software team is not aware when this might happen, at any time business can decide go to production without communicating the team, like the Schrödinger’s Cat the button can be pushed or not and the current code inside inside the version control be in production or not. For the team it is a unknown state.

If the code can be released at any time decreases the chance of postponing production readiness of the code just because the business will announce the release couple days before. In a Schrödinger’s release the team can’t take any chances, code checked-in (and green tests) is always releasable.

Spammers push technology forward too.

2012-05-09 00:00:00 +0000

Almost a month ago I got the strangest Google Alert, it said:

Web 1 new result for "Marco Valtas"
 
No Prescription Needed Pain Killers And Sildenafil ... - Marco Valtas
No Prescription Needed Pain Killers And Sildenafil. Men's Health. Shipping
Policy, Canadian Pharmacy, Erectile Dysfunction, Stop Smoking.
marcovaltas.com/?page_id=no-prescription-needed...

Obviously something wrong had happened. I wasn’t quite sure why Google thought that Canadian Pharmacy would link to my blog, but who knows? After a couple minutes digging around it became clear that something was very wrong, more and more searches on Google returned bogus links and references to my site were also from bogus links.

WordPress Spam Injection Attack

In a nutshell, spammers gain control by your WordPress platform using some vulnerability, which might be in some plugin or theme, make your site behave differently when crawled by Google (maybe other search bots too). It’s pretty clever, taking the Google page rank and attacking enough sites they start to link one to another promoting the results. I had one Israel TV website linking to my site by 36 different URLs, the TV website is also run by WordPress code.

That’s what happened with my site. If you dig into the Google there’s stories of people having to go over all data in their sites to ensure they were secure again. And the rest of the links talk about how to secure your WordPress installation.

So I did what you expected, I saved the code for evidence and took down the site until I got some evidence that it was clean back again. Going through dumps of MySQL and the code.

Trying to clean everything

I did just enough to be relatively confident that no bad code was injected around. In order to avoid any missing file I did a simple clean install of WordPress, I could go over the missing plugins afterward but the primary objective was to clean up the website from any spammer injection.

Even after checking files, doing a bit of grep around I wasn’t confident. First because I didn’t know what vulnerability was exploited. Second because for some reason if I tried to access the website using the bogus URL’s WordPress would only redirect to my home page with a nice 200 OK http code back. I wasn’t happy and I decided to dig the code a little more, this time I was determined to control WordPress redirection in details, when I’ve found the Canonical API to handle WordPress Redirecting. That was the last push I needed to ditch WordPress.

Couple notes on security

Until proven contrary Security and Convenience don’t walk together, they are almost opposite to each other. You can tell by paying attention on the most secure things you deal, lets say you bank account. If you want check your bank account online chances are you will need a token not only your password. You company might give you laptops and their policy don’t allow you to install anything into it and other security hassles, some valid, some just theater.

Taking out the social engineering, when is code against code, one thing matters simplicity. Every operation you add to your code is one more operation to be exploited, as your software grows in complexity also grows and possible security holes and there’s clever folks out there making money (it got to be money in this for exploiting my website to sell Viagra) with every invasion.

All convenience WordPress was providing me is grounded in lines and more lines of code. Themes, plugins, configuration, feeds, comments and such don’t come for free security wise speaking. And frankly I didn’t need all this stuff.

Going to Jekyll and plain old HTML pages

I knew about Jekyll by some blogs, I thought about going for it but I was too lazy to do so since my website, although important to me, it was not a priority until the attack. When I decided to change, my friend Chris Stevenson brought Jekyll to my attention. Jekyll seemed the right solution, just enough to get a website running and pretty old HTML would make less vulnerable to spammers. I won’t put all details of migrating to Jekyll there’s plenty information already in the web. Some details that made my migration a little different from the others I saw.

Backward compatibility with WordPress links

My old WordPress was configured with permanent links format like this http://marcovaltas.com/?p=107, in order to keep old links compatible with Jekyll I needed some code to proper redirect URL’s. I did a dump from the WordPress database and created a file which maps post IDs with the slugs used by Jekyll and little CGI (this exposes how old I am) script to redirect requests. I hope retire this script soon.

The dump from WordPress didn’t went smoothly

This website was migrated from Blogger to WordPress, WordPress to WordPress and finally from WordPress to Jekyll. Was written in Portuguese (just a couple of posts) and English. Guess what happened? It got mess up. Old posts with weird characters that even changing encodings didn’t solve, no proper formats and code snippets once formatted with SyntaxHighlighter have the old bracket tag. So I have to go back and fix them all, maybe delete some posts :)

There was a migration page which lists the old posts and their statuses of Pending or OK. For this, I created simple generator plugin for Jekyll that only checks if the tag migration is defined on the post, if so is considered migrated. Here’s the script:

module Jekyll

  class MigrationListGenerator < Generator
    safe true

    def generate(site)
      File.open('_includes/migrations.html','w') do |file|

        file.write("<ul>")
        site.posts.reverse.each do |post|
          next if post.date > Time.parse('2012-04-02 00:00:00 -0300')
          file.write("<li><a href='#{post.url}'>#{post.data['title']}</a> #{post.data.has_key?('migration') ? 'OK' : 'Pending'}")
        end
        file.write("</ul>")

      end
    end
  end
end

Every run, Jekyll loads the plugin and executes it. An annoying side effect is I can’t run Jekyll with the --auto flag anymore. Since it creates a file, it recognizes the site update and generates it again ad infinitum. I had no time (or too lazy) to think in a better solution. And post.html layout has also a conditional to render a little notice if the post wasn’t migrated yet (I had to remove the tag {% op %} because of formatting constraints):

if page.migration == null
<div id="migration_note">

  Looks like I didn't fix this post yet, I'm working on fixing all my posts
  after a sad hacking of my site.  You can check the progress <a
    href="/migrations.html">here.</a> 

</div>
endif

Final thoughts

Going to Jekyll wasn’t bad at all, was just the timing which wasn’t chosen by me but by the spammers. Other advantages are being free of DB, use Git to manage the files, using Vim to edit, a fresh look and some other ideas. Drawbacks, the only I can think right now is loosing the ability of posting from any computer with internet, now I have to push to the repository as a hook deploys the files.

And if you see anything weird on the site please let me know, as something maybe escape my sight.

Update: I finished the migrations so there’s no more migrations.html page.

Cheers.
Marco.

Vitosto JS - A new library in JavaScript for fast web development

2012-04-01 00:00:00 +0000

Motives

Today even the simplest application will demand from developers to know a handful of technologies and languages. Rapid prototyping libraries were created to bridge this, taking care of much of the boiler plate code necessary to put a application online, like Rails. Moreover template engines are based on the assumption that it will exists a Designer to take care of the pages, also the business logic has to be isolated from the rest of the code like persistence. Patterns like MVC, MVP and MVVM are thought to achieve this, but the sad truth is, often times applications end up with logic everywhere in these layers making growing the application harder and harder.

Vitosto JS - a new way to think about web development

With these in mind, I joined a group of friends and we started to work in a new library, Vitosto JS. View To Storage in JavaScript. The idea is lower down the amount of different layers between the user interface and the storage where the data actually is. Developers can grow their application just controlling one layer of code and having some knowledge about databases, as for now, vitosto.js can only work with MySQL databases, support for Oracle is in progress. Enough talking, more code:

Hello World:

<html>
  <head>
    <title>Vitosto JS</title>
    <script src="vitosto.js"></script>
    <script src="vitosto_config.js"></script>
  </head>
  <body>
    <h1>
      <script>$db.select("text").from("my_table");</script>
    </h1>
  </body>
</html>

vitosto.js is the library it carries all code to actually connect to the database plus a fluent API to issue queries. vitosto_config.js holds the connection information, for security it is encrypted with ROT13 algorithm for security purposes.

All the power of JavaScript is at your hands and a simple WebServer like apache will be necessary to hold your application. Here's an example of a form submission.

<html>
  <head>
    <title>Vitosto JS Form</title>
    <script src="jquery.js"></script>
    <script src="vitosto.js"></script>
    <script src="vitosto_config.js"></script>
    <script>
      processForm() {
        var name = doSomeSanityCheck($("#name").text);
        $db.insert(name).into("table");
        rewriteDomForThankYou(name);
      }
    </script>
  </head>
  <body>
    <form>
      <input id="name" type="text" maxlength="50"><input type="submit"
onClick="processForm()">
    </form>
  </body>
</html>

As will can see we can mix JQuery and other libraries with Vitosto.js.

name

Next Steps

Oracle connection driver
Support for Oracle functions and procedures

Code and API information

Download the Beta Release at: Vitosto at GitHub.

Two wrongs do not make a right

2012-01-12 00:00:00 +0000

In the software development business there's some well known currencies, these currencies are used to value an employee and the employee (dev, architect, QA or manager) struggle to achieve them as they make their employment more stable or more profitable, this fact goes probably for all work environments. Some more common is productivity (whatever way it's measured), proactivity, efficiency, leading, domain knowledge. Whatever the company values the employee will try to make himself/herself good at it.

Companies, whatever they value, have to pay attention to these values, misunderstanding what they mean and evaluating it wrong will lead to employees behaving as they're measured, which it turns out not to be the value glued in a beautiful frame on the wall.

One in particular that I want to focus is "domain knowledge". I've seem much misunderstanding of this skill inside big companies. Let's say that you work on a big company that develops software. Some old time folks would be considered valuable because the have a lot of "domain knowledge", which usually translates in knowing the technology stack and how it works and its implementations, that's supposed to be good, right?

An employee with this kind of knowledge is essential to help new comers to understand what is happening and how to deal with the application, how to improve it without breaking it and how to modify it. This is a good thing and companies are right to be worried about retaining such employees.

Gaming the system is easy though. This is where I find hard to deal with in legacy systems. Much of the systems that I saw are not so complex in essence but they are a bad coded software. Domain Knowledge experts know the way around it because they wrote the thing in first place but often do not know the business concerns behind it. It turns out that this "domain knowledge" is just a background of the past bad code written.

In such situations two wrongs do not make a right, bad code is bad code, knowing how was written doesn't make an expert it is just half of the knowledge needed, it is still missing the knowledge of the problems the software was trying to solve. I expect from an expert to know the problem and the current solution and be able to discuss new solutions for the same problems as well find out other problems. Companies have to be aware of that otherwise employees regarded as experts might be only gaming the system and holding the company back on innovative solutions.

Keeping QuickSilver up with Launchd

2012-01-11 00:00:00 +0000

OS X launchd spawn processes and manage them, you can actually make use of it to help you out on OS X automation. I had the same exactly thought that Diego Zamboni had when wrote this blog post four years ago.

QuickSilver became part of my interactions with OS X for a while but QS is not perfect and sometimes crash or needs to be restarted. As Diego felt, I felt that when QS quits is annoying go chasing it again to restart it. For this launchd can help you.

Start by disabling the Start at Login option of QS in its preferences.

After that adding the below XML as ~/Library/LaunchAgents/com.blacktree.Quicksilver.plist will do the trick of keep QS running, if it quits, it will be restarted automatically.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>Label</key>
    <string>com.blacktree.Quicksilver</string>
    <key>LimitLoadToSessionType</key>
    <string>Aqua</string>
    <key>KeepAlive</key>
    <true/>
    <key>ProgramArguments</key>
    <array>
      <string>/Applications/Quicksilver.app/Contents/MacOS/Quicksilver</string>
    </array>
  </dict>
</plist>

You can check the this Technical Note for a lot of details. But I will give a short explanation of what we have here.

Label - It's mandatory to identify the job and has to be unique, it has to match the file name (without the suffix .plist)
LimitLoadToSessionType - The context the job should run, Unix fellows this is similar to run levels. The value of Aqua means full GUI.
ProgramArguments - Is actually what we want to run. Check the man launchd.plist and execvp(3) on the Terminal for more information.

That's it, log out and log in back again and you should see QS running. If you want to test if launchd is managing your QS just quit it and you should see it running back again.

Setting up a Mac for Development

2012-01-09 00:00:00 +0000

Recently I got a new Mac and setting up for development is always fun, so I decided share some of the setups I do on a new Mac.

Is a Mac a good machine for development?

I think so and also know folks that think Linux is better, some are more use to Windows. In the end I think is not the OS which makes a better development machine (although there's space for some debate here) but how you deal with it. Take a look on The Productive Programmer by Neal Ford for some other tips.

What to keep in mind

As you setup a machine for development some things you should keep in mind. First is you want to lower the keyboard to mouse time. Taking your hands off the keyboard to point to a link in a webpage and then going back is time consuming, a developer goes from the IDE to the browser lots of time during the day.

Make the computer do what you're thinking without much effort, few keystrokes should take or do what you want. Going through menus, mouse, chasing windows on the desktop are also time consuming, unproductive and (for me) annoying.

If you do twice, think about how to do with one keystroke. If you caught yourself repeating something, think in a way to automate it (if makes sense). Apple has AppleScript and Automator which you can use to make simple scripts for mundane things.

Keep your projects and files organized, this goes for every developer. Not knowing where the files are leads you to browsing around to find them, avoid that. I have a Projects directory and an alias gp that goes to this folder, plus gp project takes me direct to the project folder. If you have some suggestion please do so (use the comments), I'm keen to know how you make your environment more smart and fast.

Now, some of the things I do on my Mac for development.

Disabling the Dashboard

I tried to use, make some meaningful use of it but never achieved, it just sit there. In Lion it turned to be a little more annoying as it shows on Mission Control view. Disabling the Dashboard is pretty easy, go to the terminal and issue this command:

defaults write com.apple.dashboard mcx-disabled -boolean YES

And restart the Dock:

killall Dock

Get iTerm 2

OS X has its Terminal application but iTerm 2 has some features that I really like, one is the Hotkey Window which you can trigger by a keyboard shortcut and it come and goes easily.

One tip for Lion users, if you use multi desktops set iTerm 2 to be available in All Desktops. You can achieve this by control + click on iTerm icon in the Dock and selecting Options - Assign To - All Desktops. This way when you invoke Hotkey Window it will not shift from desktop to desktop.

Get QuickSilver

QuickSilver is a very powerful application for action at distance. Some of you might know Alfred (check the app store), in Linux the Gnome Do and on Windows I recall Launchy.

On a Mac I prefer QuickSilver, mind that it is not just a Application launcher it can do a lot more than launch applications.

Get Dropbox

I use dropbox for a lot of file transferring and storage, specially when I change machines it helps to avoid the USB stick protocol.

Get Caffeine

A development machine that keeps dimming the monitor, going to screensaver and sleeping because you took more than five minutes to type doesn't help you. Caffeine when on don't let this happen, it's way more easy than changing Energy Saver configurations.

Get iStat Menus

This one is paid. I use to keep and eye on CPU, Memory, Hard drive access and Network usage. Also it can replace the battery level indicator and the date menu on the Mac. This is a habit that I've acquired working with Linux for long time and Bubblemon.

Get Xcode

Although I don't develop for Mac, Xcode is needed because it brings a lot of libraries for development that it's needed for a dev.

Get Home Brew

Or another package manager for Mac as MacPorts. You will need to install other packages into your Mac. Unless you're a fan of compiling everything you need something similar to apt-get to your Mac.

Get GleeBox

I can't image myself using a browser without GleeBox. It makes easier to find links, form items, issue bookmarkslets on a page and more. Is one more application to lower the keyboard to mouse time.

Turn on keyboard access to all controls

In the System Preferences go to Keyboard then keyboard shortcuts and turn All Controls. Now you can use tab to navigate through windows buttons and fields.

If you use Safari, turn "Tab" for links

It doesn't come with the "Tab" to links enabled by default, just to go to its preferences advanced and turn Press Tab to highlight each item on a webpage.

Get a real Text/Code editor

I use Vim and usually is the first thing that I install in any computer that I use. MacVim is the Vim for a Mac. Also I don't like the menu if you also don't like add to your ~/.gvimrc:

set guioptions=-T

Learn some common shortcuts of OS X

One of the good things about Apple software is it tries to be consistent over the keyboard shortcuts. The famous Command + , in almost every software that runs on a Mac takes you to the preferences of the app.

Another good one is Command + ? which takes you to the help menu, plus there's a search that can show you a entry on the menu. If the app doesn't not provide a shortcut for the action you're trying to make you can use this one to go there.