Arvind Kunday

3 Cool Leiningen Clojure libraries

arvind — Mon, 18 Apr 2016 15:58:21 +1000

lein clj-fmt Clojure formatting tool and allows you to define project standards. In short, this could potentially be a rubocop alternative for our clojure projects. Provides a way to automatically check and correct source files to thecorrect format. Tested against headphones project and works fine.To install this, add the following line to your profiles.clj

[lein-cljfmt "0.5.1"]

Look for yourself here -

https://github.com/weavejester/cljfmt

io.aviso/pretty Makes exception prettier and colourful (aka: similar to pry). Works will all our clojure projects, i’ve been using thisfor a while without any problems!To install this, add the following to your profiles.clj

 [io.aviso/pretty "0.1.20"]

Look for yourself here -

https://github.com/AvisoNovate/pretty

lein-ancientFind out-dated libraries in your project (ala - bundler-auto-update) runs a update from maven central and givesyou the latest compatible project.clj that works. You can also provide it your test command so you are sure atleastunit tests pass. To install this, add the following to your profiles.clj

[lein-ancient "0.6.8"]

https://github.com/xsc/lein-ancient

For the curious, my ~/.lein/profiles.clj looks like this

{:user {
        :doc {}
        :plugins [[lein-html5-docs "3.0.1"]
                  [lein-gorilla "0.3.5"]
                  [lein-bikeshed "0.2.0"]
                  [lein-kibit "0.1.2"]
                  [lein-cljfmt "0.5.1"]
                  [lein-cloverage "1.0.6"]
                  [org.clojars.scott/lein-nailgun "1.1.0"]
                  [io.aviso/pretty "0.1.20"]
                  [venantius/yagni "0.1.4"]
                  [lein-ancient "0.6.8"]
                  [repetition-hunter "1.0.0"]
                  [venantius/ultra "0.3.4"]
                  [jonase/eastwood "0.2.3"]
                  [lein-codox "0.9.0"]]
        :dependencies [[slamhound "1.3.1"]]}}

Why Software Estimates Don’t Work

arvind — Tue, 01 Sep 2015 20:39:30 +1000

”Just came across this wonderful article on why software estimates don’t work
http://www.innoarchitech.com/why-software-development-time-estimation-does-not-work-alternative-approaches/
TLDR:

Why Is Software Time Estimation Impossible

· The productivity and experience level of the engineer, particularly if multiple people are involved
· PTO, Late arrivals, early departures, sickness, etc.
· Unforeseen defect and customer requests, troubleshooting, challenges, system/environment issues, software/library issues, learning and ramp-up, design/architecture, required research, etc.
· The fact that software engineers notoriously underestimate
· Unforeseen issues with maintainability, architectural flaws/imperfections, scalability, performance, testability, etc.
· Time associated with spikes, R&D, design, architecture, mockup, prototype, POC, etc.
· Administrative work and non-engineering related requests

Coupled with the usage of Kanban:

In David Anderson’s book Kanban (read it), he discusses the concept of a target lead time coupled with a due-date performance metric as an alternative to estimation and target date commitment of individual development tasks. He also talks about time estimation as being considered a costly activity, while target date commitments are low-trust activities due to unkept promises stemming from inaccurate estimates.

Quoting further more:

Let me start by saying that I’d say that 99.99% of the time, projects are “behind” due to incorrect estimates and not because of product development issues. This is a result of everything discussed and is the key takeaway of this article.

Getting NAT working with Hyper-V

arvind — Tue, 01 Sep 2015 20:37:30 +1000

Recently I switched to Windows on my Surface Pro 3 full time, thanks to Windows 10, windows isn’t all that bad anymore. Given the extent of docker usage in our company, i needed docker working in my windows install. My experience with VirtualBox in Mac OSX is that it’s slow and using folder sharing brings the host machine to a crawl. So I decided to give hyper-v a try.

I set up two adapters like how I would on Virtualbox, one for NAT and one for internal network. Then i found out that NAT wasn’t supported so I used a External switch. The caveat was my home network supported multi-mac address dhcp, but the office router denied this.

Hyper-v doesn’t have a proper way to NAT Routing with Linux VM’s(or atleast not that I could find). I found a real cool way to fix this problem. I realised VMWare player supports NAT routing on Windows and given that the network adapters created by VMWare can be used by Hyper-V, I tried using that adapter in Hyper-V and alas it worked.

So here is how you get going.

1. Install VMWare player and then restart Windows(I don’t know if you need to, but hey I usually do)
2. These should create two adapters (VMWare Network vmnet1 & VMWare Network vmnet8)
3. Create a External Network in Hyper-V and you should see the vmnet8 adapter listed there.
4. Ensure VMWare NAT service is running in services.msc
5. Open the adapter properties and set connection sharing on.
6. Now in linux VM, set eth0 to resolve from dhcp and you should get a new IP Address.

Voila, internet should be working all fine and dandy. And I don’t need two adapters and one internal adapter just works great!

Introducing Cloudformer, gem for AWS Cloudformation

arvind — Fri, 25 Oct 2013 18:34:00 +1100

Cloudformer attempts to simplify AWS Cloudformation stack creation process in ruby projects by providing reusable rake tasks to perform common operations such as apply(create/update), delete, recreate on stack along with validations on templates. Task executions which enforce a stack change will wait until ROLLBACK/COMPLETE or DELETE is signalled on the stack (useful in continuous deployment environments to wait until deployment is successful). Refer github page for more information.

The list of rake tasks provided are:


rake apply           # Apply Stack with Cloudformation script and parameters(And wait till complete - supports updates)
rake delete          # Delete stack from CloudFormation(And wait till stack is complete)
rake events          # Get the recent events from the stack
rake outputs         # Get the outputs of stack
rake recreate        # Recreate stack(runs delete & apply)
rake status          # Get the deployed app status

Installation

Add this line to your application’s Gemfile:

gem 'cloudformer'

And then execute:

$ bundle

Or install it yourself as:

$ gem install cloudformer

AWS Environment configuration

Cloudformer depends on the aws-sdk gem to query AWS API. You will need to export AWS configuration to environment variables to your .bashrc/.bash_profile or your build server:

export AWS_ACCESS_KEY=your access key
export AWS_REGION=ap-southeast-2
export AWS_SECRET_ACCESS_KEY=your secret access key

Configuration

You can add cloudformer tasks to your project by adding the following to your rake file:

require 'cloudformer'
Cloudformer::Tasks.new("earmark") do |t|
  t.template = "cloudformation/cloudformation.json"
  t.parameters = parameters
end

where cloudformation/cloudformation.json is the stack json file and parameters is a hash of parameters used in the template. For a template which takes the following parameters:

"Parameters": {
  "PackageUrl": {
    "Type": "String"
  },
  "PackageVersion": {
    "Type": "String"
  }
}

the parameter hash(Ruby Object) would look like:

{
  "PackageUrl" => "http://localhost/app.rpm",
  "PackageVersion" => "123"  
}

If you have a template with no parameters, pass an empty hash {} instead.

You can find the open source code in github, any feedback/ pull requests would be awesome :)

Continuous deployment with cloud formation and cfn-hup

arvind — Wed, 23 Oct 2013 22:20:00 +1100

Cloud formation lets you create application stacks from scratch in EC2. In my project I use cloud formation to do continuous deploys using cfn-hup with every git push. If you haven’t used cloud formation before, there are plenty of tutorials on the web to get you started.

Structure:

The application is bundled into a RPM using FPM with all the application dependencies and post install scripts to be executed when the RPM is installed.

Packaging and CFN-Script:

As a part of the CI build, RPM of the app is created and uploaded to S3 tagged as ‘latest’ and the cfn-script downloads the RPM and installs it. An example looks like:

curl ‘package_url’ > /tmp/earmark-app.rpm
yum erase -y earmark-app
yum install /tmp/earmark-app.rpm
</code>
CFN-HUP Configuration:
<code>
[main]
stack=stack-name
region=ap-southeast-2

Waiting on Resource Condition:

You can use ApplicationWaitHandle & ApplicationWaitCondition to signal cloud formation deploy to wait until the installation of the package to complete. You can then use cfn-signal to indicate cloud formation stack that the particular action is complete. If cfn-signal isn’t received within a specified time period, the stack creation will fail. Here is an example WaitCondition:

“ApplicationWaitCondition” : {
    “Type” : “AWS::CloudFormation::WaitCondition”,
    “Properties” : {
    “Count” : “1”,
    “Handle” : {“Ref” : “ApplicationWaitHandle”},
    “Timeout” : “300”
    }
},
“ApplicationWaitHandle” : {
    “Type” : “AWS::CloudFormation::WaitConditionHandle”
}

You can then wait on this condition in your stack by using:

“LoadBalancer”: {
    “DependsOn”: “ApplicationWaitCondition”,
}

Signalling Cloud Formation:

You can then signal cloud formation that the install was successful using:

"/opt/aws/bin/cfn-signal -e 0 -r "Application Installation Complete." ‘", { "Ref" : "ApplicationWaitHandle" }, "’\n"

Limitations:

Cfn-hup checks for updates on the stack every 15 minutes by default and there is no way to alter the duration.
Cfn-hup cannot be used to signal stack wait during stack updates.
Cfn-hup failures aren’t propagated back to the stack events which could result in a false positivies even when update fails.

Futurespective: Influencing your team's future

arvind — Tue, 13 Aug 2013 23:23:00 +1000

–

“Our life is what our thoughts make it.” (Marcus Aurelius)

–

Any team practising(or claiming to do agile) is well aware of the positive outcomes that retrospectives bring into the project. Retrospectives helps us to refocus our priorities based on our past experiences, identifying potential areas of improvement to help us be more productive and increase delivered value. Retrospectives are derived from the parallel in the medical universe - “retrospective cohort study” where individual circumstances are assessed based on a particular outcome(in software engineering world: successful delivery of a project). Like any other representative study, it suffers from cognitive biases(selection and information bias). In software engineering, such biases results in the discussion being skewed towards perceived problems in the past few weeks(assuming retros happen that often). Typically, an action from a retrospective would be geared towards fixing a problem based on its occurrence in the past(you were trying to get more productive after all), which need not necessarily help the team achieve its goal over the next 6 months(or 2 years, depending on how far you look).

To quote Uncle Bob here, “Agile is not just processes and practices but a belief in achievement”. If agile is all about believing in achievement, we should be able to set goals as a team that we would like to achieve in the future and agree that reality is not necessarily all about harshness and things going wrong all the time. So, once in a while(either during the kick-off of a project or even in the middle of a project), it’s a good exercise to step forward and see where we would like to see ourselves as a team in the next 6 months(or 2 years) and what we would like to achieve in that time-frame. It helps the team to refocus team values and come to a agreement on a common goal and vision, helping us all to be more collaborative in how we can reach the ultimate end goal.

Enter futurespectives. A really retro, retrospective.

“Imagine yourselves in a kick-ass time machine and you decide to land in the future(6 months) and have a peek how the team is performing. You are intrigued to find out that the team has achieved all it’s set goals and objectives of the project and team member satisfaction is at it’s all time high. How did it happen? What were the things that the team did to reach nirvana?”

Given that there is no actual time machine(jeez spoilers) available, the only option you have is to walk backwards to find out how this parallel universe came into existence. Here are some of key things to think about:

What’s the *common goal* everyone was trying to move forward to?
What did we do *more*?
What did we do *less*?
What things were *we not sure* that we monitored.

Such a discussion into the perceived future makes few key assumptions:

We all agreed and understood that the team had common goals to achieve.
We were fully aware of the end goals of the project & team and were in the same page.

Though we cannot be really sure such a universe could exist, we all can agree which actions(success factors) will land us on the happy universe and what(failure factors) will land us on the not so happy parallel universe. Using the outcomes of this brainstorm, we could focus our energies to get our act together towards the happy universe.

Barebones XMonad config for Ubuntu

arvind — Wed, 14 Nov 2012 22:53:00 +1100

After countless hours of dabbling with Xmonad, a few really weird bugs, I got Xmonad to work nicely in Ubuntu 12.04 LTS. If you haven’t used/heard of Xmonad before you should check out Xmonad. It’s a simple and beautiful tiling manager written in Haskell and is configured in Haskell. I hope this post helps someone who gave it a go but backed of due to some of the weird issues relating to window focus in Java apps.

Why

Kickass multi-monitor support (xinerama) with per screen workspaces.
Simple, minimalistic and fast.
Kickass WM layouts that you can configure using Haskell.
Amazing keyboard shortcuts to re-align windows and navigate seamlessly.

Known Issues

Xmonad has some window focussing issues when it comes to Java apps (hack is available which does not work on JDK7)
Doesn’t work well with Unity 3d, so you might have to switch to Unity 2d. Alternatively if you are using classic gnome-shell, it should work well.

How to get it

If you are ubuntu you can use the traditional package manager aka apt-get to install.
```
sudo apt-get install xmonad
```
You should also get some application launcher(Unity dash works, but there are some performance issues). I use dmenu, it’s simplistic and works
```
sudo apt-get install dmenu
```

Gnome Session Config

Default install of Xmonad in Ubuntu 12.04 LTS adds the necessary session for Xmonad to work with Gnome with the exception of Default-window manager and DefaultProvider-panel. You can add them by editing the /usr/share/gnome-session/sessions/xmonad.session. You add them to session config which should look something like this.

[GNOME Session]
Name=Xmonad Unity-2D Desktop
RequiredComponents=gnome-settings-daemon;
RequiredProviders=windowmanager;panel;
DefaultProvider-windowmanager=xmonad; #this would have been missing
DefaultProvider-panel=unity-2d-panel; #this would have been missing

Issue with Intellij IDEA:

There is a known issue with XMonad not being able to interact with the GUI components when running under JAVA 7, so simplest would be to run under JAVA 6 and use ICCCMFocus#setWMName “LG3D” to ensure Idea to gain focus and play nicely with Xmonad(It’s included as part of the minimalistic xmonad config.)

Minimalistic Xmonad config

import XMonad
import XMonad.Util.Run
import XMonad.Util.EZConfig (additionalKeys)
import XMonad.Hooks.ManageDocks
import XMonad.Hooks.ICCCMFocus
import XMonad.Config.Gnome
import XMonad.Hooks.SetWMName
import XMonad.Actions.DynamicWorkspaces
myManageHook = composeAll (
  [ className =? "Unity-2d-panel" --> doIgnore
  , className =? "Unity-2d-launcher" --> doFloat
  , className =? "gnome-do" --> doFloat
  ])
main =
  xmonad $ gnomeConfig { modMask = mod4Mask
               , startupHook = setWMName "LG3D"
                       , manageHook = manageDocks <+> myManageHook <+> manageHook gnomeConfig
                       , layoutHook = avoidStruts $ layoutHook defaultConfig
                       , logHook = takeTopFocus
                       } `additionalKeys` [ ((mod4Mask, xK_d), spawn "dmenu_run -b")
                                          ]

XMonad Keymap cheat sheet

A vim style cheat sheet for Xmonad is available here.

Using OSX ? Brian McKenna (ex - Atlassian) seems to have patched Xmonad to work nicely under OSX.

Review/Opinion- 2016:Obama's America

arvind — Mon, 08 Oct 2012 22:02:00 +1100

It’s confirmed, we have proof, America will transform itself into a communist country by 2016 and Obama will serve as its most fond dictator - or so is the message of the documentary “Obama’s America 2016”. D Souza and his team attempts to walk through Obama’s past and show us how it influences his actions and policies. First forty minutes of the documentary explores young Obama’s life, influence of his absentee father and how it shaped his thoughts (base on his own book - Dreams from my Father).

He follows his life back to Kenya, Indonesia, about his father’s accidental death and about his mentor who supposedly was a pro-communist. He also points out that Obama Sr. wrote a pro-anticolonial paper and suggested increased taxes upto 100%. So? D Souza who pointed out the similarities with his childhood in India, however D Souza was able to change his opinion on pro-socialistic thought processes that he grew up with, but Obama was influenced by it, even though he hardly lived with his father. He also claims that Obama was pro-seperatist when he grew up in Hawaii, with practically no-evidence whatsoever. All this, is quite a logic stretch.

Apart from these unsubstantiated personal attacks, there are so many factual inaccuracies and careful omissions to construct the innuendo. D Souza suggests that Obama sides with radical Jihadists, that he did not help a friendly regime(Hosni Mubarak, the dictator) wants to reduce military spending(which is already 5 times as much as the next country China), reduce presence of military in Iraq and Afghanistan and hand over the power to rest of the world. Hold on, that pretty much sounds ridiculous. What do conservatives want? Wage war all the time and leave the world in a state of perpetual chaos?

Apart from policies in middle east, in the debate of national debt, he claims that Obama increased the national debt significantly, with the inner lining like, “Hey, America, Obama is trying to take all your money and give it to the poor”. Everybody who does their own fact checking and does not get all the news from fox know that when Obama came into power, bush left out a 1.2 Trillion dollar / year debt and a financial crisis on top of it. All based on the conservative policies that D Souza camp argue for.

I don’t understand the difficulty that the conservatives have in understanding Obama’s motives as he is well-meaning to the whole world except conservatives. All i can tell is America has never been or wanted to be imperialistic, pro-colonial(American Indepence from British, Hey?) and always has been a entitlement nation(Obama-care won’t make it any different. Trivia - this was first put forward by a conservative republican candidate).

P.S At-least Obama was not caught in a private dinner video talking how he will not be the president of those 53% Americans who want all these tax cuts. Romney did call 47% of America as lazy and he won’t care about them. Now it could just be gaffe, but I couldn’t resist bringing it up.

Links:

http://en.wikipedia.org/wiki/2016:_Obama’s_America

http://en.wikipedia.org/wiki/Dreams_from_My_Father

http://www.dailykos.net/archives/003167.html

http://en.wikipedia.org/wiki/Socialism_in_India

http://reason.com/archives/2006/06/06/the-rise-and-fall-of-indian-so

http://en.wikipedia.org/wiki/India_Unbound

http://en.wikipedia.org/wiki/Independence_Day_(United_States)

http://www.youtube.com/watch?v=PyJ_k2z-nTA

http://en.wikipedia.org/wiki/Hosni_Mubarak

http://en.wikipedia.org/wiki/List_of_countries_by_military_expenditures

Git Config Hacks

arvind — Fri, 17 Aug 2012 12:34:00 +1000

TL; DR - Skip to the end of the page for a quick look at my git config file

Basic hacks

Colors in Git:

Stating the obvious, git supports colors on any ANSI terminal. You can turn on git colors using

[color]
  branch = auto
  diff = auto
  status = auto
[color "branch"]
  current = yellow reverse
  local = yellow 
  remote = green
[color "diff"]
  meta = yellow bold
  frag = magenta bold
  old = red bold
  new = green bold
[color "status"]
  added = yellow
  changed = green
  untracked = cyan

This gives nice tangy colors to your git command outputs.

Change your pretty log format:

Spice up your git log by showing relative times like (4 hours ago than 2010-20-20 10:34 PM). Also the pretty format config below will squash it into a single nicely readable line

[format]
  pretty = format:%Cred%h%Creset - %Cgreen%an -%C(yellow)%d%Creset %s %Cgreen(%cr)%Creset
  date = relative

Handy hacks

Changes made yesterday:

Git has got a handy command to see the changes that were committed yesterday. You can see it by typing

git diff "@{yesterday}"

Files changed in a each commit:

Sometimes it’s handy to look at the files changed in each commit.

git whatchanged

Fairly Advanced hacks

For abbreviated commit log with branch and tag information add the following alias to your ~/.gitconfig

  log1 = log --pretty=oneline --abbrev-commit --decorate

To see the log with graph information and branches/tags where the commit is present, Add the folllowing alias to your ~/.gitconfig

lgn = "!f() { git log --graph --pretty=format:'%Cred%H%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' --date=relative $@ | git name-rev --stdin --name-only ; } ; f"

My DotFile

For those who are interested, here is my .gitconfig file

[color]
  branch = auto
  diff = auto
  status = auto
[color "branch"]
  current = yellow reverse
  local = yellow 
  remote = green
[color "diff"]
  meta = yellow bold
  frag = magenta bold
  old = red bold
  new = green bold
[color "status"]
  added = yellow
  changed = green
  untracked = cyan
[alias]
  y = diff "@{yesterday}"
  w = whatchanged
  c = commit
  st = status
  dc = diff --cached
  co = checkout
  lg = log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr)%Creset' --abbrev-commit --date=relative
  lgn = "!f() { git log --graph --pretty=format:'%Cred%H%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' --date=relative $@ | git name-rev --stdin --name-only ; } ; f"
  log1 = log --pretty=oneline --abbrev-commit --decorate

[format]
  pretty = format:%Cred%h%Creset - %Cgreen%an -%C(yellow)%d%Creset %s %Cgreen(%cr)%Creset
  date = relative

Long way to go!

arvind — Tue, 24 Jul 2012 16:05:00 +1000

Take the case of Shanti, a silver medal winner who protected her gender identity. She was stripped of her medals and laurels when she failed gender test in Doha. Unable to work elsewhere as a coach nor find herself a federal job, the medal winner now does hard labour making as much as 4$ a day. Worse part is the constant humiliation that she has been subject too just because of what she is, rather than her achievements.

The situation is not much different for other under privileged groups. The country does not count physically challenged in national census nor keeps track of their unemployment figures. While Western countries are fighting for equal marriage rights, countries like India even fail to recognize them. Only in 2009 was homosexual consensual sex was legalised by overturning the 150 year old law. Judge in his closing statement insisted that - “It cannot be forgotten that discrimination is antithesis of equality and that it is the recognition of equality which will foster the dignity of every individual.”

A country that boasts of a civilisation nurtured by harmony for at-least a few thousand years and whose citizens who are smug about the amazing achievements that india was capable of in the past, this is a shame. Is it difficult to embrace such people with love and recognise them for what they have achieved?

Inspired by the TOI article

Update:

Looks like somebody is listening. Also GAIL has announced 1 Lakh INR as a relief fund.

Will google drive succeed?

arvind — Fri, 27 Apr 2012 12:23:00 +1000

There has been lot of fuzz around the interwebs about the launch of google drive. Google is a late entrant into the online personal file syncing business while competitors like box.net and dropbox are well established and asserting dominance. Is it just another me-too product from google is it something that google values and cherishes?

Google has asserted in the past that the concept of files is dead and how it shuttered the near complete GDrive project which provided a tight integration between the local file system and online filesystem. What led google to change it’s minds about the file syncing market? As Sundar Pitchai points out, google drive is a natural evolution of google docs and collaboration being one of its key strengths.

By taking the concept of filesystem/drive to the App ecosystem, google is taking a bold step into a filesystem less future. Though it is counter intuitive, it enables a whole different class of applications that can be built apart from managing files. It lets you edit documents, presentations, diagrams and much more. All of it online and the user doesn’t have to care about where the files are stored as long as they can access them.

There are tons of apps in google chrome webstore which lets you open epubs, build mind mapping graphs, video editing(these are available only on the chrome webstore) and lot more. Sooner or later, there will be apps which cover most of the stuff we do, all online and google drive could be the center of all the data that’s being fed. It’s same as the promise of iCloud, except that iCloud is geared towards native apps than web apps.

Google drive is also important to google chrome os which forces the users to shuttle files between online apps back and forth due to a lack of a shared underlying file system. Now, google drive can act like the datasource(if the apps are integrated with google drive), people can create a document in one app and continue editing in another app seamlessly. Google executives have already confirmed that google drive will be merged into chrome os in later builds.

Let’s wait and watch how this plays out for google.

Update: Google drive arrives for chrome os in developer channel.

ruby debugger config

arvind — Mon, 23 Apr 2012 15:21:25 +1000

Whenever you notice a strange behaviour in the code or unable to assert the strange behaviour via a test, it’s helpful to launch the debugger and start inspecting the variables / try evaluating the piece of code. The REPL has so far been the best features offered by any good dynamic or functional language like ruby, haskell, clojure, python etc.

You could install ruby debugger for ruby 1.9 using the traditional gem command or adding it to the Gemfile depending on your environment.

gem install ruby-debug19

The debugger can then be invoked anywhere in the code by requiring the debugger.

require ‘ruby-debug/debugger’

The debug console is pretty advanced in terms of functionality and can be used in the same way as gdb. However, more often you use the debugger, typing eval every time gets tedious. You can avoid this by typing in

set autoeval

By doing so, every single line of code you type in the console will be immediately evaluated in the context of the currently executing code.

Also, when the debugger launches, it gives the next line of code it’s about to evaluate. In most cases, this should be fine but it helps to see few lines around the code. You can enable this by

set autolist

Once this is set, it will show the line surrounding the current code execution.

Or you can set this is the environment default by setting it up in .rdebugrc. This is how my .rdebugrc looks like this.

set autolist

set autoeval

set autoreload

For a full list of debug commands, look up the cheat sheet. Also check pivotal labs article on quickly setting up debugger for rails.

csshx - handy tool to ssh multiple boxes

arvind — Thu, 08 Mar 2012 19:11:00 +1100

One of the major pains in managing many servers is at times you would want to do some command like changing to a particular directory and tail a log file. It’s painful and time consuming having to ssh to each of these boxes manually and execute the commands over and over again. While there are tools like puppet and chef to automate software installations, it could be a over kill for such a simple situation.

I recently came across csshx (works only on mac) which perfectly solves this problem. csshx will attempt to create multiple ssh connections to list of servers you specify and give you a seperate terminal where you can key in commands which will be executed in all the other open windows(AppleScript magic).

Say for example you would like to connect to two servers 192.168.1.1 and 192.168.1.2 and tail a log file, you could do this using csshx with the following command,

csshx –login username 192.168.1.1 192.168.1.2

and voila, you have two windows with a login to each of these boxes and a red window to enter your commands. This will be copied across to all the other terminals. csshx also has an awesome set of keyboard shortcuts that you could master to get things by quickly.

You can install csshx using brew by typing the command:

brew install csshx

For people using other package managers refer their info site, or you can download the csshx script directly from google code.

Tweaks to default Hg config

arvind — Mon, 05 Mar 2012 13:17:00 +1100

The default mercurial installation does not support colours, pager or a progress bar by default. Unlike git, hg is highly customisable with python and a ton of extensions are available. Some of these extensions are bundled with mercurial and other nice ones can be found across the web.

Color Output:

Mercurial by default outputs all the diffs and logs using the default colour of the terminal. Mercurial standard library has support for colours by adding this line to .hgrc.

[extensions]

color=

Progress Bar:

Mercurial is generally fast but in certain cases(like a slow network) or a big repository, some of the commands take a considerable amount of time and mercurial command line does not produce any output until the action is complete. You can get a kickass progress bar by adding the following line in .hgrc.

[extensions]

progress=

Pager:

Most of the mercurial output is redirected to STDOUT without using less / more support. While this makes sense for default install, its painful when reading big diff logs or trying to traverse the hg graph log. You can add support to use less by adding this to your .hgrc.

[extensions]

pager=

[pager]

pager = LESS=‘FSRX’ less

ignore = version, help, update, serve, record

Git diff:

For people who are used to reading git diff formatted files, you can enable use git diff format by specifying the following to your .hgrc

[diff]

git = True

My .hgrc can be found in this gist.

Feature Toggles, Feature Branching and Branch by Abstraction

arvind — Mon, 27 Feb 2012 11:58:00 +1100

Note: For people who argue only one of it is good, please read this. Its written by Fredrick Brooks, one of the seminal guys in software engineering.

Purpose: Main aim of this article is to spur a thoughtful discussion around the topic than religious opinions or fear mongering about how other option is scary.

There is so much of debate around feature toggles and feature branches. Most of the existing literature seems to be around each one of it rather than contrasting them and giving a holistic picture. Some of the ones that do give like this seems to be missing the big picture. Here is my attack at the whole situation.

Why These?

Before we delve any deeper lets look at the various reasons around why we would need these techniques in first place.

1. Ability to release at any time. Delay in the development of a particular feature does not halt the entire production release cycle.

2. Ability for the business to hide features / AB test them / do a delayed release of a particular functionality / back out a particular feature.

For purposes of argument, this article will focus only on 1. Look here for debate around 2.

The Promise of Feature Toggles:

1. Continuously integrating changes in CI and if anything goes wrong could be reverted back.

2. Functionalities can be switched off if the integration does not go well.

3. No merge hell / Semantic merge conflicts

4. Code changes made are immediately in trunk available for the entire team for review.

To Note:

1. Feature Toggles aren’t necessarily used only in case of Large features, it could be used to hide certain functionalities that could be as simple as introducing a field to a form.

Arguments against Toggles:

1. Every Toggle added to the system needs to be checked with state on and off. So the more toggles in system would mean more combinations to test.

2. Some toggles cannot be clearly done in a clear way. In some places it could be as simple as a change in the injected object to if statements clobbered everywhere across the code.

3. Since feature toggles themselves involve code, we could accidentally be exposing something that’s potentially untested / that should not be displayed(How do you think the apple rumours(like this) are fuelled anyways?)

4. The longer the Toggle is existent in the system, the more expensive it is.

The Promise of Feature Branches:

1. Well tested code after QA’ing ends up in master.

2. Master can be deployed at any arbitrary moment.

3. Ability to track which changes went to each story/feature.

4. Ability to isolate noise when working on something big like a refactor(note: large refactoring is usually bad and should be done incrementally as much as possible).

Arguments against Branching:

1. Delays Testing(wait, am I getting wrong or qa’s don’t start testing until we call the story complete, in case of feature branching its after developer calls it complete.)

2. Merge Hell and Semantic Conflicts.

3. Integrating with CI less often.

Feature Bits

1. The arguments in feature bits of Facebook kind of invalidates the whole statement

Authoritative Articles about the subject.

http://techcrunch.com/2011/05/30/facebook-source-code/

https://www.facebook.com/video/video.php?v=10100259101684977&oid=9445547199&comments

Traveller's Ancedote, an open mail to redbus

arvind — Sat, 06 Nov 2010 19:41:15 +1100

I have been a regular user of redbus for the past one year and had some annoyances, issues with redbus. I decided to mail them and see what they feel about those issues. I got an impromptu reply from their head of marketing and I am publishing the email publicly just for the benefit of everyone.

Here is my mail followed by their reply.

Hi,

I have been a regular user of Rebus since last year and hats off to you guys,you guys provide one of the best online booking services I have ever used.As a software engineer myself, providing a usable end product to a customer is always the most difficult rather than the engineering that went to the product.

My experience with red bus usage has extremely good but some buses charge ridiculous amount of money and provide a really bad user experience. I have seen people being maltreated with rude words and the customer scolding red bus for selling shitty tickets. At times the buses are so bad that you don’t really know if you will really make it to your final destination.

Even though red bus has very little to do with any of the above, not everyone will understand that its a problem anyone will face when they try to run such a startup.

In summary the common problem I have faced with red bus so far is:

1. Quality of buses

2. Cost/Price of the tickets

I am interested to know how you guys are trying to mitigate the problem, also i have few suggestions

1. Introducing Red bus Certified Program ( Similiar to ratings but not aggregated from the users)

Redbus could tag a bunch of operators based on your own travel experiences and double check with the operators, based on the badge of the travel operator, someone can choose to buy the ticket or not. This is different from ratings as ratings data could be quite tampered by the operator itself. I have travelled in bad buses which had good ratings in redbus site.

2. Introducing something like a Customer grievance cell as a pilot who will present at some of major boarding points and classify the ratings of buses based on customer experiences. This gives a sense of good will to the customer as they have some one to go walk and talk in case he is irritated by the quality of operator during the beginning of travel itself.

Simple Improvements:

1. Changing values in drop downs don’t retain the sorted result, kind of annoying

2. Why should booked buses come on the list?(Should'nt the cancel option be more explicit or show a tool tip)

3. An option to say any seat when there are only few seats available as back and forth you don’t get to book any of the seats.

Other outstanding annoyances:

Getting an exception page when something goes wrong (/ error in some .aspx page) should not be a problem most of the time but might confuse naive users as redbus involves credit card transactions and as such.

- Arvind Kunday

And here is the reply I got from Mayank(Head of marketing)

Hi Arvind,

Thanks for all the insightful feedback and the compliments likewise.

We recognize the issues you’ve spoken of. The best thing that can be done in the situation we are in is to give our customers the freedom to rate buses and we can in turn make this information available to everyone. We can't certify any single bus because that means guaranteeing a level of service when it’s not in our control. We do plan to have a boarding point assistance -something that you mentioned in your mail.

So, to tackle the quality of the buses we have introduced two features - one is the user ratings and the other one is a -ve alert. The bad rated buses are highlighted in red. You can hide them by using a filter above the"ratings" tab. Even if you do select a poor rated bus, we’ll alert you that it’s been rated badly by other travelers and on the parameters it got rated badly. Hence we ensure that you are taking an informed decision at every step.

We are also in the process of weeding out the really bad service level operators.

On the pricing front, we have a feature that informs customers of high prices. If you search for Diwali tickets, you’ll notice that bus operators have hiked fares because of the demand and supply. Very similar to what you see in airlines. Pricing is neither regulated/dictated by redBus nor any government body and every bus operator does so at their own will. Sometimes for good reason and sometimes for profiteering. When they ply buses on some high demand days, they genuinely incur higher costs in terms of taxes so they pass on that additional cost to the customers. Of course, redBus has nothing to do with such pricing. In fact we mark it as a high price so that customers know so.

On the simple improvements you’ve suggested, point no. 2 - we show sold out buses because many of our customers would like to know if there actually is a 3 PM bus or not. Even if it’s sold out, they can use that information for future reference while planning their trip. It’s our responsibility to reflect the market, hence we show what’s sold out. You can ignore those. We will soon put a filter above the “status” tab to avoid annoyance from sold out buses.

The other improvements you’ve suggested are being worked on.

-Mayank

I am actually looking forward to meet those guys in person and try understand how they are solving these problems. Its quite challenging indeed.

Excerpts from Leslie Lamport's Interviews

arvind — Sat, 26 Dec 2009 03:22:36 +1100

Sources:

A Discussion with Leslie Lamport

An Interview with Leslie Lamport

1. When asked about some of the important problems in distributed systems.

On the theory side, recognizing a problem has been harder than solving it. Since you are asking about problems that are already recognized to be problems, the answer is probably none. On the practical side, I don’t think I have any more insight into that than most other people.

2. When asked about currently available formal methods are unable to prove correctness of large software systems, such as real operating systems. What is your advice to software developers for bridging this gap between algorithms (which can be analyzed) and full software systems?

People fiercely resist any effort to make them change what they do. Given how bad they are at writing programs, one might naively expect programmers to be eager to try new approaches. But human psychology doesn’t work that way, and instead programmers will find any excuse to dismiss an approach that would require them to learn something new. On the other hand, they are quick to embrace the latest fad (extreme programming, templates, etc.) that requires only superficial changes and allows them to continue doing things basically the same as before. In this context, it is only fair to mention that people working in the area of verification are no less human than programmers, and they also are very reluctant to change what they do just because it isn’t working.

3. The fundamental idea behind verification is that one should think about what a program is supposed to do before writing it. Thinking is a difficult process that requires a lot of effort. Write a book based on a selection of distorted anecdotes showing that instincts are superior to rational judgment and you get a best seller. Imagine how popular a book would be that urged people to engage in difficult study to develop their ability to think so they could rid themselves of the irrational and often destructive beliefs they now cherish. So, trying to get people to think is dangerous. Over the centuries, many have been killed in the attempt.

4. I think my most important contribution is my work on writing structured proofs, which is perhaps more relevant to mathematicians than computer scientists. In the unlikely event that I’m remembered 100 years from now, I expect it will be for that.

Ordered Pairs in Naive Set Theory

arvind — Sat, 26 Dec 2009 03:03:46 +1100

Ordered Pair

An ordered pair of a and b, with first co-ordinate a and second co-ordinate b is defined by,

(a,b) = {{a}, {a,b}}

The proof for the above ordered pair is given by,

If (a,b) and (x,y) are two ordered pairs, then (a,b) = (x,y) if and only if a=x and b=y.

Proof:

If in case, a = b, then, (a,b) => the singleton set of {{a}} or the singleton set of {{b}}.

Also, conversely, if (a,b) itself is a singleton set, results in {a} = {a,b}, so {b} ϵ {a,b} so which leads to a=b

On the other hand if we assume , (a,b) = (x,y) then both of them have to be singletons which can lead to

Singletons on both the sides, like {a} = {x}, so that a = x.

Also both the pairs should have atleast have one element which is not a singleton({a,b} and {x,y}), it results to {a,b} ={x,y}.

Now since we already know that b ϵ {a,b}, b is also a subset of {x,y}.

Since a=x, b also cannot be equal to x and hence, b = y.

T his completes the proof that, (a,b) = (x,y) If and only if a=x and b=y.

Cartesian product:

When A and B are two sets, we can define a set which contains the ordered pair (a,b) such that a is in A and b is in B.

Assuming above, we can conclude that {a} ϲ A and {b} c B, implies that {a,b} c A U B. Also, {a} C (A U B) and {b} C {AUB}, which implies, (a,b) C {A U B}.

There is an important observation from this, {a}, {b}, {a,b} are actually some of the members of the power set of the elements of the set {a,b}. So we can conclude that, (a,b) ϵ ‽(a,b). Its also obvious that ‽(a,b) has more elements than the ordered pair hence we can define a new set ‽(‽(a,b)) when a ϵ A and b ϵ B.

Applying the axiom of specification and axiom of extension we can obtain the set A X B which is the Cartesian product such that

A X B = {x:x=(a,b) for some a in A and some b in B}.

We can take forward the equation in a converse order for any set of ordered pairs there exists a set R such that, for sets A and B, R C (A X B).

Now, ({a},{a,b}) ϵ U R. Now, since {a,b} is one of the subsets of the ordered pair, its safe to assume that, {a,b} ϵ U R. This also assumes that a and b also belongs to UUR.

Projections

Its also often desirable to construct projections (like projection in relation algebra for example). By using the definitions,

A = { a: for some b in ((a,b) ϵ R)}

B = { b: for some a in ((a,b) ϵ R)}

Compliments in Set Theory

arvind — Tue, 22 Dec 2009 06:07:55 +1100

Relative Difference
If A and B are two sets, then the diﬀerence between A and B, the relative compliment of B in A, is the set A - B deﬁned by (note that = is just a abuse that
we use to ϵ)
A − B = xϵA : xϵ′ B
This does not necessarily mean B ⊂ A
But when the above condition holds true, then
A − B = A − (A ∩ B)
For the sake of simplicity, we assume a set U , the universe on which we can assume that all sets that we create are subsets of them.
Given a set A, we can deﬁne a set which is the absolute compliment of A, which is A’, which also provides that (A’)’ = A.
Also as with all ∅,
∅’ = E and E’ = ∅
Also, its easy to prove that,
A ∩ A′ = ∅
Also,
A ⊂ B if andonlyif , B′ ⊂ A′

Lets proceed to prove the above one. Lets assume the if part, if B′ ⊂ A′ : if xϵA, its self evident that xϵ′ A, and since B′ ⊂ A′ which clearly implies that , x ⊂ B′ which leaves us withxϵAandxϵB. Proceeding with the only if part, A ⊂ B if xϵB′ the clearly, xϵ′ B and since AϵB, its self evident that xϵ′ A, which proves the above one.
Moving forward with the De Morgans Laws,
(A ∪ B)′ = (A′ ∩ B′ )
Lets proceed to prove the above , lets assume xϵ left side then xϵ′ (A ∪ B) which
means, xϵ′ A and xϵ′ B which directly leads to xϵA and xϵB. Proceeding to the
right side, if xϵrightside then xϵ′ A and xϵ′ B which means that x is not present in
either A or B which leads to x being in A or B, hence, (A∪ B)′ = (A′ ∩ B′ ).

Hence, Demorgan Law have been proved. Similiarly, we can also prove that
(A ∩ B)′ = A′ ∩ B′
Principle of duality of Sets
One of the major implication of De morgans law is that, all theorems in Set
theory come in pairs.

So if an inclusion or an equation containing, unions, intersections and compli-
ments of a sub-set E, we replace al l the sub-sets by its compliments, inverse its
unions and intersections and reverse the inclusion , we arrive at a new theorem.

Symettric Diﬀerence
A + B = (A − B) ∪ (B − A)
This is the kin of XOR operation http : //en.wikipedia.org/wiki/Exclusivedisjunction
This is commutative and associative.
Why is theory of intersections only applicable to non ∅ sets ?
Lets just assume a set, where xϵX : xϵof ∅
Lets try to prove the other part, If this does not hold true, then ∅ must contain
a set X such that xϵX ′ , which of course is ridiculous, which proves the point.
To clear this, we can deﬁne a sub set in the Universe U, such that, ϱ(All subsets
of universe)
xϵU |xϵX f oreveryX inϱ

Asymptotic Notations

arvind — Sun, 20 Dec 2009 04:17:00 +1100

Why use Asymptotic analysis?

Since its a well known fact that for a small input set, all algorithms perform well regardless of the complexity, so to determine the complexity we need to analyze the algorithm for large values of n , typically n -> infinity. Also we need to bound the computed value between two values(a range) , or in short create a subset within which the execution time of the algorithm falls.

An asymptote of the curve is a line such that the distance between the curve and the line approaches zero as they tend to infinity. So, we use this to determine the function which is fixed while n-> infinity.

Asymptotic notations are calculated based on if the asymptotic lower and upper bounds can be calculated for a function properly.

Various Notations

Θ - when both asymptotic upper and lower bounds can be calculated and is tight.

θ - when only asymptotic upper bound can be calculated and is tight.

Ω - when only asymptotic lower bound can be calculated and is tight.

o - when the asymptotic upper bound can be calculated but cannot be tight.

ω -Only when the asymptotic lower bound can be calculated but cannot be tight.

Θ Notation:
For functions g(n) and f(n), we define a function, Θ(g(n)) such that,
Θ(g(n)) ϵ { f(n): c1,c2,n0, all non-negative constants for, 0 ≤ c1g(n) ≤ f(n) ≤ c2g(n)}

What it essentially means is that, by using the above description, we obtain a set of f(n)’s where f(n) lies between the upper and lower bounds, effectively sandwiching f(n) within c1g(n) and c2g(n).

So, we obtain a set f(n) ϵ Θ(n), for sake of convenience we use the notation, f(n) = Θ(n).

This makes, g(n) asymtotically tight bound for f(n) meaning that f(n) is equal to g(n) within a given constant factor C0.

Assumptions:

∀ f(n) ϵ Θ(n), f(n) has to be asymptotically nonnegative for sufficiently large values of n.

Also, it requires g(n) to be asymptotically nonnegative as otherwise, Θ(g(n)) = ∅

So we reach a point where, any function computed with Θ(n) has to be asymptotically nonnegative.

When computing Θ(n), lower-order terms of a function can be ignored when computing the Θ(n) as its insignificant for large values of n.
Similiarly, to compute the value of the constants c1 and c2 which define the bounds of the function, its enough if we pick a lower bound value lesser than the higher-order term and upper bound greater than the higher-order term.

For any asymptotically nonnegative function with a higher-order term 0 or if its constant we use , Θ(n0) = Θ(1). (though its sure that 1-> infinity). This notation is just used to make clarity that the function computes in constant amount of time.

θ notation:
When we have only an fixed asymptotic upper bound, we use θ(g(n)) (pronounce as big oh.)

For functions f(n) and g(n), we define a function θ(g(n)) such that,
θ(g(n)) ε {f(n), for constants , c and n0, such that 0 ≤ f(n) ≤ c.g(n) ∀ n ≥ n0}

θ(g(n)) is a subset of θ(g(n)), if it exists as Θ notation is stronger than θ notation.

-> is most commonly used as asymptotic upper bound characterizes the worst-case behavior of an algorithm.
-> we arrive at an θ notation by looking at the structure of a program

What we mean by saying a program is θ(n2) what we mean is for any value of n, above a specified n0, running time is bounded above the value of the function f(n).

Ω notation:
When we have only an fixed asymptotic lower bound, we use Ω(g(n)) (pronounced as big omega.)

For functions f(n) and g(n), we define a function Ω(g(n)) such that,
Ω(g(n)) ε {f(n), for constants, c and no, such that 0 c.g(n) ≤ f(n) for all n < n0 }

Following figure gives a clear understanding of the above notations:

Theorem 1:For functions f(n) and g(n), f(n) ε Θ(g(n)) if and only if f(n) ε θ(g(n)) and f(n) ε Ω(g(n)).o notation:The asymptotic upper bound provided by \theta notation may or may not be tight. For ex, 2n ε θ(n2) is not tightly bound. In such cases we use little-oh of g(n)For functions f(n) and g(n), we define a function o(g(n)) such that,o(g(n)) ε {f(n), for constants c and n0, 0 ≤ f(n) ≤ c.g(n) for n>n0 and n0 a constant greater than 0}ω notation:When we have have an asymptotic lower bound provided by ω notation may or may not be tight.In such case,For functions f(n) and g(n), we define a function ω(g(n)) such that,ω(g(n)) = { f(n), for constants, c and n0, 0 ≤ c.g(n) ≤ f(n)}