Ivar's rambling

Rusting, a quick dabble with the Rust language

2017-04-15T01:12:00.000+01:00

I have spent a few spare moments in the last week looking at Rust, a relative new language.

With the kids on Easter break and naturally requiring frequent attention, and a few brief actual paid client work, I have not had as long uninterrupted time focused on Rust as I would have liked. But I have managed to knock together some basic code.

Rustic Pizza

I created github.com/flurdy/rustic-pizza as my playground with rust and various web frameworks. It is very basic code and I expect I will be quite embarrassed by it in just a few months but it is a start. Rustic pizza will eventually contain several very basic web applications each in a different Rust web frameworks, all for ordering pizza. The pizzeria is a concept that I have created many times over in Java and Scala as an example for piece of framework or similar, some of which I have documented on my flurdy.com website.

Experience so far

So far I have found Rust interesting.

It perhaps use a bit too much of lazy abbreviated names for its keywords, primitives and core methods than I feel is reasonable, but that is me being pedantic. The ownership transfers, lifetimes and everything by reference is a bit of headfuck, but I think I got the hang of it. Rust does also have some of the monadic traits I am comfortable with from the Scala world, though its Futures seems to be somewhat in its infancy at the moment.

Install Rust

Rust can be installed in numerous ways, including via Brew for both Rust and Cargo, its build tool. But I recommend using Rustup, rustup.rs. Rustup lets you toggle between release and nightly version of Rust, and some frameworks require nightly builds.

Learning Rust

The main Rust website, rust-lang.org, is a good place to start to learn about Rust. Especially the "Rust book", https://doc.rust-lang.org/book/README.html, is a great source to learn about basic Rust concepts. Further reading at rustbyexample.com/index.html and aturon.github.io was very useful. As well as various blogs on Rust such as hoverbear.org and hermanradtke.com.

Once you start coding I also recommend the standard library documentation, doc.rust-lang.org/std/, to find out what methods are available, e.g. does option have map, and what is getOrElse called in rust (unwrap) etc.

Web frameworks

As I most of the time create web services or full webapps an important part of my interest is which web frameworks are available. Fortunately the Rust community also recognise the importance of this, so www.arewewebyet.org and github.com/flosse/rust-web-framework-comparison are great information on which frameworks and libraries are available with Rust.

This lead me to take a look at the newest kid on the block Rocket, rocket.rs. So far Rocket have worked very well for me.

For my Rustic Pizza I plan to also take a look at Iron, http://ironframework.io, as it is the most popular Rust web framework on github. Though it has not been updated as much lately compared to Rocket.

Hopefully if time allows I may get to look more in depth into Nickel, Conduit and Rustful as well.

Keep rusting?

Whether I will keep using Rust time will tell. Core Rust concepts such as ownership, references, memory management etc are not aspects I care to manually control and think about too much as I hope a compiler and garbage collector optimises and handles those for me. But I will keep at it so maybe Rust will accompany if not replace Scala as the main hammer in my toolbox some day.

Pair first, pair always, never occasionally

2017-03-10T01:42:00.001+00:00

Pair first

Think pair programming first by making it default for all tasks, not the optional occasional pairing which is how most places fake pairing.

Why pair

I have already ranted why pairing is so beneficial in "Pair up now". I have since worked in pair-always teams for another two years and really can’t fathom the waste of not pairing in any future projects.

Don’t do optional pairing

Many places, and I used to work at a few, do optional pairing. They say they pair occasionally as and when the team members think they want to.

I call this fake pairing. As it does not happen very often, and is more like as a second opinion or help when someone gets stuck on a task for a short time. That is not what pairing is. Please read the blog post mentioned above what pairing actually means. (code quality, knowledge share, low bus factor, best practices, egalitarian technology, feature speed)

Optional pairing lets people find excuses to not pair even when they think the tasks may be more suitable as a pairing tasks. Or wait until people are available which is wasteful as well as context switch cost for the other temporary pair member.

Optional pairing is better than no pairing, but only slightly.

Default to pair

By making it default you avoid discussion if one should pair or not. It is less ambiguous and the team just gets on with the tasks straight away. You realise most tasks can be paired on.

You also then start to see the big benefits of everyone pairing over time. You will start to notice higher quality code, that less bugs creep through and the code start to be quite lean as unnecessary code does not even get written. And the flow of tasks is steady as there is less bottlenecks of people with sole knowledge of some part of the system as the knowledge is spread across the team

You will have a team that knows each other better, that really work as a team. A team that has shared their own ideas and niche knowledge of technology. The code base now also looks more homogeneous and evolved as the team has improved their abilities unilaterally.

Non pair tasks

Sure as mentioned in my previous post there are still tasks that does not need to paired on. But there are fewer of these than you think. Mostly research tasks etc can be done alone. Or a pair may not sit together whilst ‘googling’ separately for a bit.

If someone is working solo on a task it is important that these tasks are minimised and time boxed as they often become time sinks and over complicated solutions thought of.

Odd number of team members

Often though through holiday, sickness, meetings, team rotation etc there is an odd number of members in the team making pairing everyone mathematically impossible.

You could triple up on one task. It can be done, though often end up with too many cooks in the kitchen. There may be suitably encapsulated sub tasks that the third triple member can work on his/her own and sync with the others frequently instead.

You could let the odd team member pick up a normal task as an individual. This can also work but I recommend highly against it. Many times I have seen this just result in the problems pairing was meant to solve. They get too entrenched in the task and it takes longer than planned. Or get defensive about it. Or it gets over-engineered. Or they take shortcuts or bad technology choices that would not have happened if paired on.

If there are some obvious non pair tasks to pick up then it might be a suitable filler until pair rotation happens again or a new member becomes available. Tasks such as perhaps some rare support or monitoring, or evaluate some low priority PRs.

Or they can work on some fun but non priority technical debt. Such as automating wall displays, or support scripts or tools to simplify/automate a process. It provides a nice break from the other normal tasks, and also hopefully helps the team in the future and reduces paper cuts.

Don’t chicken out

So do not chicken out by agreeing only to pair when the task's problem is suitable. Instead always plan to pair unless it is a rare hands off research task or similar.

ROC theorem: Readable, Optimised and Correct code. Pick two.

2016-11-22T14:35:00.003+00:00

ROC theorem

With database you have the well known CAP theorem. Consistency, Availability and Partition Tolerance. You can only have two. Databases have to make compromises between these pillars. You can not fully have all three.

With code you also have to decide on compromises between readable code, optimised code and correct code (ROC). And you can not have all three.

This often creates arguments between people on soap boxes from the various camps.

Correct code

Correct code, clever, terse, generic code that avoids handling a lot of edge cases. Often functional code that can be very elegant with little to no theoretic side effects. And easily composed as part of other code.

Correct code can be readable and fast, but also sometimes horrible to understand and very costly to train, write and maintain.

Optimised code

Optimised code, fast and scalable. No unnecessary cruft and takes short cuts to achieve the end results so performant from day one.

Scalable code, code you optimise to support horizontal scalable solutions, with little state and restartable.

May discourage typed system e.g. a message based Actor system, multiple layers of caching, or overuse of parallelised futures to avoid bottlenecks.

Optimised code can be "correct" code but often full of unclear and undocumented short cuts and frustratingly slow/buggy to develop.

Readable code

Readable code, simple to read and quick to understand, by people of different levels of skill. Easy to spot bugs and is maintainable by anyone. It is pragmatic in its approach and quick to develop. Can be terse if it is the most readable but often more verbose.

Readable can be "correct" as flow is easier to understand, but often not particularly performant and can be at more at risk of bugs due to more exposed code.

Not mutually exclusive

You can have all 3 pillars for some smaller sets of code. But not for whole code bases and at a cost for how much.

This is more about the priority and focus of the code you write.

Will others work on the code base, today, next week, next year? Then readable is important.
Is multiple people working on the same part of the code base? From different teams? With varied experience, or even just different locations? Then readable is important.
Is the solution used by millions? Does big O make any difference? Then consider optimised code. Note: very, very few companies/products actually need this.
Is a bottleneck been proven in production? Then optimised code is valid. But not necessarily across the code base.
Is the product business critical? With heavy integration dependants? Then correct code may be a priority.
Is the team highly skilled? Not that large, and low churn? Then correct code is an option.
Do you pair program 100%, from day 1 of onboarding? Then correct code is an option.

With unlimited time to implement and continuously heavy training, and therefore a lot money, you may achieve higher levels of 2 of them or even all 3. But that is not realistic. Pick your priorities. These are not mutually exclusive but they are at a cost of each other.

All the ROCs

You may detect my preferences. I prefer very simple and readable code, that is functional, and scalable. In that order.

I like that anyone can pick up and work on a task for most parts of a code base. I subscribe to the idea of frequent pair rotation across tasks and systems to make sure multiple people is aware of and had an input into any part of an architecture. That leads towards readable code so the overhead of swapping is low.

That a new member of our team or from another team in 6 months time can easily contribute to "our" code base for a small pull request without learning "our take" on category theory is valuable.

I prefer functional programming, with proper type checking, using functors and monads for composition. I like terse code that I can trust, but it must still be readable and maintainable by others than the original author(s).

So some overuse of higher kinded types, free monads, etc adds too much cruft for me, and risky recruitment demands. (At the moment, I am prone to evolve and may have completely changed my mind by the time you read this...)

Horizontally scalable, concurrent code is in the back of my mind of most of solutions. No state, using futures, REST principles etc are core to all my code.

But I detest premature optimisation. Only occasionally in my career have I had to modify any code to support some optimised flow. I do not work for Facebook/Twitter/Google (yet) but I have worked for financial, telcos, and games companies with enormous traffic, and still this was rarely a problem at code level.

I often spot potential optimisations and consciously say no, not yet, if it is not also the most readable and correct alternative. I even avoid parallelisation of futures if there is not yet any obvious need especially if it makes the code less readable.

Readable, Optimised and Correct code. ROC. Pick two. Or just rank them.

Contracting 101

2015-10-09T12:33:00.000+01:00

Seems a large number of people I meet are interested in becoming a contractor/consultant/freelancer. So I thought I better write a up what I did and recommend. Note this is very UK / London centric.

Information

For general information, tools, conversations and advice on contracting in the UK contractoruk.com and contractorcalculator.co.uk are very good sites. And a very dodgy contractor calculator... grid.flurdy.com/contractCalc

Forums

The forums at forums.contractoruk.com have been a great help to me. But prepare to be condescended and shot down if asking noob questions.

Association

IPSE, formerly known as PCG, the major contractor and freelancer association in the UK. ipse.co.uk

They will represent you if you have disputes with the HMRC of which they have a great track record. And they offer a range of services in general that will help you sleep at night. And discounts with Apple etc. Membership is a no brainer. You can use my referral code of IPSEMGM0921 if you want.

Insurance

You will need indemnity and public liability insurance and optionally others such as IR35 cover. QDos came recommended in the forums. Don’t under insure yourself but also don’t over insure. qdoscontractor.com

Umbrella

To soften the move to contracting world I joined an umbrella company initially. Basically the only paperwork you need to worry about then is the contract renewal. I was with contractorumbrella.com and they were great.

But eventually I took the next step to become a limited company as Umbrella taxes do add up.

Accountants

As a limited company you need an accountant. I joined Crunch as they have web site that automate almost everything on top of friendly advice. crunch.co.uk So far they have been great. Please use referral eraybyfl to receive Amazon vouchers.

Finding contracts

To find contracts is the major challenge. If you have a great network where you turn down work, then marvelous. Network at meetups and conferences. Verbally sell yourself to former colleagues as they may not be aware you could help them with a problem they have at work. Get business cards from moo.com

Otherwise global project sites such as Elance/ODesk/Upwork can turn up some work, just remember to not bankrupt yourself. I have actually had the best responses when I quoted a lot higher than anyone else.

Job sites such as jobserve.com, jobsite.co.uk, etc is the normal way to find contracts. Be aware a large chunk is simply cv fishing and not real job ads.

Contract Recruiters

If you can go direct then great, but most of the time you will have to deal with recruiters, and contract recruiters are much worse than normal recruiters, which does say something. Especially in a numbers game location such as London.

The major difference is for permie roles the recruiter will be better off if you agree a higher salary, whilst with contracts the recruiter/agency will be better off the lower they can get your rate.

And never ever believe them requiring two references. That is only so that they can extend their contact list.

Office Space

If you are not at a client I recommend using shared workspaces. I frequently use the 4 days a month for free IPSE members receive with Club Workspace club.workspacegroup.co.uk. I have also used neardesk.com and the more corporate regus.co.uk.

Good luck

Ivar Abrahamsen
Benevolent Dictator / Technical Architect Consultant

Eray by Flurdy Ltd
eray.uk | flurdy.com

Pair up now

2015-03-19T17:48:00.004+00:00

I fully believe we can deliver faster projects, with better code, exposing less risk to the company and in the end be more profitable if team members pair program.

The world was fine before

The world and businesses worked for thousands of years, even decades since computers was invented before agile methodologies took over how projects and companies was run. Projects was delivered and companies didn’t go bust.

Workable products got shipped before testing was done thoroughly, before unit testing was common and TDD was even heard of.

At the same time many teams worked well, developers wrote good code, meeting deadlines and shipping products before pair programming was a thing.

Sort of..

Were we also fine before cars replaced horses, computers replaced pen and paper, modern medicine replaced witch doctors?

Before Agile projects was delivered often late, not meeting the updated requirements, and all the other benefits with Agile. But some was delivered on time with features as desired and to budget. Just fewer and a lot slower.

And products were shipped with many bugs over and over again, with costly fixes over and over again.

Why not pair?

Double the resource cost

Many companies and managers can not see past the issue of using two people on one problem. Basically doubling the resourcing cost per story/task/whatever.

And some (not many) are aware of the teachings of the Mythical Man Month that adding more resources to a problem is not a linear reduction in time to finish the problem.

My space - my own thing

Many people and developers are also not interested in working with others on a task, especially all the time. They feel their private space violated and interfering with their work habits. I/they/most people do like to think and experiment and solve a problem on their own. And be trusted to do so.

Being forced to sit and watch, to talk to another person about every step of solving a problem is a big change in work habits and social skill requirements.

So why pair program?

3 reasons: Code Quality, Knowledge Share and Feature Speed.

Code quality

Mini plan

If two people sit together to work on a task they are quite likely to discuss the issue and basically perform a mini up-front architecture analysis of the problem. They might even apply TDD approach to solving it. In essence it should deliver a slightly more thought out solution than simply jumping in the deep end and hacking out a solution.

Human rubber duck

Explaining the problem and solution to a human rubber duck also unveils the root cause and the better solution much quicker.

Less rabbit holes

Two people are less likely though not guaranteed to go down a completely wrong rabbit hole in solving the problem. And at least quicker to realise that they are and pull out out sooner.

No short cuts

If you sit with someone you are less likely to cut corners and take short cuts in your approach and implementation. This is a very good reason for why pair programming results in better code quality. You are for example less likely to skip writing tests if someone sits with you.

Less mistakes

Simple mistakes and syntax errors are quickly spotted and corrected. Human compiler and typo spotter is often what happens but that is not really a bad thing, just don’t be offended by it.

Clean readable code

By having two people agree on the implementation then the readability and approach to the code should be cleaner and more maintainable as you both have to agree and understand the delivered implementation. This basically negates the need for extensive code review process later on as covered in my "Pair with people you like and code reviews with people you don't like" blog post

Ninja code

A lone developer often so called ninjas may also try to write smart code. Code that none else can understand nor maintain when that person leaves. A pair would prevent writing code that is unnecessarily smart. And share understanding of elegant code that is actually useful.

In the end per finished feature the code is on average more likely to be of higher quality, with more tests, more maintainable and properly solves the initial problem.

Knowledge share

One obvious benefit of pair programming is that now two people know everything about that task. And two or more now know about the product / system that the task involves.

No single point of failure

This reduces project, product and company risks and costs a lot. If one person goes on holiday or is sick there is no stoppage in working on that task or using that system as the other pair half can continue on his own.

And worse if a person leaves the company or is hit by a bus there is no panic as multiple people know about that system or feature. Ideally if a person is off for longer period or permanently a new pair is formed quickly to remove the new temporary single point of failure threat.

Best practices and showing scars

Another good knowledge share benefit of pairing up is that the two people start to share best practices in approaches to implementation or even develop new ones. They share stories of previous scars of bad practices. This should make them both better developers.

Rotate pairs - spread knowledge

If you also continuously or with some frequency rotate the members of each pair you gain even more knowledge share.

Product and system knowledge is now shared across the entire team. The risk of loosing knowledge is reduced further.

Best practices are spread around the team. Members pick up new skills and enjoy sharing their experiences. The knowledge and quality increases, and team morale should also increase.

Feature speed

Initially the feature tasks and project speed may be slower if your team members pair program. In a mathematical sense it could be halved as it ties up two members per task. And the benefits of pair programming are often more longer term.

But some implementations will go quicker as less likely to waste time on the wrong rabbit holes etc.

Speed per feature != initial implementation

In the longer term a great increase in benefit is visible. If you count bugs, planning, etc as part of the actual total feature speed, a pair programmed feature is much more likely to be of higher quality and less likely to come back over and over again with bugs. And it is bug fixes that take time.

The original implementation is often a low fraction of the total effort and time cost per feature. As Olaf describes in his "Pair programming Economics" post, implementation is perhaps only 10% of the time spent on a task.

Clean code

In addition with good clean implementation the code is more maintainable making future extension and fixes much easier and quicker.

Less procrastination

There will also be less procrastination with dealing with issues, with talking to other people, etc as two people should drive each other on to finish a task.

Less wasted time

When working on your own you may often spend a frustrating long time trying to figure out what is the actual problem, and how and where you should solve it. Sharing that with another person often cuts that waste down as it is quite likely one knows where to look already.

Quicker ramp up and less resource bottlenecks

If knowledge of a system of feature is already known by several team members if not all then starting a new related task can be ramped up quickly compared to starting on a feature or system you have never touched before. Shared knowledge also means there is no virtual bottleneck waiting for a person to be available to work on that issue if he is the only one that knows about it.

Not religious but no tricks

Fake agile

Many companies and teams say they are Agile but in reality they are not. “Being Agile” is not a black and white thing, it is a grey scale from non agile black to an unreachable white. Most projects are unfortunately quick dark grey even if they think and say they are Agile. Simply just using Atlassian’s Jira, having a daily standup, even some sort of iteration does not make you Agile. It is a lot more of fully adopting the practices and continuously try to come more agile.

Fake pair programming

The same goes with pair programming. Many say they do some sort of pair programming. Simple occasionally sitting together for a little while is not enough when most of the time they work by themselves.

You need to commit to pair programming being the default convention for every tasks, for everyone. Set adaptable conventions on how to organise pairing. And when exceptions are acceptable, not the other way round.

Shared responsibility

Agile processes means not delegating and assigning tasks to individuals, but in the end team members pull tasks from the queue and make themselves responsible for that tasks. If they pair program that assignment is for both in the pair not one person. If it goes well, both get praised, if there is a problem both stand responsible. Similar to how the team shares praise and criticism.

Personal space and time

Another important step is to allow for personal space and time away from the other half of the pair. I don’t believe 100% pairing from 9am to 5pm is a good thing, no need to be in each others arm pits all day. That would cause friction and stress on the team. There is no need to be religious about it. Its a convention not a rule.

Let people take breaks from each other. Much as Pomodoro Technique allows frequent breaks from work, pairs should take breaks from each other.

They could spend that time on researching the tasks, perform mini hacky experiments. Watching another person google around for an hour is not that productive nor fun. Allowing people to catch up with IM chats and emails, make personal calls etc will allow for a more happy and productive pair when they pair up again.

Non pair tasks

Some tasks does not need to be paired up for. Simple research, monitoring, tiny typo fixes can be done by just one person. Having a nice balance of the majority of the time is spent on important paired tasks with a few smaller non paired tasks is probably a good idea.

Effective onboarding

Taking on new team members, recruiting junior members still have a great benefit from pair programming. Sarah Mei write a great article on the benefits of "Pairing with Junior Developers". Leaving new team members to plod along fixing bugs is definitely the wrong way to onboard them.

Who to pair

Probably out of scope and can cover an entire blog post but a tip is to mix pairs. Senior with juniors. Front end with back end focused skills. Testers with developers if appropriate. And also seniors with seniors.

Remote challenge

If the team is distributed pair programming does become a challenge. However it can still be achieved. Pragmatic Bookshelf have a great book on “Remote pairing” by Joe Kutner. And a lot of tools and advice are available.

Not for everyone but for most

As I have shown here, I believe the pair programming is essential in a modern project. Sure, not pairing has worked for a long time and will still work. And for some specific teams and people it is still fine not to pair.

But pairing works better with most teams. It reduces risk, it builds team culture, increases actual velocity, makes the team enjoy their work and in the end deliver much better products.

Just approach pair programming with some common sense. It is not a factory, but don’t trick yourself into a halfway house either.

Evolve or wither slowly

2014-11-25T13:59:00.002+00:00

3 years ago (wow that went fast!) I wrote a blog post called Do not rewrite. The main points in it was to never rewrite whole applications. Never. Instead gradually rewrite one feature at the time whilst still delivering new business value.

I also wrote a more recent post about Paper cuts and broken windows. Which emphasises the importance of fixing problems straight away and not let them fester.

I want to extend these posts to include you should rewrite all the time, even when not immediately necessary.

Recap

Extracting the main points in the previous blogs why big rewrites are bad and continual rewrites are good, and to also sum the obvious why no writes at all are very bad.

Big rewrite disadvantages

Cost

No business feature - no value

Risk

Big deployments
Forgotten features

Likely to never finish
Not fully replacing old system

Support yet another system

Continual rewrite benefits

Reduced risk

Smaller delta

Quicker feedback
Modularising

Faster - less bottlenecks

Leaner

Remove features and bloat

Actually finishing

No rewrite disadvantages

Death by thousand paper cuts

Even the smallest change becomes slow and painful

Broken windows

People do not care if they introduce bugs or break other things

Staff exodus

Few wants to work with painful systems

Less obvious benefits

There are other perhaps unexpected but important benefits not raised in the previous post.

Business domain knowledge

Every time a refactor or minor rewrite takes place the knowledge of that part is refreshed and stay current in the company. If a system is not touched for a long while the knowledge of it may be forgotten especially if staff involved has moved on.

If a critical bug or urgent feature needs to be added to that system then the turn around is exponentially different between a recently updated system and one forgotten.

Technical knowledge

This is also true for the technical part of a system, the how, not just the what and why it does something. An old COBOL or complicated legacy custom build Java application will take a long time to work out compared to a contemporary tech stack application. You might even have to hire new staff or expensive contractors to fix it.

With up to date knowledge it will be a lot less risk that they might not do it properly.

Quick turnaround

If a new feature needs to be added then to any system then getting people ramped up and delivering it will be much quicker if it is on a contemporary and well known tech stack. If a new system needs to be integrated with existing systems then the API integration will be smoother and a lot quicker.

Technical migration

If a mass migration of a technology stack is needed, perhaps migrating from in house data warehouse to a public cloud based provider, or moving from monolithic to horizontally auto scaled instances, then having most systems on a contemporary and probably quite similar technology stack will speed any urgent migrations. This will avoid/reduce the need to for many deep cave explorations of old customised mystery legacy applications.

Staff retention

A very important reason to rewrite applications and features and in general keep technology up to date is staff retention. (Obviously a million other reasons exists for staff churn as well).

If you avoid death by a thousand paper cuts and broken windows staff will not mind working on the products. A negative culture will be less likely.

If you migrate to newer technologies and let people frequently learn new stuff they will be much more interested in the work and less likely to want to move elsewhere. (refer to my blog post Peak interest - the learning and sharing curve).

Recruitment

If word spreads that you keep an up to date and interesting technology stack then hiring new members of staff will be a lot easier, and you will hopefully attract more qualified candidates. On boarding will also be quick and people will be up to speed quicker with newer well known technology.

Lean technology - lean organisation

If the applications are continually refactored and evolved (as long as it is not done in a hacky ninja rock star development fashion) the architectures will become modularised, scalable and leaner in general.

Not necessarily a micro service architecture but a lean architecture that are more likely to be adaptable for the future.

Hopefully a lean architecture and highly skilled retained staff will also lead to / need a lean organisation so the business is more likely to succeed.

Rewrite when not needed

I would also suggest people and organisations refactor and rewrite when they do not see the obvious reasons for it.

I do not entirely mean rewriting applications on a whim when there is no features to be added or bugs to be fixed, after all I do emphasise the need for delivering business value along with all rewrites.

But I do think even when the obvious pain is not there that you should try to refactor and update applications. For example if the application is just one or two version behind the latest but still seems fine then still update it. It will be less painful than when it next time is 3-4 version behind with a bigger delta of change.

Rewrite encapsulation

One type of no business value rewrite I would suggest is a good idea is to encapsulate legacy systems API as soon as you come across it.

Even if you are not changing a legacy system but merely reading data from for example, then adding a contemporary facade to it straight away will be of value in the future when you do need to update it.

Experiment and rewrite experiments

Another important staff retention technique on top generally keep up to date with technology is to let them experiment with new technology.

I would avoid core systems, especially customer facing systems, but internal tools is prime candidates for field testing technology.

This will also lead to discovering technology that you can use in other applications if proven useful that otherwise would have been missed or delayed.

Naturally these experimental applications should also not be abandonware and be rewritten as often as other systems. Though they will probably always be good candidates for next set of technology experimentations.

Rewrite exploration and euthanasia

Certain system never comes involved in new features so will not be included in a normal rewrite. These will consciously have to be found and rewritten without business value. Though probably just as a contemporary facade initially.

Leaving these as abandonware is not a good idea. Either kill them or update them.

Summary

The main points was already covered in my previous blogs, but I hope people see the value of rewriting frequently to especially keep staff, organisational architecture and company in general up to date for whatever happens in the future.

A utopian expectation of all applications being up to date is too much to hope for. At the other end where companies that rarely and/or minimally update their applications, they are doomed to fail. The grey scale in between decide whether companies will wither and eventually die or survive.

Don't download the internet. Share Maven and Ivy repositories with Docker containers

2014-11-21T14:09:00.001+00:00

Downloading the internet. A term commonly used when building Maven projects. It is due to downloading your applications dependencies and their transitive dependencies. With a normal application those dependencies can add up to quite a few jars and a lot of megabytes to download. Especially on a 'clean' machine without cached dependencies in your local repository.

Running a new container in Docker sometimes feels the same as every image layer it is built on top of also has to be downloaded. Eventually these are also cached in the Docker cache.

If you then have a Maven based application in Docker you have the perfect storm of bandwidth hogging. Especially as your Docker image's Maven configuration will by default not reuse any cached dependencies and instead always download everything from Maven Central. And for each new build or launch it will re-download the internet as it knows of no local cached dependencies.

Work around, not solution

My work around is to mount my local host repositories for Maven, and for its derivative Ivy, as data volumes for the Docker container. This then avoids re-downloading the internet on every further build and launch and will also reuse dependencies I most likely have already downloaded on the host.

It is not the best solution as your images should ideally not be influenced by what you have on your host machine, but it saves a lot of time, and a lot of grief.

Vagrant

When I run my docker containers inside a Vagrant instance I first mount my host repositories by adding these 2 lines to the Vagrantfile:

config.vm.synced_folder "/Users/myusername/.m2", "/home/vagrant/.m2"

config.vm.synced_folder "/Users/myusername/.ivy2", "/home/vagrant/.ivy2"

Boot2docker

Since Docker 1.3 the OSX /Users path has been by default shared with the boot2docker VM on the same path. So to share the maven repositories you need to link the /Users path to your docker user home folder.

boot2docker ssh;

ln -s /Users/mysername/.m2 /home/docker/.m2;

ln -s /Users/mysername/.ivy2 /home/docker/.ivy2

Data volume container

For easy sharing these folders between many docker containers I mount them as a data volume container and naming it ‘maven’.

docker run -d -P --name maven \

-v ~/.m2:/root/.m2:rw \
-v ~/.ivy2:/root/.ivy2:rw ubuntu

I am basing it on the basic Ubuntu image, and it will stop immediately as no process is running. That is fine, the mounted volumes will still work.

Alternatively instead of mounting the host folders you can have a persistent container with these folders exposed as VOLUME in the Dockerfile so it is shared in a similar manner.

Launch container

docker run -d --volumes-from maven \
yourapplicationimage

Obviously you probably have other options on your docker launch, but the important bit here is the ‘--volumes-from maven’ which maps all the volumes from the data volume container called maven into this container.

Now in theory all containers with Maven and Ivy based build tools such as SBT should first look at the mounted Maven and Ivy repositories for their dependencies.

Build problem

Unfortunately if you build new images frequently you will have the same problem still as the above solution is only for running containers not building. During the build stage Docker does not allow you mount any volumes. This is so it is reproducible anywhere.

In your Dockerfile you can ADD or COPY whole repositories into your image, but that will make it very bloated with a lot of irrelevant dependencies unless you somehow construct and maintain a perfect repository. (Ps. don’t do this).

Maven repository manager

One solution that makes sense in general is to add you companies Maven repository manager’s public Maven Central mirror to the image. That way at least you will only download the intranet, not the internet.

For a Maven build add a settings.xml file to your image folder with at least these settings if using Nexus:

<mirrors>
<mirror>
  <id>Nexus</id>
  <name>Nexus Public Mirror</name>
  <url>http://nexusmachine/nexus/content/groups/public</url>
  <mirrorOf>central</mirrorOf>
</mirror>
</mirrors>

Then add this to your Dockerfile:

ADD settings.xml /root/.m2/settings.xml

Note this will be overwritten if you later during the run stage mount .m2 folder on top of it, but for most that is fine as the .m2 folder usually contain a relevant settings.xml file anyway.

For a SBT build add this repositories file to your image folder with this content:

[repositories]

local

maven-local

company-repo: http://nexusmachine/nexus/content/groups/public

scala-tools-releases

maven-central

Then add this to your Dockerfile:

ADD repositories /root/.sbt/repositories

Repository manager container

A further solution is to run a repository manager as another container and have all Maven/SBT builds refer to it instead. This avoids the involvement of the host computer yet saves any network traffic as everything is localhost.

Pull down something like https://registry.hub.docker.com/u/mattgruter/artifactory/ .

docker pull mattgruter/artifactory

Configure then run and name it:

docker run -d --name artifactory \
mattgruter/artifactory

Modify the above settings.xml and repositories to refer to you locally linked artifactory name as url instead, e.g this one liner:

company-ivy-repo: http://artifactory/artifactory/repo/,
[organization]/[module]/[revision]/[type]s/[artifact](-[classifier]).[ext]
company-repo: http://artifactory/artifactory/repo/

(The example above added it as an Ivy repository which Artifactory supports).

Hopefully these tips will avoid downloading the internet too often and save a few grey hairs.

Learning whilst writing, and relearn later

2014-11-13T16:31:00.000+00:00

Martin Fowler recently made a comment that he writes books so that he can learn about a subject.

That resonated well with me, as whilst I don’t write books I do write technical howto documents on my website and my blog. And it is true I mostly do write these documents as I am learning the topic myself.

The main reason I write and share these documents is still to help others. Hopefully someone else will be able to stand on my shoulders and have an easier time learning the same thing. I certainly use a mixture of other’s docs as I learn stuff and hopefully I reference them appropriately in my docs.

Focus and relearn

But I write howtos also to help myself. By documenting each step I keep focus on learning it properly. It also encourages me to achieve certain meaningful levels before I bounce on to the next shiny thing.

An additional unintentional benefit to me is that in e.g. 6 months, 2 years or even just a few weeks later when I need to use/learn the same technology, I have ready made revision notes for me to get up to speed instantly.

Accidental expert

People mistakenly/naïvely think I am expert on the domain of each howto I write.

On some topics covered in a few older docs I may be an experienced fountain of knowledge…, but with most I only scratched the surface. Quite a few I never touched again and have more or less completely wiped any knowledge from my little mind.

So writing these docs can be a little bit of a curse. Of my top five most visited blog posts three of them are related to Hibernate from years ago. I am not an Hibernate expert nor do I want to be, but this could trick people into assuming that I am. My most visited (by a country mile) howto doc is about setting up an email server. I know a bit about the subject but it is not my job nor interest, I merely wrote a good howto in 2003.

But in general sharing is worth it. The amount of thank notes I have received and general good feeling I get from these are gold.

Evangelical

Another benefit to myself is that I now have reference points when I evangelise about a certain topic. I can refer to my own work and words instead of soon to be forgotten spoken words.

I also starting to be confident enough to present, submit CFPs and in general talk about subjects which are often built on documents and blog posts I have previously written.

Dockerise it all - containerised addiction

2014-11-13T10:59:00.000+00:00

TL;DR: CDD - Container Driven Development

(CDL - Container Driven Life)

Occasionally you come across new interesting technology that is easy to use, and suddenly a veil is unveiled and you become aware of so many opportunities and potential. Maybe it was the same the first time you discovered programming, or OO or FP programming, message queues or NoSQL databases, or distributed source control, or moving from IDE only to proper reproducible build tools (Ant, Maven, Gradle, SBT etc). With certain tools it becomes your favourite hammer that you carry everywhere searching for nails to hit it with.

Docker is such a revelation, at least for me. And I am hammering a wide variety of nails with it. And with the hammer-nail pattern/anti-pattern some of those nails really didn’t need hammering but it was fun.

Docker history

Docker was only revealed in March 2013 by Solomon Hykes and other people at dotCloud . It got a lot of publicity in the hacker news section of the world as people could see the possibly potential in this new tangent of virtualisation. It baked and matured through 2013 as people occasionally showed examples of applying it to their workflows, but still it was too rough for most to use.

But during 2014 especially since the summer the publicity and widespread use of Docker has exploded. Downloads from Docker’s registry hub of ready made images has exponentially rocketed sine the summer of 2014 by 1387%! Now the majority of people I talk to or follow on twitter mention Docker and show examples of how they use it. Conferences and tech news are flooded with new ways to use Docker. And projects that extend or are based on Docker are multiplying all the time.

As someone who has always been interested in virtualisation, automation, reproducibility, build tools and provisioning, introducing containers like Docker has been a welcomed evolution and a revelation. Being already heavily invested in the use of Vagrant for development and systems the migration to/ combination with Docker has been smooth.

Docker addiction step 1. Exploration

First steps are usually to try the easy hello-world-ish examples. Downloading an Ubuntu image and launching bash inside a Docker container. Basically exploring Docker and its commands, which is what I cover in my how to install, basic use patterns and many handy tips with Docker howto.

Docker addiction step 2. Imagination

My howto document also takes you into step 2 which is more about creating your own basic images. Creating a simple Apache or Nginx container with a simple volumised website. Creating images ready with a Java SDK or node.js etc. I have made some simple Dockerfiles of these type of images available at github.com/flurdy/Dockerfiles.

Then you usually also wrap a simple application of your own in an image and running it on other computers. You start to understand how and where Docker can be used.

Docker addiction step 3. Diversity

Once you got one application inside a container you start to experiment with other applications. Might take a look at other types of applications and more likely support applications such as databases inside a container.

You might start to link containers to each other such as an application to a database container and some of the other handy Docker tips in my howto.

Docker addiction step 4. Proliferation

Now you have tried a mixture of applications with Docker you might spread the use to even more core applications and more. You start to link multiple containerised applications to each other.

Replacing third party integration and old legacy applications with a container image, wiring up and switching between different versions of an application via container linking, etc.

As you start to basically recreate environments using Docker containers you might start to facilitate promotion between staging environments only using container images. And eventually production is containerised.

You are probably already sharing and discovering images at work and on the internet. Using the public Docker registry hub and/or internally with tools such as Quay.io

Docker addiction step 5. Eccentricity

When comfortable with writing Dockerfiles and fully aware of the Docker way of layers and process then you start to experiment with more unconventional Docker images.

You might create Android SDK in an image, ready made SSH tunnels to apps or databases in another. You add tools such as a local maven repository manager, local DNS servers, mail servers, etc. into containers. You realise desktop applications can actually be containerised.

Eventually tools to facilitate easier and more manageable Docker life such as Fig, Panamax, Shipyard, Dokku, etc. are maybe among your toolset.

Docker addiction step 6. Ubiquitous

By now all your core applications are run/runnable inside a container.

They link to databases, message queues, 3rd party adapters, and other smaller applications that are all also inside containers.

Your testing environments, staging environments, production environment, development environment and REPL are all running inside containers.

When you create a new application or touch an old legacy application your first step is to write a Dockerfile and containerise the application. You design applications and processes from the ground up to be containerised.

This is what I call CDD - Container Driven Development.

Possibly by now you are convinced everything belongs inside a container, so a better name is perhaps CDL- Container Driven Life.

Peak Interest: The learning and sharing curve

2014-05-14T10:10:00.001+01:00

The interest curve of learning and sharing at work and its eventual peak are partly responsible for higher turnover of staff and reduced productivity if not managed well.

When you learn and share what you know at work you enjoy it. You also contribute to a very productive environment. However when you no longer learn nor share then you are not enjoying it anymore and become negative and a liability. And then leave.

About a year ago I read an article that explains everything in this blog post in a much clearer and scientifically researched way with references to specific companies. I haven’t been able to find it again, but when I do I will add a link and note to read it instead.

The problem

When you start a new role you have lots to learn. (Although sometimes you may have too much to learn and you are simply out of your depth and not able to keep you head above water).

At the same time you may have a lot experience and new thinking to share with the others in your team and company.

This learning and sharing is usually very enjoyable and rewarding, and you quickly become more and more productive in your role.

This productivity over time can simplistically be viewed in a graph like this:

However after a while in a role you learn less and less. You have absorbed the majority of the domain knowledge, you know all the people involved, and you are familiar with most of the technology stack (if in a similar line of work as me).

Your team members and colleagues have already heard most of your experiences and you are starting to even repeat yourself. The rewards for sharing are reduced. So your productivity could be viewed as flattening out:

However what tends to happen is that you enjoy your role less due to not learning and not sharing as much as before, and over time you start to not like your role and become quite negative.

This in turn start to have a negative effect on your productivity over time:

This is what I call peak interest.

The consequence

With this negative productivity trend you may start to become negative person at work. This might then spread to your colleague and the whole team starts to suffer.

In the end you probably will change jobs. So the company will have to recommence the expensive recruitment process to replace you and will loose all your knowledge.

Even if you leave quickly it affects the remaining employees negatively. Naturally they might be upset if you leave on a personal level, but also upset with the company for letting you leave, jealous if you leave for a better package elsewhere, depressed that your knowledge and capacity is gone and they have to figure out how to compensate. If many employees leave quickly all the time then the company will have a bad negative culture.

Known problem

This is in fact a known problem, and whilst I can not refer to any scientific research, I have read about this issue many times.

Major companies are fully aware of these peaks and take many actions to delay and counteract this curve for their employees.

Not applicable to all

The magnitude and curve gradient are definitely different for everyone and for every situation. And not quite as smooth. Some are quicker learners, some in a difficult role, some join a company where there is little to share, etc.

And then there are some sets of people that this hardly applies to at all. The "lifers". Not meant as a derogatory word but I fail to find a better description of this common characteristics of a set of people.

These are the people that rarely changes jobs, that are not that interested in learning anything new, nor as keen share what they know. They just get on with their job, and stay there for a long while.

Their interest curve whilst perhaps not entirely linear are certainly very much less curved. And may never peak. However my unscientific opinion is that they also make a much less productive contribution to the company over all.

Every company will have some lifers. There are many roles where this is a good trait: in jobs that are very 9-17 and never really change.

However it can be frustrating if you mix lifers and “peakers” in the same team. Developers, artists, any creative or knowledge workers need to work together, learn together and share together. If some are then learning and producing at different speeds you will have conflicts.

That being said, mixing peakers with "ninjas" is a recipe for disaster as well. Ninjas and rockstars do not have an interest curve, they have an interest seismograph. But that is for a different blog post.

Solutions

Companies do try to resist and reduce the effect of these peaks. They try to stretch out the length of the peak and create further peaks.

They can ensure the work environment is nice and comfortable. That bureaucracies are not frustrating, latest tools are available and most of all avoid stressful tasks, overtime and other counter productive situations. This will make the peak last longer and post peak decline slower.

Even then the major peak will eventually happen. Companies can then create further peaks by changing team focus, changing project, changing people, changing teams or changing role type completely. Basically avoid status quo for too long.

What they can then achieve is multiple peaks that keep you at a company for much longer.

So this avoids you having to leave the company to find another enjoyable role. However I do think eventually that undulated curve will start to fade and you might leave your role in the end.

The critical bit is then though that instead of leaving after for example 1 to 2 years you might stay for 5 or 8 years. (Time being relative to personality, industry, culture, etc).

Solution faults

Even if the company is aware of interest peaks, and want to mitigate their effects they might not always be able to.

In a small company there might not be any other team or project to move to. In a big company you might be insignificant or just too many to cater for so you are not moved around as often as needed.

Due to deadlines, business restrictions etc there might not be possible to move at the appropriate times.

Different people will have different gradient of curve and peak and companies will struggle to individually predict this.

These might lead to the negative cycle setting root and you resign.

Other issues are of course that changing everything around all the time is not appropriate either.

It may not be good to do it too often, to do it to too many people at once, to unevenly change roles for some compared to others.

People and teams do need some sort of stability so constant change is not good either.

Personal experience

I know that I go through these interest curves, and I have especially been taking notice of mine and others lately.

Whilst I do not like to change projects too often as I like to get a thorough knowledge of the domain and project, I know I also need new challenges once I have mastered a domain, technology and/or team to a sufficient level.

When I have worked with startups the curve has always been very steep, the peaks very high, but also the peaks come quickly and the need to change status quo come sooner.

When I worked with large enterprises the gradient was less steep as there was many things that took a long while to do, the technology was less challenging and pace slower. However I also reached the peaks later. But I think the productivity magnitude was much less.

As a consultant these peaks are not so much of an issue due to the normal change of clients and assignments. Some long term clients have even been helpful by moving me to new projects with them in the same assignment although perhaps less on purpose for my benefit, more for their business reasons.

My beautiful better half reminded my last year when I was moaning about being bored at work. She pointed out that exactly a year before I had told her that I really need to step up my game as there was so much to learn in my new role and that I was loving it. I had clearly reached and passed my peak for that role.

Conclusion

Peak interest happens to most people and companies. It is nothing to be ashamed about. It is in fact a nice and beneficial trait which needs to be managed.

Company need to embrace interest peaks. Let people move around the company. No questions asked. Encourage it. Better than haemorrhaging good employees, or let a bad vibe fester.

Avoid status quo. For instance make sure technology stack and skills keep continuously evolving even if not always great business value. Staff will stay interested and not need to leave find new challenges.

As an individual if you start to get bored in role, quickly request a change in your role. New project, new technology, new people, new role, whatever it is that will be a nice new challenge.

Disclaimer

Of course productivity and reasons people leave jobs are very much affected by other factors. I do believe however that peak interest of learning and sharing does contribute to both enough for it to be a significant factor.

I do not have any specific scientific research (atm), just a personal opinion backed by reading over time of similar material. If you agree, disagree or have have any research to back it up then feel free to comment or link back.

Paper Cuts and Broken Windows

2014-02-01T13:56:00.002+00:00

It is the little things that matter

We are often reminded of the facts of keeping focus, deliver a minimum viable product, lean startup principles, only invest in what returns business value and so on. Whilst all these are definitely true, but to some extent they are also false.

Delivering business value, concentrating on one task is a virtue I do insist on, however we should not abandon all other tasks and common sense. Secondly it is actually very important we do the low priority and the little ROI tasks as well. Why? Because of Paper Cuts and Broken Windows.

Note: This may apply to other professions and aspects of life in general but I am only thinking of IT development projects.

TL;DR: Fix your bugs asap, evolve systems continually, keep code clean and take pride in it

The problem: Paper Cuts

Many little bugs that in total hurts a lot.

A paper cut is a simple bug or an outstanding minor feature that is deemed not important enough to fix/implement right now. Some may have best intentions of implementing it soon/eventually but we all know from experience that it is most likely to never happen.

Over time these paper cuts adds up, soon everything you touch is already full of bugs, painful procedural steps or legacy code, and in the end your team and project may start to venture close to death by a thousand paper cuts or at least feel like it.

The problem: Broken Windows

If something is already broken, it does not matter if you break something else

Similar to paper cuts, broken windows is when a system/project/code base has some or many things wrong with it so that the current maintainers no longer feel any pride in it. It is then simple to not fix things, to ignore bugs, poor code etc.

If a continuous integration server notification of a broken build or a Nagios alert occurs for that system, people are more concerned about getting rid of the notification than actually fixing the root cause.

In an application suffering from broken windows any new feature will not be done to their best possible standard or refactored for cleaner more maintainable code. It will not be fully tested, in fact may introduce a few known bugs and limitations that will be ignored.

Also if some epics are never completely finished, it may leave an impression that you do not need to finish stories and epics properly and as such broken windows of missing features start to be an accepted practice.

Combined problem

Essential paper cuts and broken windows are about the same. Paper cuts are how much it is hurting you by slowing you down and making people resistant to do anything with it. Broken windows is about lack of pride and willpower to make an application better as it feels already partially broken.

Or worse, if your customers start to have the same paper cuts and broken windows experience then they will simply stop using the application or even your company.

Derivative problem: The costly rewrite

A system suffering from Paper Cuts & Broken Windows will eventually be so despised by the developers that they will campaign for it to be rewritten from scratch. Which leads to a long period of delivering no business value, just pure expense. Refer to my post: Do Not Rewrite.

But the new system will eventually also quickly suffer from paper cuts and broken windows if the processes are the same.

Solutions?

There is no right and wrong way to do prevent this. Eventually it is inevitable that a system will suffer from paper cuts and broken windows. However there are several ways to minimise the risk of it happening quickly, increase morale to reduce its spread and significantly delay the rot until the system has been used for so long that there is so little business value left and that it can be shut down.

Prevent death by paper cuts

Fix bugs ASAP

You cannot fix every bug and implement every feature as you will be trying to reach the impossible perfection which would cost an unlimited amount of time and money.

However the cost (in time and therefore money) of fixing a bug increases by magnitudes the longer you leave it, due to the costs of context switch, environment setup, remember data model/logic flow, knowledge share etc. A bug fixed straight away due to a very short feedback loop is invaluable, a bug fixed the same day or next day does take a little time but not much at all. Any longer than that it does become a real exponential costly tech debt.

So try and just fix bugs straight away or at least whilst still on the same specific task. It will make life so much more comfortable. Pushing out buggy features should be deplored.

For minor tasks and bugs that are still skipped or found at a later date they will still cause paper cuts. A thorough automated and manual QA procedure will reduce this but they will still occur.

Fix old bugs continually

However I have found continually picking up a few of these left over bugs and tasks will slowly reduce the amount of them on the backlog and therefore help prevent too many paper cuts.

When finishing a large complex story, try to pick up 1 or 2 bugs and minor tech debts before the next big story is started. If you finish another feature an hour before lunch, meeting or end of the day then instead of half starting another story fix a quick bug instead. This continuous self healing done in what is usually mostly lost productivity time will be invaluable over time. It is a beneficial minor procrastination.

If it turns out that bug/feature is a much larger task than you have time for now then simply just add that information to your issue tracker so that at least you have done some backlog grooming that makes it future planning easier. It may then be a valid candidate for one of the few paper cuts that is never fixed, but at least now with more data.

Development principles

Another way of preventing paper cuts is obviously to never introduce them in the first place. A bug free system does not exist, but you can reduce the frequency and impact of them by applying good procedures, architecture and code style.

Design a system as simple as possible and writing code that is clean ie KISS and SOLID principles, so that a method, class, application or system only does one thing and is very easy to understand. This significantly reduces the risk of introducing unwanted secondary effects and greatly enhances the ease of maintaining it.

Applying functional programming techniques such as avoiding mutability of objects and removing state where it is not needed will also reduce risk of unwanted secondary effects or architecture issues when scaling a system.

Designing systems and features using TDD reduces risk of bugs, ensures test coverage and also avoids implementing unnecessary features.

Introducing continuous integration delivers a very quick feedback loop so secondary effects are found whilst the developer are still aware of the context. Avoiding time spent on unintegrated feature branches is also advisable (ie max hours, never days).

Prevent lack of pride due to Broken Windows

Reducing paper cuts will prevent the broken windows feeling. Applying the principles mentioned above (KISS, SOLID, immutability, TDD, CI, etc) will reduce the risk of broken windows. And by insisting on the same principles of good code standards and sensible processes people will take more pride in their work.

But broken windows will happen. Due to unpreventable technology evolution, staff turnover and lack of knowledge transfer or just plain mistakes.

How do you fix these “windows” and when?

However windows can be fixed. And with more and more windows fixed people take pride in their systems and their work again.

Embrace automation and continuous delivery. If most parts of the process of maintaining a project is automated then any barrier to take the time and energy to fix a broken window will be very low and much more likely to happen voluntarily.

I wrote a post about a project I was on where we transformed most of what we routinely did into one button clicks on our CI server. The returned value of that investment was great, most tasks was no longer a chore, removed many bottlenecks, the risk of process typos or forgotten steps was minimised and business value feature delivery time was reduced immensely.

A good practice is to apply the same principle of picking up and fixing a broken window issue as you do with bugs, ie as soon as possible or just after finishing another story.

Another practice is to fix issues as part of story, especially if it touches the same areas. If for example a deploy often fails on a server, when you deploy to that server just fix it properly. Do not continue with your head in the sand.

With these you will slowly mend your most of your broken windows.

Little known time sinks

It is the little used and little known applications and features that becomes major time sinks when changes are needed. And as such is also the parts the team will avoid to fix. Try to prevent these from becoming this by always evolving architectures and killing features.

Evolve systems

Create your architecture from the start or bit by bit of an existing system into modularised projects and components. That way you can refactor and improve one small part at the time without big and long costly rewrite committal periods of no business value, as detailed in Do Not Rewrite.

Do not leave old systems to rot with outdated technologies and ever shrinking pool of possible maintainers. Keep evolving it, especially their integration points by for example by applying Micro Service Architecture and/or Strangler Application pattern.

Trim the fat, kill features

If an existing feature is no longer really needed, then be quick to kill it. Euthanasia to unimportant or historic features will make maintenance much easier, reduce risk and speed up delivery of new business value features.

Kill entirely an old application if nearly all its original features have already been migrated away or removed.

Bandwidth

To be able to do this you need the time to do this.

You need a strong tech lead to insist on the team to fix bugs and remove tech debt continuously, and can protect his team if they are pressured to cut corners.

You need non blinkered, smart product owner and/or project manager that understands the long term value of not doing the short term features only. A PO/PM that values the velocity of feature counts only and accepts bandwidth is also used on tech debt to achieve this. The PO/PM should not micromanage tasks and tech debt, only priorities at the epic and story level.

Henrik Kniberg’s Lean from the Trenches book describes the value of feature count above any other metric, and whether you solve tech debt (or not) and how much is up to the team and no-one else.

Summary

To reiterate the TL;DR at the top: Fix your bugs asap, evolve systems continually, keep code clean and take pride in it.

Pair with people you like and code reviews with people you don't like

2013-11-08T16:46:00.001+00:00

Pair with people you like and code reviews with people you don't like (misquoted from someone way smarter than me).

That sums it up very well for how I feel about both practices.

With large teams, especially if distributed or partially outsourced, code reviews can ensure code quality and are quite essential. It will allow you to share some knowledge and instill best practices. It will be a reminder for people that their code will be viewed by others so don’t take shortcuts.

However code review can also be a total bottleneck if over-bureaucratic. It will add an overhead for all work. If there are some high and mighty gate keepers that will stop you from pushing your code frequently then you have a complete velocity block. If your code is shit then fair enough, but if it is merely nit picking or just disagreements between styles then it is very costly. If however if any compulsory reviewer is not in your office, country, time zone or just very busy then that adds a large delay in the feedback loop.

Another issue with code reviews is that quite a few reviews are of low quality due to lack of context. They do not necessarily know all the discussions, history of why a piece of code works this way, project code style, etc. Especially if they are of the tainted ivory tower architect affliction.

In smaller, agile and especially collocated teams code reviews will flag issue unnecessarily late in the process. Just pair from the start instead to ensure no short cuts or dodgy code slips through, and automatically spread the knowledge. If you do not trust two of your developers combined then you do have a serious problem.

You can though in addition have small and short swarming/tripling/quadrupling sessions in front of 1 computer to look at especially important issues when they are worked on, not afterwards.

If you do neither code reviews nor pairing then you are in trouble.

Expanded from my own Hacker News comment https://news.ycombinator.com/item?id=6693064

Don't hire me

2013-11-07T09:33:00.000+00:00

There are many reasons not to hire me

I will say no, a lot. I am pragmatic and may find a better solution by asking what it is you actually want to achieve. Or just No

I will say its crap if it is, immediately

I will have an opinion, always. Even when based on very little facts. Sorry

I will change my opinion 180 degrees quickly if a good team convinces me I am totally wrong. This happens a lot, thankfully

If obedience is important to you don't hire me

I am in no way politically correct (not an excuse to be an asshole either) and I make terrible jokes and often “too soon”

I do not have a filter for whom to sugar coat a situation to. I will say the same whether you are a team member, manager, CEO, client or shop keeper. And always blunt and honest

I will be honest with estimates. As in they are total guesswork and only worth it to gage magnitude

If you prefer polite sales type people that tell white lies or total tales then I am not your man

I don’t clock in, I’m definitely not a morning person. If you value timeliness above value then I am not your man

I don’t clock out on the dot either but I rarely work overtime and never weekends. Too risky, if the TPS reports can't wait, don't hire me

I like collaborating, pairing, even tripling for important decisions that needs swarming

I like the team taking ownership together, I detest micro managed task delegation

If you need bums on seat and each task accountable to a person then don’t hire me

I will not jump up like a lap dog when people panic, I barely got a pulse at the best of times. If you prefer knee jerk reactions don't hire me

If you want me to be an ivory tower architect, don't. I like to work with the team to find out their best suggestion is and then work with them to implement it

Or if you have another architect of an ivory tower inclination telling me how to implement something - not going to happen

If you want to keep status quo don’t hire me, I will insist on continual evolutionary improvements. Otherwise the systems will rot, become a big ball of risk and your best developers will leave

However if you want a ninja/rockstar, I am not

If you want me to work with a ninja/rockstar I rather not. Some are okay, but most are a PITA and imperceptibly costly for the company

Insist on Windows? Don’t be ridiculous. No serious developer would accept that handicap unless desperate

Shirt & tie? No, not happening. You can’t take techies dressed as used car salesmen seriously

If you prefer drones in suites, that is not me

Many of these are valid reasons not to hire me

Some I hope are reasons to hire me

Your call

@flurdy | linkedin.com/in/flurdy | www.eray.co.uk

Need to ask what each member of your team is doing? Then your team is too big!

2013-10-01T22:06:00.001+01:00

In daily (or more frequent) stand ups that is common in agile/scrum based teams many practices the process of asking each and every attendee: what did they do yesterday, will be doing today and if there is any impediments. Many do this by default without considering alternatives.

I have previously blogged that I trust my team members and do not need to know in detail what they did yesterday. I firmly believe it is more productive to focus on the tasks on the board rather than the individual. If you do not trust every member of your team then you have bigger problems.

However task focused only works with a small “1-2 pizza” team. A small team where everyone knows what everyone is doing, where the scrum master / project manager have full understanding of what tasks is currently in progress if not already obvious from the board.

Once the team grows too large you tend to have too many tasks on the board and people are all working on different perhaps even unrelated tasks. The other team members, and especially scrum master, cannot keep up with all individual and task statuses so you need to do the token ring quiz.

The ring quiz takes time, and tend to let people’s focus drift away from the stand up and not really listen particularly well to what the others have to say after a while (I am guilty of this frequently), especially if some tasks never involve certain other members.

Also the project comes more inclined to individual tasks and not pairing and swarming together on the same tasks. The risk is then some people can disappear between the cracks or loose focus and your project start to become less efficient and agile.

So if you find you stand ups are leaning towards individual “what-you-did what-you-will-do any-impediments” interrogations then it is a strong smell of having a team that is too big.

(“1-2 pizza” is an analogy of how many can share large pizzas together. In my case 3-5 members is a good size, 8-9 is too big, and more than that is simply not agile)

One button - The aim - Nearly there

2013-08-31T13:00:00.000+01:00

An aim for a development team (and company) is to achieve one button deploys of your applications to all environment.

By achieving that it involves several solutions which in total results several direct and indirect benefits for the company. Benefits such as:

Quicker releases of new features
Frequent releases of new features
Quick and painless rollback of unwanted releases
Reliable release process
Confidence in testing in different environments
Reduced risk in deployments
Predictable roadmaps with more stable velocities
Less context switching for ops and developers
Developer time focused on delivering features instead of on processes

I joined as a Tech Lead at my current company last year. Since then my team, or more appropriately our fantastic Ops/Devops, Tools as well as other teams, have in a year made great strides towards One button deploys. We are more or less there and the company is rewarded more and more with the benefits mentioned above. We have achieved this by doing the following since last year:

Move to Git and internally hosted repository server

Subversion was once great but no longer. Whilst we used git-svn for a while, we eventually fully moved all applications’ source code to Git.

With the addition of first Stash then GitLab we removed a lot of bottlenecks, restrictions and frustrations. Code could be easily referenced without having to check out. Collaboration via Pull requests for code bases you may not be comfortable with. Code review when needed. Forking and creating new projects became instant. Project discovery and general code share became very very easy.

External configuration and one binary

The main applications in my team used to use Maven profiles to build separate binaries for each environment. This was because the configuration was baked in via Maven filters.

This is not great for QA sign off as you basically have different binaries. It also slows down deployment as you have to rebuild the binary all the time. It definitely makes it difficult to rollback releases or investigate older releases as you have to rebuild an older version from scratch. And it makes it very fragile as each machine might build it differently (different JDK, broken local Maven repositories, etc)

Moving to external configuration meant we had one binary that gets promoted through all environment. The one binary gets upload to the repository manager (initially Nexus then later Artifactory). This binary is then downloaded as part of every deploy job.

Changing configuration is a configuration change and no change to the binary.

Configuration in source control and rolled out with Puppet

With external configuration we enhanced reliability by adding these configurations into source control. This also meant configuration change became very easy. Push changes for the environment to Git and it was then automatically sync and rollout via Puppet. This removed a very annoying and typo prone bottleneck.

Production configuration changes are not activated automatically but are rolled out via a single command.

Process automation, Teamcityfy everything...

We aimed to automate as many processes as possible. Our continuous integration server, TeamCity, have a number of automated builds and a whole range of manual jobs.

Testing and building a binary automatically on every check in. Automatic deployment to development servers on successful builds.

one button jobs to deploy that binary to other environments
one button to tag a release
one button to push release binaries to environments
one button to create bugfix branch
one button to rebuild databases
one button to migrate database schemas
one button to restart servers
one button to test 3rd party APIs and environments
one button to run acceptance tests
one button to smoke test environments
one button to trigger load tests (Gatling)

We did use Fabric directly for deploy tasks but Fabric is now triggered from within TeamCity jobs. Deployinator was nice but we phased it out in preference for all deploys in TeamCity for every environment.

Test separation

There was already an extensive JUnit suite of tests in the applications my team was responsible for. However they were a mix of integration tests masquerading as unit tests which slowed down development and feedback loops.

We relabeled unit test that required frameworks (Spring, Hibernate, etc) or databases as integration tests instead. That reduced our feedback loop time.

We added separate verification builds for just integration test. We separated and migrated 3rd party test to 3rd party libraries. We added stand alone acceptance and smoke tests using Cucumber, Specs2 and Selenium.

Test data

We wrote command line and web applications with Node.js and Spray that quickly create test data and tools that review test data. This sped up development testing and proper QA testing.

3rd party mock applications

Mocking out 3rd party systems or even some internal systems entirely by creating test harness applications in integration environments that pretend to be those systems have speeded up our QA process a lot.

Removed restrictions

Restricting who can commit, run certain jobs or see certain data might sound like a sensible option, but in reality it slows down collaboration. We decided we just trust the majority more than we distrust a minority.

A great benefit was that we opened up all Git repositories to everyone. Anyone can commit, although people less involved in a project prefer to use Pull requests.

Team City jobs can be run by anyone. There is an audit trail, but there has never been an issue of anyone ran a job they should not have. It is a great help if people are in meetings or similar that anyone can push code out to the next environment for example.

Production data for some applications due to PCI and CRB restrictions, production credentials and some production jobs for our critical applications are still restricted but we try to minimise this as well.

Feature toggles

Whilst it is a good practice to keep these toggles to a minimum, adding feature toggles that can be overridden via external configuration has been a good change. We can now quickly disable broken new features. We can dark release features or we can canary release features, then via Git & Puppet enable the feature and it becomes available for all.

Content migration tools

Instead of applying content directly and manually as SQL scripts, or scp/ftp/rsync we started writing tools in Play! to help create data sets of new content, then promote those through environment, cross checking which environment the data is in.

Adding scripts that interact with 3rd party portals was also helpful.

This avoid typos, avoids forgetting to run a script in an environment. And greatly speeds up data migration.

Database migration

Many of our applications use NoSQL solutions such as Cassandra and Redis as some of our flows have to handle millions of interactions, but the core data are still mostly in PostgreSQL.

Whilst we use Flyway and DBDeploy to migrate those database schemas and stub data to some environments, we do not use it all the way to production. This is one area we need to improve.

Environment creation

One element that is important and which our Ops team is just starting to roll out is an internal PAAS/IAAS solution. One button to create a new VM environment or one button to create a database, AWS SQS queue etc. Further enhancements, such as one button to clone an existing VM or database will be nice.

No button

Obviously there were other enhancements that is not really related to "One button", such as phasing out old legacy Java applications with newer Scala applications by applying Strangler Application pattern, replacing Quartz based batch jobs with Akka and Camel, measuring new features effect with AB testing, monitor applications metrics with Graphite, log analysis via Logstash, etc.

Company profitability

All these combined has made certain part of our development and release process so easy and quick. I am sure we have covertly increased the profitability of our company as we now can release quicker, more frequently, with less broken releases or bugs in general.

Future

We still have a lot of work to do. As mentioned we are not quite there with DB migration and Environment creation. We are experimenting with using Vagrant locally and promoting to environments, ZooKeeper for better configuration change without downtime, etc. Eventually we may achieve most of the ideas behind Continuous Delivery.

Do not stand up in the morning, do it at lunch, and again

2013-04-20T14:55:00.003+01:00

The Daily Stand Up, a meeting where everyone stands up (to ensure it does not go on for too long), often referred to as the Daily Scrum by projects using Scrum methodology, is often scheduled in every morning.

And by early it is often the first thing in the morning. E.g. 9am.

There are good reasons for people picking that time slot.

* It gives people a clear idea of what they will be working on today.
* It minimises any unfocused time from starting work until the stand up.
* It ensures everyone is present and not being pulled into other meetings, leaving early etc.

However I dispute that.

Everyone may not be present.

* If you are based in a large city, various public transport delays often occur and people frequently do not make it in for the normal start time.
* Modern day work life is not like the old days of set factory shift hours. People do not need to clock in at the same time anymore. Flexible time allows people to start a 7,8,9,10 and change that every day.
* With a smaller world, companies and projects are frequently distributed across different time zones. So the work hours definitely might not match up. (Co-located teams or not discussion is for another time..)

So do you keep running the stand ups without everyone present? Or joining halfway through? Or frequently postpone it until later that day?

The time before the stand up will vary a lot.

* As people start at different times, some will make it in just before the stand up others will have been there an hour or two already.
* Members of the team might need to attend other stand ups before this stand up. E.g. Scrum-of-scrums stand ups, if the Scrum Master/project owner is split across other teams, or worse team members are split across projects...

So do you just ignore the unfocused time before the stand up?

The task plans will not be clear for the entire day.

* Time before the stand up may be unfocused. If perceived as too small to bother trying to a worthy concentration ‘zone’ or pairing session. I.e. just waste.
* They might finish or become blocked/impeded during the day, and not pick up priority tasks
* Team members might be struggling until the next stand up

Is the unfocused wasted time costly?

So how do we improve this?

I have two suggestions. And they also have other benefits.

Have the stand up time just before lunch.

E.g at 12:45 if in the UK, or 11:15 if in Norway (my two countries of reference)

Everyone will most likely be in the office whether they started early, got stuck in traffic, had to drop off the kids at school, etc.

It definitely ensures the meeting does not overrun as people really do want to go to lunch.

It does have an issue if people go lunch at different times. Lunch times however are more easily adjusted by an hour more or less than the actual working hours. (If still different then you have another problem as I am a big believer in that most of the time team members should want to go to lunch together. Socialising at lunches and other events are great team spirit builders.)

The time before the stand up is now significant so people should not be unfocused before it. They should be able to resume their task, resume pairing, enter their zone and actually do some significant work.

With members in normal working mode they will be able to quickly explain what they have been doing since the last stand up, what the important impediments are etc. I often make it in just before morning stand ups, and rarely have a good memory of what I did the day before (which is a good thing, as it means I was able to switch off from work), but also means my status contribution is not optimal. (Whether stand up are status meeting or not is another discussion).

It also blocks people from scheduling stupid meetings that overlap lunch.

Introduce more stand ups

The daily stand up does not have to be daily. To keep focused on the flow of tasks your team can introduce multiple stand ups during the day.

However I would suggest to keep one as the official one, the one you advertise to externals, the one the project owner attends. The others should be kept as informal meet ups for the members to quickly sync up, to ensure WIP limits are maintained and focused on the most imporant tasks, if any impediment have occurred during the day or if someone is not sure what to pick up/assist with next. Much like the “proper” stand up.

You need to be careful not to introduce too much of a pressure cooker, allow some natural slack time, down time during the day otherwise your team will quickly burn up and become demotivated.

The informal stand ups does not have to be at the board, it can be at a table in the kitchen etc.

Dog food

In my current team we have our official stand up at 10am. Yes not lunch, but it was what the team wanted and you go with the team agreement. Our company does not offer flexi-time, so we all need to clock in 9.30am so with traffic/commute delays people are mostly present at 10am.

We have introduced an informal stand up between the developers and QA around 1400-ish.

And a couple of kitchen breaks during the day, however these naturally also include of general banter.

Follow up

Take a look at “Cocktail Party” of stand ups that Henrik Kniberg describes.

I dont care what you did yesterday, I trust you

2013-04-20T11:37:00.001+01:00

A daily stand up is not a status update.

It is not a tool for micro management.

It is not a opportunity for line management, middle management, project management and other stakeholders to scrutinise status of anything.

It certainly is not a stage to interrogate the work ethics of individuals.

A stand up is for the team,
synchronise on what will happen today,
making sure tasks at the top are moving along to done
and to assist others(swarm) to ensure WIP blockers are removed .

Stakeholders and others can watch standups, it gives them a quick overview of the priority of the team, any impediments etc but it is not for interfering or performance scrutiny.

The board should reflect status at other times. Otherwise the project owner/scrum master can inform people what the backlog priority is.

Any micro status of what happens between the backlog and done is not for external stakeholders.

TL;DR We do not care what you did yesterday. We trust you. You would not be in the team if we did not. Lets focus and help each other on today's task instead.

Heroku commands & tips

2013-01-30T18:08:00.000+00:00

I wrote a quick howto of Heroku commands and tips I use all the time.

It covers creating instances, configurations, common add-ons such as databases & email, deployment strategies and more.

As mentioned in the document I tend to have multiple applications per project, so I always append --remote staging for example, resulting in this long deploy command:

git push staging master && \
heroku logs -t --remote staging; \
heroku open --remote staging && \
heroku logs -t --remote staging

For the eagle eyed you will have noticed the ";" which means as the log tailing never really finishes, when you notice the app is up and running you have to manually end it with control+c to proceed with opening up a browser.

Obviously these commands ties nicely into my use of Play on Heroku and the howtos I wrote on integrating both.

They are perhaps obvious commands and basic but I hope they are of use to some people.

Lean book review: Lean Architecture and Lean from the Trenches

2012-11-05T23:58:00.000+00:00

Quick review of one of two technical/project management books I read lately.

First is "Lean Architecture" by James Coplien.

Lean Architecture: for Agile Software Development

Coplien is someone I have great respect for, I have listened to a few lectures by him and read several articles. He definitely seems very knowledgeable on the subject and good in panel debates against other self entitled agilistas. Some people may recognise him for his forword in the Clean Code book by Uncle Bob. So I was looking forward to read this book.

However this book was not great. It is very wordy and repetitive. The book keeps going of on a tangent about the history of agile and lean, which while nice is not why I bought the book. Only in the last few chapters does it actually get to the point of the book, the DCI architecture style.

If you want to learn about DCI (Data, Context & Interaction) then it may be the book for you, especially if you want to pick up history of The Toyota Way, Lean and Agile. Otherwise don't bother.

Another book I read is "Lean from the Trenches" by Henrik Kniberg.

Lean from the Trenches: Managing Large-Scale Projects with Kanban

Having previously read two of his other books I was expecting a helpful book. ("Kanban and Scrum - making the most of both" & "Scrum and XP from the Trenches"). Kniberg is very much a Kanban man so I was interested in his Lean views.

And this book I thought was very good. He adopts a reflection of a large scale project for the Police in Sweden, and the aspects they learned by adopting lean practices as they went along. And then in later chapters more detailed reflections and background on subject matters. (I suspect the authoring style leads readers to think it was all accidentally knowledge gained by the team along the way, but knowing his previous experience I am sure he nudged most in the right direction).

His writing style and diagrams are very easy to follow and I finished the book in a few days read on the commute, and was very inspired. Highly recommended.

Send email via SendGrid on Heroku using Play! 2.0 with Scala

2012-05-23T02:09:00.000+01:00

If you have a Play! framework 2.0 application that you want to send email from, here are a few tips.

These tips assumes you deploy to Heroku, but other platforms should work similarly. The examples here are using Scala, but Java should work along similar lines. Finally the specifics are for the SendGrid add-on for Heroku, but other mail server providers should be fine.

First add the free option of the SendGrid add-on to your Heroku app by typing in:

heroku addons:add sendgrid:starter

Then configure your Play! app to use the mail plugin provided by Typesafe:

Add to your dependencies in the project/Build.scala file: (all on one line)

"com.typesafe" %% "play-plugins-mailer" % "2.0.2"

Then create and add these to a conf/play.plugins file: (all on one line)

1500:com.typesafe.plugin.CommonsMailerPlugin

Next configure the mail server settings. You can either add these directly to your conf/application.conf file, but I prefer to share my projects' source code, so my production settings are set via environment variables so that my username/password are not publicly available.

However for the plugin to run smpt.host must be present. Open conf/application.conf and add:

smtp.host=mock

On Heroku I append the settings to the Heroku's propriatory Procfile file. I append these settings to the Procfile to use SendGrid's servers: (all on one line)

-Dsmtp.host=smtp.sendgrid.net -Dsmtp.port=587 -Dsmtp.ssl=yes -Dsmtp.user=$SENDGRID_USERNAME -Dsmtp.password=$SENDGRID_PASSWORD

You may already have other settings in the Procfile, e.g. database URL, so be aware of the 255 char limit, and use a custom properties file instead.

web: target/start -Dhttp.port=${PORT}
-Dconfig.resource=heroku-prod.conf

The SendGrid add-on should create the environment SENDGRID_USERNAME and SENDGRID_PASSWORD variables for you.
You can verify this with:

heroku config

Finally we then create our actual application code to send email:

package notifiers
import com.typesafe.plugin._
import play.api.Play.current
import play.api.Play
import play.Logger
object EmailNotifier {
def sendMail {
val mail = use[MailerPlugin].email
mail.setSubject("Mail test")
mail.addRecipient("Joe Smith <joe@example.com>","sue@example.com")
mail.addFrom("Jon Doe <joe@example.com>")
mail.send( "Test email" )
}
def sendTestMail {
if(Play.current.mode == Mode.Prod){
Play.current.configuration.getString("smtp.host") match {
case None => Logger.debug("Email mock")
case Some("mock") => Logger.debug("Email mock")
case _ => sendMail(participant)
}
} else {
Logger.debug("Email mock")}
}
}

(In case of this posts cant parse tags replace < and > with less than and greater than tags;)

This should be all that is needed.

Play! 1.x did have a handy mock email interface for development and testing. I will try and find a suitable replacement for 2.0 and update this post when I do.

For more information

Play! 2.0: http://www.playframework.org/
Play! 2.0 docs: https://github.com/playframework/Play20/wiki
Heroku and SendGrid: https://devcenter.heroku.com/articles/sendgrid
Mail plugin for Play! 2.0: https://github.com/typesafehub/play-plugins/tree/master/mailer
Heroku environment variables: https://devcenter.heroku.com/articles/config-vars

Play! 1 & 2 command tip

2012-04-22T16:18:00.000+01:00

If you love the Play! Framework, you might be like me and have both version 1.x and 2.x installed.

Version 1.x, in my case 1.2.4, is a well established feature rich stable version. Version 2.x, in my case 2.0, is a new radically different version, that is still in its infancy but released.

You might have both installed as you have older projects using 1.x and new in development projects using 2.x, or similar.

With both version the norm is to install them and expose the main executable as "play". So how do you differentiate between which version to use for which project?

OK, it is no rocket science but here is a quick tip on how I do it:

I have Play! 1.x installed in /usr/local/lib/play-1.2.4

I have Play! 2.x installed in /usr/local/lib/play-2.0

You can add either play folders to the your PATH, e.g. in /etc/environment:

PATH="blahblabh:/usr/local/lib/play-2.0"

But I simple add them to my .bash_aliases file:

alias play1="/usr/local/lib/play1/play"
alias play2="/usr/local/lib/play2/play"

On top of that I symlink this:

$: cd /usr/local/lib;
$: sudo ln -s play-1.2.4 play1;
$: sudo ln -s play-2.0 play2

With this setup I have to make a conscious decision whether to run Play! 1 or 2, and can switch between the two very easily.

$: play1 help;
$: play2 help

Avoiding cyber squatting failure

2012-04-22T15:08:00.000+01:00

A year or two ago I misspelled a domain name of very popular site used by many developers. It came up as "Not Found" so I realised as the site at the time was still quite niche (not any more) cyber-squatters had not cottoned on to it yet.

So I registered a couple of similar domain names with a misspelled vowel. This was mainly as I thought it was quite funny at the time, but also I did not want real professional & cynical cyber-squatters to register them either.

Of course I did not know quite what to do with them so I put them as Google parked domains (AdSense for Domains) and forwarded all emails automatically to the proper domain. If ever contacted by the proper site I would just let them have the domain(s).

End of AdSense for Domains

This Spring Google closed their AdSense backed parked domain offering. And I needed to reflect what to do with the domain names and what options I have.

I do not really want to keep paying for their registration, the ads on them brought in less than £20 every year so not likely to challenge my ethical backbone either.

When some of the domain names expired this Spring, and I was initially just going to let them expire, but then from experience I know domain cyber-squatters will be scanning for expired domain names and pick them up. And they will use them far more cynically and unlikely to ever hand them over for proper usage. So I extended my registrations with those domain names.

Domain parking bad taste

In the end I moved the domains from AdSense For Domains to another Domain Parker.

But this has left a really bad taste in my mouth. I am not really any better than a cyber-squatter. I am profiting from misspellings, although less than the registration costs... The content on the parked domains are really of no assistance to the people trying to reach the proper site. So I cannot leave it as it is.

"Nice" Anti Cyber Squatting

The best solution would be if there was a friendly not-for-profit community back anti-cyber-squatter service. Offering useful content/redirection. A sort of defensive registrations the community can do on behalf of proper sites. Naturally the proper sites should really register these names themselves, but some cant or wont. But to avoid squatters taking advantage a free service like this would be handy. Of course we/I would still have to pick up the registration cost.

Landing page

But instead I think I might put up a brief comical landing page with a big button to go to the proper site. (However until I actually get round to create that, the domains are still listed with the domain parker.... )

Clone local git repository to remote server?

2012-02-22T12:34:00.009+00:00

If you have a local repository that you want to clone/copy to a remote server here is how I do it.

Perhaps you have been scaffolding, testing and initial silver bullet development and have realised the project is mature enough to share with others. Or you just want it backed up remotely.

Simple git clone local remote do not work. Destination path for git clone is always local.

What you need to do is on the remote server create a bare git repository:

remoteserver$ mkdir -p /var/local/repos/myproject;
remoteserver$ cd /var/local/repos/myproject;
remoteserver$ git init --bare;

Then add this remote to your local git repository:

localmachine$ git remote add origin \
ssh://remoteserver.example.com/var/local/repos/myproject;

Now every time you want to push to your remote server, you do a normal push:

localmachine$ git push origin master;

Although I would seriously consider using something github or gitorious instead of your own server.

Continuous Deployment via Stack Overflow

2012-02-13T00:55:00.004+00:00

I do have an interest in Continuous Deployment/Continuous Delivery. Continuous Deployment goes beyond Continuous Integration and automatically deploys builds to production all the time, not once in awhile. It goes well with agile thinking.

With frequent deployments the delta that can go wrong is very small, the feedback is very quick and the knowledge is fresh and active with the developers so fix turnarounds are immediate. By relying heavily on automated integration testing and DevOps that provide automated deployment the result is quick and painless.

This week I answered a difficult question on Stack Overflow. The question by Emwee was for advice on how to use Continuous Deployment with multitude of inter-dependant systems.

It is a tricky question and I did not have an exact answer. My answer was more along the line of how to facilitate an easier deployment by making the dependency and coupling looser and roll-outs smoother.

I referred to Duck Typing, Feature toggles, evolution scripts, version tables, symlink stage & deploy, and referenced a Hacker News discussion on how Amazon deploys their systems, a video on how Netflix builds releases in the cloud and how at IMVU they deploy 50 times a day.

I also referred to Humble and Farley's book on Continuous Delivery.

There was another reply as well by EricMinick referring to his previous answer to a similar question. Eric does in detail describe scenarios of using promotion builds into different isolated test environments and suggest solutions that his company UrbanCode provides in relation with Jenkins.

In the end Continuous Deployment is a great evolution. But with large enterprises you need to keep your tongue straight. But it is worth the investment.

Multi book reading

2012-02-03T00:39:00.008+00:00

I have a habit that I wonder if others do as well. And I wonder if it is productive or counter intuitive.

What I do is multi-book-reading. By that I mean I read several books at once.

How I do this is by having different books in different locations. And also several books in the same location. I have a preference for printed books. PDFs of books are handy occasionally for specific searches, but I really cannot read page upon page on a screen. Kindle may be nice, but I have not yet jumped on that bandwagon.

This multi-book effect is partially due to laziness, as when I am in the lounge I do not want to have to walk back upstairs to my study if I get a sudden urge to read or fetch a book. So I always have a half read book in the lounge. (Currently in the lounge I am re-reading the Kanban book by Anderson).

In my study I naturally have bookshelves loaded with computers books, especially java, and always a couple on my desk. I tend not to read too long in this room (then why is it called a study?), but a quick few paragraph while the pc is rebooting etc a few times a day. The reference books however might be referred to when needed. (Current study books are Seven Languages in Seven Weeks by Tate and Continuous Delivery by Humble & Farley)

On my bed side table there may be a fiction book. I don't really want to think too much when trying to sleep.

At work I would also have a mini library as I also tend to evangelise and lend books to colleagues. So I usually have a half read book or two or three there as well. Again for quick glances while rebooting etc, but also for longer reads during lunch if I go by myself. (Just finished ReWork by Fried & Heinemeier Hansson and Linchpin by Godin)

When I am out-and-about e.g. queuing for the till in a shop, waiting for the bus or at lunch when I forgot a book, I use the Aldiko e-book reader on my Android phone. Being a sucker for offers at O'Reilly for epub formatted books I have quite a few on my phone. Aldiko is excellent, but the small phone format is in essence rubbish for reading books over time. It is good for quick 1-5 minutes reads but no longer. And useless for reference books as the overview of the page is difficult. (97 Things Every Programmer Should Know by Henney is good for 5 minute reads)

For longer commutes by train or plane I tend to bring an actual book. If I do this regularly I should probably invest in a Kindle. (Read bits of Specification by Example by Adzic on the plane recently)

I tend do a lot of quick reads. As a stop gap filler between other events. Maybe work interrupts, maybe it is some element of ADHD, or probably our 6 months old daughter. This means I constantly waste time remembering the context of where I am in the book.

Longer reading periods is usually a holiday pattern. This would then often involve more fiction books. I read the Night's Dawn trilogy^[2][3] over the last few holidays.

Some books I might skim read chapters to get a gist of but I know I can revisit them if I need to know those sections in detail in the future. Some books I never finish. I would say a good third or more in library I have not finished or barely read. Some turned out just not interesting, some are just skim read while others are consequences of multi-book-reading and lower down the reading priority list.

So this multi-book-reading means I do read a lot of books. But it also means I do not finish enough books. Not very "kanban" or "one-piece-flow" :)