software nature

Software development and creativity

Thu, 17 Oct 2019 10:50:00 -0600

I recently encountered an interesting discussion on Twitter regarding software development being a creative field or not. There were enough passionate arguments on both the sides. Being an artist (painter) and a software professional, I do have a few thoughts on this, which are not just limited to programming.

If coding doesn't feel like painting or passionate creative writing, you have at least another level to master before calling yourself a programmer.
— James Coplien (@jcoplien) October 14, 2018

Thoughts:

Art is the process of improving the quality of an object.

The object’s value is thus increased in the process. For example, you may start with an uninteresting log of wood but, in the end you would create a beautiful vase out of that. Quality could be aesthetic or functional i.e. strength, conveying information, etc. A bit of disclaimer here: I don’t get abstract art, not a big fan of that (yet), whose theme is to not necessarily to have a purpose behind something, i.e. there could be brush strokes without necessary purpose - it would be left to the interpreter (audience) to make sense or find purpose in it. I am not talking about such art in this topic!

With software development, something is built out of nothing or mere thoughts. There is a joy (maybe due to dopamine) which one gets when a well thought, logical contraption built using imagination, hypothesis and constraints comes to life! It is very similar to creative writing even though, I am not into that field, having read stuff, I could relate to this. With software development, there are a few facets as below.

(above diagram is not all inclusive, it is a rough mind dump of what I want to say)

Craftsmanship involves designing, skills and passion to have a better outcome’ ‘implementation’, both can be equally creative and it is coupled with the analytical and skillful side of engineering. If there is any room for, and willingness to do things differently it could qualify as creative work. It’s goal is towards improving of quality of the product. This revolves around certain principles and practices which guide into making the changes in order to achieve the same. Beyond a certain point, the refinements become less of a necessity and more of a preference based on predictions and experience. This activity of improving quality is a highly creative task which increases value of the software, even though it may not be tangible at first. Effort put in the refinement and caring for the minor bits in software, shows. Take MacOS for example (haven’t tried Catalina yet!), everything looks handcrafted and made with care (including performance), while Windows could seem to be like things were put together with different goals in mind (although it does the job as well).

Doing painting especially in oil seems very similar to software development to me. One would start with a blank canvas, then by colours and shapes build a picture of what he/she sees. Oil painting remains wet for quite a while and one could paint over it, reveal paint from below layers, erase everything and start over. You could make several changes until the painting is completely dry (even after that at times). During the painting process, the artist stops and compares, introspects on the picture - makes changes until the desired outcome is achieved. Until the painting is actually done it only is in the painter’s mind, no one else gets the picture. Same is with the software, until it is built everything is intangible and everyone has their own ideas of how it might turn out to be. Many times, I think the product owner has to play this role of an artist (as example above) or a director, who has an idea of the software in their mind. This idea would be based on external(market, need etc..) and internal factors. They have to incrementally shape, tweak, push the realisation as close to the vision as possible.

Problem solving is more analytical but, also is creative, and it is equally if not more rewarding than craftsmanship. It is just like how theoretical physics and pure maths have an incredible beauty and a creative side, software development has it too. Coming up with algorithms or logic machines which do the required work for you give a great amusement and sense of satisfaction (again, dopamine). Those ‘eureka’ moments are what makes this most interesting and engaging.

Ideas and a thought process cannot be learnt however, it can be acquired. One can get inspired from other great ideas. Gathering skills building logic are a vital part of this but, ideas need to originate from one’s thinking process. In painting for example, one would struggle initially to hold the brush, mix paint properly etc. Once that is learnt, painter moves to the next level - tries to replicate paintings, imitate brush strokes, match colours. Then painters develop their own style of painting, their own palettes and brush work. After which the painter starts exploring new subjects, ideas. This is possible as the former hurdles are already crossed and there is enough mindspace and confidence to navigate one’s way. At this stage inventions (not accidental), innovations can happen.

It certainly would be difficult to run businesses with artists alone, there are limiting constraints. The hardening of ideas / notions could at times override rationality. To make it effective, creativity has to be coupled with adaptability (or as we say, pragmatism). Most time is spent in hacking, engineering and crafting a solution but, having a vision / idea in mind can make the difference. The famous cliché - magic happens when art meets engineering ;-)

Ending with a quote from the writer of Quake -

You might not think that programmers are artists, but programming is an extremely creative profession. It's logic-based creativity.” - John Romero
— Programming Wisdom (@CodeWisdom) October 16, 2018

Software rust

Tue, 06 Aug 2019 10:50:00 +0000

Rust is a well known phenomenon, it is the effect of the environmental conditions on the metal objects. Well how does this relate to software? Being aware of these factors can help avoid and mitigate such risks in product development. Also, is there anything as “curing time” in software? Lets find out…

Software may seem like an unchangeable entity once it’s ready. Right, because it’s virtual? By virtual, I mean not having a physical existence / presence. It is virtual but, in its own world it’s real. In the software world it is affected by the elements which exist there.

As it’s virtual, the expectations from it will be unreal as well. Users don’t treat software like the real thing. Example: compare a commission work for carpenter to make a chair against a commissioned work for writing software to do some accounting work. How many times would the customer go back to the carpenter for making modifications to the chair? Unless it’s a poorly built chair which doesn’t work, this will be rare. Mostly, it will be well built to the specs and customer would be reasonably happy. With software it’s quite different, as it’s virtual the raw material, the build effort is invisible. Thus, software always seems to be a work in progress (well there will be terms and definition of done) however, it’s never solid. If you know what I mean - there is no curing time (like adhesive or cement) or time required to let the paint dry. The paint seems always wet and like an oil painting you can paint over, remove paint you can even change the body/shape. It seems to always be in a work in progress state. The problem is that this is not true.

There is a phase when software is moldable and then there is a curing time, after which making changes have an effect on the solidity or robustness of the product. This curing phase may not be a tangible thing. You can get around by ensuring automated test suite passes or good coding practices and other great ideas. However, the vital thing is the knowledge. As soon as the team diverges away from the product the curing process begins. The curing process accelerator is the user of the software. The more software gets consumed, more it gets cured / starts solidifying. Like the Schrödinger’s experiment, the box has been opened, reality has taken over 😊

After this curing phase of the software, I compare its life cycle with a metal product as it exhibits similar traits. Example: A bridge built with iron can corrode / rust due to the water vapour in the air or rain. The elements in software world would be its users, the problem (domain) it’s trying to solve, third party libraries / web services, data storages, web hosts, frameworks, operating systems etc. Unless software is built as a system (including the platform its being run on) these elements are always in play and interact and affect the software. These elements listed above (it’s not a complete list) are independent and have their own life cycles, their current state however, affects the software which is deployed in the field. Example: Users found the software ‘state of the art’ when they first used it, over the years users have evolved by using other software they interact with and their expectations from the deployed software are a lot more than what they were earlier. Here the deployed software remained unchanged but, its functionality seems to have reduced (effectively). Let’s take another example: OS, let’s say the software was built using .Net 3.5 on windows XP, it all was running smoothly, one fine day Microsoft decides its discontinuing XP and thus .Net 3.5. Again, software was unchanged, its environment changed and it has a detrimental effect over it. Similarly, with data storages, third party libraries their dependencies, web services, web hosts. As another example, let’s say there is a new authentication protocol in the market, its adopted like wildfire, all the webservices which your software uses have migrated to that, and would deprecate the old one in some time. The software which is deployed would stop working after the deprecation of the older protocol unless it also uses the new protocol. These are some of the examples of elements which affect the software (that remains unchanged after deployment).

Thus, I put forth a point that software out in the field has wear and tear and needs to be maintained. Like the bridge needs repainting, metal replacements, analysis, reinforcements, software also needs to be analysed for effects of the elements and maintained. It’s possible to keep a software product rust free and performing 100% by frequently monitoring the elements which it interacts with and taking necessary actions.

Reusable components design

Wed, 26 Apr 2017 18:50:00 +0000

An adage in Software Engineering goes like

Before software can be reusable it first has to be usable.

— Ralph Johnson

Reusability is a highly desirable attribute in Software Engineering. It is known to save time (at times) and it makes robust software. It’s the magic bullet! There are various levels at which reusability can be achieved in both ‘the what’ and ‘the how’ aspects in software. Achieving high degree of reusability is hard, it takes effort in that direction. For any enterprise the relevant questions will be what is the cost of making software highly reusable, next is when to put in the effort to build reusability into software, and how much would be gained in return.

Let us take these questions one by one.

What is the cost of building in reusability?

For the uninitiated, the cost seems to be negligible. It is often underestimated and taken for granted that if good design principles / coding practices are followed, it will result into good amount of reusability. In theory, following good design practices will make code more readable and do what it is supposed to in an elegant, efficient and sophisticated way. It will not however, always make it reusable. Maintainability is the quality which should be at the centre of concern, maintainability and reuse are related in a vague way. If the software is dominating or even surviving well for over 5 or more years that means it is maintainable and a lot of code base / components are being reused, it is serving the requirements for a long duration.

Generality (spectrum of reuse) and abstractness are directly related, the more generic it is, the more abstract it becomes.

If software requirements are being thought of from a generic perspective in order to serve a wider audience, they will be more abstract and disconnected from the direct user requirements. That means the drive for developing is not coming from the core business it is something which has to be… invented. As an example, consider computer games, an engine is something which is reusable, while a game may not be. Building an engine without real requirements, trying to imagine what future games might need is a tough ask, and a bit academic in nature. It needs time and expertise to know the domain before something like this can be done. Building an engine however, is for a very high level of reuse, probably considering 100s of games being churned out using it. It is a complete software in itself whose core requirement is being reusable and doing everything what a game needs. The customers are developers. As one can observe, the scope of building an engine is exponentially growing and so it’s cost! It would be a big decision to invest into building something like this upfront.

Next level (lower) would be frameworks, then services, libraries, components and then code. The cost involved would go up as the generality increases. Building new languages, database engines, frameworks involve a huge cost. Sometimes the business might want to build engines or invent things to be ahead of the competition and maybe forecast returns in future, for that the other side of business needs to sustain the cost of building reusability.

When to put in the effort to make it reusable?

In case if someone else is waiting in the wings for a software requirement similar to the one which is being developed, then it makes sense to put in effort to make the code reusable or build components. However, prioritising reusability over fulfiling the core requirements of software is a recipe for disaster. It has to do one / few things right, rather than be an allrounder which can do a bit of everything but can’t completely do one thing. Perhaps, an analysis should be done for why there are similar requirements in different softwares. Sometimes there might be business reasons for having similar softwares. Sometimes less is more…

Form follows function

Emergence principle: Let the modules emerge

While the business requirements are being satisfied by the software a vigilant eye can always spot the components emerge from within software solutions. The components will sprout from the main features of the software and when the start to ripen on the functional software they can be easily plucked and groomed into proper shape.

There is another train of thought, to build everything in a Service Oriented Architecture (microservices). That means everything is like building blocks (lego) you can plug in anything. It sounds good in theory, however software components are not exactly plug and play, they have different interfaces - which often need customisation (adapters). Every pluggable piece has its own specifications and quirks. Not everything will be factory made. This is a reasonably good strategy however, attention should be paid to how much of code is actually consumed in the application. All code should have a purpose. There is a tendency of over engineer in case of service oriented architecture.

From a domain driven design, the focus is on accurately modelling the domain and solving the core business problem.

As long as there is loose coupling in the code, it is a conducive environment to sprout reusable components or services. This can be part of the main design principles while building the core software. Services based on contracts can always be provided on top of the core components once they are ready.

Core software will have schedules, reusability will not.

How much would be gained in return?

In theory reusable components are expected to work straight off the shelf. Realistically, there is an effort required to consume the reusable components or services to build software solutions. More out of the box’d ness the solution has the more narrow its applicability and the vice versa.

Another interesting saying in software engineering goes like:

“Telling a programmer there’s already a library to do X is like telling a songwriter there’s already a song about love.” - Pete Cordell

It’s so true, as there are so many subtle differences yet, it depends on how deep one wants to go.

Clustering using k-Means Algorithm

Mon, 30 May 2016 09:30:00 +0000

Introduction

Clustering a unique problem which falls under the category of unsupervised machine learning system. In neural networks it also is a type of classification problem, which separates the inputs into different classes.

As there is no training set available to hint or teach the system as to which point belongs to which class, this is an unsupervised learning. Only thing known to the system is how many clusters it requires to form (the k). The number of clusters can also be computed using stochastic / heuristics of initial analysis of the inputs. It is one of the simpler clustering problems. There are more advanced clustering ways like Linear Vector Quantisation (LVQs) and Self-organising maps (SOMs).

It works well when the problem is more intuitive i.e. if we can visualise different clusters as there is a good amount of separation between them (example Fig. a). It is also useful when the problem space is intermixed (example Fig. b) yet we would like to divide the space into certain clusters.

Fig a

Fig b

Clustering is a NP hard problem. k-Means is more of a heuristic algorithm, which is trying to solve this NP hard problem. This algorithm is not able to find the global minima, it converges onto a local minima. As you can try for yourself in the below implementation of the k-Means. The quality of the solution is dependent on the location of the initial guess of the centroids.

The Algorithm

Inputs

P = Set of input points where every point in P is belonging to space of n dimensions.

k = number of clusters

The algorithm will try to minimise the following function

Step 1

This is an unsupervised learning, hence we start with the analysis of input data and keep adapting our computations based on new data. Take the available input data

Step 2

Initialise the k cluster points in the input space. This initialisation is important as this alogrithm will latch onto a local minima. We are going to take random point in the bounding box of the inputs.

Step 3

Compute the distances of each point in the input space to the cluster points.

Step 4

Associate each point to the cluster point which is the nearest to it using the distances computed. You can combine step 3 and 4 for optimisation.

Step 5

Compute a centroid of the points associated with the each cluster. So there will be k new centroids. Move the cluster points to the new centroid positions.

Refer above fig.

Step 6

If there is any change in the cluster position, repeat step 3 to 5.

To achieve global minima one can use the stochastic techniques or search techniques like Simulated Annealing.

Interactive js example

Enter the number of clusters

These cluster centroids are then used to compute a voronoi region. More to come in the later posts.

Booting up my Raspberry pi 2

Sun, 03 Jan 2016 09:30:00 +0000

Raspberry pi is a small and cheap bare minimum computer. It has been around for a while. It can be connected to various devices and can function like a normal desktop computer (apart from the performance aspects). Programs written in many different languages (ruby, python, javascript, C#, C++, java… etc) can be run on the pi!

In the first incarnation of the model there were a few variants like A, A+, B, B+. Now, there comes the pi 2 and they have really put everything into this one :) You have to love this small powerhouse of a computer… It is absolutely perfect for the DIY or IOT (internet of things) applications. It comes in with a dedicated graphics processor, that means the pi can do HD graphics.

Some of highlighting features of the pi

ARM Cortex A7 quad core (900 MHz)
1GB RAM
VideoCore IV 3D graphics core
Full HDMI port
CSI Camera interface
the list goes on…

Quad core is really too much power for an embedded device (type of), if a computer is required for such usage or lower power consumption, it is recommended to use the earlier models. I had ordered the pi 2 B model.

The pi is a full computer hence, we need an OS for the pi. There are a lot of options ranging from

NOOBS
Raspbian (debain)
Windows IOT core many more are there on the pi website.

I am going with minimal amount to begin with, hence I only need the runtime - and also I wanted to checkout the Win10 IOT Core

Download the ISO image from http://ms-iot.github.io/content/en-US/Downloads.htm

Installed the msi embedded into the ISO

Then we need to burn the .ffu image from the installed setup into the micro sd card. An adapter is requried for connecting to the laptops.

Thats all… Microsoft has made things very easy. Well, even Linux is easy but I wanted test the waters of the windows 10 runtime.

I have connected the peripherals and the HDMI cable to the tele and voila, its up and running. It doesn’t do much (yet) But, there is a blinky program here which can work straight away from the user interface seen in the picture above.

Cool stuff indeed! Its got an ethernet cable, once connected it is ready to be a IOT device which can be controlled and connected to other devices. I have a very small monitor but it has the old style AV pins for connections. I will see how I can get that working along with they keyboard and mouse :)

An adventure in the Node land - first impressions

Tue, 22 Dec 2015 22:30:00 +0000

I was trying to look into Node.js which is the new phenomenon on the web these days. Lot of people (websites) are moving to Node - from ruby / java / php. Surely, I want to find out why…

I have followed Node since it’s beginning but, never really got to use it anywhere or had a chance to seriously experiment on it.

Node is effectively a new programming runtime (platform) for just about anything. The language in play is Javascript. It has various modules like ‘http’ which makes it highly useful for development of web services / apps. The runtime is very fast as it’s the V8 engine of Chrome browser. Thanks to Open source community and Google for this.

Node started with Rayn Dahl at Joyent, but now it is in the hands of a foundation. The foundation consists of members from various areas of the web. They are in charge of steering the development of node. This is good as opposed to a normal open source developer model, as everyone recognises a potential in this.

Javascript is best suited for the web, as all the client side code is javascript anyway. This makes it easy for the development of web apps as both the client and server codebase becomes javascript. Smaller technology stack. Or is it?

Javascript is a neat little programming language, which is extremely malleable. It can be moulded for any paradigm viz. it is a functional language but can be turned into an Object Oriented Language. On the other hand, this also makes it a source of a lot of problems and bugs. It is highly recommended to do production node js work using Test Driven Development.

I have been experimenting with the Chrome engine since it began, was really excited by its speed… This was back in 2011

Bezier demo

Lets have a look a how a node application works. I have downloaded node js from the nodejs website. I am planning to create just a simple hello world program that displays the current system time.

Create a file named app.js (it could be anything) with the following content

var http = require('http'); 

var server = http.createServer(function(request, response) {
    response.writeHead(200, {'Content-Type': 'text/html'})
    var d = new Date();
    var hrs = d.getHours();
    var min = d.getMinutes();
    var sec = d.getSeconds();

    var htmlContent = "<h1>Hello World!</h1><p>Time is: " + 
    hrs + ":" + min + ":" + sec;
    response.end(htmlContent);
});

server.listen(3000);

From the command line start this web service which we have written in the command line

> node app.js

After launching the browser with the http://localhost:3000 we should see the below screen

]

The difference between this and something like ruby / java or an ASP.net web application or a service is that this is a self contained - self hosted application. There is no other container requried to host this functionality. For example we have to host the ASP.net website into IIS, ruby into apache, java using jboss etc… Here in node we just have a ‘server.listen(3000)’ line. This is extremely powerful but it is also very low level. To write business applications on top of this requires considerable amount of effort. For example how do we handle routing serve static pages or resources, handle security, handle parameters…? Which has given rise to many web frameworks on the nodejs platform. One of them which seems to be quite popular is Express. More on this soon.

First impressions really are that nodejs is a growing new platform with a big user and developer base, it is actively supported, with a lot of content online. It is very lightweight and powerful runtime, which allows easy deployment also, node has got one of the best package management solutions called the node package manager (npm). Node presents a very low level js platform which gives a huge amount of power and control to the developers however, that means there has to be a reliance on middleware frameworks which will be needed to do the heavy lifting of the business applications. Thankfully there are many such frameworks now :-)

Joy of code generation using the T4 templating engine

Thu, 03 Dec 2015 19:30:00 +0000

Generating code is a truly fascinating and a very satisfying experience for me. There is something very meta about it… Effectively you write programs that write programs for you. I live by the advice (which also comes naturally to me) 3 strikes you automate. Ah well, t4 then… We will first briefly see what T4 is, then I will explain using an example for code generation.

What is T4 or TTTT?

T4 is one of the cool things Microsoft has produced, it has been around since a long time although it’s not that famous. I have been using it since about 2009 on some odd tasks / code which I found could be automated. It’s a very handy tool for generating boiler plate code.

T4 = Text Template Transformation Toolkit.

To describe it simply it’s like XSLT (Xml Transformation) or like PHP… ASP.Net’s Razor engine works on the similar principle. It can be used to generate any text output. There are two ways in which the t4 engine can be used.

Static code generation (compile time)
Dynamic code generation (at runtime)

Create a new project in Visual studio, and add the tt files into the project. I am currently using the VS Community Edition which is absolutely free and it also supports T4.

Once you create a .tt file you will likely see the below warning. This is normal, if you know its your tt file its fine.

Technically on saving the tt file, a code generator is run and that can execute all the .NET code that is embedded in the tt. This code will run with your current privileges hence, the potential to be harmful…

Understanding the tags

The tt files will contain code like below. You will see some odd tags here is a brief explanation about them.

<#= #> tags: These are control expressions, they contain only code. Any .NET code can be embedded into them. They will return a value that is written into the output of the template.

<# #> tags: These are standard control blocks, they can be staggered across the template. You can write partial code in these blocks and have pure text in between and again continue with the earlier code.

<#@ #> tags: All template directives need to be inside these.

<#+ #> tags: They contain class features, this is if we define any class within the template, you can add methods or properties etc.. within these blocks.

<#@ template debug="false" hostspecific="false" language="C#" #>
<#@ assembly name="System.Core" #>
<#@ import namespace="System.Linq" #>
<#@ import namespace="System.Text" #>
<#@ import namespace="System.Collections.Generic" #>
<#@ output extension=".txt" #>

Most of the code above is self-explanatory, the embedded language is C#, the extension type of generated file is .txt, all the other code is importing the namespaces for the embedded code.

These are known as the template directives. For more details here is the link https://msdn.microsoft.com/en-us/library/gg586945.aspx

Example

We will take a simple example to prepare a list of files in a directory, I was going to take a complex one, but it can take very long - probably I will do it in another post. The goal is to generate an XML file which contains the directory path and lists all the files in it along with its attributes.

We will only talk about static code generation here, dynamic is same in principle.

Lets say I have the below directory with following files in it.

Now, I would like to generate an XML for it.

Lets first see what the .tt file actually looks like.

There is another file which is a .txt file that has been generated by the t4 engine after parsing our .tt file, and its currently empty.

I will add some code like below to read a directory (elementary c# stuff) into the .tt file as below. Remember the <#= and the #>, anything outside those will be directly put into the output file.

<#@ template debug="false" hostspecific="false" language="C#" #>
<#@ assembly name="System.Core" #>
<#@ import namespace="System.Linq" #>
<#@ import namespace="System.Text" #>
<#@ import namespace="System.Collections.Generic" #>
<#@ import namespace="System.IO" #>
<#@ output extension=".txt" #>

<xml>
<#DirectoryInfo dir = new DirectoryInfo("c:\\tmp\\downloads");#>
	<Directory Name="<#=dir.FullName #>">
	<#foreach (var file in dir.GetFiles())
	  {#>
		<File LastWriteTime="<#=file.LastWriteTime#>">
			<#=file.Name#>
		</File>
	<#}#>
	</Directory>
</xml>

After just saving or compiling this .tt file the code generator will kick in and generate following output text or XML ;) file for us.

<xml>
	<Directory Name="c:\tmp\downloads">
		<File LastWriteTime="12/03/2015 21:31:46">
			Todo - Copy (2).txt
		</File>
		<File LastWriteTime="12/03/2015 21:31:47">
			Todo - Copy (3).txt
		</File>
		<File LastWriteTime="12/03/2015 21:31:48">
			Todo - Copy (4).txt
		</File>
		<File LastWriteTime="12/03/2015 21:31:49">
			Todo - Copy (5).txt
		</File>
		<File LastWriteTime="12/03/2015 21:31:46">
			Todo - Copy.txt
		</File>
		<File LastWriteTime="12/03/2015 21:31:46">
			Todo.txt
		</File>
	</Directory>
</xml>

That is the basic principle of the T4 templates. Static code generation is useful to generate UI code / html, generate resources / dictionaries etc. Hope it was useful.

Software developers recommended toolbox

Thu, 08 Oct 2015 20:24:00 +0000

Here is my list useful tools which have helped me in various activities involved in developing software. Most of which are indespensible :)

Code

Visual studio - can’t do any windows development without this. Visual studio premium provides really good set of tools, but the free community edition is mostly sufficient for development.

Eclipse - Really essential tool for Java / Android development for desktop / web, it supports lot of plugins and Rich client application framework. You can customise it to do any language development even ARM.

Visual studio code - this is the latest addition into my toolbox, and microsoft have done it. It’s absolutely perfect for web work / scripting quick editing.

Scite - Before visual studio code I was using this scintilla based editor as a lightweight but very powerful ruby / scripting editor

CodeMaid - Really handy free tool for cleaning up the .NET codebase. Supports batch operations as well. Supports most important customizations, could be better but still very essential.

Resharper - Probably the best developer tool (addin for Visual studio) for .NET development. It really pampers the developers, once hooked difficult to do without.

nCrunch - Concurrent test running tool. At first I thought why would I need it but, its really clever and very useful while doing test driven development. It gives a live line by line code coverage and test result while typing… imagine that.

specflow - Behaviour driven development framework and tool for .NET development in visual studio.

Git - Opensource, fast distributed version control system which understands branching properly, Source tree is brilliant application for this.

Mercurial - An alternative to Git, Tortoise Hg is a great tool for this - I was using it before GUI tools for git were not present on Windows.

Diff tools

Diffuse - good opensource python gtk based diff tool, it shows word differences, lightweight does what is supposed to and gets out of the way…

P4Merge - its a free tool from perforce, I only use it for image comparisons…

Image editing

Gimp - Fantastic tool, great effort from opensource community. This sorts my most of the raster image editing needs, I didn’t find any need for photoshop. Its got a great scripting interface Script Fu for batch editing or procedural editing.

Inkscape - Again, another opensource winner for vector graphics designing

Dia - Good free tool for doing illustrations / structured diagrams with connected elements with lot of clip art like things

Microsoft Viso - This is hands down the best diagramming tool money can buy with very big library of drawable elements. Outputs in various formats, also supports printer friendly versions. Good for professional documents.

Productivity

Powershell - Useful for automation of tasks. Powershell scripting can save a lot of time, it can interact with .NET assemblies as well

Ruby interpreter - Interactive ruby prompt is also very handy for everyday automation, scripting, file manipulation, operations on strings etc..

OneNote - Handy notebook tool which works across devices, supports clipping from internet (html copy paste), drawing etc..

Focus booster - Pomodoro technique is a great tool for improving productivity, I found this to be a great application for desktops, its free other features are not free.

Kuler - Adobe’s kuler which is perfect tool for getting colour palettes for websites / applications / designing work

TuneIn radio - Online radio to listen to while working hard, focus booster and this is a paradox :)

Microsoft Excel - Spreadsheet tool with wide applications, once if you get used to it, its very difficult to do without. The online version is free but doesn’t support scripts.

Openoffice Calc - A free alternative to Excel, not that powerful but does the job…

Task scheduler - the good old windows task scheduler can automate a lot of things, really easy and handy tool.

3D modelling

This maybe required for 3D printing, art work or game development or illustrations / walkthroughs etc.

Blender - I am a power user of blender, have been using it since the begining of it, its an inspirational opensource project. Nothing else is required for doing 3D. A 3D modeller, renderer, game and physics engine, it also supports video editing… phew. STL is supported for 3D printing. Great extensibility via the python scripting interface.

Rhino3D - Advanced tool for NURBS and surface based modelling. Not a free tool, it supports plugins.

TinkerCAD TinkerCAD is a free online CAD tool. Its not having very elaborate tools but mostly gets everything done. Good for hobby 3D printing.

The Conway's Law

Sat, 03 Oct 2015 20:24:00 +0000

This is an adage in Software Engineering, and it goes like

“organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations” — M. Conway

It’s been there since ages, even there in the Mythical Man Month by Fred Brooks. It sounds a little funny perhaps but, if we look deeper it is very natural and is a valid sociological observation.

Does the above figure look familiar?

Communication plays the most important role in software development. It sounds obvious but, for techies this has to be constantly in our mind as we tend to drift towards what we like doing. I had a change of thought with the book Domain Driven Design by Eric Evans, and his talks. The essence is that software is always facilitating a certain domain. Immediately, the focus is shifted towards the “domain” i.e. away from architecture, technology, design (all the good stuff). Hmm… well these things exist but they are driven by the domain. What we will get by this is a software that is desired by its users. It is one of the parameter of the product’s success.

Hence, we can conclude that better communication will lead to better software (it also depends on how well it will be implemented but, still its going in the right direction).

Software as a whole may be monolithic or it will be composed of various components - depending on the requirements. However, there will be parts of the software that are developed by different individuals. The way in which these individuals communicate, will affect their design.

Example

Lets say a team ‘C’ is developing a software product, team B is providing a functionality which can be integrated or consumed into the product. If teams are closely located or communicating frequently their designs will harmonious, they will not have to define their own models. On the other hand if the teams are not frequently communicating, they will come up with a more formal interface (similar to their formal communications) and each team will device their internal system which is not visible or of interest to other team apart from the common interface.

Sometimes, this system results into a cleaner design as it is separating the concerns, as the system works and everyone is happy about their components. However, if the goals of the two teams are different the designs may not be compatible. Also, if the design needs to evolve to reuse / reduce redundant code at both ends, if the changes in the system are frequent, requirements change - different teams may not respond in similar ways to them, and the design will be affected by it. For changes / scenarios like these, the teams will need to change the way they communicate.

Conway’s Law is a tool to understand as to why the design has a certain attributes, it is only a natural tendency of the system.

A handy burn-down chart excel template

Wed, 15 Jul 2015 12:01:13 +0000

A burndown chart is a common tool for reporting the progress in scrum or umbrella term “agile” world. It is a very pragmatic and simplest way to check progress of the work, if its on time, it will end at zero. It also gamifies the work if teams frequently look at it, with an aim to finish it to zero. That makes it easy to understand by anyone in the team or management. If you are using the application lifecycle management (ALM) tools like Microsoft Team Foundation Server / Jira / Mingle etc, this comes in as a featured report.

A typical burndown chart would span across a time period. I would start off with a certain quantity of stuff and progressively reducing that quantity. Examples of burndown charts in software development can be like below

Release burndown
Sprint burndown
Defects burndown

Common trends observed in the burndown charts are like below:

Above shows an underestimated amount of work or it also shows slack or not enough progress in the time period.

Above shows an overshoot and there is an offset due to more work added in between, which leads to an incomplete sprint (doesn’t end at zero).

Above shows that the work has been overestimated. Actually it has been completed faster than estimated. The team can add more work, and there is a need to re-estimate.

Generally, anyone can use burndown charts, even construction contractors, editors, movie makers, basically the people who want to manage an estimated amount of work over a period of time. I have made a simple template in MS Excel which will help chart the burndown of any work.

Using the template

Download the template from the location below. Observe the total amount of work you have started with and feed the “Starting story points” field. Provide the “Number of days” in the sprint. Now observe the work completed during each day and record the new value of the remaining work at the end of each day. I haven’t created the table for the details of work items (Tasks), as that can vary from requirement to requirement.

In the above template, there is a visual for the burndown, total amount of work completed and daily velocity. I have found that the daily velocity charting does help manage the sprint better, if the concerned people watch the day’s velocity they will notice the slack / excessive speed and apply corrections accordingly. The pie chart is a very nice way to find out what percentage of the work has been completed, generally burndown charts don’t give this type of feeling.

I hope this will be useful to people without the ALM tools. Happy work management…

Download the template here: One drive link

software nature

Software development and creativity

Software rust

Reusable components design

What is the cost of building in reusability?

When to put in the effort to make it reusable?

How much would be gained in return?

Clustering using k-Means Algorithm

Introduction

The Algorithm

Inputs

Step 1

Step 2

Step 3

Step 4

Step 5

Step 6

Interactive js example

Booting up my Raspberry pi 2

An adventure in the Node land - first impressions

Joy of code generation using the T4 templating engine

What is T4 or TTTT?

Understanding the tags

Example

Lets first see what the .tt file actually looks like.

Software developers recommended toolbox

Code

Diff tools

Image editing

Productivity

3D modelling

The Conway's Law

Example

Further reading

A handy burn-down chart excel template

Using the template