body blog - Home

rubymine

2009-07-27T16:33:00Z

scarred, but not defeated by vim, i decided to try a recently released beta of rubymine 1.5, and it’s been great so far.

massive disclaimer: i have not tried anything else for ruby/rails coding except for vim.

unlike a similar offering from netbeans, rubymine is a standalone install that seems to reuse a lot of the existing intellij idea codebase.

it was great experience out of the box – i pointed it at the local svn working copy and it verified all the installed gems (even though i am fortunate enough to run on cygwin, it recognized them all).

it has great rake support, and all rake tasks run without any modifications. however, for script/server in cygwin evironment i had to replace -e"STDOUT.sync=true;STDERR.sync=true;load($0=ARGV.shift);" with -e"STDOUT.sync=true;STDERR.sync=true;RAILS_ROOT='/cygdrive/c/project/root/dir';load($0=ARGV.shift);" in ruby arguments field. once the server runs, it also displays its log with proper color-coding.

i still run script/console from cygwin command line, since rubymine does not do readline support (i use ctrl+L, ctrl+R, tab completion, ctrl+e/ctrl+a and other goodness quite a lot in my irb).

things to love

color-coding, auto-indenting (including color-coding of matching brace/bracket/do-end block)
ctrl+shift+n/ctrl+n for finding files and getting around; with alt+f1 to show current file in different contexts/views
ctrl+f12 for current file structure
visible spaces (otherwise the ruby coding standards make the code look too squeezed to me)
ctrl+click when mousing over (jump to all kinds of things, including template names, css style names – all of it very nicely integrated)
ctrl+/ for toggling comments

rails project structure
parsing my stacktraces and linking them to the source code
autocompletion, although i do not find myself using it too much

all the usual things that idea has – svn and git support out of the box (and shelving works, just like in idea, in case you have to work with svn)
pretty sweet diff (ctrl+d) that rivals tortoisesvn visual diff
ctrl+shift+up/down to move the current line
ctrl+d to clone selection
shift+delete to delete the whole line

ctrl+shift+f12 to go full-screen
alt+number to toggle between tool windows
file structure tool window (alt+7)
alt+f7 to find usages

alt+shift+f10 to bring up run menu
shift+f10 to run current run

simple refactorings (introduce method, rename variable/method, etc).

it is pretty damn stable, and occasional errors do not kill the IDE.

things to improve

unlike netbeans, it is not a full-blown ide with ruby support, so some things that exist in idea are missing (notably, database support, some of the team communication and code sharing features, and other bells and whistles)

ctrl-q for docs is a bit wonky (frankly, i’d rather jump to matching place in the online api docs – it gives me context)
code folding fscks stuff up sometimes
still do not know how to jump to matching brace/do-end block
svn switch could not be found
ctrl+shift+f10 to run current test (and any other ad-hoc run tasks) does not work on cygwin, unless you do RAILS_ROOT trick above

i have not tried all the other stuff, like haml support, cucumber support, rspec, and rspec w/ drb.

overall feel is nice and polished – most things just work out of the box (unlike the frankenstein monster that eclipse can be sometimes – truly a Windows of IDEs).

i do believe in using “idiomatic” shortcuts with an IDE, thus i did not try any of the “compatibility” keyboard modes.

for now, i do not see myself coming back to vi for rails development – for a hundred bucks, rubymine is a great development tool.

reading: maister

2009-07-25T21:00:00Z

managing the professional services firm is my second book on consulting (first one being weinberg’s secrets of consulting).

i have been resisting the temptation to gobble up the whole book in one sitting. i am taking it slowly, giving myself time to think through each chapter, since every one is loaded with so much insight and relevant quotes, that at times i even get frustrated, not able to take it all in.

it has it all – from marketing to clients, finding work, to running a consulting business, motivating and growing people. i am only half-way through, but i am tempted to label it as a required reading for anyone starting in consulting.

here’s a quote from “motivational crisis” chapter:

Are professionals different from other types of workers? Do they need to be managed (and motivated) in special ways? While it is difficult to support the assertion that all professionals are different from all other workers, my work has led me to suspect that, when it comes to motivating forces, the average professional is different from the average worker in other environments: a difference based, I suspect, not on such things as educational levels, but on the psyche of those who choose professional careers.

The typical professional is apt to describe him or herself in the following way: “I am the type of person who gets bored easily. I hate doing repetitive sorts of work, and always like to seek out new challenges. Once I know I can do something, it tends not to satisfy me anymore.” This is, of course, a somewhat self-flattering description. In my experience, however, it is an accurate one. Professionals, certainly the best among them, are constantly driven to seek out the new, the unfamiliar, the challenging. The key word here is driven.

People who feel the (neurotic?) need to constantly and repeatedly test their skills against unfamiliar problems with an uncertain probability of success are frequently insecure, with a low sense of self-worth (never expressed in public), in constant need of external tests of their merits to prove (to themselves) that they still “got it.”

Many professionals, I would assert, are prime examples of what is now termed “The Impostor Syndrome”—successful people who live in constant dread that someone will, one day, tap them on the shoulder and say “We’ve found you out. You’ve been faking it all these years.”

Because of this, professionals tend to exhibit some clearly defined behavioral characteristics. They require continual challenge and personal growth to retain their interest, and are impatient when they do not receive it. They constantly ask themselves, and their superiors, “Am I still on track?” Because of their insecurity, and the ambiguity that surrounds the definition of “good work” in professional contexts, they need quick, repeated feedback on their performance to validate their efforts. They tend to be “scoreboard-oriented”: eager for visible, well-defined measures of success that can reassure them. They like to have unambiguous goals to shoot at. From their need to achieve self-respect by receiving the respect of others, it follows that professionals value both autonomy in their work and involvement in policy decisions, whether on engagements or firm-management matters. As much as these “rewards” are valued in their own right, they are valued more as signs that the organization trusts and respects them.

ode to vi

2009-07-24T12:26:00Z

i recently had to do a small rails project. so i did what i usually do in these cases – fired up the easiest IDE that runs anywhere – vim.

now let me reminisce a bit – i’ve been using vi on and off ever since i got my hands on linux in 1997, and i have not learned much beyond the basics over the years. it works the same way on half a dozen unixes i’ve used it on, as well as on cygwin and macs; even dreaded beasts like mks toolkit provide it. it is an indispensable cross-platform tool.

it does not require much horsepower, and it fits well with the back-to-command-line ideology of rails.

if all you are using it for is editing occasional file or two, it does everything you need out of the box. throw in basic syntax highlighting, auto-indenting, split windows, buffers – and you have enough to survive.

oh, did i mention the macho factor? it takes some effort to tame the menu-less monster of an interface with a barrage of keystrokes that appear as magic incantation to others.

vi has a peculiar physical effect – i often surprise myself when i remember certain editing commands, but at the same time i am utterly unable to remember them when standing behind someone, advising them what to type – my fingers twitch, but my higher brain functions are not firing.

curiously, this reminds me too much of some of the mainframe folks i’ve seen, or even an occasional SAP jockey. consider it a compliment – there is a lot of power in short mnemonic commands compared to drill-down menus. yes, every powerful system must have a command line, but it must degrade gracefully. with vi the discoverability of interface is pretty much non-existent, and the learning curve is steep.

my current theory is that the muscle memory vi creates leads to a particular form of addiction which explains its appeal (and perhaps the religious zeal).

when i work in a context of a project, where i constantly need to bounce around different files, vim UI starts to break down: built-in buffers support is inadequate. the editor needs to have a concept of a project i am currently working on, and, ideally, the framework i use.

i know that the usual answer is customize, customize, customize – and in this respect it follows very much a linux tradition – if you are 15yrs old, and have tons of time, and only one machine, you can spend days crafting that perfect setup that is just right. having done that a number of times, i have learned that it is just not worth my time – I switch computers often, work on client sites, bounce between different teams, so i want stuff that is reasonably workable out of the box.

i can take it easy and install some basic plugins – fuzzy finder to give me files i want fast, nerd tree for filesystem navigation, rails for rails integration, tComment for toggling comments on blocks of text.

but now i need to manage them across several machines, and perhaps i do not want to spoil my vi muscle memory that can cripple my vi-fu on that hp-ux 11.11 when i come across it (yeah right).

so perhaps i will draw the line and use some other ide for project work, leaving vi for simpler stuff.

to add some substance to this post, a few significant lines from my .vimrc:


syntax enable
filetype on
filetype plugin on

set tabstop=4
set shiftwidth=4
set expandtab

autocmd FileType ruby set shiftwidth=2|set tabstop=2|set expandtab

set number
set ai
set si

and some of the commands i use often (besides the usual navigation/editing/searching ones that are in my muscles, but refuse to be articulated):

:e! to reload the file i am currently editing
:retab i really hate those tabs

:ls to look at open buffers (the listing is a pain to read, trying to remember what those little symbols mean and matching numbers to file names)
:e filename to open a file in a new buffer
:bd to close the current buffer
:e# to bounce between two last buffers (how can i cycle between all the buffers ala alt+tab?)

ctrl+w followed by s or p to start splitting windows, then bounce around them with double ctrl+w or ctrl+w and arrows
ctrl+w followed by <number>+ or <number>- to shrink or grow the split windows
ctrl+w followed by q to close current window

zz to center screen on current line, accompanied by zt op and zb ottom
% to jump to matching brace

o and O to insert the line and switch to editing mode
I to insert at the beginning of the line (that i always forget, unlike its companion A)

m + letter to place a named mark, ' + letter to jump to the beginning of the line, ` + letter to exact position of the mark

>> and << are much easier for indenting, as opposed to my muscle-memorized number + > + enter

shift+v or v to do visual selection
once you visually selected stuff, you fold it with zf

finally, i find the whole :tabnew business utterly useless and insulting.

what i do miss in addition to project structure navigation, is easily looking up/jumping between methods, code block folding that follows language semantics, and decent tabs.

jamis buck blog entries on the subject were really informative and inspiring (especially the comments).

cloudcamp nyc

2009-06-07T14:25:00Z

i am organizing my notes from cloudcamp nyc 2009. the whole thing was done the unconference style, and i bounced around different groups for a few hours.

taxonomy

it seems that folks were struggling to come up with the taxonomy of the cloud. i see it is a continuum from bare OS instances (amazon’s EC2) to increasingly specialized services (google app engine, S3, salesforce, and even facebook).

bare OS instances are easy – nothing there is new, and what is changing is the elasticity of the capacity – you can spin up and wind down instances much faster (and recently announced amazon’s auto scaling makes it even easier). the only catch there is that you have to design for scalability yourself. but you have all the power and the flexibility, and there is no lock-in.

the latter is when you give up some of the flexibility for scalability out of the box – the design and implementation are taken care of for you – google app engine and facebook just scale. there is a lock-in there, but you might not care. somewhat unrelated, but pmarca had some things to say about it a while ago.

automation

when dealing with OS instances, these developments make it obvious that if you have not embraced automation yet, it is time to do so. the lone admin doing things by hand does not cut it anymore; even barebones shell scripts and basic packaging might not be enough.

the sad thing is that i do not see an easy, thorough, cross-platform management tool that does everything out of the box. cfengine was the closest thing back in the day, but i want something that embraces source control and declarative configuration; something that is transparent and easy to use; something that is cross-platform. chef and puppet look promising.

so there are a lot of new vendors coming out and touting their products for managing the cloud.

monitoring

same thing with monitoring – sscope was a killer app almost a decade ago, but i have not seen a compelling polished replacement (until, possibly, hyperic) – something that could scale down and up, something that can be either agentless for small easy deployments and agent-based to be able to scale beyond that. amazon’s cloud watch is a step in the right direction, but it is just the beginning.

i hope that cloud computing developments will give this industry a push it needs to clean up its act and finally come up with comprehensive convenient set of tools that is sorely overdue.

bottom line is that with clouds there will be more stuff to manage, and the demands for the management tools will become stricter.

i want commodization of these tools – monitoring, deployment, etc. ideally they will be independent of cloud provider and i want them to be simple and transparent and composable and hackable.

usage scenarios

it is almost a no-brainer these days for startups writing green-field apps to rely on amazon for their capacity. it is great, since they are not as concerned with SLAs and lock-in is not that big of a deal.

amazon truly has kick-started the industry that lurked in the shadows for so many years with all the major players trying and failing to launch a successful pay-as-you-go cloud computing service. i guess their secret was extracting a service from their own successful projects, instead of trying to come up with something new for others.

as for enterprises though, i do not see them embracing the cloud platform as it is now, however there are some scenarios possible right now, and i am sure many more will emerge:

a single batch that needs to be processed (amazon now has hadoop services ; also consider load testing from the cloud, some sort of one-off processing, or anything else that is a spike in demand could be good)

overflow capacity for something like flash traffic during holidays for retailers – extend your private infrastructure, or certain pieces of it into the cloud and wind down when you are done

POC, development – it is hard to test on the same hardware as prod, so roll your stuff out on the cloud and then for prod/uat deployment put it back inside

one-off isolated apps that can deal with current state of SLAs and lack of private clouds, or even pieces of existing apps

there is a chance for internal infrastructure folks to learn something from the cloud experience. in most large companies servers and storage have been increasingly provided as a service and billed back to the projects, but provisioning is still not elastic, and utilization is often the usual low numbers. granted, more and more are using vmware to virtualize (at least dev/uat) to increase utilization, but it is a far cry from the experience of the likes of amazon.

the ideas and some of the tech from the public clouds could and should be looked at by the internal infrastructure people, and not just the servers/storage, but also those that provide middleware services – messaging and monitoring, for instance.

clouds in your own data centers? some of the vendors seem to head this direction – give us the servers and the datacenters and we will create cloud infrastructure for you, taking care of management, monitoring, apis, provisioning, etc.

i wonder how the chargebacks in organizations will change between departments – since instead of a capital investment upfront with depreciation we can convert it to operating expense with pay for use billing – would that allow us to win anything? of course, for small companies it is great – no upfront costs, no hardware, etc.

ms azure is somewhat an outlier here – first of all, it lacks tinkerability (yes, it is a term) – the barrier for entry is too high, but at the same time it is not a vertical app, but a full-blown platform with all the development tools in place. the problem is that it is too opaque, and i would imagine it will be a while before the whole kitchen sink has proper api, billing, provisioning, etc working properly – as usual the idea is fantastic, but the implementation, knowing the track record, is something to be wary of. since windows is notoriously hard to automate, perhaps azure will finally sidestep this problem and offer that as a service, but i cannot see how it can completely do it, unless it provides something as easy and transparent as ssh+shell scripts and declarative versioned management for that.

coming up

development in several directions – granularity of service, privacy of the cloud; standards – one must talk about a certain place in the continuum – standards for infrastructure level monitoring/provisioning are perhaps possible, but once we run into more vertical services, standards make little sense, since you are buying into the provider.

there will be consolidation, since, as failures of ibm and sun showed and success of amazon shows, it is a low-margin business, and amazon knows it first-hand.

GoRuCo 2009

2009-06-06T22:11:00Z

gotham ruby confrence is the nyc’s own ruby conference organized by nyc ruby users group. it was my second time attending it. this post is an attempt to organize my own notes, as well as an attempt at feedback that i think i owe to the speakers and the organizers.

i really like the smaller focused gatherings like this with around a hundred-plus attendees – there is definitely a community spirit there, since a lot of the folks are local and already know each other.

there is also more focused discussion, since the background of the people in the room is similar. a lot of the culture is shared, a lot of the values are implied, and the conversation zips along nicely. i have stopped attending local java groups precisely for the lack of this common background – the topics are too broad, the backgrounds are too different, and it takes a lot of effort to communicate ideas.

i am a bit uneasy about my own relationship with the ruby community – i am not really contributing, nor am i doing paid ruby work. i also am a bit weary of the monoculture (look at them macboys and macgirls!) that tends to re-invent the wheel way too often. but i am there for the excitement, for the bright-eyed kids that tinker and create things – this energy is infectious and i feed off it. it is inspirational and energizing.

so why am i going to the conferences like these? in addition to the energy boost i mentioned above, there is also the trivia of learning about tools, projects, approaches; getting the feel for the zeitgeist, where things are heading, what folks are thinking. deep down inside i am always looking for the “blow your mind” experience, something that can turn a familiar topic on its head, something that can make me discover things i have never suspected existed.

GoRuCo had a nice balance – things technical and detailed, and also approaches/techniques/principles. the reject conf at the end – a series of quick lightning talks – was an icing on the cake, stuffing you full of references, pointers, tips that you could take home and work through at your own pace.

Gregory Brown: Where is Ruby really heading?

more of a book report, talking about different versions of ruby out there. for anyone following the community, none of it was a surprise. no hard data either, just his personal anecdotal experience. this part of the talk was more suited to a short user group presentation.

there were a few nice tips and personal war stories related to moving between 1.8.6, 1.8.7 and 1.9.1, unicode, side-by-side installs and very basic crude techniques to code for different versions.

these are typical growing pains – everything from the compatibility issues between versions to the curse of the system-wide install that makes one go an extra mile in order to run different apps under different ruby versions on the same box (i always preferred the semi-structured self-contained java jdk installs and jars controlled by the classpath).

there were some props to jruby (it is a real distro, not a hack to reach out for when all else fails!), mentions of ffi in jruby that allowed gregory’s project to run on windows.

Eleanor McHugh: The Ruby Guide to *nix Plumbing

this could have been a great talk, but it seems like eleanor was really hungover, so instead it was a very disconnected series of ramblings on the general subject of unix and coding. very poor delivery, and at times plain embarrassing.

only towards the end she managed to find the message for the talk, which was “you can code against kernel internals using ruby, since it makes it much easier. do not automatically assume that you need to write in C for performance – try ruby first.”

she also highlighted ruby community’s respect for bare-metal – the ability to tinker, the taste for small simple tools that do the job well, and the affinity for unix. this is exactly what attracted me to the language in the first place.

there were a lot of references that at times seemed like name-dropping: ngnix, beej’s guide to network programming, beej’s guide to unix interprocess communication, ruby/dl, duby, event machine, c10k problem, advanced unix programming book

Dan Yoder: Resource-Oriented Architecture With Waves

dan briefly talked about his waves framework that tries to present a simple DSL around HTTP and resource representation in a REST fashion.

the meat of the talk that was interesting to me dealt with REST in general, its differences from MVC, resource-oriented architecture, self-describing data, returning links to other data inside of data to aid discoverability. he mentioned how a resource identifier (e.g. URL) should not specify representation (i.e. do not add .xml to specify that you need an XML document back), but rely on client’s Accept* headers to negotiate representation (caveat being that CDNs like Akamai do not currently care about these headers, so you will always get the same content).

there was also a mention of the fact that sometimes one can view HTTP protocol as something dealing with a distributed hash table (DHT) using get/put/delete operations (with post reserved for everything else).

RDF and freebase were also mentioned.

i have a strange relationship with REST: i get it on the technical level, i sort of get it on the architecture level, but it did not fully “click” yet, perhaps due to the fact that i do not have enough practical implementations under my belt. i should go through the restful webservice book again and play around.

good talk overall, but lacking concrete examples that could have brought things more into focus.

Jake Howerton: Into the Heart of Darkness: Rails Anti-Patterns

a bit of a disappointment, since the title promised so much. i usually really liked the anti-pattern talks, since you learn nothing when things work as expected – the real learning comes when things break, and you are forced to dig in and figure out why. plus these talks also give you an idea of applicability of certain techniques, which is really a required counterpart for all patterns to begin with (beware of dartboard-driven design).

instead we were treated to a very few amusing short code snippets, but no larger patterns in the sense of fowler’s refactoring book or even rails-specific patterns.

he did mention cucumber, reek and metric_fu and some general well-known testing techniques. i liked his term “flight check” for the smoke tests that run before deployment to prod, and a notion of sandbox test environment where mocks are replaced with real classes that do destructive things without impacting the real world (like sending emails).

i also like his term “irb-driven design” for something that was copy-pasted from the exploratory irb session into the production code.

there was also some treatise of legacy code, but i think dhh’s talk on the subject was much better.

overall it was a fun, light talk that was well-received. i blame the deceiving title and the lack of focus for initial feeling of disappointment; it felt like jake was simply talking about things he does and prefers to do during development, without specific overarching theme in mind.

Sandi Metz: SOLID Object Oriented Design

sandi stole the show; she set the level that none of the speakers matched and showed what it really means to have a solid, gripping presentation that is lucid, focused, well-prepared, and superbly delivered.

she talked about SOLID design principles: Single Responsibility, Open Closed, Liskov Substitution, Interface Segregation, Dependency Inversion which all boil down to managing dependencies.

she went through the fowler’s value of design argument, and then iteratively went through a refactoring example, invoking the SOLID principles along the way.

i think the most impressive for me was how well the whole talk came together, how the arguments were presented to support the refactorings, and how well it got into my head (almost uncanny, similar to the effect the best books in head first series have).

some of the principles she kept bringing up – using the rate of change as the indicator for splitting the functionality; refactor in small steps to let the design emerge – not because you know, but because you want to find the design; red-green-refactor; mock at the seam; only mock classes i own; those that change often should depend on those that change less often.

this once again brings up the importance of developing a language to discuss design (e.g. patterns) – i think this is a crucial step for every practitioner, when something intuitive and personal (e.g. a vague code smell) becomes something that you can articulate and communicate to others.

she also mentioned uncle bob, micronaut, steve freeman and nat pryce mock objects site

Benjamin Stein: Building Cross Platform Mobile Apps with Ruby & PhoneGap

the main promise of PhoneGap is quite compelling – build cross-platform apps on the phones using javascript while taking advantage of the native features (vibrate, storage, accelerometer, sound, gps, etc).

it was a fine presentation, with a story to tell and with a perspective that put all the low-level details in context. makes me itchy to get my hands on some phone development; i was also quite impressed with their adoption of latest standards (e.g. HTML5).

Yehuda Katz: From Rails to Rack: Making Rails 3 a Better Ruby Citizen

rails3 perspective from the horse’s mouth – quite detailed look at how rails is trying to be less opinionated in its choice of frameworks to work with, and how it exposes its internals for others to integrate with (orm, rack, js frameworks). some very interesting examples of design decisions and the overall future path of the framework.

this was a bit tedious, but quite informative talk.

Lightning Talks

i really like these, since most people do not have enough material for a long talk; in fact some of the main talks earlier should probably have been half their size.

the highlights include sunlight foundation and data.gov plug, a great pair of fast talks by aman gupta on google-perftools that he tweaked to work with ruby and used on some real-world code and joe damato on tweaking the thread performance in ruby 1.8 (in retrospect, these two talks were something i wish eleanor would have done).

michael stonebraker

2009-05-26T11:55:00Z

i really enjoyed this talk by michael “map reduce is crap” stonebraker. it is a terse and mind-blowing romp through the current database landscape, focusing on inability of one-size-fits-all “dinosaurs” to deal with streaming data (like tick market data) as well as data warehousing. he has some benchmark numbers and proposes some alternatives.

i have been looking into streaming databases and column data stores for a while, but I am yet to have a production hands-on experience with them.

i always had somewhat of an inferiority complex when it came to databases, since these guys have done so much work decades ago that tackled so many problems, and it seems to have had a hard time escaping the database world and affecting CS population at large.

so this new twist in the evolutionary spiral that comes due to the availability of cheap machines, lots of RAM, and the need to process lots of data should yield interesting results. i do hope that this will lead to a whole bunch of lean and mean specialized solutions instead of (loosely quoting stonebraker), “here’s oracle, what’s your problem?” approach.

the essential drucker

2009-04-14T03:39:00Z

perhaps it is my destiny to stumble into things that i should have known (and probably have known at some point, but they only became relevant now). in any case, i am happy to add drucker to my pantheon of striking bald men (already staffed with the likes of foucault, seth godin, and crowley).

i really enjoyed this updated/revised “best of” collection of drucker essays. he is not instantly quotable – i do not think he goes after the effect of catchy parables, so it is hard to pick out the pearls to quote here.

but of course, i cannot resist:

[...] the three stonecutters [...] were asked what they were doing. The first replied, “I am making a living.” The second kept on hammering while he said, “I am doing the best job of stonecutting in the entire county.” The third one looked up with a visionary gleam in his eyes and said, “I am building a cathedral.”

perhaps proverbs and fables are a retreat for those that cannot come up with an original wording themselves, but dammit, this is the kind of stuff that you hit people over the head with to get your idea across.

anyway, i really liked his writing on building innovation into the organization. his essays on the place of the organization in the society were also quite eye-opening, especially with respect to the non-profits. he does not go into heavy theorizing – everything is quite high-level and appears simplified, but there are a lot of good observations, especially once you read them in context. yes, some of it is a bit dated, but it is easy to make a quick mental adjustment to see the core of what he is getting at.

i’ll have to pick up a few more of his books; at this rate i will be reading f. w. taylor in no time. oh boy.

manual override

2009-01-29T06:21:00Z

there is something i noticed about myself – oftentimes, when faced with a problem, i immediately jump to the technical means of solving it, which usually implies building some sort of automation. after that jump, the initial problem becomes less and less important, while i am speeding away, thinking through the technical details.

it turns out that sometimes an excel spreadsheet and a manual process is far superior solution, although it might hurt your ego and make you feel vaguely dirty.

another variation of the above is fascination with “cool” tech (for your definition of “cool”) – i noticed that sometimes i would use every opportunity to startup wireshark or Process Monitor and gloriously wallow in data, gleefully probing tiniest details. it turns out that almost always there are better tools out there that can get the job done with less noise.

so as a personal first-line defense, i opt for trying to ask the right questions early on – perhaps a manual solution would do, or a problem really is a non-issue.

it seems that too much knowledge may be a curse, but the real challenge is to keep your personal toolbox organized.

elevator test

2009-01-29T05:23:00Z

building software for a business, i established an interesting test that shows me whether i am thinking in the right direction:

off the top of your head name five most important things that technology can improve for the business as it is right now

i’ll wait – can you do it? right here, on the spot?

the way i see it – if, after having worked in a new team for a few months, you cannot do it – you have failed. you are not thinking about the business, you do not know the business, you do not care.

as technologists we are all too often drawn to the technical problems, forgetting that it is really the business that we should have in mind. the catch is that the domain of the software problems is infinite, so one can submerge forever without delivering anything of value to the business.

disclaimers apply, of course – it particularly matters for those developers working closely with the business. but then your definition of “business” might vary; the point is that sometimes a correct technical solution is the wrong one.

so while you are at it, type up this list and stick it somewhere visible, so that you can have it in front of you every day (as i’ve talked about it before).

Public Enemy No.1

2009-01-29T04:33:00Z

Alex Miller aka Pure Danger Tech has a great entry on most common concurrency bugs.

the timing is perfect – this very thing bit me today. long story short, an old app, previously perceived to be multi-threaded, was recently converted to actually be multi-threaded, and then, once traffic ramped up a bit, exhibited peculiar behavior when perfectly good dates could not be parsed. thank god it blew up, as opposed to quietly corrupting the data.

so something as innocent-looking as private static final SimpleDateFormat declaration was the culprit: java.text.DateFormat is not thread-safe.

luckily, it is easy enough to spot and reproduce (threadPoolSize and invocationCount in TestNG simplify it even further).

a pessimist would heave a mighty sigh, once again swear on the copy of JCIP to find and root out every frivolous static out there with the help of FindBugs or a simple regex.

meanwhile, there is joda-time and a promise of jsr310

but of course this whole experience still leaves you feeling cheated and dirty – why, god, why?! something so function-like and stateless in nature insists on stowing things away.

go with the flow

2009-01-05T08:20:00Z

a damn nice quote from Eric Brechner’s I. M. Wright "Hard Code" blog regarding team dynamics, specifically about retaining people:

[...] the best way to deal with turnover is to expect it and embrace it. How? Think flow, flow, floooooooow.

Think of your team as a river instead of a lake. A lake stagnates. There’s no energy or impetus to change. The same is true of groups that stagnate. They cultivate mediocrity and complacency; they abhor risk. A river is always running and changing with lots of great energy. You want a river.

a nice metaphor and it strikes a chord – building an ideal team is one thing, but perhaps it is more about building a self-sustaining culture when individuals might be coming and going. the latter seems to be a more important goal in the long run.

depth-first search

2008-12-23T06:27:00Z

decide to write a blog post
- play with formatting
  - realize that there is a new version of a blogging tool
    - start upgrading blogging tool in test in vmware instance
      - realize that new vmware server is out
        
        upgrade vmware
        
        upgrade vmware tools (never easy!)
        
        play with new vmware server options
      - realize that the new version of os is out
        
        upgrade the os
      - notice that logrotate is not working correctly
        
        fix log rotation
      - realize that rsync backups are b0rked
        
        attempt to fix them, but fail
      - upgrade db while at it
      - upgrade web server while at it
    - screw around with formatting on the upgraded blog platform
      - fail to get pagination to work (like i fail every single time)
      - fix the theme since upgrade broke it
    - start upgrading prod and realize it does not have enough physical resources
      - consider switching hosting providers
        
        really consider switching hosting providers
        
        ...
        
        ...
        
        ...
        
        ...
        
        no, no, no! there lies madness!
        
        stack overflow!
two days later write this blog post

concurrency: part 2 - actors

2008-09-19T11:59:00Z

part 1

message-passing

if shared memory makes concurrent programming difficult, what else is there that an app developer can use?

one way of representing coarse-grained parallelism is through message-passing concurrency.

the idea is pretty simple – the only way to share state between isolated components is through message passing. what happens inside the components, is their own business.

there is no global state of the whole system, unlike in shared memory program that behaves like a giant state machine. sending and receiving of messages happens concurrently together with the computations by the components themselves. this approach is much easier for the developer to reason about and it maps easily to multiple CPUs and multiple machines.

a lot of the enterprise workflow-like processing falls into this model. it is basically a pipeline of worker pools, configured independently and possibly running on separate machines. a unit of work travels through the pipeline, as it is being handed off from one set of workers to another.

actors

one of the common implementations of message-passing concurrency is actor model. i’ll take a liberty to interpret this model to fit my own developer needs, even though i am butchering decades of academic research in the process.

actors model represents well multiple computers talking to each other using network packets, components exchanging messages using JMS, independent processes or threads talking to each other using messages – basically anything where isolation is possible and interactions are loosely coupled.

usually each actor has a mailbox associated with it (often represented with a queue), where messages are stored until an actor processes them. messages map well to physical artifacts in the real world – they are immutable, and only one actor can handle a given message at a time.

actors are connected with channels; individual actors are isolated from each other – a failure of an actor does not affect another actor. no other actor can change the inner state of a given actor – the only way to communicate is through message-passing.

messaging is usually asynchronous, but synchronous messaging could also be useful.

depending on implementation, beware of deadlocks if you are using synchronous messaging. another issue to keep in mind is order of messages – depending on implementation it might not be preserved.

while some advocate “everything is an actor” approach, and I get dizzy imagining the possibilities, the pragmatic app developer in me is living in the real world among existing apps. in this case actors work best as a library for the existing language.

erlang

although i shied away from “actors everywhere” approach above, erlang is the most successful implementation that actually does just that. it is not just the language, but a whole platform that transparently runs actors within a single process as well as across multiple machines.

as this topic is heating up, one should at least read the book and play with the language. after all, a language that doesn’t affect the way you think about programming is not worth knowing, and erlang is enough of a paradigm shift to kickstart your concurrency thinking.

Tibco BusinessWorks

as i’ve described before, BusinessWorks (BW) is an example of an integration DSL that happens to use actors.

given an integration process (e.g. receive a message on JMS queue A, enrich it from a database, transform it, and send it to a JMS topic B), you describe it using BW language constructs. then it becomes an actor definition that you can deploy on an engine (really a managed JVM instance). there could be multiple engines running on multiple machines, and each engine can have many process instances (aka actors in our terminology) running inside of it. a process instance gets created from a process definition whenever a new message arrives on a queue (mailbox in actors’ terminology).

a scheduler inside the individual engine takes care of creating process instances (there could be thousands) and scheduling them on the worker threads.

all of this mapping happens at deploy time, as a developer you do not worry about it.

actors talk to each other using message-passing, thus your actor implementation does not even have to worry about threads or concurrency – you just express your integration logic. you could use shared memory, but it would not scale well, since you are limited to one JVM; nor would it be natural, since you have to use explicit language constructs; this language support for immutability is very convenient, as i have mentioned earlier

if you use a JMS server to pass messages around, it becomes a sort of a mailbox, holding messages for you in the queue. each incoming message would eventually spawn an instance of the actor, feeding it the message as an argument. multiple instances of the same actor can read from the same queue, thus achieving load-balancing.

once you recall that jms supports ~~-filters~~- selectors you have the actors implementation that curiously matches something like erlang

note that this is not fine-grained parallelism; your units of work are more coarse-grained and very loosely coupled, but fundamentally, the model is the same, and it scales like crazy achieving massive throughput.

even if you do not end up using BW, you can implement this model by hand relatively easy.

so what if i wanted more fine-grained and more efficient support for actors in my language of choice (provided i am not using erlang)?

ruby

revactor networking library includes actors implementation (also see this great intro to actors by Tony Arciery), but i have not seen a more generic approach yet.

note that ruby is really hampered by lack of proper threading support; this is why jruby guys are in a much better shape if they were to roll their own actors implementation.

scala

this is probably the most mature implementation i’ve seen (see this paper). they take advantage of scala language features to simplify the syntax and unify synchronous and asynchronous message-passing. individual actors are represented as threads or more light-weight primitives that get scheduled to run on threads in the thread pool. it is type-safe, but it relies on convention to make sure you do not mutate your messages.

although i could see where representing actors as threads could be too heavyweight for some tasks, in the case of java and scala, your mileage may vary (see this presentation from Paul Tyme).

groovy

given language features like closures and general simpler syntax, together with the fact that it sits on top of JDK that includes java.util.concurrent, one would imagine that groovy would be a perfect candidate for actors implementation. however, the only thing i found so far was groovy actors, and it seems to have been dormant for a while.

python

i do not know enough about python’s memory model and its implementation, but i suspect is suffers from the same “feature” as ruby – i.e. global interpreter lock, which means that it won’t be able to scale to multiple CPUs (and, similar to ruby, jython that builds on JVM comes to the rescue).

the only thing i’ve looked at so far is stackless python, which is a modified version of python that makes concurrency easier (see this tutorial by Grant Olson that also includes actors). it introduces tasklets aka fibers, channels, and a scheduler among other things.

java

this is where i am a bit surprised – i do not see a good drop-in-a-jar-and-go actors library blessed and used by all. there seems to be some research projects out there, but i want something that works for me now and supports in-memory zero-copy message passing, sync/async messaging, and type safety. i am OK with abiding by conventions instead of compiler checking things for me.

i suspect that the reason for this is the fact that some rudimentary form of actors can be implemented relatively easy using existing concurrency libraries, and this approach is intuitive without putting labels on it.

nevertheless, this is what i found:

jetlang is a port of a .NET library and looks at Scala actors for inspiration. it is still quite beta, but it looks promising
kilim (from one of the principle engineers of weblogic server) still seems to be a bit too much of a research project for my taste, but the theory behind it is sound

and there is a number of research projects out there:

SALSA
functional java contains actors implementation; see this blog entry as well as this one for examples
Communicating Sequential Processes for Java

bottom line

actors is a great abstraction, and “good enough” version of it is easy to implement – think about it, consider it, use it!

it helps if your language/platform supports concurrency primitives to build upon. this includes true threading support that scales to many CPUs, although we could also benefit from a standard fibers implementation, since they are more lightweight than typical threads and would allow creation of a large number of actors that later could be mapped onto threads for execution.

each language could benefit from a well thought-out actors library, since it would push developers in the right direction.

it is not right for everything though – it might not be fine-grained enough, it might not map well to problems that rely on ordering of messages or presence of any other state across multiple actors or multiple messages.

to be continued

what is on the horizon that is worth noting? what are some of the interesting research topics? what have we forgotten over the years? what other heuristics/patterns and libraries could be immediately useful?

part 1

concurrency: part 1

2008-09-12T09:10:00Z

true to the purpose of this blog, below is an attempt to organize my (admittedly very superficial) experience with concurrency.

part 2

my 10GHz CPU

you probably noticed that moore’s law does not really apply anymore when it comes to CPU speed. if it were holding up, we would have had 10GHz CPUs by now, but for half a decade we haven’t really moved past 3GHz.

that is to be expected for the current generation of hardware – the gains have to happen elsewhere. for a little while we’ll get performance boost due to increase in size and speed of the caches that would improve locality, but in the long run it seems that multiple CPUs is where the improvements are to be mined from (this also includes specialized CPUs like Cell and GPUs in general).

this means that more and more people will have to think about their applications in terms of parallel processing. this also means that optimizations will become more and more important for those workloads that cannot be parallelized and therefore will be stuck on a single CPU (for a good introduction see The Free Lunch Is Over at Dr. Dobb’s Journal).

the bottom line is that as an app developer you cannot ignore the problem any longer; to make matter worse, there is no automagical solution in the nearest future that would make your application take advantage of multiple processors.

my concurrency story

in past decade most of the stuff i’ve worked with had some sort of coarse-grained parallelism; the rest was taken care of by the underlying framework.

i started with a unix philosophy of small programs connected via pipes, each performing a simple task. a little later came in fork and signals. things were simple, and OS took care of everything.

then came the web – it was mostly stateless with the database doing all the heavy lifting when it came to shared state. we just added boxes if we needed to grow. in ETL multi-box, multi-cpu setup was also natural, and the tools were designed to conceal concurrency; same goes for integration, where concurrency was at the level of data flows, which made things rather simple.

it is only in the past year or so when i had to really dive in deeper into relatively low-level concurrent development with java.

my dog-eared copy of Java Concurrency in Practice has proved to be quite an indispensable reference. the book is a bit uneven, and the editor should have spent more time on it, but you get used to it. it is a great practical resource, especially in the presence of so much confusing and incomplete information online.

jsr-166 introduced in java5 (and the primary subject of the book) is such a productivity boost; being a part of JDK, it is a big step forward towards letting mere mortals like me really embrace concurrent programming.

i find myself using Executors convenience methods all the time: it is so easy to create a pool, and then just feed it Callable instances, getting a Future instance as a handle in return. if more flexibility is needed, i use ThreadPoolExecutor. Queues are great as a communication channel for any sort of producer/consumer scenario, anything that requires message-passing or any sort of other work hand-off. Atomics are also great – i do not have to think twice when implementing counters or any other simple data structures.

most of the time i do not even have to work with threads or low-level synchronization primitives directly – they are buried deep within the libraries. i have less nightmares, since i do not have to touch volatile as often.

at some point i’ve read both editions of doug lea’s book, but i was always hesitant to recommend it; i’d rather rely on libraries that abstracted all of this away. now that java.util.concurrent has been out for 4 years, and Java Concurrency in Practice has become a bestseller, there are no more excuses.

one thing i’ve learned though – when you think you got this stuff, you discover a whole new class of problems that make you realize how complicated all of this really is, and how truly difficult it is to write larger concurrent programs.

you really, really have to think hard about how you share your objects, how you compose them and operate on them. you need to really understand how the language and the runtime work (i find myself checking JLS quite often). this is where good OO practices like encapsulation become even more important, since you are not just risking maintenance overhead, but you are risking the correctness of your program.

now, i always told myself that programming is not an exercise in manliness. i am just an app developer; i want to ship a working code that solves customer’s problems, not spend countless hours trying to reason through non-blocking algorithms just because i decided to do something non-trivial with ConcurrentHashMap. at the same time i do not want to waste my precious CPUs, so what am i to do? shouldn’t this stuff be easier? is there something I am missing?

threads considered harmful

actually, there is no problem with threads per se; the problem is with shared state.

in a normal sequential program you only worry about the logic as it is unfolding before you – one statement after another, in order. in a concurrent program that uses threads and shared state in addition to all your usual problems you also have problem of the non-deterministic state: since at any point in time any thread can come in and mess with your data, even between the operations you considered atomic before (like counter++), the number of states that your program can be in suffers a combinatorial explosion. this makes it really hard to reason about its correctness.

your code becomes brittle, sacrificing failure isolation – one misbehaving thread can potentially harm the whole runtime (a good analogy is BSOD caused by a device driver).

in addition, things don’t compose – a transfer operation performed by a thread-safe customer between two thread-safe accounts is not going to be automatically thread-safe.

to make matter worse, some of the errors remain hidden when run on commodity 1-2 CPU IA32 hardware, but as the number of CPUs grow, or their architecture becomes less restrictive to help with concurrency, things start to break down.

for more thorough discussion see The Problem With Threads by Edward A. Lee and Cliff Click’s We Don’t Know How To Program…

now what?

a natural reaction is to forget about fine-grained parallelism and offload the hard stuff onto someone else. after all, i am an app programmer, i care about business problems, what’s all of this yak shaving about?!

in some cases we can get away with firing up individual processes to take advantage of multiple CPUs. most of the time though it means that the problem has been pushed further down the stack, which often turns out to be the database. this is the route that rails folks went, and it certainly was pragmatic approach at the time (now that they are forced to deal with efficiency, threading is back in the picture. for discussion of issues see Q/A: What Thread-safe Rails Means).

if you can get away with using individual processes, go for it (see google chrome) – you get failure isolation, you get immutability in respect to other processes (it won’t be as easy for another process to mess with your data), and as an additional benefit, you get to use all the standard tools that the OS has when it comes to managing and troubleshooting processes (as opposed to using often incomplete and idiosyncratic tools for thread management that your runtime platform of choice offers – if any).

still, as we need more and more fine-grained concurrency and as the level of concurrency increases (it is not just a handful of CPUs now, but dozens, and even hundreds), one process per task becomes too expensive (context switching, high costs of creating a new process, memory overhead, etc). so we are back to some sort of lightweight thread-like primitives running within the same process, sharing some common resources.

most of the popular languages/platforms these days provide some sort of threading and shared memory support. but as outlined above, they suffer from some fundamental problems. there are some practical things at various levels of abstractions that that can help: low-level constructs within the language/platform itself, tooling, and higher-level libraries/mini-languages

language

make immutability easier – take note of functional languages, but also make it practical. in java case, for instance, it could mean extending immutability to some core data structures (see scala collections) or making it easier to tag an instance as immutable (see ruby’s freeze; this reeks of boilerplate though) – this way errors will be caught at compile time
consider sharing data only through explicit, ideally checked at compile-time, means. thus by default nothing is shared, and in order to make something shared you have to explicitly tag it as such. ideally, this would also come with some sort of namespace support, thus limiting mutations to a sandbox (see clojure for reference)
make language safer to use when it comes to exposing shareable state (this is when something like static becomes a problem – see Shared Data Considered Harmful for an example that applies to concurrency)

tooling

static analysis tools might help, but we need to give them a bit more than just an infinite number of states. findbugs for instance, supports concurrency annotations and something like chord could also be promising. this stuff is complex though and there are limits to static analysis (and i do not even want to bring up formal proofs using process calculi)
i want more support from the platform to help me troubleshoot lock contention, deadlocks, cpu-intensive threads, and other concurrency-related infrastructure. sun’s hotspot has some rudimentary stuff in place, but i want more things out of the box (azul claims that they have always-on built-in tools in their product, but i have not played with them)
speaking of azul, i need to study them more. although perceived as a boutique solution, they are addressing issues that everyone will be facing in just a few years. seems like they ported sun’s hotspot to their hardware which allowed them to achieve scaling by automatically replacing synchronization with optimistic concurrency which scales much better. incidentally, this truism about optimistic concurrency has been obvious to database folks for decades

libraries/mini-languages

one of the approaches is to focus on your problem domain and come up with a library/language that solves your particular problem and abstracts away concurrency. web frameworks (J2EE, rails), or ETL tools, or even databases are all examples of such approaches.

this is where my interest lies as an app developer – how can i make concurrent programming easier for me, the layman.

the bottom line is that if we insist on using low-level synchronization primitives, it would be really hard to paper over the underlying complexities. right now there is no generic universal approach that will simplify concurrent programming. so at this point a pragmatic programmer is left with patterns, supporting libraries, and heuristics.

to be continued

there are some patterns (for the lack of a better word) that i found to be helpful in dealing with concurrency; there is also some stuff on the horizon that promises all sorts of benefits – is there really a silver bullet? but also there is plenty of stuff that has been with us for decades, and i would be the first one to bow my head in shame, acknowledging my ignorance.

part 2

focus

2008-08-22T15:10:00Z

this is a bit of a wandering post, but i know i think best through writing things out, and while personal journals are a great thing, i cannot force myself to write them if there is no (even hypothetical) audience.

i think as i get older, i realize that there is so much stuff out there that i am interested in, that the importance of focusing is becoming increasingly apparent.

the distractions are getting more targeted and more fine-grained – it is getting simpler to fill up all the cracks of time between activities – reddit, twitter, google reader – scan the headlines while you are waiting for the build, and boom – you’ve blown your stack and lost the context. this is when discipline often loses to muscle memory; i suppose it is the usual story of combating the addiction. all this technology i surround myself with to be more effective just makes it simpler to waste time without achieving anything.

in this sense (all other things being equal), i find pair programming to be very helpful – it is so much harder to get distracted with someone else sitting next to you.

in certain cases i’ve observed that the proverbial order of magnitude difference in productivity between individual developers can simply be attributed to the ability to focus.

this became even more apparent to me in the past year at work as i have been faced with thousands of stored procedures for a given system – some of these storprocs being 2K+ lines long and recursive(!). talking about a new level of immersion!

one behavior i am guilty of is perfectionism in the face of overarching bigger goals – trying to make something flawless all the while the solution has to be out now and has to be good enough. i think i became immune to rolling out frameworks when a quick point solution is needed, so now the battle has moved to a finer level – testing as risk management, for instance – in the absence of time i test the stuff that is most likely to contain bugs, and i test coarse-grained user-level functionality first instead of polishing up individual unit tests.

but really, all this “micro focusing” business above is not that bad and is quite manageable on day to day basis. what i am trying to figure out though is how it applies to my career in general.

my problem right now is that after a decade in IT i realize that i have some hands-on skills that would always be in demand, and that there always will be plenty of filler work that will keep me busy and provide some sort of satisfaction (the kind similar to mindless exhaustion you feel after a workout).

i know i tend to become absorbed in day to day stuff, and lose the sight of everything else. NYC is dangerous in this respect – the pace and the energy make you work harder, and one can get fixated on something with a lot more intensity. this is when i find myself performing all sorts of yak shaving feats; is it really what i should be spending my time on?

i think paul graham got it right in his essay – NYC has the drive and the ambition, and i feed on it, but i also need to be fully cognizant of its true target. doing something pointless twice as hard isn’t something i am after.

so what is it that i should be doing? so let’s see if i can talk through this.

i always took pride in knowing a great deal about a lot of technical things all the while aspiring to grasp “the bigger picture.” i enjoy being able to build things, drive them to completion, and deliver. i also know that i like dealing with people, solve organizational problems, motivate and build teams.

i really enjoy doing hands-on stuff, and i am pretty good at it; at the same time i’ve seen all too often that most of the technology problems stem from people problems. it pains me to be on the receiving end of business blunders that render my technical efforts useless.

one of the trends in large companies is increasing specialization on many levels of IT. it is inevitable to a degree, but this is precisely where my skills would be least utilized. the image that haunts me is the nine to five drone with atrophied knowledge of everything but the immediate responsibilities of everyday job, unable to survive outside of the large corporation. there are plenty of slots like that – just waiting to be filled.

i want to work in the environment where people around me do have a vision that we all share, and this is what drives our everyday work.

i also like small teams (or small companies) – you get to do a lot of different stuff, you learn much faster, and you feel the impact you make.

so ideally it comes down to focusing on my strengths and consciously building upon them, which means spending less time on everything else.

it sounds quite absolute and final (an also quite fluffy), so perhaps a better approach is to set a goal, timebox it, and focus. if it works – great, and if not – review and start again.

current work: spend less time on it; spend less time on filler work; focus on business – actually read the pile of books i have here instead of reading up on tech stuff.

spend more time on hobby projects, get back into languages/platforms i abandoned, pick a few projects and actually contribute; build the brand – push for talks at user groups, use real name in forums, blog more on technical topics.

as far as career choices go:

i’ve done the enterprise architecture thing for a couple of years before, and despite the raging pundits that the title has accumulated, i think there are great things to be done. you do not have to abandon the technical skills, but you have to realize that your success does not depend on tech anymore. you do end up building and running systems in this role, but they also include people, culture, and organization – not just technology. there is a lot to be said about systems thinking, and there is plenty more reading i need to do. i am also yet to find a good no-bullshit community that thinks along these lines (so far i’ve been really enjoying Stu Charlton’s blog for instance) and people i can work with and learn from…

...which leads me to consulting. what appeals to me here is focus on solving a particular problem (compare that to a less focused role of a regular employee), as well as a great variety of problems, and generally a higher level of expertise expected. you grow and learn so much faster, and ideally this might lead to a product idea, if you see the same problem repeated over and over again. one skill that i know i have is what jerry weinberg calls “jiggling” (see Secrets of Consulting and Are Your Lights On?) – helping people get “unstuck” by asking questions, suggesting tools, practices. often i realize that in many teams although i could do all the work myself, the team would get so much more out of me if i create an environment where people can learn and grow on their own.

another option is a product/program owner in a larger company. the biggest selling point is the diverse set of skills required and the sense of ownership. you have a bigger picture, you are responsible for it, and at the same time you need tech, people, and organization to be a success.

the refinement of the above is a startup – you build and deliver a product. your impact is tangible, you actually “own” something, and you contribute on many levels, which is a great use for a broad set of skills.

and the last option is to get a foot in the door in a larger company as a regular developer (since this is an easy lowest common denominator that is versatile enough and faceless enough) and then try and grow into some sort of a role similar to one of the above. it is terribly inefficient though – most of the energy will be spent on the wrong things.

it does help to write all of this out – having a stake in the ground is a start. let’s see how it goes from here.