“Leap – is for when you know where you’re heading and you can afford to stop everything until the new way is in place. From an architecture perspective it is a bit like starting from scratch. From the practical perspective that would mean breaking the build until a few related changes are in place”

Basically making an architecture leap means that what we did so far is not good enough for us to go forward on. It also means that the current architecture is too far from something useful to bother with taking the more evolutionary approaches to getting it there.

This sounds very much like a catastrophe and it can be if you find out you need a leap towards or after you went into full production. In my opinion, however, there are circumstances when making a leap is a valid and reasonable (and maybe even expected) strategy :

• When the project at hand is breaking new grounds (or is a new ground for the team)
• When the goal of the development phase was for a specific – e.g. The goal was time to market or better understanding the of the technology/business constraints (a.k.a. throw-away prototype).

For instance, when we started out on a new project and .NET was new (beta of 1.0) we took our best developers and sent them off-site to work with Microsoft to build a prototype architecture. This served both for getting the team up to speed on the technology and to understand the limitations of the technology so that when we designed the architecture later we made less mistakes.

It is probably needless to say but the above scenarios are likely to be at the early stages of a project. The question then is does the advice I provided in part IIIto invest some up-front design is still valid – if you are going to leap anyway, up front design seems like waste.

My opinion (and experience) is that you need to set up a working architecture for the first phase because

• You need to make some architectural decisions just to start moving – for example in one of my past projects we had several teams that were geographically dispersed, in order to begin moving forward we had to somehow divide the work so the teams would be as independent as possible – that’s an architectural decision right there.
• The initial architecture can still serve you well – For example when I worked on Biometric systems we’ve built 2 smaller scale solutions (stand-alone station and client-server solution) based on a simple(r), less scalable architecture before we turned to building the full-scale platform. We were still able to sell those systems as they provided business value
• You can’t be sure you’d even have the time to do a leap – For example in xsights, god rest her soul, our first priority was time to market, we wanted to let the marketing and sales people stop hand waving and show what we can do as soon as possible, being a startup we weren’t sure we’d have the time to change so we invested a little more time, than I would normally do, to get a good architecture rolling. So yes, we had to change it and yes it had to be a leap but it also served us to get the marketing people going on time; it was modular so a lot of the business logic survived to the new architecture. Not to mention that the architectural structure allowed us to get the team working on different parts of the solution in a way that allowed us to quickly get a fully integrated system (and get to continuous integration and automated tests)

If you are starting out, building on the assumption that you may have to abandon the initial architecture, it is important to build flexibility or at least modularity into the initial architecture. In projects where we did that, we were able to reuse most of the business logic moving from between the different architectures. Sometimes it is very simple for example, again in xsights, we moved a monorail MVC solution to a ASP.NET MVVM one in a couple of days. The other elements required more work (to strip down infrastructure dependent code) and to move business logic to a cleaner architecture which was, among other things, more evolvable – but that’s for the next post

I am looking for an interesting development management position preferably as VP R&D. Other interesting opportunities (e.g. a  chief architect position) will also be welcomed. The position should be in Israel (Hasharon, Tel-aviv or Haifa areas)

On a related note, I am also available for focused consulting engagements. By focused I mean deliverables oriented (not by the hour) engagements such as architecture evaluation, technical due diligence or architecture  workshop. You can see more details in this page. Currently I am available for consulting world-wide.

If you are interested you can grab a copy of my CV from here (English) or here (Hebrew)

Last but not least, since we are closing my team is also on the look for new jobs so if you need some great .NET people please ping me as well

For those of you scratching your head regarding last weeks “wanted ad” – yes this was a surprise for us (esp. considering that that the company was restarted with a new CEO just 4 months ago..oh well that’s life I guess :) )

You can contact me here

• Leap
• Parallel
• Simplification
• Stepping Stone
• “Just Do it”

Kent talks about these from a design and coding perspectives. Architecture is design but at a higher level and with more consequences system-wide so (I think) are are few nuances from how Kent sees this.

• “Just Do it” is what you do when you don’t exactly know where you are going and you so you’re adding features and everything but you probably heading into a big ball of mud. If possible it is better to avoid working this way, but sometimes realities take us there anyway.
• Stepping Stone is what you do when you aren’t completely sure what needs to be done but you can imagine what would make the end-goal easier to reach. The way I see it, for architects that would mean an architecture proof of concept (spike), prototype of skeleton – the three ways to evaluate architecture in code
• Simplification – is when you don’t know what you want to do and it is too expansive to directly get to an end result. This is the basis of the iterative system. start small and gradually add features as you go. From an architecture perspective it is challenging to to get to a core simplified architecture you can grow later. This is useful as long as the changes in the architecture are within the expected growth path. when it is time to make a change in the architecture you’d need to go with the other strategies.
• Parallel – is a technique to use when you know where you are going, but can’t afford to stop/halt what you already have so you keep the two designs going at the same time and migrate to the new way gradually. This is probably the best technique to use to evolve architecture as it fully considers what you already have. An analogy I like to use in describing this situation is building a new interchange on a highway. The new way would be great, but you can’t just stop traffic until everything will be in place so you’ve got to allow the old way (maybe not in full) until you can make the change safely – the same is true for a running business, nobody will halt the business until you’d get your new shiny thing up and running
• Leap – is for when you know where you’re heading and you can afford to stop everything until the new way is in place. From an architecture perspective it is a bit like starting from scratch (I’ll expand on this in the next post). From a design perspective that would mean breaking the build until a few related changes are in place

The next post will expand a little on Leap, Parallel and Simplification

IASA Israel chapter 2nd meeting
This Thursday (July 8th) at 17:00 Israel time, IASA Israeli chapter is holding its 2nd meeting. I am going to be there moderating the discussion on Architecture and Lean/Agile projects if you are around Raanana feel free to join

SEI webinar on software architecture
Another software architecture related event going on on July 8th is a SEI webinar on What’s software architecture by Rob Wojcik the webinar will take place on July 8 from 1:00 to 2:00 EDT. Looks like it is going to be an interesting listening if you are new to Software architecture.

Business Analysis Agile survay

Since my blog is also published in DDJ I get a lot of PR requests from all sorts of places, most of them end in my Junk email (if the SPAM filters doesn’t catch them first). From time to time something a little more interesting crops up. This time it was this survey ran by requirements.net , the International Institute of Business Analysis (IIBA) and capgemini.

What’s interesting about this survey is that 52% of the respondents were in business analysts (or BA management) roles and that 62% of the companies they work with had 1000+ employees. So we get some views of non-devs from large companies.

What I see in these results that Agile (with a capital A) is very much in vogue so everybody wants to say they are practicing agile even if they just moved to an iterative cycle within the same old waterfall process or as the survey word it:

“Comments from participants demonstrate that many in IT Management desire to maintain current processes and organization structure, while delivering iterative development cycles. Of course, this often results in a hybrid Agile-Waterfall methodology which is commonly referred to as “Scrummerfall” [ARGO: I guess that's pronounced Scrum-fail" :) ] or “Wagile.””

Also it seems a lot of people don’t really understand agile by thinking that SCRUM is a synonym for agile; thinking that product owner and scrum master are a logical fit to be the same person (46%) rather that dev and QA (only 11%);  leaving the business analysts as “requirements authors” (42%) rather than having them as part of the “client team” (as Subject matter experts (21%)) or product owners (16%) etc.

I guess the direction is what’s important :)

A tier is a physical boundary (versus, for example, an Edge in a SOA which is a logical boundary within the service). The implications of that are numerous. For instance, you need to consider:

• Trust–who do you let in?
• Security–what do you send out?
• Performance–you need to serialize to pass the boundary, and remote data is expensive to fetch.
• Availability–what happens if you crash?
• Manageability–can anyone see what’s your state? Help you recover?
• Temporal coupling–can you afford to make synchronous calls?
• and many similar questions.

Yet many times people think passing a tier is as simple as passing a logical layer. I should know. I made this stupid mistake more than 15 years ago in one of the first distributed systems I designed. I planned this beautiful separation of the UI controls from the business logic (I didn’t know it was called “MVC” and that someone else had figured it out ages ago, so I was pretty proud of myself). When you clicked on a button you just used metadata to say that BL should catch it. I had all this wonderful “infrastructure”that handled passing the call to its destination.

But then we wanted to take this n-layer application and put the BL in an “application server” which will handle multiple clients. Oh–now we need to move events over the wire , handle calls from multiple unrelated clients, pass a lot of data back and forth, and what about security… you can imagine the fiasco.

Thus, as Niels Bohr once said, “An expert is a person who has made all the mistakes which can be made in a very narrow field.” But you don’t have to make the same mistakes. Just remember that a tier is a natural boundary. You know what? You should think of a tier as the  edge of a cliff, right there – at the end of your application. You know you want to be careful and not  fall down.

Software Architecture books

I recently got a question from Jon :

I am wanting to make the leap from senior engineer/team lead to software architect, can you recommend any good books or resources?

Well, my current top 5 books for architects are

• Software system architecture –  Nick Rozanski and Eoin Woods.The book I wish I’ve written :) covers view and viewpoints very well
• Software architecture in practice – Len Bass, Rick Kasman and Paul Clements – A good solid introduction to software architecture from the people at SEI
• The art of system archieteting – Mark W. Maier & Eberhardt Rechtin. The best book on using heuristics with software architecture (there’s now a 3rd edition but I haven’t read it)
• Enterprise Integration patterns - Gregor Hohpe  & Bobby Woolf – Best architectural patterns catalog.
• Patterns of Enterprise Application Architecture – Martin Fowler et al – 2nd best architectural patterns catalog.

Another good source which is not a book is the IASA content repository carrying papers written by a lot of smart guys and gals (somehow I managed to  sneak a paper there as well).

Lastly there are the 10 papers every software architect should read

.NET and innovation

Jeremy Miller posted an interesting post “A train of thoughts: Innovation .NET edition“. I didn’t write any .NET OSS (my bad) so I can’t comment on the OSS side of the post but I can comment on Innovation.

The way I see this innovation grows from pain – something that bothers us. That usually happens at the fringes not at the mainstream (that doesn’t mean that if something is great it won’t become mainstream – but it isn’t born there). This has been the case with Java – where the mainstream was painful J2EE  so a lot of new stuff sprang all over the place to fix that. These days there’s still innovation on the JVM but it usually not happening in Java. .Net is very much the same, or even worse, in the sense that it came in after Java so a. its comfort zone  was wider and b. it had a lot of Java innovation to copy over (NUnit, NHibernate N..). Obviously other, less mainsteam environments (like Ruby et al) will see more innovation these days.

Bye bye iPhone

I’ve been using an iPhone 3GS for about a year now. I have to say that, for me, the platform was nice, but unusable unless Jailbroken. Even now with iOS4 copied the functionality of many of the jailbroken add-ons I used there are still quite a few reasons to Jailbreak (like Lockinfo, Grooveshark, playing flash video etc.).  There’s already a jailbreak for v.4 but I am getting tired of the way Apple handles all this and other aspects of Apples tyranny (e.g. in regards to apps approval).

Apropos innovation mentioned above, I wonder if Apple will ever understand that the jailbreak community is where the iphone platform innovation happens (starting with getting application to the iphone in the first generation…). We can see that is a common pattern for enterprises to see successful or promising innovative things and subsume them. We can see it with Jailbroken features getting into the iOS, we can see it with Twitter cannibalizing their ecosystem (making iphone client, introducing t.co as a url shortener), Microsoft with ASP MVC and  (trying to) reinvent ORM etc.

Anyway, I’ve decided to replace my (i)phone with an Android handset (most probably a Samsung Galaxy S), I wonder how that will go :)

I started writing this post as an elaborate answer to these comments but something was bothering me. Something about the comments sounded familiar , yep you’ve got it – I wrote similar things more than 5 years ago (if you’re interested in the parallels between the presentation to the comments see below). When I originally made this presentation I had about 15 years of experience in the software industry and 5 years as an Architect (by title and role) where I designed, led, reviewed, consulted and what not quite a few software projects (including some very large scale ones).

However the point of this post is not to say “been there, done that, got the t-shirt to prove it” – Rather I’d like to point out some of my observations today, i.e.  5 years and several projects later

• The construction architect metaphor is flawed – I guess all metaphors are flawed by definition but the construction in general and buildings in particular are quite far from software in the sense that software is soft and changing. The building metaphor serves well to demonstrate size and complexity will need different methods but software is not build it is written and it is abstract and it changes.  It is very presumptuous to think you can capture the right architecture for a project on the onset. Unless the project is either  simple enough, in which case it would probably benefit from an off-the-shelf architecture just the same. A or the project is a  well-trodden path being the nth in a series of projects. In the other cases software architecture will have to change and evolve with the project (you can read a series of posts I am writing on just that). Buildings don’t do that. That number of changing requirements that software projects need to endure is on a class of its own
• Enterprise architects are not software architects – After many years with a lot of debates on what software architecture is seems that most definitions today are in the IEEE-1471 area. Considering this definition is about the structure of software and not the constellation of software within an enterprise and their alignment with the overall business it should be pretty obvious that enterprise architects design the enterprise rather than software. This is not to say that it isn’t an important job just that it requires some different skills (like deep understanding of the business itself and less, for example, technological depth). project/solution architects and product line architects (to an extent) are software architects
• Architectural decisions are manifested in code and how it structured. The architect should understand  the different needs of the stakeholder and base her tradeoffs. A UML drawing as nice as it is is not an architectural decision until it is actually implemented in the system. I’ve seen a lot of system that had elaborated architecture documents which had very little resemblance to the actual architecture in the project as was manifested in the code. Architects that are far from the code are just playing with themselves
• Architects should strive to code. Whether or not an architect will write code should only depend on the number of architects vs. the size of  the team. When there aren’t enough architects to go around an architect will usually won’t write too much. In any event, the architect should know how to code and up to date on the technologies involved. The architectural constraints posed by the technologies in use are real and the best way to understand them is to write code. Things that are suggested by the architect should be reasonably feasible if you can’t code chances are what you suggest will not meat this criterion. If you don’t actively code you’d get rusty – do the math.
• Algorithms are not developed they are researched. You don’t have a concrete plan for the development of an algorithm you have several candidate ideas or approaches – some of them will work some won’t. This is true even if you adapt an existing algorithm. Some agile practices aren’t a good fit for this kind of “development” (TDD that was mentioned in one comment is an example- though a lot of tests for regression purposes are almost a must). Algorithmic research does need to responde to change as things you expected would work may need other approaches. Also, like the rest of the software the algorithms should match the changing business needs e.g. in xsights we started with 3G video calls and small(ish) image databases and now we focus on identification via clients and large image databases -the  chalenges are different. (It might be a good idea to do a post or two on architecture challenges in project with a lot of algorithmic components – let me know if this is interesting to you)
• Agile shines for new development – This isn’t an architectural point, but the issue was raised in one of the comments. When you start with a new system a lot of the requirements are flaky – and so, most likely, is your understanding of the domain. Having a development methodology that embrace change is a real boon (esp. in startups :) )

• Software architecture vs. building architecture (Gilbert comment <->slides 4-7 based on slides by Philippe Kruchten)
• Software architecture at different levels (Sadi, Arthus <-> slides 20-24)
• Software architecture vs. design (Jerome, Sadi, Arthur, Peter <->slide 17)
• Coding architects and agile (peter <-> few articles in DDJ see a discussion on InfoQ)
• etc.

* illustration : Ziggy stardust album cover

Another point is that “design” is not some exclusive architect thing. A developer is not a good developer unless she also known about  proper design. For that matter  ”mastery” of just the  technical aspects of a language without understanding the wider context of design will just help you code a lot of crap faster. So, again , a good developer knows a thing or two about design and can handle trade offs and variations on a project architecture (especially in cases described above).

This means  we need to consider a few issues – When (if at all?) do you need architects? What do they Do? What’s their relation and interaction with the developers ?

When do you need architects?

It isn’t always an obvious choice between a project that can get by with an “off-the-shelf” architecture and one that needs an architect. It would be nice if we could have something like a litmus test that would tell us if architects are needed or not. I don’t have one. The closest thing to a litmus test I have is something I call the SCLR test (pronounced scaler). SCLR stands for Size, Complexity & Limited  Resources.

• Size – well if you are going to have something  estimated at 1000 man-years or dozens of teams. I think it is pretty obvious that you can’t just use something  which isn’t made to fit. If anything there’s a need to divide the work between the teams in a way that would make sense so that you wouldn’t get a big-ball of mud. There’s also a lot of need to coordinate the efforts and keep the big-picture inline. Personally I think that it doesn’t  have to be a huge project to warrent some architects involvement. Since as Fred Brooks notes the number of interactions grows exponentially as we add more people.  In my experience trouble starts even with more modest numbers – more than 4 or even 3 different teams working concurrently is probably a good number to start thinking  about architects
• Complexity – There are many signs for complexity in a project. The vision statement can provide a hint. “Let’s design the software to support the next mars mission”, “best CRM platform ever”  – an ambitions project will not make-do with “average” architecture. Size (which  I already mentioned) is also a sign of complexity, while previously I talked about size of the project, the size of data,number of concurrent users etc. is also relevant when we’re thinking about complexity . A lot of external interfaces is another sign. Integration doesn’t seem very complicated, until you actually try to pull it off. When you have to do a lot of that in a project that’s complex. And there are many other such signs
• Limited Resources – Naturally every project has limited resources, but limited resources should be considered as a sign for architect involvement if the resources are extreme. When resources are extremely limited the tradeoffs that have to be done are more meaningful, which is why wed want people who can help with that (i.e. architects). For instance in a projects I worked on in the past we had a lot of availability and performance requirements on one hand but only so many “U”s in the rack and even limited electricity to make all this magic happen. This turned something which otherwise was a relatively standard IT project into something a lot more challenging.

Let’s assume I convinced you that some projects need architects. Convinced, you go and hire one. now what?

What do architects do?

Let’s start by looking at  “architectural decisions” – which is sure sounds like something we’d want an architect to do. I read once (I think that was Martin Fowler) that an architectural decision is a decision that in hindsiight you wished you made right. if we look at a formal definition of software architecture (say from IEEE 1471) we see that the architecture embodies the fundamental decisions about the system its components, their relations and their properties. Using this definition an architectural decision is a fundamental decision about the system (which pretty much explain why we want to make them right etc.)

Well, here are two observations on what I’ve said thus far. One is that we would want to postpone architectural decision as much as we can, since changing them will cause us a lot of headache. The problem is that in order to postpone an architectural decision we need to build flexibility into the system which is an architectural quality in itself – which might not be the top of the list if we prioritize it vs. other architectural qualities we need.

The second observation is that if we “refactor” the pretty language out from both of these definition – we can see that an architectural decision is basically a guess, hopefully that’s an educated guess but it is a guess none-the-less. and as Albert Einstein once said it is  hard to make  predictions – especially about the future.

This is why architects  breadth of knowledge – which helps explain the architect training program I posted about a few weeks ago (see Architect training program Part I and Architect training program Part II). Another aspect is experience. And to get a wider perspective it can be helpful if this experience includes other roles besides developer such as project manager or business analyst etc. Another important component is domain knowledge and understanding of the business.

Using all these you (as an architect) may come up with a reasonable architectural decision (e.g. use MVC pattern) and a design to match it and that’s it.

Well, actually, not quite since as I said earlier it is still a guess. Remember  an architectural decision (and any design for that matter) is a mirage no matter how beautiful the power point slide looks (or white board or UML sketch etc.)

Alas, power point compilers are still in the making. Which means that as an architect, you must be able to prove your point in writing – that is coding. While you are at it, you also need to know a thing or two about the technology you are using because it too has an architecture, features etc. which may (and usually do) have a significant effect on the end result. (You can read a little bit more on this in the “Architecture Deployment” paper I published a while ago).

The result of trying to postpone architectural decisions coupled with ever changing requirements and with adding details as we unfold the  architectural abstraction level to a working system, is that the architect can’t just appear at the inception of a project and disappear afterwards – they need to stick around for the game. This is especially true if you want to have an evolving architecture.

The architect’s role goes beyond making architectural decisions. In effect the role shares some of the CTO roles -though on a smaller scale i.e. withing projects rather than companies making the architect the project CTO

Tom Berray has an excellent paper describing 4 models for the role of a CTO. 3 of them can be applied to software architects (within their projects)

• “Big Thinker” – This is somewhat akin to the role discussed in the previous post.
• “External Facing Technologiest” – I usually saw this in larger projects, but it is also applicable for smaller ones. There are many occasions where the technical capabilities of the project have to be presented and/or negotiated with external stakeholders. Architects are in a good position to perform this as they should have good understanding of both the business and the technology. Additionally making architectural decisions already requires the architect to understand the different stakeholders’ needs

The third model is called “Technology Visionary and Operations Manager” – Making sure that technology works to deliver business goals – but how is that done?

In their book on organizational patterns, Jim Coplien and Neil Harrison, talk about the “Quattro Pro for Windows” (QPW) development team. According to the case study, Borland had a team of 4 architects who worked together to produce what the authors call prototypes*. 6 month later these architects were joined by additional developers to produce the product. During the development the architects kept meeting on a daily basis to coordinate their efforts (sort of like a daily stand-up in a scrum of scrums).

The situation in the QPW is probably close to the ideal architect involvement in a project – coding architects that work closely with the team, while driving technical and architectural decisions. The availability of multiple architects (but not too many – to prevent the “design by committee” effect) also enhances the overall quality of the solution.

Another aspect of the architect work is to act as a coach/mentor. It isn’t enough for the architect to “know best”. We already know that architect must also be able to reason about their recommendations/decisions, but that’s just part of the story. Helping other team members get better in what they do means that they’d be able to do their job better, they’d be able to come up with their own ideas (and get more fresh ideas into the discussions) and produce better software. Since the architect is ultimately responsible  for the quality of the solution, making others perform better should be a top priority for the architect. Being considered as a source of knowledge will help an architect perform his/her role, even when they don’t have an architect title

[This post is an edited version of several posts I wrote in 2007 and are (IMHO) worth reprinting]

But WHY  am I telling you this? For one, it’s a good talk on leadership and inspiration in itself (well worth the  18 minutes or so it would take you to watch it). The main reason, however, as the title says, is that  it also pertains to software architecture decisions.

Simon talks about “the golden circle” going from Why -> How -> What. Saying most people work from the outside in (rarely getting to the why) and leaders work from the inside our (starting with Why). It is true for us as well –  Software architecture decisions should start with why

We’re not just building an SOA (What) using mule or nservicebus (How) we’re doing that to build a flexible solution so that we can respond to changing business needs (Why).

We’re not just using a NoSQL solution (What) using Cassandra cluster (How) we’re doing that to solve a database reliability and availability problem of managing a large dataset with high write  (Why) to allow the business to support X millions of images (in xsights case) with Y concurrent users and search times less than Z seconds (a better Why)

This is the essence of why it is a very good idea to describe Quality attributes in terms of functional scenarios. We want to make sure that an architectural solution will be tied and will answer a real business scenario within the system. We don’t want to provide fault-tolerance or 99.999% uptime . That’s not interesting. We need to define why we need high-availability, what will be the consequences of the downtime loss of a business transaction? loss of 100K dollars? loss of life? – for the last case maybe 99.999% uptime is not enough for the first case maybe a 95% uptime will do – start with why. It can save you money, it will help you get to the right requirements and it will help you justify your decisions

if you don’t remember

Nonoservice is an Anti-pattern where a service is too fine grained. Nanoservice is a service whose overhead (communications, maintenance etc.) out-weights its utility.