I was trying to describe the strategic process to someone the other day and I ended up saying:

“If you have no desire to be somewhere else, then you must be happy with where you are.”
- Me

I immediately thought “what a stupid and obvious thing to say” then I thought “No, that is exactly right!”

If people are happy where they are, and have no desire to be somewhere else, then they have absolutely no motivation to move. They will stay where they are.

This in the context of strategy is why strategies are often not implemented. The people who are supposed to implement them have no desire to be “somewhere else.”

One of my professors at Wits Business school, Prof. Rasoava Rijamampianina “Rija” used to say:

“The only person who appreciates change is a baby with a wet nappy”
- Prof. Rija

This is really it. Have we as strategists spent enough time on the consequences of NOT changing. If the organisation believes it is OK where it is, or that the strategy is not addressing the right problems then change will not happen.  What’s more if the change is forced it will only be sustained as long as the leader who forced it is continuously maintaining it. As soon as that leader leaves, the fascade of change will fall off.

• The People implementing it need to believe in it.
• They need to understand why staying where they are will be disastrous.
• They need to understand why it will be “so much better” where we are going
• They need to believe in both of those.

Simple.

This doesn’t mean the strategy is the right one, or that any of the promises it makes are true. We have many examples from history of people fervently and whole-heartedly striving for something totally wrong. The strategy was wrong, the outcome disastrous but the strategy WAS IMPLEMENTED.

The skill of the strategist is two-fold then.

• Find the “best” strategy for the organisation that will have desirable outcomes.
• Convince the implementers of the need to change.

Some of my stakeholders will say, “Yes, Yes, Yes! That is exactly what we need! We agree. Let’s do that” and then they proceed to do nothing.

Intellectually they agree with the proposed strategy, but they do not believe it will be implemented, or they believe it will not address their immediate needs. This is of course the proverbial “self-fulfilling prophecy.”

Watch out for this. Those people need special attention. A strategy will not be implemented if the people who have to implement it do not believe in it as evidenced by their actions.

I have waffled a bit, but I think the point is clear:

• Make sure your stakeholders believe in the proposed strategy.
• Measure their belief by their actions, not their words.
• If you want it to be sustainable, avoid the temptation to use “Brute Force” to get it implemented.

It is probably one of those things, that if you try to find a use for it, you will fail, but one day it will suddenly be obvious.

One of my esteemed colleagues was working on an audit of a Disaster recovery capability for one of our business units.  During the course of his work he was mapping out the locations of all of the servers and the links, and since this was a high availability solution, when he was done he had drawn a picture representing a typical high availability solution, with links to a disaster recovery site for replication and clustered servers for redundancy and load balancing etc.

We decided to abstract the drawing he had made to a more generic view, listing instead of the actual server names, simply the roles they played, like “Application Server”, “Web Server” etc.  Abstracted a bit further we had a picture of what a typical high availability solution should look like in our environment.

It was a simple job to extrapolate from there what a “Continuous Availability” solution and what a Regular and Budget solution should look like.  So what we had now was a set of “Deployment Patterns” for solution hosting.  From Low-end to High-end requirements.  Next we gave them names and described their defining characteristics.  The “Budget” offering would have daily backups and if it failed we would order new hardware with up to 3 weeks of lead time. Each pattern inherited more characteristics as they became clear: Recovery time objectives, Recovery point objectives, Availability targets etc.

So now when we refer to a Bronze deployment or a Silver Deployment, everyone should immediately have in mind the same set of characteristics, and pricing level.

Then we publish a nice little poster describing the patterns we have discovered.  Patterns are discovered, not created.  Standards are created, but patterns usually already exist.  We know what “Should” be done in a given set of circumstances and what typically “Has been” done before.  The trick is seeing how things relate to one another. “Ahh, these are different types of hosting deployments, and they have different levels of availability requirements.”

Another point to note is that patterns come in sets.  We have a set of hosting patterns, a set of monitoring patterns, a set of integration patterns.  We cannot sit down with the objective of creating “Architectural Patterns” without an understanding of their objectives.

In Pattern-Oriented Software Architecture: A System of Patterns, (F. Buschmann, R. Meunier, H. Rohnert, P.Sommerlad, and M. Stal, John Wiley and Sons, 1996) three types of patterns are noted:

• An Architectural Pattern expresses a fundamental structural organization or schema for software systems. It provides a set of predefined subsystems, specifies their responsibilities, and includes rules and guidelines for organizing the relationships between them.
• A Design Pattern provides a scheme for refining the subsystems or components of a software system, or the relationships between them. It describes commonly recurring structure of communicating components that solves a general design problem within a particular context.
• An Idiom is a low-level pattern specific to a programming language. An idiom describes how to implement particular aspects of components or the relationships between them using the features of the given language.

From the Book above and some adaptations of other pattern concepts I have adapted this Pattern for Patterns:

• Name: A meaningful and memorable way to refer to the pattern, typically a single word or short phrase.
• Problems to be solved: Describes the problem to be solved, with reference to the goals and objectives as appropriate.
• Pre-context: The conditions relevant to the pattern that exist before the pattern is applied.
• Principles: Adopted rules, policies and predetermined modes of action for the pattern.
• Solution: A description of how the problem is solved.
• Products: Example products which can be used to implement the pattern.
• Resulting context: The conditions that result after the pattern is applied, including resolution of competing forces.
• Resulting context owner:    The enterprise client who owns the benefits of the resulting context. Typically this would also be the owner of the requirement for this functionality.  This may be a role rather than an individual e.g. “Operations Manager”,”Integration Committee” etc.
• Examples: Sample applications of the pattern.
• Rationale: A description of why the solution is solved in a particular way – possibly describing the key trade-offs made between competing forces.  For example:
  • Security, robustness, reliability, fault-tolerance
  • Manageability
  • Efficiency, performance, throughout, bandwidth requirements, space utilization
  • Scalability (incremental growth on-demand)
  • Extensibility, evolvability, maintainability
  • Modularity, independence, reusability, openness, composability (plug ‘n play), portability
  • Completeness and correctness
  • Ease of construction
  • Ease of use
  • … etc.  (SEE MY PREVIOUS POST ON TRIZ…)
• Related patterns: Related patterns are predecessor, successor and alternative patterns, which provide similar solutions.
• Parent pattern: If this pattern is completely contained within another pattern, the name of the containing pattern.
• Sub-patterns: Patterns which provide specialised behaviour yet are completely contained within this pattern.
• Known uses: Known applications of the pattern.
• Immediate predecessor patterns: Other patterns that contributes to the immediate pre-context of this pattern, i.e. They give inputs that are critical to the functioning of this pattern.
• Immediate successor patterns: Other patterns that depends on this pattern for their pre-context, i.e. This pattern must provide inputs to these successor patterns.
• Security considerations: Security aspects that must be considered for the pattern.  Does this pattern treat sensitive data or have capabilities that should be secured.

The Names of the various Patterns should be used in a “Framework” or Navigation matrix that will help a user to select the appropriate pattern for their particular use.  An Example of this is a set of Monitoring Patterns that we have created.  The Framework indicates monitoring “objectives” (SLA Management, Operations Management etc) as column headings and the “activities” (Instrumentation, visualisation, correlation etc) as row headings.  At the intersection of each pair is a pattern name.  Find the point in the matrix that best matches your specific intentions and you will have a specific pattern or set of patterns indicated for your business or technical problem.  It is really important to have this navigation structure in place to help users find their way through a potential maze of options.

Some Resources:

• The Open Group
• Patterns Home Page
• Patterns-Discussion FAQ
• Patterns and Software: Essential Concepts and Terminology
• Catalog of Patterns

Loosely, something that is Idempotent (Idem:Once, Potent:Having effect) will have an effect only the first time it is executed or will have exactly the same result every subsequent time it is executed.

Some say that dividing or multiplying by 1 is an idempotent operation.  In a coding sense I disagree somewhat.  Dividing or multiplying by 1 NEVER changes the result, not even the first time.  So maybe multiplying by zero is a more appropriate example of idempotence.  It changes the result the first time it is executed, but thereafter has no further effect if executed multiple times (i.e. after the first time you are multiplying zero by zero and the result will of course remain zero)

In software this translates into writing code that knows when it has already been “effective” and that if it is executed a second time will have no further effect.

An example of this is in batch processing.  A batch program written in an object oriented way could have a stack of calls to the same process (one for each item in the batch) so the batch program simply calls the same routine once for each item in the batch.  If this routine (let’s call it the batch-item-processor) is written according to idempotent principles, then calling it twice for the same batch-item will have no harmful effect, i.e. that item will not process a second time.

If you have ever had to troubleshoot batch faults you will immediately recognise that this has many benefits.  As the troubleshooter, you do not have to worry about which batch entries have already been processed and which ones have not.  Once you have found the faulty batch-item and either corrected or eliminated it, you simply resubmit the whole batch.  The batch-item-processor will not allow any batch-items to be processed twice.

You may be tempted to code an override switch so that you can force a second-potency of the batch-item-processor for a specific transaction, however it would be more effective to reverse the effects that the “Idempotency-detector” in the batch-item-processor uses to detect whether it has been processed previously.  This would roll back to a state before that transaction processed, so the new call to this batch-item-processor would correctly process for the first time “again”.

When writing an idempotent batch processor it would therefore be prudent to write a reversal processor at the same time to reverse the effects of the transaction as discussed above.  In fact I would write this in a “-1″potent way :-)   So this means it will not try to reverse transactions that have not been actually performed yet.  This means if you want to do a batch reversal you can simply submit the whole batch to the reversal processor and only transactions that have actually been processed will be reversed.  It’s the same principle in reverse.

The key here is that you give sufficient intelligence to the batch-item-processor and reversal routine to know when it is safe for them to execute.  This is a higher principle than idempotence.  Appropriate-potence: A code-object knows when it can/should process and will not process until those conditions are met.  For example if you call a step in a process which requires that previous steps have already completed, it will simply respond that it cannot run yet because these upstream processes have not yet completed.  Depending on the process you may put it then into a polling loop checking for that state to occur.  Every subsequent call simply responds that it is waiting to run and the new instance terminates itself while the original continues to wait.

NOW you can start building an intelligent network of objects that together form an optimised just-in-time no-lag process in which every step performs at the moment it needs to.  As soon as a required state is reached the dependent process will begin its tasks.  This means that to initiate a process you simply call the LAST step in the process and as it checks state, all the way up the stack it will have initiated the whole process.  If you create a monitor agent for the required state for the FIRST step in the process an event can be generated that triggers this LAST step in the process, and hence the entire end-to-end process is automated.

Another side effect of giving each step the intelligence to know when it can run is that you manage dependencies at each node, you do not have to create a complex dependency matrix, simply worry about the dependencies of each object and let the network of objects calculate the matrix in runtime.

I’m sure there are many examples of this having been done before.  Leave a comment if you have tried this, or implemented a cool example of this approach - or of course if you simply want to make a comment. Find the “Respond” link below the Article date in the Article Header.

Triz is a Russian Engineering tool that has been used for some time.  It is apparently a definitive list of design trade-offs that should be considered in any engineering design.  It Consists of 39 Features of any solution and 40 Principles that may be applied to address trade-offs of those Features.

see: http://www.triz-journal.com/

and

http://www.innovation-triz.com/

So the 39 Features are placed on both the X and Y axes of a matrix and when you need to optimize one of them you will find the list of principles (from the 40 possible principles) to apply, at the intersection point of the potentially conflicting Features.  I won’t place the matrix here as that has been done on the sites listed above.  (These 39 Features have yet to be translated into Software System analogs.  Has anyone done this yet, or does anyone know of a Software version of the list of Features anywhere?)

Some proponents of TRIZ found that in software system design these engineering and innovation principles can be applied by simply shifting the terminology from engineering terms to Systems Terms.  The 40 Principles of TRIZ for software are listed here: http://www.triz-journal.com/archives/2001/09/e/index.htm.

I believe that this approach is focused too much on software engineering and is not that clear when a solution architect or systems designer needs to apply these principles i.e. some of these analogs get into coding algorithms instead of modular design.

I have created a slant on these 40 principles that may be more useful to a solutions architect by abstracting some of the principles slightly (others are left intact as per the TRIZforSoftware site).  (Note: this list now includes suggested changes as per Dr Claude Meylan’s comment) :

1. Segmentation – Selecting an appropriate degree of Segmentation.  Some problems require a Unique approach to segmentation.
2. Interference – Will this affect other Systems or will other Systems interfere with this system
3. Local Quality – Will we have to make special exceptions to our standards to improve the quality of the solution for this specific instance?
4. Load Balancing / Resource Utilization – Is the design making optimal use of resources as well as appropriate use of load balancing/ parallel processing
5. Consolidation – Sometimes it is appropriate to perform multiple functions in a single module or sometimes processes are so similar it would make sense to consolidate them
6. Universality (mulitfunction or Personalisation) – It may be appropriate to build a module or system in a way that it can be personalized or that it can serve multiple purposes.
7. Nesting (classes within classes) – Nesting of Classes can add to reuse
8. Multi-Context (reuse within multiple contexts) - sometimes a system or module built for another purpose is actually reusable within a different context altogether
9. Pre-Processing
10. Just in Time action without lag – is every module aware of when it can start processing to avoid slack or unnecessary load.
11. Pre-compensation for reliability problems – what periodic procedures can ensure a greater degree of reliability or stability.
12. Equipotentiality – Various modules within a system must be equally scaled
13. Ability to Rollback – The system must be able to rollback transactions that cannot be fully processed.
14. Abstract / complex Data structures – Does the system really require the flexibility of Abstract data-structures or will straightforward design suffice.  Is it futureproof without these?  Will it be replaced anyway?
15. Optimal Operating Conditions -
16. What if the system fails half-way or does too much – Will it be possible to interrogate the system / data-artefacts and determine what happened / how to fix it – or is the system intelligent enough to roll back/forward to a valid state.
17. Affecting other Systems, Data or layers
18. Rate of Operation – How fast does the system need to operate?  What is the potential future requirement.
19. Scheduling – When will the system need to run?  Is Self-Scheduling part of the design?
20. Idle Time – optimal load – Is the system Idle for stretches of time and then overloaded for others?
21. Burst mode / peak load – how will the system cope with peaks in load?
22. Convert harm into benefit  - Is there some inherent problem in the process that we can use to our benefit or at least be alerted of should the event occur.
23. Feedback – How much Feedback does the system need to give? Too much will slow it down, too little will leave operators wondering if it is functioning properly.
24. Intermediary Layers : middleware – Required?  What about SOA?
25. Self-servicing (built in updating / alerting) – Does the system automatically raise alerts or check for a newer version of itself or at least submit traps to a monitoring system.
26. Copying - how easy to deploy elsewhere?
27. Disposal – will it be easy to replace or dispose of? Is the data migrateable and is the level of loose-coupling appropriate so that other systems feeding off it can be converted to talk to the replacement system easily.
28. Alternatives to this process (What if we did the whole process another way?)
29. DMA or Meta-Data structures – Are there parameters in the system that will restrict us going forward that we could address by a dynamic design.
30. Wrapper Objects – Are there existing objects we could wrap to use in the project, or will the new ones be more widely usable with some for of wrapper.
31. Required to be less than optimal – e.g. chess: beginner levels.  See Equipotentiality – a system feeding another system may need to operate less-than-optimally.
32. User Interface appearance & usability
33. Homogeneity – Container Objects – Is it all implemented “The way we do it”
34. Code self-healing / intelligence & Garbage Collection – Are appropriate clean-up processes designed-in?
35. Transformation properties - Multi-role objects – Some objects may be reusable
36. Phase transition – unexpected behaviour
37. Storage or Memory utilisation during operational cycles
38. Encryption – required?  Appropriate level.
39. Not affecting the live environment during testing - Test Harness
40. Use mulitple methods to attack the bigger problem

(adapted from TRIZ for Software: 40 engineering principles.)

This may be a useful checklist for a solution architect to check their design and optimise the proposed solution.

I’m still working on translating the 39 Features.

Good luck.  Have fun.  Any comments welcome.

Video clip:

Fiat Bravo vs. Ferrari 550 Maranello vs. Ferrari F1

The three cars in this video are clearly very different. For a formula one car the objective is easy to define. “Get around the track as fast as possible while meeting all FIA regulations and maintaining appropriate (two race) reliability of every component.”

For a Ferraro F550 it is a little more difficult to define. Probably somerhing like “Meet super-luxury standards and reliability for a road going sports car while offering super-car performance at a super-car price-point.”

For the fiat it is more likely “provide basic mass producable budget transportation”

Based on their specific goals it is obvious that we would define performance slightly differently.

For the F1 car it is about winning the constructors championship. For the GT car it is being recognized as the best luxury sports car. For the fiat it is about sales and market share in the budget category.

The point is that for three products produced by the same organisation the goals are completely different and the performance goals are consequently completely different as well.

The key is understanding your goals, understanding how you could measure success, and then target optimizing those metrics.

My view is that architecting for performance starts here. It isn’t a one- time thing either. This is why monitoring is key.

I will be speaking at an Enterprise IT Architecture Conference on the 28th and 29th of July 2008.

Attached are my Presentation Notes that I will talk to on the topic of Architecting for Performance and IT Asset Management

Asset-management-practices

Performance

The videos used in the Performance Presentation are:

Fiat vs Ferrari F550 vs Ferrari F1

F1 Safety Focus - Telemetry

Alonso in McLaren vs 3 Mercedes

This means that someone wanting a cup of coffee would have to walk 10 minutes to the central building in order to get a cup of coffee.

So a 20m break for each cup of coffee.  Now a few techies I know have up to 8 cups of coffee in a day.  Some software developers seem to be entirely fueled by coffee.  How will this end?  Are developers no longer being fueled by coffee?  Or is this just a mistake?

IBM have had their “Sametime” product for a while, and also Lotus “Connections” which is a social-networking and collaboration suite.

There are a few others aimed specifically at corporates, where one can purchase the software and host it internally and I intend to try a few of these (Including Lotus Connections) over the next few weeks.

ClearSpace is not very well known in South Africa and in fact I had never heard of Jive Software until I found their site a few weeks ago.  They are based in Portland, Oregon, and boast over 2000 customers and a coverage of 15% of the Fortune 500 Companies.

The ClearSpace 2.0 and ClearSpace Community Products are written in Java and require Java 6.

A while back I mentioned some of the potential I saw in Facebook for the Corporate.  I wasn’t saying open Facebook to corporates to use, nor that an exact replica of Facebook would be ideal in a large corporate.  What I was saying was that some of the social concepts that work for Facebook, would be very powerful when applied to corporate problems.

For example, I work on a number of projects at any given time, I have several documents in draft and pending review, I consult on various topics and sit on a number of governance and IT committees.  Keeping track of activities and actions across all of those can be an onerous administrative task and requires discipline on my part.  As a “creative” thinker, administration and discipline are not my strong points.  If projects and forums and communities-of-practice had things like facebook groups where all their documents and discussions and actions were tracked, I could simply login to MY personal page and instantly see the latest activities across all of the areas I am involved with.  That would be freedom.  I would also no longer have any excuse to say “I didn’t know” about some decision or event or outstanding action.  

Enter ClearSpace.  This is exactly what I described above.  The concept around which ClearSpace revovles is the “community”, not a document or post.  For example a document sharing site revolves around “documents”.  A Wiki revolves around the “pages” or “entries”.  A Blog revolves around the “posts”.  ClearSpace revolves around the community. Communities may exist because of Strategies, Projects, Topics of interest, anything that warrants creating a new area of focus.  One can even create hierarchies of communities and the Parent Levels can aggregate the updates and feeds of their children.  Within these communities individuals can post Blog entries, write documents, poll other users for opinions, ask questions, create discussion forums and many more.   Almost all of these allow others to respond or comment.  The major contributions can all be scored.

In addition to community blogs, individuals may also have blogs.  individuals are attributed with the scores for their contributions.  Individuals become “experts” on the topics they contribute on most, and are considered better experts when the ratings of their contributions are higher.  

Everything in the system is based on a widget concept, so if you need a new content-type for a particular community, write the widget for it, and snap it into the framework.

This is the first Corporate-community-based collaboration software that I have tried, and I am truly impressed.  

If you need to get people in your organisation to work together and you need a software solution to enable that, then ClearSpace is definitely one to consider.

Ease of Use: 9/10
Super easy, and a really great looking interface. 

Installation / Setup: 9/10
I used the embedded database to test with, so little setup was required.

Admin Pages:7/10
Lots of menus and submenus, and I couldn’t always find the setting I wanted without browsing through a number of the admin pages.

Engagement: 9/10
This pulled me in.  I loved using it, not least of all because of how good it looks and how intuitive the user interface is.

Now all the service providers have month-to-month voice offerings, but vodacom does not have a monthly data product and Virgin does not have 3G, so for data I am with MTN. This month, I needed more than my 2gig bundle and wanted to add another 2gig. Guess what? It cannot be done. How insane is that? Here is a good customer who has exceeded his usage of your product and wants to buy more and you say sorry we cannot sell you any more unless you are prepared to pay 5 times more for the additional quantity. Wow! You would think selling a larger quantity to an existing customer would be something a company would like to do. Not so MTN. Clearly their pricing structure encorages people to buy LESS of their product. This must be some innovation in marketing that we did not cover at business school.

So here’s the install process: Note in slide 3 that the install did not detect the native resolution of my monitor (Samsung 32″ LCD panel)  and I had to clear the message with the monitor’s remote control.Otherwise absolutely seemless install so far.

Now the actual file copy part:

This has dealt with the following issues:

• Paper consumption has been reduced
• They can be easily forwarded to additional people

It has left the following problems though:

• minutes clog up email inboxes
• They remain difficult to find after a while
• they can only be seen and searched by the recipients of the minutes

Enter Web 2.0

If we use an intranet/extranet based blogging solution as a method of publishing minutes, then we can solve several more problems:

• Any person with access can search the minutes for mention of specific topics
• Tags categories can be applied and searched.
• Email box clutter reduced
• there is always a place I can go to refer to old minutes
• If I use an RSS-Reader I can carry historical, searchable minutes with me.
• It is easy to comment on minutes and distribute changes.
• Many blog solutions provide a subscribe-by-email module for those users who cannot adapt.

There are however certain issues:

• Security requirements of certain minutes will mean they have to be handled differently
• Version control of changes to Blog content is not very mature, or even desirable.
• Users will need to learn to use RSS-Readers.

For the most part, this will be a great solution to make it much easier to share and distribute minutes.

In business we tend to leave the monitoring of systems as an afterthought, as a bolt-on at the end of a project, that we will do if we have some spare time. Very few business systems are designed with monitoring in mind, from the very beginning.

The operations benefit and business MIS benefit of having excellent monitoring only becomes apparent, once you have experienced it.

• The value of having all your business systems reporting client activities into your CRM system is only evident when you have an opportunity to impress an important client on your call-centre.
• The value of realtime transaction load monitoring is only evident when Christmastime loads start to hit your online transaction systems and you can see where your systems are starting to struggle.
• The value of detailed audit-logging is only evident when fraud has been committed using your online systems and you can report all the pertinent details to the police in minutes.

But it is costly to implement monitoring, because all of your code has to be modified and metrics thought through and added and the entire system tested and retested. A monitoring system can be added anytime, but the instrumentation that feeds the monitor has to be designed into your system. And it is significantly more costly to add instrumentation to a system after it is built than to have designed it in from the beginning.

Imagine again in Formula 1, if the team needed to add throttle position sensors to the car after it had been completely constructed. There probably wouldn’t be space to attach the sensors and no conduit for the wires, or sufficient processing power to record all the events or adequate bandwidth to transmit all the messages back to the pit. It will be a lot more costly to add all the necessary requirements after the car is built, than to have designed them in from the beginning. Just take one example - not enough space for the sensor. What do we do now?

• Change the design of the throttle assembly to make room?
• Buy the smallest sensor available and shave something to make room for it?
• redistribute the surrounding components so that there is room for the sensor?
• make the compartment larger and retest the aerodynamics in case they have to change as a result?

Wow! It would have been much cheaper to have designed it in from the beginning.

Ask yourself at the requirements stage of your next system:

• What do I need to measure to ensure that I have the right MIS once this thing is up and running?
• What do I need to manage in realtime when this system is running at its peak utilization.
• What information might I need to investigate failures or misuse of the system?

Don’t build it without a conscious knowledge of what it will cost to do now, versus what it will cost to add later.

Here is a copy of the Presentation used at the SPIN forum on 23rd August 2007:

F1 MITTUMAL Presentation

When they are young we make rules that they don’t necessarily understand but have to obey. It is best if they obey because they “want to” or because they love us. They could also obey because of the consequences of diobedience, but this is less optimal.

As they grow up we hope that they will realise that these behaviours are important and will be best for them in the end.

Projects are the same. Testing is important. Good design is important. The long term view is important. We can enforce adherence to these principles through consequences, but the very best reason for adherence is because the projects understand that it is best for them in the long-run.

So here’s the motive check:

Do you love your Parents?

• Projects, Are you behaving in the best interests of IT?
• Are you looking out for the good of IT?
• Do you wish good things and success for IT?
• Projects, do you love IT?

Do you love your children?

• IT, are you encouraging projects to behave right?
• Are your motives for doing this out of concern for the long-term success and good of that project?
• Remember these projects will always be your children, you can’t disown them
• IT, do you love your Projects?

Some children grow up being delinquent, but for the most part end up being good citizens who love their parents.

The motive is important. It is evident to children and they grow up to become different sorts of people, depending on whether they are loved or not.

Projects, are you the sort of children who end up being good citizens who look after their parents in their old age?

IT, are you the sort of parents who genuinely care for the long-term good of your children?

A Smart-client can address many non-functional or repeating-requirements that will not have to be developed ever again.

Smart-Client Administration
An administration Interface for the smart client is essential to take admin away from the developers and hand it over to the support desk. For instance adding a new user or promoting a piece of functionality from Testing to Production can be done via an administrative interface. Within this client we embed a lot of functionality that would previously have been done by developers. I include Authorisation, creation of Data access aliases (See section on Data Access Agent), upload of new functionality into the Smart-client database, promotions between Development, Testing, Production etc, notifications to authorised users, work-flow definition and edits, creation of aliases to access functionality (menu-item naming) and any other administration whether done by developers or support-staff. I also recommend that the admin tools be built as functionality that snaps into the smart client itself - it just makes it easier.

Authentication / Authorisation
If your organisation uses a directory service of some sort, then integrate the smart client into the directory for authentication. If you have a generic authorisation services, then use that to authorise the client functionality as well. if not, link the IDs of your directory user into an authorisation table accessed by the Smart client Administration interface to link users to the functionality they require.

Work-flow
If your organisation uses work-flow, then definitely link generic work-flow functionality into the smart client so that it too is dealt with once.

Data Access Agent
If your smart client allows plugged-in code to access data directly, then you are missing a trick here. This idea has very powerful side-effects.

Define a method by which plugged-in code asks a function provided within the smart client for some data. The smart client should then source the appropriate data and return it to the calling plug-in. Some side-effects here are that the developers need not know any of your production database user-names or passwords, and you can within the smart-client infrastructure redirect data requests to other destinations (for example to your disaster-recovery site, or a testing database)

In the implementation I have built previously, we created a lookup-table for data-sources that indicated an “allowed calling-plug-in”, “operational-mode” a “Data-source alias” and the details of each data source. This allowed us to automate a switch to “Disaster-Recovery” mode through using a predefined “operational-mode” called “DR”. The Smart-client had a process for checking against a centralised web-service for which mode it should be operating in for a particular application. The flexibility here was staggering. This also means that the change management team can promote an program from Testing to Production without any help from a technical team whatsoever.

On-line/Offline modes
A web-based client or portal has the disadvantage that offline modes are not possible or at least very difficult to implement. A smart-client can do this fairly easily and with the “data-access agent” functionality above it is even easier to achieve.

Code Generators
I have found that most user interfaces that we needed to build were in fact forms that interacted with data-tables. We used a multi-tier architecture where the front end was smart-client (containing a data-access layer), a business layer consisting of stored procedures which performed a lot of the business activities and which interacted directly with the data-tables. So we created a form-builder that created both the stored procedures and .NET forms automatically. We would point the tool at a set of data-tables and lookup tables do some configuration and click “GO”. What we got out was a stored-procedure that conformed with all the rules we had predefined and a .NET project with all the necessary forms and the interactions with the Data Access Layer and Stored Procedures. If there were no fancy custom UI interactions nececessary we could simply compile the project, and upload the DLL into our smart-client database for testing.

So no developer-time required for:

• authorisation,
• basic form-based UIs,
• access to data,
• Disaster recovery implementations,
• promotions between Dev, QA, Prod etc,
• Creation of menus
• Creation of custom workflows

among others. This represents a significant saving in the development stage of any project.

Technorati Tags: Software Factory, Development, Custom Code, Code Generation

Here are a few of the more common approaches:

1. Service orientation
Common Services (standard non-functional requirements) are implemented as SOA services. These need to be understood and published so that developers are aware of which common services they can use and exactly how to use them. SOA provides standard ways of doing this. The effect is that code should only be written for explicit functional requirements as the critical-mass of standardized non-functional requirements can be built up as needed.

Strengths

• Very loosely coupled
• Changes to a common module are immediately available to consumers
• Source-code for common services need not be seen by external developers, only the contracts for using these services needs to be known.
• Allows parallel development efforts, provided development of common services is well managed.

Weaknesses

• Governance of deployed common services is complex to manage

2. Smart Clients
The non-functional requirements are coded into a smart-client that exposes a library of functions to the functional modules which simply snap-in to the smart client framework.

Strengths

• Code for the non-functional requirements need never be seen by external developers, only the calls need to be known.
• Code executes in a distributed environment with greater flexibility to provide performance requirements.
• Allows for extremely parallel development efforts, while maintaining a common framework.
• Will scale very well.

Weaknesses

• Need to deploy the smart-client to all users when changes are made to common modules.
• Fairly tightly coupled, although functional requirements could be deployed as SOA services.

3. Common Code Libraries only
This method simply relies on the developer to re-use code from a common code library.

Strengths

• Ensures common approach to standard problems

Weaknesses

• Relies on the discipline of the developer to use the standard modules and not code their own version
• A Change to a standard module would require that all applications using that module would need to be recompiled and tested.
• Very tightly coupled.

4. Monolithic Applications
SAP for example is a single Uber-application that has many modules and performs many tasks. All of the non-functional requirements are standardised and cannot be changed. If one were to be changed, it would automatically be changed for all the other functional modules. All of the non-functional requirements are already coded into the SAP application and adding functionality involves coding the functional requirements only.

Strengths

• Similar to the Smart Client approach

Weaknesses

• Businesses can seldom function on a single piece of software
• This software will become extremely complex
• Very tightly coupled

Summary
There are numerous ways of deploying the Software-Factory Concept to speed up delivery of functional requirements, each with its own strengths and weaknesses.

I recommend either of the first two approaches (Service-orientation or smart-client) or a combination of the two. These provide most of the benefits of a software factory and less of the drawbacks.

Note that they have a management overhead. If they are not managed properly you will gain little benefit or in extreme cases if very poorly managed you may end-up worse-off than you were when you had discrete unmanaged code for each application.

Technorati Tags: Software Factory, Development, Custom Code, Code Generation,

KNOW YOUR ENVIRONMENT
If you bought a specialised mail-monitoring system you would still have to customise all the threshholds and metrics for your specific environment because your servers may be scaled differently, your mailbox structures may be configured differently etc. If you make the mistake of leaving all that up to the manufacturer of the monitoring tool, then the chances are you will be ignoring most of the info the system produces and eventually say “This thing is always wrong” and untimately it will fall into disuse. Ok that’s a worst case, but my point is you need to KNOW YOUR ENVIRONMENT. There is definite value in researching what metrics are relevant for YOU and what the appropriate threshholds are in YOUR environment. So am I saying there is no place for specialised tools. Not at all. As long as you recognise the responsibility stays with YOU to manage YOUR environment, and cannot be delegated to a tool.

Where is the Value?: Part 2
If I were a departmental manager or executive, my requirements of a management system would be completely different. I would want a very general overview of the state of affairs in the organisation. In fact I would want period summary reports and would probably never look at a realtime console. So the requirement is completely different. If the underlying monitoring systems contributed their information into a centralised warehouse, I could draw a report off that and get what I needed.

Caution: Information Overload in progress!
The question is WHAT INFO DO I NEED? That is the RIGHT question (see iRobot - the movie) to start with. Too much information can be as useless as no information, except that now you have spent money on tools (and support staff for those tools) to produce that information that you never look at. Exactly the right balance of information is critical, especially in large environments.

I used to look after all the servers for a medium-sized corporate and after about six months of harassing the systems-management guys I had exactly the reports that I wanted. I sent a monthly report to the senior IT managers exhibiting the outages and incidents that occurred with a comment from myself on each. I also had a weekly report summarising the top 10 major potential and real issues across all servers. Each week I tried to deal with the issues on the top-most-unhealthy server. I also had a daily report on the configuration and health of each server, which I archived and kept for 3 months, which I only looked at if I needed to research something later (reading that report daily would have been overload).

It shouldn’t be easy
My parting comment is that you need to get to grips with your environment and do the investigation and research into what it is that YOU need to manage in YOUR enviroment. It is extremely unlikely that someone can do that for you. You need to make the time to do it. It will save you (I say that deliberately - it may save YOU) in the long run.

Technorati Tags: Systems Management, Operations Management

Software is like a Car
Some cars have air-bags and navigation systems, but can be produced in the same factory as cars without either. Software Factories should be designed with the non-functional requirements in mind. If we consider the example of a car, the only real functional requirement is that it should transport people from A to B using public or private roads. Some of the definitions of what is considered a “car” constrain the designs (weight, number of axles etc). Other than these the remainder of a car’s features are in software terms considered non-functional requirements:

• Is it safe?
• What are its performance ratings?
• Convertible or fixed roof?
• Is it secure?
• Luxurious or Basic?
• Manual or Automatic?
• Spare Tyre or run-flat tyres with no-spare?

The Software equivalents should be fairly obvious:

• Redundant systems with fail-over capability
• Hardware requirements vs algorithmic efficiency
• User Interface for speed or ease of use
• Secure User Authentication and Authorisation methods
• Manual Installs or automatic updates
• Fat client or Thin client or both
• etc

Software is not like a Car
The Difference between Cars and Software is that potentially every piece of software has a different functional requirement, while pretty-much every car has the same functional requirement.

So in a car factory we can tool-up to produce homogeneous units with a uniform purpose, but with modular non-functional requirements that can be selected as options.

In a Software scenario the non-functional requirements remain modular and “optional” while the functional requirements have to be handcrafted or individually tooled.

So immediately the benefit of automating the numerous non-functional requirements for both software and cars is evident. There are several approaches to achieving this which will be discussed in “Software Factories: Part 2″

Standardized non-functionals are a prerequisite
The optimisation of the software coding process to ensure that non-functional requirements are not handcrafted or individually tooled is not sufficient. The greater organisation needs to accept the standardisation of these non-functional requirements before this becomes viable.

Once everyone accepts that tyres are made of rubber and filled with air, then the variables become size, durability, speed rating and aesthetics. If we cannot get everyone to accept the basic design of a tyre, our variables become infinitely more complex and we end up re-inventing the wheel with every new car design! (I know you saw that one coming).

Software is very similar in this respect. If the organisation can accept that the user interface for all web-based software looks the same and the user interface for all fat-client software looks the same, then I can derive benefit from tooling up to automate the creation of these.

An Ideal Example
I was involved in designing a software factory where all the software our company needed to produce was either form-based or generated a report. A standardized method for defining stored-procedures allowed us to automate the creation of the forms on-the-fly in the front-end and also automate the forms for inputting the report-parameters.

So the only handcrafting was the database design and the stored-procedures that interacted with the database. Everything else was written once, compiled once and deployed to everyone once. This fat-client included Authorisation and Authentication modules, data access and routing modules, presentation formatting, workflow calls, disaster recovery processes etc.

Now this was an ideal environment where we only had SQL databases and almost all software was internally developed.

BUT

We had the option of writing each discrete business requirement as a discrete and monolithic piece of custom code. We chose NOT to do that. We chose to design a software factory that allowed us to dramatically reduce the quantity of custom code written (which now only needed to support functional requirements) and we never again had to write any code to support non-functional requirements.

The time-to-market for each new business requirement was slashed, and we gained many other benefits as side-effects of this process.

So in each corporate environment where significant quantities of custom development takes place there will be some combination of non-functional requirements which can be automated to a greater or lesser extent to get the very real benefits of a Software Factory.

OK. Concept introduced! Next time we’ll talk about the various options for designing a Software Factory.

Technorati Tags: Software Factory, Development, Custom Code, Code Generation,

So I quickly Googled “How to ride a dead horse” and found a few sites quoting alternate strategies, which are worth a read.

They are all fairly similar and very funny… and unfortunately too true.

I began to wonder how those horses died. The proverb says “When you discover you are riding a dead horse…” , but there are several possible ways you got into that situation:

• Maybe the horse was dead when you got it
• Maybe you rode the horse too hard
• Maybe you forgot to feed and water the horse
• Maybe the horse got sick and you didn’t get it to a vet who could help
• Maybe someone shot the horse, because it had a broken leg (which you should have noticed if you are riding it)
• Maybe some enemy shot the horse to further his own causes
• Maybe the horse died of old age

bears some discussion…