Sam Lowe's blog on Enterprise IT

Delivering SOA Across Projects: Dealing with Shared Dependencies & Risks

sam.o.lowe@gmail.com (Sam Lowe) — Sun, 22 Jul 2007 17:07:00 +0100

I posted not long ago about professionalising the planning and delivery of SOA initiatives, making realising them more than just a gamble based on theory and good intentions.

One of the biggest pieces of this puzzle often seems to be dealing with the project delivery issues. The industrialisation of SOA if you like. The key issue is that many are not ready to deal with the shared and cross-project dependencies, the additional technology complexities, and the cross-project risk mitigation demands that enterprise-level SOA creates. Such issues create havoc with the conventional project delivery mechanisms and mindsets of the moment, but without project delivery being on-board, an enterprise-level SOA is extremely difficult to scale up to the level required for it to start paying off.

IT Project Delivery
Modern IT delivery (pre-SOA) practices that combine both robust project management and proper diligent delivery management have improved significantly over the incarnations of the 90s and before. They're particularly stronger in areas like governance, visibility and indeed success rates, learning lessons from mistakes of the past. However they've been shaped by a diet of scope definition, issue identification, risk mitigation, and industrialisation of production in traditional systems-centric world. Where each system and project has its own direct business case. Of course its not perfect, for example the delivery models of many are not as mature as they should be in how they effect global sourcing and production in order to take advantage of off-shore production whilst maintaining quality and efficiency. However, ignoring globalisation, many of the basics of today's delivery get turned on their heads by cross-project SOA initiatives.

Real cross-project SOA introduces dependencies between projects who are sharing services, coexisting in environments, or reusing central assets. It also generally means that issue resolution and risk mitigation within a project is no longer within that project's own hands. And in seeking to optimise the technology across projects, it mostly means that each project is sub-optimised, increasing the 'number of moving parts' of technology (and therefore generally the cost) beyond what is needed to realise the project's business case. A situation which often isn't exactly welcomed by the leadership of the projects concerned. And of course there are many more such changes, but you get the idea.

Some organisations have laudable ambitions for the use of architecture in reversing years of IT duplication, proliferation, conflicts and silos that business-unit-centric or project-centric technology deployment has caused, but they embark on it without a proven or complete plan as to how to deal with the road-blocks, or handle the stakeholders who's incentivisation or behaviours lie elsewhere.

1. Project Effort and Cost
For example, you have the additional factors in estimating the effort required in a project and allocating its cost. Even where a project has agreed to take on the additional technologies and development costs of the new middleware (e.g. ESBs) needed to effect SOA it will need more extra budget provision just for the dialogue needed with the central team doing the architecture and technology-usage governance (be they an EA team or called something else), and participate in the joint service designs and testing. Let alone the change control needed if a project has to comply with a late emerging technology standard, or extra regression testing required because of a change request in a parallel project which shares services. And this is on top of the actual effort taken to build the services that they need to build, and effectively feels to the project concerned like time that is not value adding and is not being spent getting on with the solution.

As often is the case, a commercial contract between two parties focuses attention on things that would otherwise be hidden, and if a 3rd party implementer is involved in delivering a particular project, these additional provisions will need to be surfaced up front, and potentially put into the contracting. That's not a nice conversation to have, but infinitely preferable to a much later conversation justifying changes in budgets when it's too late to do anything about it.

2. Design, Configuration and Release Management
There are also the additional complexities of service design, configuration, and release management in cross-project delivery. Firstly, the environments. On one hand there are more technologies that require environments (to develop->test->cut-over), and then all the technologies (new or old) require demarcation between the (develop->test) environments within their own project's control, versus the (testing->cut-over) environments in common shared environments.

Then secondly there is the new design governance process required to collectively agree a shared design that multiple projects work towards, to manage requests for changes to it during development and when its live, to coordinate integration of parallel work-streams, and to sign-off completion of delivery to the pre-agreed design. That last one is particularly troublesome as many 'design committees' that may exist around an SOA initiative may take on the design sign-off responsibility, but are often not resourced or mandated to sign-off delivery to the design. And yet without that ability the projects are at the mercy of factors outside of their control (and timeline) for sign-off, which may be incompatible with the obligations they have to the business, and their business cases.

3. Risk Management
Even with an appropriate configuration management process, and the appropriate bodies with the appropriate mandate to play the roles in it, it still is more than a little problematic to deliver federated cross-project SOA. Particularly with parallel delivery (as opposed to sharing only assets that have already been deployed) which is probably the most ambitious SOA delivery model.

Management of the risk profiles involved causes significant uncertainties for the commercial models each project is using to manage its cost. Fundamentally the budget that any project uses will be based on a certain scope, certain assumptions, and certain known risks for which it will allow contingencies. Whether there is a 3rd party prime contractor involved, 3rd party subcontractors, or only internal heads involved will affect what type commercial model is used, and therefore how the cost and contingency are estimated. But whatever the case, the project will be carrying a certain risk profile that it will look to manage so as to keep its commitment to deliver.

Of course the dependencies of the project on central activities, on existing seemingly vague assets, and on other projects activities, creates additional risks that existing techniques find it difficult to identify, and which are not within their control. This means that projects are potentially exposed in carrying risks that they find it difficult to estimate contingencies for, and that they are unable to mitigate.

Shared Services (...for Delivering Shared Services!)
But these are of course not really new problems. And they're not really peculiar to SOA. They are an aspect of executing on any shared initiative above the level of a project and business unit (BU), which is dependent on the project or BU to realise it, and therefore applies across those projects and BUs. Ultimately this boils down to being an aspect of achieving objectives across reporting lines, in a situation where those objectives include indirect (federated) business cases, in addition to the (projects' own) direct business cases.

The techniques that has most successfully been used to solve similar problems before in previous generation of IT are simple operating model concepts widely used in other areas of business, taught in most b-schools, and written about many times. They're shared services (, although I'll use the old IT term 'centre of excellence' here to save confusion between the shared services in SOA delivery from the the shared services of service oriented architecture itself).

A centre of excellence (CoE) or competency centre for SOA delivery allows the projects to all sub-contract the production of the (SOA) services to that CoE and therefore to exchange the risk that they can't manage themselves for a service-level agreement (SLA) that they can manage to. It also allows the CoE to manage the internal dependencies between the different requests being made of it, formalising and managing th change on contracts with its customers (the projects) so that it can coordinate the cost and SLAs to the risk it carries at any time. It also of course allows the CoE to drive centralised knowledge management, developing its own capabilities and professionalising its own working practices using its own revenue stream, and gaining from the economies of scale on the supply side that no one project could have experienced on its own.

Its great that its becoming acknowledged that an enterprise-level architecture capability (although often not a traditional EA group with a traditional EA framework) is key to being able to drive cross-project technology and architecture strategies and improvements. In such scenarios that EA capability is in some ways acting like a shared service for the design phase (and to a more limited degree, for the preceding demand management phase). But that is only a part of the lifecycle. A shared design is much harder to make it into reality without some level of shared delivery (construction) and shared operations (run and support). Sounds simple when it's said like that.

Technorati Tags: Enterprise IT Project Management IT Delivery Shared Services Center of Excellence Competency Center Enterprise Architecture SOA Services Architecture Service-Oriented Architecture

Survival of the loudest?: 'Social' evolution in Enterprise IT

sam.o.lowe@gmail.com (Sam Lowe) — Sat, 7 Jul 2007 19:07:00 +0100

Last year (after a recommendation from Nigel Green) I finally got round to reading Lila by Pirsig (probably better known for his previous book 'Zen and the Art of Motorcycle Maintenance'). It contains some great thoughts and insights that, whilst written for a totally different purpose and audience, I found very interesting and relevant to enterprise IT. Particularly to enterprise processes, systems, data and architecture.

Bear with me as we go a bit abstract here ... One of the models that he proposes is that the world as we know it is made up of series of 'systems' constructed one on top of another.

-He identifies the inorganic systems (how atoms, molecules, materials etc work) to be the first.
-Then he identifies the biological systems (how cells, organisms, life etc work) to be next, existing on top of the inorganic systems.
-Then he identifies the social systems (how individuals, groups, cities, cultures etc work) to be next, existing on top of the biological systems.
-Finally he identifies intellectual systems (how ideals, concepts, intellectual values etc work) to be next, existing on top of the social systems.

One of the several things he does with the model above is to point out that whilst evolution is well accepted in the biological level, it is rarely seriously considered at the social and intellectual levels. He proposes that the concept of evolution is just as appropriate in describing the change in the patterns seen in the social systems and the intellectual systems of the world, as it is to the biological ones.

He suggests that the patterns in each level have evolved on top of the patterns in the level below. That they have not been designed to serve the systems in the level above. Therefore he proposes that the social systems (e.g. cultures, cities) have evolved on top of biological patterns (e.g. man). They have not been predetermined to underpin the intellectual patterns (e.g. intellectual ideals). He proposes that social patterns may have been retrospectively formalised to support intellectual values (such as ideals of a culture), but actually the intellectual values themselves evolved from the platform provided by social patterns. In turn, he proposes that intellectual systems have evolved on top of social systems, not vice versa. Clearly, although this is not unintuitive, it challenges certain conventional wisdoms that cultures are the result of the intellectual ideals and cities are the creation of man.

You may wonder why I am describing something so metaphysical on a blog about Enterprise IT, well there is method in my madness. The example he gives about how people often consider cities to be the creation of man, that have been planned out and created as some kind of master plan in the heads of the individuals involved is a very interesting one. Pirsig rejects the idea, describing how his model suggests that actually the city at any point has evolved on top of the biological patterns involved, governed by the value systems involved, and the ways they have interacted. He rejects the idea of grand predetermined design of a complex social system such as a city (although these are my words not his).

Given how often Enterprise Architecture and IT planning are compared to city planning (inc. by me, Todd, & Villas to mention a few recent blog postings alone) it should be interesting to anyone involved in IT strategy or architecture. The equivalent of the concept in the Enterprise IT world could be that an Enterprise IT estate is not the creation of the technologies or the people involved, it does not exist as a result of any master plan, but is a complex system that has evolved as a result of the value systems that have interacted.

The obvious activity for a rational IT chap is to consider what would be the actualisation of Pirsig's models in the enterprise IT world, using an Enterprise Architect, IT planning or Portfolio Management viewpoint. Maybe the inorganic level could be the technology infrastructure, both software and hardware. Maybe the biological level could this be related to systems and data. Maybe the social level could this be the work-practices, policies, and networks/communities. Maybe the intellectual level could be the business objectives, models, and processes.

However rather than get too caught up in debating the potentially academic details of the mappings, I'm more initially interested in some of the dynamics of this model. It's not like the applicability of top-down 'intelligent design' in Enterprise IT has never been questioned before after all, but rather than just practitioner's scepticism, this gives alternative hypotheses to consider.

Of course there's enough material in such a consideration for a book in itself, but one of the dynamics that jumped out at me right away that I thought I'd mention here was that concept of social systems evolving on top of biological ones. If there is an enterprise IT equivalent of the biological level, then I suspect it's comparatively well understood and catered for compared to the social level. I have often been of the opinion that social dynamics and value systems are very badly understood in Enterprise IT, much to its detriment. Management science is better (although far from perfect) at considering the social systems that make up the organisations we work in or work with, but such matters often seem to be almost a complete blind-spot to IT.

The biological evolution of IT systems we can appreciate (although many organisations struggle with managing it), and the intellectual evolution of business models and objectives we can also appreciate (although again many struggle), but the conventional IT concept of 'the users' and the tools of functional requirements, flow-charts, use cases etc always feel like vaguely prehistoric blunt instruments for considering the social systems of an organisation. Extrapolating the concept that social systems 'evolve' on top of biological systems, based on the interaction of the value systems (and which proliferates most effectively) seems to suggest to us that we should have a far better way of describing the different communities, networks, ownerships, motivations, behaviours etc in an organisation around its use of, and opportunity for IT. One might even over-simplistically describe it as the politics around the use of IT.

Of course the re-popularisation of the internet ideals as part of the trend often referred to as Web 2.0 has brought a lot of focus to the ideas of community in consumer-focused IT, including blogs, tagging, wikis, inclusive economics etc, and many varieties of social software. The Enterprise 2.0 initiative of Andrew McAfee and others has brought an interesting perspective to how these technologies could and can be applied into enterprise IT scenarios. But even though these new technologies will very likely be important going forward, the dynamics of the social systems around IT of course apply to all system-types and all technology generations, not just social software. It can't be the preserve of a new generation of enterprise technology alone.

Sidebar: Pirsig also uses some excellent metaphors about academics that are scarily applicable to the IT industry. For example he talks about 'restaurants with 300,000 page menus and no food'. That reminds me of far too much of IT than is healthy. Another that stuck in my mind was his 'highways full of drivers too busy telling each other how to drive to actually get anywhere'. If there is an activity where that applies more than in IT then I'd love to know which? Other than politics itself of course.

BTW: I'd be interested to see what some real Pirsig experts think of this misapplication of MoQ. But please be gentle, I'm conscious that it's not exactly a pure representation...

Technorati Tags: Enterprise IT Web 2.0 Web_2.0 web-20 Enterprise 2.0 Enterprise Architecture Systems Architecture Information Systems Pirsig Robert Pirsig Lila MOQ

Placing Your SOA Bets: Making Architecture Initiatives More Than Just a Gamble

sam.o.lowe@gmail.com (Sam Lowe) — Wed, 28 Mar 2007 22:37:00 +0100

Reading this great post the other day spurred me to blog about the 'SOA lottery' that seems to be all too common.

I don't think I've ever met a decent project or delivery manager who would take on a remit without knowing what it took to be successful at it. However, in IT strategy and architecture the opposite can often be true. Once intellectually wedded to the elegance or abstract value of a new approach or principle it seems that for some the architectural approach itself becomes the goal, rather than just a means. And in many cases, the 'remit' to make this happen is taken on blindly. And by implication, without knowing what it will take for an initiative around it to be successful. It becomes almost a gamble ... will it pay off ... will it work ...

I don't believe that it is knowledge of the technologies or standards that is lacking, and it's not knowledge of the architectural frameworks or methods that is lacking either. It is a lack of knowledge of what it takes for an architecture or strategy to be perceived as being successful by all the people whose role is affected by it, or who otherwise have an interest in it. Without this it won't actually deliver anything. It will just be a good idea. And as one of my clients had on their whiteboard last week - good ideas + no results = bull....

Look at SOA for example. Even when its value is defined well (and that is rare) it is still not equally valuable to every type of project or requirement. In addition it changes the game for various IT and business people who probably don't care much about architecture. For example, IT operations, IT project delivery management and IT business relationship stakeholders to name just three. It introduces new processes, risks and dynamics into their roles that change the way they need to behave, and what they will need to prioritise.

Anybody who's been involved with traditional IT-enabled business transformations will recognise that stakeholder and change management are two of the most critical activities in making the initiative successful. However good the objectives of the project, or however clever its design, without stakeholder and change management it will not be a success. Without participation from the stakeholders it won't be delivered successfully, and without transformation of the working practices it won't be used successfully.

And yet it's actually more complex than that for an architecture initiative such as one based around SOA to be successful as the value proposition applies across programmes, rather than within one. For them it isn't about a 'direct' business case (like most business transformation programmes) whereby the cost savings or new functionality delivered by a project realise the value proposition. Instead with these types of architecture initiatives the value proposition is indirectly delivered through the effect it has on other projects, or more accurately the effect it has across an organisations portfolio of projects, systems and technologies.

Of course this makes the success of such architecture initiatives dependent on stakeholders who are not architects, and whom likely do not appreciate the ways that the architecture will affect what they do. Moreover, they will probably resist the way it will likely affect their own initiatives' business cases.

Rather than make this post longer I'll blog separately on stakeholder and change management for architecture initiatives, but a good start is to consider which horses to bet on so that the chances of success are maximised. The key question there is where the architecture style (e.g. SOA) is independently valuable, where is it marginal, where it can be used implicitly, and where it's positively unjustified. This needs to balance the conceptual strategy viewpoint that IT architects are often strong on, with the pragmatic viewpoints of the delivery stakeholders, the maintenance stakeholders, and the business ownership experience.

On that topic I was interested to see Nick Malik's recent post on 'JABOWS' that suggests it is to do with which business process the solution is related to. In particular how often the business process in question occurs, and how often it changes. I like this model but I'd suggest there should be a couple more variables beyond these first two to give the level of practicality that can help you differentiate and plan - but it's far from an exact science at this stage.

Without meaning to overuse the gambling analogy, I believe that knowing this is the key first stage to placing the right SOA bets in an organisation. And as if by magic (!) it's also the topic of a session I'm running at the Open Group's Enterprise Architecture conference in Paris on 24th April - I'd welcome any views of what would be the most important aspects to work on.

Technorati Tags: Enterprise IT Enterprise Architecture EA Service-Oriented Service Architecture SOA Open Group The Open Group Open Group Paris 2007

Architect, Functional and Technical: IT's Good, Bad and Ugly?

sam.o.lowe@gmail.com (Sam Lowe) — Wed, 17 Jan 2007 21:24:00 GMT

Despite most Enterprise IT people knowing the downsides of silo-thinking in a business, many of us are aware that IT can often act as the biggest silo of all within the business of which it is part.

The ultimate objective for many is to run IT like an independent business (commercially-driven and efficient demand, supply and resource management). But there is a trick to doing so without it starting to think like a separate organisation (obsessed with its own bureaucratic processes, pet projects and irrelevant agendas). Getting the good without the bad is a challenge that has led many to embed IT back into the business units as an alternative solution.

But there are also cliques within IT itself which tend to exist whether it is centralised or embedded. The differences between the 'management' people (focused on delivering on time and to budget), the service people (focused on keeping the kit running and the lights on), and the 'content' people (focused on the solutions and the design) are obvious. But the latter in particular itself has silos. Those of functional, technical and architect.

The functional-technical divide goes way back. Even now, organisations still tend to have separate functional and technical teams. It still seems rare for anyone to really bridge the gap. One of the great characteristics of packaged apps was that they gave frameworks through which the technical and functional teams could interact (often agnostically) in more predictable, industrialised ways. This brought them together more efficiently in terms of delivery, but did little (if anything) to bring them together in mindset.

Architect-types have sometimes thought that they could cover the whole spectrum, and also aimed to bridge the portfolio-level (the enterprise view) to the application-level (the project view). But very often this just has created a third silo - the so-called ivory tower. Additionally, with many architects having come from infrastructure-planning or bespoke-development backgrounds, in the many industries dominated by packaged apps they have had additional disadvantages in building the credibility and buy-in they've needed.

However, today's increasing focus on SOA and in general architecture above the level of the project, driving cross-project synergies and initiatives requires that the three viewpoints work together. Take composite applications for example. How can (what are by definition) composites of data, function and technology assets, mixing packaged and bespoke, be put together scalably without united functional and technical views? And how can a portfolio of them be managed without an architecture view across them that balances the delivery-proposition to each project, with the customer-proposition to each part of the business, and the ongoing ownership-proposition to the enterprise as a whole?

None of these issues are (in principle) different from those of our recent pasts, but this is the first time that the very value proposition of the approaches themselves have been contingent on these viewpoints being able to work together. And making it stick.

Technorati Tags: Enterprise IT IT Excellence Enterprise Architecture EA Service-Oriented Service Architecture SOA Composite Applications

The Inmates Taking over the Asylum? Traditional Data Management Meets Web 2.0

sam.o.lowe@gmail.com (Sam Lowe) — Sun, 7 Jan 2007 22:17:00 GMT

What does the way the public internet is evolving tell us about how Enterprise Data Management might change?

After having presented on these questions recently at DAMA Europe, I've been contacted several times about running an independent follow-up workshop in the London area in the early part of 2007 to work through some of these issues. Anyone who'd be interested in such a workshop, please drop me a line through a comment, link or the email address on my profile page.

Enterprise IT has always looked to improve efficiency through industrialisation and automation. This has often meant standardisation, centralisation, and consolidation were/are used as methods of enabling this. Enterprise Data Management has been no different and has generally followed this path. Whether you're thinking about (1) relational databases and data modelling, whether you're thinking about (2) data warehouses, the CIF and business intelligence, or whether you're thinking about (3) metadata management, business activity monitoring and corporate performance management ... all generations have held standardisation, centralisation and consolidation either at their core, or at least relied on it to a significant degree.

But outside of the enterprise, the internet of enthusiasts and consumers has had rather different dynamics. Rather than automation and industrialisation, its key objectives have tended to be more like access and collaboration. This difference has enabled interesting emergent innovations in the internet that enterprise IT has been able to absorb back into its own practices.

This applied to the original world-wide web and the eCommerce wave where the mass access to information at almost zero cost enabled new channels. New channels for organisations to interact with their customers and suppliers, new ways for systems to interact with their users, and new ways for employees to find and get at information hidden in their organisations' islands of information.

It's interesting to think about what the current wave of evolution in the internet (generally referred to as Web 2.0) might mean. One of the characteristics about what is known as Web 2.0 is that rather than the web primarily acting as a new channel providing greater access and cheaper communication, the emphasis is on the web being the platform itself, providing a basis for new levels of collaboration and cheaper participation.

For the Enterprise Data Management world there is an additional focus in this. As with the original web generation, although the channels available and information access grew, the data was generally still 'owned' in the same place it always had been. However now, the Web 2.0 evangelists and analysts talk about the user controlling their own data - which should grab the attention of all professionals involved in enterprise data, as it's a big mindset change from where things have been in the enterprise.

There are many interesting patterns in Web 2.0 that warrant analysis for clues as to how enterprise data may evolve. The Long Tail for example is very relevant, as enterprise data's current techniques and approaches are generally very expensive in terms of labour, time, and politics and so can only be applied economically (if at all) to the most common pieces of shared data, and not the huge amount of other data. The dynamic bottom-up network effects and the emergence that follows fosters behaviours and insight not normally available with the top-down governance and compliance methods of traditional enterprise data. And the user-provided tagging is very different to the rigid and controlled hierarchical taxonomies, structures and syntax that typically have been the basis of enterprise data approaches. And so on ... (I'll follow up on some of these separately).

So Web 2.0 will undoubtedly introduce new technologies like RSS, Wikis, REST, Blogs, AJAX etc into enterprise usage. But perhaps more interesting than the new technologies is the potential influences it will have on Enterprise Systems design and the way Enterprise Data Management will adapt.

For example:-

Web 2.0 EDM Influence	Challenged EDM Characteristic
Intellectual Commons	Not just Elite Experts
Emergence	Not just Upfront Grand Design
Dynamic Metadata	As well as Fixed Syntax
Tagging & Semantics	Instead of Just Hierarchies
Resources	Beyond Just Data
Links	As well as Replication
Others’ Resources as Your Platform	As well as Your Own

Many will recognise that there has been quite a lot of discussion about what Web 2.0 will mean to Enterprise IT generally (Andrew McAfee's Enterprise 2.0 threads are a fine example) but I havn't yet come across discussion on what it specifically may mean to Enterprise Data, which would seem to be one of the prime areas affected.

Technorati Tags: Enterprise IT Web 2.0 Web_2.0 web-20 Data Management Data Architecture Information Architecture Information Management

Architect Insight 2007 Event

sam.o.lowe@gmail.com (Sam Lowe) — Sat, 6 Jan 2007 15:03:00 GMT

Matt Deacon of Microsoft UK and IASA UK has just blogged about how he's tried to construct Microsoft's Architect Insight Conference at Celtic Manor in March this year so that it is as interactive and inclusive as possible, with discussions and working groups that are deliberately intended to ask questions and work through issues rather than presenting predetermined solutions. I'm looking forward to this dynamic, it should produce some interesting results, it might reduce the level of death-by-powerpoint, and maybe even keep some of the 'spin' in check.

What's particularly unusual about this event is the breadth of the agenda, with work-streams covering a broad set of areas such as 'enterprisey' strategy issues like the one I'm speaking on, through web, security, and infrastructure, as well as the software design issues (that you'd perhaps expect from an MS -organised event). It makes for an interesting mix of perspectives.

Technorati Tags: Enterprise IT Enterprise Architecture EA Software Architecture SOA Service Oriented Services Architecture Microsoft IASA

Open Group's EA London 2006

sam.o.lowe@gmail.com (Sam Lowe) — Sun, 3 Dec 2006 21:29:00 GMT

I was at the Open Group's Architecture Practitioners' Conference in central London last week. It was a good event, well organised by The Open Group and Architecting the Enterprise, with good facilities and well attended by mostly (as its title would suggested) EA practitioners. Rounding the sharp edge off the plenary sessions, Steve Redgrave closed the first day with a interesting talk on the complexities of the Olympics bid and organisation for 2012.

I'm pleased to say that unlike David Linthicum's and Todd Biske's recent observations from EAC in San Diego, there was a good amount of balanced attention to SOA as well as pure Enterprise Architecture. I would say that it did seem that most of the content was based firmly around either one or the other, but the organisers did a good job of mixing the agenda up to keep it balanced.

My slot was on how Enterprise Architects' roles need to evolve as SOA gains more traction in organisations. This wasn't to explore SOA as an architectural style, but rather to present how SOA had got many non-architects talking about architecture which is changing many non-EA methods and attitudes in Enterprise IT, and will have an important effect on the role and relationships that EAs should have within their organisations.

For example, the increase in the old need to get closer to individuals in the business units with change agendas, the need to accumulate more joint responsibility in project delivery and operations, and generally the imperative to have a hand in the ways that methods in other parts of IT are changing, rather than rising above it all. There is so much that needs involvement in fact that the need to develop ways to prioritise the use of architecture resources since good ones are so scarce is even more key. EA wouldn't want to spread itself too thinly after all.

I'm not sure of the organiser's plans for circulating the slides, but if anyone who was there wants a copy of the slides I presented, just drop me a mail using the address on my blogger profile page.

One of the other presentations on the first day that particularly caught my eye was David Robertson from IMD's presentation entitled EA: What to tell the Management Team. It drew from his work in the recent Harvard Business Press Book Enterprise Architecture as Strategy, and had some great material about engaging EA with the business and with projects, working towards making it pay for itself which was great to see. Recommended.

Technorati Tags: Enterprise IT EA Enterprise Architecture TOGAF The Open Group SOA Service Oriented Architecture Service-Oriented

Changes in the Role of the Enterprise CTO

sam.o.lowe@gmail.com (Sam Lowe) — Wed, 15 Nov 2006 20:12:00 GMT

I was talking with Ade McCormack of Auridian a few weeks ago on the changing nature of CIO and CTO roles that we were seeing in the organisations that we work with or talk to (mostly Western Europe-based).

Ade has incorporated some of his thoughts on the subject into his regular column in the FT last week, and it is definitely worth a look for anyone interested in IT leadership. If you didn't see it in print on Wednesday, then FT subscribers can get to it here, or it's on Auridian's site here.

Ade proposes that for organisations to be able to make use of IT effectively in developing their business model and their operating model, the CIO role needs to be far closer to, focused on, and more respected by the lines of business and the board. He describes that in order to free up the CIO to do that, you need to take away the operational side of the traditional CIO role. He proposes that the CTO role in organisations should change so it moves beyond the 'cleverest techie' that it tends to be now to apply its understanding of the technology to take operational accountabilities. He then proposes that this new repurposed role could then be supported by managers with the people skills and operational focus to run the IT department machine.

I like a lot of what Ade describes here, it has a lot of overlap with what I see. The CTO role as it exists in some organisations today can be a strange one. It is a term that has come from technology-centric companies such as Telcos, IT vendors etc who clearly need a technology visionary at the top of the organisation. But it has then been transposed into non-tech organisations, becoming a badge assumed by the people with the best technology expertise. But that doesn't seem appropriate for many. It can be difficult to justify the value of having a chief technical individual without accountabilities for realisation in non-tech-centric industries. Without accountabilities, CTOs' offices can become a strange kind of retained in-house IT analyst organisation, which isn't linked to value as the business would define it, and doesn't exactly help with business-IT alignment.

Of course outside of the US, organisations don't always use the terms CIO and CTO, indeed in the UK it's more common for the chief role to be IT director, however the same dynamics can still be seen. The value to decoupling operational and transformational focus, and the value of coupling technology strategy and realisation accountability both still apply.

Technorati Tags: Enterprise IT Business-IT Alignment IT Leadership IT Excellence Business-IT Engagement CIO CTO

DAMA Europe 2006 - Web 2.0 and Enterprise Data Management

sam.o.lowe@gmail.com (Sam Lowe) — Sun, 5 Nov 2006 21:54:00 GMT

I was at DAMA Europe 2006 in central London last week. I've been going for quite a few years now, and always find it thought-provoking. However there were some notable changes this year that were interesting.

The last couple of times I've spoken there it's been on more traditional Enterprise Data Architecture topics, which has been a good match - it is after all the largest independent Enterprise Data Management, Data Modelling, and Metadata Management event of the year. But this time I'd arranged with the organisers to do a bit of an off-the-wall topic drawing on Web 2.0 and asking what it might mean for Enterprise Data Management and Data Architectures.

In fact it seemed that there were similar messages starting to creep through into quite a few of the other presentations, which was very interesting. It seemed to represent a growing appreciation from some corners that the Enterprise Data Management community needs to think about managing information in ways beyond just the traditional consolidation, centralisation, replication and governance it has traditionally relied on.

I plan to put together some postings on the topics here, and having had several interesting conversations after the event with people that gave me some new ideas on how to structure this, I'd be interested to hear as well from others in the blogging community who have interests in this space.

Technorati Tags: DAMA2006 DAMA Enterprise IT Web 2.0 Web_2.0 web-20 Data Management Data Architecture Information Architecture Information Management

SOA Governance is not about Technology

sam.o.lowe@gmail.com (Sam Lowe) — Tue, 10 Oct 2006 23:48:00 +0100

I've had a few meetings recently where, much to my frustration, clever people have either explicitly or implicitly equated governing SOA or managing SOA with technology solutions that call themselves 'SOA Governance' or 'SOA Management' solutions. This has come from several corners of the industry, of course from technical people from software companies as you might expect, but also from architects at large blue-chip corporates as well. Normally in this case it's the Web Services Management and SOA Registries technologies.

But this is crazy. It seems to be a repeat of the 'IT tool syndrome' where IT people think that implementing a tool will somehow change people's behaviours. Reading Dan Creswell's comment on Steve's Dark SOA Revisited post compelled me to add comments there on a very similar topic.

So, where has this recent resurgence of the G and M words in a software context come from? I'm guessing that it's been from the recent growing realisation that if you don't have a way of managing the services you have above the level of each project or initiative or team (and the particular requirements of each) then you are extremely unlikely to be able to make SOA work (in any way more than a new type of technology plumbing) across projects, initiatives or teams. And that is very important for SOA.

But managing does not just mean storing it in a repository, putting it on a web-site for developers to look at, controlling the deployment, or monitoring the runtime. It's not that all of these things aren't useful, or important. It is that _managing_ in this context is not a technology issue. The technology may be useful, but it isn't the issue. Buying and using the tool won't change behaviours in an Enterprise environment without the processes, roles, stakeholder buy-in, incentivisation, and behaviours that will allow the tool to add efficiency.

Plus, managing in this context also has to fundamentally include proactively collaborating with each part of the business on planning, defining, evangelising about, running and improving a portfolio of services that they can care about and see value in (rather than 'technical stuff' which makes their eyes glaze over faster than you can say SOAP [or REST])

... that way when you come to manage the requirements for each service / project IMO you're far more likely to end up with shared services, consolidated services, composite services, differentiated services etc rather than a new fragmented duplicated obscure legacy of services to add to the old legacies you already have.

Technorati Tags: Enterprise IT SOA Service Oriented Services Architecture SOA Governance Web Services Management

Flat-Packed Applications - Composite Apps?

sam.o.lowe@gmail.com (Sam Lowe) — Sat, 30 Sep 2006 20:59:00 +0100

Not some new Ikea product range, but rather Owen Pettiford's thoughts on how Composite Applications have the potential to become the key 'third-way' for enterprise solutions, beyond today's well trodden 'bespoke' and 'packaged application' paths.

Of course today's conventional wisdom goes:
*If what you need is unique or not yet available then you probably need bespoke (particularly if having exactly what you need can give you a competitive advantage).
*If you need to buy-in 'best-practice', implement something that's commodising, or just need somebody else to bear the cost of maintaining/developing the solution then packaged apps are the answer.

But is it really black or white - what about all the shades of grey in between for requirements where the packaged apps aren't quite right becuase of industry or cultural specifics (but it's hardly a competitive differentiator)? What about different economic models where you don't want to bear all the cost yourself, but want to avoid funding development of a product-suite most of which you don't use? Etc etc and so on.

But Owen adds a sub-text - that it will only happen if we primarily focus on the solutions themselves with a secondary focus on the technologies and frameworks that underpin or enable them. In other words, the opposite of what the technology vendor-driven hype in the IT industry predominantly is at the moment. I think he's right.

Technorati Tags: Composite Applications Composite Apps Enterprise IT Enterprise Applications SOA Service Oriented Services Architecture

Potential Pitfalls in Technology-Enabled Business Transformations

sam.o.lowe@gmail.com (Sam Lowe) — Fri, 18 Aug 2006 00:36:00 +0100

In my experience, when Enterprises perform Business Transformation design exercises where the solution is to be enabled or underpinned by technology, a core list of the same potential pitfalls are regularly encountered. It would be interesting to see whether others come across the same types as well, so here are a few examples of what I always tend to come across:-

*Potential Pitfall*	*Ways to Avoid It*
Lack of clear business justification for all IT decisions	Retain a clear audit trail for Design Rationale through a fit-for-purpose framework that leads from Design Objectives->Design Rationale->Business case->Executive sign-off.
“Package-Think”	These exercises are almost always about more than a package, there are almost always other pieces required, needing the correct sequence of SMEs to cover all, not just the biggest and highest profile one.
Technology instead of Architecture	A solution architecture approach (rather than a technology beauty parade) is the only way to consider the implications on the design and technology decisions, and therefore deliver the best-fit technology portfolio for the solution with the most accurate and informed business case. Performing your 'as-is' analysis to surface the constraints and opportunities you have, and use of external SMEs for what’s possible.
Too Much Detail Too Early	Don't get bogged down in analysis for analysis sake in the early stages. Get enough information to understand enough to make a decision rather than know everything in detail. An 80/20 approach will identify the hot-spots. Focus down on areas that will deliver the most benefit. Move to the detail when the way forward has been agreed.
Focusing on the First Step	A balance of ‘art of the possible’ with ‘what needs to be fixed’ is necessary to design a solution which meets the requirements, but also delivers the business case through its life. Generally this requires that as well as agreeing objectives and design principles, the early phases also need to collect the hypotheses on what needs to be fixed, whilst of course doing the necessary identification of the 'as-is'.
Wish List	There is always a great temptation to use a transformation design exercise as a means to fix bug-bears which can colour / mitigate the value of the transformation. These quite likely need to be recorded, but the process must manage them so they don’t dominate. Not everyone likes the word 'strategic', but here it is important.
Reinventing the Wheel	Use best-practice 'off-the-shelf' processes as a design input, and try to stay vanilla (minimising customisation to where it's really needed for good business reasons that can't be standardised) to minimise IT costs and maximise efficiencies.
Waterfall	To avoid a sequential ‘waterfall’ where the outputs of one team become simply the input to another (with the inadequacies & risks that carries). The process needs to be iterative to repeatedly test assumptions, consider implications & and refine the design.
Lack of joined up working between IT and the Business	The work-streams need to work together to mutually inform each other, and to generate the cross-functional buy-in. Of course the IT solution will not drive the to-be business processes and operating model, but it will be an influence on them - unless you believe that what is available and what is achievable doesn't influence what is feasible and deliverable.

Technorati Tags: Business Transformation Enterprise IT Business-IT Alignment IT Excellence Solution Architecture Enterprise Architecture EA

Dare I mention the word 'Applistructure' ...

sam.o.lowe@gmail.com (Sam Lowe) — Tue, 1 Aug 2006 23:59:00 +0100

I'm not sure who ever came up with the word 'applistructure', but I think it may just be one of the worst IT jargon words ever created. I have never been able to mention it to colleagues or clients without at least raising a smirk, or more likely inducing outright ridicule.

But actually, tempted as I have always been to disregard it as some believe we should, I do actually think the concept behind the silly word is rather important. In particular, for enterprise IT departments, I believe it represents something they should be thinking very closely about - in that it represents a significant change in the capabilities, divisions, skills and focus they need moving forward.

The idea is that 'applistructure' is the blurring of the line between enterprise applications and software infrastructure, in that, rather than specific silos of applications sitting on top of the shared software and hardware infrastructures, the enterprise applications themselves become part of the shared capabilities or resources which together are almost akin to infrastructures, at a higher level. This is to say that the application components (e.g. ERP modules, CRM modules, SCM modules etc) 'join up' with the software infrastructure that underpins them (e.g. app servers, integration buses, data management hubs, portal platforms etc) to form a higher level infrastructure that businesses can exploit, share and reuse.

Before your natural cynicism causes you to see red, let me explain why I think this is important.

Traditionally the lines between the functional people and the technology people in IT have been fairly firm, particularly in the packaged applications space. Both Oracle and SAP for example have very distinct teams, often with totally separate structures, both in their clients, but actually also within the software companies themselves. Although some separation is necessary for specialisation, the chasm that tended to emerge between the two sides (I believe) caused real quantitative and qualitative issues.

I believe that the 'reunification' of functional and technical IT viewpoint (whether you call it 'applistructure' or something else) is not a bad thing. For some IT departments, it will help move them away from purely obsessing about technology, and onto the business value from the architectural combination of the two. And for others, it helps force the functional IT people to not ignore the technology, and hopefully therefore to not carry on creating solutions in isolation of (or in spite of) the silo'd, duplicated or deprecated technology involved.

Of course this is closely related to the concept of SOA. As such, 'applistructure' implies that your SOA technology platforms need to managed and designed on the terms of your applications and data . No more separation considering interfacing as being separate from applications. In SOA you need a common unified and mutually-informed approach to applications, integration, data management, user interface etc. The decoupling requires this.

The concept of 'applistructure' also helps combat the plague of pseudo-value that has come into being recently, where software infrastructure platforms try and claim value propositions of applications they support. Quite frankly I've often in the past got quite irritated when software infrastructure vendors pitch their platform technology as directly providing business benefit even though it is not a solution. You know the kind that claims "our competitive differentiator is business agility" when all their technology does is provide a means to move data with less mouse clicks. I've always found this bizarre, as though the company that makes the tyres for my car would claim that their competitive differentiator is my timeliness for meetings, when what their product really does is at most allow me to go round corners slightly faster. Of course, it's not the tool, it's how its used with regard to specific solutions that is what drives business value. And by bundling up both solution-specifics and non-specific software under a common reusable architecture, the concept has far more leverage that the disjointed combination of applications and the various types of middleware did before.

So of course, one man's infrastructure is another man's application. But moving the average perception of what the platform is up, and moving the average level of concentration towards application of these higher levels of platform, may well be a very healthy exercise. Even if the word is still stupid...

Technorati Tags: Applistructure Service-Oriented Service Architecture SOA Enterprise IT ESB BPM

Separation of Concerns

sam.o.lowe@gmail.com (Sam Lowe) — Mon, 24 Jul 2006 23:29:00 +0100

Jeff Schneider has blogged about how he's noticed that many large organisations are expending a large proportion of their (supposedly) SOA efforts into what is actually work on good old-fashioned separation of concerns.

This is an interesting question - this 'Concern Oriented Architecture' (please Jeff, no more acronyms!) is based around decoupling different pieces of configuration/code from each other so that they can each be defined and managed more easily, but primarily so that they can be shared, reused or assembled more readily - If that is the case, aren't these shared/reusable things actually shared/reusable technical _services_?

They may be, and that is the reason I think why this 'COA' is sometimes presented as being SOA. However, shared or decoupled technical services may be good technical design, but they are not directly related or aligned to any kind of service that a business provides. Jeff and others have blogged about this misdefinition before. One blogger (who shall remain nameless) has even taken to chastising a form of this as 'SOD IT' (Service Oriented Development of IT) !

However, I'm more positive about this approach. Even if it shouldn't be the destination, it doesn't mean it's not a valuable exercise. And what's more, it may well be a valid step on the maturity curve for many organisations toward SOA proper as it may constitute a modernisation of their software infrastructure that enables them to start providing more meaningful services. But (as Jeff says), it does need to be considered separate so that it is not misrepresented. If you are working on an SOA roadmap or transformation exercise, I would recommend ensuring that this is a separate work-stream, and that the business case for doing it holds water independently from the other workstreams. Make sure you ask the hard question for each component or layer, of whether each is really needed or whether it's a technology 'nice-to-have'. It may well be needed, but we all know that just as many value propositions have been blown in the past by over-engineering the technology as have by underengineering it, and sooner or later you will get asked those hard questions, so best have done the analysis before you do.

Technorati Tags: Software Architecture Service-Oriented Service Architecture SOA Enterprise IT EII ESB BPM

Business-IT Alignment: Synchronisation or Aspiration?

sam.o.lowe@gmail.com (Sam Lowe) — Tue, 18 Jul 2006 11:19:00 +0100

Reading Richard Veryard's assertion that IT should be decoupled from the business rather than aligned to it reminded me of a conversation I had recently with an freelance IT Excellence consultant on the topic of business-IT alignment.

He also, like Richard, was of the opinion that the concept of business-IT alignment was unhelpful because it implied synchronisation of IT into the business cycles, which constrained the value IT could bring to the business. Of course he was right about that being negative, and I agree with Richard's arguments about the downsides of that model. Indeed, if you were to make IT a slave to the business cycles, plodding along behind not looking around at the effects of what it was doing, not looking ahead at what it could be doing, then it would only be able to be reactive and tactical. Only thinking about how to turnaround the next request that came its way, unable to innovate, or even think strategically. This is not a good state of affairs, and is actually a bit too close for comfort for some of us to negative characteristics we see today in many organisations. These are the exact constituency for whom improving business-IT alignment was supposed to be an improvement. Have we been aspiring for something that would actually make things worse?

Well, I'm not sure that we have. And the main reason why is that I never use the term 'business-IT alignment' to imply synchronisation. I can see why alignment could be interpreted as implying synchronisation, but that isn't something to be aspired to. On the other hand, improving alignment in objectives, in priorities and language, even in intentions and mindset really are aspirations with real benefit.

And just as important, if not more, is the perception of alignment. That is to say that the lines of business believe that IT are aligned to their businesses and the priorities of the business as a whole, rather than being aligned to their own agendas, their own technology projects, and their technology suppliers. In many organisations, it's only when the lines of business perceive they can get business-focused IT expertise (innovative and/or pragmatic as required) from IT, does IT get invited to the top table.

As I have blogged about before, I do like to look at IT departments as independent IT services businesses. Any independent IT services business needs to operate and innovate asynchronously from the engagements of sales-cycles it has with its customers, but the successful one innovates only in terms of objectives or opportunities its customers have, it thinks in terms of how it can use its own resources to provide value to its clients' businesses, it needs to describe what its value-add is to the businesses of its customers, and it focuses its delivery on success of the overall business change rather than just fulfilling on the technical completion of the task it is in charge of.

We spend too much time in IT discussing semantics (and alignment may indeed not be the right word), so I don't want to reduce the discussion to one of 'which is the _correct_ meaning of the phrase', but there are some big concepts hidden within this that seem worth debating, and that go way beyond a phrase.

Technorati Tags: Enterprise IT IT Management IT Excellence IT Governance Business-IT Alignment

More on the Real Challenges in SOA

sam.o.lowe@gmail.com (Sam Lowe) — Sat, 8 Jul 2006 20:13:00 +0100

I posted around this time last year on this blog about the real issues making and breaking SOA initiatives, and how they are under-served. It was great to see it amplified by James McGovern, but I didn't follow up at the time. Thinking it was about time I did, I have been doing some surfing to see what the current state out there is. Maybe I've been looking in the wrong places but I have to say I havn't come across a great deal that would be of practical assistance to organisations in assessing and planning SOA initiatives.

Of course many on the web (probably best exemplified by the guys at OASIS) have done great work in starting to define the technology-types and architectural patterns involved in SOA, but what about other factors key to organisations realising the benefits that SOA promises?

I'm talking about factors like how SOA affects capabilities and organisational models needed, governance approaches and charging models, mandate & business engagement, migration & transisition methods, workplace infrastructure and methodologies, relationships to current technology portfolio and programme management techniques etc etc etc (this list could go on and on) ?

And in addition to all these 'do-ability' factors, are those to do with how you assess and plan the value of SOA in the first place. For example, how you identify where SOA can add the most value and where it adds the least, because it certainly doesn't add equal value everywhere, and do you really want to be investing your SOA resources in an area which doesn't deliver benefit - maybe where it proves the opposite infact? In my experience getting an answer to this kind of question is necessary to do a decent opportunity assessment, and getting a decent business case put forward.

I don't know about you but it somethimes scares me a little when I think about how, despite the fact that it's now five years since the first time I explicitly used SOA in a large-scale architecture (albeit under the rather ungainly name 'Process-based Service Model'), there still isn't nearly enough practitioner, manager, or business-level content available to people that doesn't have some software-sales spin inherent in it, or isn't about the technology?

I did come across quite an interesting (if very short and dareisay incomplete) post from Carl-Henrik Wolf Lund, which I hadn't come across before. And of course, Brenda Michelson has just convened 'BDA Community Project #1' on a closely related topic (as suggested by MarkG). So there's positive signs that a dialogue might be starting amongst blogging practitioners.

But, if these subjects are key to being able to execute on the concepts that most of the IT industry is holding aloft, why don't you hear about it more than very occasionally in the software vendor SOA hype ? Unfortunately, it is not related to selling their technology. At least not directly related. To be fair to them, I've had conversations with most of the large enterprise application & technology platform vendors on this topic over the past twelve months, and they're all interested in collaborating on these subjects, acknowledging the shortcomings of the view they're able to provide, but are honest that it's just not a competency of theirs, and therefore not going to be part of their core education or marketing strategies.

But many in the SOA crowd (and not just the vendors) need to broaden their sights from viewing SOA, or even Enterprise IT as technology. It may be stating the obvious but Enterprise IT is a business service itself after all, who's toolkit just happens to consist of technology. IT strategy gets a bad rap from many in the operational and delivery technology communities, but you can't solve the thorny SOA execution issues through technology, or through implementations alone. The operating model, governance, skills and culture of the IT department are just as important (actually more), as are IT's relationship with and attitude to the various business units.

Technorati Tags: Service-Oriented Service Architecture SOA SOA Realisation Enterprise IT Architecture Governance

Are Most Enterprise Technology Selection Exercises Flawed?

sam.o.lowe@gmail.com (Sam Lowe) — Fri, 7 Jul 2006 23:09:00 +0100

There is no single best way to select Enterprise technologies or Enterprise application packages. However, having been involved in many (on both sides of the fence, customer-side and vendor-side), it never fails to amaze me that each process I come across seems to either 1) Have been conceived from a blank sheet of paper, from first principles, or 2) Is the rolling out of an old set of templates that someone found on their hard disk which came from a very different context.

I suppose it's indicative of the youth of the IT industry that there doesn’t seem to be perceived or taught guidance on this, and instead the task is often left to people to make up as they go, or is outsourced to 3rd parties who may see the measurement of their success/quality as the amount of documentation produced! Things would be better if there was a dialogue about this, so in that spirit, here's a couple of thoughts/observations of mine to be shot down.

In my experience there really are only three ways I've come across to run a selection exercise:-

First is the traditional Request-For-Information (RFI) followed by Request-For-Proposal (RFP) process.

*Your classic paper-heavy type of affair where each party works through the night in sequence to turn embed every thought or nuance into a series of lengthy documents. You generally dredge the brains of everyone you know, trawl the internet and raid the piles of old trade-magazines that are under your desk, to come up with every product you can buy that might do something similar to some of what you think you might need. And then you start communicating with all vendors via the medium of MS Word whilst refusing to actually talk to anyone.

*Then, typically I find these work on a critical approach I.e. you look for something that is unacceptable about a particular vendor or product based on their RFI answers, and use that as a reason to knock them off the list. Eventually you're left with 2 or 3 to go to RFP with.

*This type of process is unbeatable for auditability, creating a 'whiter-than-white' approach that appears fair to all. However, it is extremely resource and time-inefficient to all sides, and often runs the risk of selecting the vendor who's best at the process (the game?) rather than has the most appropriate product.

Second is the accelerated version of the RFP-centric process, where you go straight into the RFP, with a smaller number of parties, from an independently-researched and validated shortlist.

*This process is great if you know who the contenders are, and you can get away without the paper-trail (most organisations can in my experience). Plus, the increased level of communication possible because of the shorter process and smaller number of participants means the results are more likely to be on target.

*But being able to do this is dependent on really knowing who the serious contenders are in advance. And really knowing does not mean just going for the most famous. This usually means using someone who's got real expertise _and_ experience of similar situations. Please do not just choose the few closest to the top right-hand corner of a Magic Quadrant. Even if it's the right MQ (and who's to say the big G have chunked things up the same way you should), using an MQ alone is a complete lottery. Frankly, it smacks of the IT department wanting the best toys to play with, or worse, wanting the most valuable addition to their resumes.

The third option is to run an interactive process from the start, concentrating on minimising the documentation, and instead scripting the interactions to collaboratively work though the issues, iterating towards a joint-solution that can be tendered against with no misunderstandings.

*This is of course an extension of a joint-design exercise, applied into the selection world. It can deliver the best results where the right selection cannot be reduced down to a feature-function comparison, and a qualitative decision is needed (e.g. on process issues and the user-level experience, rather than technical does have / doesn't have), bringing the business and IT communities together to assess the contenders. It also is the fastest way to get to a selection, thanks to not having the overheads of all that documentation, and instead using the time to do design and education work you’d have to do anyway.

*But, this process leaves little in the way of an audit-trail, and it can only work with a very small number of vendors/products without becoming very cumbersome. So it can only really be considered where there are only a few credible alternatives and the fact that the others are not credible can be clearly articulated (with external assistance normally).

So, there is a quick tour through the three types that I see out there. There are some interesting other thoughts from Tony Byrne here. I'd be interested to hear other people's experiences.

Technorati Tags: Enterprise IT Technology Selection RFP Process Technology Portfolio Management Application Portfolio Management

Mashups in Business - the next big thing?

sam.o.lowe@gmail.com (Sam Lowe) — Wed, 5 Jul 2006 20:38:00 +0100

After far too much Burgundy at the extremely pleasant Les Fontaines earlier in the week, conversation got rather irreverent about some of the less worldly-wise extremes of Web 2.0 - what if (just for fun) you applied the more extreme dogma to some rather inappropriate business scenarios ... mashups in Life Sciences anyone? Should have potential to transform the productivity of the industry to say the least!

Steve Jones has written some of it up in his own inimitable style.

Technorati Tags: SOA Service Orientation Mashup Mashups Business Architecture Architecture Governance

SOA OD at Sapphire?

sam.o.lowe@gmail.com (Sam Lowe) — Sun, 4 Jun 2006 15:44:00 +0100

I was at Sapphire in Paris last week, and found the process pretty useful. Met a lot of interesting people, and the client I was there with are considering HR initiatives, so some of the technology (e.g. Duet) looked potentially useful for them.

But Sapphire isn't really a techno-fest like many other IT conferences. A couple of characteristics that you couldn't help but notice and that aren't really techie at all were a) the huge number of vertical industry-solution tracks covering what seemed like every business vertical there is, and b) a huge amount of Enterprise SOA (SAP ESA as was) activity and information. Both were impressive, but the latter of course ran the risk of being a bit overwhelming.

But this is really a characteristic of the software marketing machines rather than a Sapphire-specific issue. Vinnie Mirchandani at deal architect has posted that he believes SOA is being over-promoted and maybe over-engineered (my words not his). Thomas Otter's also just got back from Sappire Paris and has has blogged that he is in some agreement with Vinnie. It is apparent that there is a risk that the heavily software-centric SOA marketing obscures the facts that it takes a lot more than some new technology, or some new approaches to applications, to produce business-valuable results from SOA. This is similar in some ways to an old posting of mine on the real issues making or breaking SOA.

It seems that for many enterprises, the major value to come from SOA isn't from the 'S', but rather from the 'A' in SOA. For many, this generation is the first time they've seriously thought about architecture at all, at least above the project-level, sometimes above the tin-level. Many will benefit from the flexibility and focus of service models, but for some that may be second to more basic value-propositions, at least in the short term. Fundamentally the focus on achieving value beyond a silo, driving commonality and collaboration, no longer sub-optimising and managing technology assets on a business-lifecycle rather than a technology one may be where much of the value comes from for some.

But I'm optimistic about SOA. Even though there were a huge amount of ESA sessions at Sapphire, I was pleasantly surprised with some of the pragmatic insight emerging (rather than the all too common "SOA will make you more flexible / agile / attractive-to-the-opposite-sex"). They were also giving out copies of Dan Woods' new book on Enterprise SOA, and I was also pleasantly surprised at the next generation of IT management and operational insight they have added there since his last. Ron liked it as well. However, the true business view of how service-orientation can add value to businesses and what is necessary to achieve it (the service-oriented enterprise if you like), was more notable in its absence.

James Governor has commented on how SAP have generally had more insight and more communication skills about the business angle than the other IT vendors, and he's right. But that said, I do believe that even business-minded IT folks and IT vendors need to be careful with the dogma about services. I believe that the real services thinking needs to come from business people if you are to avoid the cart attempting to lead the horse. Of course, the perfect solution is for business people seeking to innovate and transform their business to include those business-minded IT people in the process, as joint teams are always going to have the best chances of identifying the optimal solutions.

One last bonus at Sapphire was the Chief Architect Session on Tuesday afternoon. SAP spoke about ESA and their new BPX (Business Process eXperts) community, which was good to hear about, but there was also a refreshing item where Derek Prior from AMR talked about the importance of EA in a) providing leadership, methods and skills to actually execute on SOA, and b) bridging the business and IT gap for collaboration in SOA. Anyone who's read posts of mine before will no doubt guess how much I liked this.

Of course, this EA Session is all part of the big applications vendors increasing interest in EA (as Neil Ward-Dutton has blogged before about here), but the fact they are doing so is a great move, for them and the industry. For far too long architecture was viewed by the apps-people as being separate from the applications world, and even more separate from the apps-vendors' world. It was seen to have come from an infrastructure design and bespoke software development background. But this artificial historical division just meant that each vendor-island created its own silo of narrow architect-type roles (staffed by often excellent business-focused individuals). I believe that whilst of course you will always need specialisms, these hard divisions can't be sustained as the next generation of Enterprise IT (whatever it's called) breaks down these silos and looks for bigger and additional value-add.

Technorati Tags: SAP SAPPHIRE06 SAP ESA Enterprise IT Enterprise Architecture SOA Service-Oriented

Relating your services to your capabilities in SOA

sam.o.lowe@gmail.com (Sam Lowe) — Fri, 2 Jun 2006 19:34:00 +0100

So what exactly is a Service? If we are seeking a common definition of what SOA is (and much is made of the lack of this on the web), then we could at least start by defining the 'S' (in SOA).

There is some emerging consensus in some SOA communities that 'services are a way to gain access to, or make use, of a capability'.

What does this mean? Is it just semantics? Well actually it is important, and the reasons why are related to why services are a useful concept (and implicitly, when they're not!). I've posted a couple of thoughts as to why up on Nigel Green's Services Fabric blog.

What can ‘Agile’ tell us about how Business & IT could work together better?

sam.o.lowe@gmail.com (Sam Lowe) — Sun, 21 May 2006 13:05:00 +0100

I blogged recently about the need for joint collaborative business-IT teams in the early phases of business change or business transformation exercises. Anyone who's had exposure to IT software development will recognise similarities in the criticisms I made about current dysfunctional approaches to the criticisms of the old approach to software development, the so-called waterfall approach.

In this methodology of course, the requirements were gathered by one team before the analysis and design was carried out by another, before the development carried out by another, before the testing by another etc. It was real manufacturing production-line thinking about one group of specialists working in isolation on the task they have expertise in, before passing the work ('over-the-wall') onto the next isolated group of specialists.

It’s now well known and accepted that it never worked very well for IT software development and implementation. My feeling is that this is similar to the logic in my comments on how we need to now realise that a silo-based 'over-the-wall' approach doesn’t work well for business-IT engagement in business change design exercises either.

Of course in the software development world iterative, and more recently 'agile' methods have come into being as the dominant methods. Agile software development techniques are founded on principles like collaborative working, early engagement, iterative design and delivery, and frequent rationalised reviews. Good practices like this have demonstrated that they can help deliver more relevant software solutions that are (for example) more likely to achieve what was actually needed rather than what was initially thought to be needed, to deliver what was desirable to the user rather than the designer, and to deliver what was actually achievable rather than what was thought to be reasonable.

Now I'm not going to repurpose the word 'agile' here to my points on joint-teams, as many have borrowed that word for other domains already. But the commonaility in intention and concept is interesting. And even if (as some would point out) agile didn't really invent these concepts, for me it still illustrates some good practices that we actually need for Business-IT engagement approaches on a broader level beyond just software development.

Technorati Tags: Agile Business Relationship Management Enterprise IT

Sake, Astrophysics, and SOA?

sam.o.lowe@gmail.com (Sam Lowe) — Thu, 18 May 2006 20:11:00 +0100

My collegue Steve Jones has been on the sake again and has come up with the rather sci-fi concept of Dark SOA to explain how a lot of the hype and spin out there about SOA is simply not helpful to most of the people to whom it is addressed.

Technorati Tags: SOA Service Oriented

EA 2.0 vs EA 1.0

sam.o.lowe@gmail.com (Sam Lowe) — Sun, 7 May 2006 22:35:00 +0100

Brenda Michelson and James McGovern have recently blogged some thoughts as to what the next generation of Enterprise Architecture may hold. It's an interesting topic and one that I have also been working on recently with some of my colleagues, so here are a few of my thoughts as to what I believe 'good' should look like in the short to medium term.

Bad Old EA of the Past	Next Generation EA
Cost Centre	Self-Funding
‘One Best Way’ to do EA	Configurable, based on Desired Outcomes
Reactive	Proactive
Techies in ‘Ivory Tower’	Balanced Views to Release Benefits
Service-Provider Mindset (requirements-based)	Business-Change Mindset (collaborative engagement)
Defining & Evangelising the concept of SOA	Realising SOA (Business as Usual)
Mandating (or trying to!) the Solutions	Leveraging the Asset Base & Adapting

Technorati Tags: Enterprise IT Enterprise Architecture

Breaking Down the Walls: For Business-IT Alignment you need Joint Business-IT Teams!

sam.o.lowe@gmail.com (Sam Lowe) — Mon, 1 May 2006 13:03:00 +0100

What is required to drive the much sought-after and much talked-about Business-IT alignment? Not an easy question of course as it includes many aspects including organisational, governance, skill-set and architectural factors. But one factor that is part of the answer and I often find gets under-played is something much softer – the way that IT works (or doesn’t work) with the business in the early phases of their business change analysis and design exercises.

I spend quite a lot of time working with the business strategy & business change practices of my organisation (a large business and IT consultancy), and it often strikes me that the challenges in getting the IT consultants of a large consultancy working effectively and harmoniously with the business consulting parts of the same organisation are uncannily similar to those in getting the businesses of our clients work with their own IT organisations.

Business and IT people don't naturally use the same language, they have very different ways of working and views of the world, and they can often view each other with scepticism. Above all, they like to stay in their own worlds and throw things over the wall to each other in the form of requirements, or specifications. But the trouble with this is that for many tasks it is an inherently broken model of the world.

For example, how can the business give you their requirements when they don't know what it is possible to buy 'off-the-shelf'? How are they able to say exactly what they want when they don't know what IT assets they already have that they can use, or what constraints in their IT they have that they should be aware of before they make decisions? And how can they do that design of the business solution required when they don't know more than the superficial systems, data and infrastructure implications of any design decisions they make?

In many cases, decisions made early on with just isolated input from the line-of-business that havn’t involved the party (IT) that is required to progress the design into something that can actually be realised, are building on sand. The later engagement can uncover that the solution put together is actually not optimal after all, or that it is even not viable as a solution. Of course it’s rather embarrassing for all involved to have to rake up decisions made, to reset the communications to all the stakeholders who’ve been led to believe something else, and to revisit the business case that’s been put together on dodgy foundations. This can actually deepen the divisions between the business and IT, with IT feeling hard-done-by that they weren’t involved earlier and generally under-appreciated, and the business thinking that IT is just being awkward again and that they are ‘the tail that is wagging the dog’. So maybe as Dave Welsh has been quoted "The business is from Mars and IT are from Venus" after all and we should all just get used to it?

But this is a fundamental discontinuity of modern business-IT engagement that can further harm an often not ideal relationship, and it's totally unnecessary. Working together and communicating earlier and better can totally avoid it. So why don’t they do so? I guess that the business do not want to involve IT as they think that they don’t add to the business solution, that they focus on technical issues and factor that are irrelevant, unimportant or inappropriate to the task at hand. Fundamentally that they think that they are 'supply-side' focused (on how you build or provide IT solutions), rather than 'demand-side' focused (on how you maximise the value to the business and the wider enterprise).

IT are culpable in this as well as they (probably for fear of being over-exposed or investing their time in something they are not an expert in) often seem to want the business to get their requirements straight before they tell IT what they are (over the wall they come…). Maybe the feeling is that IT have enough to deal with without trying to work from changing requirements. Sometimes it almost seems that IT suspect that the business might try to dupe them, or somehow make them look silly so that they can criticise them more about how they don't deliver. So they seem to react by asking for absolutes, for guarantees, for an audit trail that they can point to later and say "this is what you asked for". But this attitude will always keep those that have it from the top table; how can they move from being a simple service provider, to actually being a key partner of the business with this culture and attitude?

And this is related to where the old-fashioned IT business analyst (BA) role is limited in this regard. Its shortcoming as the sole face-off to the business in design is that the BAs are great at gathering requirements and flushing out micro-level design issues, but they (mostly) don’t have the breadth and depth of knowledge of systems, data, and technology to do collaborative design. It just ends up being requirements capture, which is fine (indeed it’s required), but it’s not all that’s required. There are also issues with relying on IT project managers or particular packaged-application specialists alone in this capacity as well because of their limited breadth of knowledge and strong ‘supply-side’ delivery mindset.

In my experience by far the best solution is to use enterprise architects to bridge this gap and lead the multi-discipline IT team within the business' own early-stage pieces of work. These are not technical architects, systems architects or data architects (although they, like the IT PM or application specialist, may all have a subject matter expert role to play at this early stage), but enterprise architects who have the breadth of knowledge across all the domains needed and the business focus required to add the right value. This business-focus is needed so as to not fall into 'IT speak', or leap to an inappropriate level of detail, or to focus on what technology they're interested in as an individual (all of which cause business people's eyes to glaze over very quickly). They also have the experience of what the common implications of key big-picture design decisions are, and they have the best overview of what is available 'off-the-shelf' or in the existing assets. Finally, they critically have a methodology whereby they can pull in the necessary specialists as they are required, all under a common business-focused framework to ensure you get the right content, but in a way that is coherent and relevant to the business at this early stage.

These concepts when applied to these early-phase business change discovery and design exercises can involve just the right amount of 'down-stream' expertise in the appropriate 'upstream' activities, which maximises the likelihood that the right solution is put together, that it is appropriate, that it is also strategic to the enterprise, and above all that it and its business case are realisable.

(This is all in addition to a massive increase in buy-in and stakeholder-acceptance from having involved all parties in the definition of, and therefore to some degree the ownership of, the solution. It is my experience that there will be far less unnecessary re-questioning and general mud-slinging 'down-stream' if they're already on-board and knowledgeable about what's coming down the pipeline.)

Within the large consultancy that I work for, solving this problem so as to be able to offer joined-up early-phase business and technology services to our clients works really well (even if it hasn't always been easy), and I believe it should also be the de facto approach for the businesses and IT departments of our clients. In fact I believe it will become even more important in the new world of Service Oriented Architecture (SOA) as an enterprise service in your SOA is of vastly more value and significance if you can genuinely say that it as been collaboratively designed between the business and IT and is both relevant and useful to both.

So it's actually very simple, whilst it's not the only thing you need, if you want to work towards business-IT alignment, you're always going to need good business-IT collaboration, and the best way to achieve that is to break down the walls and use joint business-IT teams. It seems so obvious when you describe it like that.

Technorati Tags: Enterprise IT Business-IT Alignment Business IT Alignment Enterprise Architecture SOA IT Excellence Business Relationship Management

Tending to the Enterprise Garden ...?

sam.o.lowe@gmail.com (Sam Lowe) — Sat, 15 Apr 2006 22:11:00 +0100

You may have seen some of the recent questioning of the analogy of the construction industry that's often been used in the past to describe various tasks in IT.

Now of course software development is different from the wider world of Enterprise IT (Enterprise Architecture is not Software Architecture), and this is just as well as there has been some proposition is some communities that software development is like gardening. This may work as an internal 'supply-side' observation but isn't the most confidence-inspiring observation for some of Enterprise IT's customers. It implies all the wrong kind of aspects and characteristics for them, whilst the construction analogy is still quite useful.

The truth for Enterprise IT departments is that they are service providers to 'the business' of an organisation, that need to run like businesses themselves. Even if it doesn't work for software development necessarily, both the industrialisation overtones of the construction analogy, and the macro/micro trade-off analysis implications of the city-planning analogy are both useful for enterprise IT. They are both healthy aspects of maturation of a sometimes adolescent industry.

Of course it's not perfect, no analogy ever is, but then this is precisely because trying to describe the inherently virtual world of IT using terms that describe an inherently physical world is always going to be a partial success at most. In the real world everyone has a feel for the laws of physics and what is roughly possible and what is not, and how big a task it is. One of the challenges with the virtual world is that people don't have the same intuitive feel for it, so describing processes and roles that help to professionally derive this in terms people can understand is more important than it might at first appear, as long as it's not over-applied beyond where it's useful.

Technorati Tags: Enterprise IT Enterprise Architecture