The Enterprise Architect

6 questions to Maarten Steen, experienced researcher in the field of Model Driven Engineering

Wed, 08 Jul 2009 10:27:00 +0200

Recently I visited the second Dutch Model Driven Experience symposium, organized by Maarten Steen. It was interesting to meet other people in the field and to share experiences in the field of Model-Driven Architecture, Model-Driven Engineering, Domain-Specific Language design, Model-Driven Software Factories, and most important experiences with Model-Driven approaches in real-life projects. I thought it would be interesting to ask Maarten Steen to share some of his views and years of experience in the field. So, without further ado, six question to Maarten Steen:

Can you please introduce yourself?

Of course and thanks for the opportunity to present myself to your audience. You have a very interesting blog that I've been following for about a year now.

Like yourself I'm Dutch and based in the Netherlands. However, I have also lived in England and France, and I do not exclude the possibility to eventually settle in Portugal. I am married and have two small children. We are soon to set off again in our mobile home to travel through Europe like nomads as we do every summer.

I work as a researcher and consultant at Novay, the former Telematics Institute. Novay is an independent, non-profit institute for ICT-based innovation. We help organisations to innovate their products and services or the way they work by using ICT as an enabler. As a researcher and consultant I get the best of both worlds: I can both work together with researchers at universities to advance the state of the art, and I can directly work with enterprises to advise them on how to employ these novel ideas in their organisation.

In addition I try to be active in both the national and international MDD communities. At a national level I organise the annual Model Driven Experience symposium and I moderate the associated online community of practice. At an international level I am on the steering committee of the International IEEE Enterprise Computing Conference (EDOC) and have hosted this conference once in the past.

For more details, see:

Can you tell something about your research in the field of Model Driven Development?

I first encountered UML, OCL and the idea of model transformation when I was doing my PhD in England at the University of Kent at Canterbury. My own PhD research was about reconciling different formal specifications of distributed systems, which required a kind of horizontal transformation. In addition, I had the opportunity to discuss modelling issues with David Akehurst (developer of the Kent Modelling Framework and SiTra, a Java framework for Simple Transformations) and Stuart Kent (currently responsible for the Microsoft DSL tools). Mind you, this was still before the MDA initiative was launched by OMG!

Then when I first joined Novay in 1999, I worked mainly on methods, techniques and tools for conceptual modelling in various domains, such as networked enterprises, inter-organizational business processes, enterprise architecture and service-oriented architectures (SOA). So, I guess you could say that I have been researching some form of model-driven engineering (MDE) for over 10 years now.

The most concrete steps I have been able to make in de recently finished A-Muse project, where we developed a method, several DSLs and supporting tools for model-driven development of context-aware mobile services. Amongst many other lines of research, we test-drove the idea of platform-independence: is it really possible to generate PSMs and code for two different platforms - J2ME and .NET Compact Framework in our case - from a single PIM? The answer was: yes, but it ain't easy. As a side-effect we gained a lot of experience with QVT, the new model-to-model transformation standard, which we published as a kind of design patterns for model transformations. Another interesting experiment was to develop a graphical modeller and code generator for Google's Android platform using Eclipse EMF, GMF and xPand. Through this work, I got very excited about recent developments in Eclipse-based tooling for MDD (EMF, GMF, M2M, M2T). It's getting so much easier now to create the tools that you need.

My current research is focussing on applying MD-techniques in specific domains. In the VESTA project, for example, we are investigating how to model, analyse, simulate and visualise processes and scenarios for public order and safety. The idea is to use model transformations to link the various modelling, analysis and visualization tools used in the project.

What future plans does Novay have in the field of Model Driven Development?

Novay stands for networked innovation. We enable organisations to join forces and innovate together using ICT. The Model Driven Experience symposium is one example where we bring together organisations to exchange their experiences with MDD in order to inspire each other. And we intend to continue this highly successful symposium series in future years.

In addition, we are working on new projects where MD technology plays an enabling role. For example, we are currently investigating the feasibility of a project on combining business rules, business processes, services and objects in the development of enterprise systems.

What do you see as the most important research gaps in the field of Model Driven Development?

First of all, there is still a considerable gap between the state of the art and the state of practice in MDD. Practitioners rarely go beyond straightforward code generation from UML or DSL models. As a consequence these models are rarely platform-independent. MDD is mainly positioned as a method for increasing developer productivity, whereas it can be used also to get rid of the always problematic business - IT gap. Our efforts should be focussed on raising the level of abstraction at which models are made and used.

For me the most important research issues are related to this question: how to make MDE more practical? What should an MD development process look like? Which models should be made? In which order? How do we know they are consistent? How do we deal with changes? How do we manage and version models? And one issue I am particularly interested in myself is how to employ model transformation technology to relate and synchronize models from different viewpoints.

You recently organized the second Model Driven Experience conference, did you see emerging trends?

There was a lot of consensus on the advantages and pitfalls of MDD. Almost everyone agrees that MDD reduces development time and costs, increases software quality, increases maintainability and improves communication between stakeholders. On the other hand, it is generally acknowledged that MDD requires changes in culture, process, roles and skills, and that these changes cause resistance. Some also mention that MDD reduces flexibility and introduces new integration problems.

When we organised the symposium for the first time last year, most of the presentations were on experiments and small pilots with MDD. This year the presented initiatives seemed to be much more mature. Several Dutch companies, such as Sligro Food Group, ASR, Ordina, and Capgemini BAS, have a operational model-driven software factory, and others, such as ASML and Philips Healthcare, are using MDD in the development of critical software components.

Another noticeable development is that UML is hardly used anymore for modelling; almost all MDD initiatives now use some kind of DSL.

Do you have some advice for the community about how to spread the message of Model Driven Development?

One of the interesting conclusions of the discussion panel at the end of the Model Driven Experience 2009 was that most resistance against MDD comes from within the IT departments, not from the business. Business managers are easily convinced by the usual arguments of lower development and maintenance costs, better and more consistent software quality and reduced time-to-market. The software developers themselves however are often sceptic that their work can be automated. They also may feel that they loose control over the development process and the produced code.

Organisations that adopt MDD should address these fears. Switching to MDD requires a cultural change both at the business and the IT side. The business people should start feeling responsible for modelling the application domain and the required functionality. They also should learn how to work according to a structured development process. The IT people, on the other hand, should focus on generic solutions, reference architectures and the transformation to specific platform technology, and no longer feel responsible for the business analysis and application design. As with any cultural change, this won't happen overnight.

As a community we should collect and disseminate the increasing number of success stories to make people more aware of the advantages and implications of model-driven architecture and development.

8 reasons why Model-Driven Development is dangerous

Thu, 25 Jun 2009 22:44:00 +0200

Last week I gave a talk at the Hogeschool Arnhem Nijmegen as part of the conference "Information Systems the Next Generation". This article is inspired by a talk titled "Model Based Development - How to organize and architect survival of MD*" by Wiebe Wiersema given at the same conference. It was a well-balanced, realistic talk about the do's and don'ts of Model Driven Development.

While Model-Driven Development (MDD) is getting more and more attention by both tool vendors and developers, I think it's time to look at 8 reasons why MDD is dangerous. If you recognize one of these points from your practical experiences or if you think you have a solution for (almost) all points, please let me know it in the comments! Although model-driven approaches can differ a lot, I think these 8 points can be applied to almost all of them.

1. MDD actually introduces a lot of rigidity

If you're used to programming everything by hand, MDD can be quite rigid. The goal of MDD is to ‘program' on a higher level of abstraction. This means that you have to specify less and generate more. However, this also means that you can't change every little detail you want. It is, for example, often the case that generated graphical user interface are inflexible and they all look like each other.

2. Models are only flexible where flexibility has been designed

The problem with using models to directly drive the engineering of software is that they are far from flexible. First, you are limited by the kind of Model Driven Engineering tool you use. Second, you're only flexible in the parts of the solution covered by the used Domain-Specific Languages. The higher level the DSL the more commonalities are ‘hard-coded' in the framework or engine you use. Third, sometimes models are made flexible at predefined points by lower level expressions or languages. That's nice, but hopefully it is exactly done at the points you'll need it.

3. The roles of project members are quite different

You need to be aware of the fact that the roles in Model-Driven Engineering are different from the roles in traditional development. The real technical or programming part is mostly moved to the ‘meta-team' building the model-driven software factory. Building the solution is done by so-called business engineers instead of programmers. These people need to understand the business (like e.g. traditional functional designers) but they also need to express the knowledge they gather in a formal model. People able to do both these things are quite rare.

4. The modeling environment doesn't always support version control

If you have experience with building (enterprise) software applications I guess you're using a version control system. If not you should go and read this nice explanation. Existing version control systems work very well for textual programming languages. However, versioning of graphical models is a far less researched field. Most existing modeling and model-driven tools don't include a full featured versioning system. That's a pain when working with large teams.

5. The modeling tool/approach is "almost" finished at project start

The famous word "almost" is pitifully used a lot when talking about how far the model-driven software factory is finished at the start of a project. People building a model-driven software factory using DSL tools will have a lot of fun during the process. However, if the factory isn't finished and tested in practice before you are going to start a big project you will have a huge risk! If you're not scared by this article to use MDD, at least use a proven tool/approach!

6. The requirements team needs to understand what is allowed and what not

In point 1 and 2 I pointed at the rigidity and inflexibility of MDD. This is mostly dangerous from a technical perspective. However, it is even more dangerous when the requirements team, i.e. the people talking with the customer, don't understand the limitations of the used tool/approach. In the ideal situation, the people talking with the customer directly translate their knowledge in executable models. However, in big projects you often see a separation in two roles. As in traditional development this can lead to a gap. This gap can become a nasty problem if the requirements team doesn't understand the limitations of the tool/approach. For example, the requirements team reaches a consensus with the customer on a certain part of the graphical user interface (let's say a form), while it isn't possible to build that form with the model-driven tool without a lot of custom handiwork.

7. MDD is sold to the customer, but the team has no experience

While MDD is becoming more popular, customers start to ask for model-driven solutions. Actually I think MDD will become a hype (again!) very soon, and as with each hype everybody will start sticking the MDD label on their products. Doing a project in a model-driven way with an inexperienced team can lead to at least two problematic situations. First, the solution will not be build on time and it will not fulfill the customers' expectations due to most of the issues mentioned in the previous points. Second, big chance the end product isn't as easy to change as you expect from a solution built using a model-driven approach, due to misuse of the tool, not knowing its limitations, or due to using immature tools.

8. Innovation distracts

Last but not least, MDD is dangerous because innovation distracts. People tend to focus on the new tool they are using with all its cool features. Even more, people will start bragging about the great advantages they have with the new technology, in the end forgetting the projects objectives. Experiments have been done actually showing this behavior of project teams starting to use new innovative tools. The important lesson: don't forget your project management, your process, and all other things you need beside the tool or technical approach you use!

Photo by netwalker.

A Framework for Model-Driven SOA

Wed, 03 Jun 2009 17:36:00 +0200

A couple of weeks ago I gave an overview of two scientific publications researching a framework (or methodology) providing end-to-end guidance and assistance to refine and transform business models created by business experts into IT models and then to IT implementation. I tend to refer to these frameworks as Model-Driven SOA, because they focus on using models to construct an IT implementation complying to SOA principles.

However, I didn't elaborate on the practical applicability of these studies. I asked you for your opinion. From the comments on the article and the discussion in the LinkedIn MDA group triggered by this article, it became clear to me that there's a need to describe Model-Driven SOA in more detail before we can sufficiently compare different approaches and discuss the needed models, languages, and transformations.

In this article I present a generic framework describing Model-Driven SOA. I first describe the common terms in system development, like design, engineering, and architecture. After that we can apply these terms to a Service-Oriented Architecture (SOA) based development process. Finally, I want to define the requirements making this SOA system development process a Model-Driven Engineering approach.

System Development

System development is the bringing about of a new system or of changes to an existing system. The system development process comprises all activities that have to be performed in order to arrive at an implemented system. Such a development process can be described by the Generic System Development Process (GSDP) as introduced in [1] and recently described in more detail in [2]. Figure 1 exhibits the GSDP. At first sight it can looks a bit abstract, but give me a moment to explain it and I hope you'll see that it's in fact an easy and powerful way to explain the common terms in software development.

Figure 1 - The Generic System Development Process [2]

I have explained this process before in a post on the definition of architecture. So, for details I refer to that post. However, the process has slightly changed over the years as did my view on it. I will therefore briefly explain this generic system development process again.

The GSDP always concerns two systems, the object system (i.e. the system which is going to be developed) and the using system (i.e. the system that is going to use the functionality of the object system). If we, for example, want to build a human resource management (HRM) information system, this HRM information system is the object system. The using system is the HRM department. People are often referring to the using system as the problem domain, while they call the object system the solution domain.

While in a lot of cases no construction model of the using system exists, we need to reconstruct the higher level models from the implementation. This process is known as reverse engineering. Let's look again at our HRM system example. Before we can build an HRM information system supporting a certain HRM department we need to know what to support. We need to describe the using system and the only way to come to this description is to use reverse engineering. In case of the HRM department reverse engineering means talking with the people (the "implementation" of the using system) to learn how their work is performed.

The design process is split into function design and construction design. Function design starts from the construction of the using system and ends with a functional (black-box) model of the object system. This functional model describes the external behavior of the object system in terms of the using system, it doesn't talk about the construction of the object system. The functional model of an HRM information system can for example be describe with use cases.

Construction design starts with the functional model and ends with a construction (white-box) model of the object system. As shown in Figure 1 multiple construction models can exist on different abstraction levels.

The engineering of a system is the activity in which the highest level constructional model is converted into lower level models until the implementation model is constructed. Dietz defines the highest level constructional model as the ontology of a system, i.e. a model of the construction of a system that is completely independent of the way in which it is implemented [2]. This can be compared to the Platform Independent Model (PIM) in the Model Driven Architecture (MDA). In practice it is often the case that not all constructional models are defined, in most cases only the implementation model (the lowest level constructional model) is defined in a programming language (e.g. Java, C#, C++).

Implementing a system means assigning technological means to the constructional elements in the implementation model [2]. In case of an implementation model expressed in a programming language, the implementation is the compilation of that model to a specific platform. Once a system is implemented it can be put into operation.

The last part of the generic system development process is "architecture". Dietz uses the prescriptive notion of architecture, in contrast to the descriptive notion of architecture. The descriptive definition defines architecture as 'blue-prints', but the GSDP already refers to this by "constructional model" or "ontology". The prescriptive notion of architecture sees architecture as a consistent and coherent set of design principles. In the generic system development process, architecture is split into functional principles guiding the function design and constructional principles guiding the construction design.

Service-Oriented Software Development

It's time to tailor the generic system development process to a service-oriented development process. Lot's of SOA definitions exist, but I prefer to see SOA as an architectural style. Figure 2 shows the service-oriented development process, based on the generic system development process.

Figure 2 - The Service-Oriented Development Process

As shown in Figure 2 the using system is an organization (or part of an organization) which can be modeled with an organization model. Such a model can consist of process models or just textual descriptions. The organization model needs to be reverse engineered from the organization's implementation, which is in most cases a group of people and optionally already existing software.

The object system is an ICT system, to be designed and implemented to support the organization. The design, consisting of function and construction design, is guided by architecture principles. In case of a service-oriented development process the design is guided by SOA principles. A good overview of SOA principles is given by Thomas Erl in his book "Principles of Service Design" [3].

While the functional model of the software to build is defined with a set of services, the function design step consists of service identification and service specification. First the set of needed services is identified based on the organization model, next each service is specified in more detail (preferably using a standardized template). I'm just giving the outline of a generic service-oriented development process, so the details of question like "what exactly is a service?" and "how to specify a service?" are left for future ramblings.

The next step in the process is to take the functional specification consisting of service specifications and to design the construction of the needed ICT system which is going to provide these services. Following the Service Component Architecture (SCA) services can be grouped in components, which in their turn can be implemented with other components. Hence, service-oriented construction design consists of component identification. The highest level constructional model can for example be described by an SCA-based model.

The SCA also specifies how components can be implemented. Example component implementation possibilities are Java, .NET, and BPEL. So, the engineering step consists of specifying component implementations using Java, .NET, or BPEL, resulting in an implementation model consisting of artifacts described in these languages. This implementation model can be "implemented" on technology thereby resulting in the final software application, i.e. a Service-Oriented Business Application. Implementation of the implementation model means deploying all constructed artifacts to an SCA container.

Model-Driven Service-Oriented Software Development

Now what distinguishes SOA from Model-Driven SOA? I've already stated the term "model" a couple of times in the previous section and in Figure 2, but does that make the described service-oriented development process model driven? Model-driven engineering is based on formal models which are transformed into other models until the lowest level model can be executed with an engine or until code can be generated from that model (for more explanation read this introduction into model-driven software development). As we saw in the previous section the models are not necessarily formal and the implementation model is directly specified in a programming language. So, what are the requirements for Model-Driven SOA?

First, each model needs to be formal, i.e. it needs to be specified in a well-defined language. As we know from the previous the following models are needed:

Organization model - for example specified in BPMN.
Service model - mostly consisting of service specifications which are represented with a table listing some properties.
Component model - for example specified using the SCA assembly specification.
Component implementation model - each component can be specified with a set of connected Domain-Specific Languages (DSLs), meaning that a component implementation model can be considered as a multi-model.

Second, each model (except for the first model) is linked to the previous model, derived from the previous model, or a refinement of the previous model. The following steps are part of a Model-Driven SOA process:

Reverse engineering - an organization model is manually created.
Function design - a service model is derived from the organization model, whether this can be automated or not depends on how formal the organization model is. In most cases this is handiwork. Elements of the service model are linked to the organization model to enable tracing.
Construction design - a component model is derived from the service model. This is again a manual job.
Engineering - a component implementation model is a refinement of the component model. Implementation elements are specified in detail using DSLs.
Implementation - the component implementation model is implemented on technology, i.e. the models specified using DSLs are executed on high-level engines. The result is a Service-Oriented Business Application.

Multiple roles are involved in such a process using different tools. An important aspect for the success of a Model-Driven SOA approach is the ability to specify DSLs which are both high-level and directly executable. Constructing such DSLs can be quite a challenge, I gave an overview and 7 recommendations for DSL development in my previous post.

A second important aspect for Model-Driven SOA is the traceability of decisions throughout the model chain. It should be possible to trace a model element in the component implementation model back to the organization model and the other way around.

This also leads to the last important aspect for Model-Driven SOA I want to mention: change propagation. The organization model and service model are very useful for gaining a quick insight in the function of the system. The real power of these models will become available if a change in the organization model leads to an automatic impact analysis on the service model. The same should hold for changes in the service model and how they affect the organization model and component model.

If you look back at Figure 2, I think it is a good recommendation that models which are connected horizontally (organization model - service model - component model) do not automatically propagate changes, they should report inconsistencies and provide a change impact analysis. Vertically connected models (the models used in the engineering step) will need automatic change propagation to provide productivity improvements, otherwise you're half of the time busy with keeping the models consistent. In case of organization, service, and component models, you should be busy with that because in the consistency between these models lay most of the design decisions.

Conclusion

I hope this article gave you more insight in my ideas with respect to Model-Driven SOA. I have only described the framework including the needed models and transformations between these models. An important aspect to talk about in the future is the languages these models are specified in, i.e. the exact form of these models. Other aspects are needed tooling and technology (runtime engines). Moreover, every time we talk about such a model-driven approach, we shouldn't forget these 10 misperceptions and challenges of Model-Driven Development.

I'd like to hear your thoughts on this model-driven SOA framework! I'm also interested in your experiences with using a model-driven approach to build software supporting (part of) an organization. What are your important learned lessons? What challenges are you dealing with?

---------------------------
[1] J.L.G. Dietz and J.A.P. Hoogervorst. Enterprise Ontology and Enterprise Architecture - how to let them evolve into effective complementary notions. GEAO Journal of Enterprise Architecture, 1, 2007.

[2] J.L.G. Dietz. Architecture - Building strategy into design. Academic Service, The Hague, The Netherlands, 2008.

[3] Thomas Erl. Principles of Service Design. The Prentice Hall Service-Oriented Computing Series from Thomas Erl. Prentice Hall, Upper Saddle River, NJ, USA, 2007.

Photo by Ride My Pony

DSL development: 7 recommendations for Domain Specific Language design based on Domain-Driven Design

Wed, 06 May 2009 15:50:00 +0200

The term Domain-Specific Language (DSL) is heard a lot nowadays. A DSL is a language developed to address the need of a given domain. This domain can be a problem domain (e.g. insurance, healthcare, transportation) or a system aspect (e.g. data, presentation, business logic, workflow). The idea is to have a language with limited concepts which are all focused on a specific domain. This leads to higher level languages improving developer productivity and communication with domain experts. In a lot of cases it is even possible to let domain experts use the DSL and develop applications.

The question for this article is: how to develop a Domain-Specific Language?

I'll first explain the DSL lifecycle, consisting of the phases: decision, analysis, design, implementation, deployment, and maintenance. Afterwards I'll give 7 recommendations for DSL development based on my experiences with developing non-trivial DSLs.

The DSL lifecycle

According to Mernik et al. [1] the DSL life cycle consists of five development phases: decision, analysis, design, implementation and deployment. Eelco Visser [2] adds maintenance as the sixth phase in the lifecycle of DSLs. Note that in practice DSL Development isn't a sequential process, the phases should be applied iteratively.

Let's look at each phase in more detail.

1. Decision

The development of a DSL starts with the decision to develop a DSL, to reuse an existing one, or to use a GPL. If a domain is very fresh and little knowledge is available, it doesn't make sense to start developing a DSL. In order to determine the basic concepts of the field, first the regular software engineering process should be applied and a code base supported with libraries should be developed [2].

In other words: if you never have developed an application for a certain domain by hand and you have no existing code base, it isn't smart to start implementing a DSL and its associated code generators or execution engine.

The situation differs of course for non-executable DSLs. However, as you need experience with existing code for executable DSL, along the same lines you'll need a deep understanding of the domain you are modeling for non-executable DSLs.

2. Analysis

In the analysis phase the problem domain is identified and domain knowledge is gathered. The output of formal domain analysis is a domain model consisting of [1]:

a domain definition, defining the scope of the domain,
domain terminology (vocabulary, ontology),
descriptions of domain concepts, and
feature models describing the commonalities and variabilities of domain concepts and their interdependencies.

The information gathered in this phase can be used to develop the actual DSL. Variabilities indicate what elements should be specified in the DSL, while commonalities are used to define the execution engine or domain framework.

If you, for example, analyze a couple of existing code bases in a certain domain, you can split the elements of this code in two parts: the parts that differ and the parts that are the same for each code base. The static parts (the commonalities) can, depending on your implementation approach, be part of the execution engine interpreting the DSL or can be put in a domain framework which is used by the generated code. The parts that differ (the variabilities) should be specified in the DSL, these are the parts which a user of the DSL needs to ‘configure'.

Eelco Visser [2] recommends an inductive approach which, in opposite to designing the complete DSL before implementation, incrementally introduces abstractions that allow to capture a set of common programming patterns in software development for a particular domain. He also states that developing the DSL in iterations can mitigate the risk of failure. Instead of a big project that produces a functional DSL in the end, an iterative process produces useful DSLs for sub-domains early on.

In the second part of this article I will give 7 additional recommendations for the analysis and design phase of a DLS, based on my own experiences.

3. Design

Approaches to DSL design can be characterized along two orthogonal dimensions: the relationship between the DSL and existing languages, and the formal nature of the design description [1]. A DSL can be designed from scratch or it can be easier to base it on an existing language.

Mernik et al. [1] identify three different patterns of design based on existing languages:

piggyback: existing language is partially used,
specialization: existing language is restricted, and
extension: existing language is extended.

Besides the relation with existing languages the formal nature can range between:

informal: a DSL specified in natural language and/or with examples, and
formal: a DSL specified using one of the available semantic definition methods, e.g. regular expressions, grammars, etc.

It is important to decide what approach to take, however, it is maybe even more important to keep this lesson in mind [3]:

Lesson T2: You are almost never designing a programming language.
Most DSL designers come from language design backgrounds. There the admirable principles of orthogonality and economy of form are not necessarily well-applied to DSL design. Especially in catering to the pre-existing jargon and notations of the domain, one must be careful not to embellish or over-generalize the language.

Lesson T2 Corollary: Design only what is necessary. Learn to recognize your tendency to over-design.

4. Implementation

For executable DSLs the most suitable implementation approach should be chosen. Mernik et al. [1] identify seven different implementation patterns, all with different characteristics:

interpreter, DSL constructs are recognized and interpreted using a standard fetch-decode-execute cycle. With this pattern no transformation takes place, the model is directly executable.
compiler/application generator, DSL constructs are translated to base language constructs and library calls. People are mostly talking about code generation when pointing at this implementation pattern.
preprocessor, DSL constructs are translated to constructs in an existing language (the base language). Static analysis is limited to that done by the base language processor.
embedding, DSL constructs are embedded in an existing GPL (the host language) by defining new abstract data types and operators. A basic example are application libraries. This type of DSL is mostly called an internal DSL.
extensible compiler/interpreter, a GPL compiler/interpreter is extended with domain-specific optimization rules and/or domain-specific code generation. While interpreters are usually relatively easy to extend, extending compilers is hard unless they were designed with extension in mind.
commercial off-the-shelf, existing tools and/or notations are applied to a specific domain. You don't have to define your DSL, editor and DSL implementation approach yourself, you just make use of a Model Driven Software Factory. You can, for example, use the Mendix Model-Driven Enterprise Application Platform targeted at the domain of Service-Oriented Business Applications.
hybrid, a combination of the above approaches.

While the different approaches can make a big difference in the total effort to be invested in DSL development, the choice for a particular approach is very important.

5. Deployment

In the deployment phase the DSLs and the applications constructed with them are used. Developers and/or domain experts use the DSLs to specify models. These models are implemented with one of the implementation patterns presented in the previous section (e.g. the models are interpreted by an engine). Such an implementation results in working software which is used by end-users.

6. Maintenance

While domain experts themselves can understand, validate, and modify the software by adapting the models expressed in DSLs, modifications are easier to make and their impact is easier to understand. However, more substantial changes in the software may involve altering the DSL implementation. So, like any other element of software a DSL will evolve over time. Therefore having a DSL migration strategy is very important.

Besides migration strategies, I have two recommendations which alleviate the maintenance risks of DSLs:

use a good DSL tool, which at least generates editors and generators / interpreters from your language definition, and
implement models specified in different DSLs with different loosely coupled engines. In this way maintenance of a specific DSL or its compiler does not affect the whole system and the engines can make use of different technologies. In case of business applications, this calls for an architecture encouraging the development of Service-Oriented Business Applications, i.e. an application composed of multiple loosely-coupled services.

Seven Domain-Driven Design based recommendations for DSL Development

Now the lifecycle of a DSL is clear I want to share some of my experiences with the analysis and design phases of the DSL lifecycle. The other phases are left for a future article.

Before going into the details of DSL design let's try to understand the context of these experiences. First of all, they are focused on creating multiple connected DSLs, i.e. you can create models expressed with different DSLs referring to each other. For example, in a Form model you can refer to elements from your Data model. More specifically, we are talking about a set of DSLs covering all system aspects of a Service-Oriented Business Application.

Another important point in the DSLs we're talking about is that they are all aimed at non-programmer domain experts. For most cases this means domain experts can create models expressed in these DSLs, in a few cases this means they can at least read them. This of course always leads to finding a balance between flexibility and complexity.

Based on my experiences, influenced by the concepts of Domain-Driven Design [4], I have the following 7 recommendations for DSL development:

1. Capture domain knowledge in a metamodel

If you talk about models for DSLs you will stumble upon the term metamodel. For a lot of people this sounds scary enough to stop reading. However, it's just a model of the abstract structure of the language. In other words: a metamodel models the concepts of a language and their relationships. Just as you model the concepts ‘Order', ‘Product', and ‘Customer' if you are building software like an order entry portal.

A metamodel is essential for constructing a DSL. It captures the knowledge of the domain the DSL is aimed at. The model reflects how the team developing the DSL structures the domain knowledge and what they see as the most important elements. The binding of model and implementation ensures that the experiences with earlier versions of the DSL can be used as feedback in the modeling process.

2. Communicate using an ubiquitous language

The metamodel is also important for communication purposes. When designing a DSL a lot of communication is needed between the users of the language (domain experts) and the developers. The metamodel is the backbone of a language used by all team members. Because the model is bound to the implementation, developers can talk about the DSL in this language. They can communicate with domain experts without translation.

You should play with the model when talking about the DSL. If you can't talk in terms of the model about a scenario, the model should be adapted until you can. If the domain experts don't understand the model, there is something wrong with the model. Domain experts should object to terms or structures that are awkward or inadequate to convey domain understanding. Developers should watch for ambiguity or inconsistency that will trip up design.

3. Let the metamodel drive the implementation

Don't forget that a language definition is more than just a metamodel (abstract syntax). A language definition also contains a concrete syntax and semantics. When designing and implementing a DSL the concrete syntax is captured in the solution workbench, i.e. an environment in which you can specify models using the DSL with either a textual or a graphical concrete syntax. The semantics of the language are captured in the transformation rules or model interpreter (based on the used implementation pattern, see above).

It is important that the metamodel drives the implementation of the solution workbench and interpreter, i.e. the metamodel should driven the implementation of the DSL. If the implementation doesn't map to the metamodel, the metamodel is of little value. At the same time, complex mappings between metamodel and implementation are difficult to understand and in practice difficult to maintain as the design changes. A deadly gap between metamodel and implementation opens, so that insight gained in each of those activities does not feed into the other.

Therefore, design the metamodel in such a way that it reflects the implementation in a very literal way. However, demand at the same time that a single metamodel also serves the purpose of supporting the ubiquitous language. The implementation must become an expression of the metamodel, so a change to the code may be a change tot the metamodel and the other way around. To tie the DSL implementation and metamodel in such a way, usually requires DSL tools that let you generate big parts of the DSL implementation from the metamodel. Figure 1 exhibits such a scenario in which part of the solution workbench and interpreter are generated from the metamodel.

Figure 1 - Metamodel-driven DSL implementation

4. Isolate the domain

As said before, DSLs will evolve over time. In the previous recommendation we've seen that it is important to tie model and implementation, the model should drive the implementation. However, to do so you need to isolate the domain. If the domain code, representing the metamodel is diffused through the code it is very difficult to make changes to it. Changes in the GUI of your modeling environment or the infrastructure of your interpreter can actually change your domain code.

In principle the recommendations for ‘normal' software also hold for DSL implementations. Divide your code into layers and concentrate all the code related to the domain model in one layer which is isolated from GUI and infrastructure code. The domain objects should be free of the responsibility of displaying themselves, storing themselves, managing application tasks, etc. They should focus on expressing the domain model.

In the previous recommendation I stated that the model should drive the implementation, and I meant to do that as literally as possible. This is possible if you isolate the domain! Using the Generation Gap Pattern you can generate all the domain code while isolating it from your other code.

So, isolate the domain to ensure that the model can evolve to be rich enough to express the domain and to keep track of the changes in that domain.

5. Refactor continuously

Along the same lines you should refactor all the time. You should refactor while you're knowledge crunching. You should refactor while you're communicating using the metamodel. You should refactor while you're busy with implementing the DSL. You should refactor while you're generating code from you metamodel. To say it with Eric Evans [4], you should especially refactor if:

The design does not express the team's current understanding of the domain.
Important concepts are implicit in the design (and you see a way to make them explicit).
You see an opportunity to make some important part of the design suppler.

I think it doesn't need any explanation that such an approach needs close involvement of all team members including the domain experts.

6. Maintain metamodel integrity

To effectively abstract a complex domain with domain-specific models, you need more than one DSL. In complex projects multiple DSLs are usually necessary in order to cope with different concerns. In other words: multiple domain-specific models (DSMs), specified in different DSLs are needed to accurately abstract complex systems.

Total unification of the metamodel (remember: the metamodel describes the concepts of the DSL we are designing) for a large domain will not be feasible or cost-effective. The most important reason for this is that attempting to satisfy everyone with a single metamodel (and thus a single language) will lead to complex options that make the language difficult to use. This is the reason we are designing a DSL at all! Different domain experts will have a need for their own domain specific language to define their aspect of the system.

So, we need multiple domain specific languages, hence we also need multiple metamodels. However, the boundaries and relationships between different metamodels need to be marked consciously. Some recommendations on multi-DSL development:

Explicitly define the context for each metamodel, i.e. define the domain (e.g. system aspect) the DSL is designed for.
Continuously integrate the implementation of a metamodel and make the interfaces to other metamodels part of the automated tests.
Model the points of contact between the metamodels and use that model in your ubiquitous language. These points of contact define how models expressed in different DSLs can refer to each other. For example, a GUI element can refer to an element in the data model.
Think about your reference resolve strategy. If you use interpreters / engines to execute the models expressed in a DSL you can use late-binding, i.e. use soft references and resolve them at runtime. The advantage of this strategy is flexibility and adaptability. The approach usually used with code generation is early-binding, the references are explicitly reflected in the generated code. Performance can be a reason to follow this strategy.

7. Use a people-oriented approach

Executing a DSL implementation process, especially in a way as recommended in the previous points, is not easy. It requires an effective team of developers and domain experts. My last, and most important, recommendation is to use a people-first approach in DSL development. DSL development is highly creative an professional work. Developers need to make the technical decisions, they are the best people to decide how to conduct their technical work. Domain experts live the domain, hence they are best suited to decide on the applicability of the concepts of the language.

Although I strongly recommend the way of working reflected in the previous six points, the team has to decide on the process. Accepting a process requires commitment, and as such needs the active involvement of all the team.

Key take aways for DSL development

Capture domain knowledge in a metamodel
Communicate using an ubiquitous language
Let the metamodel drive the implementation
Isolate the domain
Refactor continuously
Maintain metamodel integrity
Use a people-oriented approach

----------------------

[1] Marjan Mernik, Jan Heering, and Anthony M. Sloane. When and how to develop domain-specific languages. ACM Comput. Surv., 37(4):316-344, 2005.

[2] Eelco Visser. WebDSL: A case study in domain-specic language engineering. In R. Lammel, J. Saraiva, and J. Visser, editors, Generative and Transformational Techniques in Software Engineering (GTTSE 2007), Lecture Notes in Computer Science. Springer, 2008.

[3] Wile, D. S. 2004. Lessons learned from real DSL experiments. Sci. Comput. Program. 51, 265-290.

[4] Eric Evans, Domain Driven Design: Tackling Complexity in the Heart of Software. Addison-Wesley, 2004.

Photos by Gail S and Hélio Costa

The Science of Model-Driven SOA

Tue, 14 Apr 2009 21:05:00 +0200

What do organizations need to be successful in good times and to survive in bad times? According to Richard Veryard what matters is how people and systems collaborate effectively to address emerging business opportunities and threats, in an agile and innovative manner. He calls this Organizational Intelligence.

I think for reaching Organizational Intelligence we need Model-Driven SOA. The point of Model-Driven SOA is to create technology (Service-Oriented Business Applications - SOBA) that truly supports an organization. We have to specify the business with declarative models, these models should directly lead to working software, either by direct execution (e.g. process engine, rule engine) or by transformations to executable models. Even if these transformations are manual, good tools support tracing between business requirements and implementation, thereby enabling methods to assure business-IT alignment.

However, to do Model-Driven SOA in practice we'll need some kind of framework or process describing what models and transformations we need. Although I have some ideas about this subject, let's first look at some scientific research in this field.

In the remainder of this post I present two recent (last year) scientific publications researching a framework or methodology providing end-to-end guidance and assistance to refine and transform business models created by business experts into IT models and then to IT implementation:

the Business to IT transformation Framework [1], and
the Model Driven Service Engineering Process [3].

Business to IT Transformation Framework

Stein, Kühne, and Ivanov [1] have conducted an extensive literature research in the field of model-driven business to IT transformations. They conclude that the reviewed works only cover a subset of the different criteria and present a new framework providing the building blocks of business to IT transformations from a practical point of view.

Mens and Van Gorp [2] distinguish between horizontal and vertical transformations. A horizontal transformation is a transformation where the source and target models reside at the same abstraction level. A vertical transformation is a transformation where the source and target models reside at different abstraction levels. According to Stein, Kühne, and Ivanov [1] most approaches for business to IT transformations follow a horizontal transformation strategy. A horizontal transformation starting with an abstract business process model results in an abstract orchestration model requiring significant refinement efforts to make it executable. An alternative horizontal transformation strategy is to start from a business process model augmented with technical details. However, this forces the business analyst to think in terms of executable business processes.

Due to the previous arguments, Stein, Kühne, and Ivanov [1] formulate the following axiom, which forms the basic assumption for their business to IT transformation framework:

Business process models (e.g. BPMN, EPC) must be platform independent and a platform specific IT implementation (e.g. BPEL) must be derived through a vertical transformation strategy.

Consecutively they come to the following consequences and requirements for their framework:

The business process model shall not contain any platform specific details, e. g. references to WSDL artefacts.
The business process model should use refineable proxy elements instead.
Source models should be restricted to a subset, which can be unambiguously transformed.
Full code generation is desirable but not achievable.
Target models should be comprehensible for human users.
The framework should use iterative development processes through change detection, change visualisation, merge functionalities.

Figure 1 - Business to IT transformation framework

Figure 1 exhibits the general business to IT transformation framework [1]. The framework shows three different model perspectives: process, data, and interaction perspective. These perspectives reduce cognitive complexity at design time [1]. The framework also presents two abstraction levels, the business level and the IT level, which are related with vertical transformations.

Model Driven Service Engineering Process

Anaby-Tavor et al. [3] define a model driven service engineering process aimed at providing a methodology giving end-to-end guidance and assistance to refine and transform business models created by business experts into IT models and then to IT implementation.

A service engineering process is a sequence of activities: analysis, strategy design, development, and deployment, done in this order to produce a services system [3].

Figure 2 - The Model Driven Service Engineering Process

Figure 2 presents the methodology in three layers. The upper layer shows the service engineering process, the middle layer describes the models supporting each phase of the service engineering process, and the bottom layer describes the aspects of the services system that the models affect. Anaby-Tavor et al. follow the definition of a services system given by Zang et al. [4] which states that a services system is a 6-tuple: <Inputs, Outputs, Goals, Transformation, Components, Sensors>.

In the analysis phase the system requirements are analyzed and modeled. In the next phase a strategy design is conducted to describe the static and dynamic aspects of the services system. Static enterprise models define the components' black box view. The components collaborate using services, this forms the first step of the dynamic modeling of the enterprise. During the development phase detailed executable data is added leading to execution models describing the control flow of the activities of the services. Finally, sensors use IT management models to oversee the deployment of the services system.

This time I don't want to give my opinion on these studies, I'm curious what your experiences are with translating organization models into software?

Do you use a framework? If so, what framework?

I'd like to hear your thoughts in the comments.

---------------------

[1] Sebastian Stein, Stefan Kühne, and Konstantin Ivanov. Business to IT Transformations Revisited. In Cesare Pautasso and Jana Koehler, editors, 1st International Workshop on Model-Driven Engineering for Business Process Management, pages 1-12, Milano, Italy, September 2008.

[2] Tom Mens and Pieter Van Gorp. A taxonomy of model transformation. Electr. Notes Theor. Comput. Sci., 152:125-142, 2006.

[3] A. Anaby-Tavor, D. Amid, A. Sela, A. Fisher, Kuo Zhang, and Ou Tie Jun. Towards a Model Driven Service Engineering Process. Congress on Services - Part I, 2008. SERVICES '08. IEEE, pages 503-510, July 2008.

[4] L Zhang, J Zhang, and H Cai. Services Computing: Core Enabling Technology of the Modern Services Industry. Tsinghua University Press, 2007.

5 types of Model Driven Software Development

Tue, 31 Mar 2009 21:52:00 +0200

Model Driven Software Development is getting momentum. Is Model-Driven the future of software development? Or is it the same old wine in a new bottle? I'm not going to answer this question right now. I'll first show you the different types of model driven software development using a simple metaphor: farming.

Programming - the manual work

Doing all the farming by hand, it's a craft, an art. You love it, or you don't. Everything will become as you want it to be, on your time.

In software development programming can be seen as manually creating a low level model of an application. That low level model (implementation model) is compiled to an executable or interpreted on a virtual machine.

As with tools for farming you have tools and frameworks making the job easier, but it stays handiwork. I won't define programming as model driven software development. I just mention it to sketch the starting point.

Code generation - automation

Nowadays lots of work on a farm can be automated. You have to learn how to use the machines doing the work for you, but once you have, all manual task are ‘generated' automatically.

In software development the manual work can also be automated. Instead of writing code you can create models. These models are transformed into source code. You, of course, have to learn how to model, but once you have, all the manual work of writing code is automated. This means a whole shift in application development roles.

An early, well-known example of code generation is the generation of corresponding source code for graphical user interfaces in Integrated Development Environments. Nowadays a lot of tools exist, generating full applications based on templates.

MDA - abstraction

In the previous approach we needed a specialized machine for each task. That involves a lot of learning about how to configure and operate those machines. In case of milking we have to attach the machine to each cow by hand. Can't this be automated by stepping one abstraction level higher? We just model how to attach the machine and the next part of our job is automated!

In a very basic way that's what Model Driven Architecture (MDA) is for software development. Instead of just generating code from a model, you can construct platform independent and platform dependent models. The platform independent model is transformed into a platform dependent model using model transformations. From the platform dependent model the source code is generated.

If you specify the needed functionality in a platform independent model, you can transform this functionality into models for different platforms. So, the MDA is mainly aimed at technical variability.

DSL - specializing

Farms have changed a lot over time. Started as a means for farmers to grow their own food, the focus has now changed to financial gain. That, in combination with, among others, the industrialization and growing complexity of a farm led to more specialized farms. This means that consumer products are mostly composed of materials from different farms. Examples of specialized farms are dairy farms, orchards, vineyards, stud farms, plantations, poultry farms, etc.

In software development a same movement can be seen. A lot of people argue that the UML, the language used to define the models in the MDA, is too complex. Instead of using UML the use of Domain-Specific Languages (DSLs) is promoted for model driven software development. DSLs are small languages tailored to a specific domain (a system aspect or a certain problem domain). An application isn't modeled with a single, monolithic model specified in one language, but with multiple small models specified with different DSLs. DSLs can vary in structure, variability, and notation. Most DSL approaches contain a direct translation between DSLs and their execution by using code generation or the direct interpretation of the DSLs.

MDE - from problem to solution

We can talk a lot about farming and how to automate and specialize it more, however, the real leap in productivity is in alignment. The production of raw materials need to be aligned with consumer needs. We see this happen with big companies targeting consumer needs (or manipulating them ;) ) with advertising campaigns. All production is aligned to deliver consumer products composed of multiple materials maybe coming from different farms. The focus is not on the farm and how to improve it, the focus has become consumer-centric.

In model driven software development the same can be done. In the approaches mentioned before the target was to model the application at a higher abstraction level than a programming language. We can, however, go one step further. We can also create models of the using system, i.e. creating models of the organization which is going to use the software. Once we have modeled the so-called problem domain, we can make translations to a model of the solution domain. These translations or transformations don't necessarily have to be automated, in some cases that's not even possible. However, they need to be supported by appropriate tools and methodologies.

This Model-Driven Engineering approach differs from other approaches because we can distinct between horizontal and vertical transformations. Horizontal transformations from problem to solution domain, and vertical transformations from solution domain models to executable models. We've seen in previous articles on Model-Driven SOA that such approaches are possible. However, the challenges of MDE are in appropriate tooling, methodologies, patterns, templates, reference architectures, and industry standards.

In short: MDE focuses on the problem domain, it is (or should be) business-centric.

A lot more can be said about the different approaches and how they compare to each other (and yes, I think I'll do that in the future), but for now you have a quick overview of these 5 approaches. What are your experiences with these approaches? What approach do you prefer and why?

Photos by cresk, Airborne Guy, Flint-Hill, DocBuddie, Farl.

Microblog: news and articles on Model Driven Engineering

Mon, 23 Mar 2009 21:26:00 +0200

I'm happy to announce my new microblog dedicated to news and articles on Model Driven Engineering. Model driven software development approaches are getting more and more attention, resulting in more web content. I have also contributed to this movement with a considerable amount of articles on Model Driven Architecture (MDA), Model-Driven Engineering (MDE), Domain-Specific Languages (DSL), Model Driven tooling, and Model-Driven SOA, to name a few topics.

A couple of times I've pointed at interesting articles or react on them, for example last year's discussion on UML vs. DSL or my more recent reaction on the challenges of MDD. Despite the fact that I like the dialog, I don't always have time to write a decent reaction. However, just mentioning interesting links, news, and facts, with a maximum of 140 characters doesn't take that much time. That's why I've started a microblog / twitter feed.

I've planned to do daily updates on this microblog dedicated to the latest news and articles on Model Driven Engineering. These updates will mostly consist of interesting links related to model driven software development.

Why not just posting on this blog? Because I want to keep this blog dedicated to in-depth, original content, posted with a lower frequency.

I'd like to hear what you think about this. If you stumble upon interesting news or articles about Model Driven Software Development, please let me now!

Oh... and don't forget to subscribe to your new source of model driven information ;)

What every architect should now know about the Service Component Architecture (SCA)

Wed, 11 Mar 2009 11:52:00 +0200

The definition of ‘application' is rapidly changing. As the industry is moving from using an application-centric architecture to a Service Oriented Architecture (SOA), the focus for building functionality is moving to Service Oriented Business Applications (SOBA). This means that applications are a set of services and components working together in fulfilling a certain business need. The technology, specifications, and standards for specifying these components and services may vary, and most often religious discussions are going on about the usefulness of each of them.

As regular readers may guess, my take would be to create these components and services in a model driven way using an approach to Model Driven SOA. However, even in that case you'll need some kind of programming model that defines interfaces, implementations, and references. When building an application or solution based on services, these services can be both created specifically for the application and reused from existing systems and applications. Hence, we need a programming model that specifies how to create and implement services, how to reuse services, and how to assemble or compose services into solutions.

That's exactly what the Service Component Architecture (SCA) is.

Structure of this article:

Introducing the Service Component Architecture (SCA): the who and why behind SCA.
Service Component Architecture specifications: explaining the four main elements of the SCA.
Service Component Architecture concepts: explaining the concepts (e.g. composite, component, service, wire, binding, reference, etc.) of the SCA in detail.
Deployment and the Service Component Architecture: contributions, clouds, and nodes.
Implementations of the Service Component Architecture: an overview of existing SCA implementations (open source and commercial).

Introducing the Service Component Architecture (SCA)

The SCA models the "A" in SOA. It provides a model for service construction, service assembly, and deployment. The SCA supports components defined in multiple languages, deployed in multiple container technologies, and with multiple service access methods.

The SCA consists of a set of specifications that define a model for building Service-Oriented Business Applications (SOBA). These specifications are based on the following design principles:

Independent of programming language or underlying implementation.
Independent of host container technology.
Loose coupling between components.
Metadata vendor-independence.
Security, transaction and reliability modeled as policies.
Recursive composition.

The first draft of SCA was published in November 2005 by the Open SOA Collaboration (OSOA). OSOA is a consortium of vendors and companies interest in supporting or building a SOA. Among them are IBM, Oracle, SAP, Sun, and Tibco. In March 2007 the 1.0 final set of specifications was released. In July 2007 the specifications were adopted by OASIS and they are developed further in the OASIS Open Composite Services Architecture (Open CSA) member section.

Service Component Architecture specifications

The set of specification can be split into four main elements: the assembly model specification, component implementation specifications, binding specifications, and the policy framework specification. I will shortly explain each of them in this section. In the next section I'm going to explain the concepts (e.g. what is a composite, what is a wire, etc.) of the SCA in more detail.

Assembly model specification, this model defines how to specify the structure of a composite application. It defines what services are assembled into a SOBA and with which components these services are implemented. Each component can be a composite itself and be implemented by combining the services of one of more other components (i.e. recursive composition). So, in short, the assembly model specifies the components, the interfaces (services) for each component, and the references between components, in a unified, language independent-way. I'll explain these concepts in detail in the next section. The assembly model is defined with XML files.

SCA Assembly Model V1.00 (PDF)
SCA Core XML Schema V1.0 (XSD file)

Component implementation specifications, these specifications define how a component is actually written in a particular programming language. Components are the main building blocks for an application build using SCA. A component is a running instance of an implementation that has been appropriately configured. The implementation provides the actual function of the component and can be defined with a Java class, BPEL process, Spring bean, and C++ or C code. Several containers implementing the SCA standard (meaning that they can run SCA components) support additional implementation types like .Net, OSGI bundles, etc. In theory a component can be implemented with any technology, as long as it relies on a common set of abstractions, e.g. services, references, properties, and bindings. I'll explain these abstractions in detail in the next section.

The following component implementation technologies are currently described:

SCA Java Component Implementation V1.00 (PDF), XSD file
SCA Spring Component Implementation V1.00 (PDF)
SCA BPEL Client and Implementation V1.00 (PDF)
SCA C++ Client and Implementation V1.00 (PDF), XSD file
SCA COBOL Client and Implementation V1.00 (PDF)
SCA C Client and Implementation V1.00 (PDF)

Binding specifications, these specifications define how the services published by a component can be accessed. Binding types can be configured for both external systems and internal wires between components. The current binding types described by OSOA are bindings using SOAP (web services binding), JMS, EJB, and JCA. Several containers implementing the SCA standard support additional binding types like RMI, Atom, JSON, etc. An SCA runtime should at least implement the SCA service and web service binding types. The SCA service binding type should only be used for communication between composites and components within an SCA domain. The way in which this binding type is implemented is not defined by the SCA specification and it can be implemented in different ways by different SCA runtimes. It is not intended to be interoperable. For interoperability the standardized binding types like web services have to be used.

Officially described binding types:

SCA Web Services Binding V1.00 (PDF), XSD file
SCA JMS Binding V1.00 (PDF), XSD file
SCA EJB Session Bean Binding V1.00 (PDF), XSD file
SCA JCA Binding V1.00 (PDF)

Policy framework specification, describing how to add non-functional requirements to services. Two kinds of policies exist: interaction and implementation policies. Interaction policies affect the contract between a service requestor and a service provider. Examples of such policies are message protection, authentication, and reliable messaging. Implementation policies affect the contract between a component and its container. Examples of such policies are authorization and transaction strategies.

Service Component Architecture concepts

In the previous section we've seen the different SCA specifications and what they describe. Let's now look in more detail at the concepts of the SCA.

As stated before the main building block of a Service-Oriented Business Application is the component. Figure 1 exhibits the elements of a component. A component consists of a configured piece of implementation providing some business function. An implementation can be specified in any technology, including other SCA composites. The business function of a component is published as a service. The implementation can have dependencies on the services of other components, we call these dependencies references. Implementations can have properties which are set by the component (i.e. they are set in the XML configuration of a component).

Figure 1 - SCA Component diagram

Components can be combined into assemblies, thereby forming a business solution. The SCA calls these assemblies composites. As shown in figure 2 a composite consists of multiple components connected by wires. Wires connect a reference and a service and specify a binding type for this connection. Services of components can be promoted, i.e. they can be defined as a service of the composite. The same holds for references. So, in principle a composite is a component implemented with other components and wires. As stated before, components can in their turn be implemented with composites thereby providing a way for a hierarchical construction of a business solution, where high-level services (often indicated as composite services) are implemented internally by sets of lower-level services.

Figure 2 - SCA Composite diagram

The service (or interface if you like) of a component can be specified with a Java interface or a WSDL PortType. Such a service description specifies what operations can called on the composite, including their parameters and return types. For each service the method of access can be defined. As seen before, the SCA calls this a binding type.

Figure 3 - Example XML structure defining a composite

Figure 3 exhibits an example XML structure defining an SCA composite. It's not completely filled in, but it gives a rough idea what the configuration looks like. The implementation tag of a component can be configured based on the chosen implementation technology, e.g. Java, BPEL, etc. In case of Java the implementation tag defines the Java class implementing the component.

Deployment and the Service Component Architecture

SCA composites are deployed within an SCA domain. An SCA Domain (as shown in Figure 4) represents a set of services providing an area of business functionality that is controlled by a single organization. A single SCA domain defines the boundary of visibility for all SCA mechanisms. For example, SCA service bindings (recall the earlier explained SCA binding types) do only work within a single SCA domain. Connections to services outside the domain must use the standardized binding types like webservice technology. The SCA policy definitions do also only work within the context of a single domain. In general, external clients of a service that is developed and deployed using SCA should not be able to tell that SCA was used to implement the service, it is an implementation detail.

Figure 4 - SCA Domain diagram

An SCA domain is usually configured using XML files. However, an SCA runtime may also allow the dynamic modification of the configuration at runtime.

An SCA domain may require a large number of different artifacts in order to work. In general, these artifacts consists of the XML configuration of the composites, components, wires, services, and references. We of course also need the implementations of the components specified in all kinds of technologies (e.g. Java, C++, BPEL, etc.). To bundle these artifacts the SCA defines an interoperable packaging format for contributions (ZIP). SCA runtimes may also support other packaging formats like JAR, EAR, WAR, OSGi bundles, etc. Each contribution at least complies to the following characteristics:

It must be possible to present the artifacts of the packaging to SCA as a hierarchy of resources based off of a single root.
A directory resource should exist at the root of the hierarchy named META-INF.
A document should exist directly under the META-INF directory named ‘sca-contribution.xml' which lists the SCA Composites within the contribution that are runnable.

A goal of the SCA approach to deployment is that the contents of a contribution should not need to be modified in order to install and use the contents of the contribution in a domain.

An SCA domain can be distributed over a series of interconnected runtime nodes, i.e. it is possible to define a distributed SCA domain. This enables the creation of a cloud consisting of different nodes each running a contribution within the same SCA domain. Nodes can run on separate physical systems. Composites running on nodes can dynamically connect to other composites based on the composite name instead of its location (no matter in which node the other composite lives), because they all run in the same domain. This also allows for dynamic scaling, load balancing, etc. Figure 5 shows an example distributed SCA domain, running on three different nodes. In reality you don't need different nodes (and even different components) for this example, but it makes the idea clear.

Figure 5 - A distributed SCA domain running on multiple nodes

Implementations of the Service Component Architecture

The SCA is broadly supported with both commercial and open source implementations. To give you an idea I'll list a few them below.

Open source implementations:

Apache Tuscany (official reference implementation). The current version is 1.4, released in January 2009. They are also working on a 2.0 version which they aim to run in an OSGi enabled environment.
The Newton Project.

Commercial implementations:

Do you have experience with one of these products? Do you use SCA in practice?

------------
Figure 1, 2, and 4 are taken from the SCA Assembly Model specification.
Figure 5 is taken from the Tuscany documentation.

Model Driven Engineering tools compared on user activities

Wed, 18 Feb 2009 21:48:00 +0200

In my previous post I gave you a quick overview of the roles involved in Model Driven Engineering. The question for today is how to support these roles with appropriate tooling?

Tools for Model Driven Engineering are often seen as a single category of tools. I'll argue in this article that they can be separated in two categories: DSL (Domain Specific Language) tools and Model Driven Software Factories. Both types of tools have very different user activities. However, before going into details, let's first explain the basics: what is a language workbench and what workbenches do we actually need in MDE?

What is a Language workbench?

Martin Fowler introduced the term Language Workbench a couple of years ago to refer to tooling supporting language-oriented programming. I see language-oriented programming as an important aspect of Model Driven Engineering, i.e. the meta level part of MDE. Martin Fowler defines a language workbench by naming the essential characteristics as follows:

Users can freely define new languages which are fully integrated with each other.
The primary source of information is a persistent abstract representation.
Language designers define a Domain-Specific Language (DSL) in three main parts: schema, editor(s), and generator(s).
Language users manipulate a DSL through a projectional editor.
A language workbench can persist incomplete or contradictory information in its abstract representation.

If we compare these characteristics with the roles involved in MDE, we see that all roles need to use this language workbench for their own part of the job.

Needed workbenches in Model-Driven Engineering

Let's take a step back. A language workbench as defined by Martin Fowler in principle consists of a couple of workbenches. So, what workbenches are needed in MDE? Let's start with the MDE overview picture from my previous post (see Figure 1).

Figure 1 - Overview of Model Driven Engineering

In a model driven development process the dark grey artifacts exhibited in Figure 1 need to be specified somehow. Hence, we need at least three different workbenches. A domain expert and a language engineer together define a DSL in a DSL workbench with use of a meta language. A transformation specialist and an implementation expert together define the transformation rules in a transformation workbench with use of a transformation language. They define how models expressed in the defined DSL are executed. The last workbench is the solution workbench in which the business engineer and solution architect model an application with use of the defined DSL. An overview of the different workbenches and their users is given in Figure 2.

In reality an model driven software development process isn't that straightforward. DSLs are meant to be domain-specific, i.e. they just model a system aspect. Hence, we need multiple DSLs to describe a software solution. That's why we also need the role of software factory architect (or method engineer). Someone needs to define what models and DSLs are needed in a development process and how they are connected. In principle this role defines what workbenches are combined in a single MDE tool.

Figure 2 - Overview of workbenches in MDE

Model Driven Engineering Tools

Looking at the current market, we can distinguish between two main approaches in MDE tooling: DSL tools and Model Driven Software Factories.

Let's compare them on the following points:

Workbenches: what workbenches are included in the tool.
Input: what is the input of the tool.
Output: what is the output of the tool.
Tool vendor activities: what does a tool vendor need to specify /build in order to create the tool.
Tool user activities: what does the user of the tool need to specify in order to produce the output. This is of course related to the input, but it also includes the activities not guided by the tool.
Examples: existing tools in this category (just a few examples, not an exhaustive list).

DSL tool:

Workbenches: DSL workbench and transformation workbench.
Input: DSL definitions and transformation rules.
Output: solution workbench and generator.
Tool vendor activities: meta language definition, transformation language definition, workbench implementations.
Tool user activities: DSL definitions, transformation rules, architecture definition, (domain) framework implementation.
Examples: openArchitectureWare, MetaEdit+, Microsoft DSL Tools, JetBrains Meta Programming System.

Model Driven Software Factory:

Workbenches: solution workbench.
Input: functional specification.
Output: working application.
Tool vendor activities: DSL definitions, transformation rules, architecture definition, domain framework implementation, solution workbench implementation.
Tool user activities: application modeling.
Examples: Mendix (domain: Service Oriented Business Applications).

Depending on how well the DSL tool supports the definition of multiple DSLs (referring to each other, change propagation, etc.) and transformations, you can say that a Model Driven Software Factory can be the output of a DSL tool.

DSL tool or Model Driven Software Factory?

What tool you'll need for your project depends on your specific wishes. A while ago Steven Kelly wrote an article on this subject from a financial perspective (in reaction on this post of me). He concludes with: "Building a DSL for your own use is a lot easier and cheaper, and gives greater benefits". I don't agree with this conclusion.

Building your own DSLs, i.e. using DSL tools, gives you all the flexibility you'll ever need. However, it also comes with a cost: DSL design isn't that easy. Designing a full set of DSLs for modeling all application aspects will cost a lot of effort (the DSLs will evolve along with the applications you build with them). And don't forget the training, language support, standardization, and maintenance of your languages.

A Model Driven Software Factory, on the other hand, is only useful if it precisely targets the domain you're searching a solution for. You also need to commit you to the vendor of the factory. However, the domain of a Model Driven Software Factory can be quite broad and with current business process engines, workflow engines, and business rules engines, the used languages can be both applicable to multiple problem domains and easy to understand for business engineers. With a Model Driven Software Factory you can directly start using the DSLs for modeling your application and you don't need to have the expertise in-house to define the languages, transformations, and generators yourself.

What type of tool you need depends on your situation. My advice: read and understand the characteristics of the different types of tools and make a choice for yourself.

What is your preferred choice?

Roles in Model Driven Engineering

Wed, 04 Feb 2009 12:45:00 +0200

Model-Driven Engineering, or Model-Driven Development if you like, asks for other roles than we are used to in traditional software development. I think the most important change is the addition of a meta level. Instead of translating business wishes into source code, a software engineer has to define the Domain-Specific Language (DSL) and how to transform a Domain-Specific Model (DSM) specified in that DSL into working software.

Figure 1 exhibits an overview of the artifacts involved in Model Driven Engineering (MDE). A functional specification is translated into application code by a generator. This application code uses a (domain) framework to execute. Both the application code and the framework conform to some architecture, for example a Service-Oriented Architecture (SOA). The models of the functional specification are specified in a DSL, which in its turn is specified in some meta language. The generator is configured with transformation rules, specified in a transformation language.

The question for this article is: what roles do we distinguish in MDE and how are they different from traditional software development?

Figure 1 - Overview of Model Driven Engineering

Meta level

The most important change in software development roles is the addition of a meta level. On this meta level we distinguish the following roles: domain expert, language engineer, transformation specialis, implementation / platform expert, and software factory architect.

Domain expert

A domain expert is involved in defining a DSL. The goal of a DSL is to be domain-specific, so it should capture domain knowledge. If multiple different DSLs are used for modeling an application multiple domain experts are involved in the DSL design process. Although a domain expert is necessary for constructing a usable DSL, we need another role to specify the DSL in a formal meta language: the language engineer.

Language engineer

The language engineer uses a meta language to specify the concrete syntax and abstract syntax of a DSL. For more information on metamodels see one of my previous article on defining DSLs with metamodels. Although DSL design can become quite complex, with appropriate tooling you don't have to be an experienced language engineer to define a DSL. See for example openArchitectureWare and/or MetaEdit+.

Transformation specialist

Once the DSLs are defined, we need to define how models defined in these DSLs are executed or transformed into an executable model (called ‘application code' in Figure 1). This can be done with so-called model transformation. A model transformation is configured using transformation rules, which are specified in a model transformation language (like QVT defined by the OMG). We call the role defining these transformation rules a transformation specialist.

Implementation / platform expert

While a transformation specialist knows how to use a transformation language and how to transform models into other model, an implementation (or platform) expert knows everything about executing / interpreting a model. A model is executed based on its semantics. While a model only contains the variable elements of an application (the business logic), a framework is needed for the static elements (the infrastructure). A platform expert defines and implements such a framework and ensures that it conforms to a given architecture.

Software factory architect

We've seen four roles involved with parts of the MDE process. I want to add one important role: the software factory architect. The situation shown in Figure 1 is a simplified version of an MDE process. In practice you'll have a lot of models specified in different DSL, together leading to a working software application. A software factory architect defines what models are needed, how they relate, in what order they have to be produced, and how changes are propagated. In short: he defines the methodology, hence we can also call him a ‘method engineer'. A software factory architect also defines the architecture the resulting applications should conform to. See the place of architecture in Model Driven Engineering for more information.

Application Design

Besides the new meta level roles, the traditional roles in application development also change. Application design is still a challenging task. Complex real-world problems have to be expressed in a formal model. The good thing is that this model can be at a much higher level. You don't have to be a programmer to read and write such a model. On the application level we can distinguish the following roles: business engineer, application / solution architect, test engineer.

Business engineer

Business engineer is a changed role in application development. The one translating a business problem into a formal application model. This role needs both an understanding of the problem domain and skills to express that understanding in a formal model. Because this model needs to be specified in a DSL a business engineer differs from a programmer.

Application / solution architect

Although an MDE software factory can guide business engineers a lot in how to specify an application model, they still have to decide on the application architecture. How to split functionality in components and services? That's not a trivial task. See for example the architecture requirements for Service-Oriented Business Applications. I think the role of solution architect will stay important. However, note the difference with traditional software engineering: the technical architecture is defined on the meta level. The transformations guide how models are transformed to or interpreted by technology.

Test engineer

The testing role will change a bit. Technical testing is performed at the meta level. So, test engineers can focus on functional testing. Perhaps part of this work can also be transferred to the meta level with use of model-based testing, but I think it is not possible to fully automate all the testing (automation itself is also error-prone and adds complexity).

Conclusion

We've seen the roles involved in model driven software development. Different roles doesn't mean different people in every situation. One person can fulfill multiple roles. However, we see a substantial difference with the roles in ‘traditional' software development. In practice this lead to changes in the roles people have in their job.

Do you recognize these roles? What are your experiences with people having to change to one of these roles?

--------------
Figure 1 is created by me. Other pictures by (in order of appearance): HikingArtist.com, aikwhye, jasewiththeface.