The Enterprise Architect

18 lessons learned during the 6 year roller-coaster ride called Mendix

Sat, 14 Apr 2012 20:29:00 +0200

Last week I gave a talk about the lessons learned during the exponential growth of our company the last six years. I was invited to to talk at the Appsterdam meetup in Delft about "growing pains". In other words: what lessons did we learn when our company grew from 3 to 100 employees. I enjoyed the preparations as much as the actual talk, as it forced me to step back and think about the things we did and did not last years.

I covered lessons learned in four categories: company, culture, product development, and product evolution. Attempting to cover all these subject automatically means that you stay at a high level. However, I think I was able to share some valuable experiences from the trenches. It will be interesting to dive deeper in the separate categories in the future!

The slides:

App Delivery Platform-as-a-Service - How we revolutionized the app development market

View more presentations from Johan den Haan.

Do you recognize these lessons? What are your most important lessons learned? If you have questions, don't hesitate to ask them in the comments!

Will App Delivery PaaS make cloud more relevant to the business?

Fri, 16 Mar 2012 18:35:00 +0200

I finished my last blog post by introducing a Platform-as-a-Service subcategory called "Application Delivery Platform-as-a-Service" as a way to distinguish platforms that focus on improving the entire application delivery lifecycle (and not just application development or deployment). I would like to clarify my views on Application Delivery and PaaS a bit more. My first attempt has been published on InfoQ yesterday.

The short summary: business agility is key, so focus on improving the app delivery lifecycle for maximum business value and cloud will fit in naturally (as well as agile software development, Model-Driven Development, and social collaboration).

The Need to Focus on App Delivery Lifecycle in PaaS

Cloud is the next best thing since sliced bread, isn't it? It increases responsiveness, gives a business agility, and it enables new business models. Well, it seems not. Nearly three-quarters (74 per cent) of UK IT managers think that cloud computing has no relevance to their business. Is it fear? Are security and privacy concerns the main barriers for cloud adoption as stems from most related surveys? Or should we ask the CEO instead of the CIO?

Apparently there is a mismatch between expectations (or promises) around cloud and how a large group of people perceive it. My conclusion? We should just stop talking about cloud. It doesn't help your business, cloud is irrelevant! Let's explore why.

Read the full article

Every platform will evolve into a Platform-as-a-Service (and the resulting subcategory explosion)

Thu, 01 Mar 2012 09:44:00 +0200

Platform-as-a-Service (PaaS)... I guess you heard this term quite a bit last months. I wrote about PaaS earlier when arguing that Model-Driven Engineering is essential for the success of a PaaS and when I announced the new major release of our platform.

If you look at the industry perception of PaaS the main definition is something like: it is a public cloud platform where you put code in and get a service out. This is such a broad perception that it is no surprise that new subcategories are emerging every day. I will contribute to this movement by introducing a new one too...

PaaS is a fuzzy term, some subcategories

Let's first look at some example PaaS subcategories that have been introduced lately. Gartner, for example, sees a main dinstinction between Application Platform-as-a-Service (aPaaS) and Integration Platform-as-a-Service (iPaaS). I think you can imagine what the focus of each of them is. Gartner defines aPaaS as follows:

"Highly productive, easy to learn and use development environment that delivers business applications that are customizable, changeable, capable of implementing serious business functionality and, when deployed, offered with massive scalability, high-end enterprise-class (and eyond) performance and reliability, supporting massive amounts of data, all at SMB prices."

One could argue that to deliver enterprise-class applications implementing serious business functionality implies middleware and integration capabilities, but I can see the difference between delivering an application server as-a-Service and an ESB as-a-Service.

Other, more domain-specific, species I stumbled upon recently are pPaaS and bpmPaaS. A Portal Platform-as-a-Service and a Business Process Management Platform-as-a-Service.

I think we can agree on the fact that every type of platform can have a cloud twin and thus result in a PaaS subcategory...

Let's forget the public cloud as the main requirement for a PaaS

However, is being a public cloud platform the main differentiator between a "platform" and a "Platform-as-a-Service"? In my opinion PaaS is all about abstracting (or automating) away from underlying technology layers. You can just start using the services of the platform without bothering about the configuration and maintenance of the underlying infrastructure, operating system, middleware, and runtimes.

It all started with public cloud providers delivering such platforms as a service, but that doesn't mean it is limited to such providers. In my opinion it doesn't matter where the infrastructure is. PaaS can be very usefull too if it runs on-premise or maybe in a hybrid infrastructure approach. Just how I think the open source Infrastructure-as-a-Service software project OpenStack will more and more be used in the near future to build private and on-premise cloud infrastructures.

Why every platform will evolve into a PaaS

I think PaaS is another manifestation of what we always strive for in computer science: a clean separation of concerns. Let the platform builders bother about how their platform plays well with the infrastructure, OS, and middleware. Let the users of the platform focus on delivering solutions on top of the platform. There should be a clear contract between de platform and the software running on it, which specifies the language a user can use to define the solution. As IaaS is gaining ground in private datacenters near you, I strongly believe that existing platforms will evolve into a PaaS too.

Introducing a new subcategory

I just don't want to disappoint you by stopping here. I think there are some important things missing in the current industry views on PaaS. If we want to abstract away from the lower level details and move our focus to things like application development we need to go futher than just the development level. What a business needs are apps that perfectly fit in the business. These apps need to have a short time-to-market and have to be flexible to evolve along with the business. This means the full lifecycle of such applications, from the initial idea to first deployment, and from that first deployment to long-term evolution, needs to be as agile as possible.

I don't think we can reach this with traditional development approaches. Why not abstract away from low-level code in the same way as PaaS abstracts away from infrastructure by introducing visual modeling? And if our focus should be on apps that fit the business, we need more focus on collaboration with the business. Hence, we do not only need a development and deployment platform. We also need a social collaboration platform.

Ideally a Platform-as-a-Service should support the full application lifecycle. It's main goal shouldn't be to support development, it's main goal should be to deliver applications that fit the business. From capturing requirements to developing and deploying apps, as well as gathering end-user feedback.

Hence, my PaaS subcategory input: Application Delivery Platform-as-a-Service.

Photo by Robby Virus.

Blog overview 2011

Mon, 09 Jan 2012 16:56:00 +0200

In good tradition hereby the blog overview of last year. A bit late this time, but for good reasons (see the overview of the month december).

This year I expect to write a bit more. The subjects will stay the same, and as you could sense from the posts of last year my focus will be on all phases of the application lifecycle. So, Model Driven Development will still get attention, but also cloud and Platform-as-a-Service, as well as requirements capturing, social, and agile. As my first (and only) post with video material was the second popular post last year, I will try to post more video content this year.

Top 10 search terms

These are the top 10 search terms people have use when visiting my blog via a search engine:

model driven development
model driven engineering
enterprise architect
architecture vs engineering
domain specific language
soa framework
model driven software development
service component architecture
architecture definition
service identification

Blog summary for 2011

January

Interview on Pattern Based Engineering: Lee Ackerman, asked me a couple of questions about the use of patterns at Mendix and the relation with Model-Driven Development.
Why there is no future for Model Driven Development: this post was based on a talk I did in Nantes (France) and includes the slides and a full video of my talk. The main message is that we need to focus on all phases in the application lifecycle, not just development.

February

The quest for Business Agility: Emergent Architectures: a long article which clearly describes my views on designing, modeling, and building an agile enterprise with help of emergent architectures.

March

Code Generation 2011: just an announcement of the fishbowl session I organized at the Code Generation conference.

April

Why aren't we all doing Model Driven Development yet? An analysis of the main concerns which prevent people to use MDD. Initiated an interesting discussion.

May

Language Workbench Competition 2011: A lot of new initiatives in the field of language workbenches are arising to facilitate the creation of Domain Specific Languages (DSLs) and their accompanied code generators and IDEs. They all have their own strengths and weaknesses, so a competition to learn about them sound like a great idea back in 2010... and it was! This article describes all tools presented at LWC2011 as well as the opinions of the attendees. A nice overview of DSL tool space!

June

Making Model-Driven Software Development live up to its promises: the "sound bites" of the fishbowl discussion I hosted at Code Generation 2011. It was a good discussion on the future of MDD in four sections: marketing, alignment with programming, focus on problem domain, and education.

July

The Platform-as-a-Service for the Agile Enterprise: I shared a bit more about my work this time by describing the new major release of our products. This article perfectly fits in the subjects I normally cover like Model-Driven Development, app development, agile, and cloud.

August

Managing your application landscape: engineered or emergent: this article was triggered by the Application Landscape Report of Capgemini and defines two different approaches to manage an application landscape.

September, October

Well... moving to a new house doesn't leave you with a lot of time...

November

Enterprise Agile: 3 characteristics: there is a difference between an agile team in a start-up (or enterprise!), working in some sort of vacuum, and an agile team operating within the broader context of an enterprise. This article looks at some characteristics to make the difference more clear.

December

My first... :)

Blog overviews of previous years:

Enterprise Agile: 3 characteristics

Wed, 16 Nov 2011 21:54:00 +0200

No, this is not a typo... I really mean Enterprise Agile, not the subject of some of my previous articles about Agile Enterprises. I hear you sigh: do you really need to put the enterprise moniker in front of everything? Although this is a fair question I think enterprise agile is not just a buzz word. Agile on a bigger scale, in a big enterprise, is different from agile in a small isolated team. I do not care if you call it enterprise agile, enterprise-class agile, agile at scale, or something else. My point is: there is a difference between an agile team in a start-up (or enterprise!), working in some sort of vacuum, and an agile team operating within the broader context of an enterprise. Let's look at some characteristics to make the difference more clear.

Solutions need to be fit in an existing ecosystem

There is quite a difference between building a standalone piece of software and software which needs to fit in an existing application landscape. The latter asks you to spend time on analyzing the existing systems and testing the integrations. You will have to address the inter-dependencies of applications and data.

There is also quite a difference between building applications for consumers or small organizations and enterprises. Enterprise requirements like data ownership, data security, and all kinds of compliance rules need to be taken into account. Yes, there are successful examples of tools like Yammer and DropBox with a main focus on simplicity, but as their popularity within enterprises is growing they pay more and more attention to enterprise requirements. They can stay simple and focused, but they have to pay attention to, for example, compliance and security. This means you need to spend time on these items, you need to widen your definition of done to say it in agile terms.

Multiple teams work towards a common goal

Creating software within the broader context of an enterprise also means you are just creating parts of the software landscape. Multiple teams are working in parallel towards a (hopefully) common goal. Within this context the goals and deliverables of your team need to be aligned with the other teams. Releases and release dates need to be coordinated and you probably can reuse software parts created by other teams. This means there need to be alignment with (so-called) enterprise disciplines such as portfolio management, enterprise architecture, and strategic reuse.

This will probably ask for a kind of inception phase at the start of your project to define the vision, align stakeholders, identify the initial technical strategy, secure funding, etc. Have a look at the excellent whitepaper by Scott W. Ambler and Mark Lines "Disciplined Agile Delivery: An introduction" for a more detailed description of this concept.

Agility is only possible when the whole organization adopts the mindset

Agile is not just for the software department. If you really want to adopt enterprise agile as briefly described in the previous, you will need an agile enterprise. The whole organization needs to adopt the agile mindset. If only your software department adopts the agile mindset you can crank out releases in a very fast cadence, but what about the rest of the organization?

Let's take the case of being a software product company as an example. You can create new product releases every month, but can your marketing organization handle a release every month? And what about sales, support, training, finance, etc.? In an agile enterprise the marketing and sales side of the organization is balanced with product development. In an agile enterprise the entire business is organized in a way that it can respond quickly to changes in the market. All departments are fully integrated with the overall value stream, there is end-to-end agility. Even at the executive level the focus is on smaller bets, on short feedback cycles. It is such an end-to-end approach which is an important characteristic of enterprise agile.

Conclusion

Enterprise agile is not just a buzz word in my opinion. It depends on the product maturity phase and the target user of your product, but in a lot of cases, even in startups, you will have to make the transition from agile to enterprise agile in my experience. Ideally the team grows more mature along with the process (as they will learn why they change their process), but this of course only holds for the first team members, new people join a more mature process.

Do you have experience with the continuous improvement of your working processes? What did you learn? What are the important characteristics of a mature process in your opinion? Can you relate to the characteristics mentioned above?

Photo by Matija Grguric

Managing your application landscape: engineered or emergent

Wed, 24 Aug 2011 22:05:00 +0200

Companies often have more applications than the business needs. This means IT is spending resources on legacy applications instead of focusing on business innovation and growth. This years Application Landscape Report published by Capgemini concluded:

One of the top priorities of today's CIO is to build closer alignment between IT and the business. However to achieve this goal, IT first needs to deal with its own inhibitors to change. We have found that today's application landscape is often more an obstacle to successful business and IT alignment than a testament to it. As a result, first and foremost, IT needs to review its approach to the application landscape and create long-term modernization and rationalization strategies while reaping early benefits as well. Only then can a proper foundation evolve on top of which better alignment between business and IT can flourish.

A while ago I wrote an article about how business agility asks for emergent architectures as a way to align IT with the business by creating flexible application landscapes. More recently I also wrote about the software platform an agile enterprise needs to support its business. Hopefully that's all quite interesting, but as stated in the conclusion of the report, IT doesn't seem to have the time and resources to think about these kind of changes. The first step a lot of IT executives need to take is to reduce their application portfolios. In Capgemini's survey of IT executives, 85% of respondents state that their application portfolios are in need of rationalization. The question: how?

Application landscapes can be managed in different ways. For example, using Model-Driven Enterprise Engineering (it doesn't have to be a model-driven approach of course, but that's another discussion). I would call this the "engineered" approach, which can, in a simplified way, be characterized as:

Top-down: model the organization, define services based on the organization models, group the services in components, and implement the components.
Command-and-control: the architecture board / office decides on the application portfolio and steers the implementation choices.
Based on organization models: rigid organization modeling is needed to create a proper overview of the organization and to sufficiently be able to extract the needed services from the model.
Standardize on selected technology platforms: the architecture board decides what technology platforms are allowed to be used for application development.
Consolidate applications: do not let the business build to much applications, try to merge applications, consolidate on selected technology platforms, etc.

Another way to manage an application landscape is to use, what I would call, the "emergent" approach. This approach can be characterized as:

Bottom-up: the application landscape emerges from choices by individuals and business units.
Collaboration: applications, requirements for applications, and integrations between applications are created as a collaborative effort among people from different departments and, optionally, external providers.
Based on ideation and voting: applications are retired based on what their users think of them. Users can down-vote applications and create idea's for new applications or extensions of existing applications. Based on voting within an internal social network top idea's will be selected and unpopular applications will be retired.
Standardize on APIs: technology platforms don't matter as long as they are flexible, ‘managed' (either by the own IT department or by external (cloud) providers), and provide standardized APIs to allow easy integration and mashups (i.e. a focus on interoperability).
Grow application landscape: let the application landscape emerge from "local" choices. Focus on managing the process instead of managing the result.

These are just two examples of ways to manage an application landscape. It's not an either-or choice, in practice you will probably use elements from both approaches (and more). These approaches probably are not feasible at all if they are followed literally.

Do you recognize the need for rationalization? What mix of elements would you prefer to use? How are you managing your application landscape or how do you think an application portfolio can be best managed?

Photo by M Atif Saeed

The Platform-as-a-Service for the Agile Enterprise

Tue, 19 Jul 2011 20:41:00 +0200

If you have been wondering why I was a bit quiet lately... it was for the good cause! Today we launched the third major release of the Mendix platform, which is quite a memorable moment. My team did an awesome job and when I look at the result I can only feel proud! As I have been sharing a lot of my thoughts last years, I want to take the time to explain to you what Mendix 3.0 is all about.

If you remember one of my posts a couple of months ago, you could have seen the first glimpses of our focus with this release. In my article / presentation "Why there is no future for MDD" I tried to explain that we need more than just Model-Driven Development (MDD). What we need is an approach to the full application lifecycle, not only development. MDD is necessary (in my opinion), but not sufficient. We need Agile Application Lifecycle Management (in which MDD is one of the enablers) to solve the challenges enterprises are facing nowadays.

Mendix 3.0 (or to say it as our marketing guys like it: the version 3.0 of the Mendix Agile Business Platform) is all about supporting the agile application lifecycle, from a first idea to a working application, and from a working application to long-term business agility (i.e. the evolution of an application along with the business).

Let me take one step back and explain to you the full story of our new release. Please bear with my enthousiasm as it sometimes leads to praising my own products ;)

Introduction

Walking on water and developing software from a specification are easy if both are frozen.
– Edward V. Berard

Most enterprises are continuously striving to increase "Business Agility". Business agility is the ability of an enterprise to adapt rapidly and cost efficiently in response to changes in the business environment. James McGovern stated it nicely: "Increasingly, the stability of an enterprise is rooted in its ability to be dynamic, to move fast and change quickly". Time-to-market of new products, processes, and services is key to the survival of enterprises.

For enterprises to become agile a lot of things have to become more flexible. This doesn't only ask for changes in the structure and behaviour of an enterprise itself. As the level of automation in most enterprises is only increasing, the role of IT is an important factor in enabling business agility. However, enterprise IT isn't known for its ability to be agile... existing IT back-end systems cannot cope with constantly changing business needs.

On top of that, IT needs to become more business oriented. Trends like cloud computing and an increasing demand from users to replicate consumer IT simplicity in the enterprise, ask for a new view on enterprise application development. We see a trend towards App Stores and Business Unit Application Development (e.g. spreadsheets, Microsoft Access, Lotus Notes, cloud applications), which are great ways to give business people more tools to create innovative new IT uses. This, however, confronts the IT department with significant risks like governance, integrity, and security. IT must find a way to control the app-jungle, so to speak.

The main challenges for an enterprise yearning for business agility can be summarized as:

The business needs to become more responsive to change and wants to focus on the business side of IT enabled by App Stores and cloud-based solutions.
The IT department needs to leverage and extend IT legacy and wants to control the app-jungle resulting from an increase in Business Unit Application Development.

The Mendix Agile Business Platform fulfills these wishes of both business an IT by providing a Platform-as-a-Service (PaaS) offering the right set of tools to enable agile application lifecycle management.

The application lifecycle

So much of engineering is not making better decisions, but making shorter feedback cycles.
- Kent Beck

Let’s take a closer look at application lifecycle management (ALM). ALM is a continuous process of managing the life of an application through all its phases. ALM tools need to facilitate and integrate requirements and release management, development, deployment, maintenance, etc. We want to explain this process, the needed tools, and how we facilitate it in more detail in this section, but let’s first look at the agile moniker we’ve put in front of ALM.

Agile

What do we mean with agile application lifecycle management? In 2001 seventeen people met in Utah to find some common ground to uncover better ways of developing software. They came up with the, since then historic, Manifesto for Agile Software Development, which emphasizes individuals and interactions, working software, customer collaboration, and responding to change. Agile approaches, often referred to as "lightweight" approaches, focus on delivering software in relatively short iterations while requirements evolve through collaboration. From a technical perspective things like continuous integration, test-driven development, refactoring, etc. are needed to keep the implementation agile.

We think ALM needs to be grounded in the principles of agile software development to provide us with the flexibility we need to enable business agility. The only way to really improve software engineering is to create shorter feedback cycles, to increase collaboration with all stakeholders within a software project. That’s why our platform emphasizes an agile approach. In fact, we learned that the design decisions in our platform are a key success factor in delivering agile success to customers.

Figure 1 - Agile Application Lifecycle Management.

As visualized in Figure 1 we see the application lifecycle as a circle with four phases: capture, develop, deploy, and iterate. Besides these steps we think it’s important to manage the process and its outcomes. We will explain each phase in more detail in the next sections.

Capture

A new application (version) usually starts with a first idea. This idea is crafted and detailed by discussing it with stakeholders, resulting in a set of requirements. These requirements are captured in user stories (i.e. lightweight use cases) which need to be refined and prioritized. Crafting a single user stories often involves a lot of discussion among stakeholders. It is important to support the process of (co-)creating, refining, and prioritizing user stories properly to facilitate conversations and documenting the process in the mean time.

Furthermore, user stories need to be estimated and planned in sprints. The project team needs to be facilitated with the tools to plan releases and sprints as well as monitoring tools like agile status boards and burn-down charts. As agile project approaches are gaining traction we feel the struggle of teams to properly apply the principles. We think that such teams, besides proper training and a mindset change, need the right tools, fully integrated in the application lifecycle.

Develop

Once a sprint is fully defined and the team committed to the work and deadline it is time to start developing the application. In this phase it is again important to use the right techniques to properly support an agile lifecycle. This means the way of working during development should promote communication, productivity, quality, and short iterations.

We learned that Model-Driven Development (MDD) boosts productivity as high-level modeling languages can be used to develop applications. These languages are automatically executed on advanced engines thereby by-passing the need to use labour-intensive, low-level programming languages. This also ensures short feedback cycles as the result of a model change can directly be tested in the actual application. If these modeling languages are visual (as opposed to textual) they can even provide an excellent communication mechanism to align business and IT stakeholders. That’s why we believe MDD is an essential ingredient of agile ALM.

In principle you could say you don’t develop an application, but you compose the building blocks in the Mendix platform as well as existing systems into an application using a visual language. In combination with an AppStore providing templates, widgets, plugins, and even complete business components, developing an application is more and more the art of composition.

Deploy

The next step in the life of an application is to deploy it for (acceptance) testing or production purposes. Deployment sounds like a lot of hassle, and traditionally it is. Hardware, configuration, performance, scalability, involvement of other departments, etc. If your development process was agile it is jammed by now.

In our opinion cloud deployment is essential in providing agile ALM. Deployment shouldn’t be more than a single click, with which you upload your application model to a cloud environment, resulting in a working application. We call this Model-Execution-as-a-Service. The cloud platform executing the application models should free you from sleepless nights about availability, security, performance, scalability, and other issues like these, which you often only discover after your application has been taken into production.

Iterate

Some people seem to think that we are ready now. The application is running in production, end-users are using it and we have popped the champagne. Nothing is further away from reality! The process has just been started. Only when an application is used in production end-users will start to see the impact and think about improvements. Furthermore, the business process your application supports is probably not fixed. If we want to support business agility we need to provide a mechanism to gather feedback and to turn that feedback into a new application version. In other words: we need to gather feedback and use that as input for the next cycle of capture-develop-deploy-iterate.

Manage

Since the use of high-level, visual languages for developing applications enables domain experts to be part of the development process we see an increase in so-called Business Unit Application Development. This shouldn’t be something to be afraid of. The cloud platform running the applications ensures availability, security, performance and scalability (as mentioned before). In addition it gives the IT department the right tools to manage all the applications within an organization from a single dashboard. IT can enforce workflows around deployment, testing, and configuration. In short: IT can control the app-jungle!

Mendix products

We aim to make simple things simple and complex things possible.
- Alan Kay

Mendix provides the Agile Business Platform to fully support agile application lifecycle management. From your first idea to a working application and from that working application to long-term business agility. The Agile Business Platform consists of 3 seemlessly integrated products: sprintr, the Mendix AppFactory, and the Mendix Platform-as-a-Service. Figure 2 gives a brief overview of the 3 products.

Figure 2 - Mendix Agile Business Platform, product overview.

Sprintr

Sprintr takes a lightweight, social approach to enterprise project collaboration. We believe that engaging and empowering people facilitates collaboration and therefore co-creation in any organization. Sprintr offers a unique combination of project management tools and social activity streams, offering a way to interact with colleagues, project team members, and even external people involved in projects.

By providing a collaboration platform across the enterprise, sprintr breaks down the walls between the different departments and professions. All employees are part of the same private social network and share a wall showing all the “company buzz”. Conversations can be started and turned into ideas. Ideas are organized using a voting system and can be converted into projects. Sprintr offers all the tools you expect from a social network, including two-way email integration.

Besides being a powerful social platform, sprintr also offers the tools to manage your projects in an agile way. The social backlog management tools enable project team members and stakeholders to collaborate and to create, refine, and prioritize user stories. Each user story has it’s own conversation thread which is also published on the wall to have an intuitive mix of the company buzz and the project activity streams (i.e. of unstructured and structured data). User stories can be estimated and planned in sprints. The planning overview gives team members and stakeholders the ability to monitor the progress of the current sprint using scrum boards and burn-down charts as well as to plan sprints and releases to define the future of the project.

Sprintr can be used separately from the other Mendix products and it’s feedback widget can be included in any web application. Sprintr offers an open, well-defined API enabling third-party developers to create their own feedback widgets, plugins, and any other innovative product they come up with. Take the tour for a more detailed explanation and screenshots!

AppFactory

The Mendix AppFactory gives you the ability to develop an application by using high-level visual models. This enables collaboration between business and IT, but also provides extremely short feedback cycles as the AppFactory supports one-click deploy of your models. The Mendix AppFactory consists of 3 elements:

Mendix Business Modeler: a modeling environment to design and develop your application using visual models.
Mendix Team Server: a cloud-based model repository to collaborate with team members and version your models. The Team Server is implemented as a plugin to sprintr to create a tight integration between models and requirements.
Mendix AppStore: a community marketplace to share and download business templates, widgets, themes, and technical components.

The combination of elements in the AppFactory, as well as their integration with sprintr, really empowers business engineers to be more productive in delivering complex enterprise applications.

Mendix Business Modeler

Rooted in the techniques of Model-Driven Development (MDD) the Mendix Business Modeler is a high-productivity development environment for developing complex enterprise applications. An application can be modeled using high-level languages which are directly executable on our advanced run-time engines. This ensures short feedback cycles as the result of a model change can directly be tested in the actual application.

The Mendix Business Modeler is designed to deliver enterprise-level Service-Oriented Business Applications (SOBAs). Things like a multi-tenant security model, internationalization, flows, rules, and an advanced approach to extending the model with Java code are all built-in. Applications built using the Mendix Business Modeler can run stand-alone but are also easy to integrate with your existing application landscape. Each process in Mendix can be exposed as a webservice with a single click. Integration with webservices or XML is as easy as defining a visual mapping between the XML structure and your domain model.

As we are all about collaboration, the Business Modeler has excellent team support. It features version control on the visual model level as well as the possibility to track all changes both in your model and in external resources like Java actions and custom widgets. We provide the users of our Business Modeler with the full power of version control including advanced features like branching and merging. The intuitive conflict resolution mechanism assists you in merging all kinds of model changes even if another team member changed the same property as you did.

Team Server

Written on top of Subversion and delivered as a plugin to sprintr our Team Server is designed to make the life of a Mendix Business Engineer easier. The Business Modeler is tightly integrated with the Team Server and things like creating a new project (including a versioned model repository), updating, commiting changes, merging model versions, etc. are all available from within the Business Modeler as a single click action.

The Team Server is delivered as a plugin of sprintr for a reason: it enables you to manage Team Server access from sprintr and it enables us to provide you with a revolutionary way to combine requirements, implementation, and feedback! When you commit application model changes to the Team Server from within the Business Modeler you can select the user stories (reflecting requirements) you have been working on. The Team Server will automatically create links between these user stories and the model changes you made, providing you with a way to navigate from commits to the associated requirements. Further more, with these links we create a link from a user story to a changeset and a changeset can include a link to form (if you changed a form in the changeset). While feedback also refers to a form, we can create links between feedback, forms, changesets, and user stories.

The Team Server also connects the capture and develop phase of the agile application lifecycle. When you start working on the next version of your application you just open the Mendix Business Modeler to see the user stories planned for the current sprint and start working on them. If a user story is based on user feedback, you can directly jump to the form mentioned in the metadata of the feedback and start implementing the requested change.

AppStore

The AppFactory also includes a community AppStore, fully integrated with the Business Modeler to enable 1-click downloads of business templates, widgets, themes, and technical components. Mendix community members are using the AppStore as a platform to share business templates or extensions to the Mendix Platform. Most of them are open source, but the AppStore also features commercial templates as well as full-blown business applications.

Business templates range from small, handy model templates like an enumeration with all countries or an South African ID number validator, to complete business applications like service management solutions, Product Data Management for the Fashion Industry, a HR on-boarding portal, a carbon reduction suite, and many more industry-specific business templates.

With the widgets available in the AppStore you can build amazingly rich user interfaces. Widgets are build on top of the Mendix AJAX client API which is open for everyone. The AppStore contains lots of widgets, like Google Maps, rich text editor, all kinds of charts, tree grid, document viewer, twitter, just to name a few.

The technical components in the AppStore extend the Mendix Platform by building on it’s open API. The AppStore contains a lot of useful to integrate with Microsoft Exchange, Excel, LDAP, Kerberos single sign-on, database replication, iDeal, etc. But also components like an audit trail module and a license pool module to manage SaaS subscriptions.

Mendix Platform-as-a-Service

The core of our technology is our innovative cloud platform: the Mendix Platform-as-a-Service (PaaS). The Mendix PaaS gives you the ability to provision your applications in the cloud and manage them from a central dashboard. The Mendix Cloud Dashboard is easy to use and provides an interface to the power of the advanced Mendix multi-tenant cloud architecture.

Deployment of an application is as easy as uploading your application model to the Mendix PaaS from within the Mendix Business Modeler with a single click. The application model is provisioned in one of the available AppContainers and converted into a working application by executing it on our advanced run-time engines. We call this process Model-Execution-as-a-Service.

An application running on our platform consists of multiple AppContainers representing a test, acceptance, and production environment. The Mendix Cloud Dashboard provides users with an easy transportation workflow to move application model versions up to the chain from test to acceptance to production, without touching the model.

Availability, security, performance, and scalability are provided by the platform and easy to manage for all your application from the Mendix Cloud Dashboard. The dashboard provides graphs with the scope of a day, week, month, and year showing all kinds of monitoring information like the number of active users and resource utilization. Alerts can be configured to warn you if some metric approaches a critical level or if your custom defined application health-check gives a warning.

The Mendix PaaS is both closed and open at the same time. Closed due to its advanced security and access control. Open because it features easy-to-use backup management enabling to retrieve and restore your data at any moment, and because it features open APIs to interact with you application at runtime enabling, for example, to create integrations with any existing system.

We provide you with the PaaS that runs everywhere. Our platform can run on any infrastructure and flexibly scales out over different infrastructure types, both delivered as a service and on-premise. Due to the advanced multi-tenant architecture the platform utilizes your infrastructure resources in an optimal way.

Conclusion

I think we did take a huge leap forward with our new product suite, but don't take my words for granted! Let me know what you think in the comments, visit the what's new page with more details, or give some ideas for future improvements! One thing is for sure: we will go on with our journey to create the tools to make it actually possible to deliver the software an agile enterprise needs :)

Making Model-Driven Software Development live up to its promises

Thu, 09 Jun 2011 11:04:00 +0200

Model Driven Development proponents see a lot of advantages of using MDD techniques. Higher development speed, increased quality, more cost-effective, empowering less-experienced developers, just to name a few. If you look at these promises the question arises why the whole world isn't using MDD right now? Why don't we hear a lot of MDD success stories?

In a recent article I wrote about some of the main concerns which prevent people to use MDD, as mentioned in this StackExchange thread "Why aren't we all doing Model Driven Development yet?". This article triggered quite some discussion and you should definitely read the comments! I wrote this article as a warming-up for the fishbowl discussion session I hosted at Code Generation 2011 two weeks ago. In this session, titled "Making Model-Driven Software Development live up to its promise", we discussed the future of MDD in four sections, to cover subjects like popularity / marketing, creating alignment between programming and MDD, focusing on domain experts, and education. It was a great session with lively discussions and I want to share some "sound bites" with you in this post.

Marketing

We need to convince developers to start using MDD.
It's a social change: we need to move to a separation between language designers and language users.
The success stories of MDD are not shared because of the competitive advantage. Real success stories are often very domain-specific.
Generate the boring stuff, make developers happy!
As the market grows all current developers can move to language design. Language users can be a new group of people.
MDD is too hard to explain in one sentence. Complexity is a barrier.
The message of MDD needs to be mapped to business bla, but it should be different than other tech messages.
XP (eXtreme Programming) is successfully marketed by connecting it to incremental development / agile.
We need a different name. Nobody will buy MDD, it's already dead from a marketing perspective.
Focus on agile, that's the value proposition of MDD!
We need metrics to measure the success of MDD.
MDD supercharges Agile and Lean!

Alignment with programming

In certain communities it makes no sense to distinct between modeling and programming.
There is no difference: just a language at the right abstraction level.
MDD is only useful to bridge the time between steps in 3GL languages.
The mainstream feeling about language design is that it is very complicated. People don't know that the tools have evolved.
Are we as an MDD community our own biggest enemies? All "old" discussions will always come along, e.g. textual vs. visual languages, etc.
API is a model, a DSL. People are creating abstractions all the time.
Building linguistic abstractions IS different (a lot of discussion about this statement).
There is a projection possible from an API to a language, but they are not isomorphic.
We are good at language processing, so there is a case to create a language instead of just an API.
Using or creating a language asks for a different skillset. Be aware of the difference and approach developers with a different story.

Focus on problem domain

Move languages to the language of requirements.
Make languages specific.
Involve domain experts, use the existing notations of a domain.
Do domain experts want to have a tool to build software? No, they don't want to specify all details. They have no time for it. They only want to say yes or no.
Programmers / analysts need to translate words to models and focus on the exceptions.
It can be important that domain experts can READ models.
Domain experts DO want to have a tool to build software... focus on the domain experst who are working as analyst or consultant.
Fast feedback cycles are important. "Is this what you want?"
New job title: configurators.
Focus on the middle man! The one inbetween business and IT.
There is not enough critical mass to be successful when we focus on domain experts who want to "program".
Focus on the ones who understand the domain (not only the ones in the domain / the customers).
The biggest problem is to have a domain expert available to build the language. You need someone with a very deep understanding of the domain to design a domain-specific language.
We as an MDD community failed: we didn't bridge the gap between business and IT... we need a new role now? (the middle man).

Education

Universities don't teach MDD or modeling. There are people leaving universities who are not able to read a UML diagram.
Or it leaves a bad taste because of the mandatory waterfall use cases.
The problem is a generation gap. The renew time of professors at a university is 25 years, so you can only innovate every 25 years.
Universities shouldn't just focus on preparing for the job market. They should look further.
People need to learn more than one programming language to understand the concepts as grammars and trees much better.
We need to improve productivity, hence problme domain languages are needed.
Universities should create open minded people who will accept new languages.
You cannot create a simple DSL on top of complex UML.
De-empasize UML education.
The more human languages you learn, the more open you are to new languages. Minds will open!

Thanks to all attendees for the great discussions! I guess you also have a meaning about these subjects? Post them in the comments and keep this discussion going!

Language Workbench Competition 2011

Thu, 26 May 2011 20:20:00 +0200

May 24th the first Language Workbench Competition have been held in Cambridge (UK) as a warming-up of the Code Generation conference. The idea for a language workbench competition originates from a group of people during Code Generation 2010. A lot of new initiatives in the field of language workbenches are arising to facilitate the creation of Domain Specific Languages (DSLs) and their accompanied code generators and IDEs. They all have their own strengths and weaknesses, so a competition to learn about them sounded like a great idea back in 2010... and it was!

Assignment

All participants of the workshop had to implement an assignment, consisting of the following tasks:

0. Basics
This phase is intended to demonstrate basic language design, including IDE support (code completion, syntax coloring, outlines, etc).

0.1 Simple (structural) DSL without any fancy expression language or such.
0.2 Code generation to GPL such as Java, C#, C++ or XML
0.3 Simple constraint checks such as name-uniqueness
0.4 Show how to break down a (large) model into several parts, while still cross-referencing between the parts

1. Advanced
This phase demonstrates advanced features not necessarily available to the same extent in every LWB.

1.1 Show the integration of several languages
1.2 Demonstrate how to implement runtime type systems
1.3 Show how to do a model-to-model transformation
1.4 Some kind of visibility/namespaces/scoping for references
1.5 Integrating manually written code (again in Java, C# or C++)
1.6 Multiple generators

2. Non-functional
Phase 2 is intended to show a couple of non-functional properties of the LWB. The task outlined below does not elaborate on how to do this.

2.1 How to evolve the DSL without breaking existing models
2.2 How to work with the models efficiently in the team
2.3 Demonstrate Scalability of the tools

3. Freestyle
Every LWB has its own special "cool features". In phase three we want the participants to show off these features. Please make sure, though, that the features are built on top of the task described below, if possible.

The competition

Until now, 13 different tools have been used to implement the assignments. The submissions, including a lot of documentation and explanation can be found at the LWC website. 10 of the participants joined the LWC11 in Cambridge. In a tight day schedule they all had 40 minutes to demonstrate and explain their solution as well as highlighting some of their strong points.

I will try to give you a brief overview of all the tools, but I will not write a full comparison, that's way too much work. You can check the PDFs of the submissions or play around with some of the tools to get a better understanding of the possibilities. I will just give you some (subjective) "soundbites" from LWC11 to give you a first idea about the 10 tools. Don't hit me if I missed the best feature of your tool... 10 sessions of 40 minutes on 1 day is quite a challenge for my concentration ;)

JetBrains MPS

JetBrains MPS is an opensource (Apache 2.0 license) tool. Instead of text editors it provides a projectional editor, meaning that you're directly editing the abstract syntax tree (AST) instead of a concrete syntax which needs to be parsed. You start by defining the abstract syntax. Afterwards you can define the layout of the cells that should be rendered as you edit (i.e. the projection). In most cases it looks a lot like text, and the MPS team did put a lot of effort in giving you the feeling that you're editing text when editing a projection. However, you are editing a structured format (e.g. tree, tables).

MPS is strong in language extension. The Java definition is provided with the tool and allows you to great DSL extension to Java. Code generation is also done by defining a model-to-model transformation from the AST of your DSL to the AST of Java. So, to generate a target language you will need the AST of that language (which can be quite some work to define).

Markus Voelter, who did present the MPS submission, advises to use his EMF export to be able to use Xtext to generate text from MPS models, as Xtext is stronger in that field. If you want to learn more about he model-to-model transformations you will have to play around with MPS. According to Markus, M2M in MPS is easier to implement than to explain.

MetaEdit+

Steven Kelly demonstrated the MetaEdit+ implementation of the assignment. In fact, he implemented most of the assignment from scratch during his demonstration! Very impressive and the only one who did it this way. For me, this shows the productivity and ease of use of MetaEdit+.

MetaEdit+ really focuses on vertical (i.e. business domain) and graphical Domain Specific Languages. They offer some nice features to define your graphical concrete syntax like a symbol editor with conditional options (e.g. change colors or symbols based on model values). The code generators are quite different from other tools. Very modular. A nice thing Steven demonstrated was the generation of code within the tool and the possibility to jump from the code to the model and back.

The main comment on twitter about MetaEdit+ was that its UI looks a bit old-fashioned. That's of course a matter of taste, but their next version (5.0 - to be released soon) will have a new, Windows 7 looking UI.

Mixing manual code with your modes can be done by using protected regions in the code generators.

MetaEdit+ has quite powerful language evolution. Model data is kept when changing the metamodel (depending on the case of course). Generators keep working if you used relative referrals (walk the tree, use oid's, etc.). Removing elements from the metamodel will not remove them from the models but obsolete / deprecate them.

Most validations are automatically there because you cannot model anything outside the scope of your metamodels.

Steven finished his presentation with a nice demonstration of the generator debugger giving the ability to step through the code generaters, add breakpoints, ... and that's when the egg timer (which helped us to not let this workshop evolve in a two-day event) gave the signal for a coffee break.

OOMEGA

The OOMEGA platform was presented by Christian Merenda. It's an open source platform based on Eclipse with commercial database backends. They also provide a collaborative platform at metamodels.org to share metamodels.

OOMEGA uses a strict separation between abstract and concrete syntax. Multiple textual representations are possible for one abstract structure. OOMEGA uses a very compact language to define grammars, which is nice, but you will need to get used to it. However, Christian told me they will provide an additional concrete syntax (as said they support multiple textual representations of the same language) for this language to support people to start with the tool.

Language composition is done by defining and including packages.

OOMEGA is compatible with ATL wich enables you to transform OOMEGA models with standard ATL model transformation scripts.

Multiuser options: SVN for textual models, databases as third-party extensions (locking, notifications, etc.).
Nice navigation among metamodel, model, instance model. Just click-jump.

You can define validations for your models which will prevent you from saving invalid models.

Whole Platform

The Whole Platform presentation started by emphasizing their focus on an agile approach. It includes features like hot deploy, meaning that you can change language definitions and directly see the result in the resulting modeling environment. They use model interpretation (instead of code generation) to facilitate this process. It is even possible to deploy incomplete models to test what your are currently working on.

The Whole Platform is opensource (LGPL) and delivers projectional editors. The main focus is on graphical DSLs. Even their meta-languages are graphical, e.g. the grammar definition language is graphical EBNF / railroad diagrams. They want to enable less-experienced people to be involved in language specification. There were some comments about their graphical approach, for example this statement by Jurgen Vinju:

Some graphical DSL's look like AST visualizations with edit boxes.

It is possible to have multiple projections for one language and to seemlessly switch between them. Part of the projections are automatically generated, they didn't explain how much effort you need to define a projection, and I didn't ask ;)

Multiple DSLs can be mixed / used in one model. I'm wondering about the keyboard support (MPS did put a huge effort into this) as it looks as a lot of point and click is necessary.

Model-to-model transformations can be defined using a graphical DSL (in principle it's partly based on the visual syntax of the target language). The transformation interprets the source model with an analyzer and executes the transformation specification to generate target model. A very nice feature is the visual debugger for your model-to-model transformations which features step-through debugging on the graphical transformation model level.

The Whole Platform also contains a DSL to write tests for your DSL specifications, inspired by JUnit and integrated with the JUnit plugin in Eclipse to run your tests. Tests are once again modeled using a graphical syntax. These graphical languages are nice to demo, but they can have some disadvantages as Steven Kelly mentioned:

Whole Platform looks nice, but needs huge amounts of screen real estate. Dual projectors for LWC12? :)

Everything is possible on domain / DSL level, no framework or Java code is needed (holds for more tools). Code generation is done by using text in the transformation DSL.

The Whole Platform seems to be the surprise of the day as a lot of people didn't know them. They had a nice demo and their product looks stable. As Karsten Thoms stated it:

Whole Platform is the surprise of the day. Stable tooling, nice looking, quite different.

Steven Kelly compared the Whole Platform with JetBrains MPS:

Whole Platform = gorgonzola MPS: more mature and with an Italian flavour :)

Rascal

Rascal was presented by Jurgen Vinju. Rascal is a domain specific language for source code analysis and manipulation (i.e. meta-programming). Apparently it can also be used as a language workbench in combination with the IDE Meta Tooling Platform (IMP). As Jurgen stated it: IDE construction = Metaprogramming + GUI programming, which can be done using Rascal + Eclipse IMP.

Rascal is inspired by the ASF+SDF meta-environment and will be contributed to eclipse.org in June 2011.

Algorithms can be written in the language, but Rascal itself does not have many keywords or language features that provide built-in algorithms (such as constraint solving). The notable exceptions to this statement are general parsing and disambiguation, pattern matching, general traversal and lexically scoped backtracking features. It looks like a powerfull, complete language with a strong scientific basis.

Language extensions are possible by redefining or extending definitions of concepts. Rascal features strong modularization.

It seem to be not that strong on IDE stuff. IMP has it, but the integration with Rascal seems to be immature or Jurgen didn't show it. The Rascal language itself, however, is interesting. As Meinte Boersma said it:

Very high-quality presentation by Vinju on Rascal which looks "academically inspired" but definitely reality-checked.

Spoofax

Spoofax is also (like Rascal) based on SDF (a syntax definition language) and Stratego (a transformation / rewriting language). It's based on Eclipse and provides several meta-languages for language engineering.

Every spoofax language definition is an eclipse plugin. The plugins are dynamically loaded so you don't have to start another Eclipse instance (more language workbenches based on Eclipse should support this!).

You can use Eclipse IMP in addition to Spoofax to enrich your IDE. Lennart didn't explain in detail how this works and when you'll need this.

The main advantage of Spoofax seems to be it's sophisticated approach to grammars and parsing. As Markus Voelter stated it:

Defining expression grammars in #sdf and #spoofax really is much more convenient than in #xtext!

Intentional

Mats Helander demonstrated the Intentional workbench. Intentional uses projectional editors (like JetBrains MPS), so you're editing a projection of the AST directly. According to Mats you could see it as a semantic model (as Martin Fowler calls it) instead of an AST.

Multiple projections are possible. They don't have to be complete, so it can be possible that you have to use more than one projection to complete your model.

Code generation is done by transforming your model to the AST of C# (so, in principle this is model-to-model transformation). The AST of Java, C#, and Ruby are available in the tool. If you want to generate other languages you will have to build an AST of the target language yourself or use the plain text support.

The results transformations are still editable, they are just projections in the end. Changes will also be updated in the model. So, you can offer a relation database scheme to an expert and let him use his expertise to improve the database scheme and thereby your entity model. This only works if your model-to-model transformation is bi-directional (which is only possible if the transformation keeps a 1 to 1 relationship between elements in source and target languages).

Meta-languages are only used when necessary. In most cases they are just extensions of C#. Model validation rules, for example, are specified in almost native C#.

Languages can be split in multiple files. As long as you define your concepts in the same namespace you can refer to them in the same way as you would do if they were in the same file. Mats didn't explain if Intentional supports package definitions and imports.

The type system seem to be quite strong. You can even define distances between types.

Due to Intentionals projectional nature it is possible to add the results of calculations directly into your editor. These can for example be used for testing.

Mats finished his session by showing a functional specification like projection which makes it very easy for domain experts to be involved in software specification.

At the end of the session a Twitter discussion was started about the use of having multiple projections. Some "soundbites":

Steven Kelly: Multiple projections often a solution looking for a problem. E.g. human langs don't want multiple alphabets for 1 lang

Steven Kelly: MetaEdit+ has had multiple projections for 15 years - they don't get used much. Multiple reprs of each object is used a lot more

Angelo Hulshout: Is that because mult. projection is hard to implement ( ==hard to use), or because people really don't want need it?

Steven Kelly: I think because maintaining the same thing in two views is like doing it in two places: less work to do just one

Angelo Hulshout: Unless you have multiple people with different backgrounds working on it (at different times or in parallel?) Realistic?

Steven Kelly: Precisely what we thought it would be good for, but in practice one higher-level lang is better

Essential

Essential was presented by its creator Pedro Molina. The workbench is based on practical experience and builds on the following principles:

No model infrastructure friction, use model interpretation to avoid round-trips.
Target language independence.
Minimalistic IDE.
Clear separation of Concerns (much different from Rascal, which is one language to avoid problems with integrating the concerns).
Easy integration in other development tool chains.

Essential uses continuous model interpretation for error checking. It uses StringTemplate for code generation. Model-to-model transformations are possible with the provided meta-language.

Language composition is possible in the same way as partical classes in C#.

It is possible to load plugins in the IDE that use reflection to analyze the model and can for example generate to different target languages.

The IDE looks very clean (compared to the other textual / projectional tools), or essential if you like, but according to Karsten Thoms that could be because it's quite new:

Essential shows how simple a language workbench can be at the very beginning, and then the real world requirements kick in...

Obeo Designer

Mariot Chauvin showed us Obeo Designer. The focus of this workbench is on graphical modeling languages. It's based on Eclipse and works with an Ecore metamodel.

A nice feature of Obea is the ability to use filters (or layers) on graphical editors. Quite useful to grasp certain aspects of big models in a glance. They also provide a symbol editor to create your own graphical elements for you language, including conditional styles. It is property based and compared to MetaEdit+ you have less freedom.

For code generation you can use the full power of Acceleo, which is available as an open source Eclipse project. Validations can be specified on the metamodel or in the graphical editor, depending on the scope you want.

The audience was quite positive to see a good working tool for graphical editors on the Eclipse platform.

Meinte Boersma: Do I insult Obeo when I say that their Designer provides a nice (and working!) abstraction over GMF/GEF?

Etienne Juliot: No, it isn't an insult;) But it also provides viewpoint features for complex analysis:layers, conditional styles, refine, ...

Markus Voelter: Although (I think) it still has some rough edges, Obeo Designer is the first reasonable tool for graphical editors on Eclipse

Pedro Molina: Obeo Designers brings an easy way of creating visual dsl over the EMF stack

Model-to-model transformations can be done using ATL, again an open source Eclipse project.

Multiuser support is available due to Eclipse team support integration. There is no support yet for live collaboration using CDO, but this will be part of next version, which sounds cool!

Mariot reacts on the previous twitter discussion about multiple projections and states that it is useful to have them. He gives the example of modeling for airplanes: different people aren't doing the same job, but are still working on the same model. In this case multiple projections of that model can be quite useful.

Pedro Molina makes the comparison with the Visual Studio offering by stating:

First impression: I think Obeo Designer to Eclipse is quite similar to using DSL Tools on top of Visual Studio.

Xtext

Karsten Thoms did finish the day with an demonstration of Xtext. Xtext is a workbench for creating external textual DSLs on the Eclipse platform. It has a tight integration with EMF, opening up a lot of possibilities to integrate with other tools and to, for example, mix graphical and textual languages.

The grammar is specified with a textual language, but railroad diagram are available and clickable (will select grammar elements).

The generation gap pattern is used for mixing generated with manual code. Code generation is done with Xtend 2, which is more java like than XPand, but also has some similarities. Xtend 2 definitions are compiled to Java code, against a small framework. Xtend 2 also supports model-to-model transformations.

Validation can be done using Java (the Check language, which was part of oAW is deprecated), which triggered a reaction from Steven Kelly (in 2 tweets):

Xtext implements constraints in Java - I think that's the first tool today that doesn't use a DSL for constraints. I would expect a DSL to be better. The Java looks quite verbose. Java as a fallback maybe OK, maybe a DSL for most simple checks?

Xbase, part of the Xtext offering, is a partial programming language implemented in Xtext and is meant to be embedded and extended within other programming languages and domain-specific languages (DSL) written in Xtext. It's a great starting point for expression languages in your DSLs.

The Xtext workbench is mature, but had quite some big changes in the recent 2.0 version. Karsten showed some numbers to show that Xtext is scalable and performs well for big projects. It is used in practice for many big projects, which is an even better proof than just some numbers.

In the free style part of the assignment Karsten showed us some nice features like automatic formatting, quick fixes and told us about the instance DSL compiler which can for example be used to define tests.

Conclusion

The workshop was great! Really interesting to see all these tools in one day. Markus Voelter agrees:

This workshop is *so much better* than also those "academic" workshops with boring slide shows!

It wasn't really a competition as these tools have quite different characteristics. The future will learn what characteristics are the most important ones. One conclusion: graphical DSLs work great in demonstrations. To really make a useful decision about the tool you are going to use, you have to "feel" the tool an explore its borders.

The main questions nagging me at the end of the day was whether there is a market for 10 different DSL tools / language workbenches? How big is the market? Will it grow? Markus Voelter thinks there should be a market when looking at the diversity of the tools:

Judging from #lwc11, language engineering really isn't a niche (anymore). There are many diverse tools and approaches.... very nice!

One last observation: it was nice to have a parallel virtual-conference on twitter to directly share opinions without interrupting the presenter. It was also interesting to see the tweet rate increasing when a tool was more interesting, new, or impressive.

Thanks to Angelo Hulshout and Software Acumen for organizing this great workshop!

Why aren’t we all doing Model Driven Development yet?

Sat, 16 Apr 2011 20:01:00 +0200

About a month ago Kees Dijk asked a question on the programmers StackExchange titled "Why aren't we all doing model driven development yet?". He reiterates his question also as "What do you see as the biggest problems that make you not even consider model driven development?". It shouldn't be a surprise that I'm interested in these kind of questions. The answers, in this and other discussions about this subject, are interesting food for thought. Why? Because if you only read these 15 reasons why you should start doing MDD you would become curious why we aren't doing it all by now...

Let's look at some of the main concerns which prevent people to use MDD, as mentioned in this StackExchange thread.

No Silver Bullet

The most upvoted answer on the programmers StackExchange refers to the essay of Fred Brooks "No Silver Bullet - Essence and Accident in Software Engineering". Written in 1986 and still a valuable read! Brooks follows Aristotle in dividing difficulties in essential and accidental ones. Essential tasks in software engineering are about the fashioning of the complex conceptual structures that compose the abstract software entity. Sounds abstract, but it means that constructing data sets, relationships among data items, algorithms, and invocations of functions is complex, inherent from their representation. Accidental tasks in software engineering are about the representation of these abstract elements in programming languages and the mapping of these onto machine languages within space and speed constraints.

High-level languages are solving some of the accidental difficulties and are therefore valuable, but the essence of the problem remains the same. The specification, design, and testing of conceptual constructs need to be done by people, no tool will help you with that.

Dangers of MDD

Some followers of this blog may think I'm such a hardcore fan of MDD that I will now start defending MDD against Brooks. However, let's first try to get a balanced view on MDD. Please take notice of my previous articles 8 reasons why MDD is dangerous and 8 reasons why Model-Driven approaches (will) fail.

In the first article I point at the following dangers:

MDD actually introduces a lot of rigidity.
Models are only flexible where flexibility has been designed.
The roles of project members are quite different.
The modeling environment doesn't always support version control.
The modeling tool / approach is "almost" finished at project start.
The requirements team needs to understand what is allowed and what not.
MDD is sold to the customer, but the team has no experience.
Innovation distracts.

In the second article I mention the following eight reasons for possible MDE / MDD failure:

Not targeting all goals of Model-Driven Engineering.
Only using one modeling dimension: the dichotomy between PIM and PSM.
Focusing on generating new artifacts.
Using general purpose languages.
Using custom defined domain specific languages.
Using model transformations which are not fully executable.
Not testing the model.
Insufficient tooling.

I'm mentioning these items for two reasons. First, not every approach called MDD is the same. Hence, your experience with MDD cannot be extrapolated to MDD in general. Second, MDD is indeed no silver bullet. You still need to think about the complexity of your domain and the approach you want to use. There is a lot of difference among MDD approaches and you still need to select the right approach for your situation.

Tool support and popularity

An interesting answer on the question comes from "mko": "Microsoft / Apple / Google isn't pushing it". He adds "What kind of development gets popularized has much to do with tools, backer and evangelism". How true this is was shown in 2008 when Bill Gates introduced the Oslo project thereby creating a lot of buzz in the market. However, after it's sudden (and fast) dead it's a bit quiet again. Stuart Kent recently explained that Microsoft still has an MDD strategy.

Juha Pekka also points at a popularity problem:

I believe that there are several reasons but one is for sure that MDD is not in the curriculum of universities. Typically the closest is a course that teaches modeling and there the models stay as sketches (no checking, code generation, debugging at model level). This "modeling" course often also introduces UML and students are puzzled why to learn such a large and complex notation when the value of created models is low.

Contrast this to other field of engineering like embedded-hardware developers or control engineers where students get a quite different experience. With tools like Simulink or Labview students can draw a model and then it generated you the code, or at least you can run it in simulation.

In the past universities teached compilers and parsers, but now they should teach how to make generators, implement DSLs, etc.

Some people also point at tool support as an important reason. If you develop software you want tool support for refactoring, debugging, testing, validation, etc. The workflow is also important as code generators often introduce additional steps.

Modeling != Programming?

A more fundamental question is whether we should see modeling and programming as the same thing. Berin Loritsch touches this subject by making these points in his answer:

Because there are programmers like me who get more progress and results from TDD than MDD.
Because Modeling != Programming.
Because a model that sufficiently describes a function to generate the code is no longer useful as a model.
Because there are programmers like me who only use models at a high level, and then work out the details with code. You see things differently in code than you do in modeling software.

Does MDD need to convince programmers of the usefulness of modeling? Or does MDD lead to changing roles in software development? JetBrains MPS takes another position in providing an environment to seamlessly integrate programming and modeling (e.g. Java and your DSLs).

Are there even reasons to think about MDD?

Why should we even think about using MDD? I mentioned some reasons to start using MDD before. See also a previous article about successful applications of MDD in practice. Don't limit yourself in thinking that MDD means UML + code generation. Notable success stories often involve engines interpreting models like business rule engines, business process / BPEL engines, or (not to forget ;) Mendix. In my experience most of the problems mentioned before about tool support are solved nowadays.

But, let's get back to Fred Brooks. He stated a fundamental difference between essential and accidental complexity. As an example of accidental complexity he mentions high-level languages and explains why they don't give us the big leap in productivity we want. It can solve most of the accidental complexity, and I think MDD actually does so, but we need to address the essential parts of software development. Brooks gives four suggestions addressing the essential complexity when he gives an abstract of his 1986 article in 1995 [1]:

Exploiting the mass market to avoid constructing what can be bought.
Using rapid prototyping as part of a planned iteration in establishing software requirements.
Growing software organically, adding more and more function to systems as they are run, used, and tested.
Identifying and developing the great conceptual designers of the rising generation.

In my opinion MDD actually addresses the suggestions in the "No Silver Bullet"-article. First, it removes most of the accidental difficulties and let's you focus on the essence of the design. Second, it enables rapid prototyping and if you use it as part of an Agile Application Lifecycle Management approach you also have a great way to organically grow your software.

A relevant question?

This post mainly contains questions. It's not my goal, this time, to answer them... I recently gave my opinion about the future for Model-Driven Development. Do you think questions about the usefulness of MDD are relevant? Is there a future for Model-Driven Development? If so, how can we make MDD live up to its promises? How to address the "problems" of MDD mentioned above?

Please share your thoughts in the comments and consider joining my fishbowl discussion "Making Model-Driven Software Development live up to its promises" at Code Generation 2011!

--------------------------------------
[1] Frederick P. Brooks. The Mythical Man-Month, Anniversary edition with 4 new chapters, Addison Wesley (1995)

Photo by alshepmcr

The Enterprise Architect

18 lessons learned during the 6 year roller-coaster ride called Mendix

Will App Delivery PaaS make cloud more relevant to the business?

The Need to Focus on App Delivery Lifecycle in PaaS

Every platform will evolve into a Platform-as-a-Service (and the resulting subcategory explosion)

PaaS is a fuzzy term, some subcategories

Let's forget the public cloud as the main requirement for a PaaS

Why every platform will evolve into a PaaS

Introducing a new subcategory

Blog overview 2011

Top 10 posts

Top 10 search terms

Blog summary for 2011

Enterprise Agile: 3 characteristics

Solutions need to be fit in an existing ecosystem

Multiple teams work towards a common goal

Agility is only possible when the whole organization adopts the mindset

Conclusion

Managing your application landscape: engineered or emergent

The Platform-as-a-Service for the Agile Enterprise

Introduction

The application lifecycle

Agile

Capture

Develop

Deploy

Iterate

Manage

Mendix products

Sprintr

AppFactory

Mendix Business Modeler

Team Server

AppStore

Mendix Platform-as-a-Service

Conclusion

Making Model-Driven Software Development live up to its promises

Marketing

Alignment with programming

Focus on problem domain

Education

Language Workbench Competition 2011

Assignment

The competition

JetBrains MPS

MetaEdit+

OOMEGA

Whole Platform

Rascal

Spoofax

Intentional

Essential

Obeo Designer

Xtext

Conclusion

Why aren’t we all doing Model Driven Development yet?

No Silver Bullet

Dangers of MDD

Tool support and popularity

Modeling != Programming?

Are there even reasons to think about MDD?

A relevant question?