IEnumerable

Cost of logging in your architecture

2015-04-09T00:00:00+02:00

Rebuilding QueryModels from an EventStore

Rebuilding all read-models (projections) from an EventStore can be slow, because we are processing hundreds of thousands of events with several projections. This is one area where having a boost in performance is crucial and it is where I want to understand how much our logging infrastructure impact on performance.

For this scenario I have a small batch of events (~4k) and evlog4net logging level set at DEBUG level; this will generates ~35k log messages in this reference scenario.

Projections are grouped by slots, each slot contains a certain number of projections, and each slots processes events in a separate thread (the exact architecture will be subject of future post)

First run

Here is what happened during the first run.

This is a low resolution image, but it represents how much time the system needs to rebuild all projections with no log enabled. The Y-Axis represents how many events remain to dispatch for each slot (each graph is one slot), X-Axis represents time..

This is the reference scenario, now logging is enabled at DEBUG level, both for our MongoBufferedAppender and for ColoredConsoleAppender.

What happens with all logs enabled

The situation really changed a lot:

Even if you have no X-Axis legend, it is really evident that the whole process is slowed down a lot having log enabled.

We do not need exact measurement, even if the image is a low resolution one and we have no legend on X-Axis it is evident that: Enabling logging is slowing down everything.

We have also a really strange behaviour: most of the projections consumes events lineraly, three slots are really slower (this is normal), but when the other slots finished rebuild, the three slow slots speed up considerably. This means that Logging infrastructure is somewhat bounding toghether performances of different slots.

Believe it or not, the reason is, ColoredConsoleAppender. We have multiple services running in Console applications with TopShelf; they can run as a service in production environment, but we use console appender to have a visual clue of what is happening during development/debug, when services are run as standard console applications.

Stopping ColoredConsoleAppender

The next run is done simply setting threshold level of ColoredConsoleAppender to INFO instead of DEBUG. this reduce number of logs to ~500 for the entire run. Here is the result.

The whole rebuild is really faster. It is still slower that having no log enabled but we have a real better scenario. Again, you do not need exact measurement, the impact of disabling ColoredConsoleAppender is so great that it is evident that Logging to console has a huge impact on performances.

Actually I never liked too much console appender in log4net for various reason.

It slows down your application
It is difficult to read logs, console app have a limited buffer space and logs disappear quickly.
Log disappear when the program is closed
You have no way to query log, you only have a huge sequence of text scrolling on a console app
If the program crashes usually you see a red log with the exception and the console app immediately closes befoure you can read what happened.

All these consideration lead to an obvious conclusion: Console Appender is not useful, the only purpose is showing you that the program is doing something and it is not blocked somewhere.

If you want to use logging infrastructure to have an idea of progress of your application, please convince yourself that this is not a good idea. Using logs to verify that the application is doing something has little utility and it is not the reason logs are built for.

Now it is time to the final run.

Using MongoDbAppender with writer on a dedicated thread and LooseFix

We can further improve performance if you know how log4net works internally. Our MongoDbAppender inherits from BufferingAppenderSkeleton and you can read on my english blog why it slows down performances: BufferingAppenderSkeleton performance problem.

Armed with this knowledge I’ve done a final run using LooseFix and asking mongo appender to use a dedicated thread to save logs and here is the result.

Even if the image is small, you can verify that this image is quite similar to the first one. This is the desired situation, logging infrastructure cannot be completely transparent in performances, but if the loss in performance is negligible, the advantage of logging lots of information is invaluable.

MongoDbAppender have several advantages over ColoredConsoleAppender

It is durable; logs are maintained after the application is closed
You can query to find logs with lots of conditions (even regular expression)
You can setup a simple process to move logs from various Mongo source to a single location (Ex Elasticsearch + Kibana)
If you poll log collection and read most recent 10 logs and total count you can have idea of progress (this is not a perfect solution, but at least you can use for this purpose if you want).

What we learned

This example give us some guidelines on how to handle log in production:

Logging is useful and the more information you log, the easier is to find a problem. The golden rule is: measure performance penalties in production, use only a single appender to persists logs in a durable medium that allow searching and you are ready to go.

Gian Maria.

Cost of logging in your architecture was originally published by Andrea Balducci at IEnumerable on April 09, 2015.

Setting up a good Test Fixture is not easy

2015-03-21T00:00:00+01:00

The mytical perfect Unit Test Fixture

After years of unit testing I’m really convinced that the real complexity in writing a good Unit Test is setting up a strong test fixture. Here is an example: in Jarvis we have a test that runs fine in R# test runner, in Nunit GUI test runner, in Team City build test runner, but is red when it runs inside Visual Studio Test Runner with Nunit Test adapter.

After a brief investigation we discovered that the cause was a different handling of System.Threading.Thread.CurrentPrincipal object within the various test runners. Here is our situation

What is happening inside VS?

Here is a stupid test that confirmed our suspicions.

[TestFixture]
public class verify_principal
{
    [Test]
    public void verify_principal_is_empty()
    {
        Assert.That(System.Threading.Thread.CurrentPrincipal.Identity.Name, Is.Null.Or.Empty);
    }
}

This simple test verify that the name of the identity associated to the thread used to run the test is null or empty. This test runs green with every test runner, except with Nunit test adapter in Visual Studio.

When test is executed in Visual Studio, CurrentPrincipal identity is a valid Windows Identity and it’s equal to your current Windows user, and this makes some of our test fails when executed inside visual studio. This confirms me that something different happens inside Visual Studio and I want to fix it.

Fixture Fixture Fixture

The test that failed is this one

[Test]
public void raising_events_without_user_context_should_throw()
{
	var router = new AggregateRootEventRouter<SampleAggregate.State>();
	var ex = Assert.Throws<InvalidPrincipalException>(() =>
		{
			router.Dispatch(new SampleAggregateCreated(new SampleAggregateId(1)));
		});
}

This test verifies that: if you try to raise a Domain Event without user context, the engine should throw an exception. Clearly this test fails with Visual Studio Nunit test adapter, because CurrentPrincipal.Identity is equal to current user but it runs perfectly with other test runners.

Actually we can blame VS Test Runner for this behaviour, but the real cause of the erratic test is due to an imperfect test fixture. Let’s this for a moment on the purpose of this test; it is going to verify: how our engine behave if no user context is set. It works perfectly in all test runners except Visual Studio because they runs the test without an identity set in CurrentPrincipal, but this fact is not an assumption of nunit framework.

If you believe that the real culprit is VS Test Runner, consider what happened if, before this text executes, anohter test sets a valid Principal in Thread.CurrentPrincipal property and it does not restore it to original value when the test is finished. You will end with a failure in all test runners, but the failure happens only if the test that sets principal is executed before this one. What you have is

You run all test, test is red
You run the test alone, test is green (because the test that changes CurrentPrincipal is not run)
You run all test again, test is red (ouch)
You change the name of some test and the test returns green (maybe because the test that changes CurrentPrincipal is run after the erratic test

This is called erratic test, because it is a test that can fail due to other external conditions. Such kind of tests bring real pain when they fails, because you need to understand if the test fails because the underling assumption is wrong (you have a bug) or if it’s an external situation that makes it fail (the bug is in the test). In the long run this kind of test should be either fixed or removed from your suite of test.

The above test is better refactored in this way:

[Test]
public void raising_events_without_user_context_should_throw()
{
    var old_identity = System.Threading.Thread.CurrentPrincipal;
    System.Threading.Thread.CurrentPrincipal = new GenericPrincipal(new GenericIdentity(""), new String[] {});
	var router = new AggregateRootEventRouter<SampleAggregate.State>();
	var ex = Assert.Throws<InvalidPrincipalException>(() =>
		{
			router.Dispatch(new SampleAggregateCreated(new SampleAggregateId(1)));
		});
    System.Threading.Thread.CurrentPrincipal = old_identity;
}

This test is superior for a number of reasons

It explicitly clears the user context before the test.
The test is explicitly telling you that the engine is using System.Threading.Thread.CurrentPrincipal to check current user.
This test is stable and runs correctly in every test runner.
This test is stable even if a previous test set a valid CurrentPrincipal in current Thread for some other reason.

Lesson learned

To create stable and reproducible tests, you need to be sure that the system is the desired state in Fixture Set Up phase, and with desired state I mean every external component or piece of the system that is exercised in the text. A typical error is believeing that desired fixture was already setup for you by some other “entity” like the test runner or other tests.

Gian Maria.

Setting up a good Test Fixture is not easy was originally published by Andrea Balducci at IEnumerable on March 21, 2015.

Always execute your code before moving it in production

2015-02-24T00:00:00+01:00

Code that was never executed in production

One of the greatest risk in software development is letting untested code slip in production. The general rule is: you should always execute your code and verify that it behaves as you expected before moving to production. Clearly we should prefer automated testing over manual testing, but at least we need to be sure that every part of the code was at least executed once before promote it to production.

A real example

Suppose we have this code that represents an Id based on a simple string.

public class StringId
{
    public String Value { get; set; }

    public static implicit operator String(StringId id)
    {
        return id.Value;
    }

    public override bool Equals(object obj)
    {
       //....
    }
}

And we have also a Customer class based on this type of Id

public class CustomerId : StringId
{
    public CustomerId(String rawValue)
    {
        this.Value = rawValue;
    }
}

public class Customer
{
    public CustomerId Id { get; set; }

    public String Name { get; set; }

    public String Surname { get; set; }

    public Int32 Age { get; set; }

    public Double TotalOrderAmount { get; set; }
}

Finally we have some services that handle accessing data in Sql Database with NHibernate and we have a function that return all customers. To simplify this discussion lets consider a service that keeps data in memory.

public class CustomerService
{

    public Dictionary<CustomerId, Customer> _inMemoryRepo = new Dictionary<CustomerId, Customer>();

    public void AddCustomer(Customer customer)
    {
        _inMemoryRepo[customer.Id] = customer;      
    }

    public IEnumerable<Customer> GetAllCustomers()
    {
        return _inMemoryRepo.Values
           .OrderBy(c => c.TotalOrderAmount)
           .ThenBy(c => c.Id);
    }
}

Trivial code isn’t it? Since it is super simple probably you think that there is no need for testing. If you believe that the above code is simple and correct, you missed a bug.

Let’s try to write a simple unit test that exercise the code.

[TestMethod]
public void verify_get_all_customers()
{
    CustomerService sut = new CustomerService();
    sut.AddCustomer(new Customer() { Id = new CustomerId("c1"), TotalOrderAmount = 10.0d });
    sut.AddCustomer(new Customer() { Id = new CustomerId("c2"), TotalOrderAmount = 5.0d });
    sut.AddCustomer(new Customer() { Id = new CustomerId("c3"), TotalOrderAmount = 16.0d });

    var result = sut.GetAllCustomers();
    result.Select(c => c.Id.Value)
        .Should()
        .Have.SameSequenceAs(new [] { "c2", "c1", "c3" });
}

This test is oversimplified, but it represent a similar test that you can write if the service access database trough repository interface.

This test is green, but you have a bug!

The real problem shows up only with certain test data. This new test can highlight the bug.

[TestMethod]
public void verify_get_all_customers_with_same_amount()
{
    CustomerService sut = new CustomerService();
    sut.AddCustomer(new Customer() { Id = new CustomerId("c1"), TotalOrderAmount = 16.0d });
    sut.AddCustomer(new Customer() { Id = new CustomerId("c2"), TotalOrderAmount = 10.0d });
    sut.AddCustomer(new Customer() { Id = new CustomerId("c3"), TotalOrderAmount = 16.0d });

    var result = sut.GetAllCustomers();
    result.Select(c => c.Id.Value)
        .Should()
        .Have.SameSequenceAs(new[] { "c2", "c1", "c3" });
}

Now examine the differencies: basically it is the same test as before, but now we have two customers with the same TotalOrderAmount, and the test fails with this message.

System.ArgumentException: At least one object must implement IComparable.

And this is caused by the .ThenBy LINQ instruction. This instruction tries to order objects based on Id property, but since it is a complex type, LINQ does not know how to order based on CustomerId class. The previous test never exercised the condition specified by the .ThenBy

The solution is really trivial, just implement the IComparable interface for the StringId base class and everything is green. The real problem happens when this error is discovered in production and you need to do some quick fix.

Lesson learned

This kind of problem is caused by letting slip in production code that was never executed, nor manually, nor with Unit Testing. This kind of code constitutes a time bomb in your system that can explode in every moment.

A possible solution is trying to cover all lines of code with Unit Test code coverage, but code coverage is no perfect. In the previous example if you run only the first unit test (the one that succeeds) you got 100% code coverage for CustomerService class. This happens because the code that was really not covered is inside the implementation of the ThenBy LINQ operator.

This can be a classic example on how difficult is writing Unit Tests that are able to discover every bug in your code.

Gian Maria.

Always execute your code before moving it in production was originally published by Andrea Balducci at IEnumerable on February 24, 2015.

Ubiquitous Language

2014-10-07T00:00:00+02:00

What is

Programmers should speak the language of domain experts to avoid miscommunication, delays, and errors. To avoid mental translation between domain language and code, design your software to use the language of the domain. Reflect in code how users of the software think and speak about their work. One powerful approach is to create a domain model.

The ubiquitous language is a living language. Domain experts influence design and code and the issues discovered while coding influence domain experts. When you discover discrepancies, it's an opportunity for conversation and joint discovery. Then update your design to reflect the results.

The Art of Agile Development

Is it all about terms?

Some time ago I had the pleasure to meet with some friends for a GUISA meeting.

Andrea Saltarello gave the “Ubiquitous Language Smackdown” talk in which he shared his experience. We made a very similar experience ourselves (same terms, same communication issues with our customer), but I was firmly convinced that this approach could be used only on projects and was not suitable for products where domain experts are our clients (we craft business software for SME and Public Sector).

Based on our experience every client speaks it’s own language; it can depends on the market, the geographical location (even within the same country) and company size.

Among our products there’s a Manufacturing execution system; if we talk to a mechanical parts manufacturer we can refer to “Bill of Materials”, but if we talk to a fashion manufacturer and we say “Bill of Materials” the games are over: “your product is not for us, we don’t deal with BOM”. Their bill of materials (IT: “distinta base”) is “Scheda tecnica”; I don’t know how to correctly translate it with the correct meaning in english, maybe there’s not this distinction. If we talk with a PCB assembly company, “Scheda tecnica” means “Product Sheet”.

With all these terms referring to the same piece of functionality how can we design and code our model in an Ubiquitous way?

We gave up on UL until…

No, it’s all about Meaning…

One year later I was asked to do a three days class on CQRS+ES, usually I drop these kind of requests because it’s not our business, our goal is to delivery (working) software to our customers; teaching others to do the same is community stuff.

This time I made an exception, and something happened: “Semantic… semantic… semantic…” I found myself talking all the time about domain semantics and how domain semantics can help us to shape our models.

Probably that GUISA talk was the inception, unconsciously we started to use and profit UL in a new way (thanks again Andrea).

The whole thing emerged clearly with Evenstorming.

There's a sign on the wall but she wants to be sure
Cause you know sometimes words have two meanings

Stairway to Heaven - Led Zeppelin

We can’t use Ubiquitous Language with different customers and in our code base but we can use it on a smaller scope: a single organization.
Within a single organization Ubiquitous Language is an invaluable discovery tool; Ubiquitous Language exploration has become the cornerstone of our Evenstorming process (..more on slideshare).

…and Behaviour.

As expected, when we found that the same term is used by the people in the organization with different meanings we have probably saved many hours of work; I ask the users to write down on a wide blank post-it the meaning of every single term as soon as it materialize in any command / event / readmodel post-it.

The discussion starts and leads us very quickly to domain discovery down at the rainbow’s end!

Yes, that’s the fastest way to get to the “Pot of Gold”: term => meaning => use cases => behavior.

Stay tuned for the next post, I’ll tell you how Ubiquitous Language helped us to discover behaviour and model the “Document” aggregate in our DMS.

(spoiler: it’s not a single Aggregate root)

Ubiquitous Language was originally published by Andrea Balducci at IEnumerable on October 07, 2014.

Configurations

2014-09-24T00:00:00+02:00

Why configurations are important

When you start working with .NET technologies it seems natural to place application settings inside app.config/web.config files, but this approach usually fails to be useful in long term for various reasons:

When a software is composed by multiple services it is good to have a single configuration point
Config file mixes application specific settings with .NET settings (es. assembly redirect and connection string)
Xml is not the best human readable format
Frictions for automatic deploy (web.config transform, SlowCheetah, etc)

Resist the urge for complexity

As a good rule of thumb, architectures capable of launching rocket into space should be avoided, unless launching a rocket is the real need. If you only need a better configuration manager for a set of services, simplicity should be the key.

Simplicity is the ultimate sophistication. Leonardo da Vinci

Keep It Simple, Stupid. Wikipedia

Since I love designing software architectures the risk is producing unnecessary complexity, and I’m trying to change this attitude. The goal should be designing the simplest thing that could possibly work:

Solving simple problem with complex solution, #FAIL.

Solving simple problem with simple solution, #GOOD.

Solving complex problem with complex solution, #GOOD.

Solving complex problem with simple solution, #GENIUS.

Jarvis need “now”, is moving settings away from multiple config files and having a single “configuration manager”; a json file with all the configuration is good enough to start. There are also some little pain for configuration, leading to a simple Configuration Manager class to encapsulate access to json configuration file with some facilities.

Keep in mind DevOps

Releasing sofware must be an easy and well documented procedure and Operational people should be facilitated in troubleshooting wrong configruations.

One of the pain point to address with the new configuration manager is getting rid of this code.

var sysUrl = new MongoUrl(ConfigurationManager.ConnectionStrings["system"].ConnectionString);
var sysdb = new MongoClient(sysUrl).GetServer().GetDatabase(sysUrl.DatabaseName);

This code can bite during deployment. Suppose a connection string is missing; when the service starts it chrashes without any useful log or clue of what is wrong. Logging raw exception can be good for developers, but fails to be useful for operational people. If you got a MongoException or a NullReferenceException with a stack trace you have absolutely no clue of what is wrong and where you can fix the error. One of the goal of good configuration manager is creating good logs full of useful information to troubleshoot wrong settings.

Thanks to a new configuration manager, I can simply substitute standard ConfigurationManager with the new implementation, leading to code version 2.

var sysUrl = new MongoUrl(CqrsConfigurationManager.Instance.GetSetting("connectionStrings.system"));
var sysdb = new MongoClient(sysUrl).GetServer().GetDatabase(sysUrl.DatabaseName);

The only difference is in the new CqrsConfigurationManager class; if the setting is missing I got a meaningful log: Required setting ‘connectionStrings.system’ not found in configuration: http://localhost/Jarvis/Debug/config.json. This log gives two important information: What is missing and where I should add the configuration.

Fail as first as possible (and with good log)

The situation is still not optimal because it address only missing configuration but not bad ones. What happens if the configuration is present but is wrong (es. wrong mongo connection string)? The answer is: it depends.

Usually the code will throw some exception somewhere and only when the configuration is really used. The when problem is one of the worst. If your software needs the address of a special mongo instance where it should save commands that raised exception during execution and that settings is wrong, the service starts without any error log, giving you false confidence that everything is good.

It is only when the code uses that connection string that you start having error logs, probably containing: MongoDbException: Unable to connect to server fingolfin:27017: No such host is known and absolutely no clue on how to fix it.

The rule is: I want my software to detect as soon as possible wrong configuration settings, and log every information needed to do the fix!. This lead to code version 3.

MongoDatabase sysdb = null;
CqrsConfigurationManager.Instance.WithSetting("connectionStrings.system", setting =>
{
    var sysUrl = new MongoUrl(setting);
    sysdb = new MongoClient(sysUrl).GetServer().GetDatabase(sysUrl.DatabaseName);
    sysdb.GetStats();
});

The semantic is simple; WithSetting accepts an optional lambda that should consume and/or validate the setting in a single step. If no exception is thrown or lambda returns empty string, configuration is considered to be good. If the connection string is wrong I immediately got an exception: Error during usage of configuration ‘connectionStrings.system’ - Config location: http://localhost/Jarvis/Debug/config.json - error: Unable to connect to server fingolfin:27017: No such host is known.. This log contains enough details to understand what configuration is wrong and where it should be fixed.

This is especially good if the setting is complex, ex: a comma separated list of integers.

List<Int32> parsedConfig = new List<int>();
CqrsConfigurationManager.Instance.WithSetting("coefficientList", setting =>
{
    var splitted = setting.Split(',');
    Int32 tmp;
    foreach (var number in splitted)
    {
        if (!Int32.TryParse(number, out tmp))
        {
            return "The value " + number +
                    " is not a number. I'm expecting a list of integer number comma separated.";
        }
        parsedConfig.Add(tmp);
    }
    return "";
});

If a wrong string 13,24,3O9 is found in configuration file (there is an O letter instead of zero in last number), the system will log this message: Error during usage of configuration ‘coefficientList’ - Config location: http://localhost/Jarvis/Debug/config.json - error: The value 3O9 is not a number. I’m expecting a comma separated list of integer.. Whenever a setting is consumed by the application, it is useful to consider a little variant of the famous rule found in Code for the maintainer

Always code as if the person who ends up installing your software is a violent psychopath who knows where you live!

Next post will deal on: Where do you specify location of json config file used by CqrsConfigurationManager?

Configurations was originally published by Andrea Balducci at IEnumerable on September 24, 2014.

Architecture

2014-09-18T00:00:00+02:00

Jarvis Dev stack

In the previous post we introduced Jarvis, in this we’ll talk about the development stack and the architecture.

The client is an Html5 AngularJs Single Page Application, our backend is mostly c#.

The core domains are implemented in eventsourcing and cqrs on top of NEventstore with a custom implementation of CommonDomain.
For NEventstore v5 we needed a better implementation of Mongo Persistence Engine so I started pushing our fixes on Github.. some pull requests after I was invited to join the team (thank you Damian).
That was our first giveback to the dev community, then I patched / enhanced the majority of the core libraries we used in this project.

That’s (imho) how open source is supposed to work, less complains and more pull requests.

Jarvis core domains are hosted in few (Topshelf based) windows services talking each other with Rebus and MSMQ.

The webapi services are hosted in IIS and secured with integrated authentication.

Data flow

Frontend api

Jarvis data flow is “realtime async”: our frontend api accepts and validate user commands, translate them in domain commands and push them to our commandbus for delivery.

Domain services

The destination worker handle the command, with automatic retry and failover, invoking the corresponding aggregate methods, it’s a sort of RPC over service bus: there is not a strict one to one mapping between commands and methods.

The target aggregate emits events in response to method calls to change its state, the events are persisted in NEventStore waiting for consumers: projections and process managers.

Process managers

Every process manager place a subscription on Rebus for the events it is interested in. To be more accurate a process manager can subscribe to events and messages; a message doesn’t have domain semantic and is exchanged with external high latency services. The process manager react to events sending new commands and messages.

High latency services

We need to rely on external services for text extraction, document conversion and analisys; all the cpu intensive and high latency tasks are hosted by a service worker installed on one or more machines with a simple round robin routing for task distribution.

Projections

Our projections service works in pull mode; projections are grouped by affinity; every group has a subscpription to a polling client on the evenstore and run in its own thread. Our querymodel is denormalized on MongoDb and Elasticsearch.

Push notifications

Query model updates are pushed to subscribers with SignalR and processed by AngularJs for interface updates.

What’s next

This architecture worked well for our first customer, we have over 100 concurrent users, databases, frontend api and worker services on the same vmware box. Cpu is mostly between 1-2%, ram consumption flat and the whole roundript usually takes few milliseconds.

Rebuilding all the 50+ projections from 350k+ commits takes almost 10 minutes (we are single thread), we need to improve in this area.

Our querymodel is about 4gb, we expect to grow (at least) by a 100x factor in the next two months.

We want to split our domains in isolated services and remove the friction for every deploy: we deploy twice a week, some weeks every day (or twice a day).

To achieve this goal Gian Maria is working on a configurations dispatcher service and getting rid of all the connection strings and application settings palced in the web/app.config.

What is Jarvis?

2014-09-14T00:00:00-00:00

File > New > Intranet

This is the first step of this journey, we want to tell you what we have built so far.

One year ago we started an “Intranet 2.0” project for a small bank.

The project main goal was to improve the ease of access to key informations, avoid personal (and sometimes out of sync) document folders, on user’s desktop.

We found a “Yahoo Directory” of about 4.000+ relevant documents (250.000+ pages).

To simplify document retrieval, every office developed a custom taxonomy to catalog corporate documents.

In the beginning there was Chaos,
And within this Chaos was Power,
Great Power without Form

Forget Google and try to search the internet browsing the “Yahoo Directory”.. you got it!

Our first goal was: no more phone calls to “knowledge owners” to find “that document” or endless filesystem style browsing in search of relevant informations.

So our first domain was a DMS.

library by kiarras, on Flickr.

Then we moved on:

Support Ticketing with Knowledge Base,
Corporate address book,
Home Banking workflow,
Point Of Sale workflow,
Resource scheduling system,
Corporate bookmarks,
Corporate chat & file exchange,
Active Directory sync,
Personal & team calendars,
Organisation chart,
IT asset inventory,
Analytics.

Some domains have been implemented with CQRS+ES, others with CRUD.

Next post I’ll cover the current architecture and our dev stack.

Stay tuned!

What is Jarvis? was originally published by Andrea Balducci at IEnumerable on September 13, 2014.

About this blog

2014-09-09T00:00:00+02:00

Intro

This blog is about a journey we want to share: we are building a new software product, learning day by day something new from our users, from our mistakes, from our friends.

Why we are doing this? (Quid pro quo)

No this blog is not a marketing effort, it’s a giveback to the dev community; a giveback we hope can improve our learning journey through your feedback and maybe avoid you a headache for the same mistakes we made.

Learn in the open. Have no shame to publish your code and ask questions and go after it.
— Kelly Sommers (@kellabyte) August 26, 2014

ok.. show me the code.

What you will find on this blog?

We plan to publish all the code with you (yes, the whole product.. not just code snippets) in 6-9 months when we hope to reach a full featured 1.0 release and have 500+ active paying users.

We’ll share more about the product (domain insights), tradeoffs and our current dev stack (AngularJS, .Net, NEventstore, CQRS / ES / DDD, MongoDB, Elasticsearch) in the next weeks.

So.. No code? No party! Hold on… software development is a learning process, working code is a side effetc

Stop thinking SW projects like building process, and start thinking Collaborative Learning Processes. Everything becomes clear. #agile #ux
— Alberto Brandolini (@ziobrando) September 3, 2014

That’s the point, we started this journey one year ago and we are pretty happy with this way of thinking / doing things.

We want to share what we have learned so far and what we’ll learning in the next months (and yes.. some code too).

Who we are

My name is Andrea, I run a small (4 devs) software house in Italy and I’m very active in the local dev communities.

I failed twice at blogging, this is my 3rd try and the first one in English (well beyond my comfort zone).

I asked a close friend in the dev community to join us for the next months, we have a new customer and we need to scale up to 100x in terms of data, 4x in terms of active users and a backlog with a lot of new interesting stuff.

Let me welcome on board Gian Maria.

My Name is Gian Maria, I’m an independent consultant on .NET technologies, ALM and Microsoft Team Foundation Server and I’m passionate about everything about developing.

I have a huge interest about DDD and modern software architectures and I was really amazed about the product Andrea built in really short time with its company, so when he start needing help I was really excited to jump in it. The best part of starting collaborating in this project is probably the idea of publish everything online to share [ code | opinions | comments | everything ] with DDD community.

In the end…. I’m really exited and I hope this will reflects in my blog posts, stay tuned.

Wrap-up

Me and Gian Maria we’ll keep this blog up to date with our journey, and we hope to have guest posts from our friends sharing a similar path (and some code) on other projects. Yes, we are looking at you Stefano and Alessandro.

Thank you guys

This journey wouldn’t be possible without:

Mauro Servienti: inadvertently gave me the idea for the product name on our trip to Frasassi caves;
Stefano Ottaviani: for choosing this project for his graduation thesis;
Andrea Saltarello: Ubiquitous language, Ubiquitous language, Ubiquitous language!;
Alberto Brandolini: for introducing us to Evenstorming in the early stages and for sharing a talk inspired by a subdomain of this project;
Alessandro Melchiori: the first talk about CQRS I’ve attended (as a powerpoint assistant.. “next slide please”)… you owe me a beer;
Alessandro Giorgetti: “DDD can’t work! Angular? No, KnockoutJS is fine”. Now he’s mastering DDD & AngularJs in the healthcare.

Our chats about aggregates, process managers, ubiquitous language, eventstorming, CQRS and Domain Driven Design are the building blocks of this journey.

A special mention goes to Rinat Abdullin for inspiring this blog with his HappyPancake post.

Curious? Suggestions? we are waiting your feedback and we have room in our blog backlog.

About this blog was originally published by Andrea Balducci at IEnumerable on September 09, 2014.

Wish we could say more

2014-09-04T00:00:00+02:00

09.09.2014

coming soon…

hits in meta, if you don’t have already inspected page source :P

Wish we could say more was originally published by Andrea Balducci at IEnumerable on September 04, 2014.