Let’s say our model looks like this:

In order to display a list of posts, we need the title, the author’s name and the content. So we effectively need to join posts with their authors. This is what we want:

If you don’t know how to do this in RavenDB 1.0, please go to this post.

In RavenDB 2.0 however, there is a new feature that let’s us load documents during indexing time. This means, we can load the related author document while we index the post document. Combine this features with the possibility to use an index as the read-store for your queries…

Here’s how our index looks like:

Note how we first load the author document and then store the author’s name inside our index, so we can get that value back at query time.

Here’s how a query looks like:

How awesome is that?

Please note that RavenDB does not load the author document at query time. Instead, it already has the author’s name stored in the index, so querying is as ultra-fast as if there wasn’t any relation at all.

If the author’s name changes, RavenDB will automatically recompute the index for all the posts that are related with that author. Unlike with denormalized references, we don’t need to maintain that relation!

Personally, I think this will fundamentally change how we work with RavenDB. Do you agree?

The post Joining documents in RavenDB 2.0 appeared first on Daniel Lang.

Doing email is definitively not one of the things I consider to be productive work. Think about it: when was the last time you felt like you did get a lot of things done after spending a couple of hours in Gmail? You get it, never!

This is why I personally try to spend as little time as possible in my email client, which is Outlook by the way. I know every keyboard shortcut by heart and I can literally fly through my inbox, replying, deleting and marking items with only a few keystrokes. However, there’s one type of emails that makes me want to cry. They basically go along like this:

From: Somebody (somebody@contoso.tld)
Subject: RE: Meeting tomorrow?

Hello Daniel,

Hope you’re doing good.

In regards to your proposed meeting tomorrow, yes, 4pm is just fine.

Have a nice afternoon,

Somebody

Chief Executive Sucker

Something Corporation
Checkout our new Facebook site at…

Phone: +33 44 55 66 77
Fax: +77 88 99 11 22

Email: somebody@contoso.tld
Web: www.contoso.tld

Confidentiality Warning: This e-mail contains information intended only for the use of the individual or entity named above. If the reader of this e-mail is not the intended recipient or the employee or agent responsible for delivering it to the intended recipient, any dissemination, publication or copying of this e-mail is strictly prohibited. The sender does not accept any responsibility for any loss, disruption or damage to your data or computer system that may occur while using data contained in, or transmitted with, this e-mail. If you have received this e-mail in error, please immediately notify us by return e-mail. Thank you.

Seriously? Why are you doing this to me? Only 5 minutes ago I sent you an email asking if 4pm tomorrow was a good time to do the monthly project meeting. Because you are replying to my email, there’s no need to establish context first. I know who you are, and since we send us a couple of emails every day, you don’t need to say ‘hello’ every time! Have a look at how little information your whole pile of characters actually contains.

This is what I want to receive:

Yes, 4pm is fine. See ya!

Please, do me a favor and remove all that useless waste from your emails. You don’t need a signature. You don’t need to say hello and goodbye on every email you’re sending. Don’t make me distill your emails to find what is the actual message.

Please try to say what you want to say in as few words as possible!

The post Please stop using email signatures appeared first on Daniel Lang.

What is responsive webdesign?

In the beginning was The Fixed Width, and The Fixed Width was Great, and the Great was The Fixed Width. It was not until people started to buy iPhones and Androids when troubles started to begin. Users were no longer satisfied with how our carefully crafted web pages looked on their new gadgets. So we built mobile versions of our websites, especially targeting those crippled 320 to 480 pixel devices. But then Steve gave us the iPad and – what is even worse – people loved it and navigated straight to our desktop optimized web applications. So what shall we do? Build tablet-optimized versions of our sites? Who’s going to pay for all this?

The idea of responsive webdesign was born. Instead of building three websites for different devices, let’s just build one that fits all!

Have a look at these great examples of responsiveness:

• colly.com
• seesparkbox.com
• mangomint.com (yep, that’s my own company)

Make sure you resize your browser window and note how various elements change fluently (i.e. the navigation menu).

How it works

Technically, there are few things to consider in order to make a website responsive:

• No fixed widths but ratios (i.e. how many parts in a 16 parts wide grid system)
• Content organized in floating elements that can rearrange in rows with different number of columns
• CSS media queries to selectively show/hide elements and address special issues with lower resolutions

This is just the gist of it. There’s much more to say about those techniques, but that’s not the point of this post.

I should mention that there are tons of frameworks, libraries and instructions available that help you get these things right. With all that help and a little bit of practicing, it’s actually not that hard to implement responsive layouts.

So what’s wrong with responsive webdesign?

Nothing. I just say that they cause a damn lot of work and you better be sure it’s worth the effort. Here’s a few things to consider:

• The width issue
  Think about how many times you wrote something like ‘width:240px;’ in your last website project. Forget that! Now you have to think in terms of ‘eight-columns’ or ‘six-columns’. Most of the time that works just fine, but sometimes you really want that 365px width. Oh, that’s already more than what you have on the smallest iPhone resolution…
• The spacing issue
  So let’s show a bunch thumbnails and put a slick 20px margin around. Four in a row, just so that we have the same spacing to the outer box left and right. Looks good, doesn’t it? Except when you resize the browser and everything gets turned around, then that 20 pixel look quite different and now you don’t have that nice spacing anymore. 
• The content issue
  Noticed that most responsive webdesign tutorials show simple item grids? That is because they are perfectly suited to demonstrate the idea. In the real world, however, there are landing pages, overview pages, product comparison charts, complex registration forms, etc. You really have to make sure that every piece of content adheres to the responsive principles – that means everything should be either resizable or floatable. That’s whole lot of worries for content creators.
• The programmer/designer issue
  For many year, designers only needed to hand over PSDs to us programmers. That was good because we suck at design things. With responsive layout, that gets a whole lot more complicated. Obviously designers can’t create separate PSDs for every possible screen resolution and every page. Programmers are given much more responsibility to make the final work look good. Since their perception of what looks sufficiently good can be quite different from the designers point of view, there’s usually a lot more back and forth between those two guys.

Do you really need it?

I am in no way an expert in webdesign generally and responsive design specifically. Real web developers would probably call me a script kiddie or – if they are really evil – a “windows guy”. However, I’ve worked on a couple of dozen non-trivial web applications, most of which did not implement a responsive webdesign and some more recent projects with a responsive layout – after all, I can definitely say that there’s at least a 50% increase in the total amount of time it takes to get the web related stuff done.

Also note, that responsive websites require quite different organizational structures. Content creators must be aware of the restrictions and paradigms. Designer must have a better understanding of how HTML and CSS works. The size of working batches between designers and programmers is much smaller. Communications between those two roles is chattier. All that must be taken into account.

So before you jump onto that responsive train, go ask yourself: is it really worth the effort?

What do you think? Have you had different experiences during your projects?

The post Is responsive web design really worth the effort? appeared first on Daniel Lang.

System.InvalidOperationException: Could not open transactional storage

First, let’s talk about what could possibly lead to such a problem. As you probably know at this point, RavenDB uses ESENT as it’s storage engine. ESENT is a transactional storage engine that ships with every Windows version since Windows 2000 and powers things like the Active Directory, Windows Search Index or Exchange Server. When you get the above error, there’s almost always something wrong with the ESENT files on the disk.

Personally, I’ve seen 4 different scenarios (I’m sure there are more) that could bring your database into such a corrupted state:

• Dirty shutdown
  Power outages, hardware failure or evil admin users could cause the RavenDB process to be killed unexpectedly. When this happens, there is a high chance that the ESENT data file on disk gets corrupted.
• Moving between windows versions
  If you just copy the data directory (instead of using Smuggler), you will likely end up with ESENT not being able to startup on the new machine.
• Live migration of a virtual machine
  Don’t believe hypervisor vendors like VMware or Citrix when they claim they can securely migrate a running virtual machine. They can’t. I’ve already crushed two databases while doing live migrations.
• Upgrading to RavenDB 2.0 unstable
  If you’re like me and can’t await the official realease of RavenDB 2.0, then please be aware that there could be issues when you upgrade an existing database.

I should say at this point, that RavenDB tries to automatically recover the database upon startup when it notices that it wasn’t shut down cleanly. However, that didn’t always work for me and I had to execute some manual steps to get my databases up and running.

Actually, it is quite simple – you just need to learn how to use the tool esentutl.exe which comes with Windows.

Here are a few things that you can do:

Defragmentation

This is most useful when you moved the data folder between different Windows version. Go into your data folder, open a command prompt and type:

esentutl /d Data

Soft recovery

On the disk, ESENT maintains all of the data within a single file. In addition to that, every transaction gets written to a log file. Using that log files, ESENT can re-run those operations and recreate a consistent state. Using this command, you should be able to recover from almost all damages:

esentutl /r RVN /l logs /s system

Repair

When things went really bad and both defragmentation and recovery didn’t help, you can fall back to a forced repair. Only consider this option when a recovery didn’t work. If you can, restore from a backup. If you can’t, this command can save your butt:

esentutl /p Data

Of course the best is to avoid running into these situations in the first place. However, if when you have this scaring “Could not open transactional storage” exception, it is good to know what to do!

The post How to recover a corrupted RavenDB database appeared first on Daniel Lang.

Let’s look at an issue I just ran into that caused serious data loss.

First, I have a domain model class Person which inherits Contact.

Then, there is a PersonViewModel which inherits ContactViewModel.

And my controller couldn’t be easier, it’s basically just this:

Please pay attention to how we update the person. We just load the person from RavenDB and then use AutoMapper to update the properties with the ones from the view model. RavenDB automatically tracks the changes and updates the document in the database.

Good. Now we need to configure AutoMapper. Before I show you the code, please be aware of the contact’s property CreatedOn. There are valid reasons for which I don’t want this property to be mapped when going from ContactViewModel to Contact. So we have to ignore it:

This looks great and it uses AutoMapper’s inheritance mapping feature.

Except… it doesn’t work!

The problem here is that because of how AutoMapper prioritizes the mapping, it would actually map the CreatedOn property when going from PersonViewModel to Person. In other words, it didn’t inherit the .Ignore().

What was particularly bad for me was that I only noticed this when I had a lot of documents in RavenDB which had CreatedOn set to “0001-01-01 00:00”… hmm. It doesn’t matter that I think this was a bad design decision in AutoMapper and somehow should let Jimmy Bogard now about that – the data is gone and I can only restore it from backups.

How could I have prevented that? The answer is trivially simple: I should have tested my assumption about how AutoMapper works.

Here is how a test could have looked like:

Next time, I’ll make sure I unit-test my mappings and I hope you do so as well.

The post Don’t forget to unit-test your mappings appeared first on Daniel Lang.

Searching for contacts then goes like this:

Which gives you all the contacts, both persons and companies that somehow match the term q.

Now, suppose we have a company with the name Contoso Ltd and a person John Miller who works for that company. In addition to that, we have tagged the person with ‘contoso’. How would you like the results being ordered if we search for the term ‘contoso’?

This is what RavenDB gives you by default: John Miller; Contoso Ltd

… which is not very good, because apparently you want to have the company on the first position since your search term exactly matched the company name. However, the score of John Miller was higher because we matched the term twice (in the company name and in the tags). So what we have to do is to boost the field ‘name’ in the index, right?

Let’s try this:

Unfortunately, this doesn’t work here as well. The reason for this has to do with how Lucene does scoring when using a PrefixQuery (which we actually do since we have StartsWith). If we, for instance, change the StartsWith to Equals it would give us the expected results, but this isn’t what we want.

The Real Solution™ is to do the boosting at query time, instead of index time. Let’s change the search query to this:

This time, we’re boosting the field ‘Name’ by a factor of 10 at query time. This makes that match powerful enough, so that even though John Miller has two matching terms, we will still get Contoso Ltd first because we boosted its score.

We can even further enhance this query and assign different boost factors to all of the three fields. That way we can build very powerful searching capabilities into our applications.

The post RavenDB: Field boosting with full text search appeared first on Daniel Lang.

Now this doesn’t work anymore in Windows 8!

First problem is the absence of the Compatibility tab. Second problem is that once you managed to make the tab re-appear using a registry hack it still doesn’t work quite well when you have Visual Studio 2012 installed along with Visual Studio 2010. There is also an extension called VSCommands which promises to make it work, but it did not in my case.

Here is a simple way to make Visual Studio always launch as Administrator in Windows 8:

1. Go to HKEY_CURRENT_USER\Software\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers
2. Add a String Valueand set its name to “C:\Program Files (x86)\Common Files\Microsoft Shared\MSEnv\VSLauncher.exe”. Set its value to “RUNASADMIN”.
3. Do the same again but use “C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\devenv.exe” as the name.
4. Done! Now every time you start Visual Studio it runs as Administrator!

The post Always run Visual Studio as Administrator in Windows 8 appeared first on Daniel Lang.

The free plan is limited to 15MB and it has a hard storage limit, which means that you will get exceptions on write operations as soon as you hit this wall.

I don’t want my visitor to be the first to notice that when they try to leave a comment. I also don’t want to check the size regularly on my own. What I want is an email that tells me I should enter my credit card details and upgrade to a paid plan as soon as my database reaches 13MB or so.

To do so, I decided to check the database size upon startup and send myself an email when some threshold is reached.

Admittedly, it took me some time to figure how I can query the current size of the database… Here is the code – hopefully it will be useful for someone as well:

Edit: Jonathan Channon contacted me and told me that he has already done the same before. Actually, he has a more advanced approach and deletes old documents to stay withing the limit (I don’t want to do this but it’s interesting). Take a look at his blog post here: http://blog.jonathanchannon.com/2012/09/21/nancyfx-ravendb-nerddinner-and-me/

The post Query database size in RavenHQ appeared first on Daniel Lang.

Setting up TeamCity was quick and easy, after a couple of hours I had all our projects working and guess what, builds that took more than 5 minutes before now finished within a minute or two. This was all amazing, except that we had no good backup anymore. TeamCity doesn’t provide a way to schedule automated backups but it has a REST API that one can use to build his own automated solution.

I have hacked together a small command line tool that you can use to automate TeamCity backups using its REST API. In addition to that, I automatically ship the backup files to Amazon S3. If you want to do the same, read on, I’ll show you a few simple steps.

Here is, what you need to do:

1. Configure the default TeamCity backup path by edition the backup-config.xml file in %TEAMCITY_DATA_PATH%/config/. Let’s use something like C:/teamcity-backup
   
2. Create a backup user with administrative rights. This will be the one we will use to authenticate over http later.
3. Go to https://github.com/mangomint/TCAutoBackup, clone and compile the command line tool. Now copy the files TCAutoBackup.exe and TCAutoBackup.exe.config to a new directory on your build-server, let’s say C:/teamcity-backup-tool.Make sure .NET Framework 4.5 is installed on your build-server. 
4. Edit the TCAutoBackup.exe.config file to your needs. Here’s mine:
   
   Don’t forget to run the program once to verify that everything works.
5. Now, let’s schedule the execution. One option is to use something like Windows Task Scheduler or whatever other enterprise scheduling system you prefer. Another option is to use TeamCity itself, which is what I did: Just create a new project and use the Command Line runner to run the TCAutoBackup.exefile. In addition to that, you will want to create build trigger that schedules the backup to build on a regular basis (i.e. once a day). 
6. Last thing to do is to setup offsite-shipping for your backup files. Some prefer to carry around external storage drives, others (like me) are lazy and want this to be done by someone else, preferably a computer program. I personally like CloudBerry Amazon S3 Backup. It’s easy to use and let’s you configure real-time monitoring for a folder, so that every time a new backup file is created it will ship it to your S3 bucket automatically. What is nice as well is that it sends you an email afterwards.

Congrats, you should now have automated your TeamCity backups!

Just keep in mind: You don’t have a backup until you successfully tested the restore procedure!

The post How to automate TeamCity backups using its REST API appeared first on Daniel Lang.

Pay extra attention when you use NuGet package manager and update a dependency that is used in more than one of your projects!

So, why is it broken? Let me explain…

This is our solution structure:

Both highlighted projects use the NuGet package “RavenDB.Embedded”. Recently we had an issue with a customer that was fixed in build 2024. So, I used the package manager window to update to RavenDB build 2025:

I then pushed the commit to our repository and TeamCity started the psake task to build and deploy the package. This worked great – the issue was fixed!

Since then (about 2 weeks) we shipped about 30 new builds of our software and had no problems. However, the issue re-appeared today and it was exactly the same problem that we had back then.

After some headaches I found out that for some reason we were deploying RavenDB version 2005 again, although the NuGet package clearly was on build 2025!

I have to admit – it took me some time to figure this… the problem was that NuGet update works per project and not for the whole solution! So, while one project was on version 2025, the other still was on 2005:

I can’t get my head around why I would want to update only one project but leave the other untouched. This makes absolutely no sense to me and apparently it causes some issues with msbuild as well (was shipping the newer version before but then returned for no reason).

I hope that this weird behavior will be fixed with an upcoming NuGet version (this was 2.0), but until then we must remember to update NuGet packages in every project of the solution!

The post Why NuGet package update is seriously broken appeared first on Daniel Lang.