Best Guess Theory

Awesome 3D framework for XNA?

Hudson.Akridge@gmail.com (Hudson Akridge) — Wed, 17 Feb 2010 16:42:00 +0000

Filing this under things Tuna has recommended me that I don't have time to check into yet, but which look pretty friggan sweet. I'll update later with anything I find on it. Or you guys can tell me if you've had any experience with it ;)

Converting CHM to ePUB (and a e-reader review)

Hudson.Akridge@gmail.com (Hudson Akridge) — Fri, 12 Feb 2010 22:16:00 +0000

Just got my Astak Ez Reader Pocket Pro 5", and I love it. It's the perfect size, solid resolution, has all the features, all the document support you could ever want, is very fast, has 3 different places to change the page, including link navigation, supports up to a 16gb SD card (aka a metric assload of books), no bullshit DRM and has a battery life measured in months. That's right. Months. This thing just doesn't die. It's brilliant. I've been raving about it for the week or so that I've had it now.

So on to the issue. I have a few technical books as CHM files (Old school, right?), and with the advent of ePub as a standard e-reader format, I wanted a way to convert them into some spiffy epubs so I can take advantage of text reflow (which is also supported in PDFs on the Astak). ePub is an amazing format. Low size, fantastic orientation, and chapter support. Plus, it's a standard document definition, unlike PDF, which is all over the place. It's also quite a good bit faster than PDF to navigate and turn pages on. Faster in the e-reader world typically means less processing, and less processing means longer battery life (plus a happier reader not having to wait as long for the page to turn).

Step1: CHM to HTML.

You have to get the CHM into HTML somehow. There's a few tools to do this. My favorite is ABC Amber's CHM Converter. It does cost, but it's worth it if you've got a lot of CHMs. If you don't, you can still use the trial and get away with not having watermarks crapped out all over the result (I'll explain in a few). You can also use the HH.exe in windows (if that's your platform of choice) to decompile the CHM out into a messy website in a folder of your choice (HH.EXE -decompile C:\Temp\decompile-folder C:\Temp\yourCHM.chm). You can typically convert in one of two ways:

A single HTML file
Multiple HTML files (website)

If you go option 1, it's much easier to track and manage, however because of the way ereaders work, navigating pages and chapters will be a huge pain. It'll take quite a few to turn to the next page and 10-15 seconds to navigate chapters on a 600 page book. Option 2 makes turning pages and chapter navigation significantly faster, but involves more work on your end to manage.

We want to go option 2, this is the best speed, and that counts for a lot on an eReader. Now that we've got an output directory containing our "website" we want to investigate the dir. Inside, you should find a single HTML file and a folder of the same name. Open up the folder and go to the images directory (or something similar). Remove any superfluous images such as arrow navigations or whatever else might have been included in the CHM (like header seperators and such). Feel free to also remove any images that you feel might not be of a significant benefit to you reading the book.

Now, look for a main.html or go up a level and look at the html file in the root folder. What you're looking for is the HTML file that acts as the Menu. The menu HTML file is important because it points to all of our chapters/pages for use during the next stage. It's also the place that if you used a trial of Amber CHM, you can do a search and replace for "[Trial Version]" and replace with an empty string and remove all of the "watermarking" that's done during the trial version conversion process. At this point, it's up to you to weed out any chapters you don't want to include in your ePub, such as appendixes, which can help to reduce the "noise" you'll see when navigating the ePub's menu file. Save the file if you've made any changes and continue on to the next step.

Step 2: HTML to ePUB

Going to introduce you to an awesome piece of free OSS software. Calibre. This converts from almost any format into an ePub, and it's great at it. If you're using a single HTML file from step1, then all you need to do here is "Add book" point it at the HTML file, and then "Convert To" and specify ePub as the target source. Done. However that's not what we've chosen to do if you're following along. So when you go to "Add book" in Calibre, you need to select *only* the MENU HTML FILE that you created in the first step. This file is going to tell Calibre where all of the rest of your "chapters" are.

Give Calibre a few moments to import and create a zip containing all of the referenced files. When it's completed, you should see the book appear in the list (hopefully with a size > 0 next to it if you've done it correctly). Right click and go to "Convert". Choose ePub as your output. I'd also recommend deselecting the options for a title page/image. Also select the option in the Menu section for Calibre to generate it's own menu file (it'll clean it up a bit).

Convert. You can now copy the converted ePub to your eReader (which is hopefully an Astak! =p ). Where is it? It's in the Calibre directory created for managing your books. Can't find that? Right click on the book in the Calibre list and go to "View source folder". Viola.

Now you've got a lightning quick ePub file out of a single CHM. Hopefully this helps a few people out :) Good luck!

The NHibernate Dereference pattern

Hudson.Akridge@gmail.com (Hudson Akridge) — Fri, 05 Feb 2010 22:28:00 +0000

Sometimes, NHibernate associations are so complex that relying on cascades is not always possible or optimal. Other times, you need to ensure that your session deleted object is actually removed from your in memory reference model (e.g. The loaded entities which still reference it) without having to refresh your objects from the database. I ran into this a while back when dealing with a bi-directional ManyToMany in which neither side would allow an all-delete-orphan cascade, and created a variant of the Disposable pattern to resolve it.

I'm sure that other developers implement something similar during their delete process, but I haven't run across anything formally defined. This keeps coming up, and after helping several FluentNHibernate & NHibernate users out by giving them direction I decided to do a writeup. I'm going to try and state the pattern on here for your reference and feedback. Disclaimer: I'm not saying it's perfect, or that there's not something better, I'm just saying it works for what we need, and we haven't found anything better yet.

Intent:

Called before an NHibernate Session.Delete(). Disassociates the entity from the domain model, removing all references to the entity, thereby allowing NHibernate to generate the appropriate UPDATE/DELETE statements in the database, in the order required, so there's no Foreign Key Violations. This pattern works with inheritance.

Implementation:

   1:  public interface IDereferenceable

   2:  {

   3:    void Dereference();

   4:  }

5:

   6:  public abstract class PersistedBase : IDereferenceable

   7:  {

   8:    protected bool IsDereferencing;

9:

  10:    public void Dereference()

  11:    {

  12:      Dereference(IsDereferencing);

  13:    }

  14:    protected virtual void Dereference(bool dereferencing)

  15:    {

  16:       if (dereferencing)

  17:       { return; }

  18:       IsDereferencing = true;

19:

  20:       //Dereference Children

  21:       foreach(Child child in Children.ToList())

  22:       {

  23:            RemoveChildCore(child);

  24:       }

25:

  26:       //Dereference Parents

  27:       if(parent != null)

  28:       {

  29:         parent.RemoveChild(this);

  30:         parent = null;

  31:       }

  32:    }

33:

  34:    private ICollection<Child> children;

  35:    public IEnumerable<Child> Children{ get { return children; } }

  36:    public void RemoveChild(Child child)

  37:    {

  38:        if(!IsDereferencing)

  39:        {

  40:            RemoveChildCore(child);

  41:        }

  42:    }

43:

  44:    protected void RemoveChildCore(Child child)

  45:    {

  46:        children.Remove(child);

  47:        IDereferenceable deref = child as IDereferenceable;

  48:        if(deref ! = null)

  49:        {

  50:            deref.Dereference();

  51:        }

  52:    }

  53:  }

54:

  55:  public class Concrete : PersistedBase

  56:  {

  57:    protected override void Dereference(bool dereferencing)

  58:    {

  59:       if (dereferencing)

  60:       { return; }

  61:       IsDereferencing = true;

62:

  63:       //Dereference Children

  64:       foreach(StepChild stepChild in StepChildren.ToList())

  65:       {

  66:            RemoveStepChildCore(stepChild);

  67:       }

68:

  69:       //Dereference Parents

  70:       if(stepParent != null)

  71:       {

  72:         stepParent.RemoveChild(this);

  73:         stepParent = null;

  74:       }

75:

  76:       base.Dereference(false);

  77:    }

78:

  79:    private ICollection<StepChild> stepChildren;

  80:    public IEnumerable<StepChild> StepChildren{ get { return stepChildren; } }

  81:    public void RemoveStepChild(StepChild stepChild)

  82:    {

  83:        if(!IsDereferencing)

  84:        {

  85:            RemoveStepChildCore(stepChild);

  86:        }

  87:    }

88:

  89:    protected void RemoveStepChildCore(StepChild stepChild)

  90:    {

  91:        stepChildren.Remove(stepChild);

  92:        IDereferenceable deref = stepChild as IDereferenceable;

  93:        if(deref ! = null)

  94:        {

  95:            deref.Dereference();

  96:        }

  97:    }

  98:  }

Let's go down the list:

Our abstract class/interface. This is the base class that handles basic dereference association and implements the IDereferenceable interface.
IsDereferencing - This is similar to the IsDisposing/Disposed flag on IDisposable objects. It's a way to ensure that we only Dereference an object once. This is only set within the protected Dereference method if it's passed in a !dereferencing parameter.
Dereference() - Public no-parameter. This is what is called by by the code that's about to delete the object. It's also called by other classes whenever they want to dereference this instance as a child reference. Passing in the current IsDereferencing flag prevents us from getting into a cyclic call if it's called several times by many parent classes.
Dereference(bool dereferencing) A.) We early escape if our dereferencing parameter is true, so we don't perform any dereference logic more than once B.) Dereference children first. Children are typically going to be defined as collections, but there are always exceptions. C.) We enumerate over a copy of the collection. Reason? We're going to get a collection cannot be modified during enumeration exception otherwise. D.) We call the private RemoveChildCore(child), which does not do a dereferencing check (otherwise nothing would happen) E.) We check first to see if Parent is null, this is primarily here for situations where a Parent is allowed to be null, or during testing where you may not popular all the parents of an entity F.) If parent is not null, then we call the public RemoveChild(this) method on the parent so we're no longer referenced in a parent collection. Important to note, that RemoveChild() on the parent does perform a !IsDereferencing check on the parent, just like we have for the child. After all, the Child's Dereference() could have been called by the Parent's Dereference(), so we want to respect this. G.) We null out the parent
Our collection is exposed out as a read only collection, which tells the user that we do not allow adding/removing directly on the collection. Good practice and safety precaution if you have additional association logic, which you would put into the AddChild(Child child) method (not implemented here for the sake of simplifying the example).
The public RemoveChild(Child child) checks if we're dereferencing, if we are, then we skip actually removing the child. This is there so that when children are being dereferenced, and call parent.RemoveChild(this), they do not enter a cyclic loop. If not dereferencing, call Core dereference.
RemoveChildCore(Child child) has no checks, and is private, so it should be tightly controlled by the class. This will remove the item from the private collection, cast the child as a dereferenceable, and then if the child is dereferenceable, it will proceed to call Dereference on it. NOTE: that child.Dereference() should only be called if the child dies with the parent, so if the parent is dereferenced, the child should be as well. in some cases, this is not the case, and there might only be an incidental association between them, such as is the case with a ManyToMany() association. In this case, dereferencing the child should be avoided (perhaps though you need to keep the two lists synchronized in the MTM example, so you'd call child.RemoveParents(this) in place of the child.Dereference() )
The Concrete implementation of the base class. Here we have a similar setup as the base class, except we've got additional information that needs to be dereferenced (another collection and parent, defined as StepChildren and StepParent)
Dereference(bool dereferencing). Our setup here is the same, and can be referenced from 4 A-G. However, it adds a new element at the end: H.) We call our base.Dereference(false) to continue up the dereference stack. Passing it false is important, since we know we're in the middle of dereferencing and should override the dereferencing check. If we passed the IsDereferencing field, the base Dereference would not fire, meaning that any references specific to the base class would still hold a reference to our class. As we're overriding, polymorphism dictates that our bottom level class (most concrete) will have Dereference called on it first. So it Dereferences from most specific to most generic (Covarience).

Usage:

   1:  using(var session = sessionFactory.OpenSession())

   2:  using(var transaction = session.BeginTransaction())

   3:  {

   4:    var concrete = session.Get(id);

   5:    concrete.Dereference();

   6:    session.Delete(concrete);

   7:    transaction.CommittTransaction();

   8:  }

So there you go, a way to manage the disassociations with NHibernate (or another ORM) and your object model. Hopefully this helps someone out, and if it doesn't, feel free to leave a comment if you've got any critiques or questions. Always open to feedback! (Also wrote all of the code in notepad, so if anything's off let me know and I'll correct it)

CodeMash 2010 Session Materials

Hudson.Akridge@gmail.com (Hudson Akridge) — Fri, 15 Jan 2010 18:37:00 +0000

Just finished my second session at CodeMash. Was a good time, and had a pretty packed house. As promised:

Here's my session materials

Thanks for attending, and CodeMash has been pretty amazing. Will come to Ohio again ;)

EAV in OO/NHibernate: Part 1 - Intro

Hudson.Akridge@gmail.com (Hudson Akridge) — Thu, 31 Dec 2009 16:12:00 +0000

I've been talking to a decent number of developers about the system that Robert and I are writing at GFX, and I'm met with many blank stares every time the acronym EAV pops up. I thought it might be worthwhile to try and explain what is probably the most double edged sword in data modeling I've ever run into. If you do a search for EAV problems, you'll get 99 people out of 100 telling you one horror story after another about implementing/using it in production systems. They're not crazy, or wrong. EAV architecture is a very tricky beast to tame. Unfortunately, there's not an alternative I've found out there that gives the flexibility that an EAV system offers in an RDBMS. Knowledge is power, and in order to tame the beast, you first must understand it.

EAV (Entity Attribute Value) systems have been around for a good long while, if you're unfamiliar with the concepts, check out that wiki link, and I'll do my best to sum in about a paragraph or two here:

Basically, it takes the concept of a Table, with Columns describing the table's data and a row's context, and pivot's everything. You have your standard Entity that you're describing (a Car for example), which you use rows to describe your columns (the attributes, eg. Size, Color, WeightInLbs, # of doors, etc), and a matrix of rows to describe an individual Entity's attributes (Values, eg. Midsize, Red, 2210, 4). In a RDBMS, it's easy for users to create rows, but difficult (and almost always a bad practice) for them to create columns/tables. Using EAV offers a pretty high degree of flexibility within an application, but it's not without it's downsides: Performance, Deadlocking, and an increase in Complexity are the front runners.

There's two types of EAV systems: EAV, and EAV/CR (Class Relationships). The first, is the type of EAV system you get from most modern ecommerce sites, where you can add any number of attributes to one of your products and they're all arbitrary and weakly typed. This is the simplest implementation, and (my guess) the most common. The second is where the rabbit hole opens up. In that scenario, an attribute is given a more formal definition and contains meta attribute information. This is the EAV type I'll be talking about. A good read on EAV/CR can be found here.

The large project I have been working on at my day job, is essentially an EAV/CR system for our clients. I'll quickly state the most basic requirements of the system:

Allows users to define their own Attributes, and strongly type them both in the domain and in the database.
Theoretically needs to support unlimited number of attributes a user can create to describe their entity.
Different "sets" of EAV data, so one user could have completely different data that is not shared between the context boundaries. (Context boundary could be anything, but typically is tied to each client, so every client has their own sets of attributes for entity types)
Multiple Entity types, with their own unique attributes
Extended attribute functionality that recreates many RDBMS column properties (eg. Uniqueness, Formulas, Bind able Context menus, etc.)
Must be easily searchable
Must allow for fast and scalable operations

These requirements are not without their own basic implementation challenges, but those last two bullet points, in my experience, are where the majority of the complications in building a system like this arise. EAV/CR's suffer from an exponential degradation in performance. n attributes with x entities produces nx=(v)alues. With 4,000 entities of the same type, and 200 attributes for each entity, you end up with 80,000 value rows. That's just for one entity type for one contextual boundary. Extend that with 50+ clients, and 10 entity types (80000 * 50 * 10), and you can quickly see how your v table gets out of control.

The complexities of re-implementing your own database in code causes searching to become an exercise in pivoting and endless sub-queries forking for each attribute data type you expose. Full text indexing goes from being a checkbox in most DB engines to a caching like framework you inject into your system.

In later parts of this series, I'm going to talk about some implementation challenges and a few ways I've found to overcome them in OO and NHibernate.

Join Fetch on NH queries returns dupes

Hudson.Akridge@gmail.com (Hudson Akridge) — Thu, 05 Nov 2009 21:26:00 +0000

Domain:


public class Parent
{
  public ICollection<Children> Children {get;set;}
}
public class Child
{
  public string Description {get;set;}
}

Say we want to retrieve all the Parents in HQL, but we want to join fetch to get children simultaniously, so we don't lazyload children. In my case, I knew I'd be working with children immediately after retrieving Parents. The HQL would be something like this:

SELECT p FROM Parent p JOIN FETCH p.Children

Nothing fancy. When we execute that HQL query, we're going to get a Parent row returned to us, for every child for every parent. So if we've got 3 Parents, and each one has 2 children, we will have 6 rows returned from that statment.

Each row will have all of the attributes of Parent on it (as columns), and it will also have all of the attributes of child (as columns). This is how NHibernate ensures it retrieves both objects in the database simultaneously.

So, after you execute the query, NHibernate gives you a list of 6 Parents. Each parent has a duplicate of itself in the retrieved list. Why? This is because that's what was returned, for rows, when you asked for Parents. You're getting both Parent and Child, but you're only selecting the Parent as the result from the query, NHibernate is smart enough to preload each Parent's collection of Children so when you attempt to access the collection, NHibernate does not need to go back to the database to load the collection.

So, my problem was, how to get just 3 Parent's back, instead of 6. Well, a few people, might suggest using DISTINCT in your HQL, so it'd look something like:

SELECT DISTINCT p FROM Parent p JOIN FETCH p.Children

Execute that, and you'll still get 6 Parent's, each parent, along with it's dupe. Why? Because DISTINCT is a SQL based syntax, and those rows are not dupes. Only the objects that are hydrated in the object model are. So you're not going to have any luck there.

What are your options?

1.) Use the results transformer:

query/criteria.SetResultTransformer(new DistinctRootEntityResultTransformer());

The result transformer should work after NH has retrieved the objects back from the database, and will attempt to do a comparison with your root entity (I believe on the ID) to determine if they're the same.

2.) Do it yourself in Linq with results.Distinct();

Hopefully this helps explain the situation a bit for a few people out there that might be experiencing the issue.

It's been a while

Hudson.Akridge@gmail.com (Hudson Akridge) — Sat, 17 Oct 2009 23:52:00 +0000

Things have been busy, and apologies all around for not updating this as often as I should.

I'm doing an NHibernate presentation at the Strange Loop conference in St. Louis next week, although Robert will not be co-presenting contrary to the session description. Hopefully, after I return from there, there will be more time for me to continue the NH/FNH tips and general updates. (I will end up making a post with the session materials after the session however)

Since my last post, FNH 1.0 has come out thanks in large efforts to James Gregory, Tuna's gotten into XNA and found several physics frameworks for it that I need to check out and make a few XNA related posts, and NH went to 2.1 officially.

I've also just learned that my FNH session has been accepted at CodeMash for January, so I hope to see anyone I don't see in St. Louis next week, there.

Until then!

Video: Fluent NHibernate session from Chicago's Alt.NET

Hudson.Akridge@gmail.com (Hudson Akridge) — Thu, 16 Jul 2009 13:04:00 +0000

Sergio posted the video of my session from 7/8.

http://chicagoalt.net/event/July2009Meeting060withFluentNHibernate

This one was a good bit longer than the one from Chicago Code Camp, due to no hard time restrictions. Thanks Sergio!

NHibernate Tip #3 - ID's

Hudson.Akridge@gmail.com (Hudson Akridge) — Thu, 09 Jul 2009 13:45:00 +0000

(Prequalifier, this is still a subject of much Debate, please toss in your thoughts into the comments. These are Tips that I've run across that I wanted to share, it's not to say that this is the best way to do something every time, and should not be considered the catch all solution)

A subject of much debate, especially if anyone in the debate happens to have anything resembling a background as a DBA. I'll come right out and say what I believe to be the most ideal solution in a non-legacy environment that isn't geared towards squeezing the most performance out of the database as possible:

   1:  <id name="Id"

   2:          column="PrimaryTableID"

   3:          type="System.Guid">

   4:        <generator class="assigned" />

   5:  </id>

   6:  <version name="Version"

   7:               unsaved-value="0" />

I would also recommend to have all of your objects generate the ID in the non-default constructor (you typically want to leave an empty default constructor for NHibernate for it to load an object back from the database without any "Object initialization" logic)

The above mapping is, in my opinion, the most flexible ID setup to use with NHibernate. Why?

NHibernate never has to go back to the database to "find out" what the ID is for the object, which is a very expensive operation on an Identity generation setup. A Hi-Lo setup would be a decent second choice to address this concern
Assuming you generate the ID in the non-default constructors, the ID of an object created anywhere is going to be consistent, whether you're outside of a session (such as in model tests), or in a session using hydrated objects
It significantly simplifies the equality checks you're required to do. As someone commented in my previous post, if you're unable to rely on having a unique ID at all times, your equality logic has to start incorporating additional rules to account for the pre-persistance case scenario's
Version manages when an object should be inserted through a cascade or not (as opposed to an unsaved value attribute). Version is also handy to have for other reasons that I'll bring up in a later post.

A couple of cons:

Looking at Guids in the database is not a pleasing experience, but really, you shouldn't be interacting with the database at that level. It's an implementation detail.
Using Guids in the database can increase DB fragmentation, but really, that's where I feel the Database Engine Tuning advisor (if you're using MsSql) or a DBA steps in. There are ways around this, and as far as we've been able to tell with millions of rows in a production application, the differences seem to be almost un-noticable. Still, there's no way to get around it, it's not the best performing scenario on the DB side.

To be honest, I was originally on the side of continuing to use Identity, although I was looking at a hi-lo generator algorithm with it. My background is in a more DBA-ish field, so the concept presented above was like garlic to a vampire.

Times, they be a-changin. We need to let go of some of our old DBA habits in favor of emerging frameworks/concepts which can hide persistence complexities.

NHibernate Tip #2 - Equality Overrides

Hudson.Akridge@gmail.com (Hudson Akridge) — Wed, 08 Jul 2009 13:43:00 +0000

Override Equals, GetHashCode, and the !=/== operators - This is pretty basic. If you're using NHibernate, as a general rule of thumb for a best practice, you should override all of these functions. Why? Because by default your applications will attempt to do a referential check to determine if they're equal, or if the object already exists in the collection. Implementing your own routine allows you to check the business properties of an entity, such as the ID or business data, to determine equality. To put it simply, NHibernate won't work correctly if you don't do this.

Overriding !=/== is important when you've set your associations to lazyload, because NHibernate will generate proxies for those objects to support the lazyloading concept. Those proxies rely on these two overrides to determine object equality based on business rules (again, such as the ID/business rules).

Also important to note, testing the ID property of a lazy loaded (proxied) object does not load the object from the database. The ID property is free, that's how the proxy knows about which entity to load from the database. In an ideal world, you should try and work all of your equality rules to use the ID so you can take advantage of all of NHibernate's optimizations.

NHibernate Tip #1 - Lazy Loading

Hudson.Akridge@gmail.com (Hudson Akridge) — Tue, 07 Jul 2009 14:40:00 +0000

I wanted to jot down all of the NHibernate tips and tricks that I've learned over the last year of using this thing in production, so this is everything I can remember up to current. Every day until I run out of things I've encountered, I'll be publishing another tip.

Lazy Load Everything - Unless working in a remoting type of environment where you're not guaranteed a session while working with the objects, it's best to let this default to true for all of your entities. I've been burned by this hard, there's a fundamental misunderstanding that I had when starting to pre-optimize (also a bad idea). It went something like this: If I don't lazy load an entity, my application will perform better, because it will have eagerly fetched the data that I will use 80% of the time.

The problem is that lazy loading does not eagerly fetch anything for you. All it does, is execute the same SQL statement to get the object, that it was going to when you eventually attempted to access it, except it's going to do that 100% of the time now. Setting this option to false does *not* improve performance in any way, in fact, in every case, it will actually degrade performance. This is a feature, that as far as I can tell, should only be changed to false when you've got an object graph that loses scope from the session (such as in remoting).

What most people are looking for when they set this to false on a mapping, is the "fetch=join" attribute. This will include the associated entity in the same SQL statement used to get the originating entity. I'll probably make another post about the join fetch, all I can say for now is to use that sparingly. It can improve performance, but should be set at a more granular level (A Criteria or HQL query), as opposed to the entirety of the application.

FNH session from Chicago Code Camp

Hudson.Akridge@gmail.com (Hudson Akridge) — Mon, 29 Jun 2009 03:39:00 +0000

The sound is horrible in all of these, I apologize. I've gotten several requests to upload it anyhow regardless of the sound/camera issues. Watch at your own risk ;)

http://vimeo.com/groups/18895/videos/sort:oldest

They're all broken into about 10 minute chunks, except for the last one, which was just the last 15 minutes.

NHibernate EAV system search performance results

Hudson.Akridge@gmail.com (Hudson Akridge) — Mon, 15 Jun 2009 16:09:00 +0000

We've been working on a pretty large EAV system this past year, and we're ramping up to get some real clients in it. One of it's features is a search builder that allows the end user to search on any combination of values on the entity that they've defined as attributes, and also any class specific attributes that are hardcoded on the class.

SearchBuilder's great, it works well. The problem we've had with it is performance, specifically relating to scale. E.g. 10,000 entities, with 1,000 attributes each, would be 10,000,000 rows for a user to search through, and that's just for one grouping of entities, there would be more rows than this that are actually in Sql.

Currently, we were doing a subquery of these using NHibernate's criteria using a projection with a PropertyIn. This resolves out to be an IN query in SQL, so something like this:

   1:  this_.EntityID in (SELECT this_0_.EntityID as y0_

   2:  FROM   Values this_0_

   3:         left outer join Attributes attr1_

   4:           on this_0_.AttributeID = attr1_.AttributeID

   5:         left outer join ValueEntities owner3_

   6:           on this_0_.EntityID = owner3_.EntityID

   7:  WHERE  this_0_.DiscriminatorValue = 'NumericValue'

   8:         AND attr1_.Code = 'acd3edf4-6d11-4747-8383-03a70174d72e' /* @p6 */

   9:         and attr1_.OwnerType = 'EntityType' /* @p8 */

  10:         and this_0_.NumericValue > 1 /* @p9 */)

This allows us to match entities that have values, and then check to see if the entity exists in the subselect. Now, using the IN operator is traditionally a DBA's worst nightmare, and since we were hitting some performance issues, I opted to attempt to refactor this away from a subselect after refactoring the domain model restrictions that prevented me from searching directly on the collection association in the first place. This moves into a JOIN model for each value type association (There are 4 value types in the system at the moment, int, string, datetime, bool; each strongly typed in the database, so we hold 4 different collections on the entity to associate their 4 value types so the Value property is strongly typed. The searchbuilder only creates the joins required to fulfill the search, so if only an int is being searched on, only that association will be made)

Thinking I would nab a huge performance increase, I twittered about the performance issues with the IN that I was having, and @tehlike (Tuna) responded back asking if I had tried an Exists. I had not, and to be fair, I was basing all of my assumptions off of DBA articles I read a while back. Enamored, I decided to set up a simple performance profiling of the three different ways that I could execute my EAV find: JOIN (the new way), IN (The old way), EXISTS (The suggested way).

I set up my environment with a LocalSql2005 instance. During each execution I restarted MsSql so it wouldn't page the query (otherwise the second time the query runs you'd get it back in 1-2 seconds). This in no way is a thorough performance test, or even the most ideal. These numbers should be taken with a grain of salt at best. But they do give some general indications of performance comparisons. Here's the results:

Query restriction: A single attribute value on the entity being > 1.

Baseline data: 1566 rows on the left side (entity side), 141119 rows on the right side (value sub select)

JOIN: 7s
IN: 7s
EXISTS: 8s

So, right away, the JOIN didn't give me the performance improvements that I had thought were coming. At this point, I started to get somewhat worried of what was going on behind the scenes, and if I had done something wrong. I relooked at the mappings, and the sql generated by each, and it all seemed to match up to what would by typed by hand. Next I bumped up the size of the data by 4x:

Query restriction: A single attribute value on the entity being > 1

Baseline data: 6264 rows on the left side, 564476 rows on the right side

JOIN: 26s
IN: 30s
EXISTS: 31s

Join seems to perform a little bit better, but only slightly. On a whole, the performance improvement is underwhelming. Now I complicate things by changing up the query and adding an additional attribute value to be searched on of a different type (Another join to the specific value type collection for the JOIN, and another sub-select just referencing a different value column for the IN/EXISTS)

Query Restriction: Single value on entity being > 0, and a text value being != 'NotAValue'

Baseline data: 6264 rows on the left side, 564476 rows on the right side

JOIN 35s
IN 32s
EXISTS 31

Wait...what? Join ended up slowing down significantly as you add different joins to the various value type collections and additional criteria values, wheras the subselects did not really, because SQL had already executed the sub-select for the first value, and the execution plan was already optimized when the second value subquery was being asked.

Grant you, these numbers fluxuated by a second or two when run multiple times, I'm giving the average. But IN/EXISTS seems to scale better than JOIN, in terms of adding additional restrictions, and additional value type associations.

In terms of row data scaling, the IN and Exists seemed to hold their own admirably to the JOIN, and in any complex query scenario, performed better.

So this brings me to the final decision: We're probably going to go back to the JOIN, and may look at tuning that to an EXISTS if we run into future performance issue. It's also important to note that indexing/statistics in MsSql plays a *HUGE* role in the table performance. When the properly tuned indexing was enabled for the database, all three options went to sub second response times. So in the end, why quibble about the 1% syntax performance tweaks of a problem that is 99% solved by proper database tuning.

Anyone else have anything to add? I know it's not a large amount of test data, with a stable environment, but discounting that, I feel much better about using IN's/EXISTS now.

UPDATE: Robert and I talked to Ayende at length about this (Another quick round of thanks to Ayende for being willing to dedicate the time).

It's looking like RDMS is definitely the wrong way to solve this problem. Looking at a document database, like Couch DB, or Ayende's own Divan DB. Alternatively, we can try and get something going using MsSql's Xml document column type.

Of course either of those options pose huge headaches in dealing with the search options we're giving to the client, so we're also looking at utilizing Lucene.NET to perhaps manage that.

Something tells me I'll be posing some blog entries in the near future milestoning our journey.

Why do we take on side projects?

Hudson.Akridge@gmail.com (Hudson Akridge) — Thu, 11 Jun 2009 15:51:00 +0000

I currently contribute to the following OSS projects:

Fluent NHibernate
NHibernate Query Analyzer
And a couple of other personal at home projects involving XNA.

I have a full time job, a wife, and in a band. So why do I spend what little time I have working when I get home, much less for free? For me, it's simple. I get the opportunity to give back to the community, the pride in being a part of a great team of developers who are creating fantastic products, and the chance at incredible learning opportunities.

But that's not enough is it? For many of us, programming is a hobby in the evenings. It's a way to do things the way they were *meant* to be done, without all of the bureaucracy and red tape that typically clutters up our day job. I mean, this is really a chance to flex our muscles, see what we can do when the restraints are taken off.

Some of us go barreling into the wall at 100mph , but in general, it's a way to get back to the roots that drew all of us into programming in the first place. Fun. Doing something, just because you can, and it'll make someone's life better as a result. Pushing the limits of terse code. Seeing just how fast you can actually get that cycle to complete in. Creating that framework that lets you plug in third party modules like home appliances. It's all great fun, and in my opinion, the way everything should be.

So what went wrong? Where did we go wrong during our day jobs? Why is it so difficult to find a company that's doing software development the fun way? With so many software engineers being hired out, why is there company after company that has one codethulu setup after another? Where'd the fun go?

The false prophets took it away. Who am I talking about? Anyone who got into development for the wrong reasons:

They were in sales 15 years ago and had some skills with spreadsheets at a (then) small company and became the "lead technical" guy
They went to school, didn't know what to do, and were told by the campus counselor that software was a growing industry and needed more warm bodies
The old has been that grew up knowing one language, and one way to do something, and never changed (to a man with only a hammer, every problem appears to be a nail)
Warm bodies trained to pump out code which gets charged per line (aka many "call center" developers)
The cousin of the president of the company, who's always been "Good with computers"

Generally these people have the best intentions, sometimes they don't, and they fight learning new processes and tools like a blind man sitting on a key lime pie (this is my blog, I'll make up whatever ridiculous simile's I feel like). So where did they go wrong? If we're all trying to achieve the same thing, why is it such a common obstacle to overcome to develop software the Right Way™ ?

What are your stories and reasons for getting into development, and if you're on any passion projects, what were the reasons that led to you being involved?

Chicago Code Camp session video: Update

Hudson.Akridge@gmail.com (Hudson Akridge) — Sun, 31 May 2009 21:56:00 +0000

Well, after ripping the DV, I learned a couple things about recording sessions yourself.
1.) Try and find a cheap wireless mic to feed into the audio port, because in a decent sized room, even though everyone in the room could hear me well, the camera in the back picked up the AC unit that was running next to it far, far better than it did anything coming from the front
2.) Make sure the projector screen is right next to you or behind you. Sadly I couldn't change this one, there were a couple rooms in the conference with the projector screen on the other side, and I was in one of them
3.) Make sure you get tape that fits the minutes your presentation is for. 75 min presentation, but 60 min tape. So lost 5 minutes at the 60m mark when the camera operator changed it out
4.) Make sure whoever is doing the camera knows to go to the code instead of zooming in on you whenever you're in the process of writing code, this was my fault. It was asked of me what I wanted to do, and I didn't have an answer.

I've got to fix the audio though, trying to turn it up and what not. You can hear the guy under the camera cough a couple times, and boy, does that ever get picked up by the mic...

Fluent NHibernate Session material from Chicago Code Camp

Hudson.Akridge@gmail.com (Hudson Akridge) — Sun, 31 May 2009 02:49:00 +0000

As promised, here's my session material

I had a great group participating in the session, so thanks group :) Hopefully it was as fun for all of you as it was for me.

I've also got the video for anyone who was unable to make it to the session, which I'll try and get up tomorrow. Stay tuned to my twitter or rss this for updates.

Apologies for the delays

Hudson.Akridge@gmail.com (Hudson Akridge) — Tue, 19 May 2009 14:15:00 +0000

I'm strapped for time right now working on the Fluent NHibernate session for Chicago Code Camp. I'll be posting more consistently after May 30th. Until then, if you're in the Chicago area, feel free to stop on by for that, and the many other fantastic sessions going on that day.

A Fluent NHibernate Tablename convention helper

Hudson.Akridge@gmail.com (Hudson Akridge) — Tue, 28 Apr 2009 22:23:00 +0000

So, one of my after hours duties is being a contributor/developer on the Fluent NHibernate project. This past week or so, I've been in the process of converting my day job company's production system to Fluent (from standard hbm mappings). In the process, I've created a couple of helper methods to assist me in standardizing table names, Primary Key names, and Foreign key names.

First, we start with the basic pluralizer:

   1:  public static string GetPluralOfText(string text)

   2:          {

   3:              string pluralString = text;

   4:              string lastCharacter = pluralString.Substring(pluralString.Length - 1).ToLower();

5:

   6:              // y's become ies (such as Category to Categories)

   7:              if (string.Equals(lastCharacter, "y", StringComparison.InvariantCultureIgnoreCase))

   8:              {

   9:                  pluralString = pluralString.Remove(pluralString.Length - 1);

  10:                  pluralString += "ie";

  11:              }

  12:              // ch's become ches (such as Pirch to Pirches)

  13:              if (string.Equals(pluralString.Substring(pluralString.Length - 2), "ch", StringComparison.InvariantCultureIgnoreCase))

  14:              { pluralString += "e"; }

15:

  16:              switch (lastCharacter)

  17:              {

  18:                  case "s":

  19:                      return pluralString + "es";

  20:                  default:

  21:                      return pluralString + "s";

  22:              }

  23:          }

I also added a method for stripping out some of the class name suffixes that are prevalent in the project:

   1:  public static string GetTextWithoutObjectOrientedConventions(string text)

   2:          {

   3:              string newText = text;

   4:              if (newText.Contains("Base") && string.Equals(newText.Substring(newText.Length - 4), "base", StringComparison.InvariantCultureIgnoreCase))

   5:              { newText = newText.Replace(newText.Substring(newText.Length - 4), string.Empty); }

6:

   7:              return newText;

   8:          }

And finally, my methods that tie it all together:

   1:  public static string GetTablename<TClass>()

   2:          {

   3:              return GetTablename(typeof (TClass));

   4:          }

5:

   6:          public static string GetTablename(Type tClass)

   7:          {

   8:              string tableName = tClass.Name;

   9:              tableName = GetTextWithoutObjectOrientedConventions(tableName);

  10:              tableName = GetPluralOfText(tableName);

11:

  12:              return tableName;

  13:          }

So far, it's worked out pretty well. All of my FNH conventions utilize these methods regularly. Anyhow, hope this post helps someone else out there :)

Presenting A New Acronym for Failure...

Hudson.Akridge@gmail.com (Hudson Akridge) — Tue, 28 Apr 2009 04:34:00 +0000

This contractor that I've mentioned before in passing, is really good at doing things the most absurdly ridiculous way possible. If there's an easy and a hard way to do something, he'll invent a new way that does both, backwards.

By now, everyone's familiar with AJAX (Asynchronous Javascript and XML), and if you're not, this blog may not be for you.

The application that was delivered to my company had a very peculiar feature. If you're familiar with .NET's AJAX 2.0 framework, then you're probably familiar with the Tab control. It's a very simple AJAX control, nothing fancy. Now, if you're also familiar with the ASP.NET webform's model of postbacks for controls to maintain viewstate, then you're familiar with all of the ingredients to the recipe here.

A rational, sane man, might develop a webpage using an AJAX Tab control. A rational, sane man, might develop another page using a series of tabs that take postbacks and pass some information along in viewstate. Our developmestruction hero took the road less travelled, and picked both.

That's right. He's using the AJAX Tab control, except, when you swap tabs, about half a second to a second after the tab swaps, the whole page posts back, and you end up on a different .aspx page. My co-worker Robert and I couldn't believe it. I sat there, stupefied at what was before me. I announced that this man had created Asynchronous Synchronous Javascript and XML.

To which, Robert turned to me and loudly proclaimed:

ASSJAX!

(ASynchronous Synchronous Javascript and Xml).

There's no escaping it. This term has become standard use in our office now. So thank you, anonymous contractor. You may have developed an indecipherable, bug ridden, amateur, spaghetti code system of evil that we'll have to maintain for God knows how long; but at least you gave us the greatest gift of all. ASSJAX.

Powered by the internet!

Hudson.Akridge@gmail.com (Hudson Akridge) — Wed, 22 Apr 2009 05:06:00 +0000

Just received some very important news!

HELLO:
Good Day,This is to officially inform you that Your email address has just won an ATM CARD worth 6.8 Million Dollars in the satellite software email lottery conducted by OCEANIC BANK E-MAIL PROMO INTERNATIONAL, In which e-mail addresses are picked randomly by software powered by the Internet., Please Contact Mr.mark Kelvin with this email (oceanicplc@8u8.com)

Tell +2347060481461.
VEREFICATION FORM
1 Your Full Name:
2 Your Delivery Address:
3 Your Age:
4 Your Occupation:
5 Your direct Phone Number:
6 Country
:
Regards,Mr David Mark
SENATE PRESIDENT

---

Indeed, Mr. David Mark. I shall be forwarding you the verefication information you requested, only it'll be for this guy I used to work with, who's name rhymes with Chris Wells.

Blackest Night...

Hudson.Akridge@gmail.com (Hudson Akridge) — Wed, 15 Apr 2009 03:59:00 +0000

There's a consultant that I've had to work with at my company, who's legitimately a nice guy, or tries to be. But regardless of that, he's a terrible coder. What he checks in is some of the most bug addled, overly complex, difficult to read and debug code I've ever seen.

I'll see if I can't get some permission from my employer to submit a couple of the "Golden nuggets" of his handy work, but for now, enjoy this brief glimpse at an enum I found of his for doing an item import. Honestly, I hope I never have to figure out how this works, but hey, at least it's commented for me...

   1:  Enum ItemField

   2:          Photo = 9999    'positive #'s are reserved for characteristics

   3:          Disgard = 0

   4:          LocalItemCode = -1    'negative #'s are reserved for Item table values

   5:          CompanyItemCode = -2

   6:          Name = -3

   7:          Description = -4

   8:          Vendor = -5

   9:          Bilingual = -6

  10:          Complete = -7

  11:  End Enum

Update: I have permission to post some of his code :) Prepare!

The best laid names of mice and men

Hudson.Akridge@gmail.com (Hudson Akridge) — Sat, 04 Apr 2009 23:05:00 +0000

What's in a name? If you're a person, it's there to establish your uniqueness within the community you live. If your name is Frank, and your father's name is Frank, then there's usually a Jr., Third, Fourth, etc attached to the end of your name so the people in your household know how to call which one of you down to supper. If you went to school with someone who had the same first name as you, it was often common for people to use your last name, or some abbreviation of your name to tell you apart. Your name establishes you as an entity.

If you lived back in times not too far gone, say starting around 10th or 11th century AD, your last name (surname) would typically describe one of four things about you:

1.) What you and your family did for a living. The Coopers were barrel makers. The Wagners were wagon makers or blue fuzzy teleporting mutants....

2.) Who your father was. Johnson, Peters, Myers, "The Mailman", etc.

3.) Something about your locality. The Ashley's lived by ash trees (not set on fire, or maybe they were, I'm sure some Ashley's were cremated at some point).

4.) Nicknames. Reid's were the redheads, the Armstrongs had strong arms, and the Cox's had...um...

You can even trace your family line back hundreds, if not thousands of years based on nothing more than your name. The point is, names are important. They help define us, and what we do.

The names of your variables, classes, and methods are no less important. In fact, they're probably more important, because they have no visual features in which to distinguish themselves by. They give context to your application. Well named objects help you define how your application works just like putting the correct words in the correct order helps someone read a book.

Naming is so important, that Steve McConnel dedicated 32 pages to just naming variables in chapter 11 of the foundational book "Code Complete", plus even more pages for naming methods and classes.

Back in 1999/98, Paul Haahr (blog here), wrote a foundationally solid article regarding naming that I recommend checking out.

Example time. Which set of variable names are easier to understand:

numBckWLinFtTtl01, numBckWLinFtTtl02, numBckWLinFtTtl03

standardStoreBackWallLinealFeet, prototypeStoreBackWallLinealFeet, remodeledStoreBackWallLinealFeet

So what's wrong with the first set of variables? Other than it used to be in use in one of the legacy systems at my company.

Uses a form of Hungarian notation (putting type information, along with other prefixes) into variable names. The truth is, there's simply no need to do this anymore. With modern IDE's, all it takes is hovering over your variable to get a tooltip telling you what type it is. All it does is add more letters to the front, that developers have to train themselves to ignore, so they can get at the real info
The abbreviations. Cripes. If I had a nickle for every time I've touched a system that was bogged down by abbreviations, I'd be able to form a stimulus package of my own for the government. If you feel like your name is too long, and should be shortened, do not "fix" it by abbreviating. There are only two times when abbreviation is acceptable. When it's a common abbreviation (such as URL, RAM, email, etc.); or when your code size matters, such as in the case of Javascript. If your reason is the second one, let me introduce you to an automated technique called Minification.
Ttl. Another abbreviation. Total is unnecessary in this case. Use generic words like "count" and "total" sparingly, if at all. In the majority of cases, they're too generic to add value to their use, and if the rest of your application is written well, the reader will easily be able to infer that meta piece of information.
01, 02, 03. One day, the people who append generic numeric suffixes to variable names, will be set on fire and become Ashleys. We could easily fill a book on why this is a bad practice; completely indecipherable intent, accidental confusion with integer constants (especially if one uses a variable name like that in an expression: numBckWLinFtTtl01+01), or difficulty of managing multiple variables that only differ by the number on the end. But most importantly, always program like the next guy who takes over your code may be a serial killer. Keep that in mind the next time you think this it's acceptable to have a dateLastUsed04 in the system.

Now, what about the second set of names? Well, those make more sense in the given system. I don't even have to tell you anything about how the system works, or what it does, for you to start to understand what those three variables do. That's really the point. I want to be able to look at a variable, method, or class name, and immediately understand the purpose and use of the object.

Before you create an object; spend a little bit of extra time and consideration to it's name. If you can't come up with a good name that describes the intent, then perhaps you should re-examine the purpose of your new entity. After all, if you can't explain what it's purpose is, how can you expect someone else to come along after you and figure it out?

Fluent Method Chaining

Hudson.Akridge@gmail.com (Hudson Akridge) — Fri, 27 Mar 2009 03:43:00 +0000

So, something worth noting is the concept of Method Chaining. What I mean here, is the ability to call a method on an object, have that method return us an object, which we then use to perform actions on, which returns us an object which we then use to perform actions on, which return...oh you get the idea.

In general, I find this method exceptionally useful if you're creating a suite of objects that operate together to set up a configuration. A good example of this might be from the Fluent NHibernate mapping API. First, lets check out how to do a subclass mapping without using fluent chaining:

   1:  DiscriminatorPart<string, Document> subClassDiscriminator = DiscriminateSubClassesOnColumn<string>("DiscriminatorValue");

   2:  SubClassPart<string, Document, Manager> subClassPart = null;

   3:  DiscriminatorPart<string, Document> discriminatorPart = subClassDiscriminator.SubClass<Manager>("DocumentManager", x => {subClassPart=x;} );

   4:  ManyToManyPart<Document> manyToManyPart = subClassPart.HasManyToMany(x => x.Documents);

   5:  manyToManyPart.AsMap("indexColumn");

   6:  manyToManyPart.Inverse();

   7:  manyToManyPart.Cascade.AllDeleteOrphan();

   8:  manyToManyPart.LazyLoad();

   9:  manyToManyPart.WithForeignKeyConstraintName("FK_Managers_Documents");

Not too bad in all actuality. We've all written code like the one above. But what if there was a different way to get the same thing? Lets look at the same functional code, but this time with Fluent NHibernate's chaining:

   1:  DiscriminateSubClassesOnColumn<string>("DiscriminatorValue")

   2:                  .SubClass<DocumentManager>("DocumentManager", mgr =>

   3:  {

   4:  mgr.HasManyToMany(x => x.Documents)

   5:    .AsMap("indexColumn")

   6:        .Inverse()

   7:        .Cascade.AllDeleteOrphan()

   8:        .LazyLoad()

   9:        .WithForeignKeyConstraintName("FK_Managers_Documents");

  10:     });

Let me explain the basics of what's going on here:
1: We're calling a method on our class, which returns to us a DiscriminatorPart object.
2: That DiscriminatorPart has a SubClass() method on it, which returns to us the same DiscriminatorPart that was created on line 1
4: The subclass DocumentManager is being mapped in the lambda. This is chained to call a HasManyToMany method on the DocumentManager, returning us a ManyToManyPart
5: The ManyToManyPart has a Map() method called, setting up the collection type, and returning us itself as a ManyToManyPart
6: The ManyToManyPart has an Inverse() method on it which sets a flag telling the mapping to write an inverse="true" attribute, and then returns us itself as a ManyToManyPart
7: The ManyToManyPart has a Cascade() attribute set on it which creates a CollectionCascadeExpression on the ManyToManyPart. The chain from the CollectionCascadeExpression sets an attribute for the type of cascade, in this case, AllDeleteOrphan. The AllDeleteOrphan returns us the original ManyToManyPart object that Cascade was called on
8-9: Other attributes are set via chained methods returning the ManyToManyPart

What's the difference? Hopefully it's obvious. But I'll list out the main points I can see:
- Less verbose code. Simply put, the largest benefit is that you have to write less. There's less repetition on the screen regarding accessing the same object over and over
- Reads better. Because there's less code on the screen, it's easier to focus on just the parts that convey meaning, and read intent. When debugging or reviewing code, gathering intent is most likely the highest investment you have in the code.
- More meaningful interface. We are able to specify what is allowed to be done at what point, and control the flow of access. In effect, we can guide the subscriber to our API down the "happy path".

Regarding that last point, I would highly recommend the return types to fluently chainable methods to be exclusively interfaces. This allows you to cut down on the noise and dead end paths that might appear when you return an implemented class. Even Fluent NHibernate is guilty of violating this on many occasions, so don't feel too bad if you're tossing around concretes.

How to tell when to use Fluent Method Chaining? The end result should be a much more elegant way in which to interact with your model. If you don't immediately notice the improvement, or it feels like you're having to do more work than you used to when interacting with dependant operations on your class, then go back and re-evaluate your usage of this technique. Perhaps you're using too large of an interface as the return type. Perhaps you're forcing chaining in situations where it doesn't make much logical sense to.

Not every situation needs this approach. With enough experience, you'll quickly find which scenario's work best and which are better left to traditional declarations.

NHibernate and Snapshot Isolation

Hudson.Akridge@gmail.com (Hudson Akridge) — Tue, 10 Mar 2009 13:48:00 +0000

Isolation levels make a large difference in how scalable your application is. Pick the wrong one, and your users will experience the kind of pain you see on The Daily WTF. But, if you're running into concurrency issues (row locks), or other scalability problems, it may be worth your time to take a look at the Snapshot isolation.

Typically, with NHibernate, you have four isolation strategies to choose from:

ReadUncommitted - The one you really want to stay away from unless you're 100% you know what you're doing. You can end up with cascaded failures on transactions. That in and of itself should steer you away.

ReadCommitted - The default isolation level used by NHibernate, and recommended for use in NHibernate In Action, and other such qualified sources

RepeatableRead - Really, you should get the majority of the benefits of this from using caching and versioning

Serializable - The flip side to ReadUncommitted, except this encounters scalability issues

However, NHibernate will use any isolation level supported by the database (and to that extent, ADO.NET). Which gives us "Snapshot" (recently introduced to MsSql2005+)

Lets talk briefly about the Snapshot isolation model (ref: MSDN Article):

When writing to rows, snapshot isolation actually copies the rows from the table, into the TempDB (or another comparable system if not using MsSql), performs the necessary modifications, assigning those rows to transaction ID's, then commits the snapshot rows back when it's done. While the first transaction is performing operations on the rows, it will not cause a read lock. So a ReadCommitted isolation level can actually read the rows the way they were before the snapshot isolation transaction began.

If you attempt to write to rows as a second transaction that are in a first transaction who's isolation level is set to snapshot, then the second transaction will attempt to write to the same row our isolation transaction is attempting to write to, it will raise an error and roll back the snapshot transaction.

This mode can be a major benefit, giving you some of the concurrency benefits of ReadUncommitted, while still maintaining the data integrity of a more Serializable approach.

First, you must enable snapshots on your database. For MsSql:

ALTER DATABASE MyDatabase SET ALLOW_SNAPSHOT_ISOLATION ON

ALTER DATABASE MyDatabase SET READ_COMMITTED_SNAPSHOT ON

Then you set the default isolation level in the NHibernate Configuration section. Add the following config element:

<property name="connection.isolation">Snapshot</property>

You can alternatively change the isolation level inline when a transaction first begins:

   1:  ISession session = NewSession();

   2:  session.BeginTransaction(IsolationLevel.Snapshot);

This recently (along with other tweaks) helped us solve a concurrency issue in a large application. Is it a fixall? No, absolutely not. As always, you should be pragmatic in your approach to improving performance.

AT&T was pretty close to the truth

Hudson.Akridge@gmail.com (Hudson Akridge) — Thu, 05 Mar 2009 14:12:00 +0000

In 1993 AT&T ran a couple ads explaining what you could do in the future (of course, AT&T would bring it to you).

They were pretty spot on, but to be fair, the technology trend was already heading down those paths. They of course, had the insight, because they were in the "know" due to R&D.

All in all, when you think about it, it's pretty amazing where we are from where we came in as little as 15 years. Science and technology are progressing at a rate unheard of in the history of the world. We're curing diseases faster than they can be discovered. We're increasing the average life expectancy at a rate of 40% of the time it takes to do it (thus, if you were born today, theoretically, your ALE by the time you're intended to die at age 75, will have gone up another 40% by the time you get to D-Day, assuming the same trend continues).

Of course, we were lied to. We were told we'd be making a better world. And in some respects, we have. We need more funding towards enforcing net neutrality, Stem Cell research, and exploration (space and ocean). That's our future.

The internet's wonder is it's neutral platform, what happens when everyone on earth is able to communicate at the speed of light without worry of being jailed as a result of something they've said or seen online.

Stem Cell research (and many other scientific advances that have been blue balled by government, specifically America's last administration) is the key to improving our physical well being.

Space and Ocean exploration is the answer to coping with an increasingly diminishing supply of natural resources.

It's an uncertain future. I, for one, have faith in humanity. Perhaps these dreams won't be realized in my lifetime, but if we can affect the change required to ensure that our children, and children's children have the bright future we were promised, I'll chalk that up into the win column.