Thoughts from the Wet Coast

Project Naif - 1: An Introduction

cnurse — Sun, 06 May 2012 13:33:57 -0700

(adj) naif, naïve -- marked by unaffected simplicity

I have been working for a while on a library of simple components, which I have called Project “Naif”. Naif is the masculine form of the more common adjective “naïve”. Both words are originally French and while naïve is in fairly common usage in English, naif is much less used. But they mean the same thing – marked by unaffected simplicity - as defined by Merriam Webster.

There are two major reasons for this project.

Firstly, as I have been doing quite a bit of development on Windows Phone “Mango” I have realized that some of the more sophisticated frameworks do not work on the reduced “Client Profile” that Windows Phone supports, and I needed something lightweight that would accomplish these tasks.

Secondly, I saw this as a challenge – a way to learn some new concepts and techniques.

This Project is a living project - as I need a component I will build one, but at the time of writing – my little library has the following components:

A simple lock based on the Monitor object
Thread safe collections – SynchronizedDictionary and SynchronizedList
A simple DI (dependency injection) container
A cache provider model with two simple implementations – one based on the ASP.NET runtime cache and one based on a simple Dictionary
A simple data framework which supports Unit of Work, the Repository pattern, caching and transactions
An oAuth 2 Client

For most of these components there are some quite well-known open source projects available – many of them though can be quite complex, and as stated above, my goal is to be as simple as possible.

Inspired by some of the recent micro-ORMs (Dapper, Massive and Peta-Poco) – many of these components are single classes and all of the components mentioned above are compiled into a single assembly which is currently about 35-40K in size. They may not work in every scenario, but they certainly provide a robust set of tools for my Windows Phone development.

So in this series, I am going to describe many of these components, and why I made some of the design choices.

DotNetNuke Module Development 101: 2 - What is a Module?

cnurse — Tue, 03 Apr 2012 20:47:00 -0700

In the first post in this series I pointed out that DotNetNuke places few requirements on module developers, and this is often daunting to new developers – “where do I start?” being a common refrain.

So where do you start?

Before we actually start to build our first module lets look at what constitutes a module. In the default DotNetNuke skin/template that is used when installing DotNetNuke there are a number of examples of Text/HTML modules.

As an example Figure 1 shows 3 such modules on the Home Page.

Figure 1: Examples of HTML modules on the Home Page of the Default Template

Some of the content on the page is provided by the module and some of the content is provided by the framework. In figure 1, while all three modules are HTML modules they all have slightly different “headers”.

Module 1’s heading is in a larger font than Module 2 and Module 3 doesn’t have a heading at all. This module “chrome” is provided by the container which has been applied to the module. The module header text is provided by the framework (Figure 2 – red box), the styling of the chrome is provided by the container, and the text/images in the “body” of the module are provided by the module itself (Figure 2 – blue box).

Figure 2: The components of a Module

This means that as a module developer, I don’t need to worry about the external chrome – or the module header – this is done for me, I can focus on the content inside the blue box.

IModuleControl

DotNetNuke is built on top of Microsoft’s ASP.NET Web Framework, using the Web Forms model. This model is based on Web Forms (.aspx files) and Web User Controls (.ascx files), as well as Web Controls (which are compiled as part of a class library).

Every page in DotNetNuke is a Web Form – in fact every page is the same Web Form – Default.aspx.

Sections of the page are made up of dynamically injected Controls – most of which are Web User Controls, and in most cases a module is implemented as a Web User Control (.ascx). So in order to create a module we need to first create a User Control.

But not just any User Control.

DotNetNuke needs to know certain things about a module – and conversely the module might need certain things from the DotNetNuke core, so there is an additional requirement that the User Control needs to implement a DotNetNuke interface – IModuleControl

Figure 3: The IModuleControl interface which defines a Module Control

    public interface IModuleControl
    {
        Control Control { get; }
        string ControlPath { get; }
        string ControlName { get; }
        string LocalResourceFile { get; set; }
        ModuleInstanceContext ModuleContext { get; }
    }

To make life easier for module developers, the core provides a few base classes which already implement this interface and which can be used as a base by module developers.

PortalModuleBase – this was the original base class for all modules – In DotNetNuke 5.0 the requirement was modified to implement IModuleControl, and this class was refactored to accomplish this. It exposes a lot of extra properties and methods (in addition to the 5 properties that constitute IModuleControl) – most of which can also be accessed through the ModuleContext property.
ModuleUserControlBase – this new (in DotNetNuke 5.0) base class is a simple implementation of IModuleControl, based on the UserControl base class. In addition to the 5 members of IModuleControl it adds a simple method LocalizeString which is a helper method for localization.
ModuleControlBase – this base class (introduced in DotNetNuke 5.0) is based on the WebControl base class (rather than UserControl). To use this base class the module control would be compiled as part of a class library.

Basically all we have to do then is to create a “control” which inherits from any of these 3 base classes. This is the only requirement for building a DotNetNuke Module. There is much more that DotNetNuke offers to module developers, but the minimum requirement is to create a “control” which implements the IModuleControl interface.

For most of the rest of this series we will be using the ModuleUserControlBase class as our starting point, and in part 3 of this series we will actually create our first DotNetNuke module.

DotNetNuke Module Development 101: 1 - I did it My Way

cnurse — Fri, 30 Mar 2012 11:58:06 -0700

DotNetNuke has a rich eco-system of Modules, both Open Source and Commercial. In many ways this is our biggest strength as a CMS platform. No matter what you want to do with your site – there is often a module that already does it.

In my opinion this is due to the flexibility provided by the core DotNetNuke Framework. Many other platforms are very prescriptive – as an extension developer you have to follow a fixed shopping list of rules – there is invariably only one way to create an extension, which may not work for what you want to do.

But DotNetNuke places very few requirements on what must be done to build an extension. This frees developers to be creative – to think outside the box – which I feel explains the rich ecosystem of modules that already exists and continue to be developed.

I have been working as a developer on DotNetNuke for nearly 8 years now. During that time I have built many modules for personal use as well as a number of modules that are now part of the core distributions. In addition, I have designed and created many of the hooks in the core which Module Developers can take advantage of.

Recently, I have heard comments that “Module Development with DotNetNuke is difficult”.

I don’t agree with this sentiment – but, I think that our strength (flexibility) that I described above is also our weakness. There are a multitude of ways to build DotNetNuke modules - and we don’t provide a prescriptive method of development. While this provides flexibility, it can be somewhat daunting for new DotNetNuke developers – where do I start? is a common refrain.

My goal in this series of posts is to provide some guidance – from my perspective – for both the beginner module developer and for more advanced developers, who may not be aware of all the hooks/features that DotNetNuke provides.

This is not “The Definitive Way of Module Development” – its not even “A Definitive Way of Module Development”, but (with apologies to Frank Sinatra) it is “My Way”.

Microsoft goes Open Source!!

cnurse — Tue, 27 Mar 2012 18:56:31 -0700

Well – some of it anyway !!

Today Scot Guthrie, Corporate Vice President in the Microsoft Server and Tools Business announced in a blog that some of the ASP.NET technologies that are used by web developers, including me, were to be made available under an open source license.

ASP.NET MVC has been available under a permissive license since its first release, but today that was extended to the new Web API Framework and to the ASP.NET WebPages/Razor Framework as well.

But that isn’t all.

In addition to being made available under an open source license (Apache 2.0), the code repository on codeplex.com (http://aspnetwebstack.codeplex.com/) is actually the live repository that the Microsoft development team will be using for their own source control, so we can watch as they fix bugs and make code changes.

And further, they will be taking some commits from the Community. The team are using the new Git repository support that codeplex.com introduced a few days ago, and Scott Hanselman demonstrated live on stage at DevConnections accepting a pull request from Miguel de Icaza (of mono fame). Now, don’t expect every pull request to be accepted – there will be careful review of every request and each request will need to meet a high quality bar and meet Microsoft’s goals for the platform, as the product will still be fully supported by Microsoft.

So congratulations to the ASP.NET team. Keep up the good work.

Look Mom, NoSQL! – 6. My First RavenDB Application (2)

cnurse — Fri, 18 Nov 2011 18:13:30 -0700

In my previous post in this series on NoSQL Databases, I showed how RavenDB embedded could be added to an existing ASP.NET MVC 3 application using NuGet and how it can be configured to use a folder in the App_Data folder.

In this blog I will add the code needed for my TaskController class to use RavenDB to store its data.

Everything in RavenDB goes through an instance of IDocumentStore. There are two classes in the RavenDB API which implement this interface DocumentStore and EmbeddedDocumentStore. We will be using the latter.

We can create a RavenDocumentStore class which is like the DataContext or DbContext classes in Entity Framework.

Listing 1: The RavenDocumentStore Class

public class RavenDocumentStore
{
    private static IDocumentStore instance;

    public static IDocumentStore Instance
    {
        get
        {
            if (instance == null)
                throw new InvalidOperationException("…");
            return instance;
        }
    }

    public static IDocumentStore Initialize()
    {
        instance = new EmbeddableDocumentStore
                        {
                            ConnectionStringName = "RavenDB"
                        };
        instance.Conventions.IdentityPartsSeparator = "-";

        instance.Initialize();
        return instance;
    }
}

The static property Instance ensures that we only have one instance of the DocumentStore, and the Initialize method can be called in Application_Start. We could have created the instance inside the Instance getter, but that would not be thread-safe.

Listing 2: The Application Start method

protected void Application_Start()
{
    AreaRegistration.RegisterAllAreas();

    RavenDocumentStore.Initialize();

    RegisterGlobalFilters(GlobalFilters.Filters);
    RegisterRoutes(RouteTable.Routes);
}

We are going to need an IDocumentSession to do the work. We could create a new IDocumentSession instance at the beginning of each Action method and then dispose of it at the end of each method. But MVC has two very convenient hooks to do this – OnActionExecuting and OnActionExecuted. These methods are executed before and after each action method, so we will create a new IDocumentSession in OnActionExecuting and dispose of it in On ActionExecuted. To do this we will create a BaseRavenController class to act as a base class for our TaskController (Listing 3).

Listing 3: The BasenRavenController class

public class BaseRavenController : Controller
{
    public IDocumentSession RavenSession { get; set; }

    protected override void OnActionExecuting(ActionExecutingContext filterContext)
    {
        RavenSession = RavenDocumentStore.Instance.OpenSession();
        base.OnActionExecuting(filterContext);
    }

    protected override void OnActionExecuted(ActionExecutedContext filterContext)
    {
        if (RavenSession != null)
        {
            RavenSession.SaveChanges();
            RavenSession.Dispose();
        }
        base.OnActionExecuted(filterContext);
    }
}

Finally we can then modify each action method to use the RavenSession instead of the Entity Framework DbContext. As an example the Index action (Listing 4) and the Edit actions (Listing 5) are shown.

Listing 4: The Index Action method

public ViewResult Index()
{
    var tasks = RavenSession.Query<Task>().ToList();
    return View(tasks);
}

Listing 5: The Edit Action methods

public ActionResult Edit(string id)
{
    Task task = RavenSession.Load<Task>(id);
    return View(task);
}

[HttpPost]
public ActionResult Edit(Task task)
{
    if (ModelState.IsValid)
    {
        RavenSession.Store(task);
        return RedirectToAction("Index");
    }
    return View(task);
}

That is basically it – although there is one last change that needs to be made. RavenDB uses a “key” to identify each document. By default, just like an Identity column in SQL Server, RavenDB will generate this key automatically – the default behaviour being to create a key of the form “tasks/1” – i.e. the plural of the object’s name followed by the default “/” separator and a hilo generated integer.

The default separator can be changed and we did that in Listing 1 by setting the IdentityPartsSeparator to a “-“, so that we didn’t interfere with the Url separator in our routes. And since we now have a string “key” the Id property of our Task object (and all the Action method parameters) can be changed to a string.

Figure 1 and Figure 2 show the Tasks Application running in List and Edit mode.

Figure 1: The MyTasks MVC Application in List (Index) view

Figure 2: The MyTasks MVC Application in Edit view

So that’s it – its not very hard at all to start working with RavenDB. The only challenge so far being that I cannot look into the Data Store in the same way that I can with the SQL Server tools in Visual Studio or SQL Management Studio.

However, this is not the end of this series of Blogs. I will continue to explore NoSQL databases, in particular RavenDB.

Look Mom, NoSQL! – 5. My First RavenDB Application (1)

cnurse — Thu, 17 Nov 2011 08:07:07 -0700

In my continuing series on NoSQL Databases I have been mainly describing some of the concepts behind NoSQL (and in particular Document) Databases. In this post I will start to dive into some code.

As mentioned earlier I will be using RavenDB as this is a .NET friendly database.

In this first example lets create a simple ASP.NET MVC 3 Application that uses RavenDB as its data source.

I will start with a simple “My Tasks” Application which I created by using the Internet Application template in ASP.NET MVC 3. Next I created a simple Task model class (Listing 1).

Listing 1: The Task class

    public class Task
    {
        public int TaskId { get; set; }
        public string Title { get; set; }
        public string Description { get; set; }
        public bool IsComplete { get; set; }
    }

I then used the Scaffolding available in ASP.NET MVC 3 to create a TaskController (Figure 1), as well as the Entity Framework DataContext class and all the Razor Views necessary for CRUD operations.

Figure 1: Scaffold a new TaskController class

And finally I created a SQL Server Compact Edition Database to store the data and a Tasks table with columns that matched the names and types of the Task class’s properties.

To prove that everything was wired up correctly I browsed to the application and added some data (Figure 2).

Figure 2: A List of Tasks

But the goal of this application is to use RavenDB as the database, not to create an EF Code First/SQL CE application.

As I mentioned in the 3rd post in this series, RavenDB can be configured in three different ways. I am going to use the embedded mode. Luckily, RavenDB is available as a NuGet package. To add a NuGet package to a Visual Studio project we can right-click on the References folder (Figure 3).

Figure 3: Add a new Nuget package

and select “Manage NuGet Packages”. In the Package Manager window enter Raven in the search box and install the RavenDB (Embedded) package (Figure 4).

Figure 4: Install RavenDB using NuGet

The installation adds quite a few assemblies to the bin folder.

Next we need to configure Raven, so we will create a Folder in the App_Data folder call “Database” (Figure 5),

Figure 5: Create a new Database Folder for the RavenDB database

and we will modify the connection string that RavenDB added to the web.config to point to the folder we just created.

Listing 2: Update connection string in web.config

  <connectionStrings>
    <add name="RavenDB " connectionString="DataDir = ~\App_Data\Database" />
  </connectionStrings>

So now we are all set up ready to convert our application to use RavenDB, and in the next post I will show how to hook up the existing TaskController class, so we can store our tasks in an embedded RavenDB database instead of a SQL Compact database.

Look Mom, NoSQL! – 4. And to CAP it all

cnurse — Wed, 16 Nov 2011 21:05:54 -0700

As my frequent readers would have noticed already, I have been diving into NoSQL databases recently – in particular Document Databases. But why do we need NoSQL databases in the first place. Surely the relational databases we have been using work fine – in fact they work very well in most situations.

The problem with them is they don’t always scale very well – and when I mean scale, I mean scale really big – across multiple nodes in a cluster. I alluded to this in the first post in this series, but as an aside I have decided to explore why in further detail in this post.

It all comes down to the CAP (or Consistency, Availability and Partition Tolerance) Theorem. This theorem states that when working with distributed systems you can only achieve two of the three goals, so you need to decide what is important to you.

If Consistency is most important then you would choose a relational database which excels at Consistency. Examples of when Consistency would be important are in financial systems or nuclear power stations, where it is critical that every piece of data is accounted for.

However, in most cases, “eventual consistency” is all that matters. For example in a content management system if a Page Editor updates a page it is not critical that every user sees that change immediately, as long as at some time in the near future (seconds or minutes) the update is visible.

In most applications it is more important that some version of the data is available at all times – i.e. the A or Availability of the CAP theorem. In systems that focus on Availability, server uptime is more important than the exact version of the data.

Finally, the P or Partition Tolerance defines the ability of the data to be distributed (partitioned) across multiple servers – this allows for hugely scalable systems, like Google or Facebook.

Document Databases (as with most NoSQL databases) excel at the A (Availability) and P (Partition Tolerance) corners of the triad and so are less good at Consistency - eventual consistency is good enough.

Thus Document Databases are fast and can scale, but would never be used to run a financial system or a nuclear power station.

Look Mom, NoSQL! – 3. On Raven’s Wings

cnurse — Wed, 16 Nov 2011 08:23:07 -0700

In my continuing research into NoSQL databases I needed to find a database to work with. As a .NET developer I was immediately drawn to RavenDB – a Document Database written in .NET by Ayende Rahien of NHibernate and Rhino Mocks fame.

Not only is RavenDB written in .NET but it supports LINQ – who can complain at that !! RavenDB is open source (with a commercial license option). It can be used in 3 modes:

As a Windows Service
As an IIS Application
Embedded in a .NET Application

As .NET developers, one of RavenDB’s strengths is that its Indexes can be written using LINQ queries:

from post in docs.Posts
select category in post.Categories
select new { category }

Unlike RDMS Systems, RavenDB’sIndexes do not slow CRUD performance. This is because the Indexes are processed in the background and the results persisted in a Lucene Index.

For those of you who need to compare this to how relational database systems work, a RavenDB Index is like a View – with the addition that the data is saved to disk – not dynamically generated on demand. This is one of the reasons why Document Databases are fast. You can find more information on how RavenDB indexes work in the RavenDB documentation.

Like most Document Databases RavenDB supports a RESTFull Interface. For example, assuming that there is a RavenDB installed on port 8080 as an IIS application, using something like Fiddler, we can send an HTTP GET statement"

GET http://localhost:8080/docs/bobs_address

and assuming there is a document in the database with an id of bobs_address RavenDB will return a JSON document and an HTTP 200 OK response:

HTTP/1.1 200 OK

{
  "FirstName": "Bob",
  "LastName": "Smith",
  "Address": "5 Elm St."
}

If the url does not correspond to a valid document then RavenDB will respond with the typical 404 message

HTTP/1.1 404 Not Found

As a .NET application RavenDB also has a rich Client API, which I will dive into much more detail in future posts as I build a RavenDB application.

Look Mom, NoSQL! – 2. NoSQL Database Types

cnurse — Mon, 14 Nov 2011 08:08:46 -0700

A week or so ago the topic of NoSQL databases came up at a discussion at work, so I decided it was time I learnt about this topic.

A few days ago Julie Lerman (@julielerman) tweeted that she had published an article in MSDN Magazine on Document Databases (one of the major types of NoSQL databases) and reading that article started me on my research.

I purchased the Wrox Press book “Professional NoSQL” by Shashank Tiwari for my iPad Kindle App and much of this post is gleaned from the first few chapters of that book (as well as from Julie’s article).

In a previous post I introduced the concept of NoSQL databases, and in this post I will discus the three most common types of NoSQL databases.

Sorted Ordered Column-Oriented Stores

Google, who were one of the main pioneers of the trend to NoSQL databases, due to the sheer volume of data they store uses a system (Bigtable) that stores data in a column-oriented way. This compares with typical Relational systems which are row-oriented.

Each unit of data can be thought of as a set of key-value pairs, with the unit being identified by a “primary key”. Bigtable calls this the “row key”. The units of data are sorted and ordered on the basis of this row key.

So far this isn’t really much different than a Table in a Relational model that has a primary key field and a clustered index.

What makes the store “column-oriented” is that the various pieces of information that define the “record” of data, can be divided into groups of columns or column families. For example, if we are saving information about a person, we may define first_name and last_name fields, which can be grouped in a name column family. Likewise we could define street_address, city and zip_code, which can be grouped in a address column family, and sex and age which can be grouped in a profile column family.

We now have 3 column families or buckets of information. In a column-oriented store column families are typically defined at configuration or startup but the individual columns need not be pre-defined.

Within each bucket, only key/value pairs are defined. The column key identifies the column family or bucket to use and the row key identifies the individual columns within the bucket.

Like many NoSQL databases there is not really a concept of NULL data. New columns can be added at any time as it is just another key/value pair in the bucket.

While data that relates to the same row key will often be stored in a contiguous fashion, this set up allows for data to be partitioned across multiple computer nodes.

Examples of Sorted Ordered Column-Oriented Stores include:

HBase (http://hbase.apache.org) – part of the Hadoop family – used by Facebook, StumbleUpon, Hulu, Yahoo and others and
HyperTable (www.hypertable.org) - used by Baidu – China’s biggest search engine and Rediff – India’s biggest portal.

Key/Value Stores

A Hash Map or Hash Table is the simplest data structure that can hold a set of key/value pairs. Such structures are popular because they are very efficient typically approaching O(1). The key of a key/value pair must be unique and can be easily looked up.

Examples of Key/Value Stores include:

BerkeleyDB
MemBase (www.membase.org) – build on the popular MemCached Cache – used by Zynga
Redis (http://redis.io) – used by CraigsList
Cassandra (http://cassandra.apache.org/) – an Amazon Dynamo clone, developed by Facebook and open sourced – used by Facebook, Digg, Reddit, Twitter and others

Document Databases

Document Databases are not document management systems. A document in this case is a loosely structured set of key/value pairs in documents, typically using JSON (JavaScript Object Notation), not a Word Processing Document or Spreadsheet.

One core benefit for object-oriented developers is that we can think of a document as mapping to an object, including any contained collections/objects, although in reality what we mean here are objects that are considered to be “aggregate roots”.

Document Databases treat a document as a whole – rather than splitting it into its constituent key/value pairs.

This post is getting quite long so I won’t get into further detail on this class of NoSQL databases, as I will be diving into Document Databases in more detail in future posts.

Examples of Document Databases include:

MongoDB (www.mongodb.org) – created by 10gen – used by Foursquare and Github
CouchDB (www.couchbase.com) - used by Apple, BBC, Cern

Look Mom, NoSQL! – 1. An Introduction

cnurse — Thu, 10 Nov 2011 23:34:36 -0700

Most .NET developers, whether we work on the Web with ASP.NET Web-Forms or MVC, on Windows Phone or in the browser with Silverlight, or on the desktop with WPF have worked with some form of Database.

We are for the most part familiar with SQL Server, MySQL or Oracle Relational Database Management Systems (RDBMSs) – and we are used to writing database schemas and database code using some variety of SQL.

We have come to terms with the idea of the “Object-Relational Impedance Mismatch” – the fact that objects don’t always map well to database entities, and the rise in popularity of ORM (Object Relational Mapping) Frameworks like NHibernate and Entity Framework to address this mismatch.

We also know that Relational Databases are often inefficient due to the complex “relationships” (joins) between tables. Relational Databases excel at eliminating redundancy, thus reducing the amount of data stored through Database Normalization. But normalization impacts performance, so if performance is critical we know we have to selectively de-normalize the database.

Recently, there has been more and more “noise” around the term NoSQL and NoSQL databases. But what are NoSQL databases and why are they becoming more popular.

As the topic came up at work I decided it was time that I dived into this technology to try and understand the strengths and weaknesses of these databases, and so I propose to share what I have learned in this series of Blog posts.

To get things started then.

The term NoSQL basically means what it says – NoSQL databases do not use SQL because the data is not stored in relational tables, although some people have redefined the term as an acronym NoSQL = “Not Only SQL”. Whatever the term means, NoSQL is used to describe databases that do not use a relational model and typically can store huge amounts of data – think Google, Facebook or Linked-In scale.

All NoSQL databases share common goals of speed and scalability. But the many different NoSQL databases approach these goals in different ways. In the next post I will discus two of the major categories of NoSQL Databases and some of the common implementations.