In Valid Logic

Coming Soon: Trunks

Ken Robertson — Mon, 09 Nov 2009 07:55:23 GMT

Wanted to take a moment to announce a project that I've been progressively working on over the past year, and hope to finally open up soon.

Hosted source control is a market that has really exploded in the past 18 months, with a number of providers and options out there. I'm a huge fan of source control for all code, whether big or small, but with most of the offerings, I found something lacking. There wasn't really a service that could meet all of my needs. After dwelling on it for a while, started to figure out what I was really looking for and decided to turn the idea into a reality.

Trunks started out as a realization of a few core difference between what was out there and what I was looking for.

Many other services are opinionated about their source control tool, but the feature set people are looking for in a service are the same regardless of the tool. Additionally, people often use multiple tools. I use git for my ruby development, but subversion for most of my .NET development. I don't want to have to pay for two separate services.

Most developers are tinkerers. They work on stuff here and there and have an ever growing collection of code and projects. I believe all code should be under source control and never thrown out, but over time I'd be paying more and more with other services due to the number of repositories.

Some of the services sell to companies more than individuals and are loaded up with other features. A lot of hosted source control services are bundled with extensive project management, focused around teams and collaboration on the team. I just wanted plain and simple source control.

This has lead to three of the driving ideals in Trunks:

SCM agnostic. Trunks currently supports Subversion, Git, and Mercurial, with support for Bazaar also in the works.

Unlimited private repositories. And based on focusing on individuals and their growing projects, I am still on the fence about even having public repositories (open to feedback).

Just source control. I do plan on some very basic issue tracking (a numbered todo list?), but nothing fancy. It should just be plain simple to use, pretty to look at, stable, and reliable.

More details will be coming. I've had the service up and running for almost a year, but have been sitting on it to figure out what I really wanted to do with it. I recently started dusting it off and diving back into it, polishing the points I liked and removing the annoyances I found while just using it for a few months. Hopefully over the next couple of weeks, I'll feel confident in it enough to go to a private beta and really start kicking its tires.

Keep tuned for more!

Honesty isn't overrated

Ken Robertson — Thu, 08 Oct 2009 01:23:30 GMT

Last Friday, I had to go to pick up a piece of furniture with my dad that we'd been waiting for. We'd told them we'd be in at 5pm to pack it up, and they'd said they'd have it there from their warehouse before then. Sure enough, we get there at 5pm on the dot, and it is still at their warehouse. Luckily it is only 3 block away, so we follow the owner over there to pick it up.

When we get there, he says that he could make up a million excuses about why he didn't have it there by 5, but the truth was he just forgot. His wife told him that morning it needed to be there by 5, but he forgot.

It may seem minor, but I was impressed he was so frank. Most places would act like they had no idea, it was someone else's fault, or somehow your fault. When we got there and he said it was still at their warehouse, I was rolling my eyes, but when we left, was saying no problem and didn't mind at all.

I don't get medical billing

Ken Robertson — Tue, 22 Sep 2009 15:42:53 GMT

I don't get how the medical industry can function.

So I've already posted how the hospital was gunning for their money 30 mins after my son was born back in July. So they got paid.

Then when he was coming up on two months, we started getting a trickling of other bills. The OB had some weird billing practices and we still owed on her delivery. Then we got the copay bill for the circumcision. Then we got the copay bill for the anesthesiologist. $300 here, $200 there, $400 for that one. What next, is the cook going to send me a bill for the room service?

Why can't they just do all inclusive billing? I would much rather know all my expenses up front than to get bills trickling in one after another 2 months after the fact.

But then the thing that made me nearly crap my pants. Got the mail yesterday to find something from the hospital. Open it up, and the first thing I see before opening it up all the way is "Amount Due: $14,472". I accidentally let an expletive slip with a toddler in the room. Though one you open it up and look over the fine details, you see it has three boxes: Amount due, amount due from insurance, and amount due from patient. The $14k was amount due from insurance, amount due from patient was $0. But why are they sending me a notice that insurance still hasn't paid them? I don't care. The insurance industry is just as messed up as their own billing practices. At the very minimum, they could make the statement a little bit clearer. It isn't pleasant to take it out and see the top 1/3rd first with some big dollar amount.

That's my rant for the day.

The joys of bnx2 on Debian Lenny with Dell servers

Ken Robertson — Wed, 16 Sep 2009 00:44:00 GMT

Last weekend, I was working on reinstalling Debian Lenny on one of my servers when I want into the same issue I've encountered time and time again. I have a couple of Dell PowerEdge 1950s which use Broadcom network cards, and the Broadcom firmware is classified as non-free by Debian and not included on the network install CD (maybe others, thats the only one I use). For some reason, Ubuntu includes the drivers, but Debian doesn't. My eye-rolling aside, in this case I preferred Debian over Ubuntu.

Now the problem... I'm about 45 minutes away from my servers. On all my own servers, one thing I had decided is an absolute must to have the remote management cards (DRAC5 in my 1950s) which supports remote console and virtual media. This is awesome for remotely reinstalling an operating system, but not so good when it asks you for an additional driver by removable media... and of course, when it is your network card driver.

I don't quite recall how I got through it last time, though it involved a fair amount of coursing, particularly because the installer asks for the .fw file when it really just wants the .deb.

This time around, I found a nicely prepared ISO with the deb on it, however the DRAC only support one virtual CDROM, and it didn't seem to find it if I unmounted the installation media and mounted the ISO with the deb on it.

There was also some information about PXE booting with the driver, but that sounded like an overly complicated solution.

My route? If it can't find it on the virtual cdrom, try the virtual floppy! The deb is only 102kb, so just fire up Magic ISO, create a new floppy image, copy it in, and save it. Mount it to the virtual floppy and problem solved.

Encounter the same issue? Download my floppy image!

TC5.0 Search: Adding your own types, part 2

Ken Robertson — Wed, 12 Aug 2009 00:49:00 GMT

In this post, I'll cover the specifics of the search mappers. Lets say we have a class named Book and we want to index it along with everything else Telligent Community normally indexes. I'd first create a project called BookSample and add a reference to the Telligent.Search.Mapping assembly.

Mappers are classes that inherit from the Telligent.Search.Mapping.DocumentMapper class. They have to implement two pieces, a Load() method and a Priority getter. The Load() method is used to define function blocks that are later executed to perform the mapping. The priority is to allow some overriding over the mapping process. The mappers allows support inheritance, so you could define a mapper for a Post (even though Post is an abstract class), and you could also define one for ForumPost. When a ForumPost object is mapper, both will be executed.

Additionally, you can define multiple mappers for the same type and the order of execution is set with the Priority value. All built-in mappers have a priority of 10, so if you wanted all post bodies to be indexed in lower case (for instance), you could define a mapper with a priority of 20, so it will run after the default one, remove the Body field, and then re-add it as lower case.

Our Book search mapper would look something like this:

using Telligent.Search.Mapping;

namespace BookSample
{
  public class BookMapper : DocumentMapper
  {
    public override void Load()
    {
      Map<Book>(book =>
        {
          book.ISBN.MapTo("ISBN").Key();
          book.Title.MapTo("Title");
          book.Description.MapTo("Description");
          book.Author.Name.MapTo("AuthorName");
          book.Price.ToString("0.##").MapTo("Price");
        });

      Rehydrate<Book>(isbn => Books.GetByIsbn(isbn));
    }

    public override int Priority
    {
      get { return 10; }
    }
  }
}

A few things to note:

The Load() method includes calls to the Map() and Rehydrate() methods, where you define an Action/Func block for the corresponding type. This may sound funky, since we have gotten some questions about why we just didn't have something like DocumentMapper. We didn't really want to limit a mapper definition to a single type, we wanted them to be more "scriptable", where you could clear or remove existing mappings and redefine your own, and we wanted a more fluent syntax so Map says "I'm mapping a Book" where Map says "I'm mapping an Author". It could have been done either way, we just ended up with it this way.
In the Map call, you are passed in a variable which is the actual object you will be mapping. The action block is called at a later time, and when it is, the Book that needs mapping will be passed in, so you can access is just as you would anywhere else in your code. To add a field onto the search document, all you need to do is use the MapTo() extension method on whatever you want to add and specify its name in the search document. So we have a property for book.Title and in the search document we want it named "Title", so we call book.Title.MapTo("Title").
All search documents need a Key. This is whatever the original ID was of the object. If and object is mapped and none of its mappers set a Key, it will throw an exception. This is required because if the object is later reindexed, it uses a combination of the type (Book) and the key (the ISBN value) to remove the old document from the index and add the new one. Additionally, we provide a way so if you have a search document, you can "rehydrate" it and get the original object (see later points).
Again, the MapTo just needs to be whatever final value you want added, so if you want to add something from another object, just access it. In this case, book.Author represents another class type. We don't want to index it, we just want to include the author's name. All you need to do is call book.Author.Name and then use MapTo on it. In most cases, our fields in the search document map the property name, but in this case we want to use "AuthorName" so it is a clearer it doesn't relate to the book.
And for a third time, MapTo is called on the value, so it can also be manipulated. With book.Price, it is a double and we always want it indexed with two decimal places. Simply need to call the ToString() on it and format it how you wish, then call MapTo on that.
MapTo can be used on any type, as it is an extension method for "object" and not just common types like string, int, etc. However, if MapTo is not a string, it will call ToString() on it to convert it. This is to make the mappings a little less verbose like post.PostID.ToString().MapTo("PostID"), but it also means it will adhere to some defaults and if you want a specific format, you need to convert it yourself. Additionally, if you try to map a custom class, like the Author one mentioned earlier, it will call Author.ToString(), which if you haven't defined in your class, the field will likely be useless to you.
And finally, after the call to Map, we also define Rehydrate. This takes the key we previously set in the mapping, and allows to you define a function that can take that original key and return your object. This is useful in cases where you have a search document representing the Book, but want the full object. Be aware though, it can have performance costs. If you have 20 search results and look through each one, rehydrating them to display some data, that might be 20 DB calls.

In the next post, will go over how to implement the content handler for your new type.

TC5.0 Search: Adding your own types, part 1

Ken Robertson — Wed, 12 Aug 2009 00:10:11 GMT

Although I am still on paternity leave (have 2 more weeks), I am still trying to get a few things done to stay in the swing of things. Earlier today, I saw Bruno Lopes post on Twitter enquiring about how to add your own types to the new search in Telligent Community/Telligent Enterprise 5.0.

This will be a series of posts about some of the components of the new search and some of the things you need to do to add your own objects to the index and get them back in the results. This post will focus on just the components of the new search at a high level.

In previous versions of Community Server, search had a very narrow purpose. Search Barrel and the Enterprise Search were almost entirely separate, meaning if you tried to add support for additional types, it would be specific to the form of search you were using. It had very few extensibility points for integrating, and the implementation was very specific to posts.

The new search is built more like a framework, without any specific ties to certain types of objects. It is also built to be easily extended and even replaced. It is made up of several components: mappers, content handlers, an index provider, and a search provider.

Mappers define what objects and values on those objects will be stored in the index. The index is made of a series of "search documents," which basically outlines what object it represents (eg, a ForumPost), what is original ID was (as a string, such as PostID "1234"), and a collection of fields like Subject, Body, Author, etc.

Content handlers are simple things that are used to find content that needs to be indexed, makes a call to map it to search documents, passes it over to the index provider, and then somehow stores a setting to know that piece was already indexed.

Then the index provider and search provider implement the specifics of the search backend. Included in Telligent Community is a set of providers for Solr, though you could implement your own to work with FAST, Yahoo BOSS, or whatever else you wish. The providers mainly focus on handling search documents, so you can change out the providers and they'll automatically work with your existing mappings and content handlers.

In the next post, I'll cover how to write your own mapper and some of the details about how mappers work.

Ruining a moment

Ken Robertson — Sat, 18 Jul 2009 14:24:55 GMT

Yesterday, I'd posted about my son's birth the following Friday. One thing that really irked me about the experience was the hospital ruining the moment after my son was born.

Shortly after Josh was born I was in the nursery watching the nurse bathe him, one of the other nurses came in and said the front desk was asking me to come down to handle my deductible/copayment. Josh was born at 9:34pm and my receipt was timestamped 10:13pm. So when my son was 39 minutes old, they were already wanting their money.

I couldn't believe it, and my family was standing there asking "where you going?"

When my first son was born, I don't remember the hospital asking me to pay the whole time we were there. I think they just sent me the bill. If I did need to pay there, it wasn't 30 mins after he was born, it would have been later at a insignificant time to where I wouldn't even remember it.

Sure hospitals have been hurting for money, but they could be a little more sensitive. Gee, you just welcomed a new life into the world, your going to be there for 3 days... there is plenty of time in there to have me pay rather than right afterwards.

New son, Joshua David Robertson

Ken Robertson — Sat, 18 Jul 2009 00:46:49 GMT

Last Friday, on July 10th, 2009, was pleased to welcome a new life into this world. My second son, Joshua David Robertson, was born at 9:34pm weighing 8lbs 5oz and 19.5" long. He was originally due for July 22nd, though Trish had a c-section scheduled for Monday, July 13th (our last son was a c-section, it is hospital policy to always give c-sections after you've had one).

Friday night we were getting ready for dinner when Trish started feeling some strong contractions. She called her doctor who recommend we go in to hook her up to some monitors. Low and behold, she was at 7cm already and they were rushing to get her ready for the c-section before her water broke. It was kind of crazy as there were 4-5 nurses in the room getting everything hooked up and ready to go. All and all, it was 2 hours from when we were sitting at home and Josh was born.

Initially, Josh was on the observation table for the first 12 hours. He took a gulp of fluid on his way out, so they had to suction his stomach a couple of times and watch to make sure his lungs cleared up. Apparently, it is common with c-sections. He was ok by the next morning and was hanging out in the room with Trish from then on.

On top of that, he had a true knot in his cord. This is where he wiggled his way around and developed a knot in the cord. If he had been in there longer, or if they tried a normal delivery, it could have easily tightened and cut him off. He was very lucky.

Trish and Josh came home on Monday and we've been doing very well. He is adjusting well, Nick (our 1st son, 22 months old) is getting used to him and likes helping out. Josh is a pretty good sleeper, though still has his days and nights slightly off.

This time, taking advantage of some state programs for family leave, so taking a full 6 weeks off. So I'm now one week in, my inbox is overflowing, but been enjoying it so far. Have 5 weeks left, though I don't think I'll be able to make it much longer without doing some sort of dev work. I've already chimed in on a couple email threads that I couldn't resist. Will definitely enjoy the remaining time.

TC5.0 Email Templates: An Overview

Ken Robertson — Fri, 26 Jun 2009 15:42:14 GMT

In the newest version of Telligent Community and Telligent Enterprise, we've completely rebuild the way email templates are handled to unlock a new level of possibilities and performance. I'm planning on going over several aspects of it, including highlights, usage, and extending the new templating with a series of blog posts.

Up until now, we've always used the build in System.Net.Mail.MailMessage support which is baked into the framework. While it has served us well, it had a couple of limitations that we had been bumping up against in trying to grow Mail Gateway's functionality.

The new templating is based on nVelocity templates and borrows a lot from Graffiti. If you've ever created a Graffiti theme or written a Chalk extension, you'll see a lot of similarity with the new templating. With the messages now, you actually generate the raw MIME messages. MIME stands for Multipurpose Internet Mail Extensions and it is the standardized format of email messages. By working with MIME directly, aided by a series of helpers, we are able to control aspects of the message in a fine grained manner.

First, in ability to control who we're emailing independent of the message headers. With email, who the message shows in the To headers who it isn't necessarily sent to, such as with mailing lists. Previously, Mail Gateway always sent the messages to the user, from the sender, and with the list as a special "Sender" header. This is how messages showed up in Outlook as from the user "on behalf of" the list. This type of set up didn't work well with all client, particularly the iPhone. The iPhone would always reply to the user who sent the message, not the list.

Building on the last point, since we couldn't set who a message is to independent of the message, Telligent Community would have to generate a message for each person it was emailing. So if a forum had 100 subscribers, when a new post is made, it would generate the email 100 times, queue 100 messages, and then send it out as 100 individual emails. With the new handling, we can actually generate the message just once and have it go to everyone in one swoop. Also, since mail servers often limit the number of recipients per message, we can take that message and configure it to allow 20 recipients per message. So instead of sending 100 messages, we could safely get it down to just 5.

Additionally, the variety of messages can be greatly expanded. It is now super easy to include an HTML and a plain text version of the body in the same message, then allow the email client to chose which one it will show based on its preferences. Or we can generate both at once, and then chose which one gets sent out after wards depending on user preference.

With the old support, we didn't have the ability to support subsets of functionality like meeting requests. Meeting requests are basically an attachment with the ical event, and special headers indicating a response is supported/expected. With the built in functionality, Mail Gateway could read the attachment, but not handle or send it back out as a meeting request. We still haven't added this functionality, but now the limitation from MailMessage is removed so we're one step closer.

And finally, the old templates were limiting with what was available within messages. You only had a fixed set of tokens that could be used within the email, such as [SiteName], but it was really limiting in additional fields or field manipulation. Do you want to change the format a post date is displayed in? Want to only show a certain excerpt of the post body? Want a field that wasn't available in the standard set? These things would often require code changes before, but now with nVelocity templates, you actually get the Post object, not just the common post tokens. Want to change the date format? Just use $post.PostDate.ToShortDateString(). Want the post author's display name? Use $post.User.DisplayName.

Aside from new functionality that is enabled, there are several benefits to what we already use emails for.

First, email templates themselves are greatly simplified. You no longer have a gigantic XML file with all templates, they're now template-per-file. Templates are no longer in XML, so you don't need to worry about escaping certain characters of using
instead of
or using an amperstand (&) or other characters that need to be escaped in XML. These minor things often confused users who didn't know about XML, or even seasoned developers who had a knee-jerk moment.

Second, when you need to do something more complicated, there are now some solid extensibility points. This makes it much easier to integrate advanced functionality in a simple way.

More advanced pieces of email generation is now simpler to test. With the extensibility, code is compartmentalized and easier to break down and test certain subsets. Additionally, we also plan on making it easier to unit test specific email templates, to ensure the template is rendered without errors and with the right content. I'll be covered the ways to extend the templating in upcoming posts.

Finally, for portions like forum notifications, the number of messages generated is reduced which helps to improve overall performance.

In the next post, will give an overview of ways to extend the functionality of the templates.

Telligent Community and Telligent Enterprise released!

Ken Robertson — Wed, 24 Jun 2009 00:22:00 GMT

Today, all the Telligenti have been buzzing regarding the recent release of Telligent Community and Telligent Enterprise. From telligent.com:

"Telligent, a leader in community, collaboration and social analytics software, today announced the availability of version 5.0 of Telligent Community (previously Community Server) and version 2.0 of Telligent Enterprise (previously Community Server Evolution). The latest versions of Telligent’s community and collaboration software represent the company’s ongoing commitment to innovation and enterprise collaboration."

Finally, after months of feverish development, we've reached the point of a release for what is probably our biggest release in history. Our v5.0 is unlike any of our previous major releases. We have a completely new permissions system, revamped groups, new search, new email templating, and new widget based theming.

Soon I'll be having a series of posts going over some of the bigger technical points in some of my areas of expertise. In particular with this release, I'd worked on the search implementation and the new email templating.

In previous versions of CS, we had two implementations of search, the default Search Barrel and the more powerful Enterprise Search, which was built on Lucene.Net. Now, we've changed the search to be much easier to extend and change out. Previously, search was tightly coupled to the index. You couldn't easily add new types to Enterprise Search or index additional fields.

Our new search is comprised of a series of mappers, which define how to convert an object into a searchable document, and define your own content handlers, which are used to grab objects that need to be indexed, have them mapped, and passed over to the index. You can easily build on top of existing mappers, create mappers for new types, and add new content handlers to the search process. Additionally, the search is broken out into indexing and search, allowing you to easily create your own implementations. Our included implementation is now based on Solr. Why Solr? A better question is why not Solr? It did everything we wanted and then some. It is feature packed, high performance, and super reliable. With it, we were also able to easily build in thread collapsing. Tired of the search results all being messages from the same thread? Now we can collapse down the results, combining all the posts from a single thread into just one result.

What's new with emails? Over time, we found the old email templates to be too limiting. We had one big XML file containing all the emails, people often ran into confusion when editing HTML emails (since they had to be XML escaped), and the default tokens were limiting depending on the types of customizations you'd want to make. Our new emails are entirely based on nVelocity. If you are familiar at all with the theming in Graffiti, you will be right at home in the new email templates. The templates produce the mime documents that get sent out, allowing you do manipulate the message headers, attachments, and body content. Instead of providing a fixed set of tokens like [PostDate], you get the actual objects. You can now use $post.PostDate, or call $post.PostDate.ToString() and pass in a custom formatter. Want to do more? Can easily create Chalk modules to add new macros, or TemplateModules to manipulate the nVelocity context before templating, or the processed mime object after processing (useful for adding binary attachments and such).

Want to know more? Keep posted.