But viewing a single diff isn’t always everything you want. Often, you’ll want to see all the changes made between two changesets that are farther apart—perhaps the difference between two tagged releases, or the sum of the changes made in a feature branch. With Kiln Harmony, now you can!

Simply load any changeset, and you’ll see the diff from the changeset’s first parent. Click the “Diff from another changeset…” link to search for another changeset (you can search by commit hash, tag, and branch names, or phrases in the commit message, or even filter by author and date—it’s the full power of Kiln’s search engine). Click the results to view the diff from that changeset!

You can see some examples on our demo site:

• What changed in Mercurial 2.6.1? https://mirrors.kilnhg.com/Code/Mirrors/Tools/Mercurial/History/2.6/2.6.1
• What’s new since the last Git release? https://mirrors.kilnhg.com/Code/Mirrors/Tools/Git/History/master/pu

This feature is available in Kiln 3.0.33 and higher. Sign up for a free trial and try it out!

The year: 2012. The problem: search. With a new release of Kiln, search is now forefront and center. You can zip around repositories or code with a simple tap of the keys, and boy is the future bright.

Powering search was our search engine. And powering it was our search-query parser, a couple hundred lines of code that parsed a query into a list of keywords and filters. For example, if you asked of Kiln

foo bar project:Eggs date:yesterday..now author:Tyler

Kiln finds all the commits, by people named “Tyler,” to a repository in projects named “Eggs” since yesterday with the words “foo” or “bar” in the commit message.

But try to search "foo bar" and you would be disappointed. The unspoken rule of the internet is that surrounding two words in quotation marks should make a search engine look for both words as one phrase instead of two separate words. So "foo bar" should match the string “boy I had a lot of foo bar pie” but not the string “foo and bar are two friends from way back when.” Pretty goofy rule, but the internet is a goofy place.

It’s 2012, and Kiln does not have phrase search. We left it on the cutting floor to make room for everything else we wanted, and we regret it. Life moves on.

And we cut to: the present

The year: 2013. Not having phrase search: more and more irritating. Having migrated our full-text indexing to elasticsearch, phrase searches are not only possible but easy. So you, being a developer on the Kiln team, don glasses and open the .cs file containing the query parser. Written in C# and presented for your consideration is a jumble of grammar rules and intermediate parse trees, a jungle of loops and state. A flock of crows take off from a nearby tree. You close the file.

“This seems like the ideal intern project,” you think to yourself. “It would be a shame to not allow someone else to rewrite this.”

Just then, Andrew Pritchard walks by your office. Andrew Pritchard was our summer 2012 intern who worked on a dazzling array of Kiln features, including phrase search. We will borrow a hypothetical version of him. Look at him, walking with the smooth confidence of a man not yet burdened by string parsing.

“Help us, Hypothetical Andrew Pritchard,” we said. “What do you know about parsing?”

H.A.P. points you at FParsec, a parser combinator library for F#. He begins erasing your whiteboard and drawing diagrams while you wonder what he is talking about.

“Hold on,” you say, slapping the multi-colored markers out of his hand. “I have many reservations about what’s happening right now but here’s the biggest one. F# is that that new functional-programming language from Microsoft right? Kiln is a giant ASP.NET MVC application that uses C#. There is no room for F#, Hypothetical Andrew Pritchard, you crazy lovable human being you.”

“No,” he replies. You two stare at each other for a while.

It turns out that .NET’s Common Language Runtime, plus increasingly better F# support in Visual Studio, lets you create an F# library inside your solution and reference it from a C# project. There are some quirks: ReSharper support for F# is ongoing, F# files have to be sorted in the solution tree in the order you want them to be compiled, and F# collection types map awkwardly from and to C#—to name three big ones. Overall though the experience is surprisingly pleasant. I say in the year 2013 you can (and should) alternate between F# and C# depending on the problems you are solving.

We created an F# project with the source code in this blog post if you would like to follow along. If you are not familiar with F#, fear not! By and large the F# syntax can be intuited; for a look-see, Wikipedia also has a buffet of code snippets. On my part I’ll use highly descriptive variable names and mention C# analogues to F# features when possible.

“FParsec is great, but we need F#. No biggie,” H.A.P says, shrugging his shoulders. “Besides, F# is functional, which means it’s ideal for a self-contained, computer-science-y project like string parsing.”

“It is fun.”

“You will like it.”

You are sort of convinced. In any case, he has covered your whiteboard in figures and symbols. He looks at you, then looks at the board. He walks over and gently pushes you out of your chair. You get up, brush yourself off, and read the notes on the whiteboard as he begins typing into your computer. Which notes are:

Parsers

In the world of parser combinators, a parser is a function that takes an initial state s and returns a final state s' in addition to an output object:

If the output is of type 't, we say the parser is of type 't. The black arrow shows the output.

The state is a set of facts about the world.

Let’s give ourselves a parser p that finds the string hello inside other strings. And let’s run p on the string “hello, world”. Then the state of the world prior to running p (the initial state s) as indicated by the first blue arrow has these facts:

• Our string is “hello, world”;
• Our current position is index 0;
• No errors so far;
• etc.

The state of the world after running p (the final state s') as indicated by the second blue arrow has these other facts:

• String: still “hello, world”;
• Current position: now index 5 (the comma);
• Errors: still none;
• etc.

We say that the parser has consumed the string “hello”, which means the parser has read the string “hello” and has adjusted the current position accordingly.

A parser can also fail. For example, if we run p on the string “salutations, world” an exception is raised as soon as the parser realizes there is no “hello”, and the whole thing is called off.

FParsec comes with a smörgåsbord of helper functions for parsing common chores. Some that are relevant to us:

• pchar: consumes one character of your choice. For example, pchar '"' consumes a quotation mark.
• manySatisfy: consumes as many characters as possible that matches a given function. For example, manySatisfy (fun c -> c <> ' ') consumes all non-space characters.
• many1Satisfy: like manySatisfy but will fail if it cannot consume at least one match.

These higher-order functions take arguments and return a parser. (By the way, fun c -> c <> ' ' is just a lambda. The C# equivalent, which is actually terser!, would be c => c != ' '.)

pchar '"' and manySatisfy (fun c -> c <> ' ') are parsers of type string, as indicated by their type Parser<string, unit>, which makes sense since they consume a string and then immediately output it. (You may be wondering what that second generic type argument unit is for. [unit is the same as void in C#.] We will leave the resolution to that mystery to the excellent FParsec documentation.)

Parser combinators

A parser combinator is function that takes multiple parsers and returns a new one. Every parser you’ll ever write in FParsec will be a pyramid with small simple parsers at the bottom building up to something big and wonderful at the top.

FParsec comes with a zoo of parser combinators. Some are functions, but most are operators. In F#, you can define custom operators and FParsec is nothing if not a testament to that feature. This lets us write, for example, p <|> q instead of (say, hypothetically) alternate p q. There is a trade-off though: the library can be intimidating at first. It may seem like a place of tall runes and cold magic, where the flesh and soul are tested, and light is dim, joyless. But armed with logic and courage, you will soon be able to write fairly complicated parsers in only a few terse lines.

So, using these primitives, little itty parsers get glued together into a big ol’ parser, which is what we will be doing for Kiln. You can take apart the pieces when you are reading and understanding it, and you can build up the pieces when you are writing and debugging it. You might even dare to reuse a parser or two. The same cannot be said for the kind of parsers out there in the harsh imperative world—try refactoring that while-loop with the big switch statement into modular components without messing up the big global state object.

And also: no preprocessor.

Some combinators that are relevant to us:

Sequencing with >>. and .>>

val (>>.):   Parser<'a,'u> -> Parser<'b,'u> -> Parser<'b,'u>

Here’s what you do if you’re >>.: First run p and throw away its output (black arrow). Then, following the state (blue arrows), run q and take its output.

The cousin of >>. is .>>, which throws away the second output and takes the first:

val (.>>):   Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a,'u>

Tuple sequencing with .>>. (or “two eyes, full hearts”)

val (.>>.):  Parser<'a,'u> -> Parser<'b,'u> -> Parser<('a * 'b),'u>

First run p. Then run q. Take both outputs and tuple them, which we denote by multiplying the two types together, just as F# does. (For example, a tuple of an integer and a string in F# has a type int * string, which is just shorthand for Tuple<int, string> like you might see in C#.)

Alternation with <|>

val (<|>): Parser<'a,'u> -> Parser<'a,'u> -> Parser<'a,'u>

Say you have two parsers of the same type 't. Run p first. If it fails without altering the state, run q instead.

If you recall from compilers, you can write down a grammar in a hoity-toity syntax that looks like

compound_stmt: if_stmt | while_stmt | for_stmt | try_stmt | with_stmt
if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite]
while_stmt: 'while' test ':' suite ['else' ':' suite]
for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite]
try_stmt: ('try' ':' suite
           ((except_clause ':' suite)+
            ['else' ':' suite]
            ['finally' ':' suite] |
           'finally' ':' suite))
with_stmt: 'with' with_item (',' with_item)*  ':' suite
with_item: test ['as' expr]

You can think of the <|> operator from FParsec as a rough correspondence to the | operator from Backus-Naur syntax. Note that <|> will fail if p alters the parser state. If backtracking (trying a parser without actually altering the state) is what you are looking for, seek FParsec’s excellent documentation on parsing alternatives.

Separation with sepBy

val sepBy: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>

The combinator to turn to when parsing a list of things:

1. Allocate a list of type 'p list.
2. On failure, stop and output the list.
3. On success, append the output of type 'p to the list. Then run sep and then go to step 2.

In F#, generic types’ arguments go before the generic type. So the type of the output is 'a list instead of, in C#, List<'a>. Also in F#, as you might have noticed, type variables are prefixed with a single quote because single quotes go out on Friday night and have all sorts of fun.

Function application with |>>

val (|>>): Parser<'a,'u> -> ('a -> 'b) -> Parser<'b,'u>

Run p once, which outputs an object of type 't. Pass said object to f: 'a -> 'b. Result: output of type 'b.

The code

“OK,” you say. “This is snazzy, but now what?”

H.A.P. is kneading his temples back and forth with his fingers. “This is what I am thinking,” he says. You wonder if he could think less dramatically.

• A keyword is a phrase or just a word.
  • A phrase is a bunch of characters surrounded by quotation marks.
  • A word is a bunch of non-space characters.
• A filter is a word (from a whitelist of approved words) followed by a colon followed by an argument (which is a word or phrase).
• Our input is a list of keywords and filters.

Keywords

Since elasticsearch differentiates between phrase and word searches, we need to encode the difference using types so that the code that calls into our query parser knows what to do. In F#, the way we model this is with a union type, which is analogous to union in the C programming language. So you push H.A.P. out of the way and type:

type Keyword = Word of String | Phrase of string

In F#, you should read this as “Keyword is a type with two constructors. Constructor number one is Word, which takes a string and returns a Keyword. (Constructors are just functions.) Constructor number two is Phrase, which takes a string and returns an Keyword.”

A word is a string of one or more characters that are not spaces, so we pull out the built-in many1Satisfy from earlier:

let notSpace c = c <> ' '
let word = many1Satisfy notSpace

A phrase is trickier. A phrase is a string of characters (let’s call them innards) between two quotation marks. Innards cannot contain a quotation mark unless it is escaped by a backslash. For example: "Maria said, \"I love you,\" to Mark. Mark gasped." would be a single valid phrase.

So a rigorous definition: one innard is either (1) parse a non-quotation-mark character or (2) parse a backslash followed by a quotation mark. This calls for another <|>.

let phraseEscape = pchar '\\' >>. pchar '"'
let phraseInnard = phraseEscape <|> noneOf "\""

And a phrase is just many innards surrounded by quotation marks:

let phraseInnards = manyChars phraseInnard
let phrase = between (pchar '"') (pchar '"') phraseInnards

between left right middle is another built-in parser. As the documentation for between notes, it is really just shortand for left >>. middle .>> right.

We now have two parsers of type string: word and phrase. All that remains is to lift them into the Keyword type:

let keyword = (phrase |>> Phrase) <|> (word |>> Word)

“Quiz:”, announces H.A.P. “This will not work if we switched around the arguments to <|> i.e. (word |>> Word) <|> (phrase |>> Phrase). Why?”

Who just gives out quiz questions? What kind of person does that? Quiz: What kind of person does that?

Filters

Here are all the filters we have in Kiln search:

type Filter =
    | Author of string
    | File of string
    | Project of string
    | Repo of string

Remember, the right-hand side of a type definition lists the constructors for that type and also the arguments of the constructor. For example, we can infer from this that Author has type string -> Filter.

There are three parts to a filter in the Kiln search-query syntax:

1. The name of the filter. Like author.
2. The colon.
3. The filter’s argument(s). Like Tyler in author:Tyler.
4. We also want to allow phrase searches like author:"Tyler" or author:"Tyler H". Because phrases are cool.

The code:

let filterName = (* to be written *)
let findFilter = function
    | "author" -> Author
    | "file" -> File
    | "project" -> Project
    | "repo" -> Repo
    | _ -> raise <| Exception "Invalid filter name"
let pipeline (f, x) = f x
let filter =
    (filterName |>> findFilter)
        .>> (pchar ':')
        .>>. (phrase <|> word)
        |>> pipeline

The strategy:

1. Parse the filter name. For example, given author:"Tyler", consume author.
2. Find the corresponding Filter constructor (findFilter). For example, author means Author: string -> Filter. If the filter name is not one of the ones we recognize, the parser should fail.
3. Parse a colon, throw it away (.>>). In our example, this now leaves us with just "Tyler".
4. Parse the filter argument, which can be a word or, now, a phrase.
5. With .>>., tuple the Filter constructor and the string argument together.
6. Pass the tuple to pipeline, which applies the filter constructor (string -> Filter) to the filter argument (string) and returns a Filter.
7. We now have a Parser<Filter, unit>!

Which looks like:

Remember that the blue arrows track how the string is consumed, which in this case is fairly simple, and the black arrows track how the output is manipulated, which is more complicated.

FParsec’s operators sure are a double-edged sword: the code for filter is hard to read without knowing what the operators do. But we did just compose a smart complicated parser without breaking a sweat. Overall I would say the library hurts the bright-eyed beginner but benefits the adept novice—but look at you guys, you guys are all adept novices already and I am proud of you.

Speaking of getting advanced in F#, H.A.P. has taken your wireless keyboard and typed:

let filterName =
    ["author"; "file"; "project"; "repo"]
    |> Seq.map pstring
    |> choice

“Get out of here, H.A.P. Do you ever drink water?” He stares out the window, licking his lips.

You should know that x |> f in F# is the same as f x. Futhermore, Seq.map f maps a function over a list and returns the new list, and choice [p, q, r] is roughly the same as p <|> q <|> r. While you are puzzling this all out, H.A.P. has placed one hand gently on the window, the way people do at inmate visitations.

Putting keywords and filters together

We want to store keywords and filters together in one list for simplicity’s sake, so let’s introduce another union type:

type Atom = KeywordAtom of Keyword | FilterAtom of Filter

Example: in the query

"foo bar" project:Eggs date:yesterday..now

The first atom is a phrase foo bar. The second and third atoms are filters.

And so we turn to the <|> operator one last time:

let atom = (filter |>> FilterAtom) <|> (keyword |>> KeywordAtom)

This has the type Parser<Atom, unit>, as desired. “Always satisfying to have the type of a function match the name of the function, isn’t it?” H.A.P. says as he reclines in your office chair. You were not aware your office chair had reclining capabilities.

It’s a magical world, Hobbes, ol’ buddy

H.A.P. stands up from the nap he was taking on the floor. “I am proud as well,” he says. He takes out an “A+” sticker and puts it on your hand. With a flourish of his hat—which was your hat, that he took—he leaves your office.

There is not much left to do. Now that we can parse an atom, we can parse a bunch of atoms separated by spaces, which is the parser we wanted all along:

let spaces = many1Satisfy (fun c -> c = ' ')
let parser = sepBy atom spaces

And that’s it. To run the parser, FParsec comes with a library function called run:

exception ParseError of string

let parse input =
    match run parser input with
    | Success (atoms, _, _) -> atoms
    | Failure (error, _, _) -> raise (ParseError error)

run returns a success or an error (another union type!). On success, atoms is of type atom list and we simply return it. On error, error is of type string and, since this is a toy parser, we simply raise an exception. (With match, we are using pattern matching, which alone is reason enough to try out F#. It’s the same trick that made findFilter so short.)

And what’s the type of parse? parse: string -> Atom list. That’s jazz, baby.

Sure there are some i’s to dot and t’s to cross:

• Parsing date-range filters. The date filter constructor will have the type DateTime -> DateTime -> Filter so it is not as simple as just extending filterName and findFilter.
• Part of that is also translating human date descriptions like “yesterday” and “today” into usable System.DateTime objects.
• Better error reporting, about which there is much to say.
• Using lookahead and backtracking to make the parser more robust.
• Writing a blog post, to which you commit not knowing that it requires you to learn more Photoshop than you ever wanted to.

But it is nice outside and you do love your family.

I pushed the source code to a public Kiln repository, which contains not only the code in this blog post but also some of the tests we use for our in-production parser; we are a big fan of unit testing, but who isn’t these days? (You can run the tests by hitting Test -> Run -> All Tests in Visual Studio.) I took a crack at implementing a date-range parser as a quick one-hour project but it is fairly fragile—you can do better! Play around! It’s what H.A.P. would want, with those big sad eyes of his.

And afterward, sign up for Kiln Harmony to see all the other cool stuff we put into our last release.

Hao Lian is a programmer on the Kiln team. Did you know you can create custom vibrations in iOS?

Simply enable the web hook in your Kiln account, then reference Sprint.ly items using Sprint.ly’s commands. As you push to Kiln, you’ll see your changesets linked from your Sprint.ly items.

This web hook is available in Kiln 3.0.28 and higher. Sign up for a free trial and try it out!

What’s shiny and new?

The new version of FogBugz is a single page app when it comes to the list page and the case page. What’s that mean exactly?

• a super fast list page for listing and filtering your cases
• a wicked fast case page for viewing and editing individual cases
• blazing fast search for finding the cases you’re interested in
• and blink-of-an-eye fast transitions between these pages

We’ve been dogfooding the new UI internally for a couple of months now and it has really improved our own experience of using FogBugz. We hope you’ll like it!

A Word of Caution

This is a beta, so there will be bugs. If you find one, please contact our customer service team and let them know. Furthermore, since we’re trying to get this into your hands as soon as is humanly possible, there are some caveats to the beta:

• The beta is for FogBugz On Demand accounts only.
• The beta UI has only been tested in Firefox and Chrome. Support for IE10 and Safari are on our TODO list and they should mostly work, but they might not and you will probably run into cosmetic issues.
• Customizations from the BugMonkey Plugin will not apply to the beta UI.
• With a few exceptions, existing plugins will not interact with the beta UI.
• A number of features have not yet been implemented including:
  • Custom fields
  • Quick case add on the list page
  • Graceful handling of concurrent case editing
  • Similar speed improvements and single-page-appification to Wikis, Discussion Groups, Reporting, Admin pages, etc.
  • The Working On menu
  • Searching Wiki Pages and Discussion Groups from the main search box
• With the exception of some specific text used in outgoing emails, the beta UI is not localized

Share your Feedback

We’re looking for feedback on: How the application feels, your general impressions, any missing features that impact your day-to-day workflow, and of course any bugs you find. Once you’ve started using the beta, please contact our customer service team and let them know what you think!

Once your account is enabled for the FogBugz Beta, you can enable and disable the beta UI on a per user basis from a link in the top right corner. If you later decide that you’d like your entire account taken off of the Beta, just let us know and we’ll be happy to take care of that for you.

Sign Up!

But you’re concerned about security.  There are so many public hack attempts going on these days. Now that you’ve actually written tons of code, you want to make sure it stays safe.  Can Kiln Harmony help you there?

It sure can!  In addition to all of the security features Kiln On Demand already has in place, Kiln Harmony introduces two great new features to help you secure your code: SSH and IP whitelisting.

SSH support

SSH used to be a way for people to remotely manage Unix systems, but it’s broadened out far beyond that original purpose, and is now widely used for all kinds of things, including securely and efficiently using Mercurial and Git with remote servers.  SSH can both be more secure, and more convenient, than traditional HTTPS approaches: it’s more convenient because you can use a single encryption key to access dozens of servers, and it’s more secure because the only password involved is one used on your local machine to manage your encryption key.  Dictionary attacks have no meaning in the world of properly secured SSH.

If you’re already comfortable with SSH, all you have to do is upload your public key and you’ll be good to go.  If you aren’t, check out our guide for using SSH with Kiln Harmony.

IP Whitelisting

Okay, you ask, but I actually like HTTPS just fine. What I’m really worried about is someone from a foreign country accessing my Kiln account. What about me?

We’ve got a solution for you, too. It’s called IP whitelisting, and it allows you to restrict access to your Kiln account to a specific collection of IP addresses. Say, the IP block your company uses for its public internet access.

To use IP whitelists, just have an administrator add the IP blocks that should be allowed access to your Kiln account (in IPv4 CIDR format), and then…well, actually, that’s it. As far as people outside your company know, your Kiln website doesn’t even exist.

Better understand your permissions

Permissions may not be a sexy topic, but they’re important. Even if a developer ought to be using your Kiln account, you probably still want to make sure they don’t accidentally alter the release source code two days before you ship.

Kiln has long provided a powerful, hierarchical access control mechanism to set detailed permissions on your projects and repositories, but we thought it could be a little difficult both to quickly see what someone’s permissions were, and to change them if necessary.

In Kiln Harmony, we’ve completely redone the permissions UI. You can now see at a glance who has what permissions, and changing someone’s permissions is as easy as drag-and-drop. For those of you with very large Kiln accounts, we think you’ll find the new interface also scales better, too, ensuring that Kiln Harmony can grow just as easily as your organization.

Kiln’s got your back

On top of all that, don’t forget all the existing security features in Kiln. The Activity Feed is a great way to monitor who’s pushing changes into your source, and because both Git and Mercurial identify all of their contents by SHA-1 hash, it’s incredibly difficult for a malicious hacker to replace your code without anyone knowing. And, as always, the Kiln On Demand data center is stored in a secure building staffed 24/7, so the only way for data to get out is via secure HTTPS and SSH connections. We keep your data safe.

Revised reviews and rejecting rejection

There’s a lot more in a name than most people realize. Kiln code reviews, one of the oldest pieces of the product, have offered only two real ways to mark your attitude towards a piece of code: “Approved” and “Rejected.” But developers, just like everyone else, hate rejection—especially when the thing getting rejected is our code. It makes us feel bad. Plus, most of the time, the person who marks the review as “Rejected” doesn’t really mean that the code is rejected forever and ever—just that the current version needs some work before it’ll be ready to get merged into the rest of the code base. Surely there should be some way to indicate that the code still needs some love, but that it’s headed generally in the right direction?

We know it’s a small thing, but we created a new status just for all of you: “Needs Work.” When something isn’t quite ready to merge, but isn’t actually all that bad, just press the “Needs Work” button. The reviewee will know you saw the code and formed an opinion, but you won’t sound like you’re being a mean developer and rejecting your colleague’s almost-awesome work.

We also made it easier than ever to get your code out of “Needs Review”: we now flag unread comments, so you can focus just on what’s changed since you last visited, and we allow you to use Kiln’s excellent search to add and remove changesets from a review.

Searching for the impatient

In Kiln 2.9, we introduced an entirely new search architecture based around elasticsearch that gave you instant results on changesets, code, and more. But we made one very annoying design decision: we made you press “enter” before we started showing results. But come on: in this day of rapid-fire Internet news and Google results, you want your searches now, before you’ve even finished figuring out quite what you’re searching for.

We heard you, so beginning in Kiln Harmony, we’ll start looking through changesets, code, and file names, beginning the second you start typing. Not only does this get you the results you want faster; the feedback you get as you type helps you figure out what you’re looking for in the first place. On the Kiln team, we’ve found instant searches so useful that we barely ever go to the “real” results page, and we bet you will, too.

Organizing for the harried

So, finding stuff via search is easier, but what about organizing and navigating your massive amounts of code? Kiln Harmony’s got two new features to help you manage your projects, and we think they add up to make a big difference.

First, we’ve got starred projects. Kiln already figures out which repositories you’ve used recently and adds them to the top of your “Browse Repositories” drop-down, but until now, there was no way to pin repositories you cared about, but didn’t frequently access.

Second, we’ve ditched the old modal project editing interface. Administrators can now edit projects directly. You can create and edit groups, reorganize repositories, restore deleted repositories, and manage project permissions, all without having to dive into any settings windows or activating organization mode.

Read me! Read me! Read all about it!

It’s a little thing, but a little bit of documentation can go a long way. Unfortunately, Kiln doesn’t really have any type of documentation store of its own, but many projects, at the very least, include a readme file of some variety; wouldn’t it be nice if you could quickly view that inline in Kiln?

Now you can. If you have a file named any variant of readme, readme.txt, or (for Markdown) readme.md, we’ll let you browse a formatted version of it, right on a repository’s landing page. Just click the “Read More” link. This can be especially handy for open-source projects mirrored into your Kiln installation, since those generally have very well-written Markdown-based README files that can be a great introduction to using the code.

Plays well with others

Kiln is the grand fortress where you prepare all of your code, but ultimately, that code needs to interact with lots of other services.

Beginning with Kiln Harmony, we’ve dramatically expanded our integrated services. And remember that because Kiln Harmony makes all repositories available as both Git and Mercurial, that means that you can suddenly integrate your Mercurial repositories with services that do not traditionally allow you to do so, such as Microsoft Azure. We’re excited to rapidly roll out more integration with other services leveraging exactly this functionality in the coming months.

And even more!

Kiln Harmony is a tremendously large release, and while these are some of our favorite stand-out features, there’s even more that we haven’t covered: you can subscribe to repositories, getting emails whenever new changesets are pushed. You can move Git refs and Mercurial bookmarks through drag-and-drop. You can adjust how often code reviews send you email, you can take advantage of Mercurial phases, and you can customize reviews much more flexibly right from the activities page. And we haven’t even had a chance to discuss all the new security improvements; we’ll save that ’til next time.

No matter where you look, Kiln Harmony is a tremendous step forward. We really hope you enjoy it. We know we have.

We hear you. In fact, it took us months of prototyping to reach a point where we were getting happy with the results, and lots of actually using Kiln Harmony to get all of the actual workflow-related issues right. And until that point, we ourselves weren’t 100% sure if it’d ever really be possible, so we definitely don’t blame you for having the same doubts.

What we want to do here is to provide a series of posts where we dive right down into the algorithms that make this whole thing work. In each post, we’ll take a look at one part of the system, discuss the high-level view, dive into the low-level gotchas, and hopefully convince you that, yes, while this is insanely complicated, it’s also tractable, and there aren’t any catches involved. And, as a bonus, we’ll even be hosting a Q&A with the Kiln Harmony developers next week for you to ask any questions we’re not answering in this series.

In the meantime, since today is our first day, let’s start at the top: we’ll discuss how and when Kiln Harmony engages, and then dive into how Git commit objects become Mercurial changeset entries and vice-versa.

Ah yes, one more thing: posts like this don’t really have much use for images. It’s kind of one of those text-and-source-code affairs. To make up for that, and to prevent your brain turning into those “this is your brain on drugs” fried egg shticks, I’ll try to include lots of pictures of my cats. Here comes one now:

If you push me, I’ll push you

As mentioned in our launch post, Kiln Harmony had three iron requirements: it had to be repeatable (you couldn’t get different repos on subsequent translation attempts), lossless (we couldn’t discard data just because it was hard to preserve), and idempotent (a given Git commit or Mercurial changeset had to always generate exactly the same counterpart for a given repository, with no exceptions—even across Kiln installations or with a side-trip through a site like GitHub or Google Code). This suggested to us a pretty straightforward architecture: every single repository would be stored both as Git and Mercurial on-disk, and we would write a daemon that synchronized changes between the two repositories.

The exact nature of this daemon will continue to change as we improve Kiln Harmony. At the moment, the daemon—let’s call it the Harmonizer—is written in Python, interacting with Git repositories via a customized version of Dulwich, and Mercurial repositories via a customized version of…well, since it was already written in Python, Mercurial itself. What customizations, you ask? Well, we had to customize both of these libraries to allow us to deliberately generate corrupt repositories—not something we expect would be readily accepted upstream. (That statement probably sounds both evil and insane to you, but it’s neither. We’ll make use of generating corrupt data in today’s blog post, which will give you a feel for where this comes up.)

Whenever you push to Kiln Harmony, we lock all incoming pushes from the other DVCS (e.g., if you just pushed Git, we will block any incoming Mercurial pushes), and begin translation. When the translation is complete, we run translation the other direction, in a cycle, until no work happens for one complete iteration. We then notify the website new data is available, store information about the new changesets (including the Git to Mercurial mapping), and relinquish the write lock.

So far, no rocket surgery. This is important, but mind-numbingly boring, so here’s another picture of a cat.

I’m a committer

Let’s talk about nearly isomorphic data.

Git commits and Mercurial changesets include nearly the same stuff: they both include timestamps, a committer, a description of what happened, some parents (optional), and what amounts to a pointer to what code is actually included in that version. So we have our first naive algorithm to convert: get the file contents and metadata converted somehow (we’ll cover that in another post, but for now, assume we’ve somehow already got that part working), then make a changeset or commit, using the field-for-field copy of the data in its peer.

Great idea, and it’ll work for very simple Mercurial or Git repositories. But it falls down fast in the real world.

Let’s start by just trying to figure out how to convert the valid data we’re going to come across—because, trust me, there’s going to be a lot of invalid data we’re going to have to work with, too.

Extra, extra, read all about it

Let’s start with something insanely common in the Git world that gets us into trouble: the distinction between authors and committers. In Git, the person who actually wrote the code is called the author. Let’s say that person is Sara. Sara might commit and push the code herself to the official repository, in which case the author and committer will both be her. But more likely, she’ll submit it as a pull request or send her patch to a mailing list, where someone else (let’s call him John) rebases the change on master and then pushes that to the official repository. In that case, Sara stays the author, but John is now the committer.

Problem: Mercurial changesets have only one field, the username, which corresponds most directly with a Git commit’s author. They don’t have an equivalent of the committer. Further, I’ve been glossing this a bit, but the author and the committer can also both have their own timezones and their own time stamps, so we’ll need to preserve that data somehow, too.

Thankfully, there’s a solution: both Git and Mercurial allow additional data, called extras, to be stored in their commits and changesets. So here’s what we’ll do: whenever we have data only Git can understand, we store it as an extra in the Mercurial side. When we have data only Mercurial understands, we’ll store it as an extra on the Git side. Finally, while it’s not quite applicable here, we’ll decree that explicitly stored extras that we recognize trump any data “officially” in the commit.

How does that apply here? Git authors become Mercurial usernames. Git committers, and their associated data, will become custom extras in the Mercurial changeset.

You can see these right now, if you grab the Harmony version of a Git repository that has these. Let’s take a look:

$ hg clone -U https://mirrors.kilnhg.com/Code/Mirrors/Tools/Dulwich
destination directory: Dulwich
requesting all changes
adding changesets
adding manifests
adding file changes
added 1671 changesets with 3630 changes to 355 files (+1 heads)
$ cd Dulwich
$ hg debugdata -c 3ff0539fff4f
2065f43762d222a70e915735b4d348e579ed45c6
Jelmer Vernooij <jelmer@samba.org>
1243726211 7200 -kiln-git-author:Ronald Blaschke <ron@rblasch.org> -kiln-git-commit-message:Mrn8;b7gLHX>Mg~b1n
dulwich/_objects.c
dulwich/_pack.c

Fix sentinels.

As you can see, the fields Kiln adds are all prefixed with -kiln-, to minimize the chance that any other software tries to use the same keys. We also try to minimize their use, so if the committer and author are the same, or at least share the same timestamp, we won’t store the redundant data. But worst-case, we’ve introduced just a couple extras keys: -kiln-git-author, -kiln-git-author-time, and -kiln-git-author-tz. So far, so good. (There’s also an extra key in there, but we’ll come back to that later.)

A rose by any other name would be a spatula

So are we all set for users now? Not quite. All we did was switch which tool gives us problems. Git expects usernames for committers and authors to have a set format, which is (approximately) the regex [^<]* +<(.*)>. This is all well and good, and most Mercurial repositories have usernames in this format as well, but, unlike Git, Mercurial does not require this format. Especially in repositories converted from Subversion, you’ll frequently see usernames like john or sara instead of John Example <john@example.com>.

We’ll fix this in two pieces: first, to make Git happy, we unilaterally declare that every anonymous user who has a commit in Kiln Harmony shares the free email address unknown@kiln.example.com. Second, to keep Mercurial happy on the return, we store the raw username as a Git extras field, kilnhgusername, so we can restore the verbatim name when we go back to Hg. Remember how I said earlier that extras win over “official” data? That comes up here so that Mercurial won’t pick up the munged Git username on the return: if kilnhgusername is present, it trumps the Git username on the commit.

Let’s take a look:

$ git clone https://mirrors.kilnhg.com/Code/Mirrors/Tools/Mercurial.git
Cloning into 'Mercurial'...
remote: Counting objects: 98193, done.
remote: Compressing objects: 100% (22363/22363), done.
remote: Total 98193 (delta 75822), reused 98117 (delta 75746)
Receiving objects: 100% (98193/98193), 24.33 MiB | 2.63 MiB/s, done.
Resolving deltas: 100% (75822/75822), done.
Checking out files: 100% (1015/1015), done.
$ cd Mercurial
$ git show --format=raw 12a7480271a2
commit 12a7480271a246ae990c4558a878b6628bb19550
tree 8bf68397176781c35ad65bccbeb29be8833600c3
parent 16bd8634c95aa7120aa7a7371705538b89e51576
author Stephen Darnell <unknown@kiln.example.com> 1122488324 +0800
committer Stephen Darnell <unknown@kiln.example.com> 1122488324 +0800
kilnhgrawdescription L1bhgVIVCnbZKp6AY*TBZDDR?AZ%%FWgu^GbZKvHAarjabZKp6AZTYGV{dJ3VQyq|3JL
kilnhgusername Stephen Darnell

    Add a --time command line option to time hg commands

diff --git a/mercurial/commands.py b/mercurial/commands.py
...

Okay, not so bad. Plus, I think we earned a cat. A good one. Maybe something retro?

Commits beyond description

Mercurial and Git both allow you to have descriptions to, erm, describe what you’ve done. So we just copy those, right?

Well, no. Git descriptions can have any encoding, whereas Mercurial ones have to be UTF-8. But, thankfully, we can use a similar trick here to what we used with Git authors: we’ll use changeset extras to store the Git encoding (-kiln-git-commit-encoding) and the original description’s byte sequence (-kiln-git-commit-message), and we’ll lossily transcode the Git commit message to UTF-8 so Mercurial users do see a description when they run hg log. Going the other direction, we just mark Mercurial descriptions as being UTF-8 encoded, and we’re done. Right?

Almost. Up to this point, we’ve been focusing on how we translate valid data found in Git and Mercurial repositories. But it turns out that a lot of the actual repositories out there have grossly invalid data. You see, both tools started getting used, in big ways, before their data formats were really 100% stabilized. In some cases, it’s not that the data is so much invalid as that the formats slightly changed. In others, insufficient data validation meant corrupt data could make it to disk.

How’s that apply here? Well, in theory, all Mercurial descriptions are UTF-8. In practice, you will find Mercurial repositories whose descriptions are in completely random encodings (we found lots of Latin-1, a little KOI8, some ShiftJIS, and smatters of other encodings), and, worse, you will find descriptions which are not valid in any known encoding. So now, even though “all” Mercurial descriptions are UTF-8, we also need to store the raw Mercurial description in Git (kilnhgrawdescription), and then our best-effort lossy description as the “official” Git description, so Git users have at least some chance of figuring out what was going on in a given commit. So does that work?

No. Weren’t you listening? Mercurial descriptions are UTF-8, and, nowadays, Mercurial actually enforces that. So out-of-the-box, working with the Mercurial API, even if you take the above steps, you can end up with repositories you can convert to Git, but that you cannot convert back to Mercurial.

As the French say: nom d’un chien.

Er, nom d’un chat.

Remember how I said some of our modifications to Mercurial allow us to deliberately store invalid data, and that these modifications are neither nefarious nor insane? This is why: to allow these repositories to round-trip, we need to bypass all the safety mechanisms in Mercurial, and commit a description that is a blob of bytes in an unknown encoding. See? Purely, totally, sanely logical.

Whoops, that was a Khan Academy engineer, not a cat. My bad. It won’t happen again.

So, anyway: now do we have descriptions?

Of course not!

Mercurial descriptions cannot end with a newline. Git descriptions must end with a newline. So we’ll add a newline when going to Git and remove it going to Mercurial. Right? Wrong: because Mercurial didn’t always enforce that rule. So while we do add and remove a single newline 99% of the time, the remaining 1% of the time, we store the raw Mercurial description, newline and all, in the kilnhgrawdescription extra in the Git commit. Oh, and because that will by definition have newlines, which we can’t (sanely) store in a Git extras field, we’ll also base85 encode it. Ah yes, and this will of course be another modification of Mercurial so that you can deliberately store improperly formatted descriptions that are nevertheless valid encodings.

So now are we call good with descriptions? Well, descriptions, yes, but commits in general? Not even close. Git timezones aren’t always valid, so we happily welcome our new -kiln-git-commit-tz-raw and -kiln-git-author-tz-raw overlords. Mercurial changesets end up having the same problem, so kilnhgdatetime is born unto the world. And so on and so forth.

When you’re done, you have a disgusting pile of extras on both sides, but you also have repositories that are human-friendly, and round-trip. So it’s worth it in the end.

Wait a minute. Disgusting pile of extras. Do we round-trip extras?

No, no, no, this won’t do at all!

Git extras are really more like a text blob at the end of the official commit area, while Mercurial’s are a sorted key-value store. That wouldn’t matter if we were the only ones using the extras, but there are, of course, other uses: both Git and Mercurial use their extras area to track rebases and to store what Subversion commit a changeset or commit came from, among other things. So we’re going to have to round-trip those, too.

In a maneuver that made me once jokingly refer to this product as Kiln Ouroboros, we of course solve the problem of how to preserve extras by using extras. The implementation here is incredibly uninteresting: we use prefixes to avoid having Git extras collide with Mercurial extras or use base85-encoded JSON dictionaries, depending on which direction we’re going and what we’re doing. And that ends up getting you almost all the way through commits and changesets.

But wait, there’s more!

Almost? Well, yeah. Those concepts of branching and merging that practically make DVCSes what they are? We skipped past all of that.

But I’m out of cat pictures, and you’ve got enough to think about for the time being. So let’s start tackling those concepts in our next post. In the meantime, go out, get some coffee, and enjoy knowing that the Kiln team worried about all of this so that you don’t have to.  (And, if you haven’t yet, go make a Kiln Harmony account so you can start playing with everything we just talked about yourself.)

We were locked in a year-long development marathon to bring you an amazing, secret product, and we’re ready to let you in on that secret project right now.

Today, we’re shipping Kiln Harmony. While Kiln Harmony ships with piles of new features, from SSH support to IP white-listing to better services integration and more, we want to focus on exactly one major feature today:

Every single repository in Kiln Harmony can be used from both Git and Mercurial. Simultaneously.

You read that right. The war is over, and everyone wins.

Ending the Flame War

There are quite a few hosting products right now that support both Git and Mercurial, but these tools always require you to pick which DVCS you’ll use on any given project. In other words, their support is an either/or proposition: it supports either Mercurial or Git, but not both on a given project at the same time. That means you inevitably fight it out on your team on which DVCS gets to win. Pretty soon, your Mercurial and Git users aren’t even talking to one another, the company goes bankrupt, and you’re contemplating arson for the insurance money.

That ends now. A repository in Kiln Harmony is a repository in Kiln Harmony, and each repository fully supports both Git and Mercurial. No more flame wars, no more fights. Just productive coding.

How it Works

Under the hood, Kiln Harmony has a bunch of voodoo that translates Git data to Mercurial data and vice-versa in a way that is repeatable, lossless, and idempotent. That’s a really fancy way of saying that a given Mercurial changeset will always become the same Git commit and vice-versa, even if you use that repository on another hosting service or move it to a new Kiln installation, and that we don’t lose any information in the process.

We’ll have a whole series of blog posts this week on the technology that makes that possible, but for now, I want to focus on how you can actually use Kiln Harmony.

The Magic Drop-Down

This is the yin-yang Kiln Harmony drop-down. If it is possible to fall in love with a drop-down, we know you’ll fall in love with this one. Whenever you’re exploring in Kiln—whether in a review, browsing history, looking at files, or reading a diff—simply click this little widget to switch between viewing the page in Mercurial or Git mode.

Most of the time, you’ll notice absolutely no difference—and that’s a good thing! If a file or a diff looks a given way in Mercurial, it ought to look the same way in Git. But in those few cases, like subrepositories and submodules, where Git and Mercurial have different ways to do something, you can see the difference with two clicks of a mouse.

Branching

There is one area where Git and Mercurial work pretty differently: branching. We really wanted this to be no-compromise: Mercurial and Git users shouldn’t have to change their workflows to work with each other. How do we handle that in Kiln Harmony?

First, the easy case: it turns out that many published Git and Mercurial repositories actually only ever have one branch in them to begin with. In those cases, we’ll automatically keep the Git master branch in sync with the Mercurial repository’s tip transparently. So far, so good.

What about multiple branches? That turns out to be a lot easier than you’d think. The past several versions of Mercurial have had a concept called bookmarks, which work similarly to Git refs: they can be added or deleted, they don’t have history, and you can be “on” one when you’re making changes. So, at a high level, Git branches just become Mercurial bookmarks and vice-versa.

There’s a bit of voodoo here, though, for a couple of reasons. Mercurial allows anonymous heads, which is a fancy way of saying that not all active branches necessarily have distinct names. Git doesn’t allow that. Mercurial also has a concept called named branches, which Git lacks. What do we do with those?

Kiln Harmony’s got you covered: we’ll provide bookmarks on all anonymous heads, so that all Mercurial changesets are also accessible from Git. If you use named branches, we’ll translate those into Git refs as well, so that when you refer to branch foobar, Git users will know what you’re talking about.

There’s one other little footnote, though: what about Mercurial users who have never used bookmarks before? Do you have to learn bookmarks to work with Git users through Kiln Harmony?

We actually think bookmarks are awesome and you should learn them even if you’re not using Kiln Harmony, but the general answer is, “No, you don’t.” Kiln Harmony is smart enough to infer what you meant even if you never want to play with bookmarks, or if you’re using a version of Mercurial that’s so old you don’t even have them. If you make changes that correspond to a bookmark, we’ll automatically move both the bookmark and the ref when you push.

What about everything else?

There’s a lot of magic that goes into Kiln Harmony. Rather than try to cram all that material into a little blog-post, we’ll be running a series of articles in the coming days on how Kiln Harmony works. We’ll take you under the covers into the details of our implementation, how we handle bizarre corner-cases, how we handle data formats that are more defined by their violations than their nominal structure, and more. If you’re the kind of person who loves reading about that kind of nitty-gritty detail, then you’ll probably enjoy reading our detailed articles on exactly how Kiln Harmony works. You can also sign up for the Kiln team’s live Q&A next week.

How do I get it?

Beginning today, all new Kiln On Demand accounts automatically come with Kiln Harmony. If you don’t have an account, make a free trial and you’ll have Kiln Harmony in minutes. For our existing Kiln On Demand customers, we’ll be upgrading your accounts over the next couple of weeks. If you want to jump to the front of the queue, just drop us a line and we’ll make sure you’re one of the first accounts upgraded.

Where do we go from here?

Kiln Harmony is an incredible product, and while we think you’ll love it today, we’re going to continue making it faster, more powerful, and even easier-to-use in the months to come.

In the meantime, welcome to the flamewar-free future.

So we poked around on the internet and found this technique for blurring text with CSS. (Neat!) We tailored it for Trello and created a bookmarklet that we call “Blur My Board”.

Check out the bookmarklet here!

Need an example? Here’s an unaltered screen shot of the Trello dev board:

Click the “Blur My Board” bookmarklet and here’s what it looks like afterward:

Now snap a screenshot. Refresh the page and everything goes back to normal.

If only there were some aphorism we could use that would capture the concept that images convey more information than text. Oh well, we hope you enjoy the bookmarklet at least.

If you want to show us your board, get your bookmarklet here, and after you’ve blurred it, put it online somewhere and tweet us about it @trello!

The first major step is moving all of our servers from six individual colocation cabinets to seven racks in a cage. The actual move distance is only about 10 meters but represents an upgrade in almost every way. These infrastructure upgrades will enable us to support new releases the FogBugz and Kiln teams are about to ship (teaser alert: we’ve been dog fooding them the last couple of months and they are awesome, contact us if you want to participate in the beta). The cage will provide Fog Creek a solid foundation for the future and the ability to free up enough gear to equip a second datacenter.

We are striving to keep the impact of this move as transparent as possible. However, during March we will have three significant Saturday night maintenance windows during which FogBugz and Kiln will be unavailable. Our goal is to keep these outages a short as possible, as few as possible, and during a time of low usage for a majority of customers. To paraphrase Moltke “No project plan survives first contact with the server room,” so more details on the timing and scheduling will be forthcoming. Watch this space and our status blog for more details. We are committed to the Fog Creek guarantee of not wanting your money if you are not amazingly happy. These upgrades are a huge  part of our commitment to live up to that guarantee. Please let us know if these outages materially impact your business and we will make it right.

Here are some questions you are probably asking and we have asked ourselves over the past months:

Why stay in Peer1?
Why stay in a datacenter located in one of the most expensive pieces of real estate, in a flood zone, in a shared building? All excellent questions, but as we debated the various answers we realized there was nothing wrong with our current datacenter that a second datacenter wouldn’t fix. In a disaster (natural or man made)  situation, a second geographically diverse datacenter with a tested and practiced failover procedure is our best option for providing our customers with continued service. Fog Creek has grown up with Peer1 over the past 10 years and we have a great relationship with Mike and Scott (the real heroes of Sandy) who operate the facility. So we decided to stay and make Peer1 part of our overall strategy.

Why not move to the cloud like Trello?
We aren’t parochial about choosing technology solutions. We want the best technology to solve the problem at hand (just search for cloud vs. dedicated server if you are looking for parochial arguments). Before Sandy arrived we had begun the process of moving Trello to AWS. A technology stack well suited to horizontal scaling and stellar growth made Trello an excellent candidate for cloud hosting. FogBugz and Kiln use different technology stacks and are more I/O intensive than Trello. For these and many other reasons, the cloud isn’t the right solution for FogBugz and Kiln at this time.

We will have before and after pictures and some other fun vignettes into the process. If you want to work on awesome exciting projects, we are looking for experienced unstoppable Sys Admins!