Words in Boxes

George Orwell on Writing and Thinking

2010-02-28T10:48:00.001-06:00

From Politics and the English Language:

A scrupulous writer, in every sentence that he writes, will ask himself at least four questions, thus: 1. What am I trying to say? 2. What words will express it? 3. What image or idiom will make it clearer? 4. Is this image fresh enough to have an effect? And he will probably ask himself two more: 1. Could I put it more shortly? 2. Have I said anything that is avoidably ugly? But you are not obliged to go to all this trouble. You can shirk it by simply throwing your mind open and letting the ready-made phrases come crowding in. They will construct your sentences for you – even think your thoughts for you, to a certain extent – and at need they will perform the important service of partially concealing your meaning even from yourself.

…

What is above all needed is to let the meaning choose the word, and not the other way around. In prose, the worst thing one can do with words is surrender to them. When you think of a concrete object, you think wordlessly, and then, if you want to describe the thing you have been visualizing you probably hunt about until you find the exact words that seem to fit it. When you think of something abstract you are more inclined to use words from the start, and unless you make a conscious effort to prevent it, the existing dialect will come rushing in and do the job for you, at the expense of blurring or even changing your meaning. Probably it is better to put off using words as long as possible and get one's meaning as clear as one can through pictures and sensations. Afterward one can choose -- not simply accept -- the phrases that will best cover the meaning, and then switch round and decide what impressions one's words are likely to make on another person. This last effort of the mind cuts out all stale or mixed images, all prefabricated phrases, needless repetitions, and humbug and vagueness generally. But one can often be in doubt about the effect of a word or a phrase, and one needs rules that one can rely on when instinct fails. I think the following rules will cover most cases:

(i) Never use a metaphor, simile, or other figure of speech which you are used to seeing in print.

(ii) Never us a long word where a short one will do.

(iii) If it is possible to cut a word out, always cut it out.

(iv) Never use the passive where you can use the active.

(v) Never use a foreign phrase, a scientific word, or a jargon word if you can think of an everyday English equivalent.

(vi) Break any of these rules sooner than say anything outright barbarous.

Worth rereading every once and a while.

Switcheroo version 0.4

2010-02-07T13:14:00.001-06:00

I’ve released a new version of Switcheroo, the humble incremental-search task switcher for Windows. Changes in this version include:

The list that prevents certain windows from appearing in the main window is now editable from the options dialog. (For example, “Program Manager.”)
The main window now resizes to the full width and height of the list.
Fixed a few minor bugs and tested on 32-bit Windows 7.

Essential Windows Tools

2010-01-26T06:00:00.000-06:00

When working, I like everything on my computer “just so.” I want to do things quickly and with minimal mental overhead. Here’s a list of (developer-centric) tools that make that possible:

VirtuaWin. This is hands-down the best virtual desktop manager for windows. With a bit of tweaking, you can switch between desktops instantaneously by bumping your mouse pointer at the edge of the screen while holding down the control key.
xplorer2. I use the free “lite” version. Great features:

Tabbed views (just like your browser). Keeps the taskbar clean.
Filter a directory listing quickly using ctrl-h.
When in a folder, pressing F10 and enter starts a command prompt in that folder. (With a bit more tweaking, get a cygwin bash prompt.)

Cygwin + minTTY. If you spend time on the command prompt, it’s worth it to learn your way around bash. And MinTTY’s native-windows interface – with fully resizable windows and transparency – is worth the price of admission alone. I don’t use the built-in command prompt any more, and neither should you. Great features:

Color coded directory listings and grep results.
Press control-r and search backwards through your command history.

Switcheroo. Yes, I wrote it, but what of it? Switching between (and closing) running applications using incremental search keeps me from reaching for the mouse and breaking my concentration.
Slickrun. A floating, auto-completing prompt that lets you quickly open programs, folders, and websites. With some batch-script ingenuity, the possibilities are endless. For example, I can type “vup” or “vdown” to quickly adjust the volume. Or “newmail” to write a new e-mail in Outlook. For most commands, I only have to type the first few letters. Set up tip: change the colors, make the font larger, and set it to autohide and chase your mouse cursor.
Emacs. The One True Editor. Learning it is a commitment, but worth it. If you take the plunge, grab Ctrl2Cap and remap your caps lock key as another control key. With a bit of setup, plays very well with Cygwin.
Winsplit Revolution. Not as essential with Windows 7’s built-in window positioning, but the fusion mode is still great.
Password Safe. Securely encrypt a list of all your passwords. Now you only have to remember one, which lets you online banking passwords more complex and harder to brute-force. And you’ll stop using the same one everywhere.

Knowledge and productivity are like compound interest

2010-01-22T08:05:00.001-06:00

What Bode was saying was this: ``Knowledge and productivity are like compound interest.'' Given two people of approximately the same ability and one person who works ten percent more than the other, the latter will more than twice outproduce the former. The more you know, the more you learn; the more you learn, the more you can do; the more you can do, the more the opportunity - it is very much like compound interest. I don't want to give you a rate, but it is a very high rate. Given two people with exactly the same ability, the one person who manages day in and day out to get in one more hour of thinking will be tremendously more productive over a lifetime.

– Richard Hammin, in You and Your Research

Recent Projects

2009-11-01T20:26:00.001-06:00

It's been a while since I've written here. In the interim, I've completed a few different small projects. Here they are in no particular order:

Switcheroo. The humble incremental-search task switcher for Windows.

I wrote this partly to learn a bit of WPF, and partly to scratch my own itch. I got tired of alt-tabbing through a (long) list of open windows, and really wanted something like Emacs's IDO mode buffer switching. This little program allows me to quickly switch to any window by typing in just a few characters of its title.
eXist XProc Extension Library. A set of XProc extension steps for interacting with an eXist XML database from a client. Using these steps, you can conduct common eXist management tasks from XProc - loading resources, extracting resources, querying data, etc. They fill much the same role as the eXist Ant tasks.

I wrote it as an experiment in creating a non-trivial reusable library in pure XProc. I think it came out fairly well.
Ruby Blue Visual Studio theme. I really liked the various ruby blue themes available for other editors and wanted one for Visual Studio.

XProc Roundup

2009-05-26T20:43:00.001-05:00

Interest in XProc has really picked up recently, which is exciting to see. Here's a roundup of some of the recent activity:

XProc: Step by Step. Vojtech Toman (who's working on EMC's Calumet implementation) wrote a great introduction to XProc. Calumet itself is scheduled to be released under a developer's license on June 15. I don't use any of EMC's products, but it's encouraging that such a big player in the market is embracing XProc.
XProc Tutorial. Roger Costello updated his excellent tutorial. It's a bit of a shame that it's in PowerPoint, but it's well organized and comes with several sample scripts.
Why XProc Rocks. Joel Amoussou talks about XProc and news publishing.
XML Pipelines / XProc for bioinformatics. Pierre Lindenbaum describes an interesting use of XProc.

The XProc spec is tantalizingly close to making the transition to proposed recommendation, and all the recent (and positive) attention makes me optimistic that XProc will be a standard that's actually used in the real world.

Debugging XProc Pipelines

2009-03-16T22:21:00.001-05:00

As I've worked more with XProc, I've written a couple of utility steps to help debug pipelines. Although they started as quick hacks, they've continued helping out long enough to be worth sharing. You can download the library here.

The more interesting of those is a step (wxp:assert) which asserts that the result of a given XPath expression evaluated against one document must be equal that that of a second XPath evaluated against a second document.

Practically speaking, I can assert that the output from a step must match certain expectations. If this assertion is met, the wxp:assert step functions just like a p:identity step, passing XML through its primary input to its primary output without modification. If the assertion fails, the step sends an error message to the console, and optionally throws an XProc error and terminates pipeline execution.

For example, here's an excerpt from an upconversion pipeline:

<p:xslt name="group-sections">
  <p:input port="stylesheet">
    <p:document href="group-sections.xslt"/>
  </p:input>
</p:xslt>

<wxp:debug-output step-name="6-sections-grouped" debug="true"/>

<wxp:assert label="No sections were deleted during grouping" 
            xpath-source="count(.//subsection)"
            xpath-alternate="count(.//*[self::sect-1 | self::sect-2 | 
                             self::sect-3 | self::sect-4])" 
            fail-on-error="false">
  <p:input port="alternate">
    <p:pipe port="result" 
            step="parse-initial-subsections"/>
  </p:input>
</wxp:assert>

(This also shows the wxp:debug-output step, which has probably been rendered obsolete by the latest version of Calabash. It simply functions as a p:identity step that also writes the XML it receives to disk for later review.)

This instance of wxp:assert asserts that the number of subsection elements in the primary source input port matches the number of sect-1 and sect-2 elements on the alternate input port. The source document is produced by the preceding step, and the alternate document is piped in from a step earlier in the pipeline. This is a quick way to ensure I don't do anything too lame-brained in the "group sections" step.

Here's another instance from the same upconversion pipeline. This tests the output from a XSLT step that uses regular expressions to parse out section numbers from title elements.

<wxp:assert label="No empty enums after parsing sections"
    xpath-source="count(.//enum[not(.//text()[string-length(normalize-space(.)) gt 0])])" 
    xpath-alternate="0"
    fail-on-error="true"/>

In this case, the alternate XPath expression is a constant (0), so I don't need to provide a document on the alternate port.

Using wxp:assert steps does slow down a pipline, so I usually remove them once I finish debugging the pipeline. The major exception to this rule is that I leave them active in upconversion pipelines where manual intervention is acceptable and accuracy is more important than speed.

There are a few interesting in the step's source worth checking out. Here's a summary of how wxp:assert works:

It combines the two source documents into a single document using p:pack.
That is passed to an XSL stylesheet, which uses the saxon:evaluate extension function to evalutate the two XPaths against their respective nodesets.
Depending on the value of the fail-on-error option, throw an XProc error and terminate the pipeline.
Use a final p:identity step to pipe the initial input from the step's source port to its result port.

Or course, using a Saxon extension function does tie the pipeline to Calabash (or at least, a Saxon-based processor). I'm interested in hearing ideas about alternate approaches.

Code Review: Commenting XSLT Regular Expressions

2009-02-11T18:25:00.003-06:00

You learn a lot from reading other people's code. For example, the other day I ran into a clever trick in Jeni Tennison's XSpec code for commenting regular expressions in XSLT:

<xsl:variable name="attribute-regex" as="xs:string">
  <xsl:value-of>
    \s+
    (\S+)        <!-- 1: the name of the attribute -->
    \s*
    =
    \s*
    (       <!-- 2: the value of the attribute (with quotes) -->
      "([^"]*)"  <!-- 3: the value without quotes -->
      |
      '([^']*)'  <!-- 4: also the value without quotes -->
    )
  </xsl:value-of>
</xsl:variable>

The trick is the <xsl:value-of /> instruction, which casts its contents as a string. An especially nice thing about this method is that you can refer to other variables within the declaration:

   (\S+)    <!-- 12: the name of the element being opened -->
   (        <!-- 13: the attributes of the element -->
     (      <!-- 14: wrapper for the attribute regex -->
       <xsl:value-of select="$attribute-regex" />  <!-- 15-18 attribute stuff -->
     )*
   )

Of course, to ignore all the extra white space in a regex constructed this way, you'll need to set the "x" flag in any <xsl:analyze-string />, replace(), or matches() that refers to it.

“Broken gets fixed. Shoddy lasts forever”

2009-02-04T19:35:00.001-06:00

Jack Moffett:

One of the developers I work with said this after I complained about a lingering issue in one of our products. It rings true. When deadlines are tight, and there is more work to get done than there are developers or hours in the schedule, it’s not the squeaky wheel, but the jammed one that gets the grease. The lesson, then, is to make sure it gets done right the first time. You never know when you’ll have the opportunity to revisit it.

(via Daring Fireball)

Stark and Adjustable Meanings

2009-01-20T21:07:00.001-06:00

An interview of Alan Kay:

But at PARC our idea was, since you never step in the same river twice, the number-one thing you want to make the user interface be is a learning environment—something that’s explorable in various ways, something that is going to change over the lifetime of the user using this environment. New things are going to come on, and what does it mean for those new things to happen?
... Even if the user is an absolute expert, able to remember almost everything, I’m always interested in the difference between what you might call stark meaning and adjustable meaning.
I did quite a bit of study on that over the years to understand the influence of having something that you can read. It’s known that our basic language mechanism for both reading and hearing has a fast and a slow process. The fast process has basically a surface phrasal-size nature, and then there’s a slower one. This is why jokes require pauses; the joke is actually a jump from one context to another, and the slower guy, who is dealing with the real meanings, has to catch up to it.
There have been many, many studies of this. This argues that the surface form of a language, whatever it is, has to be adjustable in some form.

Money Is Gas in the Car

2009-01-11T10:09:00.001-06:00

Tim O'Reilly talks about the importance of working on things that are important, not just things that simply make money:

First off, though, I want to make clear that "work on stuff that matters" does not mean focusing on non-profit work, "causes, or any other form of "do-goodism." Non-profit projects often do matter a great deal, and people with tech skills can make important contributions, but it's essential to get beyond that narrow box. I'm a strong believer in the social value of business done right. We need to build an economy in which the important things are paid for in self-sustaining ways rather than as charities to be funded out of the goodness of our hearts.
...

Some of you may end up working at highflying companies. Some of you may succeed, and some of you may fail. I want to remind you that financial success is not the only goal or the only measure of success. It's easy to get caught up in the heady buzz of making money. You should regard money as fuel for what you really want to do, not as a goal in and of itself. Money is like gas in the car — you need to pay attention or you'll end up on the side of the road — but a well-lived life is not a tour of gas stations!

What's Your Semantic Exit Strategy?

2009-01-08T18:08:00.001-06:00

Quoting Cafe Con Leche quoting A List Apart:

We’ll start by posing the question: “why are we inventing these new elements?” A reasonable answer would be: “because HTML lacks semantic richness, and by adding these elements, we increase the semantic richness of HTML—that can’t be bad, can it?”

By adding these elements, we are addressing the need for greater semantic capability in HTML, but only within a narrow scope. No matter how many elements we bolt on, we will always think of more semantic goodness to add to HTML. And so, having added as many new elements as we like, we still won’t have solved the problem. We don’t need to add specific terms to the vocabulary of HTML, we need to add a mechanism that allows semantic richness to be added to a document as required. In technical terms, we need to make HTML extensible. HTML 5 proposes no mechanism for extensibility.

HTML 5, therefore, implements a feature that breaks a sizable percentage of current browsers, and doesn’t really allow us to add richer semantics to the language at all.

This is an important warning for anyone creating or maintaining a custom XML document schema. Creating a tight, semantically correct element for every type of information in your existing documents is easy (and dangerously seductive); creating a mechanism by which the schema's users themselves can gracefully expand the semantic tagging as documents grow is much harder, and ultimately much more important. Be as general as you can get away with, and as specific as you dare.

Lessons from a Windows Reinstall

2009-01-06T06:45:00.000-06:00

The Windows install on my fiancee's laptop, a 3-year-old Gateway, had become unstable and slow (as Windows does if left alone for three years) so it was time to wipe the hard drive and start again. I methodically prepared, backing up all the data to two different locations — Dropbox and an external hard drive. Also, at my father's urging, I created a drive image using the free trial of Acronis True Image, which I stored on an external hard drive. I didn't think I'd need it, because the current install was so horrible that anything was preferable to it.

So, I rebooted, I popped in the Gateway system restore disk, and told it to format the drive and reinstall the operating system.

When it was done, the computer booted into Windows. Happy surprise, the restore CD didn't reinstall all of the stupid cruft software that all consumer laptops come loaded with. However, a not-so-happy-surprise — the CD also didn't reinstall any of the Gateway-specific drivers the laptop needed to function, so it couldn't use its wireless card or suspend.

No problem, I thought, I'll get them from the system restore partition on the primary hard drive. Hmm, the restore process repartitioned the drive and killed it. (Really?) Okay, no problem, I'll go to Gateway's website and download the drivers. Hmm, those don't work. (Seriously?) I'll go to the chipset manufacturer's website. Hmm, not there. (Oh, no.)

This problem was entirely my fault. I should have made absolutely sure I had the drivers I needed before wiping the disk, so I'll spare you the whole rant of WHY DOESN'T GATEWAY'S CUSTOM SYSTEM RESTORE DISK CONTAIN THE DRIVERS TO ACTUALLY, YOU KNOW, RESTORE THE SYSTEM? Because, that would be petty.

After some Internet searching, I discovered a free utility called Double Driver, which allows you to backup and restore all the drivers installed on your system. Which was great, except I'd already killed the Windows installation with the drivers I needed.

To make a long story short, here's what I did:

Created a new drive image of my partially finished reinstall with Acronis True Image.
Restored the original, pre-wipe drive image.
Booted into Windows and ran Double Driver to create an archive of all the installed drivers on my USB drive.
Restored the in-progress drive image.
Used Double Driver to restore all the previously installed drivers.

Success! (Well, Double driver actually missed one file, which I had to retrieve from the backup image. But close enough.)

Also, although Acronis True Image saved the day, I can't recommend it as a product, since its background service slowed to a crawl the two computers I installed it on. Which is too bad.

So, recapping today's lessons, if you're reinstalling Windows, you should first:

Backup all your data. Twice.
Create a drive image of your current install. It will save your bacon.
Create an archive of all your drivers using DoubleDriver. This too will save your bacon.

XProc, Part III — Turtles All The Way Down (Steps, Reuse, and Encapsulation)

2008-12-22T12:11:00.002-06:00

This is my third post on XProc. (To see the first two, go here and here.) In this post, I'll talk about the different categories of steps, how to create your own steps and reuse them across pipelines, and how this all relates to the fundamental metaphor of XProc.

One task I do frequently is removing Arbortext Editor change-tracking markup from documents. Here's a simple pipeline that does just that:

<p:pipeline type="u:accept-changes">
  <p:xslt>
    <p:input port="stylesheet">
      <p:document href="accept_changes.xslt"/>
    </p:input>
  </p:xslt>
</p:pipeline>

The fundamental unit of work in XProc is the "step," which you can think of in the same way you do subroutines in other programming languages. In general, a step takes XML as input, performs an operation on it, and outputs XML. There are three types of steps in XProc:

Atomic steps. These are the most basic building blocks of XProc pipelines, and each carries out a single fundamental XML operation. There are a number of built-in atomic steps, such as p:xslt (as shown above). These built-in atomic steps will be the foundation for almost everything you do in XProc, so learn them.
Compound steps. These are assembled from other XProc steps. Sometimes they are just a series of implicitly connected atomic steps, and sometimes they use built-in logical control structures, such as p:for-each, to control their execution.

It turns out that the above accept-changes pipeline — and in fact any pipeline you create — is also a compound step. (The fact that p:pipeline is a shortcut for p:declare-step is a dead giveaway.) This is an important idea, and we'll discuss it more below. But for now, remember this: a pipeline and a step are the same thing.
Multi-container steps. There are only two multi-container steps: p:choose and p:try. These contain two or more alternate pipelines. You cannot define your own custom multi-container step.

No doubt, in your day-to-day work, you'll never feel the need to stop and consider whether a certain step fits one category or the other. But the difference does have consequences, so it's worth spending a bit of time on it.

Let's say that you want to use the above accept-changes step as a step within a larger pipeline — for example, a pipeline that removes all of the notes from 2nd-level sections in a document. Here's how you would do that:

<p:pipeline name="delete-notes" 
    xmlns:p="http://www.w3.org/ns/xproc" 
    xmlns:atict="http://www.arbortext.com/namespace/atict"
    xmlns:u="http://www.wordsinboxes.com/xproc">

  <p:serialization port="result" encoding="utf-8" method="xml"  
    omit-xml-declaration="false"/>

  <!-- Declare accept-changes step -->

  <p:pipeline type="u:accept-changes">    
    <p:xslt>
      <p:input port="stylesheet">
        <p:document href="accept_changes.xslt"/>
      </p:input>
    </p:xslt>
  </p:pipeline>
  
  <!-- Pipeline flow starts here -->  

  <u:accept-changes name="remove-tracking-markup" />
  
  <p:delete name="delete-level-2-notes" 
    match="/chapter/section/section/note" />
     
</p:pipeline>

What's happening is that we are declaring a new step type called "u:accept-changes," and then invoking an instance of it as the first step in the pipeline. A few notes:

The u:accept-changes step declaration itself is not executed directly; only it's instance on line 21 is. More formally, a pipeline element that is the child of another pipeline element (unlike what we've seen before) is used to define a step that can be invoked later. As an analogy to other programming languages, you are defining a object type and then creating an instance of it. As a result, there is no need in this pipeline for explicit piping; the default implicit connections suffice.
A step's type attribute is different than its name attribute. The value you place in type becomes the name of the element you use to invoke it (lines 11 and 21). That instance can itself be given a name, which can then be used when controlling the flow of XML using pipes. So, a type refers to the object, and a name refers to the instance.
A step's type must be in a non-XProc namespace, which you must remember to declare.
The p:serialization element defines how a specified pipeline output port serializes XML when outputting from the pipeline. It is the equivalent of the xsl:output instruction in XSLT, and takes many of the same options.

Of course, if you're going to just reference your new step type once within the same pipeline that you've declared it in, there's not really much of a point.

The real power of declaring your own step type comes from the ability to place it in an external library and create instances of it within any number of pipelines. Here's how you do that, using p:import. First, the library document:

<p:library 
    xmlns:p="http://www.w3.org/ns/xproc" 
    xmlns:atict="http://www.arbortext.com/namespace/atict"
    xmlns:u="http://www.wordsinboxes.com/xproc">

  <p:pipeline type="u:accept-changes">    
    <p:xslt>
      <p:input port="stylesheet">
        <p:document href="accept_changes.xslt"/>
      </p:input>
    </p:xslt>
  </p:pipeline>
       
</p:library>

Note the root element p:library, which is used as a container for a collection of custom steps. If I wanted, I could have made the p:pipeline element the root element, and imported that single step, but using p:library leaves more room for future growth.

Here's the final pipeline, which imports the above library:

<p:pipeline name="delete-notes" 
    xmlns:p="http://www.w3.org/ns/xproc" 
    xmlns:atict="http://www.arbortext.com/namespace/atict"
    xmlns:u="http://www.wordsinboxes.com/xproc">

  <p:serialization port="result" encoding="utf-8" method="xml"  
    omit-xml-declaration="false"/>

  <!-- Import accept-changes step -->

  <p:import href="lib.xpl" />
  
  <!-- Pipeline flow starts here -->  

  <u:accept-changes name="remove-tracking-markup" />
 
  <p:delete name="delete-level-2-notes" 
    match="/chapter/section/section/note" />
     
</p:pipeline>

There. A nice and simple pipeline. I (and anyone else) can reference this u:accept-changes step in any number of pipelines, and never worry about how it actually works. (And, if you stored the transform inside the u:accept-changes definition using p:inline, it would be even more portable.) Later, if I decide that I'm better served by using a Python script to accept changes instead of a transform, I can simply update the step declaration (using p:exec) once in the library. My hope is that XProc will become a common language for sharing XML manipulation tasks.

It's also worth noting that if I gave this note-deleting pipeline a type attribute, then it too could be invoked as step in another pipeline. And that pipeline could be invoked as a step, and so on. Turtles all the way down.

Here's where the difference between an atomic step and a compound step comes into play. Although we defined our accept-changes step as a compound step (because it contains a subpipeline of one step), when we invoke an instance of it, that instance is an atomic step.

If that is confusing (and it sure confused me at first), it helps if you think about the step from two separate perspectives — from within, looking at the implementation details, and without, looking at what actually enters and leaves the step. We've already covered the inside. From the outside, when you call an instance, that instance is a black box which XML enters and XML leaves — exactly like a built-in atomic step. The fact that it's actually implemented by assembling XProc steps is irrelevant to the calling pipeline.

There's one more important XProc fact I've talked around but never explicitly stated. The only information that can flow between steps through pipes is XML. At first, I thought this a stifling restriction, but as I work more with XProc, I think that this prohibition is the key to its power.

Because, when you combine all these ideas, you end up with a very high level of enforced encapsulation that provides a very simple yet powerful abstraction for working with XML. It allows you to worry about what needs to be done instead of how. When any pipeline can be used as an atomic step, you can work fractally, making parts out of the same raw material as the whole, creating higher and higher levels of abstraction.

So, to summarize:

A pipeline is a step.
Although the declaration of a step may be a compound step, its invocation is not.
XML is the only information that can flow between steps through pipes.

That's it for now. If you're looking for more XProc information, Dave Pawson has been working on his introduction to XProc, which is shaping up to be a great resource.

I may start posting my XProc-related material in a more suitable space, away from my blog. I'm considering using Docbook Website system. Any thoughts?

Recently In Markup

2008-12-16T07:30:00.000-06:00

Cafe con Leche
"The W3C Core Working group has broken faith with the XML community by publishing an XML 1.0, fifth edition spec that is incompatible with all previous versions."
Joe Gregorio | BitWorking | CS Broke
"What part of the computer in front of you is fractal?"
Batch files
More than you ever wanted to know about how to write Windows batch files.
Main Page - Exslt2
The beginning of the discussion for EXSLT 2.0.
Marklogic review
Dave Pawson reviews MarkLogic ... and gives up.
Overlap, Containment and Dominance | Jeni's Musings
Let's talk about Python 3.0
"You build a nice big room-sized cage, and in one end of it you put five monkeys. In the other end you put the banana."

This Week in Google Reader

2008-12-13T08:00:00.000-06:00

Laugh-Out-Loud Cats #1006
(via HOBOTOPIA)
Teachers, Quarterbacks, and Markets
(via Uncertain Principles)
Photo
(via garfield minus garfield)
Wireframe Lamborghini
(via We Made This)

Getting Started With XProc, Part II

2008-12-07T17:27:00.001-06:00

In my previous post, I showed how to set up a batch script to run Calabash in Windows. This time I'll show how to write a useful — if simple — XProc pipeline. Here it is:

<p:pipeline xmlns:p="http://www.w3.org/ns/xproc">

  <p:xslt>
    <p:input port="stylesheet">
      <p:document href="sort.xslt" />
    </p:input>
  </p:xslt>

  <p:xslt>
    <p:input port="stylesheet">
      <p:document href="to_wordml.xslt" />
    </p:input>
  </p:xslt>

</p:pipeline>

So what happens here? When you execute it from the command line using:

runcalabash -i "source=in.xml" -o "result=out.xml' sort-and-print.xpl

this pipeline reads the input XML (in.xml), executes the first transform on it, passes the result to the second transform, executes that transform, and the saves the result as out.xml. But the syntax of XProc is clear enough you probably guessed that on your own.

The fundamental metaphor of XProc is that XML data "flows" into a pipeline and between its steps through a series of connected ports. The two most common and important ports are the primary input port (usually called "source") and the primary output port (usually called "result"). There are others, and not every step actually has both of these, but that's another discussion.

This isn't obvious in the above pipeline because all of the input and output ports, and the connections between them, are implicit. There are more explicit ways to state most of the pipeline. For example, it turns out that p:pipeline is just syntactic sugar for p:declare-step. In other words:

<p:pipeline xmlns:p="http://www.w3.org/ns/xproc" >
</p:pipeline>

is equivalent to:

<p:declare-step xmlns:p="http://www.w3.org/ns/xproc" >
  <p:input port="source" primary="true" />
  <p:input port="parameters" kind="parameter" />
  <p:output port="result" primary="true" />
</p:declare-step>

Expanding on this, here's a more explicit version of our original pipeline:

<p:declare-step name="sort-and-print" xmlns:p="http://www.w3.org/ns/xproc">
  <p:input port="source" primary="true"/>
  <p:output port="result" primary="true">
    <p:pipe step="print" port="result" />
  </p:output>
  <p:input port="parameters" kind="parameter" />
  
  <p:xslt name="sort">
    <p:input port="source">
      <p:pipe step="sort-and-print" port="source" />
    </p:input>
    <p:input port="stylesheet">
      <p:document href="sort.xslt" />
    </p:input>
    <p:input port="parameters" kind="parameter">
      <p:pipe step="sort-and-print" port="parameters" />
    </p:input>
  </p:xslt>

  <p:xslt name="print">
    <p:input port="source">
      <p:pipe step="sort" port="result"/>
    </p:input>
    <p:input port="stylesheet">
      <p:document href="to_wordml.xslt"/>
    </p:input>
    <p:input port="parameters" kind="parameter">
      <p:pipe step="sort-and-print" port="parameters" />
    </p:input>
  </p:xslt>
  
</p:declare-step>

That's quite a mouthful, but it exposes the relationships between all the steps. There are a few things worth noting. First, the p:pipe element connects the input of a step to another step. In the above pipelines, these elements are redundant, since XProc automatically connects sequential sibling steps, but they become necessary when you need to connect non-sibiling steps in larger, more complicated pipelines.

Second, every step can be given a name using the "name" attribute, and that name must be unique within the pipeline. These names can be referenced by the "step" attribute of p:pipe to explicitly connect ports. If a step isn't given an explicit name, one is generated internally by the XProc processor.

Third, the p:xslt step gets its stylesheet from a port that is just like any other port. Most of the time, you will explicitly point to a stylesheet using a p:document element, but you have a lot more flexibility than that. For example, you can programmatically create a stylesheet in one step, and then pipe it to a p:xslt to execute it. This would be a great alternative to using saxon:compile-stylesheet() and saxon:transform() within a single, multiple pass stylesheet.

Fourth, p:output works a bit differently than you might expect (as talked about recently on xproc-dev here and here). The p:output step does not define where the output for a step goes; it defines where the output comes from. So, while it may initially seem like this

<p:output port="result">
  <p:document href="out.xml" />
</p:output>

will send the output to out.xml, it instead sends the content of out.xml to the "result" output port. What this means is that where the output port of a pipeline or a step goes cannot be defined within that pipeline or step. It must be defined outside that step or pipeline — for example on the command line or in the input port of another step.

(If you want the other effect, you can use p:store, which can be thought of as the equivalent of <xsl:result-document/> in XSLT, but in that case the primary output from is actually the name of the file you are writing and not the actual content of the output.)

That's it for now. Next time: How to create and use a library of custom XProc steps.

This Week in Google Reader

2008-12-06T09:18:00.001-06:00

Obama uses Zune. Will iPod fans demand apology? | Eric 2.0
(via featuresblogs.chicagotribune.com)
Note also that the reporter that broke the story is named Santos.
Joe the Plumber and his favorite books
(via Marginal Revolution)
Also, Joe the Plumber is writing his first book.
Photo
(via garfield minus garfield)

Typography and the New York City Subway

2008-12-05T07:30:00.000-06:00

An AIGA article and a New York Times blog post about the history of the typography of the NYC subway system are both good reads:

Helvetica was originally created in Switzerland. It was a neutral typeface from a neutral country and gained runaway popularity starting in the 1960s for its modern grace. But the subway system looked elsewhere.

“It was an incredibly courageous thing to do at a time when Helvetica was riding high,” Mr. Shaw said.

That's right. Typography as an act of bravery.

(via Daring Fireball)

Getting Started With XSpec

2008-12-04T07:30:00.000-06:00

Lately I've been working with XSpec, a new testing framework for XSLT created by Jeni Tennison. I dabbled with Jeni's old testing framework, Tennison Tests, but I never could muster the discipline to use it as much as I should. XSpec makes testing frictionless, which is really how it must be if I'm going to make it a habit.

Here's how to integrate it into Oxygen:

On Windows, XSpec is launched by running a batch file. It's simple to hook up this batch file to Oxygen as an external tool so that you can run an XSpec suite by clicking a button. By being a little clever - and sticking to a consistent naming convention - we can set it up so that we can activate the tests either from the XSpec document itself, or the XSLT you're developing. The caveat is that you must save your XSpec documents in the same folder as the XSLT, and give it the same name as that transform, except with the extension "xspec" instead of "xslt." (For example, my_stylesheet.xspec tests my_stylesheet.xslt.) If you want to follow a different convention, adjust the following instructions appropriately.

Download XSpec and unzip it somewhere in your system path. I recommend getting the latest xspec.bat and transforms, since Jeni's fixed a few bugs from the initial release.
Open up xspec.bat, and edit the third line to set the CP (or class path) variable to the location where Saxon is installed. (If you don't have Saxon, get it).
Now we need to add XSpec as an external tool in Oxygen. To do so, go to Tools > External Tools > Preferences > New and fill out the fields like so:

(Make sure to change the paths to match those on your computer.)
I like to tell Oxygen to make XSpec-namespaced elements a different color. To do that, go to Options > Preferences > Editor > Colors > Elements by Prefix. This makes it easier to differentiate between XSpec instructions, and the inline XML test data.

Now you should have a button on your toolbar labeled "Run XSpec Test." If it doesn't show up, make sure to activate the "External Tools" toolbar.

Here's an example of a successful test report:

Creativity and “Courageous Sucking”

2008-12-02T07:30:00.001-06:00

From 43 Folders (emphasis mine):

I laid on the sidewalk. All the way down. On my gut on 50° of western San Francisco concrete.

And, I took my time, thinking about the aperture (all the way open for depth of field) and the available light (very little, so I put the the camera right on the ground to steady it). I snapped a dozen or more shots with slightly different settings. No idea what I was doing. People walked by, cars passed, the L barreled by, but I kept shooting until I was satisfied that I might have something. Then, I grabbed the shoe, stood up, and trotted back up the hill, triumphant, with a recovered piece of footwear, plus what I suspected might be at least one pretty good photo.

I like how it turned out.

Yeah, I know, it’s no masterpiece, but I’m proud of it for reasons of my own. Because, last night, as I was splayed prone in the fog along Taraval Street, I realized I was getting a little better at this.

Not because I’d been magically touched with mythical creativity and skill, but because for a moment I was thinking more about how to use what I’d learned to get a good photo than I was about how I might have looked while doing it. And, that felt like a small turning point.

A Man and His Chili

2008-11-14T07:30:00.001-06:00

Every man needs a chili recipe. As of a week ago, I have mine.

This is based on one from Allrecipies, but I modified it by halving the beef, adding more beans, and doubling most of the spices except for the chili powder, which I halved. To me, too much chili powder obscures the other tastes.

Ingredients

1.25 pounds of lean ground beef
30 oz of tomato juice, maybe less. Put this in last so you don't make it too soupy.
2 cups chopped onion
1/2 cup chopped celery
1/2 cup chopped green bell pepper
1/8 - 1/4 cup Chili powder
2 teaspoons ground cumin
2 teaspoons garlic powder
1 teaspoon salt
1 teaspoon ground black pepper
1/2 teaspoon oregano
1/2 teaspoon white sugar
1/4 teaspoon ground cayenne pepper (more if you want)
2 cans of red beans, drained and rinsed
1 can of pinto beans, drained and rinsed

Directions

Brown ground beef on stove. Drain and crumble.
Add all ingredients to a large kettle. Bring to a boil. Reduce heat and simmer very slowly for at least 2 hours, preferably more. If you have a crock pot, use that.
Serve with sour cream or cheese.

Getting Started With XProc and Calabash on Windows

2008-11-12T09:05:00.002-06:00

I've been excited for a long time about the coming of XProc, an XML pipelining language. A lot of my work involves running XML documents through series of transformations, which often means hacking up batch files or, for more permanent pipelines, writing an Ant build. Both of these methods are clunky at best and unmaintainable nightmares at worst.

Norm Walsh has been hard at work on his XProc implementation, XML Calabash. I've been experimenting with it, and while there are still issues to work out (it's alpha, after all), I'm already doing useful work with it. But since XProc is so new, there's almost no documentation available. There's the spec, of course, but that's aimed more towards implementers and less toward end users. Other than that, the only resources I've found are the Calabash documentation page, xproc-dev mailing list, Norm Walsh's 2007 presentation, and a few blog posts.

So this is the first in hopefully a series of posts aimed to create at least some initial public documentation about how to actually get things done using XProc. I'll post things as I learn. But caveat emptor - these shouldn't be confused for "best practices."

Before you can do anything, you have to get and install Calabash. Once you download it, place it somewhere in your system path. (Mine's in c:\home\scripts\calabash\.)

The next step is to create a batch file to run Calabash, runcalabash.bat:

@echo off
set CALABASH_HOME="%SCRIPTS%\calabash\calabash.jar"
set SAXON_HOME=%SCRIPTS%\saxon
set APTCUSTOM=%ProgramFiles%\Arbortext5.3\Editor\custom

set RUN_CALABASH=java -Xbootclasspath/p:"%APTCUSTOM%\
classes\resolver.jar";"%APTCUSTOM%\scripts\Framework";
"%SAXON_HOME%\saxon9.jar";"%SAXON_HOME%\saxon9-s9api.jar";
%CALABASH_HOME% com.xmlcalabash.drivers.Main -E
org.apache.xml.resolver.tools.CatalogResolver -U
org.apache.xml.resolver.tools.CatalogResolver

rem Slurp the command line arguments.
set CMD_LINE_ARGS=%1
if ""%1""=="""" goto doneStart
shift
:setupArgs
if ""%1""=="""" goto doneStart
set CMD_LINE_ARGS=%CMD_LINE_ARGS% %1
shift
goto setupArgs

:doneStart
%RUN_CALABASH% %CMD_LINE_ARGS%

You'll need to adjust the first few lines to match the paths in your system. CALABASH_HOME points to the location of calabash.jar. SAXON_HOME points to the Saxon jars, which Calabash requires to run. APTCUSTOM stores the path to my installation of Arbortext Editor, which contains the DTDs of the document types I work with and their XML catalog. If you don't need a catalog resolver, then you can omit this.

RUN_CALABASH pulls these together, creating the command that launches Calabash with a classpath containing all the java programs we need. I'm passing it the -E and -U options to activate the URI and entity resolvers, but again, you can leave that out if you want.

The next block compensates for the fact that Windows batch files can only take 10 parameters at once. Lame. (This code comes from ant.bat distributed with Ant).

Once that is done, you're ready to run your first pipeline (The following discussion is adapted from Norm's). He conveniently provides one with Calabash, so we'll start there. Here's the pipeline:

<p:declare-step xmlns:p="http://www.w3.org/ns/xproc">
<p:input port="source">
<p:inline>
 <doc>Congratulations! You've run your first pipeline!</doc>
</p:inline>
</p:input>
<p:output port="result"/>
<p:identity/>
</p:declare-step>

It doesn't do much - just echoes the input. To run it from the command line, type:

runcalabash pipe.xpl

That's it. You'll get back:

<doc xmlns:p="http://www.w3.org/ns/xproc">
Congratulations! You've run your first pipeline!
</doc>

If you want to direct the output to a file instead of to the command line, you can use the -o flag:

runcalabash -o "result=out.xml" pipe.xpl

Note the quotation marks. They are necessary because Windows batch files drop "=" when processing parameters. Again, lame.

If you want to change the input to a file instead of the inline document, you can use the -i flag:

runcalabash -i "source=in.xml" -o "result=out.xml" pipe.xpl

Finally, if you want to explore on your own, execute runcalabash.bat with no options to get a usage summary. Next time: an example of a pipeline that's actually useful.

Quark Joins XML Party

2008-10-30T09:39:00.000-05:00

I've lamented the apparent lack of energy in the automated XML print publishing market before. Now that Quark, a long-time desktop publishing name, has announced a new product, things might be getting more interesting. Details are scarce, but I'm not encouraged by the fact the name includes the word "Solution."

Emacs

2008-08-20T18:11:00.001-05:00

It's been almost a year since I first talked about my periodic upgrading-the-tools-I-work-with-instead-of-actually-working kicks. It's happened again. This time, the method of my pseudo-procrastination is Emacs.

Emacs, for those of you fortunate to have better things to do with your time, is the grandfather of all text editors. It was originally written in 1976 by Richard Stallman, who apparently hasn't shaved since:

Scary guy. That picture pretty much sums up everything you need to know about Emacs. That, and the manifestos.

So why Emacs? It's not because it's easy to pick up - it's learning curve is more of a learning cliff. But, if I sit in front of a computer for 8 hours a day, the time I spend learning a difficult can be amortized over hundreds or even thousands of hours. In this case, usability - the ability for a trained user to be efficient - is much more important than learnability. And Emacs, once learned, is lightning fast.

Both its strangeness and speed can be traced back to 1976. Computing then was very different than it is now. When Stallman wrote Emacs, the capability to edit lines in a text file in a different order than they existed in the file was a big deal. There were no personal computers, no mouse, and no arrow keys on keyboards. Users, mostly university researchers, logged into mainframes using text-only terminals over slow connections. This result is an interface that is strange and filled with baroque key combinations - you can use CTRL-F to move the cursor to the left - but is above all designed to be fast.

Keyboards haven't changed, and neither have people - so a program older than personal computing itself can actually be relevant today.

That's not to say there aren't problems. There are plenty. It's a free program, developed and extended by volunteers. It's missing many modern conveniences - such as drop-down Intellisense code-completion - and many of the independent extension efforts seem to have stalled - such as ECB - seem to have stalled several years ago. It took me about 5 Google searches to find downloadable code to make line numbers show up.

For me, the killer feature is progressive search. Those of you with Firefox already know how it works. As soon as you start typing your search term, it starts searching. It's an incredibly fast way to navigate a document, and Emacs does it better than any other program. Once you use it, navigating a document without it feels like swimming through Jello.

But really, for many specific applications (such as XSLT debugging), you're much better off using a paid commercial product to work in. One that was actually written in this decade. Money actually buys you things, amazingly enough.

Am I proud of learning it? Not really. In the end I might be marginally more productive, but maybe not enough to justify the loss brain real estate now devoted to remembering dozens of strange keyboard combinations. At this point it's mostly a hobby, something to distract me when I should be doing real work.