Jim's Thoughtspot

A First Look at ODRL v2

2012-01-17T15:28:00.001-07:00

With other things taking high priority over the last 6 months, this is the first opportunity I’ve had to look at the progression of ODRL Version 2.0, and evaluating where it’s improved from the earlier versions.

First things first, ODRL has migrated to the W3C as a Community Working Group. Overall, this is a good thing. It opens it up to the wider W3C community, gives greater credence to the effort and more importantly, more exposure. Well done.

On to my first impressions:

1 . The model has been greatly simplified. With ODRL 1.x, it was possible to express the same rights statement in several different ways. The obvious implication was that it was virtually impossible to build a generalized API for processing IP Rights, save running XJC on the schema, which isn't necessarily always what I want. It wasn’t all bad news though, the 1.x extension model was extremely flexible and enabled the model to support additional business-specific rights logic.

2. Flexible Semantic Model. The 2.0 model has a strong RDF-like flavor to it. Essentially, all of the entities, assets, parties (e.g., rightsholders, licensees), permissions, prohibitions, and constraints are principally URI-based resource pointers that imply semantics to each of the entities. Compared to 1.x, this is a vast improvement to its tag-based semantics, which meant that you were invariably extending either the ODRL content model, data dictionary, or both.

3. Needs More Extensibility. The current normative schema, still in draft, does need some additional design. Out of the box testing with Oxygen shows that only one element is exposed (policy). All of the other element definitions are embedded within the complexType models, which means makes it difficult to extend the model with additional structural semantics. This is extremely important on a number of fronts:

The current model exposes assets as explicit members of a permission or prohibition. Each “term” (i.e., permission or prohibition) is defined by an explicit action (print, modify, sell, display). It’s not uncommon to have a policy that covers dozens or hundreds of assets. So for each term, I have to explicitly call out each asset. This seems a little redundant. The 1.x model had the notion of terms that applied to all declared assets at the beginning of the policy (or in the 1.x semantics, rights). I’d like to see this brought back into the 2.0 model.
The constraint model is too flat. The new model is effectively a tuple of: constraintName, operator, operand. This works well for simple constraints like the following psuedo-code : “print”, “less than”, “20000”, but doesn’t work well for situations where exceptions may occur (e.g., I have exclusive rights to use the asset in the United States until 2014, except in the UK; or I have worldwide rights to use the asset in print, except for North Korea, and the Middle East). Instead, I have to declare the same constraint twice: once within a permission, and second time as a prohibition. I’d like the option to extend the constraint model to enable more complex expressions like the ones above.

Additionally list values within constraints are expressed tokenized strings within the rightOperand attribute. While completely valid to store values in this, I have a nit against these types of token lists, especially if the set of values is particularly long, like it can for things like countries using ISO-3166 codes.

I shouldn’t have to extend the whole complexType declaration in order to extend the model with my own semantics. However the current schema is structured that way. Instead, I’d like to see each entity type exposed as an “abstract” element, bound to a type, which ensures that my extension elements would have to at least conform to the base model.

Takeaways

I’m looking forward to using this with our Rights Management platform. The model is simple and clean and has a robust semantics strategy modeled on an RDF-like approach. This will make it easier to use the out-of-the-box model. That said, it’s missing some key structures that would make it easier to use and extend if I have to, but that can be address with a few modifications to the schema. (I have taken a stab at refactoring to test this theory – it’s pretty clean and I’m able to add my “wish list” extensions with very little effort.

Link: http://dl.dropbox.com/u/29013483/odrl-v2-proposed.xsd

Parallels between Punk and Anonymous

2011-12-31T15:59:00.001-07:00

Prologue: Before starting my career in the tech world 15+ years ago, I was a graduate student in Sociology studying political movements and economies.

At any rate, what’s intriguing about technology is not only about 0s or 1s, data structures, angle brackets, optimized queries or distributed architectures (don’t get me wrong, I love elegant code and design as much as any other geek) – it’s also the intended and unintended consequences it has on society at large. As the automobile and large manufacturing re-shaped our society a hundred years ago, the internet and all of the emerging technologies are transforming our social interactions today.

2011 was a landmark year. We saw “Arab Spring” unfold before us in large part because of mobile devices and social media (granted, the other necessary ingredients – anger, resentment, disenfranchisement, chronic poverty and unemployment – have been brewing for many years). The “Occupy” movement harnessed the same political, social, economic, and technological ingredients along with a sprinkling of hyper-aggressive tactics of the NYPD and transformed a seemingly innocuous protest into a worldwide meme. WikiLeaks, rightly or not, also changed the way we view government, particularly when sensitive or embarrassing information is exposed. And to that end, this year demonstrated that the combination of mobile and social technology meant that information could spread virally, beyond the full control of any one entity. This has spurred new tensions between individuals who interact with data and entities who provide and/or control the data.

In this case, I see many interesting parallels between the Punk subculture of the 1970s and early 1980s and the nascent subculture of Anonymous that is growing today. Both have emerged during periods of economic turmoil, and both have a strong anti-authoritarian sentiment that are willing to challenge the current establishment.

I love the Sex Pistols (and the Smiths, the Cure, The Damned, Souixsie and Banshees, and so on, and on, etc.). I can listen to “Anarchy in the UK”, “God Save the Queen”, or “Pretty Vacant” any time. It’s loud and raucous. It’s fun. It’s… well, rebellious. Johnny Rotten’s menacing, sarcastic vocals epitomized the political, social and philosophical undertones of the Punk subculture of the mid-to-late 1970s.

From many accounts, the Punk subculture, particularly in the UK, emerged during the mid-1970s in part because of the poor economy. Disenfranchised youths with few economic prospects gravitated to a style of music and dress that was non-conformist by nature and expressed their anger and frustration against society and government.

The ethos, or ideology of Punk is well described here (source: http://www.bunnysneezes.net/page192.html):

It is passionate, preferring to encounter hostility rather than complacent indifference; working class in style and attitude if not in actual socio-economic background; defiant, unconventional, bizarre, shocking; starkly realistic, anti- euphemism, anti-hypocrisy, anti-bullshit, anti-escapist, happy to rub people's noses in realities they don't wish to acknowledge; angry, aggressive, confrontational, tough, willing to fight — yet this stance is derived from an underlying vulnerability, for the archetypal Punk is young, small, poor, and powerless, and he knows it very well; sceptical, especially of authority, romance, business, school, the mass media, promises, and the future; socially critical, politically aware, pro-outlaw, anarchistic, anti-military; expressive of feelings which polite society would censor out; anti-heroic, anti-"rock star" ("Every musician a fan and every fan in a band!"); disdainful of respectability and careerism; night-oriented; with a strong, ironic, satirical (often self-satirical), put-on-loving sense of humor, which is its saving grace; stressing intelligent thinking and deriding stupidity; frankly sexual, frequently obscene; apparently devoted to machismo, yet welcoming "tough" females as equals (and female Punks are often as defiant of the males as of anyone else) and welcoming bisexuals, gays, and sexual experimentation generally; hostile to established religions but sometimes deeply spiritual; disorganized and spontaneous, but highly energetic; above all, it is honest.

Compare this to the first two parts of Quinn Norton’s (Wired Magazine) well-done analysis of Anonymous in “Anonymous: Beyond the Mask” (Part 1 here: http://www.wired.com/threatlevel/2011/11/anonymous-101/all/1; Part 2 here: http://www.wired.com/threatlevel/2011/12/anonymous-101-part-deux/). One of the first things this series does incredibly well is to identify Anonymous for what it is – a culture, or more accurately, a counter-culture.

Like Punk, Quinn goes on to describe the Anonymous culture:

The birthplace of Anonymous is a website called 4chan founded in 2003, that developed an “anything goes” random section known as the /b/ board.
…
Like Alan Moore’s character V who inspired Anonymous to adopt the Guy Fawkes mask as an icon and fashion item, you’re never quite sure if Anonymous is the hero or antihero. The trickster is attracted to change and the need for change, and that’s where Anonymous goes. But they are not your personal army – that’s Rule 44 – yes, there are rules. And when they do something, it never goes quite as planned. The internet has no neat endings.

What’s more, both are media savvy in their own ways, leveraging them for their own purpose. Obviously, in the ‘70s and ‘80s, the internet wasn’t even a twinkle in our eyes yet, so they relied on print and radio (typically either on small, low-band college stations or on pirate radio stations since mainstream radio stations wouldn’t give them airplay) to get their message out. Anonymous, however, have the luxury of the internet and search engines, where information is easily accessible and available:

But to be historical, let’s start with 4chan.org, a wildly popular board for sharing images and talking about them, and in particular, 4chan’s /b/ board (Really, really, NSFW). /b/ is a web forum where posts have no author names and there are no archives and it’s explicitly about anything at all. This technological format meeting with the internet in the early 21st Century gave birth to Anonymous, and it remains the mother’s teat from which Anonymous sucks. (Rule 22)

Both follow its own rules, many of which run counter to conventionally accepted protocols, and frequently meant to shock, ridicule and otherwise laugh at mainstream society.

/b/ is the id of the internet, the collective unconscious’s version of the place from which the base drives arise. There is no sophistication in the slurs, sexuality, and destruction in the savage landscape of /b/ — it is the natural state of networked man.

In this, it has a kind of innocence and purity. Terms like ‘nigger’ and ‘faggot’ are common, but not there because of racism and bigotry – though racism and bigotry are easily found there. Their use is there to keep you out. These words are heads on pikes warning you that further in it gets much worse, and it does.

Nearly any human appetite is acceptable, nearly any flaw exploited, and probably photographed with a time stamp. But /b/ reminds us that the id is the seat of creative energy. Much of it, hell even most of it, is harmless or even sweet. People reach out for help on /b/, and they find encouragement and advice. The id and /b/ are the foxholes of those who feel powerless and disenfranchised.

And like Punk, it never intended to be overtly political. Rather, the circumstances and events of the time instigated it. “The Guns of Brixton”, written by The Clash about the 1981 Brixton Riots is one of many examples. For Anonymous, its forays into political protest were spurred on by their collective belief that Julian Assange and WikiLeaks were wrongfully targeted by governments and large, multinational corporations, and that fellow “compatriots” at BitTorrent site, Pirate Bay, were wrongfully attacked. In all cases, the common thread was a belief of suppression by the establishment.

Where they differ, however, is in their means of expression. Punk is analog. It could only reach those in proximity to a radio signal (or the occasional TV appearance), a concert venue, or to a “zine”. It’s effect and impact on society at large could only scale to the number of members it could congregate in any one physical location, which meant that it could remain largely contained and isolated. On the other hand, Anonymous is digital. Its reach is unbounded and its impact on society much more significant. The virtual nature of Anonymous means that they are able to challenge mainstream society more directly with little or no impunity. With tools like the Low Orbit Ion Cannon for DDOS attacks, and with more talented hacker members able to break into corporate and government servers and stealing sensitive information from them, governments and corporations see them as a real threat.

At its essence, the Punk subculture provided its members a means of “flipping off” mainstream culture, through its music, dress, art, literature, and language. Yet, it was easy for mainstream society to ignore early punk youth, since their access to media was relatively limited. Anonymous shares this same “f--- you” attitude along with the same antipathy toward authority, yet they have the means to express their views more dramatically, and with greater reach, particularly because the internet, social media, and mobile devices enable members of Anonymous to be anywhere, or anyone.

Punk has evolved over the decades. The music has changed; the aesthetics are different, and to some extent, what was considered shocking then is widely accepted now. Yet, the idea of Punk is still here. Anonymous is just the latest manifestation of it, and it could potentially have even greater impact on society-at-large.

SOPA Will Be Our Generation’s McCarthy Witch Hunt

2011-12-14T23:14:00.001-07:00

In the late 1940s and early 1950s Joseph McCarthy was determined to eradicate the Red Scare by accusing numerous Americans of treason and being communists. It resulted in many actors being blacklisted, and resulted in the now infamous question to the “Hollywood Ten” from the House Committee on Un-American Activities – “Are you now or have you ever been a member of the Communist Party?” They exercised their 5th Amendment rights and refused to answer the question, principally because they felt their First Amendment rights were being impinged.

In its current form, the “Stop Online Piracy Act” (SOPA) would allow the Department of Justice and Copyright holders to seek injunction against websites that are accused of enabling, facilitating or engaging in copyright infringement. It doesn’t stop there: It would force search engines to remove all indexes for that site, mandate that ISPs block access to the site, and require 3rd party sites like PayPal from engaging or transacting with the offending website. All because the copyright holder (or DOJ) makes an accusation. The burden of proof is on the ISPs, the search engines and the 3rd party vendors to show that the “offending website” is not violating any copyright (So perhaps Congress should consult the 6th Amendment). The implications are severe even for websites that reference these infringing sites. They could be shut down too.

Let’s be clear, I’m not condoning piracy of any kind. Intellectual Property vis-à-vis copyright is the coin of the realm of many companies, even whole industries like Publishing, Media, Software, and yes, the Entertainment world, and they should protect their assets. They should derive value and profit from their IP. An author who pours their heart into a publication, or an artist whose performance I like should be paid. Likewise, content producers – studios, publishers, media companies – should be able to garner payment for their role in providing content. But they are looking at the whole piracy issue the wrong way.

Brute-force tactics to protect copyright have been epic failures. DRM approaches don’t work. In fact, they incite piracy, and worse, they harm the very companies they try to protect. In 2007, Radiohead released their album “In Rainbows” DRM-free. A year later, they had sold over 1.75 million copies and 1.2 million fans would buy tickets to their show. Bottom line: Locking down content doesn’t protect copyright holders. Instead, DRM tactics will end up frustrating consumers who legally purchase content but can’t use it or copy it to a new device and, as a result, diminishes revenue. And at that point, the opportunity cost of future purchases with the same DRM constraints will grow higher and higher. Media, publishing and entertainment executives know that DRM has failed, and feel that their only recourse is through SOPA.

There will always be a small percentage of consumers who will use pirated content. But it needn’t be a negative sum game. In some cases, it should be written off as a business cost in order to generate more revenue: a pirated song, might lead to the offending consumer to purchase a ticket to a concert, or to the next movie because they can’t wait. Yet, to prevent wholesale piracy, technology exists today that can protect copyrighted content: XMP (even ODRL can be serialized into XMP), digital fingerprinting for starters. By using these, along with other tools that can scan the internet for matching assets, asset producers can identify and isolate pirated copies. Then they can go after the offending sites directly.

SOPA won’t stop piracy, but it will impact everyone’s access on the Internet. And in that vein, SOPA legitimizes the piracy of 1st Amendment rights, much in the same way that McCarthyism censored free though in the 1950s, simply by accusation of copyright infringement.

NOTE: The views expressed in this post and on this blog are my own. They do not reflect the views of my employer, its employees or its partners.

Note to Fanboys: Don’t Hate the Player, Hate the Game…

2011-11-21T17:01:00.001-07:00

This is a bit of a rant. I get tired of hearing and reading fanboy comments that go along the lines of: “X rules, Y[,Z] drools…”, “You’re just a hater…" Blah. Blah Blah. It’s like listening to reverb on PA system.

My irritation stems from an article I read recently about the potential repercussions of Adobe’s move to stop development of Flash for mobile devices. The article, in my opinion, was well balanced and made the point that while Flash is on the decline, there’s plenty of room for Adobe to maneuver and claim a stake in the RIA/HTML5 world.   What struck me though were the comments. Several of them were antagonistic and claiming author bias against Flash.

The comments also struck a chord with me in that I recently ran into a buzzsaw-like argument with a client with respect to implementing and deploying a No-SQL data solution against trying to do the same thing in one of the big RDBMSs. The debate was that they felt that there wasn’t anything their current RDBMS couldn’t do that the intended NO-SQL system did. Sure, their system could, but it didn’t the specific kinds of things they wanted to do with the NO-SQL system nearly as well. In fact some of the things were bolted on with the technical equivalent of bailing wire and duct tape, and in the long run, cost them more in overhead and maintenance.

After the debate, I took some time to reflect on their argument. The underlying theme that occurred to me was this: they understood RDBMS; they didn’t understand the NO-SQL system we recommended they implement. Bottom line: Go with what you know.

Yet I’ve seen this kind of resistance to various technologies throughout my career. I’ve seen the esoteric debates between the DocBook and DITA content models and architecture, the religious orthodoxy of Windows vs. Linux vs. Mac, and more recently, the pissing contests of iOS vs. Android. The main contention between camps always seems to boil down to “mine or bigger/better/faster/cooler than yours.” My 5 year old twins do it better than anyone, but to hear it from grown-up professionals is like listening to a murder of cackling crows.

If we’re intellectually honest, all of these arguments/dogmatic disputes boil down to the same time-tested axiom: all of us will tend to gravitate to tools/technologies/practices that we’re familiar with, understand, are (reasonably) good at, scratch a particular (set of) itch(es), or just think are cool. Any variance from these, or the suggestion that something is better/faster/cooler than what exists in your comfort zone warrants unabashed trolling, simply because they don’t fit within our particular paradigm.

Tools and technology are applied to solve a specific set of problems, under a specific finite set of assumptions. Don’t like the “evil empire” Microsoft, but appreciate commodity hardware? here comes Linux; like beautiful form, closed, but controlled functionality? Mac seems a good fit. Need structured data without a lot of noise? JSON might be a good fit, however if your data is rich and structured? XML is game for it. Want a single seamless experience for your smartphone? iPhone; want to use an open-source mobile platform with many choices of devices? Android.

The point is this - When a problem veers away from these binding assumptions, or new assumptions are introduced, either the tool or technology must be modified/enhanced to fit these assumptions, or other technologies will be built to replace it.

I’m not entrenched in the idea that “all tools suck, some worse than others”. Every tool and technology has limitations - we need look no further than Joel Spolsky’s seminal work, “The Law of Leaky Abstractions.” For instance, we rely heavily on virtualized environments for our development work. Works great for most Linux and Windows environments, but you’re out of luck for Macs.   Does that mean Macs suck? For working in the virtualized environment we have, it’s a buzzkill; but overall no. We also do a lot of work with XML standards like DITA and DocBook. DITA’s great for its flexibility and reusability; but DocBook still has a place too especially for longer content components where minimalism is not applicable.

But now we can begin to boil down tools and technology down to their real “suck factor”:

In the grand scheme the evolution of technology plays out very much like Thomas Kuhn’s seminal work. In many cases, it doesn’t build on older work, but rather there is a creative destruction and replacement with new technology. During that process, there is a polarization between the two technical/philosophical camps. Eventually, as the new technology attains enough momentum through adoption, the other older technology relinquishes (perhaps not to complete obscurity, and sometimes becoming a small, niche player).

As mentioned above, all tools and technology are constrained by the the underlying assumptions they were built on, and within the bounding box of a specific problem set. Assumptions are rarely ever static – they evolve over time, and when they do, the underlying premise on which a particular tool or technology is built on will start to falter.

For example, Flash works pretty damn well on my laptop with Firefox or Chrome – it works reasonably well on my Android phone, even though it does eat up my battery. Flash basically did things that HTML + Javascript could never do (well). Along comes HTML5, and the underlying assumptions are changing, and they are building in specifications into the standard that will make it possible to create rich internet applications natively (though not right away).

Additionally, smart mobile devices are exceedingly becoming users’ primary access to the internet meaning that lightweight, small footprint applications are incredibly important. Combine these with sprinkle of animosity/frustration/angst/whatever from Steve Jobs and Apple, and the foundations on which your technology are built will inevitably weaken.

Throw in some market forces and what you think is the greatest thing since Gutenberg’s press turns out to be yet another Edsel on the trash heap of “other great ideas”. Case in point: we can argue ‘til the cows come home that BetaMax was far superior than VHS, but that and a couple of dollars will buy you a cup of coffee.

So now that I’ve gone on a somewhat random dissertation of my original rant, I’ll leave all any fanboys with the key message: Don’t hate the player, hate the game. Technology comes and goes. Assumptions change constantly. Try to keep an open mind and recognize when you’re falling into the familiarity trap. Improvise and adapt, or you’ll be left behind like yesterday’s news.

Full Disclosure
In full disclosure, and keeping with the theme of intellectual honesty:

I own an Android phone, because my carrier didn’t support iPhone at the time. I like my Android and continue to go with what I know, and like that it’s built on open source software. I think the latest generation of iPhones with Siri are pretty amazing though.

I’ve used several Linux variants throughout my career, but do most of my work on Windows because that’s what’s on my laptop, and it works well with the tools I use everyday. My last real experience with Mac was back in 1997-1998 when I was in grad school. So I won’t claim any real knowledge here.

I use Eclipse plus numerous plugins for Java development, Microsoft Visual Studio for .NET development (though SharpDevelop is pretty cool too!), and Oxygen for XML development. I prefer Notepad++ over TextPad, and I like Chrome over Firefox and use IE only when I have to.

I use JSON when I’m working with jQuery, Dojo or YUI, and I use XML for structured authoring and when I work with XML databases, XSLT, and XQuery and for things like Rights Management. I like Flex for building UIs quickly for prototypes (hey, demos are in controlled environments, right? :), but recognize its limitations when it comes to device support and will consider my options carefully in a production environment.

I like REST over SOAP over other RPC protocols. RESTEasy rocks for simple apps; Spring for bigger implementations. Eventual Consistency is in; ACID is out.

I still think HTML5 is a work in progress and needs maturity among the “Big Three” browsers and think Flash is still a few years from replacement (Firefox, IE, and Chrome/Safari – OK, I mention 4 but I lump Chrome and Safari together for their use of WebKit). While it’s still very early, I’m eager to see if Google Dart has legs and can displace Javascript (I’m not a big fan debugging other’s JS code when it comes to determining data types or scope).

I’m still trying to grok my way through XProc pipelines and tend to use XSLT 2.0 in somewhat creative ways that it wasn’t intended for, and use Ant for processing pipelines even though I know that it is IO-bound.

And finally, I’m truly into Spanish Riojas right now, and only drink Merlots or Cabernets when I have to :)

HTML5: Well, Maybe.

2011-07-23T16:41:00.001-06:00

I just finished reading an article about Roger McNamee’s bold predictions about social media. Aside from some the interesting business predictions (e.g., don’t invest in new social media startups: that train has left the station) that I mostly agree with, he is strongly emphasizing the emergence of HTML5 as the technology that will drive application development in the future. On this point, I’m not ready to throw my FlexBuilder, Visual Studio, Eclipse and Android SDK development environments in the dust bin just yet. Forget about scrapping my Notepad++ or Oxygen environments, these are keepers for the long, long term.

Yeah, HTML5 definitely has much promise: Canvas alone is just cool. I’ve seen some really interesting things done with this, and it can only get better from there. Yet one cool enhancement for a browser isn’t enough to keep my attention long term, nor is it a game-changer that will revolutionize how users interact with application interfaces.

So what kinds of things will keep me saying, “you had me at hello.”? The big deal for me is looking at the world through the publishing industry’s collective eye: Many of the big publishers are in the midst of what can considered a paradigmatic shift – while print will still be a prominent part of their business model, it won’t be the dominant model. This is a significant change. Publishers will transform themselves from content designers to media conduits.

OK, so how what will HTML5 need to have to be compelling for publishers to adopt it? I see three things, all of which are requirements for the browser vendors to reconcile:

Media Codec Standardization
Support other key technical standards (EPUB, MathML, etc.)
Form-factor scaling

Media Codec Standardization

Right now, there are myriad of audio and video standards like H.264, Ogg (Theora for audio; Vorbis for video), MP3, Speex, AAC, WAV, and so on and on and on. The problem is that none of the current browsers support a common set of these, and even when they do support them, their support varies. Until they figure that out, HTML5 will not be able to leverage its full capability and publishers will be reluctant to adopt it.

References: http://en.wikipedia.org/wiki/Comparison_of_layout_engines_(HTML5_Media);

Native Support for other Standards

OK, this one is a big, huge stretch and probably not going to happen anytime really soon. Well, OK. Ever. That said, these are the types of challenges that publishers have to face currently as well as going forward. EPUB is the biggest stretch only because it leverages HTML (and ZIP compression) anyway, but the capability to embed EPUB in an HTML container would be a big win. Yet for technical publishers, i.e., engineering, science and math publishers, there hasn’t been a good solution for displaying all manner of math equations in browsers – they’ve had to rely on either transforming the equation to a raster image (and only recently to vector images like SVG) or rely on plugins to render the equation. More recently, we’ve seen developments like MathJAX (http://mathjax.org) that rely on Javascript libraries to consume LaTeX scripts and display equations. A bit better, but not quite as elegant as leveraging structural markup.

The bottom line is that this requirement is probably more of a “nice to have,” but for STM publishers, its key to their business.

Form Factor

This is probably the most significant limitation today. It would be one thing if all applications/browsers were bound to desktops and laptops. The reality is that mobile devices, all of which have different dimensions ranging from relatively small smart phones to now tablets means that application interfaces have added challenges to support these different form factors. Today, I would be hard-pressed to recommend HTML5 UI libraries over native mobile OS UI controls.

The Future

Will HTML5 become the preeminent technology platform? My magic 8-ball on my smart phone says “Ask again later…” This resonates the same for me. I’m hopeful that HTML5 can live up to the promise and can become the common technology platform for all applications. But right now, there’s just too many holes in the various browser engines to make it practical. Don’t expect browser vendors to patch these holes quickly. In the meantime, several factors will impede HTML5 adoption: Flash, warts and all, is still largely ubiquitous. Its influence is slowly diminishing, but it won’t go away anytime soon. In addition, Javascript libraries like JQuery, YUI and Dojo are maturing, but I think we’ll need to see how they shake out over time. I’ll defer to Javascript experts to tell me which of these will become integral for HTML5 applications.

Lastly, HTML5 won’t be promoted to true standard status for another 10 – 11 years. This is a lifetime, almost an epoch, for technology. Lots can happen in that time. It’s hard to predict right now what emerging technologies will come along that will impact content and media, but chances are something will.

Update (7/23/2011 05:08 PM MST): Even more articles are coming out suggesting HTML5 will be a boom industry (see http://gigaom.com/2011/07/22/the-html5-boom-is-coming-fast/): Could be real, but could be a bubble. I’m not convinced yet that browsers are up to the task – yet.

IPRM != DRM

2011-06-23T10:49:00.003-06:00

Over the last year, I've been developing strategies that allow publishers to define and identify IP Rights. The big difference between digital rights management (DRM), and IP rights management IPRM is that DRM is about locking down assets to mitigate against piracy. IPRM is about identifying and calculating clearance to use assets for any given context, and enabling publishers to make informed decisions about using specific assets.

ODRL, or Open Digital Rights Language, is a well-established, robust, extensible XML markup designed specifically for this purpose. At it's core is the ability to define relationships between parties, assets, and permissions (i.e., print, display, execute). But it's real power is the ability to express complex permissions that include conditions and constraints. For example, "a licensee can use an asset in a printed book, but the print run is limited to 2,000 copies, and the asset creator must be given proper attribution and will receive two copies of the book prior to its release", or "the asset can be used in print, except that it can't be distributed in North Korea".

This is powerful, and gives publishers the capability to monitor and evaluate rights clearance while the product is in development. Using an XML Database and XQuery, it's relatively trivial to calculate clearances for all assets for a product and to display the information in a dashboard. Editors can monitor the progress of rights clearances against all assets and determine whether to acquire additional rights to use assets that haven't been cleared, or to use other assets instead. Publishers can also track asset usage to ensure that the proper royalties are paid. It also helps publishers in "what if" scenarios: they can easily determine the cost and feasibility of adapting a product for a different market, which will tell them how many of the existing assets are cleared for use in that market and how many remain that either need additional clearance or should be replace with other assets.

Another scenario we're working on is using ODRL for wholly-owned assets. Publishers frequently commission third parties to produce photos, images, and other rich media for which the publisher retains the rights to. They want to reuse these assets for obvious cost savings, however, they don't want to over-expose assets. Frequently, editorial teams are primarily focused on one project or program, and have little insight as to what others are doing, so it's quite possible that an image could be used by more than one product at the same time. Not that this is always a bad thing, but it can lead to over-exposure. Using ODRL to manage access to assets, using embargo dates and other usage information, editorial groups can quickly make informed decisions whether to use an asset or look for another.

Pretty cool stuff

Using DITA for Genealogical Data

2011-06-23T09:03:00.002-06:00

I’ve been working on putting my family history together for the last few years. Most of the genealogical applications have some pretty nice features, but none seemed to have all of the features I wanted. I wanted the ability to manage all of the summary information and relationships (all applications do this), but also cross reference the factual data with individual biographies. And, I want to be able to display the information in different ways and formats – not just the ones supported by any particular application.

I started looking at the format that most genealogy programs store the data into. With few exceptions, they all use GEDCOM, or GEnealogical Data COMmunication. The standard was developed by the Church of Jesus Christ of Latter-day Saints as a means of creating a portable data format to express information about individuals, families and sources (bibliography).

GEDCOM is line-delimited field format that identifies the start of a new record with the number 0. Fields within a record are identified with an incremented number. For example a first level field line would start with the number 1. A subfield (e.g, the given name of a person’s full name) would start with the next highest number. The following is an example record for an individual

  0 @1@ INDI
    1 NAME   Robert Eugene/Williams/
    1 SEX   M
    1 BIRT
      2 DATE   02 OCT 1822
      2 PLAC   Weston, Madison, Connecticut
      2 SOUR   @6@
        3 PAGE   Sec. 2, p. 45
        3 EVEN   BIRT 
          4 ROLE   CHIL
    1 DEAT
      2 DATE   14 APR 1905
      2 PLAC   Stamford, Fairfield, CT
    1 BURI
      2 PLAC   Spring Hill Cem., Stamford, CT
    1 RESI
      2 ADDR   73 North Ashley
        3 CONT   Spencer, Utah UT84991
      2 DATE   from 1900 to 1905
    1 FAMS   @4@
    1 FAMS   @9@

Other than the line/sequencing delimiters, the data structures are pretty free form, and is parser dependent. Even the field names, outside the common set supplied by GEDCOM are parser dependent. So if you use one genealogy tool, it can understand these fields, but if you try to load it in another, it blows up. Gah! Add to that, GEDCOM just isn’t that great for handling rich content like pictures in a biography.

This sounds like a job for XML.

So the first question I had to address is how to model this. I’ve looked at some the of GEDCOM XML sites, and they suffer from the same problems as the text data structure do. Just not enough rich data.

The answer I came up with was to use DITA, which has several things going for it:

I can easily mimic GEDCOM’s data structure with a specialized map
I can extend the model to support other potentially valuable metadata
I can easily model rich biographical content as a topic specialization
DITA’s numerous linking mechanisms work well for the various types of links I would need: internal references within a map, rel-tables, cross-references, external hyperlinks to third-party websites and content.

The first thing I did was to model and create a map specialization that mimics the GEDCOM data. For the sake of brevity, I’ll show a sample of a specialized map. If you want more information, ping me:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE familytree SYSTEM "file:/opt/dita/1.2/dtd1.2/genealogy/dtd/familytree.dtd">
<familytree>
    <title>Schmoe Family Tree</title>

    <individual id="I1" keys="I000001" gender="male">
        <vitals>
            <personname>
                <firstname>Joseph</firstname>
                <firstname type="nickname">Joe</firstname>
                <middlename>Aloysius</middlename>
                <lastname>Schmoe</lastname>
                <generationidentifier>III</generationidentifier>
            </personname>
            <birth>
                <date>
                    <day>1</day>
                    <month>1</month>
                    <year>1968</year>
                </date>
                <location>
                    <placename>The Stork Factory</placename>
                    <addressdetails>
                        <locality>Anytown</locality>
                        <administrativearea>Anystate</administrativearea>
                        <country>USA</country>
                    </addressdetails>
                </location>
            </birth>
        </vitals>
    </individual>

    <individual id="I2" keys="I00002" gender="male">
        <vitals>
            <personname>
                <firstname>John</firstname>
                <firstname type="nickname">Jack</firstname>
                <middlename>Michael</middlename>
                <lastname>Schmoe</lastname>

            </personname>
            <birth>
                <date><day>1</day><month>1</month><year>1948</year></date>
            </birth>
        </vitals>
    </individual>

    <individual id="I000003" keys="I000003" gender="female">
        <vitals>
            <personname>
                <firstname>Jane</firstname>
                <middlename></middlename>
                <lastname type="maidenname">Doe</lastname>
                <lastname type="marriedname">Schmoe</lastname>
            </personname>
            <birth>
                <date>
                    <day>1</day>
                    <month>1</month>
                    <year>1947</year>
                </date>
            </birth>
        </vitals>
    </individual>
    
    <family id="f1" keys="F1">
        <familymeta>
            <marriage>
                <date>
                    <day>1</day>
                    <month>1</month>
                    <year>1967</year>
                    
                </date>
            </marriage>


        </familymeta>
        <child keyref="I000001"/>
    </family>

    <familyreltable>
        <record>
            <indi>
                <personref keyref="I00001"/>
            </indi>
            <famc>
                <familyref keyref="F1"/>
            </famc>
            <fams/>
        </record>
        <record>
            <indi>
                <personref keyref="I00002"/>
            </indi>
            <famc/>
            <fams>
                <familyref keyref="F1"/>
            </fams>
        </record>
        <record>
            <indi><personref keyref="I00003"/></indi>
            <famc></famc>
            <fams>
                <familyref keyref="F1"/>
            </fams>
        </record>
        
    </familyreltable>

</familytree>

The Butterfly Effect of Oracle’s Lawsuit Against Google

2010-08-15T16:06:00.002-06:00

No one knows with any degree of certainty what the outcome will be from Oracle’s patent infringement lawsuit against Google. While the impetus of the lawsuit is largely focused on Dalvik, Google’s mobile VM based on Java, the consequences of legal action will likely reverberate through the larger Java world.

I don’t have a dog in this fight. Yet. On the one hand, Oracle appears to be hell bent on reversing Sun’s decision to release Java to the open source community, according to an InfoWorld article:

It's no secret that Larry Ellison wants to make money from Java, something Sun's execs, whom Ellison held in contempt, was never able to do. It may be that Oracle wants nothing more than a cut of Google's Android revenue -- IDC's Will Stofega told Bloomberg News that the case will probably end with Google agreeing to pay to license Oracle's patents.

On the flip side, it sure looks like Google didn't do any favors for itself either.

Whatever the case, there are potential far reaching implications of Java in the open source world. For one, it will reinvigorate the debate around software patents. More specific to my interests here, XML was, and is, heavily influenced by Java and open source. Think Xerces and Xalan, Saxon, FOP, Ant, and more recently Calabash and Calumet. It also factors in to related technologies like the APIs for eXist and MarkLogic, and fuels reference implementations of standards like DITA (see the DITA Open Toolkit).

My immediate reaction is the cat’s already out of the bag: Java is already open source, which means nothing will change for XML technologies. For now. In the short term, it’s very likely the only ones affected are Android application developers. Longer term, however, Oracle’s behavior might well impact a wide array of technologies, including XML, by deterring developers from using Java in the first place. That would be a huge detriment to XML across many technologies and industries. Let’s hope it doesn’t get to that point.

Technorati Tags: XML,Oracle,Open Source

Cancer Sucks

2010-02-27T13:40:00.003-07:00

My Grandparents died from it. My mother had it (in remission), but now she needs to have a full mastectomy because a genetic marker indicates she's at higher risk to get it again (she has surgery on March 7). And now, a friend and colleague of mine has it. This guy is healthier than 95% of other men his age, has like zero body fat, and is always encouraging everyone to ride with him (more like follow from a long distance). That's just hard to take in and process. While he's the one that has his world rocked, cancer also has a nasty side effect on family and friends. Put it bluntly: Cancer sucks.

Few other spoken words, irregardless of language or dialect, can evoke as much emotion as 'cancer'. Even as scary as influenza and other diseases like AIDS are, and as much devastation they can bring to an individual, a family, a community, and even the world, there's something about cancer that is so viscerally scary to us. For me it's because cancer is, in my non-medical, simpleton view, a mutation of cells. It's not like a virus or an infection where a foreign organism is making you sick, and you can take medication to kill off the nasty invaders - it's your own cells that, for some reason have started going haywire. That's just downright frightening to me. And that cancer is so seemingly random and unpredictable just makes that much more hard to take - you could be in the best possible physical shape and still get it.

Science and medicine have come so far when it comes to curing many forms of cancer. Many people have a good chance of living long, healthy lives if they catch the disease early. But often they have to go down through the depths of hell physically and emotionally to get to the finish line: cancer-free. Yet, they keep their eyes on the prize.

I'm pretty sure my friend will beat this. His odds are really high, and his outlook on life and his current illness are positive, which also increases his chances.

If nothing else, it's a reminder that life is too short, so live it to its fullest. It's so easy to become burdened with frustration, stress, angst, even hatred. In other words, we lose sight of the forest within the trees. I'm probably the most susceptible to this.

In short: Work hard. Play harder. Laugh more. Celebrate with your family and friends. Find something everyday that is good. A sunrise or sunset, a laugh with your kids, a hug or a kiss with your spouse or significant other, a walk in the park with your dog. It's your life - own it.

Apple to Release Tablet PC Today: e-Books are One Target Market

2010-01-27T07:35:00.002-07:00

Apple is releasing their tablet PC today. According to NPR, they plan to include an e-reader. As I mentioned in a previous post, I though that the tablet was the perfect medium for e-books, and would supplant the Kindle, Nook and other current e-book devices. Only time will tell for sure.

The move by Apple is brilliant. It already has a first class distribution model with iTunes pushing out applications and music for the iPhone and iPod. e-Books are just a natural progression.

From an XML publishing standpoint, EPUB is a relatively easy format to render. The DocBook XSLT stylesheets has an EPUB format built in. I'm not aware of one for DITA yet, but I can't imagine that it would be too far behind, or difficult to build.

Tablet PCs - What does this mean for e-Readers?

2010-01-18T21:39:00.001-07:00

HP previewed their new tablet PC at the 2010 Consumer Electronics Show. Apple will release their version later this spring. Sounds to me like this could supplant the Kindle, Nook and other e-Reader devices. It might also be the impetus for EPUB distribution. e-Readers are nice, but they're one trick ponies.

From an education publishing perspective, this could really open the door to new revenue models for schools at all levels to distribute published content. It also opens the door for new models for authors writing content.

Fun With XProc

2009-11-29T10:30:00.002-07:00

I've been so busy with clients over the last 6 months that I haven't had much time to tinker with XProc much. I took the Thanksgiving holiday week off with the hope of having a little time to dabble with the language. Up until yesterday, I didn't open my computer once (has to be a new record) since we were busy with other things. A side note: If you're in Denver before February 7th, I highly recommend you see the Ghengis Khan exhibit at the Museum of Nature and Science.

As I often do, I had already done some preliminary reading beforehand. James Sulak's blog is a must-read. Another very useful and informative website is from EMC: "XProc: Step By Step", originally authored by Vojtech Toman. Even the W3C specification is generally helpful.

The biggest hurdle for me was to stop thinking of XProc working in the same way I think of Ant. While Ant does process XML content, it isn't the tool's principle focus - Ant was principally designed as Java implementation of MAKE tools. For that purpose, Ant has become the de facto standard. Before XProc was conceived, many of us used Ant as a way to control the sequencing of complex XML publishing pipelines. I worked on XIDI, a DocBook-based publishing system at HP, which was principally based on Ant scripts; the DITA Open Toolkit is an Ant-based build environment. For the most part Ant works admirably, but there are limitations. The biggest limitation is the xslt task's static parameter declarations, and the indirect nature by which parameter values are passed to an XSL Transformation through property values. It works, but it can get kludgy pretty fast for complex stylesheets. More importantly, Ant is primarily a developer's tool that acts like a Swiss Army knife that has a tool for just about every purpose. Most of these tools work very well for very specific tasks, but they aren't intended to perform specialized tasks. For that, you'll need to create custom Ant tasks or use other specialized tools. XProc is one of these specialized tools that is designed specifically for XML processing.

So the biggest conceptual difference to grok in XProc (I like this…) is how steps are connected together to form a complete pipeline process. Rather than using target dependencies and explicit target calls like you do in Ant, XProc uses the concept of pipes to connect the output of one step to the input of the next step. It's very much like Unix shell or DOS command line pipelines. For example:

ps -ax | tee processes.txt | more

Since many steps (including the p:xslt step) can have more than one input and one or more outputs (think of xsl:result-document in XSLT 2.0) we need to explicitly bind uniquely named output streams to input streams of subsequent steps. It's very analogous to plumbing, and another way that XProc is different than Ant: Ant's tasks are very dependent on the file system to process inputs and outputs; XProc pipelines are in-memory input and output streams until you explicitly serialize to the file system.

With this I was able to create my first "real" XProc pipeline to generate Doxsl output. Here it is:

<p:declare-step name="doxsl" type="dxp:generate-doxsl-docs"
psvi-required="false"
xmlns:p="http://www.w3.org/ns/xproc"

    xmlns:dxp="urn:doxsl:xproc-pipeline:1.0">  
 <p:input port="source" kind="document" primary="true"   
     sequence="false" />   
 <p:input port="parameters" kind="parameter" primary="false"   
     sequence="true"/>   
 <p:output port="result" primary="true" sequence="false" >   
     <p:pipe step="transform" port="result"/>   
 </p:output>   
 <p:output port="secondary" primary="false" sequence="true" >   
     <p:pipe step="transform" port="secondary" />   
 </p:output>   
 <p:option name="format" select="'dita'"/>   
 <p:choose name="select-stylesheet"> 
     <p:when test="$format='dita'">  
         <p:output port="result" primary="true"   
             sequence="false" >   
             <p:pipe step="load-dita-stylesheet"   
                 port="result"/>   
         </p:output>   
         <p:load name="load-dita-stylesheet">   
             <p:with-option name="href"   
                 select="'../../dita/doxsl.xsl'" >   
                 <p:empty/>   
             </p:with-option>   
         </p:load>   
     </p:when>   
     <p:when test="$format='html'">   
         <p:output port="result" primary="true"   
             sequence="false">   
             <p:pipe port="result"
                step="load-html-stylesheet"/>   
         </p:output>   
         <p:load name="load-html-stylesheet">   
             <p:with-option name="href"   
                select="'../../html/doxsl.xsl'"/>   
         </p:load>   
     </p:when>   
 </p:choose>   
 <p:xslt name="transform">   
     <p:input port="source" >   
         <p:pipe step="doxsl" port="source"/>   
     </p:input>   
     <p:input port="stylesheet" >   
         <p:pipe step="select-stylesheet" port="result"/>   
     </p:input>   
     <p:input port="parameters">   
         <p:pipe port="parameters" step="doxsl"/>   
     </p:input>   
     <p:with-param name="debug" select="'true'"/>   
 </p:xslt>   
</p:declare-step>

Here's a diagram, built with EMC's XProc Designer. This tool is a great way to visualize and start your XProc scripts:

Essentially, I used the p:declare-step declaration so that I can declare it as a custom step (dxp:generate-doxsl-docs), which will allow it to be integrated into other pipelines. It has one option, format, which is used to specify which output format to generate (for Doxsl, 'html' and 'dita' are supported). The first step ("select-stylesheet") evaluates the format option and loads the appropriate stylesheet into the step's "result" output stream. This is used by the second step's ("transform") stylesheet port. The transform's source file (the XSLT stylesheet to be documented) is bound to the root step's source port, as is the parameters port. I also set the stylesheet's 'debug' parameter to true to inject output to the "transform" step's result port.

All of this is done in memory and not serialized to the file system. This is intentional so that other pipelines can integrate this custom step.

I've tested this with Calabash. I still need to evaluate with Calumet.

Right now, these are baby steps. I think that XProc has a lot of potential. I think the next big task is to consider an XProc implementation for DITA XML processing.

DITA’s New Keys and the 80/20 Rule

2009-05-10T16:18:00.001-06:00

Have you ever used the lorem() function in Microsoft Word? How about the rand() function? Do you know all the function keys? Most of us have used Microsoft Word for countless years and don’t know about all of the “hidden” functionality that it offers. Chances are, you’ll know a few of these, but you won’t know all of them simply because you’ve never needed them. Many of these functions are extremely powerful utilities that make Word a versatile application beyond a standard formatted text editor. But they’re available if you ever have the need.

The same is true with some of the new functionality being made available in the forthcoming DITA 1.2 release currently being worked on. Of particular interest is the introduction of keys. Keys provide a way for authors to create addresses to resources through the use of a named identifier rather than to a specific URI pointer. In other words, I can create an easy-to-remember key, like “ms-word-functions” that actually resolves to a URL “http://support.microsoft.com/kb/211982” and link to this URL using the key name in my DITA topic.

Here’s an example of how it works. In my map, I define a topicref and set the keys attribute with an identifier. I also set my href to the physical location of the resource I want to reference.

In my topic file, I can reference the key that's defined in my map:

<topic id="my.topic">
    <title>SampleTopic</title>
    <body>
        <p>
            Lorem ipsum dolor sitamet,
            consectetuer adipiscing elit. Maecenas
            porttitor conguemassa. Fusce posuere, agna
            sed pulvinar ultricies, purus
<xref keyref="ms-word-function">lectus</xref>
            malesuada libero, sit amet commodo magna eros
            quisurna.
        </p>
   </body>
</topic>

Now, when the topic is rendered, it will resolve itself to the Microsoft URL defined in my map. Pretty cool stuff. And powerful too. This has many potential uses: localizers can create translated versions of a resource using the same key reference and resolve the link to a locale-specific version of the reference. Consumers can be directed to different resources based on their profile or context within a website.

From an authoring perspective, there's another neat user story: I can reference a "yet-to-be-determined" resource via a key, and when that resource has been created, the key's definition in the map file will resolve the key reference.

Technically, a key definition doesn't need to be reside directly in the map that references that topic. It can live in an "ancestor" map that pulls in the topic indirectly by way of the map referencing that topic. In fact key values can be overridden: Let's assume that I define a key, called "company-website" in Map A that points to "www.company-a.com", and in Map B, I define the same key as"www.company-b.com". Map B also references Topic-1.dita which contains a keyref to "company-website". Map A references Map B. When the Topic-1.dita is rendered in the context of Map B as the primary map, the keyref will resolve to"www.company-b.com"; when Map A is the primary map, the same topic willreference www.company-a.com.

Map A
Key: company-website = "www.company-a.com"
- Map B
  Key: company-website = “www.company-b.com”
  - Topic-1.dita
    keyref: company-website
    resolves to: www.company-a.com
Map B
Key: company-website = “www.company-b.com”
- Topic-1.dita
  keyref: company-website
  resolves to: www.company-b.com

With all great power comes even greater responsibility. Any time a topic makes use of a key reference, that topic is explicitly binding itself to a map(or many maps), meaning that a topic is no longer a unit of information that is completely independent of any particular context in which it is assembled into. You could make the argument that any reference defined in a topic to an external resource (e.g., an image or a cross-reference to another topic) by definition creates a dependency on that topic. And arguably, the referenced (the endpoint) resource is unaware of the object that is referencing it, regardless of whether it's a topic reference or a cross-reference. But there is an additional dependency in the case of keys: Any map that references a topic with a key reference must define the key. So in a sense, not only does the map (or an ancestor map) need to know about the topic, it needs to discover what the topic is about, specifically related to any key references it points to. Consequently, somewhere along the line, at least one map must define the keys used by a topic. Did you get all that? Imagine what your XML authoring tools, CMS systems, and rendering platforms will need to do to manage this.

This is pretty sophisticated and powerful functionality. But the question is, do you need to use keys and keyrefs in order to use DITA? More importantly, will your tools need to support keys to take advantage of DITA's other capabilities? The short answer is no. In fact, I would expect that keys/keyref -enabled DITA support is still a way off for most DITA-enabled tools. Nevertheless, you can still use DITA with the current tools and get most, if not all, of what you need. Just like Microsoft Word with features like MailMerge, keys and keyref will be there if you need them, but chances are, you can get by without them for most content without ever knowing you missed it.

Finally, the possibility of defining indirect links has opens the door to many different possibilities for dynamically driven, profile- and locale-specific content. This is very cool stuff - the kind of thing XML folks like me get excited about. But from a practical standpoint, there are potential downsides too. Keys and key references add another layer of complexity to planning the authoring, deployment and management of DITA content. In reality, most tools aren't ready for this complexity just yet. So while the the standard is ahead of the game, the rest of the industry will be playing catch up. Still, Ride the Wave.

Content Management Strategies/DITA North America Conference Review

2009-04-29T13:56:00.001-06:00

I wasn’t able to attend many of the session since I was manning the Flatirons Solution booth. Yet from talking with the attendees who visited with us, here are some of the key takeaways:

DITA is here to stay. This is not news, but the key point here is that organizations are adopting the standard in earnest, as evidenced by the 150-200 attendees who came despite a bad economy, and discretionary budgets being whittled to next to nothing. This means that organizations are thinking about DITA as an integral part of their long term strategy.
DITA’s scope is not only Technical Publications. Again, not earth-shattering news. With specializations like Machine Industry, Learning Objects, and gobs of others, DITA is extending its reach to whole industries that haven’t been able to take advantage of XML before now. At the conference, I spoke to attendees in a wide range of industries including bio-tech, and manufacturing.
Shifting focus from Content Authoring to Content Management and Content Delivery Services. This is a fundamental shift. Eric Severson emphasized this point when he demonstrated that Microsoft Word could be used to create DITA for a specific class of users that aren’t the primary audience for more conventional XML authoring solutions. Obviously this raised a few eyebrows in the audience, but the point is that DITA’s architecture is such that even casual contributors, given a few minor constraints in Word, can certainly provide content that can be easily turned into DITA.
DITA will live in Middleware. This is a key point. While the focus of the conference was centered around DITA and content management, there’s more here than meets the eye. I had the opportunity to sit in on the open forum that discussed upcoming v1.2 features. Many of these features are centered around link handling (things like keyref, conref push, and conref keys [conkeyref]). There will be greater emphasis on managing all kinds of linking, including indirect links that could have significant implications on vendors’ existing architectures. While it still will be possible to manage small projects from simple file management strategies (including things like Subversion), larger projects and enterprise-wide implementations, particularly those that want to take advantage of these new features will need more sophisticated applications (read: a content management system) to manage the myriad of link strategies being made available.

Even rendering tools will need to be more sophisticated to support these new features. The DITA Open Toolkit is currently working on a new version (1.5) to support these. Other rendering applications will need to start thinking about how they plan to support these features.

I’ll have more thoughts on this particular topic later. Suffice it to say that there are some key assumptions that current DITA adopters take for granted and make impact how they design and create content in the future.
XML Authoring tools will get more complex. To support all the new features coming in DITA 1.2, DITA-aware XML authoring tools will need to be tightly integrated into middleware systems, particularly the CMS. There will also be a strong emphasis for authoring tools to handle a wide variety of link and referencing strategies. I anticipate that these applications will be more process-intensive, with larger footprints on a user’s PC. I also anticipate that the level of sophistication required to “operate” these tools will be much higher. So the emphasis for XML Authoring tool vendors will have to focus on both features and usability.

This conference was illuminating on many different facets. Even the vendors I spoke to seemed to realize that DITA is a truly disruptive technology that has changed the way the entire industry thinks about XML. In the current economic reality, this is the perfect time to be thinking about what this all means and how organizations can take advantage of these innovations in their environment. Ride the wave.

XProc and DITA: Random Thoughts

2009-04-25T15:50:00.002-06:00

I’ve been following James Sulak's Blog. He has some pretty impressive detailed discussions about using XProc. XProc is an XML pipeline processing language, specifically designed to provide instructions for processing XML content. The Recommendation specifies many different kinds of “steps” that can be assembled in virtually any order to control the sequencing and output from one step to another.

Right now, DITA’s reference implementation, the DITA Open Toolkit (DITA OT) uses Apache Ant and custom tasks to process DITA XML content. One of the principle limitations with the DITA OT is its reliance on XSLT 1.0 and extensions (particularly the Idiom FO Plugin) to handle the rendering.

With XProc-enabled tools like Calabash, it seems like DITA could easily processed using XProc, along with an upgrade of the stylesheets to 2.0.

Content Management Strategies/DITA North America Conference

2009-04-25T08:56:00.001-06:00

I’ll be attending the conference in St. Petersburg, FL. Come visit the Flatirons Solutions booth while you’re there. It should be a very interesting conference.

Eric Severson, CTO of Flatirons Solutions will be presenting a potentially “game-changing” presentation that speaks to lowering the “barrier to entry” into XML authoring. I recommend seeing this one.

DocBook Going Modular

2009-03-28T09:33:00.002-06:00

Scott Hudson, Dick Hamilton, Larry Rowland and I (AKA, “The Colorado DocBook Coalition”) recently drafted a proposal to support “modular” DocBook and presented it to the DocBook TC yesterday. In general, this proposal is in response to huge demand for DITA-like capabilities for DocBook.

Many core business factors are driving DocBook in this direction:

more distributed authoring: authors are responsible for specific content areas rather than whole manuals. Content could be authored by many different authors, even some in different organizations altogether.
content reuse: This has long been a "holy grail" of information architects: write content once, reuse in many different contexts
change management: isolate the content that has changed. This is a key driver for companies that have localization needs. By modularizing their content, they can drive down costs by targeting only the changed content for translation.

Additionally, there are additional downstream opportunities for modularized content:

dynamic content assembly: create "publications" on the fly using an external assembly file that identifies the sequence and hierarchy of modular components rather than creating a single canonical instance.

The following excerpts from the proposal detail the preliminary features (Important: these are not yet set in stone and are subject to change). The final version will be delivered with the 5.1 release.

Assemblies

The principle metaphor for Modular DocBook is the “assembly”. An assembly defines the resources, hierarchy and relationships for a collection of DocBook components. The <assembly> element can be the structural equivalent of any DocBook component, such as
a book, a chapter, or an article. Here’s the proposed content model in RelaxNG Compact mode:

db.assembly =
element assembly {
db.info?, db.toc*, db.resources+, db.relationships*
}

Resources

The <resources> element is high-level container that contains one or more resource objects that are managed by the <assembly>. An <assembly> can contain 1 or more <resources> containers to allow users to organize content into logical groups based on profiling attributes.

Each <resources> element must contain 1 or more <resource> elements.

db.resources =
   element resources {
      db.common.attributes, db.resource+
   }

Specifying Resources

The <resource> element identifies a "managed object" within the assembly. Typically, a <resource> will point to a content file that can be identified by a valid URI. However a <resource> can also be a 'static' text value that behaves similarly to a text entity.

Every <resource> MUST have a unique ID value within the context of the entire <assembly>

db.resource =
element resource {
    db.common.attributes,
    attribute fileref { text }?,
    attribute resid {text}?,
    text?
}

Content-based resources can also be content fragments within a content file, similar to an URI fragment: file.xml/#ID.

Additionally, a resource can point to another resource. This allows users to create "master" resource that can be referenced in the current assembly, and indirectly point the underlying resource that the referenced resource identifies.

For example:

The added benefit of indirect references is that users can easily point the resource to a different content file, provided that it used the same underlying fragment ids internally. It could also be used for creating locale-specific resources that reference the same resource id.

Text-based resources behave similarly to XML text entities. A content-based resource can reference a resource, provided that both the text resource and the content resource are managed by the same assembly.

assembly.xml:

...
<resource id="company.name">Acme Tech, Inc.</resource>
<resource id="company.ticker">ACMT</resource>
...

file1.xml:

<para><phrase resid="company.name"/> (<phrase resid="company.ticker"/>) is a
publicly traded company...</para>

Organizing Resources into a Logical Hierarchy

The <toc> element defines the sequence and hierarchy of content-based resources that will be rendered in the final output. It behaves in a similar fashion to a DITA map and topicrefs. However, instead of each <tocentry> pointing to a URI, it points to a resource in the <resources> section of the assembly:

Creating Relationships Between Resources

One of the more clever aspects of DITA’s architecture is the capability to specify relationships between topics within the context of the map (and independent of the topics themselves). The DocBook TC is currently considering several proposals that will enable resources to be related to each other within the assembly.

The Benefits of a Modular DocBook

There is a current mindset (whether it’s right or wrong is irrelevant) that DocBook markup is primarily targeted for “monolithic” manuscripts. With this proposal, I think there many more possibilities for information architects to create new types of content: websites, true help systems, mashups, dynamically assembled content based on personalized facets (Web 2.0/3.0 capabilities), a simplified Localization strategy like that which has been advocated in DITA.

What’s more: the design makes no constraints on the type of content resources referenced in an assembly: In fact they can be any type: sections, chapters, images, even separate books (or assemblies) to mimic DocBook’s set element.

The design takes into account existing DocBook content that currently exists as “monolithic” instances, but is flexible enough to support other applications like IMF manifests for SCORM-compliant content, making it easy to create e-Learning content.

As the first draft of the proposal, I would expect that there will be changes between now and the final spec. Yet, the core of the proposal should remain relatively intact. If you would like to get involved or have other ideas, let me know. Stay tuned.

Technorati Tags: DITA,DocBook,Modular DocBook

del.icio.us Tags: DocBook,Modular DocBook,DITA

XMetaL Reviewer Webinar

2009-02-13T10:32:00.001-07:00

I attended a webinar yesterday hosted by Just Systems for their XMReviewer product. The problem space is that conventional reviewing processes are cumbersome and inefficient, particularly when there are multiple reviewers that need to review a document concurrently. In general, most review processes rely on either multiple draft copies being sent out, one to each reviewer, and then it’s up to the author to “merge” the comment feedback into the source.

With XMReviewer, the entire review process is centralized on the XM Reviewer server: Reviewers simply access the document online, provide their comments and submit. What’s really cool is that reviewers are notified in almost real time when another reviewer has submitted their comments and can integrate their fellow reviewer’s comments into their own.

The real advantage is that authors have all reviewer comments integrated and merged into a single XML instance, and in context. Very Nice.

There’s also a web service API that allows you to integrate XMReviewer with other systems including a CMS that can automatically deploy your content to the XMReviewer server.

There are some nice general reporting/auditing features built in as well. However, I didn’t see anything that would allow me to customize the reports or to manipulate the data, but I wouldn’t consider that a show stopper.

For folks used to “offline” reviews, e.g., providing comments at home, or on a plane, this won’t work for you as it is a server application. Nonetheless, having the ability to have full control and context for review comments far outweighs the minor inconvenient requirement of being online and getting access to the server (most companies these days have VPN, so it’s not a showstopper). Though, I can envision the possibility of the server downloading and installing a small-footprint application that would allow users to review the document “offline” and being able to “submit” the comments back to the server when the reviewer is back online.

The only other limitation right now is that XMReviewer doesn’t support DITA map-level reviews in which you can provide comments on multiple topics within a map. This is currently in development for a future release – stay tuned.

Overall, XMReviewer looks great and can simplify your content review process. Check it out.

Microsoft Live Writer Convert

2009-02-11T11:39:00.001-07:00

After reading a few blogs here and there, I’ve seen a few posts about Microsoft’s Live Writer for creating blog posts. Always on the lookout for new toys and tools, I decided to download it and try it out. I gotta admit, I’m sold. This is a pretty nice application that allows me to work offline to write and edit my posts and when I am ready and able to connect, I simply push the “Publish” button and away it goes. Sweet.

It’s simple to install, and simple to configure to point to virtually any blog host out there. In short: It just works.

This is what software should be like. It should solve a particular set of problems and only those problems well without requiring massively complex installation and configuration steps. The interface should be intuitive (Live Writer is wickedly intuitive) and should help rather than hinder me in my productivity. This tool does that. Well done, Microsoft!

Implementing XML in a Recession

2009-02-09T21:27:00.001-07:00

With the economic hard times, a lot of proposed projects that would allow companies to leverage the real advantages of XML are being shelved until economic conditions improve. Obviously, in my position, I would love to see more companies pushing to using XML throughout the enterprise. We’ve all heard of the advantages of XML: reuse, repurposing, distributed authoring, personalized content, and so on. These are underlying returns on investment for implementing an XML solution. The old business axiom goes, “you have to spend money to make money.” A corollary to that might suggest that getting the advantages of XML must mean spending lots of money.

However, here’s the reality: implementing an Enterprise-wide XML strategy doesn’t have to break the bank. In fact, with numerous XML standards that are ready to use out of the box, like DITA and DocBook for publishing and XBRL for business, the cost of entry is reduced dramatically compared to a customized grammar.

And while no standard is always a 100 percent perfect match for any organization’s business needs, at least one is likely to support at least 80 percent. We often consult our clients to use a standard directly out of the box (or with very little customization) until they have a good “feel” of how well it works in their environment before digging into the real customization work. Given that funding for XML projects is likely to be reduced, this is the perfect opportunity to begin integrating one of these standards into your environment, try it on for size while the economy is slow, and when the economy improves, then consider how to customize your XML content to fit your environment.

Any XML architecture must encompass the ability to create content and to deliver it, even one on a budget. Here again, most XML authoring tools available on the market have built-in support for many of these standards, with little to no effort, you can use these authoring environments out of the box and get up to speed.

On the delivery side, these same standards, and in many cases the authoring tools have prebuilt rendering implementations that can be tweaked to deliver high-quality content, with all of the benefits that XML offers. In this case, you might want to spend a little more to hire an expert in XSLT. But it doesn’t have to break the bank to make it look good.

The bottom line: A recessionary economy is a golden opportunity to introduce XML into the enterprise. In the short term, keep it simple, leverage other people’s work and industry best practices and leave your options open for when you can afford to do more. Over time when funding returns, then you can consider adding more “bells and whistles” that will allow you to more closely align your XML strategy with your business process.

DOXSL: Reflexive Code Documentation and Testing, and other random XSLT thoughts

2009-02-06T23:22:00.007-07:00

One of the cool things about Doxsl is that I can test it on itself. Since Doxsl is an XSLT application (v2.0), I can create documentation using itself. I'll be posting these on the Sourceforge project website soon - when I finish documenting my own code. Hmmm... walking the talk and eating your own dogfood at the same time - who woulda thunk it?

There's something about reflexive tools that is just pretty cool. I built another application to document the DocBook RelaxNG schemas into DocBook.

The Doxsl DocBook stylesheets are coming along. If I can manage to get some free time at night, I might be able to finish these in about a week. The one thing I really need to do is check out xspec to see if I can write test cases against the code. I've tried XMLUnit about a year ago, but the critical difference is that it tests the artifact of the transform, rather than the code itself. Implicit testing is better than no testing at all, but it doesn't mean that it's optimal. I love JUnit and NUnit for testing my Java and .NET code, and it's great for the large enterprise-wide projects I work on. While Doxsl is just a teeny, tiny little application (tool is more like it), there is enough code right now that even simple changes can cause big problems. I'll let you know what I think about xspec when I've had a chance to tinker with it.

Another XSL application I've been working over the last year or so is an alternative to the DITA Open Toolkit. The OT is OK as a reference implementation, but it can be a bear to work with even to handle minor customizations. Part of the problem, in my opinion, is that the OT's stylesheets are dependent on the Ant scripts that drive it. In fact, it takes some fancy footwork to get the stylesheets to run outside of the ant environment. And here again, Ant is the tool for creating a consistent and reliable sequence of build steps for a development environment. Where it falls short is dealing with sophisticated XSLT applications that have lots of parameters (optional or otherwise). The parameters have to be "hardcoded" into the XSLT task. Not my idea of extensible.

Add to that: the stylesheets are still using XSLT 1.0 - ehhh. I'll use 1.0 if I have to (thanks Microsoft). There's just so much more that 2.0 provides that makes stylesheet development much, much easier. At any rate, I've been working on my own implementation of DITA using XSLT 2.0 and with relying on Ant. HTML and CHM are working, FO is the hard part. What I find interesting is that I can process a map containing over 160 topics into HTML in about 20 seconds with my stylesheets. It takes over 2 minutes with the OT! The results are anecdotal , and I haven't really tested the stylesheets on anything really big, but I like what I see so far (in fact the DOXSL website uses DITA and my stylesheets to render it).

DOXSL Shout out

2009-02-04T20:43:00.003-07:00

I recently found a post by a former client of mine, James Sulak who had some very positive feedback for my open source project, DOXSL. Check out his post. Thanks for the shout out, James!

On that front, I have been working on a new release. There are a few bugs I've discovered when I started processing some DITA stylesheets, particularly when trying to look for overrides using DITA's matching patterns (e.g., *[contains(@class,'- topic-type/element-name ')] - things got borked with speicializations when the class tokens contain the "parent" token.

There's still the other things on my roadmap to work on:

DocBook output (I have some designs in mind, specifically around modularity)
The Comment Stub Generator: this is a high priority for me
Comment Collector: This is similar to what .NET does when it compiles code documentation into a single XML file. The intent is to make the XSLT less "noisy"
There are additional validation ideas that Ken Holman suggested that I'd like to build in.

Progress is slow, given that my day job still takes precedence. Stay tuned.

It's Been Awhile

2009-02-04T20:09:00.002-07:00

It's been a long time since I've posted to this blog. Life and work have been a bit insane. Work-wise, I've been working on a very complex project that marries DITA modular architecture with a more conventional monolithic document approach. It's been an interesting excercise - one that I will discuss in more detail later.

Personally, my father passed away on December 17th. He was in ICU since the beginning of December, and had been ill for quite some time before that. Needless to say, blogging was the last thing on mind.

I did have a presentation prepared for the XML 2008 conference, called "Optimizing Schemas for Enterprise Content", which I had intended to attend, but couldn't make it due to my father's illness. Thankfully, Eric Severson graciously and adeptly stepped in for me. From all accounts, it was received well. The premise behind the paper is that schemas shouldn't be the end-all/be-all for constraining content models (e.g., validation). Instead, I offer other strategies for controlling content.

In the meantime, I'm getting back in the swing with the DITA TC, hopefully having more time to devote to moving the v1.2 spec along. I'm also working on a proposal for creating "modular DocBook." There are some interesting concepts I'm playing around with in the proposal. Hopefully, I'll have enough to present to the DocBook TC later this month.

So much to do, so little time.

Off Topic: Kate Earley's Website

2008-06-04T20:28:00.004-06:00

My sister, Kate, is an extremely talented artist (me on the other hand, have trouble drawing stick figures). She started up a new website with some of her work. Hopefully, she'll post more of her work for all to see. It's pretty amazing stuff. Here's the link: http://www.infinityillustrations.com/index.html

I Don't Care How the Sausage Is Made

2008-05-03T21:14:00.000-06:00

There's a common notion that you don't want to know how sausage is made if you like eating them. In fact, there's a butcher (reminiscent of days gone by) down the street from where I live who makes all kinds of sausage from "old family recipes". What I read into this sort of thing is the butcher is saying, "Don't ask, I'm not telling." To which my reply would be, "I won't, because it tastes good."

In much the same fashion as I don't want to know what goes into the sausage, many writers I've had the experience of working with or consulting for really don't want to know or care about angle brackets, tags, attributes, containment, content models, namespaces, DTDs, schemas or processing instructions. In other words, "Let me write, and get out of my way!"

Part of this sturm und drang over markup languages is partly the unintended consequences of sophisticated desktop publishing tools that enable virtually anyone to create publication-ready content. The problem is that structure and semantics takes a back seat to formatting and presentation. And for the last 15-plus years, DTP applications have been the main tool in the writing tool chest. It didn't matter that the underlying format was proprietary - it just worked.

But the price of "freedom" with using these tools resulted in additional overhead to ensure that "publication-ready" content conformed to internal company standards for style and format.So here comes XML, with its promise of unlocking content from format and proprietary binary formats read only by proprietary applications. Structure and semantics now matter. In so many ways, this is a great thing. Consistency can be enforced by the underlying schema - no rogue "Heading 1" styles followed by a "Heading 3" or funky new format styles that deviate from the corporate template. Better yet, no more tedious hours reformatting content from other sources like OEM content, or being limited to "one-off" copies from originals.

The rub is that the tools needed to write XML put more onus on the writer to understand the content model. Even if these tools hide or abstract the content model, there is still a significant change management issue to train users that structured content authoring is signficantly different than what they might be used to. In exchange for "creative freedom," back-end publishing systems now have a reliable content to work with. But the problem is that some folks creating the content feel like they've gotten the short end of the stick. For those that do feel short shrifted, it feels like they're right in the middle of the sausage factory.

The real challenge is to develop tools that make XML content creation simple. Users don't want to know a hyperlink or image is marked up, they simply want to identify the URL and link text. Similarly, some users don't care if the content model is using a CALS or OASIS Exchange or HTML table model, they want to insert rows and cells. In many respects, this seems to suggest that the current markup complexity is getting in the way.

Many of the tools do have WYSIWYG-ish views into the content. Yet, as Eric Armstrong pointed out, even these tools fall short and tend to confuse users when they want to insert a particular element or structure not allowed in the context of the current cursor location.In many regards, this is too much overhead for the average user. They want to have a tool that allows them to insert paragraphs, lists, tables, images and section headings. One suggestion might have a higher level abstraction layer that all tools can easily support, and build the more complex XML markup under the covers.

One possible approach might be to use HTML-like WYSIWYG editors that support basic content structures. Under the covers, it can convert the HTML into more complex structures required by grammars like DocBook and DITA. In fact, the Interoperability Framework Scott Hudson and I have worked on uses this same approach.

While this type of approach takes away some control over very specific semantics, it reduces the complexity and clutter that more structured models introduce. That doesn't preclude editing tools to be smarter about how they enable more specific semantics. Much in the same way many HTML editors display a popup dialog for users to specify the URL and text for a hyperlink, tools can provide users with other dialogs for specifying (more like classifying) an abstract span element to indicate more detailed meaning.

This is not to suggest that "non-narrative" content, like data structures or configurations that are are used by many modern applications will want to use this approach. However, publication level markup can take advantage of this type of approach to simplify how users create content, and honor their request to remain unaware of how the sausage is made and will be happier to create content in a model they do understand.