Doppler Shifted Sheep

How to avoid (DY)LD_LIBRARY_PATH with JNI

2011-11-12T15:01:00.000-05:00

Some of us are stuck with JNI. We've got a heap of code in C++. Sometimes, we have a heap of code with enough floating point computation in it that the speed advantage of native code is inescapable.

In the simple case, JNI isn't too bad. You make the header, you build the code, and you have a shared library. You can use java.library.path and System.loadLibrary, or you can use System.load and avoid any extra settings when launching java.

Things are not so nice, however, if your C++ code has dependencies on other shared libraries. Listing the directories containing those libraries in java.library.path is not enough. You'll still get exceptions claiming that your JNI library cannot be loaded. To get rid of those exception, you have to modify PATH, LD_LIBRARY_PATH, or DYLD_LIBRARY_PATH (on Windows, Linux, or OSX respectively). This leads to a world of hurt, particularly when people want to run your code inside a container such as Tomcat.

For Windows, there's a solution involving a Win32 hairball called 'delayed loading'. That's not what this posting will help you with. Perhaps I'll do a writeup some day. At Basis, we worked that out years ago.

Until now, however, we've suffered with LD_LIBRARY_PATH and DYLD_LIBRARY_PATH.

Well, we're not going to suffer any longer. The solution to these problems leaked, finally, into my consciousness, and I've built a testbed to show it off. Here it is on github:

https://github.com/bimargulies/jni-origin-testbed

The code in here shows off the existence of linker options and tools that avoid the need to set those environment variables. On Linux, the critical feature of 'ld' is '-rpath $ORIGIN'. Watch out; it takes some care to actually get the characters '$' 'O' 'R' ... into the ELF file.

On MacOSX, the situation is more complex. MacOSX has this idea that every shared library has a single, proper, installation location, called the 'install path.' Things linked to shared libraries pull that path from the Mach object file, and store it for use at runtime. If you are willing to structure your code as a Framework that follows Apple's conventions for a fixed installation, this all works great.

If not, then there turns out to be a solution. A command, 'install_name_tool', allows you to patch the location where one library (your JNI library) looks for another (its dependencies). The special token '@loader_path' expands to the location of the library itself. Thus, you can express the location of the dependencies by relative path. So long as the JNI libraries live in a fixed location relative to their dependencies, all is well.

A maven seder

2011-07-07T08:58:00.001-04:00

The other day, Stephen Connolly of the Maven PMC was moved to remark, 'Meh! there's a lot of maven haters out there...' This led me to the following musings, which might be thought of 'A Maven Seder, or, the Four Children of Maven.'

Maven may be one tool, but developers encounter it under very different circumstances. Here's a way to look at those circumstances under four headings, and perhaps reveal something about why developers end up with such wildly different attitudes.

The Developer Who Does Not Know How To Ask

Some developers find Maven by adding themselves to a project that already uses it in some sane and stable fashion. To them, it just works. They type 'mvn'. Their tests run, their jar files appear. Maybe they even follow a recipe to push a release. Likely, if they have this experience, the project they are joining has a Maven-friendly shape: sources in the standard layout, lots of dependencies, no need for complex and exotic scripting. These developers may not fall in love, but they may wonder what all the fuss is about.

The Simple Developer

The simple developer actually sets up a Maven build, but has no particular problem in doing so. His or her inputs are some java sources and some 'in classpath' files. The dependences she or he needs are all sitting out there on Maven Central. The output of the process is a jar file, or, at most, a relatively straightforward release package. He or she copies a simple pom.xml file, makes a few tweaks, and all is well. A copy of Jenkins produces instant automated builds, and a copy of Nexus or Artifactory speeds up the process. No giant stress, no giant strain. So long, of course, as nothing goes wrong. The moment this person has to graduate from 'mvn dependency:tree' to 'mvn -X', they are at risk of becoming ...

The Rebellious Developer

These are the folks who fill blogs and mailing lists with warnings to give Maven a wide berth. How do they get this way? Here are some of the ways:

They are given the job of taking a complex build with some other build system and adapting it to Maven. This is a hard job at best. It's made hard and annoying when the Maven-mavens insist on replying to all complaints and questions with variation on, "You should just restructure your entire system to fit in with the Maven way of doing things," instead of "Well, it can be a hard job to adapt a complex, highly-scripted build to Maven. Are you sure you have to? Maybe you just want to learn to publish some artifacts."
They have a build that has some incompressible complexity in it. They need to create multiple slight variations for different targets, or complex JNI, or interactions with large datasets.
They run into a problem. Maven works great when it works. When it doesn't work, often the diagnostic process leaves a great deal to be desired. Things seem to happen 'by magic.' If the ordinary log messages aren't informative, the alternative is -X, which spews a gigantic amount of content. At best, this is a needle-in-haystack situation.

The Wise Developer

At the other extreme, we have the developer who has become completely assimilated. He or she has internalized the lifecycle, and so knows exactly what to expect. Little is mysterious or surprising. Chances are, she or he has written a plugin or three, having figured out that writing plugins is often much easier to do (and debug) than convincing the more obscure options on the more obscure plugins to cooperate with each other.

This person is, of course, in danger of turning into one of the voices that ticks off the Rebellious category. Isn't that what family dynamics are like?

Lessons for the Maven Community

In my view, there are some simple lessons here. Maven evangelists should be mindful of what is realistic. Better documentation, error messages, and log message never hurt.

It's easy to write these sentiments, of course. Acting on them is another story. The Maven ecosystem is now a gigantic stack of code, and the committers for it sometimes seem like short-order cooks in a very busy diner.

How to get a reputation on StackOverflow

2009-11-15T16:21:00.004-05:00

A little while ago, I noticed that people were posting questions about Apache CXF on stackoverflow.com. It's easy enough to answer questions, but I wanted more. I wanted to be able to herd the sheep: clean up tags, edit confused questions, and generally improve the quality of the resulting knowledge base. This would require a significant store of reputation, in StackOverflow parlance, so I set out to accumulate it.

It doesn't take much observation of the site to learn that reputation comes from answering questions. Sure, if you ask really good questions, people will vote them up and your reputation will advance. However, most questions don't get upvoted, and who has time to sit around thinking of questions?

But it's not enough to answer questions. Adding the 3rd or 4th answer to a question is not going to get you votes, and votes are what you need. You have to get in there and be one of the first two to post a concise, helpful, answer.

Visiting the site, I found that it was not so easy to take a timely snipe at questions I could answer. The good candidates were buried in the giant mass of questions on subjects where I knew nothing, cared less, or both.

The first solution lept out at me: set up some 'interesting tags'. Then click on the tab that shows only unanswered questions in those tags. Seemed simple.

It didn't work. That tab is nearly entirely populated by questions that have three or four answers. They are still there because no one voted for, or accepted, any of them. In theory, that means that these are mediocre answers, and I could add a superior answer and get some votes.

No such luck. Usually, these are perfectly sensible answers. It seems as if there are not enough site users who bother to vote, and nowhere near enough question-posters who bother to accept answers. The result is usually an impenetrable clutter.

That led me to my second strategy. I started collecting a very large set of 'ignored tags'. To filter out what I didn't want to see, I had to add many, many, tags. Why? StackOverflow uses a flat taxonomy. To avoid seeing Visual Studio questions, you need to exclude about 10 different tags for 10 different versions or aspects of Visual Studio. Repeat this for all the other subjects of disinterest, and soon enough you, like I, will have a long list of ignored tags.

The reward for this labor is that the 'recent' tab under the unanswered questions is suddenly useful. Now, the most recent questions of possible interest appear at the top of the page, just waiting for a quick answer.

Sure enough, this allowed me to pile up over 500 points of reputation in a few days of visiting the site at spare moments.

My points are nothing like evenly distributed over my answers. I posted plenty of answers that collected no votes, and thus no reputation. Many got a vote or two. The big winners were two very simple answers to simple questions: A brief lesson on logarithms and quick reminder of an option to the linux mkdir command.

Go figure!

Anyhow, I've now got the privilege of fixing bad tags, and if this keeps up, I can look forward to permission to edit other people's questions.

One final hint: stay away from meta.stackoverflow.com, unless you are suffering from insomnia or have an urge to count the angels dancing on Zippy.

Mysteries of Maven

2009-01-29T14:20:00.004-05:00

Maven is a very interesting beast. It seems well-established as the successor to ant as the best practice solution for java builds. It has the cardinal virtue of doing simple things simply. Very large ant build files can collapse to very small maven pom.xml files.

It does simple things simply, and the complexity curve is not far from linear as you try more complex things. However, there's a knee in that curve, and I spent this morning with it firmly planted in a sensitive part of my anatomy.

Consider the maven-antrun-plugin. It's typical of maven that it handles the special capabilities of its competitors by absorption rather than religious argumentation. The antrun plugin allows you to embed ant build tasks in your maven build. If you don't feel like figuring out out to reexpress them in purely maven terms, you can just keep them as-is.

Now, I happened to have a multi-directory structure to move to maven, and there happened to be two places in the structure where antrun seemed the better part of valor. The first one worked fine. The second one did not. After much hair-pulling, I was able to demonstrate to myself that the dependencies that I had listed in the second case were ignored.

"Dependencies?" you ask? Maven plugins, like maven-everything-else, come with a list of dependencies that are added to their classpath. If you want to use ant facilities that aren't in the core of ant, or custom tasks, you have to package them up as maven artifacts and list them as dependencies.

Skipping to the end, there's a bug in core maven that causes all but the first set of dependencies on a plugin to be ignored when multiple projects are evaluated together. Fair enough. Things have bugs. What seemed interesting to me was how wildly opaque this bug turned out to be.

The maven-dependency-plugin didn't reveal much, since it doesn't seem interested in plugin dependencies, only module dependencies. The debug log (-X) didn't add anything to the situation. Like many debug logs, it is a compendium of things that seemed interesting to a developer chasing a particular issue at a particular time. I found myself wondering, "what would fail-soft consist of in this case? Could there be some other medium-level debug output more adapted to people debugging their POMs than to people debugging the inside of maven itself?"

Debugging is a understudied problem.

And there turn out to be two open bug reports on this against the plugin, even though it isn't the plugin's fault.

Another Voyage Through Eclipse

2009-01-17T12:12:00.004-05:00

I spend a significant fraction of my working time staring at Eclipse. I fear that I'm a lifer. Even though rumors reach my ears that Netbeans has become usable, I just can't motivate myself to consider an alternative.

Some years back, I did some development in Eclipse. I was able to convince a client of ours that the best way to set up a linguistic workbench was Eclipse. We built a set of plugins. We submitted a raft of bug reports. Even a few patches. It worked. Sadly, the client decided that they preferred .NET, so off they went. Given the problems we had getting Arabic and Hebrew text to work right in Eclipse, and my recent discoveries, I'm not entirely sure that they were crazy.

All that is background. In the last week, I set out to build a simple tool for linguistic annotation. That is, display some text, and give the user very convenient, keyboard-based, tools to mark words.

I found http://www.vogella.de/articles/RichClientPlatform/article.html, which looked to be a good solution to the fact that the books can never keep up with the ongoing redesign.

And then the fun began.

The fundamental thing I wanted in my program was an editor. Historically, Eclipse editors have tended to be more entangled with the full IDE than other kinds of components. This problem turned out not to be solved in 3.4.

The turorial explains how to use the most recent Eclipse mechanisms for setting up the File menu and all of that. Unfortunately, the tutorial does not mention that as soon as you add an editor, the simple prescription fails.

To add an editor to an RCP application, you add an extension under org.eclipse.ui.editor. And, the next thing you know, you have a file menu, whether you want one or not. Before you can add the File Open that you want, you have to get rid of the old one. All this is complicated by the transition from 'actions' to commands. Under the 'new administration', things are supposed to be organized around commands, but there remains, not surprisingly, a mountain of code, examples, and documentation from the old universe.

Thankfully, a bit-o-googling revealed a recipe for this.

Next? Make Arabic look right.

When I last worked in this environment, the top of my lap was a Windows box. (I've since spent some time chained to a copy of Ubuntu and moved on to a Macbook Pro.) There were some problems with Arabic; the SWT people hadn't thought all that much about an application running with an overall English locale that needed to do a clean job of displaying some RTL text. However, we got reasonable results on Windows.

Mac OS X? Uh, oh. SWT, it seems, has some fundamental assumptions about how systems will support RTL languages. Even though RTL languages (in general) work fine on Mac OS, there is an impedance mismatch with SWT. Net result: there is no simple way to set up a StyledText component to display Arabic or Hebrew correctly. Luckily for me, I don't absolutely need this tool to look beautiful on MacO OS.

Maven, Eclipse, Checkstyle, PMD, Oy, Vey.

2008-12-12T18:24:00.004-05:00

Code quality is a wonderful thing. You can run PMD and Checkstyle to ~~enforce consistency~~ check for inconsistencies and problems.

If you're like me, however, you really hate to edit for two hours and then get handed a long list of complaints. Or, worse, to ask your IDE to neaten up the code and find out that it's been 'neatened' to the tune of 100 Checkstyle complaints.

I use Eclipse. Eclipse has plugins for Checkstyle and PMD, and a highly configurable set of tools for formatting and code cleanup. Configurable is nice, but it's a tiresome process to establish the right configuration for all my projects and workspaces.

I wish that there was some automatic way to decorate an Eclipse workspace with all the right settings to match up with how some project uses Checkstyle and PMD.

Luckily for me, Dan Kulp of the Apache CXF Project did a vast job of reverse engineering to figure out how all the Eclipse configuration works. The Maven POMs for CXF arrange for the same Checkstyle and PMD configurations to be used for Maven and Eclipse, and set up the Eclipse formatting and cleanup options to match.

I did a bit of reverse engineering of Dan's reverse engineering here.

Aside from the goodness of this when working on CXF, I was able in an hour of so of work to adapt it to my day job.

Web Service Clients in Browsers

2008-01-23T15:38:00.000-05:00

A quick Google reveals several toolkits for talking to Web Services from JavaScript. The purpose of this post is to toot the horn of a new one. Apache CXF version 2.1 will include support for browser-resident JavaScript clients, and you can try it out, right now, by downloading a snapshot.

What's so special about the CXF JavaScript support?

To begin with, it really does handle standard-conforming web services. Several of the other packages out there lack true support for XML namespaces and XML schema. Some of them require, for example, that the messages use fixed prefixes for XML namespaces, contrary to the XML spec. It handles some of the more complex XML schema cases (xs:any or xs:anyType), though there are some extreme cases (xs:choice) that it doesn't do.

Along with the rest of CXF, it comes under an Apache license, so that those of you with an allergy to the GPL need have no fear.

It offers a very convenient mechanism for delivering the JavaScript code to the browser. A URL like:

http://myservice.com?js

delivers the actual JavaScript. Put that URL in a script tag, and you're all set.

It handles MTOM attachments. Why would you want MTOM attachments in a browser, where you can't have any binary at all? You would want them to attach non-XML-1.0 text without paying the base64 tax.

Note that this is a code-generation scheme. CXF generates a JavaScript 'class' (a constructor with some functions on its prototype) for your service, and more JavaScript classes for your bean classes. Incoming messages are deserialized from XML into these objects, and outgoing messages get the reverse treatment. It is not a dynamic scheme in which you hand in a set of parameters and the code tries to map them to some operation of a service.

Using full web services gives you much more flexibility than using REST or other simplified mechanisms. If you have a use for all of that, please give the CXF JavaScript facility a whirl.

XML 1.0 versus Web Services

2008-01-20T20:18:00.001-05:00

On the cxf-user mailing list, we see a question over and over:

"Why can't I sent an escape character to my web service?"

On further examination, the symptoms are always the same. The schema type is xs:string. The data contains a C0 control character, such as Form Feed or Escape. And the results are an error or that the character disappears. And the OP is annoyed.

The OP generally gets even more annoyed when we reveal the ugly truth: there's no good solution to this.

Why?

Long ago, when the world was young, the W3C created the specification for XML 1.0. They chose Unicode as the fundamental representation of text, so that everyone's favorite poem could be captured in an XML document.

However, they made the mistake of actually reading the Unicode specification in detail (an exercise for the insomniac if ever there was one). And they spotted the presence of the dusty, musty, old-fashioned ASCII control characters.

Having noticed them, they banished them from XML as per section 2.2. You may think I'm being bombastic with the term 'banished,' but it's really the simple truth. An XML document cannot contain any characters outside of production [2] of section 2.2. A character represented with an & is still a character. So,  is no more permitted than the same character sitting there, literally, in the document.

Here comes the nasty part. Consider a simple Web Service. The service has a WSDL, the WSDL has a schema, and the schema specifies a string. An xsd:string. The web service binding will, quite cheerfully, map this to a Java String or a C## string. Any now the fuse begins to burn...

Java Strings and C# strings hold any Unicode characters. Not just the ones that are valid in XML. xsd:string values, on the other hand, describe the XML content model, and so cannot contain control characters. By a certain logic, toolkits should refuse to map xsd:string to plain old String data types. They should map them to some class that checks for compliance with XML. You can imagine how popular that would be.

This problem seems to have become more visible of late. Why? Because more XML parsers are paying attention to section 2.2 of the specification. A few years back, all the common Java XML parsers ignored the restriction, and only the Microsoft DOM made a conspicuous point of rejecting invalid characters. Now, mainstream parsers, as used by mainstream web service toolkits, are paying attention. In the case of WoodStox, sadly, the attention being paid consists of discarding the rejects rather than diagnosing them.

Application developers are not happy. They want to send document content through their web service, without worrying about the occasional stray form feed.

What is to be done?

Well, there's XML 1.1. It does not forbid these characters. However, all of the web service specifications demand XML 1.0, and there's no sign on the horizon of any alternative. So there's no help there.

There's base64. Particularly for short strings, xsd:base64Binary is the only practical solution. Sadly, data bindings for web services don't give you much help here. You'd like to @nnotate that you want to have a Java String as the Java datatype, xsd:base64Binary as the schema datatype, and let the generated code take care of everything else. No such luck. You can call mystring.getBytes("utf-8") and pass the resulting byte[] into your service, and reverse the process on the other end.

Be careful with that UTF-8, by the way. In JavaScript, in particular, there are many base64 packages floating around that assume that the data will be ASCII.

If you have a lot of data, it's time to contemplate attachments.

I just finished teaching CXF's JavaScript client generator to handle MTOM attachments for this purpose. Half the battle was the sloppy documentation on MTOM on the web. Beware of Metro's documentation here. It has bugs in the example of the wire traffic and bugs in the schema for xmime:ContentType. Other than that it's quite informative.

By the time I was done, a Java side DataHandler with content type of 'text/plain;charset=utf-8' was mapped, bidirectionally, to a JavaScript variable in the browser, with an MTOM attachment in between.

Anyone who needs to ship arbitrary textual content through a web service has to think about this. If your application can scoop up a form feed, there is an angel with a flaming sword standing between you and the convenience of xsd:string. You have been warned.

Web Services: Contract? Code? Who's On First?

2008-01-19T09:33:00.000-05:00

Just about all of the writing on Web Services recommends contract-first development. Even packages that have extensive support for code-first, and weaker support for contract-first, feel compelled to salute this flag.

The advice sounds good. A strong contract avoids ambiguity and maximizes inter-operation. What could be wrong with that?

There is one tiny detail missing, however: the language in which you must specify this contract. Like 18th century English legal contracts written in 17th century French and medieval Latin, web service contracts are written in two languages that few really understand and which offer a wide variety of pitfalls to the unwary. These languages are XML Schema and WSDL.

This doesn't necessarily imply that contract-first development is bankrupt and code-first the only true way. Code-first has pitfalls all of its own. It does imply that you need to have a strong understanding of the interactions between the two models if you want to create and deploy services with a minimum of fuss and bother.

Before I dive in, a word of positioning. There are some cases in which the goal of a web service is to transfer a particular, complex, XML document from one place to another. I've never seen the point of this, myself. This posting is addressed to those writing all the rest of the web services in the world, which have the goal of moving some specified set of information from one place to another, which will only incidentally exist in XML as part of the process of moving it from here to there.

On with the show. What's the fuss with XML Schema? XML Schema is a specification language for arbitrary XML documents with arbitrarily complex content models. XML documents derive, as all of you know, from SGML, which was intended as a markup language. Thus, XML schema must cope with mixed content models.

I don't think that many people contemplating contract-first development need to be told to avoid mixed content model schemas.

XML Schema has a type model. If you are considering contract-first development, you have to understand it. I can promise you that this will not be an easy task. The XML Schema type model is fundamentally different than C++, Java, C#, Python, Perl, Common Lisp, and every other language with inheritance that I've ever encountered.

Stop and savor the irony. The idea of contract-first development is to use a specification that is agnostic as to programming language. It could be seen as an achievement, of a sort, to come up with a specification that is equally incompatible with all known programming languages.

This isn't the fault of the XML Schema designers. They didn't set out to build a data model for use behind many programming languages. They set out to build a specification language for XML documents.

Over and above the type inheritance model, XML schema includes several elements that map poorly to programming languages, such as xs:any and xs:choice.

My purpose here is not to write a jeremiad against XML schema. Rather, it is to suggest a practical approach to constructing web service contracts.

The best way to understand the implications of XML schema constructs is to convert them to code and see what they look like. In the Java universe, JAXB is the predominant mapping, and JAXB comes with xjc.

If you run some schemas through xjc and examine the results, you will begin to see some patterns. First and foremost, you'll see that the resulting code is infested with snails. That is to say, it has many, many, @ annotations.

Many of those annotations are redundant, especially with more modern toolkits such as CXF. Some of them are there so that you can edit the code fairly aggressively without changing the contract.

Next, you can try the opposite experiment. Write straightforward, simple, interfaces and bean types, and then run schemagen (for the beans) or a full java2ws tool (to include the interfaces and operations). Read the resulting XML schema.

Notice that straightforward code constructs lead to relatively simple, stereotypical, XML schema. Using those same constructs in schema leads to simple, readable, code. Let me give you an example of the opposite.

You might, some day, feel inclined to put the following in your schema:

<xs:any namespace="##any"/>

This allows any element to occur. I've seen code that generates schemas like this 'to allow for expansion.' Now, what happens if you map that to code with JAXB? What happens is this: you get an @XmlAnyElement annotation. There's just one problem: @XmlAnyElement doesn't correspond to '##any'. It corresponds to '##other'. In other words, the tools silently change your contract on you?

How does this happen? Well, I'm not a student of the history of JAXB, but my sense is that it's designers didn't want to tie themselves inextricably to XML Schema. They wanted a set of snails that could, perhaps, be mapped to some other schema specification, like RELAX-NG.

For example, consider arrays. In XML Schema, the closest thing to an array is the minOccurs and maxOccurs attributes of elements. Optional elements have 0, 1. Required elements have 1, 1. Arrays are generally 0, N, where N can be 'unbounded.' (Don't ask me what you get in JAXB if you specify, say, 12, 15.)

In JAXB, you have 'required' on an element. An element with required=true gets 1,1. An element with required=false gets 0,1. And a Java array gets 0,unbounded.

There are similar dances with the XML Schema 'form' attribute.

What's a person to do about all of this? Well, I offer a possible prescription.

Unless you are already a scholar of XML Schema, ignore the prescriptions of contract-first.

The first step is to design a code-first contract as as contract. Don't try to 'remote' whatever you have lying around by slapping a few snails on it. You might protest, 'Now I have extra classes all over the place and I have to copy all my data from my real objects to these special contract objects.' Well, my friend, you'd be in the same position if you used contract-first, only with a lot more snails and much more confusing code.

As a particular point here, consider using Document/Literal/Bare. That's right, bare. Not wrapped. One of the causes of confusion in code-first development is conflicts between the front-end (e.g. JAX-WS) and the data binding (e.g. JAXB). In a bare service, the front end is narrowly focussed on the interface, and the data us under the control of the binding. You don't have to worry about who wins a war between @XmlRootElement and @WebParam.

The second step is to review the schema. Pull the WSDL with the appropriate tool, and study it. Make sure you understand it. If it has strange quirks, adjust your code until it is clean.

The third step is to freeze the code. A contract should sit still until you make an organized, intentional, decision to evolve it. This is another justification for those 'extra' objects. They allow the contract to stay put while the code beneath it evolves.

If you adopt this discipline, you will find that it restricts you to a relatively simple set of constructs. Java Map objects are right out. Complex polymorphism will fall by the wayside.

This is the cost of interoperability. If you don't care about interoperability, then you don't need any of this. If Web Services are just an RPC mechanism, and you control both ends in real time, then you can write any code-first thing you like. Just don't come looking for too much help when the more obscure code-first constructs don't do precisely what you want.

So, let's review the good news. You don't have to become an XML Schema adept to build an web services that interoperates. You don't have to become a conchologist, either. You do have to be aware of the dual nature of what you do in code and schema.