I’m cheating already by assuming you already have JRuby installed as your default Ruby installation. No? Go get it!

Next step is to get bundler:

Now we need to make a home for our application, and prep it for Bundler:

Here I’m creating a new file in testapp called ‘Gemfile’ in your favorite editor. This is where we will sketch out our dependencies for Bundler to do all the hard work for us – here are the contents for this example:

Frankly, that’s it. We tell Bundler to look for gems in RubyGems core repo, and then we ask it to make sure we have Sinatra and Glassfish. Now we can create the program – create the file ‘hello.rb’, and use these contents:

So what’s special for JRuby? Absolutely nothing. We do have special sauce for Bundler, (by calling Bundler.setup prior to the require for ‘sinatra’) but trust me – you’ll be happy you used it. You’ll also make @wycats happy.

And – that’s it! Now, if you were to start this file the standard (well, bundler-standard) way, we’ll see this:

…and we can visit this URL: http://localhost:4567/hi. But, recall that our goal was to work with Glassfish, not WEBrick. All that has to change (and for folks who has done Glassfish/Rails before, this won’t be a surprise) is to run this startup instead

This time, we’ll visit this URL: http://localhost:3000/hi, and if all worked as desired, Sinatra will be crooning away. Boom goes the dynamite.

I just completed the upgrade to WordPress 3.0. So far, the upgrade appeared to go seamlessly. Everything is running, no plugins are complaining, yadayadayada. If you have a problem on the site please be sure to let me know.

If you are a WordPress user, and want to see what has changed since 2.9, here is a lovely screencast that covers the particulars:

In previous “distilling” articles, I discussed how methods are dispatched, and then how the scope of variables in each method and block is managed. The scope and dispatch rules are only part of the big picture, however. Ruby, as a programming language, must gather rich information about the execution of the program, and must be able to share this with the developer when errors occur. Furthermore, Ruby itself provides a number of kernel-level methods for accessing and manipulating the current invocation stack (such as Kernel.caller).

This article is all about how JRuby implements those concepts.

Overview

A frame in JRuby parlance is a representation of a method call, block call, eval, etc. kept for presentation to the developer. A backtrace is a representation of the active method stack at any point in time – in other words, it’s a stack of frames. In Java, this would typically be referred to as a ‘stack trace’ – at least, that’s the most direct counterpart.

It can be difficult when juggling a language implementation around in your head to realize that the trace we’re talking about is specific to the method calls in Ruby itself. JRuby may execute a number of “native” methods (code written in Java) that do not show up as part of this backtrace – the code that must run in between steps of the Ruby code executing is implementation-specific to JRuby; the Ruby developer shouldn’t care what internal magic JRuby had to do to get a method to invoke (nor would they know what to do with that knowledge if they did have it).

While it may not seem incredibly important initially, JRuby goes to great pains to be as compatible as possible with MRI in terms of what backtraces are generated (This ‘compatibility mode’ incurs a certain cost, and it may be preferrable to turn this off to give JRuby an opportunity to bypass this internal bookkeeping if, as a developer, you don’t need a backtrace to match MRI; but we’ll get into those experimental optimizations later). Backtrace information turns out to be quite important, as it is the first set of information a developer typically uses to trace execution issues in their own code; if it isn’t accurate (or at least traversable) it could easily make a small problem a big one.

Tracking Frames

I would like to mention at this point that it would be in your best interests to read the earlier Distilling JRuby articles, if you haven’t already. Method dispatching, scope, and the JIT compiler are all entertwined with the concept of frames, and I will be talking about these various relationships throughout this article.

You may recall that during the article regarding tracking variable scope, I mentioned that the ThreadContext is consulted on a number of occasions to find data in the variable table. At the time, I was talking about how variables are managed; but that same context class acts as the main source for tracking the frames of method invocation. We saw previously that when a Ruby method is dispatched, a variant of method named “preMethod{…}()” would be called on the ThreadContext class, and that would in turn tell the ThreadContext to create another DynamicScope object and put it on the top of the stack. It turns out this is exactly where the frame is managed as well. Here is a block of code I showed from the JRuby codebase in that previous article:

Note how this method not only creates a new scope to represent the method’s static scope, but also calls ‘pushCallFrame(…)’. This is where the new frame is created to represent the method that is being invoked. This frame is represented by a ‘org.jruby.runtime.Frame’ object, which is put on the top of the frame stack.

By most accounts, the Frame object in JRuby is a simple mutable Java bean. The class is relatively simple, and carries a few key pieces of information:

• The object that owns the code being invoked
• The name of the method (or block or eval) being invoked
• The visibility of the method
• The name of the file where the invocation of the frame occurred.
• The line number in the calling file where the invocation of the frame occurred.

… and that’s it! This is basically all that is required to produce a single line in a backtrace. The entire stack of frames then, in turn, represents the entire backtrace.

The Magic Line Number

While the program is executing, the line number is constantly changing. The frame has some idea of this line number, but only in terms of when the method was called in the enclosing code – it’s not a live representation. However, when you think about a running program, the number on the top of the trace is constantly changing – and on top of that, the frame that is the top of the trace is constantly changing as well – so who is keeping track of this magic number?

It turns out it’s the ThreadContext again, where an integer is kept to keep track of the most current line number (and actually the most current file, as well). In the basic (interpreted) mode of JRuby, the various AST nodes (control statements like if and while, blocks, methods, etc) all have their line number baked into them. When they are invoking, they will update the line number on the thread context. For example, here is the top part of the ‘interpret’ method on org.jruby.ast.IfNode:

For JIT compiler fans, note that it also manages the line number like the interpreted nodes, but as usual is a little more obscure. Two things are done for the JIT compiler: first – code is generated that will call into the ThreadContext to update the line number information like above (See ASTCompiler#compileNewLine). Additionally, however, the line numbers are also actually written into the generated Java bytecode using the standard label/line-number bytecode structures (This will provide distinct advantages in generating backtraces, as we will see later).

As the various code is invoked, this number is constantly being changed to represent the position from the original source. When a new method or block is invoked, that value is copied onto the frame and preserved. This allows the frame to keep track of when it lost control of the execution, while the thread context keeps track of the live line number.

Let’s take a look at a sample backtrace for a specific example:

So what this tells us is that in the method ‘do_something_else’ in another.rb, on line 7, we had a NoMethodError trying to call the method ‘call’ on a nil variable. Additionally, we know the three method calls it took to get to this point. Here is a diagram that shows what the frame stack looks like in the runtime at the moment this occurs (as usual, I’ve done some hand-wavy magic here to simplify a few less-important details…):

Note the line number mis-match

As you can see, the line number stored on the frame correlates to the position in the previous call where the invocation occurred. Also notice that the thread context carries the currently active file and line – but the method name is inferred from the top frame. This mixed relationship, while effective for the way that frames are recorded, can be confusing at first.

Managing Frames at Runtime

JRuby tries to avoid creating a huge volume of frame objects during execution; in general, the expectation is that a program is going to invoke a lot of methods during execution. If each method was represented by a frame, that would mean a lot of frame objects. To combat this, the frame objects are pre-allocated on the frame stack, and reused. Since programs are generally going to repeatedly traverse up and down the frame-stack, hovering around the same depth of execution, this is one place in Java code where pooling of objects probably makes good sense. Rather than JRuby creating thousands and thousands of frame objects, it will only create enough for the deepest level of execution per thread.

Internally speaking, the ThreadContext class keeps a growable Frame[], but in the process it also ensures that each slot is pre-filled with a ready-to-use frame object. If the allocation needs to grow, the array is increased by a capacity, the existing frame objects are moved to the new array, and the new empty slots are filled with additional Frame allocations.

When a Frame object is set up for use, some variant of the #updateFrame method is called, which basically captures all of the invocation information – it effectively behaves as a constructor:

Dispatching Options

In all of the past three articles, I have brushed by the CallConfiguration enumeration. This enum is a pretty significant lynch-pin in the dispatching and execution of program flow, as it decides a number of things about method and block invocation. Each method may be dispatched using a number of possible call configurations, based on the state of the running program, and the needs of code block being executed. This CallConfiguration decides not only whether a Frame object is required for the call, but also whether or not a Scope is required. This abstraction is very useful as both the interpreter, and the JIT-compiled code dispatch using this configuration strategy.

Just as certain methods may not require an explicit scope (no variables are mutated), some methods also don’t require frames. With both scope and frame, the primary reasons for skipping their use is performance. In the case of scope, the code which manages this is entirely transparent; you don’t care how your variables are managed, as long as they are managed.

However, with frames it’s not that simple. We’ve already discussed how method invocations can be compiled in to Java code in JRuby – effectively avoiding the overhead of making a series of reflection calls in favor for a generated block of Java code that properly preps the Ruby context, and invokes the method through the call site. In this process, a number of possible invocations can be generated – some that setup the frame constructs, and some that don’t. If you tell Ruby it can optimize away as much as possible through flags, it will generate these method connectors to be ultra-super-cool-fast.

Strictly speaking, to turn off frames in the compilation process, you simply need to set ‘jruby.compile.frameless’ to true – although to get the most speed, you could instead set ‘jruby.compile.fastest’ (which implies a number of other settings as well).

It should be noted that, by default, these settings are turned off, and both are marked as experimental. By leaving them disabled by default, it ensures that JRuby, out of the box, is compatible with MRI Ruby as it pertains to generated backtraces and frame manipulation, and is as stable as possible. Turning them on can easily break certain frameworks and libraries that expect the frame or backtrace to be consistent, and manageable. In many cases, however, your application won’t need that kind of control over the frame, and you may not care that the backtrace be exact.

Here are some general rules followed when dispatching to a method or block:

• A full frame will always be used if compatibility mode is enabled (jruby.compile.frameless is set to false).
• Certain system-level invocations (such as the ‘eval’ method) get a frame no matter what, as they are frame aware.
• In all other cases either a backtrace-only frame (a “lite” frame) will be used at most.
• If jruby.compile.fastest is set to true, then frames will not be used at all unless it is required by the running program to exist. This obviously has some impact on the readability of the execution.

As mentioned above, readability of backtraces is an issue with jruby.compile.fastest — less accurate information will be available. For example, the trace above that was used looks like this when run with fastest-compilation on:

Note that all frames but the top are completely unaware of the execution details – in many cases this may be sufficient information to fix a problem, but it is certainly less than what is available in compatibility mode.

The ‘lite’ backtrace-only frame I mentioned is basically a trimmed down representation that doesn’t hold on to references to the owning object. While this can reduce the usability of the frame (particularly for methods like ‘eval’ that may need to interact with the caller), it’s a significant optimization as it takes several objects off the object graph, preventing long-lived references to live objects in from the program flow (such as the object that is being invoked against). This will allow the GC to handle these objects sooner than may otherwise be possible.

Execution Flow

When an error occurs, the execution needs to stop and unwind from the exception to the first point it is properly handled (with a rescue, or all the way out of the program). This can make following the JRuby code complicated, as Java exception flow is used as the back-bone for the Ruby exception flow, and so the two intermingle and must be kept separate in your mind.

The primary class that represents an exception in Ruby is org.jruby.RubyException, which is the JRuby native implementation of the Exception class in Ruby. There are a number of subclasses that are constructed (such as ArgumentError, as we will see below) that let Ruby code handle errors in a typed way, but effectively everything extends this ‘Exception’ class. Now, while this is called ‘Exception’, it’s not actually a Java exception. It extends RubyObject (like all JRuby native peers), and is a representation of a Ruby exception for the runtime, but has no effect on Java as anything but a standard object.

However, RubyExceptions can be encountered during execution, and that should interrupt execution. Somehow this has to be handled in Java code. As an example, the ‘to_sym’ method on RubyString is implemented natively in Java, and that method, by contract, should throw an exception if the string is empty.

As it turns out, the easiest way to interrupt Java code like this is to use a Java exception. For this, JRuby uses the class ‘org.jruby.RaiseException’, which is, in fact, a real Java exception. As the name hints, it represents the execution of a ‘raise’ keyword in Ruby (which is roughly analogous to a Java throw, but is actually a method on the Thread class). RaiseException contains the RubyException representing the error in Ruby code.

When Ruby code invokes ‘raise’, this method will delegate through org.jruby.RubyKernel#raise, which for the most part will end up throwing a new RaiseException. Now, this is where it gets tricky to distinguish the two. Keep in mind that the RaiseException simply exists so JRuby can back up the Java code to find the right Ruby code to handle the error. On the other side of the equation, the code in JRuby follows a pattern roughly like this:

This is pseudo-code mixed from the AST RescueNode and EnsureNode, but it captures the idea. First, the code is run – then, if a RaiseException occurs, the exception is sent into a rescue block of code. Keep in mind that when the rescue code is run, the Ruby exception is unboxed so it’s directly accessible to that code block (as it always is in Ruby). The ensure code is actually handled by a separate AST node (since it may be included independently of rescue), but the concept is the same as seen above.

The JIT obviously changes how this code is actually invoked (via generated Java code), but the same general logic applies.

If an exception isn’t handled via a mechanism like raise, then the RaiseException itself is handled by the Java bootstrap (such as the executable). This has some special consequences when it comes to embedded code, and we’ll get into that shortly.

Additionally, there is a special version of RaiseException called NativeException (this also exists in MRI) – this is a special wrapper for exceptions that occur in Java code called from JRuby code. When this happens, the stack trace for those native parts is actually preserved in the Ruby stack up to the point the Ruby code invoked the Java code. Here is an example of a backtrace that was created by an exception occurring in some Java code:

Constructing Backtraces

Throughout this article, we’ve seen examples of backtraces that were (seemingly) generated off of the frame stack. To create a proper backtrace, the currently active frame stack must be copied and turned into a point-in-time snapshot of backtrace information. When an error occurs, the backtrace is captured with participation between the RaiseException, RubyException, and the ThreadContext.

When the RubyException is constructed, it asks the ThreadContext to create a backtrace, which then iterates over the current frame stack, creating a RubyStackTraceElement array. This array is then bound to the RubyException. Here is a sample of the loop that creates the backtrace array (I’ve trimmed some unnecessary details):

In the normal “framed backtrace” workflow, that’s all there is to it. That array can then be used to emit to the console, or whatever else needs to occur.

Interestingly, there are a number of other “super secret” ways the backtrace can be generated. As best as I can tell, these are entirely undocumented on the JRuby site – these are simply custom values for “jruby.backtrace.style” – these include:

• “raw” – This version provides a very explicit output of what happened, including all of the internal JRuby stack – very useful for JRuby development:

  1 2 3 4 5 6 7 8 9 10 11

  from java/lang/Long.java:419:in `parseLong'     from java/lang/Long.java:468:in `parseLong'     from Thread.java:1460:in `getStackTrace'     from RubyException.java:143:in `setBacktraceFrames'     from RaiseException.java:177:in `setException'     from RaiseException.java:119:in `<init>'     from RaiseException.java:101:in `createNativeRaiseException'     from JavaSupport.java:188:in `createRaiseException'     from JavaSupport.java:184:in `handleNativeException'     from JavaCallable.java:170:in `handleInvocationTargetEx' (... removed the rest for brevity ...)

• “raw_filtered” – Just like ‘raw’, but it omits any Java classes starting with ‘org.jruby’. This is handy if you have code-flows that go from Ruby -> Java -> Ruby -> Java, etc – and need to see the Java code intermixed. I’ve used this when coding in Swing and SWT where event hooks may go into Java, and back into Ruby.
• “ruby_framed” (the default) – This uses the internal Ruby frame stack to generate an MRI-friendly backtrace. “rubinus” is currently compatible with this version. Depending on the settings you have enabled, this can return different values (as described above).
• “ruby_compiled” – This uses the Java stack trace, and parses the compiled class names. When JRuby generates compiled invokers for methods, they will have mangled names that can be re-parsed (looking for sentinels like $RUBY$). Additionally, remember earlier how I said that the line numbers were actually compiled in to the Java code straight from the Ruby code? Well, that means the Java stack trace will automatically have the correct line numbers in it, so building the Ruby backtrace is truly just a matter of parsing the Java StackTraceElement[]. Because of the nature of the bytecode and the Java VM capturing this information, when running with jruby.compile.fastest set to true, this mode can actually return more accurate information than ruby_framed will. Note that if a method isn’t compiled, it will not show up in the Java stack, and as such the stack will only contain Java methods that were invoked (of which there may be none).
• “ruby_hybrid” (currently disabled) – This version is meant to be able to munge compiled and interpreted information together into a mega-stack-trace, allowing for compiled and interpreted methods to show up in the same stack trace, using the Java stack to (auspiciously) improve performance where possible — I’m assuming it’s commented out due to some flaw in the implementation.

Embedding JRuby Programs in Java

When embedding JRuby in Java programs, errors that occur can potentially leave the Ruby runtime altogether. When this happens, Java code is in total control. To make this transition as seamless as possible, JRuby performs some nifty tricks with traditional Java stack-traces.

Our old friend RaiseException actually generates the object-graph for a backtrace like above, and then creates a pseudo-Ruby stack in the Java code that lets a Java programmer see where in the Ruby code the error occurred. Here is the example from way up above as generated in Java code:

Other conversions for the backtrace (such as the fancy NativeException stuff) works naturally with this code as well, allowing for diversions in Ruby code to show up naturally in the Java stack.

Frame Peeking with Ruby Programs

I previously mentioned Kernel#caller, which is a method for peeking at the going-ons in the Ruby trace. Now that we understand the structure of the frames, it is probably pretty easy to see how they will be used. The implementation of org.jruby.RubyKernel#caller simply calls ThreadContext#createCallerBacktrace which is much like all of the other code we looked at, but it creates a RubyArray containing strings representing the state of the frames in the context at that time.

It’s probably also clear by now why optimizations like ‘jruby.compile.fastest’ can break these methods; the frames aren’t there for the ThreadContext to report against.

Conclusion

While the frame concepts in JRuby in and of themselves aren’t that complicated, you have to have a strong foundational knowledge of how Ruby works and how method dispatching in JRuby works to understand the code flows. I hope I’ve been able to condense the concepts into an easy enough walkthrough.

I’m by no means done with these JRuby articles — I took a little hiatus for work and personal reasons, but hope to have more coming out of the gates real soon. Here is a peek at some possible subjects:

• The Library Load Service
• Continuations (Kernel#callcc)
• Java Proxying and Support
• The New Kid on the Block: Duby

As usual, votes are welcome: http://www.realjenius.com/contact.

Stay Tuned!

It’s funny, because when I was a kid, I had no idea what the ides of march was; I just knew it carried with it a certain degree of playful dread, given the reaction of adults. As I’ve gotten older, I’ve learned a lot more about what the day actually is, and not what people seem to attribute to it.

If you have no idea what I’m talking about, I’m not surprised. It seems like the superstition around the date has faded in recent years. Historically speaking, the term ‘ides’ meant the middle of the month for either Martius (March), Maius (May), Quintillis (July), or October on the Roman calendar. On the Ides of March, the Romans actively celebrated for Mars, the god of war (who, by the way, is awesome).

But the reason that we don’t go around talking about the ides of October is not because the God of the Month was less impressive, but simply because of the historical significance of 3/15. Julius Caesar (a rather important Roman) was murdered by the Brutus’, Longinus, and a bunch of other political turds.

So over the years, the day has carried a negative connotation. It has Dark Ominous Tones.

Now, the truth is, I am no Julius Caesar. Additionally, I find it highly unlikely that anyone will stab me to death. However, I’ve had enough people come up and chatter at me about the ides doom and gloom that I can think of a few to poke with the pointy end of a stick, at least threateningly.

Incidentally – you’ll likely notice that Quintillis is the only month that doesn’t have any aural relationship to it’s English counterpart. Well, it turns out that one of Julius Caesar’s many faults (for which he got the stabby stabby) was reforming the national calendar. The Roman calendar became the Julian calendar as he worked to resolve the listing that occurred in the traditional model set up by Romulus, which had 304 days, and was then later refined by Numa (Numa) which had 355 days. The Julian calendar is identical to our modern Gregorian calendar in terms of length and usage of leap years, but was simply not shifted to match the same days of the year.

When Julius Caesar was murdered, they renamed Quintillis to Julius, and then bippity-boppity-bacon you hava July.

So anyway, that’s how I began to call myself the Roman god of war. Don’t make me smite you.

It’s amazing to me how much these platforms push “hello, world” examples that are simply not realistic web applications. After trying these short examples, developers turn around and start trying to implement a complete application, and this simple example balloons into a mess of code, and that’s without any real functionality yet in the application. A co-worker of mine is a fan of saying “[These frameworks] make the simple things trivial, and the hard things impossible”.

Historically, I’ve been known to say “If you are doing web-development in Java, use Wicket“; this was based on the fact that to my experience Wicket took the most advantages from the strongly-typed, and strongly IDE-supported, Java language, as opposed to trying to hide them behind anemic and broken templating languages that have horrid editors and basically trade one problem for another.

Recently, however, I spent some time doing some significant development with the Play Framework. I have to say that I think the Play Framework has eclipsed my Wicket fever. That’s not to say that I don’t still think Wicket is very powerful, but I have been particularly impressed with the feedback loop provided by Play. It has, without a doubt, the most direct code-test-cycle I have seen in any platform for Java (it approaches the instant feedback of Rails), and also has the distinct advantage of being stateless out-of-the-box (something Wicket is definitely not).

Play manages this feedback loop problem in a rather novel way – embedded in the framework is the Eclipse compiler for Java (ECJ). This means that when you’re coding for the play framework, you’re not sending it your class files, but rather your source files. This allows Play to recompile code in a running instance on the fly – I literally only restarted my application a handful of times while I was coding over the course of several days. It also integrates seamlessly with IDEs, and ships with an embedded HTTP runtime (no deployment is necessary during development).

There are a number of other benefits Play can provide by working with source files instead of class files. Much like Rails ability to add functionality to your application at runtime, Play can (and does) pre-process certain Java classes to add functionality.

I was further heartened to see that the next release of Play is meant to fully support Scala, which would allow for other modern language features to be used with this highly interactive framework.

It’s hard to describe all of the neat features Play provides in a few hundred words, but I would highly recommend you check it out – they have a 10 minute screencast they sells it better than I can. While I’m still convinced Java (as a language) will be surpassed for an overwhelming majority of the web-development as the language continues to stagnate, this is a compelling framework for the Java platform as a whole, even if Java isn’t your language of choice.

Proof is in the Benchmark

Performance is a very multi-faceted thing – there are so many measuring sticks (CPU, memory, I/O, startup time, scalability, ‘warm up’ time, etc). This makes quantifying a performance problem hard.

Furthermore, improvements for most performance problems typically involves making some kind of trade-off (unless you’re just dealing with bad code). The goal is to trade-off a largely-available resource for a sparse one (cache more in memory to save the CPU, or use non-blocking IO to use more CPU rather than waiting on the disk, etc).

JRuby always has a few open, standing performance bugs. It’s the nature of the beast that it is compared to MRI (the “reference” implementation), and anywhere it performs less favorably is going to be considered a bug (fast enough be damned). The performance measurement is up to the beholder, but CPU timings are generally the most popular.

JRUBY-2810 is an interesting case. IO line looping was proving to be slower than MRI Ruby; in some cases much slower. In this particular case, CPU was the closely-watched resource.

The first step I took to analyzing the problem was reproducing it. With Ruby this is usually pretty easy, as arbitrary scripts can just be picked up and executed, as opposed to Java, where all too often you have to build a special harness or test class just to expose the problem. Scripts are very natural for this, and in this particular case, the user had already provided one in the benchmarks folder that ships with the JRuby source.

Having run that file, I quickly saw the performance discrepancy reported in the bug. At this point in my experimenting, I was running inside an Ubuntu VM through VirtualBox on my Windows machine, so I think that level of indirection exasperated the numbers, so I checked my Macbook Pro as well. In both cases, the differences were significant: on Ubuntu, MRI Ruby was running the code in under 10 seconds, where JRuby was taking 30 seconds to a minute; the Macbook was still twice as slow in JRuby (12 seconds) as compared to MRI (6.5 seconds).

When faced with a big gap like this, I generally start by profiling. Running the entire process under analysis will generally grab some hotspots that need some tuning. I’m enamored with how low the barrier to entry on profiling has become on modern Java VMs (something that I think is actually a big selling point for JRuby as compared to other Ruby implementations; but I digress). To do my work here, I simply ran the benchmark, and popped open VisualVM. From there, I simply connected and performed CPU profiling (which automagically connects and injects profiling code into the running system).

In this particular case, the first problem was quickly uncovered:

Great Odin's Raven!

Clearly, a very large amount of time is being spent in ByteList.grow. I felt fairly fortunate at this point, as rarely is it this straightforward; having a performance problem reported with this singular of a hot-spot. When nearly 80% of the processing time is spent in a single method, it brings up several questions: What is ByteList? Why does IO.foreach use it? Why must it keeping ‘growing’? Did I leave the iron on? To answer these questions (most of them, anyway) you simply have to get your feet wet in the code.

Coding for Crackers

At its heart, IO.foreach (and the close counterpart, each/each_line) is simply a line iterator that hands off each line of text off to a receiving block – there are a number of caveats and subtleties built into that idea, but at its core, it allows developers to write code like this:

Deceptively, simple – isn’t it? It turns out that a lot of wrangling has to occur to make this so simple – much of it having to do with how files are encoded, and the variety of line separators that may exist. Thankfully, the good folks at JRuby have cemented this in the code fairly decently – for my part, I mostly had to draw boxes around the complex encoding and line termination algorithms, and focus on the loop and data-reading itself. Most of this was occurring in a single method (for the code-eager, this was in RubyIO#getline and its derivatives). This method is used in a number of scenarios: the traditional looping algorithms, the 1.9 inverted enumerator stuff (handing the ownership of “next” off to the caller) as well as basic calls to ‘IO.gets’. Internally, each call to getline allocates a new ByteList and copies data from the input stream into it.

This is where the high-CPU numbers started. ByteList is simply an easier-to-use wrapper around a byte[]. It backs several JRuby data structures – the most notable probably being RubyString (the Java peer for String objects in JRuby). In fact, the ByteList allocated in this routine is eventually given to a String object, and returned at the end of the call. The ‘grow’ method on ByteList (the offending code-point) is the automatic capacity increase mechanism, and does this via an an array-allocation and copy (much like ArrayList); this method uses a fairly standard 1.5x grow factor.

It’s easy to see how ByteList would be central to the benchmark since it represents the primary data structure holding the bytes from the input source, but it seemed suspicious that ‘grow’ was the offending hotspot. I would expect it to be one of the copy methods, like ‘append’, which is really where the algorithm should be spending its time (that, and ‘read’ from the input source). To understand why ‘grow’ was so cranky, I had to look more closely at the code I was invoking: the benchmark.

Understanding the Benchmark

The original benchmark used to test the ‘foreach’ performance in JRuby when 2810 was first opened performed something like 10,000 line iterations on a file with relatively short lines. Halfway through the life of this bug, those values were adjusted in this original benchmark in a way that exposed a short-coming in the JRuby line-read routine – by generating only 10 lines that were very, very long instead.

For any Ruby implementation, reading a file with particularly long lines using foreach is prohibitively expensive, as the entire line has to be read into memory as a single string object that is then shared with the code block. Normally, you wouldn’t want to read data this way if you knew that the file was structured so wide, and should probably consider a streamed-read instead. That being said, MRI Ruby performed much more admirably in this scenario, so it was something to be analyzed.

The root of the problem was this: JRuby was starting with an empty ByteList, and was then creating subsequently larger byte[]s indirectly (via ByteList.grow) – the 1.5x factor wasn’t enough, as the chunks were being read 4k at a time, and these files were significantly wider than 4k. For that reason alone, the ByteList was having to grow a number of times for each line, and when we’re talking about a byte[] several kilobytes in size, array copies are simply going to be expensive – all those together combine to make this an unfriendly performance proposition.

As I mentioned previously, the benchmark used to be a very different performance model. I decided at this point it was good to split the benchmark so that both could be run side by side, and I could see both the ‘wide’ scenario and the ‘tall’ scenario at the same time. It turned out via profiling that the tall file was experiencing pains from ‘grow’, but not nearly so badly. Even at 10,000 lines the amount of adverse memory allocation and churn was much smaller, as a single 4k allocation on each line was more than sufficient.

For reference, here is what the ‘tall’ benchmark looks like:

The only difference in the wide benchmark is the tuning parameters:

So ‘tall’ can be read as ’10000 lines, 10 sentences long’, and ‘wide’ can be read as ’10 lines, 10000 sentences long’.

Also for reference, here is what it looks like to run a benchmark using this framework – 5 iterations are run (as defined in the file), and the various aspects of CPU usage are measured. Generally, the most important number is the ‘real’ column when measuring performance between Ruby and JRuby, as the two report user/system CPU usage very differently.

After splitting the benchmarks, here is a breakdown of my two configurations:

Keep in mind I’m just rounding here; not really trying to be exact for this blog post. Check the bugs for more exact numbers.

A Solution Lurks

So, we have performance problems on tall files, and a whole lot more performance problems on wide files, particularly depending on the environment. Because of the environmental discrepencies, I spent some more time comparing the two test environments. It turned out that the Macbook Pro was simply working with a more resource-rich environment, and as such wasn’t hitting the wall as badly when allocating the new immense byte[]s. The implementation in JRuby was not degrading as well on older (or more restricted) hardware as MRI.

(It’s probably good to note here the value of testing in multiple environments, and from multiple angles)

My first pass at a solution to this problem was to consider a byte[] loan algorithm. Basically, at the start of foreach, I effectively allocated a single ByteList (byte[] container), and for each iteration of the loop, I just reused the same ByteList — eventually the byte[] being used internally would be sufficient to contain the data for each line, and would not need to grow any more (yay recycling!).

I encapsulated most of this ‘unsafe’ byte[] wrangling and copying into a small inner class called ByteListCache – at the start of the loop, the ByteListCache is created, and then it is shared for each iteration, being passed down into ‘getline’ as an optional parameter, the side effect being that the first call to ‘getline’ manually allocates a large byte[] (just like it did pre-patch), and each subsequent call can simply reuse the previously allocated byte[] that is already quite large. If the need arises to grow it more, it can, but it becomes increasingly less likely with each line.

Once the iteration is completed, the ByteListCache is dropped out of scope, ready for garbage collection. The number of calls to ‘grow’ drops dramatically with this implementation, and so did the impacts to the performance:

Unfortunately, they were only this fast because the implementation was now thoroughly broken.

Stop Breaking Crap

Okay, so I had amazing performance numbers. Except. Now over 50 ruby spec tests were failing. Oh yeah, that might be a problem. Needless to say the problem was obvious the minute I realized what I had done (I actually woke up at 6:00am realizing this, which if you know me, is a bad sign). Remember how earlier I said that the ByteList was used as a backing store for the String? Well, at the time I implemented this, that point had eluded me. I was (accidentally) creating strings with my shared bytelist, so you can probably see where that would end up creating some significant issues with data integrity.

To fix this, the solution was simple – create a perfectly-sized ByteList at the end of the line-read the exact size necessary for the String, copying into it from the shared bytelist, and then passing it in to the String constructor. Obviously this cut into my performance numbers by a percentage on each, but it also fixed the data corruption, which is nice.

The lesson learned here, obviously, is that you need to run a variety of tests (a full suite of specs if you have them) when considering bug fixes. For JRuby, that means (at a minimum) running the specs, which is easy with the Ant script:

A Word on Limited Application

Note that I isolated the use of this construct to the foreach and each_line algorithms, as these had deterministic, single-threaded behavior, and would benefit from the overhead of dealing with this additional object. The new Ruby 1.9 enumerator stuff does not use it, as there is no guarantee of single-threaded usage of the enumerator, so we can’t reuse a single byte list. Similarly, individual calls to ‘gets’ do not currently use it, for the same general reason.

Changes could be made to make this byte[] re-use more long-lasting/global – but the changes required felt a little too overwhelming for a first pass, even if they did offer potentially larger benefits.

Rinse and Repeat

Now that I had two tests, and I had seen some improvements (but not quite in the range of MRI), it was time to revisit. Re-running the benchmarks, it was fascinating to see a new prime offender – incrementlineno. It turns out that a global variable was having to be updated through a very indirect routine that contains a fixnum representing the line number in the file, and all of this heavy-weight variable updating (going through call-sites and arg file lookups) was very expensive in comparison to the rest of the iteration.

At this point, I’d spend a lot of time explaining how I improved the performance of this little gem, however the truth be told once I hit this, I simply had to inform the powers-that-be, and back up. You see, I couldn’t figure out (for the life of me) why this method was doing what it was doing; why it was so important for this line number to be set. This is one of the perils that I have verbally discussed with folks about consuming foreign code-bases. You can’t assume secret sauce is a bad thing – I had to assume it is there for a reason, even if I don’t know what it is.

It turns out, the JRuby folks didn’t know the reason either. Well, that’s not exactly true; it didn’t take long for Charles Nutter to figure out why it was there, but it was clear it was only for rare file execution scenarios, and not appropriate for the more general looping scenarios I was debugging. To follow his efforts on how he optimized that code path, you can reference his commit here: JRUBY-4117.

After his optimizations, the numbers boosted again:

Summary

I think it’s fascinating how varied the numbers are depending on the platform. This is a complicated benchmark, and as Charles Nutter mentioned to me, one problem we’ll continue to face is that we have no control element in this benchmark. You can get consistency through repetition, but there are simply too many variables to predict exactly what the outcome will be on any given platform. I find it interesting how well the Macbook handles the wide files compared to the Ubuntu VM, which just dies a slow death in comparison – this has to be a side-effect of resource starvation in the VM; but whatever the case, it’s an interesting dichotomy.

On average, the new numbers are much more competitive with MRI, even if they don’t beat it in most cases. As I learned from working with others on this bug, your mileage may vary significantly, but it’s clear from the implementation that we’re causing a lot less resource churn for very little cost (the trade off here is retained memory), and that’s generally a good sign things are going in the right direction. Certainly, profiling has shown that the effort is much more focused on reading from the input channel.

That being said, I’m sure there is more performance to be found – MRI is just a hop-skip-and-jump away!

Starting with the upcoming version 9.0, IntelliJ IDEA will be offered in two editions: Community Edition and Ultimate Edition. The Community Edition focuses on Java SE technologies, Groovy and Scala development. It’s free of charge and open-sourced under the Apache 2.0 license. The Ultimate edition with full Java EE technology stack remains our standard commercial offering. See the feature comparison matrix for the differences.

Very cool news! The impact in competition for other IDEs (namely Eclipse and NetBeans) remains to be seen, but this certainly brings another aggressive (and already well-liked competitor) to a broader market.

“For a limited time only – you can beta-test WordPress 2.9 from the comfort of your own installation!”

In the “another reason I like WordPress better for blogging” category, the folks over at WordPress.com have published a detailed article on how WordPress users can get involved in the beta-testing of WordPress 2.9 – the next major release. Included in the options for aspiring beta-testers is a plug-in that will automatically upgrade your site to the new version, and will keep you on the up-and-up.

To make is as easy as possible for you to get a beta testing install up and running we have put together a small WordPress plugin which makes it really easy to convert a test install of the latest release version of WordPress into a beta test install of the next up and coming release.  The plugin is called WordPress Beta Tester and is available to download from WordPress Extend or can be installed using the built-in plugin installer.  Please make sure you to only install this plugin on a test site, as we don’t recommend running beta versions on your normal live sites in case anything goes wrong.  You can read more about the plugin in “Making it easy to be a WordPress Tester”

I don’t want to suggest that Drupal doesn’t work for transparency in their development, or that they don’t encourage community participation; but certainly, they don’t package it up as nicely as the WordPress folks have right here. While I won’t be upgrading RealJenius.com until final drops (I did consider pulling a Picard maneuver and upgrading now), it’s nice to know that the option is there. The release schedule for 2.9 has been revisited in this entry as well.

We are aiming to release the first beta version of 2.9 around the end of October, once we have put the finishing touches on the new features, and then we switch to full on beta testing mode and your help and feedback will be very much appreciated.  During the beta program will push out new builds for automated upgrades regularly and once we feel that a suitable level of stability has been achieves we will release a release candidate, and we hope to be able to make the final release 2.9 build available in either late November or early December.

However, I think Eclipse, as a product, still has a way to go.There are a number of products I use everyday that frequently require or recommend updates and upgrades. Some good examples include:

• Mozilla Firefox
• Android and Android Applications
• WordPress
• Windows
• Mac OS X
• Songbird
• Eclipse

…and, of course, the list goes on. All of these apps have different ways to handle notifying and installing these upgrades.

Of course, both Windows and Mac OS X have built in update mechanisms, the former being the ever-controversial “Windows Update” tool, and the latter being Mac’s Apple Software Update (which has also recently been pissing people off). Firefox and Songbird use their own update mechanism that’s part of the Gecko platform. Most Mac users are also probably aware of Sparkle (or at least have seen it in action), as many Mac software distributions (like TextWrangler) use it as their update mechanism.

Your mileage may vary with the different implementations; some are more informative than others, and some are more reliable than others. But one thing can generally be said: each of these mechanisms tie in with documentation explaining what was updated, corrected, or otherwise tweaked. Some tools, like Sparkle, embed the update information in the notification dialog. Other tools, like the Firefox notifier, provide a link to the content that takes you to a rich and user-friendly webpage.

Firefox has thorough documentation online

Today when I came in to the office I decided to perform an update check in Eclipse, and was presented with a number of plug-ins that had newer replacements. At the time, I didn’t grab a screenshot, but here is an example from a brand-new download of 3.5, which has been since supplanted by 3.5.1:

So, is this a good thing?

Unfortunately, this doesn’t show off the obscure list of plug-ins my installation did earlier today, but there is still plenty I can pick on. So, what’s wrong with this picture?

• I have no information on what is being changed or improved in this update.
• The ‘Details’ section is suspiciously empty.
• The version number is unnecessarily cryptic for end users.
• The ‘More’ button simply gives me the description, copyright, and license for the plugin/feature being shown, which in this case is basically empty boxes.
• The version number is cryptic for an end user.

Now, as an Eclipse-enthusiast, I have no problem deciphering what is going on here (particularly not since I had to contrive this picture for the blog entry), however as an end-user product, this could use some work. More information could be provided, condensing what was actually improved or fixed in this release (any information at all would be nice).

I understand that Eclipse is a platform first, and a product second, however I think this is one area that could use improvement.

JIT compilers are a novel idea: take some code in an “intermediate form”, and, given some heuristics, compile it into a “more native” representation, with the expectation that the more native version will perform faster, and allow for more optimizations by the underlying platform. If some optimizations can be thrown into the more native form in the process, all the better.

Java, as an example, has had a JIT compiler for several years now. In fact, Java was, for many developers, the first time they heard the term JIT; so much so that many developers I know think the “J” in JIT stands for “Java”. In fact, JIT stands for Just-In-Time. Smalltalker’s may recognize the term “Dynamic Translation” instead.

Anyway, when the Java runtime determines code is eligible for native compilation (frequency of execution is one of the primary parameters), it diverts the execution of the code, so it can perform some fancy hoop-jumping to turn the Java bytecode into platform-specific native instructions, thereby removing any cost of bytecode interpretation, and also throwing some nifty optimizations into the compiled code. From that point forward, the native code will be used for execution, unless and until it has been invalidated by changing assumptions.

JRuby’s JIT compiler is similar in nature, the primary difference being the source and destination formats. In Java the source format is Java bytecode, and the destination format is native machine instructions. Conversely, in JRuby, the source format is the JRuby AST, and the destination format is Java bytecode. Perhaps the most fascinating aspect of the JRuby JIT compiler is that it benefits from both the initial compilation into Java bytecode, and then later, when Java may attempt to translate the JRuby-generated bytecode into native machine instructions. So effectively, it is possible to get a double-JIT on your executing code.

Generating Java bytecode from interpreted Ruby code is no small feat, however; so, without further ado, let’s start the tour of JRuby’s JIT!

There’s Compilers, and then there’s Compilers

Before we go to deep, we should do a quick overview of what is really meant by “compiling” in JRuby. Some of this discussion isn’t specific to the actual concept of “JIT” compilation; there are a few different code-paths to getting to compiled code in JRuby. One of those is definitely Just-In-Time method compilation, however JRuby can compile entire scripts as well (with Modules and Classes, and other fancy-shmancy stuff). So as this article proceeds, I will try to make a distinction on this.

When I refer to the JIT Compiler, I’m really referring to the component that is responsible for tracking method invocations and compiling them when appropriate. On the flip side, the term “Compiler” by itself belongs to the component in JRuby that can compile any Ruby script into an executable chunk of Java.

There are several tuning parameters in JRuby for the compiler as well; setting the maximum number of methods to compile, how many method calls before the compiler should be invoked, the maximum lines to compile into a single string of bytecode, etc. These settings affect different parts of the infrastructure, but for the most part, you shouldn’t have to care whether it is JIT-specific or not; in most cases it doesn’t matter.

Stay Classy, Java

So, JRuby is compiling Ruby into executable chunks of Java – that much we know; but what are they, actually? Java, as you may or may not be aware, prevents the modification of any loaded classes in the VM (security or some such nonsense; dang buzzkills). This rule prohibits JRuby from using the same class file to represent a live class in Ruby. Intuitively, it might seem logical for JRuby to group all of the methods for a single Ruby class into a single Java class; however, because of the aforementioned restriction in Java, and the facts that a.) Individual method are only compiled by the JIT at thresholds (meaning some of a class may be compile-eligible, while other parts aren’t), and b.) Ruby allows for runtime method manipulation, this single-class paradigm isn’t possible. So instead, the JRuby-to-Java compiler infrastructure is designed to turn any given JRuby AST into a generated Java class. In other words, any time a hierarchy of AST nodes that represents a Ruby script is compiled in JRuby, a new Java class is derived to match, built, loaded, and otherwise made awesome. Unlike the actual meta-class in Ruby, the AST is as static as the block of code from which it was built. If another script comes along with more AST that modifies the same class, Ruby doesn’t care – it will simply have compiled method hooks pointing to two entirely different Java classes.

The classes generated by the JRuby compiler (note I didn’t mention the word JIT here), implement the JRuby interface ‘org.jruby.ast.executable.Script’. We’ll see later how this is actually used.

That’s Some Crazy JIT!

In the first Distilling-JRuby article I detailed the class ‘DynamicMethod’. This class represents the “implementation” of a method in JRuby. As I discussed in that article, there are a ton of implementations; Figure 1 is the class-hierarchy again for those of you who’d rather not dig back to the old article. One of the arrows I have drawn on the diagram is pointing to an innocuous little class called ‘JittedMethod’. What this really represents is the cusp of the rabbit hole we are about to go down. Before we go too far down that hole, however, let’s do a quick recap.

Hello, Again Old Friend

Method invocation is the point at which code is JIT’ed in JRuby (but not the only way it gets compiled). DefaultMethod keeps an internal counter representing the call-count for that particular method. This number will be used by JRuby to determine whether or not the method is “JIT eligible”. There are somewhere between ten and four million overloaded variants (I got tired of counting) of the ‘call’ method on DefaultMethod, but needless to say they all do something like this:

(In this case, “box” is just a container for the delegate method and the call count integer pair, and the [...] denotes where the various “Arity”-supporting implementations diverge.)

As long as call count is a positive integer, the “tryJitReturnMethod” call is made, which will check in with the JIT subsystem, and will attempt to build a new method implementation. When that call completes, it returns a method implementation which this method can use, and that implementation is then invoked. If call count is negative, however, the method cached on this receiver is simply called.

The call-count integer serves multiple purposes. By simply being set negative, it effectively turns off the attempts to JIT-compile this particular method, but it also represents a counter for the number of invocations this method receives prior to JIT compilation actually being attempted. Call count is the primary metric JRuby uses to throttle JIT compilation.

The “tryJitReturnMethod” simply looks up the JITCompiler implementation using a call to Ruby#getJITCompiler(). From here, we are entering the JIT universe.

“You are in a maze of twisty passages, all alike.”

The actual org.jruby.compiler.JITCompiler class represents the heart of the entire JIT process. This class is the manager for the primary JIT compilation efforts (It is also one of JRuby’s registered MBeans – a topic I plan to discuss soon). The primary invocation point is a method called “tryJIT”, which takes the DefaultMethod instance we were just working with, as well as the method name and the current ThreadContext (there’s that context showing up again). From here, a number of contextual checks are made to see if this particular invocation is eligible for JIT:

• Is the JIT compiler enabled?
• Is the method call count above the threshold for the JIT?
• Is the JIT method-cache full?
• Has the method been excluded from JIT’ing?

After all of this, if the method still checks out, it’s time to begin the real fun of compiling (or at least, trying to). At this point I’m going to (temporarily) do some hand-waving, and let us all pretend that we’ve passed a bunch of JRuby AST to our magic fire-breathing compiler monster, and out the back-side we were given a bunch of Java bytecode that acheives the same result as the AST. To help with the transition, I have made a high-level diagram for this portion of the code.

Assuming you are still on board, remember that the goal at this point is to have a Ruby method be converted into a loadable chunk of Java code, so the bytecode we were given (which represents a Java class) needs to be loaded into a Java class at runtime. JRuby needs an efficient and ‘appropriate’ way to load the Script sub-classes into Java, and keep a handle on them so we can create new instances of our ‘method’-powering class.

The component ‘org.jruby.ClassCache’ performs both of these tasks, and is responsible for respecting the cache configuration parameters specified at startup. The method ‘cacheClassByKey(…)’ is called, and is given the bytecode from our fire-breathing monster. To load the class into the runtime, a small class called OneShotClassLoader is used, which as the name implies, is only used once. The classloader is a child of the primary JRuby class-loader (in the normal case, anyway), which means that the code generated in the script has visibility to all of the JRuby classes and dependencies (this will be important later). At the same time, it means that the class is isolated from the other scripts in its own classloading domain.

Interestingly, this class-cache does not actually hand out references to the class. It returns the class from the initial cache call, but then simply *retains* references to the classes in a weak-reference form, so that if a method goes out of scope (like if a call to ‘remove_method’ was made on a class), the reference will be dropped, and the cache will shrink by one. In other words, the primary goal of the cache is to act as a throttle, as well as an endpoint for JMX monitoring.

To create a key to represent the class in the cache distinctly, S-expressions are used. A class called SexpMaker is given the various AST elements representing the method, and it in turn generates an S-expression that represents the method. If you ever want to get a good feel for the methods stored in the ClassCache during a Ruby program execution, putting a break-point and looking at the S-expressions in the cache can be enlightening. As an example, I created a very simple Ruby class that looked like this, and made a call to it in my script elsewhere:

I then set up the JIT compiler to run in such a way that this method would be JIT’ed. Here is the S-expression generated for that method (formatting mine):

As you can see, it’s a short-hand representation of the actual AST, and it is unique for that AST structure.

Method Swapping

So we’ve made it through the journey of the compiler, and our JIT’ed code is now contained in a nice handy-dandy Script class. The JIT compiler class proceeds to create a new instance of our Script, and along with some various bookkeeping, calls DefaultMethod.swithToJitted(Script, CallConfiguration), passing in the Script object that represents the logic for this method.

This method assigns a new JittedMethod object to the default method container, and sets the call-count to a negative value, disabling subsequent JIT attempts.

Assuming the compilation has worked correctly, the actual invocation of the script is fairly straightforward. There are a variety of ‘Arity’ implementations on the Script API to line up with the DynamicMethod call methods, but for the most part they all do the same thing:

Effectively, the Script API is analogous to a general ‘Runnable’ representing a block of code; it has been specialized to handle a number of call configurations, but for the most part, it simply is a logic containing object.

From this point forward, that method is now JIT’ed unless/until it is removed from scope.

I’d Like a Side of Bytecode With My Bytecode

Okay, time for the good stuff. We’ve done enough hand-waving, it’s time to explore the compiler. Before I go any further, I should mention the –bytecode switch. If you ever want to see the bytecode JRuby generates for a chunk of Ruby code, you can simply by invoking JRuby:

It was immeasurably helpful to me in writing this article.

Compiler Components

There are several pieces and parts that all participate in the compilation process (and incidentally, many have the word ‘compiler’ in the name). That makes it a fairly complex creature to understand. If the compiler is a big furry fire-breathing monster, we’re about to dissect it and poke at the internal organs. So, let’s start with a quick break-down:

• org.jruby.compiler.ASTInspector – The ASTInspector class is effectively a statistics-gathering tool for the AST; a detective looking for certain conditions. The AST is given to this class at the start of compilation, and it looks for certain conditions in the AST that influence the overall behavior of the resulting code. One of those conditions that is scanned for is the concept of scope (which we talked about last time). Scope becomes very important, because if the code in question doesn’t need an explicit scope, the compiled code can be made much simpler; likewise if it does have some intense scoping, the compiled code has to make sure it respects that so variables aren’t leaking all over the place.
• org.jruby.internal.runtime.methods.CallConfiguration – This is an enumeration representing the type of method invocation involved at a certain level, and is calculated and returned by the ASTInspector, depending on the structures it finds. The call configuration isn’t unique to the compiler process; in fact it really is part of the scope management; but was a bit too detailed for the previous discussion. This enumeration is the actual object that performs the ‘pre/post’ invocations on the ThreadContext to setup any scope that is necessary; different work is done depending on the requirements of method being invoked. Some example call configurations are FrameFullScopeFull (meaning it needs a frame and a scope) and FrameNoneScopeNone (meaning it needs neither). We haven’t discussed the concept of ‘frame’, however it basically represents the invocation of a method in a certain context: the call frame. It keeps track of information that allows Ruby to manage the call stack beyond the scope, which we previously discussed.
• org.jruby.compiler.ASTCompiler – The ASTCompiler knows specifically how to traverse the AST, and how to then consult with other objects to translate it into an alternate representation. To handle the actual bytecode generation, the ASTCompiler hands responsibility off to the bytecode generating parts. The ASMCompiler handles the busy work of setting up the compiler hierarchy when traversing method entry/exit, closures, etc.
• org.jruby.compiler.ScriptCompiler – The ScriptCompiler interface defines the high-level hooks into the underlying bytecode generation process used by the ASTCompiler. The sole implementation of this API currently is StandardASMCompiler, which as you could guess is backed by the ASM bytecode library This class will create “sub-compilers” that know how to deal with the recursive nature of the compilation process.
• org.jruby.compiler.impl.BodyCompiler – BodyCompilers are the ‘sub-compilers’ I just mentioned. Specifically, each BodyCompiler deals with blocks of code that may carry their own scope/set-of-variables (good thing we already discussed scope!). Here is the current class-hierarchy of body compilers:
  
  The two primary categories are “root-scoped” and “child-scoped”. In our scope discussion, we called these two scenarios “local” and “block” respectively. Here is the root-scope javadoc:

  1 2 3 4 5

  /**  * Behaviors common to all "root-scoped" bodies are encapsulated in this class.  * "Root-scoped" refers to any method body which does not inherit a containing  * variable scope. This includes method bodies and class bodies.  */

• org.jruby.compiler.VariableCompiler – The variable compiler API knows how to generate bytecode for loading and storing variable values from the Ruby runtime. Body compilers create variable compilers appropriate for their scope. Certain scope rules allow for certain optimizations which will be described below.
• org.jruby.compiler.InvocationCompiler – The invocation compiler is a component that knows how to generate bytecode for invoking methods on Ruby objects in the Java memory space. All body compilers have these. This will be described in more detail below.
• org.jruby.compiler.CacheCompiler – Compiler that can translate certain references in the Java bytecode into quick lookups on the AbstractScript.RuntimeCache object, rather than having to do full lookups into the Ruby runtime (for methods, fixnums, etc). This allows the JVM more opportunities for optimization via inlining and other good stuff.
• org.jruby.compiler.impl.SkinnyMethodAdapter – Delegating implementation of the ASM MethodVisitor interface that has several convenience methods for making typed bytecode calls; this makes the code that is generating bytecode much easier to trace through than it otherwise would be (not as easy as Bitescript, but we can’t always be idealists).

Here is another view of these elements, roughly categorized by responsibility, showing some interactions:

(This is only one of many ways to show the interactions at this level, however it should give you some idea of the segregation of these components by responsibility)

The AST analysis components work with the bytecode generation libraries to recursively build up a representation of the algorithm. In the process, the bytecode generation libraries are going to “wire in” to the bytecode calls into several JRuby runtime libraries, including many we’ve already seen, such as the ThreadContext, the scope objects, and a bunch of other helper libraries that are implemented as statics everywhere.

The Nature of the Generated Code

As with everything in JRuby, there are all kinds of special fast-path hooks built into place that complicate the code for quick scanning, however we can talk about the “general” case first, and work our way into the special cases. With our previous code walkthroughs, we saw that the interpreted AST code performed the invocation using a combination of recursive node traversal and invocations of calls into the ThreadContext to manage and lookup scoped variables (creating scopes, storing values, etc). The generated code often stores and retrieves variable values the same way as we have already discussed – via collaborative lookups on the current dynamic scope. In other words, let’s say we have this method:

At this point, it’s going to be easier for me to write pseudo-Java-code to express what the generated code could do to solve certain problems. Remember that JRuby is generating Java bytecode to effectively do the same thing I’m doing here.

The pseudo-Java for this particular example could look something like this:

Now, I took several liberties with this code to make it easier to read (this isn’t valid code), but it should look a lot like the API we discussed previously in the scope article. In fact, this is the general idea of the code that is generated in many cases by JRuby. It gets more complicated than this, obviously, but this shows the most naive implementation of the JIT compiler – basically: turn the steps taken at runtime by Ruby into hard-wired Java.

If we start poking through the various compilers, we’ll see that they often make extensive use of the Ruby libraries that the AST uses (as we would expect). Here is an example from the HeapBasedVariableCompiler.assignLocalVariable(int index) method (we’ll analyze the variable compilers more deeply below):

In this case, “method” is a SkinnyMethodAdapter object as described above. This method is telling the current enclosing Java method to load the DynamicScope object from the local variable table, swap the “value to set” back onto the top of the stack, and then invoke the “setValueZeroDepthZero” method on it (which is a hardwired method for setting the variable at position 0, depth 0 with a value). That means that the DynamicScope variable has to be somewhere on the local variable stack (we’ll see how below), but beyond that it’s fairly straightforward (as these things go, anyway).

Stack vs. Heap

All of this talk about DynamicScope and looking up variables is a bit over-simplified. The JRuby JIT compiler has support hard-wired into it for dealing with variables exclusively on the local stack. This is provided via the StackBasedVariableCompiler (you can see the variable compiler hierarchy here). The various body compilers (which, if you recall, represent methods, classes, blocks, and so forth) checks with the associated ASTInspector. If the inspector says that there is no special use of closures or scope-aware method calls, the scope will then create a special stack-based variable compiler.

This stack-based variable management is a pretty significant optimization. Java is much more efficient when dealing with stack-based variables as opposed to heap references, stack memory access is bound to be faster. In interpreted JRuby execution mode there is no such thing as a stack-based Ruby variable; all of the variable tracking is synthesized by the JRuby runtime via use of the Java heap and the scope objects we previously discussed.

Here is what the local-variable assignment looks like for the stack-based variable compiler:

We’ve dropped the DynamicScope lookup, not to mention the invokeVirtual on dynamic scope. In this particular scenario, we have entirely bypassed the JRuby libraries altogether, and are simply using native Java stack to track the Ruby object/value instead. In this particular scenario, JRuby has managed to convert an operation into the direct Java counterpart, with no compromises. That is, of course, lightning fast variable assignment (much faster than several hops through methods in the DynamicScope, and doing in-heap array lookups, etc).

One of the things that the VariableCompilers are responsible for is setting up variables at the start of a method or closure. There are a number of methods on the VariableCompiler API that are consulted at the introduction of these top-level elements – here are three of those elements:

The HeapBasedVariableCompiler is going to get all of the lookup objects it needs (most notably, the DynamicScope) and stuff them into the method as local variables. More concretely, at the top of the heap-generated methods in JRuby, the bytecode generated by ‘beginMethod’ looks something like this:

(I feel the need to re-stress that this Java code is my extrapolation of the compiled bytecode – JRuby does not currently actually generate Java source, and instead has the “luxury” of dealing with Java bytecode variable indexes and the like. If you were to decompile the bytecode, it might look a little something like this, however.)

There are, of course, other things that the heap-based variable compiler may choose to do at this point to optimize itself, but those specifics aren’t particularly relevant to the idea. If you go back to our “a=5; b=a” example, you’ll see that the pseudo-Java can then simply reference ‘scope’, as opposed to calling ctx.getCurrentScope() every time.

On the other side of the fence, the StackBasedVariableCompiler (which knows it can deal with variables entirely on the Java stack), loops over all of the variables and declares them at the top of the method, assigning them to JRuby ‘nil’. In this example, if the method is going to deal with three variables (we’ll call them ‘a’, ‘b’, and ‘c’), then the stack-based compiler would start the method like this:

This allows the subsequent code to just deal with variable declaration implicitly, just like Ruby does – since the variables are already on the Java stack, they can simply be blindly assigned a value by the rest of the native bytecode – so going back to our ‘a=5;b=a’ example, it is not unreasonable to consider JRuby generating another set of lines that looks like this:

Not too shabby! In reality, JRuby can optimize even further than this (like dealing with fixnums smarter than that), but that’s beyond this discussion.

Invocation Compiler

The InvocationCompiler is responsible for create bytecode that can invoke another method. In the current JDK world, that means loading the appropriate variables on the stack to then invoke the method “IRubyObject.callMethod(…)” on the “receiver” object. In other words, if the Ruby code was:

The pseudo-Java code we want to generate would look something like this:

Of course, this is a major simplification of the actual call-site management provided by the invocation compiler, and all of this actually implemented in scary bytecode, but you get the idea. From here, the Ruby runtime takes over the method invocation, which goes through the DynamicMethod dispatching we’ve previously discussed. In that process, it may compile more code and create more class-files, but they are effectively disconnected from each other via the call-site indirection through the Ruby runtime method dispatching.

In the “amazing future” we have the concept of “invokedynamic”, which is a Java 7 bytecode instruction that allows for invocations to dynamically-generated methods in the Java space, allowing a language like Ruby to hook into the JVM’s method dispatching, in turn allowing method dispatching in the JIT-compiled code to be fully optimized. Charlie Nutter has described this far better than I can, and his article ties in nicely with the method dispatching I previously discussed.

How JIT Participates

The JITCompiler class effectively has to ‘jump in the middle’ of this compilation routine – taking a single Ruby method and translating it into an entire JIT-compiled class. The class JITClassGenerator (an inner-class to JITCompiler) is responsible for handling this special scenario.

Remember, we already have all of these concepts built for compiling arbitrary scripts – the ASTCompiler and ScriptCompiler are fully capable of handling the introduction of a method. Therefore, most of the work actually happens via the ScriptCompiler. Where-as the general compilation will make calls to “compileRoot” or “startRoot” – the JIT process makes calls to “startFileMethod”.

The primary distinction between the two invocations is described in the ScriptCompiler Javadocs:

Basically, startFileMethod performs a special pre-configuration on the generated class to make is self-sufficient for this single method, and entirely invokable through the ‘Script.run’ API. The JITClassGenerator is going to exercise the ASTInspector on the top-level of the method (just like the ASTCompiler would when it hits a method), and is going to then pass all of that information into the ScriptCompiler for construction.

From that point forward, the method is digested as it normally would be by the recursive inspection/compilation process.

Compiler Cookbook

At this point in this article, the general concepts have been thoroughly hashed out, but we’re simply missing some examples of what Java is actually built when certain scenarios are hit. So let’s walk through some. I’m going to use the same scheme from above, describing at a high-level the code that JRuby generates, and then possibly using pseudo-Java examples.

General AST Class
We’ve talked about it quite a bit already, but to cement things – if you have a ‘test.rb’ script and you compile it, you’re going to get back a class representing that script. The most natural Java code representation looks something like this:

There is, of course, other stuff that goes on. But that’s the general ‘framework’ in which the rest of the generated code works.

Creating a Method
Generally speaking, Ruby methods are translated into Java methods. However, where-as the Ruby methods are typically associated with some parent class, in Java they are simply on the Script-class for which they are compiled. When a method is hit, Ruby combines all of the stuff we previously discussed to create that method. Let’s consider this Ruby script:

JRuby is going to declare the method, embed the logic, and then invoke it. In this particular case, JRuby will be able to use stack-based variables. Let’s build on our previous class (trimming some of the irrelevant bulk):

Wow, a lot going on here – let’s break it down.

• First, we create a Java method as a peer to a Ruby method. We also assign an annotation to it so it can be tracked in a number of ways by the running system. The method name is generated; I’ve simplified it slightly here, but you get the idea.
• Then, using the logic we’ve discussed previously regarding the variable management, our method body is compiled and has a rough sketch of what we did with A and B in the ruby implementation.
• The __file__ method (which represents the invokable part of the script) is created, and is invokable through the Script API
• __file__ declares our method first. It uses a static call to RuntimeHelpers to do this, which basically looks up the class, and stuffs in a new DynamicMethod object into the RubyClass definition that can call into this script. (Keep in mind in the JIT scenario, this ‘declaration’ is skipped, and instead the body of the method is compiled into the __file__ method, which is then referenced inside a JittedMethod object).
• Next, we get a handle on a CallSite object, and then call invoke on it. From here, Ruby is going to wind back through the call-process to our compiled method object we just put in the RubyClass.

Naively, this invocation can use reflection to make the call. However, remember from the first discussion that call-site objects often result in pre-compiled “mini-classes” that have a single purpose – calling the method on an object. These mini-classes can often be jitted out of the execution, making our indirect call-site invocation nearly as fast as a direct method call – cool!

Creating a Class
Class creation in Ruby scripts is not necessarily creation (it could be augmentation/modification). As such, creating a Java Class is not actually what needs to be done in this case. Besides, even if the class is created in a compiled script, it still needs to be in the JRuby class namespace so that non-compiled code can reference it. Therefore, a method is created that can perform the class construction. Here’s a basic modification of our example:

Our example grows again in this case:

So what’s changed since our basic method-only example? Well, the generated “my_method” is identical. However, we now have a method to create the class as well. Let’s analyze this method:

• First, given the current context, find the appropriate RubyModule.
• Ask the module to get or define the class with the given name (remember, we could be updating an existing class.
• Adjust the ThreadContext to have the class as the primary scope.
• Define the method (which will be defined on the current class in scope).
• Reset the ThreadContext

If we look at our file method, you’ll see the first thing it does this time is call this “class constructor”. Note also that the file method is now using heap-based variable management because the AST has classes mixed in, causing the inspector to throw out stack-based management due to the risk of leaky vars.

After calling the construct method, the method then invokes setValue for the current scope, passing in the result of calling ‘new’ on the new class. Then, the invocation of “my_method” is performed on the object held in “x” on the scope.

Other scenarios are just extensions of all of these same ideas.

Future Plans

As fancy as this current compiler architecture is, the JRuby team is not sitting still. Currently in JRuby master, there is a significant effort underway to reimagine the underlying compilation process. One of the problems with the current approach is that the AST does not lend itself well to analysis and optimization. The current work underway derives a new intermediate representation from the AST, providing an entirely separate object hierarchy. This new representation can then be shuffled and reorganized; patterns can be derived and compressed, and all sorts of other gooey goodness can be found. The IR is designed to be analyzed by the compiler; something that could never be said for the AST.

From a component standpoint, the plan (as best as I have derived) is to replace the true “compiler” component as described above, allowing the higher-level constructs that embed that compiler into the JIT process, leaving the rest of the infrastructure around it (like the JITCompiler class) mostly unchanged.

In other words, much of the existing ASTInspector/ASTCompiler goes away, in favor of something that is more compiler-friendly – an IR scanner.

This new IR code structure will bring the JRuby JIT one step closer to what is possible in the Java JIT, where optimizations can be made by analyzing the code structure; entire branches of code (and the associated conditions) can be eliminated, loops can be optimized, etc. Of course, the options for what can be optimized in a Ruby program are often different than Java – some of the language differences could be advantageous for optimization, and some may cause serious limitations – time will tell, but I have confidence it will be interesting to see the results.

Nevertheless, the work is fairly new on this, and I’d hate to spend too much time analyzing and documenting a work in progress, but it is something to revisit in a few weeks/months. At a minimum, I suggest following the JRuby committer blogs (for a start: Charlie Nutter, Tom Enebo, Nick Sieger).

Conclusion

This article was by far the longest yet, coming in well over 6000 words (and that means if you’ve read this far you are a patient, patient individual). Short of making this a “two-parter”, I’m not sure how I could have shortened it and truly hit the hotspots of the compilation process in Ruby; however I do think my next exploration might be a touch more ‘focused’.

I haven’t entirely decided yet what’s on the chopping block next for these JRuby internals articles, but I have a couple ideas; and of course, I’m always taking input at http://www.realjenius.com/contact.

Stay tuned.