Thursday, March 12, 2009

More Compiling Ruby to Java Types

I did another pass on compiler2, and managed to wire in signature support. So let's look at a couple examples:
class MyRubyClass
def helloWorld
puts "Hello from Ruby"
def goodbyeWorld(a)
puts a

signature :helloWorld, [] => Java::void
signature :goodbyeWorld, [java.lang.String] => Java::void

In this case we have our friend MyRubyClass once again, with helloWorld and goodbyeWorld methods. You'll recall from my previous post that these two methods originally compiled as returning IRubyObject, and goodbyeWorld compiled as receiving a single IRubyObject parameter.

But with signature support, things are so much cooler! The two "signature" lines at the bottom of the class (syntax and structure are totally up for debate) associated signatures with the two methods. helloWorld receives no parameters and has a void return type. goodbyeWorld receives a single String parameter and has a void return type.

The compiler takes this new information, and produces a more normal-looking set of Java signatures:
Compiled from ""
public class MyObject extends org.jruby.RubyObject{
static {};
public MyObject();
public void helloWorld();
public void goodbyeWorld(java.lang.String);

Huzzah! There's almost nothing here to give away that we're actually dealing with Ruby code under the covers. And the code that consumes this is just as simple:
public class MyObjectTest {
public static void main(String[] args) {
MyObject obj = new MyObject();

And that's literally all there is to it. Here's a more advanced example:
class MyRubyClass
%w[boolean byte short char int long float double].each do |type|
java_type = Java.send type
eval "def #{type}Method(a); a; end"
signature "#{type}Method", [java_type] => java_type

This time we're actually *generating* the methods, looping over a list of Java primitives and eval'ing a method for each. So this is *runtime* generation of methods, like any good Rubyist loves to do. And of course, this is absolutely no problem for compiler2:
Compiled from ""
public class MyObject2 extends org.jruby.RubyObject{
static {};
public MyObject2();
public double doubleMethod(double);
public int intMethod(int);
public char charMethod(char);
public short shortMethod(short);
public boolean booleanMethod(boolean);
public float floatMethod(float);
public long longMethod(long);
public byte byteMethod(byte);

All the methods are there, just as you'd expect them! Fantastic!!! (Though the ordering is a little peculiar; I think that's because we don't have an ordered method table in our class impl. Does it matter?)

Even better, the above methods are doing the same type coercion on the way in and out that we do for any other Java-based method calling. So your integral numerics are presented to Ruby as Fixnums, floating-point numerics are Floats, and booleans come through as Ruby true or false.

There's certainly more work to be done:
  • There's no support for overloads at the moment, but I'll likely provide a method aliasing facility so you can define multiple Ruby methods and then say which one maps to which overload. And of course, you'll be able to define multiple overloads that go to the same method body if you wish.
  • I also have not wired in varargs, but it will be an easy match to Ruby's restargs. And optional arguments could automatically generate different-arity Java signatures.
  • Annotations will also be trivial to add; it's just a matter of attaching appropriate metadata and having compiler2 emit them. So you'll be able to use JavaEE 5, JUnit4, and any other frameworks that depend on having annotations present.
Of course this is all checked into JRuby trunk, so feel free to give it a try. Stop by JRuby mailing lists or IRC if you have questions. And it's all still written in Ruby; signature support bloated the compiler up to a whopping 178 lines of code, most of that for dealing with the JVM opcodes for primitive types.

This is just the beginning!

Tuesday, March 10, 2009

Compiling Ruby to Java Types

"Compiler #2" as it has been affectionately called is a compiler to turn normal Ruby classes into Java classes, so they can be constructed and called by normal Java code. When I asked for 1.3 priorities, this came out way at the top. Tom thought perhaps I asked for trouble putting it on the list, and he's probably right (like asking "prioritize these: sandwich, pencil, shiny gold ring with 5kt diamond, banana"), but I know this has been a pain point for people.

I have just landed an early prototype of the compiler on trunk. I made a few decisions about it today:
  • It will use my bytecode DSL "BiteScript", just like Duby does
  • It will use the *runtime* definition of a class to generate the Java version
The second point is an important one. Instead of having an offline compiler that inspects a file and generates code from it, the compiler will actually used the runtime class to create a Java version. This means you'll be able to use all the usual metaprogramming facilities, and at whatever point the compiler picks up your class it will see all those methods.

Here's an example:

# myruby.rb
require 'rbconfig'

class MyRubyClass
def helloWorld
puts "Hello from Ruby"
if Config::CONFIG['host_os'] =~ /mswin32/
def goodbyeWorld(a)
puts a
def nevermore(*a)
puts a

Here we have a class that defines two methods. The first, always defined, is helloWorld. The second is conditionally either goodbyeWorld or nevermore, based on whether we're on Windows. Yes, it's a contrived example...bear with me.

The compiler2 prototype can be invoked as follows (assuming bitescript is checked out into ../bitescript):

jruby -I ../bitescript/lib/ tool/compiler2.rb MyObject MyRubyClass myruby

A breakdown of these arguments is as follows:
  • -I ../bitescript/lib includes bitescript
  • tool/compiler2.rb is the compiler itself
  • MyObject is the name we'd like the Java class to have
  • MyRubyClass is the name of the Ruby class we want it to front
  • myruby is the library we want it to require to load that class
Running this on OS X and dumping the resulting Java class gives us:

Compiled from ""
public class MyObject extends org.jruby.RubyObject{
static {};
public MyObject();
public org.jruby.runtime.builtin.IRubyObject helloWorld();
public org.jruby.runtime.builtin.IRubyObject nevermore(org.jruby.runtime.builtin.IRubyObject[]);

The first thing to notice is that the compiler has generated a method for nevermore, since I'm not on Windows. I believe this will be unique among dynamic languages on the JVM: we will make the *runtime* set of methods available through the Java type, not just the static set present at compile time.

Because there are no type signatures specified for MyRubyClass, all types have defaulted to IRubyObject. Type signature logic will come along shortly. And notice also this extends RubyObject; a limitation of the current setup is that you won't be able to use compiler2 to create subclasses. That will come later.

Once you've run this, you've got a MyObject that can be instantiated and used directly. Behind the scenes, it uses a global JRuby instance, so JRuby's still there and you still need it in classpath, but you won't have to instantiate a runtime, pass it around, and so on. It should make integrating JRuby into Java frameworks that want a real class much easier.

So, thoughts? Questions? Have a look at the code under tool/compiler2.rb in JRuby's repository. The entire compiler is so far only 78 lines of Ruby code.

Saturday, February 28, 2009

Help Us Set Priorities for JRuby 1.3

With JRuby 1.2 almost out the door, I want to talk a bit about where we should go with JRuby 1.3. There's always more work to do, but in this case there's a few different directions we could probably go.

Some obvious items will continue to see work:
  • 1.9 libraries, interp, compiler, parser
  • 1.8.6 bugs
  • "Pure ruby" application support, like Rails deployment stuff (Warbler, AR-JDBC)
But there's other areas that we may want to prioritize:
  • 1.8.7 support
  • Ruby execution performance (how fast do you want it?)
  • Specific library performance (YAML, IO, Java)
  • More Java integration improvement/refactoring (esp. subclassing)
  • "Compiler #2" to produce normal Java classes from Ruby
  • Improvements to AOT compilation (all-at-once, eliminate runtime codegen)
  • Expand support for embedded/mobile platforms
And there's a number of internal chores to work on too:
  • Start generating most of the call path, to reduce duplicate code
  • Specific-arity optimizations for block yield (could be big)
  • Compiler cleanup and refactoring
  • Modularization of core classes that aren't valid on applet, Android, secured envs, etc; also may allow shipping smaller runtimes
  • More startup perf work; I have a few ideas
As always, there's way more tasks than the few of us committing to JRuby can work on, so I think we need to hear from users what's important. Any of these? Other items?

Friday, February 27, 2009

JRuby on Java ME

A number of folks have asked us over the years whether JRuby could run on a Java ME-enabled device. I've always said I believed it would be possible with enough trimming. Strip out all the libraries that ME doesn't support, and you should be left with a runnable "core" of JRuby. Roy Ben Hayun, formerly of Symbian and (I believe) now working at Sun, actually proved it possible but difficult back in 2006 with his JRubyME project, a stripped down JRuby 0.9.8. But that didn't have compiler support, and we were still dog slow in 2006. Has JRuby, with two years of work behind it, grown too complex to run on an embedded device? Is the Rhodes project right when they say JRuby is too large for this to work?

Tony Arcieri, of Revactor fame, put a bug in my ear about Java ME again this week. He's interested in running Ruby on a device hosting the Connected Device Configuration (CDC) level of Java ME. Specifically, the device in question supports CDC plus the APIs included in the Personal Basis Profile (PBP). CDC is probably about the smallest ME profile JRuby could reasonably run in, since anything lower (like CLDC, the Connected Limited Device Configuration) and you're reduced to the most basic collections, IO, and utility APIs.

So, I'm happy to announce that the JRuby CDC Project has spawned out of the JRuby codebase.

Here's what it looks like running, using Sun's reference implementation of CDC+PBP:
~/cdc/Optimized_nosym$ bin/cvm -classpath ../jruby.jar org.jruby.Main -X-C -e "[1,2,3].each {|i| puts i}"

The "cvm" is Sun's reference implementation of an embedded JVM, and this particular package includes the PBP level APIs as well. The jruby.jar here is the one I've stripped in the jruby-cdc project, but with the additional step of retroweaving it back to Java 1.3-compatible bytecode.

My intention with this is to get it running with a base set of Ruby classes but with Java integration still functional. That will allow most basic Ruby scripts to work while providing access to the rest of the Java APIs for bits I had to rip out (like IO APIs)

It's only running interpreted right now, just like the Android version of JRuby, but as in that case I expect most people will want to precompile Ruby code and ship it all as one unit.

Why bother? Well, there are still a lot of Java ME devices out there. And while the device you carry in your pocket may be growing beyond Java ME, the set-top box or Blu-Ray player in your living room is just starting to reach that level. The technology is still sound, if perhaps less obvious than an iPhone or Android. And hey, we want "Ruby everywhere," don't we?

Where do we go from here? I think this and Ruboto are proof that damn near anything is possible with JRuby. It really only took me a couple days of trimming to get this far. Fully supporting Android in Ruboto will not take long, and other "mini ruby" profiles are possible for other platforms and use cases as well. It just takes a little imagination, and maybe a few late-night hacking sessions.

Now...for my next trick...