JRuby

The JRuby community is pleased to announce the release of JRuby 1.1 RC 3Homepage: http://www.jruby.org/Download: http://dist.codehaus.org/jruby/JRuby 1.1RC3 is the third and final(?) release candidate of JRuby 1.1.  JRuby 1.1 represents a concerted ... [More] focus on speed and refinement.  Ruby code can completely compile in an Ahead Of Time (AOT) or Just In Time (JIT) mode; yielding a faster Ruby!  It also uses less memory than our previous releases.We need people to download JRuby 1.1RC3 and give us feedback. Only uncovered severe issues (common library crashing, memory leak) will hold us from  promoting this as JRuby 1.1 final.  Please try your applications out and report all issues you encounter.Highlights: - 58 issues resolved since JRuby 1.1RC2- ri/rdoc w/ documentation included in distribution- More IO corner cases fixed (popen, reopen)- Several small bottlenecks fixedWe want to thank all the people who gave us feedback this last release cycle. We ended up resolving a tremendous number of issues in a short period of time.Issues fixed since 1.1RC2:JRUBY-884       Create and consolidate extension and non-standard optionsJRUBY-1104     Our system implementation shouldn't run everything with Ruby in the name with JRubyJRUBY-1143     rails executable was inapproprite permissionsJRUBY-1173     FindBugs issuesJRUBY-1199     STI should fall back to normal invocation when visibility changedJRUBY-1259     Method access is not threadsafeJRUBY-1260     'loop' reports wrong line for positionJRUBY-1267     Java 1.5.0_12 causes "yield called out of block" error during ant test on Windows and LinuxJRUBY-1352     ActiveRecord::ConnectionAdapters::JdbcConnection#columns returns empty array on MS SqlServerJRUBY-1442     ZenTest (autotest) fails to runJRUBY-1527     Tempfile fails to clean up because of lack of finalizer supportJRUBY-1546     Rails fails to run when using C. jruby.jit.threshold=0 works fine.JRUBY-1557     Using backquotes to run commands with shell redirects sometimes passes the redirects to the called programJRUBY-1604     Rubinius Hash specs conflict with 1.9 iteration behaviorJRUBY-1609     Thread.abort_on_exception is broken on windowsJRUBY-1629     retry out of the method leads to endless loop consuming a lot of CPU resourcesJRUBY-1739     gem update --system fails - need gem version 0.9.5 for rails 2.0.2JRUBY-1799     Thread pool should have configurable min/maxJRUBY-1888     Multiple tests in JRuby OpenSSL suite are not running during test runsJRUBY-1916     Rake file prints error after running taskJRUBY-1951     -s flag not supportedJRUBY-1958     Update OS_NAMES in RbConfigLibrary with additional mappings for other likely host OSesJRUBY-1961     Stop paying any attention to a pre-existing JRUBY_HOMEJRUBY-1975     bin files with - (minus) in the name trip up the -S commandJRUBY-2035     Op assignment much slower than MRIJRUBY-2072     IO#stat on non-file objects breaks JRuby hardJRUBY-2078     $stdin.dup doesn't workJRUBY-2125     silence_stderr fails trying to create /dev/nullJRUBY-2129     UNIXServer.open fails in JRubyJRUBY-2130     instance_eval not treated specially by ASTInspector, leads to bad eval framing and scopingJRUBY-2142     improved retroweaver support for jdk 1.4JRUBY-2150     Cannot open nor stat non-ASCII file namesJRUBY-2154     jruby script/console in RoR randomly stops taking in characters when typingJRUBY-2156     Weird behavior of rails consoleJRUBY-2159     dev/null file handled incorrectly in many casesJRUBY-2164     WEBrick on Linux randomly gets empty content-length for POSTJRUBY-2166     Regression: jirb on windows doesn't provide history anymoreJRUBY-2177     attr assign as in array[1,2] = 3 is much slower than MRIJRUBY-2193     Numeric bignum float and float bignum performance very slowJRUBY-2194     Bignum performance in rubinius's bm_bignum way slower than it should beJRUBY-2195     Coercing Fixnum and Float to Bignum is really slowJRUBY-2206     "Bad file descriptor" error when running jruby and rails JA-SIG CAS ruby packageJRUBY-2216     Running AOT compiled Scripts in a Jar get a Class Not Found Exception.JRUBY-2217     Regression: Can't stop mongrel running rails by Control-CJRUBY-2218     Visibility problem introduced in r6078JRUBY-2231     TCPServer.new (and possibly other servers) should coerce port to string and do equivalent of getservbyname to get the port numberJRUBY-2237     define_method call in rcov causes RuntimeExceptionJRUBY-2241     Comipled Codewith a require can not find the spcified files in the jarJRUBY-2246     Compiler fails to compile 500 line scriptJRUBY-2255     IO reopen problem when running RIJRUBY-2257       Marshal.dump and load don't treat ints the sameJRUBY-2258     r6160 Tempfile implemenation breaks default Ruby 1.8.6 behaviorJRUBY-2259     can't convert nil into Fixnum in CSVJRUBY-2266     Reflection based execution is brokenJRUBY-2273     NPE when attempting to set Java primitive field to nilJRUBY-2274     File.lchmod causes UnsatisfiedLinkError on unsupported platformsJRUBY-2277     Module#alias_method raises exception in some cases when java support has already been loadedJRUBY-2284     Distribute rdocs in our distribution [Less]

We were just discussing GSoC a bit, and another idea occurred to me:Survey existing language implementations and how they're solving similar problems like POSIX, code generation/compilation, parsing, and so on, and work with project leads to pull out ... [More] common solutions into reusable APIs.This would be a huge help to all such language projects, since we all really want to work together but we're generally swamped with fixes and whatnot for our own projects. It would even be possible to narrow the focus a bit, such as to take on POSIX and use Tom Enebo's current library in JRuby as a base to start building out complete JVM POSIX support for all languages. If you're interested in helping JVM languages succeed, this would be a great way to help them all at once. [Less]

This week has mostly been taken up with QCon London. I spent most of Monday, Wednesday, Thursday and Friday here, and I thought that I'd take the opportunity to write up some of my impressions and thoughts about the sessions I attended.First, in ... [More] general the conference definitely didn't disappoint me. It held at least as high standard as I had expected from earlier QCon and JAOO conferences. Solid speakers, a wide range of exciting topics and lots of interesting people made for a grade A conference.I started out on the Monday with listening to my colleagues Neal Ford, Rebecca J Parsons and Martin Fowler give a tutorial on domain specific languages. I've seen bits and parts of this tutorial before, but seeing as the three speakers are working on evolving it to a full and coherent "pedagogical framework" for teaching DSLs, the current presentation had changed quite a bit since the last time. I really liked it and I recommend it to anyone interested in getting a firm grasp about what DSLs are. Having Rebecca talk about external DSLs in the context of parsers and grammars makes total sense, and the only real thing I would say was a problem was the time allotted to it. Her part of the subject was large enough that 75 minutes felt a bit rushed. Of course, I don't see how Martins or Neals parts could be compressed much more either, so maybe the subject actually is too large for a one day tutorial? Anyway, great stuff.For several reasons I decided to spend the Tuesday working from the office instead of attending tutorials again.During the Wednesday I mostly spent my time in the exhibition hall, talking to people and doing general networking. For some reason the tracks I was least interested in had all been scheduled on the same day, so I was lazy and worked on other stuff in the ThoughtWorks booth.The evening keynote on Wednesday by Martin Fowler and Jim Webber was hilarious, and also managed to get a quite important message across. I had a good time.Thursday started the session attending for me, beginning with Markus Völters presentation of XText in the DSL track. Highly informative and something that I'll keep in mind if I see something that would be benefit from it. The approach is definitely not for all problem domains, of course.After that. Venkat Subramaniam gave a talk about how to blend dynamic languages with Java. This talk was useful in explaining why you'd want to do something like this, and why it's such a powerful technique. It also served to set up my talk - which was next in that track - about evolving the Java platform. My talk went well, but I had the timing for it really messed up, so I ran out of material 10 minutes earlier than I expected. Neal Gafter was in the audience and helped out with some corrections during the talk. =)Finally I headed back to the DSL track and saw Avi Bryant talk about DSLs in Smalltalk and then Magnus Christerson and Henk Kolk talk about the Intentional Workbench. Lots of neat stuff in both of these presentations.Then there was the speakers dinner... Lots of interesting discussions with lovely people. =)And then, more quickly than I had expected, the final day of QCon arrived. Me as a Ruby person and programming language nerd had quite a good selection of tracks. I ended up seeing Ted's presentation on F#, which made me feel: wow! Microsoft took ML and did exactly what they've done to all languages on the CLR - added support for .NET objects in the mix. The talk ended with a quite strange discussion about whether F# actually helps with concurrent programming or not, and why a functional programming language has primitives that allow you to have mutable state.After that I did my talk in the Ruby track, talking about more advanced things regard JRuby. It ended up being great fun, and I spent lots of time in the talk answering questions and showing how seamlessly things work with JRuby. I ended up eating up 10 minutes of everyone's lunch time, but I had a great time and I thing most in the audience had too.Feeling happy and finished with my contributions, I ended up in the Erlang talk by Joe Armstrong. It gave a quite good overview of why Erlang was created and how it solves some of the problems in that particular problem domain. There is no doubt that Armstrong is an entertaining talker, but his buffoon image gets a little tiring and repetitive after a while. Some of the things that interested me in the talk was missing too. He started out saying that Erlang solves a particular problem, but then expanded that into something that sounded like "Erlang should be used for everything, everywhere". I tried to ask a question related to that, but the answer didn't really go in the direction I was interested in.I stayed in the languages track and saw the introduction to Scala, which is always fun, except that I'd already learned most of the things showcased. The most interesting about the presentation was the audience interest and questions.Finally I realized that my contributions were not over at all, since I'd agreed to be part of the closing Ruby panel. This ended up revolving quite a lot around the question whether Ruby and Windows ever will be a good match, if this is important, and if we really want to push Ruby into all kinds of environments.The closing panel were OK, but nothing special. It ended the day on a good note, but at that time I was tired enough to fall asleep in my chair. For some reason this always happens that last day of conferencing.Anyway. I had a great time and I look forward to being back the next time. I can definitely recommend QCon as one of the best conferences around in this industry. [Less]

I'm attending QCon in London this whole week. I'll give a talk on Thursday called "Evolving the Java platform", talking about different things that might be a part of the next generation Java platform. On Friday I give the talk "JRuby: Power on the ... [More] JVM", which is a more advanced JRuby talk than the regular introduction talks I usually give.Hope to see you around! [Less]

It's been one of those nights, my friends! An outstanding night!All day Sunday I had the stupids. It was probably related to a few beers on Saturday night, or the couple glasses of red wine before that. Whatever it was, I didn't trust myself to work ... [More] on any JRuby bugs for fear of introducing problems if my brain clicked off mid-stream. So I started playing around with my old bytecode generating DSL from a few months back. Then things got interesting.We've long wanted to have a "Ruby-like" language, probably a subset of Ruby syntax, that we could compile to solid, fast, idiomatic JVM bytecode. Not a compiler for Ruby, with all the bells and whistles that make Ruby both difficult to support and inefficient to use for implementing itself. A real subset language that produces clean, tight JVM bytecode on par with what you'd get from compiled Java. But better, because it still looks and feels mostly like Ruby.So I wrote one! And I used my bytecode library too!Let's say we want to implement our good friend fib.class Foo def fib(n) if (n < 2) n else fib(n - 2) fib(n - 1) end endendThis is normal ruby code. Given a Fixnum input, it calculates the appropriate fibbonaci number and returns it. It's slow in Ruby for a few reasons:In JRuby, it uses a boxed integer value. Matz's Ruby and Rubinius use tagged integers to improve performance, and we rely on the JVM to optimize as much as it can (which turns out to be a *lot*). But it's still way slower than using primitives directly.The comparison operations, integer math operations, and fib operations are all dynamic invocations. So there's at least a bit of method lookup cost, and then a bunch of abstraction cost. You can reduce it, but you can't eliminate it.There are many Ruby features that influence performance negatively even when you're not using them. It's very difficult, for example, to optimally store local variables when the local scope can be captured at any time. So either you rely on tricks, or you store local variables on the heap and deal with them being slow.When working with a statically-typed language, you can eliminate all of this. In Java, for example, you have both object types and primitive types; primitive operations are extremely fast and eventually JIT down to machine-code equivalents; and the feature set is suitably narrow to allow current JVMs to do amazing levels of optimization.But of course Java has its problems too. For one, it does very little guessing or "inferring" of types for you, which means you generally have to declare them all over the place. On local variables, on parameters, on return types. C# 3.0 aims to correct this by adding type inference all over, but then there's still other syntactic hassle using any C-like language: curly braces, semicolons, and other gratuitous syntax that make up a lot of visual noise.Wouldn't be nice if we could take the code above, add some type inference logic, and turn it into a fast, static-typed language?class Foo def fib(n) {n => java.lang.Integer::TYPE, :return => java.lang.Integer::TYPE} if (n < 2) n else fib(n - 2) fib(n - 1) end endendAnd there it is! This is the same code as before, but now it's been decorated with a little type declaration block (in the form of a Ruby hash/map) immediately preceding the body of the method. The type decl describes that the 'n' argument is to be mapped to a primitive int, and the method itself will return a primitive int (and yes, I know those could be inferred too...it shall be soon). The rest of the method just works like you'd expect, except that it's all primitive operations, chosen based on the inferred types. For the bold, here's the javap disassembly output from the compiler:Compiled from "superfib.rb"public class Foo extends java.lang.Object{public int fib(int); Code: 0: iload_1 1: ldc #10; //int 2 3: if_icmpge 10 6: iload_1 7: goto 27 10: aload_0 11: iload_1 12: ldc #10; //int 2 14: isub 15: invokevirtual #12; //Method fib:(I)I 18: aload_0 19: iload_1 20: ldc #13; //int 1 22: isub 23: invokevirtual #12; //Method fib:(I)I 26: iadd 27: ireturnpublic Foo(); Code: 0: aload_0 1: invokespecial #15; //Method java/lang/Object."<init>":()V 4: return}A few items to point out:A default constructor is generated, as you'd expect in Java. This will be expected to also recognize "def initialize" constructors. I haven't decided if I'll allow overloading or not.Notice the type signature for fib and all the type signatures for calls it makes are correctly inferred to the correct types.Notice all the comparison and arithmetic operations are compiled to the correct bytecodes (iadd, isub, if_icmpge and so on).And the performance is what you'd hope for:$ jruby -rjava -e "t = Time.now; 5.times {Java::Foo.new.fib(35)}; p Time.now - t"0.681$ jruby -rnormalfib -e "t = Time.now; 5.times {Foo.new.fib(35)}; p Time.now - t"27.851Here's another example, with some string operations thrown in:class Foo def bar {:return => java.lang.String} 'here' endendclass Foo # reopening classes works in the same file only (for now) def baz(a) {a => java.lang.String} b = "foo" a = a bar b puts a endendIt works, of course:$ jruby -rjava -e "Java::Foo.new.baz('Type inference is fun')"Type inference is funherefooAnd once again, the disassembled output:Compiled from "stringthing.rb"public class Foo extends java.lang.Object{public java.lang.String bar(); Code: 0: ldc #13; //String here 2: areturnpublic void baz(java.lang.String); Code: 0: ldc #15; //String foo 2: astore_2 3: aload_1 4: checkcast #17; //class java/lang/String 7: aload_0 8: invokevirtual #19; //Method bar:()Ljava/lang/String; 11: invokevirtual #23; //Method java/lang/String.concat:(Ljava/lang/String;)Ljava/lang/String; 14: checkcast #17; //class java/lang/String 17: aload_2 18: invokevirtual #23; //Method java/lang/String.concat:(Ljava/lang/String;)Ljava/lang/String; 21: astore_1 22: getstatic #29; //Field java/lang/System.out:Ljava/io/PrintStream; 25: aload_1 26: invokevirtual #34; //Method java/io/PrintStream.println:(Ljava/lang/String;)V 29: returnpublic Foo(); Code: 0: aload_0 1: invokespecial #36; //Method java/lang/Object."<init>":()V 4: return}Notice here that the operation was detected as acting on two strings, so it was compiled to call String#concat rather than try to do a numeric operation. These sorts of mappings are simple to add, and since there's type information everywhere it's also easy to come up with cool ways to map Ruby syntax to Java types.My working name for this is going to be Duby, pronounced "Doobie", after Duke plus Ruby. Duby! It has a nice ring to it. It may be subject to change, but that's what we'll go with for the moment.Currently, Duby supports only the features you see here. It's a very limited subset of Ruby right now, and the subset doesn't support all Java primitive types, for example, so there's a lot of blanks to be filled. It also doesn't support static (class) methods, doesn't wire up "initialize" methods, doesn't support packages (for namespacing) or imports (to shrink type names) or superclasses or interfaces or enums or generics or what have you. But it's already functional for simple code, as you see here, so I think it's a great start for 10 hours of work. The rest will come, as needed and as time is available.What are my plans for it? Well many of you may know Rubinius, an effort to reimplement Ruby in Ruby, modeled after the design of many Smalltalk VMs. Well, in order to make JRuby more approachable to Ruby developers, Duby seems like the best way to go. They'll be able to write mostly idiomatic Ruby code and know it will both perform at Java speeds and provide compile-type checking that all is wired up correctly. So we can avoid the overhead of "turtles all the way down" by just teaching one turtle how to speak JVM bytecode and building on that.I also hope this will lead to lots of new ideas about how to implement languages for the JVM. It's certainly attractive to language users to be able to contribute to language implementations using the language in question, and if we can come up with compilers and sub-languages like Duby it could make the JVM more approachable to a wide range of developers.Oh, and for those of you curious: the Duby compiler is written mostly in Ruby too. [Less]

With JRuby 1.1 approaching, performance looking great, and general Ruby compatibility better than its ever been, we've started to branch out into more libraries and applications. So a couple days ago, I thought I'd make good on a promise to myself ... [More] and have a look at getting RubyInline working on JRuby.RubyInline is a library by Ryan "zenspider" Davis which allows you to embed snippits of C code into your Ruby scripts. RubyInline does a minimal parse of that source, and based on the function signature you provide it wires it to the containing class as a Ruby method entry point and performs appropriate entry and exit type conversion to whatever C types you happen to use. It's particularly useful if you have a small algorithm you need to run fast, and you'd like to run it in a somewhat faster language by "throwing work over" to it.Here's the example of RubyInline from Ryan's page:class MyTest def factorial(n) f = 1 n.downto(2) { |x| f *= x } f end inline do |builder| builder.c " long factorial_c(int max) { int i=max, result=1; while (i >= 2) { result *= i--; } return result; }" endendThe interesting bit is the builder.c call, one of several functions on the C builder. Others allow you to add arbitrary preamble code, imports, and "bare" methods (with no type conversion) among other things. And as you'd expect, performance is greatly improved by writing some algorithms in C instead of Ruby.Naturally, we want to have the same thing work in Java, and ideally use the same API and the same RubyInline plumbing for the rest of it. So then, I present to you java_inline, a RubyInline builder for JRuby!This represents about four hours of work, and although it doesn't yet have a complete complement of tests and could use some "robustification", it's already working in about 100 lines of code. It's made far easier in JRuby than in C Ruby because we already have a full-features Java integration layer to handle the argument mapping.Here's a sample java_inline script to show what it looks like, similar to the fastmath.rb sample provided with RubyInline.So how does it work? Well the Ruby side of things is largely the same as RubyInline's C builder...parse signature, compose the code together and compile it, and bind the method. It wires directly into the RubyInline pipeline, so all you need to do is install RubyInline, require the java_inline.rb script and you're all set. On the Java side, it's using the Java Compiler API, provided in Java 6 implementations. (OT: This has to be the worst-designed API I've ever seen. Go see for yourself. It's cruel and unusual. I won't dwell on it.) So yes, this will only work on Java 6. Deal with it...or submit a patch to get it working on Java 5 as well :)It's not released in any official form yet, but I'll probably try to wire up a gem or something. I have to make sure I'm dotting my eyes and crossing my tees when I release stuff, even if it's only 100loc. But the repository is obviously public, so play with it, submit patches and improvements, and let me know if you'd like to use it or help work on it more. I'm also interest in suggestions for other libraries you'd like to see special JRuby support for, so pass that along too. [Less]

Upcoming event round-up!Next weekend, Tom Enebo, Nick Sieger and I will be at PyCon, checking out all the pythony goodness and hooking up with the Jython guys for some hacking.Tom and I will travel to Prague for Euruko 2008, the European RubyConf. ... [More] We'll present JRuby, unsurprisingly, and hopefully meet up with more European Rubyists we haven't talked to before.After Euruko, Tom and I will be in Edinburgh for Scotland on Rails. We're supposed to do a 90-minute talk about JRuby on Rails. We'll try to make it entertaining and informative. [Less]

The talk I gave at Google last week is now online: http://youtube.com/watch?v=PfnP-8XbJao

Today we can finally announce another exciting language-related event: we've hired Frank Wierzbicki of the Jython project and Ted Leung of OSAF (and a bunch of other stuff) to help with the Python story at Sun. Hooray!Frank Wierzbicki has been ... [More] tirelessly plodding away on Jython for years, since the early days. Even when Jython was considered by many to be "dead", Frank kept the project moving forward as best he could. Now he's been a major part of Jython's revitalization...a new release last year and rapid work to get compatibility up to current-Python levels prove that.Ted Leung has long been involved with the Open Source Applications Foundation and the Apache Software Foundation, and was one of the original developers of the Xerces Java XML parser. I don't know Ted personally, but I'm excited to be working with him, especially in light of some of this previous roles.So what does this mean for Sun? Well, it means we're serious about improving support for Python on Sun platforms. Jython is a big part of the story, since they have many challenges similar to JRuby, but a bunch of new ones as well. So we'll be looking to share key libraries and subsystems as much as possible, and we'll start looking at Jython as another driver for future JVM and platform improvement. On the non-Java side of the world, it means we'll ramp up support for Python itself. Sun has already settled on Mercurial for source control for all our OSS projects, and the package management system being worked up for Indiana is written in Python as well. But there's more we need to do.Please help me in congratulating Frank and Ted on their new roles at Sun. It's going to be another interesting year for polyglots :) [Less]

Finally the full announcement is available. We're pretty excited about this one...The University of Tokyo and Sun Microsystems Commence Joint Research Projects on High Performance Computing and Web-based Programming LanguagesImproving Efficiency and ... [More] Simplicity of Fortress, Ruby and JRuby Languages is Current Focus for Unique Academic-Corporate Collaborative ModelTOKYO and SANTA CLARA, Calif. -- February 27, 2008 -- The University of Tokyo and Sun Microsystems, Inc. (Nasdaq: JAVA) today announced two joint research projects that will focus on High-Performance Computing (HPC) and Web-based programming languages....The two research topics are:Development of a library based on skeletal parallel programming in FortressImplementation of a multiple virtual machine (MVM) environment on Ruby and JRubySome of you knew about this from Tim Bray's rather cryptic announcement at RubyConf. The basic idea here is that we on the JRuby team and folks in Professor Ikuo Takeuchi's group at the University of Tokyo will be cooperating to come up with an MVM specification and implementations for the Ruby 1.9 and JRuby codebases. Tom and I have had brief discussions with Professor Koichi Sasada (ko1, creator of YARV) about this already, and there are mailing lists and wikis already running. I've also made sure that Evan Phoenix, of Rubinius, is included in the discussions. Like JRuby, Rubinius largely has MVM features implemented, but the eventual form they'll take and API they'll present is up in the air.This should be an exciting collaboration between Sun and U Tokyo, as well as between the three most functional next-generation Ruby implementations. I'll certainly keep you posted on all MVM developments as they come up, and I welcome any comments or suggestions you might have. [Less]