Kawa in Languages shootout
Updated: Now with numbers for chameneos-redux
(chameneosredux.scm) and meteor-contest (meteor.scm) contributed by Jamison Hope.
The Computer Language Shootout
Benchmarks Game has implementations in a number of different languages
of a number of different programs.
We now have Kawa-Scheme versions of all the current programs.
You can find them in the testsuite
directory of the Kawa source code.
You can look at the raw measurements.csv or data.csv, but the table below (generated with a Scheme script) summarizes the results. The numbers are relative cpu-time, as divided by the fastest Java version. (For now I'm focusing on single-CPU performance. I'll look into multi-threaded solutions later, after I get a quad-core laptop to test with.) Each cell may have multiple numbers, indicating multiple programs for a given language and problem. In the Java column, at least one of the results will be 1.00, as expected.
| Java | Scala | Clojure | JRuby | Kawa | Kawa/FTC | |
| binarytrees | 1.00 | 1.05 1.46 1.04 | 27.36 | 1.10 | 3.72 | |
| chameneosredux | 1.00 3.92 1.85 | 0.99 4.08 | 46.43 | 0.95 | 1.96 | |
| fannkuchredux | 1.00 1.43 | 1.23 0.94 | 3.54 | 61.66 | 1.38 | 1.34 |
| fasta | 1.00 | 1.33 0.67 | 5.32 4.32 | 71.36 58.50 | 1.72 | 5.46 |
| knucleotide | 1.06 1.00 | 2.16 1.46 | 7.93 7.22 | 10.54 8.93 | 11.11 1.62 | 11.18 1.71 |
| mandelbrot | 1.00 1.01 1.06 1.09 | 1.19 1.16 1.10 | 2.28 | 186.49 | 1.05 | 1.05 |
| meteor | 2.16 1.00 | 13.25 | 41.80 26.30 | 1.25 | 2.11 | |
| nbody | 1.00 | 1.23 | 2.25 | 128.50 | 1.03 | 1.05 |
| pidigits | 1.00 | 0.30 0.83 | 1.28 | 0.63 | 0.63 | |
| regexdna | 1.29 1.00 1.00 | 1.51 1.27 1.00 | 1.63 | 4.20 2.80 1.82 | 1.05 | 1.04 |
| revcomp | 1.00 | 0.81 23.72 | 9.52 5.55 | 7.15 | 1.43 1.17 | 1.44 1.17 |
| spectralnorm | 1.00 1.05 | 1.02 1.03 | 137.54 174.76 | 1.05 | 8.98 | |
| threadring | 8.07 1.00 | 5.84 | 16.02 21.03 | 0.96 | 0.96 |
All of the languages first compile to .class files, which
are then executed by the JVM.
The Kawa column is Kawa with the default calling convention,
while the Kawa/FTC column is the same Kawa programs compiled using the --full-tailcalls flag. The latter is a bit slower, especially for programs that do many function calls, but not as bad I had feared.
(The --full-tailcalls mechanism has not been optimized/profiled
as much as I'd like, though it's generally going to be slower.)
The Kawa programs do use
type annotations, and are generally written with performance in mind.
As expected, Scala and Kawa come closest to Java performance.
Note that the Scala and Kawa versions of pidigits are actually faster
than the Java version, but that is because this test depends heavily
on bignum (infinite-precision integer) calculations, and the
standard Java BigInteger class is not as efficient as
it could be. (Alternative Java version of pidigits use the gmp library.)
Clojure is bit slower, while JRuby is substantially slower.
Version notes
The tests were run on a Lenovo ThinkPad T400, with Intel Core2 Duo CPU P8600 (2.40GHz stepping 06), and 3GB of main memory. The operating system was Fedora GNU/Linux 13, updated.
The Java VM used was OpenJDK version 1.6.0_21. The JRuby version was 1.5.3. Clojure was version 1.2.0. Scala was version 2.8.0.final. Kawa was version 1.11.