|
Mac Java Runtime v2.1 vs v2.0
CaffeineMark Performance Comparisons
Published: 2/16/99
[Update: There is a v2.1.1 update that offers higher performance and possible bug fixes at: www.apple.com/java
More reader feedback on comparisons, including PC results are in the update section below.]
Ryan Tanaka sent a performance comparison between the new MRJ 2.1 (see Apple's Software Updates) and the previous version:
"
Hi Mike,
I d/l MRJ 2.1 and it appears much faster than 2.0. Included are screenshots
of the Caffeine benchmark.
The benchmarks were done on a 7500/G3 (350/116/512k) in 24-bit color.
Ryan Tanaka
"
Summary Table of Results:
I summarized the data from Ryan's MRJ 2.1 and MRJ 2.0 graphs into a table for a side-by-side comparison between the two versions:
| MRJ 2.1 vs 2.0 Performance Summary |
| Test | MRJ 2.1 | MRJ 2.0 | Gain |
| Sieve: | 5813 | 2133 |
2.72 X |
| Loop: | 16794 | 3706 |
4.53 X |
| Logic: | 53676 | 6476 |
8.28 X |
| String: | 7226 | 421 |
17.16 X |
| Float: | 7074 | 2192 | 322.7% |
| Method: | 4652 | 3029 |
153.6% |
| Graphics: | 321 | 158 |
203.2% |
| Image: | 478 | 187 |
255.6% |
| Dialog: | 1922 | 44 |
4,368.2% |
Overall CM
Score: | 4152 | 859 |
483.4% |
Conclusion: Apple's claims of up to 5X improvements in 2.1 appear conservative based on these results. Definately a dramatic boost! There have been some compatibility problems reported however.
As noted in today's main site news, current versions of Netscape can't use the new runtime.
Updates:
More reader feedback and test data (most recent first).
Aaron Hedquist sent results from his B&W G3/350:
Brian Doerr sent the higest score I've seen so far, from his G3/333 with Apples Applet Runner 2.0:
MRJ 2.1/MS JIT/Syamntec JIT/Cyrix PC: I labelled and linked Jerry's graphs into the text below, clicking on the link will show the CaffeineMark graph.
"
Hi Mike,
This machine is a Power Macintosh 7500, with 128 MB RAM, with a Railgun
275/1 MB running at 308/154 MHz. Graphics card in this machine is an 8
MB MacTell Vision3D Pro II Lite running in Thousands of colors (16-bit
color) mode at 1152x870. The computer has 192 MB RAM and VM and RAM
Doubler were off. Mac OS version is 8.1 I benched Apple MRJ 2.1,
Symantec JITspeed (with Navigator 4.06), and Microsoft's VM (both with
and without the JIT compiler enabled). Composite results are as follows:
Apple MRJ 2.1: 4109 (wooohoooooo!!!)
Microsoft (JIT): 1845
Microsoft (No JIT): 524
Symantec JITspeed: 2659
I threw in Symantec JITspeed for obvious reasons -- it's essentially the
same JIT compiler Apple is using. But, oddly, it's quite a bit slower,
though respectable compared to anything but MRJ 2.1. If you have
philosophical objections to Microsoft products and don't want to wait
until Netscape 5 to speed up your Java, JITspeed may be a good investment
at $30 or so.
Just for grins, I also ran the benchmark on my Cyrix 200-powered Windows
95 machine, under IE 4 (display also set to Thousands of colors, the
video card is built into this machine's motherboard so I can't tell you
what it is exactly):
Cyrix 200, Win95, MSIE4: 1398
MRJ is now in the same ballpark as a PC, validating Apple's claims. Note
that Pendragon's Web page states that the highest score currently
recorded is a 3963 from a 300 MHz Pentium.
One of the interesting things, though, is that while the composite CM
score of MRJ 2.1 is very good, the PC's graphics are still significantly
faster (1180/789 for the Graphics and Image tests on Pendragon's best
results, vs. 540/289 for the same tests on my machine). The PC tested
had a Matrox Millennium installed; my Vision3D ProII Lite should actually
be significantly faster. Yet it's less than half the speed. Since the
speed of a computer's graphics display and UI updates contibutes
disproportionately to user impression of speed, this means the PC can
still _seem_ significantly faster even if the Mac is ahead overall. MRJ
could still use some work here. [The OS might also be a factor (non-native code, latency, lack of true multitasking-Mike]
I also noticed that my PC still loads and displays Web pages (and Java
applets) faster. Both machines were on the 'Net via my MediaOne cable
modem (IPNetRouter running on the Mac allowed the PC to use the modem).
The actual applet loading process took only about 2 seconds on the PC vs.
several on the Mac. Another area for improvement.
Another problem is that MRJ screws up some applets in cosmetic ways. One
chat site I visit uses a Java applet for chatting. MRJ 2.1 displays some
of the stuff _outside_ the applet's allocated screen area! Microsoft's
VM, on the other hand, colors inside the lines (so does JITspeed, for
that matter). Humorously, this problem is not limited to MRJ -- the PC
cuts off the "CaffeineMark" logo from the benchmark window.
Some small screen captures included, including one from my Windows
machine.
Jerry Kindall
Technical Writing
Manual Labor
http://www.manual.com
"
|
.gif){kind=link}
.gif){kind=link}
