A click on this Banner shows your site support to my sponsors
|
|
||||||||||||||||||||
|
OS 8.5 vs OS 8.1 MacBench 5 Graphics/Text Scores |
||||||||||||||||||||
|
Several readers wrote with comments they'd seen 2x or more lower MacBench scores under OS 8.5 for the text portion of the graphics tests. To verify this I requested tests be run on two systems, one with on-board video (a 8600) and one with a PCI graphics card (a 9600 with Thunder 3D PCI card). The results were surprising and consistent. MacBench 5.0 uses calls to standard Quickdraw/Quicktime routines as part of its tests so I have no reason to doubt these results. For more info on MacBench's test methods, see the online documentation.
At first I thought the lower text drawing speed was due to 'font smoothing' being enabled (antialiasing text - a new option in OS 8.5 that improves the look of onscreen text) but this was not the case. The Graphs below show performance with and without font smoothing enabled. The MacBench tests
uses a range of font sizes and the difference with smoothing enabled is
noticable during the test run (larger fonts look much better).
Benchmarketing: Remember like all benchmarks these primitives tests are often a small factor in real world application performance. Regular readers know I always mix benchmarks with real world tests in almost all my reviews. I'll not rant on benchmarks as being the ultimate indicator of performance (like Apple does with Bytemark - arguably the most meaningless benchmark as far as equating to real world apps performance).
Unlike Bytemark, MacBench uses actual system routine calls just like real applications do (in many cases profiling actual applications calls to the OS).
For Apple or anyone else to call that inaccurate is not being honest (sorry if the results don't always paint a rosy picture). If you take the time to read the MacBench documentation you'll see that fact is clear. Macbench uses the same calls that applications do. MacBench has proven to be a far better indicator of real world performance than Bytemark has - that fact is undeniable. How much it contributes to overall real world preformance depends on how heavily it's used and many other factors like how efficiently the application is written.
Keep in mind that these low level tests are the most basic system calls and their performance can be a tiny factor in overall
applications performance. What I'm showing here is apples for apples. Readers asked and I'm delivering a test of graphics primitives that show some OS 8.5 routines are faster, some are slower. Readers brought the text speed deltas to my attention, and I wanted to verify if the same benchmark test showed similar
results on other machines - it did.
The Macbench tests use standard system calls - just like real
apps do. Granted the font/text test are more stressful, using more
sizes and styles than most apps would (or that you would in a normal
document). But the fact remains that it's valid proof that *those routines*
have gotten much slower. This is a tiny subset of what you'd see in a normal
application, so it might be only as little as a 1% or less overall factor of the applications total performance, depending on how the application is written and how many times it calls the routine. Quality and efficiency of code can have a far greater impact on application performance. Also remember that in some cases even the lower score is more than fast enough to not be a bottleneck in actual OS use. Let's say the video hardware was able to write 500KB/sec in bandwidth but the screen mode only required 200K/Sec.
But for Apple to say it's not valid is wrong. Read the Macbench documention
that's linked in the article. Macbench is using the same calls that thousands of
actual apps are doing behind the scenes.
Test Systems :
The graphs below are self-explanitory. Only those results that showed
more than 2% difference were shown to keep the chart from being over a foot
long. Text tests were run on both machines at 1024x768, thousands colors and 1152x870, millions colors with the same relative scoring. 256 color mode tests were not run.
Thinking perhaps the onboard video played a factor in the lower text performance I requested results with a PCI video card equipped Mac, without onboard video. The results are shown below:
 From these tests and several reader reports in various Macs, it seems clear that graphics speed has increased, substantially in some operations, which text/character quickdraw routines performance is lower than OS 8.1. I've asked for a scrolling and panning test in Word 98 and Photoshop to see how much this affects actual applications performance. Watch for a update with the results to be posted here very soon. For a list of std MacBench 5.0 test results (CPU/FPU/Disk/Video) comparing OS 8.1 to 8.5, see the OS 8.5 MacBench Tests Results page posted previously. As a side note, performance in 3Dfx Quake was unchanged, but I expect RAVE and other games may see a boost based on the fact some graphics primitives were faster under OS 8.5. Word 98 Scrolling Tests: Although the test file contained far less font sizes and faces than the MacBench 5 tests, the file did contain a mix of sizes, tables, etc. and was sufficiently large (1.3MB disk file) to be of value. The test file used was the MacBench 4.0 doc file (a word 6 file). Times were recorded for the time it took to scroll through the document from top to bottom using the mouse held down on the down arrow key in Word 98. The results are listed below. "AA" denotes font smoothing enabled for text sizes above 12 point.
(OS 8.1 run from stock SCSI/HD, OS 8.5 run from UW SCSI Card/HD)
(OS 8.5 was run on a 4 Drive Raid striped volume connected to a Dual Channel UW SCSI card - OS 8.1 boot disk used the stock 4GB HD and onboard SCSI)
With the 8600 onboard video, Word 98 scrolling was faster in OS 8.5, even with font smoothing enabled. Another brand of PCI card might show different results than was seen on the 9600 as well (remember that PCI Graphics cards have shown less performance in general on the Kansas based motherboards, but also remember the same card was used for both OS versions, so relative performance scores are valid). However note the Thunder 3D was no slouch, it was almost twice as fast at the same resolution as the 8500 onboard video. I would have preferred to see OS 8.5 run from the same stock HD/SCSI as was used for OS 8.1, but that was beyond my control. Other applications could show different results. Scrolling involves more than just character rendering speed but it was the closest test I could think of to check real applications text drawing speed. Everyone loves the font smoothing in OS 8.5, and screenshots I've seen of comparisons show it really is a major improvement. It's one of many nice touches in OS 8.5, which will sell very well I expect.
Other Site Links of Interest:
Last Revised: 10/8/98 Your Source for the best in CPU/SCSI/VIDEO card reviews, daily news, and more! |
||||||||||||||||||||
|
No part of this site's content is to be reproduced in any form without written permission. |
