(Update: For comparison tests of the new Radeon 8500 AGP to the GeForce3, GeForce2MX, Radeon 7500 and GeForce4MX in a Dual 1Ghz G4 - see this Radeon 8500 AGP preview.)
Regardless of pure benchmark results, what matters most is performance in actual applications (or games). For this weekend's tests I tried to choose a small number of applications that were common and popular. (Photoshop, Appleworks, Quake3 and Unreal Tournament.) There are many others than could have been run, but with limited time I chose the most popular (based on reader interest/mails) of the programs I personally own.
Resolutions Available (w/Sony FW900)
The image below shows the list of resolutions with both cards when connected to a Sony FW900 (widescreen) CRT monitor. I'll try to post a list of OS 9 resolutions in a later update. (The GeForce4MX had higher resolutions available in OS 9 than in OS X, at least in the build shipped on the Dual G4 1GHz.)
Both cards showed higher refresh rates for some resolutions in OS 9 than in OS X, where as you can see all refresh rates are 75-85Hz Max. (However 75-85Hz is still the limit at the higher resolutions.) I personally don't see flicker at 75hz and above (but I know some people swear they need higher refresh rates).
About DVD Movie Performance:
Before I get into the results of apps/game tests, I'll comment on DVD playback observations.
In the past I've seen the best image quality with ATI's Radeon cards, compared to other brands. From what I've seen so far the same is true of these two cards. The Radeon 7500 seems (to my eyes) to have slightly better DVD movie image quality. The Radeon 7500 also had noticably lower CPU cycle usage during DVD movie playback - typically 1/2 that of the GeForce4MX. (The Radeon 7500 showed typically 10% to 20% CPU usage peaks - the GeForce4MX showed 30% to 50%. Even moving the DVD window, changing settings, etc. seemed more sluggish with the GeForce4MX. I was surprised at this - since my PowerBook G4 seemed more responsive in some respects. I'm not used to the OS X player acting almost like the OS 9 DVD player.)
I asked ATI about the features of the Radeon chip that assists DVD playback:
In terms of hardware acceleration support, it's the same as we had under OS9 for the Radeon product. We take over the chain once the macroblocks have been parsed and use the hardware to perform the inverse discrete cosine transform (iDCT), motion compensation, subpicture overlay, YUV to RGB conversion and scaling to the screen. As such, our actual driver only takes about 1-2ms (depending on the speed of your CPU) to send all this information to our hardware and execute it concurrently with the software decoder. Adapative and Temporal Deinterlacing support are not in the current shipping OSX, but we've already added the support here and it will be available in an upcoming build.
Under OS9, we've got the exact same hardware acceleration support, but since it is not a fully preemptive multitasking OS, the decoder can get locked out by other actions which results in dropped frames or macroblock corruption.
As we offload so much more from the CPU than the GeForce4MX, and since we can do it concurrently with Apples decoder, it results in substantially lower CPU usage. As you've seen from your experiments, this results in much better user responsiveness and fewer dropped frames even on OSX.
(Arshad of ATI)
I was running 1600x1200/32bit mode, and will try lower resolutions to see if that helps with the GF4MX. From this initial experience, I'd guess the DVD player is not using any special features of the NV17, but perhaps Nvidia (or Apple) can clarify this.
As far as general 2D image quality, without running two systems with identical monitors side-by-side, I really didn't notice any significant differences in the two cards as far as 2D image sharpness. (I don't think either card was razor sharp at 1600x1200 and above modes, but I spend most of my day viewing LCD displays, so that might color my impressions. I actually prefer to run the FW900 at 1600x1024 mode - where it does look at sharp as a Cinema Display.)
Photoshop 5.5 Scrolling Tests
I measured the time it took to scroll the Flowers.psd file (resampled to 300DPI) at the maximum zoom (1600%). I positioned the scroll bars at the max left and top positions, and then timed how long it took for each card to scroll the image horizontally (left to right) and then vertically (down). Display was set to 1600x1200, millions colors. The results are shown in the table below. All times are in seconds, lower numbers are faster.
The GeForce4MX was more than 20% faster at this test. (Again I suspect it's running a higher core clock speed than the Radeon 7500.)
Note: As mentioned on the intro page update, I've had a reader note problems with PShop 5.5 and the GeForce4MX. He later noted he had not applied the summer 2000 PS 5.5 Altivec core/lighting effects/multiprocessor core update available at this Adobe page. He said after updating those files his problems were solved.
AppleWorks 6.2 Scrolling Tests:
I measured the time it took to scroll from the top to the bottom of a 100 page Newsletter document. (Multiple columns with images and text on each page.)
The results are in seconds (lower numbers are faster).
The two cards were literally identical on this test. (Easily within the margin of human error in starting/stopping a stopwatch.)
Quake3 Arena Tests
Quake3 1.31beta 4 used in OS X (r_smp=1 so that both CPUs were used). All tests used the standard game options with "High Quality" settings (only the resolution was changed.) [high geometric detail, texture quality slider one notch down from max, all game options on, Trilinear Filtering, 32bit mode/textures.] No config file tweaks were used. (In fact before the tests I deleted the quake3 config file forcing a rebuild of it to remove the chance that any settings had been modified from prior tests.) Desktop mode was 1600x1200/Millions colors.
All results are in frames-per-second, higher is better.
The same results in a line graph:
Both cards have 5ns DDR video ram and almost identical rates each end of the spectrum - 640x480 and at 1600x1200. (memory bandwidth is clearly the bottleneck for both cards as resolutions rise). Based on this graph, I'd say at 640x480 the CPU is probably the limiting factor - and at the high end the memory bandwidth is the limiting factor for both cards. (They both have the same 5ns DDR ram chips.) Granted both cards provide framerates that are more than adequate at most resolutions.
Despite the "4" designation, the GeForce3 card outperforms the GeForce4MX as noted in this previous GF4MX vs. GeForce3 comparison from a 1GHz Dual G4 owner. ATI's high end retail 8500 AGP Mac edition is expected to ship later this month. (There is no retail Mac Nvidia cards available, but there's speculation there may be non-MX GeForce4's available at BTO in the future.) [Update: On Tuesday, Feb 5th there's a Press Release about the GeForce4 Ti (Titanium) being offered at BTO. The card features support for dual LCDs and has 128MB of DDR video ram. It will be offered separately for $399 later this spring according to the press release.]
Unreal Tournament Tests:
Unreal Tournament as mentioned before in previous reviews is not a really a good video card benchmark. (It seems more CPU bound than video card and delivers relatively low framerates with every video card/system I've tested, especially with the Wicked400 demo.) Unreal Tournament version (436) was used, along with the UTbench and Wicked400 demo tests. RAVE and OpenGL modes were tested with Medium detail, 32-bit mode and low audio settings. Min desired FPS was set to "0" (zero). UT was allocated 150MB of RAM.
All results are in frames-per-second (FPS), higher is better. Since the UT's timedemo stats reports min, max and average framerates, all are listed below. 2D graphics mode was set to 1280x1024, millions colors. RAVE mode delivered slightly better performance for both cards.
Wicked400 Demo Results:
Wicked400 is a recorded demo with intense action, frequent weapons firing, multiple players in a small room, etc. and is used to show perhaps worst case framerates during actual play.
The next page of this review has benchmark test results with the two cards (G4Timedemo, Cinebench 2000, RaveBench, Walker 1.2, Let1kWindowsBloom)