"Mike, I decided to upgrade my 2007 MacPro (2,1) Dual Quad Xeon 3.0GHz. I was really on the fence and was considering a new 27" iMac, but the upgrades I put in ended up costing less than a third of a new iMac.
I added an OCZ Agility 120GB solid state drive in the second optical bay. Had to run a new SATA cable from one of the two spare SATA ports on the motherboard, that was challenging because the fan assembly was in the way and not easily removable. I installed a clean copy of Lion on the SSD, the system now boots to the login screen in less than ten seconds. I really, really want a SATA III 6Gbps card that's OS X bootable.
(Mac compatible 6G SATA cards exist (NewerTech, Sonnet, Firmtek) but not bootable w/OS X. (Some are w/Windows.) If anyone knows of a 6G PCIe card bootable w/OS X, send a note.)
I also added 8GB ram (two 4GB sticks) taking me to a total of 12GB and put in a Radeon 5770 graphics card upgrade kit I bought from Apple. I am having issues with the graphics card, dark colors "flicker", bright blue lines shoot horizontally across the screen often, oh, and occasionally the screen will flash and then I will have something that looks like a layer of snow over the desktop. I have read several forum posts of others with the similar issues and Apple's response is this card is not supported in the model of MacPro. (Since their release, 5870 and 5770 card reports in older Mac pros (w/later drivers), although apple does not support it as noted on their product pages. This apple doc notes "You may notice display anomalies or issues playing back video or other HDCP-encoded content when you use an ATI Radeon HD 5770 video card in Mac Pro computers released before the Mac Pro (Mid 2010).") Disappointing, but I'm not giving up on finding a solution. Just FYI, the artifacts are much less frequent in Lion than Snow Leopard.
Adding all the upgrades and installing Lion was a bit more challenging than I anticipated. I have four 2TB drives striped as one 8TB volume (Apple SW RAID) in the machine, Lion didn't like that for some reason. Lion also will not migrate any account that's locked with File Vault. So I had a lot of shuffling of data to do before I could break the RAID apart, turn off File Vault, then migrate the accounts. I had to do the account migration manually since the SSD was too small. I created accounts with the same account info, assigned the accounts to one of the 2TB volumes and then used Carbon Copy Cloner to move the data. Took a long time....
Notes on eSATA Card w/Lion:
One last Lion note. I had a SATA card with two external drives attached for time machine backups, yeah Lion wouldn't load the driver for the SATA card, said it was incompatible. I remember posts on Xlr8yourmac discussing the issue, so I followed the link in the posting and reinstalled the driver. The external drives immediately mounted on the desktop with no issues once the driver was installed, with no reboot required.
(I asked Jason to confirm but suspect from the comments his eSATA card was Sil3132 based, as 10.6 and 10.7 installer removes those drivers and a reinstall was the first tip to try. For those that missed the past notes on this, here's a link to the SI site 3132 drivers page. I assume he installed the last 10.6.x drivers there, which have 64bit support.-Mike)
The eSATA card was 3132. I installed the 7/14/2010 Raid (v184.108.40.206) driver. (I have 2 750gb drives in the external enclosure striped with the apple soft raid.)
I have not done any benchmarks, but throughput seemed little more than double what I previously was getting with that card/drive combo. I move 6TB of data with Carbon Copy Cloner and watched the time and amount of data copied, no real hard benchmark. It's possible I had an incorrect or outdated driver in Snow Leopard, not sure, but the SATA performance with the card under Lion is significantly faster.