Having been quotes by Vlad and others (or was this others?), some more views worth reading here and some Windows Blog stuff here I thought I had better make myself a bit clearer.  1st off, this is MY opinion, not something that is sanctioned or designed by anyone else in Microsoft.  I did benchmarking for 5 years and won plenty of business by it, but I also understand when a test is Apples to Apples and not Apples to Oranges.  If the purpose of Vista was to be the fastest engine for Office 2003 (as was the test), then there is a lot that can be done to the product to make that happen, but that was not the goal, so testing it on that one goal and then saying it fails is a bit poor.  You can turn off the new features that "slow" it down if you want, but then you don't get the feature.  My car accelerates faster with the air-con turned off, but on hot days, being comfortable is more important that the acceleration, so I opt to go slightly slower rather than get there just a bit quicker.

There is more to this though, XP was designed to run on the hardware that was coming, so is Vista.  Fro Vista, it's day is yet to come! You might say "bah" to that, but this was true for XP a year after it was released, so this is not new.  (see here for XP releases that show this).  So here goes on trying to explain my point of view:

What is running on the box

Vista runs more under the hood (and graphically) than XP, so unless you believe that after 6 years bedding in XP is still hugely inefficient, you would expect that Vista would have to do more in any given second.  Yes, it would be great if I got all the extra goodies for free, but thats not happened for a while, so anyone who gets upset needs to have their expectations reset.  I mean, DOS 6.22 flys on todays hardware, as does Windows 95.  Yep, it would get shot in about 5 seconds flat, but it would perform well for the apps available back then when it was built:-)

In the last section I will discuss some of the things that could be done to make it more level to compare against.

Vista also has some performance enhancing options, but these are designed to help in specific scenarios and all take time to bed in, however do you use all the trick the OS has, or do you only use those of the older OS?

• Loading programs (SuperFetch)- optimises the disk layout weekly (if I remember) to speed OS and program loading, plus pre-loads parts of your disk to make program startup quicker
• Finding data (Windows Search)- The search engine will be chugging on the disk and CPU until it has finished.  Any test during this time is falsely loaded, but once it is in use, finding an item in Office 2007 is much faster
• Page file usage (ReadyBoost)- The ability to use USB sticks to improve page file usage can significantly improve Vista's performance.
• Disk optimisation's (ReadyDrive) - Use NV RAM on a disk to improve disk utilisation, performance and power usage

Hardware built for each OS

Hardware that is built for Vista optimisation is only just beginning to arrive, however most hardware manufacturers have spent years honing their Windows XP skills.  Windows XP was designed for hardware that shipped or was planning to ship around 6 years ago.  Some of the Vista design benefits are clearly not being benefited from yet today.  Graphics cards are only just beginning to ship with native Vista support and since the new graphics engine is a key component of Vista, that is an important difference.

Then we have the simple fact that XP was designed to fit into less memory and Vista was designed to run in a world moving to 64-bit.  Getting XP to be great in the 64-bit world is 1,000 times harder than Vista, however the simple truth is that Vista was always designed to need more memory than XP.  For example, to improve the graphics experience when playing back a video ALL video windows are stored in RAM (notice the memory in that DWM.exe process) - this adds an extra large step in the "updating the screen" process, chews memory when you have more windows open and can slow the PC down if you have a lesser graphics card.

So, would you get a machine that was truly designed for Vista (2GB+ RAM, Direct X10 graphics, ReadyBoost, ReadyDrive enabled) and do a benchmark on it using XP and Vista and not expect XP to be significantly disadvantaged?  While Vista works good on today's old hardware, it is designed to perform best on the current and coming levels of innovation, which includes these technologies as examples.

Of, and finally, ensure that the power management of both machines was set to maximum or high performance.

Options to make it more similar

OK David, so you've had your say as to why it is not the same, so what do you think would be a fairer comparison?

1. Turn off the Vista Services that could make a significant difference to performance - that way people get a choice:
1. Windows Search (or wait until it has finished and stopped indexing the main of the system)
2. Backup, Shadow Copy and Bitlocker
3. Aero Graphics
2. Enable / Use new Vista features that increase performance
1. ReadyBoost with a USB thumb drive
2. Direct X10 graphics card
3. ReadyDrive disks
4. Run the benchmark several times and make Vista think enough time had passed to do its own optimisation's to improve itself
3. Use new software that is designed for the newer OS rather than the old - since software will continue to progress, why not try things that make much better use of the newer features, search, 3d graphics and more?
4. I've offered with Vlad to build and test a streamlined version of Vista built on the basis of XP with some functionality turned off, built for speed.  We'll have to see what happens!

This still does not level out the playing field, but it does make a difference.  You can't expect two different OS's to do lots of different things and then measure just one aspect and determine the winner from that one aspect unless you get to tune all the other items that impact a system.  Formula One racing cars go much faster that their road counterparts, but they are built only for performance.  Add safety, efficiency and other aspects and suddenly many people opt for the more traditional cars rather than the ones built purely for speed.

If you feel this needs more explanation, just let me know.

thanks for listening

David