DavidOverton.com
This site is my way to share my views and general business and IT information with you about Microsoft, IT solutions for ISVs, technologists and businesses, large and small.  
David Overton's Blog » Vista SP1 beta vs XP SP3 beta and performance - what a load of old <insert your favourite derogatory term here>
Vista SP1 beta vs XP SP3 beta and performance - what a load of old <insert your favourite derogatory term here>

I saw the article at ZDNet (Windows XP outshines Vista in benchmarking test - ZDNet UK) and at various other places and decided to comment.  It compares a 1GB XP machine and a 1GB Vista machine and says that Vista is slower - both using beta service packs.  This article links back to the blog of a certain benchmarking company (http://exo-blog.blogspot.com/2007/11/windows-xp-sp3-yields-performance-gains.html) and the result was an article with quotes in it like:

"Vista, both with and without SP1, performed over two times slower than XP with SP3 in the test, taking over 80 seconds to complete the test, compared to the beta SP3-enhanced XP's 35 seconds.

Vista's performance with the service pack increased less than two percent compared to performance without SP1 — much lower than XP's SP3 improvement of 10 percent."

Now this annoyed me twice.  So did they turn off things like the pre-loading of applications, was this a fresh PC or one where the inbuilt tools could optimise themselves?  Did they turn off new features like and anti-spyware and search engine, the graphics etc?  You can't add a bundle of new features and get it all for free - they have to load and boot up time is the most likely time.

"The tests, run on a Dell XPS M1710 test bed with a 2GHz Core 2 Duo CPU and 1GB of RAM, put Microsoft Office 2007 through a set of productivity tasks, including creating a compound document and supporting workbooks and presentation materials."

Now this also annoyed me, even more. a Vista machine with the recommended MINIMUM amount of RAM, but XP with eight times it's minimum recommended.  Microsoft clearly recognised that Vista's new features require more memory, as do Office, so they have one machine that is 5 years skinny on features and another that has 5 years worth of extra features and security built in.  Come on.  What about trying it on a 2GB machine, or one that the tests have been run on a hundred times so the optimisation engines in Vista can get to work.  I've done benchmarking as a job and these runs plain stink of "wanting to get a press release out there and talking about us" rather than any real facts. 

While a Microsoft spokesperson did not comment on the memory issue, the new features or anything else, they did comment on the fact that both service packs are not finished, so conclusions drawn now are a bit early!!

"In response to the test, a Microsoft spokesperson said in a statement that, although the company understood the interest in the service packs, they are "still in development" and will continue to evolve before their release. "It has always been our goal to deliver service packs that meet the full spectrum of customer needs," the spokesperson said."

Right, had to get that off my chest, now, back to the day job.

ttfn

David

Posted Wed, Nov 28 2007 1:01 AM by David Overton
Filed under: , , ,

Comments

kliquee » Vista SP1 beta vs XP SP3 beta and performance - what a load of old <insert your favourite derogatory term here> wrote kliquee &raquo; Vista SP1 beta vs XP SP3 beta and performance - what a load of old &lt;insert your favourite derogatory term here&gt;
on Tue, Nov 27 2007 11:54 PM

Pingback from  kliquee &raquo; Vista SP1 beta vs XP SP3 beta and performance - what a load of old &lt;insert your favourite derogatory term here&gt;

kreative_96 » Vista SP1 beta vs XP SP3 beta and performance - what a load of old wrote kreative_96 &raquo; Vista SP1 beta vs XP SP3 beta and performance - what a load of old
on Wed, Nov 28 2007 3:37 AM

Pingback from  kreative_96 &raquo; Vista SP1 beta vs XP SP3 beta and performance - what a load of old

kertvista » Vista SP1 beta vs XP SP3 beta and performance - what a load of old wrote kertvista &raquo; Vista SP1 beta vs XP SP3 beta and performance - what a load of old
on Wed, Nov 28 2007 4:35 AM

Pingback from  kertvista &raquo; Vista SP1 beta vs XP SP3 beta and performance - what a load of old

Vlad Mazek - Vladville Blog » Blog Archive » Vista SP1 vs. XP SP3 Performance Stats: Flawed Samples or Market Reality? wrote Vlad Mazek - Vladville Blog &raquo; Blog Archive &raquo; Vista SP1 vs. XP SP3 Performance Stats: Flawed Samples or Market Reality?
on Wed, Nov 28 2007 9:08 PM

Pingback from  Vlad Mazek - Vladville Blog  &raquo; Blog Archive   &raquo; Vista SP1 vs. XP SP3 Performance Stats: Flawed Samples or Market Reality?

Terry wrote re: Vista SP1 beta vs XP SP3 beta and performance - what a load of old <insert your favourite derogatory term here>
on Sat, Dec 1 2007 5:02 PM

David,

Not sure why you have a huff with the results because it is a fact that Vista is considerably slower than XP, Why? well how many default tasks run the background?, all the bells and whistles have to share CPU time, and then there's all the security thats chugging away in the background.

But you know what, I remember my little old P3 450Mhz running W98SE and installing XP and feeling how painfully slow the computer went.

Its called evolving. However, these days its not about how glossy the front end is, it should be about getting the best out of each clock cycle, not just throwing extra cores at the PC to maintain the same speed.

David Overton wrote re: Vista SP1 beta vs XP SP3 beta and performance - what a load of old <insert your favourite derogatory term here>
on Sat, Dec 1 2007 5:43 PM

Terry,

why am I in a huff, because this is sensationalist benchmarking.  Your absolutely right, there is only so many cpu instructions you can fit into a second, so the more function you add, the potentially slower a system is.  That is why things like recommendd and system minimums are provided.  The original benchmark didn't even use the same version of Office in the two tests.

Now, Vista's min reqs are 8 times those of XP, so there is an up front statement that more resources are need, yet XP is given 8 times its min reqs while Vista is given 1 times - I'd hope the results were not good in those scenarios.  You have to think about the h/w that XP was released onto and the h/w that Vista has been released onto.  Of course there is some growing in to happen.

I just hate crap benchmarking (it was a career of mine for nearly 5 years) to be sold as apples to apples comparisons.

ttfn

David

Minimum System Requirements? Just Feed 8 GB (!) of RAM into Vista SP1 and then Watch it Fly, plus Windows XP SP3 | CTF Blog wrote Minimum System Requirements? Just Feed 8 GB (!) of RAM into Vista SP1 and then Watch it Fly, plus Windows XP SP3 | CTF Blog
on Sun, Dec 2 2007 8:18 PM

Pingback from  Minimum System Requirements? Just Feed 8 GB (!) of RAM into Vista SP1 and then Watch it Fly, plus Windows XP SP3 | CTF Blog

Tim wrote re: Vista SP1 beta vs XP SP3 beta and performance - what a load of old <insert your favourite derogatory term here>
on Tue, Dec 4 2007 4:11 PM

The biggest problem is that for most of us Vista hasn't added any visible functionality, to justify its thirst for resources. The graphic are nicer, and that is about it. Hibernate is differently faulty and more memory makes it even slower. That is about it.

XP gave us USB and Firewire, Vista doesn't seem to offer individuals and small businesses anything new.

David Overton's Blog wrote More on the differences between real world Vista performance and that of a benchmark
on Tue, Dec 11 2007 10:41 AM

Following up on my articles that discuss Vista performance (I still stick by my statement that it is

Craig.. Edge wrote re: Vista SP1 beta vs XP SP3 beta and performance - what a load of old <insert your favourite derogatory term here>
on Mon, Jan 28 2008 2:13 PM

Saying that its unfair to judge cus the min spec is low is stupid. All cars are judged on 0 - 60 regardless of engine size. Should we give them a head start if it a low spec???? Or just judge it on a like for like spec.

David Overton wrote re: Vista SP1 beta vs XP SP3 beta and performance - what a load of old <insert your favourite derogatory term here>
on Mon, Jan 28 2008 9:08 PM

Craig,  they are, but cars with more in / on them as standard have bigger engines.  Would you turn on the Air con to get a 0-60 in a car?  Nope.  Would you expect a heavier safer car to give you the same 0-6 with the same engine as a motor cycle or even a 5 year old Ford 1.1 engine?  Nope.  Now the newer 2GHz CPUs will give an old 3GHz CPU a run for the money, but that is true with some of the new vs old cars too.

The 0-60 ignoring the engine size, power / weight ratio and other "extras" has never been a valuable comparison - have you ever bought a car simply because of the 0-60?

You can absolutely judge a machine and an OS on the performance of a machine, but you can't take a machine that is expected to excel with XP and Expect it excell with Vista - the same was true with Dos --> Win 95, Win 95 --> 2000 and so on - this is normal for the IT industry - the extra features cost you CPU cycles, disk space and reads and memory.  If you don;t want the features, turn them off and then Vista will do better - just don't complain that it is not better than XP with all the extras turned off!!

ttfn

David

Add a Comment

(required)
(optional)
(required)
Remember Me?

(c)David Overton 2006-23