Introduction

When Windows Vista was released in early 2007, a significant amount of attention was given to the new user interface called Aero, that sets out to give Windows a slicker look and feel and promises more fluid operation.

But eye-candy aside, how does the new Aero interface stack up compared to Windows XP and Mac OS X in terms of low-level user interface efficiency? Given the previous lackluster track-record of Windows in terms of User Interface Friction, it seemed important to measure if the new release improved on some of the weak spots of previous releases, such as menu latency and mouse precision.

What is User Interface Friction (UIF)?

UIF is the resistance imposed upon a user-guided process through the operating system and the way the user interface reacts. In most cases, it has nothing to do with functionality: we use the term User Interface Friction to define the difference in fluidity and productivity that can be observed when running the same program or procedure on different computer systems, or when trying to achieve the goal on two similar digital devices.

For a detailed discussion of User Interface Friction and its impact on productivity and efficiency, please refer to the User Interface Friction Research Report published by Pfeiffer Consulting in 2006.

What were we looking for?

These User Interface Friction benchmarks are not intended as a complete, all-encompassing assessment of Windows Vista or of the new Aero user interface: the key goal of these efficiency measures was to establish how Windows Vista impacts some key areas of User Interface Friction observed in previous releases of the Windows operating system.

The benchmarks compared Windows Vista running the new Aero user interface to Windows XP SP2 on one hand, and to Mac OS X 10.4.8 on the other.

The benchmarks covered three distinct aspects of User Interface Friction: Menu latency, common desktop operations, and mouse precision and efficiency, a factor particularly important in tasks that require precise positioning of the mouse. (Please refer to the full benchmark report for more details on this subject).

Key Results

There is no doubt that the Aero user interface provides a much slicker user interface than previous releases of Windows: Vista includes a much more sophisticated rendering engine that can scale both text and images in a much more aesthetic way than, for instance, Windows XP.

In addition, some functionality provided by Windows Vista can make a user more efficient, since the operating system provides better feedback when several documents are open, and can speed up the process of switching between open windows.

But while the graphics of the new user interface are more sophisticated, Windows Vista, and particularly the new Aero user interface design, fared less well in the User Interface Friction benchmarks than Windows XP. Mac OS X came out as the clear overall winner in these benchmarks.

Problem Area 1: Menu Latency

Menu Latency remains one of the key culprits of User Interface Friction on Windows: the slight lag that Windows imposes when displaying menus and submenus may seem minimal, but these delays add up, and they certainly can make working with Windows Vista less fluid than with Windows XP or Mac OS X. On average, Windows Vista was 20% slower than Windows XP in the menu latency benchmarks conducted for the project.

Problem Area 1: Desktop Operations

Benchmarks of common desktop operations (opening folders, deleting elements, etc) also show Windows Vista/Aero at a clear disadvantage over Windows XP. Some of these issues are linked to questionable user interface design decisions. As an example, Windows Vista does not simply display the new window when a folder is opened, but uses a fade-in effect. Amusing at first, the additional lag obviously adds to the perceived (and measurable) User Interface Friction, and can become annoying over time.

Windows Vista with Aero was the slowest in these tests, scoring an average of 2.73 seconds per operation, compared to 2.34 seconds for Windows XP, and 1.50 seconds for Mac OS X. Windows Vista using the Classic user interface scored slightly better, at an average of 2.52 seconds per operation.

Problem Area 3: Mouse precision

The most preoccupying results of these benchmarks concerned mouse precision and efficiency: in these benchmarks (which measure both the time necessary to complete a series of precision tasks and the number of errors, and used the same mouse and mouse-pad for all tests), Windows Vista scored significantly worse than Windows XP. Windows Vista scored a Mouse Precision Error Coefficient of 0.52, compared with 0.40 for Windows XP, and 0.08 (!) for Mac OS X.

How important is UIF?

Next to key technology preoccupations such as security, User Interface Friction may seem a comparatively trivial aspect of an operating system. Yet UIF can result in a significant cumulative productivity loss; taking in account this factor can lead to better understanding and control of the efficiency of equipment and workflows. (See also Macintosh/Windows: Cost and Productivity Analysis, Pfeiffer Consulting 2005-2006)

Implications

As far as Windows Vista was concerned, it was important to conduct the necessary benchmarks to compare the new release with currently installed versions of Windows. In the light of the results, the core recommendation coming out of this research is simple: before deciding on an upgrade to Windows Vista, it seems wise to conduct thorough productivity analysis in a concrete workflow situation to see if User Interface Friction here would negatively impact overall efficiency. This is particularly true in creative industries such as design, publishing and photo retouching, where mouse precision is a major concern. Overall, Mac OS X remains the most fluid environment of the three systems tested.

Likewise, the viability of the Aero user interface in production environments has to be assessed in a concrete situation, to see if the proposed usability features it brings compensate for the slower user interface, or if the Classic user interface would offer higher productivity. (User input is essential here: for some users, user interface friction is barely noticeable, while others are highly susceptible to it.)

Andreas Pfeiffer
(posted February 20, 2007)