User Experience Research

How iOS 7 stacks up:
Smartphone OS User Experience Shootout

Major Points

User experience of tablets and smartphones is one of the most important aspects of the overall perceived value of the device.

The arrival of iOS 7 will have a profound impact on the market for  connected digital devices

Pfeiffer Consulting has conducted in-depth user experience benchmarks of the 5 major mobile operating systems currently in use: iOS 7, iOS 6, Android, Windows Phone 8 and Blackberry 10 


Pfeiffer Consulting


Download the complete benchmark report (release 1.1):
How iOS 7 Stacks Up: Smartphone OS User Experience Shootout (PDF)

Related Research:
Google vs. Apple vs. Amazon: The 2013 App Store Maturity Shootout (PDF)

(More information on this report is here)

 

Why is it that the arrival of iOS 7 is necessarily a momentous event for the smartphone market? Simple: Unlike any other operating system out there, it will be in the hands of millions or tens of millions of users within a few days after its launch. 

And that will make it a force to be reckoned with.

The question is, of course:
How good is it really?

How iOS 7 stacks up: Smartphone OS User Experience Shootout

Introducing: The 2013 Mobile OS User Experience Benchmarks

Whether we like it or not, smartphones have become a software game. Take any recent top-of-the-line smartphone, and you are likely to get a well-designed, fast, pleasant to use bit of hardware: fluid operation, responsive interaction, fast graphics. The difference of user experience, therefore, stems almost exclusively from the operating system, the user interface design, the application integration, the overall coherence.

This research project compares the five major mobile operating systems in use today: iOS 7, iOS 6, Android, Windows Phone 8, and Blackberry 10, and rates them in terms of user experience. 

We do not look at features, we do not compare cutting-edge options and gadgets, we only look at aspects that have a direct impact on the day-to-day user experience of an average, non-technical user.

The aspects we have surveyed and rated are the following:
cognitive load, efficiency, customization, as  well as user experience friction. Based on the results from these benchmarks we have then established an overall Mobile Operating System User Experience Index presented at the end of this document.

The benchmarks are based on the Pfeiffer Consulting Methodology for User Experience Quantification. You can find detailed information on the subject by following this link.

Key Results

 mobileoshomescreens2013

The combined results of the four different benchmarks and evaluations give iOS 7 a clear advantage in terms of of overall user experience, taking into account the context defined for these benchmarks: 
day-to-day user experience of an average, non-technical user. For details and methodology please refer to the complete report.

Overall Mobile Operating System User Experience Index

  

Individual Benchmark Results - Cognitive Load

Cognitive Load Index of Major Mobile Operating Systems

Cognitive load—the sum of elements you need to get familiar with in order to use a device spontaneously and intuitively—is one of the key aspects of user experience for a non-technical user. For this benchmark we counted the number of apps/widgets as well as other icons and user interface elements a default installation of the operating system contains. Differences between operating systems are significant. For a detailed  comparison of cognitive load elements of mobile operating systems please check the full report.

iOS7 Cognitive Load Benchmarks

 Individual Benchmark Results - Efficiency and Integration

Efficiency Rating of Major Mobile Operating Systems

Smartphone user experience depends in large part on the easy and efficient integration of different key features and services.
For this benchmark we analyzed access to key settings, integration with notifications, multitasking, and camera access, among others. While there is a core set of integration functionality all mobile operating systems share, there are significant and surprising differences between the five major players in the market.

iOS7 Efficiency Benchmarks

Individual Benchmark Results - Customization

Customization Rating of Major Mobile Operating Systems

Consumer-level customization is one of the key user experience aspects of connected digital devices. Nevertheless, the current crop of mobile operating systems differs quite significantly in terms of customization, ranging from the almost dizzying granularity of Android’s options, to the starkness of Windows Phone 8, which allows hardly any user-level customization.

iOS7 Customization Benchmarks

Individual Benchmark Results - User Experience Friction (UXF)

User Experience Friction (UXF) of Major Mobile Operating Systems

UXF is the bad stuff, the aspects of a device that can annoy you in a niggling way, or, in extreme cases, drive you crazy. Basically, UXF occurs whenever a device does not do what you expect it to do - or lacks a key feature that should be available. For this survey we took only the most obvious UXF factors into account, and rated them. For a detailed account of the UXF factors, as well as the rating method, see the full report.

iOS7 User Experience Friction Benchmarks

 

Andreas Pfeiffer,  October 2013