Introducing:

User Experience Benchmarks for Tablets and Smartphones

Major Points

User experience of tablets and smartphones is one of the most important aspects of the overall perceived value of the device.

Key aspects of the overall user experience are hardware user experience, cognitive load, user experience friction and app store sophistication.

Pfeiffer Consulting has developed methodologies to quantify key parts of the user experience.

The benchmarking project compared four medium-sized tablets: the Amazon Kindle Fire HD (7-inch), Apple’s iPad mini, the Google  Nexus 7, and the Samsung Galaxy Tab 2 (7-inch).

 


Pfeiffer Consulting


 See also: 

How iOS 7 Stacks Up:
 Smartphone OS User Experience Shootout

7-inch Tablets: What the User Experience Benchmarks Show

 

Download How iOS 7 Stacks Up: Smartphone OS User Experience Shootout  (PDF)

Download 2013 Tablet User Experience Benchmarks  (7-inch Tablets) (PDF)

Download Conceptual Framework For Quantifying Tablet User Experience (PDF)

 

 

User Experience is arguably the most important aspect of a connected digital device such as a tablet or a smartphone. 

Since these devices are very closely meshed with our everyday lives and our social interactions, a device with a bad user experience can be deeply frustrating, even inhibiting. However, discussing—let alone measuring—user experience seems almost impossible; as soon as we try to do so, we run into complex and confusing issues. As a result, the reviews and expert opinions that are meant to help us differentiate between different models and to guide us in the process of selecting the right device for our personal needs often bypass user experience issues and focus on sometimes arcane technical specifications.

Given the importance tablets and smartphones are increasingly taking in our lives, this incapacity of discussing and measuring what turns out to be one of the most important aspect of the overall value proposition is slowly turning into a serious problem.

We need to be able to discuss objectively what value these devices bring the user, or we are going to be increasingly swamped by hundreds of different models we can not really tell apart any more.

Since 2012, Pfeiffer Consulting has embarked on an ongoing, independent research project to establish some objective criteria for comparing, discussing and measuring the user experience of digital devices. And in doing so, it became immediately clear that it was not enough to discuss some aspects of user experience, but that we would need to establish a coherent conceptual framework that would provide a solid foundation for the research, and could grow and expand as we tackle different devices and future developments.

The reports presented here outline the conceptual framework and benchmark methodology, as well as the first set of results from the 7-inch tablet user experience benchmarks. These are just the very first step in this exciting new area of technology analysis. More will come soon...

I hope you enjoy the reports. We really want to get a discussion going on this subject. Please let us know what you think.

 

Andreas Pfeiffer,

April 2013

 

  • Yanbo JIA

    Epic failure IMHO. there are lots of bashing against WP8 and BB10, you guys are simply unaware of the importance of ‘live tiles’ and ‘gesture control’….and just subconsciously using iOS as a golden standard because that’s what you are familiar with… I cannot stop laughing when reading “iOS displays 24 apps on its home screen, while WIndows Phone 8 barely manages 12″… Seriously? Don’t you guys even know the tiles on WP8 can be resized? WP8 can easily display 24 apps on one page of the home screen, if you don’t know, just Bing it. This is just one of those laughable conclusions that you made in the report….

  • Jeff

    I’m quite shocked at how this report is highly biased and on topics that are speculative to most people. I’m also surprised that Andreas and Raphael would even associate their last names with this report. Who is really doing this assessment? The numbers you’ve generated associated with each benchmark seem to be completely meaningless and fabricated for a false result.
    I don’t think your “strategic analysis” was very in-depth on this one. Your analysis is completely irrelevant. And I highly doubt that you guys have ACTUALLY been specialists in this field since 1998.
    This is awful work and you should be ashamed of yourselves. This is all coming from a company who their own web site developer can’t even spell his name correctly on the site he created. And he is on your expert committee? Oh and none of the images work on your “Research” page. Some of the links are broken. Yeah, you guys REALLY know what you’re doing.
    I also noticed Apple is one of your clients. Apparently ass-kissing is your way of keeping your jobs!

  • Jeff

    By the way, have you heard of all the users complaining of motion sickness using iOS 7? How many points does that score on your scale? Or is vomit not considered an accurate benchmark?

  • Dan

    I couldn’t agree more with how lame this is. Let’s compare a generalization of Android with a specific iOS version. What are the comparative phone specs? What version of Android was compared?

    When you say “Samsung”, you’re only referring to maybe, say 100+ different phones…

    This was certainly humorous, and I only read it because a friend of mine who is an Apple minion sent it to me, almost certainly because I’m an Android 4.3 guy lately, and he just upgraded his iPhone 5 to iOS 7. We’re engineers, and got a good pants-wetting laugh from this (well, I dd, at least) LOL

    Talk about subjective, unscientific stupidity… this should be featured on Talk Soup!

  • quarterpipe

    The link in the iOS7 document that says “You can find detailed information about the “Pfieffer Consulting Methodology for User Experience Quantification by following this link” brings the user here.

    Where is the detailed information? What weightings and elements are used in each of your index ratings? Cognitive Load Index for example is 6 times higher in Samsung than in iOS7 but there is no transparency in how you arrived at this number. This means that this information should be disregarded since there is no evidence to back this assertion up.

Back to top