The positive buzz continues for Apple as a new study by m:metrics shows that iPhone users are significantly more likely to use every type of media on their phones when compared to other “smart phone” users. This includes watching mobile TV/video, accessing social networking sites (e.g., Facebook), and performing a search or viewing news/other Web content. And, iPhone users are more than 3 times as likely to use their phone to listen to music when compared to other smart phone users.

While this should generally be no surprise to anyone on the planet that has read about, heard about, seen, or used Apple’s “must have” appliance, it does raise some interesting questions that aren’t really answered by the press release (competitors take note).
The author of the press release casually passes over the contribution of “the attributes of the device itself” and emphasizes AT&T’s unlimited data plan as a key contributer to the increased use of media content …
“While the demographics of iPhone users are very similar to all smartphone owners, the iPhone is outpacing other smartphones in driving mobile content consumption by a significant margin,” said Donovan. “In addition to the attributes of the device itself, another important factor to consider is the fact that all iPhones on AT&T are attached to an unlimited data plan. Our data shows that once the fear of surprise data charges is eliminated, mobile content consumption increases dramatically, regardless of device.”
While it makes sense that the unlimited data plan could contribute to increased use (who passes up the all you can eat buffet?), the recent New York Times article nor the press release really mention much about the interface or user experience other than the fact that widgets for Google Maps and YouTube seem to drive usage of those apps.
It would have been great to know more about the “why’s,” for example:
1) How much of iPhone’s media usage can be attributed to the user-interface itself? Not just the content widgets but the unique aspects of iPhone’s touch screen, screen size, etc.?
2) Conversely, is the lower incidence of usage by other smart phone users related to problems/issues with their phones’ interface?
3) How much of this usage is driven by self described “early adopters,” i.e., those who are predisposed to using rich media in the first place?
4) Is the iPhone driving usage or the other way around: what percentage of users are “first timers” – i.e., their iPhone usage is the first time they’ve used some of these media applications on a smart phone?
5) How much is attributable to users accessing content via WiFi vs. AT&T’s network (many users have complained about the slowness of AT&T’s network)?
6) And, importantly, do these numbers represent claimed usage or as m:metrics promises in its About section, “actual mobile content consumption?” (According to The New York Times the results are from a survey of more than 10,000 adults. Did the researchers sit over the shoulders of all 10,000 people to make sure they were using the applications claimed?) How much of claimed usage can be attributable to overstated usage? Look, if I shelled out $500 for a phone and someone asked me if I was using all the cool stuff that came with it I might be embarrased to admit that besides the music, I haven’t really gotten around to using all that other cool stuff.
Claimed usage or not, these are impressive numbers and represent a cold slap to Apple’s competitors. Now if they can just open up the phone to other wireless providers …
iPhone videos:
You can always listen to “Blender” (the Collective Soul album) but will the iPhone blend?:
http://youtube.com/watch?v=qg1ckCkm8YI
Fun with iPhone’s SDK (just released in Feb):
http://youtube.com/watch?v=u5xA8-XvjNk
Can you use it as a hot plate too?
http://youtube.com/watch?v=jihVOLrZ4UA
Kindle Fire Under Fire: Really Jakob?
December 12, 2011Kindle Fire Under Fire or Biased Attack?
Alas, another one of our heroes has fallen in our estimation.
Coming off of two weeks of user experience interviews in which we tested a client’s software on touch screen devices, I read a recent article in The New York Times about the Kindle Fire with keen interest. In his Personal Tech post, the author, David Streitfeld, builds his case that the Kindle Fire is doomed based on an analysis of Amazon user reviews posted on the retailer’s website, and a study conducted by usability guru Jakob Nielsen. Nielsen, a highly regarded expert on UX (and considered by many to be the founding father of usability), tested the Fire and concludes…
“I feel the Fire is going to be a failure. I can’t recommend buying it.”
Strong stuff. I was imagining a rigorous test of the device among several current and new users, perhaps a take-home evaluation so users could try it out and see what it could do (as we did in our client’s unrelated product test). Although Nielsen doesn’t specify the time users spent with the device (you can read his article here: Kindle Fire Usability Findings), he does specify the sample size. Guess how many? 100, 20, 10?
Try n=4.
I know. I had the same reaction. Can you call a complex, deep-featured device like the Kindle Fire a “failure” after only 4 in-person interviews in a UX lab? I agree with Nielsen that “qualitative studies generate deeper insights,” but with 4 people? And to be that definitive in your conclusions you would think those four people lived with the device for at least two weeks to see just how much of a failure it was for them. Yet it is clear that this was a lab test performed to “collect video clips.” Nielsen goes on to say in his own article …
“Our studies of Kindle Fire weren’t intended to advise consumers on whether to buy a Fire device. Our goal was to discover design guidelines for companies that are building websites, apps, or content that their customers might access on a Fire.”
If this is true, why the definitive “failure” quote that made its way into an article read by millions who might be considering buying the Kindle Fire? I am especially peeved by this, after having just conducted several interviews with users testing a product that had a much narrower feature/functionality footprint. Nielsen should be taken to task for making such broad conclusions based on so little data.
I was also disappointed in The New York Times for taking short cuts when making such broad claims. Building most of your article around online user comments, which tend to trend negative, a lab study among 4 users and a few off-the-cuff remarks from one “expert” is just sloppy journalism, especially when you get to pick and choose the comments that help build your case.
Full disclosure: I just bought the Kindle Fire last week and although there are serious problems with its user experience, I am finding it quite enjoyable, especially once I discovered how to open the device and sideload some additional Android apps. I now have the Nook and Kindle e-reader running on it. In order to get non App-store apps on the iPad 2 you’d have to jail break it. Amazon makes opening up the Fire easy with a single menu toggle.
Whether you agree with Nielsen’s assessment or not, the Times article should be a wake-up call to any of my colleagues who feel a desire to give comments to the press based on such light “qualitative” data. The paper’s standards obviously aren’t quite up to the ones we strive for in our own studies, I’m afraid. And, the piece caught one of our most esteemed researchers looking rather flat footed. The last quote in the NYT article is from Nielsen again:
“If I were given to conspiracy theories, I’d say that Amazon deliberately designed a poor Web browsing user experience to keep Fire users from shopping on competing sites.”
Hmm. Not only is this questionable since Amazon allows you to “open” the device up to competing apps and competing browsers, but given Nielsen’s own admission that “Apple is one of the companies that sent the most attendees” to his conferences, you might conclude the same re: his review of the Kindle Fire.
UPDATE: 12/13/11
After some harsh responses from critics Jakob Nielsen has posted a rebuttal. His insights into the user experience are sound but his argument re: the small sample is still unconvincing. Check it out yourself: Rebuttal to Critics