Usability Heatmaps, Eyetracking vs Mousetracking
Recently I was recommended taking a look at CrazyEgg which could be a possible candidate for creating heatmaps of user navigation. Rather than to be specific to CrazyEgg, I would like to comment on the general principles and pitfalls when using the methodology that CrazyEgg uses.
Heatmap (from CrazyEgg demo site)
As I understand, it works by injecting a javascript on the page, then tracking mouse movement and clicks. Here are my immediate comments for the concept:
- Sounds like a good idea for “poor mans tests”. You can do tests with low effort, no test lead, and on all your users. It can’t get more quantitative :)
- Tracks mouse movement not eye movement. Results will be skewed compared to “traditional” eyetracking tests
- Results depend on how users use the mouse. Results will be skewed by users that don’t use mouse, users that use a laptop with trackpad or another device for moving the cursor around on screen. More on that later.
- However, if logfiles or your current statistics tool fail to give you the answers, or if traditional eyetracking is not possible, the CrazyEgg method could give you some valuable insight.
- Also, there could be more niche-like usage scenarios: For instance, are there any elements in the user interface that users think are clickable but are not?
Mouse vs. trackpad
In my opinion, there is a difference in the results collected by mouse-tracking because it’s skewed by the mouse (or “pointing device” to put it in academic terms).
Some pointing devices are simple and natural to use and others are not. For instance, I move my mouse around with the hand, sometimes I “read” or “look for information” with my mouse.
When I use my trackpad on the laptop, however, it’s more when I really need to: When I’ve found where to click (and it’s too far away for using the TAB key), I use my index finger to move the cursor. A trackpad gives me more tensions and strain in the underarm and it’s harder and more time-consuming to target the right spot.
Your opinion
I’d love to hear your comments on this. What are your opinions? Any testing data to back it up? Any other considerations to put into the equation?
Technorati Tags: eyetracking, usability, heatmap, heatmaps, crazyegg, quantitative, testing, mouse, tracking, eye tracking, mouse tracking, mouse, trackpad
December 21st, 2006 at 11:31 (GMT-1)
By the way, Thanks to Christian Dalager for bringing this up and asking for comments.
December 21st, 2006 at 12:27 (GMT-1)
One consideration is the power of peripheral vision – something that not even eye-tracking accounts for. Peripheral vision is very important since it is what your brain uses to command you eye to focus on something – and subsequently click on.
It reacts to similar shapes, odd colors/shadows/objects, anything that stands apart and movement. Designing for peripheral vision is too me more important than designing for eye-tracking or mouse tracking.
I got a research paper on it somewhere, I will see if I can find it again and post a link.
December 21st, 2006 at 12:36 (GMT-1)
Thomas, I think that eye-tracking is still a poor replacement for brain-tracking :)
We could do a much better job if only we had the tools (or abilities) to tap into the user’s brain. Then we could say goodbye to think-aloud-tests, eyetracking tests, considerations regarding peripheral vision, etc.
Well, as the prediction season is about to start, let me just “predict” that it will probably not happen in 2007 either.
In that perspective, I guess peripheral vision is still an important factor in the equation, especially because — as you say — not even eyetracking accounts for that.
December 21st, 2006 at 13:47 (GMT-1)
Glad you brought this up; I’ve been wondering about it the past couple of days.
In your opinion, what can this tool do that log files or tracking cannot? As I see it, it’s an opportunity to present results in a more visual way (to clients). But is there something I’m missing, is there more to it? (apart from the issue of users clicking on unclickable elements, which actually is a good point).
December 21st, 2006 at 13:49 (GMT-1)
But, do not forget future time tracking – than we will know what people will think before they think it, and use their peripheral vision, eye-tracking and mouse tracking :)
(sorry, this discussion is getting side-tracked).
I do not, as such, think that CrazyEgg is bad. It provides a good illustration of what decisions people make. Eye-tracking does not provide this – it show you what people noticed, but not necessarily what they think about it. They might look at a specific button, but you do not know if they decided to click on it.
The optimal solution would be to use both. Eye-tracking for recording interest and top choices, and CrazyEgg to record the decisions made afterwards.
December 21st, 2006 at 13:57 (GMT-1)
BTW: I do agree that CrazyEgg is a very poor replacement for Eye-tracking
December 21st, 2006 at 14:11 (GMT-1)
Trine, You can also see target areas (or lack hereof) – for instance if people continually click just outside a hotspot (because they think it is bigger than it really is).
It is useful for systems that does not load a new page – mostly in web applications and all these new AJAX applications.
For visual presentations – flash movies/films – it can provide navigation clues.
And, if your site has more than one way to get to another page, you can see which path they take.
December 21st, 2006 at 18:11 (GMT-1)
There seems to be a bug in the reports for Firefox on OS X.
Here’s two screenshots from a short tracking test for my weblog:
In safari
In firefox.
The safari-variant is also what you see on windows + firefox.
December 21st, 2006 at 22:37 (GMT-1)
I think this kind of tools are useful for analyzing another kind of data: what are your users actually clicking on. And what are they not clicking. Traditional server logs can show you what pages are being requested and how are they being reached: by clicking a link, by referrer, etc. But if you have two links to a page on the homepage, for instance, you can’t possibly know which one was clicked. Unless you place a URL parameter, like a unique ID. This is where click tracking might be useful. You can see what was clicked, and how frequently. As I see it, this is not about competing with eye tracking techniques. This is not a technique for observing user focus, or attention, is for one simple thing: what is perceived as click-able (and what is not ) and what is being actually clicked on.
I had previously written a post about this matter, but it is in Spanish: Registro de Clics y Comportamiento de los Usuarios.
December 22nd, 2006 at 22:11 (GMT-1)
We had originally wanted to do eye-tracking in addition to click-tracking.
We looked into it, and decided that the potential security risks outweighed all of the potential benefits of secretly video recording all of your web visitors.
The fact that javascript has no window.eyePosX and eyePosY properties was only a minor inconvinience ;)
(Actually, IE7 probably _does_ have those values burried in there somewhere)
January 10th, 2007 at 16:25 (GMT-1)
[…] Jan CrazyEgg Heat Maps January 10th, 2007 under Usability. Jesper has an interesting post describing some ofthe pros and cons of using CrazyEgg to generate heat maps (a color coded map of where the clicks are happening) on your site. Results depend on how users use the mouse. Results will be skewed by users that don’t use mouse, users that use a laptop with trackpad or another device for moving the cursor around on screen. […]
March 20th, 2008 at 15:47 (GMT-1)
[…] run out of time now so I have to leave you off with an interesting article about web heatmaps and invite you to try some of these heatmaps and see if they provide any useful […]
April 12th, 2009 at 18:46 (GMT-1)
[…] อ่านรายละเอียดเพิ่มเติมที่นี่) […]
May 20th, 2009 at 10:13 (GMT-1)
[…] According to Jesper Rønn-Jensen, there is a difference in the results collected by mouse-tracking because it’s skewed by the mouse (or “pointing device” to put it in academic terms). You can read more about at: http://justaddwater.dk/2006/12/21/usability-heatmaps-eyetracking-vs-mousetracking/ […]