I wrote a blog in 2016 about text vs graphics for presenting information, where I tried to summarise the issues and evidence.   Recently, a colleague asked me what I “really think” about this issue; ie, what do I believe is the case, even if I dont have solid evidence to back up my beliefs.  I’ll try to set down some thoughts in my blog, starting off with the question of how effective information graphics are for people who are not domain experts.   I focus on decision support, where the goal is to help people make good decisions.

Of course, graphics (and text) are also often used for other purposes, such as persuasion.  For example, when a financial advisor tried to convince me to buy a stock fund (unit trust) which gave him a high commission, he showed me a performance graph which showed the fund doing very well in the past.  This graph was pretty dubious from a decision-support perspective because past performance does not say much about future performance, and also because the advisor carefully choose the time period (you can make any stock fund look good by choosing the right start and end date).  But the advisor was not using the graph to help me make a good decision, he was using it to encourage me to buy a fund which gave him a large kickback (excuse me, commission).  I am not going to look here at persuasion, my focus is on genuine decision support, where the the goal is to help the user make the best decision.

“Man in the Street” struggles with even simple visualisations

As mentioned in my previous post, there is good evidence that experts are better at interpreting and utilising data visualisations than non-experts.  But I suspect this effect is stronger than many people think: an average person will struggle to understand even a very simple graph, and a novice professional (eg, doctor or engineer) will struggle if the information is not clear-cut and visually prominent.

Looking first at the average person or “man in the street”, it is unfortunately true that many people have very poor numeracy skills.  For example, in the US and UK, a quarter of adults do not have “OECD Level 2” numeracy, which means they cannot compare information presented in a simple table.  More than 5% do not even have Level 1 numeracy, which means they cannot identify the most popular holiday destination on a simple bar chart.

So in other words, if we want to effectively present information to 90% (or even 80%) of the general population, we cant go much beyond simple bar charts.   Even very simple line graphs (such as item 3 in the example OECD numeracy questions)  will not be reliably understood by the general population.    Which is presumably why financial advisors show line graphs when they try to sell stocks; the graph looks impressive and factual, but many people wont be able to understand it, which means they cannot ask awkward questions.

In a comment on my earlier blog, Verena Rieser cited a study she had done which showed that people struggled to make effective use of even very simple weather graphics.   I think this is due to the same problem; the general population can only make effective use of extremely simple graphics and information visualisations.

Domain Novices Struggle with Messy Data

How well do novice/junior professionals (eg, new doctors or engineers) interpret graphs. especially if they are stressed or under time pressure (which is common)?  Certainly we expect them to do worse than experienced professionals, and we saw exactly this in the Babytalk project.  But how much worse?

I am not aware of relevant studies in this area, so I am definitely speculating here.  But my belief, which is based on working with junior professionals in a number of contexts, is that junior professionals struggle to deal with noise.  If they are presented with visualisations of clean data, they’re reasonably good at figuring out what is going on (although not as good as experienced professionals).  However, if they are presented with visualisations of noisy data, they struggle to differentiate between real signal and noise, especially if they are under pressure.  An experienced professional is very good at identifying and ignoring noise; a junior professional is not.

Also, in my experience junior professionals find it harder to draw insights from data.  An experienced professional will look at a visualisation and explain what is happening inside the person or machine being monitored; a novice will describe the data, but will find it much more difficult to use the data to understand what is happening inside the system being monitored.

I also suspect that some novice/junior professionals are less likely to adjust a visualisation (change the data channels being displayed, change time scale, etc) than experienced professionals.  For such people, it is important that the initial visualisation communicates the key information in a way which is visually perceivable (correct scale, amongst other things).

Conclusion

Domain experts with years of experience are very good at utilising information visualisations for decision support; they can identify key patterns and use this to infer what is happening inside the system being monitored.  However novice or junior professionals are much less proficient at using visualisations, and often would benefit from seeing information presented in others as well, such as an NLG text summaries.  And many ordinary people struggle to make sense of anything more complicated than a simple bar chart; such people definitely would benefit from having information presented in words as well as in pictures.