Balancing Data with Intuition

Balancing Data with Intuition by Jon Yablonski:

While data is inherently non-subjective, there are a number of ways it can be manipulated. Firstly, the way user tests are constructed or conducted has the potential to skew the results. Secondly, how we interpret data can be skewed by our own subjective bias. Misinterpreted data can quickly lead to us solving problems that don’t exist or creating new ones unknowingly.

This is a great, short read by Jon Yablonski. In it, he breaks down when data is useful in design and illustrates potential pitfalls.

My evolution as a designer was through product management and software development, not formal design schooling. I wasn’t exposed to rigorous data collection and research strategies. Early on I didn’t plan for how data could be used to better inform direction. Because I didn’t plan for it, when I did collect data, it was seldom actionable. That didn’t stop people from trying, of course, but as the article says, the data always seemed too open to interpretation.

Recently there have been a lot of articles trying to explain what designers do, especially user experience designers. For me, design has always been the ability to creatively explore solutions while holding the user’s viewpoint in your head. That sounds too simple. How do you know when you’ve got it right? Well, you don’t really. You are always making an educated guess.

This is because “why” is hard to measure. “What” can be measured, and measured at scale. Recording click-rates for 100 users is just as easy as 10. The more results, the clearer any possible patterns are to see. Scale helps “what” become visible.

“Why” is not only hard, but trying to answer “why” at scale is almost impossible because every user adds effort. It takes many more resources to evaluate “why” for 100 users than it does 10. Instead, you have to rely on your intuition.

If you’re not a designer, that is not a satisfying answer. Intuition doesn’t seem clinical enough. Too much possibility for bias. But, designers know there is bias and we are prepared for it. We look for it and challenge each other over it. Evangelists of data collection over intuition often don’t realize the biases introduced just by the collection methods used.

This is not to say that data has no place. Data is just another tool, a piece in the design process. Used intentionally, data helps validate assumptions. Data should be used to guide decisions, not make them. Like Yablonski says, the best approach is to balance data and intuition.