Product

Avoiding User Research Pitfalls

November 22, 2010

Trip Here

I’ve always been a big advocate for incorporating user research into the product management process.

Why is that? As I wrote in an earlier blog post on market research tools, the rise of remote user testing and complex web site tracking tools have paved the way for many expansion stage software companies to implement those tests and acquire a significant amount of user feedback at little cost. What used to be a costly process only performed by large software companies is now something that smaller growth stage businesses can harness.
So why do senior managers not advocate user research to every product manager? Rich user feedback and user experience research can be the panacea for all of the typical issues that plague enterprise software products. Those tools would help companies address lack of user friendliness, long implementation processes, complex configurations, and customization requirements. However, user research has its pitfalls as well, and that is probably the reason why it is not always advocated across the board.

From my experience, I’ve discovered that a few significant errors can lead companies to believe that performing user research is more trouble than its worth. Here are three user research pitfalls that can undermine an product manager’s efforts:

Not Having the Right Goal for User Research

Typically, user research helps uncover the user’s mental model while they interact with the software. It also identifies how the software interface is organized. Conversely, user research does not help uncover intrinsic, persistent issues within the software’s interface. The reason for that is the wide range of users with varying experience levels using the software interface in totally different ways.

That’s particularly true for enterprise software, which is designed for a very specific purpose and set of users. As a result, it typically includes a richer user interface than modern consumer software tools (with the exception of Microsoft Office). Novice users are expected to be confused in that environment, thus negating the discovery power of user research.

Therefore, user research serves a very specific goal that contributes to the product management process and it usually succeeds when it is held to that very narrow objective. Utilizing user research for other goals might not be optimal.

Performing User Research in a Vacuum

With more remote user research tools available, it is very easy to perform do-it-yourself research projects. While that process democratizes the field, it also allows for some very common missteps. If it’s a company’s first time performing user research, it’s likely that a DIY approach will encourage that company to commit to the process before first consulting someone that’s performed it before. Those companies are also likely to fail to put their results in context with other customer research data.

Here’s the main issue with those two problems: User research results often live in a vacuum with no context. In other words, people fail to look for possible reasons that might rationalize the results or dig for other results that would corroborate them. Instead, they simply take the user research results without context and try to craft a solution for them. That’s skipping a huge step and it often leads to well-intended but misplaced action.

Making Decisions Based Purely on User Research

Even the rich insights gleaned from user research need to be tempered with other viewpoints. For example, companies should consider a heuristics evaluation by user experience experts. After all, the data that the users are responding to might not truly reflect the most important or most commonly used product feature.

By prioritizing fixing a usability defect over adding new functionality, companies can actually cause the the product to stagnate unnecessarily. That’s particularly true for more complex software. Due to the limited nature of user research, only a small subset of features can be exposed to the test participants. Their feedback is then invariably confined to those features and any related ones, while more important issues or functionality might be missed.

Use It, Don’t Abuse It

Part of the user research process involves asking the right questions, too. As Jared M. Spool asks on his blog at User Interface Engineering, what good is user research if the results you get only create more confusion or unnecessary work for the company? Spool offers up three questions companies should avoid asking during their user research efforts, explaining why they can be troublesome.

It’s not difficult to avoid those pitfalls. Companies simply need to possess the right perspective on the modern use and abuse of user research. If you use it correctly, it can help you better your product and user experience. If you abuse it, user research can do more harm than good.

Chief Business Officer at UserTesting

Tien Anh joined UserTesting in 2015 after extensive financial and strategic experiences at OpenView, where he was an investor and advisor to a global portfolio of fast-growing enterprise SaaS companies. Until 2021, he led the Finance, IT, and Business Intelligence team as CFO of UserTesting. He currently leads initiatives for long term growth investments as Chief Business Officer at UserTesting.