How I Survived (and Loved) Rocket Surgery

How does one survive Rocket Surgery? Is it something like Brain Surgery, or is it more Rocket Science?

The short answer is “Neither of the Above”. If you are not familiar with the term, then let me quickly explain its origins: it is term used by Steven Krug, as part of the title for his book “Rocket Surgery Made Easy“, in which he did not write about any form of surgery, nor any rocketry. Instead, the book used Rocket Surgery as a metaphor for the mythically expensive, complex usability testing process, which Krug has broken down and reconstituted as simple, cheap yet incredibly effective user testing methodology.

His approach emphasizes short, straightforward tests that can be done with a wide range of testers that allow product designer and user experience designer to get quick feedback and fresh ideas to improve a software/website incrementally.

This week, I ran the first usability test myself, with our team intrepid Market Research Intern, Rene Miller. The application I wanted to test is an internal analytics tool used by our team to capture and analyze complex set of data. It is under constant use by our team and thus we are always improving it and making it more useful and intuitive to us. However, I wanted to follow through with the Rocket Surgery methodology and give it a try with Rene, who hireto was not aware of the software and how it worked.

The setup of the test was simple:

– I reused the templates provided in the book, printed the checklists, the introduction templates etc. I wrote a test script with 9 tasks which vary in length and complexity, but is nothing mind boggling or impossible to do. I thought the test would take about 1 hr, but it ended up taking a total of 2.5 hrs to go through.

– I set Rene up in the testing terminal, which is essentially my Mac Book Airlaptop, connected via Thunderbolt cable to a 27in Mac monitor. Below is a photo of the actual setup

– Before the test itself, Rene was given some time to watch some instructions video for the application to familiarize himself with the tools, but he was not given access to the software itself

– To record the session, I installed a trial version of Silverback. I had previously created a small video with this tool and thorugh that it was a great screen recorder for Usability testing purposes

– I went through the checklists again to make sure that I did not miss anything important, and then asked Rene to start. The test was on!

The Testing Process

One thing I cannot emphasize enough is that time really flies in these tests. Because of the interactive nature of the format, the level of details I was asking from Rene, and the free-flowing nature of some of the tasks, we ended up having to split the session into 2 session, one about 1 hr long and the other 1.5 hr long, to accomplish all the tasks in the script.

I was nervous about the ongoing, spoken feedback as the tester carried out the tasks, because having been so close to the software itself, I was afraid I would take some of the feedback too personally and felt discouraged with the number of issues that would be uncovered (which I was sure would be plenty).

However, it was actually great! I got to see how someone brand new experiments with the software and accomplishes cool things, and I did not feel too hurt about the issues that he raised, because I know that those are important and will help improve the product tremendously.

The Results

So surviving Rocket Surgery is not so hard – it was a great time that actually flew by too quickly. Why I loved it? Because it opened me up to a totally different customer experience and user mindset from my own experience and other team members’ experiences. Rene’s newness to the application allows him to point out usability issues that our team has overlooked before, simply because they were too used to it.

Not only that many usability concerns were uncovered, new ideas for improvement and productivity were created, on the spot, or right after, during the recap of the test. It seems that as long as the participants are intellectually engaged, we will have really valuable feedback and innovative ideas for the product.

I will be writing more about how to turn these insights into concrete product improvements ideas and the challenge of prioritizing those in the backlog.