Labcast: The Impact of Peer Code Reviews in Product Dev

In our latest episode of Labcast, we talk product development with Jason Cohen, founder of WP Engine. He is also known as the founder and former CEO of SmartBear Software, Inc., a company that provides popular tools for software testing and development.

In this podcast, Jason takes some time to discuss the important role peer code review plays in the product development process, how SmartBear tools can help, and what young companies should consider when building out their product teams.

The Impact of Peer Code Reviews in Product Dev

For more from Jason, visit his A Smart Bear blog site and follow him on Twitter @asmartbear.

PODCAST TRANSCRIPT

Brendan Cournoyer: Hello again, everyone, and welcome to this episode of Labcast. Today we are joined by entrepreneur Jason Cohen, founder of WP Engine, a premium WordPress hosting provider. He’s also a partner with Capital Thought, mentor at Capital Factory, and founder and former CEO of Smart Bear Inc. Jason, how are you doing today?

Jason Cohen: Great, thanks for having me.

Brendan: Thanks for being on. I want to let our audience know this is actually take two. About six weeks ago, I think, you called in for a podcast, everything went great, and then we had what we in the industry call a podcast fail, where we lost the audio, and unfortunately weren’t able to publish that. So, thanks very much for graciously agreeing to come back on and chat with us again.

Jason: Sure.

Brendan: So, the first thing I want to talk about, the company that you were first known for, I guess, was Smart Bear, Inc. Smart Bear was a peer code review tool, and you also wrote a book on peer code review techniques. My first question is, what does this mean, peer code review, and why is it important?

Jason: It just means people reviewing each other’s work, just like authors and editors do, and for the same reasons, to catch problems and train new people on how things work here. If teams are physically distributed, it’s a good way to keep in touch with certain things, or for parts of the code base, or parts of your process that are more risky, where you want to spend extra time on quality. For example¸ if you’re in a feature freeze and just fixing bugs, it’s really important not to introduce new ones at that point. You might want to spend extra time, not throwing new bugs over the transom to QA. Or, for example, a core library that’s used by the whole application, you may want to spend extra time to make sure that stays nice and tight, intelligent, scalable, documented, and everything else, whereas something on the periphery maybe is less important.

It’s a tool. Some people literally review every line of code they ever write. Other people say that’s excessive. There’s usually some amount of code where it is not excessive, or some scenario, like when you have a new hire and you’re trying to bring him up to speed, where it’s exactly the right tool to do whatever you’re trying to do there.

Brendan: So Smart Bear was a software product that was designed to build better software products and best practices, right?

Jason: Well, it was just to facilitate code review. The problem is you can waste a lot of time on it. The traditional Fagan-style review involves printing out code and going to meetings. That obviously takes a lot of time. So, although a tool cannot magically make the code good, and a tool, furthermore, can’t even find bugs or make developers smarter, there are a few ways in which a tool can certainly save time or aggravation.

For example, by integrating the version control, you don’t have to find the differences in the code yourself. You can just say, “I just want to review this change set,” or something like that. As another example, if your company requires reports like, how much time did we spend on code review, what did we find, did we fix them or delay them, and blah, blah, blah, all that kind of stuff, the tool can write that report for you, which is obviously nice. So there are certain mundane aspects of it a tool can help with, and that’s why people use our software.

Brendan: You were a developer, and you’ve been a consultant in the past. Did this idea behind Smart Bear and peer code review in general come from your own experiences, seeing a need for something like this?

Jason: Actually, no, not at all. The company, Smart Bear got started with a product called Code Historian, which was a data mining tool for version control. My big idea was that version control was “Write many, read never.” That is, there’s this big corpus of interesting information in there about which files are changing, when, who’s touching them, and all this stuff. You don’t really look at that history. There’s not a good way to visualize it. There’s not a lot of information you can glean with the normal tools. You pretty much just use it to make sure that two people don’t stomp on a file at the same time, and that’s about it, most of the time that you use version control. I thought there was a bunch of interesting data there.

It turns out that there probably is not that much interesting data there. That wasn’t a great idea. But what happened was people were using these code visualization tools as part of a code review process, and I didn’t understand that at first. So I got these feature requests, which I thought were very strange. Someone would say, “Hey, I’m looking at this set of file diffs and that’s cool. Hey, is there a way for me to export these in some way, like a PDF or something?” I thought that was bizarre, because I was thinking, “No, you’ve got it all wrong. It’s an interactive tool. It’s got data. It’s got graphs. You don’t want to do that.” But I said okay, and it allows you to export it in HTML format.

Then they said, “That’s great. I e-mailed those around, and is there a way for me to write on it and send it back?” Again, keep in mind, I wasn’t thinking about code review, so I was just confused. But, as you can tell, my customers literally pulled me in this direction of, “No, we want to talk about the code. That’s what we’re doing with it. That’s the value in it.” So, I stumbled upon, or you could say discovered – that would be better for my ego, if you said discovered or invented – but the truth is, stumbled upon, and was dragged into this market for peer code review, which at the time didn’t exist.

In other words, there essentially was not any software to facilitate code review. That either means it’s a terrible idea for a company and that’s why no one’s doing it, or it’s a big opportunity because no one’s doing it yet, but people actually want it. It turned out that it was the latter. Once again, you could say that I was prescient, and give me a lot of credit, or you could say I was lucky in that I found it. I’m sure it’s a little of both, actually. Nevertheless, that’s what happened, and so we became the premier, and eventually not the only tool in the code review space.

Brendan: It’s an interesting story, too, because it speaks from a product standpoint. Maybe you come in with one idea and you think, “This is the way we want to go. This is the market we want to target,” and then just by listening to customers and figuring out what their needs are, it puts you in a different direction and tests your assumptions. Sometimes that’s really what you need to do to create a product that’s going to be really impactful.

Jason: I think that happens most of the time. I think most of the time the idea you have going in is not the one that actually really works. By works, I don’t mean getting the first ten customers that you might do on any idea. I mean the first hundred or thousand customers, usually it’s morphed into something else. Again, if you look at histories of various companies, you’ll almost always find that, or at least you’ll find it more often than you don’t.

Brendan: I also want to ask a little bit about, from a software development standpoint, a product team standpoint, what are some of the common mistakes that folks make, where a tool like this or just peer code review techniques can really help.

Jason: Well, it’s difficult to say what mistakes people make when they’re writing software. I would say this. If you take a new hire, and you say “How are we going to get this person up to speed in our code base,” obviously they’re going to read a lot of code, but eventually, of course, the only way you learn is by jumping in and doing it. That’s also how you break stuff, because of course that person doesn’t know all the gotchas and dependencies and whatnot. So one way is to do pair programming for a while, which I think is not a bad idea. But ultimately you want to release them on the code base, let them go, and yet keep an eye on what they’re doing, so that they don’t check anything in to any important branch without it being reviewed, and for them to learn and get up to speed.

I guarantee you that if you’re not doing that now, and if you try it, for the next hire or maybe a current one that’s just getting going, if you try code review with them and just feel the speed. I suppose you could measure it in different ways, although it’s hard to measure someone’s learning. But just see what happens, see how fast they transform into a person who can produce pretty good code or do trivial fixes quickly, and that sort of thing, showing that they’re getting up to speed. See if it feels like it goes faster.

I actually feel like that metric, how it feels, isn’t as bad as everyone says. Everyone wants stuff that’s measurable, which is fine, if you can do that. But my feeling is that code review is so impactful and makes such a big difference, that you should be able to feel it. In other words, if you can’t feel it, and you need detailed metrics to see the little change, then I would say don’t do it, because that’s not a big enough change to warrant this pretty significant change in your process, and definitely a significant use of your time. It better be a massive change, to take that much of your time. It’s hard to make the argument that you shouldn’t do it, in that scenario, and if you just try it, you’ll see for yourself.

Brendan: Now to switch gears just a little bit. Here at OpenView, we invest primarily in younger companies at the expansion stage, but we also create a lot of content for early and startup stage businesses. As far as a young company goes, when it comes to building a product team, what are some of the things that you look for when it comes to building that team out in those processes?

Jason: I think it’s been said before, but I think the Joel Spolsky’s “Smart and Gets Things Done” really are the two things to look for in a new team, because every single member of the team has to contribute in a herculean way, as much as two other people might. You might add a third thing, that’s culture. Once you have a team of ten people, the culture is set, and hopefully that’s because you’ve hired for culture, and in fact fired for culture. But, as they say, even if you’re not controlling the culture, you still have one. That’s of course why you want to control for it. When you have a small team, just one or two or three people, the culture is not set, so you want to know what your goals are there, and be careful with those first few hires, that they’re going to match it. Even one person out of three who doesn’t fit, it will cause a lot of strife.

Brendan: Now, before we wrap up, I’d also like to ask, in your experience, are there any things that many young companies could do a better job at when it comes to product development? Are there certain strategies that you prefer, that you would recommend to businesses, as far as finding success?

Jason: No, because it depends on the business completely. It depends on your goals with the company. It depends on the team makeup. It depends on a lot of things. I don’t think, if there were a magic bullet way to produce software that other people wanted to buy and make all the correct tradeoffs in terms of which features to do, whether to fix that bug, how to interface with the customer and how to get the word out, if there were magic bullets for that, then it would be easy to do startups. So no, I don’t think that there’s one thing.

Brendan: Jason, thank you very much for taking the time to talk with us today. Before we wrap things up, we’ve talked a lot about Smart Bear and peer code review. I’d also like to give you a chance to tell us a little about some of the things you’re doing now, WP Engine in particular, and how people can find more from you and more information.

Jason: Sure. You can find me on my blog, which is blog.asmartbear.com, and you can find my contact information there too, or asmartbear on Twitter. About 18 months ago I started a company called WordPress Engine, WP Engine. It’s a premium WordPress hosting company. If you’re sick of self-hosting, or you’re about to move off a cheap hosting thing and it’s time to get serious, or maybe your company homepage is WordPress, so you care that the thing is fast, has good uptime, doesn’t get hacked, gets upgraded automatically, and all the kind of stuff you would expect from any managed service, that’s us. We’re kind of like Heroku for WordPress.

Brendan: Very cool. Is that wpengine.com?

Jason: Right.

Brendan: Excellent, cool. Thanks very much again for taking the time, and hopefully we can do this again some time soon.

Jason: Sure, thanks.