Craigslist Goes to War Against the Open Web

Before I get up on my soapbox, the facts:

Over the last few months, Craigslist has been cracking down on mashups of their data. Because Craigslist itself does not offer an API, these mashups rely on 3^rd party data providers such as 3taps, which uses a web spider to scrape Craigslist postings from the public domain and repackages them as an API. PadMapper, the target of much of Craiglist’s ire, mashes up the 3taps API with geo-location data from the Google Maps API to display Craigslist as a map.

Craigslist, to put it nicely, does not like this practice. They took legal action earlier this year to shut down PadMapper.

As this lawsuit works itself out, Craigslist has also pursued other ways to impede PadMapper. On October 2^nd, Craigslist sent a cease and desist letter to OpenView  portfolio company Mashery, which managed 3tap’s unlicensed Craigslist API. Mashery, not wanting to get in the middle of a messy legal battle, promptly complied and severed their relationship with 3taps.

While it’s not a kill-shot for 3tap or PadMapper, it certainly makes their lives harder and opens the door for Craigslist’s recently launched mapping application to take back some of PadMapper’s traffic. Craigslist will no doubt continue to pursue similar legal channels to impede their unwelcome partners.

Although I’m fully prepared to bash their legal strategy, let me first say that I love Craigslist.

I’ve found multiple apartments, bought and sold more furniture than I can keep track of, and even found a sublettor on Craigslist, who I’ve been friends with ever since. I love the company’s simple UI, focus on functionality and substance over glitz and glamor, and especially its lack of commercialization. In a prophetic interview with UBS tech analyst Ben Schachter in 2006, Craigslist CEO Jim Buckmaster was asked how the company planned to make money on the back of impressive user growth.

“That definitely is not part of the equation,” the ironically named Buckmaster replied, while Wall Street analysts everywhere spat out their coffee.

But despite Craigslist’s public persona as a quasi-non-profit disinterested in making money, its recent litigation against 3taps, PadMapper, and Mashery can’t be viewed as anything but profit-maximizing. Craigslist wants the economic value of its data all to itself, even if that’s at the expense of the technology community, and even — gasp — its users. If, like Buckmaster says, all Craigslist wants to do is help people find apartments, shouldn’t PadMapper get a gold star? Instead, they get a cease and desist order.

Hypocrisy aside, Craigslist, just like every other private company, is entitled to try to make money on their proprietary data if that’s what they want to do. And while there’s some legal debate over who owns the postings that’s over my head, it certainly feels to me like Craiglist should have the right to restrict 3rd party access to their postings. Even so, as a long-term business strategy I think Craigslist’s litigation efforts are misguided for two reasons:

1) It’s incredibly easy to scrape Craigslist. Much easier, in fact, than to litigate against someone scraping Craigslist. That means that even if Craigslist succeeds in shutting down 3taps, you can be sure another dozen websites will be waiting in the wings. The situation is reminiscent of the music industry’s never-ending battle with Filesharing websites. Within about a week of Napster’s bankruptcy, Kazaa had stepped in to fill the vacuum, followed closely by Limewire, Mediafire, etc. In the end, litigation only serves as a temporary fix to a long-term problem. Craigslist would be better served by beefing up their virtually non-existent security than fighting a never-ending, uphill battle against web scraping.

2) They’re missing out on the benefits of open data. Many unabashed capitalist companies are very public with their data, not as a service to the community, but selfishly to keep their company ahead of the curve:

• Realizing that developers outside the four walls of their own organization are talented too, Twitter has vastly benefited from the functionality 3^rd party developers like Hootsuite have added to their platform. Despite drawing heat for the recent restrictions on their API, Twitter remains more transparent than most with their data.
• The more applications use Twitter to power their app, the more Twitter becomes the internet-wide authority on social media. Paradoxically, opening up their data has strengthened their competitive moat.

So even if Craigslist finds itself on the right side of the law in their battle with PadMapper and 3taps, their war against the open web is destined to fail. Like skinny dipping in a receding tide, the internet has an unmistakable tendency to expose more information than its users/owners intend. Companies that have resisted this inevitability have — like the music industry — endured decades of frustration, wasted millions in legal fees, and seen their brands erode. Almost without exception, the companies that have benefited from this error in judgment have been the ones that most embrace the open web.

I’m not saying Craigslist should immediately roll over and play dead while PadMapper makes millions off their data. But they should be looking for ways to work with 3taps and PadMapper, instead of against them. That could mean opening up and charging for a firehose API, like Twitter does, or some sort of revenue-sharing agreement.

Craigslist, we love you. Don’t be the next Tower Records.