Open access paper, closed data

A new paper in Science Advances by Kicheli-Katz and Regev (2016) is interesting in several ways. It’s a very interesting look at gender bias using eBay auction data.

I’ve used data from eBay myself (Faulkes 2015), and it was hard going. I visited the website roughly daily and pulled auction data into a spreadsheet by hand for each listing. Nothing was automated.

Kicheli-Katz and Rege, however, got data directly from eBay. And they got over one million transactions to analyze. That’s an awesome sample that gives them a lot of statistical power. I was curious to know more, but found this text in several places in the paper.

According to our agreement with eBay, we cannot use, reproduce, or access the data.

I raised my eyebrows at that a bit. All sorts of questions bubbled into my mind. The acknowledgements section provided more detail, however:

eBay has provided written assurance that researchers wishing to replicate the work would be afforded access to the data under the same conditions as the authors.

Okay. That’s more helpful information, and I wish it was in the main body of the text, rather than buried at the end of the acknowledgements. On the Science Advances webpage, it is the second to last thing on the page (the last is the copyright notice).

Clearly, social media companies realize that their data is scientifically valuable and want to use that for their own gain. OKCupid used to write amazing blog posts based on their data that give you a hint of how rich the kinds of questions and answers you can tackle are using large social media datasets. (Although OKCupid also got flak for running unregulated experiments on its users, and rightfully so.)

The eBay disclaimer runs counter to the major trends we are seeing in scientific publication: more transparency, more access to raw data. I am not sure how confident I am with eBay’s promise to share the data with other researchers. From the point of view of reading this paper, eBay is mostly a big black box.

Update, 12 April 2017: I emailed the eBay representative listed as a contact in the paper on 26 February 2016, inquiring about working with eBay’s research lab.

I never got a reply.

References

Kricheli-Katz T, Regev T. 2016. How many cents on the dollar? Women and men in product markets. Science Advances 2(2): e1500599. http://dx.doi.org/10.1126/sciadv.1500599

Faulkes Z. 2015. Marmorkrebs (Procambarus fallax f. virginalis) are the most popular crayfish in the North American pet trade. Knowledge and Management of Aquatic Ecosystems 416: 20. http://dx.doi.org/10.1051/kmae/2015016

Picture from here.