A couple of weeks ago the Washington Post ran an article about gun violence in which Abigail Vegteran, an assistant professor of political science at Berry College, and Alexandra Middlewood, an assistant professor of political science at Wichita State University, discuss a recent paper they published in Social Science Quarterly.
There was something fishy about their paper.
As per PSR, the authors take eight separate polls on gun violence from 1999 to 2018. They predict whether or not respondents endorse this phrase: “Government and society can take action that will be effective in preventing shootings" (i.e., a dichotomous dependent variable) using respondent age as their main independent variable.
For the earliest four polls (1999, 2000, 2001, 2011) they use OLS and they find no significant effect of age. For the latest four polls (2012, 2015, 2017, 2018) they use logistic regression and they do find a significant effect of age.
From this, they conclude that young people make up a "massacre generation" that reacts differently to shootings than older people.
The problem, of course, is that there is no reason for them to use one type of regression in the first half of the surveys, and a different type of regression for the second half. Nowhere in the paper do they justify this methodological choice. Hmmmmmm.
So I emailed the authors, and after a few emails back and forth, we worked it out. They say:
After talking to SSQ and the owners of the data, we’ve learned that as long as we only upload the variables we used in our analysis, we are in accordance with SSQ policy and the policy of the proprietors of the data. All eight datasets and our code can be found on Dataverse under Replication Data for: Massacre Generation: Young People and Attitudes About Mass Shooting Prevention.
Okay, great, all their code and data is now uploaded here.
I would like to reiterate that after running the models once more just be sure, there was no statistical difference between OLS and logistic regression. That being said, we appreciate your desire to check and welcome your replication process. Moreover, we understand why not using the same technique was concerning. As a result, we’ve contacted SSQ and requested that they issue a correction when the article goes to print explaining that we made a mistake in not using the same technique for both set of models. We’ve also requested that they include tables displaying our results using both methods for all eight models in an online appendix in order to alleviate this concern that others may share. We sincerely thank you for bringing this issue to our attention! We welcome any opportunity to clarify our approach and ensure transparency.
Okay, great, they acknowledge that “we understand why not using the same technique was concerning” and have issued a correction in the SSQ.
This is not a major correction, since it wasn’t a major problem with the paper. Everything is now cleared up.
See how easy that was? They had nothing to hide, so they answered in a transparent and helpful manner. Contrast this with, say, the AJPS who refuse to investigate fabricated data in their journal and instead choose to send me legal threats for investigating:
This is outstanding. I can't tell you how refreshing it is to deal with honest and direct scholars for once, who provide simple and sufficient answers. I can't tell you how frustrating it is to deal with a constant stream of researchers who hide their data and lie and scheme about their code. It has made me very jaded.
I will now replicate their entire paper, but the code is now public and the correction has already been issued, so it should go smoothly.
Ryan Enos Has Not Tweeted In Twelve Days
All Ryan Enos had to do was explain his data and code in a straightforward and honest manner like Vegteran and Middleton did. But instead of answering any questions about his data he has instead dragged this on for several months, obfuscated, lied, and whined.
Now he may (?) have brought down the President of Harvard.
Let’s go over the timeline.
June 6, late afternoon: Karlstack publishes an update on the AJPS investigation of the Enos case, revealing that a political scientist verified the claims of the whistleblower report and deems the irregularities concerning enough to warrant an investigation.
June 8, early afternoon: Harvard President Lawrence S. Bacow announces that he will step down, with no reason provided, after an unusually short presidential tenure.
June 9, late morning: The last tweet from Ryan Enos before 12 days (and counting) of extremely atypical Twitter silence.
Enos is addicted to Twitter.
This twitter break is unprecedented.
Coincidences?
Perhaps not…. stay tuned.
UPDATE:
Good on them for dealing with the issue like adults! And good on you for your public approbation. It's often easy to slip into only scolding bad behavior, but equally important to praise the good. So good on you too :)
We’ve always had guns in America.. they aren’t new.
We didn’t have mass shootings until recently.