Page 1 of 1

Transparency in APD Research

Posted: Sun Apr 24, 2016 2:58 pm
by rvalell1
As a scholar of American political development I have to express my enormous gratitude for this deliberative process. DART strikes me as a threat, on balance, to APD scholarship -- yet APD scholarship has had a genuinely productive impact on political science and shows every sign of continuing to have such an impact. I worry greatly that DART will scare off talented younger scholars from producing APD work.

Re: Transparency in APD Research

Posted: Tue Apr 26, 2016 11:48 pm
by AlanJacobs
Thanks, Rick Vallely, for putting implications for APD scholarship on the agenda. I'd be interested in hearing more from APD scholars about a.) what specific transparency practices are common, feasible, or of value for the kinds of analysis and evidence typically employed in the study of political development and b.) what the limits or costs of transparency might be when it comes to these forms of analysis and evidence.

Re: Transparency in APD Research

Posted: Mon May 02, 2016 3:17 pm
by Guest
DA-RT seems to be a solution in search of a problem as relates to both American Political Developmental scholarship (in general) and historical based study of the U.S. Presidency and Supreme Court (more specifically). It certainly threatens to drive the wedge deeper between "Americanists" and other scholars utilizing qualitative methods.

I've always supported Gerring's call to add rigor to APD scholarship, however DA-RT seems excessively burdensome and (dare I say it) a bit heavy-handed and thought chilling. As a graduate student, I was attracted to the qualitative method by not only its inherent flexibility but the challenge it creates for individual researchers to 'make' their case without simply resorting to tests of statistical significance, formal models, or DA-RT standards.

There are many good ways to increase the rigor of APD research, instituting DA-RT does not seem to be one of them.

Re: Transparency in APD Research

Posted: Sat May 14, 2016 10:28 am
by TimButhe
Thank you, Rick, for starting this thread focused on APD (American Political Development) scholarship. As we prepare to start moving from the at-large deliberations of stage 1 to stage 2, where deliberations are supposed to continue, primarily, in 12-15 working group so as to allow more differentiated discussions of research transparency in specific research traditions, the Steering Committee would very much welcome the opportunity to hear from more APD scholars--at all levels--especially with an eye toward two sets of questions:

1) What do you consider to be the most important issues for APD research transparency that require separate discussion during the 2nd stage? As you consider this overarching question, you might want to think about: What does research transparency mean for you in APD research? About which elements of your research do you try to be particularly transparent? What kind of transparency do you most value in APD work that you read? What do you see as the most important costs, risks, and other specific concerns that keep you and other APD scholars from being as transparent as you might otherwise wish to be. [The answer may be: None; but the question is worth asking.] Are there particular tools or practices you've found to be particularly useful (in your own work or the APD research of others) to achieve transparency without undue costs or risks?

2) Is APD scholarship as such is a good delimiter for a separate working group? Or would APD scholars consider other, more cross-cutting issues more sensible as working group foci?

Re: Transparency in APD Research

Posted: Fri May 20, 2016 9:17 am
by rvalell1
The questions that Alan and Tim have posed for APD researchers and scholars are extremely important -- and suggest that those of us in APD ought to up our game. For most of us transparency is in the apparatus, in the self-consciousness with which we discuss our evidence, and in the robustness of our causal stories. But we could do more. One scholar of whom I am aware who models the kind of transparency that would put us closer to where qualitative methods and comparativists are now is Daniel Galvin. Another is David Mayhew, who is scrupulous in this regard. Such practices as putting datasets online should become standard in APD. We might also post methodological or bibliographic essays to support publications. Images from archival research could be made available as well, if restrictions do not apply. "Active citation" may not be feasible in all cases but we should try harder.

Many of us -- being book writers -- look forward to "author meets critics" panels, and such panels might incorporate discussions about transparency.

Finally, CLIO, the Politics and History newsletter, might feature pieces about we wrestle with these questions in our work.

Re: Transparency in APD Research

Posted: Tue May 24, 2016 6:14 pm
by shamira
Writing as Clio editor:
I'd be happy to publish pieces on transparency in APD research in the upcoming issue. If anyone here is interested in writing something, you can touch base with me by email or in the Clio group on Facebook.

Writing as me:
I'd add the "Archival Research and Coding" appendix in Catherine Paden's Civil Rights Advocacy on Behalf of the Poor to the list of transparency models in APD. I think we're generally very good about documenting sources for specific claims with extensive footnote references to individual documents, but it's rare to see something like Paden's explicit discussion of how she approached document collection and analysis. Thinking through what it would entail to make such discussions par for the course in APD publications would probably be an important conversation. I'd also be interested in having archivists weigh in on having researchers make even unrestricted document images available, especially if "available" means essentially published, perhaps behind a subscription paywall, on a proprietary website.

Re: Transparency in APD Research

Posted: Wed May 25, 2016 12:18 pm
by robmickey
One important research transparency issue for APD scholarship is how to strike the balance between the benefits of better documentation for empirical claims through some form of hyperlinked citations and the risk of discouraging scholarship for which comprehensively providing such documentation is prohibitively costly.

If a comprehensive version of DA-RTish transparency requirements were the norm when I was starting my diss, I probably wouldn't have done it (not necessarily a bad counterfactual, for me or for the world). It would've taken extra years to scan and turn into an "active citation" every single empirical claim I make. And I would have incurred such prohibitive costs even if I had worked "only" with archival evidence. But I relied heavily on thousands of newspaper articles on microfilm. Most of these regional newspapers aren't easily retrievable for most US university-based researchers, such as those available through ProQuest Historical Newspapers, and probably never will be.

If I have to choose between original, exciting research that doesn't/can't comply with transparency norms versus a world in which those norms are rules with which all are compliant, but the exciting research doesn't exist, I'd choose the former. Obviously, the real choice isn't so stark; it's a case-by-case thing for journal editors (AND university press editors, some of whose reviewers will presumably start imbibing these norms). If the status quo is democracy, then greater research transparency could be polyarchy--something to strive for, but not at all costs. What would be good to develop, therefore, is a sensible and practical set of guidelines that would allow researchers to determine (and would help most reviewers and readers agree) which claims are sufficiently critical that it is worth the costs of making the source materials available (legal and ethical constraints permitting).

This seems to be an issue that is not peculiar to APD scholarship and I see no reasons why the guidelines for APD scholars would or should differ, so I vote for cross-cutting working group foci.