To participate, you may either post a contribution to an existing discussion by selecting the thread for that topic (and then click on "Post Reply") or start a new thread by clicking on "New Topic" below.
The transition to Stage 2 of the deliberations is currently underway but will take some time to complete. In the meantime, we very much welcome additional contributions to the existing threads in this forum.
For instructions on how to follow a discussion thread by email, click here.
University of Texas at Austin
- Posts: 2
- Joined: Sat Apr 09, 2016 8:28 pm
Insofar as ours is an empirical endeavor the goal of transparency would seem to matter greatly. If an argument is based on some sort of evidence (be it interviews, observations, texts, pictures, shards, whatever) then it is a problem if the author cannot produce the evidence, as well as the form of analysis the researcher used to reach his/her conclusions. I grant that it is not always possible to do this, and i'll discuss some of these problems below. But my point is (I think) a simple one: transparency is good, ceteris paribus.
The debate over DA*RT rests on the ceteris paribus conditions. What sacrifices are entailed, and are they worth the costs (or in what situations might they be worth the cost)?
In answering this question there seems to be a common view that the obstacles are greater for quals than for quants. I agree, but only to a point. Let me give some examples...
1. Protecting sources. This is a persistent - and very real - worry about transparency. But it is certainly not unique to qual work; indeed, there are elaborate procedures for protecting the anonymity of survey respondents. Nor does it apply to all qual work, e.g., to research based on archives or published work (as Lisa Wedeen notes). Protecting sources, in my view, runs orthogonal to the qual/quant divide.
2. Costs for the researcher (time and money, if RA's are involved). Again, this is a persistent and very real concern. But it is certainly not unique to qual research. To be sure, it may be somewhat more time-consuming for qual researchers to prepare their notes (or other material) for archiving, including anonymizing sources, where necessary. But this is a matter of degrees, i would think. And the process is eased (i would hope) by the Qualitative Data Repository hosted at IQMR.
3. Proprietarial material. I can think of at least one obstacle to transparency that is found more commonly on the quant side than the qual side. Sometimes data used for research is owned by someone else. While it may be used, perhaps in an aggregate format, the owner of the data does not consent to make the data public. (I imagine there are examples of this in qual work but i can't think of any off the top of my head.) An example in the quant world is the influential World Bank Governance Indicators, developed by Daniel Kaufmann and collaborators. The data used to construct these indices is mostly proprietarial and therefore is not released with each version of the dataset. People are troubled by this - as they should be - but that doesn't seem to prevent publications in (top) scholarly journals.
Other examples might be discussed but i think the point is clear. There are costs to achieving greater research transparency, and they fall on all researchers. Perhaps the burden is somewhat heavier on the qual side, but we ought not imagine that the quantoids are getting off lightly.
By way of conclusion, i would suggest that we focus on the specific methodological issues that are raised by DA*RT rather than whether the work under review is quant or qual. It is perfectly reasonable to insist that no transparency initiative should compromise the confidentiality of sources, for example. But it does not seem reasonable to say that qualitative work should be exempt from the DA*RT, as some contributors to this symposium seem to suggest.
For those who are unhappy with standards adopted by the APSR or other journals, or by APSA, i would issue a challenge: let's work to improve them.
Univ of New Mexico
- Posts: 16
- Joined: Wed Apr 06, 2016 9:20 am
School of International Service, American University
- Posts: 3
- Joined: Thu Apr 07, 2016 1:39 pm
The "let's work to improve this" spirit is what seems to drive this deliberative process, though insofar as this is unfolding well behind the decisions already made by influential journals, the playing field is less than level.
- Posts: 31
- Joined: Fri Feb 26, 2016 11:39 pm
All of this raises two important and now increasingly urgent question:
(1) For the kind of research you do, what are the aspects of research transparency that are most in need of differentiated deliberations during the 2nd stage of the QTD process? (Please be sure to specify the "kind of research you do")
And more generally: (2) How should the stage 2 working groups be delineated or defined?
Please feel free to weigh in here or by posting a reply on the general thread about how the working groups should be defined.
university of michigan--ann arbor
- Posts: 3
- Joined: Sat May 14, 2016 8:04 pm
Qualitative Transparency Deliberations
- Site Admin
- Posts: 1
- Joined: Thu Feb 11, 2016 4:54 pm
robmickey wrote:Transparency is a great idea, and we should embrace much of this movement. I appreciate what John Gerring is up to and what he says here. But I don't think Andy Moravcsik's claim that qual work is in crisis because of transparency and replicability is a persuasive one. Qual work is considered by many scholars as unscientific largely for reasons having to do with number of cases, degrees of freedom, and the (perceived) absence of (good) rules/norms about the specification of causal claims and provision of evidence about them. Telling them where our sources come from won't matter. So I support qualitative researchers' adaptation of this movement on terms that make sense for qual. work, but I think the motivation should be a principled commitment to transparency, rather than the hope that "now maybe they'll like us." That's not going to happen short of a much larger transformation of qualitative political science (or of mainstream scholars' understanding of what science is).
This is a test post.
Signatures are fun.