Substantive Dimensions of the Deliberations

Forum rules

We encourage contributors to the Discussion Board to publicly identify by registering and logging in prior to posting. However, if you prefer, you may post anonymously (i.e. without having your post be attributed to you) by posting without logging in. Anonymous posts will display only after a delay to allow for administrator review. Contributors agree to the QTD Terms of Use.

Instructions
To participate, you may either post a contribution to an existing discussion by selecting the thread for that topic (and then click on "Post Reply") or start a new thread by clicking on "New Topic" below.

The transition to Stage 2 of the deliberations is currently underway but will take some time to complete. In the meantime, we very much welcome additional contributions to the existing threads in this forum.

For instructions on how to follow a discussion thread by email, click here.

John Gerring
University of Texas at Austin
Posts: 2
Joined: Sat Apr 09, 2016 8:28 pm

Beyond the qual/quant divide

PostSat Apr 09, 2016 9:33 pm

Having read some of the previous posts - and paid some attention to the debate carried out in other venues over DA*RT - i have lots of thoughts, but let me try to boil things down so i don't take up too much of everyone's time...

Insofar as ours is an empirical endeavor the goal of transparency would seem to matter greatly. If an argument is based on some sort of evidence (be it interviews, observations, texts, pictures, shards, whatever) then it is a problem if the author cannot produce the evidence, as well as the form of analysis the researcher used to reach his/her conclusions. I grant that it is not always possible to do this, and i'll discuss some of these problems below. But my point is (I think) a simple one: transparency is good, ceteris paribus.

The debate over DA*RT rests on the ceteris paribus conditions. What sacrifices are entailed, and are they worth the costs (or in what situations might they be worth the cost)?

In answering this question there seems to be a common view that the obstacles are greater for quals than for quants. I agree, but only to a point. Let me give some examples...

1. Protecting sources. This is a persistent - and very real - worry about transparency. But it is certainly not unique to qual work; indeed, there are elaborate procedures for protecting the anonymity of survey respondents. Nor does it apply to all qual work, e.g., to research based on archives or published work (as Lisa Wedeen notes). Protecting sources, in my view, runs orthogonal to the qual/quant divide.

2. Costs for the researcher (time and money, if RA's are involved). Again, this is a persistent and very real concern. But it is certainly not unique to qual research. To be sure, it may be somewhat more time-consuming for qual researchers to prepare their notes (or other material) for archiving, including anonymizing sources, where necessary. But this is a matter of degrees, i would think. And the process is eased (i would hope) by the Qualitative Data Repository hosted at IQMR.

3. Proprietarial material. I can think of at least one obstacle to transparency that is found more commonly on the quant side than the qual side. Sometimes data used for research is owned by someone else. While it may be used, perhaps in an aggregate format, the owner of the data does not consent to make the data public. (I imagine there are examples of this in qual work but i can't think of any off the top of my head.) An example in the quant world is the influential World Bank Governance Indicators, developed by Daniel Kaufmann and collaborators. The data used to construct these indices is mostly proprietarial and therefore is not released with each version of the dataset. People are troubled by this - as they should be - but that doesn't seem to prevent publications in (top) scholarly journals.

Other examples might be discussed but i think the point is clear. There are costs to achieving greater research transparency, and they fall on all researchers. Perhaps the burden is somewhat heavier on the qual side, but we ought not imagine that the quantoids are getting off lightly.

By way of conclusion, i would suggest that we focus on the specific methodological issues that are raised by DA*RT rather than whether the work under review is quant or qual. It is perfectly reasonable to insist that no transparency initiative should compromise the confidentiality of sources, for example. But it does not seem reasonable to say that qualitative work should be exempt from the DA*RT, as some contributors to this symposium seem to suggest.

For those who are unhappy with standards adopted by the APSR or other journals, or by APSA, i would issue a challenge: let's work to improve them.

Post Reply

Mala Htun
Univ of New Mexico
Posts: 16
Joined: Wed Apr 06, 2016 9:20 am

Re: Beyond the qual/quant divide

PostSun Apr 10, 2016 8:36 pm

you should check out the other thread about the costs of compliance with data access requirements. for work based on interviews, this would involve transcription and anonymization, which is time consuming and expensive. on that thread,several folks pointed out that the costs could be prohibitive and that they personally would face huge barriers to continuing with qualitative work. i dont think we want to make a comparison with quant work (totally agree there are costs there too!) so much as to discuss how to manage the compliance costs. for me, this is important for its effects on the incentive structures underlying various forms of inquiry in the profession. see also my essay in the CP newsletter, just released last week. thanks! mala

Post Reply

Jonathan Fox
School of International Service, American University
Posts: 3
Joined: Thu Apr 07, 2016 1:39 pm

Re: Beyond the qual/quant divide

PostSun Apr 24, 2016 2:34 pm

I second Mala Htun's about the compliance costs, especially for those do numerous informal interviews as part of full immersion in ethnographic field research.

The "let's work to improve this" spirit is what seems to drive this deliberative process, though insofar as this is unfolding well behind the decisions already made by influential journals, the playing field is less than level.

Post Reply

Tim Buthe
HfP/Technical Univ of Munich & Duke University
Posts: 32
Joined: Fri Feb 26, 2016 11:39 pm

Re: Beyond the qual/quant divide

PostSat May 21, 2016 11:48 pm

Thank you to John, Mala, and Jonathan, for your contributions to this thread. I appears that you all agree (as do probably many others) that we do not want to reify the qualitative-quantitative divide--and that there are numerous different kinds of "qualitative" scholarship. Next week, we will start to move from the at-large deliberations of stage 1 of the QTD to stage 2. During stage 2, all of the original threads remain available, but the deliberations are supposed to continue, primarily, in 12-15 separate working groups, so as to allow more differentiated discussions of research transparency in specific qualitative research traditions.

All of this raises two important and now increasingly urgent question:

(1) For the kind of research you do, what are the aspects of research transparency that are most in need of differentiated deliberations during the 2nd stage of the QTD process? (Please be sure to specify the "kind of research you do")

And more generally: (2) How should the stage 2 working groups be delineated or defined?

Please feel free to weigh in here or by posting a reply on the general thread about how the working groups should be defined.

Post Reply

robert mickey
university of michigan--ann arbor
Posts: 3
Joined: Sat May 14, 2016 8:04 pm

Re: Beyond the qual/quant divide

PostWed May 25, 2016 12:27 pm

Transparency is a great idea, and we should embrace much of this movement. I appreciate what John Gerring is up to and what he says here. But I don't think Andy Moravcsik's claim that qual work is in crisis because of transparency and replicability is a persuasive one. Qual work is considered by many scholars as unscientific largely for reasons having to do with number of cases, degrees of freedom, and the (perceived) absence of (good) rules/norms about the specification of causal claims and provision of evidence about them. Telling them where our sources come from won't matter. So I support qualitative researchers' adaptation of this movement on terms that make sense for qual. work, but I think the motivation should be a principled commitment to transparency, rather than the hope that "now maybe they'll like us." That's not going to happen short of a much larger transformation of qualitative political science (or of mainstream scholars' understanding of what science is).

Post Reply

Fora Admin
Qualitative Transparency Deliberations
Site Admin
Posts: 1
Joined: Thu Feb 11, 2016 4:54 pm

Re: Beyond the qual/quant divide

PostMon Aug 22, 2016 4:59 pm

robmickey wrote:Transparency is a great idea, and we should embrace much of this movement. I appreciate what John Gerring is up to and what he says here. But I don't think Andy Moravcsik's claim that qual work is in crisis because of transparency and replicability is a persuasive one. Qual work is considered by many scholars as unscientific largely for reasons having to do with number of cases, degrees of freedom, and the (perceived) absence of (good) rules/norms about the specification of causal claims and provision of evidence about them. Telling them where our sources come from won't matter. So I support qualitative researchers' adaptation of this movement on terms that make sense for qual. work, but I think the motivation should be a principled commitment to transparency, rather than the hope that "now maybe they'll like us." That's not going to happen short of a much larger transformation of qualitative political science (or of mainstream scholars' understanding of what science is).


This is a test post.

Post Reply
A user can add any text they want to their signature. :D
Signatures are fun.