III.2. Interpretive methods

Forum rules

To download the working group's draft report, select the "DRAFT REPORT" announcement. Please provide comments or other feedback on the draft via the first topic-thread "Comments on Draft Report ..." You may also continue to view and add to the earlier threads. Please log in first to have your post be attributable to you. Anonymous posts will display only after a delay to allow for administrator review. Contributors agree to the QTD Terms of Use.

Tim Buthe
Duke University
Posts: 55
Joined: Fri Feb 26, 2016 11:39 pm

Comments on Draft Report of Working Group III.2

PostTue Aug 29, 2017 10:56 am

Please use this thread to share feedback on the draft report

Post Reply


Guest

Re: Comments on Draft Report of Working Group III.2

PostFri Sep 08, 2017 1:31 pm

This report is admirably clear and concise. It does a fine job of highlighting the stakes of this debate for scholars who do not share the assumptions of the mainline tendencies in political science while helpfully pointing to the political stakes that might animate these differences. As such, this report shows why scholars should be concerned about this initiative not only because it imposes onerous requirements on researchers, and especially on early career scholars, but also because it risks narrowing the field of political science and foreclosing lines of criticism—political as well as epistemological—that take aim at the methodological commitments of those who favor this initiative.

The point here is not that interpretive scholars should be outraged by the mainstream of the discipline's inattentiveness to interpretivist scholarship. Nor is it to malign the excellent scholarship conducted by many mainstream and/or positivist political scientists who share the assumptions underlying this initiative. Instead, my view is that scholarly communities ought to be broad churches, graduate students and faculty ought to be conversant in both positivist and non-positivist approaches to the study of politics, and we should have enough presumptive generosity when reading our colleagues' work to accept that we may simply come to different judgments over what any given data point means. Sometimes people will be wrong, of course; sometimes they will misunderstand the context for a piece of evidence or simply misread it. Good citational practice, careful reading, and critical distance will allow that to come to light and be refuted. And even if it is not refuted, it will ultimately be worked over and refashioned into something very different by later scholars whose own worlds will condition how they (creatively mis)read the work of scholars today. No data repository is likely to alter this commonplace in academic research, nor is it obvious why we should want it to.

Post Reply


Tim Buthe
Duke University
Posts: 55
Joined: Fri Feb 26, 2016 11:39 pm

Re: Comments on Draft Report of Working Group III.2

PostSun Sep 10, 2017 1:13 pm

Thank you to the author of this anonymous post for providing this thoughtful feedback. In case you are monitoring the thread, may I ask you to clarify (to avoid confusion) what you mean by "this initiative" when you write:

Guest wrote: ... this report shows why scholars should be concerned about this initiative ....


In the QTD, we strive to have the inclusiveness and the intellectual generosity for which you call and hope to foster better understanding and mutual respect among positivistic and non-positivist scholars of politics, so I trust you do not mean the QTD (as part of which, moreover, this very report was written).

Thanks again,
Tim Büthe, QTD co-chair

Post Reply


Daragh Grant
Harvard University
Posts: 2
Joined: Thu Apr 07, 2016 5:50 pm

Re: Comments on Draft Report of Working Group III.2

PostTue Sep 12, 2017 11:19 am

Dear Tim,

apologies for the lack of clarity; by "this initiative" I meant to reference the Data Access and Research Transparency proposals as well as the Journal Editors' Transparency Statement (so, "these initiatives" would have been more precise).

I appreciate the work of those organizing and contributing to the Qualitative Transparency Deliberations, and certainly didn't mean to imply that there was a lack of collegiality in this deliberative process. Nevertheless, I broadly agree with the authors of this particular draft report when they begin by quoting Tim Pachirat's observation that "Proponents of the Data Access and Research Transparency (DA-RT) initiative, the Journal Editors’ Transparency Statement (JETS) and indeed the Qualitative Transparency Deliberations have tended to assume that 'transparency is a universal value and that its institutionalization does not trouble, challenge, reorder or impose an explicit or implicit hierarchy of worth on the ontological and epistemological diversity of existing research communities and traditions...[transparency is cast] as a strictly neutral vessel, which scholars from every research tradition can implement and enforce,'" and then go on to question "the adequacy of conceptions of transparency informing these initiatives."

And I am not sure why my original post appeared anonymously. I had intended it to appear with identifying information when I posted it.

Best regards,

Daragh Grant
Lecturer, Committee on Degrees in Social Studies
Harvard University

Post Reply


Timothy Pachirat

Re: Comments on Draft Report of Working Group III.2

PostMon Sep 18, 2017 3:42 pm

Posted by Timothy Pachirat (who can't login properly because he can't recover his password!):

I'll post my own reactions to this Working Group Report separately below, but I do want to offer a small and friendly amendment to Daragh Grant's post: the quotation mark for my observation is misplaced--my quote actually picks up at "transparency is a universal value...." The first part of Daragh's quotation, mistakenly attributed to me, is actually the language of the Working Group Draft Report itself (they go on to quote me later in that sentence).

My original quote does not and could not have referenced the Qualitative Transparency Deliberations (QTD), since QTD didn't even exist when I wrote the Tyranny of Light essay in 2015. That said, to the extent that QTD functions in both form and substance to agenda-set the importance of transparency and to suggest/argue/advance/implement the idea that such transparency is a neutral, universal good that can and should be unpacked and applied by different research communities, then I would agree with the authors of the Working Group and with Daragh that the quote is also applicable to QTD.

For reference, the full quote from paragraph two of my Tyranny of Light essay is:

"DA-RT proponents argue that they are simply reinforcing a key universal value—transparency—and that they are not doing so in any way that troubles, challenges, reorders, or imposes an explicit or implicit hierarchy of worth on the ontological and epistemic diversity of existing research communities and traditions within the discipline. DA-RT, in this view, is a strictly neutral vessel, at its core an a-political or depoliticized set of guidelines which scholars from every research tradition should then take and decide for themselves how to best implement and enforce."

(Obviously, the essay then goes on to contest this understanding of transparency.)



[quote="djgrant"]Dear Tim,

apologies for the lack of clarity; by "this initiative" I meant to reference the Data Access and Research Transparency proposals as well as the Journal Editors' Transparency Statement (so, "these initiatives" would have been more precise).

I appreciate the work of those organizing and contributing to the Qualitative Transparency Deliberations, and certainly didn't mean to imply that there was a lack of collegiality in this deliberative process. Nevertheless, I broadly agree with the authors of this particular draft report when they begin by quoting Tim Pachirat's observation that "Proponents of the Data Access and Research Transparency (DA-RT) initiative, the Journal Editors’ Transparency Statement (JETS) and indeed the Qualitative Transparency Deliberations have tended to assume that 'transparency is a universal value and that its institutionalization does not trouble, challenge, reorder or impose an explicit or implicit hierarchy of worth on the ontological and epistemological diversity of existing research communities and traditions...[transparency is cast] as a strictly neutral vessel, which scholars from every research tradition can implement and enforce,'" and then go on to question "the adequacy of conceptions of transparency informing these initiatives."

And I am not sure why my original post appeared anonymously. I had intended it to appear with identifying information when I posted it.

Best regards,

Daragh Grant
Lecturer, Committee on Degrees in Social Studies
Harvard University[/quote]

Post Reply


Timothy Pachirat

Re: Comments on Draft Report of Working Group III.2

PostMon Sep 18, 2017 4:52 pm

Posted by Timothy Pachirat (who cannot login properly because he can’t recover his password):

Thank you to Lisa Bjorkman, Lisa Wedeen, Juliet Williams, and Mary Hawkesworth for their labor on this superb report. Three highlights, among others:

1. The report’s eloquent refusal of a discipline-wide implementation of transparency standards, no matter how devolved and differentiated by research community.

2. The report’s show-don’t-tell application of interpretive methods (including semiotics, deconstruction, genealogy, and perhaps also a subconscious sprinkling of psychoanalysis) to the rise of transparency-talk itself, demonstrating the ways in which DA-RT and its attendant narratives serve, among other undesirable things, to:

a) “Defend the stability of liberal politics….and permit positivist political science to occupy the position of authorized (because disinterested) discoverer, teacher, enforcer of what counts as true of justified statements about politics” (p. 7 quoting Wedeen 2016, pp. 34-35).
b) “Reassert commitments to openness and transparency while burying substantive concerns about political bias [against the study of race and inequality] under guidelines for data archives and adherence to methodological strictures” (p. 9), and
c) Promote an ongoing “inattention to the ways in which scholarly production is tied to our ‘aspirations to the kind of power that is presumed to accompany…science’ (Foucault 1980, p. 84)” (p.9).

3. The report’s recommendation that (drumroll please)…..in response to the particular transparency anxieties and desires articulated by DA-RT exactly nothing needs to be changed at a discipline-wide level and the report’s affirmation that existing peer review standards as exemplified by Perspectives on Politics under the prior editorship of Jeffrey Isaac are working very well, thank you very much.

My question: How do we (we being all of us united in this unholy matrimony that is political science) move forward in the face of an ongoing discipline-wide roll-out of something that more than a few of us refuse?

(And, as this Working Group Report makes resoundingly clear, refuse in the entirety of its conceptualization, theorization, and politicization, not just in secondary aspects of its implementation that can be ironed out through deliberation, no matter how inclusive, no matter how differentiated.)

I don’t have the answer, but I do have this story.

At a 2017 APSA business meeting of the Interpretive Methods Conference Related Group, a well-meaning member of the APSR editorial team attended and rose to speak. His goal, a laudable one, was to reassure scholars doing interpretive work that their work was welcome at the APSR and to encourage submissions to the journal.

“But what about the fact that APSR signed onto JETS? What are we to make of that given JETS’ intrinsic hostility to interpretive approaches?” the editorial team member was asked.

“Oh, not to worry,” the APSR editorial team member replied. “Although we have officially signed on to JETS [the Journal Editor’s Transparency Statement, perhaps the most explicit way DA-RT proponents are exercising a disciplinary power over political scientists] and publically reaffirm our commitment to DA-RT, we plan to treat interpretive and other potentially ‘problematic’ qualitative research submissions on a case-by-case basis, possibly relaxing some of JETS standards for such submissions. We can work it out. You just have to trust us.”

Faced with a wondrously polyvocal, polyamorous, and rebellious political science, is DA-RT becoming the crocodile that eats its own tail? To keep alive an illusion of universal and neutral “transparency,” must its proponents now make recourse to the very thing that provoked many of the anxieties that first called DA-RT into existence? That is, to trust itself?

Post Reply


Guest

Re: Comments on Draft Report of Working Group III.2

PostMon Nov 06, 2017 9:05 pm

As an interpretivist, I have followed this thread and read the report closely. I like a lot of the report, like the section on reframing priorities. It is important to point out that demands for "transparency" and claims about following best transparency practices CAN be used instrumentally as a way to exercise power. And the section on "transparency as a political concept" makes a strong case for concerns about such a use of transparency. It is also important to make it clear that transparency is not a universally shared value. And that the language of transparency has, in the wake of DART and JETS, come to be understood among many interpretivist political scientists as hostile to their research tradition. In addition, it is probably very helpful to remind non-interpretivist readers of this report of the diversity of interpretivist research logics. And to identify common beliefs as well as different elements of various non-positivist research logics.

Although I like most of the report and thank the authors for it, the report also contains some passages that I consider problematic and closer to political rhetoric than to the thoughtful interpretivist scholarship I admire and seek to do myself:
- Timothy Pachirat's statement about the assumptions made by certain colleagues, quoted in the beginning, seems more assertion than based on any discernible interpretivist method.
- There are many non-interpretivist faculty and PhD students in my department. None of them believe in the "veridical understanding" attributed to them in the Isaac quote.

What I find really unsatisfying about the report is that is makes no effort to explore the seemingly obvious common ground between Isaac's pluralistic call for scholars to explain why and how the perspective they have chosen "ought to be considered illuminating" and to explain those "methods and techniques" as well as explain the "evidence to support their explanation," as well as his demand that scholars "offer an account of why the perspective that has been presented is interesting, important and fruitful for further thinking about things that matter" (quoted as part of the final recommendation) and the proclaimed goals of transparency advocates. Much, but not all, interpretivist scholarship does very well in meeting Isaac's expectation. Much non-interpretivist scholarship, by contrast, does not meet those expectations in my reading. I often wish authors would do more in this regard. So I do not agree with Pachirat that "exactly nothing needs to be changed." That's the intellectual case for exploring the common ground. There's also a strategic and professional case for doing so: As a graduate student who's just starting out in the profession, I'm not really interested in a lifelong crusading antagonism vis-a-vis non-interpretivists. Of course, I strongly disagree on some fundamentals with my PhD student friends who take a more "positivist" approach, but as a pluralistic interpretivist, I also recognize that I do not have the only valid viewpoint, and I want to continue to engage with them for mutually fruitful gain. Exploring common ground, as well as clarifying our differences, allows us to do this.

Post Reply


Peri Schwartz-Shea
University of Utah
Posts: 8
Joined: Thu Oct 12, 2017 6:03 pm

Re: Comments on Draft Report of Working Group III.2

PostTue Nov 14, 2017 11:44 am

Peer review, not DA-RT, not JETS!

Colleagues, I am writing to comment on the 8-27-2017 draft report QTD Working Group III.2, Interpretive Methods. Thanks to Professors Bjorkman, Wedeen, Williams, and Hawkesworth for all of the excellent work that has gone into this document. I want to go on record endorsing the report’s statements that:

[JETS’] “fixating on data transparency… perpetuates the false belief that the professional currently lacks standards to ensure research integrity” (p. 9).

“Our recommendation is that the peer review system, already in place, is sufficient to insure research integrity (Isaac 2015; Trachtenberg 2015” (p. 10).

This recommendation is important because a major set of questions that proponents of the “transparency project” have yet to answer are: How is DA-RT meant to relate to peer review? Why isn’t peer review sufficient? If it is not, what specific elements need improvement and why are DA-RT and, especially, JETS the proper responses?

I would also add that DA-RT and, particularly the required exemptions of JETS, promotes an approach that is not only unnecessary but actively harmful to scholarly values of autonomy, creativity, and collegiality.

Despite flaws, the major advantage of peer review is that it involves the application of quality assessment standards by expert readers in a project-centered manner. DA-RT and JETS encourage researchers and, especially, graduate students to aspire to general, abstract standards if they want to escape the exemptions stigma and the possibility that their work cannot even arrive at the peer review stage. Self-censorship of potentially career-risky projects is already happening under the contemporary, prior-review IRB regime and now DA-RT/JETS closes the circle. We should not underestimate the ways in which these systems will discourage new researchers from taking up difficult questions and unorthodox methods. What these systems communicate is that researchers are not to be trusted unless they become ciphers – transparent to the world, to all imagined readers. DA-RT/JETS fails to recognize the developmental arc of a scholarly career and the ways in which our research practices develop and deepen over the years as we develop into bona fide experts. [See Flyvbjerg’s (2001, 10-24), discussion of Dreyfus’ model of the development of expert knowledge.] The projected-centered, peer review system honors and respects scholarly expertise; it is a system that communicates that researchers should develop substantive knowledge with the promise that their projects will be assessed by others with similar substantive backgrounds—who understand the ins and outs of the questions and methods in the area—and who will then apply scholarly standards that fit those projects. Well-functioning peer review critically assesses a project in ways that support and elevates the scholar’s efforts.

Moreover, the DA-RT and JETS conception of “analytic transparency,” promotes the impossible. The admonition that researchers (emphasis added) provide “a full account [of] how they draw their analytic conclusions from the data, i.e., clearly explicate the links connecting data to conclusions” flies in the face of understandings of the practice of experts, which involves tacit knowledge (see the citation above to the section in Flyvbjerg, 2001). As Margaret Keck observes in note 23 of the QTD Working Group II.1, Research with Text-Based Sources:

“To analyze documents and interviews, I rely not just on language skills but on knowledge accumulated from 35 years of work in a region….”

Peer review relies on experts of a similar caliber to assess her research claims—why we read and trust peer-reviewed articles. Most scholars have neither the time nor the in-depth expertise to assess whether a text she references (say, a portion revealed by a link via active citation) is “properly” interpreted.

Peer review, not DA-RT, not JETS!

References
Flyvbjerg, Bent. 2001. Making Social Science Matter. Cambridge, UK: Cambridge University Press.

Post Reply



Return to “III.2. Interpretive methods”