III.3. Ethnography and participant observation

Forum rules

To download the working group's draft report, select the "DRAFT REPORT" announcement. Please provide comments or other feedback on the draft via the first topic-thread "Comments on Draft Report ..." You may also continue to view and add to the earlier threads. Please log in first to have your post be attributable to you. Anonymous posts will display only after a delay to allow for administrator review. Contributors agree to the QTD Terms of Use.

Tim Buthe
Duke University
Posts: 55
Joined: Fri Feb 26, 2016 11:39 pm

Comments on Draft Report of Working Group III.3

PostTue Aug 29, 2017 2:46 am

Please use this thread to share feedback on the draft report

Post Reply


Timothy Pachirat

Re: Comments on Draft Report of Working Group III.3

PostMon Oct 02, 2017 12:27 am

I want to thank Jillian Schwedler, Erica Simmons, and Nicholas Rush Smith for their labor on the QTD Working Group Report on Ethnography and Participant Observation. I post this feedback with their blessing and in a spirit of respect for their work as ethnographers and people who think carefully about methodology and method. I am also thankful to them for taking on the rather thankless task of wrangling ethnographers in political science (a rowdy bunch, to be sure) and trying to represent our collective voice within a mandated discussion that many of us see, at best, as originating in desires and anxieties quite foreign to our own.

The report has many excellent qualities, including a clear outline of some of the scaffolding that should be a part of all good and persuasive ethnographic work. As you might imagine, though, I am firmly opposed to the introduction of the term "transparency" into ethnographic evaluative criteria, no matter how strongly qualified, no matter with how many caveats.

Everything covered in this report is, I believe, much better subsumed under the notion of reflexivity, a concept with a long theoretical and practice-based history in ethnographic work across multiple disciplines. This history, for example, subsumes the entirety of the list in Ben Read's otherwise excellent QTD post on ethnography and transparency, which the working group report cites from. Indeed, scholars familiar with reflexivity as a central norm of ethnographic practice and evaluation would not be wrong to feel surprised and perhaps not a little perplexed by the strange resurrection of reflexivity in the guise of “transparency” as if it were a novel topic.

Nor is this an argument over “mere” words. I believe there is much at stake here for those of us who do ethnographic work in political science. To put it simply, transparency conceals and evacuates power while reflexivity foregrounds it as an object of ethnographic analysis.

(For some of the ways “transparency” conceals and evacuates power, please see my Tyranny of Light essay; see also the superb QTD Interpretive Methods Working Group Report for a genealogy of “transparency” and a sophisticated argument for why it ought not be adopted as a universal standard in the discipline. For extensive discussions and/or enactments of reflexivity in political science, see, for example, essays in Ed Schatz’s Political Ethnography: What Immersion Contributes to the Study of Power; essays in Dvora Yanow and Peregrine Schwartz-Shea’s Interpretation and Method: Empirical Research Methods and the Interpretive Turn; Fred Schaffer’s Elucidating Social Science Concepts; Lee Ann Fujii’s Interviewing in Social Science Research; Ido Oren’s Our Enemies and US; Robert Vitalis’ White World Order, Black Power Politics; Anne Norton’s 95 Theses; Mary Hawkesworth’s Embodied Power; or my forthcoming Among Wolves: Ethnography and the Immersive Study of Power. Several recent superb ethnographies produced by political scientists also powerfully showcase what the practice of reflexivity brings to the study of power. To name but a few, see, for example: Lisa Wedeen’s Peripheral Visions; Severine Autesserre’s Peaceland; Lee Ann Fujii’s Killing Neighbors; Sarah Weibe’s Everyday Exposure; Regine Spector’s Order at the Bazaar; Erica Simmons’ Meaningful Resistance; Erin Beck’s How Development Projects Persist; Mneesha Gellman’s Democratization and Memories of Violence; Sarah Parkinson’s fieldwork-based articles on militant organizations; Robin Turner’s work on nature tourism and rural politics in southern Africa; Nicholas Rush Smith’s forthcoming book on vigilantism and the South African state; and my own Every Twelve Seconds.)

By sticking with reflexivity, ethnographers retain the centrality of investigating how *power* is at work, not only in our loci and objects of study (our fieldsites and the theoretical preoccupations we put them in conversation with), but in constituting the instruments, methods, and relationships of research themselves. Further, if we insist on reflexivity--if we refuse to substitute for it the anti-politics machine that is "transparency"--we retain ethnography's potential to serve as a site of critique for the ways power is at work in political science, and in transparency-talk in particular: at work, for example, to occlude key dimensions of potential domination, including in the relationships between the researcher and the researched. How ironic, after all, that in its original manifestation DA-RT addressed only so-called transparency *between* researchers and had to be forcibly reminded that researchers also incur obligations of reciprocity, trust, and fairness to the worlds they purport to describe, represent, and analyze (see, for e.g., the QTD Working Group reports on Research Ethics and Vulnerable and Marginalized Populations)?

I am fearful that despite its many excellent qualities, the report as it is currently written will serve to advance transparency as a concept and a disciplining paradigm for ethnographic work. Despite its nuanced attempts to qualify the term transparency and its many reminders that not all ethnographers will agree with the term, the report nonetheless ultimately accedes to the very "translation" project that DA-RT and QTD call for. I'm fearful of the consequences of that translation project for the ability of political ethnographers to produce original, daring, and provocative work. And, I’m even more fearful of the consequences of a continued bleeding of analyses of power from how the discipline thinks about its own methods.

I encourage the authors of this report to refuse the “translation” of transparency. Although it may seem an expedient and collegial way to enter a conversation whose pre-set agenda revolves around “transparency,” such translation puts much at risk insofar as these QTD Working Group Reports have the potential to become performative utterances, taken up as “policy” in unexpected places and in unexpected ways (and with the added “legitimacy” bestowed by widespread deliberation). Instead, I encourage the authors of this report to educate the wider political science community about longstanding practices and standards of reflexivity in ethnographic work. Transparency-talk conceals and evacuates power, but by exploring the theorization and practice of reflexivity in ethnographic work, ethnographers have an opportunity to once again foreground the ways power is always at work to constitute our objects of study and the instruments we use to study them. It would be worth imagining, for example, what would happen if QTD were renamed QRD: Qualitative Reflexivity Deliberations, and what might result if the enormous energy, resources, and talent currently conscripted in the service of transparency were instead redirected to thinking systematically about how power is at work to constitute political science’s objects of study as well as its collective desires and anxieties as a discipline.

Finally, in closing, I would once again point participants in the QTD process to the report from the Working Group on Interpretive Methods as an exemplar for how to participate in these QTD discussions while remaining firm in a refusal of their defining, authorizing term: transparency.

Reflexivity, not transparency!

Post Reply


Ed Schatz

Re: Comments on Draft Report of Working Group III.3

PostTue Oct 10, 2017 9:29 pm

Tim Pachirat is right that notions of reflexivity could be further developed here; that's a terrific suggestion.

I am less sure how far I'd go toward throwing out the word "transparency," which he argues has been so tainted by DA-RT as to be beyond remedy. I understand and appreciate Tim's position, but I worry that if we cede too much linguistic ground we also cede intellectual ground. Imagine a hypothetical report that replaced "transparency" entirely; I fear it would be read--cursorily, because that's how people "read" these days--as advancing criteria for ethnographic work that need not apply to non-ethnographic work. The implication would be: non-ethnographic work is transparent, ethnographic work is something else. I know that Tim's point is the opposite--that reflexivity should be front-and-center in all research traditions--but I fear that is not how it'd be received.

Having said this, I am all for the idea of a Qualitative Reflexivity Deliberations (QRD). It'd be fantastic to encourage formal modelers, survey researchers, and everyone else to engage in reflexivity.
--Ed Schatz

Post Reply


Peri Schwartz-Shea
University of Utah
Posts: 8
Joined: Thu Oct 12, 2017 6:03 pm

Re: Comments on Draft Report of Working Group III.3

PostTue Nov 14, 2017 1:27 pm

Colleagues, I am writing to comment on the 8-25-2017 draft report QTD Working Group III.3, Ethnography and Participant Observation. Thanks to Professors Schwedler, Simmons, and Rush for all of the excellent work that has gone into this document. I want to go on record endorsing a couple of the report’s statements.

First, the report briefly examines the Boston College incident (p. 9), which deserves wider discussion in the discipline more broadly. I would add a reference, Palys, Ted and Lowman, John. 2012. Defending Research Confidentiality “To the Extent the Law Allows”: Lessons from the Boston College Subpoenas. Journal of Academic Ethics 10: 271-97. Relatedly, in the technical realm, in a period of astounding data insecurity, I have yet to see proponents of DA-RT/JETS seriously address the hacking problem for sensitive data stored in “trusted digital repositories.”

Second, I want to endorse Timothy Pachirat’s October 1st post recommending the concept of reflexivity over transparency (or openness). “Transparency” encourages a “laying bare” of the scholarly self, implying that our human role in knowledge production is meant to disappear; “reflexivity” is a concept that acknowledges human embodiment in all of its complexity and asks that researchers actively reflect on how embodiment affects the knowledge they produce (as the report analyzes on page 4).

Third, the report implies that DA-RT and JETS may produce “negative consequences for knowledge production” (p. 9), focusing particularly on the ways in which JETS may segregate ethnographic research “to only certain venues.”

I would add to this point that DA-RT and, particularly the required exemptions of JETS, promotes an approach that is not only unnecessary but actively harmful to scholarly values of autonomy, creativity, and collegiality. The DA-RT and JETS approach that qualitative and interpretive work needs to be “exempted” and “accommodated” (the latter term used on p. 10 of the report) marks that research as somehow deficient – before it even gets to peer review!

A key problem with where we are in the overall discussion about DA-RT and JETS is that a major set of questions has yet to be squarely addressed: How is DA-RT meant to relate to peer review? Why isn’t peer review sufficient? If it is not, what specific elements need improvement and why are DA-RT and, especially, JETS the proper responses?

Here is why these questions matter. Despite flaws, the major advantage of peer review is that it involves the application of quality assessment standards by expert readers in a project-centered manner. DA-RT and JETS encourage researchers and, especially, graduate students to aspire to general, abstract standards if they want to escape the exemptions stigma and the possibility that their work cannot even arrive at the peer review stage. Self-censorship of potentially career-risky projects is already happening under the contemporary, prior-review IRB regime and now DA-RT/JETS closes the circle. We should not underestimate the ways in which these systems will discourage new researchers from taking up difficult questions and unorthodox methods. What these systems communicate is that researchers are not to be trusted unless they become ciphers – transparent to the world, to all imagined readers. DA-RT/JETS fails to recognize the developmental arc of a scholarly career and the ways in which our research practices develop and deepen over the years as we develop into bona fide experts. [See Flyvbjerg’s (2001, 10-24), discussion of Dreyfus’ model of the development of expert knowledge.] The projected-centered, peer review system honors and respects scholarly expertise; it is a system that communicates that researchers should develop substantive knowledge with the promise that their projects will be assessed by others with similar substantive backgrounds—who understand the ins and outs of the questions and methods in the area—and who will then apply scholarly standards that fit those projects. Well-functioning peer review critically assesses a project in ways that support and elevates the scholar’s efforts.

Moreover, the DA-RT and JETS conception of “analytic transparency,” promotes the impossible. The admonition that researchers (emphasis added) provide “a full account [of] how they draw their analytic conclusions from the data, i.e., clearly explicate the links connecting data to conclusions” flies in the face of understandings of the practice of experts, which involves tacit knowledge (see the citation above to the section in Flyvbjerg, 2001). As Margaret Keck observes in note 23 of the QTD Working Group II.1, Research with Text-Based Sources:

“To analyze documents and interviews, I rely not just on language skills but on knowledge accumulated from 35 years of work in a region….”

Peer review relies on experts of a similar caliber to assess her research claims—why we read and trust peer-reviewed articles. Most scholars have neither the time nor the in-depth expertise to assess whether a text she references (say, a portion revealed by a link via active citation) is “properly” interpreted.

Peer review, not DA-RT, not JETS!

References
Flyvbjerg, Bent. 2001. Making Social Science Matter. Cambridge, UK: Cambridge University Press.

Post Reply



Return to “III.3. Ethnography and participant observation”