Page 1 of 1

[From Steering Committee] What is openness/transparency/explicitness in violence/contentious politics research?

Posted: Mon May 16, 2016 7:39 pm
by separkinson
At this point in the QTD, I think it’s fairly clear that many scholars who employ field-based approaches when studying violence and contentious politics (among other topics, e.g. authoritarianism, race/ethnicity, inequality) are strongly opposed to “data sharing” or “transparency” in the form of providing interview transcripts or field notes, even if redacted or de-identified. Many people have voiced serious reservations rooted in both ethical (e.g. threats to research subjects) and methodological concerns (e.g. it is an inappropriate standard for many research traditions, such as ethnography). In more recent iterations of the debate, other pitfalls of data sharing for violence researchers have emerged. In the recent Comparative Politics section newsletter, Lee Ann Fujii notes, for example, that depositing materials such as photographs and postcards may reproduce violent processes (e.g. if one were to deposit formerly popular postcards depicting lynchings), and imply a specific type of politics in its own right.

My sense from violence scholars is that they constantly engage with questions of openness, transparency, honesty, and ethics; this happens both individually in publications, as a community during classes, workshops, and conferences, and via IRB processes (as flawed as they are). What do people see as the norms in this realm? For example, I generally think that people who write and review this type of work are accustomed to seeing an author explain why they worked with a translator from a particular social group; that they allowed participants to choose interview sites for safety; how they coped with challenging power dynamics; and whether or not interviews was recorded. In my experience, the simple phrase “interviews were not recorded” potentially signals information regarding the sensitivity of the topics discussed, a researchers’ attention to participants’ comfort and safety, and the type of material a researcher may have gathered as a result. Scholars may discuss their position in the field or how their personal status influenced research (gender, nationality, and race being three of the most frequently mentioned factors). People may ask what scholars told research participants about their reasons for choosing certain topics, their funding sources, the potential moral hazards embedded in their work, or their professional goals (an aspect of openness and honesty that has remained relatively untouched in these discussions). In other words, the idea of transparency—towards other scholars and towards participants--doesn't seem to be a new concept for scholars who study violence and contentious politics. However, it does feel as though we may often use different language to discuss its various aspects and face varying incentives in the publication process (see Sean Yom’s comments elsewhere on this site).

Due to the intellectual, political, and physical difficulty of much violence research, I’d infer that many of us are most interested in the evaluating the conditions of possibility for a scholar’s claims rather than reviewing their “data” in the form of transcripts. Data—including interviews and field notes--can be faked by those intent on doing so. It’s much harder to convincingly fake depth of field experience/knowledge or an ethical sensibility. How do people feel about these statements?

When I review articles, particularly those based on fieldwork in violent contexts, I have been trained to look for factors such as the relationship between the time a scholar spent in their field site and their stated methodological approach (e.g. if they were on the ground for a grand total of two weeks and use the term “ethnography”); any geographic vagueness or knowledge of geographic variation/specificities; whether the researcher spoke the local language; if/how they describe any translators, RAs, or facilitators; where they lived; where they conducted interviews; and how the political environment might have influenced their work (e.g. the final stages of my own book research in Lebanon were carried out during the Syrian Civil War, which influenced what topics I could safely discuss). What do you look for to evaluate this type of work?

If a researcher emphasizes trust and the intimacy of their relationships, I then look for evidence of that closeness (e.g. repeated interviews/interactions, access to people’s homes, presence at invite-only events, empathy). If they mention in-depth interviews or ethnography, I look for appropriate citations, but do not expect a primer on methods explained in detail elsewhere. If the goal of openness/transparency/explicitness is to foster evaluation and future research, what kinds of evaluation approaches serve these goals?

For violence scholars (and, again, plenty of others) who conduct intensive or immersive field research in any methodological tradition, there seems to be agreement that it is often impossible (and frequently undesirable) to capture the entirety of field experiences in one article or one book. To paraphrase many: How does one even begin to distill ten years of engagement with a field site into a concise explanation of how it influenced a discrete analytical claim? What might more realistic expectations entail?

I wonder if there are unarticulated reasons that many violence scholars naturally produce a body of work that emphasizes theory, method, and ethics to differing degrees in different publications (but overall paints a picture of the scholar's approach); specifically, I wonder if there are lessons to be learned for other topical foci. Fujii, for example, has several publications on research methods/ethics that can be read alongside her book. So do violence scholars such as Elizabeth Jean Wood, Timothy Pachirat, and Jesse Driscoll (among many others too numerous to list). To me, the complementarities between openness/method/ethics/evidence/claims are more productively and deeply explored if the researcher has flexibility in how to frame and write about them, rather being forced into a single model. How would you respond?

Re: [From Steering Committee] What is openness/transparency/explicitness in violence/contentious politics research?

Posted: Tue May 17, 2016 12:43 pm
by mneesha
Sarah Parkinson’s summary above of the gathered responses to date resonates with my own reading of this site (many violence scholars and ethnographers are strongly opposed to data sharing DA-RT-style), and I want to add my own thoughts on a few of the questions she poses.

What are the norms for engaging with issues of openness, transparency, honesty and ethics for ethnographers? Parkinson lists many of the central features that I also look for, which center around many different kinds of explanation. Authors need to explain (sometimes very briefly and with keywords, depending on the kind of publication) many of the aspects of how they carry out their ethnographies: for how long, with whom, why translators or not, why these respondents and not others, reflexivity and positionality, etc, as she describes well above.

I personally look for a match between the methods, the characteristics described, and the type (level and depth) of analysis derived (ie white male with translator doing ethnography in rural indigenous female Mexican cooperative should explain limitations of positionality and how there are levels of conversation not being captured, and the study shouldn’t purport to explain “how rural indigenous women think” without deep reflection on limitations).

I also look for overly perfect research design and findings. Politics and communities are messy, so publications that report unblemished processes tend to make me concerned. Was ALL the information really available? The researcher had access to everything they needed and it all matched up perfectly? Regarding honesty, I think admitting study limitations, mistakes, and absences is an important way to fully account for what the study may not be able to do, and then to explain why it can’t. I personally would like to see more of this in published work, as I think the quest for perfection in publishing actually pushes authors to be less honest, and this is an element of academic culture that this deliberate process has the potential to address or at least begin related conversations about.

Regarding transparency, Parkinson’s statement that many of us are more interested in “evaluating the conditions of possibility for a scholar’s claims rather than reviewing their “data”” resonates strongly with me. Brief but comprehensive explanation is vital, as ultimately it is the researcher’s credibility (not just reputation but their ability to articulate the inglorious minutia of their project), rather than their raw data, that gives me confidence they are analyzing what they see accurately. As Parkinson and many other contributors to QTD have pointed out, providing raw data does not prevent falsified data, but comprehensive reflection on research design and honest reflection on problems within the project, through explanation, can provide the kind of transparency we need. Many people are already doing this, but the DA-RT conversation has pushed us to articulate it.

However, one concern I have is that our potential consensus on the need for articulation will lead to a new “magic formula” for explanation. I think that heterogeneity in approaches is the way to honor the diverse range of methods and contexts we all engage in. Thanks for keeping such an important discussion going