Substantive Dimensions of the Deliberations

Forum rules

We encourage contributors to the Discussion Board to publicly identify by registering and logging in prior to posting. However, if you prefer, you may post anonymously (i.e. without having your post be attributed to you) by posting without logging in. Anonymous posts will display only after a delay to allow for administrator review. Contributors agree to the QTD Terms of Use.

Instructions
To participate, you may either post a contribution to an existing discussion by selecting the thread for that topic (and then click on "Post Reply") or start a new thread by clicking on "New Topic" below.

The transition to Stage 2 of the deliberations is currently underway but will take some time to complete. In the meantime, we very much welcome additional contributions to the existing threads in this forum.

For instructions on how to follow a discussion thread by email, click here.

Alyssa Grahame
UMass Amherst
Posts: 4
Joined: Sun May 01, 2016 4:49 pm

Transparency, openness, & ethnography in democratic contexts

PostMon May 02, 2016 10:44 am

There has been quite a bit of discussion about what DA-RT would mean for ethnographic research, and for research taking place under less than ideal conditions, such as in authoritarian regimes, or with otherwise vulnerable research subjects. Here I wish to add in some of the implications for those of us working with living populations even in “ideal” circumstances — for example, conducting ethnographic research in an “advanced” democracy.

I see DART/JETS as, at its core, a conversation about research ethics, which I suspect we don’t talk about enough in political science.

Research ethics invites thinking about our obligations as researchers. To whom are we obligated? For those of us who work with living populations, the answer is obvious: we have the obligation not to cause harm to our “human subjects.” To this end, in the US, our practices are governed by IRBs, which assess our protocols for such risks. IRBs are by no means perfect mechanisms for research oversight, for example, protocol requirements seem much more geared towards the concerns of medical and psychological studies than they do towards studies based on everyday interactions. But the process of obtaining IRB approval for my research did make me think about research ethics very carefully.

I have often grappled with the role of transparency in my own, ethnographically-driven dissertation research on the politics of the financial crisis in Iceland. My field research methods training took place in my university’s anthropology department, and research ethics were major considerations in project and IRB protocol design. Traditionally in ethnographic writing, place names and names of people appearing in the study are changed to protect confidentiality. But as ethnographic research has become more participatory, anthropologists have moved away from the old conventions. Indeed, the terminology has changed from “research subjects” to “research participants.” Along with this, there has been some movement toward using the real names of people and places, in appropriate circumstances. This is not only for the sake of transparency but also in recognition of the agency of the participants involved in the study, including acknowledging that they are not merely anonymous “subjects” but rather actual people with whom the researcher has forged an ongoing relationship.

These practices are by no means appropriate for all ethnographic studies, but they were for mine. In my IRB informed consent document, I offer interview participants the option to be represented by their full name, first name only, or I can attempt to anonymize them, if I want to quote them in my writing. In the first option, I would run the quote by them before using it attached to their full name. The protocol recognizes that anonymity and confidentiality are distinct concepts, that the former may be impossible in a context like Iceland, and that personal harm is unlikely to derive from using real names. I strongly doubt that this would be appropriate, for example, for research in Turkey that has even a whiff of potential political controversy. Even in my case, it does not make putting interview transcripts in a shareable repository ethical. For researchers working with real people, their ethical obligations are to *those people* before and beyond obligations to research transparency or replicability.

I spent over a year getting to know my interlocutors so that they would speak freely with me. And getting to know them entailed not only repeated conversations, but also familiarity with everyday existence in their city, with the rhythms of ordinary and extraordinary politics, with the way that prices in the grocery store rose or fell with the exchange rate. I would not expect someone without the same familiarity with my field site (and not just the site as such, but at that particular time) to glean the same interpretations from my interview transcripts as I would.

Furthermore, my interviews capture what my participants were thinking at that time. It is entirely possible that someone could go and interview the same people, asking the same questions, and find that those people have revised their positions and thinking since that time. I know I have. My research may be not be conducive to replication, but that does not make it invalid. Rather, it makes my findings a product of a particular time, place, and interactions that are also intimately linked to my own positionality as a researcher.

Iceland is a country of 320,000 people. It is widely considered to be one of the most democratic countries in the world. Yet anonymity is nearly impossible there. It is a society that highlights the advantages — and disadvantages — of transparency because in some ways it is a fact of life. Before the 2008 financial crisis, public protest was seen by many as socially undesirable and somewhat shameful. Some of my participants described wearing ski masks or skulking in the back of the initial post-crisis demonstrations so that they would not be seen, either by others in the crowd or by falling into scenes captured and circulated by the media. Among other concerns, they feared retaliation from their employers, who might hold divergent political opinions or disapprove of contentious practices. Those who did not see themselves as dependent upon Icelandic society for their livelihoods — often artists — became the most visible organizers of collective action. Similarly, I found that many people were reluctant to share their perspective or opinion about topics on which they did not feel they had the authority to speak, even though, as it often turned out, they did have a valuable perspective to share! The point is that transparency does not necessarily promote openness.

I do not believe that this project would have been possible if part of the expectation was that interview transcripts and field notes would have been made publicly accessible. In this project, I believe that there is no way to remove identifying information from an interview transcript while retaining any meaningful data.

For all of their limitations, the advantage of IRBs is that they approach research ethics on a case-by-case basis. A protocol that might be appropriate for one study may not be appropriate in the next. In describing my own project, I have tried to show the ways in which I attempt to navigate the ethic of transparency. Giving participants a choice in how they are represented in my work is one way of accomplishing greater transparency, not only for my audience but also for my participants. But the local conditions, which I only learned about as a consequence of taking part in Icelandic life, contraindicate sharing whole transcripts or field notes. Can DA-RT standards be made sensitive enough to these considerations without imposing another layer of bureaucracy on researchers working with living populations? I suspect that the answer is no.

Post Reply

Kathy Cramer
University of Wisconsin-Madison
Posts: 4
Joined: Fri May 06, 2016 2:10 pm

Re: Transparency, openness, & ethnography in democratic contexts

PostFri May 06, 2016 2:39 pm

This is beautifully said. I also believe that my work would not be possible if I had to ask the people I spent time with to gather data had to give me permission to make the transcriptions of our conversations publicly accessible. I recently completed a project that required that I repeatedly walked into gas stations, diners, restaurants and other gathering places, introduced myself, and asked for access to conversations, as well as permission to record them for transcription purposes. I could have added in a request to make those transcriptions publicly available, but I would have felt arrogant and overly intrusive doing so. Also, I suspect they would have laughed me out the door.

Part of what I was studying was attitudes toward higher education. I routinely encountered a perception that university faculty are aloof, arrogant, and take but do not give back. It would have done none of us a service if my first few minutes wth them actually reinforced that perception.

Post Reply

Nicholas Rush Smith

Re: Transparency, openness, & ethnography in democratic contexts

PostMon May 09, 2016 7:52 pm

I agree with the comments on this thread that even for researchers working in open, democratic contexts there should be concerns for researchers about DA-RT. Many of the threads in this deliberative process have raised concerns about doing work in authoritarian or violent contexts – concerns that are absolutely valid. One inference that DA-RT proponents might draw, however, would be that work in democratic contexts would not generate challenges for human subjects protections were DA-RT to be implemented. Yet a recent controversy around data access suggests that even in democratic contexts there may be ethical challenges created by data transparency and that such ethical challenges make it very difficult to give research subjects a fair assessment of the risk they would face by participating in a study – a risk assessment that would likely become even more difficult were DA-RT to become the model for circulating data.

By now many are familiar with discussions concerning Alice Goffman’s celebrated but controversial book, “On the Run,” about how being on the run from the law shapes the life paths of young African-American men in a Philadelphia neighborhood. The book first came to national prominence because of the new light it shed on the American carceral state and its effects. The book reappeared in national news after a series of articles criticizing Goffman’s research activities and the veracity of some of the accounts in the book. Much of the criticism circulated around issues of research transparency as some of the accounts in question could not be verified because she had destroyed her field notes to prevent them from being subpoenaed upon publication of the book.

There is much to say about the book and how it relates to data access. For the purposes of this discussion, however, I would like to focus on one point: the difficulty of subject anonymization and, specifically, that even when utilizing best practices which go beyond the standards typically deployed in political science it may still be possible to identify research subjects. Much discussion of DA-RT has circulated around the challenges related to human subjects concerns and the possibility of alleviating those concerns with sufficient anonymization. How such anonymization would take place and what its effectiveness would be has been somewhat less discussed, however. The Goffman case suggests how difficult such anonymization can be.

The Challenges of Subject Anonymization

The majority of Goffman’s research was drawn from experiences among a small group of young men engaged in various aspects of the drug trade. To protect these men while still relaying detail about their lives she gives them pseudonyms – a standard practice for such ethnographic research. She also gives the neighborhood in which they live a pseudonym – a standard practice in sociology but a practice that goes somewhat beyond the anonymization standards that might be used in political science research. In other words, Goffman arguably does more to hide the identities of her subjects than most political scientists would typically do.

Despite these relatively advanced practices, at least two commentators on her book were able to determine the specific individuals about whom she wrote. In one instance, a reporter found and visited the home of one her informants, based solely on clues found in Goffman’s book about the likely location of the neighborhood in which she worked. My point is not to criticize Goffman’s anonymization practices. Again, her efforts likely went beyond what political scientists would typically do in such research. The point, rather, is to show how difficult it can be to anonymize sources, even in final published work. Part of the appeal of Goffman’s work is her command over detail – the look of streets, the color of homes – details that were confirmed by the reporter who found her sources. But these details also enabled others to determine who her sources were, undermining the anonymity typically granted to human subjects in such research.

This raises troubling questions for DA-RT and how it would be rolled out. It would not be enough to simply de-identify interviews or field notes. Identifying details, as the Goffman example suggests, can be carried in seemingly innocuous details about place, time, or housing color. This presents enormous challenges for researchers as they work to produce final published work where they are able to take time to craft narratives to protect subjects’ anonymity by obscuring details. Determining how to hide identifying details with interview or field note data at a much larger scale would present enormous logistical hurdles for researchers and would only increase the likelihood of having identifying details unintentionally becoming publicly available. In other words, while at first glance de-identifying data may sound like a straightforward practice, it is anything but.

This is particularly true in an age of rapidly proliferating access to public information on the internet (one of the individuals claiming to have identified her subjects seemingly did so via such data) to give a realistic account of the risks that subjects would face were notes or recordings to be made available. Even if a fieldworker were doing work on a less legally challenging subject, there are good reasons to provide anonymity to sources even in democratic contexts, as the first post in this thread suggested. But the increased access to information at large makes promises of such anonymity increasingly difficult to sustain amidst the rapid profusion of easily and publicly available information – a trend that DA-RT would likely perpetuate were it to become the discipline’s standard. As research is conducted under such circumstances, this places researchers in a difficult position of having to give research subjects a sense of the risks involved in a research project, while having no real basis upon which to make such risk assessments as information openly circulates. The Goffman affair, therefore, raises important issues that would need to be clarified were DA-RT to continue and were JETS to be implemented.

Post Reply

Alyssa Grahame
UMass Amherst
Posts: 4
Joined: Sun May 01, 2016 4:49 pm

Re: Transparency, openness, & ethnography in democratic contexts

PostWed May 11, 2016 2:49 pm

Thanks to both for the replies and to Kathy for the kind words. I agree with both contributors' responses and elaborations.

Both responses invite further consideration of how researchers relate to their "subjects" or "participants" in various contexts. Both Kathy Cramer's post and the Guest's post about Alice Goffman's work speak to a conceit that what researchers produce is somehow fundamentally separate from and irrelevant to the lives of the people who feature in their research. This all-too-prevalent assumption is a folly precisely for the reasons given in both replies.

These issues also raise questions about the status of truth claims vis-a-vis the value of transparency and research relationships. To expand on the point about Goffman, who in particular has been criticized for altering the timelines or certain details of her participants' lives: She has replied that this was done to protect their anonymity. Her critics insist that this somehow makes her account less "true" or "accurate" and therefore less valid and valuable as a contribution to knowledge. I think this criticism completely misses the point of her work, and of much ethnographic work more broadly.

Post Reply

Nicholas Rush Smith
CUNY - City College
Posts: 6
Joined: Thu Apr 07, 2016 12:05 pm

Re: Transparency, openness, & ethnography in democratic contexts

PostMon May 16, 2016 1:44 pm

Alyssa Grahame makes an important point about the unintended effects of the focus on the truth value of Goffman’s work and ethnographic work more broadly. I think it worth elaborating the point because it highlights two interrelated questions relevant to DA-RT. Those are, that the Goffman episode a) reveals different standards of transparency among people writing from different epistemological standpoints and that b) when such standards collide possible deleterious effects can occur, crowding out broader discussions of research findings.

As background, Goffman was accused of various forms of malfeasance in the wake of her book’s publication – from having unwittingly committed a felony to fabricating details through inconsistency in the presentation of events – the latter of which, as Grahame notes, she explained as necessary to protect subjects’ identities. The accusations were sufficiently serious that Goffman wrote a memo rebutting the allegations to an internal university board that cleared her of any wrongdoing. Nonetheless, the seeming “scandal” has played itself out on everything from professional message boards to the pages of the New York Times.

So how does this relate to DA-RT? At their hearts, the accusations are about transparency and reporting of data in published work. A number of the accusers (one anonymous and two professors of law) assert that it is basically impossible to know if Goffman fabricated data because she destroyed the data upon which her claims are based – an act she justifies as necessary to protect her notes from being subpoenaed and potentially being used against her subjects.

Intended to protect her subjects, this act and the reaction to it reveals a crucial insight about research transparency and its effects. Specifically, it shows how different epistemological communities approach data storage and transparency, with some of her critics arguing that the destruction of notes and her obscuring of details makes it difficult to fact check her research, similar to what would be common for journalists. Arguably, this difference in assumptions about best practices for data storage and sharing is what escalated the Goffman episode to the level of a scandal. The destruction of her notes – and it is unclear even if she still had the notes whether or not she would be able to share them under IRB protocols – also makes it very difficult to defend herself from her accusers, even as questions around her research practices have become nationally publicized. It is worth reiterating that the scandal has mushroomed even as an internal board at her university cleared her of wrongdoing.

There is thus an unintended effect of the questions about transparency of which it is worth taking note: the creation of an apparent scandal targeted at the researcher that has detracted from the discussion of her research findings. The focus on transparency has enabled accusations of fabrication to take the place of social “fact,” steering the conversation about her work towards its alleged lack of veracity and away from her findings. Much of the public discussion around Goffman’s book has been to examine the degree to which her ethnographic work is accurate or not – a discussion that has not faded even as reporters have been able to find the specific individuals with whom she interacted. This has come at the necessary expense of a discussion of her findings – findings that are themselves scandalous.

Goffman shows the brutal effects of hyper-policing on the lives of young African-American men, effects that negatively impact the lives of their friends and family members who may not themselves be involved in illegal activity. It is an outrageous story about the failures of the American democratic system that deserves serious discussion. But this conversation has been buried in the reporting on her book, which tends to focus on questions of transparency. In other words, the seeming “scandal” around her book’s apparent lack of transparency has come at the expense of a discussion of the real scandal: the consequences of America’s policing practices.

In reflecting on such a case, it is worth asking what effects DA-RT and JETS would have on the conversations that emerge out of the research that political scientists do and whether a focus on the transparency or research process might come at the expense of a focus on research findings. Of course, living up to ethical standards in the conduct of research and reporting of research findings are important. However, the Goffman affair shows the multiplicity of such standards and the ways in which arguments across different epistemic communities about what the appropriate standards should be might detract from larger goals of disseminating and discussing research findings.

Nicholas Rush Smith
CUNY – City College

PS In the interest of transparency, I authored the previous post on Goffman in this thread. I thought I was logged into this website and had not intended to post anonymously.

Post Reply

Guest

Re: Transparency, openness, & ethnography in democratic contexts

PostTue May 17, 2016 10:16 am

I have done a large number of elite interviews in advanced industrialized democracies. I am entirely convinced that, had I been required to inform my interview subjects that I would be making public full transcripts of my interviews, the interview subjects would have either (a) refused to talk to me, or (b) provided me with very bland, party-line answers.

Post Reply

Tim Buthe
HfP/Technical Univ of Munich & Duke University
Posts: 32
Joined: Fri Feb 26, 2016 11:39 pm

Re: Transparency, openness, & ethnography in democratic contexts

PostTue May 17, 2016 10:29 am

Thank you all for your very helpful, detailed posts.

Let me also strongly encourage everyone, but especially those of you who are tenured, to post "on the record" (i.e., logged in). It is very helpful for the Steering Committee (and maybe for others, but I can't speak for them) to get a sense not just of the range of experiences and positions but also of the range of backgrounds of the scholars participating in these deliberations. This would also help by allowing us to contact posters with follow-up questions. For instance, the last, anonymous post above (by Guest on May 17 at 10:16 am) might be more helpful as a contribution to the separate thread on interview research, but we try not to move posts without asking the authors first, and here we have no way of contacting the author.

If you have reasons not to want to participate on the record, you might consider still providing some more information, such as "PhD student at an R1 research university on the West coast" or "postdoc at a teaching-oriented university in Europe" or "assistant professor at liberal arts college in the Midwest"

Post Reply

Guest

Re: Transparency, openness, & ethnography in democratic contexts

PostTue May 17, 2016 3:12 pm

Hi Tim, Thanks for following up- I am the 5/17, 10:16 am poster-- I prefer to post anonymously as I do not have tenure, but feel free to move my post to a more appropriate location. [Assistant professor at an R1].

Post Reply