Qualitative Transparency Deliberations

First, a note of thanks to Tim Büthe and Alan Jacobs and others for their hard work building this discussion. These posts have been valuable in helping explain the possible impacts of this initiative. They also raise new questions. Since I’m writing in response to Andrew Bennett’s email request for graduate student comments, I’ll post under the “Graduate Student concerns” thread, but as James Heilman said, graduate students have mostly the same concerns as faculty. My three points below build on the excellent points made by others.

1. A solution in search of a problem? Unpacking the problem(s)

I am sympathetic to those who suggest DA-RT may be a solution in search of a problem. It’s hard to know how to participate in this discussion without engaging with this question first. For example, some have suggested that these discussions will increase the quality (or perhaps the credibility) of qualitative research in general, easing the path to publication. Others suggest just the opposite, that DA-RT would place additional burdens on qualitative researchers and do little to increase the quality of their work, and potentially harm the quality. If the problem is quality, then is it that: (1) poor quality research is getting published? (2) good quality research isn’t getting published? (3) good quality research isn’t getting produced? (4) good quality research is being produced and even published, but authors feel they have to misrepresent how they did their work? (5) Or is it the problem something else, maybe not related to quality at all? Posing questions like these might shift the discussion beyond changes individual researchers and journals might need to make, to consider other intervention points in the system. For example, although some have discussed potential effects of DA-RT on graduate training, what changes might be made to graduate training to address the underlying problems that motivated DA-RT?

In general, I’m wondering whether explicitly unpacking the problem(s) would help contextualize potential solutions, and perhaps prompt new ideas beyond data access and transparency.

2. Unintended Consequences: Potential chilling effects, mutually reinforcing each other, and potential implications for graduate training

I appreciate and agree with the posts from those who raise concerns about unintended consequences. DA-RT might disincentivize scholars from tackling certain types of research, as several posts suggested. Potentially, this could mean fewer experts available to teach qualitative methods courses. Other posts suggest DA-RT could disincentivize graduate students from pursuing qualitative dissertations, an effect that seems highly likely given the pressure to complete degree programs quickly. Potentially, this would mean fewer students taking qualitative methods courses. Departments may respond by reducing their qualitative courses offerings. For some departments including my own, would that mean going from one elective course in qualitative methods to… none? It’s particularly worrisome that these effects may reinforce each other, undermining the discipline’s capacity and willingness to tackle complex research questions where qualitative and inductive approaches excel.

3. Against requirements. For reflections. And for new methods scholarship.

I agree with the “against requirements” post by Kurt Weyland, and others suggesting informal and voluntary guidelines. It’s seems too likely that a set of formalized standards and requirements, whether these are intended to guide journal editors, scholars or students, risks being overly burdensome, have unequal impacts, is likely to squelch innovation, and could even be misused.

The exchange of posts between Margaret Keck and Alan Jacobs and others is one alternative. Identifying good examples of methodological reflections like Keck’s could be one of the most useful outcomes of these discussions for those of us new to qualitative research. Perhaps examples like this could be highlighted in the next generation of methods books and articles, ones that could be referenced when and where applicable.

Thank you; much appreciated. I look forward to seeing future posts from you and your colleagues.

Thanks for the questions. I'll be sure to raise them with my peers as well as think on them and post a lengthier reply on here.

Thank you for sharing these concerns and reflections, James. Very much appreciated.

On behalf of the Steering Committee, let me also encourage you and other graduate students to go beyond responding to DA-RT and think of the deliberations on these pages as an opportunity to participate in shaping our professional norms more broadly. You might do so by speaking directly to questions such as: What kind of transparency practices (or "explicitness" practices, as Craig Parsons has suggested) do you value in other scholars' work? What kind of transparency practices would you like to see for the kind of research you do in your own dissertation? And what do you see as the limits of transparency for that kind of work?

I haven't seen a topic started by a graduate student yet or one that is specifically about graduate students concerns (but there is a good one about how faculty might have to change graduate student training: viewtopic.php?f=10&t=67) so I figured I'd start one and hopefully encourage more graduate student responses. As a graduate student who has talked with peers in my department and other universities about DART my impression is that graduate students have the same concerns as faculty posting on here about protecting sources, the extra time required to make qualitative data transparent and the potential new divisions and reinforcement of existent divisions between methodologies and subfields in political science. However, there might be some issues that are of special concern to graduate students and junior faculty.

I recently got an email conversation started amongst graduate students in my department and there is a wide-spread concern amongst us all about how DART will affect our career opportunities that are opened up based on publications. There is a desire from us graduate students to know more about DART and so we have recently started to figure out what type of roundtable discussions we can have with our faculty so that we are better informed about the practical implications of DART and how we can do our best to be as transparent as possible. Some of us are concerned about how DART will affect our likelihood of publishing in top-tier journals. Publication in these journals is a big positive on our CVs when we apply for jobs after finishing our dissertation and will remain important as we work towards tenure. This is especially important for graduate students who are not completing PhDs at the best of the best graduate programs. Grad students from outside the most well-respected programs know that publication in a well respected journal in combination with an excellent dissertation will make their applications competitive with graduate students from top-tier programs.

I know a lot of discussion on here has been about how DART can reinforce or create new divides in our discipline, such as discouraging qualitative researchers from publishing in certain journals. I just want to emphasize this can have effects on the career opportunities of graduate students.

I also would like to know more about the specific transparency practices that could create problems specifically for graduate students. One graduate student (in the post linked to at the beginning of this post) mentioned that most graduate students are learning on the fly when they are doing field research. Learning how to take field notes, compile them and knowing what data needs to be gathered in order to answer a question are already difficult tasks. Having to then put this all in a format that can be made available to others is another task that will require time that could be spent on finishing a dissertation. This is not an argument to exempt graduate students but to point out one way in which DART specifically affects graduate students.