I.1. Ontological/Epistemological Priors

Forum rules

We encourage contributors to the Discussion Board to publicly identify by registering and logging in prior to posting. However, if you prefer, you may post anonymously (i.e. without having your post be attributed to you) by posting without logging in. Anonymous posts will display only after a delay to allow for administrator review. Contributors agree to the QTD Terms of Use.

Instructions
To participate, you may either post a contribution to an existing discussion by selecting the thread for that topic (and then click on "Post Reply") or start a new thread by clicking on "New Topic" below.

For instructions on how to follow a discussion thread by email, click here.

Marcus Kreuzer
Villanova University
Posts: 26
Joined: Sat Apr 16, 2016 9:48 am

Topic 3: Elaborating Multiple/alternative Journal Criteria

PostWed Sep 07, 2016 9:32 pm

The transparency discussion ultimately connects to practices and consequences “on the ground” in the form of evaluative criteria that journals may apply to article submissions. Our last group of questions speaks to the same terrain of possible epistemological concerns over transparency standards, but attempts to focus the discussion on criteria that journals could apply—asking participants to point out criteria that don’t fit their work or to sketch others that do.
[1] Do you prefer to evaluate the validity or significance of your own work (or of whatever you see as good, publishable work) in ways that do not fit with visual metaphors of transparency? Can you describe a feature of your work that you perceive might lead it to being disadvantaged by transparency standards as you understand them?
[2] Thinking of whatever journal or press you think best fits your work, is it possible and/or desirable to draw up criteria for the production and presentation of good research for that venue (allowing, perhaps, that such criteria would define typical or common features of good research of this sort, without insisting that all good and publishable research had all such features)? If so, please describe at least one such criterion and what it would look like as a statement of that journal’s policy.
[3] Would you like to see the editors of journals or presses you respect presenting their venues and related rules/practices as participating in a more expansive and pluralistic version of a transparency initiative; as pursuing a separate but in some ways parallel initiative to elaborate standards for the kind of research you value; or would you prefer that editors/journals/presses avoid taking any such steps at this time? Please describe how your preferred option might work.
If you respond to any specific question, please identify it as Question 1, 2, or 3. This will help structure the discussion.

Post Reply

Zoe Marks
University of Edinburgh
Posts: 3
Joined: Mon May 09, 2016 7:37 am

Re: Topic 3: Elaborating Multiple/alternative Journal Criteria

PostTue Dec 27, 2016 8:09 pm

I'd like to offer a few thoughts that have arisen in conversation with colleagues and which I haven't (yet) seen elsewhere on the forum. They seem to loosely fit here...
By way of introduction, my research is on armed groups, gender, violence, and post-conflict development; I rely primarily on extended/repeat qualitative interviews and fieldwork, found archives, and more recently, quantitative social network and survey data in sub-Saharan Africa.

In response to Q[2], I'm uneasy with the idea that there would be an ever-narrowing set of journals in which my work, particularly vis-a-vis epistemological and methodological approach, is welcome and publishable. Often, the DA-RT discussion has been worrying in that it seems to be compounding the silo'ing of research, and potentially further ghettoising qualitative research and limiting the frequency with which epistemological and ontological cross-pollination can occur within the pages of a single journal. I regularly consult the pages of, e.g., APSR, JCR, JPR, World Politics, African Affairs, and Signs, when working on violence research, despite all having different intellectual gestalts. I hope any DA-RT driven policies and norms stridently avoid further formalising and entrenching the centrifugal forces that already pull our research out of conversation with one another.

Perhaps to the latter part of Q[2], then, I'd like to see journals officially promote *adaptive* transparency policies that prioritize: (1) intellectual clarity, vis-a-vis communication internal and external to the discipline; and (2) ethical rigour - both to the premise of knowledge production, and to research subjects and participants. To Q[3], I welcome a more expansive, pluralistic, and rigorous conception of transparency, which I don't see requiring a new initiative, but rather, statements of clarification or revision. Given the conversation DA-RT has generated, I think inaction would be a missed opportunity. Statements and policies, to the extent that they have changed or are changing in the past year, should be proactive in articulating the ethos and willingness of journal editors and other gatekeepers to cultivate best practices in concert with their authors, and to share insights, compromises, and innovations devised to preserve clarity, concision, and efficient use of academic time/talent/resources.

Finally, related to all of three questions is a concern I have about genuinely multi method work, particularly that which seeks to expand or exploit hybrid epistemologies. For positivist or positivist-leaning research, mixed method research is becoming increasingly popular, from hardcore survey and RCT econometricians, to people like myself, who hail from a qualitative and induction-friendly environment, but need more systematic and expansive data that we end up having to go generate ourselves. While many colleagues across these forums have noted the perceived double standard of quant. researchers not justifying every coding interpretation made in constructing a dataset, rather less has been said about how the qualitative historical, institutional analysis and interviews that underpin standard QMMR survey design, for example, would hop-scotch through DA-RT hurdles.

The scenario gets even more complex for work I'm currently doing, which generates hundreds of quantitative social networks (mapped by hand, ergo also qualitative), paired with in-depth interviews, participant observation, and survey data designed to be interpreted alongside the aforementioned network data. Under DA-RT guidelines, I could probably publish the cleaned, anonymised dataset from the surveys, seemingly performing 'transparency' whilst also protecting vulnerable informants in a conflict-affected country. But, it would be epistemologically disingenuous. The research is built through an iterative data collection and analysis process that relies on multiple frames of understanding and engaging with the research environment and research question. Yet, it seems implausible and unreasonable to have all data sources in mixed methods research required to meet separate, rigorous standards for each respective methodological or data-driven silo, much less articulate interrelated but separate epistemological premises.

I'm wondering if the moderators of this forum, or if others across the QTD committee have been part of conversations on mixed methods research, and what the current thoughts are with regard to state of the art? Are we to pick a dominant method and epistemological framework and perform transparency that hews most closely to our claims in a given paper? Or, are we to have a more abstract and comprehensive approach to transparency that articulates the messier aspects of the process, wherein most critical junctures are found and alternative theories/claims confirmed or rejected? (i.e. the junctures that give way to decisions that most profoundly affect the outcomes and analysis; see also, Paul Staniland's comment on the violence thread, among others' similar requests for this type of transparency.)

In its current formulation of access and transparency, DA-RT seems to deeply disincentivize and possibly pose outright barriers to multi-source, multi-method data generation and triangulation, and to the kind of work that relies on multiple or hybrid epistemologies to test the limits of our explanatory theories. I'd hate for our profession to close down some of the most potentially innovative and creative avenues of ontological, epistemological, and methodological inquiry simply because the burden of entry is too high. (Assuming undue burden when one has to devise a transparent, anonymous, and safe public repository of, e.g., private archives, interview transcripts and tapes, survey results, and social network maps...as opposed to, say, secondary data scraped from voter records, consumer reports, social media, and censuses.)

In mulling over alternative ways forward, and observing the discussion for the past few months, I'm currently most optimistic about the possibility of developing an academic version of an artist's statement, which could be linked to any peer-reviewed publications (and of which there could be different iterations, as per pluralism in a given individual's oeuvre). In a few short paragraphs, we could begin to share and overtly articulate the ontological and epistemological assumptions that underpin our work, guide our decisions, and shape how we do - or don't - interpret different types of data. Of course any given publication would have a nuts and bolts description of how data was collected and analysed, and the interpretive tools used to connect findings with theory. But, the additional piece of a researcher's précis might fill in the blanks where previous academic generations once simply knew one another and their work. As Sarah Parkinson mentioned on a thread in the first round, within our subfields, many of us know one another's work through an amalgamation of different publications, conversations, and other outlets that provide a broader view of how qualitative research has been conducted, often over years, if not decades. In an ever expanding research market, however, perhaps a friendly, purpose driven blurb could help close the gap.

Post Reply