Qualitative Transparency Deliberations

Interesting link. It discusses how authors manipulate on occasion the review process and what journals are doing to address it. It also claims that such manipulations are one of the reasons why we see more and more journal retractions. This raises the question of whether the peer review process shou...

Two possible solutions come to mind: 1) The author submitting a TRAX and increasing his/her transparency should be rewarded by being the right to respond to any reviewer's critiques drawn from the TRAX. This might involve restructuring the review process. Maybe reviewers would be invited to comment ...

What is epistemologically desirable with regards to research transparency and what is logistically/economically possible at the publishing end do not always line up and create a real conflict for implementing DA-RT. And it certainly seems to be a tension which the original DA-RT guidelines overlooke...

Thanks for this post which wonderfully identifies the misalignment in the current DA-RT initiative between the transparency expectations for individual scholars and the transparency disincentives of the broader production of knowledge. [For more on this see Topic 2 in this cluster] This misalignment...

Yes, my understanding is that historians use discretion in how they employ footnotes rather than following a protocol. Maybe the one exception is there insistence that footnotes have page numbers. By the way, I extrapolated those three uses from Anthony Grafton's delightful "The Footnote: Curio...

Interesting takes here on the value of whether or not to report interviews, or textual sources for that matter, that were not accessible. I think Tasha is correct to point out that process tracing and Bayesian analysis substitutes the frequency of evidence for what is called its probative value. (i....

Let me respond with respect that the problems that "analytical transparency" tries to solve. I think that the problems that data access and production transparency address are more self-explanatory. The DA-RT guidelines are particularly brief and generic about the goals of analytic transpa...

There is a widespread perception that the three transparency dimensions (e.g. data access, analytical transparency, production transparency) identified by the DA-RT initiative capture not only a narrow statistical research practices but also cover only the small, test-related stage of the broader kn...

Thanks, Hillel. I would like to underscore two important implications of your post. First, your two suggestions for unreported findings are spot on. Many of the works that I enjoy reading are enjoyable in part because they report more null findings and model specifications. Being taken along for the...

The structuring effect of the current publication template is a huge part for the limited transparency and for defining transparency narrowly in terms of sharing data files and computer codes. What the current deductive, hypothesis testing template does is to define transparency narrowly in terms of...

I would like to touch on one specific and little discussed type of evidence: unreported findings . The APSA Guidelines list under data access the importance to make available data unreported in the final article. Having easy access to such unreported data is important to counter problems like p-hack...

When you cite your sources, what types of information do you include either in-text or in the associated footnotes/endnotes in order to help readers understand how the evidence you are using supports the analytical, descriptive, or causal claims you are making? What types of information, and how mu...

This is an interesting question: in political science, I would not say that we have necessarily been trained or encouraged to include these types of observations. There also is not a lot of room, given the word limits of many journals and/or the discouraging of footnotes, to elaborate critically on...

The transparency discussion ultimately connects to practices and consequences “on the ground” in the form of evaluative criteria that journals may apply to article submissions. Our last group of questions speaks to the same terrain of possible epistemological concerns over transparency standards, bu...

Political science is multi-faceted and multi-sited field of inquiry. Diverse epistemological assumptions inform research excellence and subfield-specific practices with profound effects on the construction of the objects of research. This online discussion might help explicate the manifold epistemic...

I. Points of Departure Academic disciplines, like Political Science, are evolving and conflicted institutions and discourses. With the discursive domains and institutional pressures of such academic operations, individuals and groups struggle to make the world knowable by mobilizing certain techniq...

Given that we are among footnote aficionados here, just a quick plug for Anthony's Grafton marvelous history "The Footnote: a Curious History. " He traces the history of the footnote back to the 17th century's scientific revolution and Rene Descartes disparagement of historical knowledge a...

Thanks for pointing out the AJPS badges. While I share your concerns, the idea of badges is intriguing if the could be earned for more than just data access and replication. Let's assume that you could earn badges for the type of research transparency that is more relevant to qualitative research, t...

I like Evan's emphasis that qualitative transparency standards are still emerging and thus will require experimentation. I think experimentation could easily be extended beyond data dissemination. It would be particularly encouraging if journals themselves would lead the way with several possible ex...

This is a response to Alan's four posts up where he pulled together other posts on whether journal editors or reviewers should be transparency gatekeepers. Here are a few suggestions on how DA-RT could be integrated more fully in the review process and thereby shift transparency evaluations from edi...

Different Model of Research Transparency : DA-RT front-ends research transparency by trying to make the review process more rigorous. At a recent CES roundtable, Gary Marks (UNC, Chapel Hill) pointed out that traditionally research is evaluated after publication through further replication and by r...

There are lots of suggestions made across various posts. Is there any way to pull them together in a single posts or even hyperlink them?

Dear Rudy, Your and various other posts imply the DA-RT involves imposing quantitative transparency standards on qualitative research, a kind of KKV déja vue all over again. I can share this concerns with some of the requirements especially the data repository/access requirement. But is there not an...

FOOTNOTES! I think we need to underscore the importance of proper and consistent footnotes as a transparency practice. Ideally, these would include page numbers where the information came from, which too many texts these days do not do. mala Two additional suggestions on footnotes: 1) The historian...

Two quick observations regarding the point Nancy raises of whether the CP Newsletter embodies a certain ideology: 1) Most people posting work in a qualitative tradition and generally we share horror stories of how our qualitative colleagues butchered a particular case study, lack contextual understa...

Regardless of their methodological orientations, scholars differ tremendously in how they specify their test and alternative hypotheses, that is, they vary in how they position their causal inference in the broader literature. If I correctly understand the DA-RT guidelines, this would fall under ana...