Qualitative Transparency Deliberations

[Nikhar Gaikwad Yale University]

Thank you for these very helpful suggestions, Taylor.

This is Taylor Boas from Boston University, posting as "Guest" since I don't have a login. I'm just reviewing this discussion now. There are a lot of ideas circulating, but I think one of the clearest concerns how to convey details regarding textual sources (or any type of non-library source) given space constraints. In my mind, the answer is clear: online appendices. These have become almost de rigueur in quantitative research given the vast number of alternative specifications, robustness tests, and so on that scholars are expected to present, but which don't fit in the main body. The same could easily be done for details on qualitative sources. Many journals host appendices on their websites, with a link right next to the full-text link for the paper. Since these are non-copyrighted, they can be legally duplicated on a scholar's own personal space (and, in the case of journals that don't host appendices, a scholar's personal web space provides an alternate home). Those publishing in journals that don't host appendices and who lack web space and don't want to pay for it could upload an appendix to a repository such as Dataverse.
Marcus Kreuzer points out that detail in Appendices will be less accessible than footnotes, which is true, but one can minimize the burden with a) a short footnote (or textual mention) in the article stating that more detail is in the Appendix; b) a clearly stated location for the Appendix (e.g., URL of the author's website) in the article's "page 1" footnote alongside acknowledgments and such; c) a detailed table of contents in the Appendix itself. Surely those seeking to visit an archive and examine the same material as the author will be willing to take the extra time to look up additional details in a separate document online.

[Nikhar Gaikwad Yale University]

I would like to follow up on this important set of questions. It is worth distinguishing between two forms of transparency-related online documentation.

The first is the online appendix. As Taylor Boas mentioned, online appendices can afford qualitative and multi-methods scholars facing publishing-related word limits with the space to provide detailed context on their sources. In cases where journal word limits preclude scholars from including lengthy footnotes, scholars can provide shorter footnotes within the text of the article, while also directing readers to an online appendix containing additional information related to data access, production transparency, and analytical transparency. One drawback about this practice, as some discussants have pointed out above, is that it impedes access to information; readers are required to navigate a second and potentially cumbersome online document to get access to additional contextual information regarding data sources. Taylor points out that this might not be too big a cost, since only interested readers (for example, those seeking to revisit the source documentation in the relevant archives) would need to review the online appendix. By contrast, general interest readers would be able to access abridged information about the article's sources within the main article itself. An alternate approach, as suggested by Marcus Kreuzer above, would be to require journals to increase word limits so that lengthy footnotes can be included within the text of the manuscripts.

The second form of online documentation pertains to a replication repository. This would be analogous to the replication dataset that most journals now require as a condition for publishing quantitative studies. The initial suggestion here was for qualitative scholars to provide copies (scans, photographs, etc.) of the source materials that they rely upon for making evidentiary claims. We have already heard a number of opinions on this board about the pros and cons of creating replication repositories for qualitative and archival materials. For example, some archives have strict copyright policies that preclude the dissemination of original sources. Additionally, the time and effort required for digitizing and circulating certain forms of qualitative sources might place an undue burden on scholars, especially junior scholars and graduate students.

My sense is that it is worth evaluating the merits and demerits of both forms of documentation separately. As the guest post below aptly notes, the motivating question for us to ask is whether and how these practices help make our work more transparent. Please refer to the discussion on the benefits and costs of transparency for additional opinions related on this topic (see, e.g., https://www.qualtd.net/viewtopic.php?f=17&t=140#p887 and https://www.qualtd.net/viewtopic.php?f=17&t=140#p795)

For text-based sources, what do you imagine having in the online appendix?
Would it be a description of the sources used and an extended discussion of how the scholar assessed the evidentiary value of different documents or authors?
Would it be a way to shorten footnotes in the article itself and provide full documentation online?
Would it include images of documents or other text sources?

I think the online appendix is an intriguing idea, but am not sure if it solves the fundamental problem that the assumption is that all scholars have the time/resources to take scans or images of all their documents, nor does it answer whether such an endeavor actually makes our work more "transparent."

This is Taylor Boas from Boston University, posting as "Guest" since I don't have a login. I'm just reviewing this discussion now. There are a lot of ideas circulating, but I think one of the clearest concerns how to convey details regarding textual sources (or any type of non-library source) given space constraints. In my mind, the answer is clear: online appendices. These have become almost de rigueur in quantitative research given the vast number of alternative specifications, robustness tests, and so on that scholars are expected to present, but which don't fit in the main body. The same could easily be done for details on qualitative sources. Many journals host appendices on their websites, with a link right next to the full-text link for the paper. Since these are non-copyrighted, they can be legally duplicated on a scholar's own personal space (and, in the case of journals that don't host appendices, a scholar's personal web space provides an alternate home). Those publishing in journals that don't host appendices and who lack web space and don't want to pay for it could upload an appendix to a repository such as Dataverse.
Marcus Kreuzer points out that detail in Appendices will be less accessible than footnotes, which is true, but one can minimize the burden with a) a short footnote (or textual mention) in the article stating that more detail is in the Appendix; b) a clearly stated location for the Appendix (e.g., URL of the author's website) in the article's "page 1" footnote alongside acknowledgments and such; c) a detailed table of contents in the Appendix itself. Surely those seeking to visit an archive and examine the same material as the author will be willing to take the extra time to look up additional details in a separate document online.

[Nikhar Gaikwad Yale University]

Several scholars weighed in on practices related to the documentation of text-based sources in Stage I of the QTD process. I have included below links to some posts that are relevant to the topics discussed in this thread:

Active citation versus the meaty footnote: Response 1, Response 2, Response 3, Response 4, Response 5, Response 6, Response 7, Response 8

DA-RT: effect on graduate training

Graduate student concerns

Data access--discourage original data production?

Who are the gatekeepers? Editors or reviewers?

Data access and “right to first use” of newly collected quantitative data