Substantive Dimensions of the Deliberations
We encourage contributors to the Discussion Board to publicly identify by registering and logging in prior to posting. However, if you prefer, you may post anonymously (i.e. without having your post be attributed to you) by posting without logging in. Anonymous posts will display only after a delay to allow for administrator review. Contributors agree to the QTD Terms of Use.
Instructions
To participate, you may either post a contribution to an existing discussion by selecting the thread for that topic (and then click on "Post Reply") or start a new thread by clicking on "New Topic" below.
The transition to Stage 2 of the deliberations is currently underway but will take some time to complete. In the meantime, we very much welcome additional contributions to the existing threads in this forum.
For instructions on how to follow a discussion thread by email, click here.
-
Nancy Hirschmann
The University of Pennsylvania - Posts: 6
- Joined: Wed Apr 06, 2016 1:59 pm
CP newsletter: editorial or ideology?
Post Reply
-
Paolo Spada
Southampton - Posts: 11
- Joined: Wed Apr 06, 2016 8:24 pm
Re: CP newsletter: editorial or ideology?
Post Reply
-
Mala Htun
Univ of New Mexico - Posts: 16
- Joined: Wed Apr 06, 2016 9:20 am
Re: CP newsletter: editorial or ideology?
http://comparativenewsletter.com/da-rt
Post Reply
-
Rudra Sil
University of Pennsylvania - Posts: 6
- Joined: Thu Apr 07, 2016 4:50 pm
Re: CP newsletter: editorial or ideology?
http://comparativenewsletter.com/files/ ... ng2016.pdf
Post Reply
-
Marcus Kreuzer
Villanova University - Posts: 26
- Joined: Sat Apr 16, 2016 9:48 am
Re: CP newsletter: editorial or ideology?
1) Most people posting work in a qualitative tradition and generally we share horror stories of how our qualitative colleagues butchered a particular case study, lack contextual understanding, overlook boundary conditions, and generally engage the type of historical tourism that makes theorizing easy if not necessarily valid. I wonder to what extent such horror stories are little more than earlier incarnations of DA-RT. Qualitative scholars might be transparent when it comes to their data collection and analysis but they generally lack transparency when it comes to their theorizing and other pretesting activities. They take full advantage of the logical positivists division of analysis falling into a domain of confirmation and a domain of discovery where the latter is so serendipitous that it does not require any research transparency. But the entire push for process tracing is to go past this false dichotomy and require quantitative scholars to demonstrate greater research transparency in their theorizing. DA-RT, if broadly defined, could inconvenience (or improve depending of where you stand) quantitative scholars just as much as qualitative ones.
2) I also think that it is important to place DA-RT in the broader context where the natural and social sciences have been experiencing what is widely referred to as a replication crisis. Christie Aschwand captured the underlying problems by observing that it is easier to get results than it is to get answers. I would highly recommend two articles by her posted on FiveThirtyEight: Science isn't Broke and http://fivethirtyeight.com/features/sci ... nt-broken/ and Failure is Moving Science Forward http://fivethirtyeight.com/features/fai ... e-forward/
Post Reply
-
Rudra Sil
University of Pennsylvania - Posts: 6
- Joined: Thu Apr 07, 2016 4:50 pm
Re: CP newsletter: editorial or ideology?
Post Reply
-
Marcus Kreuzer
Villanova University - Posts: 26
- Joined: Sat Apr 16, 2016 9:48 am
Re: CP newsletter: editorial or ideology?
Your and various other posts imply the DA-RT involves imposing quantitative transparency standards on qualitative research, a kind of KKV déja vue all over again. I can share this concerns with some of the requirements especially the data repository/access requirement.
But is there not another way of looking at this.
Couldn't you see a scenario where, if qualitative transparency standards are formalized and made more explicit (we are still quite a ways from this), quantitative scholars would see DA-RT just as much as an imposition on their research as qualitative scholars do right now. Quantitative scholars increasingly do mix-method research and, as you and Amel Ahmed have pointed out elsewhere, the case studies in this research frequently don't meet the transparency standards of qualitative scholars. Furthermore, analytical transparency also involves more detailed discussion regarding the theorizing and test construction stage whose quality once again requires detailed qualitative knowledge. Bottom line, quantitative research involves more than just data collection and data analysis. Those pre-testing activities are covered by the transparency standards widely used by qualitative scholars. So the inconvenience (or benefits) that DA-RT imposes will be symmetrical if qualitative scholars succeed in formalizing their transparency standards more fully.
Post Reply
-
Rudra Sil
University of Pennsylvania - Posts: 6
- Joined: Thu Apr 07, 2016 4:50 pm
Re: CP newsletter: editorial or ideology?
You mentioned my piece with Amel Ahmed - well one of the things we emphasized was that much qualitative research has no business being mixed with quantitative work given the quite different fundamental objectives and foundational premises. Thus, we argued that the world of mixed-method research, if it is turned into a standard practice, will basically accommodate only one kind of qualitative scholarship, that proceeding from a worldview that has more in common with that of quantitaive work (i.e. small-N, Mill-style comparison, or process-tracing done explicitly to match the variables that seem to be most strongly correlated in a statistical analysis). Here, we have a DA-RT program that not only assimilates qualitative work to quantitative work (at least enough so that the kind of data from field notes is seen as similar) on the grounds that more transparency is required for replicability. True, that can apply to certain forms of qualitative work. But, not all. More importantly, in a Lakatosian world, far more important than replication of a single study would be to develop theories backed by evidence that simply are more convincing and explain more - people in various qualitative traditions do that by challenging each other's arguments on the basis of different kinds of empirical data. I fail to see what will be added to this process that justifies making a junior scholar struggling to fund fieldwork at a small liberal arts college find the time and energy to scan all the field notes (while carefully treading around IRB requirements) into repositories. There will never be perfect transparency in this business, but efforts to push the envelope a little further carry more risks than benefits for the whole range of research communities that some of us wish to keep afloat within a more inclusive political science.
Post Reply
-
Robert Kaufman
Rutgers University - Posts: 1
- Joined: Thu May 19, 2016 2:39 pm
Re: CP newsletter: editorial or ideology?
I believe that the debate over DA-RT has been a very healthy one, which has been joined with professionalism and good will by participants on all sides of the issue. This spirit was very well reflected in the recent symposium in the Comparative Politics Newsletter, and I would like to weigh in with a few thoughts of my own. At the outset of the debate about DA-RT, I was inclined to be sympathetic to the effort to specify the norms of knowledge production in our discipline, and I believed (and still do) in the sincerity of promises to respect the diverse research traditions in comparative politics. Upon further reflection, however, I have come around to the view, expressed by several critics who participated in the CP symposium and in the contributions by the editors of CPS and World Politics, that the costs of the enterprise outweigh the potential benefits.
In principle, the DA-RT initiative pursues the laudable goal of making the empirical bases of knowledge claims clear and available to other researchers. Moreover, as several participants have noted, providing a public accounting of the data and methods should be an appropriate step for quantitative researchers, although there is some room for legitimate debate about the time frame in which original data sets should be made available to others. The problem comes with proposals to extend and adapt these norms to the various approaches to qualitative research. Efforts to do so may be both possible and desirable in some instances, but the professional disincentives, ethical limits, and practical costs – particularly to young researchers – can be overwhelming.
To me, the most critical stumbling block is the Journal Editors Statement (JETS). It is one thing for members of the Association to debate and even try to frame norms of data access and research transparency. It is quite another for journal editors to add another filter to the already demanding gauntlet of peer reviews and editorial judgments that a paper must survive in order to reach publication. Editors, to be sure, do serve as gatekeepers, as Deborah Yashar has noted, and they have an obligation to insist on rigorous reviews. But it is far from clear that adding new requirements to the review process – even ones that are flexible and respectful of alternative research traditions – will improve on existing practices. The great risk, of course, is that it will result in a more limited and less creative research environment.
This risk is especially high when there is so little agreement about the specific standards that might be generally applicable across the research communities within comparative politics. The evidentiary standards for arriving at and documenting inter-subjective claims need to emerge through inter-subjective processes. Such processes, to be sure, are shaped by the hierarchical structure of our profession – and probably should be. Still, for the time being, it seems prudent to rely on the norms of research transparency and data accessibility that emerge from within the diverse research communities themselves, rather than ask journal editors and their boards to elaborate additional hoops for authors to pass through.
Post Reply
-
Tim Buthe
HfP/Technical Univ of Munich & Duke University - Posts: 32
- Joined: Fri Feb 26, 2016 11:39 pm
Re: CP newsletter: editorial or ideology?
What are the aspects of research transparency for which you would most want to see an opportunity for differentiated deliberations during stage 2 of the QTD? Please don't hesitate to post even if you might only mention your top 2-3, and please be as specific as you can be. Adding a sentence or two to explain why will make your post even more valuable but such elaboration is by no means required.
Thanks!
Post Reply