How the Rule of Transparency Can Become Self-Destructive for Political Science
Posted: Fri Dec 23, 2016 5:29 pm
Marcus Kreuzer et al. pose several thoughtful and central questions that deserve careful consideration if the DA-RT “transparency” challenge to non-positivistic methods of political science inquiry is to be fully understood. (See their post introducing this thread at viewforum.php?f=13)
Kreuzer also reminds us that DA-RT proponents have already had the APSA formally adopt a rule of methodology to govern the work of all political scientists. He writes,
As articulated in the 2012 APSA Guide to Professional Ethics in Political Science, “Researchers have an ethical obligation to facilitate the evaluation of their evidence-based knowledge claims through data access, production transparency and analytic transparency so that their work can be tested and replicated.” (See his post “Setting the Stage” at viewtopic.php?f=13&t=134)
But I argue that for Interpretive Political Science, which uses multiple methods, “testing and replication” are the key terms of which to beware.
In a letter dated 1-7-16, 20 APSA presidents expressed their concern over some of the implications for research of the Journal Editors Transparency Statement (JETS). The letter was sent to the editors of 27 journals who signed the statement. But the ideal of the replicability of research was not an object of critical examination. Indeed, they praised JETS for upholding the ideal as a part of “the long efforts to improve research transparency and facilitate replication in political science. Many of us have strongly supported these efforts.” Which efforts they characterize as “the replication and transparency movement,” at
http://www.politicalsciencenow.com/lett ... uidelines/
This letter and that ethics percept show how deeply ingrained positivistic assumptions about science are in our profession’s ethos. The assumptions are that transparency is important as a way to facilitate replication, and replication is important as a way to test and evaluate the validity of research knowledge claims. These notions are derived from a positivistic idealized model of natural science, which Clarke and Primo call the Hypothetical-Deductive Model (see my article in Perspectives on Politics*).
Consider the term “transparency” as having two significant uses. As an ethical ideal, transparency implies honesty, and no one can deny the importance of honesty in science. But as a methodological ideal, insofar as transparency implies the replicability of studies in political science, it is deeply problematic.
For the DA-RT positivists, professional ethics requires that journal articles describe their methods, and have an appendix and/or stored information that would facilitate replication. But this principle is a threat to multi-method Interpretive Political Science, both because subjects have privacy rights, and because interpretation relies very much on empathy. Empathic interpretation is a highly personal research practice. Hence, it is generally too unique to be replicated. It requires a different set of standards for verification. (For more on how unique interpretive studies are threatened by the replicability requirement see my proposed Community Transparency Statement for Interpretive Political Science on this site at viewtopic.php?f=21&t=195#p800)
Of course, fundamental to any science is the principle of Reason requiring that all studies be methodical. In line with this principle, knowledge claims from interpretive studies can be made “transparent” by discussing the methods used. Even if such studies cannot be replicated, they can be criticized. The criticism of the methods and criticism of the knowledge claims derived therefrom can be conducted both prior to publication and thereafter. Along with the experts, any informed persons can make their critiques of a study known. Through this criticism, interpretive knowledge claims can be validated in such degree as there is consensus within the community of experts. Because unique studies cannot, by definition, be replicated, replication is an inappropriate professional standard.
As I have mentioned, at viewtopic.php?f=18&t=200#p846, Alice Goffman’s study, On the Run, is an excellent example of empathic interpretation. She is fully transparent about the methods of her study. While her study is too dependent on her unique abilities and circumstances to ever be exactly replicated, her methods and findings can be criticized and validated in whole or in part.
Under the rule of replicability, a study like Goffman’s would have to be denied recognition as important for political science due to its uniqueness. Because it would exclude important knowledge, this is a self-destructive rule rather than a helpful rule for our profession.
Why is Goffman's study an exemplar of method and of important knowledge for political science? The answer is readily given within the context of David Easton’s definition of political science as the study of the political system, particularly as to how a political system is able to persist over time.^ Goffman’s study was of an aspect of the US criminal justice system (CJS). The CJS is a subsystem of the political system. It can be understood as having the function of aiding or facilitating the persistence of the political system.
Goffman’s study exposed some deleterious unintended consequences of a CJS public policy; namely, a major city’s “get tough on crime” policing policy. As knowledge for political science, her study raises the question as to whether the continuation of such a policy aids or threatens the persistence of the political system. Such knowledge is vital to the political system. If its policies are headed towards self-destruction, then it should be made aware of that. In this way, political science studies can serve as an alert system for the political system. Goffman’s study started out as a pure research project without expectation as to its results. The results were a discovery made in the course of the study.
Whether or not the city involved corrects its CJS public policy, Goffman’s study illustrates Easton’s vision of political science as being empirical, methodical, and able to produce knowledge which is of vital importance to the political system.
One of the unintended consequences of the positivistic insistence that replicability is the key to validating political science knowledge is the continuing decline of esteem for the profession among political professionals and the public. Our work is often criticized as heavy on method but light on heft. (See my discussion of this in my paper on Good Work in Political Science at https://independent.academia.edu/WilliamJKelleherPhD )
Therefore, the APSA ought to jettison the DA-RT project as a threat to the best interests of the profession. Additionally, APSA resources should be used to inform its membership of the vital role Interpretive Political Science can play in furthering the interests of the profession, especially as to its esteem in society.
William J. Kelleher, Ph.D.
*A Model Discipline, and How “Good Work” Hurts Political Science William Kelleher Journal: Perspectives on Politics / Volume 13 / Issue 2 / June 2015 DOI: https://doi.org/10.1017/S1537592715000262 Published online: 18 June 2015, pp. 446-448
https://www.cambridge.org/core/journals ... 8C738B9BA5
^David Easton, A Systems Analysis of Political Life; A Framework for Political Analysis; The Political System
Kreuzer also reminds us that DA-RT proponents have already had the APSA formally adopt a rule of methodology to govern the work of all political scientists. He writes,
As articulated in the 2012 APSA Guide to Professional Ethics in Political Science, “Researchers have an ethical obligation to facilitate the evaluation of their evidence-based knowledge claims through data access, production transparency and analytic transparency so that their work can be tested and replicated.” (See his post “Setting the Stage” at viewtopic.php?f=13&t=134)
But I argue that for Interpretive Political Science, which uses multiple methods, “testing and replication” are the key terms of which to beware.
In a letter dated 1-7-16, 20 APSA presidents expressed their concern over some of the implications for research of the Journal Editors Transparency Statement (JETS). The letter was sent to the editors of 27 journals who signed the statement. But the ideal of the replicability of research was not an object of critical examination. Indeed, they praised JETS for upholding the ideal as a part of “the long efforts to improve research transparency and facilitate replication in political science. Many of us have strongly supported these efforts.” Which efforts they characterize as “the replication and transparency movement,” at
http://www.politicalsciencenow.com/lett ... uidelines/
This letter and that ethics percept show how deeply ingrained positivistic assumptions about science are in our profession’s ethos. The assumptions are that transparency is important as a way to facilitate replication, and replication is important as a way to test and evaluate the validity of research knowledge claims. These notions are derived from a positivistic idealized model of natural science, which Clarke and Primo call the Hypothetical-Deductive Model (see my article in Perspectives on Politics*).
Consider the term “transparency” as having two significant uses. As an ethical ideal, transparency implies honesty, and no one can deny the importance of honesty in science. But as a methodological ideal, insofar as transparency implies the replicability of studies in political science, it is deeply problematic.
For the DA-RT positivists, professional ethics requires that journal articles describe their methods, and have an appendix and/or stored information that would facilitate replication. But this principle is a threat to multi-method Interpretive Political Science, both because subjects have privacy rights, and because interpretation relies very much on empathy. Empathic interpretation is a highly personal research practice. Hence, it is generally too unique to be replicated. It requires a different set of standards for verification. (For more on how unique interpretive studies are threatened by the replicability requirement see my proposed Community Transparency Statement for Interpretive Political Science on this site at viewtopic.php?f=21&t=195#p800)
Of course, fundamental to any science is the principle of Reason requiring that all studies be methodical. In line with this principle, knowledge claims from interpretive studies can be made “transparent” by discussing the methods used. Even if such studies cannot be replicated, they can be criticized. The criticism of the methods and criticism of the knowledge claims derived therefrom can be conducted both prior to publication and thereafter. Along with the experts, any informed persons can make their critiques of a study known. Through this criticism, interpretive knowledge claims can be validated in such degree as there is consensus within the community of experts. Because unique studies cannot, by definition, be replicated, replication is an inappropriate professional standard.
As I have mentioned, at viewtopic.php?f=18&t=200#p846, Alice Goffman’s study, On the Run, is an excellent example of empathic interpretation. She is fully transparent about the methods of her study. While her study is too dependent on her unique abilities and circumstances to ever be exactly replicated, her methods and findings can be criticized and validated in whole or in part.
Under the rule of replicability, a study like Goffman’s would have to be denied recognition as important for political science due to its uniqueness. Because it would exclude important knowledge, this is a self-destructive rule rather than a helpful rule for our profession.
Why is Goffman's study an exemplar of method and of important knowledge for political science? The answer is readily given within the context of David Easton’s definition of political science as the study of the political system, particularly as to how a political system is able to persist over time.^ Goffman’s study was of an aspect of the US criminal justice system (CJS). The CJS is a subsystem of the political system. It can be understood as having the function of aiding or facilitating the persistence of the political system.
Goffman’s study exposed some deleterious unintended consequences of a CJS public policy; namely, a major city’s “get tough on crime” policing policy. As knowledge for political science, her study raises the question as to whether the continuation of such a policy aids or threatens the persistence of the political system. Such knowledge is vital to the political system. If its policies are headed towards self-destruction, then it should be made aware of that. In this way, political science studies can serve as an alert system for the political system. Goffman’s study started out as a pure research project without expectation as to its results. The results were a discovery made in the course of the study.
Whether or not the city involved corrects its CJS public policy, Goffman’s study illustrates Easton’s vision of political science as being empirical, methodical, and able to produce knowledge which is of vital importance to the political system.
One of the unintended consequences of the positivistic insistence that replicability is the key to validating political science knowledge is the continuing decline of esteem for the profession among political professionals and the public. Our work is often criticized as heavy on method but light on heft. (See my discussion of this in my paper on Good Work in Political Science at https://independent.academia.edu/WilliamJKelleherPhD )
Therefore, the APSA ought to jettison the DA-RT project as a threat to the best interests of the profession. Additionally, APSA resources should be used to inform its membership of the vital role Interpretive Political Science can play in furthering the interests of the profession, especially as to its esteem in society.
William J. Kelleher, Ph.D.
*A Model Discipline, and How “Good Work” Hurts Political Science William Kelleher Journal: Perspectives on Politics / Volume 13 / Issue 2 / June 2015 DOI: https://doi.org/10.1017/S1537592715000262 Published online: 18 June 2015, pp. 446-448
https://www.cambridge.org/core/journals ... 8C738B9BA5
^David Easton, A Systems Analysis of Political Life; A Framework for Political Analysis; The Political System