Transparency in Security Studies
Posted: Thu Apr 07, 2016 4:35 pm
About me:
I am an International Security scholar who works in active conflict zones, with a particular focus on modern jihadist insurgencies. I research the intersection between criminal business networks and militant armed groups in civil wars. My research uses a mixed methods approach: I conduct interviews with clandestine actors and I also run large surveys for regression analysis. I have published both quantitative and qualitative articles, including in International Security and Security Studies. My work is positivist and with a heavy rational choice theoretical orientation. I write from that perspective.
About DA-RT:
I wholly support the DA-RT initiative in requiring all quantitative researchers to release their datasets, coding, and replication materials upon moment of publication. This is especially important for security scholars, as our research often speaks directly to policy-makers. Withholding quantitative replication data is unethical and dangerous.
With respect to our qualitative research, however, the DA-RT initiative has failed to consider serious ethical and security considerations around releasing field notes. There are three key points I offer in this regard:
1. RESEARCH ETHICS BOARDS: Qualitative research conducted by security scholars often involves interviewing high-risk individuals. This work necessarily requires extreme levels of confidentiality, especially as some of respondents may be actively involved in violent or illicit activities. In such cases, our Research Ethics Boards very rightly require that we take great measures to ensure the confidentiality of such participants. Interviews with higher risk respondents often cannot be done with any technology in the room, such as using a computer for typing notes. In many case, only hand-written notes, with no identifiable information recorded, are ethical and appropriate.
Even post-interview transcription has restrictions, as digital security can be compromised by hackers and government agencies; this is especially important when both states or non-state actors are actively seeking information about certain militarized individuals and groups. A full digital transcript that records information about dates, locations, and other details that could be used to piece together the respondent's identity could put the individual and the entire research team at risk. Without a sound guarantee that the notes will be protected, it would be impossible to meet the requirements of confidentiality set forth in the ethics review process.
2. POLICY IMPLICATIONS: If security specialists are not given the right to complete our field work in a safe and ethical fashion, then we will stop doing this type of work entirely. Frankly, the consequences of losing the type of knowledge we bring to the table will be extreme. All security specialists who do field work in war zones are producing results that have immediate policy implications. We regularly advise governments on the impact of their decisions on the outcomes of violent insurgencies.
What we bring to the table is a granular, ground-level perspective, which complements system-level, large-N studies. For example, statistical studies show us how drone strikes affect insurgent groups with different organizational structures, but without rigorous qualitative research that investigates what the internal organization of each of those groups is, these results are rendered unusable. These are insights that policymakers need in order to devise effective solutions for some of the most pressing international security crises in the world today.
3. LOST DATA: Those of us who have spent years working in conflict zones have already made promises about confidentiality to our respondents, which we are bound to keep. For example, I promised every Somali business elite I met in Mogadishu that no one else would ever see my notes. Keeping this information in confidence is therefore an ethical requirement, which was stipulated by my Research Ethics Boards. We who have spent years in the field have extraordinary vats of qualitative data that were acquired before the DA-RT initiative began, and which we have yet to publish from. This would mean that years of hard-fought research that could help to save human lives would be laid to waste. Countless future publications would be left in desk drawers, never to be seen.
A Better Standard for Security Scholars:
I am in complete agreement with the overarching principle that all qualitative social scientists (most especially those working on international security) should be held to a higher standard of transparency. The DA-RT initiative, however, misses the most obvious and shockingly lack of transparency that qualitative researcher are actually guilty of. Rather than releasing interview notes (which is dangerous and inappropriate), I propose that the correct course of action on qualitative data transparency should be centred on three key areas: (1) research ethics, (2) data security, and (3) positionality. To that end, I heartily support a new requirement that leading journals require ALL researchers to include a brief online methodological appendix responding to these three key issues.
1. ETHICS: Researchers working with human subjects should be required to provide a methodological appendix that discusses their research ethics process, as well as formally disclose any ethical or security challenges faced during their research process. Too many scholars suppress this information; disclosure should be made mandatory.
2. DATA SECURITY: Researchers with confidential respondents should be required to discuss how they ensured the security of their data, with special consideration to the challenges of protecting information in our digital age. This conversation is essential to ensure the safety of respondents in the long-term, and will help future scholars to think sensibly about these issues.
3. POSITIONALITY: Researchers from ALL backgrounds should be required to comment in a methodological appendix how their intersectional positionality (gender, race, etc.) affected both their research process and results. I am appalled that it is more often female and minority scholars who discuss their positionality, whereas very few of my male and white colleagues in security studies feel obligated to do this. All qualitative data is affected by the researcher's positionality, whether the scholar is working with government elites or impoverished refugees; addressing this issue explicitly should be a professional requirement for all publication in leading journals.
Thank you for taking this initiative.
[5/19: QTD moderator change in the topic/thread title, with prior approval from the original poster, from "Security Research and DA-RT" to "Transparency in Security Studies" to reflect the broader focus of both the original post and the thread]
I am an International Security scholar who works in active conflict zones, with a particular focus on modern jihadist insurgencies. I research the intersection between criminal business networks and militant armed groups in civil wars. My research uses a mixed methods approach: I conduct interviews with clandestine actors and I also run large surveys for regression analysis. I have published both quantitative and qualitative articles, including in International Security and Security Studies. My work is positivist and with a heavy rational choice theoretical orientation. I write from that perspective.
About DA-RT:
I wholly support the DA-RT initiative in requiring all quantitative researchers to release their datasets, coding, and replication materials upon moment of publication. This is especially important for security scholars, as our research often speaks directly to policy-makers. Withholding quantitative replication data is unethical and dangerous.
With respect to our qualitative research, however, the DA-RT initiative has failed to consider serious ethical and security considerations around releasing field notes. There are three key points I offer in this regard:
1. RESEARCH ETHICS BOARDS: Qualitative research conducted by security scholars often involves interviewing high-risk individuals. This work necessarily requires extreme levels of confidentiality, especially as some of respondents may be actively involved in violent or illicit activities. In such cases, our Research Ethics Boards very rightly require that we take great measures to ensure the confidentiality of such participants. Interviews with higher risk respondents often cannot be done with any technology in the room, such as using a computer for typing notes. In many case, only hand-written notes, with no identifiable information recorded, are ethical and appropriate.
Even post-interview transcription has restrictions, as digital security can be compromised by hackers and government agencies; this is especially important when both states or non-state actors are actively seeking information about certain militarized individuals and groups. A full digital transcript that records information about dates, locations, and other details that could be used to piece together the respondent's identity could put the individual and the entire research team at risk. Without a sound guarantee that the notes will be protected, it would be impossible to meet the requirements of confidentiality set forth in the ethics review process.
2. POLICY IMPLICATIONS: If security specialists are not given the right to complete our field work in a safe and ethical fashion, then we will stop doing this type of work entirely. Frankly, the consequences of losing the type of knowledge we bring to the table will be extreme. All security specialists who do field work in war zones are producing results that have immediate policy implications. We regularly advise governments on the impact of their decisions on the outcomes of violent insurgencies.
What we bring to the table is a granular, ground-level perspective, which complements system-level, large-N studies. For example, statistical studies show us how drone strikes affect insurgent groups with different organizational structures, but without rigorous qualitative research that investigates what the internal organization of each of those groups is, these results are rendered unusable. These are insights that policymakers need in order to devise effective solutions for some of the most pressing international security crises in the world today.
3. LOST DATA: Those of us who have spent years working in conflict zones have already made promises about confidentiality to our respondents, which we are bound to keep. For example, I promised every Somali business elite I met in Mogadishu that no one else would ever see my notes. Keeping this information in confidence is therefore an ethical requirement, which was stipulated by my Research Ethics Boards. We who have spent years in the field have extraordinary vats of qualitative data that were acquired before the DA-RT initiative began, and which we have yet to publish from. This would mean that years of hard-fought research that could help to save human lives would be laid to waste. Countless future publications would be left in desk drawers, never to be seen.
A Better Standard for Security Scholars:
I am in complete agreement with the overarching principle that all qualitative social scientists (most especially those working on international security) should be held to a higher standard of transparency. The DA-RT initiative, however, misses the most obvious and shockingly lack of transparency that qualitative researcher are actually guilty of. Rather than releasing interview notes (which is dangerous and inappropriate), I propose that the correct course of action on qualitative data transparency should be centred on three key areas: (1) research ethics, (2) data security, and (3) positionality. To that end, I heartily support a new requirement that leading journals require ALL researchers to include a brief online methodological appendix responding to these three key issues.
1. ETHICS: Researchers working with human subjects should be required to provide a methodological appendix that discusses their research ethics process, as well as formally disclose any ethical or security challenges faced during their research process. Too many scholars suppress this information; disclosure should be made mandatory.
2. DATA SECURITY: Researchers with confidential respondents should be required to discuss how they ensured the security of their data, with special consideration to the challenges of protecting information in our digital age. This conversation is essential to ensure the safety of respondents in the long-term, and will help future scholars to think sensibly about these issues.
3. POSITIONALITY: Researchers from ALL backgrounds should be required to comment in a methodological appendix how their intersectional positionality (gender, race, etc.) affected both their research process and results. I am appalled that it is more often female and minority scholars who discuss their positionality, whereas very few of my male and white colleagues in security studies feel obligated to do this. All qualitative data is affected by the researcher's positionality, whether the scholar is working with government elites or impoverished refugees; addressing this issue explicitly should be a professional requirement for all publication in leading journals.
Thank you for taking this initiative.
[5/19: QTD moderator change in the topic/thread title, with prior approval from the original poster, from "Security Research and DA-RT" to "Transparency in Security Studies" to reflect the broader focus of both the original post and the thread]