Substantive Dimensions of the Deliberations

Forum rules

We encourage contributors to the Discussion Board to publicly identify by registering and logging in prior to posting. However, if you prefer, you may post anonymously (i.e. without having your post be attributed to you) by posting without logging in. Anonymous posts will display only after a delay to allow for administrator review. Contributors agree to the QTD Terms of Use.

Instructions
To participate, you may either post a contribution to an existing discussion by selecting the thread for that topic (and then click on "Post Reply") or start a new thread by clicking on "New Topic" below.

The transition to Stage 2 of the deliberations is currently underway but will take some time to complete. In the meantime, we very much welcome additional contributions to the existing threads in this forum.

For instructions on how to follow a discussion thread by email, click here.

winters

DA-RT Approach Misses Important Aspects of Qualitative Research

PostMon Apr 25, 2016 3:43 am

What is the essence of interpretation and analysis in Comparative Politics? A big part of it is not just having "data" or "evidence." It is also the investment of years building the capacity even to know what you are observing and to recognize its significance in context. This is what makes people authorities in a field. It is the accumulation of experience and expertise that allows for meaningful interpretation in the first place. And it is rooted in a mastery of languages, cultures, histories, and other nuances of places, populations, groups, and communities. Colleagues working on the US (who set DA-RT in motion) take these skills and insights for granted because they only focus on "home-America" where everything is relatively familiar.

Someone in an Indonesian, Vietnamese, or Filipino legislature could say or do something that has an apparent meaning to US-focused observers, but a very different (or possibly even opposite) meaning to someone who has a subtle understanding of these places. Speaking just for myself, it has taken decades to develop the knowledge needed for interpreting the not-always-evident evidence. And this didn't just come from reading books and articles over the years (which someone could presumably "cram" to replicate). It came also from hundreds of conversations, experiences, observations, people telling their stories, and listening to people from these places trying to interpret as well. And it came from lots of mistakes. You learn to triangulate and piece disparate elements together to make sense of things that are very hard to figure out.

This is related to a second point already touched on by Mimi Keck. Part of the problem with the whole survey-dataset-IRB image of research is that many of us have made scores of visits to the places we study that are not necessarily part of a formal research plan or episode. And yet during those visits, you had a chance to meet a labor leader, an NGO activist, or whomever. And during those conversations, interviews, lunches, or wedding receptions (even funerals and circumcisions!), you took out your notebook and wrote down what was said -- possibly as summary or maybe scribbling direct quotations. You knew what you were hearing was important, but you knew you might not need it or use it immediately. By my last count, I have 18 such notebooks filled with material, 49 (long) recordings of people I talked to from presidents and prime ministers to newspaper editors and oligarchs to peasant protesters and garbage pickers. And I have many more recordings I dictated to myself after various meetings, just to capture what was said while it was still fresh. It's an ordered hodge-podge of things that I have reviewed several times and drawn from for my work.

To "replicate" this, someone would have to replicate the entire range of my encounters with the places and people I've spent decades studying. Indeed, some people are fairly close approximations -- because they are themselves experts and authorities on these same places. Thankfully, a key part of the process of publishing includes editors sending our writings to such very informed people for their critical assessments.

The DA-RT initiative represents a hopelessly narrow slice of what good research is, how useful information is gathered, and how illuminating debate is advanced. Sometimes research involves IRB-cleared survey instruments and planned research episodes. But many times research happens when it does and looks nothing like what is in the minds of the people pushing this agenda. I take the honesty and integrity of colleagues in the academy as high unless proven otherwise. I can question their research and conclusions based on many dimensions of the work (what it asks, what it fails to ask, what evidence is present or lacking) without having to say "prove to me the evidence is actually real or accurate." For some limited kinds of research or analysis, appending datasets or other materials may make sense. But for others, it simply does not. If you want to replicate Ben Anderson's insights about the Javanese concept of power or reproduce his analyses in Imagined Communities, I invite you read what he read, go where he went, talk to the people he engaged with, and retrace the pilgrimages he made.

Jeffrey Winters
Northwestern University

Post Reply