small

IS MORE DELIBERATION ABOUT DA-RT REALLY SO GOOD?

The controversy within American political science about DA-RT—an initiative claiming to remedy a deficiency in the accessibility and transparency of data analysis—continues apace.

Well over 20 journals have committed themselves to codifying new requirements for article submissions, in a statement now known as JETS (Journal Editors Transparency Statement). A small group led by six distinguished scholars—Nancy Hirschman, Mala Htun, Jane Mansbridge, Kathy Thelen, Lisa Wedeen, and Elisabeth Wood—organized a petition drive that attracted over one thousand signatures, asking these editors to slow down and include a range of interlocutors in further deliberation. Most of the journal editors refused, some rather intemperately. Three APSA Presidents—Jennifer Hochschild, Rodney Hero, and David Lake—publicly then expressed support for the decision of the APSR to move forward with the new procedures. (These statements, and others of relevance, can be found at http://dialogueondart.org.)

Most of the opposition to DA-RT, including the petition, has centered on a broadly methodological question: does DA-RT really take seriously the work of qualitative and interpretivist scholars, work that does not easily lend itself to data archiving, and that also involves complex legal and ethical relationships with human subjects? This question lies at the heart of the fine symposium published in the Spring 2015 issue of the Qualitative and Multi-Method Newsletter, edited by Tim Buthe and Alan Jacobs. And it lies behind the admirable effort of Buthe and Jacobs to organize a major process of electronic deliberation about DA-RT and JET under the auspices of APSA’s Organized Section for Qualitative and Multi-Method Research. The Group of Six recently circulated a letter to all of those who signed their petition, encouraging them to support and to participate in such a deliberation . They wrote:

“We see the QMMR process as taking as its point of departure a consideration of the first principles of transparency in scholarship, not the DA-RT/JET Statement, which was based on a particular and limited interpretation of those principles.  Given this starting point, some people who signed our earlier letter are likely to argue that those first principles are already adequately incorporated in present scholarly practices.  This position must be represented in the . . .  deliberations . . . . This process is probably the most participatory and open process of consultation in the history of the APSA.  We think it has been well designed to capture the nuances of experience and thought throughout the membership.   Having signed the ‘delay’ letter, you know that the topic is of great importance.  At this stage, the . . .  consultation process is our best tool to demonstrate to other members of the profession and the journal editors the kinds of concerns and insights that APSA members have about these issues.”

To be clear, this letter did not endorse DA-RT. It endorsed further deliberation about “data access and research transparency”—the “problem” that DA-RT identified, defined, or perhaps, depending on your point of view, even created. The letter also announced that this effort to promote further deliberation would be seconded by an open letter to JETS editors by 20 APSA Presidents. This letter, recently posted online , also calls for further deliberation, noting that while research transparency is important, many important issues remain to be worked out, and such issues require further thinking, conversation, and planning. The statement urges caution in the interests of success of the effort itself.

“The commitments expressed in the JETS are a milestone in the long efforts to improve research transparency and facilitate replication in political science. Many of us have strongly supported these efforts. Indeed, APSA formally adopted a commitment to research transparency and data access in its Ethics Guidelines in 2012. We welcome the journals’ commitments to move towards concrete implementation of the broad policy statements. We appreciate your joining in this difficult task. As (in many cases) former editors, we are sympathetic to the labor involved.”

I have been an outspoken opponent and critic of DA-RT (see especially here), and many colleagues and friends have asked me what I think of these recent developments. In what follows I will say what I think. It’s important to note that I am speaking as an individual. In my capacity as Editor in Chief of Perspectives on Politics I have publicly explained on numerous occasions why the journal under my editorship will not participate in DA-RT. That position stands. And the primary basis of that position is that Perspectives is a particular journal whose distinctive mission as “a political science public sphere” involves prioritizing broader forms of political science research and writing and a less expert-centered conception of the public vocation of the discipline. We long have taken seriously general principles of research transparency (see here) and broader principles of scholarly recognition.

And we see no need to institute further stringent and anxiety-generating requirements in order to sustain the intellectual credibility of our journal. Here I wish to put those particular editorial statements aside, and offer a personal response centering on a political approach to the current controversy and to the question at hand.

In his classic, The Semi-Sovereign People, E.E. Schattschneider wrote that: “the definition of the alternatives is the supreme instrument of power.” This is now a commonplace of political science. The insight is relevant to almost all questions of politics. One place where the insight has been mobilized with particular valence has been the discourse of deliberative democracy in contemporary political theory. The notion that inclusive and rational deliberation is a central means of democratic legitimacy is widespread in political science. But it is also now widely appreciated that deliberation always takes place in a situation infused by power and motivated by interests, passions, and value commitments. There is no neutral deliberation and there is no deliberation that proceeds ab ignitio or from scratch.

We know this. Let’s apply this, briefly, to the evolving discussion of DA-RT and JET.

American political science has long been organized on the basis of a serious commitment to “scientific practice,” and its journals have long taken seriously principles of methodological clarity, evidentiary argumentation, and rigorous peer-review. A few years ago a small group of well-meaning political scientists decided that they wished to upgrade this level of seriousness. Why? Because they noted a few issues of fraud in other scientific fields, most notably psychology, and they wanted for political science to be an unimpeachable science that learned from the problems of psychology and modeled the practices of more “scientific” disciplines like psychology and economics who were elevating their attention to “transparency.” In addition, it was very important to their efforts to defend NSF funding of political science that political science be preeminently scientific. Thus “DA-RT” was created. DA-RT is not simply a valorization of research accessibility and honesty. It is a very particular construction of the issue. Its premise is that we face a scientific credibility problem, and we need new ideas—Data Access and Research Transparency—and new practices—JET—in order to solve the problem. For the past four of five years, the proponents of this idea have worked diligently and sincerely, largely below the radar screen, to promote this idea. In 2012 the APSA Council voted to incorporate generic language about voluntary transparency into the APSA Code of Professional Ethics. I was present as a Council member. Many did not understand why this was necessary. Some, including myself, were very skeptical. But the language seemed innocuous—Who could oppose honesty? Does anyone advocate fraud?—and it prescribed no specific procedures, only a general ethnical norm, and it passed. APSA members were thus enjoined to be ethical in this way. In 2014 DA-RT surfaced, as a more public, and political, effort to institutionalize these ideas in the editorial practices of major journals.

A few people, myself included, opposed this effort on very broad grounds, insisting that DA-RT was animated by a positivistic conception of knowledge; that its proponents, ironically, had not demonstrated empirically that there was any serious problem with current practices that required a major movement to substantially revise those practices; that this effort to codify new “data” practices could only seem to be threatening to many scholars whose work is not well described by the DA-RT vocabulary; and that a serious disciplinary concern with publicity and credibility would raise bigger questions about the insularity and un-readability of much of the most “scientific” political science research.

But most of those who shared these concerns about DA-RT chose a more “reasonable” and conciliatory response. Why oppose DA-RT, they said? Aren’t the DA-RT proponents sincere, and aren’t they onto something? Who can be against accessibility and transparency? Why be so oppositional? Why not engage our well-meaning colleagues in dialogue? Isn’t what we really need more inclusive deliberation? And if we can have such deliberation, can’t we come up with new rules that include us too? And won’t we all be better off as a result?

This was a well-meaning effort that made sense. It led to a meeting of the principals on “both” sides at the 2015 APSA meeting in San Francisco. Many participants thought a more “deliberative” approach had been accepted by all in attendance. Then the JET journal editors declared they were moving ahead. And APSA leaders said and did nothing. So a petition calling for delay and deliberation was circulated and shared. Then the petition was rejected by most of the JET editors and APSA leaders issued public statements that seemed provisionally to support DA-RT and unambiguously supported the APSR’s commitment to institute DA-RT.

Now a new round of deliberation is being organized, this time on the basis of elaborate electronic technologies. The Group of Six endorses the deliberation. And now, apparently, the Group of Twenty APSA Presidents does too.

All things considered, this is good. Deliberation can (almost) never be bad. And it is likely that further deliberation will generate some compromises, and a greater sense that those who employ qualitative and interpretive approaches will feel included, heard, and recognized.

The DA-RT initiative will thus be refined, and at the same time further legitimated.

Such reasoned deliberation and compromise is enormously appealing, especially since the refusal of such deliberation and compromise has no obvious practical implication beyond saying “no!” and it surely feels obstinate to refuse to talk and to simply say “no!”

In addition, to say “no” seems to put one in the position of saying “I am against data access and research transparency,” or even perhaps “I am against science, and think that we should have no standards at all.” And who wants to say this? Surely not anyone who wants to be a serious participant in US political science.

And so the Group of Six call for further deliberation has appeal.

And so, for those who have asked me, I must declare that I am not against it.

But neither am I for it.

Why?

Because I believe that we political scientists ought to be sufficiently reflexive about our programmatic disciplinary discussions about “science” to take seriously what we actually know about human affairs and about politics when we undertake our own deliberations.

And we ought to be both respectful and honest.

And it seems to me that these deliberations are infused with power and are not simply matters of pure reason. And that the proponents of DA-RT have won the institutional “war of ideas,” and have set the agenda, and have left those who are skeptical or critical to demand a place at the table and to argue about tactics. And it also seems to me one of the reasons why the proponents of DA-RT have won is because so many of the skeptics have been so darned collegial and reasonable and deliberative, and so indifferent to the forms of power being exercised by those whose principal and sincere commitment is to the upgrading of the “scientific” credentials of political science and the institution of ever-more demanding scientific methods—measures that are not universally embraced or even sought by most political scientists.

The recent statement by the 20 APSA Presidents makes this “victory” of the  logic of DA-RT clear. It urges more deliberation so that DA-RT can be successful, and it declares that JET is a “a milestone in the long efforts to improve research transparency and facilitate replication in political science.” Read these words carefully. They represent more than a position on DA-RT. They represent an interpretation of the recent history of political science. They declare that DA-RT represents a fulfillment of a long-standing mission to render political science more scientific.

Now, I am certain that many of those who signed this statement do not really believe this. But they signed anyway, because in a way the sentiments seem unobjectionable, and in any case the best way to convince partisans to support deliberation is by avowing commitment to their goals and asking them to talk with you about the best means of achieving these goals. It makes sense. It perhaps promotes further deliberation. But it also throws in the towel before the conversation begins. Or, to be more precise, it concedes that the towel was thrown in long ago, when it was first decided that the most “reasonable” way to respond to DA-RT was to refine its logistics rather than declare opposition, articulate an alternative vision, and build support for this vision.

What would such a more “radical” position have entailed?

I really do not, now, wish to outline an alternative “political strategy” within the discipline. Those who have been paying attention know that I have long maintained that Perspectives on Politics represents a different conception of political science; that the journal was the outcome of an earlier argument about political science; and that by very publicly declaring and enacting its conception of a “political science public sphere,” the journal was publicizing that it could be seen as one space that supported a broader and more “public” political science. But it would be inappropriate to say more than this as an editor, and I have not said more than this. And I am confident that those who have worked with me—from all corners of the discipline, including experimentalists, modelers, and survey researchers—will attest that our journal has been above all committed to scholarly seriousness and fairness and intellectual quality.

As an editor I cannot say more. As an individual political scientist whose editorial tenure will end in May 2017, I can say more. But for a variety of reasons I won’t do so here and now.

But I will say this: the ongoing discussion of DA-RT and JET is a political discussion, debate, and contest, with implications for the distribution of power, benefits, and values within the discipline, and with implications for the broader public import of the discipline. Will political science become more preoccupied with expertise and methodological proficiency and propriety, or will it become more dialogic and genuinely public? Will it maintain a strong humanistic, normative, and critical dimension, or will it come more technocratic and, frankly, anti-humanistic? Many important questions are at stake.

While I will not articulate a more political approach to or strategy regarding the issues at stake, I would like to pose this question: why have so many of us bought into the idea that right now data access and research transparency –a “problem” constructed by DA-RT– is the most pressing challenge facing political science publishing and indeed, in terms of time and energy, the most pressing challenge facing political science?

Why are we talking about DA-RT, and urging further deliberation about new requirements for journal publishing, when we could be talking about:

*          How to encourage more political science journals to become more interesting and more readable by fellow political scientists, other social scientists, and indeed broader reading publics?

*          How to seriously build incentives for better writing among political scientists, at every level of the discipline? Instead of worrying about how poorly written work is linked to more rigorous data archiving, why not worry about why so much work is poorly written? Is this too not a problem of practices, institutions, incentives, editorial policies?

*          How to institutionalize better means of communication linking specialized political science journals, generalist political science journals, and broader forms of media and public discussion—a question that includes but is not reducible to the question of new business models that render scholarly journals publicly accessible?

*          How to incorporate serious attention to the ethical relationship between our inquiry and the subjects of our inquiry. I mean here more than basic concerns about “human subjects” that animate institutional review boards. Such concerns are ethically and intellectually crucial. But the question I am posing goes beyond the “research ethics” and bureaucratic requirements of ethnography and fieldwork. It is the old question of the value-basis of all political science. How do our research problems get defined, and what is the ethical and political significance of our choices? From where does our funding come? What values animate our work, individually as scholars and collectively as a discipline? What values should animate our work? And in situations where values conflict, what kinds and forms of judgment and institutionalized practice best help us to navigate them in a responsible way?

Why not focus attention on any one of these questions, or on all of them together, rather than on new codes of article submission and data archiving?

Indeed, pressing such questions opens out onto even broader questions:

*          How can the research and writing that we promote as a discipline speak most meaningfully to the intellectual and pedagogical and communitarian needs of the vast majority of political scientists who are not employed by Research-1 universities, who do not have time for active research agendas, and who do not have access to the institutional and financial resources that would allow them to become dedicated practitioners of DA-RT? We know that increasing numbers of our colleagues spend most of their time teaching many courses, in often insecure situations, and in institutions that do not prize or reward sustained research agendas. The journal New Political Science recently ran a special issue on these questions, “The Future of Higher Education and American Democracy” (vol. 36, issue 4, 2014). Why have none of these issues made their way into the discussion of political science publishing and “publicity?” Why have these issues received no attention from APSA at a time when “data archiving” has come to loom so large?

*          Does a renewed emphasis on methodological rigor—something that is now promoted by most qualitative methodologists as much as it is by quantitative methodologists, which is one reason why some of the leaders of the qualitative methods community have been leaders of DA-RT—really best serve most of our graduate students, as they prepare for careers in community colleges, liberal arts colleges, and public universities ever more committed to “relevance” and the bottom line, and as they prepare to enter a labor market characterized by precarity and insecurity?

*          Is the most pressing lack for most political scientists the lack of more rigorous procedures of data access, or is it lack of access to good jobs, or to serious participation in public discourse in a world increasingly suffused by reality TV, insular digital communities, and non-stop self-absorption, self-monitoring, and self-promotion by cell phones and other digital technologies?

*          If we believe that political science is a universal form of inquiry that addresses all political challenges throughout the world—as we claim to believe—then how do our practices of research, writing, and publication seriously engage the rest of the world, and especially the vast parts of the world where political scientists and other academics labor amidst institutional impoverishment, political harassment, repression and violence? Do we really think that the most pressing intellectual challenge for political scientists in Romania or Hungary, Brazil or Mexico, China or Indonesia, Sudan or South Africa, Egypt or Turkey, is DA-RT? Do most political scientists in the world give a shit about this? Do they even know about it much less care about it? And if not, why are we focusing our energies on it? And why do some of our most distinguished colleagues refer to this focus as a “milestone” of intellectual probity?

My critics will say I am now no longer criticizing DA-RT, but simply changing the topic.

Yes, exactly!

I am asking this: why are we talking about this topic now?

Why are we not spending our time, energy, and scarce resources discussing bigger and more important questions related to the public role of political science and the worldly challenges facing political scientists in our effort to meaningfully combine research and writing, teaching, service, and public engagement?

Why indeed.

The question is a political one, and the answer is a political one.

What would happen if political scientists–and especially those who are skeptical of more bureaucratic requirements, and who care about interpretation and critique and intersubjectivity and “the public and its problems”– began to think of it as a political question?

And what would happen if they—if we—not simply began to think about this politically, but to act politically?

About Jeffrey C. Isaac

Jeffrey C. Isaac

Jeffrey C. Isaac is the James H. Rudy Professor of Political Science at Indiana University, Bloomington. He has served as Editor in Chief of Perspectives on Politics since 2009, and served as Book Review Editor since 2005.

Jeffrey C. Isaac @
 

Check Also

Secularism Cover

Political Secularism, Religion and the State

Religion is intertwined with politics in every country in the world, though how this manifests is unique in each country. For example, in Germany – a state that still levies a religious tax for members of recognized religions – recent court cases addressed whether the government could regulate the ritual of circumcision for Jews and Muslims and whether it could ban the religious headwear of Muslim women. Much of the Muslim world is in a state of civil war over the role religion should play in government and which interpretation of Islam should be dominant.