Socrates_teaching_Perikles-Nicolas_Guibal-IMG_5308

Further Thoughts on DA-RT

What’s up with DA-RT? What happened regarding DA-RT at the APSA meeting in San Francisco? Is it true, as one prominent editor has been quoted, that DA-RT is a fait accompli, and that “the train has already left the station?”

In recent weeks I have been repeatedly asked these questions.

DA-RT, for those who need a refresher, stands for Data Access and Research Transparency: an important – and now somewhat controversial – initiative, pioneered by Arthur Lupia and Colin Elman, to codify new rules and requirements for political science journals in the name of promoting transparency, replication, and the testing of scientific results. DA-RT has proceeded slightly beneath the disciplinary radar screen for a number of years. In October 2014 DA-RT went public, organizing a conference, producing a joint statement that has now been adopted by 25 journals, and posting the statement, along with supplementary materials, on a website: http://www.dartstatement.org .

I have found myself at the center of much recent DA-RT discussion—and thus the addressee of the questions cited above—because I am the editor-in-chief of one of APSA’s flagship journals, Perspectives on Politics: A Political Science Public Sphere, and because I have taken a very public stance against DA-RT. In October 2014 I wrote an open letter to DA-RT organizers explaining why I could not and would not sign their statement, and in June 2015 I made that letter the centerpiece of a 13,000- word lead essay in Perspectives, entitled “For a More Public Political Science.”

I offered three basic reasons for my initial reluctance to sign the DA-RT statement:

(1)  The statement seemed to have major, and arguably controversial and troubling, implications, and represented a significant departure from established practice. It also seemed to require much serious deliberation – both among editors and editorial boards and among other disciplinary groupings and associations – before moving forward.

(2)  In my experience as editor of Perspectives, DA-RT is a solution in search of a problem. We have done quite well with our existing processes of rigorous peer review – which often involve reviewer critiques of data or requests for more data – and editorial scrutiny and judgment; we have always encouraged authors to share data in appendices consistent with their own best judgments; and there was thus no reason to adopt the DA-RT principles.

(3)  Perspectives on Politics has a distinctive mission: to promote a broad and pluralistic political science public sphere. I was concerned that the DA-RT framework, principles, and potential rules would privilege certain kinds of methods and formats over others, and would they would have a broader “chilling” effect, producing anxieties and concerns among many scholars that their work was no longer welcome or considered properly “scientific.”

In my June 2015 essay I linked these objections together via a fourth objection: that DA-RT was part of a broader agenda to promote “a resurgent neopositivism” in political science, and that Perspectives on Politics is a journal that represents a different conception of political science that places a premium on a different kind of publicity: the kind that comes not from enforcing data access in the name of greater methodological rigor, but from encouraging the development of more broadly interesting political science research and writing that is publicly relevant, intelligible and readable.

My June essay struck a chord with many. I have been informed that by September 1, it had been downloaded or viewed over 17,000 times (making it the second-most widely viewed piece published by the three APSA journals in 2014-2015). As a result, I have been included in many listserv discussions; have been invited to consult with editors and editorial boards deliberating about DA-RT; have been invited to give numerous talks at universities; and have been involved in a range of APSA discussions of DA-RT, including a workshop organized by the Qualitative and Mixed Methods Section at this year’s APSA meeting and a private, by invitation only, meeting at APSA that was organized informally to bring together DA-RT proponents, critics, and APSA leadership.

Indeed, at this year’s APSA conference there was extraordinary attention devoted to DA-RT. At least five meetings devoted to DA-RT were on the conference program, including the annual Saturday morning Editors’ Breakfast organized by APSA itself, as well as a day-long meeting bringing together DA-RT advocates and prominent proponents of qualitative methods who were troubled by DA-RT. DA-RT was also discussed at the informal leadership meeting noted above. And though it was not on the official agenda of the APSA Governing Council, it came up for discussion numerous times at the Council, including a suggestion, made by one Council member, that the topic of DA-RT must be placed on the agenda of the Spring 2016 Council meeting.

What’s up with DA-RT in the wake of all of this attention?

As far as I know, there have been no official statements of clarification from APSA. At the same time, I think that it is fair to note the following:

  1. The seemingly ineluctable “consensus” on DA-RT has been at least temporarily halted. While Perspectives is thus far the only major journal of which I am aware that has publicly opposed DA-RT, a number of other important journals have either demurred or are in the process of considering, or in one case even reconsidering, the adoption of DA-RT principles.
  1. In San Francisco it was clearly acknowledged by APSA leaders, informally, that DA-RT as such had never been vetted by the APSA Governing Council; that the topic needed to be brought before the Council; and that until there was serious Council deliberation on the topic, DA-RT could not be considered an official APSA-endorsed initiative. This position, which I describe as an “understanding,” has not (yet) been articulated in any formal sense. But I would submit that this much is clear: while the APSA Council voted in 2012 in support of the general value of voluntary research transparency, and while APSA staff has offered some support to DA-RT organizers, APSA as an organization has never officially adopted or unequivocally endorsed DA-RT. Most people who are aware of DA-RT are not aware of this. But it is an important fact, and it was clarified in San Francisco.
  1. DA-RT is thus a private initiative, pioneered by some exceptionally prolific, energetic, and highly respected colleagues, who have succeeded in creating momentum for the idea, organizing events to promote the idea, and persuading many journal editors to sign on to the idea. Have these editor-signatories done due diligence, or brought DA-RT before their own editorial boards for serious deliberation, or consulted properly with relevant stakeholders or institutional sponsors? These are legitimate questions that different editors will no doubt answer differently. I hope that the readers and supporters of the DA-RT journals ask such questions. But regardless of whether or not such questions are put to the individual editors, in no sense can these editors be seen as acting in accordance with an APSA mandate. There is no official APSA mandate.
  1. In recent months there has also been some very serious scholarly and intellectual discussion of DA-RT, including critical discussion of DA-RT’s limits and biases as developed and outlined. Many colleagues have raised questions about whether DA-RT is consistent with many so-called “qualitative” and ethnographic research practices. Some have raised questions about whether the very idea of DA-RT is grounded in a naturalistic and positivistic conception of inquiry that is most appropriate to quantitative research. The Spring 2015 issue of the Qualitative and Multi-Method Research Newsletter contains a collection of fine articles on the practical and philosophical problems that DA-RT seems to pose to qualitative and multi-method researchers. I would encourage anyone interested in coming to a well-informed opinion on DA-RT to read this issue of the Newsletter.
  1. Many very prominent political scientists have been raising these concerns informally, and in some cases formally, via a series of joint letters calling for general caution in the adoption of DA-RT principles. In doing so, they have raised particular concerns about one issue of crucial administrative, legal, and ethical importance: the need for political scientists, especially those whose work involves field research, to place both standard university human subjects commitments, and ethical obligations to vulnerable human subjects, above any possible DA-RT requirement for the disclosure of confidential data. While I am not in a position to publicize these letters, at least one such letter, signed by some of the top figures in our discipline, was sent to APSA leadership prior to the San Francisco meetings, and at least one other letter was sent to all of the editors who have publicly signed on to DA-RT. I have it on good authority that these concerns have been effectively communicated to relevant journal editors, and that they indeed may bear fruit in two ways:

(i) by encouraging the editors – who as DA-RT signatories agreed to adopt new editorial guidelines by January 15, 2016 – to delay the adoption of new guidelines before some of the knotty practical and ethical concerns are given more attention;

(ii) by successfully persuading both the DA-RT organizers and the affiliated editors that the DA-RT principles must do more to recognize the priority of human subjects concerns.

The jury remains out on these matters. Indeed, much of what I have mentioned is taking place informally and below the radar. In general, I think it is fair to say that there is a strikingly ironic disconnect between the DA-RT calls for “transparency” and the opacity of the discussions, maneuvers, and decisions whereby DA-RT has acquired the misleading and unearned appearance of ineluctability and disciplinary authority.

In the meantime, the discussion proceeds, and I am writing this post in the hope that more political scientists will both pay attention to the discussion and contribute to it.

It seems clear that the DA-RT organizers have succeeded in thrusting the question of “data access and research transparency” to the center of political science journal discussion, and that all journal editors, myself included, are now required to pay attention to it.

Attentiveness to data and analytic integrity has indeed always been important to the peer review processes of any serious political science journal. At the same time, greater attentiveness, in moderation, can hardly be a bad thing.

It also seems clear that the resistance to DA-RT that has been mounted from many quarters—some voicing caution on practical or procedural grounds, some leveling more fundamental questions—has had an effect, and that as some version of DA-RT—or at least some more specific version of what we might call “dart”—is codified, those who codify it will have greater sensitivity to the practical and ethical challenges faced by qualitative and interpretive researchers who can less easily “disclose data” than can quantitative researchers whose data is easier to list, reference, post, and test via replication.

In short, the DA-RT initiative, from the start a savvy and well-intentioned effort to reshape political science publishing, has been recognized for what it is—a form of disciplinary politics—and it has been politicized. Lines have been drawn, and crossed. A range of conversations will continue to take place. And some compromises will be made. If the relevant disciplinary institutions and leaders organize this process well, then this can be worked out with a spirit of integrity, mutual respect, and genuine deliberation among equals. And because the discipline, like all disciplines, is complex, diverse, pluralistic, and porous, this will never be worked out in One Single Way. A range of journals, institutions, etc., will deal with this issue in a variety of ways, in accordance with their own mandates, constituencies, and processes of decision.

This is as it should be. And APSA, as the peak disciplinary association in the U.S., may well take a position. It is hoped that if it does take a position, the position is sensitive to the diversity of its members and constituent units. But in any case, APSA is an organization with a bureaucracy. It is not a scientific or intellectual authority. It is true, it wields much power and institutional authority; among many other things, it supports Perspectives on Politics, and its staff of seven, and its editor—me.

And this leads me to my final point: APSA cannot speak with one voice, even if its Governing Council—a small group of notables, selected mainly by cooptation, who rotate on and off the Council with regularity while most members pay no attention at all—were to speak with one voice. APSA is pluralistic to its core. And at its core are three “flagship” journals. And one of those flagship journals—Perspectives on Politics—will remain opposed to DA-RT at least for the length of my tenure and that of our editorial board.

The source of this opposition is both practical and principled.

On the one hand, there is no compelling argument that what political science and its journals most need right now is new forms of bureaucratic regulation.

On the other hand, it is pretty clear, at least to me, that the desire to institute new forms of regulation is linked to a broader desire to ratchet up the “scientific” identity of political science, by placing a premium on methodological rigor, replicable behavioral results, and “evidence-based policy” engagement. This desire is surely a legitimate aspiration for those who share it. But it does not represent political science institutionally, and it certainly does not represent what is best in political science as a tradition of inquiry.

Perspectives on Politics has for some time stood for something different. Seven years ago we branded the journal as “A Political Science Public Sphere.” For us publication, and being “public,” does not refer to bureaucratic rules of data disclosure. It refers to the value of publicity and of vigorous intellectual engagement – within our scholarly community, among and between diverse scholarly communities, and between the academic world and the broader public world.

DA-RT, and its bureaucratic “accessibility” and “transparency,” does nothing to support this mission. In some ways it is a distraction.  But in other ways it is a direct impediment, because it promotes a specialized, expert, and instrumentalized conception of political inquiry and political knowledge. Such a conception has great cache in our discipline. In all likelihood it will have greater cache in the future. But Perspectives stands for something else, and it is my hope, and my determined purpose, that it will continue to stand for a more authentically public political science for a long time to come.

*I would like to thank James Moskowitz his helpful comments.

About Jeffrey C. Isaac

Jeffrey C. Isaac

Jeffrey C. Isaac is the James H. Rudy Professor of Political Science at Indiana University, Bloomington. He has served as Editor in Chief of Perspectives on Politics since 2009, and served as Book Review Editor since 2005.

Jeffrey C. Isaac @
 

Leave a Reply