The resolution itself would have been cutting-edge five years ago. The text is weak compared to policies at leading institutions: it “endorses the storage and preservation of scholarly publications in Florida State University’s open access institutional repository”, directs the libraries to “develop policies and procedures”, and calls for an annual report. Unfortunately, that’s as far as it goes. Based on the plain text of the resolution, there’s no mandatory deposit, the key element of successful open access policies. As a result, we can expect compliance to be weak. However, as Micah Vandegrift, Scholarly Communications Project Manager at FSU, notes, it’s a first step. Hopefully this resolution will spark a dialog, creating greater awareness and understanding, leading to the adoption of a mandatory policy in the near future. Meanwhile, kudos to the FSU faculty and those who worked to develop this policy.

Of course, I would be remiss not to add that my alma mater, the University of Florida, also has taken some positive steps in the open access arena lately (see: this, this, this).

On July 17, I submitted these comments on the proposed revisions:

I am disappointed that the National Science Foundation’s proposed Merit Review Principles and Criteria removes the criteria’s reference to the broad dissemination of results. NSF should retain and strengthen this consideration of merit review, as I wrote in my previous comments to the Task Force on Merit Review; see http://www.gavinbaker.com/2011/03/15/nsf-review/. Broad dissemination must be a high priority for NSF-funded research.

If NSF proceeds with the proposed criteria, it should ensure that its guidance for investigators, reviewers, and staff resolutely expresses the importance of ensuring the broadest possible access to research results. As I wrote in my previous comments, this should include updating the list of representative activities provided to investigators, particularly to encourage investigators to provide open access to their publications.

Thank you for the opportunity to comment on the proposed merit review criteria. Please do not hesitate to contact me with any questions.

What do you see as the strengths and weaknesses of each criterion?

The current Broader Impacts merit review criterion includes the consideration, “Will the results be disseminated broadly to enhance scientific and technological understanding?” NSF should retain and strengthen this consideration of the merit review.

Broad dissemination must be a high priority for NSF-funded research. As a federal agency operating with taxpayer funding, NSF has a responsibility to maximize return on investment by removing barriers to access for the scientific community, as well as to ensure access for taxpayers. In addition, broad and equitable dissemination of scientific information advances the goals of the Universal Declaration of Human Rights, which affirms that “everyone has the right freely to … share in scientific advancement and its benefits,” and the International Covenant on Economic, Social and Cultural Rights, in which governments agree to take the steps “necessary for the conservation, the development and the diffusion of science;” the United States is a signatory to both documents.

Under the current criteria, which are more than a decade old, proposals may be inadequately reviewed with regard to the dissemination consideration. During that time, the landscape of scholarly publishing and information has undergone significant changes. In particular, the exceptional opportunities created by the Internet behoove NSF to ensure that its funded research takes full advantage of the new technology to maximize cost-effective dissemination.

At present, broad dissemination of results is promoted through two avenues at NSF: the merit review criteria, which investigators must address in their proposals, and the Award and Administration Guide (AAG), which governs projects after an award is issued. This dual approach is beneficial and should be maintained and strengthened.

NSF establishes minimum requirements for dissemination in Chapter VI.D.4 of the AAG. In particular:

• “Investigators are expected to promptly prepare and submit for publication … all significant findings from work conducted under NSF grants;” and
• “Investigators are expected to share with other researchers, at no more than incremental cost and within a reasonable time, the primary data, samples, physical collections and other supporting materials created or gathered in the course of work under NSF grants.”

This policy is underpinned by Sec. 7011 of the America COMPETES Act of 2007, which requires NSF to enforce the standards by making any researcher who fails to comply ineligible for future funding.

Additionally, Sec. 7010 of the 2007 COMPETES Act requires NSF to make project reports freely available to the public online, along with citations to any publications resulting from NSF funding. This provision of the law is implemented by Chapter II.E.3 of the AAG, which requires grantees to submit a report describing the project outcomes, written specifically for the public, to be made freely available via Research.gov.

In addition to the baseline standards of the AAG, investigators are required to address the dissemination consideration of the Broader Impacts merit review criterion. This dual approach is beneficial because it encourages creative approaches to dissemination beyond the minimum, as appropriate to the proposed activity. Moreover, by including dissemination in the merit review criteria rather than only in post-award requirements, NSF ensures that investigators plan for dissemination beginning from the proposal stage, a valuable way to keep dissemination in mind throughout the project’s life cycle.

Unfortunately, both aspects of this approach currently are insufficient. Both the dissemination requirements of the AAG and the dissemination consideration of the Broader Impacts criterion should be updated and strengthened.

While NSF is to be recognized for its leadership in recently requiring investigators to develop data management plans, in other regards the AAG is outdated and should be reformed. Most importantly, NSF has not implemented a requirement that its funded investigators provide public access to resulting peer-reviewed manuscripts, rather than merely the citations to those publications. Among federal science agencies, the National Institutes of Health (NIH) is the current leader in this area, having adopted a mandatory public access policy as required by Sec. 218 of the Consolidated Appropriations Act, 2008. Dozens of other public and private research foundations worldwide have successfully adopted similar policies. NSF should incorporate a similar mandatory policy in the AAG. Furthermore, NSF should improve on the NIH policy by reducing the “embargo” period when manuscripts can be withheld from public access from the current maximum of twelve months to a six-month maximum, as several other research funders worldwide have done.

However, as this comment is directed to the Task Force on Merit Review, I will focus on the dissemination consideration of the Broader Impacts criterion, rather than the requirements of the AAG. To be clear, even if the dissemination requirements of the AAG are strengthened, the dissemination consideration of the merit review criteria also should be retained and strengthened.

The first regard in which the dissemination consideration of the merit review criteria is inadequate is its construction. Currently, broad dissemination is a consideration only of the Broader Impacts criterion, not the Intellectual Merit criterion. However, dissemination is properly understood as fundamental to both the intellectual merits of the proposed activity as well as its broader impacts.

The thrust of the current consideration is that maximizing the social value of the funded project requires communicating the project’s results to relevant audiences outside the research community, such as industry and policymakers, and to the public to enhance the public understanding of science. This is useful strategy to increase the broader impact of NSF funding and should be maintained. In particular, it supports the requirements of Sec. 526 of the America COMPETES Reauthorization Act of 2010, which directs NSF’s Broader Impacts criterion to advance the goal of “increased public scientific literacy.”

However, even within the academic community, the broad dissemination of research results cannot be taken for granted. Access barriers imposed by the high and rising cost of serials and monographs can significantly hamper the circulation of knowledge. These barriers can be particularly imposing to researchers and students at smaller institutions and in developing countries. Additionally, reticence to share data or materials with other researchers, or delays in doing so, also hinder the progress of science.

Thankfully, online technologies enable innovative approaches to the broad dissemination of research information which previously was only shared in small circles. These approaches already have begun to bear fruit. For instance, the NIH-supported Alzheimer’s Disease Neuroimaging Initiative was recently highlighted by the New York Times for its innovative approach to data sharing which is already being emulated. Given the promise of openness, the Board should ensure that the merit review criteria promote the broadest possible dissemination of results.

What changes, if any, would you like to see made to the merit review criteria?

If the Board retains the current criteria, it should add a consideration for broad dissemination in the Intellectual Merit criterion, in addition to the current consideration in the Broader Impacts criterion. Such a consideration might read, “Will the proposed activity ensure the broadest possible access to its results within its own field or across different fields?”

Alternatively, the Board might adopt a single consideration that addresses broad dissemination both within and beyond the research community.

What role should the institution play to ensure that the intellectual merit and broader impacts in NSF proposals can be realized?

NSF should provide additional guidance to proposers on how best to address the dissemination consideration. The Task Force will be aware of Sec. 526 of the COMPETES Reauthorization Act, which directs NSF to better inform proposers as well as staff and reviewers about the Broader Impacts criterion’s requirements, among other changes. The process of implementing these statutory provisions offers a timely opportunity for NSF to suggest “proven strategies and models” for cost-effective broad dissemination.

Currently, NSF provides a list of examples of representative activities for the dissemination consideration. These examples should be updated to better take advantage of proven strategies. Most critically, NSF should encourage investigators to provide open access to their publications by publishing them in an open access journal or by depositing them in an open access repository immediately upon publication, under an open copyright license. NSF could also encourage investigators to consider the impact of their publishers’ policies and pricing on the broad dissemination of their research. Further, NSF should encourage investigators to post their data online for free public access, and to freely distribute online any software created resulting from NSF funding under a free software license, depositing both in appropriate repositories as applicable.

Beyond mere suggestions of activities, NSF should refer investigators to resources on how to accomplish these activities, such as existing guides prepared by the library community and others. NSF should also encourage institutions to play a more proactive role in supporting investigators in maximizing the dissemination of their research, including collaboratively sharing resources and strategies between institutions.

Additionally, NSF should ensure that these resources and guidance are available to its reviewers and staff, and that they duly take into account these activities in reviewing proposals’ commitment to broad dissemination of research.

What impact, if any, has NSF’s two review criteria had on how you think about developing your research projects?

Any other comments?

Thank you for the opportunity to comment on the merit review criteria. Please do not hesitate to contact me with any questions.

So what’s the point of AcaWiki when almost all articles have abstracts, which are summaries and usually available gratis? Well, AcaWiki summaries are also libre (CC Attribution license), so they invite reuse: mashup, translation, and so on. They’re also editable, so they can evolve and be improved.

Abstracts vary widely, usually shaped by the journal’s format: sometimes they’re several paragraphs, something just a few sentences. They might outline the methodology or they might not. They are usually written at the level of specialists in that field, so they may or may not be much use to other readers.

There’s room for improvement and innovation in the world of summary, in other words. For instance, Emerald launched a program asking authors to provide a summary highlighting potential applications. RNA Biology requires its authors to write up their findings on Wikipedia. BMJ publishes only one-page abridgments in its print edition, with the full article available online.

For a more direct comparison, see WikiSummary, which predates AcaWiki but covers only political science.

Two other points of comparison: journalism / press releases and Wikipedia.

Press releases are gratis; science journalism may or may not be gratis; both are rarely libre. They only cover new studies: good luck finding coverage of an article from 1989. They rarely provide a full citation to the original article. They often discuss only the findings, with little consideration of methodology. They frequently focus on studies with controversies or practical applications, rather than new theories or research methodologies. In reporting the most interesting (a.k.a. most titillating) of the findings, journalism sometimes distorts the impression of the overall study. Meanwhile, press releases try to paint the most positive picture. Since they’re written for a general audience, and often not written by someone with a background in the field, they may be too general.

If we consider research blogging in this category, conversely, the writing may be too technical. It may be more commentary or critique than summary.

Wikipedia is gratis and libre. It’s written for non-specialists (in theory), but can also go into more detail. The main difference from AcaWiki is that most academic papers will not be “notable” enough to merit their own Wikipedia page; even if someone wrote them, they would probably get deleted. As an encyclopedia, Wikipedia provides a higher-level overview. There could be some other conflicts with Wikipedia policies, such as those against publishing original research or authors writing about themselves or their work.

All of the aforementioned resources have their uses, but as we can see, AcaWiki has its niche. I hope it thrives there.

In disclosure, I did some paid work for AcaWiki some months ago, but am not actively involved in the project.

Robert B. Townsend, assistant director for research and publications of the American Historical Association, said he was skeptical of the compact, at least based on what was released Monday.

“My ambivalence is the utter lack of clarity, and the tendency in most open access discussions to treat the science journals as normative,” he said. “The lack of recognition of the vast differences between disciplines makes this look like more of the usual one-size-fits-all open access thinking that prompted our efforts on the [National Humanities Alliance] report. I hope that report will have some effect on their thinking, if and when these universities try to turn their words into deeds, but I am not optimistic.”

It’s hard to guess the exact context of Dr. Townsend’s comments, but his comments here seem to be directed against the OA movement in general. He doesn’t criticize any specific aspect of the compact. It’s hard to interpret his intent as anything but obstructionism: “Stop — we haven’t figured out how to make this work for the humanities yet.”

The NHA report is generally a thoughtful look at publishing in the social sciences and humanities, with a particular idea as to how OA might work there. (The biggest flaw of the NHA report, which I mentioned in my comments on OAN, is ironically the biggest flaw of the OA compact: They devote all their energy to publication charges and pay only lip service to any other funding models — despite the fact that fewer than one-third of OA journals use publication charges.) Of course, societies in HSS are justified in trying to manage the transition to OA in the way that least disrupts the journals, but to date they have in general been overly cautious.

One could go farther and say that a little disruption can sometimes be a good thing. The NHA study concluded that the per-article costs in HSS journals are three times that of STM journals. I have yet to see any evidence that HSS journals are three times as valuable to their readers. In another context, those numbers would elicit adjectives like “bloated” and “inefficient”. Of course, in academia every field is exceptional and no one would suggest such a thing here. Each discipline must be allowed to retain its peculiar traditions, regardless of cost, because by God that is just the way things are done. But I digress.

There is indeed one size that fits all when it comes to scholarly knowledge, and that is that scholarly knowledge ought to be free. That certainly does not mean that every journal, publisher, or discipline must use the same revenue model, but they all need to get us to same outcome. The HSS community has not seriously contested that principle, but so far they have shown a lack of vision and creativity in getting us there (a charge I’ve levied against others before). If HSS societies are “not optimistic” about the OA compact or other current OA efforts, I should like to see them undertake more experiments of their own, rather than criticizing without proposing real alternatives. OA works for thousands of journals, hundreds in the social sciences and humanities, for many different types of journals and with many different revenue models. Let’s stop dragging our feet and make it happen.

[T]his study replicated [...] a study [...] that had first been conducted at The University of Alabama in 1996 by Carl Boening.

What follows is that the structure of the document and extensive lengths of text appears to have directly borrowed, as well. The data is original.

Does that meet the standards of originality and substance that we associate with a doctorate? Many online commenters don’t think so; obviously, the granting university and review committee (which included 2 of the same members as the earlier study) thought so.

But is it a contribution to the scientific discourse? I think so. In another context — one without the traditional expectations of a dissertation — would this behavior be inappropriate? I don’t think so.

I won’t make the argument for students, who are still trying to “prove” that they have learned and mastered the research methodology — but for professional researchers, I don’t see any reason why standards should hew to the traditional structures of a publication, foreclosing the possibility of publications which utilize more extensive copying and linking.

It appears that citation patterns already are changing due to online access to publications. Debate remains about exactly how the patterns are changing, but it’s clear that change is happening.

Not all change is good, but not all change is bad, either.

The bogeyman of plagiarism is meant to guard against appropriating another’s work as one’s own without giving them due credit. It would be a shame if the taboo of plagiarism scared the research community away from the evolution and experimentation that can come with new technologies.

The Law Library of Congress began harvesting legal blawgs in 2007. The collection has grown to more than one hundred items covering a broad cross section of legal topics.

Questions:

• What are the criteria for inclusion here? (It’s apparently curated, not an opt-in service like I suggested previously. Note that the two strategies are not incompatible.)
• The archived pages are openly accessible (just like pages in the Internet Archive). This is contrary to concerns which some had raised about copyright in either the blog or third-party content (like blog designs). Is there a special exemption for LOC? Or are they relying on the same argument as the Internet Archive (or Google’s cache, for that matter)? (P.S. Is the Internet Archive’s argument fair use or a specific statutory exemption? What about Google?)
• Did LOC ask the authors of these blogs for permission before archiving them, or do they consider it part of their general duty to collect and preserve? (For that matter, do the authors of these blogs even know they’re being archived by LOC?) Does it matter, either as a matter of etiquette or of law?
• The archiving frequency for all the blogs seems to be monthly. Should it be more often, less often, dependent on the blog (some are updated more frequently than others) …?
• Are other branches of LOC harvesting blogs in those subject areas, too? Are other national libraries doing this?
• What’s the history of the program? What’s the case they make for doing this? What’s it cost?

I don’t do much with preservation, so I plead ignorance. Is anybody preserving scholarly blogs (or other “grey” online scholarly resources), other than general-purpose national archiving and the Internet Archive? (Is it en vogue to do Web archiving for certain domains/types of contents vs. general purpose?)

Also, a suggestion: What about an opt-in archiving service for scholarly blogs? (Not opt-in for an individual page, as with WebCite, but for an individual blog/site, to be harvested on a recurring basis.) I don’t suggest it as the end of a solution, but as the beginning of one. An interested library or archive (or consortium) could, I assume, provide such a service fairly cheaply (e.g., someone could build the service in their basement over a weekend). The opt-in could be a simple Web form, asking for the URL of the site and some metadata, maybe also getting a license to provide open access to the preserved copies if the original goes dark. (P.S. Preservation/copyright experts: Is such a license needed?) This might provide a higher level of service than general-purpose Web archiving, e.g. the ability to categorize sites by scientific domain or topic, more frequent/robust archiving than is accorded to pictures of cats with captions, etc.

I imagine I’m glossing over an entire body of literature on the topic. Hopefully someone will let me know where I’m wrong!

Instead of databases becoming available as a result of Freedom Of Information Act requests, government officials should be required to justify why any public data should not be freely available to the taxpayers who paid for its creation.

It struck me that there’s very different politics in these two different scenarios:

• “We need information about topic X. We should build a database of data/documents about topic X.” Later: “Hey, now that we have this useful database, maybe we should share it with the public.”
• “Let’s build a database about topic X for the public.”

In the former case, you’re making an argument about value to the institution. Once the information trove has been established for a limited audience, there’s then (especially where public money is involved) an equity concern about other audiences who might benefit from the information but don’t have access.

In the latter case, it’s an argument about the nebulous value to the “public”. Sometimes, a do-gooder idea just doesn’t get the traction as an idea framed as necessary to achieve a self-interested goal.

So that brings us to scientific publications. The most effective route to open access is requirements by research funders and institutions that their authors provide OA to their manuscripts accepted for publication. Growing handfuls of funders and institutions have adopted these policies, but they’re still only handfuls. Why?

There seem to be three main clusters of concerns about these policies:

• Concerns about the infrastructure: paying for and maintaining a repository, locus of deposit, digital preservation, etc.
• Concerns about author effort: the burden on authors of depositing a copy of their manuscripts
• Concerns about open access: the impact of OA on scholarly publishers, etc.

There are other concerns that don’t easily fit into these categories (e.g. about the proliferation of conflicting/unedited versions of the same publication, how to enforce compliance, scientific interference by the funder/institution, etc.) but most of people’s concerns really seem to be about the first three (and most especially about the latter).

What if the argument was about creating internal repositories? We take concerns about open access off the table (as well as the conflicting versions concern). What’s left are entirely questions of bureaucracy, technology, and a little money. The tech is stable and cheap (if not very sexy); it’s not hard to educate authors about the requirement, and deposit itself only takes a few clicks.

For those minor costs, we get these benefits:

• Research assessment: Guaranteed, single-source access to authors’ publications for tenure/promotion/grant review
• Internal access: Guaranteed access to publications by your institution’s authors to your researchers and students
• Publicity: Provide a list of publications (metadata) by your institution’s authors to the public and other researchers — while avoiding the cluster of questions about open access
• Experience: Develop the infrastructure and familiarity with the service, for authors and readers as well as administrators. (This will make it easier later to suggest you “flip the switch” to OA as the default.) Even better if your repository provides services that actually help authors, like author pages or facilitated grant reporting.
• Open access where desired: Even if it’s not mandatory, the institution can encourage authors to make their publications OA and provide support to help them through the process. For skeptics, this should sound like the best of both worlds: support for authors who want OA but no requirement for authors who aren’t sure. This approach is very unlikely to achieve high rates of OA, but I think it’ll get better results than where self-archiving isn’t mandated. Authors can also send individual copies via the request a copy button.
• Other uses: Of course, your repository can also house student theses and dissertations, faculty educational resources, digitized content from the library and archives, and administrative and historical documents about the institution.

Doesn’t that sound like the “essential infrastructure” we were promised repositories would be?

If my idea works (and I think it will), you’ll develop a low-cost, high-value repository. Then (more) people will start to ask, hey, why are keeping this locked down; why don’t we open it all up? Cue Rep. Honda… (This doesn’t much happen now at institutions without requiring self-archiving, since the repositories have so little content they’re not of much value — accomplishing little other than driving repository managers into therapy.)

I think of this as a twist on the immediate deposit/optional access strategy (or the dual deposit/release strategy). In those models, it’s framed as an open access strategy. Moreover, open access is the default, with an optional embargo.

This is preferable at institutions and funders where the political will for an open access mandate exists. But I don’t think that’s yet everywhere, unfortunately.

Funders and institutions not ready for an open access mandate should be able to adopt a self-archiving mandate for their internal repository, though. I don’t know how they do research assessment without it.

This is effectively the policy commonly applied to theses and dissertations: require submission (deposit) of ETDs, provide access within the institution, provide an option for open access.

For funders and institutions without the wherewithal to adopt an OA mandate yet, it’s probably the best compromise step. If an OA mandate is low feasibility but high impact, and deploying a repository without requiring self-archiving is high feasibility but low impact, this should be somewhere in between: feasible enough to get it done even within a skeptical institution, but impactful enough to make it more worthwhile.