I welcome the opportunity to comment on the Department of Health and Human Services’ (HHS) advance notice of proposed rulemaking (ANPRM) on human subjects research protections. Modernizing federal regulations to better protect human subjects could advance science and bolster public trust in the research system, strengthening the economy and improving health while upholding human rights.

I submit these comments in my personal capacity, representing solely my own opinion. However, my background may inform the reading of these comments. I have participated in research as a human subject, including survey research as well as greater-than-minimal risk research. Professionally, I research government transparency, including topics such as scientific integrity, focusing on the role of information in improving lives and increasing democratic accountability. Previously, my work focused on public access to scientific information. In addition, I am a graduate student in the School of Library and Information Studies at Florida State University, studying information seeking and use.

In my opinion, several of the information management reforms proposed in the ANPRM could enhance the protection of human subjects and increase public trust in the research system. These changes could benefit the research system by aiding in the recruitment of human subjects. In addition, HHS should consider other reforms not specifically proposed in the ANPRM.

I recommend that HHS consider amending the federal rules on human subjects protection to:

1. Respect and enhance scientific openness;
2. Empower research participants and their communities;
3. Minimize and mitigate information risks; and
4. Collect the data necessary for system oversight.

1. Human subjects research protections should respect and enhance scientific openness

Openness is a fundamental characteristic of science, and human subjects research protections should respect and enhance scientific openness. I applaud HHS’ consideration of increasing the reuse of existing data and biospecimens (see Question 23). HHS should seek ways to expand appropriate sharing of data and biospecimens, while mitigating information risks to such sharing (see #3).

Openness can strengthen human subjects protections in other ways as well. Articles 20 and 21 of the Helsinki Declaration prohibits the repetition of a study where the outcome is already known, to avoid exposing human subjects to unnecessary risk. As a result, prompt and widespread communication of results is necessary to ensure human subjects are protected, along with sharing and reuse of data and biospecimens. To avoid unnecessarily exposing human subjects to the risks of research, HHS should ensure that the results of human subjects research are rapidly and effectively disseminated to the research community, such as under the National Institutes of Health (NIH) Public Access Policy.

2. Human subjects research protections should empower research participants and their communities

Although human subjects protection is the responsibility of the research and oversight community rather than of the subject, federal regulations should empower subjects to protect themselves to the greatest extent possible. HHS’ consideration of mechanisms to improve informed consent is particularly important in this regard (see Questions 35-53). HHS may also wish to consider readability and comprehension testing to improve consent forms. HHS should also ensure that subjects and potential subjects are informed of potential risks through public access to the proposed database of adverse events reporting (see Question 69).

With regard to the extension of federal regulations to some non-Federally funded research (Question 71), without prejudice to the resolution of the overall question, it is important that subjects and potential subjects be informed of the protections applicable to the context. HHS should require studies to inform subjects whether or not the federal human subjects research protections apply to the study, with a brief description of said protections and a reference to an easy-to-understand website with additional information.

Because the goal of human subjects research is ultimately to improve health, it is important that human subjects research protections leverage opportunities to do so, both for the subjects specifically and for their communities. Access to information is a key method for doing so. For instance, results should be returned to research participants whenever possible, including publications describing overall results of the study (see Question 18). Public access to research results (see #1) and to trial information (see #4) also would advance this aim.

3. Human subjects research protections should minimize and mitigate information risks

The adequate protection of research participants’ privacy is critical to the functioning of the research system and an important factor in the successful recruitment of human subjects. To prevent the damage that data breaches could wreak on the research system, human subjects research protections should ensure proper data security, such as by establishing mandatory standards for data security as proposed in the ANPRM. In addition, promptly notifying individuals of data breaches, as proposed in the ANPRM, is important to mitigate the impact of any security failures.

However, while it may be necessary to reexamine the role of institutional review boards (IRBs) in ensuring data security, HHS should be hesitant to eliminate such a role altogether. Through their approval and oversight, IRBs are the institutional guarantors of human subjects protection, and HHS should be cautious to remove vital aspects of that protection from the boards’ purview. HHS should carefully consider the most effective role of IRBs in ensuring data security: for instance, whether inspector certification of compliance and retrospective IRB audits would ensure protection.

4. Human subjects research protections should collect the data necessary for system oversight

The ANPRM correctly identifies the need for effective data collection and information management in conducting oversight of human subjects research protections. However, HHS should consider expanding required reporting where necessary, in addition to streamlining existing reporting requirements.

For instance, with regard to Question 70 re: ClinicalTrials.gov, the recent report of International Research Panel of the Presidential Commission for the Study of Bioethical Issues identifies needed improvements to trial registration and reporting, including increased reporting and public access to information.

In another example where increased reporting and transparency could strengthen oversight, Menikoff in 2010 proposed that investigators should disclose their consent forms. Such disclosure, the author argued, would ultimately improve the quality of consent forms, in addition to allowing potential subjects to more easily locate appropriate trials.

Sincerely,

Gavin Baker

The resolution itself would have been cutting-edge five years ago. The text is weak compared to policies at leading institutions: it “endorses the storage and preservation of scholarly publications in Florida State University’s open access institutional repository”, directs the libraries to “develop policies and procedures”, and calls for an annual report. Unfortunately, that’s as far as it goes. Based on the plain text of the resolution, there’s no mandatory deposit, the key element of successful open access policies. As a result, we can expect compliance to be weak. However, as Micah Vandegrift, Scholarly Communications Project Manager at FSU, notes, it’s a first step. Hopefully this resolution will spark a dialog, creating greater awareness and understanding, leading to the adoption of a mandatory policy in the near future. Meanwhile, kudos to the FSU faculty and those who worked to develop this policy.

Of course, I would be remiss not to add that my alma mater, the University of Florida, also has taken some positive steps in the open access arena lately (see: this, this, this).

On July 17, I submitted these comments on the proposed revisions:

I am disappointed that the National Science Foundation’s proposed Merit Review Principles and Criteria removes the criteria’s reference to the broad dissemination of results. NSF should retain and strengthen this consideration of merit review, as I wrote in my previous comments to the Task Force on Merit Review; see http://www.gavinbaker.com/2011/03/15/nsf-review/. Broad dissemination must be a high priority for NSF-funded research.

If NSF proceeds with the proposed criteria, it should ensure that its guidance for investigators, reviewers, and staff resolutely expresses the importance of ensuring the broadest possible access to research results. As I wrote in my previous comments, this should include updating the list of representative activities provided to investigators, particularly to encourage investigators to provide open access to their publications.

Thank you for the opportunity to comment on the proposed merit review criteria. Please do not hesitate to contact me with any questions.

What do you see as the strengths and weaknesses of each criterion?

The current Broader Impacts merit review criterion includes the consideration, “Will the results be disseminated broadly to enhance scientific and technological understanding?” NSF should retain and strengthen this consideration of the merit review.

Broad dissemination must be a high priority for NSF-funded research. As a federal agency operating with taxpayer funding, NSF has a responsibility to maximize return on investment by removing barriers to access for the scientific community, as well as to ensure access for taxpayers. In addition, broad and equitable dissemination of scientific information advances the goals of the Universal Declaration of Human Rights, which affirms that “everyone has the right freely to … share in scientific advancement and its benefits,” and the International Covenant on Economic, Social and Cultural Rights, in which governments agree to take the steps “necessary for the conservation, the development and the diffusion of science;” the United States is a signatory to both documents.

Under the current criteria, which are more than a decade old, proposals may be inadequately reviewed with regard to the dissemination consideration. During that time, the landscape of scholarly publishing and information has undergone significant changes. In particular, the exceptional opportunities created by the Internet behoove NSF to ensure that its funded research takes full advantage of the new technology to maximize cost-effective dissemination.

At present, broad dissemination of results is promoted through two avenues at NSF: the merit review criteria, which investigators must address in their proposals, and the Award and Administration Guide (AAG), which governs projects after an award is issued. This dual approach is beneficial and should be maintained and strengthened.

NSF establishes minimum requirements for dissemination in Chapter VI.D.4 of the AAG. In particular:

• “Investigators are expected to promptly prepare and submit for publication … all significant findings from work conducted under NSF grants;” and
• “Investigators are expected to share with other researchers, at no more than incremental cost and within a reasonable time, the primary data, samples, physical collections and other supporting materials created or gathered in the course of work under NSF grants.”

This policy is underpinned by Sec. 7011 of the America COMPETES Act of 2007, which requires NSF to enforce the standards by making any researcher who fails to comply ineligible for future funding.

Additionally, Sec. 7010 of the 2007 COMPETES Act requires NSF to make project reports freely available to the public online, along with citations to any publications resulting from NSF funding. This provision of the law is implemented by Chapter II.E.3 of the AAG, which requires grantees to submit a report describing the project outcomes, written specifically for the public, to be made freely available via Research.gov.

In addition to the baseline standards of the AAG, investigators are required to address the dissemination consideration of the Broader Impacts merit review criterion. This dual approach is beneficial because it encourages creative approaches to dissemination beyond the minimum, as appropriate to the proposed activity. Moreover, by including dissemination in the merit review criteria rather than only in post-award requirements, NSF ensures that investigators plan for dissemination beginning from the proposal stage, a valuable way to keep dissemination in mind throughout the project’s life cycle.

Unfortunately, both aspects of this approach currently are insufficient. Both the dissemination requirements of the AAG and the dissemination consideration of the Broader Impacts criterion should be updated and strengthened.

While NSF is to be recognized for its leadership in recently requiring investigators to develop data management plans, in other regards the AAG is outdated and should be reformed. Most importantly, NSF has not implemented a requirement that its funded investigators provide public access to resulting peer-reviewed manuscripts, rather than merely the citations to those publications. Among federal science agencies, the National Institutes of Health (NIH) is the current leader in this area, having adopted a mandatory public access policy as required by Sec. 218 of the Consolidated Appropriations Act, 2008. Dozens of other public and private research foundations worldwide have successfully adopted similar policies. NSF should incorporate a similar mandatory policy in the AAG. Furthermore, NSF should improve on the NIH policy by reducing the “embargo” period when manuscripts can be withheld from public access from the current maximum of twelve months to a six-month maximum, as several other research funders worldwide have done.

However, as this comment is directed to the Task Force on Merit Review, I will focus on the dissemination consideration of the Broader Impacts criterion, rather than the requirements of the AAG. To be clear, even if the dissemination requirements of the AAG are strengthened, the dissemination consideration of the merit review criteria also should be retained and strengthened.

The first regard in which the dissemination consideration of the merit review criteria is inadequate is its construction. Currently, broad dissemination is a consideration only of the Broader Impacts criterion, not the Intellectual Merit criterion. However, dissemination is properly understood as fundamental to both the intellectual merits of the proposed activity as well as its broader impacts.

The thrust of the current consideration is that maximizing the social value of the funded project requires communicating the project’s results to relevant audiences outside the research community, such as industry and policymakers, and to the public to enhance the public understanding of science. This is useful strategy to increase the broader impact of NSF funding and should be maintained. In particular, it supports the requirements of Sec. 526 of the America COMPETES Reauthorization Act of 2010, which directs NSF’s Broader Impacts criterion to advance the goal of “increased public scientific literacy.”

However, even within the academic community, the broad dissemination of research results cannot be taken for granted. Access barriers imposed by the high and rising cost of serials and monographs can significantly hamper the circulation of knowledge. These barriers can be particularly imposing to researchers and students at smaller institutions and in developing countries. Additionally, reticence to share data or materials with other researchers, or delays in doing so, also hinder the progress of science.

Thankfully, online technologies enable innovative approaches to the broad dissemination of research information which previously was only shared in small circles. These approaches already have begun to bear fruit. For instance, the NIH-supported Alzheimer’s Disease Neuroimaging Initiative was recently highlighted by the New York Times for its innovative approach to data sharing which is already being emulated. Given the promise of openness, the Board should ensure that the merit review criteria promote the broadest possible dissemination of results.

What changes, if any, would you like to see made to the merit review criteria?

If the Board retains the current criteria, it should add a consideration for broad dissemination in the Intellectual Merit criterion, in addition to the current consideration in the Broader Impacts criterion. Such a consideration might read, “Will the proposed activity ensure the broadest possible access to its results within its own field or across different fields?”

Alternatively, the Board might adopt a single consideration that addresses broad dissemination both within and beyond the research community.

What role should the institution play to ensure that the intellectual merit and broader impacts in NSF proposals can be realized?

NSF should provide additional guidance to proposers on how best to address the dissemination consideration. The Task Force will be aware of Sec. 526 of the COMPETES Reauthorization Act, which directs NSF to better inform proposers as well as staff and reviewers about the Broader Impacts criterion’s requirements, among other changes. The process of implementing these statutory provisions offers a timely opportunity for NSF to suggest “proven strategies and models” for cost-effective broad dissemination.

Currently, NSF provides a list of examples of representative activities for the dissemination consideration. These examples should be updated to better take advantage of proven strategies. Most critically, NSF should encourage investigators to provide open access to their publications by publishing them in an open access journal or by depositing them in an open access repository immediately upon publication, under an open copyright license. NSF could also encourage investigators to consider the impact of their publishers’ policies and pricing on the broad dissemination of their research. Further, NSF should encourage investigators to post their data online for free public access, and to freely distribute online any software created resulting from NSF funding under a free software license, depositing both in appropriate repositories as applicable.

Beyond mere suggestions of activities, NSF should refer investigators to resources on how to accomplish these activities, such as existing guides prepared by the library community and others. NSF should also encourage institutions to play a more proactive role in supporting investigators in maximizing the dissemination of their research, including collaboratively sharing resources and strategies between institutions.

Additionally, NSF should ensure that these resources and guidance are available to its reviewers and staff, and that they duly take into account these activities in reviewing proposals’ commitment to broad dissemination of research.

What impact, if any, has NSF’s two review criteria had on how you think about developing your research projects?

Any other comments?

Thank you for the opportunity to comment on the merit review criteria. Please do not hesitate to contact me with any questions.

In late 2006 or early 2007, I was looking for ways to get students interested in open access. I had started to become versed in the topic myself a few months earlier, after my library announced it planned to cut subscriptions around the same time the Federal Research Public Access Act was introduced for the first time. At the time, there were no resources for students and no student organizations meaningfully engaged with the issue. I helped the Alliance for Taxpayer Access scrape together some basic information for and about students, but no one paid much attention.

At some point, I had the idea of picking a day to try to focus student attention on open access. We’d choose a date and ask our few student allies to organize some activities to speak out on the issue. This became the National Day of Action for Open Access.

We didn’t have much lead time to plan, and few resources. Not a lot of people participated — but a few did. There wasn’t much attention, but we did get an article in the Washington Post, where I went completely off-message. (Coincidentally, the reporter was Rick Weiss, who later edited Science Next, which included an essay by me about open access.) It was a start.

By the next year, I was consulting for SPARC. We decided to revive the concept, but shifted the schedule and the focus: not just students, we wanted everybody to make noise about open access. For Open Access Day 2008, we had more time and more resources. In organizing it, I dropped the ball too many times, but thankfully someone was always there to pick it up. The response was much bigger; we made a splash.

After 2008, the organizers made two strategic decisions which I disagreed with at the time but were absolutely right. One was to expand the day to a week to make scheduling easier. The other was not to organize a central event, but instead to rely more on the partners and hosts to take more initiative. I was afraid we’d have insufficient focus and momentum. Instead, we let a hundred flowers blossom. The more flexible schedule, along with an increased role for partnerships — and our experience and increased visibility from the first time around — combined to make Open Access Week the most vibrant outing yet. The breath and depth of activities worldwide, along with a number of high-profile announcements timed for the week, are truly remarkable. I haven’t been very involved since the early strategic planning, so I can’t claim much credit. But I am thrilled and impressed with the outcome.

Most personally touching for me are the events in Cuba. Growing up in Florida, Cuba was only 90 miles across the strait but impossibly far culturally. There is no direct fiber optic link, nor even direct postal service, between Cuba and the U.S.; as an American, I need special permission from my government to travel there. Reportedly, only 2% of Cubans have Internet access. So it was a revelation to realize that our message of open access to scholarship had resonated in Cuba. For me, it’s a symbol of what open access is all about: the free exchange of knowledge and ideas worldwide.

Happy Open Access Week. May it be the first of many.

So what’s the point of AcaWiki when almost all articles have abstracts, which are summaries and usually available gratis? Well, AcaWiki summaries are also libre (CC Attribution license), so they invite reuse: mashup, translation, and so on. They’re also editable, so they can evolve and be improved.

Abstracts vary widely, usually shaped by the journal’s format: sometimes they’re several paragraphs, something just a few sentences. They might outline the methodology or they might not. They are usually written at the level of specialists in that field, so they may or may not be much use to other readers.

There’s room for improvement and innovation in the world of summary, in other words. For instance, Emerald launched a program asking authors to provide a summary highlighting potential applications. RNA Biology requires its authors to write up their findings on Wikipedia. BMJ publishes only one-page abridgments in its print edition, with the full article available online.

For a more direct comparison, see WikiSummary, which predates AcaWiki but covers only political science.

Two other points of comparison: journalism / press releases and Wikipedia.

Press releases are gratis; science journalism may or may not be gratis; both are rarely libre. They only cover new studies: good luck finding coverage of an article from 1989. They rarely provide a full citation to the original article. They often discuss only the findings, with little consideration of methodology. They frequently focus on studies with controversies or practical applications, rather than new theories or research methodologies. In reporting the most interesting (a.k.a. most titillating) of the findings, journalism sometimes distorts the impression of the overall study. Meanwhile, press releases try to paint the most positive picture. Since they’re written for a general audience, and often not written by someone with a background in the field, they may be too general.

If we consider research blogging in this category, conversely, the writing may be too technical. It may be more commentary or critique than summary.

Wikipedia is gratis and libre. It’s written for non-specialists (in theory), but can also go into more detail. The main difference from AcaWiki is that most academic papers will not be “notable” enough to merit their own Wikipedia page; even if someone wrote them, they would probably get deleted. As an encyclopedia, Wikipedia provides a higher-level overview. There could be some other conflicts with Wikipedia policies, such as those against publishing original research or authors writing about themselves or their work.

All of the aforementioned resources have their uses, but as we can see, AcaWiki has its niche. I hope it thrives there.

In disclosure, I did some paid work for AcaWiki some months ago, but am not actively involved in the project.

[T]his study replicated [...] a study [...] that had first been conducted at The University of Alabama in 1996 by Carl Boening.

What follows is that the structure of the document and extensive lengths of text appears to have directly borrowed, as well. The data is original.

Does that meet the standards of originality and substance that we associate with a doctorate? Many online commenters don’t think so; obviously, the granting university and review committee (which included 2 of the same members as the earlier study) thought so.

But is it a contribution to the scientific discourse? I think so. In another context — one without the traditional expectations of a dissertation — would this behavior be inappropriate? I don’t think so.

I won’t make the argument for students, who are still trying to “prove” that they have learned and mastered the research methodology — but for professional researchers, I don’t see any reason why standards should hew to the traditional structures of a publication, foreclosing the possibility of publications which utilize more extensive copying and linking.

It appears that citation patterns already are changing due to online access to publications. Debate remains about exactly how the patterns are changing, but it’s clear that change is happening.

Not all change is good, but not all change is bad, either.

The bogeyman of plagiarism is meant to guard against appropriating another’s work as one’s own without giving them due credit. It would be a shame if the taboo of plagiarism scared the research community away from the evolution and experimentation that can come with new technologies.

I don’t do much with preservation, so I plead ignorance. Is anybody preserving scholarly blogs (or other “grey” online scholarly resources), other than general-purpose national archiving and the Internet Archive? (Is it en vogue to do Web archiving for certain domains/types of contents vs. general purpose?)

Also, a suggestion: What about an opt-in archiving service for scholarly blogs? (Not opt-in for an individual page, as with WebCite, but for an individual blog/site, to be harvested on a recurring basis.) I don’t suggest it as the end of a solution, but as the beginning of one. An interested library or archive (or consortium) could, I assume, provide such a service fairly cheaply (e.g., someone could build the service in their basement over a weekend). The opt-in could be a simple Web form, asking for the URL of the site and some metadata, maybe also getting a license to provide open access to the preserved copies if the original goes dark. (P.S. Preservation/copyright experts: Is such a license needed?) This might provide a higher level of service than general-purpose Web archiving, e.g. the ability to categorize sites by scientific domain or topic, more frequent/robust archiving than is accorded to pictures of cats with captions, etc.

I imagine I’m glossing over an entire body of literature on the topic. Hopefully someone will let me know where I’m wrong!