In a report issued today, the GAO examined evidence that abstinence education programs are effective on measures such delaying sexual activity and other justifications for their receipt of federal funds. In addition, GAO noted that although HHS and the states are required to assess the scientific accuracy of materials used in abstinence-until-marriage education programs, but this appears not to have been done in many cases. GAO noted that as “of October 2006, HHS’s ACF—which awards grants under two programs that account for the largest portion of federal spending on abstinence education—did not review its grantees’ education materials for scientific accuracy, nor did it require grantees of either program to do so.” And there are more failures.
The report notes that studies have failed to meet basic standards for evaluating the effectiveness of the programs. I guess it’s faith-based science?
GAO first laid out the requirements for a valid experimental design.
Most of the efforts of HHS, states, and other researchers to evaluate the effectiveness of abstinence-until-marriage education programs included in our review have not met certain minimum criteria that experts have concluded are necessary in order for assessments of program effectiveness to be scientifically valid. . . . The reports of two panels of experts, as well as the experts we interviewed in the course of our previous work, generally agreed that scientifically valid studies of a program’s effectiveness should include the following characteristics:
• An experimental design that randomly assigns individuals or schools to either an intervention group or control group, or a quasi-experimental design that uses nonrandomly assigned but well-matched comparison groups.
• Studies should assess or measure changes in biological outcomes or reported behaviors instead of attitudes or intentions.
However, GAO found that these standards are not being met by HHS, the states or other researchers.
Many of the efforts by HHS, states, and other researchers that we identified in our review lack at least one of the characteristics of a scientifically valid study of program effectiveness. Most of the efforts to assess the effectiveness of these programs have not used experimental or quasi-experimental designs with sufficient follow-up periods and sample sizes.
For example, ACF used, according to ACF officials, grantee reporting on outcomes in order to monitor grantees’ performance, target training and technical assistance, and help grantees improve service delivery. However, because the outcomes reported by grantees have not been produced through experimentally or quasi-experimentally designed studies, such information cannot be causally attributed to any particular abstinence-until-marriage education program.
Further, none of the state evaluations [GAO] reviewed that had been completed included randomly assigned control groups.
Similarly, some of the journal articles that we reviewed described studies to assess the effectiveness of abstinence-until-marriage programs that also lacked at least one of the characteristics of a scientifically valid study of program effectiveness. In these studies, researchers administered questionnaires to study participants before and after they completed an abstinence-until-marriage education program and assessed the extent to which the responses of participants changed. These studies did not compare the responses of study participants with a group that did not participate in an abstinence-until-marriage education program.
Many of the “studies” were aspirational rather than studying actual effectiveness
Most of the efforts we identified in our review used reported intentions and attitudes in order to assess the effectiveness of abstinence-until-marriage programs. For example, as of 2006, neither ACF’s community-based grantees nor OPA’s AFL grantees were required to report on behavioral or biological outcomes, such as rates of intercourse or pregnancy. Similarly, the journal articles we reviewed were more likely to use reported attitudes and intention s— such as study participants’ reported attitudes about premarital sexual activity or their reported intentions to remain abstinent until marriage — rather than their reported behaviors or biological outcomes to assess the effectiveness of abstinence-until-marriage programs.
Some studies that met valid experimental criteria fail to show that abstinence programs achieve desired behavior
Among the assessment efforts we identified are some studies that meet the criteria of a scientifically valid effectiveness study. However, results of these studies varied, and this limits the conclusions that can be drawn about the effectiveness of abstinence-until-marriage education programs. Some researchers have reported that abstinence-until-marriage education programs have resulted in adolescents reporting having less frequent sexual intercourse or fewer sexual partners. For example, in one study of middle school students, participants in an abstinence-until-marriage education program who had sexual intercourse during the follow-up period were 50 percent less likely to report having two or more sexual partners when compared with their nonparticipant peers. In contrast, other studies have reported that abstinence-until-marriage education programs did not affect the reported frequency of sexual intercourse or number of sexual partners. For example, one study of middle school students found that participants of an abstinence-until-marriage program were not less likely than nonparticipants at the 1 year follow-up to report less frequent sexual intercourse or fewer sexual partners. Experts with whom we spoke emphasized that there were still too few scientifically valid studies completed as of 2006 that could be used to determine conclusively which, if any, abstinence-until-marriage programs are effective.
Statutory requirements were not put in place when grant recipients disagreed with them
GAO found that grant recipients claimed that they were not bound by legal requirements to provide accurate information. Part of the problem, though, is due to an unclear statute.
During the course of our work on abstinence-until-marriage education, we identified a federal statutory provision—section 317P(c)(2) of the Public Health Service Act—relevant to the grants provided by HHS’s State Program, Community-Based Program, and AFL Program. This provision requires that educational materials prepared by HHS’s grantees, among others, that are specifically designed to address STDs, contain medically accurate information regarding the effectiveness or lack of effectiveness of condoms in preventing the diseases the materials are designed to address.
At the time of our review, an ACF official reported that materials prepared by abstinence-until-marriage education grantees were not subject to section 317P(c)(2). However, we concluded that this requirement would apply to abstinence-until-marriage education materials prepared by and used by federal grant recipients, depending upon the substantive content of those materials. In other words, in materials specifically designed to address STDs, HHS’s grantees are required to include information on condom effectiveness, and that information must be medically accurate.
Therefore, we recommended in a letter dated October 18, 2006, that HHS reexamine its position and adopt measures to ensure that, where applicable, abstinence education materials comply with this requirement.
The study is Abstinence Education: Assessing the Accuracy and Effectiveness of Federally Funded Programs GAO-08-664T, April 23, 2008
Comments
Add Comment