Yesterday, the Obama administration apparently thought it the better part of valor not to continue to push any statistic about 98% of Catholic women--e.g., that they "have used" non-NFP contraception. So they took down the link to that claim.
Today, though, the Guttmacher institute has decided to double down on that statistic. Humorously enough, when Rachel Jones, one of the authors of the Guttmacher report that kicked off this discussion, was initially contacted by the Washington Post blog, she seemed a tad...impatient, not to say clueless, about any statistical shortcomings in her previous report. She seemed to think she could dismiss the matter with a little snark about not including "89-year-old women."
The first thing you should notice is that the new table is based on "unpublished tabulated data from the NSFG 2006-2008." Translation: We plebes can't check this stuff out independently. Rachel & co. have it on their computers from the Centers for Disease Control and expect you to take their statements as gospel. Let's please continue to bear in mind what I have said in many comments on the thread: Jones did not "do a study." Guttmacher did not "do a study." They are using and interpreting data from a huge study on many topics done by the Centers for Disease Control.
The second thing you should notice is that there isn't a lot of additional detail to this table beyond the original bare assertion about 99% of all women having used non-NFP contraception and 98% of Catholic women having done so (sometime, somewhere). About the only thing this adds is the total size of the population surveyed in the 2006-2008 large survey plus some alleged breakdowns by other religious groups (such as mainline and evangelical Protestants).
So this is a very, very slightly dressed-up version of the original assertion, made by people who are not the original researchers, based on data to which they are not giving the public access in any detail.
Oh, and one more thing to notice as a preliminary: This additional material was produced in haste, in the last couple of days, by the Guttmacher Institute, for an avowedly political purpose, in the midst of a political controversy over their previous unsubstantiated claims. Not exactly the sort of situation terribly conducive to scientific care, objectivity, and accuracy in the analysis of the NSFG data.
Okay, to further details. First, here are the...
Things we already knew that bear repeating
--Virgins are excluded. Since the claim was always about "women who have ever had sexual intercourse," virgins were always excluded. Still, it's worth noting again, since this is supposed to have something to do with whether people are following Catholic Church sexual teachings.
--Only women between the ages of 15-44 are included.
--The highest statistic given by Guttmacher and the administration (the "98% of [non-virgin, ages 15-44] Catholic women") is about whether women have ever used a method of contraception. Ever. As in perhaps only once. This is not about whether women (including Catholic women) are presently using contraception, about their on-going habits, about what they can be expected to continue doing. I noted this from the beginning, anticipating that perhaps the "ever-used" statistic would be something Guttmacher would try to double down on. Actually, current use is a much more interesting and pertinent thing to examine. A statistic based on tabulating data concerning what a woman has ever done over possibly decades of her life risks giving the impression that it describes behavior that is regular, typical and on-going when it could easily be including statistics about behavior that is, at most, occasional or sporadic, or even highly unusual or unrepeated. In the context of a public policy debate where the point is supposed to have something to do with women's on-going use for contraception, the pertinence of an "ever-used" statistic is dubious in the extreme. (It goes without saying that it is irrelevant to the religious liberty issue.)
This is especially noteworthy since the Guttmacher patter attempts to talk about future expectations in its conclusion:
Importantly, the vast majority of women who are at risk but are not using contraceptives have used a method in the past and will most likely do so again in the future.
Head-shaking is called for. This sentence occurs in the course of attempted damage control concerning the revelation that 11% of sexually active women who are "at risk of" pregnancy and aren't actively trying to get pregnant are nonetheless using no method of birth control. Whatever else it may do, a statistic about how many women have ever used contraception at any point in their lives tells us little about the conclusion, "Women not presently using contraception even though they are sexually active and not trying to get pregnant will most likely use contraception again in the future." Indeed, one can easily imagine situations in which a woman at one life stage used contraception but did not do so later (e.g., after marriage, after converting to Catholicism, after finding that she enjoyed motherhood, after deciding that her fertility wasn't all that high, after deciding that she doesn't enjoy using contraception, after realizing how much her husband wanted children, or whatever).
Rachel Jones is pretty adamant about not getting it on this point. In response to Ezra Klein of the Washington Post blog, she said,
But the policies being implemented right now are ones that don’t effect [elderly women]. Right here and now, we’ve got 98 percent who have ever used a contraceptive method. Those are who will be impacted by this
See what she is doing? She is assuming that a 44-year-old woman who has "ever used" contraception, perhaps decades ago, is more "Impacted" by legislation mandating funding for contraception now than an 89-year-old woman. How does that work? If neither woman wants contraception anymore, then even if the 44-year-old is still fertile, she has no more use than the 89-year-old has for contraception! Jones is absolutely insistent on assuming that if a woman has ever used contraception and is presently of child-bearing age, she will "need" it again. That simply doesn't follow.
It's not even scientific.
But there's more. There are also...
Reasons to think even this qualified and irrelevant statistic is at least somewhat inaccurate
That is, specific reasons, in addition to those already noted about the haste with which these "supplements" were produced and the political context.
Because we do not have anything like the underlying data on which this new little chart is based, and in particular don't have the religious data, it's impossible for the moment to do further checking directly on the claim about Catholic women.
What we do have, however, is this document, brought to my attention by a reader in an earlier thread. This CDC report apparently doesn't include any religious data (so Guttmacher is claiming to get that from unpublished stats). However, take a look at Table 1 on p. 18, concerning what methods all women have ever used to avoid pregnancy, and look at the column for the 2006-2008 survey, which is what Guttmacher claims to be basing its statement on. Behold! We do find that 99.1% of total women have "ever used" "any method" of contraception. We also find a list of "contraceptive methods" in the chart below.
The alert reader will notice that the listed percentages of women who have used methods sum to far more than 100%. That's to be expected, since we're looking at "ever-used" statistics. A woman might quite easily have used more than one "contraceptive method" and was permitted to list more than one. You will also notice that no overlap information is given, which makes it impossible to tell from this table what the probability was that, say, a woman would have ever used the birth control pill given that she had ever used NFP or vice versa. Which brings me to something very interesting indeed: Natural Family Planning is simply listed as another method of contraception!
What does this mean? This means that, while this table could portray a situation in which 99% of women in the survey had ever used non-NFP methods (including some who had also used NFP), this table from the CDC could also include women in the 99% total who had only ever used NFP.
There is quite a high percentage of women in table 18 from the CDC who had ever had sexual intercourse with a condom: That is 93%, the most popular non-NFP method listed. So this means that at least 93% of the women in the survey had "ever used" a non-NFP method--condoms, at least. The next most popular method was the birth control pill, at about 82%.
The point is, however, that the most one can say from table 18 is that 93% of women had definitely (ever, at some point in their lives) used a non-NFP method of contraception. Perhaps no more. Because the 99% is of "all methods" including NFP.
Is it possible that Guttmacher has some additional data, not available here, that gives them sufficient information to tell them for sure that the 99% really is entirely composed of women who "have ever used" non-NFP methods? It's possible. But look at table 53 in this 2002 CDC document on an earlier cycle of the NSFG. It has the same features--methods summing to more than 100%, no overlap data available, NFP counted as a "contraceptive method." I think this gives us some reason to believe that at some point along the line the Guttmacher tabulators simply didn't notice that NFP is being listed as a "contraceptive method."
How much would such a mistake have affected the 98% "ever used" claim for Catholics? Well, again, we don't know, because we don't have public data this detailed by religion. It seems not too implausible that, as in the actual published CDC documents, condoms would be the most popular "ever used" non-NFP method and that that number would be lower than the total "ever used" numbers. How much lower? Again, we don't know. Might the number of "ever used condoms" (in their lives) for Catholic women still be in the 90%'s? Yes, it might. But 98% sounds so cool...
One more point: Condoms are evidently the most popular non-NFP type of contraception in the "ever-used" statistics, but they are cheap and are not what we are hearing about as required insurance coverage, needed "medical care," etc. Withdrawal is also a popular method in the ever-used table, and is cost-free. These points are relevant to Rachel Jones's silly comment to Ezra Klein that 98% of Catholic women are "impacted" by this legislation on the basis of their past use, ever, of any contraceptive method.
Summary of chief points:
*"Ever-used" statistics are even more removed from relevance to public policy matters than current use statistics (and neither addresses religious liberty issues).
*The new brief chart just published by Guttmacher is little more than the original bare assertion by Guttmacher dressed up in little squares. Insufficient further data has been provided to allow us to check its accuracy, and its accuracy is at least somewhat called into question by the haste and the politicized context in which it was produced in response to a critique of the earlier document.
*A comparison to independently published data casts doubt on the 99% ever-used non-NFP Guttmacher claim for all women. There is some reason to believe that such a claim cannot be substantiated at above 93% (for condoms) for all women. This point puts question marks over the 98% "ever used non-NFP methods" for Catholic women as well.