5.1.5 The Varieties of Evidence

Part: 
Two
Chapter: 
5

We have looked at a wide range of types of evidence. Some has come from personal experience of witnesses, some from professionals whose orientation is primarily clinical, some from experimental social scientists, and some from other forms of empirical science. We have not categorically refused to consider any type of evidence, choosing instead to hear it all, consider it all, and give it the weight we believe in the final analysis it deserves. No form of evidence has been useless to us, and no form is without flaws. A few words about the advantages and disadvantages of various types of evidence may help to put into perspective the conclusions we reach and the basis on which we reach them.

Most controversial has been the evidence we have received from numerous people claiming to be victims of pornography, and reporting in some way on personal experiences relating to pornography. In later portions of this Report concerned with victimization and with the performers in pornographic material we discuss this evidence in more detail. We have considered this firsthand testimony, much of it provided at great personal sacrifice, quite useful, but it is important to note that not all of the first-hand testimony has been of the same type.

Some of the first-hand testimony has come from users of pornography, and a number of witnesses have told us how they became "addicted" to pornography, or how they were led to commit sex crimes as a result of exposure to pornographic materials. Although we have not totally disregarded the evidence that has come from offenders, in many respects it was less valuable than other victim evidence and other evidence in general. Much research supports the tendency of people to externalize their own problems by looking too easily for some external source beyond their own control. As with more extensive studies based on self-reports of sex offenders, evidence relying on what an offender thought caused his problem is likely to so overstate the external and so understate the internal as to be of less value to us than other evidence.

Most of the people who have testified about personal experiences, however, have not been at any point offenders, but rather have been women reporting on what men in their lives have done to them or to their children as a result of exposure to certain sexually explicit materials. As we explained in the introduction, we do not deceive ourselves into thinking that the sample before us is an accurate statistical reflection of the state of the world. Too many factors tended to place before us testimony that was by and large in the same direction and concentrated on those who testified about the presence rather than the absence of consequences. Nevertheless, as long as one does not draw statistical or percentage conclusions from this evidence, and we have not, it can still be important with respect to identification and description of a phenomenon. Plainly some of these witnesses were less credible or less helpful than others, but many of the stories these witnesses told were highly believable and extremely informative, leading us to think about possible harms of which some of us had previously been unaware. Many witnesses have urged us to draw conclusions about prevalence exclusively from anecdotal evidence of this variety, but we have refused to do so. But that we have refused to make invalid statistical generalizations does not mean that we cannot learn from the stories of those with personal experiences. Many of their statements are summarized in the victimization section of this Report, and we urge people to consider those statements as carefully as we did. We can and we have learned from many of these witnesses, and their testimony has provided part of the basis for our conclusions. As in many other areas of human behavior, the most complete understanding emerges when a phenomenon is viewed from multiple perspectives. One important perspective is the subjective meaning that individuals attribute to their own experiences. This perspective and the unique experiences of individuals are less amenable to objective or statistical inquiry than certain other perspectives, and thus can be valuably examined through the kinds of witnesses whose statements we summarize later in this Report.

The evidence provided by clinical professionals carries with it some of the same problems. Although filtering the evidence through a trained professional, especially one who described to us the experience of numerous cases, eliminates some of the credibility problems, the problem of statistical generalization remains. Because people without problems are not the focus of the clinician's efforts, evidence from clinical professionals focuses on the aberrational. Consequently, clinical evidence does not help very much in answering questions about the overall extent of a phenomenon, because it too is anecdotal, albeit in a more sophisticated way and based on a larger sample. Still, clinical evidence should not be faulted for not being what it does not purport to be. What it does purport to be is sensitive professional evaluation of how some people behave, what causes them to behave in that manner, and what, if anything, might change their behavior. Clinical evidence helps us to identify whether a problem exists, although it does not address the prevalence of the problem. We have looked at the clinical evidence in this light, and have frequently found it useful.

The problems of statistical generalization diminish drastically when we look to the findings of empirical social science. Here the attempt is to identify factors across a larger population, and thus many of the difficulties associated with any form of anecdotal evidence drop out when the field of inquiry is either an entire population, some large but relevant subset of a population, or an experimental group selected under some reliable sampling method.

Some of the evidence of this variety is correlational. If there is some positive statistical correlation between the prevalence of some type of material and some harmful act, then it is at least established that the two occur together more than one would expect merely from random intersection of totally independent variables. Some of the correlational evidence is less "scientific" than others, but we refuse to discount evidence merely because the researcher did not have some set of academic qualifications. For example, we have heard much evidence from law enforcement personnel that a disproportionate number of sex offenders were found to have large quantities of pornographic material in their residences. Pornographic material was found on the premises more, in the opinion of witnesses, than one would expect to find it in the residences of a random sample of the population as a whole, in the residences of a random sample of non-offenders of the same sex, age, and socioeconomic status, or in the residences of a random sample of offenders whose offenses were not sex offenses. To the extent that we believe these witnesses, then there is a correlation between pornographic material and sex offenses. We have also read and heard evidence that is more scientific. Some of this evidence has related to entire countries, where researchers have looked for correlations between sex offenses and changes in a country's laws controlling pornography or changes in the actual prevalence of pornographic materials. Other evidence of this variety has been conducted with respect to states or regions of the United States, with attempts again being made to demonstrate correlations between use or non-use of certain sexually explicit materials and the incidence of sex crimes or other anti-social acts.

Correlational evidence suffers from its inability to establish a causal connection between the correlated phenomena. It is frequently the case that two phenomena are positively correlated precisely because they are both caused by some third phenomena.

We recognize, therefore, that a positive correlation between pornography and sex offenses does not itself establish a causal connection between the two. It may be that some other factor, some sexual or emotional imbalance, for example, might produce both excess use of pornographic materials as well as a tendency to commit sex offenses. But the fact that correlational evidence cannot definitively establish causality does not mean that it may not be some evidence of causality, and we have treated it as such. The plausibility of hypothesized independent variables causing both use of pornography and sex offenses is one factor in determining the extent to which causation can be suggested by correlational evidence. So too is the extent to which research design has attempted to exclude exactly these possible independent variables. The more this has been done, the safer it is to infer causation from correlation, but in no area has this inference been strong enough to justify reliance on correlational evidence standing alone.

The problem of the independent variable drops out when experiments are conducted under control group conditions. If a group of people is divided into two subgroups randomly, and if one group is then exposed to a stimulus while the other is not, then a difference in result between the stimulus group and the control group will itself establish causation. As long as the two groups are divided randomly, and as long as the samples are large enough that randomness can be established, then any variable that might be hypothesized other than the one being tested will be present in both the stimulus group and the control group. As a result, the stimulus being tested is completely isolated, and positive results are very strong evidence of causation.

The difficulty with experimental evidence of this variety, however, is that it is virtually impossible to conduct control group experiments outside of a laboratory setting. As a result, most of the experiments are conducted on those who can be induced to be subjects in such experiments, usually college-age males taking psychology courses. Even a positive result, therefore, is a positive result only, in the narrowest sense, for a population like the experimental group. Extrapolating from the experimental group to the population at large involves many of the same problems as medical researchers encounter in extrapolating from tests on laboratory animals to conclusions about human beings. The extrapolation is frequently justified, but some caution here must be exercised in at least noting that the extrapolation requires assumptions of relevant similarity between college age males and larger populations, as well as, in some cases, assumptions of causality between the effects measured in the experiment and the effects with which people are ultimately concerned.

Perhaps more significantly, enormous ethical problems surround control group experiments involving actual anti-social conduct. If the hypothesis is that exposure to certain materials has a causal relationship with rape, for example, then the "ideal" experiment would start with a relatively large group of men as subjects, would then divide the large group randomly into two groups, and would then expose one of the two groups to the pornographic materials and the other to control materials. Then the experimenter would see if the stimulus groups committed more rapes than the control group. Of course such an experiment is inconceivable, and as a result most experiments of this variety have had to find a substitute for counting sexual offenses. Some have used scientific measures of aggression or sexual arousal, some have used questionnaires reflecting self-reported tendency to commit rape or other sex offenses, some have used experiments measuring people's willingness to punish rapists, and some have used other substitutes. With respect to any experiment of this variety, drawing conclusions requires making assumptions between, for example, measured aggression and an actual increased likelihood of committing offenses. Sometimes these assumptions are justified, and sometimes they are not, but it is always an issue to be examined carefully. One final point about the experimental evidence presented to us is in order. Even with control group experiments, the ultimate conclusions will depend on the ability of the researcher to isolate single variables. For example, where there is evidence showing a causal relationship between exposure to violent pornography and aggressive behavior, the stimulus as just described contains two elements, the violence and the sex. It may be that the cause is attributable solely to the violence, or it may be that the cause is attributable solely to the sex. Good research attempts to examine these possibilities, and we have been conscious of it as we evaluated the research presented to us.