**************************************************************************************

Australian Journal of Psychology Vol. 22, No. 3, 1970, pp. 253-260.

(With three post-publication addenda following the original article)

THE DEVELOPMENT AND VALIDATION OF A BALANCED DOGMATISM SCALE



JOHN J. RAY

Macquarie University

Abstract

A "balanced" Dogmatism scale was produced by writing entirely new items to express sentiments opposed to those accepted by the archetypical Dogmatic. These items are not reversals of old ones. The scale of 36 items showed a reliability of .91 on the standardization (student) sample and .78 on an adult community sample. Validity was demonstrated for the scale as a whole and for the two halves separately.

INTRODUCTION

Since the publication of the California 'F' scale (Adorno et al., 1950) two major criticisms have been levelled against it as a measure of authoritarianism. 1. That it is ideology specific and slanted so as to cause conservatives automatically to obtain high scores; 2. That its all-positive wording causes acquiescence and authoritarianism to be confounded.

The first of these criticisms was largely met by the Rokeach (1960) dogmatism scale and Rokeach (1967), claims that his scale is sufficiently free from response bias to render a balanced version unnecessary. Although this may be true to some extent, it does seem desirable to lay the doubts about "the acquiescent personality" completely to rest where-ever it is possible to do so. The voluminous literature on the one-way wording of the 'F' scale is sufficient to inspire in every user of scales the desire to preclude from the beginning a "response set" interpretation of his own results.

Four attempts at writing balanced `D' scales have been made-by Peabody (1961), Haiman and Duns (1964), Stanley and Martin (1964) and Haiman (1964). The first three above all employed the method of taking an existing item and attempting to re-write it so as to reverse its meaning. All three were unsuccessful. Haiman (1964) however, did produce a reliable balanced scale. An important part of his method was to write some entirely new items rather than rely on reversals of existing ones. Most authors have been reluctant to do this, but the failure of the reversed item method seems to make it necessary. The only other alternative is the forced choice method (Berkowitz & Wolkon, 1964) but the basic assumption of this method is that the alternatives offered are opposites -- and it is precisely this that the failure of reversed item scales has shown not to be the case.

Haiman's thirty-item scale contained five `F' scale items, five reversed `F' scale items, five 'D' scale items, five reversed 'D' scale items plus five positive and five negative items written by himself. Unfortunately the ten items he himself contributed seem largely indistinguishable from what would normally be considered conservatism items. Haiman does state that it was his aim to write items having a specific political reference. We are told that these ten items make a disproportionate contribution to the scale's homogeneity, i.e. what this group of items measures is closest to what the scale as a whole measures.

It will be seen that Haiman's scale has irretrievably blurred the important distinctions between authoritarianism, conservatism and dogmatism that the original authors (Rokeach, 1960; Adorno et al., 1950) wished to make. For this reason, the labelling of his scale as measuring "open-mindedness" does seem rather arbitrary. Haiman appears to have put back the ideological specificity that Rokeach was at pains to take out.

Nevertheless, there is much to be said for Haiman's bold attempt to write new items and in this paper a scale will be presented wherein all negative items are new in content. In order to meet Rokeach's original aim of reducing ideology specificity, all positive items will be drawn solely from existing Dogmatism scales. Unlike Haiman's scale, no `F' scale or conservatism items will be included.

METHOD

Reliability Study

As a preliminary step, the forty item 'D' scale of Rokeach and the nine item Australian revision by Anderson & Western (1967) were administered to a group of 114 first-year psychology students at the University of Sydney. These results were combined and used to item-analyse the original scale and thus produce the most reliable form possible of the 'D' scale before further work with it was undertaken. The scale proved to be most reliable (.82) when all but 25 items were dropped. This was an improvement over the .78 reliability observed for the 49 item scale.

In the next step, the concept of Dogmatism and its potential opposites were discussed with a group of third year students in a course on attitudes conducted at Macquarie University. The opposite of dogmatism was agreed to be "open-mindedness" and a total of fifty-nine items was written by the students to tap this concept. By obtaining the items in this way it was hoped that some initial content validity could be guaranteed. These items were then included with the twenty-five 'D' scale items (mentioned above) in a questionnaire administered to forty-six second-year day and evening students at Macquarie University. Responses to each item were scored from 5 (strongly agree) to 1 (strongly disagree). The negative items were then divided into three sets, of twenty, twenty and nineteen items each and successively combined with the set of twenty-five `D' items and the usual item-total correlations computed. This procedure allowed new items to be selected with some view to their correlation with one another as well as their correlation with the original items. From these three versions of the balanced scale the twenty-four most successful negative items were chosen and combined with the twenty-four strongest `D' scale items ("strength" being assessed from correlations with the total score on the 'D' items alone). This `scale' was then reduced to a balanced thirty-six item length by dropping the items showing a low correlation with total scores on the combined scale.

Validation Study

Since the 'D' scale is taken as an already well-established instrument for which we are interested to provide a balanced version, it might seem that the only validation necessary is the information that the negative items correlate as well with the positive items as the positive items themselves correlate with the scale total. This, however, is not completely true. The whole reason for constructing a balanced scale was the claim that the `D' scale might really be measuring acquiescent tendency, and if this is so the empirical correlates with the 'D' scale heretofore observed might have been due to the characteristics of acquiescent people rather than of dogmatic people.

In other words, it should now be possible to examine for the first time empirical correlates of Dogmatism with the influence of acquiescence as an artifact removed.

The study undertaken was of the "criterion groups" type. Ideally, this type of study requires that the two groups differ on no other characteristic than that purportedly measured by the scale. For Dogmatism, such groups are very hard to find. The two groups eventually decided on were active Methodists and active Humanists. It was argued that both of these would have an equivalent high level of education and both would have a strong commitment to humanitarian ideals. The only difference would be the Methodists' adherence to a more or less clearly defined body of dogma. It was also decided to take the relatively unusual step for this type of study of going outside the University for subjects. It is a rather stringent test of a scale to administer it to a population for which it was not standardized but it was desired to eliminate the criticism (which can so often be levelled at studies in this area) that the relationships observed represent a psychology of university students rather than of the general population.

Through local Methodist church study groups and local Humanist meetings, responses were obtained from 120 people, sixty-three Methodists and fifty-seven Humanists. From each, measurements were obtained on the following seven variables: 1. Occupational Status-scored 1 to 7 where a high score indicates low status: 2. Education-scored 1 to 4 where a high score indicates high educational level: 3. Political Preference-scored 5 to 1 where a high score indicates conservatism of choice: 4. Belief scored 1 for Methodists and 2 for Humanists: 5. Dogmatism total score -- scored 180 to 36: 6. Dogmatism positive score -- scored 90 to 18: 7. Dogmatism negative score-scored 90 to 18. For all three 'D' scale scores, a high score indicates a highly dogmatic response.

Yeasaying

To answer questions about the empirical relationship between acquiescence and dogmatism (Frandsen, 1967) the Methodist-Humanist sample was divided into yeasayers (n = 80), naysayers (n = 4) and "normals" (n = 36). The basis for this classification was that 20 or more responses of "agree" or "Disagree" (out of 36) allocated the respondent to "yeasayers" or "naysayers" whilst "normals" were the residue. The means for all three groups were calculated on three "scales"-the first being formed of the eighteen Rokeach items, the second of the 18 balancing items and the third of the total thirty-six items.

RESULTS

The reliability obtained on the norming sample was .91. It should be noted that the scale is naturally balanced at this length, i.e. identical criteria for item selection were applied to both positive and negative items. The very high reliability obtained was in fact required, being a corollary of the other two requirements that only the strongest `D' scale items be used and that the negative items be equally as strong as the positive items. Such a high initial reliability is also important because the original norming maximizes both "error" and "true" inter-item correlation. Therefore some drop is always experienced on re-administration to another group of a freshly constructed scale. The items of the new scale are presented in Table 1.


TABLE 1

The Items of the Balanced `D' Scale. (The last 18 are negative items.)

1. Man on his own is a helpless and miserable creature.
2. There is so much to be done and so little time to do it in.
3. It is better to be a dead hero than a live coward.
4. A man who does not believe in some great cause has not really lived.
5. It is only when a person devotes himself to an ideal or cause that life becomes meaningful.
6. A person who gets enthusiastic about too many causes is likely to be a pretty "wishy-washy" sort of person.
7. When it comes to differences of opinion in religion we must be careful not to compromise with those who believe differently from the way we do.
8. A group which tolerates too much difference of opinion among its own members cannot exist for long.
9. There are two kinds of people in this world: those who are for the truth and those who are against the truth.
10. A person who thinks primarily of his own happiness is beneath contempt.
11. Most of the ideas which get printed nowadays aren't worth the paper they are printed on.
12. In this complicated world of ours the only way we can know what's going on is to rely on leaders or experts who can be trusted.
13. Most people just don't know what's good for them.
14. Of all the different philosphies that exist in the world there is probably one which is more in accord with reality than any of the others.
15. The person who is extremely tolerant of widely different and even conflicting viewpoints probably has few opinions of his own.
16. It is annoying to listen to a speaker or teacher who seems unable to make up his mind about what he really believes.
17. For most questions there is only one right answer once a person is able to get all the facts.
18. Although many details still remain to be worked out, we now have definite answers to most practical problems in life.
19. It's unfortunate that we have censorship in this country.
20. Live and let live is a good motto in life.
21. It doesn't matter much what religion a person follows.
22. There is good in everyone.
23. There is something to be appreciated in all forms of art.
24. I usually try to keep a fairly open mind on most issues.
25. It is possible that there are many facets to the "truth".
26. It is usually a help to get a new slant on an old problem.
27. There are few decisions which don't bear some re-consideration.
28. Criticism can perhaps be useful if it results in a reconciliation of opposing views.
29. It is usually unwise to indulge in generalizations since individual circumstances often alter cases.
30. Many problems have more than one acceptable solution.
31. What may be all right for one person isn't necessarily right for his neighbour.
32. The Churches don't give enough weight to individual conscience.
33. It is impossible to generalize about members of other races and cultures.
34. People cannot be expected to stick to the same opinions month after month.
35. Even intelligent people often change their opinions after hearing others' views.
36. People cannot be blamed for some inconsistencies in what they think.

The correlation between the positive and negative halves was .71.

The correlations observed with validity criteria etc. are given in Table 2. The reliability of the total scale for the second sample was .78. The reliability of the negative items considered as a separate scale was .64. For the positive items it was .80. This latter statistic indicates that the original `D' scale items function very similarly on both university and non-university samples.


TABLE 2

Intercorrelation of Seven Variables on 120 Subjects

........................Educ......Polit.....Belief......'D' tot........'D' pos........'D' neg.


Status.............. -.65....... .21....... .06.......... .24............. .29............. .03
Education..................... -.25....... .09......... -.33............ -.37............ -.09
Political Pref............................. -.53.......... .34............. .29............. .24
Belief........................................................ -.51............ -.40............ -.41
'D' total.......................................................................... .88............. .66
'D' positive........................................................................................ .22


Using a contrast-wise error-rate approach, all correlations above .16 are significant at the < .05 level.

Note from Table 2, that as planned, the Methodists and Humanists were empirically not differentiated by degree of education (r = .09) or occupational status (r = .06). Why these two variables are related to scores on the "positive" scale but not to scores on the "negative" scale is unclear but it is in any case not disturbing to the goals set by the study. To the extent to which we might expect dogmatism to be equally distributed throughout the population it is in fact a positive feature.

Results of the yeasaying study appear in Table 3. As the "naysayer" group turned out to be so small, it was felt that the only comparisons of interest were those between the "yeasayers" and the "normals". The yeasayers had significantly higher scores on the Rokeach items and the total scale but significantly lower scores on the balancing items (Rokeach items -- t (114) = 9.28; Balancing items t (114) = 2.42; Total scale t (114) = 4.34). This reduced difference for the balancing items and the reversed direction appears to be at least partly a scoring artifact. It is of course the difference on the total scale which is free of artifact and of most interest here.


TABLE 3 Mean Scores and S. D.s on Three Dogmatism "Scales" for Naysayers, Yeasayers and "Normals" Among the Methodist-Humanist Sample

Group................ Rokeach items.........Balancing items...........Total Scale


Normals...............43.36 (5.74) ..............41.58 (5.89)................84.94 (10.95)
Yeasayers............56.36 (9.08)...............38.62 (6.60)................94.98 (13.46)
Naysayers............36.00 (0.70)...............45.25 (3.34)................81.25 ( 3.70)



DISCUSSION

The reliability on the student sample was all that might be desired of a scale. Given this high initial level, the drop observed in the second study was still not such as to render the reliability unsatisfactory.

As a validation experiment, the second study was successful in that the correlation between Belief and Dogmatism was .51, which is significant and in the predicted direction. The significant correlation with political preference is contrary to Rokeach's expectation but consonant with his findings (Rokeach, 1956). It is in any case a feature common to both negative and positive halves of the scale. It is not unparalleled in other published studies (Direnzo, 1968; Kirtley & Harkness, 1969) and Rokeach (1960) has claimed that such a feature is not inconsistent with the validity of a dogmatism measure. Dogmatism is a variable of great interest in its own right (Rokeach and Fruchter, 1956) and the fact that it does empirically turn out to correlate weakly with right wing ideology is a fact about the world (albeit a fact disappointing to the hope that dogmatism might turn out to be the non-ideological component of authoritarianism). The charge by Simons (1968) that the relationship between dogmatism and conservatism is artifactual rests on the not surprising finding that dogmatism items are rated by student judges to be more characteristic of right wing ideology. This finding was one we might have expected from the correlations between dogmatism and ideology already observed among students and can just as well be explained by saying that the students have accurate perceptions of what goes on in the world as by saying that the dogmatism items have a built-in right wing bias. The drop in reliability (although the reliability observed here is in fact identical with that for the all-positive scale quoted earlier) was of course somewhat disappointing. This drop was almost entirely contributed by the negative items, as can be seen in the correlation of only .22 between the two halves of the scale. It still remains true, however, that what is measured by the scale is the intersection of what is measured by the negative and positive halves. The scale here is unlike the balanced 'F' scales that have been produced in that the reliability of these scales is normally very low (Berkowitz & Wolkon, 1964). In any case, this low relationship of the two halves is only true for a population unlike that for which the scale was designed. On the university student sample, the two halves correlated .71. This compares with .17 on Stanley and Martin's (1964) scale (student sample) and a range between .21 and .11 on Peabody's (1961) scale (student sample). What appears to have happened is that for non-university subjects this study has replaced the weaker half of the `D' scale by an equally weak but negative set of items. This does not affect the validity of the instrument as the correlation with belief is approximately the same for each half of the scale.

In interpreting the results of the acquiescence study it must be borne in mind that the aim adopted was to remove any systematic or artifactual influence of acquiescent response set on the final total score. The stronger aim of presenting a scale with which acquiescent tendency does not correlate significantly as a matter of empirical fact was deemed to be inappropriate here. It would in fact be to create another artifact -- one directly opposite to that originally attacked. Because the published literature does offer opposing findings on the relationship of acquiescence (yeasaying) to open-mindedness (Frandsen, 1967), results bearing on this question were presented here. As it turns out, when the acquiescence artifact in Dogmatism measurement was removed by use of the new scale, Dogmatic people are shown still to be more acquiescent (t (114) = 4.34). Users of this scale may therefore wish to partial out yeasaying score in experimental applications. This has not been possible heretofore because, on a unidirectional scale, the yeasaying score and the scale score are essentially identical.

CONCLUSION

A balanced and valid 'D' scale has been constructed with reliability between .91 (for the norming sample of students) and .78 (for an adult population sample). A validity coefficient of .51 was obtained.

REFERENCES

ADORNO, T. W., FRENKEL-BRUNSWIK, E., LEVINSON, D. J. & L SANFORD, R. N. The Authoritarian Personality, New York: Harper & Row, 1950.

ANDERSON, D. S. BC WESTERN, J. S. An inventory to measure students' attitudes, St. Lucia, Brisbane: University of Queensland Press, 1967.

BERKOWITZ, N. H. & WOLKON, G. H. A forced choice form of the 'F' scale-free of acquiescent response set. Sociometry, 1964, 27, 54-65.

DIRENZO, G. J. Dogmatism and presidential preferences in the 1964 elections, Psychological Reports, 1968, 22, 1197-1202.

FRANDSEN, K. D. Haiman's revised open-mindedness scale: A comparative study of response patterns, Speech Monographs, 1967, 34, 389391.

HAIMAN, F. S. A revised scale for the measurement of open-mindedness, Speech Monographs, 1964, 31, 97102.

HAIMAN, F. S. & DUNS, D. F. Validations in communicative behaviour of attitude-scale measures of dogmatism. Journal of Social Psychology, 1964, 64, 287-297.

KIRTLEY, D. & HARKNESS, R. Some personality and attitude correlates of dogmatism. Psychological Reports, 1969, 24, 851-854.

PEABODY, D. Attitude content and agreement set in scales of Authoritarianism, Dogmatism, Anti-Semitism and Economic Conservatism. Journal of Abnormal and Social Psychology, 1961, 63, 1-11.

ROKEACH, M. Political and religious dogmatism: An alternative to the authoritarian personality. Psychological Monographs, 1956, 70, Whole no. 425.

ROKEACH, M. The open and closed mind, New York: Basic Books, 1960.

ROKEACH, M. Authoritarianism scales and response bias: Comment on Peabody's paper. Psychological Bulletin, 1967, 67, 349-355.

ROKEACH, M. & FRUCHTER, B. A factorial study of dogmatism and related concepts. Journal of Abnormal and Social Psychology, 1956, 53, 356-360.

SIMONS, H. W. Dogmatism scales and leftist bias, Speech Monographs, 1968, 35, 149-153.

STANLEY, G. & MARTIN, J. How sincere is the Dogmatist? Psychological Review, 1964, 71, 331-333.

J. J. Ray, School of Behavioural Sciences, Macquarie University, North Ryde, N.S.W, 2113.

(Manuscript received 24 March, 1970)



POST-PUBLICATION ADDENDA

1. The pos/neg correlations from the second study above have proved more typical in subsequent applications of the scale. See:

Kirton, M.J. Ray's balanced dogmatism scale re-examined. Brit. J. Soc. & Clin. Psychol. 1977, 16, 97-98.

Ray, J.J. (1979) Is the Dogmatism scale irreversible? South African Journal of Psychology 9, 104-107.




2. Subsequent articles germane to the matters discussed above are as follows:

Ray, J.J. (1972) A new balanced F scale -- And its relation to social class. Australian Psychologist 7, 155-166.

Ray, J.J. (1972) Are conservatism scales irreversible? British J. Social & Clinical Psychology 11, 346-352.

Ray, J.J. (1974) Balanced Dogmatism scales. Australian Journal of Psychology 26, 9-14.

Ray, J.J. (1979) A short balanced F scale. Journal of Social Psychology, 109, 309-310.

Ray, J.J. (1980) Acquiescence and the Wilson Conservatism scale. Personality & Individual Differences, 1, 303-305.

Ray, J.J. & Pratt, G.J. (1979) Is the influence of acquiescence on "catchphrase" type attitude scale items not so mythical after all? Australian Journal of Psychology 31, 73-78.

Ray, J.J. (Unpublished) ACQUIESCENT RESPONSE TENDENCY: An update and some data on the invalidity of the Dogmatism scale



3. SCALE FORMAT: Replication is one of the cornerstones of science. A new research result will normally require replication by later researchers before the truth and accuracy of the observation concerned is generally accepted. If a result is to be replicated, however, careful specification of the original research procedure is important.

In questionnaire research it has been my observation that the results are fairly robust as to questionnaire format. It is the content of the question that matters rather than how the question is presented (But see here). It is nonetheless obviously desirable for an attempted replication to follow the original procedure as closely as possible so I have given here samples of how I presented my questionnaires in most of the research I did. On all occasions, respondents were asked to circle a number to indicate their response.




Go to Index page for this site

Go to John Ray's "Tongue Tied" blog (Backup here or here)
Go to John Ray's "Dissecting Leftism" blog (Backup here or here)
Go to John Ray's "Australian Politics" blog (Backup here or here)
Go to John Ray's "Gun Watch" blog (Backup here or here)
Go to John Ray's "Education Watch" blog (Backup here or here)
Go to John Ray's "Socialized Medicine" blog (Backup here or here)
Go to John Ray's "Political Correctness Watch" blog (Backup here or here)
Go to John Ray's "Greenie Watch" blog (Backup here or here)
Go to John Ray's "Food & Health Skeptic" blog (Backup here or here)
Go to John Ray's "Eye on Britain" blog (Backup here or here)
Go to John Ray's "Immigration Watch" blog. (Backup here or here).
Go to John Ray's "Leftists as Elitists" blog (Not now regularly updated -- Backup here or here)
Go to John Ray's "Marx & Engels in their own words" blog (Not now regularly updated -- Backup here or here)
Go to John Ray's "A scripture blog" (Not now regularly updated -- Backup here or here)
Go to John Ray's recipe blog (Not now regularly updated -- Backup here or here)
Go to John Ray's "Some memoirs" (Occasionally updated -- Backup here)

Go to John Ray's Main academic menu
Go to Menu of recent writings
Go to John Ray's basic home page
Go to John Ray's pictorial Home Page (Backup here or here).
Go to Selected pictures from John Ray's blogs (Backup here or here)
Go to Another picture page (Best with broadband)



Note: If the link to one of my articles is not working, the article concerned can generally be viewed by prefixing to the filename the following:
http://pandora.nla.gov.au/pan/42197/20060526/jonjayray.tripod.com/