**************************************************************************************

Personality & Individual Differences, Vol. 6, No. 5, pp. 655-656, 1985.

ACQUIESCENCE AND RESPONSE SKEWNESS IN SCALE CONSTRUCTION: a paradox



J. J. RAY

Sociology, University of New South Wales, P.O. Box 1, Sydney, 2033 NSW, Australia

(Received 26 November 1984)

Summary

Kirton's version of the Wilson C Scale is primarily a version wherein items of the C Scale that attract very skewed responses have been eliminated. Contrary to what seems obvious, it is shown that the Kirton scale shows more, rather than less, influence from acquiescent response bias. Although such influence was not a serious problem with this particular scale, the finding does give grounds for believing that some skewed items may make a valuable contribution to an additive (`Likert') scale.

INTRODUCTION

If we have an attitude or personality scale wherein there are three response options (e.g. 'Yes', '?' 'No') for each item, it seems logically necessary that a tendency towards acquiescent response bias in the data from that scale will be associated with skewness in the responses to at least some of the items in the scale. The distribution of answers to at least some items will be skewed towards the high end (the 'Yes' end) of the 3-point scale. It would seem, then, that the common item-analysis procedure of eliminating from the preliminary version of a scale those items showing excessive skewness (i.e. items wherein most respondents answer 'Yes' or 'No') should, inter alia reduce the influence of acquiescent response bias on the final form of the scale. The items most affected by acquiscence bias will, by definition, have been removed.

The inter alia above is, however, important. As part of another debate on acquiescence effects (Ray and Pratt, 1979; Feather, 1980), Ray (1982) argued that not all skewness is the outcome of meaningless acquiescence and that some skewed items can make a valuable contribution to a scale. He argued that the skewed responses to some items are highly meaningful and that such items perform a valuable function in sorting out the sheep from the goats as far as the construct measured by the scale is concerned. Thus in an additive scale of conservatism, if there is one item expressing conservative sentiments with which most respondents agree, those people who disagree would have to be 'real' radicals. The identification of radicals provided by that item would be an especially certain one and to throw away that item would be throwing away valuable discrimination power. Proof that skewed items can be valuable was offered by showing that some skewed items had high correlations with the total score on the scale of which they formed part. The data used to illustrate this were from administrations of the (balanced) Wilson (1973) Conservatism Scale.

It might, however, be contended that such usefulness in skewed items is probably rare and that the routine elimination of skewed items during scale construction should continue. Some test of whether the routine elimination of skewed items is overall beneficial or detrimental would therefore be useful.

METHOD AND RESULTS

As it happens, there does exist a short form of the Wilson Conservatism (C) Scale which was constructed almost entirely by eliminating from the original form of the scale those items which showed skewed responses -- The Kirton (1978) C Scale. A comparison of this scale with the original should therefore be of considerable interest.

It was reported at the time of its construction that the Kirton scale showed a similar reliability (alpha) to the Wilson original, even though it was 20 items shorter than the 50 of the original. This represents a substantial gain in internal consistency (mean inter-item correlation) and appears to justify the item-selection method used. What, however, can be said of the new scale's openness to acquiescent responding? If (as should 'obviously' be the case) the deletion of skewed items has also led to the deletion of items that attracted indiscriminate acquiescence, the shorter scale should have far fewer acquiescence effects visible in it.

There are two ways that the influence of indiscriminate acquiescence can be detected in a balanced scale: the correlation between the positively and negatively keyed items (rPN) will be low and the alpha of the scale scored for acquiescence (i.e. without normal reverse-scoring of negatively-worded items) will be high. See Martin (1964), Ray and Pratt (1979) and Ray (1983).

The two statistics mentioned were therefore calculated using the data on which the Kirton (1978) C Scale had originally been constructed [*]. The rPN for the Kirton C Scale was -0.63 vs -0.72 for the Wilson C Scale. This indicates, if anything, more rather than less influence of acquiescence in the Kirton C Scale. The reliability (alpha) of the two scales scored for acquiescence was 0.21 for the Kirton scale and 0.15 for the Wilson C Scale. Again the evidence indicates more acquiescence in the Kirton C Scale.

As these results were rather counter-intuitive, the possibility that some artifact might be involved seemed worth considering. An obvious possibility is that Kirton C Scale was not exactly balanced. It had 16 positive and 14 negative items. To overcome this a new 28-item form of the Kirton C Scale was constructed using the same criteria as originally employed, which deleted two positive items and left an exactly balanced 28-item scale. This scale had an rpN of -0.63 and an acquiescence a of 0.23. This scale was, then, even worse than the 30-item scale as far as acquiescence contamination was concerned. Using Hoyt's (1941) approach to testing the significance of alpha, the significant level of alpha for the given sample size (286) is 0.18 (p < 0.01). Thus the original Wilson C Scale had no significant internal consistency as a measure of acquiescence but both the Kirton C Scales did have.

DISCUSSION

It has been shown that, contrary to almost all expectation, the elimination of skewed items in the course of scale construction leads to a final scale that is more, rather than less, open to influence from acquiescent response bias. This can only be explained by the theory mentioned earlier -- that many skewed items are highly meaningful to respondents and make a valuable contribution to the final scale. At least on the present occasion, it seems that the skewed items eliminated were on balance more likely to be skewed for reasons of their meaning than for reasons of acquiesence bias. Skewness is on balance more likely to be meaningful than not. Elimination of skewed items tends to reduce the proportion of highly meaningful items in a scale and thus leads to a higher aggregate influence of meaningless acquiescence in the final scores on the scale.

This is not to say that scale constructors should in future ignore the problem of skewed responses to their scale items or that the Kirton C Scale in particular is misconceived. As it happens, the Kirton (1978) data shows remarkably little influence from meaningless acquiescence by the standard of what has been reported for some other administrations of the Wilson C Scale (Ray, 1980). Thus, even though the Kirton C Scale showed more influence from acquiescence bias than did the Wilson C Scale, it still remains true that such influence is minimal and compares very favorably with other scales. Nor is this finding peculiar to one sample. In a replication of the original study, another community sample of 276 people in S.E. England received the Kirton C Scale [*]. The rPN on this occasion was still a high -0.57 and the acquiescence a a low 0.25. The alpha when the scale was scored for content was 0.82. There is nothing to cavil at in these statistics.

The lesson from the present work then would appear to be that skewed items can sometimes be valuable and should not be routinely eliminated. Instead, skewness of responses in any item should be considered in conjunction with the correlation of that item with the total score on the scale. There may well be a considerable number of occasions when a skewed item shows a relatively high item-total correlation. Such items might well be profitably retained instead of being routinely eliminated.

REFERENCES

Feather N. T. (1980) Conservatism, acquiescence and the effects of sample heterogeneity. Aust. J. Psychol. 32, 11-16.

Hoyt C. J. (1941) Note on a simplified method of computing test reliability. Educ. psychol. Measur. 1, 93-95.

Kirton M. J. (1978) Wilson and Patterson's Conservatism Scale: a shortened alternative form. Br J. soc. clin. Psychol. 17, 319-323.

Martin J. (1964) Acquiescence: measurement and theory. Br J. soc. clin. Psychol. 3, 216-225.

Ray, J.J. (1980) Acquiescence and the Wilson Conservatism scale. Personality & Individual Differences, 1, 303-305.

Ray, J.J. (1981) Sample homogeneity, response skewness and acquiescence: A reply to Feather. Australian Journal of Psychology 33, 41-46.

Ray, J.J. (1983) Reviving the problem of acquiescent response bias. Journal of Social Psychology 121, 81-96.

Ray, J.J. & Pratt, G.J. (1979) Is the influence of acquiescence on "catchphrase" type attitude scale items not so mythical after all? Australian Journal of Psychology 31, 73-78.

Wilson G. D. (1973) The Psychology of Conservatism Academic Press, London.

---------------------------

[*] Data kindly supplied by M. Kirton.

POST-PUBLICATION ADDENDUM

Other research tends to show that acquiescent tendency invalidates almost all published attitude scale findings -- even if it was conducted using balanced scales. See Ray (1985)

Ray, J.J. (1985) Acquiescent response bias as a recurrent psychometric disease: Conservatism in Japan, the U.S.A. and New Zealand. Psychologische Beitraege 27, 113-119.






Go to Index page for this site

Go to John Ray's "Tongue Tied" blog (Backup here or here)
Go to John Ray's "Dissecting Leftism" blog (Backup here or here)
Go to John Ray's "Australian Politics" blog (Backup here or here)
Go to John Ray's "Gun Watch" blog (Backup here or here)
Go to John Ray's "Education Watch" blog (Backup here or here)
Go to John Ray's "Socialized Medicine" blog (Backup here or here)
Go to John Ray's "Political Correctness Watch" blog (Backup here or here)
Go to John Ray's "Greenie Watch" blog (Backup here or here)
Go to John Ray's "Food & Health Skeptic" blog (Backup here)
Go to John Ray's "Leftists as Elitists" blog (Not now regularly updated -- Backup here)
Go to John Ray's "Marx & Engels in their own words" blog (Not now regularly updated -- Backup here)
Go to John Ray's "A scripture blog" (Not now regularly updated -- Backup here)
Go to John Ray's recipe blog (Not now regularly updated -- Backup here or here)

Go to John Ray's Main academic menu
Go to Menu of recent writings
Go to John Ray's basic home page
Go to John Ray's pictorial Home Page (Backup here)
Go to Selected pictures from John Ray's blogs (Backup here)
Go to Another picture page (Best with broadband)