re-analysis of the Commission on Community Interrelations experiments

Answering an Antisemitic Remark, 1945 to 1949

A project of the Lab for the Global Study of Antisemitism, Anne Tanenbaum Centre for Jewish Studies, University of Toronto. Ron Levi and Sol Goldberg.

X in

6 publications · 1,130 checks on the transcription · 47 data tables · 3 attitude experiments, 1946 · 7 preference rounds, 1945–1946 · 1 replication, 1949

The sources

Source	What it contributes
Citron, A. F., I. Chein and J. Harding. 1950. "Anti-Minority Remarks: A Problem for Action Research." Journal of Abnormal and Social Psychology 45: 99-126.	The three 1946 attitude-change experiments.
Selltiz, C., A. F. Citron, J. Harding, O. Rosahn and M. H. Wormser. 1950. "The Acceptability of Answers to Anti-Semitic Remarks." International Journal of Opinion and Attitude Research 4: 353-390.	Seven rounds of preference tests, from the first pilots in 1945 to the vote of the Experiment III audiences, which no other source prints by answer.
Citron, A. F. 1952. Experiments on Answering Anti-Minority Remarks. PhD dissertation, Columbia University.	Session-level data, variances and the distribution of individual change for the 1946 experiments; the preference study; two training experiments.
Citron, A. F. and J. Harding. 1950. "An Experiment in Training Volunteers to Answer Anti-Minority Remarks." Journal of Abnormal and Social Psychology 45: 310-328.	The published training study, with the adjusted group means the dissertation omits.
Harding, J., A. F. Citron and E. King. 1953. "An Experimental Study of Answers to Anti-Negro Remarks." Journal of Social Psychology 37: 3-17.	The 1949 replication with an anti-Black remark.
Marcuse, F. L. 1951. "A Note on 'Anti-Minority Remarks: A Problem for Action Research.'" Journal of Abnormal and Social Psychology 46: 603.	A Cornell class of 160 students saw the two calm answers; of the 90% who judged which was more effective, all chose individual differences. No attitudes were measured, and the order of the plays is not reported.

Context

Chein, I., S. W. Cook and J. Harding. 1948. "The Field of Action Research." American Psychologist 3: 43-50. The Commission's statement of method, adapted from a talk at the University of Chicago on 10 August 1946, between the second and third attitude experiments. It holds no data on answering remarks. It defines experimental action research as research comparing "the relative effectiveness of various action techniques", with situations as alike as possible and measurements before and after, and says such research "can provide, under favorable circumstances, a definitive test of a specific hypothesis". The answering experiments followed that design: they compare answers with each other rather than with an audience that heard no answer.

How the rounds line up

The article and the dissertation number the same work differently.

Selltiz et al. (1950)	Citron (1952)	What happened
Rounds I to IV	pilot rounds, no data printed	Testing kinds of reply in four settings; a Round I discussion is dated 26 November 1945
Round V	Round IV	The preference experiment: 532 people chose between two answers
Round VI	a footnote (p. 41)	Peoria, Illinois: 67 people
Round VII	Rounds V and VI	The Experiment III audiences of the 1950 article: 520 people, October to December 1946

Where the printed numbers disagree

The transcription keeps every number as printed. Where a printed number cannot be reproduced from the authors' own tables, the verifier pins the discrepancy and the codebook explains it. Among them:

The Experiment III preference vote: the 1950 article counts 157 of 293 calm choices for the non-Jewish answerer and 141 of 210 for the Jewish answerer; the dissertation prints 155 of 293 and 144 of 212.
The dissertation's t of 4.51 for the drop in trainees' ratings; its own table gives 2.88.
The training chapter's "31 per cent" of women who could not answer at all; no whole number of the 33 women tested gives 31 per cent.
The preference article's 155 interviewees in its fifth round, where its own group counts sum to 158.

The full lists are in each codebook's discrepancy section.

How the re-analysis works

Every value was read from page images of the sources. Five scripts then recompute every printed total, mean, percentage and test statistic that the transcribed cells allow, and the published numbers must agree within rounding.

Verifier	Checks	Pass
`verify_dataset.py`	362	362
`verify_preference.py`	389	389
`verify_training.py`	128	128
`verify_hck1953.py`	159	159
`verify_selltiz.py`	92	92

Attitude change is expressed in standard deviations (Cohen's d): each mean change is divided by the within-group standard deviation the authors report. Estimates from separate experiments are combined with inverse-variance weights. Differences between the 1946 and 1949 studies are tested with a z test on independent samples. Preference is modelled by logistic regression on which answer each audience chose, with terms for the order of hearing and, in 1946, for calm against indignant delivery.

The answers were assigned to whole audiences rather than to individuals. Where the authors report the variation between audiences, it is no larger than the variation within them, which supports treating people as the unit; the permutation test on the findings page uses audiences instead.