Page 170 - [Uma_Sekaran]_Research_methods_for_business__a_sk(BookZZ.org)

154 EXPERIMENTAL DESIGNS

concentrating on a different set of behaviors after the treatment. The frame of
measurement of behaviors (in a sense, the measuring instrument) has now
changed and will not reflect the change in behaviors that can be attributed to the
treatment. This is also true in the case of physical measuring instruments like the
spring balance or other finely calibrated instruments that might lose their accuracy
due to loss of tension with constant use, resulting in erroneous final measurement.
In organizations, instrumentation effects in experimental designs are possible
when the pretest is done by the experimenter, treatments are given to the exper-
imental groups, and the posttest on measures such as performance is done by dif-
ferent managers. One manager might measure performance by the final units of
output, a second manager might take into account the number of rejects as well,
and a third manager might also take into consideration the amount of resources
expended in getting the job done! Here, there are at least three different measur-
ing instruments, if we treat each manager as a performance measuring instrument.
Thus, instrumentation effects also pose a threat to internal validity in experi-
mental designs.

Selection Bias Effects

The threat to internal validity could also come from improper or unmatched
selection of subjects for the experimental and control groups. For example, if a
lab experiment is set up to assess the impact of working environment on
employees’ attitudes toward work, and if one of the experimental conditions is
to have a group of subjects work for about 2 hours in a room with some mild
stench, an ethical researcher might disclose this condition to prospective sub-
jects, who may decline participation in the study. However, some volunteers
might be lured through incentives (say a payment of $70 for the 2 hours of par-
ticipation in the study). The volunteers so selected may be quite different from
the others (inasmuch as they may come from an environment of deprivation) and
their responses to the treatment might be quite different. Such bias in the selec-
tion of the subjects might contaminate the cause-and-effect relationships and
pose a threat to internal validity as well. Hence, newcomers, volunteers, and oth-
ers who cannot be matched with the control groups would pose a threat to inter-
nal validity in certain types of experiments.

Statistical Regression
The effects of statistical regression are brought about when the members chosen
for the experimental group have extreme scores on the dependent variable to
begin with. For instance, if a manager wants to test if he can increase the “sales-
manship” repertoire of the sales personnel through Dale Carnegie–type programs,
he should not choose those with extremely low or extremely high abilities for the
experiment. This is because we know from the laws of probability that those with
very low scores on a variable (in this case, current sales abilities) have a greater
probability of showing improvement and scoring closer to the mean on the
posttest after being exposed to the treatment. This phenomenon of low scorers