Page 17 - TEST_MONOGRAPH 2018_+cover_Float

ies that included evidence for the practices in each topic evidence were not operational definitions. Rather, they
area. The number of citations provided by the topic area were general descriptions of studies or research reviews.
workgroups ranged between 20 (interaction) and 125 For example, many of the papers classified as [research]
(instruction) (Mean = 54, SD = 39). The studies were syntheses were in fact narrative reviews that did not in-
categorized by the workgroup leaders (?) as descriptive, clude explicit analyses of practice-outcome relationships
correlation, quasi-experimental, or experimental studies, (e.g., Case-Smith & Holland, 2009; Finello, 2011; Horn
or as meta-analyses or [unspecified types of] syntheses. & Kang, 2012; Justice & Kaderavek, 2004; Keen, 2007;
The descriptions proved problematic for a number of King et al., 2003) but rather were selective reviews of
reasons. For example, descriptive studies were described studies to support author assertions about particular
as “studies that include statistics that summarize and/or types of practices.
describe the data characteristics of a sample population, Based on all of the above concerns about the re-
or phenomenon being studied.” These types of studies, search evidence for the 2014 DEC recommended prac-
however, would not likely include practice-outcome re- tices, the author reexamined all the cited evidence us-
lationship data and therefore would not include evidence ing operational definitions of different types of research
for the effectiveness of an intervention practice. syntheses (Dunst, 2016). I focused specifically on sys-
The author, as part of a review and analysis of the tematic reviews and meta-analyses because these types
papers provided as evidence for the practices in each of syntheses typically include aggregated evidence from
topic area, could not replicate the categorization for four multiple studies where replicated findings bolster con-
reasons. First, large numbers of papers included no prac- tentions about practice-outcome relationships (e.g., Ba-
tice-outcome evidence whatsoever. For example, Bailey nerjee, Movahedazarhouligh, Millen, & Luckner, 2018;
et al. (2004) was cited as evidence for an assessment Jasny, Chin, Chong, & Vignieri, 2011; Simons, 2014).
practice involving professionals and family members The four types of research reviews were narrative re-
working together to gather assessment information, but views, summative reviews, systematic reviews, and
that report includes no evidence for this practice nor was meta-analyses. The definitions are shown in Table 1-2.
the purpose of the study to do so. Second, papers that Very few of the sources of evidence cited as syntheses
included evidence were quite often not evidence for the or meta-analyses met either stringent (meta-analyses)
practice to which it was assigned. For example, Dunn or liberal (systematic reviews) definitions of these types
et al. (2012) was cited as evidence for a teaming prac- of research reviews. Meta-analyses were considered
tice also involving practitioners and family members stringent evidence if effect sizes for practice-outcome
working together, but the research in that study includes relationships were reported, whereas systematic reviews
findings from a study “to coach parents in strategies to were considered liberal evidence if there was at least
support their child’s participation in everyday activities some explicit focus on the description of practice-out-
and routines” (p. 520). Third, quite a few studies were come relationships. The number of reviews that were so
investigations of older individuals with disabilities and classified is shown in Figure 1-3. The only topic areas
not infants, toddlers, or preschoolers. There were, for for which there was evidence for replicated practice-out-
example, research syntheses cited as evidence for an come relationships were the instruction and family topic
environment practice for providing infants, toddlers, areas. The findings confirm an earlier conclusion that the
and preschool children opportunities for movement and 2014 DEC recommended practices cannot be considered
physical activity, but the syntheses included studies of evidence-based or evidence-informed as determined by
school-age children and youth but no preschoolers (e.g., the fact that so few research reviews include practice-
Janssen & LeBlanc, 2010; Johnson, 2009; Taub & Greer, outcome data.
2000). Fourth, many of the papers categorized as meta- Closer examination of the cited evidence found
analyses or syntheses were in fact not research reviews. considerable variation in the percent of total number
For example, Cara’s Kit (Milbourne & Campbell, 2007) of articles that were either systematic reviews or meta-
was cited as a research synthesis for an environment analyses. As reported earlier, the number of citations for
practice but this document includes no evidence what- the different topic areas ranged between 20 (interaction)
soever for the practices constituting the focus of creating and 125 (instruction). This is shown in Figure 1-4. The
adaptations for everyday routines and activities. These percents ranged between zero (transitions) and 32%
are but a few examples of the disconnect between the (family) with the majority of evidence not including rep-
2014 DEC recommended practices and cited evidence licated results reported in either systematic reviews or
for the practices. meta-analyses. Again, except for the family practices,
At least one reason why the categorization of the the preponderance of citations for the 2014 DEC recom-
papers could not be replicated is the fact that the “defi- mended practices cannot be considered sufficient evi-
nitions of types of evidence” used to classify the cited dence for the practices .
1

9