D-2: DISTINGUISH BETWEEN INTERNAL AND EXTERNAL VALIDITY ©
Target Terms: Internal Validity, External Validity
Internal Validity
내적타당도 : 실험적 연구에 있어서 주어진 실험처치(experimental treatments)가 정말로 실험효과를 가져왔느냐 하는 정도. 이 내적 타당도에서는 주어진 실험처치가 이 실험에서 정말로 어느 정도 실험효과를 가져왔느냐에 대한 질문의 답이 된다. 한 실험연구의 내적 타당도는 필수적인 요건이 되며, 이러한 내적 타당도를 높이기 위해서는 다른 잡다한 요인들이 종속변인에 우연적으로 영향을 주는 일이 없도록 실험연구를 잘 통제해야 될 것이다.
Definition: An experiment shows convincingly that changes in behavior are a function of the intervention/treatment and NOT the result of uncontrolled or unknown factors.
Example in clinical context: A behavior analyst implements a DRA procedure to support a client who engages in skin picking. The skin picking behavior responds favorably to the intervention. When the DRA is removed, the target behavior returns. When the intervention is put in place a second time, the behavior returns to low rates. This fact pattern strongly suggests that the DRA intervention (and nothing else) was primarily responsible for the reduction in skin picking behavior.
Example in supervision/consultation context: A behavior analyst is consulting in a classroom and implements a direct instruction methodology during literacy time. No other classroom or homework practices are altered. After several weeks of receiving this new instruction, the students show significant improvement in their reading performance. The behavior analyst concludes that the instructional method is the reason why the student’s academic performance is improving and is not the result of uncontrolled variables.
Why it matters: Without high internal validity, cause-and-effect relationships cannot be discovered.
Behavior analytic literature places an emphasis on within-subjects designs, wherein research participants serve as their own controls. This is a fantastic way of answering clinical questions about individuals. When applying behavior analytic research to our own work, we carefully select findings that have direct bearing on our own clinical problems (for example, by matching the function of problem behavior).
However, because behavior analytic studies usually have small numbers of participants, and because we tend not to use between-subjects (groups) designs, it can take longer to build external validity, and our work can be confusing to other fields which rely on large numbers of participants to “even out” individual variables.
External Validity
외적타당도 : 외적 타당도란 이 실험결과를 어느 정도 일반화할 수 있느냐에 대한 답이 된다.
Definition: The degree which a study’s results are generalizable to other subjects, settings and/or behaviors not included in the original study.
Example in clinical context: A behavior analyst is implementing a new intervention from a study that they read in a peer reviewed journal. The individual participant variables (developmental level, topography and function of behavior, for example) are a good match with the behavior analyst’s current client. The analyst replicates the intervention steps with their client and achieves similar favorable results. This supports the study’s external validity, since the results from the study have been replicated with a different subject.
Example in supervision/consultation context: A study is conducted on a systematic way to teach consultees how to conduct functional assessments independently. All subjects in the study learned to complete functional assessments using the methods described in the study. Subsequently, numerous other studies replicated the methods and found similar results across participants and settings. This is strong evidence in support of the original study’s external validity.
Why it matters: Research findings are clinically useless unless they can convincingly demonstrate (1) that the methods were responsible for the observed changes, and (2) that the methods can work across participants and contexts not included in the original study. Science is always building and correcting itself, and replication is a vital – if unglamorous – part of the scientific process!
Internal Validity
Internal validity is the degree to which the changes in the dependent variable truly result from the manipulation of the independent variable and not from other causes (Cooper, Heron, and Heward, 2007).
Questions to ask yourself: Did the study control for confounding and extraneous variables to an acceptable degree? Could the results have been caused by other unknown events or influences?
External Validity
External validity is the degree to which the results yielded by the study can be generalized to other target behaviors, populations, etc. (Cooper, Heron, and Heward, 2007).
Questions to ask yourself: Would the results of this study likely be similar or not in other populations of individuals? Would the results likely be similar or not in other settings? Would the results likely be similar or not if related behaviors were studied using the same or similar independent variables?
