IOA
- the degree to which 2 or more independent observers report the same values after measuring the same events.
- collect IOA data at least 20% of the time during each phase of a study or program implementation.
- if your program includes 10 hours of direct service per week, you should complete IOA for at least 2 hours per week.
- Any IOA scores over 80% are ideal.
Prerequisite for obtaining valid IOA
- observers must score independently
- observers must use the same measurement system
- observers must measure the same event
Benefit of using IOA
- IOA helps us detect things like observer drift and increases belivability in the data(shich makes us and our clients more confident in what we're doing!)
- it does take quite a bit of out time,
- it increases or decreases the confidence that the target behavior was clear
- it gives confidence that variability in data is not a function of which observers were on duty, therefore implying that changes in data actually reflect changes in behavior.
Unscored Interval IOA
If you have an interval in which BOTH observers recorded as SCORED you don't use it in the Unscored Interval IOA calculation. When calculating Unscored Interval IOA you are only looking at intervals with AT LEAST 1 unscored interval. This IOA is calulated by dividing the number of intervals both recorders recorded an occurrence by the number of intervals in which AT LEAST one recorder recorded an occurrence and multiplying by 100
Mean count IOA
- is calculated by finding the average of the mean percentage of agreement between each of the counts.
- this is tedious and takes the most work
Exact Count-Per-Interval IOA
- is found using the percentage of intervals in which 2 observers recorded the same count.
- this event recording IOA method is also the strictest.
- a scored interval in Exact Count IOA only occurs when two values from the observers match EXACTLY.
Trial-by-Trial IOA
- is calculated by dividing the number of trials of agreement by the total number of trials and multiplying by 100.
- For Trial-by-Trial you are only recording the trial as successful or unsuccessful.
- you collect data using +/- scoring instead of actual values.
Total Count-Per-Interval IOA
- is calculated by dividing the smaller sum of the counts by the larger sum of the counts and multiplying by 100