For many years, behavior analysts who were responding to other professionals about non-science-based treatments had a standard line, almost a catch phrase; “show me the data.” Perhaps a bit obnoxious, but many were frustrated that despite overwhelming evidence of its effectiveness, there was minimal acceptance of applied behavior analysis (ABA).
Now that ABA enjoys much wider acceptance, I hear this much less often. In addition, it has been years since I’ve heard a special education teacher, speech pathologist, or other professional complain about the need to collect data. It is just an accepted part of practice of most professionals working with children with special needs. I still hear people say, “show me the data,” but now it is in the context of clinical decision making.
On this blog, I’ve frequently discussed how data collected by ABA professionals can be misleading in a wide variety of ways. But when data are collected by people who don’t have the expertise in the design of data collection systems, things are even more likely to be extremely misleading, as there are often fundamental flaws in those systems.
In some professions, it is possible to get a tremendous amount of information simply by looking at the data. That’s because there are standardized ways of collecting the measurements, and it is easy for any professional to compare those measurements. Some examples include investors evaluating company profits and loss statements, doctors diagnosing cancer, or construction workers using an architect’s blueprints.
We need that level of precision in ABA, too. It would be a wonderful thing if any BCBA could pick any other BCBA’s data and have a clear picture of what’s happening. Unfortunately, we aren’t there yet. So, what to do until the real thing comes along? Instead of saying, “show me the data,” say, “show me the video.” Often, in five minutes of video (or live observation), it is possible to get more information than hundreds of pages of data and reports. There is no substitute to actually seeing what’s happening.