A critical component of programs based on the principles of applied behavior analysis is the commitment to data-based decision making. Part of this commitment involves deciding what data we should collect. I teach two important criteria for making that decision:
Data must be meaningful.
The data must truly capture how well the client is doing and help guide programming.
Data must be practical.
Teachers and parents should be able to collect the data.
Unfortunately, balancing these criteria can lead to conflict. I have frequently seen schools and even programs designed by BCBAs collect superficial types of data that aren’t very useful in helping to make decisions. Why is this?
It is quite possible to design a data collection system that might work in a scientific laboratory with 2-way mirrors and professionals collecting data on sophisticated computer programs, rewinding videos as needed to get all the relevant data. That doesn’t mean it can be done in a classroom or in a home setting with multiple siblings present.
In practice, when conflict comes between “practical” and “meaningful,” the winner will almost always be practical. After all, not having data is going to get you in major trouble. But rarely do people dig into the details and evaluate how meaningful or useful the data collected are in making decisions. If you do, you will find that in many cases, poorly thought out data leads to misleading conclusions.
Currently, though many of us could use some POOGI in this area, I fear this problem is instead getting worse. That’s because of a relatively new third criterion:
We want to record the data electronically.
In addition to creating practical and meaningful data collection systems, you have to make sure this data can be collected using an app. Certainly, collecting data electronically has many advantages and is extremely useful in many situations. But this technology is very unlikely to lead to more focus on how useful the data collected are for the child’s program. I see many programs twisting what they want to do so that it fits into the data collection system that has been purchased.
It is only a matter of time before our paper data sheets are as common as typewriters. In general, that’s a good thing. The improvements in efficiency and effectiveness from these systems are apparent. Just be careful to keep in mind the importance of collecting meaningful data as well as creating a practical data collection system. The goal is not to make the child’s programming fit into the technology. The goal is for the technology to assist us in making the child’s program better. Too often it isn’t working that way.