BCBAs tend to have a real love of data and graphs. There are no situations in behavior analysis where we can work without having data. Most of the time, we naturally want to use the best procedures available. We aren’t working with fat grants and usually can’t have multiple people watching sessions behind one-way mirrors recording data on sophisticated laptop computers as often happens in important research projects. In addition to the high cost, there is also practicality — we are in homes, classrooms, and community settings. Still, we try to use the best procedures available under the practical constraints that we have. Often, this is a mistake.
Why? Don’t you believe in POOGI? Shouldn’t we do the best we can do? The problem comes when we compromise on using the best intervention programs available in order to collect more data.
The basic rule on designing intervention programs is to first design the best possible teaching procedures available. Only after you have the determine the best procedures for the individual client should you decide on the data collection. The data collection procedure must give you all the information you need, and the data must be valid and reliable, without requiring compromises in the teaching procedure.
For example, when teaching conversation skills, sometimes behavior analysts ask the RBT or the parent to write down everything the child says. Now, those data might be interesting! We might see if the child is using language learned in speech sessions spontaneously in conversations at lunch. We might detect patterns in conversation topics, etc. etc. But if you watch those sessions, it will usually quickly become apparent that the teacher can’t really do a great job teaching conversation when they are trying to write down every word the child says. You need a different procedure.
Another common example is many instructional programs involve some type of prompting and fading. Now, often it would be interesting to know exactly what type of prompt was used on each instructional opportunity. But I rarely attempt to do this. When I’ve designed teaching sessions this way, frequently it is obvious that this data collection method decreases the ability of the instructor to teach well. Effective teaching depends on careful observation and getting your timing exactly right. Most of the time this isn’t done well when instructors are focused on writing down every prompt used.
POOGI doesn’t mean trying to improve everything all the time. POOGI means trying to improve the right things. The design of the intervention takes priority. POOGI that. Make the intervention the best it can be. Then make sure the data are good enough to make decisions, but don’t go so overboard that you reduce the quality of your program. It happens more than you think.