Every reasonable behavior analyst has a lot of “stories.” A story will typically go something like this: When we started with Johnny, he wasn’t toilet trained, had no language, and engaged in severe aggression, self-injury, and tantrums. Now, after an intensive two years of intervention, Johnny is toilet trained, he says 30 words, and problem behaviors only occur rarely.
What’s wrong with that story?
The problem with stories like this is we have no way to know if Johnny did well in the intervention. Maybe this program was fantastic and a lower quality program would have achieved much less. But maybe this program sucked and even a reasonably high quality program could have achieved 10X as much.
Unfortunately, there is only a small amount of research on predictors of progress. Which means right now, no one can conduct an assessment and tell us what we should reasonably expect from a high-quality service. There are no standards for results. Instead, standards tend to focus on process. That involves things like “Is the curriculum science based? Are the staff trained well?” That’s good, but not good enough.
Behavior analysts tell their “stories” and think everything is great! They don’t have any reason to think they need to improve urgently. I can tell you from hard experience that many of us definitely do. The lack of standards for results is the enemy of the Poogi.
The long-term solution is to determine measurements of what “appropriate” or “reasonable” progress looks like. We need clear standards. Without them there is simply no way to do thorough quality control as a field.
But while we are waiting for the long-term solution, there is a LOT we can do in the short-term. Always be skeptical of your stories. Constantly seek additional opinions. Look at research on how others implement the same interventions. Is there something we can learn? Measure the rate of progress. Is it improving over time?
Constantly ask-What have we done to POOGI today?