We can see whether a behavior changed on the graph, but not whether a program was effective. In behavior analysis effectiveness does not just mean that the behavior changed. It means the change made a socially significant difference in the person’s life. When you look at the graph and you see the child learned ten words, that’s great. But did you make a significant difference in the child’s life? If not, the program was not effective.
After graduate school, one of the first cases that I took on involved an adolescent with extremely dangerous aggressive behaviors. At baseline, there were hundreds of aggressive behaviors per day. Often staff were hurt. It was a dangerous place to work. He always had a staffing ratio of 2:1. Through intensive effort, we dramatically reduced this behavior and increased participation in a wide variety of activities. In fact, most days he had zero episodes of problem behavior. I graphed this data on a per month basis and it looked amazing! After a year, when I left that job, he had only 3 or 4 episodes per month.
The graphs appeared to tell a great story. The behavior was reduced more than 99% and participation in a variety of activities was up substantially. But really, this program wasn’t very effective. That’s because the 3 or 4 episodes that happened each month often involved someone going to the hospital or emergency room. It had no substantial impact on his quality of life. He still had a restrictive staffing ratio. He sill had limited opportunities to participate in the community.
Now, it’s not that what we did was worthless. Obviously, the program had a big impact. It’s just that what we did was just the beginning of what was needed. Just changing the behavior isn’t sufficient. We must change the behavior enough that it has a real substantial impact on the person’s life or in the long run it won’t be worth much.