Author: Barry Morgenstern

When we go to the doctor for a medical test, often we want to know whether we “have” or “do not have” a particular medical condition. For example, the patient wants to know if he has cancer, COVID-19, or strep throat. The expected answer is either “yes” or “no.” Sure, some medical tests are more accurate than others, and any test might be wrong some of the time. But, still, they seem useful in the medical profession.

Now in behavior analysis, we sometimes act as if we are doing something similar to medicine. We are testing for the presence vs. the absence a behavior (or a cause of a behavior), and those results are used to make decisions. In reality, we just about never do that.

In behavior analysis, we can certainly demonstrate the presence of behavior or causes of a behavior. For example, we might do assessments and show that a student can label all his colors, washes her hands, or engages in problem behaviors in order to escape instructional demands.

But unlike the medical profession, in behavior analysis, there is no way to demonstrate that the absence of a behavior means that the behavior never occurs, or that a behavior in a learner’s repertoire it is never caused by a certain event.  That’s because there always may be a situation that we haven’t tested where the learner engages in the behavior.

Understanding this is one key to avoiding misleading data and making poor decisions. For example, a common problem I’ve observed is that a BCBA starts working with a learner and conducts an initial assessment of skills. After intervention, a reassessment shows that the child has mastered hundreds of new skills very quickly. Now, that might be the case–sometimes. But it’s more likely that the child already had many of those skills in his or her repertoire. He just didn’t demonstrate them on the initial assessment due to problem behaviors or other reasons. Now that the team has improved problem behaviors and attention, the child has the opportunity to demonstrate many more skills.

In a similar manner, people will argue that a behavioral assessment demonstrated that a child does not have problem behavior motivated by attention since the assessment showed the behavior only occurred under escape conditions. That’s just silly, especially if you consider how this is typically tested. What happens is a child is left with little to do, then an adult (often unknown to the child) pretends they are busy. If the child engages in any problem behavior, the adult provides a reprimand. Of course, a child might not engage in any problem behaviors under those conditions, but still be the class clown when all the other kids are laughing at his antics in the classroom. That’s one reason I don’t do these types of assessments any longer. Greg Hanley and colleagues have demonstrated a better way.

In Behavioral Assessment, we can show that a behavior or cause of a behavior does occur. But we always must be careful about stating that a behavior doesn’t occur. All you really know is that under the circumstances you tested, the behavior doesn’t occur. That’s a critical difference, and misunderstanding this point leads to bad treatment decisions.

Obviously, when we want to know how well a child is doing, we want to see the data. The catch phrase of a BCBA is often “show me the data.” The problem is that sometimes those data are not summarized in a very useful way. In many cases, data summaries are downright misleading. For example, if we want to know how a particular child is doing, we are probably not going to look through dozens of graphs or hundreds of pages of raw data, we need a summary. Summaries often include things like this:

• Problem behaviors are down 90%
• 50 New Words Learned
• etc.

Now, sometimes when I’m doing an evaluation, and I look closely at these types of data, I find that the data were collected very poorly, or that the staff can’t replicate these changes during an observation. But today, I’m not talking about bad data collection. Today, I’m assuming that these data are 100% accurate and reliable.

The problem is they may or may not be very meaningful. The problem behaviors may be down 90%, but only if the preferred staff is there, only if the team dramatically reduces the math presented, and never says “no” to requests. The child may have learned 50 new words, but only uses them if you practice in a distraction-free area, remind him to “get ready” before asking, and provide tangible rewards for answering.

In order to tell if something is meaningful, you need the generalization data. That’s what is important:

Problem Behavior (Examples)

Does the child still have low levels of problem behavior when:

• The regular staff person is out sick?
• When there is a fire drill?
• When there is a change in schedule?

New Words (Examples)

• Does the child use the words in spontaneous speech? When novel pictures are used?
• Do novel people understand the new words?

As a general rule, it is much better to make fewer behavior changes, but choose behavior changes that truly make a difference. Make sure that each behavior change is really useful to the learner. Don’t get carried away with absurd numbers of goals or graphs.

It is easier to measure the quantity rather than quality. It can look impressive–50 new words! But in the long run, the quality of the behavior changes is what makes the difference in the learner’s life.

A common complaint from parents and teachers is that the child is doing “great” in therapy, but they don’t use the skills they learn in therapy “in real life.” Now, there might be a lot of reasons why a particular skill occurs in therapy, but fails to generalize to natural environments. I won’t try to cover them all here as there are too many possibilities, and most of them are well known by BCBAs. Today, I’m just talking about one of those possibilities because it is not well known, easy to overlook, and often the data hides the problem. The problem may be that the child is just not very good at the skill, despite what the data seem to show.

For example, remember when you were first learning to drive. At the beginning, you had to think about everything. Put pressure on the gas, where to hold the steering wheel, and when to push the break. Of course, we all realize when you are driving with someone at that level of skill, you should probably be afraid. Now, you don’t do any of that. You just get in the car and drive; you aren’t thinking about it any longer. When you reach a high level of skill at a task so that you can perform it beautifully without thinking, you have attained what is called fluency.

Many times, I’ve seen children who have “mastered” a particular skill, but actually aren’t very good at the skill yet. A common example involves children who have learned a skill through discrete trial teaching. This might be anything from labeling pictures, pronouns, prepositions, or a wide range of other possibilities. Often, a skill would be considered “mastered” if the child achieves 100% accuracy for a few sessions. We should realize, though, that this is the equivalent to student drivers who are still thinking about everything they are doing. Although they can drive, they aren’t likely to do well under difficult “real-world” driving conditions. Likewise, not reaching fluency is often a major reason why a child can perform a skill while in therapy, but does not generalize the skill to “real life.” This is true for a wide variety of skills in all the relevant domains–social skills, language skills, motor skills, play skills, and more.

The subset of Applied Behavior Analysis practice that emphasizes this type of work is Precision Teaching. For more information, my friend Rick has written the book on the topic.

Once, I was at a school observing a program where a child was engaged in a social skills program to learn conversation skills:

Student (reading from a script): “What are you going to do on your vacation?”

Conversation Partner: “I’m going to Miami.”

Student: “What are you going to do in your Ami?”

I think this is a fascinating response. Not just because it is an amusing anecdote. But because it highlights a fundamental problem with the way social skills and language instruction is often conducted.

This student clearly had clearly learned a social “rule.” Something like, “After the initial conversation starts, ask a follow up question.” The problem here was the student didn’t seem to have any interest in having a conversation. He didn’t care at all about where the conversation partner was going on vacation or even talking to that person at that time. If he really wanted to know about his conversation partner’s vacation plans, his follow up question would have been “What is an ‘Ami’?” The student was working with a motivation system where if he completed a series of activities, he would earn a reward. He was just trying to finish his activities to get the reward.

Now, there is a lot of literature with different approaches to attempt to solve this type of problem. As always, more research is needed. But the key lesson for practitioners is that just getting the response to look right doesn’t mean that you have taught anything useful. Probably, the student in the above scenario would have looked great 90% of the time. The school could have shown many of his conversations at conference presentations as examples of how well their social skills program works.

But we know better now. Someone may learn to have a superficial conversation and look appropriate. But surely it won’t generalize to natural situations or friendships unless they are engaging in conversation because they care about the answers, not simply earning the contrived reward at the end of the conversation.

If you missed it, here is Part 1.

In 1995, I submitted my master’s thesis on Functional Communication Training (FCT), and ever since I have usually been successful implementing FCT. FCT is still one of the most commonly implemented treatments for problem behaviors. Until about 5 years ago, I would not have believed that I had a lot of opportunity for Poogi in this area. I thought my practice was excellent.

Then, I heard Greg Hanley talk about the new research on Practical Functional Assessment. It has transformed my practice. I didn’t fully buy in at first, but now I’m convinced that it is a major breakthrough in the treatment of problem behaviors. A large number of studies have already been published, with more coming out all the time. At the time of this writing, Practical Functional Assessment is gaining in popularity, but not yet fully accepted by the field.

The basic ideas of Practical Functional Assessment are extremely simple, yet the difference in practice is profound. I’m convinced that BCBAs who adopt these procedures will achieve dramatically superior results as compared to using the available knowledge prior to the first Practical Functional Assessment study published in 2014.

Now, I take training courses to upgrade my skills as often as possible.  Currently, I’m taking the advanced course with Shannon Ward, which is fantastic. An amazing amount of progress has been made since 2014. In fact, I just took an all-day training last October, and many new improvements have been made since then.

I have always preached that no matter how good you are, there is always dramatic room for POOGI. In this training, Greg discusses his goal of dramatic improvement every six months. It is a great model for all of us to work toward achieving—a perfect example of the POOGI.