Author: Barry Morgenstern

One of the most common complaints of applied behavior analysis is that while the behavior changes are successful, they often fail to generalize across people, places, settings, examples, or situations. Now, there are many possible reasons for the failure to generalize, and I won’t be covering them all here. Today I just want to cover one that is surprisingly common: The behavior change was not mastered under practical conditions.

After a behavior change has been completed, there are two important questions to ask. These questions were adapted for children with autism and other developmental disabilities from this audio recording from Eli Goldratt, which has nothing to do with autism, special education, or behavior analysis. You never know where good ideas will come from.

1. What does the adult have to do for the child (or help the child do) that they would not have to help a typically developing child of the same age do with regard to this behavior?
2. What does the adult have to avoid doing with this child with regard to this behavior that they would not have to avoid with a typically developing child of the same age?

Collectively, I call the answers to those questions the limitations. The limitations are, in my view, the most common reason for failure to generalize otherwise mastered skills.

For example, when skills that are mastered in discrete trial teaching, they often do not generalize. Why?  When you observe the RBT (Registered Behavior Technician) you find things like:

• The RBT makes sure that they have the child’s attention before each trial.
• The RBT provides enthusiastic praise after each trial.
• The RBT provides a tangible reward (e.g., access to the iPad after every 10 trials.)

Now, there is nothing necessarily wrong with those types of procedures to establish a behavior. But don’t expect them to occur spontaneously without a lot more work to make the skills practical so that they will be naturally reinforced under real-world conditions.

In a similar manner, the same thing happens with behavior reduction programs. The problem behavior might be at zero, but it doesn’t generalize to novel people or situations. Why? You will often find things like:

• The parent always offers choices before presenting any demands.
• The RBT reminds the student of the contingencies before each class at school.
• The expectations of the activity are reduced for the student.

Again, nothing necessarily wrong with these types of procedures to reduce a significant problem behavior. Just don’t expect to get generality to novel situations unless you do a lot more work to teach the alternative behaviors that will meet a natural contingency.

Once you get the hang of this idea and start looking for limitations, you will see them everywhere. One of the keys to good programming is overcoming all the important limitations to get skills to the point where they are likely to occur under natural situations.

Atul Gawande describes an amazing experiment that dramatically improved surgery results in a wide variety of hospitals in many different countries. The results showed substantial improvements including reductions in deaths, major complications, and return visits to the hospital. The truly amazing part is the intervention was so simple. They got surgery teams to use a safety checklist.

Now, of course, BCBA’s are no strangers to checklists. Still, I think there are huge lessons that behavior analysts can learn from this experiment. When designing the checklist, the team considered all kinds of possibilities to determine what should and should not go on the checklist. One of the things that the team considered were fires in operating rooms. When a fire breaks out during an operation, it is extremely dangerous, can cause serious injury, and the hospital is liable for damages. Considering how serious that problem can be, the team made a fascinating decision – they decided against including any checklist items to prevent fires!

Why would they make a decision like that? Simple. Their focus was to make the checklist practical. Preventing fires would substantially increase the checklist length. They strongly suspected that adding this to the checklist might decrease their effectiveness by making surgeons much less likely to use it. In addition, although fires happen, they are incredibly rare. Compared to the other very common problems they were trying to prevent (infections, bleeding, and unsafe anesthesia), they decided fires weren’t worth addressing.

When we select what changes to make, it is important to realize that it is possible to make anything sound extremely important when you consider it isolation or hear a dramatic story, like a fire during surgery. Sure, sometimes the problem is extremely important. Both the staff and the clients have limited time and attention. Working on too much at the same time is a sure way to fail. When we consider all of the client needs, is this really the most important thing we should be addressing?

In applied behavior analysis, the term social validity is often used as a synonym for satisfaction ratings even though the concept of social validity includes much more than that. Parents, teachers, and others involved in the treatment are often asked for their opinions on goals, methods used, and the outcomes. These data are extremely important. It is quite possible for the objective data to look great, but — if the parent doesn’t agree with the goals, the teacher in the classroom isn’t happy with your methods, the staff don’t like your supervision, or the funder doesn’t thinks the outcomes are significant enough — it is only a matter of time before we run into trouble. The problem is that people sometimes don’t tell you when they are unhappy. Therefore, soliciting the feedback from all the relevant stakeholders is very important to a successful treatment.

On the other hand, it is easy to get caught up in a positive reinforcement feedback loop. I’ve seen many situations where a client isn’t doing very well, yet all the stakeholders seem happy with the services being provided. Sometimes we work with clients living in low socioeconomic conditions who simply have low expectations. The BCBA might think there is no urgency to Poogi as everyone is very satisfied with what they are doing in the moment. Despite not doing much, the parent, teachers, and others are heaping huge praise on them. Everyone responds to reinforcement.

The problem is that (in my experience) satisfaction ratings can change dramatically over time. Many parents with adult children wish they did more to prepare during the teenage years even though they were very happy with the services at the time. Many parents in early intervention programs were very happy at the time, but only later become disillusioned with the services when they learn other things that might have been provided. It’s bad for the individual client and bad for the profession as a whole. Don’t be satisfied with satisfaction.

Of course, we want parents to be satisfied. That’s essential. But we also have to be effective. They are not the same. For BCBAs, effective does not mean simply that the parents are satisfied or that the behavior has changed. It means that the intervention made a socially significant difference in the person’s life.  It is quite possible to have one, but not the other. It is essential that you have both.

I know, I know — every BCBA is super busy. There are just too many cases to see and not enough time to see them. In my view, conversations with staff should be a major part of your Organizational Behavior Management (OBM) plan. Although hard to prove, I believe it dramatically improves program quality. It definitely improves staff satisfaction.

Staff often place a very high value on interacting with supervisors about training, discussing next steps for the client, brainstorming problems, opportunities to address concerns, overcoming obstacles, asking questions, learning new things, professional growth, and just getting to know the supervisor personally. Supervisors often enjoy and appreciate how much they can learn from these conversations. In addition, supervisors get the opportunity to set expectations, provide reinforcement, monitor performance, and give corrective feedback.

We train supervisors not to use “sandwich feedback.” Sandwich feedback is where you start with a positive statement, (e.g., I appreciate you showing so much enthusiasm teaching Mark!). Then slip in the corrective feedback (e.g., when I observed you running the behavior plan, I noticed you forgot step #5 blah blah). Then end with a positive statement (e.g., You are such a hard worker, I know he is going to make great progress working with you!). Sandwich feedback is problematic because it is mostly to make the supervisor feel better about giving the feedback. For the person receiving the feedback, they ignore the positive statements and focus on the corrections.

The general rule to avoid this problem is never mix positive reinforcement with corrections in the same interaction. Separate them in time while making the positive much more frequent. That is good advice, most of the time. But I think you can take this advice too far. If you sit down to have an in-depth conversation to solve a problem, undoubtedly both positive and negative things will come up. If you are working together to solve a problem, that’s often highly reinforcing to staff. If you worry about mixing your positives and negatives, you can’t have a meaningful conversation.

I believe these conversations are so important that I make sure they go on the schedule. I don’t believe you can maintain high performance without them. Sometimes, we think we can replace conversations with staff with emails, text messages, training days, or brief positive statement when we catch the staff doing something right. It’s just not the same. I recently went to a workshop by John Austin on conversations as a primary management intervention with similar (but not identical) ideas.

Sometimes, what is necessary to be successful seems obvious, but you can’t get the resources you need due to political forces at play in the particular situation. This is particularly common in school consultation.

“Well, we asked for 1-1 support but the school district said no.”

“I told them we needed a space to work, but the principal told me there is no space available.”

“The paraprofessional is not physically able to keep up with him, but the special education director said we have to train her.”

In these situations, I’d say that we have to advocate for the child, even though someone is bound to say “We can’t do that; they won’t like it.”

In this situation, many behavior analysts will compromise. In general, it’s a big mistake. If you compromise on what the student obviously needs to get along, in all likelihood the student will not be successful. Then, when it doesn’t work, no one is going to remember that you said it wouldn’t work with this para, without space, or without a 1-1. It is going to be your fault anyway.

But if you push, get what the student needs, and you are successful with the student, people won’t remember the little fights at the beginning. In fact, often the most dedicated supporters will come only after they see the dramatic improvements the program has made. Usually (but not always), when a student is dramatically successful, you will win the support of everyone.

So, the general rule is to advocate for everything you need to be successful right from the beginning. Don’t worry that someone might not like the recommendation if it is something critical the child needs. Yes, despite how socially savvy you might be, this might ruffle some feathers. Richard Foxx told us that a consultant needs a persuasive personality.

Fear of that reaction can cause us to back down. Don’t do that. Of course, we want to have good social validity in the places we work. Sure, some BCBAs have a reputation for not behaving well – not that I know anyone like that. Sure, use your best social skills, but get what the child needs. Bad compromises are bad for everyone. Even if you have short-term social validity, you won’t maintain it when the program isn’t successful. You have nothing to lose.