Author: Barry Morgenstern

The Importance of Second Opinions

A parent recently called me for advice. The problem the family was facing is that they were losing their current services due to changes in funding. The family would no longer have their current BCBA and team, with whom the parents thought the child had made excellent progress. Now, there would be a new BCBA and a new team. The parent was extremely concerned. These services would occur at a school, so it wouldn’t be easy to observe. How will she know if the school-based BCBA and school staff are doing a good job?

In my experience, the parent is definitely justified in being concerned about this problem. I have seen great services delivered in school systems, and I’ve seen terrible services delivered in school systems. Of course, sometimes the answer is obvious. For example, if there are frequent safety issues due to dangerous problem behaviors, that is a warning sign that there is likely something wrong with the services. On the other hand, if the child is making friends, getting good grades, and doing well on standardized tests, that is pretty obvious something is going right. But usually it isn’t so simple. As a parent, it isn’t always easy to tell the difference between low quality and high quality.

Sometimes, people will suggest looking at the data. That is certainly helpful, but insufficient for numerous reasons that I’ve discussed frequently on this blog. Even an expert can’t really evaluate the program quality by simply looking at data, even if there are no misleading aspects of the data and it was collected with perfect accuracy, which is NEVER the case.

Sometimes, people will suggest asking other parents. That is helpful too, but also insufficient. Parent reports are often good indicators, but certainly not always. Several times I remember doing an evaluation where I thought the services were of very low quality, yet the parents were completely satisfied. Other times, I’ve seen parents upset with the quality when in my view the services were excellent.

Sometimes, people will suggest looking at how services are generalizing to home. This can be very helpful. If you are seeing generalization to home, that’s certainly a great sign. But the lack of generalization could be caused by a lot of potential factors. Certainly, the problem should be addressed, but it isn’t possible to draw a straight conclusion that services are poor based on this evidence alone.

In my view, this is an area where parents frequently need a second opinion. I often suggest that parents have an independent outside expert BCBA provide an evaluation of the program. This offers several advantages. First, it allows parents to get a realistic picture of the quality of the services. Second, even if the services are of high quality, there is always room for Poogi. Having another set of eyes can often lead to great ideas for further improvements. Third, unfortunately, the way the world works is that is that the “squeaky wheel gets the grease.” Often, schools have limited resources. Simply having an outside evaluator come in to review a program can improve the quality. Fourth, if the program is of low quality, you have the opportunity to make changes early and get started on a POOGI.

It is important to pick the right type of expert. You are looking for someone who will give an honest opinion. I’ve seen some evaluations that are from parent advocates, basically echoing whatever the parents want them to write in the report. In a similar manner, I’ve seen some evaluators that will pretty much put in whatever the school district wants them to say. Usually, these aren’t BCBA’s who must adhere to our ethical code.

Unfortunately, it isn’t always easy to find an expert to do this type of work. At least in Connecticut, where I live, I know people who do an excellent job providing these types of services are often booked months or years in advance. We need more people doing this type of work. If you live in Connecticut, contact me if you need a recommendation. But realistically, there are long waiting lists for all the excellent evaluators.

If you have worked in behavior analysis awhile, you are sure to hear the criticism at some point that applied behavior analysis is no different from animal training. That’s just silly. Of course, there are many, many differences between teaching humans versus training animals.

But there is an element of truth. There are some aspects of learning that have been shown to be universal across species. So, it doesn’t matter the organism that you are teaching. Some of the same teaching procedures can be useful.

I was lucky that one training I attended probably stole a game from the animal trainers–the shaping game. I’ve been using this game in training sessions for more than twenty years, and have found it to be an extremely useful in training staff and sometimes parents. When shaping a behavior, if you are off even one second on your timing of reinforcement delivery, you can lose a lot of the benefits of the treatment. But timing is difficult to practice. This game is an effective way to introduce the topic and get people started on learning this essential practice.

How to Play the Shaping Game:

1. One person is sent out of the room and will be the learner.
2. The group that is still in the room decides on something to “teach” the learner. No embarrassing tasks allowed.
3. When the person comes in, there is only ONE person designated as the teacher. If you don’t explain this well, the whole group starts attempting to teach.
4. The learner who left comes back in, and the teacher can only clap to provide reinforcement to the learner. No other teaching procedures allowed (e.g., prompting).
5. Everyone in the room claps when the learner is successful.

A Few Tips for Trainers:

1. The easiest to teach tasks are using an object in a typical way (e.g., sit in a chair, write with a pencil, or turn off a light switch). The tasks that are the hardest are multiple step tasks and using items in an atypical fashion (e.g., tape a pencil to the wall).
2. It is generally better to start with difficult tasks at the start of the shaping game with experienced people who won’t get upset during the game. This can model several things for the group.
   • First, there is no expectation that the learner will get it quickly. It is often tempting to start with a simple demo, but this will mess things up later when people are unable to learn their tasks easily and get upset.
   • Second, experienced people can model that you have to move around and try things. The hardest person to teach is someone who just stands there and shrugs “I don’t know what to do.”
   • Third, this is a FUN game. There is no need to get frustrated; we are all learning.
3. It is generally very helpful to point out the three critical features modeled in #2 after the first demo.
4. Be sure to make the point that the feeling of not knowing what to do is how our students might feel frequently.
5. A good shaping game trainer discusses the details of what happened during each round, specifically looking for opportunities to point out what happened when timing was off by just the slightest amount. You can always find them. Do this with humor, and you will create a memorable experience with insights that people will not forget.
6. Sometimes, if someone is struggling, it is helpful to send the learner out of the room and do a live coaching session with the teacher about what they might do to Poogi their teaching. Common suggestions include increasing the frequency of reinforcement, intensity of reinforcement, or improving the way the skills are task-analyzed.
7. As the trainer, be sure to take an opportunity to be a learner. People appreciate it and you can model appropriate trainer behaviors.

I have found that many people genuinely love to play the game, and are eager to participate to help train new staff. Granted, it is hard to measure whether it truly improves staff performance in actual work with children. But at least it gives you something to prompt when you are coaching a staff person. It seems helpful to be able to say something like, “Remember when we were playing the shaping game and…”

It’s up to you to decide whether or not you disclose that this activity was probably stolen from the animal trainers.

When I worked as a 1-1 with children with autism, the school once asked me to substitute for an absent staff person. Sure, there was some paperwork with some basic information about the student I would work with for the day, but otherwise no one was there to give me all the details I would need. As soon as I arrived and met the student, he started to shoot off a round of non-stop questions. “What time will we be going to lunch today?” “Where will we be working today?” and on and on. I explained that I didn’t know, but I’d find out. The student correctly concluded, “You don’t know a lot of things.” Luckily for me, the student had enough communication skills that I could figure out the information he wanted and get through the day successfully. But over the years, I’ve seen many, many staff in many schools that were not so lucky. Not knowing basic information can often lead to severe problem behaviors, safety issues, and other emergencies.

Many years later when I became a supervisor, I became extremely interested in this topic. There is a tremendous amount of information that is known by staff, but is not documented anywhere. It lives in the minds of the staff. That’s a problem for many reasons:

• It can become an emergency when the staff person is sick, leaves, goes on vacation, maternity leave, and sometimes even goes to the bathroom.
• Staff are reinforced for keeping the problem behavior low, but are often using techniques that might not be in the student’s long-term best interest.
• The information that is in the staff person’s heads are often a goldmine that can lead to the best goals that we should be setting for the student.
• It can be extremely difficult to find out this information since the knowledge has become second nature to the staff. Even if they want to help, they are often unable to accurately report everything that they do in order to be successful with the student.

The key lesson is that it is worth putting substantial time and effort into the documentation of all the procedures needed in order for a student to be successful. It will reduce emergencies, improve generalization, and help select better goals. Of course, it is hard to do because there is no one pressuring you to get it done. But the effort is more than worth it.

…swims like a duck, and quacks like a duck doesn’t mean it is a duck.

Sometimes, BCBAs think we are talking in plain English, but we really aren’t. Many people, including many BCBAs are confused about what we mean when we say “those behaviors are the same” and “those behaviors are different.” This common misunderstanding can really inhibit communication, and has even led to serious criticisms that don’t make any sense.

I once heard a famous critic of behavior analysis argue that there is a huge difference between a child who looks at an adult in order to gain a reward and a child who looks at an adult out of love. Of course, no behavior analyst would disagree. That’s not a criticism, that’s common sense. The critic clearly didn’t understand how behavior analysts define when behaviors are the same and when they are different.

Let’s explain this concept using Plain English and avoiding technical jargon. BCBAs consider two behaviors the same if they occur for the same reason (see note below). Even if the two behaviors look very different, if they occur for the same reason, they are the same behavior. For example, a child may engage in many different problem behaviors (e.g., self-injury, aggression, tantrums, fall to the floor, screaming), but in most cases they usually occur for the similar reasons, and are therefore not really different behaviors.

On the other hand, sometimes behaviors may look almost identical. But if they occur for different reasons, they are different behaviors. For example, if you are reading a book for enjoyment, that is a different behavior than reading a textbook because there is an exam tomorrow morning.

Therefore, in behavior analysis, the duck test doesn’t work. Often you can’t tell just by observing why a behavior is occurring. If the behavior isn’t occurring for reasons that are natural, it likely won’t last in the long run.

We need to be very careful when we say things like he is doing so well sharing his toys, playing with friends, participating in class, eating his vegetables, or refraining from problem behaviors. Those things might look great. But if the child is doing it for contrived reasons, that’s the wrong behavior and not likely to last.

So, the critic may have a point. He is wrong on the behavior analysis because he doesn’t understand the jargon. But he may be correct that in practice, progress may be exaggerated because we often don’t report on the reason why those new positive behaviors are occurring.

NOTE: For the picky BCBAs out there- I explained it this way in order to avoid explaining response classes, topography vs function, and other difficult to understand concepts. That’s way beyond the score of this post. 

Let’s say we are teaching a learner to wash his hands. Probably, we’ll start by creating a list of the steps the child will need to learn to wash his hands. The first step in this sequence might be to turn on the water, and so on. Often, practitioners will measure how well the child is doing by measuring the “prompt level.” For example, a child might require a physical prompt to turn on the water. In this case, the therapist or parent might have to hold the child’s hand to show them how to turn on the water. After some teaching, the child might do it with just a gesture towards the faucet or a verbal reminder to turn on the water. Finally, we hope that the child turns on the water without any prompts. This allows the practitioner to determine if the child is improving or if he or she needs less intrusive prompts over time.

I find this to be an extremely common measurement procedure in practice, but when I look through the new edition of the White Book, this procedure isn’t in there. This book is generally considered the flagship textbook in Applied Behavior Analysis. Why does our flagship textbook not cover such a commonly used procedure? I suspect it is likely because it is a very poor way to measure progress.

I don’t know how the use of prompt-level data became so frequently used in practice. I used it myself for many years. But later, I realized there were much, much better ways to measure progress.

Prompt-level data is problematic for the following reasons:

• A prompt is not the child’s behavior, but the teacher’s behavior. The teacher is the person who determines what prompt to use and when to use it. This data often varies dramatically based on who is doing the prompting.
• Prompt-level data often interferes with good teaching by requiring the therapist to record a lot of data instead of focusing on the learner.
• Effective teaching requires the therapist to completely focus on the learner and to provide the reinforcement at just the right time. That feedback is the primary mechanism required for teaching. This procedure makes the teacher focus on the prompt rather than the reinforcement.
• Prompt-level data can often exaggerate progress. It is relatively easy to show a reduction in the intensity of prompts, but it can be difficult to obtain true independence.

What do to instead? Simple. Just measure each step in the list of skills and record if the child did it independently or did not do it independently, regardless of what the teacher did to help. This produces much more reliable data, and makes it easier for the teacher to focus on teaching.