POOGI for ABA

The Importance of Second Opinions

A parent recently called me for advice. The problem the family was facing is that they were losing their current services due to changes in funding. The family would no longer have their current BCBA and team, with whom the parents thought the child had made excellent progress. Now, there would be a new BCBA and a new team. The parent was extremely concerned. These services would occur at a school, so it wouldn’t be easy to observe. How will she know if the school-based BCBA and school staff are doing a good job?

In my experience, the parent is definitely justified in being concerned about this problem. I have seen great services delivered in school systems, and I’ve seen terrible services delivered in school systems. Of course, sometimes the answer is obvious. For example, if there are frequent safety issues due to dangerous problem behaviors, that is a warning sign that there is likely something wrong with the services. On the other hand, if the child is making friends, getting good grades, and doing well on standardized tests, that is pretty obvious something is going right. But usually it isn’t so simple. As a parent, it isn’t always easy to tell the difference between low quality and high quality.

Sometimes, people will suggest looking at the data. That is certainly helpful, but insufficient for numerous reasons that I’ve discussed frequently on this blog. Even an expert can’t really evaluate the program quality by simply looking at data, even if there are no misleading aspects of the data and it was collected with perfect accuracy, which is NEVER the case.

Sometimes, people will suggest asking other parents. That is helpful too, but also insufficient. Parent reports are often good indicators, but certainly not always. Several times I remember doing an evaluation where I thought the services were of very low quality, yet the parents were completely satisfied. Other times, I’ve seen parents upset with the quality when in my view the services were excellent.

Sometimes, people will suggest looking at how services are generalizing to home. This can be very helpful. If you are seeing generalization to home, that’s certainly a great sign. But the lack of generalization could be caused by a lot of potential factors. Certainly, the problem should be addressed, but it isn’t possible to draw a straight conclusion that services are poor based on this evidence alone.

In my view, this is an area where parents frequently need a second opinion. I often suggest that parents have an independent outside expert BCBA provide an evaluation of the program. This offers several advantages. First, it allows parents to get a realistic picture of the quality of the services. Second, even if the services are of high quality, there is always room for Poogi. Having another set of eyes can often lead to great ideas for further improvements. Third, unfortunately, the way the world works is that is that the “squeaky wheel gets the grease.” Often, schools have limited resources. Simply having an outside evaluator come in to review a program can improve the quality. Fourth, if the program is of low quality, you have the opportunity to make changes early and get started on a POOGI.

It is important to pick the right type of expert. You are looking for someone who will give an honest opinion. I’ve seen some evaluations that are from parent advocates, basically echoing whatever the parents want them to write in the report. In a similar manner, I’ve seen some evaluators that will pretty much put in whatever the school district wants them to say. Usually, these aren’t BCBA’s who must adhere to our ethical code.

Unfortunately, it isn’t always easy to find an expert to do this type of work. At least in Connecticut, where I live, I know people who do an excellent job providing these types of services are often booked months or years in advance. We need more people doing this type of work. If you live in Connecticut, contact me if you need a recommendation. But realistically, there are long waiting lists for all the excellent evaluators.

If you have worked in behavior analysis awhile, you are sure to hear the criticism at some point that applied behavior analysis is no different from animal training. That’s just silly. Of course, there are many, many differences between teaching humans versus training animals.

But there is an element of truth. There are some aspects of learning that have been shown to be universal across species. So, it doesn’t matter the organism that you are teaching. Some of the same teaching procedures can be useful.

I was lucky that one training I attended probably stole a game from the animal trainers–the shaping game. I’ve been using this game in training sessions for more than twenty years, and have found it to be an extremely useful in training staff and sometimes parents. When shaping a behavior, if you are off even one second on your timing of reinforcement delivery, you can lose a lot of the benefits of the treatment. But timing is difficult to practice. This game is an effective way to introduce the topic and get people started on learning this essential practice.

How to Play the Shaping Game:

1. One person is sent out of the room and will be the learner.
2. The group that is still in the room decides on something to “teach” the learner. No embarrassing tasks allowed.
3. When the person comes in, there is only ONE person designated as the teacher. If you don’t explain this well, the whole group starts attempting to teach.
4. The learner who left comes back in, and the teacher can only clap to provide reinforcement to the learner. No other teaching procedures allowed (e.g., prompting).
5. Everyone in the room claps when the learner is successful.

A Few Tips for Trainers:

1. The easiest to teach tasks are using an object in a typical way (e.g., sit in a chair, write with a pencil, or turn off a light switch). The tasks that are the hardest are multiple step tasks and using items in an atypical fashion (e.g., tape a pencil to the wall).
2. It is generally better to start with difficult tasks at the start of the shaping game with experienced people who won’t get upset during the game. This can model several things for the group.
   • First, there is no expectation that the learner will get it quickly. It is often tempting to start with a simple demo, but this will mess things up later when people are unable to learn their tasks easily and get upset.
   • Second, experienced people can model that you have to move around and try things. The hardest person to teach is someone who just stands there and shrugs “I don’t know what to do.”
   • Third, this is a FUN game. There is no need to get frustrated; we are all learning.
3. It is generally very helpful to point out the three critical features modeled in #2 after the first demo.
4. Be sure to make the point that the feeling of not knowing what to do is how our students might feel frequently.
5. A good shaping game trainer discusses the details of what happened during each round, specifically looking for opportunities to point out what happened when timing was off by just the slightest amount. You can always find them. Do this with humor, and you will create a memorable experience with insights that people will not forget.
6. Sometimes, if someone is struggling, it is helpful to send the learner out of the room and do a live coaching session with the teacher about what they might do to Poogi their teaching. Common suggestions include increasing the frequency of reinforcement, intensity of reinforcement, or improving the way the skills are task-analyzed.
7. As the trainer, be sure to take an opportunity to be a learner. People appreciate it and you can model appropriate trainer behaviors.

I have found that many people genuinely love to play the game, and are eager to participate to help train new staff. Granted, it is hard to measure whether it truly improves staff performance in actual work with children. But at least it gives you something to prompt when you are coaching a staff person. It seems helpful to be able to say something like, “Remember when we were playing the shaping game and…”

It’s up to you to decide whether or not you disclose that this activity was probably stolen from the animal trainers.

When I worked as a 1-1 with children with autism, the school once asked me to substitute for an absent staff person. Sure, there was some paperwork with some basic information about the student I would work with for the day, but otherwise no one was there to give me all the details I would need. As soon as I arrived and met the student, he started to shoot off a round of non-stop questions. “What time will we be going to lunch today?” “Where will we be working today?” and on and on. I explained that I didn’t know, but I’d find out. The student correctly concluded, “You don’t know a lot of things.” Luckily for me, the student had enough communication skills that I could figure out the information he wanted and get through the day successfully. But over the years, I’ve seen many, many staff in many schools that were not so lucky. Not knowing basic information can often lead to severe problem behaviors, safety issues, and other emergencies.

Many years later when I became a supervisor, I became extremely interested in this topic. There is a tremendous amount of information that is known by staff, but is not documented anywhere. It lives in the minds of the staff. That’s a problem for many reasons:

• It can become an emergency when the staff person is sick, leaves, goes on vacation, maternity leave, and sometimes even goes to the bathroom.
• Staff are reinforced for keeping the problem behavior low, but are often using techniques that might not be in the student’s long-term best interest.
• The information that is in the staff person’s heads are often a goldmine that can lead to the best goals that we should be setting for the student.
• It can be extremely difficult to find out this information since the knowledge has become second nature to the staff. Even if they want to help, they are often unable to accurately report everything that they do in order to be successful with the student.

The key lesson is that it is worth putting substantial time and effort into the documentation of all the procedures needed in order for a student to be successful. It will reduce emergencies, improve generalization, and help select better goals. Of course, it is hard to do because there is no one pressuring you to get it done. But the effort is more than worth it.