POOGI for ABA

Inclusion of children with autism in general education classrooms can have wonderful academic and social benefits. But those benefits don’t always occur. I often see parents and educators focusing on whether or not the child is in the classroom, not if the child is actually learning any skills while there. I’ve seen many situations where the child is present, but isn’t learning much. Sometimes nothing at all. That would suggest the services being offered are not appropriate, but whether or not that’s true is hard to measure.

There is no situation where we can provide appropriate behavior analytic services without collecting data. Sure, schools usually require some regular testing. That might be appropriate in some cases, but often the testing is too infrequent to use as an on-going measurement for how much learning occurs in the classroom. I have seen many BCBA’s struggle with this problem. Many measurement systems I have seen simply fail to capture the data needed to make decisions. It can be difficult to measure the effects of inclusion for several reasons:

• In a general education classroom (especially in the younger grades), the child is often prompted a lot by general education teachers, special education teachers, and other classroom helpers. If the child is not given enough opportunity to respond independently, progress is difficult to measure.
• One of the benefits of inclusion is social opportunities. But often, peers will help their friends with autism with academic tasks, which can limit opportunities for data collection.
• General education teachers often move through topics quickly. There might not have been enough opportunities for assessment as the teacher has moved on to a new lesson.
• One common comment from teachers is that “all the children are having difficulty with XXX,” with the implication that we don’t have to measure whether the instruction was effective for the student with autism because ineffective instruction is being delivered to all the students.
• Often IEPs include modifications that can make evaluation difficult. For example, often the child is given multiple choices to fill in the blank questions. This can make evaluation difficult because, as we know, multiple choice questions can be super-easy or super-difficult depending on the contrast between the choices:

2 +3 = __
(a) 786
(b) 2941
(c) 5

There is a relatively simple (but not easy) solution to this problem. It isn’t easy because it takes significant time and coordination with school teams, but I have found it will consistently give the data needed to make important educational decisions. I call it the Classroom Learning Measure. Here is how to do it:

1. The BCBA meets with the general education teacher to obtain the most important objectives that the children are expected to acquire in upcoming lessons.
2. The BCBA creates a simple assessment that measures the objective.
3. Give the assessment to child before the teacher begins teaching this objective, without any prompting or support.
4. The child goes to class with all the supports designed by the IEP team (e.g., push in services from special education teacher, speech, modifications, etc)
5. Give the assessment to the child again after the teacher has taught the topic, without any prompting or support.
6. In a few situations, I’ve been lucky enough to get approval to have all children in the classroom tested in this way so that we can see how well the child with autism is learning compared to his or her peers.

Once these data are collected, teams now have valuable information. In some cases, teams will find that the services are effective. But in my experience, many will find that the services are not effective, and significant modifications are needed in order to produce successful academic outcomes. In my view, it is not sufficient that a child is “present” in the classroom. The child has a right to acquire skills too. Too often, we aren’t checking to see if this is occurring.

In applied behavior analysis, BCBAs often write a document that describes how to teach a learner a particular skill or respond to a behavior. This is often called a “program.”  One of the core features of behavior analysis is that the written procedures in our programs are very detailed. In research, that is so the procedures can be replicated by other researchers. In practice, the high level of detail is there to help guide staff or parents in producing a behavior change. While having a well-written program can be useful, it often creates a misleading impression that making a behavior change is easy.

There are many difficult-to-learn skills that people instantly realize that you can’t learn just from reading how to do it such as riding a bike, driving a car, or performing surgery. Making a behavior change is like those examples. We cannot expect untrained people be able to implement the procedures in our programs correctly solely from reading a document. In fact, our research overwhelmingly shows that people cannot do that. Only after you go through a significant amount of training will you be able to pick up a behavior change plan and implement it properly, no matter how well it is written.

So how should BCBAs respond to requests for suggestions of what to do at home?  On the one hand, of course we want the skills we teach to generalize to the home. On the other hand, there is significant risk that implementation by untrained people is risky at best. I think we have a few good options:

1. Depending on the skill, it might be appropriate to take the time to do the full training for parents. We make sure the parents are trained to criterion and that the practices are appropriate and practical for everyday use before we let the parents implement the program.
2. Depending on the skill, it might be appropriate to ask the parents to implement only a part of the intervention. We explain to the parents that the child will begin by learning parts of the skill, but isn’t ready for full implementation quite yet. We then make sure the parents are fully trained to implement the part of the intervention that is appropriate.
3. Depending on the skill, it might be appropriate to ask the parents to wait on implementation at home. We can explain that the skill is very fragile and we would like to ensure the implementation procedures are successful before we do the training for the home.

Any of those might be appropriate depending on the details of what any particular child needs. If Option 3 is selected, however, we need to realize since the parents have a problem at home, they are probably going to seek out other temporary solutions while waiting for our intervention to be “ready.” This may be why we see BCBAs offer families “resources” that they should know from the research are overwhelmingly unlikely to be successful. When a parent asks what they can do at home, it is an extremely important question, but we often don’t think through how best to handle it. It can affect the child’s progress, parent satisfaction, and all of the relationships involved. A high-quality program takes the time to think through the family situation and figure out the best approach to program implementation rather than advising them to try procedures that are unlikely to work or are difficult to put into practice.

There are many stories of workers who are afraid to tell their supervisor something is wrong. Co-pilots are afraid to tell the pilot about a critical safety issue. Nurses are afraid to tell the surgeon he is operating on the wrong leg. Sometimes, they are afraid to speak up even in life threatening situations.

In those field, there has been some research and training to address this important issue. The basic problem seems to be that there is no reinforcement for speaking up. Indeed, there are significant punishments, as apparently (at least in some situations) surgeons and pilots tend to act in an extremely vengeful manner toward anyone who challenges their authority. Leaders in the aviation and medical fields took steps to encourage people to speak up since there were clear-cut cases where people died, even though the co-pilot or the nurse clearly knew that there was a problem and the tragedy could have been averted.

Although behavior analysts are not usually in life threatening situations, we have the same issues that pilots and surgeons have—our staff are sometimes afraid to tell us something is wrong. Instead, staff training usually focuses on the importance of following the procedure exactly to maximize client gains. We rarely encourage staff to give feedback on what can be improved, and often they are too afraid to speak up when they see problems.

From what I’ve seen, it is a rare organization or BCBA that really puts an emphasis on the Poogi and encourages staff or parents to speak up when they see problems. But how can you tell if they are afraid to speak up? One warning sign is when a parent or a staff person says,

“But you said…”

What causes a parent or staff person to use this phrase? This phrase is defensive. The staff person is afraid of getting blamed for something going wrong. For example, let’s say a child is failing to learn a skill or a behavior problem is not improving. During the problem-solving discussion, the BCBA might ask the staff something like, “Why are you using that procedure?” The staff think they are doing the right thing, and interpret the BCBA’s question as an accusation.  That leads to an almost instinctive response, “But you said…”

The key issue that I’m interested in here is not who is at fault. What concerns me is if there is an obvious problem, why is the staff not bringing it to the BCBA and having a conversation? Although hard to prove, my view is if staff aren’t bringing problems to you, it is often because they are afraid. If you hear people saying, “But you said…”, that’s a clear sign they are afraid of getting blamed for problems, and it’s time for a review of how staff can be encouraged to speak up.