POOGI for ABA

Preference assessments are a set of procedures that are often used to select what items people with disabilities prefer in order to use those items as reinforcers when teaching. I do think that the research on preference assessments has produced some valuable knowledge for the field, but I’m skeptical that the procedures themselves are very useful for practitioners. Nevertheless, I’ve sat through many, many conference presentations where they act as if you would have to be crazy not to use such a well-validated set of procedures.

Preference assessments take significant time. There are a variety of different methods of selecting preference, but they all take time away from instruction. This recent research compared formal preference assessments with “in-the-moment reinforcer analysis” and found them both to be effective, but the in-the-moment reinforcer analysis was substantially more efficient. Of course, if you don’t have to spend a lot of time on the preference assessment and can just teach, you get more done in less time.

In the moment reinforcer analysis is basically just looking at the child to determine if he was interested by using things like affect—did the child appear happy? Did the child make statements to suggest he was enjoying the item? There are other procedures such as rotation of reinforcers to make sure the child doesn’t get bored with an item and conditioning new items.

You might say “of course.” But that didn’t stop me from using formal assessments for a long time.

In addition to saving time, staff will be much more effective in the long run if you stop using preference assessments:

• Good therapists need to learn to assess the child in the moment. There are many situations where it simply won’t be possible to do a preference assessment. We work in classrooms, homes, and community settings. If you can’t assess “in-the-moment” you just won’t be effective.
• Good therapists need to learn procedures like conditioning new things as reinforcers and rotating reinforcers. These will help substantially improve results over time.
• Good therapists need to be creative and try a lot of things. If you had to put everything into a formal preference assessment before you tried it, the odds of trying lots of things will go down dramatically.
• Good therapists recognize that there are thousands of small subtle ways that the therapist interacts with the child that we can’t always measure exactly. (e.g., the way the therapist smiles, flips her hair, voice volume, tone, or plays with the child and the object). These interactions can make a huge difference in the effectiveness of a specific thing as a reinforcer.

As usual, more research is needed. While we are waiting, I think there is enough evidence and logic on the side of “in-the-moment reinforcer analysis” to justify that recommendation for practitioners.

The What Am I Doing Wrong Procedure

Most supervisors claim to want feedback from staff, but in my experience it’s very rare for staff to actually provide it. I have written before about how staff sometimes are afraid to tell you when something is wrong. It is a hard problem to solve, as there is no obvious way to know if someone is upset about something and they don’t tell you. In my view, this is a major obstacle that will prevent Poogi. You need the people you supervise to speak up and tell you when something isn’t right. Your programs, your supervision, nothing in your whole organization is perfect. One of the biggest things that we don’t often take advantage of is people’s knowledge about what needs to be improved.

On another front, this is a critical skill that your supervisees will need in their future careers. When people are sitting in meetings, listening to ideas, collaborating with other professionals, and many other activities, the ability to speak up and clearly communicate when something isn’t right — either ethically or scientifically — is a critical skill.

A procedure I’ve used that at least seems effective (it’s hard to know for sure) at increasing the amount of feedback I receive is the “What am I doing wrong here?” procedure. Specifically, I will present something at a staff meeting that sounds as good as I can make it, but doesn’t make sense scientifically or ethically. Then I see if anyone speaks up. If no one does, I might prompt. “Does anyone see anything wrong here?” Then, I enthusiastically reinforce the staff person when they provide feedback. Tell people you are going to do this on a regular basis. Also, if you are the supervisor and you make a natural mistake, it’s great to poke a little fun at it — “Oh, I meant to do that!”

Eventually, people realize that your goal is the POOGI. Once people see giving feedback has both natural reinforcers (we all want to see the program improve) and tangible rewards, you will increase the amount of feedback you receive.

Warning! There will be more wrong than you can imagine. Not only that, it will go on forever as there is no end to the POOGI. It’s not easy, but it’s worth it.

BCBAs tend to have a real love of data and graphs. There are no situations in behavior analysis where we can work without having data. Most of the time, we naturally want to use the best procedures available. We aren’t working with fat grants and usually can’t have multiple people watching sessions behind one-way mirrors recording data on sophisticated laptop computers as often happens in important research projects. In addition to the high cost, there is also practicality — we are in homes, classrooms, and community settings. Still, we try to use the best procedures available under the practical constraints that we have. Often, this is a mistake.

Why? Don’t you believe in Poogi? Shouldn’t we do the best we can do? The problem comes when we compromise on using the best intervention programs available in order to collect more data.

The basic rule on designing intervention programs is to first design the best possible teaching procedures available. Only after you have the determine the best procedures for the individual client should you decide on the data collection. The data collection procedure must give you all the information you need, and the data must be valid and reliable, without requiring compromises in the teaching procedure.

For example, when teaching conversation skills, sometimes behavior analysts ask the RBT or the parent to write down everything the child says. Now, those data might be interesting! We might see if the child is using language learned in speech sessions spontaneously in conversations at lunch. We might detect patterns in conversation topics, etc. etc. But if you watch those sessions, it will usually quickly become apparent that the teacher can’t really do a great job teaching conversation when they are trying to write down every word the child says. You need a different procedure.

Another common example is many instructional programs involve some type of prompting and fading. Now, often it would be interesting to know exactly what type of prompt was used on each instructional opportunity. But I rarely attempt to do this. When I’ve designed teaching sessions this way, frequently it is obvious that this data collection method decreases the ability of the instructor to teach well. Effective teaching depends on careful observation and getting your timing exactly right. Most of the time this isn’t done well when instructors are focused on writing down every prompt used.

POOGI doesn’t mean trying to improve everything all the time. POOGI means trying to improve the right things. The design of the intervention takes priority. POOGI that. Make the intervention the best it can be. Then make sure the data are good enough to make decisions, but don’t go so overboard that you reduce the quality of your program. It happens more than you think.