Author: Barry Morgenstern

That’s not a particularly interesting question for several reasons, but mostly because if you are like most people, you will gain it all back within six months to a year. When articles say most people, they usually mean a majority of people–maybe 60%. But when talking about weight loss, it is something like 97- or 98%. Really, “most” is not the right word. Maybe we should say “nearly everyone.”  You worked hard–you ate salad; you went to the gym–but the results didn’t last over time. There are a variety of reasons why losing weight is so difficult. But at least we have data to demonstrate the extreme challenge you are likely to face.

­We see a similar phenomenon in behavior analysis when working with children with autism and other developmental disabilities–the results of interventions often don’t last. But the weight loss literature has a major advantage over behavior analysis. They have a lot of data demonstrating that in the vast majority of cases, weight loss interventions are completely ineffective in the long run. This wealth of data exists because it is so easy to collect. All they need is a scale and people willing to come to the office for weigh-ins, and boom–in two minutes they can weigh each subject and report the maintenance data.

That’s not the case with autism and other developmental disabilities. Many, many interventions for language, social skills, and especially severe problem behaviors do not last over time. But getting data on which interventions, how often, and under what conditions they do or do not maintain is especially difficult.

I believe that solutions to long-term weight loss are developing in the medical literature. But a necessary first step was recognizing how likely failure is to happen with the standard procedures.

If we want to Poogi as a field, we have to figure out a way to collect these types of data. Many people who work with young children think their program is great, and maybe it is, but you don’t know until you look at what happens to the kids after they leave your influence. The people who work in adult programs know the truth of how well we did in the early years.

If you don’t know how you are doing, it is hard to be on a POOGI. Ideally, we would regularly follow up with families a few years after they left our influence. That would give tremendous insights. But most importantly, as a field, we need data on how likely our interventions maintain under practical conditions.

 

When teaching high-level cognitive skills, it can be incredibly difficult to determine when you are finished. For example, let’s say we are teaching a learner the concept of before / after.  We might do this by showing the learner a sequence of pictures like this:

We might teach this by asking the learner to:

1. Point to what happened before the boy rinsed his hands.
2. Point to what happened after the boy applied soap.
3. Tell me what happened before the boy scrubbed his hands.
4. Tell me what happened after the boy turned off the water.

After the learner acquires this skill with many sequences and shows generalization to novel, untaught sequences, we might think we are done. He now “understands” before / after. He can do it with sequences we have never shown him before! This skill is mastered! Time to celebrate.

But if you have been around for a while, you probably know it is too soon to celebrate. We are just getting started. We now need to teach:

When did the boy turn on the water?

The expected answer might be:

Before he applied soap.

OK, done! The learner mastered the before / after questions with “when” and it was integrated with the previous four questions. He showed generalization to novel examples that we didn’t teach. He has truly got it now!

No, probably not. Go ahead and celebrate some, but there’s still more to do. How about things like this:

What did the boy do before he dried his hands?

Or:

Before you touch your nose, clap.

Clap after you touch your nose.

Or:

The learner is playing a game with a group of friends. Who went before Joe? Who goes after Sally?

What if we combine it with other difficult concepts, like pronouns?

Who gets a turn before you? Whose turn is it after me?

And what about remote events:

What are you going to do after school?

What did you do before school today?

This kind of problem is present in virtually any teaching situation from preschool through graduate school. If you are interested in a deep dive, check out the book Theory of Instruction. Unfortunately, that book requires a few lifetimes to study. For some immediate teaching resources for elementary school children, check out the Direct Instruction programs written by Engelmann and colleagues.

Behavior analytic services should only be delivered in the context of a professional relationship. Nothing written in this blog should be considered advice for any specific individual. The purpose of the blog is to share my experience, not to provide treatment. Please get advice from a professional before making changes to behavior analytic services being delivered. Nothing in this blog including comments or correspondence should be considered an agreement for Dr. Barry D. Morgenstern to provide services or establish a professional relationship outside of a formal agreement to do so. I attempt to write this blog in “plain English” and avoid technical jargon whenever possible. But all statements are meant to be consistent with behavior analytic literature, practice, and the professional code of ethics. If, for whatever reason, you think I’ve failed in the endeavor, let me know and I’ll consider your comments and make revisions, if appropriate. Feedback is always appreciated as I’m always trying to Poogi.

There is an old saying that predictions are difficult, especially about the future. Many times, when we run into problems, we blame random chance as the cause of the problem. For example, we might have scheduling problems due to random unpredictable events:

• Three staff are out sick on the same day.
• A critical staff person leaves for more money or to go to graduate school.
• Several staff members are on maternity leaves.

Sometimes the children we serve have problems that we blame on random chance:

• He was asking to go outside, but we had to say no because it is raining.
• I was there to do the session, but he had to use the bathroom and I missed most of my time with him.
• We were working on the iPad, and the battery ran out just when we were going to work on the reading program.

Sure, random unpredictable things do happen and you can’t prevent all of them. The mistake is thinking that the event was unpredictable and there is nothing that I could have done to prevent all the problems that the random event caused. I just have to deal with it.

No, you don’t. You might not be able to predict the time exactly, but you know that sometimes multiple staff will get sick on the same day–at least occasionally. What’s the plan to handle that? Why can’t the child be asked to go to the bathroom before critical sessions?

Having a plan–a system in place to handle these types of “unpredictable” events–often means the difference between success and failure.

Schools are supposed to provide children with disabilities with FAPE (Free Appropriate Public Education). But frequently, parents of children with disabilities and school districts don’t agree on the meaning of the word “appropriate.” 

There is a lot of legal opinion on this topic that I don’t intend to get into here. For now, the main point is that there isn’t an agreed-upon standard to determine if a program is appropriate. 

 

There are some generally accepted rules. For example, the school district is not required to provide the “best,” only what is appropriate. On the other hand, if a child is making no progress or very minimal progress, the program is clearly “not appropriate.” This can sometimes be an adversarial process; schools and parents sometimes turn to outside evaluators to help make the determination. 

 

When you have a subjective standard like this, it is impossible to prevent huge biases from entering into the decision. This is especially true when the results of an evaluation can mean very large sums of money (e.g., if the evaluation causes a school district to outplace a student to a private school for example) and loss of control over a program. 

 

I used to be one of the expert evaluators that would give an opinion on whether a program was “appropriate” or “not appropriate.” Although it can sometimes help children get better programming, I rarely agree to do these anymore. I just didn’t enjoy the work. I’d rather be the person doing the programming, and then let someone else evaluate my work. I’ll Poogi more that way, too. 

 

The key lesson is that these decisions are rarely made solely based on the data (even if the issues are primarily decided by BCBAs). I’ve seen terrible programs survive an evaluation as appropriate. I’ve seen excellent programs be deemed not appropriate. It is not enough to master data analysis; we must learn to work in complex social environments too. 

 

It was my first day of a new job. I walked into a super-confrontational meeting between parents and a school district. The discussion was heated. The district was very unsatisfied with the current behavioral consultation and the parents were extremely happy with the current consultants. During the discussion a school district leader asked me, “How long will it take for you to learn about this student and take over the consultation?”

That was a very tough question. There were about 15-20 people in the room. I had met a few of them earlier in the day. I didn’t know much about the student besides what I heard during this difficult meeting. I don’t even know what the rest of my caseload will be like and how much time I can devote to the case. If I say it will take quite some time, I’m sure to upset school district leaders who are furious with the behavioral consultants. If I say I can do it quickly, I’m sure to upset the parents who I haven’t even met outside of this heated discussion. Everyone is emotional and on edge. What to do?

“Well, this is my first day and I think I can remember where the bathroom is located.”

After that, I was able to talk a bit about how I would get to learn about the student without making any specific commitments about when I would complete the project. I was able to build rapport with the parents and school district leaders in subsequent weeks. Many other times, I haven’t been so lucky.

The problem is that as BCBAs we are supposed to make recommendations based on the scientific literature. Often, however, we are asked to make recommendations where it isn’t easy to point to the literature that would justify specific recommendations.

• How many hours of consultation per week are required for the student to have an “appropriate” program?
• Does this student need a 1-1 Registered Behavior Technician in order to have an “appropriate” program?
• Is this goal “medically necessary”?

We can certainly interpret the scientific literature to make recommendations like this even if there is no study exactly on point. We can collect data to inform these decisions. We can and should do both of those things.

I would suggest that in addition, it is important to realize that in the real-world, decisions like these are often made on the basis of arm wrestling. It is more like buying a used car than a debate on the science. Sure, data and scientific literature can help, but it usually only has a small impact on how decisions are usually made. That’s because many times there is no obvious science-based answer to these types of questions and different BCBAs are likely to disagree. Knowing the science and how to collect data to evaluate decisions is necessary, but not sufficient to handle these situations well. Learning to negotiate the complex social environment is essential too.