POOGI for ABA

Just about everyone who enters fields like education, psychology, counseling, or behavior analysis has good intentions. We want to do the right thing and make a significant difference in the lives of the people we serve. But in one well-known study on mentoring for at-risk youth, the results of the long-term follow-up were just awful. On every measure, including how long they lived, job and marriage satisfaction, mental health, physical health, alcoholism, the at-risk youth who had mentors performed worse. In other words, if you were an at-risk youth, you would be much better off if these people trying to help you just left you alone. In fact, there appeared to be a dose relationship; the more mentoring you received, the worse you did in life.

One particularly interesting finding is that many, many of the men who were in the mentoring group remember their mentors fondly, giving them credit for keeping them out of prison and improving the quality of their life. Yet, most of them were probably wrong. Likely, they would have done better without any mentoring at all.

Now, this is not an argument to avoid trying to help people in trouble. But realize that just because you have good intentions and you attempt to help doesn’t mean that you will succeed. Most professionals refuse to believe that their intervention might be harmful in the long run.

BCBA’s (like all other professions) don’t think that this argument applies to us. And behavior analysis has a better argument than many fields. First, in behavior analysis, we have long-term studies that show great improvements for many children with autism. Second, we have a culture of constant data-based decisions. If our interventions are not effective, we make changes based on the individualized needs of the child.

The problem is that most of those long-term studies were not done under the practical conditions you and I face every day. Most weren’t done in public schools with countless rules and requirements that are far from optimal. They weren’t spending precious therapy time on all kinds of notes, filling out forms with procedure codes, and other stuff that health insurance companies require. In the old days, we met as a team–usually weekly. Insurance companies don’t pay for that, so it typically doesn’t happen any longer. Does that make a difference in the long-term outcomes?

Even if all those differences don’t change outcomes much, some children don’t have very positive long-term outcomes. I think this is particularly true in the treatment of severe problem behaviors. Things can look good for a long time before they fall apart.

We need long-term research data. Unfortunately, that’s difficult and extremely expensive.  In the meantime, we can think long-term and avoid being overconfident.

A long time ago, in a galaxy far, far away, I had a very intense job. Actually, I’ve had more than one very intense job.

At one job, I was a waiter, and Monday mornings were generally a bit slow. In fact, so slow that often, the manager and I were the only ones working. The manager had to leave to make bank deposits on Mondays, leaving me alone to run the restaurant. Most of the time this wasn’t a problem. A customer would come in and I’d pour the coffee, throw the eggs down on the grill, butter the toast, take cash at the cashier, and on to the next customer. Even with just 4 or 5 customers, it was a bit hectic, but I could handle it. I made $2 per hour, plus tips. Occasionally, the manager would leave, and I’d get a flood of people—I could hardly breathe until the manager returned. Since the restaurant was in a vacation area, sometimes the same family came in for breakfast, lunch, and dinner, and I served them all three times because I’d sometimes work 12-hour shifts from 9 AM-9 PM. I didn’t have time to sit down once and they’ve all eaten their third meal.

The thing is, I liked when it was super-busy. Not just because we were paid mostly in tips, but also when things were that intense, I would get into a flow, and time just flew by. I would get to work, and in the blink of an eye, my 12-hour shift was over. Working intensely without interruptions feels good.

Later, I worked as a supervisor in a program for children with autism which was intense in different ways. I was again super-busy—I had a huge caseload. And while I didn’t yet have a cell phone, I did have email. I often received a lot of emails that were important, urgent, and needed to be answered quickly. This was before the BACB (Behavior Analysis Certification Board), and we didn’t have things like limits on caseloads. The amount of work (and email) I had was staggering.

Yet, the work at the program for children with autism didn’t feel like the work at the restaurant. It didn’t fly by quickly. It felt intense and stressful most of the time. Sometimes I’d get that feeling of flow like at the restaurant, but not usually. As I’m learning from Cal Newport’s new book, that might have been partially caused by the email. Email can cause stress, anxiety, and generally be an ineffective way to get work done. Answering emails is much different than serving customers at the restaurant or working intensely with a client. You don’t get that positive feeling of flow from email.

There are many lessons to learn here. Be suspicious of how email / text messages are influencing your work. I have argued before that sometimes we make a huge mistake when we think that an email or a text is a substitute for a conversation. I’ve also argued that having your phone on while working with clients will substantially decrease the quality of your work.

I’ve been working on workflow systems for many years, and it is still a work in progress. You are unlikely to learn how to organize yourself, handle enormous loads of email, text messages, and prioritize your workload in a graduate school class or a CEU course. But these are skills all professionals need to learn. There are lots of potential ways to improve, but just realizing that these things can have a big impact on your effectiveness is a good first start.

That’s not a particularly interesting question for several reasons, but mostly because if you are like most people, you will gain it all back within six months to a year. When articles say most people, they usually mean a majority of people–maybe 60%. But when talking about weight loss, it is something like 97- or 98%. Really, “most” is not the right word. Maybe we should say “nearly everyone.”  You worked hard–you ate salad; you went to the gym–but the results didn’t last over time. There are a variety of reasons why losing weight is so difficult. But at least we have data to demonstrate the extreme challenge you are likely to face.

­We see a similar phenomenon in behavior analysis when working with children with autism and other developmental disabilities–the results of interventions often don’t last. But the weight loss literature has a major advantage over behavior analysis. They have a lot of data demonstrating that in the vast majority of cases, weight loss interventions are completely ineffective in the long run. This wealth of data exists because it is so easy to collect. All they need is a scale and people willing to come to the office for weigh-ins, and boom–in two minutes they can weigh each subject and report the maintenance data.

That’s not the case with autism and other developmental disabilities. Many, many interventions for language, social skills, and especially severe problem behaviors do not last over time. But getting data on which interventions, how often, and under what conditions they do or do not maintain is especially difficult.

I believe that solutions to long-term weight loss are developing in the medical literature. But a necessary first step was recognizing how likely failure is to happen with the standard procedures.

If we want to Poogi as a field, we have to figure out a way to collect these types of data. Many people who work with young children think their program is great, and maybe it is, but you don’t know until you look at what happens to the kids after they leave your influence. The people who work in adult programs know the truth of how well we did in the early years.

If you don’t know how you are doing, it is hard to be on a POOGI. Ideally, we would regularly follow up with families a few years after they left our influence. That would give tremendous insights. But most importantly, as a field, we need data on how likely our interventions maintain under practical conditions.