POOGI for ABA

Certainly, I’m no expert when it comes to the stock market. But I do think there is something we can all learn from technical analysis because it superficially seems similar to what we do in behavior analysis. Technical analysis refers to attempting to make predictions about stock market prices from historical stock market data. Essentially, people are analyzing graphs and other data to try to make predictions about the future. That may seem like what we do in behavior analysis. But it really isn’t, or it shouldn’t be.

A deep dive into technical analysis is both beyond the scope of this blog and my current abilities, but there is minimal evidence that it is effective, and there seems to be problems with the data in studies that found it to be effective.

Don’t we do the same thing in behavior analysis? We look at data on historical performance and then use that data to make predictions on future performance. If a child has increasing levels of problem behavior, we might change something to try to decrease problem behavior. If the child has had no problem behavior in months, we might assume that nothing needs to be done about the behavior.

A likely reason why technical analysis doesn’t work to predict the stock market is that there are literally millions of variables that may impact the price of a stock. Things like politics, labor negotiations, weather patterns that affect the price of raw materials, or whether I need to sell my shares due to a family emergency. All of those things can have an impact on the price of a stock. Therefore, the relationship from previous patterns in a graph are probably not a good predictor of future performance. But an occasional win by chance can lead to superstitious behavior, and make people think they can predict the future when they probably can’t.

But, in behavior analysis this shouldn’t be the case. In behavior analysis there is a clear relationship between what we do and behaviors observed. We specifically decide when to implement the behavior plan. So, if problem behavior is high and I implement a behavior plan on Wednesday, and then get an immediate drop in the problem behaviors, that’s much better than technical analysis because I controlled when to do the intervention.  It is probable that my intervention worked and was not due to chance. Although behavior analysts understand that the results could be due to chance, there are a variety of methods in single subject design that make that very unlikely.

Unfortunately, in practice it is easy to mess this up and fall into a pattern of doing something like technical analysis. Graphs of behavior without the analysis are not useful for making predictions. As an example, if we collect data on problem behaviors across the whole school day, there are so many variables that might impact the data that it will be hard to know what is causing what (e.g., a peer made a negative comment, there was a fire drill, the assignment was hard today, his pencil broke, the teacher usually calls on him frequently but didn’t today, etc. etc.).

For me, the lesson of technical analysis is that decisions based on data that don’t show “the why” are not effective.  It is easy to believe you are making scientific decisions based on relevant data, but instead fall into superstitious patterns. In behavior analysis, we also make decisions based on data, but it is absolutely essential that we know WHY those data went up, down, or are staying stable. If we don’t know “the why,” the data are essentially useless for making predictions about the future, especially when there are any changes to a program.

Most organizations working with children with autism and other developmental disabilities have policies around phone usage. Specifically, individuals working with clients should not have their phones on while working with clients. You shouldn’t be checking your email, social media, or texts. This makes sense–if you are distracted, performance will go down substantially. But there often is a big exception to this rule.

Despite having this rule for the direct 1-1 staff, many organizations do allow the BCBAs to have their phones on during the day. This is especially true if the BCBA is working across many locations, and people are likely to contact them with “emergencies.” This may not seem like a huge conflict; you just quickly respond to a text or two. No big deal.

While we don’t have data on how distractions impact BCBA performance specifically, I believe the substantial declines we’ve seen for other skills like driving are very likely to hold up for behavior analysts as well. Many BCBAs I’ve talked to about this topic simply don’t believe it impacts their performance. I think they are being delusional. But even if they accept that it does disrupt their performance, they believe there is nothing that can be done about it and just accept it as a “fact of life.” Clients will be upset if I don’t respond right away. Staff will be upset if I don’t respond quickly.

If you are constantly bombarded with texts, questions, and emails, that almost certainly means that your systems are not well designed. There should be a clear system for handling routine and predictable events that don’t require people to contact you for every little thing. But even if you have reasonable systems in place, communication with clients and staff can become overwhelming and reduce effectiveness.

I have been enjoying Cal Newport’s podcast, and I think he gives the answer to this problem in the first episode. Cal argues what people want is clarity on how they will be able to communicate with you and not necessarily constant accessibility.

When working with parents or staff, have a clear system for how to communicate (e.g., we meet every Tuesday at 10, notes of the meeting are emailed within 24 hours, etc.). When those types of systems are in place, people will not mind so much if you are not constantly available by text. Problems often arise when schedules are unpredictable and people don’t know when they are going to get to talk to you. That is what leads to constant demands that you respond immediately, help, provide support, etc. Think clarity of communication, not constant accessibility.

Setting up a communication system that provides clarity for parents and staff is a tremendous amount of work, but the huge benefits are worth it.

If you make a bad decision, it is very hard to learn from and make improvements based on that bad decision. My friend Alan Barnard just gave this great interview where he describes this, and he gives lots of excellent advice on making decisions. Alan isn’t a BCBA, but I’ve always found his insights to be extremely valuable.

It is so hard to learn from bad decisions because there is often a long time period between implementing the bad decision and receiving feedback on the decision’s impact. If you tried a procedure and received feedback immediately, it would be much easier to learn. Behavior analysts know this is a problem, and we take a very specific action to ensure that we can learn from bad decisions even if the feedback is greatly delayed–collecting data.

We collect data so that if we make a bad decision (e.g., a particular intervention is not effective), we can make a change quickly. This has allowed us to be dramatically more effective with our clients than if we didn’t use data.

Unfortunately, even that can fail if you collect data on the wrong things. If you teach A, when B was what would really make a big difference for the long-term best interest of the client, you might not realize it for a long time. Making sure you get feedback is important, but it’s still easy to make bad decisions if you haven’t thought through the long-term best interest of the client.