Quality over Quantity – POOGI for ABA

Obviously, when we want to know how well a child is doing, we want to see the data. The catch phrase of a BCBA is often “show me the data.” The problem is that sometimes those data are not summarized in a very useful way. In many cases, data summaries are downright misleading. For example, if we want to know how a particular child is doing, we are probably not going to look through dozens of graphs or hundreds of pages of raw data, we need a summary. Summaries often include things like this:

Problem behaviors are down 90%
50 New Words Learned
etc.

Now, sometimes when I’m doing an evaluation, and I look closely at these types of data, I find that the data were collected very poorly, or that the staff can’t replicate these changes during an observation. But today, I’m not talking about bad data collection. Today, I’m assuming that these data are 100% accurate and reliable.

The problem is they may or may not be very meaningful. The problem behaviors may be down 90%, but only if the preferred staff is there, only if the team dramatically reduces the math presented, and never says “no” to requests. The child may have learned 50 new words, but only uses them if you practice in a distraction-free area, remind him to “get ready” before asking, and provide tangible rewards for answering.

In order to tell if something is meaningful, you need the generalization data. That’s what is important:

Problem Behavior (Examples)

Does the child still have low levels of problem behavior when:

The regular staff person is out sick?
When there is a fire drill?
When there is a change in schedule?

New Words (Examples)

Does the child use the words in spontaneous speech? When novel pictures are used?
Do novel people understand the new words?

As a general rule, it is much better to make fewer behavior changes, but choose behavior changes that truly make a difference. Make sure that each behavior change is really useful to the learner. Don’t get carried away with absurd numbers of goals or graphs.

It is easier to measure the quantity rather than quality. It can look impressive–50 new words! But in the long run, the quality of the behavior changes is what makes the difference in the learner’s life.

Behavior analytic services should only be delivered in the context of a professional relationship. Nothing written in this blog should be considered advice for any specific individual. The purpose of the blog is to share my experience, not to provide treatment. Please get advice from a professional before making changes to behavior analytic services being delivered. Nothing in this blog including comments or correspondence should be considered an agreement for Dr. Barry D. Morgenstern to provide services or establish a professional relationship outside of a formal agreement to do so. I attempt to write this blog in “plain English” and avoid technical jargon whenever possible. But all statements are meant to be consistent with behavior analytic literature, practice, and the professional code of ethics. If, for whatever reason, you think I’ve failed in the endeavor, let me know and I’ll consider your comments and make revisions, if appropriate. Feedback is always appreciated as I’m always trying to Poogi.