One thing that I learned from the book The Goal is it’s possible, even probable that when you have contingencies attached to data, people will engage in behaviors that are not necessarily in the best interest of the client.
For example, one teacher told me that he worked in a program where the number of trials per day was publicly posted. Now, in general, he thought this intervention had positive effects. For the most part, more well-delivered trials will promote more learning. But this intervention did produce a nasty side effect. Specifically, there was great resistance from the staff when programs were suggested that made it difficult to get a high rate of trials.
So in this program everyone was happy to work on reading that required frequent responses, math problems, and discrete trial instruction. But what about changes a child might need to make – that are absolutely critical- but don’t produce a high rate of trials? Mands for Information? Problem Solving? No one wants to work on those programs; their trial-rate graph will look terrible.
A cardinal rule that I try to follow is: If we implement this measurement, will it encourage everyone to engage in behaviors that are in the best interest of the client? Or ask, “Is there any way staff can show improvement in this measurement while doing something that is not in the client’s best interest?” If so, either don’t implement the measurement or add other procedures / measurements to attempt to prevent the negative effects.