Model Monitoring Part 3: Detecting Drift in Model Structure, Model Coefficients, and Real-World Data
Previous discussions of model monitoring (Part 1 and Part 2) have focused more on user experience and application development. But the first and, in some ways, foremost consumers of model monitoring tools are the data scientists themselves. Data-driven models are highly dependent on the quality of the data given to them for training, as well as on the consistency of the data given to them in deployment. If the training data or the real-world test data changes in some material way, data scientists need to have the tools to detect those changes as soon as they might begin impacting the performance of those applications, or even before.
Machine learning models for artificial intelligence applications require constant retraining. The real-world data for deploying these models is a moving target. People change the way the interact with each other, computer systems, and the world at large. Language use changes subtly, with new words being introduced and new language patterns and word frequencies changing. The user base of an application will gradually change over time. For all of these reasons, models should not remain static, but should be retrained from time to time with new training data.
As models are retrained on new data, data scientists need to track whether their new models are similar enough to the previously deployed models to believe that the models will continue to work on the applications they have been performing well on to that point. The process for doing that analysis depends on what kind of models the data scientists are using.
If they are using statistical regression models, tracking the change in the coefficients and performing well-understood cross-validation tests should detect significant changes in those models. If the new models pass those tests, then all is good. But what should they do if they fail those tests? Data scientists need tools to dive into the changes in the new models, to understand what has changed in the regression coefficients, what features of the data are likely to have caused those changes, and if the users of those models should be concerned about the changes in the models or if they are unlikely to have a negative impact on the performance of the applications that use them.
In the case where data scientists are using deep learning techniques, or other non-linear learning methodologies that produce less humanly explainable models, the problem of tracking drift in models is even more acute. One can use cross-validation methodologies to detect changes in behavior of the new models versus the old models, by simply applying the new and old data sets to the new and old models to see if there are noticeable differences in key performance indicators, e.g. task completion, accuracy, recall, etc. But if differences are detected, it’s less clear how to identify the root cause of the change, much less to determine if the changes are a cause for concern. This is where good and focused model monitoring software is critical. The model monitoring software must be developed in a way which anticipates the learning techniques used in building the models, so it can be manipulated to understand the parts of the models that have changed, what features of the data are likely to have caused the changes, and whether the new models are likely to behave substantially differently in the real-world applications they are deployed. The less explainable the models are, the more difficult this task will be. Facile and nimble visualization tools are critical in these cases to allow data scientists to dive into the innards of the models they have built to understand the important features of these models, in order to diagnose whether the changes in the models are idiosyncratic and unimportant, or significantly impactful on performance.
One way to avoid exposure to model changes before they impact production performance is to have monitoring tools that act directly on the data itself. Data monitoring tools are critical for detecting changes in the real-world data that an application processes, to anticipate any future problems when retraining new models and to avoid having applications begin to fail because the world is changing more quickly than the training algorithms for the model can handle.
Monitoring data involves more than just tracking superficial statistics. Data monitoring software needs to have access to information about what features of the data are important for the performance of the source models. Without this input, the data monitoring is more likely to produce lots of false alarms as innocuous features of the data change over time, drowning out information about what changes in the data might be of greater concern.
For example, stock trading systems depend on many different features of price movement patterns, and there is a great amount of variability in trading patterns that are just part of the real world of financial markets. However, some trading systems can be sensitive to the way price movements are distributed over the course of the trading day. Those patterns can be impacted by when derivative markets open and close, and how trading changes on those derivative markets. For instance, US stock markets can be impacted by trading on stock index futures markets and stock options markets. In the past, those markets have changed their operating hours, their liquidity profiles, and the kinds of derivative products offered on their exchanges. The changes in those derivative markets can have a significant residual impact on the trading of the underlying stocks, which could cause dramatic changes in the markets that retraining algorithms might not be able to adapt to quickly enough to avoid negative impacts on application performance. Trading system data scientists need data monitoring tools which will be sensitive to detecting these kinds of systematic changes in real-world data so that they can be alerted to the need to intervene in the automated retraining of the models. How to react to these changes in the real world can be more art than science, but science is needed to detect the need to react.
Data science is fundamentally about data. Data scientists depend on deep understanding of the data that goes into those models in order for them to be effective at building useful models for solving real-world problems. Tracking the behaviors of the model is useful and important, but waiting for data changes to filter into those models is irresponsible if those changes materially change the performance of applications in ways that harm the customers or users of those applications. Data monitoring solutions can help data scientists protect the users of their models from data changes by alerting them to changes in the real-world data that might have an impact on models before the impact is painfully felt.
More News & Insights
