I like seeing patterns across domains. Consider machine learning models and employees. Both models and people can be appraised.
What is an employee appraisal?
Employee “performance appraisal has three basic functions: (1) to provide adequate feedback to each person on his or her performance; (2) to serve as a basis for modifying or changing behavior toward more effective working habits; and (3) to provide data to managers with which they may judge future job assignments and compensation” (Levinson, 1976).
How is that like a machine learning evaluation?
All three apply to model performance. Model evaluation is one of the six tenets of the CRISP-DM framework. And the value of a model changes with time, hence, it is alive. See “concept drift” in Wikipedia or three reasons why models go out of sync (Pannu & Moore, 2017). Similarly, like employees, periodic checkups are good for models. The express goal is changing the model behavior and measuring usefulness for future assignment.
Humans are needed for employee appraisals, but maybe we can remove the personnel from the process of model evaluation. Consider automated machine learning (AutoML). “AutoML aims to automate the entire ML workflow” (Open Data Science, n.d.). Further, I suggest that machine learning can gain from general software development methods. For example, software methods that look to thrive on what used to be known as problems, for example, the Agile Manifesto or Antifragile Software (Kapalko, 2018).
The bottom line.
Appraisals are an important tool for your company. Disregarding appraisals is not the way because we know they have value. Consider a review of the appraisal process. A thoughtful appraisal can add value to the company and the people involved. Appraisals are a source of information.
Similarly, look for model evaluation. Consider checking the machine learning models that are aging. Further, look at the process of model review. As a result, company value can be increased. And improve the health for the people involved.
Let’s get appraising!
