Opening the black box: Attempts to understand the results of machine learning models

Sophisticated machine learning models (like GBMs and Neural Networks) produce better predictions than simpler models (like linear or logistic regression), but sophisticated models do not produce interpretable 'effects' that specify the relationship between predictors and and outcome. This is because sophisticated models can learn non-linear, interactive, or even higher level relationships between the predictors and outcome without being explicitly specified. In many settings it is often important to understand, as best as possible, how 'black box' models are producing because:1. If users do not understand how a prediction is being made, they may not trust the model/prediction enough to act upon the model's suggestions. Significant business value can be derived from understanding what drives an outcome of interest (e.g. purchase or churn) in order to make product changes to accentuate or minimize desired effects 3. Understanding how predictors relate to an outcome can inform subsequent feature generation that can improve a model's predictive power. This talk will discuss two methods that have been proposed to better understand machine learning models: simulating changes in input variables (the R ICEbox package) or building a simpler model locally around specific predictions (the Python LIME package).