black box models

A black-box machine learning algorithm is one were the internal details are either not available or too complex to comprehend. A deep neural network is a good example. With potentially billions of weights and nodes, it is not easy to understand what is going on inside. In some cases it may be acceptable to simply treat the black-box model as an oracle, giving answers that are trusted, even if not understood. However, in other cases the decisions being made may be safety-critical or there may be issues of legal responsibility; in such cases, explainable AI is essential.