Instabilities in a dynamic process refer to a tendency to move rapidly between very different states without settling into a stable state. It can occur in natural or artificial processes if there is positive feedback. It can also occur during the training phase of machine learning if the rate of training is too great meaning that each each training example pulls the system too far.
Used on Chap. 8: page 153