In a new paper, a trio of Columbia University researchers propose a novel framework and hierarchical predictive model that learns to identify what is predictable from unlabelled video.
The paper Learning the Predictability of the Future introduces a hierarchical predictive model for learning what is predictable from unlabelled video. Inspired by the observation that people often organize actions hierarchically, the researchers designed the approach to jointly learn a hierarchy of actions from unlabelled video while also learning to anticipate them at the right level of abstraction. The model thus will predict a future action at the concrete level of the hierarchy when it is confident, and, when it lacks confidence, will select a higher level of abstraction to improve confidence.
Here is a quick read: Columbia University Model Learns Predictability From Unlabelled Video