Blog: Finding signal in the noise - Why some forecasts improve and others don‘t
Humans have always tried to predict the future. During the hunter-gatherer era, people predicted things such as plant and food availability, weather, and animal movement. Today, with the rise of AI, prediction has entered a completely new dimension. We are no longer only making predictions ourselves but we are also teaching machines to make increasingly sophisticated predictions. Figure A. Popularity of the search term prediction in Google Trends With the rise of big data, one question becomes central for the art of prediction: How can we find the signal in this vast amount of data and use it to improve our predictions? As I was reading Nate Silver's book "The Signal and the Noise", I became more interested in this question. First, it would be good to define what a prediction is. A prediction can be defined as a specific, testable claim about a future observable outcome. A prediction usually has four important attributes: Target: What observable outcome are we predic...