TensorFlow - coursera

Time Series & Prediction

Once we have a model and a period, then we can evaluate the model on it, and we'll need a metric to calculate their performance. So let's start simply by calculating the errors, which is the difference between the forecasted values from our model and the actual values over the evaluation period. The most common metric to evaluate the forecasting performance of a model is the mean squared error or mse where we square the errors and then calculate their mean. Well, why would we square it? Well, the reason for this is to get rid of negative values. So, for example, if our error was two above the value, then it will be two, but if it were two below the value, then it will be minus two. These errors could then effectively cancel each other out, which will be wrong because we have two errors and not none. But if we square the error of value before analyzing, then both of these errors would square to four, not canceling each other out and effectively being equal. And if we want the mean of our errors' calculation to be of the same scale as the original errors, then we just get its square root, giving us a root means squared error or rmse. Another common metric and one of my favorites is the mean absolute error or mae, and it's also called the main absolute deviation or mad. And in this case, instead of squaring to get rid of negatives, it just uses their absolute value. This does not penalize large errors as much as the mse does. Depending on your task, you may prefer the mae or the mse. For example, if large errors are potentially dangerous and they cost you much more than smaller errors, then you may prefer the mse. But if your gain or your loss is just proportional to the size of the error, then the mae may be better. Also, you can measure the mean absolute percentage error or mape, this is the mean ratio between the absolute error and the absolute value, this gives an idea of the size of the errors compared to the values. If we look at our data, we can measure the MAE using code like this. The keras metrics libraries include an MAE that can be called like this. With the synthetic data we showed earlier, we're getting about 5.93, let's consider that our baseline.