Page 51 - FINAL CFA II SLIDES JUNE 2019 DAY 3

LOS 9.g: Contrast in-sample and out-of-sample forecasts and READING 9: TIME SERIES ANALYSIS
compare the forecasting accuracy of different time-series
models based on the root mean squared error criterion.

In-sample forecasts are within the range of data (i.e., time/test period) used to estimate the model: How accurate is our model
in forecasting the actual data used to it? (See Predicted vs. Actual Capacity Utilization in our Trend analysis example).

Out-of-sample forecasts are made outside of the sample period: How accurate is our model in in forecasting the y variable
value for a time period outside the period used to develop the model? They help test relevance (i.e., predictive power) in the real
world. Most research employs in-sample forecasts only.

Root Mean Squared Error criterion (RMSE):
Compares the accuracy of autoregressive models in forecasting out-of-sample values.

• For both models (AR(1) and AR(2)), calculate the RMSE (the square root of the average of the squared errors) for the out-of-
sample data: the lower the better! Note that the model with the lowest RMSE (for in sample data) is not necessarily same for
out-of-sample data!
LOS 9.h: Explain the instability of coefficients of time-series models.

Instability or non-stationarity –tendency for dynamic financial and economic
conditions causing regression coefficients in one period to differ from those of
another period.
• Shorter time series more stable than longer time; and
• But shorter also means less reliable.
You must manage tradeoff between increased statistical reliability from longer time
series and the increased coefficient stability from shorter time series.

Key issues: Have there been regulatory changes? Any dramatic
change in the underlying economic environment?

If yes, historical data may not provide a reliable model. Merely examining the significance
of the autocorrelation of the residuals will not indicate whether the model is valid.

Also examine whether the data is covariance stationary!