Switching coefficients or automatic variable selection: an application in forecasting commodity returns

In this paper, we conduct a thorough investigation of the predictive ability of forward and backward stepwise regressions and hidden Markov models for the futures returns of several commodities. The predictive performance relative a standard AR(1) benchmark is assessed under both statistical and economic loss functions. We find that the evidence that either stepwise regressions or hidden Markov models may outperform the benchmark under standard statistical loss functions is rather weak and limited to low-volatility regimes. However, a mean-variance investor that adopts flexible forecasting models (especially stepwise predictive regressions) when building her portfolio, achieves large benefits in terms of realized Sharpe ratios and mean-variance utility compared to an investor employing AR(1) forecasts.