ARIMA forecast for timeseries is one step ahead. Why? (Solved Interview Problem)

Question from interviewer
 

I'm trying to forecast timeseries with ARIMA (Autoregressive Integrated Moving Average (ARIMA) model). As you can see from the plot, the forecast (red) is one step ahead of the expected values (input value colored in blue). How can I synchronize the two?



The code I used:


history = [x for x in train]
predictions = list()

for t in range(len(test)):
    model = ARIMA(history,order=(2, 2, 1))
    model_fit = model.fit(disp=0)
    output = model_fit.forecast(alpha=0.05)
    yhat = output[0]
    predictions.append(yhat)
    obs = test[t]
    history.append(obs)
    print('predicted=%f, expected=%f' % (yhat, obs))

rmse = sqrt(mean_squared_error(test, predictions))
print('Test RMSE: %.3f' % rmse)
# plot
plt.plot(test, color='blue')
plt.plot(predictions, color='red')
plt.show() 

Answer
It appears you are using Python's statsmodel ARIMA .

There are two options:

1) Change order=argument. The p, d, or q hyperparameters might be causing that result.
2) Manually lag the predictions: predictions = predictions[1:]

What you are seeing in the plot can be interpreted differently while considering the time dimension on x-axis with future being on the right and past on the left.

Your predicted curve (the one colored in red) is FOLLOWING your input (blue) curve. At time t, if blue goes down, then at time t+1, red will go down. You can see this behavior happening twice very clearly in the image once around t = 10 and then second time around t = 20. Also, the magnitude of change in the predicted curve is not as big/low as the input curve.

Tags: Technology,Machine Learning,