Allow xreg and newxreg to take different values for each series in forecast.gts()

[robjhyndman]

Currently, the following code will work:

htseg2x <- matrix(rnorm(16*17),nrow=16,ncol=17)
htseg2nx <- matrix(rnorm(10*17),nrow=10,ncol=17)
forecast(htseg2 , h=10, fmethod="arima", xreg=rnorm(16), newxreg=rnorm(10))

But this only allows the same xreg and newxreg vectors to be applied to all time series. We need to allow a matrix to be passed for each argument.

[earowang]

a matrix can be passed to xreg and newxreg now

htseg2x <- matrix(rnorm(16*17),nrow=16,ncol=17)
htseg2nx <- matrix(rnorm(10*17),nrow=10,ncol=17)
forecast(htseg2 , h=10, fmethod="arima", xreg=htseg2x, newxreg = htseg2nx)

[robjhyndman]

This is trickier than I thought. There are several possibilities:

1. The same vector regressor is applied to all series.
2. A different vector regression is applied to each series.
3. The same matrix regressor is applied to all series.
4. A different matrix regressor is applied to each series.

The old code handled 1. The new code handles 1 & 2. It should be easy to
handle 3, but if the number of regressors is the same as the number of
series, then it will not be obvious whether 2 or 3 is required. For 4, we
would need an array argument.

Any ideas how to make the interface simple while handling all 4 cases?

---

Rob J Hyndman
www.robjhyndman.com

On Thu, Jul 17, 2014 at 5:05 PM, Earo Wang [email protected] wrote:

Closed #6 #6.

—
Reply to this email directly or view it on GitHub
#6 (comment).

[dashaub]

Could we use lists to handle the case where a different matrix or vector should be applied to each level? A list of length equal to the number of levels could contain a vector or matrix to apply to the series in the same column position in the hts/gts. If the xreg/newxreg is a vector or matrix, that vector/matrix would be applied to every level of the series.

[ghost]

Hi,
Is there a workaround for this issue? If yes, could you please point me to it?

I am trying to forecast demand across a geographical hierarchy (Cities, States, Countries). I am using external regressors like temperature, humidity etc. which assume different values for different cities. Hence, I need to pass a matrix of external regressors for each bottom node of the hierarchy.

Also, how can we aggregate external regressors from one hierarchy level to another (eg. Cities to States)? Sometimes we would need to take their sum, and sometimes average.

I apologize if this is the wrong forum to ask this question. If a similar topic exists in another forum, please point me to it.

Any help would be appreciated.
Thanks

[gabrielcaceres]

Maybe a new optional argument could be added to indicate what series each regressor corresponds to, and if not given, xreg is applied to all series.

I'm thinking along the lines of how the gts object is created. Either a list with entries for each regressor and the series/groups it corresponds to, or an indicator matrix of size n-series x m-regressors.

That way xreg would just be a single large matrix, and specific columns can be assigned to any given (and possibly multiple) series, including at different levels like in the question above.

If, say, using an indicator matrix, 1 and 3 would be the default when NULL, 2 would just be a diagonal matrix, and a denser one could address 4.

[amarchin]

I am using the hts package for a project and I thought that if I provided a matrix regressor I would be in case 3. So Am I using the package incorrectly?

[123saga]

@robjhyndman : thanks for amazing package, Can anyone help me on how to pass "different matrix regressor is applied to each series." ? or is this still not supported with the current version of the package ?
Thanks a ton !

[robjhyndman]

You currently have to create your own forecasts in a loop and then use combinef()

[123saga]

@robjhyndman , thanks for your response, I will try the following steps, please let me know if I am missing anything.

1. I will need to create series at the least granular level and create a gmatrix accordingly.

2. But the forecast will be made at individual series for all levels using respective xreg and newxreg matrix in for loop.

3. Create fcasts matrix and use combinef() to get optimal forecasts at all levels.

Thanks,
Sagar

[123saga]

Hi @robjhyndman ,

I have tried combinef() as directed.
Generated the individual forecasts and executed the following.
combinef(as.matrix(REV_FORECAST),keep="gts",groups=gps,algorithms = "lu")
and got this error.
Error: Argument fcasts requires all the forecasts.

I believe I have all the forecast avaialble as per the group object that i created. Could you please let me how can i fix this issue.

dim(gps)
[1] 3 8764
dim(REV_FORECAST)
[1] 8 8764

Thanks,
Sagar

[robjhyndman]

See the example for combinef(). Your groups matrix should not have the same number of columns as the forecast matrix.

[123saga]

@robjhyndman,

I generated forecasts separately at lowest level granularity. do i need to manually create forecasts for interactions between groups as well ?

• For example:
  I have 7 unique values in group1, 1252 in group2 and created 8764 (1252*7) forecasts
  and now trying group them using combinef().

with your comment, I understand that I need to create additional forecasts

• 1 at top level (aggregating all the series together)
• 1252 (at group 2 level) &
• 7(at group 1 level)).

But when I execute forecast using:
gps <- rbind( rep(1:1252,each=7), # group 1 rep(1:7,each=1252), # group 2 rep(1:(1252*7),each=1) # group 1 X group 2 )

Followed by :
gy <- gts(y, groups=gps) and
print gy

Grouped Time Series 5 Levels Number of groups at each level: 1 1252 7 1252 8764 Total number of series: 11276 Number of observations per series: 28

1 (top level) 1252 (aggregate by group1) 7 (aggregate by group2) 1252 (what are these series? ) 8764(series at combination of group1 and 2 )

At what level I should generate the other 1252 forecast and in what order should arrange before passing the forecast matrix to combinef() .

Sorry If I missing anything here, thanks a lot for your time.

[123saga]

Can anyone provide me insight on how to generate individual forecasts and use by combinef() to get optimal forecast considering there groups present in data.

[123saga]

@robjhyndman,
I figured out the issue, I created groups object that does not align with my setup. Thank you !

[lirilkumar]

This is trickier than I thought. There are several possibilities:

1. The same vector regressor is applied to all series.
2. A different vector regression is applied to each series.
3. The same matrix regressor is applied to all series.
4. A different matrix regressor is applied to each series.

The old code handled 1. The new code handles 1 & 2. It should be easy to
handle 3, but if the number of regressors is the same as the number of
series, then it will not be obvious whether 2 or 3 is required. For 4, we
would need an array argument.

Any ideas how to make the interface simple while handling all 4 cases?

Rob J Hyndman
www.robjhyndman.com

On Thu, Jul 17, 2014 at 5:05 PM, Earo Wang [email protected] wrote:

Closed #6 #6.
—
Reply to this email directly or view it on GitHub
#6 (comment).

Hi There,
Thanks for Amazing package.
any updates on 4th point ?