Task vs output

[yanshen0]

Hi all,

I have some confusion about "task" vs "output": what exactly is the difference between them? How can there be single-output-multi-task models? I have read through the docs and some related papers, but still fail to understand the difference. Some concrete examples would be appreciated! Thanks!

[Balandat]

You should think of "output" purely from an API perspective: The output of the model is multi-dimensional. This doesn't say anything about how these outputs are modeled. For instance, if fit on training data with multiple trailing dimensions, SingleTaskGP produces a batched model in which the each of the dimensions (outputs) is modeled independently under the hood. It's basically the same functionality as if you fitted multiple models, one for each output dimension (though it will be faster b/c it exploits batching under the hood.

When we talk about "multi-task" models, we generally talk about modeling the correlation between the different tasks (e.g. via an ICM kernel). Such model cannot be split as in the example above.

Now, again from a BoTorch Model API perspective, you can have a multi-task model return just one of the modeled tasks (often called the "target task") as an output, even though internally the model needs to deal with all tasks. Or you could have it return all (or a subset of) tasks as outputs.