Combining kernel functions together

[ovr4]

🚀 Feature Request

I’d like to create a covariance module that combines two or more kernel functions together (the resulting kernel function would be a matrix addition involving all the kernel functions)?
Example: Covariance Module = Matern5(x, x’) + RBF(x, x’) + IndexKernel(x, x’) (where each kernel handles different features of x)
In this scenario, the hyperparameters registered for each kernel function would be optimized together as they now contribute to one combined kernel function. I know it isn’t a popular approach to have multiple kernel functions describing the same data, but I’d love to hear opinions about how to implement something like this.

Motivation

Is your feature request related to a problem? Please describe.

The motivation behind this is that some features in my training data are categorical of type string while some are categorical of type float. We’d like to describe the string types using the indexkernel and the float types using a kernel like Matern or RBF.

Pitch

Describe the solution you'd like

Example: Covariance Module = Matern5(x, x’) + RBF(x, x’) + IndexKernel(x, x’) (where each kernel handles different features of x)
Describe alternatives you've considered

I haven’t seen any alternatives in the repo, so I was wondering if you’d be able to implement a kernel combination function. Or point me in the direction of already available alternatives.

Are you willing to open a pull request? (See CONTRIBUTING)

Additional context

[wjmaddox]

This can be done natively in gpytorch, so an example that's integrated in Botorch is

from botorch.models import SingleTaskGP
from gpytorch.kernels import MaternKernel, RBFKernel, IndexKernel

train_x = torch.randn(40, 6)
train_x[train_x[:,-1] >0] = 1
train_x[train_x[:,-1] < 0] = 0
# the active dims argument sets the dimensions used for each kernel
# Edit: note that the IndexKernel requires a num_tasks argument -- we have two tasks here
covar_module = MaternKernel(active_dims=[0]) + RBFKernel(active_dims=[1,2,3,4]) + IndexKernel(num_tasks=2, active_dims=5)

train_y = torch.randn(40, 1) # some data

model = SingleTaskGP(train_x, train_y, covar_module=covar_module)

[ovr4]

Thanks for the quick response. I'll try this out. Thanks!