how to optimize discrete inputs

[taikichi-pc]

Thank you for a lot of helpful library for Bayesian Optimization.

def get_next_point(init_x, init_y, best_init_y, bounds, candidates_num = 1):
        mxsingle_gp = MixedSingleTaskGP(init_x, init_y, cat_dims=[2, 3])
        mll = ExactMarginalLogLikelihood(likelihood = mxsingle_gp.likelihood, model = mxsingle_gp)
        fit_gpytorch_model(mll)
        model = qExpectedImprovement(model = mxsingle_gp, best_f = best_init_y)
        optimize_acqf_mixed(
                         acq_function = model
                         bounds = bounds,
                         fixed_features_list = [{3: 1}, {3: 2}, {2: 1}, {2: 2}],
                         q = candidates_num,
                         num_restarts = 200,
                         raw_samples = 512,
                         options = {"batch_limit": 5, "maxiter": 200}
        return candidates

I give fixed_features_list = [{3: 1}, {3: 2}, {2: 1}, {2: 2}] in order to fix discrete features(indices are 2&3, 0&1 are continuous), but only indice[3] of candidate is fixed. Then indice[2] is float like continuous.

Like this, When I'd like discrete inputs(>=2 this case x2, x3) candidate, how should I use it?
Thanks for reading the beginner's article.

[Balandat]

So doing fixed_features_list = [{3: 1}, {3: 2}, {2: 1}, {2: 2}] will loop over the four dicts in the list, perform optimization with the features as specified in the respective dict fixed, and then pick the best value across all four dicts.

If you want to optimize the continuous features and fix the discrete ones, you should be doing sth like
fixed_features_list = [{2: 1, 3: 1}], which will rune one optimization with features 2 and 3 both fixed to 1. If you want multiple combinations of fixed features you can add additional dicts to the list.

[taikichi-pc]

Thank you for your reply! I. under stood.
I'd like to know little bit more detail.

feature 2 = (0 or 1) and feature 3 = (1 or 2). I want to get next point from these two all combination.
In above case, which one below should I use?
fixed_features_list = [{2: 0, 3: 1}, {2: 0, 3: 2}, {2: 1, 3: 1}, {2: 1, 3: 2}]
fixed_features_list = [{2: 0, 3: 1, 3: 2}, {2: 1, 3: 1, 3: 2}]
fixed_features_list = [{2: 0, 2: 1, 3: 1, 3: 2}]

I appreciate your quick reply.

[Balandat]

You’ll want to use the first of these options.

[taikichi-pc]

Thank you so much!
I'm sorry for the continuous questions.
At last , Could you reply below question about optimize_acqf_discrete()

It is another near answer box.

[taikichi-pc]

Last one. Thank you a lot!

I'd like to use this optim_acqf below,

def optimize_acqf_discrete(
    acq_function: AcquisitionFunction,
    q: int,
    choices: Tensor,
    max_batch_size: int = 2048,
    unique: bool = True,
    **kwargs: Any,
) -> Tuple[Tensor, Tensor]:

Only int(discrete) features, no continuous feature.
For example, the number of features is two(feature1 and feature2).
feature1 = (0 or 1) and feature2 = (1 or 2). I want to get next candidate point from these two all combination.
?? In this case, how should I use above function(mainly choices and unique)?
?? In this case How should I modify below code?

mxsingle_gp = MixedSingleTaskGP(init_x, init_y, cat_dims=[0, 1])
mll = ExactMarginalLogLikelihood(likelihood = mxsingle_gp.likelihood, model = mxsingle_gp)
fit_gpytorch_model(mll)

I'm sorry my poor English and
appreciate your kindness and cooperation.

[Balandat]

So unique just means that if q>1 this function returns different choices (which is generally desirable here).

choices just needs to be a tensor of possible choices to consider. So in your example with values {0,1} and {1, 2} for features 1 and 2, respectively, you'd generate a tensor like

choices = torch.tensor(list(itertools.product([(0, 1), (1, 2)])))

that contains all possible combinations