physbo.search.discrete package

Module contents

class physbo.search.discrete.History

Bases: object

export_all_sequence_best_fx()

Export all fx and actions at each sequence.

(The total number of data is total_num_research.)

Returns:

• best_fx (numpy.ndarray)

• best_actions (numpy.ndarray)

export_sequence_best_fx()

Export fx and actions at each sequence. (The total number of data is num_runs.)

Returns:

• best_fx (numpy.ndarray)

• best_actions (numpy.ndarray)

export_valid()

Export the valid (successfully evaluated) observations.

Returns:

• actions (numpy.ndarray) – Indexes of the actions of the valid observations.

• fx (numpy.ndarray) – Objective values of the valid observations.

load(filename)

Load the information of the history.

Parameters:

filename (str) – The name of the file which stores the information of the history

save(filename)

Save the information of the history.

Parameters:

filename (str) – The name of the file which stores the information of the history

show_search_results(N)

property time_get_action

property time_run_simulator

property time_total

property time_update_predictor

property valid_mask

Mask of valid observations (True) vs failed ones (False).

An observation is failed when its objective value is not finite (NaN or +-Inf). Failed observations are kept in the history but excluded from the training data and the best-value tracking.

Return type:

numpy.ndarray of bool, shape (total_num_search,)

write(t, action, time_total=None, time_update_predictor=None, time_get_action=None, time_run_simulator=None)

Overwrite fx and chosen_actions by t and action.

Parameters:

• t (numpy.ndarray) – N dimensional array. The negative energy of each search candidate (value of the objective function to be optimized).

• action (numpy.ndarray) – N dimensional array. The indexes of actions of each search candidate.

• time_total (numpy.ndarray) – N dimenstional array. The total elapsed time in each step. If None (default), filled by 0.0.

• time_update_predictor (numpy.ndarray) – N dimenstional array. The elapsed time for updating predictor (e.g., learning hyperparemters) in each step. If None (default), filled by 0.0.

• time_get_action (numpy.ndarray) – N dimenstional array. The elapsed time for getting next action in each step. If None (default), filled by 0.0.

• time_run_simulator (numpy.ndarray) – N dimenstional array. The elapsed time for running the simulator in each step. If None (default), filled by 0.0.

class physbo.search.discrete.Policy(test_X, config=None, initial_data=None, comm=None, rng=None)

Bases: CheckpointMixin

Single objective Bayesian optimization with discrete search space

Parameters:

• test_X (numpy.ndarray or physbo.Variable) – The set of candidates. Each row vector represents the feature vector of each search candidate.

• config (SetConfig object (physbo.misc.SetConfig))

• initial_data (tuple[np.ndarray, np.ndarray]) – The initial training datasets. The first elements is the array of actions and the second is the array of value of objective functions

• comm (MPI.Comm, optional) – MPI Communicator

• rng (None, "legacy", int, or numpy.random.Generator, optional) – Random number generator specification (default: “legacy”). None or “legacy” uses the global numpy.random state and is bit-compatible with the historical behavior. An int seed or a Generator switches to a policy-owned generator whose state is stored on the policy (and thus included when the policy is pickled). Under MPI, pass the same specification on all ranks.

actions: ndarray

Array of action indices that have not been searched yet.

Initially contains all candidate indices, and decreases as actions are evaluated.

bayes_search(training=None, max_num_probes=None, num_search_each_probe=1, predictor=None, is_disp=True, simulator=None, score='TS', interval=0, num_rand_basis=0, optimizer=None, unify_method=None, ard=False, ucb_beta=1.0)

Performing Bayesian optimization.

Parameters:

• training (physbo.Variable) – Training dataset.

• max_num_probes (int) – Maximum number of searching process by Bayesian optimization.

• num_search_each_probe (int) – Number of searching by Bayesian optimization at each process.

• predictor (predictor object) – Base class is defined in physbo.predictor. If None, blm_predictor is defined.

• is_disp (bool) – If true, process messages are outputted.

• simulator (callable) – Callable (function or object with __call__) Here, action is an integer which represents the index of the candidate.

• score (str or callable) – TS, EI, PI, UCB, or a callable called positionally as score(predictor, training, test) returning a numpy.ndarray of shape (test.X.shape[0],), with larger values preferred. Empty test data skips the callable. Under MPI, the callable evaluates local points and must not perform collective communication.

• interval (int) – The interval number of learning the hyper parameter. If you set the negative value to interval, the hyper parameter learning is not performed. If you set zero to interval, the hyper parameter learning is performed only at the first step.

• num_rand_basis (int) – The number of basis function. If you choose 0, ordinary Gaussian process run.

• optimizer (optimizer object, optional) – This is for compatibility with the range-based Policies. This is not used.

• ard (bool) – If True, use Automatic Relevance Determination (ARD) for the Gaussian kernel. Default is False.

• ucb_beta (float) – Trade-off parameter between exploitation and exploration used when score == "UCB". The score is fmean + ucb_beta * fstd; larger values favor exploration. Default is 1.0.

Returns:

history

Return type:

history object (physbo.search.discrete.results.history)

config: SetConfig

Configuration object containing settings for learning and search.

export_history()

Returning the information of the history.

export_predictor()

Returning the predictor dataset

export_training()

Returning the training dataset

get_kernel_length_scale()

Return the Gaussian kernel length scale(s) (width) of the predictor.

With ARD, returns one length scale per input dimension; otherwise a single value.

Returns:

Length scale(s). Shape (num_dim,) when ARD is used, (1,) otherwise. None if the predictor is not set or not a GP with Gaussian kernel.

Return type:

numpy.ndarray or None

get_num_dim()

Return the input dimension (number of features) of the search space.

Returns:

The number of dimensions. None if neither training nor test data is set.

Return type:

int or None

get_permutation_importance(n_perm: int, split_features_parallel=False)

Calculating permutation importance of model

Parameters:

• n_perm (int) – number of permutations

• split_features_parallel (bool) – If true, split features in parallel.

Returns:

• importance_mean (numpy.ndarray) – importance_mean

• importance_std (numpy.ndarray) – importance_std

get_post_fcov(xs, diag=True)

Calculate covariance of predictor (post distribution)

Parameters:

• xs (physbo.Variable or np.ndarray) – input parameters to calculate covariance shape is (num_points, num_parameters)

• diag (bool) – If true, only variances (diagonal elements) are returned.

Returns:

fcov – Covariance matrix of the post distribution. Returned shape is (num_points) if diag=true, (num_points, num_points) if diag=false.

Return type:

numpy.ndarray

get_post_fmean(xs)

Calculate mean value of predictor (post distribution)

Parameters:

xs (physbo.Variable or np.ndarray) – input parameters to calculate mean value shape is (num_points, num_parameters)

Returns:

fmean – Mean value of the post distribution. Returned shape is (num_points).

Return type:

numpy.ndarray

get_score(mode, *, actions=None, xs=None, predictor=None, training=None, parallel=True, alpha=1, ucb_beta=1.0)

Calcualte score (acquisition function)

Parameters:

• mode (str or callable) – TS, EI, PI, UCB, or a callable called positionally as mode(predictor, training, test) returning a numpy.ndarray of shape (test.X.shape[0],), with larger values preferred. Empty test data skips the callable but still participates in MPI gathering. The callable must not perform collective communication. alpha and ucb_beta apply only to string modes.

• actions (array of int) – actions to calculate score

• xs (physbo.Variable or np.ndarray) – input parameters to calculate score

• predictor (predictor object) – predictor used to calculate score. If not given, self.predictor will be used.

• training (physbo.Variable) – Training dataset. If not given, self.training will be used.

• parallel (bool) – Calculate scores in parallel by MPI (default: True)

• alpha (float) – Tuning parameter which is used if mode = TS. In TS, multi variation is tuned as np.random.multivariate_normal(mean, cov*alpha**2, size).

• ucb_beta (float) – Trade-off parameter between exploitation and exploration used if mode = UCB. The score is fmean + ucb_beta * fstd. Default is 1.0.

Returns:

f – Score defined in each mode.

Return type:

float or list of float

Raises:

RuntimeError – If both actions and xs are given

Notes

When neither actions nor xs are given, scores for actions not yet searched will be calculated.

When parallel is True, it is assumed that the function receives the same input (actions or xs) for all the ranks. If you want to split the input array itself, set parallel be False and merge results by yourself.

history: History

History object storing search results including chosen actions,

objective function values, and timing information.

load(file_history, file_training=None, file_predictor=None)

Loading files about history, training and predictor.

See save for the difference from the checkpoint API (save_checkpoint / load_checkpoint).

Parameters:

• file_history (str) – The name of the file that stores the information of the history.

• file_training (str) – The name of the file that stores the training dataset.

• file_predictor (str) – The name of the file that stores the predictor dataset.

mpicomm: Any | None

MPI communicator for parallel computation.

If None, MPI is not used (single process).

mpirank: int

Rank of the current MPI process. Set to 0 if MPI is not used.

mpisize: int

Number of MPI processes. Set to 1 if MPI is not used.

new_data: Variable | None

New data that has been added to training but not yet to predictor.

Set to None after predictor is updated.

predictor: Any | None

Predictor used for Bayesian optimization.

Base class is defined in physbo.predictor. If None, predictor is not yet initialized.

random_search(max_num_probes, num_search_each_probe=1, simulator=None, is_disp=True)

Performing random search.

Parameters:

• max_num_probes (int) – Maximum number of random search process.

• num_search_each_probe (int) – Number of search at each random search process.

• simulator (callable) – Callable (function or object with __call__) from action to t Here, action is an integer which represents the index of the candidate.

• is_disp (bool) – If true, process messages are outputted.

Returns:

history

Return type:

history object (physbo.search.discrete.results.history)

save(file_history, file_training=None, file_predictor=None)

Saving history, training and predictor into the corresponding files.

This persists the search results in a portable form (the files can be loaded by runs with a different number of MPI processes). To suspend and later resume an execution bit-exactly (including the random number generator state), use save_checkpoint / load_checkpoint instead.

Parameters:

• file_history (str) – The name of the file that stores the information of the history.

• file_training (str) – The name of the file that stores the training dataset.

• file_predictor (str) – The name of the file that stores the predictor dataset.

seed: int | None

Seed parameter for np.random. Set by set_seed() method.

set_seed(seed)

Setting a seed parameter.

In the legacy RNG mode (default), this seeds the global numpy.random state, as before. In the Generator mode (rng= given as int or Generator), the policy-owned generator is re-created from the seed and the global state is not touched.

Parameters:

• seed (int) – seed number

• -------

test: Variable

The set of candidate search points.

Each row of X represents the feature vector of each search candidate.

training: Variable

Training dataset containing pairs of input (X) and output (t).

Stores all evaluated data points.

write(action, t, X=None, time_total=None, time_update_predictor=None, time_get_action=None, time_run_simulator=None)

Writing history (update history, not output to a file).

Parameters:

• action (numpy.ndarray) – Indexes of actions.

• t (numpy.ndarray) – N dimensional array (1D) or N x k dimensional array (2D). The negative energy of each search candidate (value of the objective function to be optimized). Will be normalized to (N, 1) shape internally. A non-finite value (NaN or +-Inf) marks a failed evaluation: the action is recorded in the history and consumed, but the observation is excluded from the training data and the best-value tracking.

• X (numpy.ndarray) – N x d dimensional matrix. Each row of X denotes the d-dimensional feature vector of each search candidate.

• time_total (numpy.ndarray) – N dimenstional array. The total elapsed time in each step. If None (default), filled by 0.0.

• time_update_predictor (numpy.ndarray) – N dimenstional array. The elapsed time for updating predictor (e.g., learning hyperparemters) in each step. If None (default), filled by 0.0.

• time_get_action (numpy.ndarray) – N dimenstional array. The elapsed time for getting next action in each step. If None (default), filled by 0.0.

• time_run_simulator (numpy.ndarray) – N dimenstional array. The elapsed time for running the simulator in each step. If None (default), filled by 0.0.

Note

In this function, (X, t, Z) is added to the training data set (self.training), but not yet to predictor (self.predictor). self.newdata means such data. To add (X, t, Z) to the predictor, call self._update_predictor() after self.write().