この jupyter notebook ファイルは ISSP Data Repository (develop branch) から入手できます。
多目的最適化
最適化したい目的関数が複数 (\(p\) 個) ある場合は、多目的最適化をおこないます。
本チュートリアルでは、「解」は目的関数の組 \(y = (y_1(x), y_2(x), \dots, y_p(x))\) を意味することに注意してください。
解の大小関係 \(\prec\) を以下のように定義します。
\(y \prec y^{'}\Longleftrightarrow \forall \ i \le p, y_i \le y^{'}_i \land \exists \ j \le p, y_j < y^{'}_j\)
（最大化問題における） パレート解 とは、上記の大小関係の上で、自身よりも大きな解がないような解を指します。
すなわち、任意の目的関数の値を改善しようとした場合、他の目的関数のうちどれかひとつは悪化するような解です。
目的関数間にトレードオフが存在する場合は、パレート解は複数存在するため、それらを効率的に求めることが課題となります。
PHYSBOでは、パレート解を効率的に求めるためのベイズ最適化手法を実装しています。
[1]:
import numpy as np
import matplotlib.pyplot as plt
import physbo
%matplotlib inline
テスト関数
本チュートリアルでは、多目的最適化のベンチマーク関数である VLMOP2 を利用します。
目的関数の数は2つです。
\[\begin{split}\begin{split}
y_1(\vec{x}) &= 1 - \exp\left[-\sum_{i=1}^N\left( x_i - 1/\sqrt{N}\right)^2\right] \\
y_2(\vec{x}) &= 1 - \exp\left[-\sum_{i=1}^N\left( x_i + 1/\sqrt{N}\right)^2\right]
\end{split}\end{split}\]
\(y_1\) と \(y_2\) はそれぞれ \(x_1 = x_2 = \cdots x_N = 1/\sqrt{N}\) と \(x_1 = x_2 = \cdots x_N = -1/\sqrt{N}\) に最小値があり、その値はともに０です。また、上界は1 です。
PHYSBO は最大化問題を仮定するため、-1を掛けたものをあらためて目的関数とします。
参考文献
Van Veldhuizen, David A. Multiobjective evolutionary algorithms: classifications, analyses, and new innovations. No. AFIT/DS/ENG/99-01. AIR FORCE INST OF TECH WRIGHT-PATTERSONAFB OH SCHOOL OF ENGINEERING, 1999.
[2]:
def vlmop2_minus(x):
n = x.shape[1]
y1 = 1 - np.exp(-1 * np.sum((x - 1/np.sqrt(n)) ** 2, axis = 1))
y2 = 1 - np.exp(-1 * np.sum((x + 1/np.sqrt(n)) ** 2, axis = 1))
return np.c_[-y1, -y2]
探索候補データの準備
入力空間 \(\vec{x}\) は 2次元とし、[-2, 2] × [-2, 2] の上で候補点をグリッド状に生成します。
[3]:
import itertools
a = np.linspace(-2,2,101)
test_X = np.array(list(itertools.product(a, a)))
[4]:
test_X
[4]:
array([[-2. , -2. ],
[-2. , -1.96],
[-2. , -1.92],
...,
[ 2. , 1.92],
[ 2. , 1.96],
[ 2. , 2. ]], shape=(10201, 2))
[5]:
test_X.shape
[5]:
(10201, 2)
simulator の定義
[6]:
class Simulator(object):
def __init__(self, X):
self.t = vlmop2_minus(X)
def __call__( self, action):
return self.t[action]
[7]:
simu = Simulator(test_X)
関数のプロット
2つの目的関数をそれぞれプロットしてみましょう。
1つ目の目的関数は右上にピークがあり、2つ目の目的関数は左下にピークがあるようなトレードオフがある状態となっています。 （星はピークの位置です）
1つ目の目的関数
[8]:
plt.figure()
plt.imshow(simu.t[:,0].reshape((101,101)), vmin=-1.0, vmax=0.0, origin="lower", extent=[-2.0, 2.0, -2.0, 2.0])
plt.title("objective 1")
plt.colorbar()
plt.plot([1.0/np.sqrt(2.0)], [1.0/np.sqrt(2.0)], '*')
plt.show()
2つ目の目的関数
[9]:
# plot objective 2
plt.figure()
plt.imshow(simu.t[:,1].reshape((101,101)), vmin=-1.0, vmax=0.0, origin="lower", extent=[-2.0, 2.0, -2.0, 2.0])
plt.title("objective 2")
plt.colorbar()
plt.plot([-1.0/np.sqrt(2.0)], [-1.0/np.sqrt(2.0)], '*')
plt.show()
最適化の実行
Policy のセット
多目的最適化用の
physbo.search.discrete_multi.Policy を利用します。num_objectives に目的関数の数を指定してください。[10]:
policy = physbo.search.discrete_multi.Policy(test_X=test_X, num_objectives=2)
policy.set_seed(0)
通常の
physbo.search.discrete.policy (目的関数が1つの場合）と同じく、random_search または bayes_search メソッドを呼ぶことで最適化を行います。基本的なAPIや利用方法は
discrete.policy とおおよそ共通しています。ランダムサーチ
[11]:
policy = physbo.search.discrete_multi.Policy(test_X=test_X, num_objectives=2)
policy.set_seed(0)
res_random = policy.random_search(max_num_probes=50, simulator=simu)
0001-th step: f(x) = [-0.99973003 -0.62334035] (action = 836)
Pareto set updated.
the number of Pareto frontiers = 1
0002-th step: f(x) = [-0.99789981 -0.99866427] (action = 9404)
Pareto set updated.
the number of Pareto frontiers = 2
0003-th step: f(x) = [-0.99090897 -0.46609239] (action = 4664)
Pareto set updated.
the number of Pareto frontiers = 1
0004-th step: f(x) = [-0.92633083 -0.29208351] (action = 4780)
Pareto set updated.
the number of Pareto frontiers = 1
0005-th step: f(x) = [-0.67969326 -0.99981691] (action = 9566)
Pareto set updated.
the number of Pareto frontiers = 2
0006-th step: f(x) = [-0.45601619 -0.99848443] (action = 8852)
Pareto set updated.
the number of Pareto frontiers = 2
0007-th step: f(x) = [-0.92670204 -0.71508873] (action = 6088)
0008-th step: f(x) = [-0.58233995 -0.99952931] (action = 7060)
0009-th step: f(x) = [-0.99848229 -0.96780195] (action = 473)
0010-th step: f(x) = [-0.80479332 -0.99994946] (action = 7573)
0011-th step: f(x) = [-0.99700074 -0.99847873] (action = 1406)
0012-th step: f(x) = [-0.9992592 -0.93891121] (action = 6061)
0013-th step: f(x) = [-0.19963873 -0.93357674] (action = 5722)
Pareto set updated.
the number of Pareto frontiers = 2
0014-th step: f(x) = [-0.98046765 -0.99294428] (action = 2309)
0015-th step: f(x) = [-0.99602549 -0.98620358] (action = 7989)
0016-th step: f(x) = [-0.99957128 -0.9973798 ] (action = 8484)
0017-th step: f(x) = [-0.52191048 -0.72845916] (action = 5405)
Pareto set updated.
the number of Pareto frontiers = 3
0018-th step: f(x) = [-0.99916441 -0.40869572] (action = 2742)
0019-th step: f(x) = [-0.99480122 -0.7565076 ] (action = 1266)
0020-th step: f(x) = [-0.63329589 -0.63329589] (action = 5200)
Pareto set updated.
the number of Pareto frontiers = 4
0021-th step: f(x) = [-0.95437918 -0.80142908] (action = 2487)
0022-th step: f(x) = [-0.99899466 -0.96646532] (action = 269)
0023-th step: f(x) = [-0.19473522 -0.99445365] (action = 8044)
Pareto set updated.
the number of Pareto frontiers = 5
0024-th step: f(x) = [-0.99969529 -0.52395588] (action = 2135)
0025-th step: f(x) = [-0.59106078 -0.79258035] (action = 6206)
0026-th step: f(x) = [-0.78231041 -0.99997141] (action = 9079)
0027-th step: f(x) = [-0.99955573 -0.99930147] (action = 396)
0028-th step: f(x) = [-0.99825097 -0.99875436] (action = 9403)
0029-th step: f(x) = [-0.65387719 -0.99938669] (action = 9456)
0030-th step: f(x) = [-0.26132949 -0.87913689] (action = 5816)
Pareto set updated.
the number of Pareto frontiers = 6
0031-th step: f(x) = [-0.74523952 -0.7724917 ] (action = 6301)
0032-th step: f(x) = [-0.33644513 -0.97772424] (action = 5530)
0033-th step: f(x) = [-0.99777557 -0.93373833] (action = 6470)
0034-th step: f(x) = [-0.9725524 -0.6296562] (action = 2277)
0035-th step: f(x) = [-0.99924826 -0.84674905] (action = 353)
0036-th step: f(x) = [-0.99743705 -0.17848438] (action = 2549)
Pareto set updated.
the number of Pareto frontiers = 7
0037-th step: f(x) = [-0.99994916 -0.78013199] (action = 326)
0038-th step: f(x) = [-0.99887174 -0.86936149] (action = 5458)
0039-th step: f(x) = [-0.99362699 -0.99769786] (action = 9309)
0040-th step: f(x) = [-0.84889217 -0.99973225] (action = 10156)
0041-th step: f(x) = [-0.95713719 -0.09067194] (action = 4073)
Pareto set updated.
the number of Pareto frontiers = 7
0042-th step: f(x) = [-0.17190645 -0.91382463] (action = 6120)
Pareto set updated.
the number of Pareto frontiers = 6
0043-th step: f(x) = [-0.98406208 -0.94467587] (action = 7289)
0044-th step: f(x) = [-0.93974303 -0.92444262] (action = 2698)
0045-th step: f(x) = [-0.98414342 -0.84762781] (action = 1780)
0046-th step: f(x) = [-0.99699986 -0.98166426] (action = 7684)
0047-th step: f(x) = [-0.3868143 -0.99930896] (action = 8060)
0048-th step: f(x) = [-0.97964062 -0.65554315] (action = 2075)
0049-th step: f(x) = [-0.99907307 -0.73466786] (action = 4450)
0050-th step: f(x) = [-0.99720717 -0.85352507] (action = 865)
目的関数の評価値(の array) とそのときの action ID が表示されます。
また、パレート解集合 (Pareto set) が更新されたときにメッセージを表示します。
Pareto set が更新された際に中身を表示したい場合は、
disp_pareto_set=True と指定します。Pareto set は1つ目の目的関数値の昇順でソートされています。
[12]:
policy = physbo.search.discrete_multi.Policy(test_X=test_X, num_objectives=2)
policy.set_seed(0)
res_random = policy.random_search(max_num_probes=50, simulator=simu, disp_pareto_set=True)
0001-th step: f(x) = [-0.99973003 -0.62334035] (action = 836)
Pareto set updated.
current Pareto set = [[-0.99973003 -0.62334035]] (steps = [1])
0002-th step: f(x) = [-0.99789981 -0.99866427] (action = 9404)
Pareto set updated.
current Pareto set = [[-0.99973003 -0.62334035]
[-0.99789981 -0.99866427]] (steps = [1 2])
0003-th step: f(x) = [-0.99090897 -0.46609239] (action = 4664)
Pareto set updated.
current Pareto set = [[-0.99090897 -0.46609239]] (steps = [3])
0004-th step: f(x) = [-0.92633083 -0.29208351] (action = 4780)
Pareto set updated.
current Pareto set = [[-0.92633083 -0.29208351]] (steps = [4])
0005-th step: f(x) = [-0.67969326 -0.99981691] (action = 9566)
Pareto set updated.
current Pareto set = [[-0.92633083 -0.29208351]
[-0.67969326 -0.99981691]] (steps = [4 5])
0006-th step: f(x) = [-0.45601619 -0.99848443] (action = 8852)
Pareto set updated.
current Pareto set = [[-0.92633083 -0.29208351]
[-0.45601619 -0.99848443]] (steps = [4 6])
0007-th step: f(x) = [-0.92670204 -0.71508873] (action = 6088)
0008-th step: f(x) = [-0.58233995 -0.99952931] (action = 7060)
0009-th step: f(x) = [-0.99848229 -0.96780195] (action = 473)
0010-th step: f(x) = [-0.80479332 -0.99994946] (action = 7573)
0011-th step: f(x) = [-0.99700074 -0.99847873] (action = 1406)
0012-th step: f(x) = [-0.9992592 -0.93891121] (action = 6061)
0013-th step: f(x) = [-0.19963873 -0.93357674] (action = 5722)
Pareto set updated.
current Pareto set = [[-0.92633083 -0.29208351]
[-0.19963873 -0.93357674]] (steps = [ 4 13])
0014-th step: f(x) = [-0.98046765 -0.99294428] (action = 2309)
0015-th step: f(x) = [-0.99602549 -0.98620358] (action = 7989)
0016-th step: f(x) = [-0.99957128 -0.9973798 ] (action = 8484)
0017-th step: f(x) = [-0.52191048 -0.72845916] (action = 5405)
Pareto set updated.
current Pareto set = [[-0.92633083 -0.29208351]
[-0.52191048 -0.72845916]
[-0.19963873 -0.93357674]] (steps = [ 4 17 13])
0018-th step: f(x) = [-0.99916441 -0.40869572] (action = 2742)
0019-th step: f(x) = [-0.99480122 -0.7565076 ] (action = 1266)
0020-th step: f(x) = [-0.63329589 -0.63329589] (action = 5200)
Pareto set updated.
current Pareto set = [[-0.92633083 -0.29208351]
[-0.63329589 -0.63329589]
[-0.52191048 -0.72845916]
[-0.19963873 -0.93357674]] (steps = [ 4 20 17 13])
0021-th step: f(x) = [-0.95437918 -0.80142908] (action = 2487)
0022-th step: f(x) = [-0.99899466 -0.96646532] (action = 269)
0023-th step: f(x) = [-0.19473522 -0.99445365] (action = 8044)
Pareto set updated.
current Pareto set = [[-0.92633083 -0.29208351]
[-0.63329589 -0.63329589]
[-0.52191048 -0.72845916]
[-0.19963873 -0.93357674]
[-0.19473522 -0.99445365]] (steps = [ 4 20 17 13 23])
0024-th step: f(x) = [-0.99969529 -0.52395588] (action = 2135)
0025-th step: f(x) = [-0.59106078 -0.79258035] (action = 6206)
0026-th step: f(x) = [-0.78231041 -0.99997141] (action = 9079)
0027-th step: f(x) = [-0.99955573 -0.99930147] (action = 396)
0028-th step: f(x) = [-0.99825097 -0.99875436] (action = 9403)
0029-th step: f(x) = [-0.65387719 -0.99938669] (action = 9456)
0030-th step: f(x) = [-0.26132949 -0.87913689] (action = 5816)
Pareto set updated.
current Pareto set = [[-0.92633083 -0.29208351]
[-0.63329589 -0.63329589]
[-0.52191048 -0.72845916]
[-0.26132949 -0.87913689]
[-0.19963873 -0.93357674]
[-0.19473522 -0.99445365]] (steps = [ 4 20 17 30 13 23])
0031-th step: f(x) = [-0.74523952 -0.7724917 ] (action = 6301)
0032-th step: f(x) = [-0.33644513 -0.97772424] (action = 5530)
0033-th step: f(x) = [-0.99777557 -0.93373833] (action = 6470)
0034-th step: f(x) = [-0.9725524 -0.6296562] (action = 2277)
0035-th step: f(x) = [-0.99924826 -0.84674905] (action = 353)
0036-th step: f(x) = [-0.99743705 -0.17848438] (action = 2549)
Pareto set updated.
current Pareto set = [[-0.99743705 -0.17848438]
[-0.92633083 -0.29208351]
[-0.63329589 -0.63329589]
[-0.52191048 -0.72845916]
[-0.26132949 -0.87913689]
[-0.19963873 -0.93357674]
[-0.19473522 -0.99445365]] (steps = [36 4 20 17 30 13 23])
0037-th step: f(x) = [-0.99994916 -0.78013199] (action = 326)
0038-th step: f(x) = [-0.99887174 -0.86936149] (action = 5458)
0039-th step: f(x) = [-0.99362699 -0.99769786] (action = 9309)
0040-th step: f(x) = [-0.84889217 -0.99973225] (action = 10156)
0041-th step: f(x) = [-0.95713719 -0.09067194] (action = 4073)
Pareto set updated.
current Pareto set = [[-0.95713719 -0.09067194]
[-0.92633083 -0.29208351]
[-0.63329589 -0.63329589]
[-0.52191048 -0.72845916]
[-0.26132949 -0.87913689]
[-0.19963873 -0.93357674]
[-0.19473522 -0.99445365]] (steps = [41 4 20 17 30 13 23])
0042-th step: f(x) = [-0.17190645 -0.91382463] (action = 6120)
Pareto set updated.
current Pareto set = [[-0.95713719 -0.09067194]
[-0.92633083 -0.29208351]
[-0.63329589 -0.63329589]
[-0.52191048 -0.72845916]
[-0.26132949 -0.87913689]
[-0.17190645 -0.91382463]] (steps = [41 4 20 17 30 42])
0043-th step: f(x) = [-0.98406208 -0.94467587] (action = 7289)
0044-th step: f(x) = [-0.93974303 -0.92444262] (action = 2698)
0045-th step: f(x) = [-0.98414342 -0.84762781] (action = 1780)
0046-th step: f(x) = [-0.99699986 -0.98166426] (action = 7684)
0047-th step: f(x) = [-0.3868143 -0.99930896] (action = 8060)
0048-th step: f(x) = [-0.97964062 -0.65554315] (action = 2075)
0049-th step: f(x) = [-0.99907307 -0.73466786] (action = 4450)
0050-th step: f(x) = [-0.99720717 -0.85352507] (action = 865)
結果の確認
#### 評価値の履歴
[13]:
res_random.fx[0:res_random.num_runs]
[13]:
array([[-0.99973003, -0.62334035],
[-0.99789981, -0.99866427],
[-0.99090897, -0.46609239],
[-0.92633083, -0.29208351],
[-0.67969326, -0.99981691],
[-0.45601619, -0.99848443],
[-0.92670204, -0.71508873],
[-0.58233995, -0.99952931],
[-0.99848229, -0.96780195],
[-0.80479332, -0.99994946],
[-0.99700074, -0.99847873],
[-0.9992592 , -0.93891121],
[-0.19963873, -0.93357674],
[-0.98046765, -0.99294428],
[-0.99602549, -0.98620358],
[-0.99957128, -0.9973798 ],
[-0.52191048, -0.72845916],
[-0.99916441, -0.40869572],
[-0.99480122, -0.7565076 ],
[-0.63329589, -0.63329589],
[-0.95437918, -0.80142908],
[-0.99899466, -0.96646532],
[-0.19473522, -0.99445365],
[-0.99969529, -0.52395588],
[-0.59106078, -0.79258035],
[-0.78231041, -0.99997141],
[-0.99955573, -0.99930147],
[-0.99825097, -0.99875436],
[-0.65387719, -0.99938669],
[-0.26132949, -0.87913689],
[-0.74523952, -0.7724917 ],
[-0.33644513, -0.97772424],
[-0.99777557, -0.93373833],
[-0.9725524 , -0.6296562 ],
[-0.99924826, -0.84674905],
[-0.99743705, -0.17848438],
[-0.99994916, -0.78013199],
[-0.99887174, -0.86936149],
[-0.99362699, -0.99769786],
[-0.84889217, -0.99973225],
[-0.95713719, -0.09067194],
[-0.17190645, -0.91382463],
[-0.98406208, -0.94467587],
[-0.93974303, -0.92444262],
[-0.98414342, -0.84762781],
[-0.99699986, -0.98166426],
[-0.3868143 , -0.99930896],
[-0.97964062, -0.65554315],
[-0.99907307, -0.73466786],
[-0.99720717, -0.85352507]])
パレート解の取得
[14]:
front, front_num = res_random.export_pareto_front()
front, front_num
[14]:
(array([[-0.95713719, -0.09067194],
[-0.92633083, -0.29208351],
[-0.63329589, -0.63329589],
[-0.52191048, -0.72845916],
[-0.26132949, -0.87913689],
[-0.17190645, -0.91382463]]),
array([40, 3, 19, 16, 29, 41]))
解（評価値）のプロット
これ以降、図示する空間が \(y = (y_1, y_2)\) であり \(x = (x_1, x_2)\) ではないことにあらためて注意してください。
赤のプロットがパレート解です。
[15]:
def plot_pareto_front(res):
front, front_num = res.export_pareto_front()
dominated = [i for i in range(res.num_runs) if i not in front_num]
points = res.fx[dominated, :]
plt.figure(figsize=(7, 7))
plt.scatter(res.fx[dominated,0], res.fx[dominated,1], c = "blue")
plt.scatter(front[:, 0], front[:, 1], c = "red")
plt.title('Pareto front')
plt.xlabel('Objective 1')
plt.ylabel('Objective 2')
plt.xlim([-1.0,0.0])
plt.ylim([-1.0,0.0])
[16]:
plot_pareto_front(res_random)
劣解領域 (dominated region) の体積を計算
パレート解ではない解、すなわち、「自らよりも優れた解 \(y'\) が存在する解 \(y\) 」 を劣解と呼びます (\(\exists y' y \prec y'\))。
解空間（の部分空間）のうち、劣解の占める空間である劣解領域の体積は、多目的最適化の結果を示す指標のひとつです。
この値が大きいほど、より良いパレート解が多く求まっていることになります。
res_random.pareto.volume_in_dominance(ref_min, ref_max) は、 ref_min, ref_max で指定された矩形(hyper-rectangle) 中の劣解領域体積を計算します。
[17]:
res_random.pareto.volume_in_dominance([-1,-1],[0,0])
[17]:
np.float64(0.2376881844865093)
ベイズ最適化
多目的の場合の bayes_search では、score には以下のいずれかを指定します。
HVPI (HyperVolume-based Probability of Improvement)
EHVI (Expected Hyper-Volume Improvement)
TS (Thompson Sampling)
以下、score を変えてそれぞれ 50回 (ランダムサーチ10回 + ベイズ最適化 40回) 評価を行います。
HVPI (HyperVolume-based Probability of Improvement)
多次元の目的関数空間における非劣解領域 (non-dominated region) の改善確率を score として求めます。
参考文献
Couckuyt, Ivo, Dirk Deschrijver, and Tom Dhaene. 「Fast calculation of multiobjective probability of improvement and expected improvement criteria for Pareto optimization.」 Journal of Global Optimization 60.3 (2014): 575-594.
[18]:
policy = physbo.search.discrete_multi.Policy(test_X=test_X, num_objectives=2)
policy.set_seed(0)
policy.random_search(max_num_probes=10, simulator=simu)
res_HVPI = policy.bayes_search(max_num_probes=40, simulator=simu, score='HVPI', interval=10)
0001-th step: f(x) = [-0.99973003 -0.62334035] (action = 836)
Pareto set updated.
the number of Pareto frontiers = 1
0002-th step: f(x) = [-0.99789981 -0.99866427] (action = 9404)
Pareto set updated.
the number of Pareto frontiers = 2
0003-th step: f(x) = [-0.99090897 -0.46609239] (action = 4664)
Pareto set updated.
the number of Pareto frontiers = 1
0004-th step: f(x) = [-0.92633083 -0.29208351] (action = 4780)
Pareto set updated.
the number of Pareto frontiers = 1
0005-th step: f(x) = [-0.67969326 -0.99981691] (action = 9566)
Pareto set updated.
the number of Pareto frontiers = 2
0006-th step: f(x) = [-0.45601619 -0.99848443] (action = 8852)
Pareto set updated.
the number of Pareto frontiers = 2
0007-th step: f(x) = [-0.92670204 -0.71508873] (action = 6088)
0008-th step: f(x) = [-0.58233995 -0.99952931] (action = 7060)
0009-th step: f(x) = [-0.99848229 -0.96780195] (action = 473)
0010-th step: f(x) = [-0.80479332 -0.99994946] (action = 7573)
Start the initial hyper parameter searching ...
Done
Start the hyper parameter learning ...
0 -th epoch marginal likelihood -3.209523518535507
50 -th epoch marginal likelihood -3.2561436911697434
100 -th epoch marginal likelihood -3.257729815869572
150 -th epoch marginal likelihood -3.259506317055296
200 -th epoch marginal likelihood -3.2618022406678744
250 -th epoch marginal likelihood -3.2650078673751812
300 -th epoch marginal likelihood -3.2702146544567237
350 -th epoch marginal likelihood -3.2831467787386135
400 -th epoch marginal likelihood -3.3349571842670835
450 -th epoch marginal likelihood -3.4598938485764457
500 -th epoch marginal likelihood -3.638891636652417
Done
Start the initial hyper parameter searching ...
Done
Start the hyper parameter learning ...
0 -th epoch marginal likelihood -0.39855118433009995
50 -th epoch marginal likelihood -1.1073617181516324
100 -th epoch marginal likelihood -1.48359552534615
150 -th epoch marginal likelihood -1.6402679273119745
200 -th epoch marginal likelihood -1.723529184935276
250 -th epoch marginal likelihood -1.7923359112412909
300 -th epoch marginal likelihood -1.8557670557806043
350 -th epoch marginal likelihood -1.914971589977279
400 -th epoch marginal likelihood -1.9707008234172179
450 -th epoch marginal likelihood -2.023644146709623
500 -th epoch marginal likelihood -2.0742748668215913
Done
0011-th step: f(x) = [-0.3901523 -0.82431347] (action = 5911)
Pareto set updated.
the number of Pareto frontiers = 2
0012-th step: f(x) = [-0.03511246 -0.9676084 ] (action = 6830)
Pareto set updated.
the number of Pareto frontiers = 3
0013-th step: f(x) = [-0.71403166 -0.55036801] (action = 5096)
Pareto set updated.
the number of Pareto frontiers = 4
0014-th step: f(x) = [-0.57031134 -0.69398025] (action = 5403)
Pareto set updated.
the number of Pareto frontiers = 5
0015-th step: f(x) = [-0.95456998 -0.13930743] (action = 4274)
Pareto set updated.
the number of Pareto frontiers = 6
0016-th step: f(x) = [-0.97105694 -0.03456373] (action = 3769)
Pareto set updated.
the number of Pareto frontiers = 7
0017-th step: f(x) = [-0.8209077 -0.3783301] (action = 4589)
Pareto set updated.
the number of Pareto frontiers = 8
0018-th step: f(x) = [-0.2166426 -0.8977806] (action = 6118)
Pareto set updated.
the number of Pareto frontiers = 9
0019-th step: f(x) = [-0.8884927 -0.23687603] (action = 4283)
Pareto set updated.
the number of Pareto frontiers = 9
0020-th step: f(x) = [-0.47482314 -0.76211783] (action = 5507)
Pareto set updated.
the number of Pareto frontiers = 10
Start the initial hyper parameter searching ...
Done
Start the hyper parameter learning ...
0 -th epoch marginal likelihood -20.47851706054039
50 -th epoch marginal likelihood -20.781777717781775
100 -th epoch marginal likelihood -21.074743824515078
150 -th epoch marginal likelihood -21.361819816983022
200 -th epoch marginal likelihood -21.643265562211905
250 -th epoch marginal likelihood -21.91931561518443
300 -th epoch marginal likelihood -22.190181963505218
350 -th epoch marginal likelihood -22.456056664334906
400 -th epoch marginal likelihood -22.717113628442412
450 -th epoch marginal likelihood -22.973511398909388
500 -th epoch marginal likelihood -23.22539459785565
Done
Start the initial hyper parameter searching ...
Done
Start the hyper parameter learning ...
0 -th epoch marginal likelihood -19.220067456548204
50 -th epoch marginal likelihood -19.781203528558475
100 -th epoch marginal likelihood -20.16078232129641
150 -th epoch marginal likelihood -20.476091523476207
200 -th epoch marginal likelihood -20.77894833455666
250 -th epoch marginal likelihood -21.075707387253228
300 -th epoch marginal likelihood -21.367080634555183
350 -th epoch marginal likelihood -21.65332404734539
400 -th epoch marginal likelihood -21.93463854018043
450 -th epoch marginal likelihood -22.211199410392915
500 -th epoch marginal likelihood -22.483162429093504
Done
0021-th step: f(x) = [-0.00146848 -0.97868044] (action = 6834)
Pareto set updated.
the number of Pareto frontiers = 11
0022-th step: f(x) = [-9.82978321e-01 -3.32414921e-04] (action = 3264)
Pareto set updated.
the number of Pareto frontiers = 12
0023-th step: f(x) = [-0.65290909 -0.61133266] (action = 4999)
Pareto set updated.
the number of Pareto frontiers = 13
0024-th step: f(x) = [-0.76211783 -0.47482314] (action = 4693)
Pareto set updated.
the number of Pareto frontiers = 14
0025-th step: f(x) = [-0.30569255 -0.85754934] (action = 5814)
Pareto set updated.
the number of Pareto frontiers = 15
0026-th step: f(x) = [-0.11566871 -0.93445881] (action = 6223)
Pareto set updated.
the number of Pareto frontiers = 16
0027-th step: f(x) = [-0.93445881 -0.11566871] (action = 3877)
Pareto set updated.
the number of Pareto frontiers = 16
0028-th step: f(x) = [-0.85754934 -0.30569255] (action = 4386)
Pareto set updated.
the number of Pareto frontiers = 17
0029-th step: f(x) = [-0.61133266 -0.65290909] (action = 5101)
Pareto set updated.
the number of Pareto frontiers = 18
0030-th step: f(x) = [-0.52191048 -0.72845916] (action = 5305)
Pareto set updated.
the number of Pareto frontiers = 19
Start the initial hyper parameter searching ...
Done
Start the hyper parameter learning ...
0 -th epoch marginal likelihood -42.17175308256856
50 -th epoch marginal likelihood -43.80465098138666
100 -th epoch marginal likelihood -45.02627488559782
150 -th epoch marginal likelihood -45.96539736747549
200 -th epoch marginal likelihood -46.77369057378866
250 -th epoch marginal likelihood -47.54071032744218
300 -th epoch marginal likelihood -48.29092418080571
350 -th epoch marginal likelihood -49.02943117865013
400 -th epoch marginal likelihood -49.75844919206087
450 -th epoch marginal likelihood -50.47935398601936
500 -th epoch marginal likelihood -51.19300046940765
Done
Start the initial hyper parameter searching ...
Done
Start the hyper parameter learning ...
0 -th epoch marginal likelihood -41.719173831287456
50 -th epoch marginal likelihood -43.05994290558705
100 -th epoch marginal likelihood -44.130659234469974
150 -th epoch marginal likelihood -45.04560599026428
200 -th epoch marginal likelihood -45.85761716495073
250 -th epoch marginal likelihood -46.625463272431865
300 -th epoch marginal likelihood -47.377755215418944
350 -th epoch marginal likelihood -48.12173437412513
400 -th epoch marginal likelihood -48.85871974035222
450 -th epoch marginal likelihood -49.58901390368543
500 -th epoch marginal likelihood -50.312730818888525
Done
0031-th step: f(x) = [-0.42677852 -0.79293383] (action = 5509)
Pareto set updated.
the number of Pareto frontiers = 20
0032-th step: f(x) = [-0.79293383 -0.42677852] (action = 4591)
Pareto set updated.
the number of Pareto frontiers = 21
0033-th step: f(x) = [-0.91382463 -0.17190645] (action = 4080)
Pareto set updated.
the number of Pareto frontiers = 22
0034-th step: f(x) = [-0.72845916 -0.52191048] (action = 4795)
Pareto set updated.
the number of Pareto frontiers = 23
0035-th step: f(x) = [-0.69201544 -0.56755251] (action = 4897)
Pareto set updated.
the number of Pareto frontiers = 24
0036-th step: f(x) = [-0.95078624 -0.06762287] (action = 3774)
Pareto set updated.
the number of Pareto frontiers = 25
0037-th step: f(x) = [-0.17190645 -0.91382463] (action = 6120)
Pareto set updated.
the number of Pareto frontiers = 26
0038-th step: f(x) = [-0.35362632 -0.83371008] (action = 5712)
Pareto set updated.
the number of Pareto frontiers = 27
0039-th step: f(x) = [-0.25896196 -0.87874951] (action = 5916)
Pareto set updated.
the number of Pareto frontiers = 28
0040-th step: f(x) = [-0.06762287 -0.95078624] (action = 6426)
Pareto set updated.
the number of Pareto frontiers = 29
Start the initial hyper parameter searching ...
Done
Start the hyper parameter learning ...
0 -th epoch marginal likelihood -68.14422496160624
50 -th epoch marginal likelihood -69.74821313678456
100 -th epoch marginal likelihood -71.16364516263113
150 -th epoch marginal likelihood -72.45633416637565
200 -th epoch marginal likelihood -73.7042727789136
250 -th epoch marginal likelihood -74.9391823161883
300 -th epoch marginal likelihood -76.16638542804971
350 -th epoch marginal likelihood -77.3865858356882
400 -th epoch marginal likelihood -78.60006687735121
450 -th epoch marginal likelihood -79.80705478031906
500 -th epoch marginal likelihood -81.00774602550848
Done
Start the initial hyper parameter searching ...
Done
Start the hyper parameter learning ...
0 -th epoch marginal likelihood -67.44927460858037
50 -th epoch marginal likelihood -69.06511428073524
100 -th epoch marginal likelihood -70.50986339369791
150 -th epoch marginal likelihood -71.83302009477606
200 -th epoch marginal likelihood -73.1047262763335
250 -th epoch marginal likelihood -74.35727970975769
300 -th epoch marginal likelihood -75.59977538619195
350 -th epoch marginal likelihood -76.83435149512748
400 -th epoch marginal likelihood -78.06151201943835
450 -th epoch marginal likelihood -79.28144610933279
500 -th epoch marginal likelihood -80.49427803124193
Done
0041-th step: f(x) = [-0.83371008 -0.35362632] (action = 4488)
Pareto set updated.
the number of Pareto frontiers = 30
0042-th step: f(x) = [-0.58937008 -0.67252288] (action = 5202)
Pareto set updated.
the number of Pareto frontiers = 31
0043-th step: f(x) = [-0.54457573 -0.71034774] (action = 5304)
Pareto set updated.
the number of Pareto frontiers = 32
0044-th step: f(x) = [-0.86857621 -0.28270771] (action = 4285)
Pareto set updated.
the number of Pareto frontiers = 33
0045-th step: f(x) = [-0.63212056 -0.63212056] (action = 5100)
Pareto set updated.
the number of Pareto frontiers = 34
0046-th step: f(x) = [-0.67252288 -0.58937008] (action = 4998)
Pareto set updated.
the number of Pareto frontiers = 35
0047-th step: f(x) = [-0.40208972 -0.80711969] (action = 5610)
Pareto set updated.
the number of Pareto frontiers = 36
0048-th step: f(x) = [-0.7454381 -0.49811725] (action = 4794)
Pareto set updated.
the number of Pareto frontiers = 37
0049-th step: f(x) = [-0.49811725 -0.7454381 ] (action = 5406)
Pareto set updated.
the number of Pareto frontiers = 38
0050-th step: f(x) = [-0.77770464 -0.45044788] (action = 4692)
Pareto set updated.
the number of Pareto frontiers = 39
パレート解のプロット
ランダムサンプリングと比較して、パレート解が多く求まっていることが分かります。
[19]:
plot_pareto_front(res_HVPI)
劣解領域体積
[20]:
res_HVPI.pareto.volume_in_dominance([-1,-1],[0,0])
[20]:
np.float64(0.32877907991633726)
EHVI (Expected Hyper-Volume Improvement)
多次元の目的関数空間における非劣解領域 (non-dominated region) の改善期待値を score として求めます。
参考文献
Couckuyt, Ivo, Dirk Deschrijver, and Tom Dhaene. 「Fast calculation of multiobjective probability of improvement and expected improvement criteria for Pareto optimization.」 Journal of Global Optimization 60.3 (2014): 575-594.
[21]:
policy = physbo.search.discrete_multi.Policy(test_X=test_X, num_objectives=2)
policy.set_seed(0)
policy.random_search(max_num_probes=10, simulator=simu)
res_EHVI = policy.bayes_search(max_num_probes=40, simulator=simu, score='EHVI', interval=10)
0001-th step: f(x) = [-0.99973003 -0.62334035] (action = 836)
Pareto set updated.
the number of Pareto frontiers = 1
0002-th step: f(x) = [-0.99789981 -0.99866427] (action = 9404)
Pareto set updated.
the number of Pareto frontiers = 2
0003-th step: f(x) = [-0.99090897 -0.46609239] (action = 4664)
Pareto set updated.
the number of Pareto frontiers = 1
0004-th step: f(x) = [-0.92633083 -0.29208351] (action = 4780)
Pareto set updated.
the number of Pareto frontiers = 1
0005-th step: f(x) = [-0.67969326 -0.99981691] (action = 9566)
Pareto set updated.
the number of Pareto frontiers = 2
0006-th step: f(x) = [-0.45601619 -0.99848443] (action = 8852)
Pareto set updated.
the number of Pareto frontiers = 2
0007-th step: f(x) = [-0.92670204 -0.71508873] (action = 6088)
0008-th step: f(x) = [-0.58233995 -0.99952931] (action = 7060)
0009-th step: f(x) = [-0.99848229 -0.96780195] (action = 473)
0010-th step: f(x) = [-0.80479332 -0.99994946] (action = 7573)
Start the initial hyper parameter searching ...
Done
Start the hyper parameter learning ...
0 -th epoch marginal likelihood -3.209523518535507
50 -th epoch marginal likelihood -3.2561436911697434
100 -th epoch marginal likelihood -3.257729815869572
150 -th epoch marginal likelihood -3.259506317055296
200 -th epoch marginal likelihood -3.2618022406678744
250 -th epoch marginal likelihood -3.2650078673751812
300 -th epoch marginal likelihood -3.2702146544567237
350 -th epoch marginal likelihood -3.2831467787386135
400 -th epoch marginal likelihood -3.3349571842670835
450 -th epoch marginal likelihood -3.4598938485764457
500 -th epoch marginal likelihood -3.638891636652417
Done
Start the initial hyper parameter searching ...
Done
Start the hyper parameter learning ...
0 -th epoch marginal likelihood -0.39855118433009995
50 -th epoch marginal likelihood -1.1073617181516324
100 -th epoch marginal likelihood -1.48359552534615
150 -th epoch marginal likelihood -1.6402679273119745
200 -th epoch marginal likelihood -1.723529184935276
250 -th epoch marginal likelihood -1.7923359112412909
300 -th epoch marginal likelihood -1.8557670557806043
350 -th epoch marginal likelihood -1.914971589977279
400 -th epoch marginal likelihood -1.9707008234172179
450 -th epoch marginal likelihood -2.023644146709623
500 -th epoch marginal likelihood -2.0742748668215913
Done
0011-th step: f(x) = [-9.00606888e-04 -9.80950206e-01] (action = 6935)
Pareto set updated.
the number of Pareto frontiers = 2
0012-th step: f(x) = [-0.54603076 -0.71127314] (action = 5404)
Pareto set updated.
the number of Pareto frontiers = 3
0013-th step: f(x) = [-0.7666416 -0.48481035] (action = 4993)
Pareto set updated.
the number of Pareto frontiers = 4
0014-th step: f(x) = [-0.24028954 -0.95990013] (action = 5626)
Pareto set updated.
the number of Pareto frontiers = 5
0015-th step: f(x) = [-0.95713719 -0.09067194] (action = 4073)
Pareto set updated.
the number of Pareto frontiers = 6
0016-th step: f(x) = [-0.30791078 -0.85800445] (action = 5914)
Pareto set updated.
the number of Pareto frontiers = 7
0017-th step: f(x) = [-0.97621647 -0.0052244 ] (action = 3467)
Pareto set updated.
the number of Pareto frontiers = 8
0018-th step: f(x) = [-0.86941464 -0.28728372] (action = 4185)
Pareto set updated.
the number of Pareto frontiers = 8
0019-th step: f(x) = [-0.67252288 -0.58937008] (action = 4998)
Pareto set updated.
the number of Pareto frontiers = 9
0020-th step: f(x) = [-0.24028954 -0.95990013] (action = 7226)
Pareto set updated.
the number of Pareto frontiers = 10
Start the initial hyper parameter searching ...
Done
Start the hyper parameter learning ...
0 -th epoch marginal likelihood -17.256832207034954
50 -th epoch marginal likelihood -17.70987258414165
100 -th epoch marginal likelihood -17.978813161119295
150 -th epoch marginal likelihood -18.20902541636026
200 -th epoch marginal likelihood -18.42887383064057
250 -th epoch marginal likelihood -18.642976990250432
300 -th epoch marginal likelihood -18.852196397138677
350 -th epoch marginal likelihood -19.056813755107562
400 -th epoch marginal likelihood -19.257038614163122
450 -th epoch marginal likelihood -19.453048039816018
500 -th epoch marginal likelihood -19.644993031408248
Done
Start the initial hyper parameter searching ...
Done
Start the hyper parameter learning ...
0 -th epoch marginal likelihood -16.974228421583867
50 -th epoch marginal likelihood -17.412773390586732
100 -th epoch marginal likelihood -17.6957396729131
150 -th epoch marginal likelihood -17.947485717899337
200 -th epoch marginal likelihood -18.19073249547074
250 -th epoch marginal likelihood -18.427704212860476
300 -th epoch marginal likelihood -18.658813644519498
350 -th epoch marginal likelihood -18.884284104555697
400 -th epoch marginal likelihood -19.10431964671836
450 -th epoch marginal likelihood -19.31910873872588
500 -th epoch marginal likelihood -19.528823244919753
Done
0021-th step: f(x) = [-0.45044788 -0.77770464] (action = 5508)
Pareto set updated.
the number of Pareto frontiers = 11
0022-th step: f(x) = [-0.15266624 -0.92125491] (action = 6221)
Pareto set updated.
the number of Pareto frontiers = 10
0023-th step: f(x) = [-0.80711969 -0.40208972] (action = 4590)
Pareto set updated.
the number of Pareto frontiers = 11
0024-th step: f(x) = [-0.98483911 -0.00295954] (action = 3163)
Pareto set updated.
the number of Pareto frontiers = 12
0025-th step: f(x) = [-0.91382463 -0.17190645] (action = 4080)
Pareto set updated.
the number of Pareto frontiers = 13
0026-th step: f(x) = [-0.61133266 -0.65290909] (action = 5201)
Pareto set updated.
the number of Pareto frontiers = 14
0027-th step: f(x) = [-3.32414921e-04 -9.82978321e-01] (action = 6936)
Pareto set updated.
the number of Pareto frontiers = 15
0028-th step: f(x) = [-0.72845916 -0.52191048] (action = 4895)
Pareto set updated.
the number of Pareto frontiers = 16
0029-th step: f(x) = [-0.3783301 -0.8209077] (action = 5711)
Pareto set updated.
the number of Pareto frontiers = 17
0030-th step: f(x) = [-9.82978321e-01 -3.32414921e-04] (action = 3264)
Pareto set updated.
the number of Pareto frontiers = 17
Start the initial hyper parameter searching ...
Done
Start the hyper parameter learning ...
0 -th epoch marginal likelihood -40.56517293280312
50 -th epoch marginal likelihood -41.830958947827085
100 -th epoch marginal likelihood -42.67299215293333
150 -th epoch marginal likelihood -43.40527956813362
200 -th epoch marginal likelihood -44.119164651126496
250 -th epoch marginal likelihood -44.82488438364018
300 -th epoch marginal likelihood -45.52360072557966
350 -th epoch marginal likelihood -46.21583030400157
400 -th epoch marginal likelihood -46.90190490077359
450 -th epoch marginal likelihood -47.58205426649934
500 -th epoch marginal likelihood -48.2564517202437
Done
Start the initial hyper parameter searching ...
Done
Start the hyper parameter learning ...
0 -th epoch marginal likelihood -38.64708359440495
50 -th epoch marginal likelihood -40.29723156259631
100 -th epoch marginal likelihood -41.50105212639616
150 -th epoch marginal likelihood -42.39506537989113
200 -th epoch marginal likelihood -43.16816415460113
250 -th epoch marginal likelihood -43.90802002518212
300 -th epoch marginal likelihood -44.634574259968524
350 -th epoch marginal likelihood -45.35124261625998
400 -th epoch marginal likelihood -46.05893403939048
450 -th epoch marginal likelihood -46.75815946038287
500 -th epoch marginal likelihood -47.44928513058065
Done
0031-th step: f(x) = [-0.00295954 -0.98483911] (action = 6937)
0032-th step: f(x) = [-0.06762287 -0.95078624] (action = 6426)
Pareto set updated.
the number of Pareto frontiers = 18
0033-th step: f(x) = [-0.23687603 -0.8884927 ] (action = 6017)
Pareto set updated.
the number of Pareto frontiers = 19
0034-th step: f(x) = [-0.00295954 -0.98483911] (action = 7037)
0035-th step: f(x) = [-0.9830327 -0.00352624] (action = 3164)
0036-th step: f(x) = [-0.83371008 -0.35362632] (action = 4488)
Pareto set updated.
the number of Pareto frontiers = 20
0037-th step: f(x) = [-0.49811725 -0.7454381 ] (action = 5406)
Pareto set updated.
the number of Pareto frontiers = 21
0038-th step: f(x) = [-9.00606888e-04 -9.80950206e-01] (action = 6835)
Pareto set updated.
the number of Pareto frontiers = 22
0039-th step: f(x) = [-0.98483911 -0.00295954] (action = 3263)
0040-th step: f(x) = [-0.00557976 -0.98649648] (action = 7038)
Start the initial hyper parameter searching ...
Done
Start the hyper parameter learning ...
0 -th epoch marginal likelihood -60.35218224451241
50 -th epoch marginal likelihood -62.31324230703647
100 -th epoch marginal likelihood -64.03507521142387
150 -th epoch marginal likelihood -65.58859324220903
200 -th epoch marginal likelihood -66.99580206729125
250 -th epoch marginal likelihood -68.30020077573657
300 -th epoch marginal likelihood -69.54647337575287
350 -th epoch marginal likelihood -70.76184092943669
400 -th epoch marginal likelihood -71.95882264318004
450 -th epoch marginal likelihood -73.14338125074447
500 -th epoch marginal likelihood -74.31855291263867
Done
Start the initial hyper parameter searching ...
Done
Start the hyper parameter learning ...
0 -th epoch marginal likelihood -62.25021556098837
50 -th epoch marginal likelihood -63.99696283844089
100 -th epoch marginal likelihood -65.48521482593569
150 -th epoch marginal likelihood -66.82111661606083
200 -th epoch marginal likelihood -68.08675696269277
250 -th epoch marginal likelihood -69.31967194576625
300 -th epoch marginal likelihood -70.53336812110848
350 -th epoch marginal likelihood -71.73451334728779
400 -th epoch marginal likelihood -72.92618059866014
450 -th epoch marginal likelihood -74.10947899659575
500 -th epoch marginal likelihood -75.28474615660663
Done
0041-th step: f(x) = [-9.80950206e-01 -9.00606888e-04] (action = 3265)
Pareto set updated.
the number of Pareto frontiers = 23
0042-th step: f(x) = [-0.8884927 -0.23687603] (action = 4183)
Pareto set updated.
the number of Pareto frontiers = 24
0043-th step: f(x) = [-0.00146848 -0.97868044] (action = 6834)
Pareto set updated.
the number of Pareto frontiers = 25
0044-th step: f(x) = [-0.93445881 -0.11566871] (action = 3877)
Pareto set updated.
the number of Pareto frontiers = 26
0045-th step: f(x) = [-0.98107174 -0.00727443] (action = 3165)
0046-th step: f(x) = [-0.00352624 -0.9830327 ] (action = 6836)
0047-th step: f(x) = [-0.77770464 -0.45044788] (action = 4692)
Pareto set updated.
the number of Pareto frontiers = 27
0048-th step: f(x) = [-0.00352624 -0.9830327 ] (action = 7036)
0049-th step: f(x) = [-0.98649648 -0.00557976] (action = 3162)
0050-th step: f(x) = [-0.63212056 -0.63212056] (action = 5100)
Pareto set updated.
the number of Pareto frontiers = 28
パレート解のプロット
[22]:
plot_pareto_front(res_EHVI)
劣解領域体積
[23]:
res_EHVI.pareto.volume_in_dominance([-1,-1],[0,0])
[23]:
np.float64(0.3200467412741881)
TS (Thompson Sampling)
単目的の場合の Thompson Sampling では、各候補(test_X)について、目的関数の事後分布からサンプリングを行い、値が最大となる候補を次の探索点として推薦します。
多目的の場合は、サンプリングした値についてパレートルールの上で最大となる候補、つまりパレート最適な候補の中からランダムに1つ選択して次の探索点とします。
参考文献
Yahyaa, Saba Q., and Bernard Manderick. 「Thompson sampling for multi-objective multi-armed bandits problem.」 Proc. Eur. Symp. Artif. Neural Netw., Comput. Intell. Mach. Learn.. 2015.
[24]:
policy = physbo.search.discrete_multi.Policy(test_X=test_X, num_objectives=2)
policy.set_seed(0)
policy.random_search(max_num_probes=10, simulator=simu)
res_TS = policy.bayes_search(max_num_probes=40, simulator=simu, score='TS', interval=10, num_rand_basis=5000)
0001-th step: f(x) = [-0.99973003 -0.62334035] (action = 836)
Pareto set updated.
the number of Pareto frontiers = 1
0002-th step: f(x) = [-0.99789981 -0.99866427] (action = 9404)
Pareto set updated.
the number of Pareto frontiers = 2
0003-th step: f(x) = [-0.99090897 -0.46609239] (action = 4664)
Pareto set updated.
the number of Pareto frontiers = 1
0004-th step: f(x) = [-0.92633083 -0.29208351] (action = 4780)
Pareto set updated.
the number of Pareto frontiers = 1
0005-th step: f(x) = [-0.67969326 -0.99981691] (action = 9566)
Pareto set updated.
the number of Pareto frontiers = 2
0006-th step: f(x) = [-0.45601619 -0.99848443] (action = 8852)
Pareto set updated.
the number of Pareto frontiers = 2
0007-th step: f(x) = [-0.92670204 -0.71508873] (action = 6088)
0008-th step: f(x) = [-0.58233995 -0.99952931] (action = 7060)
0009-th step: f(x) = [-0.99848229 -0.96780195] (action = 473)
0010-th step: f(x) = [-0.80479332 -0.99994946] (action = 7573)
Start the initial hyper parameter searching ...
Done
Start the hyper parameter learning ...
0 -th epoch marginal likelihood -3.209523518535507
50 -th epoch marginal likelihood -3.2561436911697434
100 -th epoch marginal likelihood -3.257729815869572
150 -th epoch marginal likelihood -3.259506317055296
200 -th epoch marginal likelihood -3.2618022406678744
250 -th epoch marginal likelihood -3.2650078673751812
300 -th epoch marginal likelihood -3.2702146544567237
350 -th epoch marginal likelihood -3.2831467787386135
400 -th epoch marginal likelihood -3.3349571842670835
450 -th epoch marginal likelihood -3.4598938485764457
500 -th epoch marginal likelihood -3.638891636652417
Done
Start the initial hyper parameter searching ...
Done
Start the hyper parameter learning ...
0 -th epoch marginal likelihood 1.6231643893527536
50 -th epoch marginal likelihood 0.9161844782048458
100 -th epoch marginal likelihood 0.5685391937325868
150 -th epoch marginal likelihood 0.4352074454732371
200 -th epoch marginal likelihood 0.4046905941601331
250 -th epoch marginal likelihood 0.40039478628316516
300 -th epoch marginal likelihood 0.4000108419175117
350 -th epoch marginal likelihood 0.3999892703426049
400 -th epoch marginal likelihood 0.399988546087064
450 -th epoch marginal likelihood 0.3999885330679991
500 -th epoch marginal likelihood 0.3999885329713031
Done
0011-th step: f(x) = [-0.92567449 -0.99858284] (action = 4035)
0012-th step: f(x) = [-0.40231263 -0.99932643] (action = 7860)
Pareto set updated.
the number of Pareto frontiers = 3
0013-th step: f(x) = [-0.98603674 -0.99111929] (action = 2004)
0014-th step: f(x) = [-0.99813428 -0.80708194] (action = 5259)
0015-th step: f(x) = [-0.1290265 -0.97076113] (action = 6030)
Pareto set updated.
the number of Pareto frontiers = 2
0016-th step: f(x) = [-0.31326331 -0.99785753] (action = 8451)
0017-th step: f(x) = [-0.06477106 -0.98577901] (action = 6537)
Pareto set updated.
the number of Pareto frontiers = 3
0018-th step: f(x) = [-0.00727443 -0.98107174] (action = 6735)
Pareto set updated.
the number of Pareto frontiers = 3
0019-th step: f(x) = [-0.23644203 -0.9003642 ] (action = 6318)
Pareto set updated.
the number of Pareto frontiers = 4
0020-th step: f(x) = [-0.6174769 -0.950085 ] (action = 4518)
Start the initial hyper parameter searching ...
Done
Start the hyper parameter learning ...
0 -th epoch marginal likelihood -10.078833143340244
50 -th epoch marginal likelihood -11.372542035317903
100 -th epoch marginal likelihood -12.096279974270505
150 -th epoch marginal likelihood -12.690922552401627
200 -th epoch marginal likelihood -13.219727233175465
250 -th epoch marginal likelihood -13.681451256767232
300 -th epoch marginal likelihood -14.074623220134443
350 -th epoch marginal likelihood -14.402312179038432
400 -th epoch marginal likelihood -14.671730361070669
450 -th epoch marginal likelihood -14.892691354177359
500 -th epoch marginal likelihood -15.075844833346665
Done
Start the initial hyper parameter searching ...
Done
Start the hyper parameter learning ...
0 -th epoch marginal likelihood -15.112224858853768
50 -th epoch marginal likelihood -16.181986185590105
100 -th epoch marginal likelihood -16.833787053279423
150 -th epoch marginal likelihood -17.438417148847172
200 -th epoch marginal likelihood -18.007972646158954
250 -th epoch marginal likelihood -18.51649515565874
300 -th epoch marginal likelihood -18.945546768437563
350 -th epoch marginal likelihood -19.28948877533065
400 -th epoch marginal likelihood -19.55626174975576
450 -th epoch marginal likelihood -19.762790969117418
500 -th epoch marginal likelihood -19.927950829796334
Done
0021-th step: f(x) = [-0.47482314 -0.76211783] (action = 5407)
Pareto set updated.
the number of Pareto frontiers = 5
0022-th step: f(x) = [-0.99728354 -0.22242321] (action = 3050)
Pareto set updated.
the number of Pareto frontiers = 6
0023-th step: f(x) = [-0.70361786 -0.5838437 ] (action = 5297)
Pareto set updated.
the number of Pareto frontiers = 7
0024-th step: f(x) = [-0.65048192 -0.65048192] (action = 5500)
Pareto set updated.
the number of Pareto frontiers = 8
0025-th step: f(x) = [-0.5757763 -0.69787234] (action = 5503)
Pareto set updated.
the number of Pareto frontiers = 9
0026-th step: f(x) = [-0.49811725 -0.7454381 ] (action = 5406)
Pareto set updated.
the number of Pareto frontiers = 10
0027-th step: f(x) = [-0.19329537 -0.90599453] (action = 6119)
Pareto set updated.
the number of Pareto frontiers = 11
0028-th step: f(x) = [-0.99990551 -0.74008536] (action = 2630)
0029-th step: f(x) = [-0.35569141 -0.83424136] (action = 5812)
Pareto set updated.
the number of Pareto frontiers = 12
0030-th step: f(x) = [-0.52191048 -0.72845916] (action = 5405)
Pareto set updated.
the number of Pareto frontiers = 13
Start the initial hyper parameter searching ...
Done
Start the hyper parameter learning ...
0 -th epoch marginal likelihood -33.03566521657211
50 -th epoch marginal likelihood -34.21087943010022
100 -th epoch marginal likelihood -35.26691136227762
150 -th epoch marginal likelihood -36.1692116919876
200 -th epoch marginal likelihood -36.9095962531232
250 -th epoch marginal likelihood -37.52053460682146
300 -th epoch marginal likelihood -38.05614980685903
350 -th epoch marginal likelihood -38.55795777844784
400 -th epoch marginal likelihood -39.04407269367799
450 -th epoch marginal likelihood -39.51955099606786
500 -th epoch marginal likelihood -39.98554713717896
Done
Start the initial hyper parameter searching ...
Done
Start the hyper parameter learning ...
0 -th epoch marginal likelihood -40.362135886235095
50 -th epoch marginal likelihood -41.21185747261464
100 -th epoch marginal likelihood -41.71091427454593
150 -th epoch marginal likelihood -42.17891729040115
200 -th epoch marginal likelihood -42.63020005251695
250 -th epoch marginal likelihood -43.067870534527344
300 -th epoch marginal likelihood -43.494469044584584
350 -th epoch marginal likelihood -43.91164175202283
400 -th epoch marginal likelihood -44.32037337251974
450 -th epoch marginal likelihood -44.72129928652454
500 -th epoch marginal likelihood -45.11490990259672
Done
0031-th step: f(x) = [-0.67356913 -0.590682 ] (action = 5098)
Pareto set updated.
the number of Pareto frontiers = 14
0032-th step: f(x) = [-0.22412685 -0.89875721] (action = 6218)
Pareto set updated.
the number of Pareto frontiers = 14
0033-th step: f(x) = [-0.08294372 -0.9457967 ] (action = 6325)
Pareto set updated.
the number of Pareto frontiers = 14
0034-th step: f(x) = [-0.52496048 -0.73019147] (action = 5205)
0035-th step: f(x) = [-0.89875721 -0.22412685] (action = 4382)
Pareto set updated.
the number of Pareto frontiers = 14
0036-th step: f(x) = [-0.84649506 -0.40332189] (action = 3988)
Pareto set updated.
the number of Pareto frontiers = 15
0037-th step: f(x) = [-0.80711969 -0.40208972] (action = 4590)
Pareto set updated.
the number of Pareto frontiers = 15
0038-th step: f(x) = [-0.69201544 -0.56755251] (action = 4997)
Pareto set updated.
the number of Pareto frontiers = 15
0039-th step: f(x) = [-0.28270771 -0.86857621] (action = 5915)
Pareto set updated.
the number of Pareto frontiers = 16
0040-th step: f(x) = [-0.0706017 -0.95094348] (action = 6526)
Pareto set updated.
the number of Pareto frontiers = 17
Start the initial hyper parameter searching ...
Done
Start the hyper parameter learning ...
0 -th epoch marginal likelihood -56.21546310712146
50 -th epoch marginal likelihood -58.25622836432893
100 -th epoch marginal likelihood -59.75108821359915
150 -th epoch marginal likelihood -61.078458173593916
200 -th epoch marginal likelihood -62.26921857098807
250 -th epoch marginal likelihood -63.345215370503524
300 -th epoch marginal likelihood -64.3448690638929
350 -th epoch marginal likelihood -65.30367860992325
400 -th epoch marginal likelihood -66.24167890691578
450 -th epoch marginal likelihood -67.16646905837175
500 -th epoch marginal likelihood -68.08025664281062
Done
Start the initial hyper parameter searching ...
Done
Start the hyper parameter learning ...
0 -th epoch marginal likelihood -65.71713782535159
50 -th epoch marginal likelihood -66.80270167547336
100 -th epoch marginal likelihood -67.74510791293163
150 -th epoch marginal likelihood -68.65801740864794
200 -th epoch marginal likelihood -69.55912214517568
250 -th epoch marginal likelihood -70.44948901839517
300 -th epoch marginal likelihood -71.3294562629428
350 -th epoch marginal likelihood -72.19939545215031
400 -th epoch marginal likelihood -73.05968426337662
450 -th epoch marginal likelihood -73.91069526436273
500 -th epoch marginal likelihood -74.75279082216392
Done
0041-th step: f(x) = [-0.85800445 -0.30791078] (action = 4286)
Pareto set updated.
the number of Pareto frontiers = 18
0042-th step: f(x) = [-0.95545924 -0.05507465] (action = 3673)
Pareto set updated.
the number of Pareto frontiers = 18
0043-th step: f(x) = [-0.92175727 -0.15807186] (action = 4179)
Pareto set updated.
the number of Pareto frontiers = 19
0044-th step: f(x) = [-0.13297899 -0.92804454] (action = 6222)
Pareto set updated.
the number of Pareto frontiers = 20
0045-th step: f(x) = [-0.15807186 -0.92175727] (action = 6021)
Pareto set updated.
the number of Pareto frontiers = 21
0046-th step: f(x) = [-0.7462514 -0.4997207] (action = 4894)
Pareto set updated.
the number of Pareto frontiers = 22
0047-th step: f(x) = [-0.40399998 -0.80773592] (action = 5510)
Pareto set updated.
the number of Pareto frontiers = 23
0048-th step: f(x) = [-0.63329589 -0.63329589] (action = 5200)
Pareto set updated.
the number of Pareto frontiers = 23
0049-th step: f(x) = [-0.98131247 -0.01990033] (action = 3565)
Pareto set updated.
the number of Pareto frontiers = 24
0050-th step: f(x) = [-0.78401544 -0.46604924] (action = 4992)
Pareto set updated.
the number of Pareto frontiers = 25
パレート解のプロット
[25]:
plot_pareto_front(res_TS)
劣解領域体積
[26]:
res_TS.pareto.volume_in_dominance([-1,-1],[0,0])
[26]:
np.float64(0.31732213827454625)
付録：全探索
random_search で max_num_probes に全データ数 (N = test_X.shape[0]) を渡すと手軽に全探索できます。全データの評価には時間がかかるため、あらかじめデータ数を減らしておきます。
[27]:
test_X_sparse = np.array(list(itertools.product(np.linspace(-2, 2, 21), repeat=2)))
simu_sparse = Simulator(test_X_sparse)
policy = physbo.search.discrete_multi.Policy(test_X=test_X_sparse, num_objectives=2)
policy.set_seed(0)
N = test_X_sparse.shape[0]
res_all = policy.random_search(max_num_probes=N, simulator=simu_sparse)
0001-th step: f(x) = [-0.98370924 -0.99074733] (action = 361)
Pareto set updated.
the number of Pareto frontiers = 1
0002-th step: f(x) = [-0.99523084 -0.2246305 ] (action = 91)
Pareto set updated.
the number of Pareto frontiers = 2
0003-th step: f(x) = [-0.99987153 -0.64661672] (action = 46)
0004-th step: f(x) = [-0.69972145 -0.9998078 ] (action = 413)
Pareto set updated.
the number of Pareto frontiers = 3
0005-th step: f(x) = [-0.92045011 -0.23557331] (action = 196)
Pareto set updated.
the number of Pareto frontiers = 3
0006-th step: f(x) = [-0.99617304 -0.98813681] (action = 338)
0007-th step: f(x) = [-0.97604003 -0.76975911] (action = 256)
0008-th step: f(x) = [-0.82194903 -0.44805925] (action = 218)
Pareto set updated.
the number of Pareto frontiers = 4
0009-th step: f(x) = [-0.99987659 -0.99987659] (action = 420)
0010-th step: f(x) = [-0.97555848 -0.99552177] (action = 383)
0011-th step: f(x) = [-0.88371066 -0.99999225] (action = 437)
0012-th step: f(x) = [-0.82093385 -0.82093385] (action = 280)
Pareto set updated.
the number of Pareto frontiers = 5
0013-th step: f(x) = [-0.58998965 -0.99856763] (action = 390)
Pareto set updated.
the number of Pareto frontiers = 5
0014-th step: f(x) = [-0.70979198 -0.83517034] (action = 181)
Pareto set updated.
the number of Pareto frontiers = 6
0015-th step: f(x) = [-0.99829019 -0.98356972] (action = 36)
0016-th step: f(x) = [-0.44333209 -0.96709773] (action = 225)
Pareto set updated.
the number of Pareto frontiers = 6
0017-th step: f(x) = [-0.99980113 -0.99964985] (action = 399)
0018-th step: f(x) = [-0.96595598 -0.99935088] (action = 427)
0019-th step: f(x) = [-0.99516257 -0.95351517] (action = 56)
0020-th step: f(x) = [-0.98403021 -0.52428851] (action = 214)
0021-th step: f(x) = [-0.99968938 -0.51472752] (action = 67)
0022-th step: f(x) = [-0.88792973 -0.98833743] (action = 364)
0023-th step: f(x) = [-0.99960175 -0.88600183] (action = 10)
0024-th step: f(x) = [-0.2246305 -0.99523084] (action = 349)
Pareto set updated.
the number of Pareto frontiers = 7
0025-th step: f(x) = [-0.99728354 -0.22242321] (action = 90)
Pareto set updated.
the number of Pareto frontiers = 8
0026-th step: f(x) = [-0.90611507 -0.70896582] (action = 138)
0027-th step: f(x) = [-0.99995095 -0.76244576] (action = 25)
0028-th step: f(x) = [-0.98843647 -0.3935306 ] (action = 193)
0029-th step: f(x) = [-0.95021293 -0.95021293] (action = 320)
0030-th step: f(x) = [-0.90611507 -0.70896582] (action = 258)
0031-th step: f(x) = [-0.81837051 -0.98926465] (action = 365)
0032-th step: f(x) = [-0.94682748 -0.10038325] (action = 175)
Pareto set updated.
the number of Pareto frontiers = 7
0033-th step: f(x) = [-0.96709773 -0.44333209] (action = 115)
0034-th step: f(x) = [-0.91108736 -0.99830471] (action = 187)
0035-th step: f(x) = [-0.99994244 -0.72122752] (action = 65)
0036-th step: f(x) = [-0.40038762 -0.89013777] (action = 223)
Pareto set updated.
the number of Pareto frontiers = 7
0037-th step: f(x) = [-0.70057384 -0.99966256] (action = 412)
0038-th step: f(x) = [-0.91745532 -0.99949228] (action = 429)
0039-th step: f(x) = [-0.29032185 -0.97617587] (action = 326)
Pareto set updated.
the number of Pareto frontiers = 8
0040-th step: f(x) = [-0.76579837 -0.99918182] (action = 410)
0041-th step: f(x) = [-0.9999968 -0.91531622] (action = 2)
0042-th step: f(x) = [-0.94637208 -0.83375864] (action = 118)
0043-th step: f(x) = [-0.64661672 -0.99987153] (action = 394)
0044-th step: f(x) = [-0.99948501 -0.99505129] (action = 336)
0045-th step: f(x) = [-0.91838861 -0.9951763 ] (action = 385)
0046-th step: f(x) = [-0.99227074 -0.28627554] (action = 92)
0047-th step: f(x) = [-0.99948501 -0.99505129] (action = 16)
0048-th step: f(x) = [-0.72437945 -0.99945311] (action = 411)
0049-th step: f(x) = [-0.99993226 -0.81366367] (action = 126)
0050-th step: f(x) = [-0.72826929 -0.99087791] (action = 206)
0051-th step: f(x) = [-0.99959374 -0.99778272] (action = 357)
0052-th step: f(x) = [-0.40038762 -0.89013777] (action = 283)
Pareto set updated.
the number of Pareto frontiers = 9
0053-th step: f(x) = [-0.90826315 -0.99999653] (action = 418)
0054-th step: f(x) = [-0.09782226 -0.96971359] (action = 306)
Pareto set updated.
the number of Pareto frontiers = 8
0055-th step: f(x) = [-0.98926465 -0.81837051] (action = 75)
0056-th step: f(x) = [-0.99964985 -0.99980113] (action = 421)
0057-th step: f(x) = [-0.96595598 -0.99935088] (action = 167)
0058-th step: f(x) = [-0.5969236 -0.95805394] (action = 204)
0059-th step: f(x) = [-0.97389649 -0.91908164] (action = 98)
0060-th step: f(x) = [-0.43214402 -0.99936004] (action = 332)
0061-th step: f(x) = [-0.98370924 -0.99074733] (action = 101)
0062-th step: f(x) = [-0.98403021 -0.52428851] (action = 94)
0063-th step: f(x) = [-0.76579837 -0.99918182] (action = 250)
0064-th step: f(x) = [-0.64661672 -0.99987153] (action = 354)
0065-th step: f(x) = [-0.83517034 -0.70979198] (action = 259)
0066-th step: f(x) = [-0.91908164 -0.97389649] (action = 342)
0067-th step: f(x) = [-0.89013777 -0.40038762] (action = 157)
Pareto set updated.
the number of Pareto frontiers = 9
0068-th step: f(x) = [-0.99999091 -0.8635332 ] (action = 43)
0069-th step: f(x) = [-0.15766112 -0.9970573 ] (action = 330)
0070-th step: f(x) = [-0.99918182 -0.76579837] (action = 30)
0071-th step: f(x) = [-0.99074733 -0.98370924] (action = 339)
0072-th step: f(x) = [-0.99943574 -0.99943574] (action = 400)
0073-th step: f(x) = [-0.81629344 -0.99887005] (action = 229)
0074-th step: f(x) = [-0.86773446 -0.9955598 ] (action = 386)
0075-th step: f(x) = [-0.92022365 -0.56459205] (action = 237)
0076-th step: f(x) = [-0.9951763 -0.91838861] (action = 55)
0077-th step: f(x) = [-0.99778272 -0.99959374] (action = 83)
0078-th step: f(x) = [-0.99996438 -0.82750022] (action = 105)
0079-th step: f(x) = [-0.85793518 -0.97397059] (action = 163)
0080-th step: f(x) = [-0.99945311 -0.72437945] (action = 29)
0081-th step: f(x) = [-0.98830423 -0.93616616] (action = 77)
0082-th step: f(x) = [-0.01711023 -0.98935582] (action = 308)
Pareto set updated.
the number of Pareto frontiers = 10
0083-th step: f(x) = [-0.56459205 -0.92022365] (action = 303)
0084-th step: f(x) = [-0.65160403 -0.99785708] (action = 249)
0085-th step: f(x) = [-0.55457668 -0.99911619] (action = 391)
0086-th step: f(x) = [-0.99853885 -0.99202523] (action = 37)
0087-th step: f(x) = [-0.91908164 -0.97389649] (action = 142)
0088-th step: f(x) = [-0.99970416 -0.99908292] (action = 378)
0089-th step: f(x) = [-0.99911619 -0.55457668] (action = 149)
0090-th step: f(x) = [-0.99505129 -0.99948501] (action = 104)
0091-th step: f(x) = [-0.17190645 -0.91382463] (action = 264)
Pareto set updated.
the number of Pareto frontiers = 11
0092-th step: f(x) = [-0.56459205 -0.92022365] (action = 203)
0093-th step: f(x) = [-0.38659518 -0.99878288] (action = 311)
0094-th step: f(x) = [-0.99964985 -0.99980113] (action = 41)
0095-th step: f(x) = [-0.70896582 -0.90611507] (action = 182)
0096-th step: f(x) = [-0.99017418 -0.09267839] (action = 152)
Pareto set updated.
the number of Pareto frontiers = 12
0097-th step: f(x) = [-0.99481892 -0.52157619] (action = 72)
0098-th step: f(x) = [-0.94513852 -0.99815828] (action = 406)
0099-th step: f(x) = [-0.99949659 -0.55330867] (action = 128)
0100-th step: f(x) = [-0.9985717 -0.28016277] (action = 109)
0101-th step: f(x) = [-0.38833643 -0.99786316] (action = 290)
0102-th step: f(x) = [-0.79699336 -0.99997619] (action = 396)
0103-th step: f(x) = [-0.81366367 -0.99993226] (action = 434)
0104-th step: f(x) = [-0.85466737 -0.99971163] (action = 251)
0105-th step: f(x) = [-0.99856763 -0.58998965] (action = 50)
0106-th step: f(x) = [-0.93742384 -0.39868067] (action = 216)
0107-th step: f(x) = [-0.28627554 -0.99227074] (action = 268)
0108-th step: f(x) = [-0.01990033 -0.98131247] (action = 307)
Pareto set updated.
the number of Pareto frontiers = 13
0109-th step: f(x) = [-0.38659518 -0.99878288] (action = 371)
0110-th step: f(x) = [-0.97934917 -0.97934917] (action = 100)
0111-th step: f(x) = [-0.65160403 -0.99785708] (action = 389)
0112-th step: f(x) = [-0.90531099 -0.94621941] (action = 321)
0113-th step: f(x) = [-0.93742384 -0.39868067] (action = 136)
0114-th step: f(x) = [-0.99852214 -0.99916062] (action = 61)
0115-th step: f(x) = [-0.28016277 -0.9985717 ] (action = 351)
0116-th step: f(x) = [-0.99642687 -0.99884734] (action = 402)
0117-th step: f(x) = [-0.99987659 -0.99987659] (action = 20)
0118-th step: f(x) = [-0.52157619 -0.99481892] (action = 248)
0119-th step: f(x) = [-0.99949659 -0.55330867] (action = 48)
0120-th step: f(x) = [-0.09267839 -0.99017418] (action = 328)
0121-th step: f(x) = [-0.85793518 -0.97397059] (action = 343)
0122-th step: f(x) = [-0.99918182 -0.76579837] (action = 190)
0123-th step: f(x) = [-0.70979198 -0.83517034] (action = 281)
Pareto set updated.
the number of Pareto frontiers = 14
0124-th step: f(x) = [-0.98833743 -0.88792973] (action = 76)
0125-th step: f(x) = [-0.99270087 -0.99270087] (action = 360)
0126-th step: f(x) = [-0.99829019 -0.98356972] (action = 316)
0127-th step: f(x) = [-0.99908292 -0.99970416] (action = 62)
0128-th step: f(x) = [-0.92022365 -0.56459205] (action = 137)
0129-th step: f(x) = [-0.99884734 -0.99642687] (action = 358)
0130-th step: f(x) = [-0.99971163 -0.85466737] (action = 189)
0131-th step: f(x) = [-0.43696618 -0.99653688] (action = 269)
0132-th step: f(x) = [-0.01990033 -0.98131247] (action = 287)
Pareto set updated.
the number of Pareto frontiers = 15
0133-th step: f(x) = [-0.97964062 -0.65554315] (action = 95)
0134-th step: f(x) = [-0.99830471 -0.91108736] (action = 33)
0135-th step: f(x) = [-0.85466737 -0.99971163] (action = 431)
0136-th step: f(x) = [-0.99780154 -0.99780154] (action = 60)
0137-th step: f(x) = [-0.99704052 -0.72671998] (action = 212)
0138-th step: f(x) = [-0.80212129 -0.99622705] (action = 207)
0139-th step: f(x) = [-0.43696618 -0.99653688] (action = 369)
0140-th step: f(x) = [-0.76975911 -0.97604003] (action = 344)
0141-th step: f(x) = [-0.73362301 -0.53100224] (action = 199)
Pareto set updated.
the number of Pareto frontiers = 15
0142-th step: f(x) = [-0.99930674 -0.38484896] (action = 88)
0143-th step: f(x) = [-0.96643645 -0.98917271] (action = 122)
0144-th step: f(x) = [-0.82093385 -0.82093385] (action = 160)
0145-th step: f(x) = [-0.94513852 -0.99815828] (action = 166)
0146-th step: f(x) = [-0.39868067 -0.93742384] (action = 224)
0147-th step: f(x) = [-0.98131247 -0.01990033] (action = 153)
Pareto set updated.
the number of Pareto frontiers = 15
0148-th step: f(x) = [-0.63212056 -0.63212056] (action = 220)
Pareto set updated.
the number of Pareto frontiers = 14
0149-th step: f(x) = [-0.81837051 -0.98926465] (action = 185)
0150-th step: f(x) = [-0.52157619 -0.99481892] (action = 368)
0151-th step: f(x) = [-0.95805394 -0.5969236 ] (action = 236)
0152-th step: f(x) = [-0.90531099 -0.94621941] (action = 141)
0153-th step: f(x) = [-0.98980427 -0.99939737] (action = 125)
0154-th step: f(x) = [-0.98917271 -0.96643645] (action = 78)
0155-th step: f(x) = [-0.95351517 -0.99516257] (action = 144)
0156-th step: f(x) = [-0.99999091 -0.8635332 ] (action = 23)
0157-th step: f(x) = [-0.89771579 -0.89771579] (action = 140)
0158-th step: f(x) = [-0.29634835 -0.87107548] (action = 263)
Pareto set updated.
the number of Pareto frontiers = 13
0159-th step: f(x) = [-0.5969236 -0.95805394] (action = 324)
0160-th step: f(x) = [-0.96875158 -0.99815304] (action = 405)
0161-th step: f(x) = [-0.99887005 -0.81629344] (action = 211)
0162-th step: f(x) = [-0.99778272 -0.99959374] (action = 423)
0163-th step: f(x) = [-0.53100224 -0.73362301] (action = 241)
Pareto set updated.
the number of Pareto frontiers = 14
0164-th step: f(x) = [-0.60034647 -0.7730085 ] (action = 201)
0165-th step: f(x) = [-0.73362301 -0.53100224] (action = 219)
Pareto set updated.
the number of Pareto frontiers = 15
0166-th step: f(x) = [-0.95021293 -0.95021293] (action = 120)
0167-th step: f(x) = [-0.9998078 -0.69972145] (action = 27)
0168-th step: f(x) = [-0.72122752 -0.99994244] (action = 395)
0169-th step: f(x) = [-0.96709773 -0.44333209] (action = 215)
0170-th step: f(x) = [-0.10038325 -0.94682748] (action = 285)
Pareto set updated.
the number of Pareto frontiers = 16
0171-th step: f(x) = [-0.9955598 -0.86773446] (action = 54)
0172-th step: f(x) = [-0.99468465 -0.99698104] (action = 81)
0173-th step: f(x) = [-0.99698104 -0.99468465] (action = 359)
0174-th step: f(x) = [-0.09009546 -0.99440334] (action = 329)
0175-th step: f(x) = [-0.40208972 -0.80711969] (action = 242)
Pareto set updated.
the number of Pareto frontiers = 17
0176-th step: f(x) = [-0.91838861 -0.9951763 ] (action = 165)
0177-th step: f(x) = [-0.99468465 -0.99698104] (action = 381)
0178-th step: f(x) = [-0.99516257 -0.95351517] (action = 296)
0179-th step: f(x) = [-0.99815828 -0.94513852] (action = 34)
0180-th step: f(x) = [-0.99878288 -0.38659518] (action = 129)
0181-th step: f(x) = [-0.98407554 -0.16481416] (action = 113)
0182-th step: f(x) = [-0.09782226 -0.96971359] (action = 266)
Pareto set updated.
the number of Pareto frontiers = 18
0183-th step: f(x) = [-0.98407554 -0.16481416] (action = 173)
0184-th step: f(x) = [-0.97604003 -0.76975911] (action = 96)
0185-th step: f(x) = [-0.97555848 -0.99552177] (action = 123)
0186-th step: f(x) = [-0.98356972 -0.99829019] (action = 404)
0187-th step: f(x) = [-0.96467434 -0.99999957] (action = 440)
0188-th step: f(x) = [-0.99916062 -0.99852214] (action = 379)
0189-th step: f(x) = [-0.99815304 -0.96875158] (action = 35)
0190-th step: f(x) = [-0.09267839 -0.99017418] (action = 288)
0191-th step: f(x) = [-0.88600183 -0.99960175] (action = 430)
0192-th step: f(x) = [-0.99966256 -0.70057384] (action = 148)
0193-th step: f(x) = [-0.22242321 -0.99728354] (action = 350)
0194-th step: f(x) = [-0.99653688 -0.43696618] (action = 171)
0195-th step: f(x) = [-0.97934917 -0.97934917] (action = 340)
0196-th step: f(x) = [-0.94637208 -0.83375864] (action = 278)
0197-th step: f(x) = [-0.99936004 -0.43214402] (action = 68)
0198-th step: f(x) = [-0.91531622 -0.9999968 ] (action = 438)
0199-th step: f(x) = [-0.23339717 -0.95468937] (action = 245)
0200-th step: f(x) = [-0.85258693 -0.99998271] (action = 436)
0201-th step: f(x) = [-0.83375864 -0.94637208] (action = 322)
0202-th step: f(x) = [-0.89013777 -0.40038762] (action = 217)
Pareto set updated.
the number of Pareto frontiers = 19
0203-th step: f(x) = [-0.72201886 -0.99989894] (action = 414)
0204-th step: f(x) = [-0.82896507 -0.99980725] (action = 432)
0205-th step: f(x) = [-0.99704052 -0.72671998] (action = 52)
0206-th step: f(x) = [-0.02268252 -0.96719113] (action = 286)
Pareto set updated.
the number of Pareto frontiers = 18
0207-th step: f(x) = [-0.86698033 -0.99855947] (action = 408)
0208-th step: f(x) = [-0.99999225 -0.88371066] (action = 63)
0209-th step: f(x) = [-0.98813681 -0.99617304] (action = 382)
0210-th step: f(x) = [-0.99087791 -0.72826929] (action = 74)
0211-th step: f(x) = [-0.66040447 -0.66040447] (action = 240)
0212-th step: f(x) = [-0.23557331 -0.92045011] (action = 284)
0213-th step: f(x) = [-0.99887005 -0.81629344] (action = 31)
0214-th step: f(x) = [-0.94482572 -0.99940249] (action = 188)
0215-th step: f(x) = [-0.81419262 -0.99988107] (action = 293)
0216-th step: f(x) = [-0.44333209 -0.96709773] (action = 325)
0217-th step: f(x) = [-0.99980725 -0.82896507] (action = 8)
0218-th step: f(x) = [-0.97389649 -0.91908164] (action = 298)
0219-th step: f(x) = [-0.99284468 -0.62472681] (action = 213)
0220-th step: f(x) = [-0.95063572 -0.73057679] (action = 117)
0221-th step: f(x) = [-0.99853885 -0.99202523] (action = 337)
0222-th step: f(x) = [-0.99973531 -0.58647807] (action = 107)
0223-th step: f(x) = [-0.96875158 -0.99815304] (action = 145)
0224-th step: f(x) = [-0.99989894 -0.72201886] (action = 106)
0225-th step: f(x) = [-0.99994244 -0.72122752] (action = 45)
0226-th step: f(x) = [-0.52428851 -0.98403021] (action = 226)
0227-th step: f(x) = [-0.97617587 -0.29032185] (action = 114)
0228-th step: f(x) = [-0.91745532 -0.99949228] (action = 209)
0229-th step: f(x) = [-0.95805394 -0.5969236 ] (action = 116)
0230-th step: f(x) = [-0.72122752 -0.99994244] (action = 375)
0231-th step: f(x) = [-0.99785708 -0.65160403] (action = 191)
0232-th step: f(x) = [-0.98833743 -0.88792973] (action = 276)
0233-th step: f(x) = [-0.9970573 -0.15766112] (action = 110)
0234-th step: f(x) = [-0.99940249 -0.94482572] (action = 252)
0235-th step: f(x) = [-0.65554315 -0.97964062] (action = 205)
0236-th step: f(x) = [-0.3935306 -0.98843647] (action = 347)
0237-th step: f(x) = [-0.94307317 -0.99999878] (action = 419)
0238-th step: f(x) = [-0.99911619 -0.55457668] (action = 49)
0239-th step: f(x) = [-0.99988107 -0.81419262] (action = 7)
0240-th step: f(x) = [-0.62472681 -0.99284468] (action = 367)
0241-th step: f(x) = [-0.72671998 -0.99704052] (action = 228)
0242-th step: f(x) = [-0.99815828 -0.94513852] (action = 274)
0243-th step: f(x) = [-0.95468937 -0.23339717] (action = 195)
0244-th step: f(x) = [-0.99552177 -0.97555848] (action = 317)
0245-th step: f(x) = [-0.76975911 -0.97604003] (action = 184)
0246-th step: f(x) = [-0.98926465 -0.81837051] (action = 255)
0247-th step: f(x) = [-0.51472752 -0.99968938] (action = 353)
0248-th step: f(x) = [-0.99997802 -0.81260125] (action = 24)
0249-th step: f(x) = [-0.91108736 -0.99830471] (action = 407)
0250-th step: f(x) = [-0.99989894 -0.72201886] (action = 26)
0251-th step: f(x) = [-0.99786316 -0.38833643] (action = 70)
0252-th step: f(x) = [-0.99980725 -0.82896507] (action = 168)
0253-th step: f(x) = [-0.99728354 -0.22242321] (action = 130)
0254-th step: f(x) = [-0.99999653 -0.90826315] (action = 22)
0255-th step: f(x) = [-0.7730085 -0.60034647] (action = 179)
0256-th step: f(x) = [-0.99855947 -0.86698033] (action = 32)
0257-th step: f(x) = [-0.82750022 -0.99996438] (action = 335)
0258-th step: f(x) = [-0.94307317 -0.99999878] (action = 439)
0259-th step: f(x) = [-0.89800614 -0.82042409] (action = 279)
0260-th step: f(x) = [-0.95745348 -0.97583482] (action = 341)
0261-th step: f(x) = [-0.98813681 -0.99617304] (action = 102)
0262-th step: f(x) = [-0.58998965 -0.99856763] (action = 270)
0263-th step: f(x) = [-0.85994015 -0.56582803] (action = 158)
0264-th step: f(x) = [-0.99936004 -0.43214402] (action = 108)
0265-th step: f(x) = [-0.55330867 -0.99949659] (action = 312)
0266-th step: f(x) = [-0.97617587 -0.29032185] (action = 194)
0267-th step: f(x) = [-0.97397059 -0.85793518] (action = 277)
0268-th step: f(x) = [-0.2246305 -0.99523084] (action = 289)
0269-th step: f(x) = [-0.72826929 -0.99087791] (action = 366)
0270-th step: f(x) = [-0.16481416 -0.98407554] (action = 267)
0271-th step: f(x) = [-0.98917271 -0.96643645] (action = 318)
0272-th step: f(x) = [-0.8635332 -0.99999091] (action = 397)
0273-th step: f(x) = [-0.99830471 -0.91108736] (action = 253)
0274-th step: f(x) = [-0.81260125 -0.99997802] (action = 416)
0275-th step: f(x) = [-0.99884734 -0.99642687] (action = 38)
0276-th step: f(x) = [-0.99523084 -0.2246305 ] (action = 151)
0277-th step: f(x) = [-0.99999878 -0.94307317] (action = 21)
0278-th step: f(x) = [-0.81260125 -0.99997802] (action = 376)
0279-th step: f(x) = [-0.76244576 -0.99995095] (action = 415)
0280-th step: f(x) = [-0.99966256 -0.70057384] (action = 28)
0281-th step: f(x) = [-0.94621941 -0.90531099] (action = 299)
0282-th step: f(x) = [-0.99855947 -0.86698033] (action = 232)
0283-th step: f(x) = [-0.99996438 -0.82750022] (action = 5)
0284-th step: f(x) = [-0.99999225 -0.88371066] (action = 3)
0285-th step: f(x) = [-0.93616616 -0.98830423] (action = 363)
0286-th step: f(x) = [-0.98356972 -0.99829019] (action = 124)
0287-th step: f(x) = [-0.99935088 -0.96595598] (action = 13)
0288-th step: f(x) = [-0.72437945 -0.99945311] (action = 271)
0289-th step: f(x) = [-0.99943574 -0.99943574] (action = 40)
0290-th step: f(x) = [-0.70057384 -0.99966256] (action = 292)
0291-th step: f(x) = [-0.97397059 -0.85793518] (action = 97)
0292-th step: f(x) = [-0.99995095 -0.76244576] (action = 85)
0293-th step: f(x) = [-0.98060895 -0.99934904] (action = 426)
0294-th step: f(x) = [-0.99074733 -0.98370924] (action = 79)
0295-th step: f(x) = [-0.99202523 -0.99853885] (action = 403)
0296-th step: f(x) = [-0.52428851 -0.98403021] (action = 346)
0297-th step: f(x) = [-0.89800614 -0.82042409] (action = 139)
0298-th step: f(x) = [-0.82194903 -0.44805925] (action = 178)
Pareto set updated.
the number of Pareto frontiers = 19
0299-th step: f(x) = [-0.99987153 -0.64661672] (action = 86)
0300-th step: f(x) = [-0.99785708 -0.65160403] (action = 51)
0301-th step: f(x) = [-0.99934904 -0.98060895] (action = 14)
0302-th step: f(x) = [-0.80212129 -0.99622705] (action = 387)
0303-th step: f(x) = [-0.87107548 -0.29634835] (action = 177)
Pareto set updated.
the number of Pareto frontiers = 18
0304-th step: f(x) = [-0.82750022 -0.99996438] (action = 435)
0305-th step: f(x) = [-0.39868067 -0.93742384] (action = 304)
0306-th step: f(x) = [-0.91531622 -0.9999968 ] (action = 398)
0307-th step: f(x) = [-0.99481892 -0.52157619] (action = 192)
0308-th step: f(x) = [-0.94682748 -0.10038325] (action = 155)
Pareto set updated.
the number of Pareto frontiers = 19
0309-th step: f(x) = [-0.97583482 -0.95745348] (action = 99)
0310-th step: f(x) = [-0.44805925 -0.82194903] (action = 222)
0311-th step: f(x) = [-0.9955598 -0.86773446] (action = 254)
0312-th step: f(x) = [-0.98980427 -0.99939737] (action = 425)
0313-th step: f(x) = [-0.88600183 -0.99960175] (action = 230)
0314-th step: f(x) = [-0.95468937 -0.23339717] (action = 135)
0315-th step: f(x) = [-0.82042409 -0.89800614] (action = 301)
0316-th step: f(x) = [-0.99934904 -0.98060895] (action = 294)
0317-th step: f(x) = [-0.85994015 -0.56582803] (action = 238)
0318-th step: f(x) = [-0.99968938 -0.51472752] (action = 87)
0319-th step: f(x) = [-0.28627554 -0.99227074] (action = 348)
0320-th step: f(x) = [-0.72201886 -0.99989894] (action = 334)
0321-th step: f(x) = [-0.87107548 -0.29634835] (action = 197)
Pareto set updated.
the number of Pareto frontiers = 20
0322-th step: f(x) = [-0.99087791 -0.72826929] (action = 234)
0323-th step: f(x) = [-0.66040447 -0.66040447] (action = 200)
0324-th step: f(x) = [-0.99940249 -0.94482572] (action = 12)
0325-th step: f(x) = [-0.99949228 -0.91745532] (action = 11)
0326-th step: f(x) = [-0.99971163 -0.85466737] (action = 9)
0327-th step: f(x) = [-0.58647807 -0.99973531] (action = 333)
0328-th step: f(x) = [-0.99998271 -0.85258693] (action = 84)
0329-th step: f(x) = [-0.80711969 -0.40208972] (action = 198)
Pareto set updated.
the number of Pareto frontiers = 19
0330-th step: f(x) = [-0.3935306 -0.98843647] (action = 247)
0331-th step: f(x) = [-0.99949228 -0.91745532] (action = 231)
0332-th step: f(x) = [-0.95351517 -0.99516257] (action = 384)
0333-th step: f(x) = [-0.81366367 -0.99993226] (action = 314)
0334-th step: f(x) = [-0.83517034 -0.70979198] (action = 159)
0335-th step: f(x) = [-0.82042409 -0.89800614] (action = 161)
0336-th step: f(x) = [-0.82896507 -0.99980725] (action = 272)
0337-th step: f(x) = [-0.95745348 -0.97583482] (action = 121)
0338-th step: f(x) = [-0.99935088 -0.96595598] (action = 273)
0339-th step: f(x) = [-0.38484896 -0.99930674] (action = 352)
0340-th step: f(x) = [-0.96719113 -0.02268252] (action = 154)
Pareto set updated.
the number of Pareto frontiers = 20
0341-th step: f(x) = [-0.58647807 -0.99973531] (action = 393)
0342-th step: f(x) = [-0.99786316 -0.38833643] (action = 150)
0343-th step: f(x) = [-0.99939737 -0.98980427] (action = 315)
0344-th step: f(x) = [-0.98131247 -0.01990033] (action = 133)
Pareto set updated.
the number of Pareto frontiers = 21
0345-th step: f(x) = [-0.70896582 -0.90611507] (action = 302)
0346-th step: f(x) = [-0.99642687 -0.99884734] (action = 82)
0347-th step: f(x) = [-0.9985717 -0.28016277] (action = 89)
0348-th step: f(x) = [-0.99856763 -0.58998965] (action = 170)
0349-th step: f(x) = [-0.23339717 -0.95468937] (action = 305)
0350-th step: f(x) = [-0.10038325 -0.94682748] (action = 265)
Pareto set updated.
the number of Pareto frontiers = 22
0351-th step: f(x) = [-0.91382463 -0.17190645] (action = 176)
Pareto set updated.
the number of Pareto frontiers = 22
0352-th step: f(x) = [-0.99440334 -0.09009546] (action = 131)
0353-th step: f(x) = [-0.99973531 -0.58647807] (action = 47)
0354-th step: f(x) = [-0.88792973 -0.98833743] (action = 164)
0355-th step: f(x) = [-0.99505129 -0.99948501] (action = 424)
0356-th step: f(x) = [-0.96971359 -0.09782226] (action = 134)
0357-th step: f(x) = [-0.7730085 -0.60034647] (action = 239)
0358-th step: f(x) = [-0.62472681 -0.99284468] (action = 227)
0359-th step: f(x) = [-0.96643645 -0.98917271] (action = 362)
0360-th step: f(x) = [-0.99997802 -0.81260125] (action = 64)
0361-th step: f(x) = [-0.99284468 -0.62472681] (action = 73)
0362-th step: f(x) = [-0.97583482 -0.95745348] (action = 319)
0363-th step: f(x) = [-0.83375864 -0.94637208] (action = 162)
0364-th step: f(x) = [-0.94482572 -0.99940249] (action = 428)
0365-th step: f(x) = [-0.86698033 -0.99855947] (action = 208)
0366-th step: f(x) = [-0.98843647 -0.3935306 ] (action = 93)
0367-th step: f(x) = [-0.99552177 -0.97555848] (action = 57)
0368-th step: f(x) = [-0.99980113 -0.99964985] (action = 19)
0369-th step: f(x) = [-0.29032185 -0.97617587] (action = 246)
0370-th step: f(x) = [-0.99959374 -0.99778272] (action = 17)
0371-th step: f(x) = [-0.60034647 -0.7730085 ] (action = 261)
0372-th step: f(x) = [-0.56582803 -0.85994015] (action = 202)
0373-th step: f(x) = [-0.88371066 -0.99999225] (action = 377)
0374-th step: f(x) = [-0.98830423 -0.93616616] (action = 297)
0375-th step: f(x) = [-0.61718446 -0.99986083] (action = 374)
0376-th step: f(x) = [-0.98060895 -0.99934904] (action = 146)
0377-th step: f(x) = [-0.7328647 -0.7328647] (action = 180)
0378-th step: f(x) = [-0.99999957 -0.96467434] (action = 0)
0379-th step: f(x) = [-0.99815304 -0.96875158] (action = 295)
0380-th step: f(x) = [-0.99988107 -0.81419262] (action = 147)
0381-th step: f(x) = [-0.23557331 -0.92045011] (action = 244)
0382-th step: f(x) = [-0.99960175 -0.88600183] (action = 210)
0383-th step: f(x) = [-0.99698104 -0.99468465] (action = 59)
0384-th step: f(x) = [-0.99622705 -0.80212129] (action = 233)
0385-th step: f(x) = [-0.94621941 -0.90531099] (action = 119)
0386-th step: f(x) = [-0.99227074 -0.28627554] (action = 172)
0387-th step: f(x) = [-0.8635332 -0.99999091] (action = 417)
0388-th step: f(x) = [-0.95063572 -0.73057679] (action = 257)
0389-th step: f(x) = [-0.44805925 -0.82194903] (action = 262)
0390-th step: f(x) = [-0.9998078 -0.69972145] (action = 127)
0391-th step: f(x) = [-0.7328647 -0.7328647] (action = 260)
0392-th step: f(x) = [-0.99999878 -0.94307317] (action = 1)
0393-th step: f(x) = [-0.99270087 -0.99270087] (action = 80)
0394-th step: f(x) = [-0.56582803 -0.85994015] (action = 282)
0395-th step: f(x) = [-0.99440334 -0.09009546] (action = 111)
0396-th step: f(x) = [-0.22242321 -0.99728354] (action = 310)
0397-th step: f(x) = [-0.9951763 -0.91838861] (action = 275)
0398-th step: f(x) = [-0.99617304 -0.98813681] (action = 58)
0399-th step: f(x) = [-0.51472752 -0.99968938] (action = 373)
0400-th step: f(x) = [-0.99970416 -0.99908292] (action = 18)
0401-th step: f(x) = [-0.99916062 -0.99852214] (action = 39)
0402-th step: f(x) = [-0.99622705 -0.80212129] (action = 53)
0403-th step: f(x) = [-0.55457668 -0.99911619] (action = 291)
0404-th step: f(x) = [-0.55330867 -0.99949659] (action = 392)
0405-th step: f(x) = [-0.09009546 -0.99440334] (action = 309)
0406-th step: f(x) = [-0.43214402 -0.99936004] (action = 372)
0407-th step: f(x) = [-0.69972145 -0.9998078 ] (action = 313)
0408-th step: f(x) = [-0.99878288 -0.38659518] (action = 69)
0409-th step: f(x) = [-0.99780154 -0.99780154] (action = 380)
0410-th step: f(x) = [-0.65554315 -0.97964062] (action = 345)
0411-th step: f(x) = [-0.73057679 -0.95063572] (action = 183)
0412-th step: f(x) = [-0.53100224 -0.73362301] (action = 221)
Pareto set updated.
the number of Pareto frontiers = 23
0413-th step: f(x) = [-0.99986083 -0.61718446] (action = 66)
0414-th step: f(x) = [-0.72671998 -0.99704052] (action = 388)
0415-th step: f(x) = [-0.86773446 -0.9955598 ] (action = 186)
0416-th step: f(x) = [-0.76244576 -0.99995095] (action = 355)
0417-th step: f(x) = [-0.99202523 -0.99853885] (action = 103)
0418-th step: f(x) = [-0.85258693 -0.99998271] (action = 356)
0419-th step: f(x) = [-0.99017418 -0.09267839] (action = 112)
0420-th step: f(x) = [-0.73057679 -0.95063572] (action = 323)
0421-th step: f(x) = [-0.81419262 -0.99988107] (action = 433)
0422-th step: f(x) = [-0.9999968 -0.91531622] (action = 42)
0423-th step: f(x) = [-0.97964062 -0.65554315] (action = 235)
0424-th step: f(x) = [-0.99945311 -0.72437945] (action = 169)
0425-th step: f(x) = [-0.99653688 -0.43696618] (action = 71)
0426-th step: f(x) = [-0.99998271 -0.85258693] (action = 4)
0427-th step: f(x) = [-0.81629344 -0.99887005] (action = 409)
0428-th step: f(x) = [-0.99997619 -0.79699336] (action = 44)
0429-th step: f(x) = [-0.99852214 -0.99916062] (action = 401)
0430-th step: f(x) = [-0.96971359 -0.09782226] (action = 174)
0431-th step: f(x) = [-0.38833643 -0.99786316] (action = 370)
0432-th step: f(x) = [-0.99939737 -0.98980427] (action = 15)
0433-th step: f(x) = [-0.29634835 -0.87107548] (action = 243)
Pareto set updated.
the number of Pareto frontiers = 24
0434-th step: f(x) = [-0.16481416 -0.98407554] (action = 327)
0435-th step: f(x) = [-0.93616616 -0.98830423] (action = 143)
0436-th step: f(x) = [-0.99993226 -0.81366367] (action = 6)
0437-th step: f(x) = [-0.28016277 -0.9985717 ] (action = 331)
0438-th step: f(x) = [-0.98935582 -0.01711023] (action = 132)
Pareto set updated.
the number of Pareto frontiers = 25
0439-th step: f(x) = [-0.99908292 -0.99970416] (action = 422)
0440-th step: f(x) = [-0.92045011 -0.23557331] (action = 156)
0441-th step: f(x) = [-0.89771579 -0.89771579] (action = 300)
[28]:
plot_pareto_front(res_all)
[29]:
res_all.pareto.volume_in_dominance([-1,-1],[0,0])
[29]:
np.float64(0.30051687493437484)