{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# PHYSBO の基本\n",
    "\n",
    "## はじめに\n",
    "\n",
    "\n",
    "本チュートリアルでは例として、Cuの安定した界面構造の探索問題を扱います。 目的関数の評価にあたる構造緩和計算には、実際には1回あたり数時間といったオーダーの時間を要しますが、本チュートリアルでは既に評価済みの値を使用します。問題設定については、以下の文献を参照してください。\n",
    "\n",
    "- S. Kiyohara, H. Oda, K. Tsuda and T. Mizoguchi, “Acceleration of stable interface structure searching using a kriging approach”, Jpn. J. Appl. Phys. 55, 045502 (2016).\n",
    "\n",
    "---\n",
    "\n",
    "それではサンプルデータを用いて、各手順を実際に行ってみましょう。\n",
    "\n",
    "はじめに、PHYSBOをインポートします。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-12-04T06:02:05.943971Z",
     "start_time": "2020-12-04T06:02:05.507138Z"
    }
   },
   "outputs": [],
   "source": [
    "import physbo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 探索候補データの準備\n",
    "\n",
    "まず、データを読み込みます。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-12-04T06:02:05.950047Z",
     "start_time": "2020-12-04T06:02:05.945622Z"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "def load_data():\n",
    "    A =  np.asarray(np.loadtxt('data/s5-210.csv',skiprows=1, delimiter=',') )\n",
    "    X = A[:,0:3]\n",
    "    t  = -A[:,3]\n",
    "    return X, t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-12-04T06:02:06.105894Z",
     "start_time": "2020-12-04T06:02:05.961463Z"
    }
   },
   "outputs": [],
   "source": [
    "X, t = load_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以下、N: 探索候補の数  , d: 入力パラメータの次元数  とします。\n",
    "\n",
    "X は N x d 次元の行列で、各行は探索候補である各パラメータセット (d 次元のベクトル) を表します。  \n",
    "t は N 次元のベクトルで、各探索候補の負のエネルギー(最適化したい目的関数の値) と対応します。\n",
    "※ 通常、ベイズ最適化を行う際は、Xのみを与え、tは存在しない状況からスタートします。そのため、実際に利用する際には、tの値はベイズ最適化の候補提案を受け、simulatorで評価することではじめて値が得られます。ここではチュートリアルのためにtをあらかじめ与えています。  \n",
    "\n",
    "**PHYSBO では最適化の方向は「最大化」だと仮定します。**  \n",
    "\n",
    "そのため、元々の問題設定は「エネルギー最小化」ですが、PHYSBOで最適化を行うにあたって、目的関数値にマイナスを掛けて「負のエネルギーの最大化」問題として扱っています。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-12-04T06:02:06.115603Z",
     "start_time": "2020-12-04T06:02:06.107365Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0. , 1. , 0. ],\n",
       "       [0. , 1. , 0.1],\n",
       "       [0. , 1. , 0.2],\n",
       "       ...,\n",
       "       [8. , 1.5, 3.4],\n",
       "       [8. , 1.5, 3.5],\n",
       "       [8. , 1.5, 3.6]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-12-04T06:02:06.120704Z",
     "start_time": "2020-12-04T06:02:06.116918Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-1.01301176, -1.01487066, -1.02044168, ..., -1.11680203,\n",
       "       -2.48876352, -2.4971452 ])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "探索パラメータのスケールを合わせるため、X の各列についてそれぞれ、平均が0, 分散が 1 となるように標準化します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-12-04T06:02:06.127129Z",
     "start_time": "2020-12-04T06:02:06.121967Z"
    }
   },
   "outputs": [],
   "source": [
    "X = physbo.misc.centering( X )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-12-04T06:02:06.132603Z",
     "start_time": "2020-12-04T06:02:06.129255Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-1.71079785, -1.46385011, -1.68585446],\n",
       "       [-1.71079785, -1.46385011, -1.59219588],\n",
       "       [-1.71079785, -1.46385011, -1.4985373 ],\n",
       "       ...,\n",
       "       [ 1.71079785,  1.46385011,  1.4985373 ],\n",
       "       [ 1.71079785,  1.46385011,  1.59219588],\n",
       "       [ 1.71079785,  1.46385011,  1.68585446]])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## simulator の定義\n",
    "\n",
    "PHYSBO に与える simulator クラスを定義します。  \n",
    "`__call__` メソッドの返り値が、action を与えたときの目的関数値となります。  \n",
    "action は探索候補の ID (0, 1, ..., N-1) を表します。\n",
    "\n",
    "本チュートリアルでは、action が与えられたときに、既に計算された t の値をそのまま返すだけの simulator を定義しています。  \n",
    "他の問題に適用する際は、simulator クラスをカスタマイズしてください。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-12-04T06:02:06.136850Z",
     "start_time": "2020-12-04T06:02:06.134076Z"
    }
   },
   "outputs": [],
   "source": [
    "class simulator:\n",
    "    def __init__( self ):\n",
    "        _, self.t = load_data()\n",
    "    \n",
    "    def __call__( self, action ):\n",
    "        return self.t[action]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 最適化の実行"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### policy のセット\n",
    "\n",
    "まず、最適化の `policy` をセットします。  \n",
    "\n",
    "`test_X` に探索候補の行列 (`numpy.array`) を指定します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-12-04T06:02:06.149593Z",
     "start_time": "2020-12-04T06:02:06.143075Z"
    }
   },
   "outputs": [],
   "source": [
    "# policy のセット \n",
    "policy = physbo.search.discrete.policy(test_X=X)\n",
    "\n",
    "# シード値のセット \n",
    "policy.set_seed(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`policy` をセットした段階では、まだ最適化は行われません。\n",
    "`policy` に対して以下のメソッドを実行することで、最適化を行います。\n",
    "\n",
    "- `random_search`  \n",
    "- `bayes_search`\n",
    "\n",
    "これらのメソッドに先ほど定義した `simulator` と探索ステップ数を指定すると、探索ステップ数だけ以下のループが回ります。\n",
    "\n",
    "i) パラメータ候補の中から次に実行するパラメータを選択\n",
    "\n",
    "ii) 選択されたパラメータで `simulator` を実行\n",
    "\n",
    "i) で返されるパラメータはデフォルトでは1つですが、1ステップで複数のパラメータを返すことも可能です。\n",
    "詳しくは「複数候補を一度に探索する」の項目を参照してください。  \n",
    "\n",
    "また、上記のループを PHYSBO の中で回すのではなく、i) と ii) を別個に外部から制御することも可能です。つまり、PHYSBO から次に実行するパラメータを提案し、その目的関数値をPHYBOの外部で何らかの形で評価し（例えば、数値計算ではなく、実験による評価など）、それをPHYSBOの外部で何らかの形で提案し、評価値をPHYSBOに登録する、という手順が可能です。詳しくは、チュートリアルの「インタラクティブに実行する」の項目を参照してください。\n",
    "\n",
    "### ランダムサーチ\n",
    "\n",
    "まず初めに、ランダムサーチを行ってみましょう。\n",
    "\n",
    "ベイズ最適化の実行には、目的関数値が2つ以上求まっている必要があるため（初期に必要なデータ数は、最適化したい問題、パラメータの次元dに依存して変わります）、まずランダムサーチを実行します。   \n",
    "\n",
    "**引数**  \n",
    "\n",
    "- `max_num_probes`: 探索ステップ数  \n",
    "- `simulator`: 目的関数のシミュレータ (simulator クラスのオブジェクト) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-12-04T06:02:06.380266Z",
     "start_time": "2020-12-04T06:02:06.154735Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "res = policy.random_search(max_num_probes=20, simulator=simulator())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "実行すると、各ステップの目的関数値とその action ID、現在までのベスト値とその action ID が以下のように出力されます。\n",
    "\n",
    "```\n",
    "0020-th step: f(x) = -1.048733 (action=1022)\n",
    "   current best f(x) = -0.963795 (best action=5734) \n",
    "```\n",
    "\n",
    "\n",
    "### ベイズ最適化\n",
    "\n",
    "続いて、ベイズ最適化を以下のように実行します。\n",
    "\n",
    "**引数**  \n",
    "\n",
    "- `max_num_probes`: 探索ステップ数  \n",
    "- `simulator`: 目的関数のシミュレータ (simulator クラスのオブジェクト)  \n",
    "- `score`: 獲得関数(acquisition function) のタイプ。以下のいずれかを指定します。\n",
    "    - TS (Thompson Sampling)  \n",
    "    - EI (Expected Improvement)  \n",
    "    - PI (Probability of Improvement)  \n",
    "- `interval`:  \n",
    "指定したインターバルごとに、ハイパーパラメータを学習します。  \n",
    "負の値を指定すると、ハイパーパラメータの学習は行われません。  \n",
    "0 を指定すると、ハイパーパラメータの学習は最初のステップでのみ行われます。  \n",
    "- `num_rand_basis`: 基底関数の数。0を指定すると、Bayesian linear modelを利用しない通常のガウス過程が使用されます。  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-12-04T06:03:19.374318Z",
     "start_time": "2020-12-04T06:02:06.382690Z"
    },
    "code_folding": [],
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "res = policy.bayes_search(max_num_probes=80, simulator=simulator(), score='TS', \n",
    "                                                  interval=20, num_rand_basis=5000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 結果の確認\n",
    "\n",
    "探索結果 res は history クラスのオブジェクト (`physbo.search.discrete.results.history`) として返されます。  \n",
    "以下より探索結果を参照します。\n",
    "\n",
    "- `res.fx` : simulator (目的関数) の評価値の履歴。\n",
    "- `res.chosen_actions`: simulator を評価したときの action ID (パラメータ) の履歴。  \n",
    "- `fbest, best_action= res.export_all_sequence_best_fx()`: simulator を評価した全タイミングにおけるベスト値とその action ID (パラメータ)の履歴。\n",
    "- `res.total_num_search`: simulator のトータル評価数。\n",
    "\n",
    "各ステップでの目的関数値と、ベスト値の推移をプロットしてみましょう。  \n",
    "`res.fx`, `best_fx` はそれぞれ `res.total_num_search` までの範囲を指定します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-12-04T06:03:19.620065Z",
     "start_time": "2020-12-04T06:03:19.375626Z"
    }
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-12-04T06:03:19.774131Z",
     "start_time": "2020-12-04T06:03:19.621947Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7fe0d9310580>]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD3CAYAAADxJYRbAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAABCdElEQVR4nO2deXgc1ZX231PVrX21Fku2Zcv7vgsbDF4wYIITSAJZIYFAiAlZHSczX74szGSyswTCfDCEZEjiSYAQmLBvtrENAWwj492WbdnyrtWyrMVaernfH9VVXd1dvUhqdftWn9/z6IEuVavq+lade+57zz2HhBBgGIZhUgMl2TfAMAzDJA42+gzDMCkEG32GYZgUgo0+wzBMCsFGn2EYJoVwJPsGIlFcXCwqKyuTfRsMwzBSsX379hYhRInV7y5qo19ZWYnq6upk3wbDMIxUENHxcL9jeYdhGCaFYKPPMAyTQrDRZxiGSSHY6DMMw6QQbPQZhmFSCDb6DMMwKQQbfYZhmBSCjT6T0qzf34iTrReSfRsMkzAu6s1ZFzP/8/4xnG7rwfdWTIJDTZ2xs+F8D8ryM5J9G3HhdFs37lxbjVGFmXjh65ejKCd9yK/p9nht+7w0nO+By+PFiIJMqArhfLcL6/Y34rU99Wjq6IVTJThUBTfNG4nPXjI62bc7YM53u9DU3oOJw3NjOv9MWzceXHcI31g+AWOKsof47qJjz6cvATy2+Sge23wE33xqB3rdnpi/J4TAf206gmt+sxnnuvqG8A7jzzuHm3HpLzfg/SNnQ37X2tUHj1eugjyv720AADS19+Krf9mOPrd3SK/3wbFWzPz3N7F+f+OQXicZvLz7DBbf+xYW37sRU3/8OpY/sAlVP1uH7/19F2oaOlCck4bsdAeaO3rx4+f34XRbd9LuVQiBbzz5IZ7cemJA3//x83vxqcfehzeG510IgR8/vxd/334KN/9+a1LbrdNvo09EDiL6NhE1x3DuGiLaQkRrIh1LFJ29bnzjyQ8HPZ1v7ujF6bZuzB1dgNf2NuDOP1fjQp876vdcHi++/9we/Pr1Ghxu6sS2Y60Bv+/qdeMnL+3D2c7eQd3fUKG/JK/sORNw/GTrBVz6yw346MPvYMOBRiSyGpvHK7DjxDm0DODf7LU99Zhanof7Pj0LHxw7hx89vyfivfe4PHh62wm8VTMwo7316Fl0uzz4+pMfojqo72Xmya0n8M2ndmBORQF+deNM3H5FJSaV5uJLiyrx/Ncvxz//z5X44+0L8D9fXoi/3LkQAPDIxtqk3e/++na8vLse//biXuw/096v77Z09uK1vfU43+1CfXtP1PPX7W/EhpomfO6SCrT3uHDL77egKYbvDSUD8fQdALYC2B3pJCKqADBGCHEpgOVENMHq2ACuP2C2Hz+Hl3fX48ltAxvhdXaebAMA/GDlVNx70yy8W9uC7/xtZ8TvnDp3AV/64zb8rfokvrp0PJwqYceJtoBzNtQ04Y/vHsOf3w+bNiNptHb1Yf2BRhAB6/c3BXg5L+46gz63F119bnz5z9X4zO/ex4mzidHJH95wGJ989D1U/Ww95v90HW57YhvaLkSfQTWc70H18XNYOaMMH58zEt9cPgHPVJ/CM9UnQ8690OfGH945iiX3bsT3/3cP7nvj0IDu9UBDB4bnpWNkQSbu+NMHONjQMaC/M1jcHi8ON3bgYIP209QxcCP0u81H8IN/7MHSSSVYe8dCfG7BaPzf66bisS/Oxw8/Og1zKgpARMb5Iwsy8bkFFXjmg5MBztf7R87iL1uOJ8RheGlXPVSFUJCVhtV/24Eel3+m3tXrDvgczN+rT8Hl0e7xcGPk/rvQ58ZPXtqPycNz8dNPzMCf71iA5o5e3PKHrWjvccWnMQOg30ZfCNEjhNgCIFrvrADwNBHNB1AKYHGYYwmjtqkTgDatH8zDtetkG1SFMGNEPj5zSQW+eOkYbDrYDLcnUB7ocXnwp3frcNN/vYcrfr0RW4+24t5PzcL3r5uCaSPysePEuYDzP6jTvL+/V59MuFQihIDLE17eeH7Habg8AquWjENDew/2nD5v/O6lXWcwb3QB3vruMvzsEzNQU9+Be17cO+T3XNvUgUc31eLqqaX48cem4dLxRdh8qBlbjkb3ot/Yp0k7180sBwB85+pJmDQ8By/tqg8591tP7cTPXjmA8SU5mD4iL+K/UyRq6tsxe1QB/nzHAmSmqbj1ia1DOqt7eMNhfOmP20KO/+dbtbjmwbdx7UPazxW/3hjTTDWYc119+NXrNbh2+nA8/sUqZKapMX3va8smQFEI//nWYQDa8/PF/96KHz2/F/e+cTDmd9PjFXj7UDO+9dQOLPzFevzi1QM4fyGyMRVC4OXdZ3D5hGI88OnZONTYiV+/XoMLfW78dv1hXPLz9Vj0q7fw3PZTIffh9Qo8te0EppRpWr5uT8Lx2w2HcbqtGz/75Aw4VQXzRhfi4c/PxeGmTrx9KKpQMmRENfpEdCsRrTf9xCrLlALoA3APgFUAhoc5Fny9VURUTUTVzc3x/YfRO6mupQsHo4zSkdh5sg1TynKNh3zWqAL0ur04drYr4LxHN9bi31/aj65eN/7l2snY+L1l+ExVBQBgbkUBdp86HzBQbKtrRW66A/Xne/D24cQ+FH967xgu++UGS4MmhMAz1Scxe1Q+7l46HqpCeHO/ZjQPNXagpqEDN8weAaeq4AuXjsHXrpyATQebsf34uZC/FY13a1vwjx2nop7n9Qr84H/3IivNgV/dNAtfvmIs7vvULBAhJg/61T31mDQ8BxNKcwAAikJYMHYYdp5sCxhwe90evHO4GbdeNgZPrboUE0pzQgb3WOhxeVDX0oUp5XmoGJaFx74wH43tvXhjX+xS0S9ePWCsQ8TChgON2HSwGQfq/RKG1yvw7PZTmD+mEI/eMg+fX1CBPrcXnb39N/pHWzohBPDZSyqQ5ojdfyzLz8AXFo7Bcx+exm/ePIhvPb0D80YX4rNVFfivTUfw0PrDUf9GbVMHlty7Ebc+sQ2bDzVjclkefv/OUSy9fyOe+GddWKdp16nzOHWuG9fPKseSSSX40qJK/PHdY1hy70Y8uP4QlkwsQWVRFr779124+fdbcbTZb9j/WduCE60XcPey8SjOSYto9I82d+K/36nDp+ePwiWVw4zj80YXAtDWkSK3r3PIHL+oPSWEWCuEuNr085sY/3YzgEcA/BBAOoCWMMeCr/e4EKJKCFFVUmKZDnrAHGnqxLjibBABr+2J/eUx4/UK7DrZhjkVBcaxaSPyAAD7gvTBrXWtmD0qH6+vXoKvXzkBFcOyjN/NHV2AbpfHGHzOdfXhYGMH7rhiLIqy0/D0ICWo/uDyePH420fR0tmH5o7Qh3Hv6XbUNHTg01UVKMhKw4LKYVjnW4x8cecZKAR8dNYI4/zbFo1BUXYaHlrfPxmkx+XB6r/txD0v7Iu6SPZM9UlsO9aKH6ycgmJf1E1WmgOjh2XhYGNknba5oxfbjrXiuhnlAcfnjS5EZ68bh5v8g8be0+3odXuxaHwRAEBVCO4BvIyHGjvgFcBUn5c4p6IAw/PS8d6RkFfAkh6XB4+/fRTfenoHdp9qi3q+xyuMZ+u57f5BdGtdK063dePWy8Zg5cxyzB5VAABwe/rfproWTZ4ZW5zT7+9+ddk4OFXCw2/VYumkEvz5jgX45Y0z8en5o/DbDYejav6/eu0gOnpceOTmedj2w6uw9o4FePmbV2D6iDz8x8v78eA662fvpV1nkKYqWDG9DADw/eumYHZFAcYUZePZr16Gx744H89+dRF+/skZ2HvmPK7/z39iwwHtWX9y6wkMy07DR2aUYXxJDg5HMPo7TrTB7RW4a+n4gOMFWU44VUKTxXum03C+Bzf913v46cv7I/4bDJShjN5ZB+CUEGIvgBsBvB/mWMI40tyJBWOH4ZIxw/rlMZk52tKJjl43ZpuM/oTSHKSpCvabPCqPV2Dv6fMBg4MZfcTXdf0PfAt7V0wsxk3zR2HDgSZLrbXH5cEtf9iCh9Yfipv++ea+RtSf167VaLHI9Ez1SaQ7FFw/WzPsK6YPx6HGTtS1dOHFXWewaHwxSnL94Y5ZaQ7cvWw83jncgm11sS9Y/nXrCTR39KKjx43a5vAvVFNHD37x6gEsGDvMmDnpTB6ei5oonv4b+xogBLByZqDRnz9G65MPj7cZx/QF1/ljNG/NqSgDMpA19do9TSnXHAQiwqLxxXj/yNmYokCO+9ZIPF6Bu/5nu+XgbOZE6wX0uLzIcCp4fudpYwb3jx2nkJ2mYsU0zeipiqa3D8zod0JVCKMKM/v93dLcDNzzsem44/KxhjSkKIRf3TQL188egfvfPBhWM99z6jzWH2jEnYvH4aOzypHu0Gbc00fk4y9fXohPzx+FRzfVhsw0vV6BV3bXY8mkYuRnOgEAGU4VL3z9cjx39yJU+TxyRSHcsnAM3vzOEowtycada6tx3xs1WHegEZ+aPwrpDhUTh+fgcGNH2HewyyeXFWQ5A44TEUpy0sOuo3i9At/9+070ub249bIxMf5r9o+4GX0iKiCih/TPQojjALYQ0VYAHUKIfVbH4nX9aJzr6sPZrj5MKM3BdTPLcLCxI2DqFiu6kZ5rMuZOVcGkspyASIAjzZ3o6vNg1qgCWDGqMBPFOWkBRj/NoWDWqHx8pqoCbq/Ac9tPh3zvmeqTeLf2LB5afxir/7azX+Gi4fjTe3VI903Pgz2QHpcHL+w8jetmlBkvyjXTNFXu/jcO4kTrBdwwewSCuWXhGJTkpuM36w7GdA89Lg8e23wE44q1OOZw0lBtUyc+89j76HF78YtPzgxYJASAKWW5ONbSFXEx7rW99RhXko1JwwM91NHDslCUnRZw7Q+OncO44mxjUFNVgtsbXt45de4CfvnaAXzrqR0BMtD++nZkOlWMMc32Fo0vwlnfDC8adS2adPiLT87AuQt9+PqTH0ZcWzjYoD2LX1k8Di2dfXj7UDN6XB68tqcBH5lRbkiTTt+egUhtCsexlgsYPSzL+Bv95eaFo3HP9dMCpCFVIfzHDdOR5VTDyjwPrj+E/Ewnbr+8MuR3RIR7rp+GEQWZ+O4zO9Flkq22nziHhvYew3mJRnl+Jv5+1yKsnFGORzYegccr8PkF2v6CiaW5aO9xoznMmowul+Wkh26FKsnLCDtoP/FuHd6tPYsff2waxpX0fwYVCwM2+kKIq4M+twkhVgcdu18IsVAI8bNIxxKB7jmOL83BR2ZoXs5rPm+/tqkTN/9+C96tjT7V3nWqDbnpDowP6pBp5XnYf6bdGPl3+SJ8Zofx9IkIcyoKseOkZmC21bViTkUB0h0qJpTmYEHlMPztgxMBnkSPy4NHNx5B1ZhC/OtHJuOFnWfwhT9sHVS8/97T5/HBsXP4ku8FCg4ne7e2Be09btw0f5RxbFRhFqaPyMMre+qRpiq41vfvaSYzTcXXlo3HlqOtxvQ4En/ZchzNHb34xY0zUZjlxIcWRn/zoWZ88tF30dnrxlNfWWjo8WYml+XBK8Ivsl3oc2PL0VZcO70sZMAgIswdXWgssHu9AtuPt6KqstA4xxlG3jl+tgtf/Z/tWHLvRvxu81G8uOsMqk1tqGlox+SyXCiK/5qLJhQDAN6z2PcQjG70V84sx69vmoVtda24/83wA2pNQweIgK8sGYdh2Wl47sNTWLe/ER29btw4b6RxnkP1efoDkKyOtnShsigr+on9pDA7DXdcMRav7KkPWI8AtPW0t2qasGrJOORmOC2/n5vhxAOfno3jrRfw81cPGMdf2nUG6Q4FV00NWUoMS2aaiv/8/Fx8/7op+ObyCRjrc0r0Z6+20fo56+p1Q1XIcKbMlOamWxr9/Wface/rB3HNtOH4/IKKkN/Hi5TZnKUbgQklOSjPz8ScigK8vrcBb/sMyXtHzuL37xyN+nd2nmzDrIr8gJcX0Iz+2a4+w1PWBwfdc7Vi7ugCHG3uwum2buw9046FY/0LPp+9pALHzl4IkKGeqT6JhvYefOeaSfjasgl4+PNzsfNkW8SXP5iXd5/Bd/62E8d9i85/eu8YstJU3L10PBQK9fSP+WSF6SPyA47r3v7SySXGDCCYzy8YjcqiLNy5thr3vLA3bJhad58Hj20+ikXji3DpuCLMHV2ID4Mim9btb8Ttf9yGkQWZeP7rlxtySzCTfZp5OIlH30QWzljNG1OAoy1dONfVhyPNnTh3wWVM+wHAoVrLO09uO4E39zfgq0vHY/2aJUhzKEaEkBACNQ0dmFoeuINzZEEmKouy8F4Mzsaxli4U56QjN8OJj88Zic9UjcIT/6zDsZYuy/MPNnSgsigbeRlO3DB7BNbvb8La94+hLC8Dl44r8rfH9xxHmjU0tffg44+8iyOmmbEQAsfPdg1Iz4+FO68Yh9x0R8i60IPrDqEwy4nbFlVG/P7CcUX4yuJxeHLrCVz9m8246oFN+NsHJ3HV1FJL7zsSikL46tLx+O6KycaxiT6jH07X7+r1ICtNDXEsAKAkNz3kPfN6BdY8sxP5WU786sbQGWw8SSmjn+FUMLJA0x9XzizDntPncfufPsDIgkx8Ys4IvFvbEjHkq8flQU19h6VOP81nFHWJZ/ep85g5KnRwMDN3tPZ39GiDBSaj/9FZ5Zg+Ig/f/ttObDzYZHj5CyqHGYuKN8wegSsmFOP9o9E9RZ2nt53EP3acxjUPvo17X6/Bi7vO4KZ5o1CQlYbinPSQqIL6tm5kOBUUBmmTH51ZDodCIZq6mQynihe/eQVuu6wSf9lyHFc9sNlStvnr1uNo6ezF6qsnAQDmjS7AkeaugHj7//7nUYweloXn7l6EUYXhvcvKoiykORRD3gimvVubdocbqIy1lpPn8MEx7V7N0RcOxVre6XV5kZ3uwL9+ZAomlOZiycRivLlP26jW0N6DtgsuTPXp+WYWTSjG1rrWqBFBdS1dGFvsb/f3rp0Mp6rgV6/VWJ5f09CByb40AZ+aPwp9Hi8+OHYOn5g70tDxtfZoJiBSpMire+qx62RbwIytqaMXF/o8AfcUT/KznPjy4rF4Y18j9p4+j44eFx7bfASbDzVj1ZLxMRnu766YhDsuH4tJw3MwpSwP104vw9eWxWdrUEluOvIyHGFnlJ297rD3WJqbjtauvoAd4I0dPahp6MDXlo0f8nQgtjT6mw424cr7N6HVJHvUNnViXHGOYYSvm1GONIeCKyeX4tm7F+H2y8fC5RFGKKLOz1/Zj0c21hoLs26vMCIezOhe3P76dvS6PThQ3x5Wz9eZNaoACgFPbTsBVSHD4ACawfzrnQsxsTQHd63djn99djca2nuw+uqJAV7AJWOH4WhzV8w7UmsaOnDVlFKsmDYcj246gj63F7ct0haMhudloDFogan+fA9G5GeGeB4Th+di+4+uMTz+cORlOPHvN0zHC1+/Aj19HjxlEZW0+VAzppbnGYOe3/C2AdAWl7fWteLjc0YiO8rL7lAVTCzNCevp67ONvDDSwOxRBVAVwvbj51B9rBXFOWkBswKHSpaevtvrNbxmAFgxrQyn27qx70y7fxG3zMLojy9CZ68bu037HqyoO9tlSAuAthB699LxeH1fA7YGDfrdfR4cO9tlzHqmj8gzYsvN0o7eHgDGhiMrNtQ0AQB2nfTf49FmbYZRGWEmO1juuGIs8jOd+MaTH+KyX76FX71Wg8vGFcW8wJnuUHHP9dPw6C3z8cgt8/Dw5+dixsj86F+MASLChNKcgEgvMxf63GGf1dJcLXfV2S7/O3v6nJaeYewQ/nvq2NLoF+eko66lC2/u8xvwI82dARpwxbAsbPvBVfj9rfORk+7ArFH5GFmQiVf3+DfnbDl6Fr9/pw73vXEQX/rjNrzle/jn+Dx0M7kZTowpysL+M+04UN8Bl0dgTkXkBywn3YFJw3Nxoc+DGSPzQx6Sgqw0/PXOhZhclosXd53BgsphuGx8UcA5uiQUy7b+s529aOnsxWXji/D/bp6Hv965EA98ejYmlGoGoTQ31NM/c74b5QXWCdbys6wNpxUzR+VjWE6apUfb5/YiL8Pf9tkV2mCo6/qv7qmHEMD1s8tDvmvF5LLcsLH67d0+ox/G089MUzGtPA8fHm/DB8dbcUnlsIABz6EocHtFSNSG2yMCEqldNbUUCgFv7mvAAd+sQzfCZi7zSS2RJJ6OHheaO3pDDOydi8ehPD8DP3/1QEAE0OGmDggBw9ATEdZcMwm3X16JSUFJwnRPP9xMo7PXja2+zW76TnQAxp6UoTRSeRlOfOPKCTh1rhtXTS3FC1+/HE+tujTqwJ8oJpbmRvD0PWHvUw8KML9rp3xGP9IsNl7Y0uhPH5GHMUVZeNWnh3f3eXC6rTtk4a8gK814oYkIK2eW4Z+1LTjvMwy/XX8YJbnp+OnHp2NrXSse3XQEIwsyjZE6mGnledhf324s4kbz9AFgrs+rXWBaLAy+x798eSFuWTgaP/n49BCPe+bIAqQ7FGyri74RSjeEuvG5fEJxwAJtaV5GSChZfVsPyvP7H5JnhUMhuCxkBI9XGB4nAGSnOzClLM/Q9V/adQZTy/OMwSkaU8py0dTRa7nA3d6jyTvhPH1Ak5e2Hz+Hk63dAXq+3gb9ns24vQJOk6dflJOOSyqH4Y19jThQ34GRBZmWklJRTjqmludFXMzVwzWD14cy01T8y7WTsfvUebywyx/pVRPUzwCwYnoZ/u366SF/W/93Dyfv/PNwC/o8Xlw9tRSn27qNBci6li6kORSMiNOzEY47F4/F3p9ci99+bm7YoIhkMXF4Dlo6+yyfs65eN3LSrXcol+pG36Tr64nYdPl5KLGl0SciXDejHO/VtqDtgrYgJwQsoz3MrJxZDpdHYN3+Rmyra8X7R8/iriXj8MXLKvG/dy/C+JJsXD21NOz3p5Xnoa6lC+8daUFxTjrKY0hBrOv6l1RaL0wCmkf980/OtNSE0xwK5lQUGHH+kdBDA608TkB7GM929RmLei6PF40dPRgRp1TKDkWBx1IaEVCVwEdx3pgC7DzRhhNnL+DDE20xe/mAFsEDWC/m+j398N7ivDGF6PP9G1wSNBg7jBDHYE8/NGXytdO10OB3DjeHLOKaWTS+CNXHz4UNMz3aEl5K+cSckZgxMg+/XX/Y8PYPNnQgw6nElMbXqcs7YYz+WzWNyM1w4I4rxgLwR6XV+SJ3Iq1ZxQMiQoYztvQOiWa8HsFjEfrd1etGdloYeSdPM/rmCJ5T5y6gKDst5lQWg8GWRh/QFhrdXoE39zcaUQfBYZbBzKkoMCSe3244hOKcdNyyUNMPZ4zMx4bvLsO/3xDqLenoO3M3HGjCnIr8mFbgPzarHD9cORVXTgk/mERjwdhh2HfmfNSt9AcbOlCY5URJmIWi4XkZEALG+kBjew+EAMrj5H2E283q9noDvGRA0/W7+jx40Be98bGZscVWA/5drzUWi7n6LC5cuJ9+bQDI8kk9ZsJFu7iCZiuAtokNANouuCz1fJ3LJxShz+3FljAL8nqETqWFEdcjS46dvYD1voXWmoZ2TBqeG7BgGw7VWMgNlXe8XoG3apqxdFIJ5lRoax27fLuBNaOf/NzwycSI4LEI2+zsDa/pF2Xrnr5/Vn3qXPeANrkNBNsa/Rkj8zCqUDPgtU2dUAiojBJpoM0QyrDpYBPerdW8/OCRN5Ih142+2ytiknYAbffqV5aMG/AGF0CbJXgFAmLbH91Ui7XvHws4r6ahA5PLcsO2oTRIa9R36cYyY4kFh0qWxsXtESEGSje8/9hxGrMrCjC6H/HgJbnpKMxyWur67T0u5KY7IhrEUYWZKM1Nx7zRhSHeezg5xO3xwhk0WxlVmIUZI7VnwmqWpqPvaH504xHLHZ51LV0YkZ8R1uP9yPQyjCzIxB/+WQdAG9wnx1jgwz+IhV53z+nzaOnsxdVThyMrTVt/0nMTnTh7AWNLUtvoj8jPRKZTtdT1u3rdyA4j76Q5FAzLTguRd0ay0R8cRISPzizHu7Ut2H78HMYUZRvbtSOxclY5vAIoyk7DLZf2r7pPWV4GhmWnAQi/KWsomDemEAr5UznsPX0e971xEA9v8E/5vV6BQ40dET1Ofdqpp2I4E2edMZynH6zpA8CYIm13LABcPyt2aQfQ+n5ymXU6hvZud9hFXPP3H/vifPzk41YauPbKBBtJq4EL0Awy4HcIrMhwqvjm8gnYdqwVmy2yL9a1dEWMknGoCm6/vBLb6lqx8WATWjr7wkp4wRg7ci2M/oaaJigELJ2k5cCaU5GPXSfbcKatG30eL8amuKevKOEjeLr6wi/kAoEbtIQQOH2uOyF6PmBjow9oaXNdHoH3jpyNKu3ozBlVgGWTS/B/rpuCrDCaXDiIyJADZsUpNCwWctIdmD4iH9vqWiGEwE9f3u+Tafqw94wWZne6rRsX+jwRjcHwPM2j1z0Qw9OP08MYLm+Nx0LT13fHAtqehf4ypSxPS3IWNMi097iQmxG9X+eNLrR8ZsIt5Lq8wtDHzXz5inF44ktVUaNcPnfJaIwqzMR9bxwMuedjQeGaVnz2kgrkpDvw4+e1lNaRBnczRu4dixnYWzWNmD+mEIW6IzOqAO09bmw8qEWxJSK88GJnYmlOiKfv8njR5/YiJ4L9MG/QaunsQ6/bm5DIHcDmRn+2LwwTiL6Iq6MohD/dviDipqNIrJxZjmunDzdelERxSaWWFvjl3fXYWteKNddMAhGwsUbzHHWvNzhkz0xRdhrItCu3vq0buRmOfu9gDIeqkGWUiMtC0weAu5eNw79dP21A0UOTy7RQWD0UTqe92xXV049EOE3f47WufZuZpmL5lOjb/tMcClZfPQn7zrTjdVOo8bmuPrRdcEU1sLkZTnzukgqjvbF7+tYJ1xrO92Dv6faAe9dDlf+xQ4sUYqOvLa7Xn+8J2Gil5/uJ5OmX5Kaj2TejTmTkDmBzo69r9AAwPkH6480LR+N3X6xKyLXMLBhbiF63F99/bjcmDc/B15aNx5yKArzl88oORogV13Goim9Xrk/e8W3MiheOMMnKPGGkkfljhuH2y8cO6Fp6MrUjQZEV7T3uiOGa0XCGid5xeUTA5qyB8Mm5IzGhNAcPvHnQiJuv60c8/Jcur4SqEIqy0wKynkbCESbhmr4Ibs47NLE0F1lpKnacaEN2mhrzNeyMvlNdDxAAIidb0ynNzUBzZy+EEDh1TgvJZU0/TnyqahSG56VHDIm0A3o8eVefBz/86DQ4VAXLJ5di96k2tHT24mBjJ0YVZkb12ktN084zbeE3Zg2E8NE7oZr+YNH3UgRnQdQ8/YHPXFRD3gk0km6Pd1CL8frf/t6KSTjS3GWU9Kzrx87XUYVZuH1RZcxZJAH/zMVqEAOADNM6mKqQsaO1sjh7SPPDyII+azTnlerq1UJvo3n6Lo9A2wWXsRuXjX6cmFKWh60/uHpIt4tfDBTnpGPmyHxcPXW4sfB25ZRSCAG8fagZBxvaY4roGJ6XYSzk1p+P38YswLebNaymH18Doi+otwZtnGnvdg3S07eOdonXwHXt9DIsmVSCX7x6ALVNHTh2tguqQqiIUe/90cemRQwrDsYw+iEL09qgFtwmPe+U3d+nWNGNvpWnnxUmegcI3KB16lw38jIcg3ou+4PtjX4q8cxdl+HRW+YZn6eV56EkNx1v7GvA0eaumHRe3dPvcXnQ2tUXt41ZgGZgLDV9j9dIBxAvstJUZDiVgBq0Hq9AR687bLK1WFAV62iXeMg7gCZJ3v+pWchKc+CbT+1ETUMHRhVm9qscYX/wRyOF7jsAEDJ70fNORcoem0rkWxh9vd5wZHnHv0FLC9dMzCIuwEbfVmSmqQHGQVEIV04uwbr9jXB7RcxGv6Wz11gQjFfkDhC+AInHGx+DaYaIUJSt7TDW6dRTMAxmIVe1jnZxx3HgKs3LwH2fmoUD9e1Yt79xSDdBhU0r4RsEgiOSqioLkeFUwlaESzV0o9/ebZZ3fAu5UaJ3AG2D1ukEbswC2Ojbnisnl0J/n2MJ4yv17crd68v6OCKOmr4jgqavxlnTBzSJxyzv+DNsDlzT1zdgBbfDaq/BYLhq6nB88VJtN/hQRsmEK6Kiz2SCI5KG52Vg+4+uwfJB7CC3E7okEyjvaJp+RE/fFB596tyFhEXuAMDFka6OGTIun1hseHOxGA992qlnVIxr9E4ETT/enj4AFOWk4Wyn3+ifj5JhMxbC1ZR1eQe/kBvMDz86FR09rpBavvFEH8SC5R0995BVKO3FkuXyYiCipx9B089JdyArTcXhRq2saiI9fe49m5OX4cRl44vQdsEVky6sb9DSjX7ZEGv6QghtETTOmj6gefrmvCjRcunHgjOsvBP/gSvDqeKhz82N698MRlEIROHlHau9B4yfNIeCTKdquZAbbXAszU03yqWy0WfiyoOfnROxHJ4ZPRXD/jPtKMpOi2uGQ03TD5VFAAyNp5+dhrNdWiw0ERlVs+IRsmm5kCupgXQqimU0EhAavcOEkpfpCDD6kerjminJTTeqs40suIgXconIQUTfJqLQJCGh5z5MRJuJ6AnTsQYi2uT7md/f6zP9R0vzHJsnUZyTDiJteh/PGH3AutSgblyGQtMvyklHj8uLC32axqpPwQcTvRNuc1Zw5SyZ0KqBBUXv+AaBNEkHskSSn+kMMfrZYerjmjHX5bjYF3IdALYC2B3pJCLKBvCsEGIpgEYiWuz71etCiGW+n+0DuD4zhDhVxUh0Fs8YfcA6n/5QevrBsfqGvBOP6B2PhbwjqVdstWnOkHckHcgSSX6m05hFAtoGyVhSl+gRPFlpKgr6UYVusPTb6AsheoQQWwCEL6qpndclhHjb97EVgB4wvYSI3iGiB4i39F2UlPg8kHjG6AN6GoYwUSJDoOnrg5cettne7QIRIibCioaRe8fC04/3Qm6icKpKyAxMb1+8N83ZEUtPvx9Gf2RBaA3qoSTqU0pEtxLRetPPmv5cgIgmApgnhNjmO7RaCLEY2oxhpcX5q4iomoiqm5ujKkjMEDDcp+uPiHMYmVXCNd3YDIWXXOQrFqNv0GrvcSM33TGoak+OMEVHhmIhN1E4lNBi7y6PF06VONVCDORlOEMWcmMx+nqkXCKlHSCGhVwhxFoAawfyx4loBIB7Adxm+nsv+v73FQBzfP81X+9xAI8DQFVVVcTZBDM06A9jPDdmAXqN3ODslEPnUVp5+oORdgD/4GRe+DQikCT19K32T8Rzs5ndyct0hoRsxiLv6LH6icq5ozPUvfoAgLuEEO0AQETDiGi573dVAGqH+PrMANDDNuMt76gKQQgE5It3J1jTH2x+E7+nH9oGq5h2GXCoiuVCrqxrFIkmP9OJjl638Ux09XqQFUOtW71saSIjd4A4Gn0iKiCih0yfFwBYBuAZX6TODQDaAdxMRJsBTAXwQryuz8SPisIsKARUDIvvw2gV+eL39OPvfwTn39GqZg0uStlqIVeXRoYiAikROFSyXKPgyJ3Y0KPBOnyBAp0xevrjSrKxZFKJkSAxUQz4DRBCXB30uQ3AatPnbQCsthLeOdBrMonhE3NHYmp5nuHxxwvVIs+LK0yOl3gQnH/nfLcrap3kaFjVlNXXJYJr5MqCQ6GQqCqZo5ESjTnTZkFWGi70xabpZzhVrL1jwVDfXghyPqXMkJLmUDBzVPzLPfpzt/u95KHU9IHA/DtxkXdUC3nHyFMjp5F0KBbRO56h2SVtR4IzbXb1Rq6Pm2y4V5mEYbWbdSg1fSAw/05cFnKNkE2/kXR55U5Z4FQpZEeuHr3DRMds9PvcXvR5vMiJkHcn2cj5lDJS4kiwpg/4PX23x4uuPk8cFnItBi6P3Au54UJpZR3EEo0/6Zo7pvq4yYZ7lUkYVrnbXWEqNMULPf9Oe8/g8+4AptmKpbwj5+vkUJXQIioS7ztINPozdb7bFXOytWQi51PKSIkaQdMfOnlHy7/TcF4rATlYT5+I4AzKVWPIO5IaSaflTmnvkFXrshtmeaerL3oBlWTDvcokDEtpJAELuQBw7KxWYHwwydZ0guUQY+CSVANXFcUigRx7+rGS6VThVEkz+kZRdNb0GSaipj9UkSL6rty6Fs3oD3YhFwhNRWxIVJJGuziV0CybfW7W9GOFiIz8O7qmH0ucfrLgXmUSRlI0fd+ux6PNutEf/MsYXOvXWMiV1NPXUiuHevqyticZ5GU40d7j4oVchjGTFE0/SN4ZrKYP6HHtoZuzZPWMreL0OfdO/9Dz73Syp88wfpKq6cdT3gleyJU8ZNMq5bXLw55+fwiWd9jTZxgkR9M38u909UFVCNkxJMKKhhqUitjIvSOr0bcoWC9zfYBkYBh9X5W2WBKuJQvuVSZhJEPT1/PvAEBehiMu+eG1oiN2kncsylhKXPM3Geh1crt63XDEUB83mVy8d8bYjmRo+oBf4omHtAOEGkk7LuT2ebzSylXJIN+k6Wenx8e5GCrY6DMJIxmaPqDl3wHis4gLWMg7XslDNi125HKWzf6Rn+mEVwBN7b0X9SIuwEafSSBWqZWHWtMHzJ5+fF7GYHnHJbmnz7l3Bo++6e/M+e6LemMWwEafSSBWRVTcQ6zpA/6wzXh6+mbPWPf0pV3ItSii4vIIlnf6gf5snWnrvqgjdwA2+kwC8Xv6ZoM59Jq+vkErXkbfqVLQYrTu6cv5OjmV0HKJbg9H7/QH3dNv6ey7qPPuAGz0mQRiVXVqqIuoAPGXd4JDHOXPvUPwBtUudklc6D0ZmIMEWN5hGB9Wmr47AZq+Lu/EI9kaoMsh5ugd2RdyQ9NFcxGV/pEfYPTZ02cYAEnU9HV5J44hm9byjpxG0r9pTusLj1dACHkHsWRgfrY4eodhfETS9NUhjGsePSwLWWkqJpTkxOXvaUVH7LU5C/D3xVBvmLMjuekO6I+w7Tx9InIQ0beJqDmGcxuIaJPvZ77v2Boi2kJEawZyw4y8hNP0FQKUIdb0d//bCiyaUByXv6d5+qG5d2TNPx+8f0I3/mmSDmLJQFHICBSwo6fvALAVwO4Yzn1dCLHM97OdiCoAjBFCXApgORFNGMD1GUkJp+knQkaIpxfuUBXL3DvSGn1d3vF5+ImQ3OyIruvHI7/TUNLvN0EI0SOE2AJARD0ZWEJE7xDRA6TtS14B4Gmf118KYHHwF4hoFRFVE1F1c3PUyQQjEeESrslmXBxK4EKuR/Y4/RB5R+6av8lCjw7Lkt3TJ6JbiWi96ac/ssxqIcRiaLODldAMfR+AewCsAjA8+AtCiMeFEFVCiKqSkpJ+XIq52HFYaPouj1c6Y+lQCB5PYHijU6WLOt9KJPyefqCmz5uz+ofu6V/s8k7UuxNCrAWwdiB/XAjxou9/XwEwB0AzgEcA3AkgG0DLQP4uIydqGE1fNlnEoSoBO1hlLziiRx3psxc3e/oDwpB3LnKjP2S9SkTDiGi572MVgFoA6wCcEkLsBXAjgPeH6vrMxYdVamW3V0CVzGA6lNAiKrJJVGaC11p04y9rCGqy8Hv6NtP0w0FEBUT0kOlQO4CbiWgzgKkAXhBCHAewhYi2AugQQuyL1/WZix81SDsGAI+EFZqCK025vV7pZitm9FmKyxPo6XMahv6hR+9c7J7+gO9OCHF10Oc2AKtNn93QZJzg790P4P6BXpeRFyIKDXf0yqfpO9XQNAwySyHGjtwgTV/mgSwZ5BnROxe30Zf3SWWkJDgXvYyafnAqYtkzUgbPwIyFXIkHsmQwY2Q+KoZloti3A/xi5eIekhjboVWdCtb05TKYTiU0947cnn5QnL7kCeSSxdJJJXjnX5dHPzHJyPukMlIS7CVrmr5cj6GqKBDCvPAp90Ju2DQMki2wM7HBvcokFK3qVGABEtk8fYeRldK/g9UpsYF0BGXZlL3mLxMZeZ9URkqCPX23hJp+8MKn2yOfRGVG9+j98g5r+naGe5VJKA6FQjZnyWYwVSVwB6vbK1/YqRnd03d5gtMwyNsmJjxs9JmEogaVGnR7EpNwLZ44g+UdyYuI6//+Ho7eSQm4V5mE4lSU0I1NknmUhhxiSlAmm0RlJnSNQu6soUxk2OgzCUUN2pwlY8hmcLSL7EXEncaOXPb0UwHuVSahqBaavmwepeEZm+LaZZutmFHVwOynHKdvb9joMwnFYaHpy5ZwLThbqOzyjjOoPW729G0N9yqTUBwWmr5skS+6MdQHL49X7tTKwZWzjELvEreJCQ/3KpNQghOuyazpm7NSyiyFhMu9I3ObmPCw0WcSiq00fVP+eZmlEGfwjlzW9G2NvE8qIyV20PT9ce0mT1+ygctM8I5cf7lEufqFiQ3uVSahBGv6Hgl3szqsFnIl9vRDQ1AFFAIUiQcyJjzyPqmMlIRq+jImXAtcyJW9cpaiEBQyFVGRXK5iIsM9yySU4CIqMiZc8+eq0QYvj+QLuYBe7N0vV7HRty/cs0xCCa4v65FS0w8qL2gDz9ihEDymHbmyD2JMeOR+UhnpUBUlJLWyfJp+YO4d2RdygcCKZi4Jk+AxsdPvniUiBxF9m4iao5w3mog2+X5OEtEnfMcbTMfnD/C+GUlxKiR9ERVzlk0hhC8Ng9xG0qkqpn0H8m2YY2JnIDVyHQC2Atgd6SQhxAkAywCAiF4D8IbvV68LIb40gOsyNsAOmr5qknd071jmwuhAYHEb2XMJMZHpt3sihOgRQmwBIKKeDICIxgE4LYTo9h1aQkTvENEDRBTyZBHRKiKqJqLq5uaIkwlGQsyavtcrIASk0/SNQuJeYQxgquRGUvP0/Zq+7GsUTHii9iwR3UpE600/a/p5jc8CeNr0ebUQYjG0GcPK4JOFEI8LIaqEEFUlJSX9vBRzsRPsUQLy7fz0e/pef2lByQauYLTB2BS9I3l7mPBElXeEEGsBrB3ENVYAuM/09170/e8rAOb4/sukCA5FCanFKpu8Y07D4LZJaUE1YCGXo3fszJAO5z5pp0EI4fZ9HkZEy32/rgJQO5TXZy4+zFEi+n+lW8g1pS3QY9ulX8g1DcYuGyxMM+GJW88SUQERPRR0+OMAXjJ9bgdwMxFtBjAVwAvxuj4jB6pJ0/dIWpZPtfD0ZV/Idaj+BXa3xyt9e5jwDCR6BwAghLg66HMbgNVBxx4M+uwGcOdAr8nIj8NS05fLq3QqoQu5srUhmIAZGO/ItTXcs0xCcfg2Z2nx7ZJr+mZ5R7I2BONQFaM/XBIWq2dih40+k1DMGR2NcEfJDKY5y6ZH0gikYBymOgccsmlvuGeZhOIvwi2vwSQiI/TUqDIleYijuc6BHdJKMOGR+0llpCPA09eNvoQGU1UILq/Xv5Ar2cAVjDmUlj19e8M9yyQUo+qUR15NH9CidQLaILmRdKp+eYfTMNgbuZ9URjqMXPQmL1k2TR/QFz6FYShlHLjMBOyU5ugdW8M9yyQU3cDLrOkD+sKn198GyY2+uYiKi7Ns2ho2+kxCsYumry98Ggu5knvGTlP2U5fHK2WfMLHBPcskFNWk6cvsJTsULSulXRZyzcVt3DYo/8iEh40+k1CcAZq+5iXLqelrWSn9i9Fyv0raQq5/cxZr+vaFe5ZJKGZNX9bUyoA/bYHLJp6+uc4Bx+nbGzb6TEIxFxX3GFk25XsM9bh23dOXcbZiRm+PXv6RPX37wj3LJBRD0w9YyJXPYOoLuX5NX+5XSZ+5GOUfJZ+5MOGR+0llpMNfgMSv6Usp7/jKC8osUZlxqArcHvtEIzHh4Z5lEop1yKZ8BlPzjE0Dl4QSlRm9PXbZbMaER+4nlZEO1TaaPvk8Y3vIIQ6V4BVAn9tX85c9fdvCPcskFIdNNH2nLw2DfXLvaPff4/IAkF+uYsIj95PKSIddNH1VIa2Iik3kEH0Gpht99vTtC/csk1DMIZuyFkYHNDnH7ZV7V7EZ/f67DaMvd3uY8LDRZxKKalrI9Uice0f1afoy7yo2o3v23X0+eUfCPmFiY0A9S0QPE9FmInoiynlriGgLEa2JdIxJHSw1fQm9Sr2mrMsr4FQJRPK1wYzKnn7K0G+jT0TZAJ4VQiwF0EhEi8OcVwFgjBDiUgDLiWiC1bHB3DwjH5aavoReslPfzGSTjJS6ke9x2SMElQmPo79fEEJ0AXjb97EVQG+YU1cAeJqI5gMoBbAY2iATfKzW/CUiWgVgFQCMHj26v7fHXOTYRdNXFcUI2ZRxphKMbuQ5esf+RB3OiehWIlpv+lnjOz4RwDwhxLYwXy0F0AfgHmhGfHiYYwEIIR4XQlQJIapKSkoG1Cjm4sWyiIqEXqXTlGVTxplKMLqR1+WdNI7esS1RPX0hxFoAa83HiGgEgHsB3Bbhq80AHgFwJ4BsAC2+41bHmBRBN/D6jlwiOT19h0rGBjPZY/QBf78YC7k2aBNjzUB79gEAdwkh2iOcsw7AKSHEXgA3Ang/zDEmhfB7+l6fHi6fwQf0IipanL5T0jaYCfb0Wd6xLwNZyF0AYBmAZ4hoExHd4DteQEQP6ecJIY4D2EJEWwF0CCH2WR2LRyMYeTCKqPi8ZBm9fEBbm/DoC7k28IodwZuzJJTcmNgYyELuNgDlFsfbAKwOOnY/gPujHWNSh+AiKjLq+QCgqgSXV8DltclCLqdhSBnkfOMYaTFr+h6JDabTV1PW7fHawit2cpx+yiD/08pIRYCmL3Hki15ExeWRV6IyY2zO6uMsm3aHe5ZJKLqRd3m0qlOyGkyzBm4HrzhU3mHTYFe4Z5mEoigEheTX9M1G0g4G0hkUvWOHiCTGGvmfVkY6HIoivabv9/TllajM+OUd9vTtDvcsk3BUhXyavl3kHflfI6OIipujd+yO/E8rIx0OlXyavrxesu4Jd7s8tjCQjiBPn9Mw2BfuWSbhGBubvELK+rhAoKcv68BlJiThmg3axFgj5xvHSI1q0vRljXzxL+TaI7VycBoGWWU3JjryP62MdDhsoOkb+efdNpF3VL+8Y4eiMEx42OgzCUfPUCmzpq8PVkLYYyOTX96xx8yFCQ/3LpNwHHrVKYk9fbNhlHXgMqN7+n0ery1mLkx42OgzCUf1LeRqmr6cj6DZ0Nshpt2cP4gjd+wN9y6TcLTNWXJr+mZv2A6evhowiMnfHiY8bPSZhKMq8mv65hmKHYykM2AQY7NgZ7h3mYSj1ZeVu4iK+b5llajMEJHRJlnDaJnYkP9pZaRDNW3OktWrdNpM3gH8A5kd1iiY8HDvMglH1/TlTrhmlnfs8RrpmTXtMogx1tjjaWWkQtf0XR6vPeQdSdsQjD54pTnYLNgZ7l0m4ThMmr6sXqVZx1clna0E42BPPyUYkNEnooeJaDMRPdHf84iogYg2+X7mD+T6jNzYIeFaoKcvZxuC0aU2u8hVjDX97l0iygbwrBBiKYBGIlrcz/NeF0Is8/1sH/CdM9Jih4RrAQu5krYhGH2dQtY+YWKj30ZfCNElhHjb97EVQG8/z1tCRO8Q0QPEWZ1SEj3hmsyavkO130Ku4enbZObCWBO1d4noViJab/pZ4zs+EcA8IcS2KN8PPm+1EGIxAAeAlRbnryKiaiKqbm5u7neDmIsfPeGazJq+w44LuRynnxI4op0ghFgLYK35GBGNAHAvgNsifdfqPCHEi77/fQXAHN9/zdd7HMDjAFBVVSWi3R8jH4EJ1+T0Ks1GX9bZSjD64rQdNpsx4Rlo7z4A4C4hRHt/ziOiYUS03Pe7KgC1A7w+IzGqopgSrslpMM2Sjl2MJG/OSg0GspC7AMAyAM/4InBu8B0vIKKHopzXDuBmItoMYCqAFwbdAkY6HArB5fFKnYYhMMumnG0IRjf2dpGrGGuiyjvB+LT5covjbQBWRzsPwJ39vSZjLxwqodft1f5fUgPjsGGCMmNHrk0GMcYaezytjFQ4FDIKcMur6ZvlHXsYSZZ3UgPuXSbhqIpiePqyGkxVIegBx3Yxkk6Wd1ICezytjFSY5QNZNX3AvxNXVokqGL1f7LIwzVjDvcsknIAqTRIbTNVmuWocLO+kBNy7TMJxBsS4y/sI2i1XDadhSA3s8bQyUqEG5KKX18D4NzPJ2wYznIYhNeDeZRKOXYqK++Ude7xGDg7ZTAns8bQyUqHaJIWB02a5ahw2m7kw1rDRZxJOwG5Wib1kvXiKzAOXGSdH76QE3LtMwrFLCgOnsfBpj9eIN2elBty7TMJRzbnoJfaS/dE78rbBjBG9I3GfMNFho88kHLukJVaNzVn2eI2cNgtBZazh3mUSjl00fb8GLu/AZUblOP2UQN43jpGWgJBNiQ2M3XawOjlOPyXg3mUSTsDmLInlHYfdcu+wp58SsNFnEo5dNH3/DlZ522CGE66lBty7TMKxi6avD1gyD1xmeEduaiDvG8dIi100faeqwKkSiORtgxl9bULmgZiJDvcuk3Dso+mTrQykw2ZpJRhr7PPEMtJgJ01f5plKMKzppwYD6l0iepiINhPRE1HOayCiTb6f+b5ja4hoCxGtGci1GflRbaLpOxRF6plKMEYlMBsNZEwo/X7jiCgbwLNCiKUAGolocYTTXxdCLPP9bCeiCgBjhBCXAlhORBMGeN+MxJjlA1ViA6N5+vIOWsGoCnv6qUC/e1cI0SWEeNv3sRVAb4TTlxDRO0T0AGmrXSsAPO3z+ksBRBowGJti1vRlzvMypSwX00fkJfs24sa4kmwU56ShNDc92bfCDCFRjT4R3UpE600/a3zHJwKYJ4TYFuHrq4UQiwE4AKyEZuj7ANwDYBWA4RbXW0VE1URU3dzcPIAmMRc7dtH0Vy0Zjz/dviDZtxE35o4uRPWPrkFBVlqyb4UZQhzRThBCrAWw1nyMiEYAuBfAbVG++6Lvf18BMAdAM4BHANwJIBtAi8V3HgfwOABUVVWJaPfHyIddNH2GkZGBvnEPALhLCNEe7gQiGkZEy30fqwDUAlgH4JQQYi+AGwG8P8DrMxJjF02fYWRkIAu5CwAsA/CMLyrnBt/xAiJ6yHRqO4CbiWgzgKkAXhBCHAewhYi2AugQQuwbbAMY+bBLnD7DyEhUeScYn4ZfbnG8DcBq02c3NBkn+Lz7Adzf3+sy9iEwDQMbfYZJJCyoMgnHLoXRGUZG2OgzCcdhKihul7w1DCMLbPSZhGO3PPQMIxNs9JmEo0s6bPQZJvGw0WcSjsNmeegZRibY6DMJx6g4xTleGCbh8FvHJBzW9BkmebDRZxKObuvZ6DNM4mGjzyQcIoJDIU7BwDBJgI0+kxRUm5UaZBhZ4LeOSQpO1V5VpxhGFtjoM0lBVYhDNhkmCbDRZ5KCQ7FXUXGGkQU2+kxS0Dx9fvwYJtHwW8ckBaeqSF0fl2FkhY0+kxRY02eY5MBGn0kKrOkzTHJgo88kBdb0GSY58FvHJAUHx+kzTFLod41chokHX1s2HnmZzmTfBsOkHAMy+kT0MIDZAI4IIe4Ic85oAGt9H8cD+KYQ4nkiagBQ4zv+XSHE9oHcAyM3188ekexbYJiUpN/yDhFlA3hWCLEUQCMRLbY6TwhxQgixTAixDMBeAG/4fvW6fpwNPsMwTGLpt9EXQnQJId72fWwF0BvpfCIaB+C0EKLbd2gJEb1DRA8QV8VmGIZJKFGNPhHdSkTrTT9rfMcnApgnhNgW5U98FsDTps+rhRCLoUlLKy2ut4qIqomourm5OfaWMAzDMFEhIUT/v0Q0AsAjAG4TQrRHOXcjgGuEEO6g4ysAzBFC3Bvuu1VVVaK6urrf98cwDJPKENF2IUSV1e8GGrL5AIC7YjD44wA06AafiIYR0XLfr6sA1A7w+gzDMMwAGMhC7gIAywA8Q0SbiOgG3/ECInoo6PSPA3jJ9LkdwM1EtBnAVAAvDOSmGYZhmIExIHknUbC8wzAM03+GQt5hGIZhJOSi9vSJqBnA8QF+vRhASxxvRxZSsd2p2GYgNdudim0G+t/uMUKIEqtfXNRGfzAQUXW46Y2dScV2p2KbgdRsdyq2GYhvu1neYRiGSSHY6DMMw6QQdjb6jyf7BpJEKrY7FdsMpGa7U7HNQBzbbVtNn2EYhgnFzp4+wzAMEwQbfYZhmBTClkafiNYQ0RY9I6jdIaKHiWgzET3h+5wS7Sei2UT0mu//U6XNX/D19YtElJkK7SaiHxDR20T0HBE5iOizRLSViO5L9r3FE1/bvu3bn6QfC+nfwfa57Yw+EVVA25hwKYDlRDQh2fc0lIQpamP79hORAuDLAJyp0udElAHgegBXArgR2oYdW7ebiNIAXCqEWALgEIAFAK4XQiwEkGNK4GgHHAC2AtgNWNuyeDzrtjP6AFYAeJqI5gMoBWBZ2csuWBS1WYTUaP9XAPzR9/+p0ucLAXRDq0J3D1Kg3UKIPgBpRDQHwCQA5QBe8ZVjnQJgaRJvL64IIXqEEFsA6NE1Vv076D63o9EvBdAH7aVYBWB4cm8nMehFbaD1qa3bT0RlAEYKIXb4DqVKn5cDyAZwLYCxSJ12vwPgXgBNANIAnAfwGwC3wb5tBqz7d9B9bkej3wytwMsPAaQjBfJ0+Ira3AvgLqRG+28CcA0RbQIwH0An7N9mAOgCsFkI4QWwGYAXNm83EV0CIE8IsQLAUQAeAD8H8DtopVpt12YTVu/yoN9vOxr9dQBOCSH2QtM930/y/SQCc1Eb27dfCPGIEOIyIcQyANsBvAibt9nHdmjTewCYDmAL7N/uMgB61b2zAEoAnAOwHlqb30vSfSUCq3d50O+37Yy+EOI4gC1EtBVAhxBiX7LvaSgJLmoDYDZSqP1A6vS5EOIMgA+I6D0ATiHEZti/3a8BqPQVXroBwJ8A/B7ABwCm+X5vS6ye63g867wjl2EYJoWwnafPMAzDhIeNPsMwTArBRp9hGCaFYKPPMAyTQrDRZxiGSSHY6DMMw6QQbPQZhmFSiP8PgrkiCTjeWLEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(res.fx[0:res.total_num_search])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-12-04T06:03:19.873280Z",
     "start_time": "2020-12-04T06:03:19.775506Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7fe0d93eaaf0>]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD3CAYAAADxJYRbAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAATbElEQVR4nO3dX4xcZ3nH8e/j7DoBk9RA1vmDHIeShApQErABk9YEouD0gnARVLUVNFwApr2oEu0lSFYl6EWpqSIqtyJIUFlqFQG9SUujKKlgkzZZg8OfkBQIFaqpC8RjiPEmxPbMnKcXs+ssm13vznh3J7vP9yONNOfsOXPeN+P95d13nvNOZCaSpBo2DLsBkqTVY+hLUiGGviQVYuhLUiGGviQVMjLsBpzNxRdfnFdeeeWwmyFJa8pjjz12LDPH5vvZSzr0r7zySg4dOjTsZkjSmhIRhxf6mdM7klSIoS9JhRj6klSIoS9JhRj6klSIoS9JhRj6klTIS7pOX1pp//79p/nu/x4fdjOkF7nm0gt577WXL/vrGvoq63Sn4Y57vsOzpzpEDLs10m9677WXG/rScvrWT57h2VMdPvcn27nljZcOuznSqhhoTj8ixiNiMiLGFznusxHxcETsnbXvgxExERH3RsTLBrm+tBwmnmoxsiG44XWvHnZTpFXTd+hHxFZgW2buBG6KiKsWOO7NAJm5C3hTRGyNiAuAW4F3A7dl5vODN106NxM/bLF92yu58ILRYTdFWjWDjPR3A/dExHZgC7BrgeOuBr47/fxrwLXA24HngfuBvQucJ624oydO8l8/O8GNr593IUJp3Vo09CPi9oh4cOYBXAqcphfae4BLFjj1+8CN0893AhcBlwGbgFuA10bE9fNcb09EHIqIQ61Wq9/+SEvy0I+OAXDjNYa+alk09DPzQGbePPMAngb2A58AzgeOLXDe94AfR8T9wCjwK+A5YCIzG2ACeP08592dmTsyc8fYmL+QWhkTT7UYu/B83nDZRcNuirSqBpneeQA4kplPALcBj57l2E9l5i1AAgeBx+hNDwG8EfjRANeXzkm3SR7+UYsbrxkjrNVUMX2HfmYeBiYj4iAwlZlPAkTE5oi4a+a4iLgQ+GpETAD3ZeYvMvOnwDcj4hFgNDO/tSy9kPrw+JHjHP9126kdlTRQnX5m7gP2zdl3HLhz1vYUvbn7ued+EvjkINeVlsPEUy02BPzeVRcPuynSqvPmrAF96yfP8OVDR4bdDA3goadaXLd1M6/ctHHYTZFWnaE/oC/+5//wb9/7Ga8yONacAP74rVcMuxnSUBj6Azp64iRvuWIzX/7TG4bdFElaMpdWHlBr6hRbLrxg2M2QpL4Y+gM6OnWKsQvPH3YzJKkvhv4Afn26w7OnOmy5yNCXtLYY+gM4euIUAGOvMPQlrS2G/gCOTvVCf8tFzulLWlsM/QG0ZkLfOX1Ja4yhP4CjUycBQ1/S2mPoD+Do1ClGNgSvfLk3ZklaWwz9ARw90SvX3LDBFRolrS2G/gCOTp10akfSmmToD6A1dYox78aVtAYZ+gM4OnXKG7MkrUmGfp/a3YZfPnfaG7MkrUmGfp+OPTtzY5ahL2ntMfT7NLMEgytsSlqLDP0+HfVuXElrmKHfpzN34zq9I2kNMvT7dPTEKSLgYj/IlbQGGfp9aj17ile9fCOj5/mfTtLaY3L1aWYJBklaiwz9PrWmTrqOvqQ1y9Dv09GpU1buSFqzDP0+NE1Or7tj6Etamwz9Pjzz69N0mnSkL2nNGij0I2I8IiYjYnyR4z4bEQ9HxN5Z+z4eEQ9FxD9HxMgg1x+WF27Mck5f0trUd+hHxFZgW2buBG6KiKsWOO7NAJm5C3hTRGyNiI3Azsx8J/AUsGPwpq++M9+N641ZktaoQUb6u4F7ImI7sAXYtcBxVwPfnX7+NeDazDwNbIyI64FrgB/OPSki9kTEoYg41Gq1BmjeynEJBklr3aKhHxG3R8SDMw/gUuA0sBfYA1yywKnfB26cfr4TuGj6+cPAp4GjwPG5J2Xm3Zm5IzN3jI2N9dOXFffCF6I7vSNpbVo09DPzQGbePPMAngb2A58AzgeOLXDe94AfR8T9wCjwq4h4K3BRZu4Gfgz8wTL1Y1UcPXGKC88f4WUbzxt2UyRpIIN8kPoA8PuZ+URE/BVw4CzHfiozOxHxj8BB4AagM/2zXwCvGuD6A2t3G2792//g/555fqDzn293ueLVL1/mVknS6uk79DPz8HTlzkHgXzLzSYCI2Az8RWbeOb19IfCViLgA+Hxm/iIi7gP+KCImgGeADyxTP5Zk6mSHH/x8ip2//SrecNlvDfQaN7zu1cvcKklaPQOVTGbmPmDfnH3HgTtnbU8Bt8w5psMqB/1snW4DwHuvvZwP7tw2rGZI0tCUujmr3SQAo+fFkFsiScNRKvRnRvojG0p1W5LOKJV+7W5vpD/iSF9SUaVCv9P0Rvp+AYqkqkqlX2dmpL/Bkb6kmkqFfrvrSF9SbaXSr9M4py+ptlKh37Z6R1JxpdJvZk7fOn1JVdUKfat3JBVXKv2s05dUXanQf2F6p1S3JemMUun3wge5jvQl1VQy9B3pS6qqVPpZpy+pulqhb52+pOJKpV/bOn1JxZUK/Zk6/RHn9CUVVSr92q6yKam4UqFvnb6k6kqlX6dpiIDzHOlLKqpU6Le7yaiVO5IKK5WAnW5jjb6k0mqFfpPO50sqrVQCtruNNfqSSisV+p1uejeupNJKJWDbOX1JxfUd+hExEhF3RERrCceOR8RkRIyfbd9qaTunL6m4QRJwBDgIPH62gyJiK7AtM3cCN0XEVfPtG+D6A+t0G+/GlVTaSL8nZOZJYDIicpFDdwP3RMR2YAuwi97/ZObu++/ZJ0XEHmAPwBVXXNFv886q3U3X3ZFU2qIJGBG3R8SDsx5LnZbZApwG9tIL8UsW2PcbMvPuzNyRmTvGxsaW2o8l6TRW70iqbdGRfmYeAA4M8NotYD/wEWATcGx6/3z7VkWvesfQl1TXSs51PAAcycwngNuARxfYt2p61TtO70iqa9kSMCI2R8RdM9uZeZje3P9BYCozn5xv33Jdfyl6d+Q60pdUV98f5M7IzJvnbB8H7pyzbx+wb7F9q6XTbRg5f+AuS9KaV2quo911pC+ptlKh32kal2GQVFqpBOx0k9GRUl2WpN9QKgHbTcOoJZuSCqsV+p10wTVJpZUK/U5jnb6k2kolYO87ch3pS6qrVOh3vCNXUnGlErDdOKcvqbZSod/pNoxapy+psDIJ2DRJkzjSl1RamdBvNw2AX5coqbQyCdjp9r7oy/X0JVVWL/Qd6UsqrEwCvjC940hfUl1lQv+F6Z0yXZakFymTgO2uI31JKhP6naY30rd6R1JlZRJwZqRvnb6kyuqFvnP6kgork4AzH+Q6py+psjqh38xM75TpsiS9SJkEbM+M9L0jV1JhZULfO3IlqVDotxurdySpTOif+SDX6h1JhZVJwI51+pLUf+hHxEhE3BERrSUcOx4RkxExPmvfZyNiIiK+0O+1z0W7sWRTkgYZ6Y8AB4HHz3ZQRGwFtmXmTuCmiLgqIjYBX8nMG4GnI2LXANcfSMebsySp/9DPzJOZOQnkIofuBu6JiO3AFmBXZj6XmQ9N//yXwKm5J0XEnog4FBGHWq1F/5hYsjNz+iOGvqS6Fk3AiLg9Ih6c9Rhf7JxpW4DTwF5gD3DJrNe8GnhLZn5j7kmZeXdm7sjMHWNjY0u81OLOrKdvnb6kwkYWOyAzDwAHBnjtFrAf+AiwCTgGEBGXA58GPjTAaw7MOn1JWtnqnQeAI5n5BHAb8Oj0/s8AH8vMEyt47RdxlU1JWsbQj4jNEXHXzHZmHgYmI+IgMJWZT0bE24B3AV+KiK9HxPuW6/qLaVunL0mLT+8sJDNvnrN9HLhzzr59wL5Z298ALhv0mufCOn1JKnRz1kyd/ogf5EoqrEzod7oNIxuCCENfUl11Qr9Jp3YklVcm9Nvdxg9xJZVXJgU7XUf6klQn9JvGG7MklVcmBdvddAkGSeWVCf1O15G+JJVJwXaTrqUvqbwyod/pNow60pdUXJkUtHpHkgqF/ulu47dmSSqvTAp2us7pS1Kd0G8c6UtSmRRsO6cvSXVCv9NYvSNJZVKw003X0pdUXpnQb1unL0l1Qt/19CWpUuh30+odSeWVScHe9I4jfUm1lQl9p3ckqVDo+0GuJBUK/d4yDGW6K0nzKpOCvWUYnN6RVFuJ0M/M6WUYSnRXkhZUIgU7TQL4HbmSyus79CNiJCLuiIjWEo4dj4jJiBifs/+6iLiv32sPqtPthb4jfUnVDZKCI8BB4PGzHRQRW4FtmbkTuCkirprevwH4MDA6wLUH0m4aAOv0JZXXd+hn5snMnARykUN3A/dExHZgC7Brev9HgS8udFJE7ImIQxFxqNVa9I+JJTkz0nd6R1Jxi4Z+RNweEQ/Oeowvds60LcBpYC+wB7gkIi4FXpOZ317opMy8OzN3ZOaOsbGxJV7q7Drd3kjf6R1J1Y0sdkBmHgAODPDaLWA/8BFgE3AMeD/wnoh4J3BdRPxZZv79AK/dl/bMB7lO70gqbiWHvg8ARzLzCeA24NHM3J+Z78jMdwGPrUbgw6yRvguuSSpu2VIwIjZHxF0z25l5GJiMiIPAVGY+uVzX6lf7TPWOI31JtS06vbOQzLx5zvZx4M45+/YB+5Zy/krqnKnecaQvqbYSKWj1jiT1lAj99vSc/uhIie5K0oJKpOALyzCU6K4kLahECrbP1Ok7vSOptiKhb52+JEGR0LdOX5J6SqSgdfqS1FMi9K3Tl6SeEilonb4k9ZQI/TN1+o70JRVXIgVn6vSd05dUXY3Qt3pHkoAioW+dviT1lAj9meodvzlLUnUlUrBt9Y4kAUVCv3NmeqdEdyVpQSVSsNM0bAg4z5G+pOJKhH67m87nSxJlQr9h1FG+JNUI/U63caQvSRQJ/XaT1uhLEkVCv9NtvBtXkigT+um6O5JEkdDvTe+U6KoknVWJJOxN7zjSl6QSoW+dviT1lEjCTtNYvSNJDBD6ETESEXdERGsJx45HxGREjM/a98GImIiIeyPiZf1efxCdbjq9I0nAyIDnHAQeP9tBEbEV2JaZOyPiXyPiXuAIcCvwbmBDZnYGuH7f2t6cJUnAACP9zDyZmZNALnLobuCeiNgObAF2AW8HngfuB/bOd1JE7ImIQxFxqNVa9I+JJek0yUZDX5IWD/2IuD0iHpz1GF/snGlbgNP0wn0PcAlwGbAJuAV4bURcP/ekzLw7M3dk5o6xsbGl9uOsesswOL0jSYtO72TmAeDAAK/dAvYDH6EX9MeA54CJzGwiYgJ4PfCdAV67L+1uekeuJLGy1TsPAEcy8wngNuBR4DF60z4AbwR+tILXP6PdtXpHkmAZQz8iNkfEXTPbmXkYmIyIg8BUZj6ZmT8FvhkRjwCjmfmt5br+2XQa6/QlCQar3gEgM2+es30cuHPOvn3Avjn7Pgl8ctDrDsL19CWpp8Tw1wXXJKmnRug31ulLEhQJ/XY3nd6RJIqEvl+XKEk9JZKw3TinL0lQJPQ73YZRb86SpPUf+k2TNIkjfUmiQOi3mwbAr0uUJAqEfqfbWwzUZRgkqVDou+CaJBUI/dPdmekdR/qStO5DvzM9p2+dviSdw4JrL2U/+PkJ/vyfvg30FlsD/I5cSWKdhv4FI+dx9SWvOLN9/dbN/O5VFw+xRZL00rAuQ//Kizfxdx/YPuxmSNJLjhPdklSIoS9JhRj6klSIoS9JhRj6klSIoS9JhRj6klSIoS9JhURmDrsNC4qIFnD4HF7iYuDYMjVnrajYZ6jZb/tcR7/93paZY/P94CUd+ucqIg5l5o5ht2M1Vewz1Oy3fa5jOfvt9I4kFWLoS1Ih6z307x52A4agYp+hZr/tcx3L1u91PacvSfpN632kL0maxdCXpELWbehHxHhETEbE+LDbstIi4rMRMRERX5jeLtH3iLguIu6bfl6lzx+cfq/vjYiXVeh3RHw8Ih6KiH+OiJGI+MOIOBgRfz3sti2n6b7dMX1/0sy+F72/5/qer8vQj4it9G5O2AncFBFXDbtNKyUiNgFfycwbgacjYhcF+h4RG4APA6NV3u+IuAC4FXg3cBu9G3bWdb8jYiOwMzPfCTwFvA24NTPfDrwiIm4aagOX1whwEHgc5s+x5fi3vi5DH9gN3BMR24EtwK4ht2fFZOZzmfnQ9OYvgRuo0fePAl+cfl7l/X478DxwP7CXAv3OzNPAxoi4HrgGuAz4akRcAfwOcOMQm7esMvNkZk4CM9U1872/5/yer9fQ3wKcpveLsQe4ZLjNWXkRcTXwFnrv6brue0RcCrwmM789vavK+30ZsAm4BXgtdfr9MPBp4CiwEfgV8DfAh1i/fYb5399zfs/Xa+i3gP3AJ4DzWedrdUTE5fR+KT5Gjb6/H3hPRHwd2A48y/rvM8BzwERmNsAE0LDO+x0RbwUuyszdwI+BLvCXwOeAU6zDPs8y3+/yOf9+r9fQfwA4kplP0Jv7fHTI7VlpnwE+lpknKND3zNyfme/IzHcBjwH3ss77PO0xen/eA7wRmGT99/tSoDP9/BfAGPAM8CC9Pj8ypHathvl+l8/593tdhn5mHgYmI+IgMJWZTw67TSslIt4GvAv40vTI9zqK9H1Glfc7M38KfDMiHgFGM3OC9d/v+4ArI2ICeB/wD8DngW8Cb5j++bo037/r5fi37h25klTIuhzpS5LmZ+hLUiGGviQVYuhLUiGGviQVYuhLUiGGviQV8v/uHKYPCihLbgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "best_fx, best_action = res.export_all_sequence_best_fx()\n",
    "plt.plot(best_fx)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 結果のシリアライズ\n",
    "\n",
    "探索結果は `save` メソッドにより外部ファイルに保存できます。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-12-04T06:03:19.880149Z",
     "start_time": "2020-12-04T06:03:19.874622Z"
    }
   },
   "outputs": [],
   "source": [
    "res.save('search_result.npz')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-12-04T06:03:19.883409Z",
     "start_time": "2020-12-04T06:03:19.881419Z"
    }
   },
   "outputs": [],
   "source": [
    "del res"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "保存した結果ファイルは以下のようにロードします。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-12-04T06:03:19.892245Z",
     "start_time": "2020-12-04T06:03:19.884981Z"
    }
   },
   "outputs": [],
   "source": [
    "res = physbo.search.discrete.results.history()\n",
    "res.load('search_result.npz')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}