{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# PHYSBO の基本\n",
    "\n",
    "## はじめに\n",
    "\n",
    "\n",
    "本チュートリアルでは例として、Cuの安定した界面構造の探索問題を扱います。 目的関数の評価にあたる構造緩和計算には、実際には1回あたり数時間といったオーダーの時間を要しますが、本チュートリアルでは既に評価済みの値を使用します。問題設定については、以下の文献を参照してください。\n",
    "\n",
    "- S. Kiyohara, H. Oda, K. Tsuda and T. Mizoguchi, “Acceleration of stable interface structure searching using a kriging approach”, Jpn. J. Appl. Phys. 55, 045502 (2016).\n",
    "\n",
    "---\n",
    "\n",
    "それではサンプルデータを用いて、各手順を実際に行ってみましょう。\n",
    "\n",
    "はじめに、PHYSBOをインポートします。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import physbo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 探索候補データの準備\n",
    "\n",
    "まず、以下を実行してサンプルデータをダウンロードしてください。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import urllib2\n",
    "import ssl\n",
    "import numpy as np\n",
    "\n",
    "ssl._create_default_https_context = ssl._create_unverified_context\n",
    "        \n",
    "def load_data():\n",
    "    A =  np.asarray(np.loadtxt('data/s5-210.csv',skiprows=1, delimiter=',') )\n",
    "    X = A[:,0:3]\n",
    "    t  = -A[:,3]\n",
    "    return X, t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "X, t = load_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以下、N: 探索候補の数  , d: 入力パラメータの次元数  とします。\n",
    "\n",
    "X は N x d 次元の行列で、各行は探索候補である各パラメータセット (d 次元のベクトル) を表します。  \n",
    "t は N 次元のベクトルで、各探索候補の負のエネルギー(最適化したい目的関数の値) と対応します。\n",
    "※ 通常、ベイズ最適化を行う際は、Xのみを与え、tは存在しない状況からスタートします。そのため、実際に利用する際には、tの値はベイズ最適化の候補提案を受け、simulatorで評価することではじめて値が得られます。ここではチュートリアルのためにtをあらかじめ与えています。  \n",
    "\n",
    "**PHYSBO では最適化の方向は「最大化」だと仮定します。**  \n",
    "\n",
    "そのため、元々の問題設定は「エネルギー最小化」ですが、PHYSBOで最適化を行うにあたって、目的関数値にマイナスを掛けて「負のエネルギーの最大化」問題として扱っています。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0. , 1. , 0. ],\n",
       "       [0. , 1. , 0.1],\n",
       "       [0. , 1. , 0.2],\n",
       "       ...,\n",
       "       [8. , 1.5, 3.4],\n",
       "       [8. , 1.5, 3.5],\n",
       "       [8. , 1.5, 3.6]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-1.01301176, -1.01487066, -1.02044168, ..., -1.11680203,\n",
       "       -2.48876352, -2.4971452 ])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "探索パラメータのスケールを合わせるため、X の各列についてそれぞれ、平均が0, 分散が 1 となるように標準化します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = physbo.misc.centering( X )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-1.71079785, -1.46385011, -1.68585446],\n",
       "       [-1.71079785, -1.46385011, -1.59219588],\n",
       "       [-1.71079785, -1.46385011, -1.4985373 ],\n",
       "       ...,\n",
       "       [ 1.71079785,  1.46385011,  1.4985373 ],\n",
       "       [ 1.71079785,  1.46385011,  1.59219588],\n",
       "       [ 1.71079785,  1.46385011,  1.68585446]])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## simulator の定義\n",
    "\n",
    "PHYSBO に与える simulator クラスを定義します。  \n",
    "`__call__` メソッドの返り値が、action を与えたときの目的関数値となります。  \n",
    "action は探索候補の ID (0, 1, ..., N-1) を表します。\n",
    "\n",
    "本チュートリアルでは、action が与えられたときに、既に計算された t の値をそのまま返すだけの simulator を定義しています。  \n",
    "他の問題に適用する際は、simulator クラスをカスタマイズしてください。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "class simulator:\n",
    "    def __init__( self ):\n",
    "        _, self.t = load_data()\n",
    "    \n",
    "    def __call__( self, action ):\n",
    "        return self.t[action]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 最適化の実行"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### policy のセット\n",
    "\n",
    "まず、最適化の `policy` をセットします。  \n",
    "\n",
    "`test_X` に探索候補の行列 (`numpy.array`) を指定します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# policy のセット \n",
    "policy = physbo.search.discrete.policy(test_X=X)\n",
    "\n",
    "# シード値のセット \n",
    "policy.set_seed(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`policy` をセットした段階では、まだ最適化は行われません。\n",
    "`policy` に対して以下のメソッドを実行することで、最適化を行います。\n",
    "\n",
    "- `random_search`  \n",
    "- `bayes_search`\n",
    "\n",
    "これらのメソッドに先ほど定義した `simulator` と探索ステップ数を指定すると、探索ステップ数だけ以下のループが回ります。\n",
    "\n",
    "i) パラメータ候補の中から次に実行するパラメータを選択\n",
    "\n",
    "ii) 選択されたパラメータで `simulator` を実行\n",
    "\n",
    "i) で返されるパラメータはデフォルトでは1つですが、1ステップで複数のパラメータを返すことも可能です。\n",
    "詳しくは「複数候補を一度に探索する」の項目を参照してください。  \n",
    "\n",
    "また、上記のループを PHYSBO の中で回すのではなく、i) と ii) を別個に外部から制御することも可能です。つまり、PHYSBO から次に実行するパラメータを提案し、その目的関数値をPHYBOの外部で何らかの形で評価し（例えば、数値計算ではなく、実験による評価など）、それをPHYSBOの外部で何らかの形で提案し、評価値をPHYSBOに登録する、という手順が可能です。詳しくは、チュートリアルの「インタラクティブに実行する」の項目を参照してください。\n",
    "\n",
    "### ランダムサーチ\n",
    "\n",
    "まず初めに、ランダムサーチを行ってみましょう。\n",
    "\n",
    "ベイズ最適化の実行には、目的関数値が2つ以上求まっている必要があるため（初期に必要なデータ数は、最適化したい問題、パラメータの次元dに依存して変わります）、まずランダムサーチを実行します。   \n",
    "\n",
    "**引数**  \n",
    "\n",
    "- `max_num_probes`: 探索ステップ数  \n",
    "- `simulator`: 目的関数のシミュレータ (simulator クラスのオブジェクト) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "res = policy.random_search(max_num_probes=20, simulator=simulator())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "実行すると、各ステップの目的関数値とその action ID、現在までのベスト値とその action ID が以下のように出力されます。\n",
    "\n",
    "```\n",
    "0020-th step: f(x) = -1.048733 (action=1022)\n",
    "   current best f(x) = -0.963795 (best action=5734) \n",
    "```\n",
    "\n",
    "\n",
    "### ベイズ最適化\n",
    "\n",
    "続いて、ベイズ最適化を以下のように実行します。\n",
    "\n",
    "**引数**  \n",
    "\n",
    "- `max_num_probes`: 探索ステップ数  \n",
    "- `simulator`: 目的関数のシミュレータ (simulator クラスのオブジェクト)  \n",
    "- `score`: 獲得関数(acquisition function) のタイプ。以下のいずれかを指定します。\n",
    "    - TS (Thompson Sampling)  \n",
    "    - EI (Expected Improvement)  \n",
    "    - PI (Probability of Improvement)  \n",
    "- `interval`:  \n",
    "指定したインターバルごとに、ハイパーパラメータを学習します。  \n",
    "負の値を指定すると、ハイパーパラメータの学習は行われません。  \n",
    "0 を指定すると、ハイパーパラメータの学習は最初のステップでのみ行われます。  \n",
    "- `num_rand_basis`: 基底関数の数。0を指定すると、Bayesian linear modelを利用しない通常のガウス過程が使用されます。  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "res = policy.bayes_search(max_num_probes=80, simulator=simulator(), score='TS', \n",
    "                                                  interval=20, num_rand_basis=5000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 結果の確認\n",
    "\n",
    "探索結果 res は history クラスのオブジェクト (`physbo.search.discrete.results.history`) として返されます。  \n",
    "以下より探索結果を参照します。\n",
    "\n",
    "- `res.fx` : simulator (目的関数) の評価値の履歴。\n",
    "- `res.chosed_actions`: simulator を評価したときの action ID (パラメータ) の履歴。  \n",
    "- `fbest, best_action= res.export_all_sequence_best_fx()`: simulator を評価した全タイミングにおけるベスト値とその action ID (パラメータ)の履歴。\n",
    "- `res.total_num_search`: simulator のトータル評価数。\n",
    "\n",
    "各ステップでの目的関数値と、ベスト値の推移をプロットしてみましょう。  \n",
    "`res.fx`, `best_fx` はそれぞれ `res.total_num_search` までの範囲を指定します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x1a1c5ae050>]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAD8CAYAAABzTgP2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xd8W/W5+PHPI8l2vOKVxLHjxM4O2cPZ7ISVUgIUWiilYbThtoVCW3op7b23ve2vt73QQge9QFqgQCmUFUYYIcxAgOyQOHsnjp3EM17xkPT9/aFhyZYsJZLjRHrer5dfto6OpK8snfOc5/t8z/eIMQallFLKw9LTDVBKKXV60cCglFLKjwYGpZRSfjQwKKWU8qOBQSmllB8NDEoppfxoYFBKKeVHA4NSSik/GhiUUkr5sfV0A05Gnz59TFFRUU83Qymlzihr166tNMb0DbXeGRkYioqKWLNmTU83Qymlzigisj+c9bQrSSmllB8NDEoppfxoYFBKKeVHA4NSSik/GhiUUkr50cCglFLKjwYGpZRSfjQwRNl7W4+wt7Kxp5sRlhW7Kik/drynmxFztpTVcfhYc6flm0qP8fmeqh5o0YlrsTtobLH3dDPOWKv3VfPYJ3v5aEcFZbXHOdMuoRxxYBCRa0Vks4g4RaS4i/UuFZHtIrJLRH7is3ywiKwUkZ0i8i8RSYy0TT2lprGVW59cwxUPfXLCOwBjDP9ceYBHP9rdTa3zV3LoGN94bCV/em9Xt71GY4udV9YfwunseqOwO5zd1oZTbeeReq76vxXcv3R7p/v+9+1tfONvK/l0V2UPtCx8Tqfh5idW85WHPz3jdminA7vDye3/XMevlmxhweOrmPXb97n2kc9otZ853/NoZAwlwNXA8mAriIgV+AtwGTAauF5ERrvv/l/gQWPMcKAGuDUKbTohZbXHeeyTvRFvBOsP1gCQYLXwzcdW8foXZWE9rrnNwd0vbOSnizdx/9LtHG91+N1/tL457OcKhzGG/359M8bAlvK6qD1vR3/5YBd3/WsDH2w/GnSdbYfrmPyrZTz92b5ua8ep0tzm4PvPbaDF7uRIXeeMobKhBbvTcNs/1rLjSH0PtDA8L6w9yKe7q9h2uJ71B2uDrud0Go4GeJ/rD9Rw0xOr2Fx2rDubedp6d+tRjtS18LtrJ/CvhTP44UUjWLO/hj+8u6Onmxa2iAODMWarMabz4ZG/acAuY8weY0wr8BwwX0QEuBB40b3ek8CVkbbpRD23+iC/WrKFdQdqInqetftrsFqEN75/NhMHZnLHs+t5LcQO/WB1E19b9DkvrSvlwlH9sDsNGzpsjI9+tIc7nl3PptLobGivbyxn9b4a8jN6sf1wHY4QR/Qno7HFzj8+d519/8+VBwKu43Qafra4hLpmO79aspWSQ2f2juS+t7eztbyO3N5JVDW2drq/sqGV80b0pVeClZufWM3R+s471e7y0Ps7+csHu9h1tOuAVFHfwq/f2MrkQZkkJ1h5YU1p0HX/sXI/03/zHk99ts+7bPvhem56YjUfbq/ga49+zoog2ZExhsc/2cvDH+7m5XWlfLqr8pT+Pxpb7KzdX83Tn+/nD+/uoDrA53Wynlm5n7yMXlw5MZ/pQ3L4/pzhfK14II98tJs1+6r91j3W1Ma6AzW8sOYgL6/r/L/eeaSeC3//IX9+bydNraeua+9UzZU0ADjoc7sUmA7kALXGGLvP8gGnqE1eO91Hb69tKGNKYfZJP8+afTWMye9NXkYyT906jUv/sJzF60q5YkK+33qVDS28uLaUt0sOs+FgLamJVh69cQrTB2cz8ZfLWLu/mplDc7zrf7rb1S319Of7uO+aCSfdPoCmVjv/88ZWxuT3ZsHMIv79pY3srWxkWL+0E3qeF9YcZPH6Qzx+01R6JVg73f/8moPUNds5d0Rf3t9+lNKaJgqyUvzWeXFtKWv31/DTeaN4YsU+7nh2Pa/fcTZpSafPFF4V9S0seHwVv7pyLFMKs/zuKz92nPpmO8kJVjaXHePxFXtZMLOQ5jZnpyzJ6TTUNLUydkBvfnzJSL766Gd875l1PH/bTFzHR91nb2Ujv3vHdbR6/9LtDOmbys+/PIbzRnSeS+2XS7bQ3Obkvmsm8H8f7GLJF2X8/MujA37GK/dWYwz816ubKT/WzPVTB3HjYytJsll44d9m8h+LS7jpiVXcf80Erpzkv1mvO1DDL5ds6fScub2TGDcgg7vmjmDsgIwo/QfgQFUTD767g/1VjRysOU5FfYvf/Us2lvOPW6fTP6MXAPsqG3lixV5uO28o+ZnJYb/OvspGPt5ZyQ/mjsBmbT/u/s8vj+bTPZX88PkvePPOc1i9r5qHP9zNqr3+gWLyoCyK+qR6by/dfJg9FY38ftkOnvp8P3fOGc7Xpg4kwdq95eGwnl1E3hWRkgA/88N8nUDffNPF8kBtWCgia0RkTUVFRZgvG57t7sDwxqbyk+7vbnM4+aK0lsmDXDuPXglWphRms+nQsU5dVLf/cx2/fWsbDqfhx5eMZOkPzuWSMf3JTElkRG4aq/e1Zy41ja1sLa8jJdHKqxvKqG2K7Mjm4Q93c7iumV9cMYYxA3oDsLVDd9LW8rqQBfQvSmv5dHcV973dOVm0O5w8vmIvxYVZ/ObqcQjw3KqDfuvUNLbym7e2MrUoi2+dPYQ/fG0i+6sa+a9XSyJ6f9H2+hdlbCmv4/6l2/yWl9Y0ccHvPuTiB5dzzn0f8G//WMfI3HTunXcW2WmJ1DS1+n3udc1tOJyG7NQkxg7I4N55Z7F6Xw0rO+wYusPbJYcBeOk7s/jV/DHYHYafv1rSqfbzwfajvP5FGd+9YCjD+qVxTXEB9S12lm4+HPB5Nx86xtyzcvn69EE8/OFuLvnDclrsTp6+dTpTi7J5/t9mMqUwi7v+taFTze3VDWUk2Sys+ukc3v/RefzzW9P5z8tHM3NIDqv31fAfr3T9PThUe5yD1U1h/w8WfbybJRvLSLJZuWBkX+6+eAR/+2Yxn/7kQp5bOIPDx5q55pFP2VPRwKLlrvfy5Gf7eSfAe/+/D3dx+z/X8d1n1nLb02v8egWeXXUAq0X42tSBfo9JS7LxwFcncrCmidm/fZ+bn1jNweomfnjRCB5bUMyTt0wDYOVe///Tqn01jMxN56XvzKQoJ4X/eKUk6OcRTWEFBmPMXGPM2AA/r4b5OqWA73+qACgDKoFMEbF1WB6oDYuMMcXGmOK+fUPOGhu25jYH+6uaGJGbRmVD60lvqFvL62huc1Jc1H5UOW5AbyobWjns0w/b3OZg7f4abjt3CK/fcTbfu2CY35F0cVE26w7UeLt3PBvUT+edRYvdyYtrg6f2oWokh4818+jyPVwxIZ+pRdkM65eGzSKdAsNtT6/l+8+u7/K5PO17fMVePt3t312wdPMRDlYf51vnDGFAZjIXjurHc6sP0uYTdH/71jZXF9KVY7FYxJtyv7zuEG9uKu/ytZ1O0y3dX4Es2ViG1SJ8vqea1T7dAL9/ZwfGwANfncDvrp3A/1w1jqdvnUavBCs5qYm0OQx1ze2pv6drKSfVNbbi2ikFZKUk8LeP955wm6obW/n5qyUca2oLa/23Nx9mQkEGUwqzuHFmEXdfMpJ9VU18tKP9AKvN4eS/X9vM0L6pfOf8oQDMGJzDgMzkgN+5uuY29lU1MXFgBr++cix3XzyC3sk2nrh5KiP7pwOQkZzA32+eRnZqIo9/stfvtd7YWM7c0bn0692LIX3TmDWsD7eePZg/XDeJO+cMZ8PB2qBdix/tqODiBz7i5r+vDuv9t9qdLNlYzqVj83h24Qzuu2YCt184nLmjc8nPTGbGkBz++e3pNLTYmfvAR/zPm9s4Z3gfRKCmw//Y4TT8bul2VuyqZOeRBkoO1fH9Z9fzx3d30tzm4Pk1B7norFxv5uFralE2d188koHZydx/zXg++vEFfH/OcOaclcu5w/vQJy2RlXuq/V5r3f4aiouymFKYzfO3zeTpW6dx2di8sN53JE7VcNXVwHD3CKRE4DrgNePak30AXONebwEQbrCJij0VjTichm+fM4TURCuvbTi5Iu/a/a6jfN/uhnEFrlTYtzZQcugYbQ7TqVvCo7gwi/pmu7c4+enuKlISrXxt6kCKC7N4+vP9AUf5bDhYy9Rfv8ffVwQvov/lg1043VkKQJLNyrB+aX4F6ANVTRyobmLToWMcqAp+RGZ3GHJSExnSJ5W7n/+CumbXBmSMYdHHeyjKSeGi0bkA3DC9kMqGFpZtOYLd4eTBZTv415qD3Hr2YEb17+19zjsuHM7wfmn86b2dXQa5ax/9jPG/WMo3/raSB5ft4NlVB/jr8j08sGwHz606QHObI+hjT8Sh2uOsO1DLd84bSk5qIg+97xrBVXLoGIvXH+KWswdz9eQCrplSwNenD6Jfb9fOICfNtfOvamjvrvD0YWe7A0OvBCs3zijkvW1H2FPRcELtemLFXp78bD+PrwgdVMpqj/PFwVouGdvfu+yysf3J7Z3E3z/d5132r9UH2VfVxE/nnUWSzdVtZLEIX5lSwCcBhjVvPuT6zowZkIGIcPuFw/n83jnejNmjV4KV66YO5N2tRyitcX2fVuyqpKqxlfkdulg9vjKlgF4JFp5Z2XmG6OdXH+SWv6/GYQy7jjaENdx6+Y4KapvauGpS4NcDGF+QyQu3zeTs4X358/WT+Os3i+ndK6FThl53vA2ncX1Xl/3wPD64+3yunjyAB9/dwbWPfEZNUxs3zBgU9HW+d8EwltxxDtcWDyTR1r77FRGmDc72OzDdWl5HQ4udaYOzveucM7wvVkv3dj1CdIarXiUipcBM4A0RWepeni8ibwK4awi3A0uBrcDzxpjN7qe4B/ihiOzCVXN4LNI2nYid7mLc+IJMLh7Tn7dKyv2GlYXbtbRmv6uYm5fR3h85Oi8Di+B35OMpcE8aFDgwTC1yfQk8RarP9lQxbXA2CVYLN84sZH9VE8t3+nelGWP4nze2UtXYwi9e38I9L22kxe6/czxUe5znVh/g2uKBDMxuz1BG5/X2yxg+8SkWvlkS/Mjd4TSkJFn5/VcncKS+he/8Yy2/WrKFO5/bwBcHa7n1nCHeL/C5I/oyIDOZRz/azTWPfMYf39vJVZMG8IO5I/ye02oRvn3uELYdrufjnYGLllvK6li7v4ZxBRlUNbbyp/d3cu/Lm/j1m1v503s7+cnLm5j92/f583s7Q3a7rd5XzUUPfBR0vP6bG13v/9riAm49ZzAf7ajgi4O1/OatrWSlJHiPrDvKTk0C8CtoeoKEJ2gA3DiziASLJawdvEeL3cGzq1zF/Kc/399pBFtHnq6QS8e0B4YEq4Ubphfy0Y4Kdlc00NRq54/v7WRqURYXjurn9/hrJhdgDLy87pDfcs+Io7H57XWAYLWSG2YUAvCMexDCaxvK6N3LxnkjA2f+GckJzJ8wgFfWl3kPOMB1YPPvL21k1tAcnr51OgArdoUeFr54wyGyUxM5Z3jXPQ3Dc9N56pZpfHlCPiJCVkoC1R0yhuom/wCfaLPw+2sncMeFw9h06BhFOSnMHtonZJsCmT44x6+LzFN/8OwTTqVojEpabIwpMMYkGWNyjTGXuJeXGWPm+az3pjFmhDFmqDHm1z7L9xhjphljhhljrjXGtAR6ne6y40g9NoswuE8qV0zIp67ZzvIdFdgdTn79xhbG/mJpp66WQNbtr2FyhywgOdHK8H7pbPINDPtrGZidTN/0pIDPU5CVTL/0JNbsr+FoXTO7jjYwc4irEH3p2P70SUv0jvbx+HBHBav2VfPzy0fz/QuH8fyaUq5f9LnfkMmH3t+FINx+4TC/x56V15sjdS3endiKXZX0792LCQUZvLExeGBocxpsFguTBmVx98Uj+XxPNc+tOsDqfdXMHJLDNZMLvOtaLcLXpw/ii9Jj7K1s5KGvT+LBr00kObFzQXP+xHz6pSexaPmegK+7eH0pNovwfzdM4a07z2Hjzy/m059cyKZfXMye/5nHcwtnMK4gg98v28HcB5Z3OWTyg21H2Xm0gdKawEedSzaWMW5ABoU5qdw4o5CM5ATufG49K3ZV8f05w+ndKyHg4zzdRb4jk9q7kto/977pSVw5KZ8X15ZSE2RUTMex729tOkxlQyu3XzCM6sZWXgwwksVv/ZLDjMxNZ0hf/8EF108bRKLVwlOf7uOJFfuoqG/hnktHddq5D8pJYfrgbF5aW+qXxZUcOkb/3r2Cfo99DchM5qLRuTy36gDHmtpYuvkw88bleTOTQL4xo5DjbQ5edndjPbfqAPcv3c5Vkwbw+E1TmTIoi5zUxJDnhNQ3t/HuliNcPj7vhAu2WamJnQ4uPJ9TVmp7gBcRfnTxSB5bUMwfr5uE5SSP6KcPcQUAT9awZn81AzKTT6j4HS1xf+bz9sMNDO6TSqLNwuxhfchMSeCZlfv5xmMr+evHe0P264MrXS8/1hywe2jsgAw2HarDGIMxhnUHajql275EhKlF2azZV8Nn7vrCLPcRSJLNynVTB/HetqMs2ejq8nI6Dfe9vZ1B2Sl8fXohP7x4JA/fMJlth+v50p8+YeWeKg5WN/HCmoNcN20gAzp8yc7Kay9AO52GT3dXMntYH740Pq/L7iSH0+nNCL5z/lB2/r/L2PzLS/ns3jk8u3BGp53+N2cWcvfFI1h617lcPj54Sp9ks3Lz7MF8squyUx+z3eHklQ1lXDCqn/eILb1XAvmZyaT3SsBiEWYMyeHvN09jyR1nk2gVrlv0eaeRHx7bD7uyxZoAmcWBqia+KD3G5ePzvK9z8+wi9lU1UZiTwg3TC4O+B09W4JsxVDd4dij+weRb5wyhuc0ZsNvkzU3lTPrlO37Dl5/8bB9D+qTyw4tGMGFgJn/7eE/QektlQwur91X7dSN59E1P4vIJeby4tpRHPtrN3LP6URzkyHT+xAHsqWxk2+H2oa6bDh1j7IDeAdcPZMHMImqa2vjB8xtobHVwxcTg3wFwdcNOKMjgHysP8OH2o/zslRLOHdGX+64ZT4LVgsUizByawye7Krvsdny75DAtdmenUVHhyEpJ7PTd8HYJpnQ+D3fOWblMGJh5wq/jMaJfOpkpCazcU4UxhlV7a7zdSKdaXAWG+uY2by3AY+fRekbkuopliTYLl43N44PtFaw/UMsDX53AxaNzef2Lsk4bX0V9i7ev3/OcxQGGuroK0C0cqWuh7FgzR+tbugwM4KpTHKo9zkvrDtG7l43R+e0b4G3nDaG4MIs7nl3Ps6sO8PrGMraW1/Gji0d4+ywvG5fHK9+bTe9eNr7+t5V8+6k1WCzCd88f1um1zspzvfctZXVsKa+jpqmNs4fneAtcwbqT7A6DzefIKNRRUnqvBG6/cHjAolxHX58+iNREa6esYcXuKirqW/jK5NAb+dgBGbzwnVn0TU/ixsdW8v62I53W8YxGC9TltGSTK/B+aXx7oe/mWYMZX5DBL748xq9/uCNP0PKtMVQ1tpKeZOt0lDwiN51zhvfh2Q6jtsAVuBpbHSx8ag2HjzWzsbSW9QdquXFmIRaLcNu5Q9hf1RRw5AzAu1uO4DT+3Ui+bp41mMZWBw0tdu52150CuXhMLhZxZR/gOgdgT2XjCQ0nnTk0h2H90nh/21FyeycxfXBOyMfcMKOQXUcbWPjUWkbkpvN/N0z2O+qfPawPR+tb2N1FjebVDWUU5qQw6SR22JkpCdQ0+ncleQJFxwAfDRaLMK3IVWfYV9VEZUNLj3QjQZwFhp+/uplb/r7am54fb3VwoLrJGxgAbpldxJxR/Xj5u7O4enIB8ycO4Gh9i99wuw0Ha5nxm/dY8MQqjtY1s3Z/DckJVkblpXd6TW8B+tAx1rkDSKjA4PkyLN9RwfQhOX7FpvReCTx1y3TOHd6Xe1/exH+8UsJZeb35coej8BG56bx6+2wuOiuXbYfruWH6oIA75Zy0JHJ7J7G1vM57MtKsoX0YmJ3ChIKMoCOEHE7TbUWwjOQEvj59EG9sKvcWLAFeXldKRnICF3ToBw9mQGYyL9w2k2H90rjjn+v9umUaWuzeLqSOI08AlnxRzqRBmX4jxjJSEnjt9rNDvn6SzUpakq1TV1J2WuDZXooLszlUe9xv1BZARUMLyQlWGlvsfPupNSxavoeURCtfmeLqprtkTH8Kc1J4dPmegEfNb28+TGFOijf4dzSuIINLx/TnpllFfoMAOuqTlsS0wdm85f4ubC2vwxj/+kIoIsKCma4s68vj88P67nx5fD6ZKQmuUU03FXc6v8XTlx+sznCkrpkVuyuZP3HASZ0rkh0wY3B9V7JTu2fmnulDcjhQ3eQdBDO1qOt9RXeJq8Bw+YQ8jh1v42N38XbX0QaMgRG57f2vw3PTeeymqYxxf+kvHNWPtCQbr25oL77d9/Y2UhKtrN5XzaV//Jilmw8zYWBGwD5MTwF6U2kt6w7U0CvBEjCA+DorL50Ud1fMrKGdj6ySE6389ZvFfGl8HvXNdv790pEBj9jTeyXw8Dcm88y3pnPPpaO6eL3ebCmv45NdlQzvl0aue3TNvHF5bCw9FnC8uN3pnzFE2y1nD8YicMez6zla30yDezz95eO77pvuKCctie+eP4zGVodfrWi7T7dIx41/f1UjW8rruuzyCv26if5dSY0t3tpDR55++qoG/3ZU1rcwKDuFP143iZKyYyzZWM7Vkwd4axtWi/Ctswez4WBtp+6yivoWVuyq5JIx/bvcKT5y4xR+/uUxId/PZWPz2Hm0gV1H671dfCd6AtpXphRw/bRBLJhVFNb6yYlWXvy3Wbx2+2y/QR0eg3JSGJidHPTs6hfXlmIMXBmi2yqYrNREmlodfqPcappaSbJZSA5wwl80THd3HT3x6V6yUhJO+MTTaImrwHDO8L5kpSTwqjsae4aEDs8NvqPulWDlkjH9eavkMM1tDj7ZWcmnu6u4a+4IltxxNrm9ewWtL4Dryz2sX5orYzhQy/gBmSGLYDarhUmDXKnvrCAjHBJtFv583SQ++vH5XDAy+BGsiDB7WJ+AZ656nJXXm11HG1i9r5rZw9pfb944d3dSgKyhOzMGgLyMZP503SS2lddzxZ9X8MA7O2huc3K1T1E7XJMLXf9L3ylPfOcq6lj43VPhOrnP8xmcjOzURL8dfVVDq3e0UkeewNDxbNzKhhb6pCcyd3Qu9142itREKwtmFvmtc82UgfRJS+QvH/pPvvj4ir3YnYbrOpxodbIucXdHvbXpMCVldfRJSyS3d+jCs6+URBu/uXqc36i4UIb1S/MOAw5k9tA+fLanqtPowbrmNv768R7OHdG3U+E9XJkprgBc65NRVje2kp2a2G1nq5+V15v0XjZqm9ooLsru9rPig4mrwJBgtXDZuDyWbTlCY4vrXIFEq4WinK6/qPMn5lPfbOfD7Ue5f+k28jN6ccP0QQzrl84r35vFb68ex61nDwn6+HEDMvmi9Bhbyo4xqTC8nc3l4/OZODDTL5vpyGIRCnNSg94frtF5vbE7Dc1tTr/AMDA7hTH5vflwe+czze1OJzZL9359LhuXx4vfmYnVIjy+Yi9FOSlMPomddV5GMnkZvVh3oL2Iu/1wPamJVnJ7J3XqSqp01wb6pp3Yjs9XTmqiX1dSdWNryIyhosF/rqCKhhb6uNuw8NyhrPuvizodxCQnWvnWOUNY7h5KC66d4j8+28+8cXknvVPsqH9GL6YUZvFWyWFKDh1jrPv8hZ42a1gf6pvtlJT5jxz828d7qW1q48cXB6+dhOIpMPtmlLVNrWQFKDxHi9VdZwC8v3tCXAUGgPkT8jne5uDdrUfYcaSeIX1T/eY0CWTW0Bz6pCXy369v4YvSY9w1d4T3CDzJZuW6aYO67HMcN6A31Y2ttDlMyPqCx/XTBvHK92afko3PMzLJahHvkDmPfulJNAaYvKu7MwaPMfkZvHb7bL48IZ8fXDTipP8fkwZlst4nY9h2uI4R/dPJTk3qVHz2Di0NUhMIR3ZqItWNrgBjjHEdaQZ5vqAZQ32rX3AK1oX2DfdQ2oc+cJ2A9/Rn+6lvsfOd8wKfZ3GyLhvbny3ldWw/Un9C9YXu5Olq9e1Oqm5s5bGP9zBvXH9vje9kZAYIDNWNrd1SePY1wz08fWoPjUiCOAwMU4uyycvoxasbythxpMF7+n5XbFYLl4/Pp/xYM0P7pnJ1GKNifPl+OSPpnugug/uk0ivBwoSCjE5j860WC3ZH58Km3WmwWU/NEWNOWhJ/vn4S8yee/PyKkwdlUVpznKP1zRhj2H64nlH908lKSeiUMVS5i74piSc/mV9OWhLVja75kuqO27E7TdCMoY87YPgGhsYWO8fbHPQJ4zyBtCQbt8wezLItR1h3oIbHP9nLeSP6RnUSOnCdRwO4Cs8nMFS1O/VJS2JU/3TeKin3/v8e/nAXx9sc/PCiESEe3TVPAPAdmVTT1NatGQPA9dMH8cBXJzAhgqAWqbgLDBaLcMWEfJbvqOBQ7XG/EUlduWZKATaLcM+lo0JmGB15CtCuk9dCD9c81awW4d8vGcUdc4Z3us9mkYDj5E9VxhAtnoC8bn8tFQ0t1DS1MSI3PeBY9aqG1oiyBcBvvqSqxs5nPftKslnJSE7wCwyev/uE2Z1106wi0pJsfOvJNVQ1tvLdIGdlR6IgK4Xx7p1VtINOJG6cWciWsjrOue99/vOVEp78bD9XTSpgWL/wtu1gAnUleWoM3SktycbVkwt6tKsu7gIDwBUT87G7d3bhBoaxAzLY8POLuTjImPCuJCdaKS7M7rJI3NNuOXtwwPbZrEKbs/O0IB3PYzjdjcnPIMEqrD9Y4x2RNLK/64Si2o41hsZWciKoL0D7cMbqxlafeZKCP2ff9CSO+gQGb50jjIwBXENpF8wqpLqxlSmFWd12YtRNs4qYNTSn04mSPemG6YW896PzuXx8Pv9cdQBjDHfN7XyQc6I8XUmerka7w8mx492fMZwOTp+J70+h0Xm9GdYvjV1HG7os7nYUyXUCnl04I+Ac46e7rjKG7i4+R1OvBCtj8jNYv7/W228/MjedT1OqqG1qxek03iG/1Y0t5EaY2XkCS3VjS6eZVQPpm5bklzF4AkPJSNKqAAAbmElEQVSfE8hcbj17CJ/srOTHl4zstqPNqycXnNTIsO42uE8qv7t2AnfOGU5NU+sJjXwKJtFmITXR6j13ofZ4957DcDo5c7bsKBIRbpxRSGFOCgOzIv8ChcNqkZOeQ6UnBa8xOLGeohpDtEwalMnGQ7VsLqujT1oSOWlJZKYk4DT4TdbmGloaeVcSuK7a1nFm1UD6pidR0dC5K+lERkZlpyby6u1ne4uX8WhgdgrjC6JXx/OdLynQPEmxKi4DA7jm7vnoxxeckTvrU6nrjOHM+t9NHpRFc5uTdzYfZpR70IFnZ+0pQBtj3DWG6HUleabGCBkYfGsMDa2IxMfR6eksKyXRO6Oq5zsSaJ6kWBO3geF0GIN9JrBZBXugGsMZVnyG9gJ0Y6vDW1vK6lBgrG+x0+pwnlAXTiB+gaGxlbQkW5cnGfZNT6Kp1eGdAryyoYXslMQTHuigoivTZ9RadWP3zZN0utFvneqSzSLeQr2vMzFjGJDpmtIc8GYM7We3ujZ6z9nKkY5K6pXgmi+psqElrJEsni4jT9ZQUd8S9ogk1X2yfbuSPBPoacag4p3VYsER5DwG6xlUfAZXlug5wdBz/oo3Y3AXGL0X1OliBFG4XCe5tYYVGPr19pz97Hp9z3QYqmdlpbTPeeXNGDQwqHiXEGS46pmYMYDranK9e9kY7h6N1rErqTJKGQO0B4bKhuDTYXh0PPu5sqEloik5VHRkpiRQ32zH7nBS09hKcoI14AWmYk1cDldV4bMGKT7bHc4zrsYAcN3UgVw5Kd97VnN6LxsWaQ8MnqPCaHTj9ElL5FBtM9WNLYzN7/pMYd+uJGOMdiWdJjyZXu3xNqqbuv/kttNFRBmDiFwrIptFxCkixUHWGSgiH4jIVve6d/rc9wsROSQiG9w/8wI9h+o5wWoM3T3tdnexWMRvqguLRchMSfQWGD1dSdHoLvDMl1QdxglzWSmJWC1CRX0Lja0OmtucYU2HobqXd76kxlZqTsE8SaeLSDOGEuBq4NEu1rEDPzLGrBORdGCtiCwzxmxx3/+gMeZ3EbZDdROb1YIxnafAsDvNGXceQzBZKQntxefGVnr3snV5hbZwZacmcaTOU7PoOtBYLEKftEQq6luoPIlzGFT3aJ8Wo43qUzBP0ukiom+/MWarMWZ7iHXKjTHr3H/XA1uBk58NTZ1SnmDQccjqmVpjCCQrJdFbfK5siF4Xjm8wCKcLwnOSm6cArRlDz/OMWqtpaqVWu5K6h4gUAZOAlT6LbxeRjSLyuIj0zHXsVFCenb9vncEY484gYmPsQqbPRHrRmEDPw/d5gk257cszLUZl/YlPh6G6h+cs5xr36DLNGNxE5F0RKQnwM/9EXkhE0oCXgLuMMZ6rajwMDAUmAuXA77t4/EIRWSMiayoqOl84RnWP9oyhPTB4gkTsZAwJ7YGhsSUqQ1XBP0sI1ZUE7Wc/n+gEeqr7eLqSKupbqG+2x03GELLGYIyZG+mLiEgCrqDwjDHmZZ/nPuKzzl+BJV20YxGwCKC4uLhzNVR1C89lSH3nS/IEiTNxVFIgWamu4rNnOoypRVHKGHwCTDhTbPRNT6KyoYUjdS2u6TDi5Oj0dJacaCXJZmFvpetyr1kp8VF87va+AHHNPfEYsNUY80CH+/J8bl6Fq5itTiOBagyxljFkpiTQanfS2Oqgpin0OQfh8u1KCitjSEvC7jTsPFqv02GcRrJSEtld0eD6O04yhkiHq14lIqXATOANEVnqXp4vIm+6V5sN3AhcGGBY6n0isklENgIXAD+IpD0q+gLVGGIuY3Afme+taMRpwju6D4en2yEl0drlPEkefd1TfW8tr9dupNNIVmoieypcGUO8ZHERDVc1xiwGFgdYXgbMc//9CQS+FIEx5sZIXl91P1uArqRYyxg8gWFXhesCPtEqPvdKsJKaaA37KNMTDA5UN3H2sD5RaYOKXFZKAlvLXZMbxkvGoGc+qy7ZAhSfPd1KsdLV4ek33nXU1V0QreIzuEYjhXuU6Zsl6Iik04fvSCQtPitFe3eRI4ZrDJ6jwJ1HXIEhmjvlUf17h12w9A0M2pV0+vA92zkzTorPGhhUlwJmDI7YqjF4NnZPgTFaNQaAR74xJexLuqYmWklOsHK8zaHzJJ1GPBlDWpKNJFvsT6AHOruqCqHLGkOMTImRmeza8PdXNWERyEyO3lHhiVzSVUS8mYIGhtOHZ76keJknCTQwqBAC1xg8GUNsfH0SbRbSkmzYnYbs1MQevdyr50JCOh3G6SPbHRDiZUQSaGBQIcRDjQHau5OiWXg+GZ6MQSfQO314MoZMDQxKuXi6i9ocnUclxUqNAdr7kaM1VPVkebuS9Optpw3PdyNeRiSBBgYVgs3dXeSI4bmSoH1kUjQLzydjVP/e9E1Piqtui9Od57OIlwn0QEclqRACTaIXa2c+Q/u5DNGaDuNkXTd1IF+ZMiBmzhGJBdlpiYjEVxangUF1yVt8dgSqMcTOzstzNNjTJ5ZZLEKSJT6GRJ4p0pJsPHHTVCYOzOzpppwyGhhUlzw1hlg+jwF8is9a9FUBnD+yX0834ZSKnUM+1S0C1Rjap8SIncAQjwVGpYLRwKC6FDc1Bk/xWQODUhoYVNcSrAFqDI7YG5V03oi+fOf8oYwviJ9+ZKWC0RqD6lK8ZAwZyQncc+monm6GUqcFzRhUl7o+j0G/PkrFIt2yVZesAYarxuKZz0qpdpFe2vNaEdksIk4RKe5ivX3uS3huEJE1PsuzRWSZiOx0/86KpD0q+hICDFeNxTOflVLtIs0YSoCrgeVhrHuBMWaiMcY3gPwEeM8YMxx4z31bnUasXVzzOZaGqyql2kUUGIwxW40x2yN4ivnAk+6/nwSujKQ9Kvo8dYTAGYP2RCoVi07Vlm2Ad0RkrYgs9Fmea4wpB3D/jq/TC88AtgDDVWNxVJJSql3I4aoi8i7QP8BdPzPGvBrm68w2xpSJSD9gmYhsM8aE0/3k246FwEKAQYMGnchDVQSsEqDG4A4SWmNQKjaFDAzGmLmRvogxpsz9+6iILAam4apLHBGRPGNMuYjkAUe7eI5FwCKA4uJiE2w9FV0Wi2CRwDUGq9YYlIpJ3d6VJCKpIpLu+Ru4GFfRGuA1YIH77wVAuBmIOoVsVovfhXp0VJJSsS3S4apXiUgpMBN4Q0SWupfni8ib7tVygU9E5AtgFfCGMeZt932/BS4SkZ3ARe7b6jRjs4jfpT21xqBUbItoSgxjzGJgcYDlZcA89997gAlBHl8FzImkDar7WS2io5KUiiO6ZauQbBbxXoMB2jMGTRiUik0aGFRINqulQ8bgxGYRRDQyKBWLNDCokALVGLS+oFTs0sCgQupUY3AYHZGkVAzTwKBCSrBaOtUYNGNQKnZpYFAhWS3S6ZrPNqt+dZSKVbp1q5BsFvFegwFcw1U1Y1AqdmlgUCFZOw5X1RqDUjFNA4MKqfNwVc0YlIplGhhUSLZONQZDgtYYlIpZunWrkKwWoc2hNQal4oUGBhVSgjXAqCQNDErFLA0MKiSrRWsMSsUTDQwqpEA1Bs0YlIpdGhhUSFpjUCq+aGBQIXWqMTiMXotBqRimW7cKyWqx+AUGzRiUim0aGFRINovQ5jftthObVQODUrEq0ms+Xysim0XEKSLFQdYZKSIbfH7qROQu932/EJFDPvfNi6Q9qnvYLILDoRmDUvEioms+AyXA1cCjwVYwxmwHJgKIiBU4hP91oh80xvwuwnaobmSz+l+PQUclKRXbIgoMxpitwIlc4nEOsNsYsz+S11WnVsdptzVjUCq2neoaw3XAsx2W3S4iG0XkcRHJCvZAEVkoImtEZE1FRUX3tlL5sVksfsNVXRmDlqeUilUht24ReVdESgL8zD+RFxKRROAK4AWfxQ8DQ3F1NZUDvw/2eGPMImNMsTGmuG/fvify0ipCHU9w04xBqdgWsivJGDM3Sq91GbDOGHPE57m9f4vIX4ElUXotFUXWTjUGnStJqVh2KvsDrqdDN5KI5PncvApXMVudZlxXcPM/wU0zBqViV6TDVa8SkVJgJvCGiCx1L88XkTd91ksBLgJe7vAU94nIJhHZCFwA/CCS9qjuYXOf4GaMKzjYnUbPY1AqhkU6Kmkx/kNPPcvLgHk+t5uAnADr3RjJ66tTw9Nt5HAHBK0xKBXbdGiJCsnqzg483Ul2h1NHJSkVw3TrViF5MgZPYHDoCW5KxTQNDCokT3bgmRbD7jTeLEIpFXs0MKiQbN6uJNdJbpoxKBXbNDCokKw+XUnGGFfGoDUGpWKWbt0qpAR3ELA7DZ7TGTRjUCp2aWBQIXkyBofDeLuTdLiqUrFLA4MKyVNjaHM6vXMmacagVOzSwKBC8o5KchrvkFXNGJSKXRoYVEje4rPDeIesasagVOzSwKBCaj/BzdmeMVj1q6NUrNKtW4XkOyWG1hiUin0aGFRICX41Bh2VpFSs08CgQvKrMWjGoFTM08CgQvKdEkNHJSkV+zQwqJB8Z1dtzxj0q6NUrNKtW4XkO7tqm0NrDErFuogDg4jcLyLbRGSjiCwWkcwg610qIttFZJeI/MRn+WARWSkiO0XkXyKSGGmbVHRZfYarejKGBJ12W6mYFY2MYRkw1hgzHtgB3NtxBRGxAn8BLgNGA9eLyGj33f8LPGiMGQ7UALdGoU0qimw+w1W1xqBU7Is4MBhj3jHG2N03PwcKAqw2DdhljNljjGkFngPmi4gAFwIvutd7Ergy0jap6PK95rPWGJSKfdHeum8B3gqwfABw0Od2qXtZDlDrE1g8y9VpxBME7A6D3aEZg1KxzhbOSiLyLtA/wF0/M8a86l7nZ4AdeCbQUwRYZrpYHqgNC4GFAIMGDQqj1SparNbONQab1hiUillhBQZjzNyu7heRBcDlwBxjTKAdeykw0Od2AVAGVAKZImJzZw2e5YHasAhYBFBcXBwweKjukWDxrTHoqCSlYl00RiVdCtwDXGGMaQqy2mpguHsEUiJwHfCaO4h8AFzjXm8B8GqkbVLRZQ1YY9DAoFSsikaN4SEgHVgmIhtE5BEAEckXkTcB3NnA7cBSYCvwvDFms/vx9wA/FJFduGoOj0WhTSqKPDWGNoeOSlIqHoTVldQVY8ywIMvLgHk+t98E3gyw3h5co5bUacpTY3D4XcFNRyUpFat061Yh2Sx6HoNS8UQDgwrJ5je7qtNvmVIq9mhgUCFZfTMGPY9BqZingUGFJCLYLOJfY9DzGJSKWRoYVFisFtEag1JxQgODCovNIh2u4KZfHaVilW7dKixWi7iv+awZg1KxTgODCkuC1eKeK0lHJSkV6zQwqLBY3V1JmjEoFfs0MKiw2NzFZ4fDcwU3/eooFat061ZhsVktOJyGNnfGoAmDUrFLA4MKizdjcDqxWQTXxfeUUrFIA4MKi6vG4MTuNFpfUCrGaWBQYbFZLd4ag45IUiq2aWBQYbH5nMegGYNSsU0DgwqL1SK0OVxzJdl0RJJSMU23cBUWzRiUih8aGFRYbFb/UUlKqdgVUWAQkftFZJuIbBSRxSKSGWCdgSLygYhsFZHNInKnz32/EJFD7mtFbxCReR0fr04PNotFRyUpFScizRiWAWONMeOBHcC9AdaxAz8yxpwFzAC+JyKjfe5/0Bgz0f3T6ZrQ6vTgmUTP4dRRSUrFuogCgzHmHWOM3X3zc6AgwDrlxph17r/rga3AgEheV516Cdb26zFoxqBUbItmjeEW4K2uVhCRImASsNJn8e3urqjHRSSri8cuFJE1IrKmoqIiGu1VJ8CbMTiMXotBqRgXcgsXkXdFpCTAz3yfdX6Gq8vomS6eJw14CbjLGFPnXvwwMBSYCJQDvw/2eGPMImNMsTGmuG/fvmG9ORU9NouFNq0xKBUXbKFWMMbM7ep+EVkAXA7MMcaYIOsk4AoKzxhjXvZ57iM+6/wVWBJmu9Up1l5jcOr1npWKcZGOSroUuAe4whjTFGQdAR4DthpjHuhwX57PzauAkkjao7qPTWsMSsWNSDuLHwLSgWXu4aaPAIhIvoh4RhjNBm4ELgwwLPU+EdkkIhuBC4AfRNge1U18r/mso5KUim0hu5K6YowZFmR5GTDP/fcnQMA9iTHmxkheX506VovFmzFo8Vmp2KZbuApLglVwOD1zJWnGoFQs08CgwuJ7zWetMSgV2zQwqLB4ruBmd+hcSUrFOg0MKixWi8U7JYZmDErFNg0MKiyuKTGcWnxWKg7oFq7CYrUITgNtDqdmDErFOA0MKiyeukJLm9YYlIp1GhhUWDyX82yxOzRjUCrGaWBQYfFkCc1tOleSUrFOA4MKiydL0IxBqdingUGFxZMxOA06KkmpGKdbuAqLp8YAaMagVIzTwKDC4hsMdFSSUrFNA4MKi28w0IxBqdimgUGFxbcrSTMGpWKbBgYVFv+MQb82SsUy3cJVWPxqDHoeg1IxLdJrPt8vIttEZKOILBaRzCDr7XNfwnODiKzxWZ4tIstEZKf7d1Yk7VHdx6bFZ6XiRqQZwzJgrDFmPLADuLeLdS8wxkw0xhT7LPsJ8J4xZjjwnvu2Og3pcFWl4kdEgcEY844xxu6++TlQcIJPMR940v33k8CVkbRHdR/NGJSKH9GsMdwCvBXkPgO8IyJrRWShz/JcY0w5gPt3vyi2R0WRb5ZgtWppSqlYZgu1goi8C/QPcNfPjDGvutf5GWAHngnyNLONMWUi0g9YJiLbjDHLT6Sh7oCyEGDQoEEn8lAVBQlWzRiUihchA4MxZm5X94vIAuByYI4xxgR5jjL376MishiYBiwHjohInjGmXETygKNdtGMRsAiguLg44Ouo7uM7RFVrDErFtkhHJV0K3ANcYYxpCrJOqoike/4GLgZK3He/Bixw/70AeDWS9qjuozUGpeJHpJ3FDwHpuLqHNojIIwAiki8ib7rXyQU+EZEvgFXAG8aYt933/Ra4SER2Ahe5b6vTkO+5C5oxKBXbQnYldcUYMyzI8jJgnvvvPcCEIOtVAXMiaYM6NfwzBi0+KxXLdAtXYdEag1LxQwODCovWGJSKHxoYVFj8agw6V5JSMU0DgwqLXqhHqfihgUGFxaY1BqXihgYGFRabVUclKRUvdAtXYdFLeyoVPzQwqLBojUGp+KGBQYXFt/tIr+CmVGzTwKDCYrUI4o4HWmNQKrbpFq7C5ulC0hqDUrFNA4MKmycgaI1BqdimgUGFLcHdhaQZg1KxTQODCptnKgwtPisV2zQwqLBpjUGp+KCBQYWtvcagXxulYplu4SpsNq0xKBUXIr3m8/0isk1ENorIYhHJDLDOSPdlPz0/dSJyl/u+X4jIIZ/75kXSHtW9PLUFHZWkVGyLNGNYBow1xowHdgD3dlzBGLPdGDPRGDMRmAI0AYt9VnnQc78x5s2Oj1enD6vWGJSKCxEFBmPMO8YYu/vm50BBiIfMAXYbY/ZH8rqqZ3iGq2rGoFRsi2aN4RbgrRDrXAc822HZ7e6uqMdFJCuK7VFRphmDUvEhZGAQkXdFpCTAz3yfdX4G2IFnunieROAK4AWfxQ8DQ4GJQDnw+y4ev1BE1ojImoqKipBvTEWfzSruOZM0MCgVy2yhVjDGzO3qfhFZAFwOzDHGmC5WvQxYZ4w54vPc3r9F5K/Aki7asQhYBFBcXNzV66huYrWIZgtKxYFIRyVdCtwDXGGMaQqx+vV06EYSkTyfm1cBJZG0R3WvBItF6wtKxYFIawwPAenAMvdw00cARCRfRLwjjEQkBbgIeLnD4+8TkU0ishG4APhBhO1R3UgzBqXiQ8iupK4YY4YFWV4GzPO53QTkBFjvxkheX51aNqtoxqBUHNAzn1XYbBbBZtWvjFKxTrdyFTar1hiUigsaGFTYbFpjUCouaGBQYfOcx6CUim0RFZ9VfLlheiHlx473dDOUUt1MA4MK28yhnQaWKaVikHYlKaWU8qOBQSmllB8NDEoppfxoYFBKKeVHA4NSSik/GhiUUkr50cCglFLKjwYGpZRSfqTri66dnkSkAth/kg/vA1RGsTlninh83/H4niE+33c8vmc48fddaIzpG2qlMzIwREJE1hhjinu6HadaPL7veHzPEJ/vOx7fM3Tf+9auJKWUUn40MCillPITj4FhUU83oIfE4/uOx/cM8fm+4/E9Qze977irMSillOpaPGYMSimluhBXgUFELhWR7SKyS0R+0tPt6Q4iMlBEPhCRrSKyWUTudC/PFpFlIrLT/Turp9sabSJiFZH1IrLEfXuwiKx0v+d/iUhiT7cx2kQkU0ReFJFt7s98Zqx/1iLyA/d3u0REnhWRXrH4WYvI4yJyVERKfJYF/GzF5U/ufdtGEZkcyWvHTWAQESvwF+AyYDRwvYiM7tlWdQs78CNjzFnADOB77vf5E+A9Y8xw4D337VhzJ7DV5/b/Ag+633MNcGuPtKp7/RF42xgzCpiA6/3H7GctIgOA7wPFxpixgBW4jtj8rP8OXNphWbDP9jJguPtnIfBwJC8cN4EBmAbsMsbsMca0As8B83u4TVFnjCk3xqxz/12Pa0cxANd7fdK92pPAlT3Twu4hIgXAl4C/uW8LcCHwonuVWHzPvYFzgccAjDGtxphaYvyzxnXlyWQRsQEpQDkx+FkbY5YD1R0WB/ts5wNPGZfPgUwRyTvZ146nwDAAOOhzu9S9LGaJSBEwCVgJ5BpjysEVPIB+PdeybvEH4N8Bp/t2DlBrjLG7b8fi5z0EqACecHeh/U1EUonhz9oYcwj4HXAAV0A4Bqwl9j9rj2CfbVT3b/EUGCTAspgdkiUiacBLwF3GmLqebk93EpHLgaPGmLW+iwOsGmuftw2YDDxsjJkENBJD3UaBuPvU5wODgXwgFVc3Skex9lmHEtXvezwFhlJgoM/tAqCsh9rSrUQkAVdQeMYY87J78RFPaun+fbSn2tcNZgNXiMg+XF2EF+LKIDLd3Q0Qm593KVBqjFnpvv0irkARy5/1XGCvMabCGNMGvAzMIvY/a49gn21U92/xFBhWA8PdoxcScRWsXuvhNkWdu2/9MWCrMeYBn7teAxa4/14AvHqq29ZdjDH3GmMKjDFFuD7X940xNwAfANe4V4up9wxgjDkMHBSRke5Fc4AtxPBnjasLaYaIpLi/6573HNOftY9gn+1rwDfdo5NmAMc8XU4nI65OcBORebiOJK3A48aYX/dwk6JORM4GPgY20d7f/lNcdYbngUG4Nq5rjTEdC1tnPBE5H7jbGHO5iAzBlUFkA+uBbxhjWnqyfdEmIhNxFdwTgT3AzbgO+GL2sxaR/wa+hmsE3nrgW7j602PqsxaRZ4Hzcc2gegT4OfAKAT5bd5B8CNcopibgZmPMmpN+7XgKDEoppUKLp64kpZRSYdDAoJRSyo8GBqWUUn40MCillPKjgUEppZQfDQxKKaX8aGBQSinlRwODUkopP/8fL4Sde17Zf9wAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(res.fx[0:res.total_num_search])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x1a1c6e41d0>]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAD8CAYAAABzTgP2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAFZ9JREFUeJzt3X2wXPV93/H3Rw+AMXUIIJ5phGM1rqZ1MHNDIWkbG5MpD57IpqGG2C5JTWky8cR12yS0tM00LVM87SRtWkqrxrh0xg5QOzJqLRsH6jHuxCWI2K4FKkZDEiNDzMU2BjlF3N399o89V9q9d6+uuLvSRXverxnm3vOwe35nj/h97u98zzmbqkKSpHlrVrsBkqRXF4NBkjTEYJAkDTEYJElDDAZJ0hCDQZI0xGCQJA0xGCRJQwwGSdKQdeO8OMkpwN3ARuCPgL9RVd8Zsd6HgKuayX9eVXc38wP8C+AaoAvcXlW/udx2TzvttNq4ceM4TZek1nnkkUeeq6oNy603VjAANwEPVNWtSW5qpn9lcIUkVwEXAhcAxwOfT/LpqnoB+BngPOCNVdVLcvrhbHTjxo3s3LlzzKZLUrsk+ePDWW/cU0lbgDub3+8E3jFinc3A56uqU1XfA74CXN4s+3ng16qqB1BVz47ZHknSmMYNhjOq6hmA5ueov/i/AlyR5MQkpwFvpT9KAPhB4F1Jdib5dJJNS20oyY3NejtnZ2fHbLYkaSnLnkpKcj9w5ohFNx/OBqrqs0l+BPg9YBb4ItBpFh8PvFRVM0muBu4A/soS77MV2AowMzPjI2El6QhZNhiq6rKlliX5ZpKzquqZJGcBI08FVdUtwC3Naz4GPNEs2gt8ovl9G/CRV9B2SdIRMO6ppO3A9c3v1wP3Llwhydokpza/vwl4E/DZZvEngUub338c+NqY7ZEkjWncq5JuBe5J8j7g6/QvOyXJDPBzVXUDsB74Qv/KVF4A3lNVnYHXfzTJB4F9wA1jtkeSNKaxgqGqvgW8bcT8nTSdfFW9RP/KpFGvf56D9zdIkl4Fxh0xaIF7v/wNZl/cv9rNkLQKOr2i0+3xcrfgCH1t8vU/upFTTzr+iLz3PINhgp769p/ygbu+vNrNkPQq0D97Pnk/ecE5BsOx5NkXXwLg9ndfyI9tOm2VWyPpaFu3Jqxfu4Z1a0KOVDIcBQbDBM2fQjrvlBN53QnrV7k1krQyPl11gmb3vQzA6X/myA7zJOlIMhgmaPbF/SRwymuPW+2mSNKKGQwT9Ny+/Zxy4nGsW+vHKunYZQ82QbMv7ue0I3y1gCQdaQbDBD23bz8brC9IOsYZDBPUHzFYX5B0bDMYJqSqHDFImgoGw4Ts29/hpbmeNQZJxzyDYUKea+5hcMQg6VhnMEzI/F3PjhgkHesMhgl5bl8/GBwxSDrWGQwT4ohB0rQwGCbkuX37WePjMCRNAYNhQmZf3M8prz2etWuO3UftShIYDBPjPQySpoXBMCHe9SxpWhgME/LcvpcdMUiaCgbDBFQVsy/uZ4NXJEmaAgbDBLzwUoeXuz1HDJKmgsEwAd7DIGmaGAwT4F3PkqaJwTABjhgkTRODYQIcMUiaJgbDBMy+uJ+1a8LJr1m/2k2RpLGNFQxJTknyu0meaH5+/xLrfSjJrua/dw3Mf1uSP0jy5ST/K8kbxmnPanluX//mtjU+DkPSFBh3xHAT8EBVbQIeaKaHJLkKuBC4APhLwC8leV2z+Hbg3VV1AfAx4B+P2Z5V0b/r2dNIkqbDuMGwBbiz+f1O4B0j1tkMfL6qOlX1PeArwOXNsgLmQ+L7gKfHbM+q8K5nSdNk3GA4o6qeAWh+nj5ina8AVyQ5MclpwFuB85plNwA7kuwF3gvcutSGktyYZGeSnbOzs2M2e7IcMUiaJuuWWyHJ/cCZIxbdfDgbqKrPJvkR4PeAWeCLQKdZ/EHgyqp6KMkvAb9OPyxGvc9WYCvAzMxMHc62j4Zer/jW93yyqqTpsWwwVNVlSy1L8s0kZ1XVM0nOAp5d4j1uAW5pXvMx4IkkG4AfrqqHmtXuBj7zSndgtX33/80x1y1HDJKmxrLBsIztwPX0TwFdD9y7cIUka4GTq+pbSd4EvAn4bLP4+5L8uar6GvATwO4x27Mi/23nU/zaf3+MlQxDetV/lSMGSdNi3GC4FbgnyfuArwPXACSZAX6uqm4A1gNfSALwAvCequo06/1t4BNJesB3gL81ZntWZNc3vsv+bo/3XvwDK3r9CevX8JYf2jDhVknS6hgrGKrqW8DbRszfSVMrqKqX6F+ZNOr124Bt47RhEuZ6xetOWMc/efvIZkpSq3jnM9Dp9vyuZklqGAxAp1usW+NHIUlgMADQ6RXr1zpikCQwGADo9HqsW+tHIUlgMAAw1y3WWWOQJMBgAPrF53WeSpIkwGAA+jUGi8+S1GdvSP+qJIvPktRnMNAUnx0xSBJgMABN8dkRgyQBBgMwP2IwGCQJDAagufPZ+xgkCTAYAO98lqRBBgPNfQwWnyUJMBgAi8+SNMhgwOKzJA0yGLD4LEmD7A1pis+OGCQJMBiA+Yfo+VFIEhgMQP87ny0+S1KfwcD85aoGgySBwUCvV/QK72OQpEbre8NOrwC881mSGgZDrwdg8VmSGq3vDee6/RGDNQZJ6mt9MHS6zYjBYJAkwGCg29QYPJUkSX2t7w3nLD5L0pCxgiHJNUkeTdJLMnOI9S5P8niSPUluGph/fpKHkjyR5O4kx43TnpU4eCqp9RkpScD4I4ZdwNXAg0utkGQtcBtwBbAZuC7J5mbxh4DfqKpNwHeA943ZnlfsQPHZEYMkAWMGQ1XtrqrHl1ntImBPVT1ZVS8DdwFbkgS4FPh4s96dwDvGac9KHLhc1RGDJAFHp8ZwDvDUwPTeZt6pwPNV1Vkwf6QkNybZmWTn7OzsxBrXccQgSUPWLbdCkvuBM0csurmq7j2MbYzqcesQ80eqqq3AVoCZmZkl13ulvPNZkoYtGwxVddmY29gLnDcwfS7wNPAccHKSdc2oYX7+UWXxWZKGHY3e8GFgU3MF0nHAtcD2qirgc8BPNetdDxzOCGSiLD5L0rBxL1d9Z5K9wCXAp5Lc18w/O8kOgGY08H7gPmA3cE9VPdq8xa8Afy/JHvo1hw+P056VsPgsScOWPZV0KFW1Ddg2Yv7TwJUD0zuAHSPWe5L+VUurptNzxCBJg1r/Z/L8VUnrHTFIEmAwHCw+O2KQJMBg8FlJkrRA64NhfsSw1lNJkgQYDAeLz34fgyQBBsPB4rPfxyBJgMEw8J3PjhgkCQyGA3c+e7mqJPW1vjc8UHx2xCBJgMFg8VmSFjAYLD5L0pDW94adXo8E1jpikCTAYGCuWxaeJWlA63vETrfnaEGSBhgMvfIeBkkaYDD0ehaeJWlA63vETre8VFWSBrQ+GOa65YhBkga0vkfs9iw+S9Kg1gfDnMVnSRrS+mDodHvexyBJA1rfI3a6jhgkaVDrg6F/Kqn1H4MkHdD6HrHb63m5qiQNaH0wzHkfgyQNaX0wdLre+SxJg1rfI/qsJEka1vpg6J9Kav3HIEkHjNUjJrkmyaNJeklmDrHe5UkeT7InyU0D8z/azN+V5I4k68dpz0pYfJakYeP+qbwLuBp4cKkVkqwFbgOuADYD1yXZ3Cz+KPBG4C8CrwFuGLM9r5j3MUjSsHXjvLiqdgMkh+xYLwL2VNWTzbp3AVuAx6pqx/xKSX4fOHec9qzEnI/dlqQhR6NHPAd4amB6bzPvgOYU0nuBzyz1JkluTLIzyc7Z2dmJNc7HbkvSsGVHDEnuB84csejmqrr3MLYxqtetBdP/AXiwqr6w1JtU1VZgK8DMzMzC16/YXNc7nyVp0LLBUFWXjbmNvcB5A9PnAk/PTyT5VWAD8HfG3M6KWHyWpGFH40/lh4FNSc5PchxwLbAdIMkNwF8Drquq3lFoyyIWnyVp2LiXq74zyV7gEuBTSe5r5p+dZAdAVXWA9wP3AbuBe6rq0eYt/iNwBvDFJF9O8k/Hac9KWHyWpGHjXpW0Ddg2Yv7TwJUD0zuAHSPWG2v7k2DxWZKGtfpP5apqHonR6o9Bkoa0ukfs9voXNzlikKSDWh0MnflgsPgsSQe0Ohjmuv0LofzOZ0k6qNU9YqfriEGSFmp3MBw4ldTqj0GShrS6R+z0+qeSLD5L0kHtDoauVyVJ0kKtDoYDxWdPJUnSAa3uEb1cVZIWa3cwHDiV1OqPQZKGtLpHtPgsSYu1OhjmvI9BkhZpdTB0LD5L0iKt7hE7PkRPkhYxGPDOZ0ka1Ooecf5UkiMGSTqo1cFg8VmSFmt1MMxfrmrxWZIOanWP6LOSJGmxdgdDU3x2xCBJB7W6R5wvPq91xCBJB7Q6GOZ8iJ4kLdLqYOj4nc+StEire0S/81mSFmt3MFh8lqRFWt0jWnyWpMVaHQxzPkRPkhYZKxiSXJPk0SS9JDOHWO/yJI8n2ZPkphHL/12SfeO0ZSU63R7r1oTEYJCkeeOOGHYBVwMPLrVCkrXAbcAVwGbguiSbB5bPACeP2Y4V6fTKwrMkLTBWMFTV7qp6fJnVLgL2VNWTVfUycBewBQ6Exr8CfnmcdqxUp1teqipJCxyNXvEc4KmB6b3NPID3A9ur6pnl3iTJjUl2Jtk5Ozs7kYZ1ej3WOmKQpCHrllshyf3AmSMW3VxV9x7GNkb1vJXkbOAa4C2H8R5U1VZgK8DMzEwdzmuWM9ct1jlikKQhywZDVV025jb2AucNTJ8LPA28GXgDsKcp/p6YZE9VvWHM7R22TrfHekcMkjRk2WCYgIeBTUnOB74BXAv8dFU9ysBIJMm+oxkKAF2Lz5K0yLiXq74zyV7gEuBTSe5r5p+dZAdAVXXo1xLuA3YD9zShsOrmehafJWmhsUYMVbUN2DZi/tPAlQPTO4Ady7zXSeO0ZSU63Z53PUvSAq3+c3muW6zzOUmSNKTVvWKnZ/FZkhZqdTB0e+VzkiRpgVYHw1y3530MkrRAq3vFTtfLVSVpoVYHw1zP4rMkLdTqXrHT7bHeGoMkDWl1MHjnsyQt1upgsPgsSYu1ulf0i3okabF2B4OP3ZakRVrdK8752G1JWqTVwWDxWZIWa3UwWHyWpMVa3St2fFaSJC3S7mDwsduStEire8U5H7stSYu0Nhh6vaIKawyStEBre8W5Xg/Aq5IkaYHWBkOnWwAWnyVpAYPB4rMkDWltrzh/KsnisyQNa20wdHvzp5Ja+xFI0kit7RXnuk3x2RqDJA1pbTAcrDEYDJI0qL3BcOBy1dZ+BJI0Umt7xU5TY/A7nyVpWHuDwctVJWmksXrFJNckeTRJL8nMIda7PMnjSfYkuWlgfpLckuRrSXYn+cVx2vNKWHyWpNHWjfn6XcDVwH9aaoUka4HbgJ8A9gIPJ9leVY8BPwOcB7yxqnpJTh+zPYdt/lSSxWdJGjZWMFTVboDkkJ3rRcCeqnqyWfcuYAvwGPDzwE9XVa95v2fHac8rcXDE4KkkSRp0NHrFc4CnBqb3NvMAfhB4V5KdST6dZNNSb5Lkxma9nbOzs2M3av4GN+98lqRhywZDkvuT7Brx35bD3Maonrean8cDL1XVDPCfgTuWepOq2lpVM1U1s2HDhsPc9NIsPkvSaMueSqqqy8bcxl76dYR55wJPDyz7RPP7NuAjY27rsFl8lqTRjsafyw8Dm5Kcn+Q44Fpge7Psk8Clze8/DnztKLQHsPgsSUsZ93LVdybZC1wCfCrJfc38s5PsAKiqDvB+4D5gN3BPVT3avMWtwF9P8lXgXwI3jNOeV8LisySNNu5VSdvonwJaOP9p4MqB6R3AjhHrPQ9cNU4bVsrisySN1to/ly0+S9Jore0VD3zns8VnSRrS2mDwO58labTWBsOB4rOnkiRpSGt7RYvPkjRaa4Oh43c+S9JIre0VvfNZkkZrbTB0usWawBqDQZKGtDYY5no9C8+SNEJre8Zut/y+Z0kaobXB0OmVIwZJGqG1PeNct2fhWZJGaG0wdLrlI7claYSxnq56rLl521f5/T/8NgB/8sJLvO6E9avcIkl69WlVMJx98mvYdMZJAGw64yQufv2pq9wiSXr1aVUw/MJb37DaTZCkV73W1hgkSaMZDJKkIQaDJGmIwSBJGmIwSJKGGAySpCEGgyRpiMEgSRqSqlrtNrxiSWaBP17hy08Dnptgc44VbdzvNu4ztHO/3efD8wNVtWG5lY7JYBhHkp1VNbPa7Tja2rjfbdxnaOd+u8+T5akkSdIQg0GSNKSNwbB1tRuwStq4323cZ2jnfrvPE9S6GoMk6dDaOGKQJB1Cq4IhyeVJHk+yJ8lNq92eIyHJeUk+l2R3kkeTfKCZf0qS303yRPPz+1e7rZOWZG2SLyX5H830+Ukeavb57iTHrXYbJy3JyUk+nuT/Nsf8kmk/1kk+2Pzb3pXkt5OcMI3HOskdSZ5Nsmtg3shjm77fbPq2/5PkwnG23ZpgSLIWuA24AtgMXJdk8+q26ojoAH+/qv48cDHwC81+3gQ8UFWbgAea6WnzAWD3wPSHgN9o9vk7wPtWpVVH1r8FPlNVbwR+mP7+T+2xTnIO8IvATFX9BWAtcC3Teaz/C3D5gnlLHdsrgE3NfzcCt4+z4dYEA3ARsKeqnqyql4G7gC2r3KaJq6pnquoPmt9fpN9RnEN/X+9sVrsTeMfqtPDISHIucBXwW810gEuBjzerTOM+vw74q8CHAarq5ap6nik/1vS/efI1SdYBJwLPMIXHuqoeBL69YPZSx3YL8F+r738DJyc5a6XbblMwnAM8NTC9t5k3tZJsBN4MPAScUVXPQD88gNNXr2VHxL8BfhnoNdOnAs9XVaeZnsbj/XpgFvhIcwrtt5K8lik+1lX1DeBfA1+nHwjfBR5h+o/1vKWO7UT7tzYFQ0bMm9pLspKcBHwC+LtV9cJqt+dISvJ24NmqemRw9ohVp+14rwMuBG6vqjcD32OKThuN0pxT3wKcD5wNvJb+aZSFpu1YL2ei/97bFAx7gfMGps8Fnl6lthxRSdbTD4WPVtXvNLO/OT+0bH4+u1rtOwJ+DPjJJH9E/xThpfRHECc3pxtgOo/3XmBvVT3UTH+cflBM87G+DPjDqpqtqjngd4AfZfqP9bylju1E+7c2BcPDwKbm6oXj6Bestq9ymyauObf+YWB3Vf36wKLtwPXN79cD9x7tth0pVfUPq+rcqtpI/7j+z6p6N/A54Kea1aZqnwGq6k+Ap5L8UDPrbcBjTPGxpn8K6eIkJzb/1uf3eaqP9YClju124G82VyddDHx3/pTTSrTqBrckV9L/S3ItcEdV3bLKTZq4JH8Z+ALwVQ6eb/9H9OsM9wB/lv7/XNdU1cLC1jEvyVuAf1BVb0/yevojiFOALwHvqar9q9m+SUtyAf2C+3HAk8DP0v+Db2qPdZJ/BryL/hV4XwJuoH8+faqOdZLfBt5C/ymq3wR+FfgkI45tE5L/nv5VTH8K/GxV7VzxttsUDJKk5bXpVJIk6TAYDJKkIQaDJGmIwSBJGmIwSJKGGAySpCEGgyRpiMEgSRry/wFzHgS42N6eSwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "best_fx, best_action = res.export_all_sequence_best_fx()\n",
    "plt.plot(best_fx)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 結果のシリアライズ\n",
    "\n",
    "探索結果は `save` メソッドにより外部ファイルに保存できます。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "res.save('search_result.npz')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "del res"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "保存した結果ファイルは以下のようにロードします。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "res = physbo.search.discrete.results.history()\n",
    "res.load('search_result.npz')"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}