{ "cells": [ { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "![](aux/titlepage.png)" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "# Part II: Automatic Differentiation with Zygote.jl\n", "\n", "![Zygote.jl](aux/Zygote.png)" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Why I personally use Julia\n", "\n", "![Julia](aux/WhyIUseJulia.png)" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "For me, this becomes particularly apparent in the context of automatic differentiation. Here, Julia's main AD system, the [`Zygote.jl` package](https://github.com/FluxML/Zygote.jl), offers a particularly flexible and straightforward way of getting gradients of essentially arbitrary pieces of code. I will illustrate this with a modelling example in a COVID-19 context in the following.\n", "\n", "The notebook is an adaptation of the notebooks in [this repository](https://github.com/maren-ha/DifferentiableProgrammingForStatisticalModeling) and slides from a presentation I gave about this at the Statistical Computing Workshop at Reisensburg Castle in July 2022." ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Exemplary modelling challenge\n", "\n", "First, let's load and install the necessary packages as usual. The commented-out code will read from the Project.toml and Manifest.toml files and try to load the same environment as the one used by the author (i.e., the exact same version of Julia and its packages). This is good practice to ensure reproducibility, but its not necessary for the notebook to run and might result the user having to deal with installing different versions of a package. Your results should match the ones on this page; if they don't, try running the commented-out code.\n", "\n", "```julia" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [], "source": [ "using Pkg;\n", "#Pkg.activate(\"aux/\")\n", "#Pkg.add([\"GLM\", \"ProgressMeter\", \"StatsBase\", \"Zygote\"])\n", "#Pkg.instantiate() # uncomment if you want to run this notebook yourself under the same environment as the author\n", "#Pkg.status()\n", "\n", "using CSV \n", "using DataFrames\n", "using GLM \n", "using LinearAlgebra\n", "using ProgressMeter\n", "using Random\n", "using Statistics\n", "using StatsBase\n", "using StatsPlots\n", "using VegaLite\n", "using Zygote " ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "At the beginning of the pandemic, our institute was asked by the Robert Koch Institute, the main health authority that monitored the COVID-19 pandemic in Germany, and the Federal Ministry for Health, to build a prediction model for COVID-19 ICU demand. The data we were given was a newly-set-up intensive care registry, where each ICU in Germany was required to report daily their numbers of COVID-19 patients in ICUs and normal wards, alongside a bunch of other information. Even though daily reporting was compulsory, the data was still very messy, with many ICUs not reporting at all, and many others reporting only sporadically.\n", "\n", "To get an idea what the data looked like, let's look at an exemplary hospital from the registry during a period in spring - early summer 2020. \n", "\n", "Note that the values shown in `:currentcases` have been imputed based on last-observation-carried-forward whenever a report has been missing, as indicated by a $0$ in the `:reported`column. " ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/html": [ "

10 rows × 3 columns

datecurrentcasesreported
DateFloat64Bool
12020-04-160.00
22020-04-170.01
32020-04-180.01
42020-04-190.01
52020-04-202.01
62020-04-210.01
72020-04-221.01
82020-04-232.01
92020-04-242.01
102020-04-252.00
" ], "text/latex": [ "\\begin{tabular}{r|ccc}\n", "\t& date & currentcases & reported\\\\\n", "\t\\hline\n", "\t& Date & Float64 & Bool\\\\\n", "\t\\hline\n", "\t1 & 2020-04-16 & 0.0 & 0 \\\\\n", "\t2 & 2020-04-17 & 0.0 & 1 \\\\\n", "\t3 & 2020-04-18 & 0.0 & 1 \\\\\n", "\t4 & 2020-04-19 & 0.0 & 1 \\\\\n", "\t5 & 2020-04-20 & 2.0 & 1 \\\\\n", "\t6 & 2020-04-21 & 0.0 & 1 \\\\\n", "\t7 & 2020-04-22 & 1.0 & 1 \\\\\n", "\t8 & 2020-04-23 & 2.0 & 1 \\\\\n", "\t9 & 2020-04-24 & 2.0 & 1 \\\\\n", "\t10 & 2020-04-25 & 2.0 & 0 \\\\\n", "\\end{tabular}\n" ], "text/plain": [ "\u001b[1m10×3 DataFrame\u001b[0m\n", "\u001b[1m Row \u001b[0m│\u001b[1m date \u001b[0m\u001b[1m currentcases \u001b[0m\u001b[1m reported \u001b[0m\n", "\u001b[1m \u001b[0m│\u001b[90m Date \u001b[0m\u001b[90m Float64 \u001b[0m\u001b[90m Bool \u001b[0m\n", "─────┼────────────────────────────────────\n", " 1 │ 2020-04-16 0.0 false\n", " 2 │ 2020-04-17 0.0 true\n", " 3 │ 2020-04-18 0.0 true\n", " 4 │ 2020-04-19 0.0 true\n", " 5 │ 2020-04-20 2.0 true\n", " 6 │ 2020-04-21 0.0 true\n", " 7 │ 2020-04-22 1.0 true\n", " 8 │ 2020-04-23 2.0 true\n", " 9 │ 2020-04-24 2.0 true\n", " 10 │ 2020-04-25 2.0 false" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "#load data \n", "all_hospitals_df = CSV.read(\"aux/data_all_hospitals.csv\", DataFrame)\n", "myhospital = all_hospitals_df[all_hospitals_df[!,:hospital] .== 40, [:date, :currentcases, :reported]]\n", "first(myhospital, 10)" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "We replace all numbers in `:currentcases` that have been imputed by last-observation-carried-forward with `NaN`, i.e., all values in rows where `:reported` is equal to $0$, and plot the number of current cases over the time interval to visualize the missingness pattern." ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "application/vnd.vega.v5+json": { "$schema": "https://vega.github.io/schema/vega/v5.json", "axes": [ { "aria": false, "domain": false, "grid": true, "gridScale": "y", "labels": false, "maxExtent": 0, "minExtent": 0, "orient": "bottom", "scale": "x", "tickCount": { "signal": "ceil(width/40)" }, "ticks": false, "zindex": 0 }, { "aria": false, "domain": false, "grid": true, "gridScale": "x", "labels": false, "maxExtent": 0, "minExtent": 0, "orient": "left", "scale": "y", "tickCount": { "signal": "ceil(height/40)" }, "ticks": false, "zindex": 0 }, { "grid": false, "labelFlush": true, "labelOverlap": true, "orient": "bottom", "scale": "x", "tickCount": { "signal": "ceil(width/40)" }, "title": "Date", "zindex": 0 }, { "grid": false, "labelOverlap": true, "orient": "left", "scale": "y", "tickCount": { "signal": "ceil(height/40)" }, "title": "Number of prevalent COVID-19 cases", "zindex": 0 } ], "background": "white", "data": [ { "name": "source_0", "values": [ { "currentcases": null, "date": "2020-04-16", "reported": false }, { "currentcases": 0, "date": "2020-04-17", "reported": true }, { "currentcases": 0, "date": "2020-04-18", "reported": true }, { "currentcases": 0, "date": "2020-04-19", "reported": true }, { "currentcases": 2, "date": "2020-04-20", "reported": true }, { "currentcases": 0, "date": "2020-04-21", "reported": true }, { "currentcases": 1, "date": "2020-04-22", "reported": true }, { "currentcases": 2, "date": "2020-04-23", "reported": true }, { "currentcases": 2, "date": "2020-04-24", "reported": true }, { "currentcases": null, "date": "2020-04-25", "reported": false }, { "currentcases": 3, "date": "2020-04-26", "reported": true }, { "currentcases": null, "date": "2020-04-27", "reported": false }, { "currentcases": 4, "date": "2020-04-28", "reported": true }, { "currentcases": null, "date": "2020-04-29", "reported": false }, { "currentcases": 5, "date": "2020-04-30", "reported": true }, { "currentcases": null, "date": "2020-05-01", "reported": false }, { "currentcases": null, "date": "2020-05-02", "reported": false }, { "currentcases": null, "date": "2020-05-03", "reported": false }, { "currentcases": 4, "date": "2020-05-04", "reported": true }, { "currentcases": 3, "date": "2020-05-05", "reported": true }, { "currentcases": 2, "date": "2020-05-06", "reported": true }, { "currentcases": 3, "date": "2020-05-07", "reported": true }, { "currentcases": null, "date": "2020-05-08", "reported": false }, { "currentcases": null, "date": "2020-05-09", "reported": false }, { "currentcases": null, "date": "2020-05-10", "reported": false }, { "currentcases": 5, "date": "2020-05-11", "reported": true }, { "currentcases": null, "date": "2020-05-12", "reported": false }, { "currentcases": null, "date": "2020-05-13", "reported": false }, { "currentcases": null, "date": "2020-05-14", "reported": false }, { "currentcases": null, "date": "2020-05-15", "reported": false }, { "currentcases": null, "date": "2020-05-16", "reported": false }, { "currentcases": null, "date": "2020-05-17", "reported": false }, { "currentcases": 1, "date": "2020-05-18", "reported": true }, { "currentcases": 1, "date": "2020-05-19", "reported": true }, { "currentcases": null, "date": "2020-05-20", "reported": false }, { "currentcases": null, "date": "2020-05-21", "reported": false }, { "currentcases": null, "date": "2020-05-22", "reported": false }, { "currentcases": null, "date": "2020-05-23", "reported": false }, { "currentcases": null, "date": "2020-05-24", "reported": false }, { "currentcases": null, "date": "2020-05-25", "reported": false }, { "currentcases": 2, "date": "2020-05-26", "reported": true }, { "currentcases": null, "date": "2020-05-27", "reported": false }, { "currentcases": 3, "date": "2020-05-28", "reported": true }, { "currentcases": null, "date": "2020-05-29", "reported": false }, { "currentcases": null, "date": "2020-05-30", "reported": false }, { "currentcases": null, "date": "2020-05-31", "reported": false }, { "currentcases": null, "date": "2020-06-01", "reported": false }, { "currentcases": null, "date": "2020-06-02", "reported": false }, { "currentcases": null, "date": "2020-06-03", "reported": false }, { "currentcases": null, "date": "2020-06-04", "reported": false }, { "currentcases": null, "date": "2020-06-05", "reported": false }, { "currentcases": null, "date": "2020-06-06", "reported": false }, { "currentcases": null, "date": "2020-06-07", "reported": false }, { "currentcases": 2, "date": "2020-06-08", "reported": true }, { "currentcases": 2, "date": "2020-06-09", "reported": true }, { "currentcases": 1, "date": "2020-06-10", "reported": true }, { "currentcases": 3, "date": "2020-06-11", "reported": true }, { "currentcases": null, "date": "2020-06-12", "reported": false }, { "currentcases": null, "date": "2020-06-13", "reported": false }, { "currentcases": null, "date": "2020-06-14", "reported": false }, { "currentcases": 2, "date": "2020-06-15", "reported": true }, { "currentcases": null, "date": "2020-06-16", "reported": false }, { "currentcases": null, "date": "2020-06-17", "reported": false }, { "currentcases": 2, "date": "2020-06-18", "reported": true }, { "currentcases": 3, "date": "2020-06-19", "reported": true }, { "currentcases": null, "date": "2020-06-20", "reported": false }, { "currentcases": null, "date": "2020-06-21", "reported": false }, { "currentcases": null, "date": "2020-06-22", "reported": false }, { "currentcases": null, "date": "2020-06-23", "reported": false }, { "currentcases": null, "date": "2020-06-24", "reported": false } ] }, { "name": "data_0", "source": "source_0", "transform": [ { "as": "date", "expr": "toDate(datum[\"date\"])", "type": "formula" } ] }, { "name": "data_1", "source": "data_0", "transform": [ { "expr": "(isDate(datum[\"date\"]) || (isValid(datum[\"date\"]) && isFinite(+datum[\"date\"]))) && isValid(datum[\"currentcases\"]) && isFinite(+datum[\"currentcases\"])", "type": "filter" } ] } ], "height": 150, "marks": [ { "encode": { "update": { "defined": { "signal": "isValid(datum[\"date\"]) && isFinite(+datum[\"date\"]) && isValid(datum[\"currentcases\"]) && isFinite(+datum[\"currentcases\"])" }, "description": { "signal": "\"date: \" + (timeFormat(datum[\"date\"], '%b %d, %Y')) + \"; currentcases: \" + (format(datum[\"currentcases\"], \"\"))" }, "opacity": { "value": 0.8 }, "stroke": { "value": "#004b96" }, "x": { "field": "date", "scale": "x" }, "y": { "field": "currentcases", "scale": "y" } } }, "from": { "data": "data_0" }, "name": "layer_0_marks", "sort": { "field": "datum[\"date\"]" }, "style": [ "line" ], "type": "line" }, { "encode": { "update": { "ariaRoleDescription": { "value": "point" }, "description": { "signal": "\"date: \" + (timeFormat(datum[\"date\"], '%b %d, %Y')) + \"; currentcases: \" + (format(datum[\"currentcases\"], \"\"))" }, "fill": { "value": "#004b96" }, "opacity": { "value": 0.6 }, "size": { "value": 100 }, "x": { "field": "date", "scale": "x" }, "y": { "field": "currentcases", "scale": "y" } } }, "from": { "data": "data_1" }, "name": "layer_1_marks", "style": [ "point" ], "type": "symbol" } ], "padding": 5, "scales": [ { "domain": { "fields": [ { "data": "data_0", "field": "date" }, { "data": "data_1", "field": "date" } ] }, "name": "x", "range": [ 0, { "signal": "width" } ], "type": "time" }, { "domain": { "fields": [ { "data": "data_0", "field": "currentcases" }, { "data": "data_1", "field": "currentcases" } ] }, "name": "y", "nice": true, "range": [ { "signal": "height" }, 0 ], "type": "linear", "zero": true } ], "style": "cell", "width": 700 }, "image/png": "iVBORw0KGgoAAAANSUhEUgAAAuMAAADyCAYAAADnaMuzAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzdd3Rb9fk/8LeGJe+9RzzjmWEnIYHEhCxDoEBDIKSkkzJCoVDKCLO/0k0olBYKpw17Hmgp60uAxE4owUkaEsfOdIZXvIdk2bIly1r394diOYot+dqxbNl6v87htLq60n38iWQ/+uj5PB+JIAgCiIiIiIhowkknOwAiIiIiIm/FZJyIiIiIaJIwGSciIiIimiRMxomIiIiIJgmTcSIiIiKiScJknIiIiIhokohKxt955x38+c9/hl6vx+LFixESEoLnn3/e3bEREREREU1rEjF9xlesWIEFCxZg3rx5uOmmmxAeHg6z2QyVSgUfH5+JiJOIiIiIaNoRNTPe2tqKtLQ0VFRUYNasWSguLoZWq0V7e7u74yMiIiIimrbkYk4KCAjAN998g8rKSsydOxdKpRIAOCtORERERHQBRM2Mr1y5Eu+++y7Ky8tRVFSEI0eOICQkBJGRke6Oj4iIiIho2hJVM24ymfDuu+8iODgYa9aswZ49e9De3o7rrrtuImIkIiIiIpqWRCXjANDc3Izi4mIsXrwYCoUCfn5+iI6Odnd8RERERETTlqia8SNHjmDZsmXo7OzExx9/jP/85z84cuQIysvL3R0fEREREdG0Japm/Pe//z1CQ0MRHh4OAPjxj3+MQ4cOQa/XuzU4IiIiIqLpTFQyXllZiZ///OeIiooCACQmJkIQBHR2dro1OCIiIiKi6UxUMh4ZGYmjR48CAIxGI1566SXIZDLWjBMRERERXQBRNeM33HAD7rrrLgDA+vXrIQgCrrvuOigUCrcGR0REREQ0nYlKxm+//XY0NTXhtddeg06nQ1FREV588UV3x0ZERERENK2Jbm1IRERERETjS1TN+DvvvIM///nP6OvrQ2FhIUJDQ/HCCy+4OzYiIiIiomlN1Mz4ypUrMX/+fMyfPx/f+973EBoaCqvVCpVKBR8fn4mIk4iIiIho2hE1M97S0oK0tDRUVFQgLy8PO3bsgFarRXt7u7vjIyIiIiKatkQt4PT398fu3btx7NgxzJ07F0qlEgA4K05EREREdAFEzYyvXLkSb7/9NsrLy1FUVISjR48iJCQEkZGR7o6PiIiIiGjaElUzbjQa8c477yAoKAjXX3899u7di7a2Nlx33XUTESMRERER0bQ0qtaGGo0GGo3GfjspKYmlKkREREREYySqZvz06dNYu3Ytjh496nC8qqoK6enpbgmMiIiIiGi6E1Uz/rvf/Q5VVVXw8/NDUlISZDIZ0tPT7Qs5iYiIiIho9EQl40eOHMGmTZuQmZmJl156Cd988w0SExORmJjo7viIiIiIiKYtUcm4yWRCQkICAgMDcezYMURGRuKbb75BZ2enu+MjIiIiIpq2RCXjcXFxaGxsRHZ2Nu6//35kZ2dDLpezTIWIiIiI6AKIWsB5//33Q6fT4Qc/+AEOHDiApqYmPPHEEwgICHB3fERERERE09aoWhsSEREREdH4EVWm8vjjj+O2226z3/7nP/+JoqIitwVFREREROQNRCXjn332GWJiYuy3Z86ciZKSErS1tbktMCIiIiKi6U5UMi6Xy9HY2Gi/3dDQYD9ORERERERjIyqbLigowKuvvgqz2YyIiAi88cYbSEpKQkREhLvjIyIiIiKatkQt4KypqcGSJUvQ2toKwDYj/vbbb2P9+vVuD5CIiIiIaLoS3U2lo6MDn376KXQ6HZYvX47Zs2e7OzYiIiIiomnNLa0NDx48iOLiYvvt5cuXY+HCheN9GfJCgiBg38l27DrWgvr2HgBAQkQALp0VhyU5sZBIJJMcIbnS02fEFwfqcahWjc6efgT4ypGdGIbV85OQGBk42eERERFNOLeswNyzZw/27t2Lq666CgDg6+vrjsuQlxEEAW/sOIm9Jxy7+NS196BuZw+OndHgttU5kDIh90jtXX145qMKdOmM9mM9fSbsP92O8hoVbl+di7mpXIdCRETexW3tUBYsWIDbb7/dXU9PXmjfyfYhifi5DlZ3YNfRUCybHT+BUZFYr+844ZCIn8tsseL1khP43Q8XItDXZ4IjIyIimjyiWhuOxZEjR/Dyyy+jvLzcXZcgL1N6vGVczqGJ19KpQ3WL1uU5+n4zDlapJigiIiIizzDizLjVasXOnTtRWVkJAMjOzsayZcvg4+N89io5ORlNTU1oamrCU089hbvvvht33333+EVNXqm5Uz/iOS2degiCwNpxDyPm3852ns7NkRAREXkWlws4m5ubcfXVVw+Z3c7JycEXX3yB5OTkES9QVVWF5cuX2zcKAoDS0lLs3r3b4byoqCgsW7ZstPGTF/nDRyeg77e4PEcuk+KJG7KZjHuYY409eLe0fsTzLsmMwNXzYicgIiIiosmRlpbmcNvlzPhvfvMbHDlyBJdddhnq6+vR2NiItWvX4oMPPsBPf/pT7NixY8QL+vj4wGJxTKAKCwtRWFjocGzz5s1DgpsINTU1k3LdqcDTxiYnVYvTzd0uz0mKDER6errbY/G0sfEkw42Nf5genx3uGvGx+dkpSEubvjX/fN04x7FxjmPjHMfGOY6Nc5M5NjU1NUOOuUzGDxw4gEceeQS//e1vodPpEBERgZ/85CdYsGABNm3ahO7uboSEhAx53OOPP46goCAolUq8+uqruPPOO8fvpyCvdWle3IjJ+KWz4iYoGhqN2DB/zIwPcfnv56+UY35G9ARGRURENPlcLuAMDAxEWVkZTp48iZ07d8JoNKK3txeLFy+GIAjQaodfkLV27VoolUpYrVa88MILePzxx90SPHmXhZnRWJLrvIRhwcwoXJrHZNxT/WRVNsIClcPe5yOX4uZV2QjwdVuDJyIiIo/k8i/f+vXrcdddd+Hzzz8HAPj7+2Px4sX4+uuv4ePjg6ioqGEfN2/ePMybN2/8oyWvJpFI8KMVWchJCsPXR5pR39ELQRCQGBmIS/PicEl2DGvFPVhksC8e/958bDvYgIoaFVTdBgT5+yArIRRXLpiB+PCAyQ6RiIhowrlMxu+44w50d3fj/fffh7+/P379618jPj4eOTk5+OSTT7iZD02Ki2ZG46KZLGeYigJ9fXD94jRcv5h1jERERMAIybhUKsUjjzyCRx55xOF4fn4+8vPz3RoYEREREdF0N6ZNfw4cOIAFCxagsbFxvOMhIiIiIvIaY0rGe3p6UFZWhv7+/vGOh4iIiIjIa7gsU8nIyEB1dfVExUJERERE5FXGNDNOREREREQXzmUyHh8fjw0bNkAQBIf/du7cOVHxERERERFNWy6T8aKiInR0dAw5HhQUhPnz50OpHH4DDyIiIiIiGpnLmvFf/epXwx5fsGABDhw44JaAiIiIiIi8xahqxpubm1FSUoK+vj53xUNERERE5DVGlYxv27YNRUVFaG5udlc8REREREReg91UiIiIiIgmCZNxIiIiIqJJMqpk/Pvf/z46OzuRmprqrniIiIiIiLyGy24qA3bv3o0dO3bAaDRi7ty5WLNmDaRSTqoTEREREV0Il8m4IAi44447sGXLFofj8+bNQ0lJCcLCwtwaHBERERHRdOZyevtf//oXtmzZAl9fX6xZswYbNmxAXFwcDh48iAcffHCiYiQiIiIimpZczoz/5z//gY+PD0pLSzF//nwAQHd3NxYuXIh///vfeOmllyCRSCYkUCIiIiKi6cblzHhVVRWWLFliT8QBICQkBD/+8Y+h1WqhVqvdHiARERER0XTlcmZcp9MhNTUVGo3G4XhwcLD9/sjISPdFR0REREQ0jblMxi0WCz788EN8+OGHw95vNptdPnlnZydeeuklrF+/HikpKWOPkqYsg9GC7eUNOHC6HSqtAUofGTLiQnDF/CRkxIVMdnhuodUb8WVZAw7VqqDuMSDIT4GshFBcdVEy4sP9Jzs8IiIi8iCiWhuO1aOPPopPP/0U8+fPZzLuhXr6jHjmo8No6dTZj+n7zThcp8aRM2psuGwmls6Kn8QIx19blx7PfHQI3Tqj/ZhWb8T+0+2oqFXh9ityMSc1YhIjJCIiIk8yYs24IAhO/0tPT3f62G+//Rb9/f2YO3fuuAdNU8P7u6odEvFzCQLw3q4qtHXpJzgq93q95KRDIn4uk9mK13echM5gmuCoiIiIyFO5Zeces9mMBx98EH/84x/d8fQ0Bej7zThY3eHyHItVwN7KtgmKyP2a1DrUtGpdnqMzmFBeo5qgiIiIiMjTuUzGMzIyIJFInP5XXV097OP+9re/Ye3atYiLi3NL0OT5WjV6WKzCiOc1O5k5n4qG+1msVgG9582EN6unz89MREREF0YiCILTjCkjI8Npwg3YyljOL1WxWq2IiYnBvffeC7lcjldffRVLly7Fww8/bD+3tLQUu3fvHvJ869atG+vPQR6mQa3HP4prRzwvNzEY3y9MmoCI3O9wfTfe39Nov91rsODbqk74KmRYnBluP74kKwJXFcRORohEREQ0ydLS0hxuu1zAWVZWBqvV6vT+kJDhu2E88MADLoMoLCxEYWGhw7HNmzcPCW4i1NTUTMp1p4ILGZvYBDPe39854uz4nMxkpKVNvcW9w42Nb6geXxwdLFMJCgJ8zmihN1oAuS+C/RUAgIKcNKSlTd9knO8p5zg2znFsnOPYOMexcY5j49xkjk1NTc2QYy6T8UcffRTLly/HNddcA6VSKeoiUqkUDz30kP32f//7X6xfv97lYk+afvyVcszPiMK3p9qdniOXSXFJTswERuVe8eH+SI8LQXVLNwBAIgHiwwNQ26ZFo1qHXH8FAnzlKEhnb34iIiKycVkzvm3bNqxbtw6xsbH40Y9+hJKSErioahnWunXr2NbQS61fmoH48IBh75NKJLjpsgxEh/hNcFTu9ZOVWQgNGPzgmhBp+/kbVb3wkUvx06Ic+Cvd2lGUiIiIphCXyfgf/vAHFBUVoaenB2+99RaKioqQm5uLP/3pT2hsbHT1ULuf/vSnyMjIGJdgaWoJ9PXBw+sKcM3CFMSFB0AukyLQ1wcF6ZF48Pp8FOZOvwW+0aF+ePx787AqPxFRIX6ICfVHoK8PBAG4aelMzEoOH/lJiIiIyGu4nKJbv3491q9fj5aWFrz33nt45513UFZWhkcffRSPP/44ampqkJycPFGx0hSk9JHh6oXJuHqh97xOgvwUWFeYjnWFttKsJ/9djg92V6OiVoUludO3VpyIiIhGT1Sf8bi4OPzyl7/Ep59+iltvvRWArWuK2Wx2a3BE00FRQSIAYPvBhlGXeREREdH0NmLxam9vLz766CO8/fbb2LFjBywWCwBg9uzZCA4OdnuARFNdQVokokL80KTWobJBg9wZLFUhIiIiG5cz4/fcc4998eb27dsRFhaGe+65B2VlZTh8+DCioqImKk6iKUsqlWDFnAQAQElF0yRHQ0RERJ7EZTL++eefw2AwYNWqVfjXv/6FpqYm/O1vf8O8efMmKj6iaWGgVGXbwXqWqhAREZGdy2T8t7/9LVpaWlBcXIyVK1dCrVa73ASIiIY3NzUCsWH+aOvqw7F6zWSHQ0RERB7CZTK+YcMGlJWVIS8vDxEREYiPj0dMTAyeeuopzu4RjYJEIsGKubZSle0HGyY5GiIiIvIULpPx/fv349prr8Xx48ftx1QqFR566CH8+c9/dntwRNNJUb6tVKWkohFWKz/MEhER0QjJ+IsvvgiLxYJnn30WHR0d6O3txdatWxETE4PnnntuomIkmhZmJYcjPjwA7d19OHKmc7LDISIiIg/gMhk/fvw4vvOd7+Dee+9FZGQkAgICcNVVV+GRRx5BU1MTuru7JypOoilPIpFgZb6tVKW4nKUqREREJHLTn/OxXpxobIrykwAAxeWNsPJ9RERE5PVcJuO5ubnYunUrnn32WbS2tqK7uxtbt27Fk08+icTERISEhExUnETTQu6MMCRFBULdY0BFjXqywyEiIqJJ5jIZv+uuuyCTyXDfffchLi4OoaGhuPrqq9HW1oZf/OIXExUj0bSycq5tISdLVYiIiMhlMr5gwQJ8+umnyMvLsx+LiorCU089hfvvv9/twRFNRwMbAJVUNMLCripEREReTT7SCatXr8bq1auh0WhgMBgQExMDqXRMpeZEBCArIRQpMUGoa+vBweoOXDQzerJDIiIiokniMqvetWsXNm7ciLa2NoSFhSEuLg5SqRQPP/ww/vGPf0xUjETTzmCpSuMkR0JERESTyWUy/sILL+B///sfYmJiHI6bzWY88sgjsFqtbg2OaLoaKFXZeaiJpSpERERezGUyfvLkScydO3fI8VmzZqGrqwvt7e1uC4xoOsuIC0FabDC6dP3Yf4rvIyIiIm/lMhkPCgrCmTNnhhyvq6uDRCJBUFCQ2wIjmu5W5Z8tValgVxUiIiJv5TIZz8vLw65du/CnP/0JOp0OJpMJn3/+Of76178iJSUFAQEBExUn0bRzeYFtA6CvDjfDZGbJFxERkTdy2U3lzjvvxOuvv45HH30Ujz76KHx8fGAymQAAv//9750+7syZM9i2bRu6urqQk5ODq6++GhKJZHwjJ7do69Lj8/31ONHYhYZWFZLj2zE7OQJXLpiBsEDlZIc3rcSF+8NPKcOJRg3Wb96O5OggZMSH4PKCJMyMH9uGWvtOtmHX0RbUd/RAAJAQEYDC3DgU5saBb0Gi0TtYrcJXh5twpr0HVkFAXJg/FufE4rLZ8ZDyTUVOVNSosPNwE+raemCxCogL98cl2bFYPoevG7GqW7qx7WADqlq06Os3IyrEF/MzonDFvBnwVcgmO7xx5TIZnzNnDj7++GPccccdOHPmDEwmE4KCgvDYY4/hrrvucvq4ffv2ob29HZGRkXjuueewc+dOPPvss+MePI2vk41deGHrUfSbLAAAQRDQ2dOPr48240BVO+5bMxeJkYGTHOX00Gsw4ZkPKyCVSGEyW1Hf0YuoED8crlXjSJ0a31s6E8tmx4t+PkEQ8NZXp7D7eKvD8bq2HtS19eDomU5svDKXfwSIRuG9XVX46nCTw7H6jl7Ud1ThSF0n7vxOHuQytvolR//ZXYPt523q1tDRi4aOKhyuVePnV8+Cj5yvG1e+OdaCd/97GlZhsMFBW1cfPj9Qj4PVKjywNh9Bfj6TGOH4GvHVsHr1atTW1qK6uhrHjh2DSqXCQw895HKm+8Ybb8Tjjz+OO+64Ay+88AK++OKLcQ2axl+/yYJXiivtifj5dAYzXtl+wuGNQWP3r2+q0dypR2KkrdSruVMH69muKoIAvL+rCi2dOtHPd6CqY0gifq6KGtWQpIKInDtUq3b5njlW38nWpDTE8XrNkET8XCcaNfjyINcJudLe1Yf3dlU5zTdaNXq8t6tqgqNyL1EfzSQSCdLS0pCbmwuFQiHqicvLy/Hcc89h48aNuO666y4oSHK/I3VqdOuMLs9p7tShtrVngiKavgxGC8qqOgAAgb4+CA1QwGS2oq2rz36OVRCw90Sb6OcsPeY8ER88p2X0wRJ5qdLjI79fxJxD3uWbY80jnlN6rAUCJ7ac2nuiFWaL63VUB6s6oDOYJigi93Pr9yQWiwUBAQG47LLL3HkZGgctmr6RT4ItIacL09ald/hFkxBhK/1pUvc6nNfcqRf9nOc/Vqsf+sGqVdPHbzaIRGpWj/y7TqU1wMjF13SO8/+WDvdtc5euH33G4b+FJqBF4/i3r6fPhPP/clkFwWECa6pzWTN+IQoKClBQUIBbbrkFmZmZaG5uhlRqy/1LS0uxe/fuIY+pqalxVzguTdZ1PUlrazu0Wu2Q4+cfa25uRo3f9HkDXIixvm6aOvscxjXUV4DVakVDRw8yon0hPVsBplYLoq+h1WrR02ebJegxmLH3VCfCAnxwUXqY/RypRILamtoJWcjJ95RzHBvnPGlsurq6oO11/W0hANTV1UIudf+bypPGxtN40thoNJ3Qavvtt0uOtEMqlWBZbhTOfZnU1tbCbwIWIXrS2IilUqnsfyMFAfjqeAekEgkWZ4ZDcU6tfUNDA6D3G/N1PGlsRCXjR48eRUREBOLi4kZ9AUEQ0NvbC6vVak/GCwsLUVhY6HDe5s2bkZaWNurnv1A1NTWTcl1Po7EG4UB9v8MxrVaL4OBgh2MLZs1EWpzjMW90Ia+b+EQLwvd32mfHgwFEBPdB02uA3iJDfLitjnz2zCTR18hJ7cWJRg0AoKGrE1KpFBEhAQ7/fgkRAUhPd/9rne8p5zg2znna2OSm6XG4Tu3ynMhgX2RmpLs9Fk8bG0/iaWOTl26A4WwZYq/BBEik8FP6IDRk8HdxaIACuVkZbu8y52ljI9acDikaum173LRq9LAKEgT5KxAZHmo/RyqRYMHsTAT4jm0R52SOzXAfAkSVqaxZswavvvqq/fb+/fshkUhQXV097Pn33nsvfve73+G5557DFVdcgZtvvhlyudsm4WkczE6JQGiA6/UACREBSItlIn6hfBUyLMiIcjiWEOkPAGhU2b4al0klWJwTK/o5L80b/KDcdPbr9fM735x7DhG5Jub9snSW+I5H5B3Ofd1oz67DOr/rx6V58Wz37MIl2TH2LkWNA3/PIhz/ns3PiBpzIu6JXCbjarUaZWVl6O/vR3NzM8rKylBWVobi4mLbg6XDP3zjxo0ICwuD2WzGH/7wBzz//PPjHzmNK4VciluvyIXSZ/ivzQJ9fXDrFbnsVT1Obrw03SFZTooMhEQiQatGD4tgxfeWzkRsmL/o55ufEYlL8+LQ1duP3j4TfBVyRAT72u+flx6Fy0bRKpHI281JjcDKuQlO75+VHG7fRZdoQE5SGFbPnwEA0J4tHQz2V5x3f9KkxDZVRIX4YcNlMwEB9q5iCZGDm0zGhftj/VL3fyM1kVxOV3/++ef40Y9+BAB48cUX8eKLL9rv8/X1RWzs8DN3OTk5yMnJGccwaSLMjA/B4+vn48uD9Ther0FvjwQRQb6YkxqB1fOTEBrATX/GS4CvDx66oQDF5Q3Yf7odHd0GRIX4orfPhKV58Vg6a3Sz2BKJBD9YnomKGjV8FXIkRwdCKZciMTIQhblxWJwTw5kYolG68dIMzIwPxVdHbJv+WCwCEiIDsCQnDoV5sezbT8O67pJUpMcG4/5X9kAqlSA8UInk6CBckh3DzaJEWpIbi3pVD74sr0eovxKhAUpEhfhhfkYkLi9IcjpxOFW5TMbj4uKwatUq7N69G/Hx8UhNTQUABAUF4Yc//CH8/MZeOE+eKTrUDz9akQVg6tabTRUKuRTfuSgZ37koGQDw7ten8ZePDuHYmc4xPZ8gCDjV1IWYUD/8/Y5LkZ8WOZ7hEnmlgvRIFKTzvUSjMyc1AuFBSiRFBuLPtyxGdmLoyA8iB8fOaBAV7Ie7r5mNH6/Mmuxw3MplMr5q1SqsWrUKd999N5YvX461a9dOVFxEXmdVfiL++slhfHOsBTqDGQG+o1tncaxeg+ZOHSKDfTEnJcJNURIR0UjMFtvOylKpBMnR3Ll6tIxmq71nu6tyselC1F/7559/HnV1dfjss8/Q1zfY1u7KK69EYCBfZETjITrED7OTw3GoVo3S4y24Yt7o6gqLz+76dvm8JEgnoNUaERENr76jFyazFYmRgfBTsIHFaJUet01K5c4IG9KMYDoS9QrZsmUL7rzzTlgsjk3qq6qqmIwTjaOigiQcqlWjuLxhVMm4IAjYeXbr7lVzuaiMiGgy1bTa+mSnswPZmBSXNwIAivK9Y7GrqGT8n//8J0JCQrB8+XKEhoZCJrMVzp/fg5qILkxRQSL+8tEh7KlsQ6/BhECRrZsO13WipVOPmFA/zE4Jd3OURETkij0ZjwuZ5EimHoPRgtJjLZBIJFiZP/1LVACRyXhUVBTWr1+PTZs2uTseIq8WEeSL/LQIHKxWYdfRZly1IFnU484tUWHXFCKiyVXV3A0ASIsNmuRIpp5vjjWjz2jGnNQI+yZ4052oZHzNmjV47rnnkJeX59DOcNasWVAq2e6OaDwVFSThYLUKxeWNopJxq3WwRMVbvtIjIvJkNW22mXFulDd6xRW2EhVvKrkUlYw//fTTqK6uxtVXX+1wvKqqCunp06vxOtFkW5WfiKc/rMDeE23Q6o0OG0YMp6JWjfauPsSH+yMnie2ziIgmk8lsRUNHL2RSCVJiODM+Gvp+M3Yfb4VEIsEKL+iiMkBUMr5kyRJ7j/Fzsc840fgLC1RifkYUvj3Vjq+PNuOahSkuzx8sUZnBEhUioklW194Di1VAcnQQFPLptTmNu+062ox+kwUF6ZGj2oV6qhOVjL/xxhvujoOIzrEqPxHfnmpHcXmjy2TcahXw1UCJSoH3fKVHROSpBhdvskRltLyti8oAqdgTX375ZaxatQpff/01nn32Wbz55pvujIvIq62YmwCZVIJvT7WjW2d0el5ZdQdUWgMSIwORlcASFSKiyVbdYlu8mR7LTiqjoTOYsfdEK6RSCZbPiZ/scCaUqGT89ddfx2233YYdO3agq6sLRqMRmzZtgiAI7o6PyCuFBihxUWY0zBYrvjrS5PS8gVmEyzkrTkTkEQZmxtlJZXT+e6QJRrMVBWmRiArxrjJoUcn4G2+8gTVr1iA52dbZYdWqVWhra0NnZ6dbgyPyZgNf0w0k3OezWAX894htu+CiAu/6So+IyFNVs8f4mNhLVLxwcklUMq5Wq7Fs2TL4+voCgH1G3GQyuS8yIi+3fE48fORS7D/Vjs4ew5D7B44nRwdhZjx/6RMRTTaj2YomlQ5ymRRJXrCN+3jp6TNh36k2SCUSrJjDZHxYmZmZePPNN6HValFcXIzbb78doaGhiImJcXd8RF4r2F+BhZnRsAqDM+DnKq6wdVG5Yh5nxYmIPEFtmxZWQcCMqED4yEUvy/N6Xx1ugslsxYKZUQgP8r79a0S9Uh588EEcO3YMLS0teOGFF1BeXo5HH32UbdSI3GygVGX72faFA8wWqz1BX5XvfbMIROyzboEAACAASURBVESeqLqFJSpjMVii4p2TS6JaGy5atAj79u3Du+++i97eXlxxxRW49tpr3R0bkddbPicBf/xXGQ5Wq9DR3Wdf1LLvpK3LSnpcMHd4IyLyEAOdVLh4Uzyt3ogDVe2QSSVYNtu7uqgMEJWMA8DcuXMxd+5cd8ZCROcJ8JXj4uwY7Dragq8ON+HGSzMADJaoeFsvViIiT1bDxZujtuOQrURlcU4swgK9r0QFGKFMZf78+QgPD3f6X21t7UTFSeS1BhLukgpbi0OT2YpdR1sAACvzvWe7YCIiTzfYY5zfWIo1sIu0N5dcupwZ7+7uhkajcXq/1Wod94A8VU2rluUANCkumx0PpY8MFbUqtHf14USjBlq9EZkJIUiN4WuSiMgT9BnNaO3qg0IuQyI7qYii6e1HWVUH5DKp15aoACMk41VVVWN6UpVKhc8++wxqtRqLFi1CYWHhmJ5nsnXrjPiirB6H69TYfbwFi7JikJ0YhivnJyEuPMBt1xUEAftOtePrI81o6OgFACRGBuLSvDgszolxWDh79EwnSioaUdumhdkiICbUDxdnx2DFnATIZVzJPR34KmRIjAhE6fEW/OCZHejS9aOtS49rLkqZ7NCIPIrJYkVxeQO+PdWOjm4DfGRSpMUGo6ggETlJYZMdHk0wfb8ZX5bVo7xGBVW3Af5KOTITQnHFvCSkxAzWdJstVpRUNGLfyTa0dxsgl0mQFhuMVfmJyJsRLvp6ta09sFoFJMcGQib17AYXfUYzvixrwMEq2y7OfkoZMhNCsXr+DKREu7fe/VCtGjsPNaGuXYuTjV1oVPdiUWYsAnx93HpdTya6Zry3txeVlZXo7u62H1uyZAn8/IbukvSLX/wCMTExSEtLw8aNG3H33XfjjjvuGJ+IJ0hzpw5/+egQevpsvdSb1Dpoevux72QbDlZ14K6rZ7nll7sgAK+VnMS+k20Ox2vbtKht0+J4gwa3XJ4NqUSC//u2Dp99e8bhvCa1Dv/ZXYOKGjV+ce1sKH1k4x4jTRyzxYq/f3YUvQYjjGYL6tp60GswwmS24lRLF/5dWo11hemTHSbRpOszmvGXjw6h/uwEBmB7/xyr78TxBg3WLk7F5V7aqcEbaXr78fSHFVBpB/do6DWYcLC6AxU1KtxclI2FmdEwGC346yeHUdumtZ9ntgDH6zU4Xq/Bdy9OwVULkkVds7p1YPGmZ39j2aXrx9MfHkJHd5/9mM5gRnm1Codq1PjxqixcnOWe1tUf7a3Fl2X19tv1Hb0wmqzoM5rx108O455rZ8PHCycSRf3E+/fvR2pqKhYuXIiioiL7f83NQ3sfA8Bbb72Fv/zlL/j5z3+OF198EW+//fa4Bu1ugiDg1e0n7Im41SqgR29C79nbJosVr2yvhMFoGfdr7z3ROiQRP9eB0+0oPdaKqpZubN1f7/S86pZu/N+3deMeH02sL8vqUdmgQVx4AOQyKTS9BpjMVoQFKhHo64OSikYcPcOdcIk+3FPrkIifSxAEfLinxun9NP28/dUph0T8XFZBwFs7T0LT249P9tU6JOLn+3TfGfuizJEMLt707GT8na9OOyTi57IKAt7eeQqdPf3jft0TjV0OibjBZIFKa4BMKkVcuD9ONXXhcxd5zXQmKhl/8sknIZFIEBISgkWLFiEsLAwXXXQRAgOHr4mSSgeftq6uDgkJU2uR2Zn2XjSoBn9p9xhMsAqCPTkHbLtFHapVjfu1S4+3iDqn9FiLfSdUZ/ZUtsE6wjnk2UqPtwIAZFIJYsP87ccTIgPOOWfk1wzRdGYyW11OYgC2bx13873iFTS9/ThW73y9G2DbKfN/J9uwt3Kk142A3Wd/D49kKvQY79YZceSM2uU5JosV+065Hpex+OaY4wRuk1oHQbCV1w7MhpceHzm3mY5ElamcPn0av/zlL/H+++/jiSeeQGZmJtasWTPiDpxtbW34zW9+g08//XRcgp0ozZ06h9s9eiMAWy/MhIiAc87Tj/+11Y7Pue1gPXQGs8MxiVQChUyKfpPjzHxRQSKC/BT22zqDCd06o9e2Cprq+oxmaHoHZycSIgLQqOqFRCJBYsTgB+HzXzNE3kal7Rvy+3A45/9up+mpuVM3JKFrVuvwv/M+sH1xoB59Rse/rwAQG+aPxTmxDs8nxsDMeJoHL6xv0eghJtdtVo//e+X85+zRGyGRSBzyKq3eCF2/GYFeVj8uKhmXSCQIDw9HSEgIduzYAYlEgqNHj6K9vR3R0dHDPqa3txfXXHMNnnzyScyaNcvhvtLSUuzevXvIY2pqasbwI1y486/b3NwFrXbwa6n2zl5YrVZ0aHqREDJYg93e3oaamvH9BKfVdqPvbPmLts9s/yDgwAqYYYXV6njt3t5eCCbHf9IzZ85A4yd6acAQk/VvMhW4e2z6TVaH12GADJBAQIBCCnO/HtqzebpCMHjcv5OnxeNJODbOjXVsOrT9Du8VZzr9LFN2/Kdq3BPh/LFpatUNeT3o9f1DOsCZLeZhu8K1dPaiU9MNucy2CFOjHPl1o++3oL6tCz4yCfq1bajpaR/LjzLuzo+7oa1X1HtFpZKO+2uuq6sL2nNKh9IiFYgLDoVCbnGIqa62Dv5K969386T3lKgsLTExEdXV1cjPz8fTTz+Np59+GkFBQQgJGf6rGJPJhHXr1mHDhg248cYbh9xfWFg4pMPK5s2bkZaWNoYf4cLU1NQMua4ssAfFlT3228bmPkilUvRbJQgOHvzEm5+dirS08V3kkJ3abf+q60ynGlKpFBnxIZiTEmE/Jzk6CAkRAdhT6fqrswBfH8zJnQmpZGyruocbG7KZqLFJjlc5zI4nRhoQFqh0eB3mpUd61L8TXzfOcWycu5CxSTJbEbVPPeLs+OyZCVNy/Pm6cW64sQmPMeLDco3D7HhwMJCRFOVw3nWLU/HlgQaH2fHdx1vR1qWH1iRFSpitq8isjLgRx/9wrRoKhQLZiaHISPeMRfXDjU1krBH/OagZcXZ8TmYy0tJmjGs8een90J92/JBy/ncIwf4K5GVnOHSNc4fJfE8N9yFAVM347373O1xzzTV49NFHcf311+OSSy7B22+/DaVy+PKHu+++G83NzfD398eWLVvw3nvvXVjkE2xGVCCSz2nto+2zzU7r+kz22ehgfwXmpkaO+7WXzhrss9l09quxxAjHNoqX5sWhMDd2xBfrkpzYMSfi5BnOfT0AthaX8cO8Hoi8mY9cOmL3B4nE9juRpr/QAAVmJbtuSaj0keHirFgsznF83Qysx2lS2f7+SiQSFIr4HVtzdhGop3dSCfZXOEzuDcdHJsXF2ePfTUXM36pL8+Lcnoh7IlHJeENDAwoKChATE4MPPvgAe/bswbXXXuv0/Pz8fFx55ZWoqalBTU0NGhoaxi3giSCRSPDTomwE+ytgsQr2mu2BRZwKuRS3XJ4DX8X4f42yKDMai3Ni0dljgN5ghr9SjvAgX/v9F82MRmFuHNLjQnDtIud9pjPiQnCNi/tpari8wLHPbVy4P/yV8nPuTxpVH1yi6Wrt4jSH3tHnkkgkuGFJOpKiuBGLt/jB8pmIChnaehmwLYj/0YoshAYo8N2LUx0S6PjwAEilEnR098FotmLNxSmi+m4PdlLx3MWbA76/LBPRTsZGKpHghysy3bLWLCsx1GWbSNs+LuM7Gz9ViCpTuf/++9He3o6bbroJP/zhD0fcxGeq9RQfTmyYP371vfl4ZfsJbN1/BhYLIJMCiZEBePD6fMSE+o/8JGMgkUjw45VZOFjVAV+FDCkxQVDIpUiKCsKleXG4OCsaAx8ar1pga84/sOmPyWJFbJg/Ls6KwfI5CR6/6QCNTC6T4ufXzMKuoy3YU9mKZrUOMpkEKdFBWDE3EXNTXc9wEHkLX4UMD1yXj52Hm7DvZBvauvqgkEuRHheCywuSkJng+UkSjZ/QACUeWz8P2w424GCVCiptHwJ8feyb/sw4+8FM6SPD/dfNxc5DTdh3yva6iQ8PgLrHgIK0CKwWmRwOdlLx7JlxAAgJUOCx9fOx7WADyqraodLaNkSaGR+C1fNnOFQGjLfvXpyCtNgg7DjUhLq2HpitVsSF+eOS7FgsmxPvtd/mi0rGb775Zvzzn//Eli1bsGXLFsybNw+33XYbbr75ZqelKtNBsL8CWYmhDit902JD3JaID7BaBZxu6UZMqD9evHOpy5nP3BlhyJ3BneWmM6lEgmWz4716q2AiMXzkUlwxLwlXzOPmPgT4KeRYc3Eq1lyc6vI8uUyKy+cl4fKzr5vPvj2DJ97dj8qGLtHXsndSiZ0aH/p8FTJ89+IUfPfiif8GfXZKBGaPUCrjbUSVqTz22GOoq6vDV199hVtuuQXHjh3Dz372MzQ2Nro7vklX3WLbUWvg0+7AbXeqqFWhvasP8eH+yOUWzkRERBNm+ZwEKORSlFV3ON0c51xavREd3X3wV8oRFzZ8+QeRK6L3HG1ubsa+ffuwb98+9Pfbujt4Q5H9wFdPRfm2T8w1LnbqGi8lFU22axYkecUYExEReYoAXzkuzo6B1Srgq8NNI54/kCekxgTzbzaNiahkfP369UhOTsbDDz+M+vp6bNy4Efv37/eKVksDXz0tn5MAH7kUTWodDMaRN5cYK6tVwM5Dtm8cigr4VSsREdFEG5iAG5gcc2Vw8abn14uTZxJVM15WVoaCggLcfvvt2LBhAwIDvWNFur7fjBaNHgq5bSFlUmQgalq1qGvvQXZiqFuuWVbdAZXWgMTIQLddg4iIiJy7bHY8lD4ye9lodKjz8pPqVlv5qqe3NSTPJWpmvKSkBAcOHMDtt9/uNYk4ANS2aSEIAlJigiCTSuwtiwY+BbtDcbltVvzygkS3XYOIiIic81fKsTgnFlargB2HXK+PG1y8yWScxkZUMh4fH4977rkHMTEx+OSTT/DYY4/hD3/4g7tjm3TntypKiw06e9w9izitgoD/HmkGwBIVIiKiyVSUb5sUG5gkc6am1bZjd0b81OikQp5HVDL+zDPP4Pnnn0d7u20b05SUFDz11FOwWq1uDW6y2T/txtiS8fRY986M7z/Vjs4eA5KjgzCTb2oiIqJJs3RWPPwUchw504mWTv2w53Tp+tHZY0CArxxRwb7DnkM0ElHJ+GeffYaf/exnyMrKAgBccskl0Gq16OjocGtwk+38HbUG2xu6JxnfXm7bqZQ9comIiCaXr0KGJbmxEAQBO510VRn8Bj2EnVRozEQl4319fQ6dUwZmyOVyUes/p6yq5oFFGbbylKSoQCh9ZGjR6KHvN4/rtcwWq71EZVU+68WJiIgm20DJ6PaDDcPeb0/GWS9OF0BUMp6fn4/nnnsOHR0dePnll/H9738fM2bMQETE9N1BSWcwo0NrgK9Chvhw2w6cUokEydFBEAQBtePcb/zbU+3o1hmRHhfMRSBEREQe4NK8OAT4ynGsvhONqt4h97OTCo0H0TtwmkwmdHZ24rPPPkNnZyeefvppd8c2qapbuyEIAlJjgiGVDn71NPCGG+9SlYEFIgO9TYmIiGhyKeRSFObGAQB2Hm4ecv/55axEYyGqziQ9PR1HjhzBF198AZ1Oh2XLliE7O9vdsU2qgWT7/E+7AyUr47mI02S24uujtjf5yvyEcXteIiIiujBFBUnYdrABxeUN+NGKTIf7as92UuHMOF0IUcl4ZmYmVqxYgX/84x/ujsdjOPu0645e43tPtEKrNyIzIQSpMXxDExEReYrFOTEI9PVBZYMG9R09mBFlm5RTaQ3o0vUj2F+BSHZSoQsgqkzlkksuQW/v0Fqp6Wygl/j5izLS7WUq49drfGC7XZaoEBEReRaFXIals2ylKiUVgz3HByftOIlGF0bUzHh2djb+3//7fxAEAXl5eZDJZACAjRs3IjR0em7Z7mxHrfjwAPgp5Gjr6kNPnwlBfj4XdB2j2YpdR9lFhYiIyFMVFSTh8wP1KKlowk+LcgAMTsqxRIUulKhk/JVXXoHZbMa7777rcPyGG26Ylsm4Vm+ESmuAv1KO2DA/h/ukUglSYoJQ2aBBbasWc1IvrKPM7uMt6DWYkJMUhqSowAt6LiIiIhp/l2THINhfgVNNXaht0yI1JnhwZjyWizfpwohKxh988EF0dXUNOT5dWxueu3hzuCb+abHBqGzQoHocknF7F5UClqgQERF5IrlMimWz4/HpvjqUVDTitityUc0yFRonopLxjRs3AgD0ej30ej0iIyPdGtRkG6lv6Hgt4jQYLSg93gIAWDEn/oKei4iIiNxnVX4iPt1Xh20HG3DbFbmodVLOSjRaohZwqtVqXH311QgICEBUVBRyc3Px7bffuju2SVNztlWRs76hA+0NL3QR5+7KVuj7zZiVHI7ESJaoEBEReapFWTEIC1Sirq0Heypb0dNnQmiAEmGByskOjaY4UTPj9913H7Zu3YqIiAjI5XJUVlbiuuuuQ0NDA6TSofl8V1cXvv76a5w8eRLf/e53kZWVNe6Bu1ONfWY8aNj7B+rDRrvxj8Uq4KvDTdh7ohWtmj58e6oNbV19WLt4+tXdE9Ho9BpM+OJAPSpqVOjs6UeArxxZiaG4cn4yEiMD7OcZzVYUlzdg/+kOtHfpofSRIS02GJcXJCErkb9LiNxFKoFtzVi9Brf87St09hggl0mw72QbFmZGD1vWSiSGqJnxw4cPY+PGjWhvb0drayv++Mc/orm5GWq1etjzt2/fjrfeegtvvfUWjhw5Mq4BT4SBJNvZoozYMD/4K+VQ9xjQrTOKek6TxYq/fnIY/y6tRqNKh36Txf6/lQ0afLS3dtziJ6KpRaU14PfvlaGkohEqrQFWQUBPnwkHTnfgT/8+iPJqFQBA32/GU/8px6f76tDSqYPFKkDfb8bRM5149pPDDm3XiGj8WAUBW76shEprQL/ZcvZ9arvv1eITeGPHSQiCMLlB0pQlKhlfvHgxUlJS7LPg6enpmD17NqKiooY9/8Ybb8QHH3yA/Pz88Yt0gnTp+qHp7Uegrw+iQoZv4i+RSOyb84itG9/67RmcahpcBNvSqYPZYkV4kBL+Sjm+LKvHicahi2SJaPp7veQENL39w95ntljx+o4T6Okz4oPd1WjoGH7PB0EQ8J/dNWhUedeeEEQTYdfRFhys7kBUiB98FYNFBcF+CgDA3hNt2HeyfbLCoylOVDIulUrx61//GmvXrsVNN92EW2+9FfHx8di4cSM2btyIjo4Od8c5YaqaXXdSGTCwelpMMi4IAnZXtjoca1TrAACJEYO14qXHWkYdLxFNbW1depxudr3+xGC0YN/JDnw7wh97qyBgT2XbeIZHRIC92YIEQHy4v/14kL9iyDlEoyWqZvyLL76A0WjERx99ZD+2bds2+//ftGmT01nyqUbsjloDizurRSTjvQYztPrBchazxYo2TR8kEgniIwbf1M2durGETERTWLNaP+RYfUcvDpx2TLy/LGuAvt/kcCw8yBfLZjt2YuLvEaLxJQgCWjoH36cJEYH2XCH4nGSc7z0aK1HJ+PPPP4/eXudffcbExIzqoqWlpdi9e/eQ4zU1NaN6nvFy7nX3H69Ff38/gmT9LuPxtfagv78f5ScbUFPjOnHX9Zuh1Q4m7S1dBpjMZoQFKGAy6GEy2I4HyEyTNgbOeFo8noRj4xzHxrnzx6apqcfh9wMA9PUZYLVaHY6ZzZYhx0wm05DHqtXWKTv+UzXuicCxcc7dYyMIArRaLcwW2/tPKQF8ztYVGPS9OPsnHJZ+ucf9O3laPJ7Ek8ZGVDJ+5ZVXjutFCwsLUVhY6HBs8+bNSEtLG9friFFTU+NwXU1/PZRKJS6ek4G0tGinjwuM6IPy0xp06DBi3IIgIDFWbZ8dP9bcB6lUitS4UAQHDybyuWnRkzIGzpw/NjSIY+Mcx8a54cYmMKIP/3fYcb1IcHAwsmY4/v5ZV5iOj/fWwmRxTMjPN3tmwpQcf75unOPYODdRY5OV0uWwXmNGjBE9fSaHv+GZCaEe9e/E141zkzk2w30IEFUzPlpNTU3YsmULTp8+jZKSErz++uvuuIxbiC1TiQ7xQ5CfD7p0/ejsMbg8VyKRoDA3FoCtRKW9S28rUQkPcDjv0ry4C4iciKai6BA/ZCa43k7bXynHoqxoXJzt+ltIqUSCJbn8PUI03s7/+5wQEehQogLA/neeaLTckoz39fWhpqYGy5YtQ2hoKOrq6txxmXGn0hqg1RsR7K9ARNDwnVTONdhRpWfEc69akIzsxDA0q/WwWAVEBfvCVyE75/4Z7BFM5KV+siob4UHDbxwil0nxk1XZCPJT4PolaZgRNfwGYRKJBOsK05EQETDs/UQ0dpfmxWHBzMFvqyKDfREbOrjma0luLBZmOv82ncgVUWUqo5WRkYEnn3zSHU/tVgOz4hlOdt48X0Z8CA7XqVHd2o0FM10vYPWRS/GL785G6bEWKOQyzIgOgp9CjpSYIKzKT8Ss5PALjp+IpqaIIF88vn4BviyrR0WtCqpuA4L8fZAZH4qrLkq2d2/wU8ix6YYClJQ34ttTbWjvNkDpI0N6bDAun5eEmfHifncR0ehIJRLcenk28maE4ZtjLWhU9SIpKhAzogJx2ex4XDSTiTiNnctkPCMjA7/5zW/w8ssvY926dbjzzjsnKq5JUdU8sPOm6xKVAakxth06xfYa7+0zoVmjQ2JkAN55YCXCRcy+E5F3CPCV4/olabh+ies6Rh+ZFFcumIErF8yYoMiICLB9+7Q4JxaLc1iOQuPLZTIul8tRWlqKU6dO4cCBAygpKXG4f8mSJfDz83NrgBNpIKlOiw0Sdb69vWGLuGT8q8PNMJmtuDg7hok4EREREY08M/6Pf/wDAPDaa6/htddec7i/qqoK6enp7otugtW0nS1TiRdXuz2ajX8AoLiiAQBQlJ80huiIiIiIaLpxmYw/88wziIqKwgcffICUlBTk5OQ43B8YOPxCoqlIEATUnJ3hHig/GUlEkC9CA5To0vVDpTUgMtj5bHeXrh/7T7VDLpMO2aSDiIiIiLyTy2Q8KysLr732GgIDA7F8+XKsXbt2ouKacO3dBvQaTAgPUiIscPiuBsNJiw3GweoOVLd0u0zGdx5qgsUqYEluDEICFE7PIyIiIiLvIaq14fPPP4/k5GTcf//92LhxI/7973+7O64JV91iW7yZHju6bgQDiz1HqhsvLm8EwBIVIiIiIhokqrVhaWkpVqxYAZPJBADYsmULfvvb3+JXv/qVW4ObSAPJtNhOKgMGFnu6qhvX9PbjYHUHfORSLJ3FDTmIiIiIyEbUzPjTTz+N0NBQPPvss3j11Vcxf/58PPXUU7BaXW/LPJWI3XnzfAOLPQcWfw6npKIRFquAS7Jjh+zYRURERETeS9TMeE1NDW6//Xbce++9AIDk5GSsXLkSbW1tiIubHjO91a2j6zE+YGBmvLqlG4IgQCKRDDlnoERlVX7CBUZJRERERNOJqJnx6OhofPnll6isrERzczPee+89yGQyREREuDu+CSEIAurabFvajzYZDw1QIjzIFzqDGe3dhiH3q3sMqKhRQSGXYuksdlEhIiIiokGiZsZvuukm3HrrrcjNzbUfW79+PRSK6VFy0aLpg77fjKgQvzGVkaTHBqOzx4Dqlm7EhDpuglRc3girIGBJbjwCfX3GK2QiIiIimgZEJeM333wz1Go13njjDeh0OqxatQpPP/20u2ObMPZOKqOsFx+QFhuM/afbUd2iHbJNbnH52Y1+ChIvLEgiIiIimnZEJeNSqRSbNm3Cpk2b3B3PpBhYvJkWM7Zk3NlOnO3dfThyphO+ChkKc6dHbT0RERERjR9RNePT3eDM+Oh6jA+w9xo/uwh0QHF5I6xWAYW5cfBXivrcQ0RERERehMk4zu0xHjSmxw8k8bWtPbBaBfvxwRIVbvRDREREREN5fTJutQqoa++BRCJB6ig7qQwI8vNBVIgf+oxmtHb1AQBaNXocq9fAVyHD4pyY8QyZiIiIiKYJUcn4hg0b8NRTT7k7lknR1t2PfpMFMaF+F9TtxF6qcrbkZXt5IwRBwNK8ePgpWKJCREREREOJSsbVajWam5vdHcukqFfbZrJH21/8fOnnJeMDJSqXz2OJChERERENT9SU7Y033oiHHnoIixYtQmZmpv34rFmzoFQq3RbcRKhX6QEAGWNcvDlgoG68prUHTWodTjR2wV8pxyXZLFEhIiIiouGJSsb/9Kc/Qa1WY8OGDQ7Hq6qqkJ6e7pbAJspAMn6hM+MDiz+rW7qx/WADBEHAstkJUPrILjhGIiIiIpqeRCXjS5YsQWpq6pDjfn5+w5xtc+rUKXz55ZfIysrCFVdcMfYIARyuVWPHoUbUtffAbBEQG+aPRZnRWDE3AXLZYKVNTasW2w424HRzN/rO7qg5Lz0Sq+fPgK9iMClu1ejxRVk9Khs02HqwBf1WCU40aFCYF4vQgNHP9BuMFlQ2dKG5U4dGlQ5l1R3o6zcjNyn0gn5uIiIiIpreRCXjb7zxxqie9MSJE7jmmmtw33334ZlnnsGxY8dw3333jSnAT/5Xi88P1Dsca1T1olHVi0O1avziu3OgkEux+3gr3v7qFKzCYGvBti5b0n2wWoUH1s5FsL8ClQ0avLj1KIxmKwQB0PVbIJHKUNmkwe/fO4j7rpuL+HB/0fFp9UY889EhtGr08JFJoTebodYaoJBL8dXhZiREBuLSPG74Q0RERERDiW5t+Morr6CoqAhff/01/vKXv+DNN990eu6LL76Ie+65Bz/72c/w7rvv4plnnoHFYhl1cKeauoYk4ueqaunGZ9/WQaU14N2vTzsk4udq69Lj3a9Po99kwavFJ2A0WwEAvQYTrALgr5RDLpWip8+IV7Yfh5OnGdZ7u6rQqrGVugT7K+zH48IDAInt/vaz7Q6JiIiIiM4lKhl/4403cOutt6KkpARdXV0wmUx46KGHUYoSCwAAIABJREFUIDjJWg8fPowFCxYAACIjIxEQEIDW1tZRB1d6fOTH7Klsw97KNpgtVpfnHapRY++JVmj1Rvuxnj7b/z83iW5U6VDXrh3y+OHoDGaUV6vst899nsTIAACA2WLF/062iXo+IiIiIvIuospUXn/9daxZswbl5eUAgFWrVuHhhx+GRqNBeHj4kPM1Gg2CggZ3swwODoZarUZCQsKogmvu1A05tr28Eb19Rodj//qmCn1G84jP9/mBM9D3Dz0v2N+xv3izWo/UmJEXdLZq9A6z8cF+tmRcIZchKniwnn64n4OIiIiISFQyrlarsWbNGlRWVgKAfUbcaDQOe35YWBi02sHZZa1Wi4iICPvt0tJS7N692+Ex/v7+2Lx5s8Oxb+qV6OqXOBzr7pbDaHE8Zu4HTK4nxs+eJ8BklQw5rmptREXnYDmMrGU/SoNHLqtR90lQ0Ti44NNglqCvTw5fpRWHDx+yH++osqL78GcjB0hERERE01Zubi7S0tIcjolKxjMzM/HWW29Bq9WipKQEe/bsQWhoKGJihu+hPWfOHBw4cACLFy+GSqWCTqdDbGys/f7CwkIUFhaOeN2o4hPYd16JR/555wT5KXDZrDh8tv+My+eSSiT43mXpePe/VQ7HKyoqkJ/v+KwPrisQNTOuM5jwwCt77bPjFquAln11WJATi+jQwZnxqxem4JqFySM+n6fZvHkzHnroockOwyNxbJzj2DjHsXGOY+Mcx8Y5jo1zHBvnPG1sRCXjDz74IJYuXQqj0Yi///3vAGw/iEQydJYZAO644w5ce+21kEql+Pjjj/HLX/4SMtno+21fmhc3JBk/X2FeLBbnxmJbeQNMZufT4/lpEbg4KxZb99ejWzf8jD4AJEUFIiVaXM/xAF8fzMuIxIHTHQAAmVSCsAAlokIGE3G5TIqLs6JFPR8REREReRdRCzgXLVqEffv24cEHH8Qdd9yBjz/+GJs2bXJ6fm5uLrZu3Qqr1YoHHngADzzwwJiCmxkfgqtdzChnJoTiOxclIyLIF99flgmpkw8HsWH+2LAsE0ofGW4pynG6EU+Qnw9uKcqBk6cZ1veWZiDunFaI2Umh9sdLJRJsuGymQ3JORERERDRA1Mw4AGRkZGDlypXQ6/X2TimuZGVlISsr64KCA4BrFqYgNSYYOw41ora1B2arFbGh/rgkJxbLZsdDJrVlvpdkxyAu3B/bys5u+mM0IzLYF/MzonB5QZJ905+sxFA8tn4evixrwPH6TkgARAb7Yk5qBK6cP8OhI4oYQX4KPLJuHraXN6CsSgW5TApfHxlmJoTginlJospdiIiIiMg7iUrGDx06hCuvvBItLS0AAKVSiRdeeAG33HKLW4MbMCs5HLOSh3ZtOV9KdBA2Xpk74nkxof748UrbB4XSmRYUFi66oPiUPjJcszAF1yxMuaDn8TRLliyZ7BA8FsfGOY6Ncxwb5zg2znFsnOPYOMexcc7TxkYiOGsWfo7Vq1ejuLgYl19+OUJDQ7F161aYTCZ0dXVBqRz99vFERERERATInnjiiSdGOumFF17A6tWr8f777+OGG27AzJkz8d577+H++++Hr6/vRMTpkkajwXPPPYfs7Gz4+4vfyn44PT09KC4uxieffILAwECHLjBtbW148803cfToUaSlpXnEzz6S7u5u/O1vf0NmZiYCAv5/e/cd1lSy9wH8GwRlXUWwLYsSViBiAUUFFLFgXb2Iiq7i4rUsS3HtEFGvXa+uomsDywWx4KqEIl1RbFjAAooQkCoiIIIIQaSIEOb9g5ezZAOIkkh05/M8PA85J5mc+eXMZHJmzsy3LU7L398fly5dQnl5ObS0tET237t3D2fPnkVaWho4HA7atv24IT+fy+HDh/Hy5Uv06dOH2Xb69Gnw+XwMGDCgxelXV1fDx8cHd+7cgbq6Ojp06AAA4PP58PX1xe3bt1FdXQ0NDdmbYefo0aPIyclB3759mW1nzpzB48ePMXDgwBan/+jRIwQEBCAhIQGDBg0S2RcTEwNfX18AQM+ePVv8XpLm5uaGrKwskdicO3cODx8+FJuR6VPExsYiICAA8fHxGDx4MLP91KlTuHz5MiIjIxEZGQk+nw9DQ8MWv58kubu749mzZ+jX76+eSS8vL8TExIh9zh+LEILIyEicP38efD4fWlpaYnXvs2fP4O7ujv79++Obb2TrHh0PDw+kp6ejf//+zDZvb2/cv39f5HP+VGFhYQgKCkJMTAxUVVWhrKwMAHj16hUuXLiA0NBQaGpqiqz9IWtOnDgBNpvd4u9vABAKhfDz88OtW7fQs2dPJt8nT55EeHg4U44SEhKaNeS2tZ06dQo9evRo8fc3UDt7XEBAAOLi4ho892S5HDXE09MTampqLY4NIQRRUVFMHaOpqcnkv3450tLSklo5atYNnNbW1sjJyUFVVRWA2g/MwsKCKfStzdfXF1u3bgWPx2txWpGRkfDw8GAqyzqFhYUYNmwYysrKkJ+fj8mTJ0Mo/PBc5K3t/Pnz2Lp1K86dO9fitNavX4/IyEh888032LBhA7Zt28bsO3HiBBYtWoR27dohMTERWVlZTaTUun7//Xc4OTmhpqZ29p3i4mI4Ojri6NGjEkn/119/xeXLl1FaWooxY8bg7du3AIDr16+joqIC7dq1w7Jly+Dh4SGR95OkXbt2YdWqVaiurl0cq6SkBA4ODjh8+LBE0t+wYQOioqLg4uIisv3KlSuwtraGoqIili5diuDgYIm8nyTt3r0bXC6XqQdLS0vh4OAAV1dXiaS/adMm3L17FwcOHBDZrqSkBBUVFaioqCAuLg7h4eESeT9J+uOPP8Dlcpm1J8rLy+Ho6IiDBw+2OO2YmBhs2rQJNTU1SElJgYGBAVOm6qxcuRK7d+9GYWFhi99P0vbt2wdHR0dUVlYCACoqKuDo6Ij9+/dLJP2QkBAoKiqitLQUJiYmePr0KQDAx8cHQUFBOHr0qEzXxwBw4MABvHr1SiJp/fbbbwgJCUFFRQXGjBmD4uJiAKLl6PHjxzJZjhri4uLySSuYN2Tz5s24e/cu9u3b1+B+WS5HDTl06BByc3NbnE5sbCw2bNgAoVCItLQ0DBkyhFkrx9vbG0FBQThy5Aiys7Nb/F6NIk0wNTUlmpqaRFNTk8jJyREVFRWipqZGAJAePXqQ58+fN/Xyz2bMmDHExcWFmJiYMNsKCgqIl5cXuXPnDtm/fz95/Pgxs+/EiRPk2bNn5MiRI+T+/fsNprl8+XLi5ubGPPb09CTz5s1jHltaWpLQ0FAp5EayJkyYQFxcXMjQoUOZbYWFheTs2bMkKiqK7N+/nzx69IjZd+rUKZKRkUGOHj1K7t69K5KWUChk/k9MTCQcDocQQsjbt2+Jqqoqyc3NlXJuJENNTY1YWVmRGzduEEIIOX78OLGysiLDhw8nhBBSXl5O/P39ibOzMzl//jyTb29vb5KTk8Okc+3aNRIbGyuSdlZWFmGz2cxr7O3tybFjx8SOwd/fn8yYMUMq+WsJNptNrKysyJUrVwghtee9lZUVMTIyIoQQUlFRQQICAoizszPx9fVl8unr60uysrKYdG7cuCFyXtWXlJRE9PX1RbZNnDiRXL58mRBCyL1790TKsqzQ1NQkVlZWJCwsjBBCyNmzZ4mVlRUZPHgwIYSQd+/ekcDAQOLs7Ex8fHyY2Jw/f55kZmYy6dy8eZPExMQ0+B5paWlEV1e30WMwNzcn3t7eksqSxHA4HGJlZcXUiTwej1hZWZGBAwcSQv6Kze7du4m3tzeprq4mhNSWg/qxuXXrFomOjhZJu369Qwgh48aNYz6DuvfauHEj6dOnD0lLS5NK/lqib9++xMrKigQGBhJCCPHz8yNWVlakf//+hBBCKisrSXBwMNm9ezfh8XhMbAICAkhGRgaTzp07dxr9vqqzcOFCcvLkSZFtpqamYnW5rNHT0yNPnjwhQqGQHD58mNmenJxMwsPDCSGEXLx4kTx58oR4enqSo0ePkrKyMrF08vLyiJqaGhPD5cuXk0OHDok9b8qUKcTHx0dKuZGsQYMGkfj4eEIIIa6ursz21NRUcunSJUIIIWFhYSQxMZGcPn2aHD16lJSWljaaXkZGBunbt6/YdlkvRw0xMDBgvoPrxyYtLY2pIy5fvkwSEhLIn3/+SY4cOULevn0rls7f65iJEyeKte9GjRr1wfLXEk1eGc/OzkZGRgYyMjJQU1MDgUDA/Ap58eIFc4WoNeXm5uL58+dYunQpCgoK8Px57eI/r169wuLFixEREQEDAwPY2NggLq52VcyDBw9i9erV0NbWRvfuzZsDXE5OTiS/VVVViI2NlXyGJCg/Px9paWlYsmQJSkpKmCsmr1+/xpIlS3DlyhUYGhrC3t4ejx49AgC4urpi1apV0NLSElvUSU7ur9Pl+fPn6NGjB4DaG3w5HA7y8vJw+vRpZGZmfqYcfjpLS0t4e3sDqL2CNGvWLGZfeno6kpKSoKGhgYsXLzILA+Tm5uLYsWPM81asWCHWO5SQkAB9fX0mVkZGRsx5BwDXrl3D3r17sWnTJkybNk1q+WuJ+rHx9vbG7NmzmX0ZGRlITEyEhoYGwsPDweVyAQB5eXlwc3Njnufg4PBR3Xnx8fFMl7GBgQHi4+MlkRWJayo2mZmZ4PP50NDQwPXr17FixQoAtXVR/V4XR0dHZujSxygsLERUVBTMzc1bmAvp+HtsLC0tmX1ZWVmIj48Hm81GREQEli1bBgAoKCgQ6XXhcrliXc716x2hUIjs7GxmGNObN2/g4uKC//znP1LLlyQ0FZvs7GzExsaCzWbjzp07+O233wAARUVFIr0uTk5OjQ4dOHHiBDZs2ICIiAiMHj1aijmRrpqaGuzcuZN5zOfzmV4yPz8/2NnZQVVVFQKBAHZ2dmKvT0xMhJ6eHrOuyd/rX6C2HN29exdTpkyRYk6kY8eOHcz/iYmJCAwMBAAEBATA1tYW3bt3R0lJyUdPrvGllKOm1I9NUlIS/P39AQBBQUGwsbFB165dUV5ejoULF4q9tn4dU1NTI1LHfC5NNsYfPnyIoqKiRv969er1uY6zUd7e3pg1axZYLBYsLS1Fhqp06tQJ69atw4gRI+Dk5AR3d3dm36pVqzBhwgT88EPzZkCZMmUKoqOjsWbNGqxatQqxsbEoKiqSeH4kycfHBzNnzoScnBzmzJkjEpuOHTti48aNMDExwdq1a0UaUlwuFxMmTGj08y0pKQGXy2WGqeTl5SEjIwM7d+7E8+fPMWrUKMTExEg3cy00atQoREVFITc3FxUVFWCz2cw+PT095svS0NAQAQEBAIB58+aBx+OhpqYG9+/fR/fu3cXOH4FAINIIVVJSEjtPqquroa6uLpFxxtIwfPhwPHjwAHl5eXjz5o3IedCvXz/8/PPPAGobzUFBQQCAuXPnwtfXFzU1NXj48CGUlJSgra3d7PesH7c2bdqAEMJ068uSoUOHIjY2Fvn5+SgoKACHw2H26ejoYO7cuWCxWBg0aBBCQkIAAD///DP8/f0hFAoRFxcHRUXFT5r21c/PD1OmTJHZsZyGhobg8/nIz89Hbm6uSB45HA7mzZsHFosFfX19XLhwAQAwZ84cBAQEoLq6Gnw+H/Ly8iJj8v9u165dGDx4MHR1dQEAGzduxOrVq2U2JnUGDRqE5ORk5OfnIzMzU2T8uJaWFhYsWAA5OTno6ekhLCwMADB79mwEBwejqqoKSUlJqK6uhp6eXqPvUVlZieHDh6Nbt25Sz09rWbhwISZOnIg1a9YgMjJSbH9z6l9fX1+Ym5vL/DnzsebPn48ff/wRTk5OuHfv3ke99kspR5/q3//+NyZNmgQul4vo6Ogmn7t7927o6upK5B6pj9FkY7xTp07MGKuOHTuitLQUAoGA+ZOFMdM8Hg+vX7+Gs7Mz8vPzRRqcPXr0YFYJVVdXR05ODrOvfuOrOZSVlRETE4MhQ4bgxx9/xOzZs/H9999LJhNSwuPxUFRUBGdnZ+Tl5YnERk1N7ZNi8/79e8yYMQNLly7FyJEjAQCdO3cGIQReXl7YuHEjnJyccPz4cSnlSjLk5OQwevRoLF68GDNnzhTZFxISgpkzZ+Lly5dgsVgQCAQAgC5dukBfXx83b97EyZMn8csvv4ilq6KiIjKe9c2bN+jc+a9pOceNG4c1a9Zg27ZtWLx4sZRy1zJycnIYN24cFi1ahBkzZojsu3jxIiwsLJhpTuvGY6qoqMDQ0BDXrl1rNDZNUVFRYcboVVdXg8ViyeRMTSwWCxMnToS9vT2mT58usi88PBzTpk3DixcvAPwVm06dOmH48OEIDw/HyZMnG7wy0xxeXl7MDyFZNWnSJNjb24v1+ly9ehXm5uZMbEpKSlBTUwMlJSWMHDkSly9f/mBs6sZu1vVOvXz5Ej4+PkhOToazszMKCwvh7u6O169fSy+DLWBmZgZ7e3uxno2IiAj861//YsajlpeX4/379+jQoQPGjh2LixcvfjA21tbW2LNnD7S0tHDkyBGp5qM11X3nysnJMfe11Peh+hf4MsrRp6iLDYvF+qi22ZdWjj5F/bZa3b1iDfHz88P58+dbpf3SrBs4w8PD8d1334HNZkNLS4v5a+2bQp4+fYr8/Hzm6pSmpiZKSkqQlJQEoLZLva7AJicnQ11dnXkt62OW2fx/ysrKmD17NsaOHYuQkBCYmppKIBfSkZmZiezsbObqlIaGBt69ewc+nw+g9ibcumE3zY1NTU0NFixYAGNjY5GGZL9+/aCoqMh09SgoKDR5wssKS0tLhISEiAxRAYDAwEDs2LEDK1euhIWFBd69e8fss7GxwZEjRxAWFibWUAUAXV1dPH78mMl/dHR0g7+wa2pqxG5CkyV1sak/DAOo7fLbtm0bVq5ciZ9++gnl5eXMPhsbG/zvf/9DaGioWEw/ZMCAAUxvSkxMTJNXAFubpaUlgoODG4zN5s2b4eDgAEtLS5SVlTH7bGxs4ObmhsDAQLHXNUd2djZSUlIwbty4Fh+/NNXFpv4wDAAIDg7Gxo0b4eDggDlz5ojFxt3dHf7+/mKvq3Pt2jVs374doaGhzBAfRUVFODg4SC8zEmZpaYmgoKAGY7N+/Xo4OjrCyspKLDbHjh2Dr69vsxqQhBCZrlc+RF5eHqTebMt1P/qbq1+/fuDz+Uxj9O/1b3Z2NtLS0mS+HDWmbvgN8PGxacyXVo4aU9ejCnxabG7cuIEtW7YgNDS0VWYeataiP1u2bEFxcTF0dHRExjq29pUrLy8vzJkzhxnTC9Teqc7j8WBpaQkWi4XFixdj6NCh2LlzJzOGqCmvX7+Gv78/EhISmO4tW1tbsFgs2NjYQE9PDxcuXMCwYcNkbnqx+ng8HmbPni0Sm6qqKvB4PCxYsABt2rTB4sWLYWxsjJ07d8LHx+eDaW7ZsgX379+Hqakp3N3d0bZtWyxcuBDfffcdxo8fD1tbWwwZMgS7d++WyOwt0mZiYsJU2vULr56eHs6fP4/u3bvDzc1NZIrGsWPHws7ODpMmTWpwGi51dXWMGjUKv/zyC/r27YuIiAjs2bMHQG2X/ODBg8FiseDu7g5HR0cp5/DTDR06lIlN/W7eutioqakx50Cd0aNHw9bWFuPGjWt0qqnAwEDEx8czV19MTEzQv39/ODo6gsvl4unTp/Dw8MDmzZulm8EWGDJkCPNj68mTJ8x2PT09+Pv7g81mw8PDQ6R+NDExga2tLUaPHg0lpYZX5Q0ODkZ8fDyKiorg7u4OY2Nj5kcJj8fDrFmzIC/f7EWTW4W+vj4Tm5SUFGa7np4eAgICoKmpCQ8PD5HzZvjw4UhPT8fIkSPRqVMnsTQzMjJgYWGBJUuWMGNkTU1N0bt3b5H67dSpU7Czs0PXrl2llb0W0dXVZRoLdffvALWxCQwMRO/evXHixAkoKCgw+4yMjJCVlYVhw4ZBRUVFLM24uDjs27cPBgYGyMnJgaenJyIiIgAAqampiIiIQG5uLoKCgpCTk4OffvpJyrn8NIQQ5iLQ4MGD4eLiAg6HgzNnzsDIyKjZ6aiqqmLy5MmYP38+9PX1cenSJdSfvdnLywuzZs0SadTKuvqxMTIywr59+9CvXz+cPn36o4c6hoSEgM/no7i4GO7u7hg2bBgGDBjwRZWj+urHxtjYGHv37oWenh48PT1FhoJ9SGZmJqZPn45FixYxQy9Hjx4NHR0dpKSk4ObNm3j58iUCAwORlZUllXLUrHnGeTweTE1NER4eDjs7O+avsS+VzyU5ORnTp08XOWnYbDby8/Pxww8/IDo6GmvXrkV6ejrWrl3LzCFdWVmJoUOHNjgPtkAgwOXLl9GjRw8oKSlBIBBg/PjxTNdPQUEBpk6dCi6X+0lX1z+X5ORkTJs2TWT8IJvNRl5eHrS0tBAZGYkNGzYgNTUVq1evZuYCrqyshJGRUYM/tF68eIFu3bqhuLgYAoEAb9++xZgxYwDUdsFWV1ejsLAQmzdvbvHcwtJUWVmJkSNHijRsampq0K5dOxgaGsLIyAgVFRVITU2FtbU1lJWVmSE5LBYLAQEBWL16daM3eJibm+P9+/dgsVhwdnZmukl79uyJ3NxcyMvLg8vlYurUqdLP7EeqrKyEiYmJSIOAEIK2bdvCyMgIBgYGqKqqQkpKChYuXIjOnTuLxCY4OBiOjo6NDnW6fv06Xr9+DT09PQgEAvTq1Qvq6urQ1taGkZERnj17Bjs7O4wfP/6z5Pdj1I3JrV9vEEKgoKCAoUOHMo30pKQkLFiwAF27dhWJTWhoKJYvX97ofSo3btzAq1evMHDgQAgEAmhoaDBxTE5OxowZM2R2PHBdbOrXG4QQyMvLY9iwYcycxk+ePMG8efPQrVs3jBgxgqlDL1y4gKVLlzZ4n0pRURHTeKobIsnhcMSGCVZWVsLY2Fgic1VLUt1x1Z8bnRCCNm3awNjYGPr6+mjTpg0SExMxd+5cqKqqwsTEhOlpDAsLw6JFi8TWdQBqh2XIy8sjPz8fampq2L9/P3N+ZWZmIjIykhmDLxQKYWxs/Bly/PF27NgBBwcHfPvtt5gyZQoSExNRVVUFa2trdO3aFX369EFVVRX69OnDfN/X1eN/Z2ZmBqFQCKFQiF27domUmeTkZFhYWMhsOWrIzp07sWzZMnTs2BFmZmZ48uQJ3r9/D2tra3Tp0gV9+/ZFVVUVdHR0mHw1FpuIiAjk5+czdQybzRZb70JWy1FDdu3ahcWLF6NTp04wMzNDcnIyKisrxWLD4XCYyToaio1AIACLxWKGpQoEAmhra0NNTU2kHBFCUFNTg2HDhkk8L81agfPOnTuYP38+1q9fL/LBmZiYyOyA/ydPnmDZsmW4du1aax+KzElNTYWtrS1u3rzZ2ofyRcnJycHZs2cRGBiIu3fvtvbhyJQXL17g3Llz8PX1xYMHD1r7cGTKy5cv4eXlhTNnzuDhw4cy/SP+c3v58iV4PB5Onz6NR48e0djUU3cP1PHjxxEXF/dVxubZs2dwcXHBzZs3mRm9qFrPnz+Hi4sLrly58tV+/p8qKysLrq6uCAsLA5/P/ypi06w+z/T0dGRlZcHGxkZse0O/1mVB165dMWfOnNY+DJnUuXNnWFlZtfZhfHEEAgGqq6vh5+fX2ocic4qLi/H+/ftmDQX7p3nz5g3Ky8sRGBj4VXxpSFJJSQnKyspobBpQUlKCkpISBAcHf7WxKSsrg7a2NtavX9/ahyJzysrK0KtXL1y9evWr/fw/VXl5OdhsNq5du/bVxKZZV8YHDRqElJQUGBoaisw97erqKjYXNUVRFEVRFEVRzdOsK+NaWlowMjISmYuaoiiKoiiKoqiWaVZjfPjw4di6dStUVVWZVReB2tkhWvsmToqiKIqiKIr6UjVrmIq2trbIVEx1ZHnMOEVRFEVRFEXJumZdGbe3t0dhYaHYdmVlZYkfEEVRFPVxoqKicPv2beaxioqKyDzlFEVRlOxq1pVxiqIoSnZt374dGzduFNtuYWGBs2fPNmsKWh0dHaSmpiI/P5+Zk5eiKIqSvmZdGd+wYQMKCgrEtm/fvv2Lmjyfoijqa7Z06VLs2rUL0dHRsLW1RUBAANauXYuDBw/i1atXuHr1KnJyciAvLw8OhwMzMzPIycnh1KlTzEqrrq6u6NChA1auXIl27dohMTERt27dYhZLk9WFYyiKor5UdMw4RVHUF67uyvjSpUvh6uoKALh06RImT54MZWVlFBUV4c8//8T69evxww8/4NWrV0hNTYW5uTmCg4MxdepUXLp0CVVVVdDQ0ECbNm0QFxeH48ePg8vlomfPnmjfvj2SkpKwatUq7Nmzp5VzTFEU9fVo1pVxV1dXlJaWAgDevXsHNzc3dOnShc4xTlEUJaP69esHoHZBpvLyclhaWoLD4SAhIQFFRUVwcXFBSEgI8vLyEBwczAxTefDgAbp3747S0lKsXr0aHTp0gL29PeTk5HDgwAH88ccfcHR0FFuOnqIoivo0zWqMT548WeTxgAEDYGJiAkVFRakcFEVRFNUyCQkJAGpv5mzfvj2cnJywd+9eDBgwADo6OqiurgZQuyS9qqqq2OufPXuG9+/fo0OHDoiNjQUAjBw5EkDt6pC0MU5RFCUZzWqM79mzB0KhECwWC2VlZbh9+zbKyspQVFREb/ShKIqSIWVlZXjw4AFWrFgBAJg/fz5YLBZCQkIAACEhIVBTU4O2tjYAoG6kooKCAvN6AOjVqxcUFBQgFAqZ1Zarq6vh5+cHHR2dz50tiqKor1azGuNubm5iY8ZHjhxJG+IURVEy5NChQzh06BDz2MLCAjt37gQAjBgxAqmpqZg9ezYUFBTw9u1bkdcOGjQIiYmJMDAwQO/evXH37l3s2bMHXC4Xffr0AYfDQXp6OqqqqjBnzpzPmi+KoqivWbNu4HRzc0NxcTGA2qsnvXr1gpmZGdq2bSv1A6QoiqJvB/RuAAAB0UlEQVSadv36dYSHh0NBQQEdOnRA586dYWRkhIEDBzLPqaqqgre3N16/fo0xY8bgwYMHePr0KZYtW4YePXrg7du38PLyQlZWFtq3b49169YBAJKSknD79m0UFxfj+++/x+jRo8Fms1srqxRFUV8dOs84RVEURVEURbWSJoepLFiwALm5uY3u9/T0hJqamsQPiqIoiqIoiqL+CZpsjEdGRjY4v3idiooKiR8QRVEURVEURf1TNNkY9/T0FGlwp6Sk4Pfff0dubi5UVVXRsWNHqR8gRVEURVEURX2tmjVm/MWLF9i2bRtOnDgBRUVFLFmyBOvWrYOSktLnOEaKoiiKoiiK+io12RgXCARwdnaGi4sLhEIhFi5ciP/+9790SkOKoiiKoiiKkoAmG+O6urpITEyEoqIifv31V6irq4vst7e3h7KystQPkqIoiqIoiqK+Rk02xrW1tZu8gTM9PR1aWlpSOTCKoiiKoiiK+to1eQOnk5MTs9hPQ7p06SLxA6IoiqIoiqKofwq66A9FURRFURRFtRK51j4AiqIoiqIoivqnoo1xiqIoiqIoimoltDFOURRFURRFUa2ENsYpiqIoiqIoqpXQxjhFURRFURRFtZL/A1YG8UqAzUvyAAAAAElFTkSuQmCC", "image/svg+xml": [ "Apr 19Apr 26May 03May 10May 17May 24May 31Jun 07Jun 14Jun 21Date012345Number of prevalent COVID-19 cases" ], "text/plain": [ "@vlplot(\n", " width=700,\n", " height=150,\n", " mark={\n", " point={\n", " filled=true,\n", " size=100,\n", " color=\"#004b96\",\n", " opacity=0.6\n", " },\n", " color=\"#004b96\",\n", " opacity=0.8,\n", " type=\"line\"\n", " },\n", " encoding={\n", " x={\n", " field=\"date\",\n", " axis={\n", " title=\"Date\"\n", " }\n", " },\n", " y={\n", " field=\"currentcases\",\n", " axis={\n", " title=\"Number of prevalent COVID-19 cases\"\n", " }\n", " }\n", " },\n", " data={\n", " values=...\n", " }\n", ")" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "myhospital[findall(x -> x == false, myhospital[:,:reported]),:currentcases] .= NaN;\n", "\n", "figure1=myhospital|> @vlplot(\n", " width=700, height=150,\n", " mark={:line, point={filled=true, size=100, color=\"#004b96\", opacity = 0.6}, color=\"#004b96\", opacity=0.8}, \n", " x={field=:date, axis={title=\"Date\"}},\n", " y={field=:currentcases, axis={title=\"Number of prevalent COVID-19 cases\"}}\n", ") " ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Now, imagine standing at say, May 31, having had several days of missing reports, and being tasked with making a prediction for what's going to happen next...? \n", "\n", "![](aux/PredictionNeeded.png)" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "So here's what we did: We developed a model to predict increments of daily cases, i.e., *changes* in cases from one day to the next. Since the pandemic dynamics differed wildly between different regions and larger and smaller hospitals, we needed individual models for each hospital. We used a simple linear model: \n", "\n", "![](aux/IncrementModel.png)" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "To account for missing daily reports in our model, we needed to figure out what to do when the number of prevalent cases from the day before is missing. If the number of cases from the day before hasn’t been reported, we take the value from the day before that, and add the model prediction for the change from day $t-1$ to $t$, to we just propagate the model prediction forwards in time from the last observed value, until we have a new observation available: \n", "\n", "![](aux/IncrementModel2.png)\n", "\n", "To illustrate what this means in practice, let's look at different situations in some imaginary hospitals: \n", "\n", "![](aux/HospitalsMissing.png)\n", "\n", "The problem with this is that it leads to higher-order terms in the coefficients, and thus a non-linear optimisation problem: \n", "\n", "![](aux/NonlinearOptProb.png)" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "But, even though maximum likelihood estimation suddenly becomes rather challenging and we cannot write down the model in closed form, it *is* quite straightforward to write down the model loss function as a computer program -- and that's all we need to do automatic differentiation. At least with a flexible system like `Zygote.jl`, where we can differentiate pretty much everything that we can write as code (provided it's differentiable, of course), and thus still estimate model parameters." ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "loss (generic function with 1 method)" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "function loss(y, z, r, β)\n", " sqerror = 0.0 # squared error\n", " firstseen = false # set to true after skipping potential missings \n", " last_y = 0.0 # prevalent cases from previous time point\n", " contribno = 0.0 # number of non-missing observations\n", "\n", " for t = 1:length(y)\n", " # skip missings at the start until first reported value\n", " if !firstseen \n", " if r[t] == 1\n", " firstseen = true\n", " last_y = y[t]\n", " else\n", " continue\n", " end\n", " else # make a prediction for the current increment\n", " pred_dy = β[1] + β[2] * last_y + β[3] * z[t-1] \n", " if r[t] == 1\n", " dy = y[t] - last_y\n", " sqerror += (dy - pred_dy)^2\n", " contribno += 1.0\n", " last_y = y[t]\n", " else\n", " last_y += pred_dy\n", " end \n", " end\n", " end\n", " return sqerror/contribno # return MSE over all reported time points end\n", "end" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "So let's see how it's done in Julia! We use automatic differentiation to optimize the model parameters, $\\beta$, by gradient descent on the loss function. " ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "# get the data for our chosen hospital\n", "myhospital = all_hospitals_df[all_hospitals_df[!,:hospital] .== 40, :]\n", "y, z, r = myhospital[1:end-1,:currentcases], myhospital[1:end-1,:incidences], myhospital[1:end-1,:reported]\n", "# set aside the last time point in the data to evaluate our prediction on: \n", "last_currentcases, lastreported = myhospital[end,:currentcases], myhospital[end,:reported]\n", "\n", "# initialize the parameters\n", "β = [0.0, 0.0, 0.0]\n", "losses = [] # save losses during training\n", "for epoch=1:1000\n", " curgrad = gradient(arg -> loss(y, z, r, arg), β)\n", " β .-= [0.001, 0.01, 0.001] .* curgrad[1]\n", " push!(losses, loss(y, z, r, β))\n", "end" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "We can now visualise the loss: " ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "application/vnd.vega.v5+json": { "$schema": "https://vega.github.io/schema/vega/v5.json", "axes": [ { "aria": false, "domain": false, "grid": true, "gridScale": "y", "labels": false, "maxExtent": 0, "minExtent": 0, "orient": "bottom", "scale": "x", "tickCount": { "signal": "ceil(width/40)" }, "ticks": false, "zindex": 0 }, { "aria": false, "domain": false, "grid": true, "gridScale": "x", "labels": false, "maxExtent": 0, "minExtent": 0, "orient": "left", "scale": "y", "tickCount": { "signal": "ceil(height/40)" }, "ticks": false, "zindex": 0 }, { "grid": false, "labelFlush": true, "labelOverlap": true, "orient": "bottom", "scale": "x", "tickCount": { "signal": "ceil(width/40)" }, "zindex": 0 }, { "grid": false, "labelOverlap": true, "orient": "left", "scale": "y", "tickCount": { "signal": "ceil(height/40)" }, "zindex": 0 } ], "background": "white", "data": [ { "name": "source_0", "values": [ { "x": 1, "y": 1.7804587348273626 }, { "x": 2, "y": 1.5284681238473774 }, { "x": 3, "y": 1.5213312897587425 }, { "x": 4, "y": 1.5172458110455926 }, { "x": 5, "y": 1.5135734062883326 }, { "x": 6, "y": 1.5100518242768475 }, { "x": 7, "y": 1.5065936343532005 }, { "x": 8, "y": 1.5031819154830508 }, { "x": 9, "y": 1.4998121189043703 }, { "x": 10, "y": 1.4964827402930512 }, { "x": 11, "y": 1.4931929129671753 }, { "x": 12, "y": 1.4899419192725551 }, { "x": 13, "y": 1.4867290851017532 }, { "x": 14, "y": 1.4835537572187785 }, { "x": 15, "y": 1.4804152985604835 }, { "x": 16, "y": 1.4773130868320348 }, { "x": 17, "y": 1.47424651394791 }, { "x": 18, "y": 1.471214985579184 }, { "x": 19, "y": 1.4682179207613837 }, { "x": 20, "y": 1.4652547515129817 }, { "x": 21, "y": 1.462324922470798 }, { "x": 22, "y": 1.4594278905367082 }, { "x": 23, "y": 1.4565631245367099 }, { "x": 24, "y": 1.4537301048912172 }, { "x": 25, "y": 1.4509283232963706 }, { "x": 26, "y": 1.4481572824158648 }, { "x": 27, "y": 1.4454164955828928 }, { "x": 28, "y": 1.442705486511837 }, { "x": 29, "y": 1.4400237890193202 }, { "x": 30, "y": 1.4373709467542723 }, { "x": 31, "y": 1.4347465129366692 }, { "x": 32, "y": 1.4321500501046127 }, { "x": 33, "y": 1.429581129869458 }, { "x": 34, "y": 1.42703933267867 }, { "x": 35, "y": 1.424524247586143 }, { "x": 36, "y": 1.4220354720296937 }, { "x": 37, "y": 1.4195726116154777 }, { "x": 38, "y": 1.4171352799090722 }, { "x": 39, "y": 1.4147230982329912 }, { "x": 40, "y": 1.412335695470391 }, { "x": 41, "y": 1.4099727078747577 }, { "x": 42, "y": 1.4076337788853566 }, { "x": 43, "y": 1.405318558948244 }, { "x": 44, "y": 1.4030267053426388 }, { "x": 45, "y": 1.4007578820124815 }, { "x": 46, "y": 1.3985117594029786 }, { "x": 47, "y": 1.3962880143019833 }, { "x": 48, "y": 1.3940863296860269 }, { "x": 49, "y": 1.3919063945708525 }, { "x": 50, "y": 1.3897479038662939 }, { "x": 51, "y": 1.3876105582353533 }, { "x": 52, "y": 1.385494063957334 }, { "x": 53, "y": 1.3833981327948959 }, { "x": 54, "y": 1.3813224818648986 }, { "x": 55, "y": 1.379266833512906 }, { "x": 56, "y": 1.3772309151912412 }, { "x": 57, "y": 1.3752144593404554 }, { "x": 58, "y": 1.3732172032741194 }, { "x": 59, "y": 1.3712388890668152 }, { "x": 60, "y": 1.3692792634452322 }, { "x": 61, "y": 1.367338077682259 }, { "x": 62, "y": 1.3654150874939859 }, { "x": 63, "y": 1.363510052939517 }, { "x": 64, "y": 1.3616227383234987 }, { "x": 65, "y": 1.359752912101295 }, { "x": 66, "y": 1.357900346786702 }, { "x": 67, "y": 1.3560648188621454 }, { "x": 68, "y": 1.3542461086912676 }, { "x": 69, "y": 1.3524440004338365 }, { "x": 70, "y": 1.3506582819629027 }, { "x": 71, "y": 1.3488887447841351 }, { "x": 72, "y": 1.3471351839572696 }, { "x": 73, "y": 1.3453973980196063 }, { "x": 74, "y": 1.343675188911488 }, { "x": 75, "y": 1.3419683619037093 }, { "x": 76, "y": 1.3402767255267896 }, { "x": 77, "y": 1.3386000915020557 }, { "x": 78, "y": 1.3369382746744842 }, { "x": 79, "y": 1.3352910929472506 }, { "x": 80, "y": 1.3336583672179219 }, { "x": 81, "y": 1.3320399213162684 }, { "x": 82, "y": 1.330435581943623 }, { "x": 83, "y": 1.3288451786137592 }, { "x": 84, "y": 1.327268543595232 }, { "x": 85, "y": 1.325705511855149 }, { "x": 86, "y": 1.3241559210043266 }, { "x": 87, "y": 1.3226196112437856 }, { "x": 88, "y": 1.3210964253125566 }, { "x": 89, "y": 1.319586208436758 }, { "x": 90, "y": 1.3180888082798983 }, { "x": 91, "y": 1.3166040748943857 }, { "x": 92, "y": 1.3151318606741935 }, { "x": 93, "y": 1.3136720203086638 }, { "x": 94, "y": 1.312224410737407 }, { "x": 95, "y": 1.3107888911062748 }, { "x": 96, "y": 1.309365322724369 }, { "x": 97, "y": 1.3079535690220614 }, { "x": 98, "y": 1.3065534955100022 }, { "x": 99, "y": 1.3051649697390766 }, { "x": 100, "y": 1.3037878612612999 }, { "x": 101, "y": 1.3024220415916132 }, { "x": 102, "y": 1.3010673841705593 }, { "x": 103, "y": 1.299723764327817 }, { "x": 104, "y": 1.298391059246568 }, { "x": 105, "y": 1.297069147928679 }, { "x": 106, "y": 1.295757911160666 }, { "x": 107, "y": 1.2944572314804375 }, { "x": 108, "y": 1.2931669931447767 }, { "x": 109, "y": 1.291887082097562 }, { "x": 110, "y": 1.2906173859386878 }, { "x": 111, "y": 1.2893577938936835 }, { "x": 112, "y": 1.2881081967840058 }, { "x": 113, "y": 1.2868684869979812 }, { "x": 114, "y": 1.2856385584623935 }, { "x": 115, "y": 1.28441830661469 }, { "x": 116, "y": 1.2832076283757963 }, { "x": 117, "y": 1.282006422123523 }, { "x": 118, "y": 1.280814587666544 }, { "x": 119, "y": 1.2796320262189398 }, { "x": 120, "y": 1.278458640375291 }, { "x": 121, "y": 1.2772943340862974 }, { "x": 122, "y": 1.276139012634925 }, { "x": 123, "y": 1.2749925826130544 }, { "x": 124, "y": 1.2738549518986273 }, { "x": 125, "y": 1.2727260296332747 }, { "x": 126, "y": 1.271605726200415 }, { "x": 127, "y": 1.2704939532038098 }, { "x": 128, "y": 1.26939062344657 }, { "x": 129, "y": 1.268295650910595 }, { "x": 130, "y": 1.2672089507364421 }, { "x": 131, "y": 1.2661304392036081 }, { "x": 132, "y": 1.2650600337112183 }, { "x": 133, "y": 1.2639976527591084 }, { "x": 134, "y": 1.262943215929301 }, { "x": 135, "y": 1.2618966438678485 }, { "x": 136, "y": 1.2608578582670498 }, { "x": 137, "y": 1.2598267818480287 }, { "x": 138, "y": 1.258803338343656 }, { "x": 139, "y": 1.2577874524818236 }, { "x": 140, "y": 1.2567790499690425 }, { "x": 141, "y": 1.2557780574743787 }, { "x": 142, "y": 1.254784402613699 }, { "x": 143, "y": 1.253798013934234 }, { "x": 144, "y": 1.2528188208994435 }, { "x": 145, "y": 1.2518467538741793 }, { "x": 146, "y": 1.250881744110139 }, { "x": 147, "y": 1.2499237237316052 }, { "x": 148, "y": 1.248972625721459 }, { "x": 149, "y": 1.2480283839074677 }, { "x": 150, "y": 1.2470909329488356 }, { "x": 151, "y": 1.2461602083230146 }, { "x": 152, "y": 1.245236146312767 }, { "x": 153, "y": 1.2443186839934781 }, { "x": 154, "y": 1.2434077592207076 }, { "x": 155, "y": 1.24250331061798 }, { "x": 156, "y": 1.2416052775648043 }, { "x": 157, "y": 1.2407136001849222 }, { "x": 158, "y": 1.239828219334775 }, { "x": 159, "y": 1.2389490765921871 }, { "x": 160, "y": 1.2380761142452643 }, { "x": 161, "y": 1.2372092752814936 }, { "x": 162, "y": 1.2363485033770492 }, { "x": 163, "y": 1.2354937428862955 }, { "x": 164, "y": 1.234644938831484 }, { "x": 165, "y": 1.2338020368926363 }, { "x": 166, "y": 1.232964983397621 }, { "x": 167, "y": 1.2321337253124025 }, { "x": 168, "y": 1.2313082102314703 }, { "x": 169, "y": 1.2304883863684473 }, { "x": 170, "y": 1.2296742025468594 }, { "x": 171, "y": 1.2288656081910794 }, { "x": 172, "y": 1.228062553317429 }, { "x": 173, "y": 1.2272649885254414 }, { "x": 174, "y": 1.2264728649892804 }, { "x": 175, "y": 1.2256861344493144 }, { "x": 176, "y": 1.2249047492038339 }, { "x": 177, "y": 1.2241286621009242 }, { "x": 178, "y": 1.223357826530473 }, { "x": 179, "y": 1.222592196416325 }, { "x": 180, "y": 1.2218317262085714 }, { "x": 181, "y": 1.2210763708759749 }, { "x": 182, "y": 1.2203260858985248 }, { "x": 183, "y": 1.2195808272601236 }, { "x": 184, "y": 1.218840551441402 }, { "x": 185, "y": 1.2181052154126524 }, { "x": 186, "y": 1.2173747766268903 }, { "x": 187, "y": 1.2166491930130294 }, { "x": 188, "y": 1.2159284229691791 }, { "x": 189, "y": 1.215212425356051 }, { "x": 190, "y": 1.2145011594904818 }, { "x": 191, "y": 1.213794585139061 }, { "x": 192, "y": 1.2130926625118745 }, { "x": 193, "y": 1.2123953522563449 }, { "x": 194, "y": 1.2117026154511814 }, { "x": 195, "y": 1.2110144136004275 }, { "x": 196, "y": 1.2103307086276114 }, { "x": 197, "y": 1.2096514628699904 }, { "x": 198, "y": 1.208976639072894 }, { "x": 199, "y": 1.2083062003841598 }, { "x": 200, "y": 1.2076401103486583 }, { "x": 201, "y": 1.206978332902914 }, { "x": 202, "y": 1.2063208323698094 }, { "x": 203, "y": 1.2056675734533777 }, { "x": 204, "y": 1.205018521233681 }, { "x": 205, "y": 1.2043736411617714 }, { "x": 206, "y": 1.2037328990547331 }, { "x": 207, "y": 1.2030962610908076 }, { "x": 208, "y": 1.2024636938045927 }, { "x": 209, "y": 1.2018351640823242 }, { "x": 210, "y": 1.2012106391572295 }, { "x": 211, "y": 1.2005900866049566 }, { "x": 212, "y": 1.1999734743390766 }, { "x": 213, "y": 1.199360770606659 }, { "x": 214, "y": 1.1987519439839127 }, { "x": 215, "y": 1.1981469633719006 }, { "x": 216, "y": 1.197545797992322 }, { "x": 217, "y": 1.1969484173833558 }, { "x": 218, "y": 1.1963547913955763 }, { "x": 219, "y": 1.1957648901879256 }, { "x": 220, "y": 1.1951786842237573 }, { "x": 221, "y": 1.1945961442669324 }, { "x": 222, "y": 1.194017241377984 }, { "x": 223, "y": 1.193441946910336 }, { "x": 224, "y": 1.1928702325065825 }, { "x": 225, "y": 1.1923020700948244 }, { "x": 226, "y": 1.1917374318850606 }, { "x": 227, "y": 1.1911762903656375 }, { "x": 228, "y": 1.19061861829975 }, { "x": 229, "y": 1.190064388721996 }, { "x": 230, "y": 1.1895135749349839 }, { "x": 231, "y": 1.188966150505995 }, { "x": 232, "y": 1.1884220892636879 }, { "x": 233, "y": 1.1878813652948597 }, { "x": 234, "y": 1.1873439529412546 }, { "x": 235, "y": 1.1868098267964184 }, { "x": 236, "y": 1.1862789617026004 }, { "x": 237, "y": 1.185751332747703 }, { "x": 238, "y": 1.185226915262276 }, { "x": 239, "y": 1.1847056848165538 }, { "x": 240, "y": 1.1841876172175394 }, { "x": 241, "y": 1.183672688506129 }, { "x": 242, "y": 1.1831608749542792 }, { "x": 243, "y": 1.1826521530622183 }, { "x": 244, "y": 1.1821464995556943 }, { "x": 245, "y": 1.1816438913832656 }, { "x": 246, "y": 1.1811443057136324 }, { "x": 247, "y": 1.1806477199330028 }, { "x": 248, "y": 1.180154111642501 }, { "x": 249, "y": 1.1796634586556114 }, { "x": 250, "y": 1.1791757389956585 }, { "x": 251, "y": 1.1786909308933258 }, { "x": 252, "y": 1.1782090127842055 }, { "x": 253, "y": 1.17772996330639 }, { "x": 254, "y": 1.1772537612980918 }, { "x": 255, "y": 1.1767803857952992 }, { "x": 256, "y": 1.1763098160294665 }, { "x": 257, "y": 1.175842031425237 }, { "x": 258, "y": 1.1753770115981934 }, { "x": 259, "y": 1.1749147363526486 }, { "x": 260, "y": 1.1744551856794583 }, { "x": 261, "y": 1.1739983397538707 }, { "x": 262, "y": 1.173544178933404 }, { "x": 263, "y": 1.1730926837557532 }, { "x": 264, "y": 1.172643834936725 }, { "x": 265, "y": 1.172197613368205 }, { "x": 266, "y": 1.1717540001161508 }, { "x": 267, "y": 1.1713129764186097 }, { "x": 268, "y": 1.1708745236837712 }, { "x": 269, "y": 1.1704386234880382 }, { "x": 270, "y": 1.17000525757413 }, { "x": 271, "y": 1.1695744078492096 }, { "x": 272, "y": 1.1691460563830351 }, { "x": 273, "y": 1.1687201854061413 }, { "x": 274, "y": 1.1682967773080368 }, { "x": 275, "y": 1.167875814635435 }, { "x": 276, "y": 1.1674572800905056 }, { "x": 277, "y": 1.167041156529145 }, { "x": 278, "y": 1.1666274269592785 }, { "x": 279, "y": 1.1662160745391774 }, { "x": 280, "y": 1.165807082575806 }, { "x": 281, "y": 1.165400434523182 }, { "x": 282, "y": 1.1649961139807665 }, { "x": 283, "y": 1.1645941046918724 }, { "x": 284, "y": 1.1641943905420922 }, { "x": 285, "y": 1.1637969555577496 }, { "x": 286, "y": 1.1634017839043698 }, { "x": 287, "y": 1.163008859885169 }, { "x": 288, "y": 1.162618167939568 }, { "x": 289, "y": 1.1622296926417193 }, { "x": 290, "y": 1.1618434186990578 }, { "x": 291, "y": 1.1614593309508692 }, { "x": 292, "y": 1.1610774143668758 }, { "x": 293, "y": 1.160697654045844 }, { "x": 294, "y": 1.1603200352142038 }, { "x": 295, "y": 1.1599445432246935 }, { "x": 296, "y": 1.159571163555017 }, { "x": 297, "y": 1.1591998818065188 }, { "x": 298, "y": 1.1588306837028777 }, { "x": 299, "y": 1.1584635550888152 }, { "x": 300, "y": 1.158098481928824 }, { "x": 301, "y": 1.1577354503059067 }, { "x": 302, "y": 1.1573744464203382 }, { "x": 303, "y": 1.1570154565884359 }, { "x": 304, "y": 1.15665846724135 }, { "x": 305, "y": 1.156303464923871 }, { "x": 306, "y": 1.155950436293247 }, { "x": 307, "y": 1.155599368118017 }, { "x": 308, "y": 1.1552502472768653 }, { "x": 309, "y": 1.1549030607574793 }, { "x": 310, "y": 1.1545577956554316 }, { "x": 311, "y": 1.1542144391730698 }, { "x": 312, "y": 1.1538729786184228 }, { "x": 313, "y": 1.1535334014041196 }, { "x": 314, "y": 1.1531956950463225 }, { "x": 315, "y": 1.1528598471636722 }, { "x": 316, "y": 1.1525258454762473 }, { "x": 317, "y": 1.1521936778045343 }, { "x": 318, "y": 1.1518633320684146 }, { "x": 319, "y": 1.1515347962861584 }, { "x": 320, "y": 1.1512080585734341 }, { "x": 321, "y": 1.1508831071423293 }, { "x": 322, "y": 1.1505599303003857 }, { "x": 323, "y": 1.15023851644964 }, { "x": 324, "y": 1.1499188540856837 }, { "x": 325, "y": 1.149600931796727 }, { "x": 326, "y": 1.149284738262681 }, { "x": 327, "y": 1.1489702622542457 }, { "x": 328, "y": 1.1486574926320094 }, { "x": 329, "y": 1.1483464183455652 }, { "x": 330, "y": 1.148037028432627 }, { "x": 331, "y": 1.1477293120181662 }, { "x": 332, "y": 1.147423258313553 }, { "x": 333, "y": 1.1471188566157113 }, { "x": 334, "y": 1.1468160963062795 }, { "x": 335, "y": 1.1465149668507841 }, { "x": 336, "y": 1.1462154577978252 }, { "x": 337, "y": 1.145917558778264 }, { "x": 338, "y": 1.1456212595044304 }, { "x": 339, "y": 1.1453265497693297 }, { "x": 340, "y": 1.1450334194458653 }, { "x": 341, "y": 1.144741858486068 }, { "x": 342, "y": 1.1444518569203344 }, { "x": 343, "y": 1.1441634048566744 }, { "x": 344, "y": 1.143876492479968 }, { "x": 345, "y": 1.1435911100512284 }, { "x": 346, "y": 1.1433072479068778 }, { "x": 347, "y": 1.143024896458027 }, { "x": 348, "y": 1.142744046189765 }, { "x": 349, "y": 1.1424646876604603 }, { "x": 350, "y": 1.1421868115010638 }, { "x": 351, "y": 1.1419104084144223 }, { "x": 352, "y": 1.1416354691746053 }, { "x": 353, "y": 1.1413619846262286 }, { "x": 354, "y": 1.1410899456837957 }, { "x": 355, "y": 1.1408193433310407 }, { "x": 356, "y": 1.140550168620281 }, { "x": 357, "y": 1.140282412671777 }, { "x": 358, "y": 1.1400160666730978 }, { "x": 359, "y": 1.1397511218784968 }, { "x": 360, "y": 1.139487569608291 }, { "x": 361, "y": 1.139225401248249 }, { "x": 362, "y": 1.138964608248987 }, { "x": 363, "y": 1.138705182125369 }, { "x": 364, "y": 1.1384471144559152 }, { "x": 365, "y": 1.138190396882215 }, { "x": 366, "y": 1.137935021108353 }, { "x": 367, "y": 1.1376809789003306 }, { "x": 368, "y": 1.1374282620855038 }, { "x": 369, "y": 1.137176862552023 }, { "x": 370, "y": 1.1369267722482763 }, { "x": 371, "y": 1.1366779831823466 }, { "x": 372, "y": 1.1364304874214648 }, { "x": 373, "y": 1.136184277091481 }, { "x": 374, "y": 1.1359393443763253 }, { "x": 375, "y": 1.1356956815174928 }, { "x": 376, "y": 1.1354532808135203 }, { "x": 377, "y": 1.1352121346194732 }, { "x": 378, "y": 1.1349722353464429 }, { "x": 379, "y": 1.1347335754610397 }, { "x": 380, "y": 1.1344961474848982 }, { "x": 381, "y": 1.1342599439941898 }, { "x": 382, "y": 1.1340249576191321 }, { "x": 383, "y": 1.1337911810435106 }, { "x": 384, "y": 1.1335586070042056 }, { "x": 385, "y": 1.1333272282907176 }, { "x": 386, "y": 1.1330970377447054 }, { "x": 387, "y": 1.1328680282595245 }, { "x": 388, "y": 1.1326401927797736 }, { "x": 389, "y": 1.1324135243008397 }, { "x": 390, "y": 1.1321880158684587 }, { "x": 391, "y": 1.1319636605782677 }, { "x": 392, "y": 1.1317404515753753 }, { "x": 393, "y": 1.131518382053923 }, { "x": 394, "y": 1.1312974452566644 }, { "x": 395, "y": 1.1310776344745375 }, { "x": 396, "y": 1.130858943046249 }, { "x": 397, "y": 1.1306413643578601 }, { "x": 398, "y": 1.130424891842375 }, { "x": 399, "y": 1.1302095189793386 }, { "x": 400, "y": 1.129995239294433 }, { "x": 401, "y": 1.1297820463590813 }, { "x": 402, "y": 1.129569933790054 }, { "x": 403, "y": 1.1293588952490816 }, { "x": 404, "y": 1.129148924442469 }, { "x": 405, "y": 1.1289400151207143 }, { "x": 406, "y": 1.1287321610781318 }, { "x": 407, "y": 1.12852535615248 }, { "x": 408, "y": 1.1283195942245912 }, { "x": 409, "y": 1.128114869218006 }, { "x": 410, "y": 1.1279111750986137 }, { "x": 411, "y": 1.1277085058742917 }, { "x": 412, "y": 1.127506855594551 }, { "x": 413, "y": 1.127306218350188 }, { "x": 414, "y": 1.127106588272934 }, { "x": 415, "y": 1.1269079595351141 }, { "x": 416, "y": 1.1267103263493052 }, { "x": 417, "y": 1.126513682968 }, { "x": 418, "y": 1.1263180236832737 }, { "x": 419, "y": 1.126123342826452 }, { "x": 420, "y": 1.1259296347677876 }, { "x": 421, "y": 1.1257368939161336 }, { "x": 422, "y": 1.1255451147186242 }, { "x": 423, "y": 1.125354291660358 }, { "x": 424, "y": 1.1251644192640846 }, { "x": 425, "y": 1.1249754920898916 }, { "x": 426, "y": 1.1247875047348983 }, { "x": 427, "y": 1.1246004518329522 }, { "x": 428, "y": 1.1244143280543235 }, { "x": 429, "y": 1.1242291281054109 }, { "x": 430, "y": 1.1240448467284418 }, { "x": 431, "y": 1.1238614787011836 }, { "x": 432, "y": 1.1236790188366483 }, { "x": 433, "y": 1.123497461982811 }, { "x": 434, "y": 1.123316803022321 }, { "x": 435, "y": 1.1231370368722238 }, { "x": 436, "y": 1.1229581584836779 }, { "x": 437, "y": 1.1227801628416838 }, { "x": 438, "y": 1.1226030449648075 }, { "x": 439, "y": 1.1224267999049085 }, { "x": 440, "y": 1.1222514227468752 }, { "x": 441, "y": 1.1220769086083566 }, { "x": 442, "y": 1.1219032526395007 }, { "x": 443, "y": 1.121730450022691 }, { "x": 444, "y": 1.1215584959722944 }, { "x": 445, "y": 1.121387385734399 }, { "x": 446, "y": 1.1212171145865657 }, { "x": 447, "y": 1.1210476778375755 }, { "x": 448, "y": 1.1208790708271819 }, { "x": 449, "y": 1.1207112889258641 }, { "x": 450, "y": 1.1205443275345839 }, { "x": 451, "y": 1.1203781820845466 }, { "x": 452, "y": 1.120212848036958 }, { "x": 453, "y": 1.1200483208827903 }, { "x": 454, "y": 1.1198845961425463 }, { "x": 455, "y": 1.1197216693660297 }, { "x": 456, "y": 1.1195595361321116 }, { "x": 457, "y": 1.1193981920485052 }, { "x": 458, "y": 1.1192376327515379 }, { "x": 459, "y": 1.1190778539059296 }, { "x": 460, "y": 1.1189188512045705 }, { "x": 461, "y": 1.1187606203683 }, { "x": 462, "y": 1.1186031571456923 }, { "x": 463, "y": 1.1184464573128383 }, { "x": 464, "y": 1.1182905166731336 }, { "x": 465, "y": 1.1181353310570654 }, { "x": 466, "y": 1.1179808963220066 }, { "x": 467, "y": 1.117827208352003 }, { "x": 468, "y": 1.1176742630575731 }, { "x": 469, "y": 1.1175220563754995 }, { "x": 470, "y": 1.117370584268631 }, { "x": 471, "y": 1.11721984272568 }, { "x": 472, "y": 1.1170698277610258 }, { "x": 473, "y": 1.1169205354145164 }, { "x": 474, "y": 1.1167719617512772 }, { "x": 475, "y": 1.1166241028615127 }, { "x": 476, "y": 1.1164769548603233 }, { "x": 477, "y": 1.116330513887508 }, { "x": 478, "y": 1.1161847761073824 }, { "x": 479, "y": 1.1160397377085896 }, { "x": 480, "y": 1.1158953949039183 }, { "x": 481, "y": 1.115751743930118 }, { "x": 482, "y": 1.1156087810477202 }, { "x": 483, "y": 1.1154665025408579 }, { "x": 484, "y": 1.1153249047170877 }, { "x": 485, "y": 1.1151839839072153 }, { "x": 486, "y": 1.1150437364651198 }, { "x": 487, "y": 1.1149041587675796 }, { "x": 488, "y": 1.1147652472141052 }, { "x": 489, "y": 1.114626998226764 }, { "x": 490, "y": 1.1144894082500154 }, { "x": 491, "y": 1.114352473750544 }, { "x": 492, "y": 1.1142161912170918 }, { "x": 493, "y": 1.114080557160297 }, { "x": 494, "y": 1.11394556811253 }, { "x": 495, "y": 1.1138112206277335 }, { "x": 496, "y": 1.1136775112812616 }, { "x": 497, "y": 1.1135444366697227 }, { "x": 498, "y": 1.1134119934108218 }, { "x": 499, "y": 1.1132801781432067 }, { "x": 500, "y": 1.113148987526312 }, { "x": 501, "y": 1.1130184182402076 }, { "x": 502, "y": 1.1128884669854466 }, { "x": 503, "y": 1.112759130482917 }, { "x": 504, "y": 1.1126304054736895 }, { "x": 505, "y": 1.1125022887188751 }, { "x": 506, "y": 1.1123747769994738 }, { "x": 507, "y": 1.1122478671162332 }, { "x": 508, "y": 1.112121555889503 }, { "x": 509, "y": 1.1119958401590944 }, { "x": 510, "y": 1.111870716784137 }, { "x": 511, "y": 1.1117461826429393 }, { "x": 512, "y": 1.1116222346328513 }, { "x": 513, "y": 1.1114988696701267 }, { "x": 514, "y": 1.1113760846897835 }, { "x": 515, "y": 1.1112538766454745 }, { "x": 516, "y": 1.1111322425093468 }, { "x": 517, "y": 1.111011179271914 }, { "x": 518, "y": 1.1108906839419224 }, { "x": 519, "y": 1.11077075354622 }, { "x": 520, "y": 1.1106513851296274 }, { "x": 521, "y": 1.1105325757548103 }, { "x": 522, "y": 1.1104143225021506 }, { "x": 523, "y": 1.110296622469621 }, { "x": 524, "y": 1.1101794727726602 }, { "x": 525, "y": 1.110062870544048 }, { "x": 526, "y": 1.109946812933782 }, { "x": 527, "y": 1.1098312971089572 }, { "x": 528, "y": 1.109716320253643 }, { "x": 529, "y": 1.1096018795687645 }, { "x": 530, "y": 1.1094879722719826 }, { "x": 531, "y": 1.1093745955975773 }, { "x": 532, "y": 1.1092617467963286 }, { "x": 533, "y": 1.1091494231354029 }, { "x": 534, "y": 1.1090376218982363 }, { "x": 535, "y": 1.1089263403844205 }, { "x": 536, "y": 1.1088155759095915 }, { "x": 537, "y": 1.1087053258053143 }, { "x": 538, "y": 1.1085955874189743 }, { "x": 539, "y": 1.1084863581136657 }, { "x": 540, "y": 1.108377635268082 }, { "x": 541, "y": 1.1082694162764073 }, { "x": 542, "y": 1.1081616985482112 }, { "x": 543, "y": 1.108054479508337 }, { "x": 544, "y": 1.1079477565967997 }, { "x": 545, "y": 1.1078415272686806 }, { "x": 546, "y": 1.107735788994022 }, { "x": 547, "y": 1.1076305392577246 }, { "x": 548, "y": 1.1075257755594436 }, { "x": 549, "y": 1.1074214954134902 }, { "x": 550, "y": 1.1073176963487268 }, { "x": 551, "y": 1.1072143759084705 }, { "x": 552, "y": 1.1071115316503914 }, { "x": 553, "y": 1.1070091611464163 }, { "x": 554, "y": 1.1069072619826297 }, { "x": 555, "y": 1.106805831759177 }, { "x": 556, "y": 1.1067048680901688 }, { "x": 557, "y": 1.1066043686035862 }, { "x": 558, "y": 1.106504330941187 }, { "x": 559, "y": 1.1064047527584089 }, { "x": 560, "y": 1.1063056317242792 }, { "x": 561, "y": 1.1062069655213231 }, { "x": 562, "y": 1.10610875184547 }, { "x": 563, "y": 1.1060109884059637 }, { "x": 564, "y": 1.105913672925273 }, { "x": 565, "y": 1.1058168031390017 }, { "x": 566, "y": 1.105720376795801 }, { "x": 567, "y": 1.1056243916572797 }, { "x": 568, "y": 1.1055288454979197 }, { "x": 569, "y": 1.1054337361049864 }, { "x": 570, "y": 1.1053390612784453 }, { "x": 571, "y": 1.1052448188308763 }, { "x": 572, "y": 1.105151006587389 }, { "x": 573, "y": 1.1050576223855393 }, { "x": 574, "y": 1.1049646640752442 }, { "x": 575, "y": 1.1048721295187036 }, { "x": 576, "y": 1.1047800165903137 }, { "x": 577, "y": 1.10468832317659 }, { "x": 578, "y": 1.104597047176083 }, { "x": 579, "y": 1.1045061864993022 }, { "x": 580, "y": 1.1044157390686329 }, { "x": 581, "y": 1.10432570281826 }, { "x": 582, "y": 1.1042360756940899 }, { "x": 583, "y": 1.1041468556536709 }, { "x": 584, "y": 1.10405804066612 }, { "x": 585, "y": 1.1039696287120437 }, { "x": 586, "y": 1.1038816177834616 }, { "x": 587, "y": 1.1037940058837352 }, { "x": 588, "y": 1.1037067910274911 }, { "x": 589, "y": 1.1036199712405457 }, { "x": 590, "y": 1.1035335445598338 }, { "x": 591, "y": 1.1034475090333358 }, { "x": 592, "y": 1.1033618627200041 }, { "x": 593, "y": 1.1032766036896933 }, { "x": 594, "y": 1.1031917300230876 }, { "x": 595, "y": 1.1031072398116308 }, { "x": 596, "y": 1.1030231311574559 }, { "x": 597, "y": 1.1029394021733168 }, { "x": 598, "y": 1.1028560509825172 }, { "x": 599, "y": 1.1027730757188448 }, { "x": 600, "y": 1.1026904745265005 }, { "x": 601, "y": 1.1026082455600332 }, { "x": 602, "y": 1.1025263869842725 }, { "x": 603, "y": 1.1024448969742622 }, { "x": 604, "y": 1.1023637737151928 }, { "x": 605, "y": 1.1022830154023384 }, { "x": 606, "y": 1.1022026202409922 }, { "x": 607, "y": 1.1021225864463986 }, { "x": 608, "y": 1.1020429122436928 }, { "x": 609, "y": 1.1019635958678364 }, { "x": 610, "y": 1.1018846355635532 }, { "x": 611, "y": 1.101806029585268 }, { "x": 612, "y": 1.1017277761970456 }, { "x": 613, "y": 1.101649873672525 }, { "x": 614, "y": 1.1015723202948646 }, { "x": 615, "y": 1.1014951143566756 }, { "x": 616, "y": 1.1014182541599657 }, { "x": 617, "y": 1.1013417380160773 }, { "x": 618, "y": 1.10126556424563 }, { "x": 619, "y": 1.1011897311784595 }, { "x": 620, "y": 1.1011142371535618 }, { "x": 621, "y": 1.1010390805190324 }, { "x": 622, "y": 1.1009642596320117 }, { "x": 623, "y": 1.1008897728586247 }, { "x": 624, "y": 1.100815618573928 }, { "x": 625, "y": 1.10074179516185 }, { "x": 626, "y": 1.100668301015136 }, { "x": 627, "y": 1.1005951345352953 }, { "x": 628, "y": 1.1005222941325425 }, { "x": 629, "y": 1.1004497782257452 }, { "x": 630, "y": 1.100377585242369 }, { "x": 631, "y": 1.100305713618423 }, { "x": 632, "y": 1.1002341617984082 }, { "x": 633, "y": 1.1001629282352618 }, { "x": 634, "y": 1.1000920113903074 }, { "x": 635, "y": 1.1000214097332002 }, { "x": 636, "y": 1.0999511217418774 }, { "x": 637, "y": 1.099881145902504 }, { "x": 638, "y": 1.099811480709424 }, { "x": 639, "y": 1.099742124665109 }, { "x": 640, "y": 1.0996730762801068 }, { "x": 641, "y": 1.0996043340729922 }, { "x": 642, "y": 1.0995358965703166 }, { "x": 643, "y": 1.0994677623065607 }, { "x": 644, "y": 1.0993999298240829 }, { "x": 645, "y": 1.0993323976730716 }, { "x": 646, "y": 1.0992651644114977 }, { "x": 647, "y": 1.0991982286050659 }, { "x": 648, "y": 1.0991315888271669 }, { "x": 649, "y": 1.0990652436588315 }, { "x": 650, "y": 1.0989991916886817 }, { "x": 651, "y": 1.098933431512886 }, { "x": 652, "y": 1.0988679617351114 }, { "x": 653, "y": 1.098802780966479 }, { "x": 654, "y": 1.098737887825518 }, { "x": 655, "y": 1.0986732809381197 }, { "x": 656, "y": 1.098608958937494 }, { "x": 657, "y": 1.0985449204641229 }, { "x": 658, "y": 1.098481164165718 }, { "x": 659, "y": 1.098417688697175 }, { "x": 660, "y": 1.0983544927205313 }, { "x": 661, "y": 1.0982915749049222 }, { "x": 662, "y": 1.0982289339265365 }, { "x": 663, "y": 1.098166568468577 }, { "x": 664, "y": 1.0981044772212138 }, { "x": 665, "y": 1.098042658881546 }, { "x": 666, "y": 1.0979811121535574 }, { "x": 667, "y": 1.097919835748076 }, { "x": 668, "y": 1.0978588283827322 }, { "x": 669, "y": 1.0977980887819176 }, { "x": 670, "y": 1.097737615676745 }, { "x": 671, "y": 1.0976774078050093 }, { "x": 672, "y": 1.0976174639111438 }, { "x": 673, "y": 1.0975577827461818 }, { "x": 674, "y": 1.09749836306772 }, { "x": 675, "y": 1.0974392036398748 }, { "x": 676, "y": 1.0973803032332459 }, { "x": 677, "y": 1.0973216606248768 }, { "x": 678, "y": 1.0972632745982156 }, { "x": 679, "y": 1.0972051439430788 }, { "x": 680, "y": 1.0971472674556113 }, { "x": 681, "y": 1.0970896439382498 }, { "x": 682, "y": 1.0970322721996841 }, { "x": 683, "y": 1.0969751510548222 }, { "x": 684, "y": 1.0969182793247514 }, { "x": 685, "y": 1.0968616558367017 }, { "x": 686, "y": 1.0968052794240104 }, { "x": 687, "y": 1.0967491489260863 }, { "x": 688, "y": 1.096693263188371 }, { "x": 689, "y": 1.0966376210623072 }, { "x": 690, "y": 1.0965822214052996 }, { "x": 691, "y": 1.0965270630806825 }, { "x": 692, "y": 1.096472144957683 }, { "x": 693, "y": 1.0964174659113877 }, { "x": 694, "y": 1.096363024822706 }, { "x": 695, "y": 1.0963088205783387 }, { "x": 696, "y": 1.096254852070743 }, { "x": 697, "y": 1.096201118198097 }, { "x": 698, "y": 1.0961476178642677 }, { "x": 699, "y": 1.0960943499787787 }, { "x": 700, "y": 1.0960413134567757 }, { "x": 701, "y": 1.0959885072189919 }, { "x": 702, "y": 1.0959359301917198 }, { "x": 703, "y": 1.0958835813067747 }, { "x": 704, "y": 1.0958314595014644 }, { "x": 705, "y": 1.095779563718557 }, { "x": 706, "y": 1.0957278929062488 }, { "x": 707, "y": 1.0956764460181336 }, { "x": 708, "y": 1.0956252220131697 }, { "x": 709, "y": 1.0955742198556495 }, { "x": 710, "y": 1.09552343851517 }, { "x": 711, "y": 1.0954728769665991 }, { "x": 712, "y": 1.0954225341900488 }, { "x": 713, "y": 1.0953724091708419 }, { "x": 714, "y": 1.0953225008994822 }, { "x": 715, "y": 1.095272808371626 }, { "x": 716, "y": 1.095223330588052 }, { "x": 717, "y": 1.0951740665546306 }, { "x": 718, "y": 1.0951250152822956 }, { "x": 719, "y": 1.0950761757870149 }, { "x": 720, "y": 1.0950275470897617 }, { "x": 721, "y": 1.0949791282164856 }, { "x": 722, "y": 1.0949309181980829 }, { "x": 723, "y": 1.094882916070371 }, { "x": 724, "y": 1.0948351208740574 }, { "x": 725, "y": 1.094787531654713 }, { "x": 726, "y": 1.0947401474627452 }, { "x": 727, "y": 1.0946929673533674 }, { "x": 728, "y": 1.0946459903865744 }, { "x": 729, "y": 1.0945992156271147 }, { "x": 730, "y": 1.0945526421444605 }, { "x": 731, "y": 1.094506269012786 }, { "x": 732, "y": 1.0944600953109362 }, { "x": 733, "y": 1.0944141201224016 }, { "x": 734, "y": 1.094368342535293 }, { "x": 735, "y": 1.094322761642314 }, { "x": 736, "y": 1.0942773765407343 }, { "x": 737, "y": 1.0942321863323672 }, { "x": 738, "y": 1.0941871901235394 }, { "x": 739, "y": 1.0941423870250682 }, { "x": 740, "y": 1.0940977761522361 }, { "x": 741, "y": 1.0940533566247648 }, { "x": 742, "y": 1.09400912756679 }, { "x": 743, "y": 1.0939650881068372 }, { "x": 744, "y": 1.0939212373777967 }, { "x": 745, "y": 1.0938775745168985 }, { "x": 746, "y": 1.0938340986656891 }, { "x": 747, "y": 1.093790808970006 }, { "x": 748, "y": 1.0937477045799544 }, { "x": 749, "y": 1.0937047846498833 }, { "x": 750, "y": 1.093662048338361 }, { "x": 751, "y": 1.0936194948081521 }, { "x": 752, "y": 1.093577123226193 }, { "x": 753, "y": 1.0935349327635708 }, { "x": 754, "y": 1.093492922595498 }, { "x": 755, "y": 1.0934510919012892 }, { "x": 756, "y": 1.0934094398643417 }, { "x": 757, "y": 1.0933679656721074 }, { "x": 758, "y": 1.093326668516075 }, { "x": 759, "y": 1.0932855475917451 }, { "x": 760, "y": 1.0932446020986069 }, { "x": 761, "y": 1.0932038312401202 }, { "x": 762, "y": 1.093163234223689 }, { "x": 763, "y": 1.0931228102606407 }, { "x": 764, "y": 1.0930825585662074 }, { "x": 765, "y": 1.0930424783594996 }, { "x": 766, "y": 1.0930025688634872 }, { "x": 767, "y": 1.0929628293049791 }, { "x": 768, "y": 1.0929232589146 }, { "x": 769, "y": 1.09288385692677 }, { "x": 770, "y": 1.0928446225796848 }, { "x": 771, "y": 1.0928055551152926 }, { "x": 772, "y": 1.0927666537792764 }, { "x": 773, "y": 1.0927279178210296 }, { "x": 774, "y": 1.0926893464936394 }, { "x": 775, "y": 1.092650939053865 }, { "x": 776, "y": 1.092612694762116 }, { "x": 777, "y": 1.0925746128824354 }, { "x": 778, "y": 1.0925366926824769 }, { "x": 779, "y": 1.0924989334334863 }, { "x": 780, "y": 1.0924613344102816 }, { "x": 781, "y": 1.0924238948912346 }, { "x": 782, "y": 1.0923866141582494 }, { "x": 783, "y": 1.0923494914967447 }, { "x": 784, "y": 1.0923125261956335 }, { "x": 785, "y": 1.0922757175473061 }, { "x": 786, "y": 1.092239064847608 }, { "x": 787, "y": 1.092202567395824 }, { "x": 788, "y": 1.0921662244946586 }, { "x": 789, "y": 1.0921300354502146 }, { "x": 790, "y": 1.09209399957198 }, { "x": 791, "y": 1.0920581161728065 }, { "x": 792, "y": 1.0920223845688892 }, { "x": 793, "y": 1.0919868040797533 }, { "x": 794, "y": 1.0919513740282327 }, { "x": 795, "y": 1.091916093740452 }, { "x": 796, "y": 1.091880962545811 }, { "x": 797, "y": 1.0918459797769653 }, { "x": 798, "y": 1.091811144769809 }, { "x": 799, "y": 1.0917764568634565 }, { "x": 800, "y": 1.0917419154002272 }, { "x": 801, "y": 1.091707519725627 }, { "x": 802, "y": 1.0916732691883302 }, { "x": 803, "y": 1.0916391631401632 }, { "x": 804, "y": 1.0916052009360884 }, { "x": 805, "y": 1.0915713819341872 }, { "x": 806, "y": 1.0915377054956412 }, { "x": 807, "y": 1.091504170984718 }, { "x": 808, "y": 1.0914707777687542 }, { "x": 809, "y": 1.0914375252181376 }, { "x": 810, "y": 1.091404412706292 }, { "x": 811, "y": 1.0913714396096617 }, { "x": 812, "y": 1.0913386053076921 }, { "x": 813, "y": 1.0913059091828186 }, { "x": 814, "y": 1.0912733506204464 }, { "x": 815, "y": 1.0912409290089367 }, { "x": 816, "y": 1.0912086437395905 }, { "x": 817, "y": 1.0911764942066322 }, { "x": 818, "y": 1.0911444798071963 }, { "x": 819, "y": 1.091112599941309 }, { "x": 820, "y": 1.0910808540118748 }, { "x": 821, "y": 1.091049241424661 }, { "x": 822, "y": 1.091017761588281 }, { "x": 823, "y": 1.0909864139141818 }, { "x": 824, "y": 1.0909551978166265 }, { "x": 825, "y": 1.0909241127126807 }, { "x": 826, "y": 1.090893158022197 }, { "x": 827, "y": 1.0908623331678013 }, { "x": 828, "y": 1.090831637574877 }, { "x": 829, "y": 1.0908010706715516 }, { "x": 830, "y": 1.09077063188868 }, { "x": 831, "y": 1.0907403206598334 }, { "x": 832, "y": 1.0907101364212823 }, { "x": 833, "y": 1.0906800786119826 }, { "x": 834, "y": 1.0906501466735639 }, { "x": 835, "y": 1.0906203400503114 }, { "x": 836, "y": 1.0905906581891558 }, { "x": 837, "y": 1.0905611005396563 }, { "x": 838, "y": 1.0905316665539904 }, { "x": 839, "y": 1.090502355686936 }, { "x": 840, "y": 1.0904731673958594 }, { "x": 841, "y": 1.0904441011407044 }, { "x": 842, "y": 1.090415156383975 }, { "x": 843, "y": 1.0903863325907233 }, { "x": 844, "y": 1.0903576292285366 }, { "x": 845, "y": 1.0903290457675248 }, { "x": 846, "y": 1.0903005816803046 }, { "x": 847, "y": 1.0902722364419912 }, { "x": 848, "y": 1.0902440095301784 }, { "x": 849, "y": 1.0902159004249325 }, { "x": 850, "y": 1.0901879086087747 }, { "x": 851, "y": 1.090160033566672 }, { "x": 852, "y": 1.0901322747860198 }, { "x": 853, "y": 1.0901046317566345 }, { "x": 854, "y": 1.0900771039707373 }, { "x": 855, "y": 1.0900496909229436 }, { "x": 856, "y": 1.090022392110249 }, { "x": 857, "y": 1.0899952070320178 }, { "x": 858, "y": 1.0899681351899717 }, { "x": 859, "y": 1.0899411760881759 }, { "x": 860, "y": 1.0899143292330282 }, { "x": 861, "y": 1.0898875941332458 }, { "x": 862, "y": 1.0898609702998547 }, { "x": 863, "y": 1.0898344572461758 }, { "x": 864, "y": 1.089808054487816 }, { "x": 865, "y": 1.089781761542654 }, { "x": 866, "y": 1.0897555779308286 }, { "x": 867, "y": 1.0897295031747283 }, { "x": 868, "y": 1.0897035367989794 }, { "x": 869, "y": 1.089677678330434 }, { "x": 870, "y": 1.0896519272981593 }, { "x": 871, "y": 1.0896262832334245 }, { "x": 872, "y": 1.089600745669692 }, { "x": 873, "y": 1.0895753141426043 }, { "x": 874, "y": 1.0895499881899733 }, { "x": 875, "y": 1.0895247673517698 }, { "x": 876, "y": 1.089499651170112 }, { "x": 877, "y": 1.0894746391892531 }, { "x": 878, "y": 1.0894497309555728 }, { "x": 879, "y": 1.0894249260175655 }, { "x": 880, "y": 1.0894002239258285 }, { "x": 881, "y": 1.089375624233053 }, { "x": 882, "y": 1.0893511264940114 }, { "x": 883, "y": 1.0893267302655492 }, { "x": 884, "y": 1.089302435106572 }, { "x": 885, "y": 1.0892782405780357 }, { "x": 886, "y": 1.0892541462429375 }, { "x": 887, "y": 1.0892301516663037 }, { "x": 888, "y": 1.0892062564151799 }, { "x": 889, "y": 1.0891824600586217 }, { "x": 890, "y": 1.0891587621676837 }, { "x": 891, "y": 1.0891351623154082 }, { "x": 892, "y": 1.0891116600768167 }, { "x": 893, "y": 1.089088255028901 }, { "x": 894, "y": 1.0890649467506093 }, { "x": 895, "y": 1.0890417348228403 }, { "x": 896, "y": 1.089018618828431 }, { "x": 897, "y": 1.088995598352148 }, { "x": 898, "y": 1.088972672980678 }, { "x": 899, "y": 1.088949842302615 }, { "x": 900, "y": 1.0889271059084553 }, { "x": 901, "y": 1.088904463390586 }, { "x": 902, "y": 1.0888819143432733 }, { "x": 903, "y": 1.088859458362656 }, { "x": 904, "y": 1.0888370950467356 }, { "x": 905, "y": 1.0888148239953652 }, { "x": 906, "y": 1.0887926448102418 }, { "x": 907, "y": 1.0887705570948962 }, { "x": 908, "y": 1.0887485604546847 }, { "x": 909, "y": 1.0887266544967789 }, { "x": 910, "y": 1.088704838830157 }, { "x": 911, "y": 1.0886831130655954 }, { "x": 912, "y": 1.0886614768156577 }, { "x": 913, "y": 1.0886399296946887 }, { "x": 914, "y": 1.088618471318803 }, { "x": 915, "y": 1.088597101305877 }, { "x": 916, "y": 1.0885758192755404 }, { "x": 917, "y": 1.0885546248491673 }, { "x": 918, "y": 1.0885335176498665 }, { "x": 919, "y": 1.0885124973024753 }, { "x": 920, "y": 1.0884915634335468 }, { "x": 921, "y": 1.0884707156713462 }, { "x": 922, "y": 1.0884499536458394 }, { "x": 923, "y": 1.0884292769886832 }, { "x": 924, "y": 1.0884086853332207 }, { "x": 925, "y": 1.0883881783144698 }, { "x": 926, "y": 1.0883677555691162 }, { "x": 927, "y": 1.0883474167355056 }, { "x": 928, "y": 1.0883271614536336 }, { "x": 929, "y": 1.088306989365139 }, { "x": 930, "y": 1.0882869001132955 }, { "x": 931, "y": 1.0882668933430035 }, { "x": 932, "y": 1.0882469687007803 }, { "x": 933, "y": 1.0882271258347564 }, { "x": 934, "y": 1.088207364394663 }, { "x": 935, "y": 1.0881876840318268 }, { "x": 936, "y": 1.0881680843991595 }, { "x": 937, "y": 1.0881485651511533 }, { "x": 938, "y": 1.0881291259438717 }, { "x": 939, "y": 1.0881097664349402 }, { "x": 940, "y": 1.0880904862835399 }, { "x": 941, "y": 1.0880712851504013 }, { "x": 942, "y": 1.0880521626977933 }, { "x": 943, "y": 1.0880331185895191 }, { "x": 944, "y": 1.0880141524909062 }, { "x": 945, "y": 1.0879952640687998 }, { "x": 946, "y": 1.087976452991555 }, { "x": 947, "y": 1.0879577189290304 }, { "x": 948, "y": 1.0879390615525795 }, { "x": 949, "y": 1.0879204805350438 }, { "x": 950, "y": 1.0879019755507449 }, { "x": 951, "y": 1.0878835462754803 }, { "x": 952, "y": 1.0878651923865101 }, { "x": 953, "y": 1.0878469135625557 }, { "x": 954, "y": 1.0878287094837908 }, { "x": 955, "y": 1.0878105798318318 }, { "x": 956, "y": 1.0877925242897353 }, { "x": 957, "y": 1.087774542541986 }, { "x": 958, "y": 1.0877566342744944 }, { "x": 959, "y": 1.0877387991745875 }, { "x": 960, "y": 1.0877210369310004 }, { "x": 961, "y": 1.0877033472338729 }, { "x": 962, "y": 1.0876857297747393 }, { "x": 963, "y": 1.087668184246526 }, { "x": 964, "y": 1.0876507103435389 }, { "x": 965, "y": 1.0876333077614606 }, { "x": 966, "y": 1.0876159761973436 }, { "x": 967, "y": 1.0875987153496025 }, { "x": 968, "y": 1.087581524918007 }, { "x": 969, "y": 1.087564404603677 }, { "x": 970, "y": 1.0875473541090737 }, { "x": 971, "y": 1.087530373137996 }, { "x": 972, "y": 1.0875134613955715 }, { "x": 973, "y": 1.087496618588251 }, { "x": 974, "y": 1.0874798444238032 }, { "x": 975, "y": 1.0874631386113052 }, { "x": 976, "y": 1.08744650086114 }, { "x": 977, "y": 1.0874299308849875 }, { "x": 978, "y": 1.0874134283958192 }, { "x": 979, "y": 1.087396993107892 }, { "x": 980, "y": 1.087380624736741 }, { "x": 981, "y": 1.0873643229991758 }, { "x": 982, "y": 1.087348087613271 }, { "x": 983, "y": 1.0873319182983627 }, { "x": 984, "y": 1.0873158147750417 }, { "x": 985, "y": 1.0872997767651458 }, { "x": 986, "y": 1.087283803991757 }, { "x": 987, "y": 1.087267896179193 }, { "x": 988, "y": 1.0872520530530017 }, { "x": 989, "y": 1.0872362743399562 }, { "x": 990, "y": 1.0872205597680475 }, { "x": 991, "y": 1.0872049090664804 }, { "x": 992, "y": 1.0871893219656654 }, { "x": 993, "y": 1.0871737981972163 }, { "x": 994, "y": 1.08715833749394 }, { "x": 995, "y": 1.087142939589834 }, { "x": 996, "y": 1.087127604220081 }, { "x": 997, "y": 1.087112331121039 }, { "x": 998, "y": 1.087097120030243 }, { "x": 999, "y": 1.0870819706863912 }, { "x": 1000, "y": 1.0870668828293453 } ] }, { "name": "data_0", "source": "source_0", "transform": [ { "as": "x", "expr": "toNumber(datum[\"x\"])", "type": "formula" } ] } ], "height": 200, "marks": [ { "encode": { "update": { "defined": { "signal": "isValid(datum[\"x\"]) && isFinite(+datum[\"x\"]) && isValid(datum[\"y\"]) && isFinite(+datum[\"y\"])" }, "description": { "signal": "\"x: \" + (format(datum[\"x\"], \"\")) + \"; y: \" + (format(datum[\"y\"], \"\"))" }, "stroke": { "value": "#4c78a8" }, "x": { "field": "x", "scale": "x" }, "y": { "field": "y", "scale": "y" } } }, "from": { "data": "data_0" }, "name": "marks", "sort": { "field": "datum[\"x\"]" }, "style": [ "line" ], "type": "line" } ], "padding": 5, "scales": [ { "domain": { "data": "data_0", "field": "x" }, "name": "x", "nice": true, "range": [ 0, { "signal": "width" } ], "type": "linear", "zero": false }, { "domain": { "data": "data_0", "field": "y" }, "name": "y", "nice": true, "range": [ { "signal": "height" }, 0 ], "type": "linear", "zero": true } ], "style": "cell", "title": { "frame": "group", "text": "Loss" }, "width": 800 }, "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0MAAAD0CAYAAABZ5/2/AAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nO3deXxU9b3/8ffsk8k6WSAkBCEokcoiIvEWqa34Q/pTwZ8KrrWtelFrHyiuXW4vt7V9XOX6UFu1vT6uS731qqVa9XrV1g2wgrcqyiY7RNasZF9ny/n9MZkxCZmwTDJzcF7Px8NHcpaZ8zmfpsm8+Z7zPRbDMAwBAAAAQIqxJrsAAAAAAEgGwhAAAACAlEQYAgAAAJCSCEMAAAAAUhJhCAAAAEBKIgwBABJm2rRpslgsuvbaa5NdCgAAhCEAAAAAqYkwBAAwjWAwqGXLlmnq1KlKT09XSUmJLr74Ym3cuDG6z+uvv67y8nJlZmYqLS1N48aN04033hjdft999+nkk0+W2+1WVlaWJk2apMcffzwZpwMAMDl7sgsAACDijjvu0KOPPiqPx6PLL79cmzdv1muvvaZVq1Zp/fr1Ki4u1uWXXy6/369bbrlF+fn5qqio0LZt2yRJq1at0k9/+lMVFRXpzjvvlNVq1caNG7Vz584knxkAwIwIQwAAUwiFQnriiSckSQ8//LBuvPFG+f1+jR07VlVVVXrhhRd02223yefzyWazqaSkRNOnT9cPf/hD5eXlSZLa2tokSU6nU+PGjdPUqVP1k5/8RB6PJ2nnBQAwLy6TAwCYQmVlpbq6uiRJp512mqRwqCkrK5MkVVRUKD09XQ899JA8Ho/uuecenXfeeSooKNCll14qv9+vuXPn6tprr9WBAwe0aNEilZeXKy8vT/fdd1/SzgsAYF6EIQCAKRQVFcntdkuStmzZIkkKBALasWOHJGncuHGSpFtvvVX19fXatm2bnn/+eWVkZOiVV17R+++/L5vNpj/84Q9qbW3V2rVrde+996qrq0tLly6V3+9PzokBAEyLy+QAAAn35ptv6swzz4wu5+Tk6N1339UNN9yg3/72t7r99tu1Zs0abdmyRZWVlcrKytLVV18twzBUUlKiuXPn6qSTTlJDQ4N8Pp8kacSIEVqxYoXuvvtufeMb39DIkSO1YcMGSZLX65XD4UjKuQIAzIswBABIuIaGBjU0NESX8/PzJUm//vWvVVZWpieeeEIvvviivF6v5s2bp3vvvVfjxo2TYRiaMWOG3nnnHdXV1clms+lrX/uaFi9erKlTp2rr1q3Kzc3VH//4RzU2Nio7O1uzZ8/Wr371K1kslmSdLgDApCyGYRjJLgIAAAAAEo17hgAAAACkJMIQAAAAgJREGAIAAACQkghDAAAAAFISYQgAAABASiIMAQAAAEhJCX/O0P/8z//otNNOS/RhAQAAAKSo0tLSAdcnPAxt2bJF8+bNS/Rhj1pFRUXMZpkB9cWH+uJDffGhvvhQX3yoLz7UFx/qiw/1xaeioiLmtoReJnfwUKvWt47Svf/5QSIPCwAAAACHSWgY6u421BZyqbqhLZGHBQAAAIDDJDQMOezhwwWC3Yk8LAAAAAAcJsFhyCZJCgRDiTwsAAAAABwmsWHIFj6cnzAEAAAAIMkSGoacjsjIEJfJAQAAAEiuhIYhe8/IUDBEGAIAAACQXAkPQxZLOAx1dxuJPDQAAAAA9JHQMCRJFoVDEKNDAAAAAJIp4WHI2hOGmEQBAAAAQDIlPgxZwmGISRQAAAAAJFMSRobCAiFGhgAAAAAkTxLuGQqPCAUCjAwBAAAASJ4kXCYX/srIEAAAAIBkSt7IEPcMAQAAAEiipE2gwGxyAAAAAJIpaVNrMzIEAAAAIJmSd88QI0MAAAAAkoiRIQAAAAApKfETKEQfusrIEAAAAIDkSd7IUIiRIQAAAADJk7Sptf0BRoYAAAAAJE/Cw5At+tBVRoYAAAAAJE8SRoa4ZwgAAABA8iVtAoUgs8kBAAAASCImUAAAAACQkuyDbQwGg9q6datGjx4tr9c74D6bNm2S3++PLk+fPn3QA0YeusoECgAAAACSKWYY+tOf/qSbbrpJ3d3deuqpp7RgwYIB97vooot08skny2q1ymKx6O233x70gNae2eTau/yD7gcAAAAAwylmGJozZ46qqqq0aNGiI77JX/7yFzmdzqM6oMsaHhF677M9SnM5dPm3Jio7w32U5QIAAADA0Ih5z5DX65XbfXQhpbW1tc+lcoPJdXToe3OnSJJeW7ND//jAG3ry9XWqb+k8qtcDAAAAwFCIewIFj8ejOXPmaOzYsbrxxhvV3X3kiREWfGuifrP4fJ31tWL5AkH995odWvTA63rkz5/oQF1LvCUBAAAAwBFZDMMwBtvh2muv1cUXXxzznqFAICCHw6Guri7NnTtXP/zhD3X55ZdLklavXq01a9Yc9pqFCxdGvz9Y36F31ldp/ReNMgxDFotFk8bk6FuTR+rkUZnxnBsAAAAAqLS0dMD1g84mdzQcDockye1265xzztHu3buj22bNmqVZs2b12X/ZsmV9iiktlb4xY5IqD7XqldXbteLTPdq0t1Gb9jbq5GKv5p89QbMmj5HDnphZwCsqKmI2ywyoLz7UFx/qiw/1xYf64kN98aG++FBffKgvPhUVFTG3xUwYgUBAjY2N8vv9am9vV1NTkySps7NTDz30kCSpurpaK1asUE1NjdasWaNnn31W55xzznEVWZSfqR/+vzP19I8u0tX/Z5KyM9zadbBRD/3pI12/7H/0X+9s4r4iAAAAAEMm5sjQunXrdMstt0iSHn30UT3zzDNauXKl/H6/XnnlFd1xxx0KhUL63e9+pz179igvL08PPPCAzj777LgKys5w66rzTtNl3zxV76/fp9c/3KGKqiYtX7FFL63aqrMmFmtu+XhNO2WkLBZLXMcCAAAAkLpihqHy8nKtXbv2sPXZ2dn64IMPJEnFxcV66aWXhqUwp92mOWeO05wzx2nznjq9+fdd+vDzA/pwc/i/kd50nT+jVOdNH6e8rLRhqQEAAADAV1fc9wwlwmljC3Ta2AI1tXXpnbVf6K2Pd6umsV3Pvr1Jz737uc44pVDnTR+nsyYWJ+zeIgAAAAAnthMiDEXkZLi18FsTteCbp2rDrhq9tbZCH205qLXbq7R2e5Uy0pz6xpQSnTttrE4dky+uogMAAAAQywkVhiIsFotOP6VQp59SqNYOv1at36P3Pt2j3ZWN+stHu/WXj3arMDdD3zx9jM6ZMkZjRmYnu2QAAAAAJnNChqHeMj1OzZs5QfNmTtDemmat/GyP3t+wT9UNbVq+YouWr9iisYXZmjV5jGZNKVFxPs8uAgAAAPAVCEO9nTQyW9//v1P1vW9P0edf1On99Xv14eYD2lPdrD3Vm/Rf72zSuFE5OntSiWZOGq2SEVnJLhkAAABAknylwlCExWLR5NIRmlw6QjdfPF3rdlZr9ab9+mjLQX1R1aQvqpr0X+9sUnFBpr5+2midNbFYZSW5TNUNAAAApJCvZBjqzW6zasapRZpxapGCoW6t31WjDz/fr4+2VupgXateWrVVL63aqpwMt8onFmmM16ai0UG5nV/51gAAAAApLaU+8dttVp1ZNkpnlo1Sd7ehzXvq9NGWg/r7loOqaWzX259USJL+c8VuTRo3QmeWjdL0slHcZwQAAAB8BaVUGOrNav3yUrp/vGia9lQ365NtlfpgfYX21rVr3c5qrdtZrSdeX6fC3AydMaFQ004p1JTxI+RxOZJdPgAAAIA4pWwY6m9sYbbGFmZr+hiX8kcW67Od1Vq7vUrrdlaruqFNb/59l978+y7ZrBZNKMnT1JNHaur4kSoryeNBrwAAAMAJiDA0gKx0l751+kn61uknyTAM7TrYqM92VGv9rmpt21evrXsPaeveQ/rje5vlctg08aT86CjTKaNzZbcRjgAAAACzIwwdgcVi0Smjc3XK6FxdMftr6vIHtamiVht21WjD7hrtrWnW+l01Wr+rRpLkdtpVVpKnSeMKdNq4Ak0oyZPLYUvyWQAAAADojzB0jNxOe3R2Oklqbvfp8y9qtamiTpsqarW/tlkbdoeDkhSetGF8sVcTx+Rr4th8nTomT7mZack8BQAAAAAiDMUtO92lsyeV6OxJJZLC4Wjznjpt/qJOW/bUqaKqSdv31Wv7vnq9unq7JGmEN12njsnTqWPyNKEkT6WjvNx3BAAAACQYYWiIZae7NPO00Zp52mhJUqcvqO37DmnL3kPauq9eO/bVq7axXbWN7frbhn2SwqNHpUU5OmV0XviSvGKvRhdkyWrlIbAAAADAcCEMDbM0l12nn1Ko008plCQZhqF9tS3atq9e2/cd0o79DdpX26Id+xu0Y39D9HVup13ji7waX+zVycVejS/yqrggK1mnAQAAAHzlEIYSzGKx6KSR2TppZLbmziiVFB492nmwQTv212vXgUbtPNig2sb28OV2e+qir3U6bBrldWvi2AaNG5WjcaNyNLYwR2ku/mcEAAAAjhWfok0gzWXXlNIRmlI6Irqupd2n3ZWN2nWwUbsPNqqiqlHVDW3aW9uuvbW7o/tZLFJhbobGFeZoTM+zksaOzNGovAwuswMAAAAGQRgyqax0l6adUqhpPZfXSVJ7V0BrPt2iLqWpoqpJX1Q1aX9ts6rq21RV36YPNx+I7uu02zS6IFNjRmZrzMhslYzI0pgRWSrMJSQBAAAAEmHohJLudujkUZkqLS2Nrgt1GzpY16I91c3aU92kvTXN2lvdrNqmdlVUNamiqqnPezjsVhXnZ2n0iEyNLshSSUGWRhdkqig/U24nPw4AAABIHXz6PcHZrJbo6M85U8dE13f6gtpX26x9Nc3aX9uivTXNOlDbqtqmdu2pbtKe6qbD3is/26Pi/EwVF2SqKC9DxQVZKsrL0Ahvuuw2pv4GAADAVwth6CsqzWVXWUmeykry+qzv8ge1v7ZFB+padKCuNfq1qr5Vh5o7dKi5I/rA2Aib1aIR3nSNysvQqLxMFeama1RuhgrzMlSYmyGXw5bIUwMAAACGBGEoxbid9vCzjEbn9lnf3W2otqldBw+1qvJQqw4ealNlz/d1zR3R+5Kk6sPe05vpVmFuhgpz0+VUQKc2WjTSm66R3nTlZXtk4x4lAAAAmBBhCJIkq9XSE2gyNH3CqD7bgqFu1TS0q7K+VZX1baqub1NVQ5uqG9pU29iuxtYuNbZ2aeveQ5Kkt9ZVRl9rs1qUl+XRCK9HI7zp4f9yPCrI9ig/x6MROelyMrIEAACAJCAM4YjsNquKC8L3EvVnGIYONXeqprFdNQ1t2lpxQH7DqZrGNtU2dqi+pVO1Te2qbWqXvqgb4N2l7HSX8rI9ys9O00hvunIz05SX7VFBjkd5WWnKy0ojMAEAAGDIEYYQF4vFooKccHCZNK5A47xGn9nugqFu1TV1qLapPfy1sV21TR2qawrfn1TX1K7mdp+a232qqGyMeZxMj1N5WWnKz/bIm5mm/Ow05WalKTez52uWWznpbqYNBwAAwFEjDGFY2W3WnokXMmLu09jaFZ28IRyQOlXfElnuVENLp1o7/Grt8GtPdXPM97FaLcpJdysn0628rDTlZLiUl+VRdoZLuZlpyslwq7W5S0X+INOIAwAAgDCE5PNmuuXNdB82qUNvja1damjtVH1zp+pbOnu+71BTm0+HmjvU2Nql5vbwPg2tnYOOMkmb5HLYlJMRPm52uls5GS7lZLqVne5Sdnp4fVa6S1kel7LTXYw4AQAAfAURhnBCiASm8UXemPsEQ91qbvepoaUzGp4aW7vU1NalhpZONbf7VNvQqrauoHyBUPg+p8b2ozp+drorGo6y0l1fLkfWeZzK9LiUneFSpscpj8sxVKcOAACAYUIYwleG3WaNTrgQS0VFhUpLS9XlD0aDUnO7Lzyy1PN9U3uXmlq71NLhV3O7Ty099zQ1t/uOqZbMnoCUmebs+d6pjDRX9PtMj1OZaU5lpDmV4Ql/BQAAQOIQhpCS3E77Ee9lijAMqaW9Vzjq8Km5LRySWjrCX1s7/OH17T61dvjU6QtGpxw/FlaLlOHZqMw0p9LTHOGglOZUutuh9J6vGWlOZbjD29N7vnpcDqWnOeS0M+seAADA0SIMAUdgsUjZGW5lZ7hVcpSvCYa61dLhU1uHXy0dfrV2+NTWGZ4EorXTr9Z2n1o7/WrrDKitw6e2zoBaO8MhqqVnNOp42G1Wpbsd8rgd4QDldkaXPe5waEpz2eVxO5Thdka/T3PZleYKv8bjcnCPFAAASAmEIWAY2G3W8LTfmbEv2RvIrt0VKigsVlunX+09Aam9M6D2roDaO/1q7wqordOvtq5wkOroCu/X1rM9ct/UsVzSNxCXwya3y6E0p10ZaQ65neGwFAx0aeSGBrmddrlddrmddqW7w/s5HbZooHI5bHL1bHM57HLYrXHVAwAAMBwIQ4CJWC3qmdHOdVyv9wdD6ugKh6fI1/Yuvzp6ljt8QXX6vtzW0fN9py+oTl9QHb7w/r5ASL5ASANOZL674djPy2qRxxUOVS6HTWk9QcrpsMnjcsjltMtptyrd7ZTTYZPDblWG2ymHwxbd32GzyuN2ytWz3eNyyNYzEgYAAHA8CEPAV4jTbpMzIzxteDy6/EF1+cMBKRKaunxB7TtQqYxsrzp9AXX5Qz0hKrzdFwjJ5w+qrbMnTPlD4df5gwqGusMjWp3+ITrTvhx2q1wOu+xWye3aqow0RzSAOezhQBUOUeFgZbOGQ5TFalG62yGb1aI015df7Tar3E57z1dbv2V+bQIA8FXBX3UAh3E7wyM3Of3ml8h3dam0tPSY3y/UbURDUyDQrU5/OET5g6HoqFQgGA5PPn9IgWBI7V3h7f5AeH0g2K3OntAVCHarvcuvUMiIbgsEe4JW+/AErt4ioSgSniwWRUeo0t1OWSyK3nsVCWEOu1Wd7W3y7myTp2ff/vtYLZbotkhoiwSy3u9NKAMAYGjw1xTAsLNZLdGZ8YZDJDRt31mhouLRau8KKBQKh65AMBQdtQqEwoEqFDLU3hVQt2GooyugUHd3OJCFuuXrGcnq8gcVCHbLFwj1Wg5FlyOjXMd+f1bVkJ57JDRFRsIkRUezJMnjdspqUZ/tkZE0SdEgJkmd7S3K3x8+nwy3U+qZRyPd7ZDFEl7oPcFG5NiSlOa0y2YLr3fabXI6wu9JcAMAmBl/oQCc8Jx2m5x2m7wZzqOaLj1ekXAUCVyGIbV3BSQpGpI6esJWJIQFgt2qqqlVTo5Xnb6gJKm9yy/DkHyBcPDq7g6PdElSpy+oUHd3NJD13j8SyiL7RRzvLIR9HRyC94itd7DqHZp6j65Jil7eGOFxOdTZ2aH0v1f3CXbhfb8Md1I44Nl7TdrRfxKP3gFwoOXIiN/hdX+53L8+STIM4+iaAAAwDcIQABwju80aHeXKPobXVVTYjusyw8F0+ALq7jailyJK6rlPK/zBPBKgIqNfUt8wFQlrklRVc0hZWeEz6n1/V1uXX+r5nN/eFYh+6I8cW4qEt/D3/kBI/mD4/SPBMSISGo/fsU/gkVhro9+FA17f0dCBglY4zPUNVrFG1CKXSvbmcoYnGOmv/0hsU1NTn8s0+9ZlVZpr4I8EvUcJ+xy3X4jsLTxSOPAskv2DZUR9q0+5bV3RUc3+BgqgABAvwhAAnMB6j5Ac7yyEERUVFUMe1vrrHaZ6h6beo2tS38AmhUfaqqqrNXLkyD4jaOF9u+ULfBm4fP7wpYzR5Z6Rt9jL4fvUIvoHOElq6+wb4iKXZsY6t8g5DTRpSLxT38dnaC/THHobj/uVR7ok82gu2ex9iWl/Pp9PLteuo3oW29EGt6MOeJaeUDqIpqYm5exokxS+hPVonxc3UEA/ksHCbiy1tQ1q6U4/ptdEDBa8j/71g9db3+qTp6Et5vbIJDpDgWBvLoQhAEDC9JkK/dgew6UKj1+lpUf76OPE6x8mwwGvbxjqPYIX0ftSyIiBApk0cLjqPbr35cF7RvR6aWho6HOZ5tEcT+o7MtjnuP1CZG+RyzwHEg6Nh68PBAIKGZY+QbbP9n4Bub/e9/INn45hfv94mT3s7k52AUdw/GE8MT6JuSXewDgYm82qtCP8Q0IgEJDDsTXuY3ncDg3Hc98XX3ByzG2EIQAAhoHFcvilalL8I3jHKxEjf/GIt77+I4T9DRb4Irr6jSr2dvDgQRUXF/e5PDR2LYMHt4jeo6ODGiDc9tfQ0KDc3FxJ4TB6pBojBgroR9LZc8/ksWhra5dsx/dcuMGC99G/fvB6wx/mY9d3POcc81hH+fNxLCLPBxwuAz538PAqhu34w4kwBAAATngOu1UO+/DMWClJVl+jSou9w/b+8fqqh93hdqLXF29gHMzR/EPC/v37VVIS/8h9ePKhuN/mcL7GmJsIQwAAAMAJLPJw8WTpaHKpMHf4Z3M9XhUVscPQ0NwJBgAAAAAnGMIQAAAAgJQUMwwdPHhQjz32mG655RZ99tlnMd/A7/frkUce0Y9//GNt2rRpWIoEAAAAgKEWMwxt2LBBu3bt0tq1a1VRURHzDW644Qbt3r1b5eXluuSSS7Rv375hKRQAAAAAhlLMCRQuuOACXXDBBbr22mtjvriurk7vvvuuDhw4IJvNps2bN+uZZ57R0qVLh6VYAAAAABgqcd0ztHPnTpWVlclmC89eMXXqVG3dGv8DlwAAAABguMUVhlpaWuTxeKLL6enpampqirsoAAAAABhucT1nKD8/v0/4aWxsVEFBQXR59erVWrNmzWGvG+weJDOgvvhQX3yoLz7UFx/qiw/1xYf64kN98aG++Ji9vljiCkOnnnqqdu/erba2NmVkZOiDDz5QeXl5dPusWbM0a9asPq9ZtmzZCf2E32SjvvhQX3yoLz7UFx/qiw/1xYf64kN98aG++AwW1GKGoT179mj58uXavHmzDMPQgQMHtGTJEjU0NGjKlCk6cOCAMjIytHjxYl1wwQWaMWOGVqxYoV/84hfDchIAAAAAMJRihiGHwyGv16ubb75ZkuR0OiVJHo9H9957b3S/n/3sZ5ozZ46qqqq0dOlSZWdnD3PJAAAAABC/mGGouLhYN95442Hr3W63rr/++j7rzjrrrKGvDAAAAACGUVyzyQEAAADAiYowBAAAACAlEYYAAAAApCTCEAAAAICURBgCAAAAkJIIQwAAAABSEmEIAAAAQEoiDAEAAABISYQhAAAAACmJMAQAAAAgJRGGAAAAAKQkwhAAAACAlEQYAgAAAJCSCEMAAAAAUhJhCAAAAEBKIgwBAAAASEmEIQAAAAApiTAEAAAAICURhgAAAACkJMIQAAAAgJREGAIAAACQkghDAAAAAFISYQgAAABASiIMAQAAAEhJhCEAAAAAKYkwBAAAACAlEYYAAAAApCTCEAAAAICURBgCAAAAkJIIQwAAAABSEmEIAAAAQEoiDAEAAABISYQhAAAAACmJMAQAAAAgJRGGAAAAAKQkwhAAAACAlEQYAgAAAJCSCEMAAAAAUhJhCAAAAEBKIgwBAAAASEmEIQAAAAApiTAEAAAAICURhgAAAACkJMIQAAAAgJREGAIAAACQkuxH2iEUCslms8Xc3tzcrO7u7uiy1+sdmsoAAAAAYBjFDEN+v1/XX3+91q1bJ7fbreeee06nnnrqYftNmTJFNptNFotFVqtVO3fuHNaCAQAAAGAoxAxDTz/9tKxWqzZv3qy3335bN998s1atWjXgvtu2bZPT6Ry2IgEAAABgqMW8Z+j111/XVVddJUmaM2eOtm7dqqampoQVBgAAAADDKWYYqqqqUlFRkSTJYrGoqKhIVVVVA+572mmnaezYsfrlL385PFUCAAAAwBCzGIZhDLShvLxcjz/+uM444wxJ4XuDXnrpJU2YMKHPfnV1dSooKNChQ4c0Z84c/epXv9KFF14oSVq9erXWrFlz2HsvXLhwqM8DAAAAAAZUWlo64PqY9wwVFxdr//79OuOMMxQKhVRZWRkdKeqtoKBAkpSfn6958+Zpw4YN0TA0a9YszZo1q8/+y5Yti1mMGVRUVFBfHKgvPtQXH+qLD/XFh/riQ33xob74UF98ToT6Yol5mdyll16qJ598UsFgUH/84x9VXl6ujIwMdXV16emnn5YkNTQ0aPv27ZKk/fv36+WXX9b06dOHuHwAAAAAGHoxR4auueYabdy4UZMmTdKYMWP0+9//XpLU0dGhpUuX6vrrr1dLS4u++93v6sCBA8rOztaiRYs0d+7chBUPAAAAAMcrZhiyWq164IEH9MADD/RZn5ubqwMHDkiSxo4dq48++mh4KwQAAACAYRDzMjkAAAAA+CojDAEAAABISYQhAAAAACmJMAQAAAAgJRGGAAAAAKQkwhAAAACAlEQYAgAAAJCSCEMAAAAAUhJhCAAAAEBKIgwBAAAASEmEIQAAAAApiTAEAAAAICURhgAAAACkJMIQAAAAgJREGAIAAACQkghDAAAAAFISYQgAAABASiIMAQAAAEhJhCEAAAAAKYkwBAAAACAlEYYAAAAApCTCEAAAAICURBgCAAAAkJIIQwAAAABSEmEIAAAAQEoiDAEAAABISYQhAAAAACmJMAQAAAAgJRGGAAAAAKQkwhAAAACAlEQYAgAAAJCSCEMAAAAAUhJhCAAAAEBKIgwBAAAASEmEIQAAAAApiTAEAAAAICURhgAAAACkJMIQAAAAgJREGAIAAACQkghDAAAAAFISYQgAAABASiIMAQAAAEhJhCEAAAAAKYkwBAAAACAlEYYAAAAApCTCEAAAAICUZB9s444dO/TXv/5VZWVlmjt37oD7dHd365VXXlFNTY3mz5+v0aNHD0uhAAAAADCUYo4Mbdu2TRdeeKEcDocefPBBPfTQQwPud+edd2r58uUKBoOaPXu2amtrh61YAAAAABgqMUeGfve73+nWW2/VD37wAy1cuFBTp07VbbfdJpvNFt2nqalJzz//vPbv3y+n06nq6mo988wzuueeexJSPAAAAAAcr5gjQxs3btSZZ54pScrPz1d6erqqq6v77LN9+3ZNmDBBTqdTklReXsl2Li8AAA4GSURBVK4NGzYMY7kAAAAAMDRihqHGxkZlZmZGl7OyslRfX3/EfRoaGoahTAAAAAAYWjEvk/N6vWppaYkut7S0KC8v77B9Wltbo8vNzc3Kzc2NLq9evVpr1qzp8xqHw6Fly5bFXTgAAAAAHElBQYFKS0sH3mjEsHjxYuM3v/mNYRiGUVdXZxQVFRnBYLDPPo2NjcaIESMMn89nGIZh/OQnPzGWLVsW6y0NwzCM+++/f9DtyUZ98aG++FBffKgvPtQXH+qLD/XFh/riQ33xOZHrizkydPPNN2v+/PmyWq169dVXdfvtt8tms6m5uVkXXXSRPvjgA+Xk5OjKK6/U1VdfrZkzZ+rFF1/U6tWrhyvUAQAAAMCQiXnP0Ne+9jW98cYb6u7u1l133aW77rpLUvgyt3PPPTe638MPP6yrrrpKLpdLK1as0MiRI4e/agAAAACI06APXS0rK1NZWVmfdR6PR/fee2902Wq16rLLLhue6gAAAABgmNh+/vOf/zzRBx0zZkyiD3lMqC8+1Bcf6osP9cWH+uJDffGhvvhQX3yoLz4nan0WwzCMBNcCAAAAAEk36GVyw2nHjh3661//qrKyMs2dOzcpNXz22Wd65513osvnnnuuysvLJUnt7e166aWXFAqFdOmllyonJ2fY62lqatL777+v7du36+KLL+5zieJg9axatUobNmzQeeedp0mTJiW8Pp/Pp1//+tfR/UaMGKHrrrsu4fUdOnRIr7/+uurr63XWWWdp1qxZ0W1m6F+s+szSv7179+qtt95SU1OTJk6cqIsuukgWi0WSOfoXqz6z9C+ioaFBTzzxhK644gqNHTtWkjn6F6s+s/TveH8fJ7s+s/RPkjo7O/Xcc8+psbFRM2fO1Nlnny3JHP2LVZ9Z+vfqq69q+/btfdbdfffdslqtpuhfrPoCgYAp+idJf/vb3/Txxx+roKBACxculMfjkWSen7+B6jPLz58kvf/++/r00081ffp0ffOb34yuN0v/Bqovnv4N9pm3t+7ubr3yyiuqqanR/PnzNXr06Oi2jRs3auXKlTrjjDP0jW98I7q+sbFRf/7zn+VyubRgwQKlpaUNem4xJ1AYTtu2bdOFF14oh8OhBx98UA899FAyytCHH36o//3f/5XX65XX65Xb7Y5uu/jii7Vx40bt3btX559/vkKh0LDX8/bbb+vZZ5/Vs88+q02bNvXZFqueZ555Rv/0T/8kt9utyy+/XJ988knC6+vq6tLDDz8c7WNWVlZ0WyLru+2227Rx40a5XC7ddNNNevzxx6PbzNC/WPWZpX8fffSRamtrlZWVpUceeUR33HFHdJsZ+herPrP0L+KnP/2pfvOb32jXrl3RdWboX6z6zNK/4/l9bIb6zNK/zs5OzZ49W5988oncbrfeeuut6DYz9C9WfWbpX3p6erSGyspKPffcc7Jawx+RzNC/WPWZpX8vvPCClixZopEjR+rDDz/UvHnzotvM0L9Y9Zmlf//+7/+uH/3oR8rLy9P999+vp59+OrrNDP2LVV88/RvsM29vd955p5YvX65gMKjZs2ertrZWUvh38pVXXim326177rlHzz//vCQpEAho9uzZqqys1KeffqoFCxYc+QQTNsF3L4sXLzYeeeQRwzBiP8MoER599FHjl7/85WHrP/74Y2PGjBnR5Ysuush48803E1bXd77zHePFF188qnrKysqMbdu2GYZhGMuXLzeuvvrqhNfX1NRklJWVDbhvIusLhULR71etWmWcffbZhmGYp3+x6jNL/3rbvn17tCaz9C9WfWbq34cffmh873vfM7797W8b77zzjmEY5urfQPWZpX/H8/vYDPWZpX+PPfaY8d3vfvew9WbpX6z6zNK/3pYsWWL867/+q2EY5ulfrPrM0r/rr7/e+P3vf28YhmH4fD7D4/EYhmGe/sWqzyz9mz59uvHxxx8bhmEY1dXVRmlpqWEY5ulfrPqGon/9P1P2NtjzTBcsWBB93ZYtW4zJkycbhmEY//3f/23Mnz+/T+3r1q0b9PySMjK0ceNGnXnmmZKk/Px8paenq7q6OhmlaNOmTXryySe1bt26AeuTpPLycm3YsCEZ5Q1aj8/nU2VlZXRoccaMGVq/fn1Samxra9OTTz6p1157TYFAQJISXl/kX/Ekac+ePSouLpZknv7Fqk8yR/8kad26dXrkkUd000036ZJLLpFknv7Fqk8yR/+CwaB+/OMf6/777++z3iz9i1WfZI7+Scf2+9gs9Unm6N/KlSs1d+5c/fnPf9Zrr72mYDAoyTz9i1WfZI7+RYRCIf3pT3/SVVddJck8/YtVn2SO/l1wwQVatmyZHnroIS1atEhLly6VZJ7+xapPMkf/rFZr9P8TwWBQFRUVamlpMU3/YtUnDW//tm/frgkTJsjpdErq+3m8d28mTpyovXv3KhgMHtdn+KSEocbGRmVmZkaXs7KyVF9fn/A6TjrpJJWWlurgwYO64oor9Oijj0oKX1Pfv76GhoaE1xcRq56B+piMOm02m+bOnatDhw7pqaee0rnnnqtgMJi0+mpqavSLX/xC//zP/yzJfP3rX5/Z+hcKhZSenh69Jths/etfn1n698gjj+jSSy9VYWFhn/Vm6V+s+szSv2P9fWyW+szSv+rqav3bv/2bPv/8cz3//POaN2+eDMMwTf9i1WeW/kWsXLlSpaWl0fv9zNK/WPWZpX//8A//oClTpqi7u1sdHR1qbm6WZJ7+xarPLP37wQ9+oJtuukkPPvigFi1aFP1cbJb+xapvuPs32Pv035aRkaHGxsbj+gyflAkUvF5vNFFKUktLi/Ly8hJex7x586LXjV5zzTU699xztXjxYuXm5uqLL76I7tfc3Kzc3NyE1xcRqx6v16vW1tbo+paWlqTUmZGRoaeeeiq6PHPmTK1evVpf//rXE15fW1ub5s2bp/vvvz96s56Z+jdQfWbq37Rp0zRt2jTdcMMNmjBhgiorK03Vv4HqM0P/uru7dd9992nJkiVatmyZKioqtHz5co0bN84U/RusvvHjxye9f9Kx/z5O9M9frPrM8PMnhf+uzp8/X/fcc48Mw9D48eO1c+dO0/QvVn0TJkwwRf8iXnjhhT6jLmbpX6z6zPLzt3jxYt18883Re1pKSkp02223maZ/seobOXKkKfp33XXX6YwzztDnn3+u3/72t5oyZYoKCwtN079Y9aWlpQ1r//qfZ+/P45EsEckPbW1t8nq9ys3NPew1EydOHPQ4SRkZmjJlitauXSspPMNWe3v7Yf9amWgOhyN6U9rkyZOj9UnSJ598oqlTpyartJj1uFwuFRUVadu2bZKkjz/+OKl1RtjtdoVCoYTX5/f7ddlll+naa6/V5ZdfHl1vlv7Fqq+/ZPWvN8Mw1NbWpu7ubtP0L1Z9/SWrf3fddZfs9sP/fcks/YtVX39m+Pk7mt/HZqmvv2T1b+rUqXI4HJIki8Uiu90uwzBM079Y9fWXzJ+/rq4uvfbaa1q4cGF0nVn6F6u+/pLVv5qamj7/+1osFlP9/YhVX3/J/PmbOnWqrrnmGu3cuVOTJ09WWlqaafoXq77+hqp/TU1NMgxDZWVl2rFjh/x+v6S+n8d7Z4ktW7ZozJgxstvtx/UZPikjQzfffLPmz58vq9WqV199VbfffrtsNlvC6/jZz36mzMxMuVwuPf3007rlllskha8vzMjI0JIlS5SRkaGqqiqdf/75w17PwYMH9cYbb2jnzp1699131dbWpu9///uD1nPPPffouuuu03e+8x099thjeuaZZxJe31/+8hetXLlSJ510kj766CM1NzdHp3RNZH233nqrqqur5XK59B//8R/KysrSlVdeaZr+xarPLP1bsmSJ8vLylJ2dreeff17XXXed7Ha7afoXqz4z9M9qtepHP/pRdHnVqlW64oorNH78eI0fPz7p/RusPjP0Tzq+38dmqM8s/Vu0aJEuvPBCOZ1Obd68WUVFRTrllFNktVpN0b9Y9Zmlf5L05ptvqry8XAUFBdF1Zvn5i1WfWfq3YMEC3Xrrrbr11lu1evVqTZw4UaNGjdKoUaNM0b9Y9Zmlf6+99prWr18vm82mp556Sn/4wx8kmefnL1Z98fQv1mdKSRo1apSqq6uVk5OjK6+8UldffbVmzpypF198UatXr5YUnqF30aJFqq2t1bPPPhv9G3fBBRdo6dKl+pd/+Rc1NDQoPz9fp59++qDnl7SHrm7fvl1vvfWWJkyYoG9/+9vJKEGfffaZ/va3v6m7u1szZszoM0d5ZF73YDCoyy67LCHPGdq1a5eefPLJ6LLb7dbPf/7zI9azcuVKbdy4UbNnz9bkyZMTXl9dXZ1ef/111dbWqqSkRJdcckmffzFIVH2PP/649uzZE13Oy8vT3XffLckc/YtVn1n6t3XrVr333nvy+/2aOnWqzjvvvOg2M/QvVn1m6V9vTz/9tM455xydfPLJkszRv1j1maV/x/v7ONn1maV/UnhiljfeeEOZmZm64oor5HK5JJmjf7HqM1P/Xn31VeXn5/d5Rp1knv4NVJ+Z+vfee+9p3bp1Kiws1MKFC0338zdQfWbpX1VVlV5++WVZrVadf/75Gj9+fHSbGfoXq754+hfrM+UXX3yhG264QStWrJD05XOGqqurNX/+fJWUlERfs2HDBq1atUrTpk3TOeecE13f0NCgl19+WU6nUwsXLjzic4aSFoYAAAAAIGLz5s3q7OzsMyPccCMMAQAAAEhJSZlAAQAAAACSjTAEAAAAICURhgAAAACkJMIQAAAAgJT0/wGO2aYHegoxKAAAAABJRU5ErkJggg==", "image/svg+xml": [ "0501001502002503003504004505005506006507007508008509009501,0000.00.51.01.5Loss" ], "text/plain": [ "@vlplot(\n", " width=800,\n", " title=\"Loss\",\n", " mark=\"line\",\n", " encoding={\n", " x={\n", " type=\"quantitative\",\n", " field=\"x\",\n", " title=nothing\n", " },\n", " y={\n", " type=\"quantitative\",\n", " field=\"y\",\n", " title=nothing\n", " }\n", " },\n", " data={\n", " values=...\n", " }\n", ")" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "@vlplot(:line, x = {collect(1:1000), type=\"quantitative\"}, y={losses, type=\"quantitative\"}, width=800, title=\"Loss\")" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Now, we can do this for all hospitals, and compare the different benchmark models and their prediction errors. \n", "\n", "We initialise the parameters `betas` and `lrbetas` for both the increment model and the standard linear regression model estimated via classical maximum likelihood. For each model, we also initialize an array for the predictions from this model for each hospital. " ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "1281" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "nhospitals = length(unique(all_hospitals_df[:,:hospital]))" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [], "source": [ "betas = collect([0.0; 0.0; 0.0] for i in 1:nhospitals)\n", "lrbetas = collect([0.0; 0.0; 0.0] for i in 1:nhospitals)\n", "\n", "prederrs_mean = Array{Union{Float64, Missing}}(missing, nhospitals)\n", "prederrs_modmean = Array{Union{Float64, Missing}}(missing, nhospitals)\n", "prederrs_zero = Array{Union{Float64, Missing}}(missing, nhospitals)\n", "prederrs_linreg = Array{Union{Float64, Missing}}(missing, nhospitals)\n", "prederrs_incrmod = Array{Union{Float64, Missing}}(missing, nhospitals);" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "121.561528 seconds (1.58 G allocations: 63.377 GiB, 24.63% gc time, 5.99% compilation time)\n" ] } ], "source": [ "@time for i = 1:nhospitals\n", " # get the covariates and info about missing reports\n", " curhospital = filter(row -> row[:hospital] == i, all_hospitals_df)\n", " y, z, r = curhospital[1:end-1,:currentcases], curhospital[1:end-1,:incidences], curhospital[1:end-1,:reported]\n", " last_currentcases, lastreported = curhospital[end,:currentcases], curhospital[end,:reported]\n", "\n", " # run the gradient descent via automatic differentiation/differentiable programming\n", " β = betas[i]\n", " for epoch=1:150\n", " curgrad = gradient(arg -> loss(y, z, r, arg), β)\n", " β .-= [0.001, 0.01, 0.001] .* curgrad[1]\n", " end\n", " betas[i] = β\n", "\n", " # calculate last increment based on the held-out final time point \n", " lastincr = last_currentcases - y[end]\n", "\n", " # calculate the prediction of the fitted models for this last increment\n", " incrmod_pred = β[1] + β[2] * y[end] + β[3] * z[end]\n", "\n", " # calculate the prediction of the mean and modified mean models\n", " mean_pred = mean(diff(y))\n", " modmean_pred = y[end] == 0.0 ? 0.0 : mean_pred\n", "\n", " # coarse heuristic: if the fit goes wrong, use the mean model prediction as fallback\n", " if isnan(incrmod_pred) || any(abs.(β) .> 10)\n", " incrmod_pred = mean_pred\n", " end\n", "\n", " # fit standard linear regression model using the last-observation-carried-forward imputation in the :currentcases\n", " slrdf = DataFrame(dy = diff(y), y = y[2:end], z = z[2:end])\n", " ols = lm(@formula(dy ~ y + z), slrdf) # standard estimation using maximum likelihood\n", " lrbeta = coef(ols) \n", " lrbetas[i] = lrbeta\n", " # calculate prediction of the standard linear regression model for the last increment \n", " linreg_pred = lrbeta[1] + lrbeta[2] * y[end] + lrbeta[3] * z[end]\n", "\n", " # include in the prediction error array only if the value at the last time point has actually been reported\n", " if lastreported \n", " prederrs_mean[i] = (lastincr - mean_pred)^2\n", " prederrs_modmean[i] = (lastincr - modmean_pred)^2\n", " prederrs_zero[i] = (lastincr - 0)^2\n", " prederrs_linreg[i] = (lastincr - linreg_pred)^2\n", " prederrs_incrmod[i] = (lastincr - incrmod_pred)^2\n", " end\n", "end" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "We can now combine the prediction errors for all models in a dataframe and calculate summary statistics. " ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "text/html": [ "

5 rows × 6 columns

ModelSSEFirstQuartileMedianThirdQuartileMaximum
StringFloat64Float64Float64Float64Float64
1meanmodel89.23880.00.00.0029.0
2modmeanmodel84.34210.00.00.09.0
3zeromodel86.00.00.00.09.0
4linregmodel181.0680.00.0010.01530.227
5incrmodel74.74570.00.00.0016.079
" ], "text/latex": [ "\\begin{tabular}{r|cccccc}\n", "\t& Model & SSE & FirstQuartile & Median & ThirdQuartile & Maximum\\\\\n", "\t\\hline\n", "\t& String & Float64 & Float64 & Float64 & Float64 & Float64\\\\\n", "\t\\hline\n", "\t1 & meanmodel & 89.2388 & 0.0 & 0.0 & 0.002 & 9.0 \\\\\n", "\t2 & modmeanmodel & 84.3421 & 0.0 & 0.0 & 0.0 & 9.0 \\\\\n", "\t3 & zeromodel & 86.0 & 0.0 & 0.0 & 0.0 & 9.0 \\\\\n", "\t4 & linregmodel & 181.068 & 0.0 & 0.001 & 0.015 & 30.227 \\\\\n", "\t5 & incrmodel & 74.7457 & 0.0 & 0.0 & 0.001 & 6.079 \\\\\n", "\\end{tabular}\n" ], "text/plain": [ "\u001b[1m5×6 DataFrame\u001b[0m\n", "\u001b[1m Row \u001b[0m│\u001b[1m Model \u001b[0m\u001b[1m SSE \u001b[0m\u001b[1m FirstQuartile \u001b[0m\u001b[1m Median \u001b[0m\u001b[1m ThirdQuartile \u001b[0m\u001b[1m Maximum \u001b[0m\n", "\u001b[1m \u001b[0m│\u001b[90m String \u001b[0m\u001b[90m Float64 \u001b[0m\u001b[90m Float64 \u001b[0m\u001b[90m Float64 \u001b[0m\u001b[90m Float64 \u001b[0m\u001b[90m Float64 \u001b[0m\n", "─────┼────────────────────────────────────────────────────────────────────────\n", " 1 │ meanmodel 89.2388 0.0 0.0 0.002 9.0\n", " 2 │ modmeanmodel 84.3421 0.0 0.0 0.0 9.0\n", " 3 │ zeromodel 86.0 0.0 0.0 0.0 9.0\n", " 4 │ linregmodel 181.068 0.0 0.001 0.015 30.227\n", " 5 │ incrmodel 74.7457 0.0 0.0 0.001 6.079" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "nonmissinginds = findall(x -> !ismissing(x), prederrs_mean)\n", "prederrs_df = DataFrame(hospitals = nonmissinginds,\n", " meanmodel = Float64.(prederrs_mean[nonmissinginds]),\n", " modmeanmodel = Float64.(prederrs_modmean[nonmissinginds]),\n", " zeromodel = Float64.(prederrs_zero[nonmissinginds]),\n", " linregmodel = Float64.(prederrs_linreg[nonmissinginds]),\n", " incrmodel = Float64.(prederrs_incrmod[nonmissinginds])\n", ")\n", "models = names(prederrs_df[!,2:end])\n", "summed_squared_errors = sum.(eachcol(prederrs_df[!,2:end]))\n", "medians = round.(median.(eachcol(prederrs_df[!,2:end])), digits=3)\n", "firstquartile = round.(quantile.(eachcol(prederrs_df[!,2:end]), 0.25), digits=3)\n", "thirdquartile = round.(quantile.(eachcol(prederrs_df[!,2:end]), 0.75), digits=3)\n", "maxvals = round.(maximum.(eachcol(prederrs_df[!,2:end])), digits=3)\n", "summary_df = DataFrame(Model = models, \n", " SSE = summed_squared_errors,\n", " FirstQuartile = firstquartile,\n", " Median = medians, \n", " ThirdQuartile = thirdquartile,\n", " Maximum = maxvals\n", ")" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "If you want to know more details, you can [read the paper](https://doi.org/10.1080/00031305.2021.2002189) ([arXiv version here](https://arxiv.org/abs/2012.05722)) or have a closer look at the [GitHub repository](https://github.com/maren-ha/DifferentiableProgrammingForStatisticalModeling) with the code + Jupyter notebooks to reproduce all analyses and images. \n", "\n", "![](auxiliary/TAS.png)" ] } ], "metadata": { "kernelspec": { "display_name": "Julia 1.9.1", "language": "julia", "name": "julia-1.9" }, "language_info": { "file_extension": ".jl", "mimetype": "application/julia", "name": "julia", "version": "1.9.1" }, "orig_nbformat": 4 }, "nbformat": 4, "nbformat_minor": 2 }