diff options
| author | SIPB | 2024-12-08 20:47:03 -0500 |
|---|---|---|
| committer | SIPB | 2024-12-08 20:47:03 -0500 |
| commit | ad1458c018cc8ce1264819629c884748b78ce907 (patch) | |
| tree | a5533445a195c51e38e2bb3b4fbd757fcc03a0ef | |
| parent | 6ae42b74e177f31b6dbcd06a1ae29be34deac8bb (diff) | |
Commit more stuff
| -rw-r--r-- | blog.md | 105 | ||||
| -rw-r--r-- | index.html | 100 | ||||
| -rw-r--r-- | plots/train-dist.html | 8 | ||||
| -rw-r--r-- | transformer_shortest_paths.ipynb | 2742 |
4 files changed, 2592 insertions, 363 deletions
@@ -4,6 +4,8 @@ mkzip: zip project.zip index.html img title: "Off-Distribution Generalization of Transformers for Shortest Paths" --- +<!-- Guidelines: https://www.dropbox.com/scl/fi/bet8enscln8ue36kd8t17/final_project_guidelines.pdf?rlkey=knd19cnumk51ho1y9crno56ib&e=2&dl=0 --> + <style> body { /* Shrink margins */ @@ -19,12 +21,6 @@ body { } </style> -<!-- -BTW we should run a spell checker on this at some point before submitting -note to self ---> -<!-- Guidelines: https://www.dropbox.com/scl/fi/bet8enscln8ue36kd8t17/final_project_guidelines.pdf?rlkey=knd19cnumk51ho1y9crno56ib&e=2&dl=0 --> - <div style="text-align:center"> Anthony Wang, Alek Westover, Kevin Zhao @@ -35,9 +31,10 @@ Anthony Wang, Alek Westover, Kevin Zhao ### Motivation -Neural networks are capable of impressive feats of off-distribution generalization. For instance, a program trained to convert sketches of cats into realistic looking pictures of cats can draw a cat with three eyes, if the sketch has three eyes, even if such an image never occured in its training data. However, neural networks also often learn non-robust features that cause it to perform very poorly off-distribution (e.g., adversarial examples for an image classifier). In this project, we will investigate the question of when transformers generalize off-distribution through via a case study on a simple synthetic task. More specifically, the goal of our project is to make progress towards answering the following question: +Neural networks are capable of impressive feats of off-distribution generalization. For instance, a program trained to convert sketches of cats into realistic looking pictures of cats can draw a cat with three eyes, if the sketch has three eyes, even if such an image never occurred in its training data. However, neural networks also often learn non-robust features that cause it to perform very poorly off-distribution (e.g., adversarial examples for an image classifier). In this project, we will investigate the question of when transformers generalize off-distribution through via a case study on a simple synthetic task. More specifically, the goal of our project is to make progress towards answering the following question: -> Suppose you train a model to be truthful on task A, and to produce coherent answers on a related task B.\ +> Suppose you train a model to be truthful on task A, and to produce coherent answers on a related task B. +> > Will the model learn to be truthful on task B? Here's a more specific version of this question, @@ -70,31 +67,11 @@ In order to investigate generalization of this type in transformers, we decided  </div> -<!--  --> - Obviously, if you actually want to compute shortest paths in a graph, you should use the standard graph algorithm BFS (breadth-first search). Our motivation for using this problem to study generalization was two-fold: -1. It's very simple to generate data and ground truth labels for this task. -2. There is a simple circuit which computes shortest paths (exactly), by doing something akin to BFS. In fact, we implemented this simple circuit by hand in the section [Handcrafted BFS Transformer](#handcrafted-bfs-transformer). Our hope was that our model can also find a simple circuit that generalized well. - - -<!-- -ANT: -RE t<7 -The model couuld've decided to generalize for t > 7 too, but it didn't -so id say -we tested generalization for all t in 3, 15 -it succeeded for t in 3,7 -failed for t in 8 to 15 -reason is because it could do "if statements" instead of realizing that it should just be looking at the dor product -mmm -i feel like the fact that it works for 3,7 and not 8,15 is kind of compelling -anyways this is just a comment explaining how im going to / think you should pitch the generalization section -(it couldve generalized all the way, but didnt, so what we learened is generalization sometimes happens but sometimes doesnt +1. It's very easy to generate data and ground truth labels for this task. -yeah that's the intuition -can't really do interpretability to figure out if that's actually what's going on ---> +2. There is a simple circuit which computes shortest paths (exactly), by doing something akin to BFS. In fact, we implemented this simple circuit by hand in the section [Handcrafted BFS Transformer](#handcrafted-bfs-transformer). Our hope was that our model can also find a simple circuit that generalized well. ### Findings @@ -118,7 +95,7 @@ The remainder of our post is structured as follows: 1. [Handcrafted BFS Transformer](#handcrafted-bfs-transformer): To start, we explicitly construct by hand weights for a transformer to solve the shortest paths problem. We also investigate the "robustness" of our construction by showing that, starting from our constructed solution it's possible to recover low loss after perturbing the weights via gradient descent. This gives a proof-of-concept that a transformer could encode a simple generalizable solution to the problem, and gives some reason to believe that a transformer could learn such a solution. -2. [Training](#training): Next, we discuss the training. First we describe the exact data generation process and model architecture. Then we discuss hyperparameter settings and optimization choices that were important to get the model to train efficiently. Finally, we investigate some interesting phenomena that occured during training, such as phase transitions in loss for different path lengths. +2. [Training](#training): Next, we discuss the training. First we describe the exact data generation process and model architecture. Then we discuss hyperparameter settings and optimization choices that were important to get the model to train efficiently. Finally, we investigate some interesting phenomena that occurred during training, such as phase transitions in loss for different path lengths. 3. [Generalization](#generalization): Finally we discuss the phenomena that we wanted to study -- generalization! We show that some generalization does occur, but other generalization doesn't. We explain why we think this happens. @@ -187,7 +164,7 @@ TODO: Switch to fancy mpld3 plots We'll represent an $n$ vertex, $m$ edge unweighted, undirected graph as sequence of the endpoints of the $m$ edges, so $[a_1,b_1,a_2,b_2,\ldots,a_m,b_m]$ represents a graph with the edges $\{(a_i,b_i)\}$ where $a_i < b_i$ for $1 \leq i \leq m$. All sequences are padded to the same length using the padding token $0$. -The full input to our model additionally includes the target vertex $t$ after the padding tokens. The label to an input is the length of the shortest path from $1$ to $t$. If no such path exists, we define the length to be $n+1$ which represents infinity. For example, the input $[1, 3, 3, 4, 2, 4, 2, 3, 0, 0, 0, 0, 3]$ has the label $2$. +The full input to our model additionally includes the target vertex $t$ after the padding tokens. The label to an input is the length of the shortest path from $1$ to $t$. If no such path exists, we define the length to be $n$ which represents infinity. For example, the input $[1, 3, 3, 4, 2, 4, 2, 3, 0, 0, 0, 0, 3]$ has the label $2$. <div style="text-align:center">  @@ -197,8 +174,6 @@ We have a dataset for each step. 1. **Pre-train data**: For each $n \in [3,15]$, we generated an equal number of graphs on $n$ vertices, with $t = 2$. Each graph was created by choosing $n$ random edges. To balance the dataset, we removed some graphs with shorter paths to obtain the distribution shown below. -TODO: Add axes titles, switch to bar plot - $include`raw="html"` plots/train-dist.html 2. **Fine-tune data**: For each $n \in [3,7]$, we generated an equal number of graphs on $n$ vertices each with a random $t$ on the shortest path from $1$ to $2$. Again, each graph was created by choosing $n$ random edges. @@ -209,7 +184,6 @@ We wrote some Python code to generate the data during the training loop, but Pyt ### Architecture - We used a standard transformer architecture. To ensure that it can in theory learn BFS, we ensured that the number of layers in our transformer exceeds the diameter of the input graphs. Since the order of the edges in the input doesn't matter, we did not use positional encodings. Each edge $(a,b)$ is embedded to dimension $d$ where the first $\frac{d}{2}$ elements are the learned embedding of $a$ and the last $\frac{d}{2}$ elements are the learned embedding of $b$. For the target vertex $t$, we pair it with the special token $TARGET$ and embed $(t,TARGET)$ in the same way. @@ -221,25 +195,25 @@ Since the order of the edges in the input doesn't matter, we did not use positio For our training run, we used the following specifications: -| Hyperparameter | Value | -| ------------------------ | --------------- | -| Model dimension | $64$ | -| Layers | $11$ | -| Heads | $2$ | -| Epochs | $100$ | -| Batch size | $2^{15}$ | -| Dropout | $0$ | -| Training examples | $10^8$ | -| Parameters | $550433$ | -| Learning rate | $5\cdot10^{-4}$ | -| Loss | MSE | -| Optimizer | Adam | +| Hyperparameter | Value | +| ----------------------- | --------------- | +| Model dimension | $64$ | +| Layers | $11$ | +| Heads | $2$ | +| Epochs | $200$ | +| Batch size | $2^{15}$ | +| Dropout | $0$ | +| Total training examples | $2\cdot10^8$ | +| Parameters | $550433$ | +| Learning rate | $5\cdot10^{-4}$ | +| Loss | MSE | +| Optimizer | Adam | The number of bits required to store the model parameters in float32 is around $1.76\cdot10^6$. The number of possible graphs on 15 vertices generated using our procedure is approximately $$\frac{\binom{15}{2}^{15}}{15!} \approx 1.59\cdot10^{18}.$$ This is because there are $\binom{15}{2}$ choices for each of the 15 edges and we don't care about the order of the edges. This is only an approximation because some edges might be duplicated. Each graph has an answer between 1 and 15 which requires around 4 bits, so memorizing all the answers requires requires $4\cdot1.59\cdot10^{18} = 6.36\cdot10^{18}$ bits, which is $3.61\cdot10^{12}$ times larger than our model size. This implies that in order to get really low loss, our model needs to do something other than brute memorization. -A single training run takes roughly eight hours to run on a Radeon 7900 XTX graphics card. +A single training run takes roughly two hours to run on a Radeon 7900 XTX graphics card. One pitfall we encountered during training is that we initially used bfloat16 to save VRAM, but our huge batch size caused loss-of-precision problems and made training very difficult. It took us two weeks to debug this until we found that switching to float32 improved training significantly. @@ -249,19 +223,19 @@ Here is our training loss curve: $include`raw="html"` plots/train-loss.html -Here is a "confusion matrix"-like plot, showing for each true label, the probability distribution over classifications that the model gives. +Here is a "confusion matrix"-like plot, showing for each true label, the probability distribution over classifications that the model gives: $include`raw="html"` plots/train-hist.html -One pattern we consistently noticed during training is that the model often gets stuck and plateaus for many epochs before rapidly decreasing. For instance, this happened between epochs 100 and 300 in the graph above: +One pattern we consistently noticed during training is that the model often gets stuck and plateaus for many epochs before rapidly decreasing. For instance, this happened between epochs 100 and 300 in the graph below: -$include`raw="html"` plots/grokking.html +$include`raw="html"` plots/train-lens.html Such **phase change** phenomena are ubiquitous in deep learning and for simple algorithmic tasks we can sometimes interpret them [^2]. **TODO ANT: training curves for 1, 2, 3 length paths. and add some text about why this is an interesting finding.** -$include`raw="html"` plots/len-loss.html + ## Generalization @@ -273,14 +247,33 @@ $include`raw="html"` plots/tune-loss.html $include`raw="html"` plots/tune-hist.html -$include`raw="html"` plots/test-onpath-largetarget.html - $include`raw="html"` plots/test-onpath-smalltarget.html +$include`raw="html"` plots/test-onpath-largetarget.html + $include`raw="html"` plots/test-small-any.html -$include`raw="html"` plots/test-large-any.html +$include`raw="html"` plots/test-large-any-smalltarget.html + +$include`raw="html"` plots/test-large-any-largetarget.html + +<!-- +ANT: +RE t<7 +The model couuld've decided to generalize for t > 7 too, but it didn't +so id say +we tested generalization for all t in 3, 15 +it succeeded for t in 3,7 +failed for t in 8 to 15 +reason is because it could do "if statements" instead of realizing that it should just be looking at the dor product +mmm +i feel like the fact that it works for 3,7 and not 8,15 is kind of compelling +anyways this is just a comment explaining how im going to / think you should pitch the generalization section +(it couldve generalized all the way, but didnt, so what we learened is generalization sometimes happens but sometimes doesnt +yeah that's the intuition +can't really do interpretability to figure out if that's actually what's going on +--> ## Conclusion @@ -254,6 +254,7 @@ <h1 class="title">Off-Distribution Generalization of Transformers for Shortest Paths</h1> </header> +<!-- Guidelines: https://www.dropbox.com/scl/fi/bet8enscln8ue36kd8t17/final_project_guidelines.pdf?rlkey=knd19cnumk51ho1y9crno56ib&e=2&dl=0 --> <style> body { /* Shrink margins */ @@ -268,11 +269,6 @@ body { margin: auto; } </style> -<!-- -BTW we should run a spell checker on this at some point before submitting -note to self ---> -<!-- Guidelines: https://www.dropbox.com/scl/fi/bet8enscln8ue36kd8t17/final_project_guidelines.pdf?rlkey=knd19cnumk51ho1y9crno56ib&e=2&dl=0 --> <div style="text-align:center"> <p>Anthony Wang, Alek Westover, Kevin Zhao</p> <p>{xy,alekw,kevinmz}@mit.edu</p> @@ -282,7 +278,7 @@ note to self <p>Neural networks are capable of impressive feats of off-distribution generalization. For instance, a program trained to convert sketches of cats into realistic looking pictures of cats can draw a cat with three -eyes, if the sketch has three eyes, even if such an image never occured +eyes, if the sketch has three eyes, even if such an image never occurred in its training data. However, neural networks also often learn non-robust features that cause it to perform very poorly off-distribution (e.g., adversarial examples for an image classifier). @@ -292,8 +288,8 @@ synthetic task. More specifically, the goal of our project is to make progress towards answering the following question:</p> <blockquote> <p>Suppose you train a model to be truthful on task A, and to produce -coherent answers on a related task B.<br /> -Will the model learn to be truthful on task B?</p> +coherent answers on a related task B.</p> +<p>Will the model learn to be truthful on task B?</p> </blockquote> <p>Here’s a more specific version of this question, inspired by a <a href="https://www.alignmentforum.org/posts/BxersHYN2qcFoonwg/experimentally-evaluating-whether-honesty-generalizes?commentId=dsDA2BWpHPdgLvaXX">blog @@ -360,38 +356,20 @@ class="math inline">2</span> for graphs with <span class="math inline">n <div style="text-align:center"> <p><img src="img/test.svg" /></p> </div> -<!--  --> <p>Obviously, if you actually want to compute shortest paths in a graph, you should use the standard graph algorithm BFS (breadth-first search). Our motivation for using this problem to study generalization was two-fold:</p> <ol type="1"> -<li>It’s very simple to generate data and ground truth labels for this -task.</li> -<li>There is a simple circuit which computes shortest paths (exactly), -by doing something akin to BFS. In fact, we implemented this simple -circuit by hand in the section <a +<li><p>It’s very easy to generate data and ground truth labels for this +task.</p></li> +<li><p>There is a simple circuit which computes shortest paths +(exactly), by doing something akin to BFS. In fact, we implemented this +simple circuit by hand in the section <a href="#handcrafted-bfs-transformer">Handcrafted BFS Transformer</a>. Our hope was that our model can also find a simple circuit that generalized -well.</li> +well.</p></li> </ol> -<!-- -ANT: -RE t<7 -The model couuld've decided to generalize for t > 7 too, but it didn't -so id say -we tested generalization for all t in 3, 15 -it succeeded for t in 3,7 -failed for t in 8 to 15 -reason is because it could do "if statements" instead of realizing that it should just be looking at the dor product -mmm -i feel like the fact that it works for 3,7 and not 8,15 is kind of compelling -anyways this is just a comment explaining how im going to / think you should pitch the generalization section -(it couldve generalized all the way, but didnt, so what we learened is generalization sometimes happens but sometimes doesnt - -yeah that's the intuition -can't really do interpretability to figure out if that's actually what's going on ---> <h3 id="findings">Findings</h3> <p>Our main findings were as follows:</p> <ol type="1"> @@ -449,7 +427,7 @@ solution.</p></li> First we describe the exact data generation process and model architecture. Then we discuss hyperparameter settings and optimization choices that were important to get the model to train efficiently. -Finally, we investigate some interesting phenomena that occured during +Finally, we investigate some interesting phenomena that occurred during training, such as phase transitions in loss for different path lengths.</p></li> <li><p><a href="#generalization">Generalization</a>: Finally we discuss @@ -636,8 +614,8 @@ class="math inline">0</span>.</p> to an input is the length of the shortest path from <span class="math inline">1</span> to <span class="math inline">t</span>. If no such path exists, we define the length to be <span -class="math inline">n+1</span> which represents infinity. For example, -the input <span class="math inline">[1, 3, 3, 4, 2, 4, 2, 3, 0, 0, 0, 0, +class="math inline">n</span> which represents infinity. For example, the +input <span class="math inline">[1, 3, 3, 4, 2, 4, 2, 3, 0, 0, 0, 0, 3]</span> has the label <span class="math inline">2</span>.</p> <div style="text-align:center"> <p><img src="img/finetune.svg" /></p> @@ -652,14 +630,13 @@ class="math inline">t = 2</span>. Each graph was created by choosing we removed some graphs with shorter paths to obtain the distribution shown below.</li> </ol> -<p>TODO: Add axes titles, switch to bar plot</p> <style> </style> -<div id="fig_el1200021399424053838726786165297"></div> +<div id="fig_el3734471397923808578566893788369"></div> <script> function mpld3_load_lib(url, callback){ var s = document.createElement('script'); @@ -674,7 +651,7 @@ if(typeof(mpld3) !== "undefined" && mpld3._mpld3IsLoaded){ // already loaded: just create the figure !function(mpld3){ - mpld3.draw_figure("fig_el1200021399424053838726786165297", {"width": 640.0, "height": 480.0, "axes": [{"bbox": [0.125, 0.10999999999999999, 0.775, 0.77], "xlim": [0.29999999999999993, 15.7], "ylim": [0.0, 7533.75], "xdomain": [0.29999999999999993, 15.7], "ydomain": [0.0, 7533.75], "xscale": "linear", "yscale": "linear", "axes": [{"position": "bottom", "nticks": 9, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}, {"position": "left", "nticks": 9, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}], "axesbg": "#FFFFFF", "axesbgalpha": null, "zoomable": true, "id": "el120002139942361034016", "lines": [], "paths": [{"data": "data01", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361126032", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data02", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361031376", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data03", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942360791184", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data04", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361121472", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data05", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361120896", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data06", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361123824", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data07", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361123392", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data08", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361127904", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data09", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361129296", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data10", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361129056", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data11", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361128624", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data12", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361127376", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data12", "xindex": 2, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361126752", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data12", "xindex": 3, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361126320", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data13", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361129872", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}], "markers": [], "texts": [], "collections": [], "images": [], "sharex": [], "sharey": []}], "data": {"data01": [[1.0, 0.0], [1.933333396911621, 0.0], [1.933333396911621, 7175.0], [1.0, 7175.0]], "data02": [[1.933333396911621, 0.0], [2.866666793823242, 0.0], [2.866666793823242, 6576.0], [1.933333396911621, 6576.0]], "data03": [[2.866666793823242, 0.0], [3.799999952316284, 0.0], [3.799999952316284, 4624.0], [2.866666793823242, 4624.0]], "data04": [[3.799999952316284, 0.0], [4.733333587646484, 0.0], [4.733333587646484, 3021.0], [3.799999952316284, 3021.0]], "data05": [[4.733333587646484, 0.0], [5.666666507720947, 0.0], [5.666666507720947, 2275.0], [4.733333587646484, 2275.0]], "data06": [[5.666666507720947, 0.0], [6.599999904632568, 0.0], [6.599999904632568, 1653.0], [5.666666507720947, 1653.0]], "data07": [[6.599999904632568, 0.0], [7.5333333015441895, 0.0], [7.5333333015441895, 492.0], [6.599999904632568, 492.0]], "data08": [[7.5333333015441895, 0.0], [8.466667175292969, 0.0], [8.466667175292969, 119.0], [7.5333333015441895, 119.0]], "data09": [[8.466667175292969, 0.0], [9.399999618530273, 0.0], [9.399999618530273, 28.0], [8.466667175292969, 28.0]], "data10": [[9.399999618530273, 0.0], [10.333333015441895, 0.0], [10.333333015441895, 5.0], [9.399999618530273, 5.0]], "data11": [[10.333333015441895, 0.0], [11.266666412353516, 0.0], [11.266666412353516, 1.0], [10.333333015441895, 1.0]], "data12": [[11.266666412353516, 0.0, 12.199999809265137, 13.133333206176758], [12.199999809265137, 0.0, 13.133333206176758, 14.066666603088379], [12.199999809265137, 0.0, 13.133333206176758, 14.066666603088379], [11.266666412353516, 0.0, 12.199999809265137, 13.133333206176758]], "data13": [[14.066666603088379, 0.0], [15.0, 0.0], [15.0, 6799.0], [14.066666603088379, 6799.0]]}, "id": "el120002139942405383872", "plugins": [{"type": "reset"}, {"type": "zoom", "button": true, "enabled": false}, {"type": "boxzoom", "button": true, "enabled": false}]}); + mpld3.draw_figure("fig_el3734471397923808578566893788369", {"width": 640.0, "height": 480.0, "axes": [{"bbox": [0.125, 0.10999999999999999, 0.775, 0.77], "xlim": [-1.19, 16.19], "ylim": [0.0, 7642.95], "xdomain": [-1.19, 16.19], "ydomain": [0.0, 7642.95], "xscale": "linear", "yscale": "linear", "axes": [{"position": "bottom", "nticks": 11, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}, {"position": "left", "nticks": 9, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}], "axesbg": "#FFFFFF", "axesbgalpha": null, "zoomable": true, "id": "el373447139797053788464", "lines": [], "paths": [{"data": "data01", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381535360", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data02", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792380856416", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data03", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139797054022832", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data04", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381530944", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data05", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381539632", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data06", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139797053780624", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data07", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381532192", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data08", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381536896", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data09", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381539872", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data10", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381532480", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data11", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139797053797824", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data01", "xindex": 2, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381533200", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data01", "xindex": 3, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381532240", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data01", "xindex": 4, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139797054021056", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data01", "xindex": 5, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381532576", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data12", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381527536", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}], "markers": [], "texts": [{"text": "Label", "position": [0.5, -0.07921476671476674], "coordinates": "axes", "h_anchor": "middle", "v_baseline": "hanging", "rotation": -0.0, "fontsize": 10.0, "color": "#000000", "alpha": 1, "zorder": 3, "id": "el373447139792380852176"}, {"text": "Count", "position": [-0.10161850358422941, 0.5], "coordinates": "axes", "h_anchor": "middle", "v_baseline": "auto", "rotation": -90.0, "fontsize": 10.0, "color": "#000000", "alpha": 1, "zorder": 3, "id": "el373447139792380853616"}], "collections": [], "images": [], "sharex": [], "sharey": []}], "data": {"data01": [[-0.4, 0.0, 10.6, 11.6, 12.6, 13.6], [0.4, 0.0, 11.4, 12.4, 13.4, 14.4], [0.4, 0.0, 11.4, 12.4, 13.4, 14.4], [-0.4, 0.0, 10.6, 11.6, 12.6, 13.6]], "data02": [[0.6, 0.0], [1.4, 0.0], [1.4, 7279.0], [0.6, 7279.0]], "data03": [[1.6, 0.0], [2.4000000000000004, 0.0], [2.4000000000000004, 6614.0], [1.6, 6614.0]], "data04": [[2.6, 0.0], [3.4000000000000004, 0.0], [3.4000000000000004, 4434.0], [2.6, 4434.0]], "data05": [[3.6, 0.0], [4.4, 0.0], [4.4, 2907.0], [3.6, 2907.0]], "data06": [[4.6, 0.0], [5.3999999999999995, 0.0], [5.3999999999999995, 2324.0], [4.6, 2324.0]], "data07": [[5.6, 0.0], [6.3999999999999995, 0.0], [6.3999999999999995, 1578.0], [5.6, 1578.0]], "data08": [[6.6, 0.0], [7.3999999999999995, 0.0], [7.3999999999999995, 469.0], [6.6, 469.0]], "data09": [[7.6, 0.0], [8.4, 0.0], [8.4, 133.0], [7.6, 133.0]], "data10": [[8.6, 0.0], [9.4, 0.0], [9.4, 27.0], [8.6, 27.0]], "data11": [[9.6, 0.0], [10.4, 0.0], [10.4, 4.0], [9.6, 4.0]], "data12": [[14.6, 0.0], [15.4, 0.0], [15.4, 6999.0], [14.6, 6999.0]]}, "id": "el373447139792380857856", "plugins": [{"type": "reset"}, {"type": "zoom", "button": true, "enabled": false}, {"type": "boxzoom", "button": true, "enabled": false}]}); }(mpld3); }else if(typeof define === "function" && define.amd){ // require.js is available: use it to load d3/mpld3 @@ -683,7 +660,7 @@ if(typeof(mpld3) !== "undefined" && mpld3._mpld3IsLoaded){ window.d3 = d3; mpld3_load_lib("https://mpld3.github.io/js/mpld3.v0.5.10.js", function(){ - mpld3.draw_figure("fig_el1200021399424053838726786165297", {"width": 640.0, "height": 480.0, "axes": [{"bbox": [0.125, 0.10999999999999999, 0.775, 0.77], "xlim": [0.29999999999999993, 15.7], "ylim": [0.0, 7533.75], "xdomain": [0.29999999999999993, 15.7], "ydomain": [0.0, 7533.75], "xscale": "linear", "yscale": "linear", "axes": [{"position": "bottom", "nticks": 9, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}, {"position": "left", "nticks": 9, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}], "axesbg": "#FFFFFF", "axesbgalpha": null, "zoomable": true, "id": "el120002139942361034016", "lines": [], "paths": [{"data": "data01", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361126032", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data02", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361031376", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data03", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942360791184", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data04", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361121472", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data05", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361120896", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data06", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361123824", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data07", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361123392", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data08", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361127904", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data09", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361129296", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data10", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361129056", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data11", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361128624", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data12", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361127376", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data12", "xindex": 2, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361126752", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data12", "xindex": 3, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361126320", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data13", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361129872", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}], "markers": [], "texts": [], "collections": [], "images": [], "sharex": [], "sharey": []}], "data": {"data01": [[1.0, 0.0], [1.933333396911621, 0.0], [1.933333396911621, 7175.0], [1.0, 7175.0]], "data02": [[1.933333396911621, 0.0], [2.866666793823242, 0.0], [2.866666793823242, 6576.0], [1.933333396911621, 6576.0]], "data03": [[2.866666793823242, 0.0], [3.799999952316284, 0.0], [3.799999952316284, 4624.0], [2.866666793823242, 4624.0]], "data04": [[3.799999952316284, 0.0], [4.733333587646484, 0.0], [4.733333587646484, 3021.0], [3.799999952316284, 3021.0]], "data05": [[4.733333587646484, 0.0], [5.666666507720947, 0.0], [5.666666507720947, 2275.0], [4.733333587646484, 2275.0]], "data06": [[5.666666507720947, 0.0], [6.599999904632568, 0.0], [6.599999904632568, 1653.0], [5.666666507720947, 1653.0]], "data07": [[6.599999904632568, 0.0], [7.5333333015441895, 0.0], [7.5333333015441895, 492.0], [6.599999904632568, 492.0]], "data08": [[7.5333333015441895, 0.0], [8.466667175292969, 0.0], [8.466667175292969, 119.0], [7.5333333015441895, 119.0]], "data09": [[8.466667175292969, 0.0], [9.399999618530273, 0.0], [9.399999618530273, 28.0], [8.466667175292969, 28.0]], "data10": [[9.399999618530273, 0.0], [10.333333015441895, 0.0], [10.333333015441895, 5.0], [9.399999618530273, 5.0]], "data11": [[10.333333015441895, 0.0], [11.266666412353516, 0.0], [11.266666412353516, 1.0], [10.333333015441895, 1.0]], "data12": [[11.266666412353516, 0.0, 12.199999809265137, 13.133333206176758], [12.199999809265137, 0.0, 13.133333206176758, 14.066666603088379], [12.199999809265137, 0.0, 13.133333206176758, 14.066666603088379], [11.266666412353516, 0.0, 12.199999809265137, 13.133333206176758]], "data13": [[14.066666603088379, 0.0], [15.0, 0.0], [15.0, 6799.0], [14.066666603088379, 6799.0]]}, "id": "el120002139942405383872", "plugins": [{"type": "reset"}, {"type": "zoom", "button": true, "enabled": false}, {"type": "boxzoom", "button": true, "enabled": false}]}); + mpld3.draw_figure("fig_el3734471397923808578566893788369", {"width": 640.0, "height": 480.0, "axes": [{"bbox": [0.125, 0.10999999999999999, 0.775, 0.77], "xlim": [-1.19, 16.19], "ylim": [0.0, 7642.95], "xdomain": [-1.19, 16.19], "ydomain": [0.0, 7642.95], "xscale": "linear", "yscale": "linear", "axes": [{"position": "bottom", "nticks": 11, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}, {"position": "left", "nticks": 9, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}], "axesbg": "#FFFFFF", "axesbgalpha": null, "zoomable": true, "id": "el373447139797053788464", "lines": [], "paths": [{"data": "data01", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381535360", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data02", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792380856416", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data03", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139797054022832", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data04", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381530944", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data05", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381539632", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data06", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139797053780624", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data07", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381532192", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data08", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381536896", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data09", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381539872", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data10", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381532480", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data11", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139797053797824", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data01", "xindex": 2, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381533200", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data01", "xindex": 3, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381532240", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data01", "xindex": 4, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139797054021056", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data01", "xindex": 5, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381532576", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data12", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381527536", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}], "markers": [], "texts": [{"text": "Label", "position": [0.5, -0.07921476671476674], "coordinates": "axes", "h_anchor": "middle", "v_baseline": "hanging", "rotation": -0.0, "fontsize": 10.0, "color": "#000000", "alpha": 1, "zorder": 3, "id": "el373447139792380852176"}, {"text": "Count", "position": [-0.10161850358422941, 0.5], "coordinates": "axes", "h_anchor": "middle", "v_baseline": "auto", "rotation": -90.0, "fontsize": 10.0, "color": "#000000", "alpha": 1, "zorder": 3, "id": "el373447139792380853616"}], "collections": [], "images": [], "sharex": [], "sharey": []}], "data": {"data01": [[-0.4, 0.0, 10.6, 11.6, 12.6, 13.6], [0.4, 0.0, 11.4, 12.4, 13.4, 14.4], [0.4, 0.0, 11.4, 12.4, 13.4, 14.4], [-0.4, 0.0, 10.6, 11.6, 12.6, 13.6]], "data02": [[0.6, 0.0], [1.4, 0.0], [1.4, 7279.0], [0.6, 7279.0]], "data03": [[1.6, 0.0], [2.4000000000000004, 0.0], [2.4000000000000004, 6614.0], [1.6, 6614.0]], "data04": [[2.6, 0.0], [3.4000000000000004, 0.0], [3.4000000000000004, 4434.0], [2.6, 4434.0]], "data05": [[3.6, 0.0], [4.4, 0.0], [4.4, 2907.0], [3.6, 2907.0]], "data06": [[4.6, 0.0], [5.3999999999999995, 0.0], [5.3999999999999995, 2324.0], [4.6, 2324.0]], "data07": [[5.6, 0.0], [6.3999999999999995, 0.0], [6.3999999999999995, 1578.0], [5.6, 1578.0]], "data08": [[6.6, 0.0], [7.3999999999999995, 0.0], [7.3999999999999995, 469.0], [6.6, 469.0]], "data09": [[7.6, 0.0], [8.4, 0.0], [8.4, 133.0], [7.6, 133.0]], "data10": [[8.6, 0.0], [9.4, 0.0], [9.4, 27.0], [8.6, 27.0]], "data11": [[9.6, 0.0], [10.4, 0.0], [10.4, 4.0], [9.6, 4.0]], "data12": [[14.6, 0.0], [15.4, 0.0], [15.4, 6999.0], [14.6, 6999.0]]}, "id": "el373447139792380857856", "plugins": [{"type": "reset"}, {"type": "zoom", "button": true, "enabled": false}, {"type": "boxzoom", "button": true, "enabled": false}]}); }); }); }else{ @@ -691,7 +668,7 @@ if(typeof(mpld3) !== "undefined" && mpld3._mpld3IsLoaded){ mpld3_load_lib("https://d3js.org/d3.v5.js", function(){ mpld3_load_lib("https://mpld3.github.io/js/mpld3.v0.5.10.js", function(){ - mpld3.draw_figure("fig_el1200021399424053838726786165297", {"width": 640.0, "height": 480.0, "axes": [{"bbox": [0.125, 0.10999999999999999, 0.775, 0.77], "xlim": [0.29999999999999993, 15.7], "ylim": [0.0, 7533.75], "xdomain": [0.29999999999999993, 15.7], "ydomain": [0.0, 7533.75], "xscale": "linear", "yscale": "linear", "axes": [{"position": "bottom", "nticks": 9, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}, {"position": "left", "nticks": 9, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}], "axesbg": "#FFFFFF", "axesbgalpha": null, "zoomable": true, "id": "el120002139942361034016", "lines": [], "paths": [{"data": "data01", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361126032", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data02", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361031376", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data03", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942360791184", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data04", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361121472", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data05", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361120896", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data06", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361123824", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data07", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361123392", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data08", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361127904", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data09", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361129296", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data10", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361129056", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data11", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361128624", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data12", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361127376", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data12", "xindex": 2, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361126752", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data12", "xindex": 3, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361126320", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data13", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361129872", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}], "markers": [], "texts": [], "collections": [], "images": [], "sharex": [], "sharey": []}], "data": {"data01": [[1.0, 0.0], [1.933333396911621, 0.0], [1.933333396911621, 7175.0], [1.0, 7175.0]], "data02": [[1.933333396911621, 0.0], [2.866666793823242, 0.0], [2.866666793823242, 6576.0], [1.933333396911621, 6576.0]], "data03": [[2.866666793823242, 0.0], [3.799999952316284, 0.0], [3.799999952316284, 4624.0], [2.866666793823242, 4624.0]], "data04": [[3.799999952316284, 0.0], [4.733333587646484, 0.0], [4.733333587646484, 3021.0], [3.799999952316284, 3021.0]], "data05": [[4.733333587646484, 0.0], [5.666666507720947, 0.0], [5.666666507720947, 2275.0], [4.733333587646484, 2275.0]], "data06": [[5.666666507720947, 0.0], [6.599999904632568, 0.0], [6.599999904632568, 1653.0], [5.666666507720947, 1653.0]], "data07": [[6.599999904632568, 0.0], [7.5333333015441895, 0.0], [7.5333333015441895, 492.0], [6.599999904632568, 492.0]], "data08": [[7.5333333015441895, 0.0], [8.466667175292969, 0.0], [8.466667175292969, 119.0], [7.5333333015441895, 119.0]], "data09": [[8.466667175292969, 0.0], [9.399999618530273, 0.0], [9.399999618530273, 28.0], [8.466667175292969, 28.0]], "data10": [[9.399999618530273, 0.0], [10.333333015441895, 0.0], [10.333333015441895, 5.0], [9.399999618530273, 5.0]], "data11": [[10.333333015441895, 0.0], [11.266666412353516, 0.0], [11.266666412353516, 1.0], [10.333333015441895, 1.0]], "data12": [[11.266666412353516, 0.0, 12.199999809265137, 13.133333206176758], [12.199999809265137, 0.0, 13.133333206176758, 14.066666603088379], [12.199999809265137, 0.0, 13.133333206176758, 14.066666603088379], [11.266666412353516, 0.0, 12.199999809265137, 13.133333206176758]], "data13": [[14.066666603088379, 0.0], [15.0, 0.0], [15.0, 6799.0], [14.066666603088379, 6799.0]]}, "id": "el120002139942405383872", "plugins": [{"type": "reset"}, {"type": "zoom", "button": true, "enabled": false}, {"type": "boxzoom", "button": true, "enabled": false}]}); + mpld3.draw_figure("fig_el3734471397923808578566893788369", {"width": 640.0, "height": 480.0, "axes": [{"bbox": [0.125, 0.10999999999999999, 0.775, 0.77], "xlim": [-1.19, 16.19], "ylim": [0.0, 7642.95], "xdomain": [-1.19, 16.19], "ydomain": [0.0, 7642.95], "xscale": "linear", "yscale": "linear", "axes": [{"position": "bottom", "nticks": 11, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}, {"position": "left", "nticks": 9, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}], "axesbg": "#FFFFFF", "axesbgalpha": null, "zoomable": true, "id": "el373447139797053788464", "lines": [], "paths": [{"data": "data01", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381535360", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data02", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792380856416", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data03", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139797054022832", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data04", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381530944", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data05", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381539632", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data06", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139797053780624", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data07", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381532192", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data08", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381536896", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data09", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381539872", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data10", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381532480", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data11", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139797053797824", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data01", "xindex": 2, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381533200", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data01", "xindex": 3, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381532240", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data01", "xindex": 4, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139797054021056", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data01", "xindex": 5, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381532576", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data12", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381527536", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}], "markers": [], "texts": [{"text": "Label", "position": [0.5, -0.07921476671476674], "coordinates": "axes", "h_anchor": "middle", "v_baseline": "hanging", "rotation": -0.0, "fontsize": 10.0, "color": "#000000", "alpha": 1, "zorder": 3, "id": "el373447139792380852176"}, {"text": "Count", "position": [-0.10161850358422941, 0.5], "coordinates": "axes", "h_anchor": "middle", "v_baseline": "auto", "rotation": -90.0, "fontsize": 10.0, "color": "#000000", "alpha": 1, "zorder": 3, "id": "el373447139792380853616"}], "collections": [], "images": [], "sharex": [], "sharey": []}], "data": {"data01": [[-0.4, 0.0, 10.6, 11.6, 12.6, 13.6], [0.4, 0.0, 11.4, 12.4, 13.4, 14.4], [0.4, 0.0, 11.4, 12.4, 13.4, 14.4], [-0.4, 0.0, 10.6, 11.6, 12.6, 13.6]], "data02": [[0.6, 0.0], [1.4, 0.0], [1.4, 7279.0], [0.6, 7279.0]], "data03": [[1.6, 0.0], [2.4000000000000004, 0.0], [2.4000000000000004, 6614.0], [1.6, 6614.0]], "data04": [[2.6, 0.0], [3.4000000000000004, 0.0], [3.4000000000000004, 4434.0], [2.6, 4434.0]], "data05": [[3.6, 0.0], [4.4, 0.0], [4.4, 2907.0], [3.6, 2907.0]], "data06": [[4.6, 0.0], [5.3999999999999995, 0.0], [5.3999999999999995, 2324.0], [4.6, 2324.0]], "data07": [[5.6, 0.0], [6.3999999999999995, 0.0], [6.3999999999999995, 1578.0], [5.6, 1578.0]], "data08": [[6.6, 0.0], [7.3999999999999995, 0.0], [7.3999999999999995, 469.0], [6.6, 469.0]], "data09": [[7.6, 0.0], [8.4, 0.0], [8.4, 133.0], [7.6, 133.0]], "data10": [[8.6, 0.0], [9.4, 0.0], [9.4, 27.0], [8.6, 27.0]], "data11": [[9.6, 0.0], [10.4, 0.0], [10.4, 4.0], [9.6, 4.0]], "data12": [[14.6, 0.0], [15.4, 0.0], [15.4, 6999.0], [14.6, 6999.0]]}, "id": "el373447139792380857856", "plugins": [{"type": "reset"}, {"type": "zoom", "button": true, "enabled": false}, {"type": "boxzoom", "button": true, "enabled": false}]}); }) }); } @@ -753,7 +730,7 @@ class="math inline">(t,TARGET)</span> in the same way.</p> </tr> <tr class="even"> <td>Epochs</td> -<td><span class="math inline">100</span></td> +<td><span class="math inline">200</span></td> </tr> <tr class="odd"> <td>Batch size</td> @@ -764,8 +741,8 @@ class="math inline">(t,TARGET)</span> in the same way.</p> <td><span class="math inline">0</span></td> </tr> <tr class="odd"> -<td>Training examples</td> -<td><span class="math inline">10^8</span></td> +<td>Total training examples</td> +<td><span class="math inline">2\cdot10^8</span></td> </tr> <tr class="even"> <td>Parameters</td> @@ -799,8 +776,8 @@ class="math inline">4\cdot1.59\cdot10^{18} = 6.36\cdot10^{18}</span> bits, which is <span class="math inline">3.61\cdot10^{12}</span> times larger than our model size. This implies that in order to get really low loss, our model needs to do something other than brute memorization.</p> -<p>A single training run takes roughly eight hours to run on a Radeon -7900 XTX graphics card.</p> +<p>A single training run takes roughly two hours to run on a Radeon 7900 +XTX graphics card.</p> <p>One pitfall we encountered during training is that we initially used bfloat16 to save VRAM, but our huge batch size caused loss-of-precision problems and made training very difficult. It took us two weeks to debug @@ -811,20 +788,19 @@ significantly.</p> <p>$include<code>raw="html"</code> plots/train-loss.html</p> <p>Here is a “confusion matrix”-like plot, showing for each true label, the probability distribution over classifications that the model -gives.</p> +gives:</p> <p>$include<code>raw="html"</code> plots/train-hist.html</p> <p>One pattern we consistently noticed during training is that the model often gets stuck and plateaus for many epochs before rapidly decreasing. For instance, this happened between epochs 100 and 300 in the graph -above:</p> -<p>$include<code>raw="html"</code> plots/grokking.html</p> +below:</p> +<p>$include<code>raw="html"</code> plots/train-lens.html</p> <p>Such <strong>phase change</strong> phenomena are ubiquitous in deep learning and for simple algorithmic tasks we can sometimes interpret them <a href="#fn2" class="footnote-ref" id="fnref2" role="doc-noteref"><sup>2</sup></a>.</p> <p><strong>TODO ANT: training curves for 1, 2, 3 length paths. and add some text about why this is an interesting finding.</strong></p> -<p>$include<code>raw="html"</code> plots/len-loss.html</p> <h2 id="generalization">Generalization</h2> <h3 id="fine-tuning-results">Fine Tuning Results</h3> <p>After receiving our initial results, we fine-tuned with a learning @@ -833,11 +809,31 @@ are shown in the images below.</p> <p>$include<code>raw="html"</code> plots/tune-loss.html</p> <p>$include<code>raw="html"</code> plots/tune-hist.html</p> <p>$include<code>raw="html"</code> -plots/test-onpath-largetarget.html</p> -<p>$include<code>raw="html"</code> plots/test-onpath-smalltarget.html</p> +<p>$include<code>raw="html"</code> +plots/test-onpath-largetarget.html</p> <p>$include<code>raw="html"</code> plots/test-small-any.html</p> -<p>$include<code>raw="html"</code> plots/test-large-any.html</p> +<p>$include<code>raw="html"</code> +plots/test-large-any-smalltarget.html</p> +<p>$include<code>raw="html"</code> +plots/test-large-any-largetarget.html</p> +<!-- +ANT: +RE t<7 +The model couuld've decided to generalize for t > 7 too, but it didn't +so id say +we tested generalization for all t in 3, 15 +it succeeded for t in 3,7 +failed for t in 8 to 15 +reason is because it could do "if statements" instead of realizing that it should just be looking at the dor product +mmm +i feel like the fact that it works for 3,7 and not 8,15 is kind of compelling +anyways this is just a comment explaining how im going to / think you should pitch the generalization section +(it couldve generalized all the way, but didnt, so what we learened is generalization sometimes happens but sometimes doesnt + +yeah that's the intuition +can't really do interpretability to figure out if that's actually what's going on +--> <h2 id="conclusion">Conclusion</h2> <p>In this post we’ve investigated solving the shortest paths problem with a transformer. We’ve shown that this problem has a simple solution diff --git a/plots/train-dist.html b/plots/train-dist.html index 54aee21..8f2a132 100644 --- a/plots/train-dist.html +++ b/plots/train-dist.html @@ -4,7 +4,7 @@ </style> -<div id="fig_el1200021399424053838726786165297"></div> +<div id="fig_el3734471397923808578566893788369"></div> <script> function mpld3_load_lib(url, callback){ var s = document.createElement('script'); @@ -19,7 +19,7 @@ if(typeof(mpld3) !== "undefined" && mpld3._mpld3IsLoaded){ // already loaded: just create the figure !function(mpld3){ - mpld3.draw_figure("fig_el1200021399424053838726786165297", {"width": 640.0, "height": 480.0, "axes": [{"bbox": [0.125, 0.10999999999999999, 0.775, 0.77], "xlim": [0.29999999999999993, 15.7], "ylim": [0.0, 7533.75], "xdomain": [0.29999999999999993, 15.7], "ydomain": [0.0, 7533.75], "xscale": "linear", "yscale": "linear", "axes": [{"position": "bottom", "nticks": 9, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}, {"position": "left", "nticks": 9, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}], "axesbg": "#FFFFFF", "axesbgalpha": null, "zoomable": true, "id": "el120002139942361034016", "lines": [], "paths": [{"data": "data01", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361126032", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data02", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361031376", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data03", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942360791184", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data04", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361121472", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data05", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361120896", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data06", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361123824", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data07", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361123392", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data08", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361127904", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data09", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361129296", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data10", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361129056", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data11", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361128624", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data12", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361127376", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data12", "xindex": 2, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361126752", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data12", "xindex": 3, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361126320", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data13", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361129872", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}], "markers": [], "texts": [], "collections": [], "images": [], "sharex": [], "sharey": []}], "data": {"data01": [[1.0, 0.0], [1.933333396911621, 0.0], [1.933333396911621, 7175.0], [1.0, 7175.0]], "data02": [[1.933333396911621, 0.0], [2.866666793823242, 0.0], [2.866666793823242, 6576.0], [1.933333396911621, 6576.0]], "data03": [[2.866666793823242, 0.0], [3.799999952316284, 0.0], [3.799999952316284, 4624.0], [2.866666793823242, 4624.0]], "data04": [[3.799999952316284, 0.0], [4.733333587646484, 0.0], [4.733333587646484, 3021.0], [3.799999952316284, 3021.0]], "data05": [[4.733333587646484, 0.0], [5.666666507720947, 0.0], [5.666666507720947, 2275.0], [4.733333587646484, 2275.0]], "data06": [[5.666666507720947, 0.0], [6.599999904632568, 0.0], [6.599999904632568, 1653.0], [5.666666507720947, 1653.0]], "data07": [[6.599999904632568, 0.0], [7.5333333015441895, 0.0], [7.5333333015441895, 492.0], [6.599999904632568, 492.0]], "data08": [[7.5333333015441895, 0.0], [8.466667175292969, 0.0], [8.466667175292969, 119.0], [7.5333333015441895, 119.0]], "data09": [[8.466667175292969, 0.0], [9.399999618530273, 0.0], [9.399999618530273, 28.0], [8.466667175292969, 28.0]], "data10": [[9.399999618530273, 0.0], [10.333333015441895, 0.0], [10.333333015441895, 5.0], [9.399999618530273, 5.0]], "data11": [[10.333333015441895, 0.0], [11.266666412353516, 0.0], [11.266666412353516, 1.0], [10.333333015441895, 1.0]], "data12": [[11.266666412353516, 0.0, 12.199999809265137, 13.133333206176758], [12.199999809265137, 0.0, 13.133333206176758, 14.066666603088379], [12.199999809265137, 0.0, 13.133333206176758, 14.066666603088379], [11.266666412353516, 0.0, 12.199999809265137, 13.133333206176758]], "data13": [[14.066666603088379, 0.0], [15.0, 0.0], [15.0, 6799.0], [14.066666603088379, 6799.0]]}, "id": "el120002139942405383872", "plugins": [{"type": "reset"}, {"type": "zoom", "button": true, "enabled": false}, {"type": "boxzoom", "button": true, "enabled": false}]}); + mpld3.draw_figure("fig_el3734471397923808578566893788369", {"width": 640.0, "height": 480.0, "axes": [{"bbox": [0.125, 0.10999999999999999, 0.775, 0.77], "xlim": [-1.19, 16.19], "ylim": [0.0, 7642.95], "xdomain": [-1.19, 16.19], "ydomain": [0.0, 7642.95], "xscale": "linear", "yscale": "linear", "axes": [{"position": "bottom", "nticks": 11, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}, {"position": "left", "nticks": 9, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}], "axesbg": "#FFFFFF", "axesbgalpha": null, "zoomable": true, "id": "el373447139797053788464", "lines": [], "paths": [{"data": "data01", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381535360", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data02", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792380856416", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data03", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139797054022832", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data04", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381530944", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data05", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381539632", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data06", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139797053780624", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data07", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381532192", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data08", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381536896", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data09", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381539872", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data10", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381532480", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data11", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139797053797824", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data01", "xindex": 2, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381533200", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data01", "xindex": 3, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381532240", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data01", "xindex": 4, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139797054021056", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data01", "xindex": 5, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381532576", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data12", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381527536", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}], "markers": [], "texts": [{"text": "Label", "position": [0.5, -0.07921476671476674], "coordinates": "axes", "h_anchor": "middle", "v_baseline": "hanging", "rotation": -0.0, "fontsize": 10.0, "color": "#000000", "alpha": 1, "zorder": 3, "id": "el373447139792380852176"}, {"text": "Count", "position": [-0.10161850358422941, 0.5], "coordinates": "axes", "h_anchor": "middle", "v_baseline": "auto", "rotation": -90.0, "fontsize": 10.0, "color": "#000000", "alpha": 1, "zorder": 3, "id": "el373447139792380853616"}], "collections": [], "images": [], "sharex": [], "sharey": []}], "data": {"data01": [[-0.4, 0.0, 10.6, 11.6, 12.6, 13.6], [0.4, 0.0, 11.4, 12.4, 13.4, 14.4], [0.4, 0.0, 11.4, 12.4, 13.4, 14.4], [-0.4, 0.0, 10.6, 11.6, 12.6, 13.6]], "data02": [[0.6, 0.0], [1.4, 0.0], [1.4, 7279.0], [0.6, 7279.0]], "data03": [[1.6, 0.0], [2.4000000000000004, 0.0], [2.4000000000000004, 6614.0], [1.6, 6614.0]], "data04": [[2.6, 0.0], [3.4000000000000004, 0.0], [3.4000000000000004, 4434.0], [2.6, 4434.0]], "data05": [[3.6, 0.0], [4.4, 0.0], [4.4, 2907.0], [3.6, 2907.0]], "data06": [[4.6, 0.0], [5.3999999999999995, 0.0], [5.3999999999999995, 2324.0], [4.6, 2324.0]], "data07": [[5.6, 0.0], [6.3999999999999995, 0.0], [6.3999999999999995, 1578.0], [5.6, 1578.0]], "data08": [[6.6, 0.0], [7.3999999999999995, 0.0], [7.3999999999999995, 469.0], [6.6, 469.0]], "data09": [[7.6, 0.0], [8.4, 0.0], [8.4, 133.0], [7.6, 133.0]], "data10": [[8.6, 0.0], [9.4, 0.0], [9.4, 27.0], [8.6, 27.0]], "data11": [[9.6, 0.0], [10.4, 0.0], [10.4, 4.0], [9.6, 4.0]], "data12": [[14.6, 0.0], [15.4, 0.0], [15.4, 6999.0], [14.6, 6999.0]]}, "id": "el373447139792380857856", "plugins": [{"type": "reset"}, {"type": "zoom", "button": true, "enabled": false}, {"type": "boxzoom", "button": true, "enabled": false}]}); }(mpld3); }else if(typeof define === "function" && define.amd){ // require.js is available: use it to load d3/mpld3 @@ -28,7 +28,7 @@ if(typeof(mpld3) !== "undefined" && mpld3._mpld3IsLoaded){ window.d3 = d3; mpld3_load_lib("https://mpld3.github.io/js/mpld3.v0.5.10.js", function(){ - mpld3.draw_figure("fig_el1200021399424053838726786165297", {"width": 640.0, "height": 480.0, "axes": [{"bbox": [0.125, 0.10999999999999999, 0.775, 0.77], "xlim": [0.29999999999999993, 15.7], "ylim": [0.0, 7533.75], "xdomain": [0.29999999999999993, 15.7], "ydomain": [0.0, 7533.75], "xscale": "linear", "yscale": "linear", "axes": [{"position": "bottom", "nticks": 9, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}, {"position": "left", "nticks": 9, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}], "axesbg": "#FFFFFF", "axesbgalpha": null, "zoomable": true, "id": "el120002139942361034016", "lines": [], "paths": [{"data": "data01", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361126032", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data02", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361031376", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data03", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942360791184", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data04", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361121472", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data05", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361120896", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data06", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361123824", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data07", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361123392", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data08", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361127904", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data09", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361129296", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data10", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361129056", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data11", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361128624", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data12", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361127376", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data12", "xindex": 2, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361126752", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data12", "xindex": 3, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361126320", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data13", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361129872", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}], "markers": [], "texts": [], "collections": [], "images": [], "sharex": [], "sharey": []}], "data": {"data01": [[1.0, 0.0], [1.933333396911621, 0.0], [1.933333396911621, 7175.0], [1.0, 7175.0]], "data02": [[1.933333396911621, 0.0], [2.866666793823242, 0.0], [2.866666793823242, 6576.0], [1.933333396911621, 6576.0]], "data03": [[2.866666793823242, 0.0], [3.799999952316284, 0.0], [3.799999952316284, 4624.0], [2.866666793823242, 4624.0]], "data04": [[3.799999952316284, 0.0], [4.733333587646484, 0.0], [4.733333587646484, 3021.0], [3.799999952316284, 3021.0]], "data05": [[4.733333587646484, 0.0], [5.666666507720947, 0.0], [5.666666507720947, 2275.0], [4.733333587646484, 2275.0]], "data06": [[5.666666507720947, 0.0], [6.599999904632568, 0.0], [6.599999904632568, 1653.0], [5.666666507720947, 1653.0]], "data07": [[6.599999904632568, 0.0], [7.5333333015441895, 0.0], [7.5333333015441895, 492.0], [6.599999904632568, 492.0]], "data08": [[7.5333333015441895, 0.0], [8.466667175292969, 0.0], [8.466667175292969, 119.0], [7.5333333015441895, 119.0]], "data09": [[8.466667175292969, 0.0], [9.399999618530273, 0.0], [9.399999618530273, 28.0], [8.466667175292969, 28.0]], "data10": [[9.399999618530273, 0.0], [10.333333015441895, 0.0], [10.333333015441895, 5.0], [9.399999618530273, 5.0]], "data11": [[10.333333015441895, 0.0], [11.266666412353516, 0.0], [11.266666412353516, 1.0], [10.333333015441895, 1.0]], "data12": [[11.266666412353516, 0.0, 12.199999809265137, 13.133333206176758], [12.199999809265137, 0.0, 13.133333206176758, 14.066666603088379], [12.199999809265137, 0.0, 13.133333206176758, 14.066666603088379], [11.266666412353516, 0.0, 12.199999809265137, 13.133333206176758]], "data13": [[14.066666603088379, 0.0], [15.0, 0.0], [15.0, 6799.0], [14.066666603088379, 6799.0]]}, "id": "el120002139942405383872", "plugins": [{"type": "reset"}, {"type": "zoom", "button": true, "enabled": false}, {"type": "boxzoom", "button": true, "enabled": false}]}); + mpld3.draw_figure("fig_el3734471397923808578566893788369", {"width": 640.0, "height": 480.0, "axes": [{"bbox": [0.125, 0.10999999999999999, 0.775, 0.77], "xlim": [-1.19, 16.19], "ylim": [0.0, 7642.95], "xdomain": [-1.19, 16.19], "ydomain": [0.0, 7642.95], "xscale": "linear", "yscale": "linear", "axes": [{"position": "bottom", "nticks": 11, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}, {"position": "left", "nticks": 9, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}], "axesbg": "#FFFFFF", "axesbgalpha": null, "zoomable": true, "id": "el373447139797053788464", "lines": [], "paths": [{"data": "data01", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381535360", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data02", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792380856416", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data03", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139797054022832", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data04", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381530944", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data05", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381539632", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data06", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139797053780624", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data07", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381532192", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data08", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381536896", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data09", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381539872", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data10", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381532480", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data11", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139797053797824", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data01", "xindex": 2, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381533200", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data01", "xindex": 3, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381532240", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data01", "xindex": 4, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139797054021056", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data01", "xindex": 5, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381532576", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data12", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381527536", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}], "markers": [], "texts": [{"text": "Label", "position": [0.5, -0.07921476671476674], "coordinates": "axes", "h_anchor": "middle", "v_baseline": "hanging", "rotation": -0.0, "fontsize": 10.0, "color": "#000000", "alpha": 1, "zorder": 3, "id": "el373447139792380852176"}, {"text": "Count", "position": [-0.10161850358422941, 0.5], "coordinates": "axes", "h_anchor": "middle", "v_baseline": "auto", "rotation": -90.0, "fontsize": 10.0, "color": "#000000", "alpha": 1, "zorder": 3, "id": "el373447139792380853616"}], "collections": [], "images": [], "sharex": [], "sharey": []}], "data": {"data01": [[-0.4, 0.0, 10.6, 11.6, 12.6, 13.6], [0.4, 0.0, 11.4, 12.4, 13.4, 14.4], [0.4, 0.0, 11.4, 12.4, 13.4, 14.4], [-0.4, 0.0, 10.6, 11.6, 12.6, 13.6]], "data02": [[0.6, 0.0], [1.4, 0.0], [1.4, 7279.0], [0.6, 7279.0]], "data03": [[1.6, 0.0], [2.4000000000000004, 0.0], [2.4000000000000004, 6614.0], [1.6, 6614.0]], "data04": [[2.6, 0.0], [3.4000000000000004, 0.0], [3.4000000000000004, 4434.0], [2.6, 4434.0]], "data05": [[3.6, 0.0], [4.4, 0.0], [4.4, 2907.0], [3.6, 2907.0]], "data06": [[4.6, 0.0], [5.3999999999999995, 0.0], [5.3999999999999995, 2324.0], [4.6, 2324.0]], "data07": [[5.6, 0.0], [6.3999999999999995, 0.0], [6.3999999999999995, 1578.0], [5.6, 1578.0]], "data08": [[6.6, 0.0], [7.3999999999999995, 0.0], [7.3999999999999995, 469.0], [6.6, 469.0]], "data09": [[7.6, 0.0], [8.4, 0.0], [8.4, 133.0], [7.6, 133.0]], "data10": [[8.6, 0.0], [9.4, 0.0], [9.4, 27.0], [8.6, 27.0]], "data11": [[9.6, 0.0], [10.4, 0.0], [10.4, 4.0], [9.6, 4.0]], "data12": [[14.6, 0.0], [15.4, 0.0], [15.4, 6999.0], [14.6, 6999.0]]}, "id": "el373447139792380857856", "plugins": [{"type": "reset"}, {"type": "zoom", "button": true, "enabled": false}, {"type": "boxzoom", "button": true, "enabled": false}]}); }); }); }else{ @@ -36,7 +36,7 @@ if(typeof(mpld3) !== "undefined" && mpld3._mpld3IsLoaded){ mpld3_load_lib("https://d3js.org/d3.v5.js", function(){ mpld3_load_lib("https://mpld3.github.io/js/mpld3.v0.5.10.js", function(){ - mpld3.draw_figure("fig_el1200021399424053838726786165297", {"width": 640.0, "height": 480.0, "axes": [{"bbox": [0.125, 0.10999999999999999, 0.775, 0.77], "xlim": [0.29999999999999993, 15.7], "ylim": [0.0, 7533.75], "xdomain": [0.29999999999999993, 15.7], "ydomain": [0.0, 7533.75], "xscale": "linear", "yscale": "linear", "axes": [{"position": "bottom", "nticks": 9, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}, {"position": "left", "nticks": 9, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}], "axesbg": "#FFFFFF", "axesbgalpha": null, "zoomable": true, "id": "el120002139942361034016", "lines": [], "paths": [{"data": "data01", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361126032", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data02", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361031376", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data03", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942360791184", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data04", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361121472", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data05", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361120896", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data06", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361123824", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data07", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361123392", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data08", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361127904", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data09", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361129296", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data10", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361129056", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data11", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361128624", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data12", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361127376", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data12", "xindex": 2, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361126752", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data12", "xindex": 3, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361126320", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data13", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el120002139942361129872", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}], "markers": [], "texts": [], "collections": [], "images": [], "sharex": [], "sharey": []}], "data": {"data01": [[1.0, 0.0], [1.933333396911621, 0.0], [1.933333396911621, 7175.0], [1.0, 7175.0]], "data02": [[1.933333396911621, 0.0], [2.866666793823242, 0.0], [2.866666793823242, 6576.0], [1.933333396911621, 6576.0]], "data03": [[2.866666793823242, 0.0], [3.799999952316284, 0.0], [3.799999952316284, 4624.0], [2.866666793823242, 4624.0]], "data04": [[3.799999952316284, 0.0], [4.733333587646484, 0.0], [4.733333587646484, 3021.0], [3.799999952316284, 3021.0]], "data05": [[4.733333587646484, 0.0], [5.666666507720947, 0.0], [5.666666507720947, 2275.0], [4.733333587646484, 2275.0]], "data06": [[5.666666507720947, 0.0], [6.599999904632568, 0.0], [6.599999904632568, 1653.0], [5.666666507720947, 1653.0]], "data07": [[6.599999904632568, 0.0], [7.5333333015441895, 0.0], [7.5333333015441895, 492.0], [6.599999904632568, 492.0]], "data08": [[7.5333333015441895, 0.0], [8.466667175292969, 0.0], [8.466667175292969, 119.0], [7.5333333015441895, 119.0]], "data09": [[8.466667175292969, 0.0], [9.399999618530273, 0.0], [9.399999618530273, 28.0], [8.466667175292969, 28.0]], "data10": [[9.399999618530273, 0.0], [10.333333015441895, 0.0], [10.333333015441895, 5.0], [9.399999618530273, 5.0]], "data11": [[10.333333015441895, 0.0], [11.266666412353516, 0.0], [11.266666412353516, 1.0], [10.333333015441895, 1.0]], "data12": [[11.266666412353516, 0.0, 12.199999809265137, 13.133333206176758], [12.199999809265137, 0.0, 13.133333206176758, 14.066666603088379], [12.199999809265137, 0.0, 13.133333206176758, 14.066666603088379], [11.266666412353516, 0.0, 12.199999809265137, 13.133333206176758]], "data13": [[14.066666603088379, 0.0], [15.0, 0.0], [15.0, 6799.0], [14.066666603088379, 6799.0]]}, "id": "el120002139942405383872", "plugins": [{"type": "reset"}, {"type": "zoom", "button": true, "enabled": false}, {"type": "boxzoom", "button": true, "enabled": false}]}); + mpld3.draw_figure("fig_el3734471397923808578566893788369", {"width": 640.0, "height": 480.0, "axes": [{"bbox": [0.125, 0.10999999999999999, 0.775, 0.77], "xlim": [-1.19, 16.19], "ylim": [0.0, 7642.95], "xdomain": [-1.19, 16.19], "ydomain": [0.0, 7642.95], "xscale": "linear", "yscale": "linear", "axes": [{"position": "bottom", "nticks": 11, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}, {"position": "left", "nticks": 9, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}], "axesbg": "#FFFFFF", "axesbgalpha": null, "zoomable": true, "id": "el373447139797053788464", "lines": [], "paths": [{"data": "data01", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381535360", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data02", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792380856416", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data03", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139797054022832", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data04", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381530944", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data05", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381539632", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data06", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139797053780624", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data07", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381532192", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data08", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381536896", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data09", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381539872", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data10", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381532480", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data11", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139797053797824", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data01", "xindex": 2, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381533200", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data01", "xindex": 3, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381532240", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data01", "xindex": 4, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139797054021056", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data01", "xindex": 5, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381532576", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}, {"data": "data12", "xindex": 0, "yindex": 1, "coordinates": "data", "pathcodes": ["M", "L", "L", "L", "Z"], "id": "el373447139792381527536", "dasharray": "none", "alpha": 1, "facecolor": "#1F77B4", "edgecolor": "none", "edgewidth": 1.0, "zorder": 1}], "markers": [], "texts": [{"text": "Label", "position": [0.5, -0.07921476671476674], "coordinates": "axes", "h_anchor": "middle", "v_baseline": "hanging", "rotation": -0.0, "fontsize": 10.0, "color": "#000000", "alpha": 1, "zorder": 3, "id": "el373447139792380852176"}, {"text": "Count", "position": [-0.10161850358422941, 0.5], "coordinates": "axes", "h_anchor": "middle", "v_baseline": "auto", "rotation": -90.0, "fontsize": 10.0, "color": "#000000", "alpha": 1, "zorder": 3, "id": "el373447139792380853616"}], "collections": [], "images": [], "sharex": [], "sharey": []}], "data": {"data01": [[-0.4, 0.0, 10.6, 11.6, 12.6, 13.6], [0.4, 0.0, 11.4, 12.4, 13.4, 14.4], [0.4, 0.0, 11.4, 12.4, 13.4, 14.4], [-0.4, 0.0, 10.6, 11.6, 12.6, 13.6]], "data02": [[0.6, 0.0], [1.4, 0.0], [1.4, 7279.0], [0.6, 7279.0]], "data03": [[1.6, 0.0], [2.4000000000000004, 0.0], [2.4000000000000004, 6614.0], [1.6, 6614.0]], "data04": [[2.6, 0.0], [3.4000000000000004, 0.0], [3.4000000000000004, 4434.0], [2.6, 4434.0]], "data05": [[3.6, 0.0], [4.4, 0.0], [4.4, 2907.0], [3.6, 2907.0]], "data06": [[4.6, 0.0], [5.3999999999999995, 0.0], [5.3999999999999995, 2324.0], [4.6, 2324.0]], "data07": [[5.6, 0.0], [6.3999999999999995, 0.0], [6.3999999999999995, 1578.0], [5.6, 1578.0]], "data08": [[6.6, 0.0], [7.3999999999999995, 0.0], [7.3999999999999995, 469.0], [6.6, 469.0]], "data09": [[7.6, 0.0], [8.4, 0.0], [8.4, 133.0], [7.6, 133.0]], "data10": [[8.6, 0.0], [9.4, 0.0], [9.4, 27.0], [8.6, 27.0]], "data11": [[9.6, 0.0], [10.4, 0.0], [10.4, 4.0], [9.6, 4.0]], "data12": [[14.6, 0.0], [15.4, 0.0], [15.4, 6999.0], [14.6, 6999.0]]}, "id": "el373447139792380857856", "plugins": [{"type": "reset"}, {"type": "zoom", "button": true, "enabled": false}, {"type": "boxzoom", "button": true, "enabled": false}]}); }) }); } diff --git a/transformer_shortest_paths.ipynb b/transformer_shortest_paths.ipynb index 8da9ce4..25fd2a7 100644 --- a/transformer_shortest_paths.ipynb +++ b/transformer_shortest_paths.ipynb @@ -183,7 +183,7 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 4, "execution_state": "idle", "metadata": {}, "outputs": [], @@ -200,35 +200,35 @@ }, { "cell_type": "code", - "execution_count": 95, + "execution_count": 4, "execution_state": "idle", "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "(tensor([[1, 3, 1, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", - " 0, 0, 0, 0, 0, 0, 2],\n", - " [1, 5, 3, 4, 1, 3, 4, 5, 3, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", - " 0, 0, 0, 0, 0, 0, 2],\n", - " [2, 9, 1, 3, 4, 8, 8, 9, 3, 9, 5, 7, 6, 7, 1, 6, 0, 0, 0, 0, 0, 0, 0, 0,\n", - " 0, 0, 0, 0, 0, 0, 2]]),\n", - " tensor([ 1., 15., 3.]),\n", - " tensor([[False, False, False, False, True, True, True, True, True, True,\n", - " True, True, True, True, True, True, True, True, True, True,\n", - " True, True, True, True, True, True, True, True, True, True,\n", + "(tensor([[ 5, 11, 9, 10, 3, 4, 6, 13, 3, 9, 9, 13, 1, 10, 6, 8, 1, 2,\n", + " 6, 13, 4, 5, 1, 4, 2, 10, 0, 0, 0, 0, 2],\n", + " [ 4, 10, 1, 10, 2, 7, 6, 9, 4, 5, 4, 6, 5, 7, 3, 5, 0, 0,\n", + " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2],\n", + " [ 9, 10, 3, 7, 1, 3, 5, 6, 13, 14, 1, 15, 4, 6, 4, 11, 2, 11,\n", + " 6, 13, 9, 15, 7, 14, 10, 12, 3, 15, 0, 0, 2]]),\n", + " tensor([1., 5., 8.]),\n", + " tensor([[False, False, False, False, False, False, False, False, False, False,\n", + " False, False, False, False, False, False, False, False, False, False,\n", + " False, False, False, False, False, False, True, True, True, True,\n", " False],\n", " [False, False, False, False, False, False, False, False, False, False,\n", - " True, True, True, True, True, True, True, True, True, True,\n", + " False, False, False, False, False, False, True, True, True, True,\n", " True, True, True, True, True, True, True, True, True, True,\n", " False],\n", " [False, False, False, False, False, False, False, False, False, False,\n", - " False, False, False, False, False, False, True, True, True, True,\n", - " True, True, True, True, True, True, True, True, True, True,\n", + " False, False, False, False, False, False, False, False, False, False,\n", + " False, False, False, False, False, False, False, False, True, True,\n", " False]]))" ] }, - "execution_count": 95, + "execution_count": 4, "metadata": {}, "output_type": "execute_result" } @@ -239,13 +239,13 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 12, "execution_state": "idle", "metadata": {}, "outputs": [ { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjAAAAGdCAYAAAAMm0nCAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAqu0lEQVR4nO3de3BUZZ7/8U8upAmX7kg03WQJkFlcIYgXwIUe1F00Sw/TuroEZ3AQWUAtqEZNsgsxu5hx0DGIowgqZPBCmBpYhSpxJCnAGCCsQ7gYNg6gRFyjYSZ2x11NGhhJQtK/P36Ts7SCkpCkecL7VXWq7PN8zznfpyukP56cczoqFAqFBAAAYJDoSDcAAADQXgQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxYiPdQFdpbW1VbW2t+vfvr6ioqEi3AwAAzkMoFNLx48eVnJys6Ohzn2fpsQGmtrZWKSkpkW4DAAB0wLFjxzRo0KBzjvfYANO/f3/pL2+A3W6PdDsAAOA8BINBpaSkWJ/j59JjA0zbn43sdjsBBgAAw3zf5R9cxAsAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgnNhIN2CioY8UR+S4ny7xRuS4AABcbDgDAwAAjEOAAQAAxiHAAAAA4xBgAACAcdoVYIYOHaqoqKhvLT6fT5J06tQp+Xw+JSYmql+/fsrIyFAgEAjbR01Njbxer/r06aOkpCQtWLBAp0+fDqvZuXOnRo8eLZvNpmHDhqmwsLAz5goAAHqIdgWY/fv36/PPP7eWkpISSdJdd90lScrKytLmzZu1ceNGlZWVqba2VlOmTLG2b2lpkdfrVVNTk3bv3q21a9eqsLBQeXl5Vk11dbW8Xq8mTpyoyspKZWZm6r777tO2bds6b9YAAMBoUaFQKNTRjTMzM1VUVKSjR48qGAzqiiuu0Pr16zV16lRJ0pEjRzRixAiVl5dr/Pjx2rJli2677TbV1tbK6XRKkgoKCpSTk6MvvvhCcXFxysnJUXFxsQ4dOmQdZ9q0aaqvr9fWrVvPu7dgMCiHw6GGhgbZ7faOTvGsuI0aAICucb6f3x2+BqapqUm//e1vNXv2bEVFRamiokLNzc1KT0+3aoYPH67BgwervLxcklReXq5Ro0ZZ4UWSPB6PgsGgDh8+bNWcuY+2mrZ9nEtjY6OCwWDYAgAAeqYOB5g333xT9fX1+ud//mdJkt/vV1xcnBISEsLqnE6n/H6/VXNmeGkbbxv7rppgMKivv/76nP3k5+fL4XBYS0pKSkenBgAALnIdDjCvvPKKJk+erOTk5M7tqINyc3PV0NBgLceOHYt0SwAAoIt06KsEPvvsM73zzjt64403rHUul0tNTU2qr68POwsTCATkcrmsmn379oXtq+0upTNrvnnnUiAQkN1uV3x8/Dl7stlsstlsHZkOAAAwTIfOwKxZs0ZJSUnyev/votIxY8aoV69eKi0ttdZVVVWppqZGbrdbkuR2u3Xw4EHV1dVZNSUlJbLb7UpLS7NqztxHW03bPgAAANodYFpbW7VmzRrNnDlTsbH/dwLH4XBozpw5ys7O1o4dO1RRUaFZs2bJ7XZr/PjxkqRJkyYpLS1NM2bM0Pvvv69t27Zp0aJF8vl81tmTuXPn6pNPPtHChQt15MgRrVy5Uhs2bFBWVlZnzhsAABis3X9Ceuedd1RTU6PZs2d/a2zZsmWKjo5WRkaGGhsb5fF4tHLlSms8JiZGRUVFmjdvntxut/r27auZM2dq8eLFVk1qaqqKi4uVlZWl5cuXa9CgQXr55Zfl8XguZJ4AAKAHuaDnwFzMeA4MAADm6fLnwAAAAEQKAQYAABiHAAMAAIzToefAAACAi8Olel0mZ2AAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDh8lYBBIvW4aF0Ej4wGAOBMnIEBAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYp90B5k9/+pPuueceJSYmKj4+XqNGjdJ7771njYdCIeXl5WngwIGKj49Xenq6jh49GraPL7/8UtOnT5fdbldCQoLmzJmjEydOhNX84Q9/0E033aTevXsrJSVFS5cuvZB5AgCAHqRdAearr77ShAkT1KtXL23ZskUffPCBnnnmGV122WVWzdKlS7VixQoVFBRo79696tu3rzwej06dOmXVTJ8+XYcPH1ZJSYmKioq0a9cuPfDAA9Z4MBjUpEmTNGTIEFVUVOjpp5/WY489ptWrV3fWvAEAgMFi21P81FNPKSUlRWvWrLHWpaamWv8dCoX03HPPadGiRbrjjjskSb/5zW/kdDr15ptvatq0afrwww+1detW7d+/X2PHjpUkPf/88/rxj3+sX/3qV0pOTta6devU1NSkV199VXFxcRo5cqQqKyv17LPPhgUdAABwaWrXGZi33npLY8eO1V133aWkpCRdf/31eumll6zx6upq+f1+paenW+scDofGjRun8vJySVJ5ebkSEhKs8CJJ6enpio6O1t69e62am2++WXFxcVaNx+NRVVWVvvrqq7P21tjYqGAwGLYAAICeqV0B5pNPPtGqVat05ZVXatu2bZo3b54eeughrV27VpLk9/slSU6nM2w7p9Npjfn9fiUlJYWNx8bGasCAAWE1Z9vHmcf4pvz8fDkcDmtJSUlpz9QAAIBB2hVgWltbNXr0aD355JO6/vrr9cADD+j+++9XQUFB13V4nnJzc9XQ0GAtx44di3RLAACgi7QrwAwcOFBpaWlh60aMGKGamhpJksvlkiQFAoGwmkAgYI25XC7V1dWFjZ8+fVpffvllWM3Z9nHmMb7JZrPJbreHLQAAoGdqV4CZMGGCqqqqwtZ99NFHGjJkiPSXC3pdLpdKS0ut8WAwqL1798rtdkuS3G636uvrVVFRYdVs375dra2tGjdunFWza9cuNTc3WzUlJSW66qqrwu54AgAAl6Z2BZisrCzt2bNHTz75pD7++GOtX79eq1evls/nkyRFRUUpMzNTTzzxhN566y0dPHhQ9957r5KTk3XnnXdKfzlj86Mf/Uj333+/9u3bp9///veaP3++pk2bpuTkZEnSz372M8XFxWnOnDk6fPiwXn/9dS1fvlzZ2dld8R4AAADDtOs26htuuEGbNm1Sbm6uFi9erNTUVD333HOaPn26VbNw4UKdPHlSDzzwgOrr63XjjTdq69at6t27t1Wzbt06zZ8/X7feequio6OVkZGhFStWWOMOh0Nvv/22fD6fxowZo8svv1x5eXncQg0AACRJUaFQKBTpJrpCMBiUw+FQQ0NDp18PM/SR4k7dnwk+XeKNdAsAgLOI1GdSV30unO/nN9+FBAAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGCcdgWYxx57TFFRUWHL8OHDrfFTp07J5/MpMTFR/fr1U0ZGhgKBQNg+ampq5PV61adPHyUlJWnBggU6ffp0WM3OnTs1evRo2Ww2DRs2TIWFhRc6TwAA0IO0+wzMyJEj9fnnn1vLu+++a41lZWVp8+bN2rhxo8rKylRbW6spU6ZY4y0tLfJ6vWpqatLu3bu1du1aFRYWKi8vz6qprq6W1+vVxIkTVVlZqczMTN13333atm1bZ8wXAAD0ALHt3iA2Vi6X61vrGxoa9Morr2j9+vW65ZZbJElr1qzRiBEjtGfPHo0fP15vv/22PvjgA73zzjtyOp267rrr9PjjjysnJ0ePPfaY4uLiVFBQoNTUVD3zzDOSpBEjRujdd9/VsmXL5PF4OmPOAADAcO0+A3P06FElJyfrBz/4gaZPn66amhpJUkVFhZqbm5Wenm7VDh8+XIMHD1Z5ebkkqby8XKNGjZLT6bRqPB6PgsGgDh8+bNWcuY+2mrZ9nEtjY6OCwWDYAgAAeqZ2BZhx48apsLBQW7du1apVq1RdXa2bbrpJx48fl9/vV1xcnBISEsK2cTqd8vv9kiS/3x8WXtrG28a+qyYYDOrrr78+Z2/5+flyOBzWkpKS0p6pAQAAg7TrT0iTJ0+2/vuaa67RuHHjNGTIEG3YsEHx8fFd0d95y83NVXZ2tvU6GAwSYgAA6KEu6DbqhIQE/c3f/I0+/vhjuVwuNTU1qb6+PqwmEAhY18y4XK5v3ZXU9vr7aux2+3eGJJvNJrvdHrYAAICe6YICzIkTJ/Tf//3fGjhwoMaMGaNevXqptLTUGq+qqlJNTY3cbrckye126+DBg6qrq7NqSkpKZLfblZaWZtWcuY+2mrZ9AAAAtCvA/Ou//qvKysr06aefavfu3fqnf/onxcTE6O6775bD4dCcOXOUnZ2tHTt2qKKiQrNmzZLb7db48eMlSZMmTVJaWppmzJih999/X9u2bdOiRYvk8/lks9kkSXPnztUnn3yihQsX6siRI1q5cqU2bNigrKysrnkHAACAcdp1Dcwf//hH3X333frf//1fXXHFFbrxxhu1Z88eXXHFFZKkZcuWKTo6WhkZGWpsbJTH49HKlSut7WNiYlRUVKR58+bJ7Xarb9++mjlzphYvXmzVpKamqri4WFlZWVq+fLkGDRqkl19+mVuoAQCAJSoUCoUi3URXCAaDcjgcamho6PTrYYY+Utyp+zPBp0u8kW4BAHAWkfpM6qrPhfP9/Oa7kAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgnHZ9mSMuXT3tuzYAAGbjDAwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABjnggLMkiVLFBUVpczMTGvdqVOn5PP5lJiYqH79+ikjI0OBQCBsu5qaGnm9XvXp00dJSUlasGCBTp8+HVazc+dOjR49WjabTcOGDVNhYeGFtAoAAHqQDgeY/fv369e//rWuueaasPVZWVnavHmzNm7cqLKyMtXW1mrKlCnWeEtLi7xer5qamrR7926tXbtWhYWFysvLs2qqq6vl9Xo1ceJEVVZWKjMzU/fdd5+2bdvW0XYBAEAP0qEAc+LECU2fPl0vvfSSLrvsMmt9Q0ODXnnlFT377LO65ZZbNGbMGK1Zs0a7d+/Wnj17JElvv/22PvjgA/32t7/Vddddp8mTJ+vxxx/Xiy++qKamJklSQUGBUlNT9cwzz2jEiBGaP3++pk6dqmXLlnXWvAEAgME6FGB8Pp+8Xq/S09PD1ldUVKi5uTls/fDhwzV48GCVl5dLksrLyzVq1Cg5nU6rxuPxKBgM6vDhw1bNN/ft8XisfZxNY2OjgsFg2AIAAHqm2PZu8Nprr+nAgQPav3//t8b8fr/i4uKUkJAQtt7pdMrv91s1Z4aXtvG2se+qCQaD+vrrrxUfH/+tY+fn5+sXv/hFe6cDAAAM1K4zMMeOHdPDDz+sdevWqXfv3l3XVQfk5uaqoaHBWo4dOxbplgAAQBdpV4CpqKhQXV2dRo8erdjYWMXGxqqsrEwrVqxQbGysnE6nmpqaVF9fH7ZdIBCQy+WSJLlcrm/dldT2+vtq7Hb7Wc++SJLNZpPdbg9bAABAz9SuAHPrrbfq4MGDqqystJaxY8dq+vTp1n/36tVLpaWl1jZVVVWqqamR2+2WJLndbh08eFB1dXVWTUlJiex2u9LS0qyaM/fRVtO2DwAAcGlr1zUw/fv319VXXx22rm/fvkpMTLTWz5kzR9nZ2RowYIDsdrsefPBBud1ujR8/XpI0adIkpaWlacaMGVq6dKn8fr8WLVokn88nm80mSZo7d65eeOEFLVy4ULNnz9b27du1YcMGFRcXd97MAQCAsdp9Ee/3WbZsmaKjo5WRkaHGxkZ5PB6tXLnSGo+JiVFRUZHmzZsnt9utvn37aubMmVq8eLFVk5qaquLiYmVlZWn58uUaNGiQXn75ZXk8ns5uFwAAGCgqFAqFIt1EVwgGg3I4HGpoaOj062GGPsKZoO7y6RJvpFsAgItapD6Tuur38/l+fvNdSAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGaVeAWbVqla655hrZ7XbZ7Xa53W5t2bLFGj916pR8Pp8SExPVr18/ZWRkKBAIhO2jpqZGXq9Xffr0UVJSkhYsWKDTp0+H1ezcuVOjR4+WzWbTsGHDVFhYeKHzBAAAPUi7AsygQYO0ZMkSVVRU6L333tMtt9yiO+64Q4cPH5YkZWVlafPmzdq4caPKyspUW1urKVOmWNu3tLTI6/WqqalJu3fv1tq1a1VYWKi8vDyrprq6Wl6vVxMnTlRlZaUyMzN13333adu2bZ05bwAAYLCoUCgUupAdDBgwQE8//bSmTp2qK664QuvXr9fUqVMlSUeOHNGIESNUXl6u8ePHa8uWLbrttttUW1srp9MpSSooKFBOTo6++OILxcXFKScnR8XFxTp06JB1jGnTpqm+vl5bt249776CwaAcDocaGhpkt9svZIrfMvSR4k7dH87t0yXeSLcAABe1SH0mddXv5/P9/I7t6AFaWlq0ceNGnTx5Um63WxUVFWpublZ6erpVM3z4cA0ePNgKMOXl5Ro1apQVXiTJ4/Fo3rx5Onz4sK6//nqVl5eH7aOtJjMz8zv7aWxsVGNjo/U6GAx2dGq4iPS0f5gAgM7R7ot4Dx48qH79+slms2nu3LnatGmT0tLS5Pf7FRcXp4SEhLB6p9Mpv98vSfL7/WHhpW28bey7aoLBoL7++utz9pWfny+Hw2EtKSkp7Z0aAAAwRLsDzFVXXaXKykrt3btX8+bN08yZM/XBBx90TXftkJubq4aGBms5duxYpFsCAABdpN1/QoqLi9OwYcMkSWPGjNH+/fu1fPly/fSnP1VTU5Pq6+vDzsIEAgG5XC5Jksvl0r59+8L213aX0pk137xzKRAIyG63Kz4+/px92Ww22Wy29k4HAAAY6IKfA9Pa2qrGxkaNGTNGvXr1UmlpqTVWVVWlmpoaud1uSZLb7dbBgwdVV1dn1ZSUlMhutystLc2qOXMfbTVt+wAAAGjXGZjc3FxNnjxZgwcP1vHjx7V+/Xrt3LlT27Ztk8Ph0Jw5c5Sdna0BAwbIbrfrwQcflNvt1vjx4yVJkyZNUlpammbMmKGlS5fK7/dr0aJF8vl81tmTuXPn6oUXXtDChQs1e/Zsbd++XRs2bFBxMXf+AACA/69dAaaurk733nuvPv/8czkcDl1zzTXatm2b/uEf/kGStGzZMkVHRysjI0ONjY3yeDxauXKltX1MTIyKioo0b948ud1u9e3bVzNnztTixYutmtTUVBUXFysrK0vLly/XoEGD9PLLL8vj8XTmvAEAgMEu+DkwFyueA4MLwW3UAEzR0x43cb6f33wXEgAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA48RGugHgYjT0keKIHPfTJd6IHBcATMMZGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOO0KMPn5+brhhhvUv39/JSUl6c4771RVVVVYzalTp+Tz+ZSYmKh+/fopIyNDgUAgrKampkZer1d9+vRRUlKSFixYoNOnT4fV7Ny5U6NHj5bNZtOwYcNUWFh4IfMEAAA9SLsCTFlZmXw+n/bs2aOSkhI1Nzdr0qRJOnnypFWTlZWlzZs3a+PGjSorK1Ntba2mTJlijbe0tMjr9aqpqUm7d+/W2rVrVVhYqLy8PKumurpaXq9XEydOVGVlpTIzM3Xfffdp27ZtnTVvAABgsKhQKBTq6MZffPGFkpKSVFZWpptvvlkNDQ264oortH79ek2dOlWSdOTIEY0YMULl5eUaP368tmzZottuu021tbVyOp2SpIKCAuXk5OiLL75QXFyccnJyVFxcrEOHDlnHmjZtmurr67V169bz6i0YDMrhcKihoUF2u72jUzyrSD1mHj0fXyUAoL162lefnO/n9wVdA9PQ0CBJGjBggCSpoqJCzc3NSk9Pt2qGDx+uwYMHq7y8XJJUXl6uUaNGWeFFkjwej4LBoA4fPmzVnLmPtpq2fZxNY2OjgsFg2AIAAHqmDgeY1tZWZWZmasKECbr66qslSX6/X3FxcUpISAirdTqd8vv9Vs2Z4aVtvG3su2qCwaC+/vrrs/aTn58vh8NhLSkpKR2dGgAAuMh1OMD4fD4dOnRIr732Wud21EG5ublqaGiwlmPHjkW6JQAA0EViO7LR/PnzVVRUpF27dmnQoEHWepfLpaamJtXX14edhQkEAnK5XFbNvn37wvbXdpfSmTXfvHMpEAjIbrcrPj7+rD3ZbDbZbLaOTAcAABimXWdgQqGQ5s+fr02bNmn79u1KTU0NGx8zZox69eql0tJSa11VVZVqamrkdrslSW63WwcPHlRdXZ1VU1JSIrvdrrS0NKvmzH201bTtAwAAXNradQbG5/Np/fr1+t3vfqf+/ftb16w4HA7Fx8fL4XBozpw5ys7O1oABA2S32/Xggw/K7XZr/PjxkqRJkyYpLS1NM2bM0NKlS+X3+7Vo0SL5fD7rDMrcuXP1wgsvaOHChZo9e7a2b9+uDRs2qLiYu38AAEA7z8CsWrVKDQ0N+vu//3sNHDjQWl5//XWrZtmyZbrtttuUkZGhm2++WS6XS2+88YY1HhMTo6KiIsXExMjtduuee+7Rvffeq8WLF1s1qampKi4uVklJia699lo988wzevnll+XxeDpr3gAAwGAX9ByYixnPgYGJeA4MgPbiOTAAAACGIMAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOO068scAXStnvZIcADoKpyBAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcdodYHbt2qXbb79dycnJioqK0ptvvhk2HgqFlJeXp4EDByo+Pl7p6ek6evRoWM2XX36p6dOny263KyEhQXPmzNGJEyfCav7whz/opptuUu/evZWSkqKlS5d2dI4AAKCHaXeAOXnypK699lq9+OKLZx1funSpVqxYoYKCAu3du1d9+/aVx+PRqVOnrJrp06fr8OHDKikpUVFRkXbt2qUHHnjAGg8Gg5o0aZKGDBmiiooKPf3003rssce0evXqjs4TAAD0ILHt3WDy5MmaPHnyWcdCoZCee+45LVq0SHfccYck6Te/+Y2cTqfefPNNTZs2TR9++KG2bt2q/fv3a+zYsZKk559/Xj/+8Y/1q1/9SsnJyVq3bp2ampr06quvKi4uTiNHjlRlZaWeffbZsKADAAAuTZ16DUx1dbX8fr/S09OtdQ6HQ+PGjVN5ebkkqby8XAkJCVZ4kaT09HRFR0dr7969Vs3NN9+suLg4q8bj8aiqqkpfffXVWY/d2NioYDAYtgAAgJ6pUwOM3++XJDmdzrD1TqfTGvP7/UpKSgobj42N1YABA8JqzraPM4/xTfn5+XI4HNaSkpLSiTMDAAAXkx5zF1Jubq4aGhqs5dixY5FuCQAAdJFODTAul0uSFAgEwtYHAgFrzOVyqa6uLmz89OnT+vLLL8NqzraPM4/xTTabTXa7PWwBAAA9U6cGmNTUVLlcLpWWllrrgsGg9u7dK7fbLUlyu92qr69XRUWFVbN9+3a1trZq3LhxVs2uXbvU3Nxs1ZSUlOiqq67SZZdd1pktAwAAA7U7wJw4cUKVlZWqrKyU/nLhbmVlpWpqahQVFaXMzEw98cQTeuutt3Tw4EHde++9Sk5O1p133ilJGjFihH70ox/p/vvv1759+/T73/9e8+fP17Rp05ScnCxJ+tnPfqa4uDjNmTNHhw8f1uuvv67ly5crOzu7s+cPAAAM1O7bqN977z1NnDjRet0WKmbOnKnCwkItXLhQJ0+e1AMPPKD6+nrdeOON2rp1q3r37m1ts27dOs2fP1+33nqroqOjlZGRoRUrVljjDodDb7/9tnw+n8aMGaPLL79ceXl53EINAAAkSVGhUCgU6Sa6QjAYlMPhUENDQ6dfDzP0keJO3R8QaZ8u8Ua6BQAdFKnPpK76vXG+n9895i4kAABw6SDAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOPERroBAJE39JHiiBz30yXeiBwXgPk4AwMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMExvpBgBcuoY+UhyR4366xBuR4wLoPJyBAQAAxrmoA8yLL76ooUOHqnfv3ho3bpz27dsX6ZYAAMBF4KINMK+//rqys7P185//XAcOHNC1114rj8ejurq6SLcGAAAi7KINMM8++6zuv/9+zZo1S2lpaSooKFCfPn306quvRro1AAAQYRflRbxNTU2qqKhQbm6utS46Olrp6ekqLy8/6zaNjY1qbGy0Xjc0NEiSgsFgp/fX2vjnTt8ngO4zOGtjRI576BeeiBwXPVukPpO64vP1zP2GQqHvrLsoA8z//M//qKWlRU6nM2y90+nUkSNHzrpNfn6+fvGLX3xrfUpKSpf1CQDt4Xgu0h0Anaerf56PHz8uh8NxzvGLMsB0RG5urrKzs63Xra2t+vLLL5WYmKioqKiI9tbZgsGgUlJSdOzYMdnt9ki30+2Y/6U9f/EeXPLzF+9Bj55/KBTS8ePHlZyc/J11F2WAufzyyxUTE6NAIBC2PhAIyOVynXUbm80mm80Wti4hIaFL+4w0u93e435w24P5X9rzF+/BJT9/8R702Pl/15mXNhflRbxxcXEaM2aMSktLrXWtra0qLS2V2+2OaG8AACDyLsozMJKUnZ2tmTNnauzYsfrbv/1bPffcczp58qRmzZoV6dYAAECEXbQB5qc//am++OIL5eXlye/367rrrtPWrVu/dWHvpchms+nnP//5t/5kdqlg/pf2/MV7cMnPX7wHl/z8JSkq9H33KQEAAFxkLsprYAAAAL4LAQYAABiHAAMAAIxDgAEAAMYhwBgiPz9fN9xwg/r376+kpCTdeeedqqqqinRbEbNkyRJFRUUpMzMz0q10qz/96U+65557lJiYqPj4eI0aNUrvvfdepNvqFi0tLXr00UeVmpqq+Ph4/fVf/7Uef/zx7/2+FJPt2rVLt99+u5KTkxUVFaU333wzbDwUCikvL08DBw5UfHy80tPTdfTo0Yj129m+a/7Nzc3KycnRqFGj1LdvXyUnJ+vee+9VbW1tRHvubN/3M3CmuXPnKioqSs89d2l8ZwUBxhBlZWXy+Xzas2ePSkpK1NzcrEmTJunkyZORbq3b7d+/X7/+9a91zTXXRLqVbvXVV19pwoQJ6tWrl7Zs2aIPPvhAzzzzjC677LJIt9YtnnrqKa1atUovvPCCPvzwQz311FNaunSpnn/++Ui31mVOnjypa6+9Vi+++OJZx5cuXaoVK1aooKBAe/fuVd++feXxeHTq1Klu77UrfNf8//znP+vAgQN69NFHdeDAAb3xxhuqqqrSP/7jP0ak167yfT8DbTZt2qQ9e/Z87+P3e5QQjFRXVxeSFCorK4t0K93q+PHjoSuvvDJUUlIS+ru/+7vQww8/HOmWuk1OTk7oxhtvjHQbEeP1ekOzZ88OWzdlypTQ9OnTI9ZTd5IU2rRpk/W6tbU15HK5Qk8//bS1rr6+PmSz2UL/8R//EaEuu8435382+/btC0kKffbZZ93WV3c613vwxz/+MfRXf/VXoUOHDoWGDBkSWrZsWUT6626cgTFUQ0ODJGnAgAGRbqVb+Xw+eb1epaenR7qVbvfWW29p7Nixuuuuu5SUlKTrr79eL730UqTb6jY//OEPVVpaqo8++kiS9P777+vdd9/V5MmTI91aRFRXV8vv94f9W3A4HBo3bpzKy8sj2lukNDQ0KCoqqsd/D96ZWltbNWPGDC1YsEAjR46MdDvd6qJ9Ei/OrbW1VZmZmZowYYKuvvrqSLfTbV577TUdOHBA+/fvj3QrEfHJJ59o1apVys7O1r/9279p//79euihhxQXF6eZM2dGur0u98gjjygYDGr48OGKiYlRS0uLfvnLX2r69OmRbi0i/H6/JH3r6eROp9Mau5ScOnVKOTk5uvvuu3vklxuey1NPPaXY2Fg99NBDkW6l2xFgDOTz+XTo0CG9++67kW6l2xw7dkwPP/ywSkpK1Lt370i3ExGtra0aO3asnnzySUnS9ddfr0OHDqmgoOCSCDAbNmzQunXrtH79eo0cOVKVlZXKzMxUcnLyJTF/nFtzc7N+8pOfKBQKadWqVZFup9tUVFRo+fLlOnDggKKioiLdTrfjT0iGmT9/voqKirRjxw4NGjQo0u10m4qKCtXV1Wn06NGKjY1VbGysysrKtGLFCsXGxqqlpSXSLXa5gQMHKi0tLWzdiBEjVFNTE7GeutOCBQv0yCOPaNq0aRo1apRmzJihrKws5efnR7q1iHC5XJKkQCAQtj4QCFhjl4K28PLZZ5+ppKTkkjr78p//+Z+qq6vT4MGDrd+Ln332mf7lX/5FQ4cOjXR7XY4zMIYIhUJ68MEHtWnTJu3cuVOpqamRbqlb3XrrrTp48GDYulmzZmn48OHKyclRTExMxHrrLhMmTPjWrfMfffSRhgwZErGeutOf//xnRUeH/z9XTEyMWltbI9ZTJKWmpsrlcqm0tFTXXXedJCkYDGrv3r2aN29epNvrFm3h5ejRo9qxY4cSExMj3VK3mjFjxreuB/R4PJoxY4ZmzZoVsb66CwHGED6fT+vXr9fvfvc79e/f3/obt8PhUHx8fKTb63L9+/f/1vU+ffv2VWJi4iVzHVBWVpZ++MMf6sknn9RPfvIT7du3T6tXr9bq1asj3Vq3uP322/XLX/5SgwcP1siRI/Vf//VfevbZZzV79uxIt9ZlTpw4oY8//th6XV1drcrKSg0YMECDBw9WZmamnnjiCV155ZVKTU3Vo48+quTkZN15550R7buzfNf8Bw4cqKlTp+rAgQMqKipSS0uL9XtxwIABiouLi2Dnnef7fga+Gdp69eoll8ulq666KgLddrNI3waF8yPprMuaNWsi3VrEXGq3UYdCodDmzZtDV199dchms4WGDx8eWr16daRb6jbBYDD08MMPhwYPHhzq3bt36Ac/+EHo3//930ONjY2Rbq3L7Nix46z/7mfOnBkK/eVW6kcffTTkdDpDNpstdOutt4aqqqoi3Xan+a75V1dXn/P34o4dOyLdeqf5vp+Bb7qUbqOOCvXkx1gCAIAeiYt4AQCAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADDO/wPP0WtNPThbKQAAAABJRU5ErkJggg==", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkcAAAGwCAYAAACjPMHLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAzDUlEQVR4nO3df1zV9d3/8ScIHPAHh9Q4yESl2RJM87ee2Vom89So1WRbbmYutS690AKuS41ramY/MDd/FslcJu5aXmXXbbqUEhFTM/FHGOWPZLUsXAZsMzhpCQjn+8d3fK73SWuKwAfkcb/dPreb5/1+nfd5vdWOzz7n8+EE+Hw+nwAAACBJCrS7AQAAgJaEcAQAAGAgHAEAABgIRwAAAAbCEQAAgIFwBAAAYCAcAQAAGILsbqA1qKur08mTJ9WpUycFBATY3Q4AALgIPp9Pn3/+uaKjoxUYePHngwhHF+HkyZOKiYmxuw0AANAAJ06cUPfu3S+6nnB0ETp16iT98zc3PDzc7nYAAMBF8Hq9iomJsf4dv1iEo4tQ/1FaeHg44QgAgFbmUi+J4YJsAAAAA+EIAADAQDgCAAAwEI4AAAAMhCMAAAAD4QgAAMBAOAIAADAQjgAAAAyEIwAAAAPhCAAAwEA4AgAAMBCOAAAADIQjAAAAA+EIAADAQDgCAAAwBNndAJpfr4dzmmztjxYmNtnaAAA0B84cAQAAGAhHAAAABsIRAACAgXAEAABgIBwBAAAYCEcAAAAGwhEAAICBcAQAAGAgHAEAABhsDUe9evVSQEDAeUdycrIk6ezZs0pOTlaXLl3UsWNHJSUlqayszG+NkpISJSYmqn379oqMjNTMmTN17tw5v5odO3Zo0KBBcjgc6t27t7Kzs5t1nwAAoPWwNRwdOHBAn376qXXk5eVJkn76059KklJTU7Vp0ya9/PLL2rlzp06ePKmxY8daz6+trVViYqKqq6u1Z88erV27VtnZ2Zo3b55Vc/z4cSUmJmrUqFEqKipSSkqKpkyZotzcXBt2DAAAWroAn8/ns7uJeikpKdq8ebPef/99eb1eXX311Vq3bp1+8pOfSJKOHTumuLg4FRQUaMSIEXrttdd0++236+TJk3K5XJKkrKwszZ49W3/7298UEhKi2bNnKycnR4cPH7ZeZ9y4caqoqNCWLVsu2EdVVZWqqqqsx16vVzExMaqsrFR4eHiT/z40Nb5bDQDQFni9Xjmdzkv+97vFfPFsdXW1/vCHPygtLU0BAQEqLCxUTU2NEhISrJo+ffqoR48eVjgqKChQv379rGAkSR6PR9OmTdORI0c0cOBAFRQU+K1RX5OSkvK1vWRkZOjRRx9top0CANA6tZX/uW4xF2Rv3LhRFRUV+uUvfylJKi0tVUhIiCIiIvzqXC6XSktLrRozGNXP1899U43X69WXX355wV7S09NVWVlpHSdOnGjEnQIAgJasxZw5Wr16tW677TZFR0fb3YocDoccDofdbQAAABu0iDNHH3/8sbZt26YpU6ZYY1FRUaqurlZFRYVfbVlZmaKioqyar969Vv/4X9WEh4crLCysyfYEAABapxYRjtasWaPIyEglJv7f542DBw9WcHCw8vPzrbHi4mKVlJTI7XZLktxutw4dOqTy8nKrJi8vT+Hh4YqPj7dqzDXqa+rXAAAAMNkejurq6rRmzRpNnDhRQUH/9ymf0+nU5MmTlZaWptdff12FhYW677775Ha7NWLECEnSmDFjFB8frwkTJuidd95Rbm6u5syZo+TkZOtjsalTp+rDDz/UrFmzdOzYMT377LNav369UlNTbdszAABouWy/5mjbtm0qKSnRpEmTzptbunSpAgMDlZSUpKqqKnk8Hj377LPWfLt27bR582ZNmzZNbrdbHTp00MSJE7VgwQKrJjY2Vjk5OUpNTdXy5cvVvXt3Pffcc/J4PM22RwAA0Hq0qJ9z1FI19OcktFRt5VZMAEDjam3/fjT032/bP1YDAABoSQhHAAAABsIRAACAgXAEAABgIBwBAAAYCEcAAAAGwhEAAICBcAQAAGAgHAEAABgIRwAAAAbCEQAAgMH2L55F29Davo8HANB2ceYIAADAQDgCAAAwEI4AAAAMhCMAAAAD4QgAAMBAOAIAADAQjgAAAAyEIwAAAAPhCAAAwEA4AgAAMBCOAAAADIQjAAAAA+EIAADAQDgCAAAwEI4AAAAMhCMAAAAD4QgAAMBAOAIAADAQjgAAAAyEIwAAAAPhCAAAwEA4AgAAMBCOAAAADIQjAAAAA+EIAADAYHs4+uSTT3TPPfeoS5cuCgsLU79+/fTWW29Z8z6fT/PmzVO3bt0UFhamhIQEvf/++35rnDp1SuPHj1d4eLgiIiI0efJknT592q/m3Xff1fe+9z2FhoYqJiZGixYtarY9AgCA1sPWcPTZZ59p5MiRCg4O1muvvaajR49q8eLFuuqqq6yaRYsWacWKFcrKytK+ffvUoUMHeTwenT171qoZP368jhw5ory8PG3evFm7du3SAw88YM17vV6NGTNGPXv2VGFhoX79619r/vz5WrVqVbPvGQAAtGxBdr74U089pZiYGK1Zs8Yai42NtX7t8/m0bNkyzZkzR3feeack6fe//71cLpc2btyocePG6b333tOWLVt04MABDRkyRJL09NNP64c//KF+85vfKDo6Wi+88IKqq6v1/PPPKyQkRH379lVRUZGWLFniF6IAAABsPXP0yiuvaMiQIfrpT3+qyMhIDRw4UL/73e+s+ePHj6u0tFQJCQnWmNPp1PDhw1VQUCBJKigoUEREhBWMJCkhIUGBgYHat2+fVXPTTTcpJCTEqvF4PCouLtZnn312Xl9VVVXyer1+BwAAaBtsDUcffvihVq5cqWuvvVa5ubmaNm2aHnzwQa1du1aSVFpaKklyuVx+z3O5XNZcaWmpIiMj/eaDgoLUuXNnv5oLrWG+hikjI0NOp9M6YmJiGnXfAACg5bI1HNXV1WnQoEF68sknNXDgQD3wwAO6//77lZWVZWdbSk9PV2VlpXWcOHHC1n4AAEDzsTUcdevWTfHx8X5jcXFxKikpkSRFRUVJksrKyvxqysrKrLmoqCiVl5f7zZ87d06nTp3yq7nQGuZrmBwOh8LDw/0OAADQNtgajkaOHKni4mK/sT//+c/q2bOn9M+Ls6OiopSfn2/Ne71e7du3T263W5LkdrtVUVGhwsJCq2b79u2qq6vT8OHDrZpdu3appqbGqsnLy9N1113nd2ccAACAreEoNTVVe/fu1ZNPPqkPPvhA69at06pVq5ScnCxJCggIUEpKih5//HG98sorOnTokO69915FR0frrrvukv55punWW2/V/fffr/379+vNN9/U9OnTNW7cOEVHR0uSfvGLXygkJESTJ0/WkSNH9NJLL2n58uVKS0uzc/sAAKAFsvVW/qFDh2rDhg1KT0/XggULFBsbq2XLlmn8+PFWzaxZs3TmzBk98MADqqio0I033qgtW7YoNDTUqnnhhRc0ffp0jR49WoGBgUpKStKKFSuseafTqa1btyo5OVmDBw9W165dNW/ePG7jBwAA5wnw+Xw+u5to6bxer5xOpyorK6+I6496PZzTZGt/tDCxxbwmAKBxtbb38ob++23714cAAAC0JIQjAAAAA+EIAADAQDgCAAAwEI4AAAAMhCMAAAAD4QgAAMBAOAIAADAQjgAAAAyEIwAAAAPhCAAAwEA4AgAAMBCOAAAADIQjAAAAA+EIAADAQDgCAAAwEI4AAAAMhCMAAAAD4QgAAMBAOAIAADAQjgAAAAyEIwAAAAPhCAAAwEA4AgAAMBCOAAAADIQjAAAAA+EIAADAQDgCAAAwEI4AAAAMhCMAAAAD4QgAAMBAOAIAADAQjgAAAAyEIwAAAAPhCAAAwEA4AgAAMBCOAAAADLaGo/nz5ysgIMDv6NOnjzV/9uxZJScnq0uXLurYsaOSkpJUVlbmt0ZJSYkSExPVvn17RUZGaubMmTp37pxfzY4dOzRo0CA5HA717t1b2dnZzbZHAADQuth+5qhv37769NNPrWP37t3WXGpqqjZt2qSXX35ZO3fu1MmTJzV27Fhrvra2VomJiaqurtaePXu0du1aZWdna968eVbN8ePHlZiYqFGjRqmoqEgpKSmaMmWKcnNzm32vAACg5QuyvYGgIEVFRZ03XllZqdWrV2vdunW65ZZbJElr1qxRXFyc9u7dqxEjRmjr1q06evSotm3bJpfLpQEDBuixxx7T7NmzNX/+fIWEhCgrK0uxsbFavHixJCkuLk67d+/W0qVL5fF4mn2/AACgZbP9zNH777+v6OhoXXPNNRo/frxKSkokSYWFhaqpqVFCQoJV26dPH/Xo0UMFBQWSpIKCAvXr108ul8uq8Xg88nq9OnLkiFVjrlFfU7/GhVRVVcnr9fodAACgbbA1HA0fPlzZ2dnasmWLVq5cqePHj+t73/uePv/8c5WWliokJEQRERF+z3G5XCotLZUklZaW+gWj+vn6uW+q8Xq9+vLLLy/YV0ZGhpxOp3XExMQ06r4BAEDLZevHarfddpv16/79+2v48OHq2bOn1q9fr7CwMNv6Sk9PV1pamvXY6/USkAAAaCNs/1jNFBERoe985zv64IMPFBUVperqalVUVPjVlJWVWdcoRUVFnXf3Wv3jf1UTHh7+tQHM4XAoPDzc7wAAAG1DiwpHp0+f1l/+8hd169ZNgwcPVnBwsPLz86354uJilZSUyO12S5LcbrcOHTqk8vJyqyYvL0/h4eGKj4+3asw16mvq1wAAADDZGo7+8z//Uzt37tRHH32kPXv26Mc//rHatWunn//853I6nZo8ebLS0tL0+uuvq7CwUPfdd5/cbrdGjBghSRozZozi4+M1YcIEvfPOO8rNzdWcOXOUnJwsh8MhSZo6dao+/PBDzZo1S8eOHdOzzz6r9evXKzU11c6tAwCAFsrWa47++te/6uc//7n+8Y9/6Oqrr9aNN96ovXv36uqrr5YkLV26VIGBgUpKSlJVVZU8Ho+effZZ6/nt2rXT5s2bNW3aNLndbnXo0EETJ07UggULrJrY2Fjl5OQoNTVVy5cvV/fu3fXcc89xGz8AALggW8PRiy+++I3zoaGhyszMVGZm5tfW9OzZU6+++uo3rnPzzTfr7bffbnCfAACg7WhR1xwBAADYjXAEAABgIBwBAAAYCEcAAAAGwhEAAICBcAQAAGAgHAEAABgIRwAAAAbCEQAAgIFwBAAAYCAcAQAAGAhHAAAABsIRAACAgXAEAABgIBwBAAAYCEcAAAAGwhEAAICBcAQAAGAgHAEAABgIRwAAAIYguxsAmkqvh3OaZN2PFiY2yboAgJaBM0cAAAAGwhEAAICBcAQAAGAgHAEAABgIRwAAAAbCEQAAgIFwBAAAYCAcAQAAGAhHAAAABsIRAACAgXAEAABgaFA4uuaaa/SPf/zjvPGKigpdc801jdEXAACALRoUjj766CPV1taeN15VVaVPPvmkMfoCAACwRdClFL/yyivWr3Nzc+V0Oq3HtbW1ys/PV69evRq3QwAAgGZ0SeHorrvukiQFBARo4sSJfnPBwcHq1auXFi9e3LgdAgAANKNLCkd1dXWSpNjYWB04cEBdu3Ztqr4AAABscUnhqN7x48cbvxMAAIAWoMG38ufn5+u//uu/NGXKFE2aNMnvaIiFCxcqICBAKSkp1tjZs2eVnJysLl26qGPHjkpKSlJZWZnf80pKSpSYmKj27dsrMjJSM2fO1Llz5/xqduzYoUGDBsnhcKh3797Kzs5u4K4BAMCVrkHh6NFHH9WYMWOUn5+vv//97/rss8/8jkt14MAB/fa3v1X//v39xlNTU7Vp0ya9/PLL2rlzp06ePKmxY8da87W1tUpMTFR1dbX27NmjtWvXKjs7W/PmzbNqjh8/rsTERI0aNUpFRUVKSUnRlClTlJub25CtAwCAK1yDPlbLyspSdna2JkyYcNkNnD59WuPHj9fvfvc7Pf7449Z4ZWWlVq9erXXr1umWW26RJK1Zs0ZxcXHau3evRowYoa1bt+ro0aPatm2bXC6XBgwYoMcee0yzZ8/W/PnzFRISoqysLMXGxloXisfFxWn37t1aunSpPB7PBXuqqqpSVVWV9djr9V72PgEAQOvQoDNH1dXV+u53v9soDSQnJysxMVEJCQl+44WFhaqpqfEb79Onj3r06KGCggJJUkFBgfr16yeXy2XVeDweeb1eHTlyxKr56toej8da40IyMjLkdDqtIyYmplH2CgAAWr4GhaMpU6Zo3bp1l/3iL774og4ePKiMjIzz5kpLSxUSEqKIiAi/cZfLpdLSUqvGDEb18/Vz31Tj9Xr15ZdfXrCv9PR0VVZWWseJEycuc6cAAKC1aNDHamfPntWqVau0bds29e/fX8HBwX7zS5Ys+ZdrnDhxQg899JDy8vIUGhrakDaajMPhkMPhsLsNAABggwaFo3fffVcDBgyQJB0+fNhvLiAg4KLWKCwsVHl5uQYNGmSN1dbWateuXXrmmWeUm5ur6upqVVRU+J09KisrU1RUlCQpKipK+/fv91u3/m42s+ard7iVlZUpPDxcYWFhl7hzAABwpWtQOHr99dcv+4VHjx6tQ4cO+Y3dd9996tOnj2bPnq2YmBgFBwcrPz9fSUlJkqTi4mKVlJTI7XZLktxut5544gmVl5crMjJSkpSXl6fw8HDFx8dbNa+++qrf6+Tl5VlrAAAAmBoUjhpDp06ddP311/uNdejQQV26dLHGJ0+erLS0NHXu3Fnh4eGaMWOG3G63RowYIUkaM2aM4uPjNWHCBC1atEilpaWaM2eOkpOTrY/Fpk6dqmeeeUazZs3SpEmTtH37dq1fv145OTk27BoAALR0DQpHo0aN+saPz7Zv3345PVmWLl2qwMBAJSUlqaqqSh6PR88++6w1365dO23evFnTpk2T2+1Whw4dNHHiRC1YsMCqiY2NVU5OjlJTU7V8+XJ1795dzz333Nfexg8AANq2BoWj+uuN6tXU1KioqEiHDx8+7wtpL8WOHTv8HoeGhiozM1OZmZlf+5yePXue97HZV9188816++23G9wXAABoOxoUjpYuXXrB8fnz5+v06dOX2xMAAIBtGvzdahdyzz336Pnnn2/MJQEAAJpVo4ajgoKCFvcziwAAAC5Fgz5WM7/8VZJ8Pp8+/fRTvfXWW5o7d25j9QYAANDsGhSOnE6n3+PAwEBdd911WrBggcaMGdNYvQEAADS7BoWjNWvWNH4nAAAALcBl/RDIwsJCvffee5Kkvn37auDAgY3VFwAAgC0aFI7Ky8s1btw47dixw/res4qKCo0aNUovvviirr766sbuEwAAoFk06G61GTNm6PPPP9eRI0d06tQpnTp1SocPH5bX69WDDz7Y+F0CAAA0kwadOdqyZYu2bdumuLg4ayw+Pl6ZmZlckA0AAFq1Bp05qqurU3Bw8HnjwcHBqqura4y+AAAAbNGgcHTLLbfooYce0smTJ62xTz75RKmpqRo9enRj9gcAANCsGhSOnnnmGXm9XvXq1Uvf/va39e1vf1uxsbHyer16+umnG79LAACAZtKga45iYmJ08OBBbdu2TceOHZMkxcXFKSEhobH7AwAAaFaXdOZo+/btio+Pl9frVUBAgH7wgx9oxowZmjFjhoYOHaq+ffvqjTfeaLpuAQAAmtglhaNly5bp/vvvV3h4+HlzTqdT//Zv/6YlS5Y0Zn8AAADN6pLC0TvvvKNbb731a+fHjBmjwsLCxugLAADAFpcUjsrKyi54C3+9oKAg/e1vf2uMvgAAAGxxSeHoW9/6lg4fPvy18++++666devWGH0BAADY4pLC0Q9/+EPNnTtXZ8+ePW/uyy+/1COPPKLbb7+9MfsDAABoVpd0K/+cOXP0xz/+Ud/5znc0ffp0XXfddZKkY8eOKTMzU7W1tfrVr37VVL0CAAA0uUsKRy6XS3v27NG0adOUnp4un88nSQoICJDH41FmZqZcLldT9QoAANDkLvmHQPbs2VOvvvqqPvvsM33wwQfy+Xy69tprddVVVzVNhwAAAM2oQT8hW5KuuuoqDR06tHG7AQAAsFmDvlsNAADgSkU4AgAAMBCOAAAADIQjAAAAA+EIAADAQDgCAAAwEI4AAAAMhCMAAAAD4QgAAMBAOAIAADAQjgAAAAyEIwAAAAPhCAAAwGBrOFq5cqX69++v8PBwhYeHy+1267XXXrPmz549q+TkZHXp0kUdO3ZUUlKSysrK/NYoKSlRYmKi2rdvr8jISM2cOVPnzp3zq9mxY4cGDRokh8Oh3r17Kzs7u9n2CAAAWhdbw1H37t21cOFCFRYW6q233tItt9yiO++8U0eOHJEkpaamatOmTXr55Ze1c+dOnTx5UmPHjrWeX1tbq8TERFVXV2vPnj1au3atsrOzNW/ePKvm+PHjSkxM1KhRo1RUVKSUlBRNmTJFubm5tuwZAAC0bEF2vvgdd9zh9/iJJ57QypUrtXfvXnXv3l2rV6/WunXrdMstt0iS1qxZo7i4OO3du1cjRozQ1q1bdfToUW3btk0ul0sDBgzQY489ptmzZ2v+/PkKCQlRVlaWYmNjtXjxYklSXFycdu/eraVLl8rj8diybwAA0HK1mGuOamtr9eKLL+rMmTNyu90qLCxUTU2NEhISrJo+ffqoR48eKigokCQVFBSoX79+crlcVo3H45HX67XOPhUUFPitUV9Tv8aFVFVVyev1+h0AAKBtsD0cHTp0SB07dpTD4dDUqVO1YcMGxcfHq7S0VCEhIYqIiPCrd7lcKi0tlSSVlpb6BaP6+fq5b6rxer368ssvL9hTRkaGnE6ndcTExDTqngEAQMtlezi67rrrVFRUpH379mnatGmaOHGijh49amtP6enpqqystI4TJ07Y2g8AAGg+tl5zJEkhISHq3bu3JGnw4ME6cOCAli9frrvvvlvV1dWqqKjwO3tUVlamqKgoSVJUVJT279/vt1793WxmzVfvcCsrK1N4eLjCwsIu2JPD4ZDD4WjknQIAgNbA9jNHX1VXV6eqqioNHjxYwcHBys/Pt+aKi4tVUlIit9stSXK73Tp06JDKy8utmry8PIWHhys+Pt6qMdeor6lfAwAAwGTrmaP09HTddttt6tGjhz7//HOtW7dOO3bsUG5urpxOpyZPnqy0tDR17txZ4eHhmjFjhtxut0aMGCFJGjNmjOLj4zVhwgQtWrRIpaWlmjNnjpKTk60zP1OnTtUzzzyjWbNmadKkSdq+fbvWr1+vnJwcO7eOK1Cvh5vu79RHCxObbG0AgD9bw1F5ebnuvfdeffrpp3I6nerfv79yc3P1gx/8QJK0dOlSBQYGKikpSVVVVfJ4PHr22Wet57dr106bN2/WtGnT5Ha71aFDB02cOFELFiywamJjY5WTk6PU1FQtX75c3bt313PPPcdt/AAA4IJsDUerV6/+xvnQ0FBlZmYqMzPza2t69uypV1999RvXufnmm/X22283uE8AANB2tLhrjgAAAOxEOAIAADAQjgAAAAyEIwAAAAPhCAAAwEA4AgAAMBCOAAAADIQjAAAAA+EIAADAQDgCAAAwEI4AAAAMhCMAAAAD4QgAAMBAOAIAADAQjgAAAAyEIwAAAAPhCAAAwEA4AgAAMBCOAAAADIQjAAAAA+EIAADAQDgCAAAwEI4AAAAMhCMAAAAD4QgAAMAQZHcDABqu18M5TbLuRwsTm2RdAGgNOHMEAABgIBwBAAAYCEcAAAAGwhEAAICBcAQAAGAgHAEAABgIRwAAAAbCEQAAgIFwBAAAYCAcAQAAGAhHAAAABlvDUUZGhoYOHapOnTopMjJSd911l4qLi/1qzp49q+TkZHXp0kUdO3ZUUlKSysrK/GpKSkqUmJio9u3bKzIyUjNnztS5c+f8anbs2KFBgwbJ4XCod+/eys7ObpY9AgCA1sXWcLRz504lJydr7969ysvLU01NjcaMGaMzZ85YNampqdq0aZNefvll7dy5UydPntTYsWOt+draWiUmJqq6ulp79uzR2rVrlZ2drXnz5lk1x48fV2JiokaNGqWioiKlpKRoypQpys3NbfY9AwCAli3IzhffsmWL3+Ps7GxFRkaqsLBQN910kyorK7V69WqtW7dOt9xyiyRpzZo1iouL0969ezVixAht3bpVR48e1bZt2+RyuTRgwAA99thjmj17tubPn6+QkBBlZWUpNjZWixcvliTFxcVp9+7dWrp0qTwejy17BwAALVOLuuaosrJSktS5c2dJUmFhoWpqapSQkGDV9OnTRz169FBBQYEkqaCgQP369ZPL5bJqPB6PvF6vjhw5YtWYa9TX1K/xVVVVVfJ6vX4HAABoG1pMOKqrq1NKSopGjhyp66+/XpJUWlqqkJAQRURE+NW6XC6VlpZaNWYwqp+vn/umGq/Xqy+//PK8XjIyMuR0Oq0jJiamkXcLAABaqhYTjpKTk3X48GG9+OKLdrei9PR0VVZWWseJEyfsbgkAADQTW685qjd9+nRt3rxZu3btUvfu3a3xqKgoVVdXq6Kiwu/sUVlZmaKioqya/fv3+61XfzebWfPVO9zKysoUHh6usLCw8/pxOBxyOByNvEsAANAa2HrmyOfzafr06dqwYYO2b9+u2NhYv/nBgwcrODhY+fn51lhxcbFKSkrkdrslSW63W4cOHVJ5eblVk5eXp/DwcMXHx1s15hr1NfVrAAAA1LP1zFFycrLWrVunP/3pT+rUqZN1jZDT6VRYWJicTqcmT56stLQ0de7cWeHh4ZoxY4bcbrdGjBghSRozZozi4+M1YcIELVq0SKWlpZozZ46Sk5Otsz9Tp07VM888o1mzZmnSpEnavn271q9fr5ycHDu3DwAAWiBbzxytXLlSlZWVuvnmm9WtWzfreOmll6yapUuX6vbbb1dSUpJuuukmRUVF6Y9//KM1365dO23evFnt2rWT2+3WPffco3vvvVcLFiywamJjY5WTk6O8vDzdcMMNWrx4sZ577jlu4wcAAOex9cyRz+f7lzWhoaHKzMxUZmbm19b07NlTr7766jeuc/PNN+vtt99uUJ8AAKDtaDF3qwEAALQEhCMAAAAD4QgAAMBAOAIAADAQjgAAAAyEIwAAAAPhCAAAwEA4AgAAMBCOAAAADIQjAAAAA+EIAADAQDgCAAAwEI4AAAAMQXY3AKD16PVwTpOt/dHCxCZbGwAuBWeOAAAADIQjAAAAA+EIAADAQDgCAAAwEI4AAAAMhCMAAAAD4QgAAMBAOAIAADAQjgAAAAyEIwAAAAPhCAAAwEA4AgAAMBCOAAAADIQjAAAAA+EIAADAQDgCAAAwEI4AAAAMhCMAAAAD4QgAAMBAOAIAADAQjgAAAAyEIwAAAAPhCAAAwEA4AgAAMNgajnbt2qU77rhD0dHRCggI0MaNG/3mfT6f5s2bp27duiksLEwJCQl6//33/WpOnTql8ePHKzw8XBEREZo8ebJOnz7tV/Puu+/qe9/7nkJDQxUTE6NFixY1y/4AAEDrY2s4OnPmjG644QZlZmZecH7RokVasWKFsrKytG/fPnXo0EEej0dnz561asaPH68jR44oLy9Pmzdv1q5du/TAAw9Y816vV2PGjFHPnj1VWFioX//615o/f75WrVrVLHsEAACtS5CdL37bbbfptttuu+Ccz+fTsmXLNGfOHN15552SpN///vdyuVzauHGjxo0bp/fee09btmzRgQMHNGTIEEnS008/rR/+8If6zW9+o+joaL3wwguqrq7W888/r5CQEPXt21dFRUVasmSJX4gyVVVVqaqqynrs9XqbZP8AAKDlabHXHB0/flylpaVKSEiwxpxOp4YPH66CggJJUkFBgSIiIqxgJEkJCQkKDAzUvn37rJqbbrpJISEhVo3H41FxcbE+++yzC752RkaGnE6ndcTExDThTgEAQEvSYsNRaWmpJMnlcvmNu1wua660tFSRkZF+80FBQercubNfzYXWMF/jq9LT01VZWWkdJ06caMSdAQCAlszWj9VaKofDIYfDYXcbAADABi32zFFUVJQkqayszG+8rKzMmouKilJ5ebnf/Llz53Tq1Cm/mgutYb4GAABAvRYbjmJjYxUVFaX8/HxrzOv1at++fXK73ZIkt9utiooKFRYWWjXbt29XXV2dhg8fbtXs2rVLNTU1Vk1eXp6uu+46XXXVVc26JwAA0PLZGo5Onz6toqIiFRUVSf+8CLuoqEglJSUKCAhQSkqKHn/8cb3yyis6dOiQ7r33XkVHR+uuu+6SJMXFxenWW2/V/fffr/379+vNN9/U9OnTNW7cOEVHR0uSfvGLXygkJESTJ0/WkSNH9NJLL2n58uVKS0uzc+sAAKCFsvWao7feekujRo2yHtcHlokTJyo7O1uzZs3SmTNn9MADD6iiokI33nijtmzZotDQUOs5L7zwgqZPn67Ro0crMDBQSUlJWrFihTXvdDq1detWJScna/DgweratavmzZv3tbfxAwCAts3WcHTzzTfL5/N97XxAQIAWLFigBQsWfG1N586dtW7dum98nf79++uNN964rF4BAEDb0GKvOQIAALAD4QgAAMBAOAIAADAQjgAAAAyEIwAAAAPhCAAAwEA4AgAAMBCOAAAADIQjAAAAA+EIAADAQDgCAAAwEI4AAAAMhCMAAAAD4QgAAMBAOAIAADAQjgAAAAyEIwAAAAPhCAAAwBBkdwMA8E16PZzTZGt/tDCxydYG0Hpx5ggAAMBAOAIAADAQjgAAAAyEIwAAAAPhCAAAwEA4AgAAMBCOAAAADIQjAAAAA+EIAADAQDgCAAAwEI4AAAAMhCMAAAAD4QgAAMBAOAIAADAQjgAAAAxBdjcAAC1Jr4dzmmztjxYmNtnaABoPZ44AAAAMhCMAAABDmwpHmZmZ6tWrl0JDQzV8+HDt37/f7pYAAEAL02bC0UsvvaS0tDQ98sgjOnjwoG644QZ5PB6Vl5fb3RoAAGhB2swF2UuWLNH999+v++67T5KUlZWlnJwcPf/883r44Yftbg9AG8ZF4EDL0ibCUXV1tQoLC5Wenm6NBQYGKiEhQQUFBefVV1VVqaqqynpcWVkpSfJ6vc3UcdOqq/qiydb+ut+jK+k12WPTaCmv2Rb2CDRUa/u7Wr+mz+e7tCf62oBPPvnEJ8m3Z88ev/GZM2f6hg0bdl79I4884pPEwcHBwcHBcQUcJ06cuKTc0CbOHF2q9PR0paWlWY/r6up06tQpdenSRQEBAbb15fV6FRMToxMnTig8PNy2PppSW9ij2sg+2eOVgT1eGdrqHn0+nz7//HNFR0df0lptIhx17dpV7dq1U1lZmd94WVmZoqKizqt3OBxyOBx+YxEREU3e58UKDw+/Yv9y12sLe1Qb2Sd7vDKwxytDW9yj0+m85DXaxN1qISEhGjx4sPLz862xuro65efny+1229obAABoWdrEmSNJSktL08SJEzVkyBANGzZMy5Yt05kzZ6y71wAAANSWwtHdd9+tv/3tb5o3b55KS0s1YMAAbdmyRS6Xy+7WLprD4dAjjzxy3kd+V5K2sEe1kX2yxysDe7wysMdLE+C75PvbAAAArlxt4pojAACAi0U4AgAAMBCOAAAADIQjAAAAA+GoFcnMzFSvXr0UGhqq4cOHa//+/Xa31GgyMjI0dOhQderUSZGRkbrrrrtUXFxsd1tNauHChQoICFBKSordrTSqTz75RPfcc4+6dOmisLAw9evXT2+99ZbdbTWa2tpazZ07V7GxsQoLC9O3v/1tPfbYY5f+3U0tyK5du3THHXcoOjpaAQEB2rhxo9+8z+fTvHnz1K1bN4WFhSkhIUHvv/++bf021Dfts6amRrNnz1a/fv3UoUMHRUdH695779XJkydt7flS/as/S9PUqVMVEBCgZcuWNWuPl+ti9vjee+/pRz/6kZxOpzp06KChQ4eqpKTkol+DcNRKvPTSS0pLS9MjjzyigwcP6oYbbpDH41F5ebndrTWKnTt3Kjk5WXv37lVeXp5qamo0ZswYnTlzxu7WmsSBAwf029/+Vv3797e7lUb12WefaeTIkQoODtZrr72mo0ePavHixbrqqqvsbq3RPPXUU1q5cqWeeeYZvffee3rqqae0aNEiPf3003a31mBnzpzRDTfcoMzMzAvOL1q0SCtWrFBWVpb27dunDh06yOPx6OzZs83e6+X4pn1+8cUXOnjwoObOnauDBw/qj3/8o4qLi/WjH/3Ill4b6l/9WdbbsGGD9u7de8lfq9ES/Ks9/uUvf9GNN96oPn36aMeOHXr33Xc1d+5chYaGXvyLXO6XuqJ5DBs2zJecnGw9rq2t9UVHR/syMjJs7auplJeX+yT5du7caXcrje7zzz/3XXvttb68vDzf97//fd9DDz1kd0uNZvbs2b4bb7zR7jaaVGJiom/SpEl+Y2PHjvWNHz/etp4akyTfhg0brMd1dXW+qKgo369//WtrrKKiwudwOHz/8z//Y1OXl++r+7yQ/fv3+yT5Pv7442brqzF93R7/+te/+r71rW/5Dh8+7OvZs6dv6dKltvTXGC60x7vvvtt3zz33XNa6nDlqBaqrq1VYWKiEhARrLDAwUAkJCSooKLC1t6ZSWVkpSercubPdrTS65ORkJSYm+v15XileeeUVDRkyRD/96U8VGRmpgQMH6ne/+53dbTWq7373u8rPz9ef//xnSdI777yj3bt367bbbrO7tSZx/PhxlZaW+v19dTqdGj58+BX7/lOvsrJSAQEBLeq7NS9XXV2dJkyYoJkzZ6pv3752t9Po6urqlJOTo+985zvyeDyKjIzU8OHDv/HjxQshHLUCf//731VbW3veT/N2uVwqLS21ra+mUldXp5SUFI0cOVLXX3+93e00qhdffFEHDx5URkaG3a00iQ8//FArV67Utddeq9zcXE2bNk0PPvig1q5da3drjebhhx/WuHHj1KdPHwUHB2vgwIFKSUnR+PHj7W6tSdS/x7SV9596Z8+e1ezZs/Xzn//8ivqi1qeeekpBQUF68MEH7W6lSZSXl+v06dNauHChbr31Vm3dulU//vGPNXbsWO3cufOi12kzXx+C1iM5OVmHDx/W7t277W6lUZ04cUIPPfSQ8vLyLu2z71akrq5OQ4YM0ZNPPilJGjhwoA4fPqysrCxNnDjR7vYaxfr16/XCCy9o3bp16tu3r4qKipSSkqLo6OgrZo9tXU1NjX72s5/J5/Np5cqVdrfTaAoLC7V8+XIdPHhQAQEBdrfTJOrq6iRJd955p1JTUyVJAwYM0J49e5SVlaXvf//7F7UOZ45aga5du6pdu3YqKyvzGy8rK1NUVJRtfTWF6dOna/PmzXr99dfVvXt3u9tpVIWFhSovL9egQYMUFBSkoKAg7dy5UytWrFBQUJBqa2vtbvGydevWTfHx8X5jcXFxl3SXSEs3c+ZM6+xRv379NGHCBKWmpl6xZwPr32PawvuPjGD08ccfKy8v74o6a/TGG2+ovLxcPXr0sN6DPv74Y/3Hf/yHevXqZXd7jaJr164KCgq67PchwlErEBISosGDBys/P98aq6urU35+vtxut629NRafz6fp06drw4YN2r59u2JjY+1uqdGNHj1ahw4dUlFRkXUMGTJE48ePV1FRkdq1a2d3i5dt5MiR5/0Ihj//+c/q2bOnbT01ti+++EKBgf5vne3atbP+j/VKExsbq6ioKL/3H6/Xq3379l0x7z/16oPR+++/r23btqlLly52t9SoJkyYoHfffdfvPSg6OlozZ85Ubm6u3e01ipCQEA0dOvSy34f4WK2VSEtL08SJEzVkyBANGzZMy5Yt05kzZ3TffffZ3VqjSE5O1rp16/SnP/1JnTp1sq5lcDqdCgsLs7u9RtGpU6fzrqHq0KGDunTpcsVcW5Wamqrvfve7evLJJ/Wzn/1M+/fv16pVq7Rq1Sq7W2s0d9xxh5544gn16NFDffv21dtvv60lS5Zo0qRJdrfWYKdPn9YHH3xgPT5+/LiKiorUuXNn9ejRQykpKXr88cd17bXXKjY2VnPnzlV0dLTuuusuW/u+VN+0z27duuknP/mJDh48qM2bN6u2ttZ6H+rcubNCQkJs7Pzi/as/y68GvuDgYEVFRem6666zoduG+Vd7nDlzpu6++27ddNNNGjVqlLZs2aJNmzZpx44dF/8il3kXHZrR008/7evRo4cvJCTEN2zYMN/evXvtbqnRSLrgsWbNGrtba1JX2q38Pp/Pt2nTJt/111/vczgcvj59+vhWrVpld0uNyuv1+h566CFfjx49fKGhob5rrrnG96tf/cpXVVVld2sN9vrrr1/wv7+JEyf6fP+8nX/u3Lk+l8vlczgcvtGjR/uKi4vtbvuSfdM+jx8//rXvQ6+//rrdrV+0f/Vn+VWt8Vb+i9nj6tWrfb179/aFhob6brjhBt/GjRsv6TUCfK35x7oCAAA0Mq45AgAAMBCOAAAADIQjAAAAA+EIAADAQDgCAAAwEI4AAAAMhCMAAAAD4QgAAMBAOALQZmVnZysiIuKy1wkICNDGjRsbpScA9iMcAWjVfvnLX7a67/gC0LIRjgAAAAyEIwBXrCVLlqhfv37q0KGDYmJi9O///u86ffr0eXUbN27Utddeq9DQUHk8Hp04ccJv/k9/+pMGDRqk0NBQXXPNNXr00Ud17ty5ZtwJgOZEOAJwxQoMDNSKFSt05MgRrV27Vtu3b9esWbP8ar744gs98cQT+v3vf68333xTFRUVGjdunDX/xhtv6N5779VDDz2ko0eP6re//a2ys7P1xBNP2LAjAM0hwOfz+exuAgAa6pe//KUqKiou6oLo//3f/9XUqVP197//XfrnBdn33Xef9u7dq+HDh0uSjh07pri4OO3bt0/Dhg1TQkKCRo8erfT0dGudP/zhD5o1a5ZOnjwp/fOC7A0bNnDtE3CFCLK7AQBoKtu2bVNGRoaOHTsmr9erc+fO6ezZs/riiy/Uvn17SVJQUJCGDh1qPadPnz6KiIjQe++9p2HDhumdd97Rm2++6XemqLa29rx1AFw5CEcArkgfffSRbr/9dk2bNk1PPPGEOnfurN27d2vy5Mmqrq6+6FBz+vRpPfrooxo7dux5c6GhoU3QOQC7EY4AXJEKCwtVV1enxYsXKzDw/19euX79+vPqzp07p7feekvDhg2TJBUXF6uiokJxcXGSpEGDBqm4uFi9e/du5h0AsAvhCECrV1lZqaKiIr+xrl27qqamRk8//bTuuOMOvfnmm8rKyjrvucHBwZoxY4ZWrFihoKAgTZ8+XSNGjLDC0rx583T77berR48e+slPfqLAwEC98847Onz4sB5//PFm2yOA5sPdagBavR07dmjgwIF+x3//939ryZIleuqpp3T99dfrhRdeUEZGxnnPbd++vWbPnq1f/OIXGjlypDp27KiXXnrJmvd4PNq8ebO2bt2qoUOHasSIEVq6dKl69uzZzLsE0Fy4Ww0AAMDAmSMAAAAD4QgAAMBAOAIAADAQjgAAAAyEIwAAAAPhCAAAwEA4AgAAMBCOAAAADIQjAAAAA+EIAADAQDgCAAAw/D9PxKffezrd3wAAAABJRU5ErkJggg==", "text/plain": [ "<Figure size 640x480 with 1 Axes>" ] @@ -255,32 +255,33 @@ } ], "source": [ - "plt.hist(mkbatch(2**15)[1].cpu(), bins=MAX_VTXS)\n", - "with open(\"train-dist.html\", \"w\") as f:\n", + "bins = torch.bincount(mkbatch(2**15)[1].to(torch.uint8).cpu())\n", + "plt.bar(range(len(bins)), bins)\n", + "plt.xlabel(\"Label\")\n", + "plt.ylabel(\"Count\")\n", + "with open(\"plots/train-dist.html\", \"w\") as f:\n", " mpld3.save_html(plt.gcf(), f)" ] }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 10, "execution_state": "idle", "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "(array([2.4747e+04, 6.5340e+03, 1.2840e+03, 1.8300e+02, 2.0000e+01]),\n", - " array([1., 2., 3., 4., 5., 6.]),\n", - " <BarContainer object of 5 artists>)" + "Text(0, 0.5, 'Count')" ] }, - "execution_count": 7, + "execution_count": 10, "metadata": {}, "output_type": "execute_result" }, { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjkAAAGdCAYAAADwjmIIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAkD0lEQVR4nO3dfVSUdf7/8Rc3DZgxQ94AspJabiJ5GxpOllvJcVRyY3N31VwXjerUATelvKE8aG2F2fGkHU3X2iPtObGpndUKEiVMWBPvMFaxdLP0YKuDlskIW6gwvz9+ca3zFSsUmfj4fJxznW3m+sw177n27PJsmLkI8Hq9XgEAABgm0N8DAAAAXAlEDgAAMBKRAwAAjETkAAAAIxE5AADASEQOAAAwEpEDAACMROQAAAAjBft7AH9qaGjQ0aNHFRYWpoCAAH+PAwAAfgKv16vTp08rOjpagYEXf7/mqo6co0ePKiYmxt9jAACAS3DkyBF17dr1ovuv6sgJCwuTvj9Jdrvd3+MAAICfwOPxKCYmxvo5fjFXdeQ0/orKbrcTOQAAtDE/9lETPngMAACMROQAAAAjETkAAMBIRA4AADASkQMAAIxE5AAAACMROQAAwEjNipzs7GwNHjxYYWFhioiIUHJysg4cOOCz5q677lJAQIDP9uijj/qsqaysVFJSkq699lpFRERoxowZOnfunM+azZs369Zbb1VISIh69uypnJycC+ZZunSpunfvrtDQUCUkJGjHjh3Ne/UAAMBYzYqc4uJipaWladu2bSosLNTZs2c1YsQI1dbW+qx7+OGHdezYMWtbsGCBta++vl5JSUk6c+aMtm7dqjfeeEM5OTnKysqy1hw6dEhJSUm6++67VV5ermnTpumhhx7Shg0brDWrVq1SRkaG5s6dq927d6t///5yuVw6fvz45Z0RAABghACv1+u91AefOHFCERERKi4u1rBhw6Tv38kZMGCAFi1a1ORj1q9fr3vvvVdHjx5VZGSkJGn58uWaNWuWTpw4IZvNplmzZik/P18VFRXW48aPH69Tp06poKBAkpSQkKDBgwdryZIl0vd/bDMmJkZTp07V7Nmzf9L8Ho9HDodD1dXVXPEYAIA24qf+/L6sz+RUV1dLkjp06OBz/5tvvqlOnTqpT58+yszM1H//+19rX2lpqfr27WsFjiS5XC55PB7t27fPWpOYmOhzTJfLpdLSUknSmTNnVFZW5rMmMDBQiYmJ1pqm1NXVyePx+GwAAMBMl/y3qxoaGjRt2jQNHTpUffr0se5/4IEH1K1bN0VHR2vPnj2aNWuWDhw4oH/84x+SJLfb7RM4kqzbbrf7B9d4PB59++23+uabb1RfX9/kmv3791905uzsbD3zzDOX+pIBAEAbcsmRk5aWpoqKCm3ZssXn/kceecT65759+6pLly4aPny4Pv/8c910002XN+1lyszMVEZGhnW78a+YAgAA81xS5KSnpysvL08lJSXq2rXrD65NSEiQJB08eFA33XSToqKiLvgWVFVVlSQpKirK+s/G+85fY7fb1a5dOwUFBSkoKKjJNY3HaEpISIhCQkKa+WoBAEBb1KzI8Xq9mjp1qtauXavNmzerR48eP/qY8vJySVKXLl0kSU6nU88//7yOHz+uiIgISVJhYaHsdrvi4uKsNe+//77PcQoLC+V0OiVJNptN8fHxKioqUnJysvT9r8+KioqUnp7enJd0xXSfne/vEa4Kh+cn+XsEAMDPVLMiJy0tTbm5uXrnnXcUFhZmfYbG4XCoXbt2+vzzz5Wbm6vRo0erY8eO2rNnj6ZPn65hw4apX79+kqQRI0YoLi5OkyZN0oIFC+R2uzVnzhylpaVZ77I8+uijWrJkiWbOnKkHH3xQmzZt0urVq5Wf/79wyMjIUEpKigYNGqTbbrtNixYtUm1traZMmdKyZwgAALRJzYqcZcuWSd9/Tfx8K1eu1OTJk2Wz2fTBBx9YwRETE6OxY8dqzpw51tqgoCDl5eXpsccek9PpVPv27ZWSkqJnn33WWtOjRw/l5+dr+vTpWrx4sbp27arXX39dLpfLWjNu3DidOHFCWVlZcrvdGjBggAoKCi74MDIAALg6XdZ1ctq6K3mdHH5d1Tr4dRUAXH1a5To5AAAAP1dEDgAAMBKRAwAAjETkAAAAIxE5AADASEQOAAAwEpEDAACMROQAAAAjETkAAMBIRA4AADASkQMAAIxE5AAAACMROQAAwEhEDgAAMBKRAwAAjETkAAAAIxE5AADASEQOAAAwEpEDAACMROQAAAAjETkAAMBIRA4AADASkQMAAIxE5AAAACMROQAAwEhEDgAAMBKRAwAAjETkAAAAIxE5AADASEQOAAAwEpEDAACMROQAAAAjETkAAMBIRA4AADASkQMAAIxE5AAAACMROQAAwEhEDgAAMBKRAwAAjETkAAAAIxE5AADASEQOAAAwEpEDAACMROQAAAAjETkAAMBIRA4AADASkQMAAIxE5AAAACMROQAAwEhEDgAAMBKRAwAAjETkAAAAIxE5AADASEQOAAAwEpEDAACMROQAAAAjETkAAMBIRA4AADASkQMAAIxE5AAAACM1K3Kys7M1ePBghYWFKSIiQsnJyTpw4IDPmu+++05paWnq2LGjrrvuOo0dO1ZVVVU+ayorK5WUlKRrr71WERERmjFjhs6dO+ezZvPmzbr11lsVEhKinj17Kicn54J5li5dqu7duys0NFQJCQnasWNH8149AAAwVrMip7i4WGlpadq2bZsKCwt19uxZjRgxQrW1tdaa6dOn67333tOaNWtUXFyso0eP6v7777f219fXKykpSWfOnNHWrVv1xhtvKCcnR1lZWdaaQ4cOKSkpSXfffbfKy8s1bdo0PfTQQ9qwYYO1ZtWqVcrIyNDcuXO1e/du9e/fXy6XS8ePH7/8swIAANq8AK/X673UB584cUIREREqLi7WsGHDVF1drc6dOys3N1e//e1vJUn79+9X7969VVpaqiFDhmj9+vW69957dfToUUVGRkqSli9frlmzZunEiROy2WyaNWuW8vPzVVFRYT3X+PHjderUKRUUFEiSEhISNHjwYC1ZskSS1NDQoJiYGE2dOlWzZ8/+SfN7PB45HA5VV1fLbrdf6mloUvfZ+S16PDTt8Pwkf48AAGhlP/Xn92V9Jqe6ulqS1KFDB0lSWVmZzp49q8TERGtNbGysbrjhBpWWlkqSSktL1bdvXytwJMnlcsnj8Wjfvn3WmvOP0bim8RhnzpxRWVmZz5rAwEAlJiZaa5pSV1cnj8fjswEAADNdcuQ0NDRo2rRpGjp0qPr06SNJcrvdstlsCg8P91kbGRkpt9ttrTk/cBr3N+77oTUej0fffvutvvrqK9XX1ze5pvEYTcnOzpbD4bC2mJiYS335AADgZ+6SIyctLU0VFRV66623WnaiKygzM1PV1dXWduTIEX+PBAAArpDgS3lQenq68vLyVFJSoq5du1r3R0VF6cyZMzp16pTPuzlVVVWKioqy1vzfb0E1fvvq/DX/9xtZVVVVstvtateunYKCghQUFNTkmsZjNCUkJEQhISGX8pIBAEAb06x3crxer9LT07V27Vpt2rRJPXr08NkfHx+va665RkVFRdZ9Bw4cUGVlpZxOpyTJ6XRq7969Pt+CKiwslN1uV1xcnLXm/GM0rmk8hs1mU3x8vM+ahoYGFRUVWWsAAMDVrVnv5KSlpSk3N1fvvPOOwsLCrM+/OBwOtWvXTg6HQ6mpqcrIyFCHDh1kt9s1depUOZ1ODRkyRJI0YsQIxcXFadKkSVqwYIHcbrfmzJmjtLQ0612WRx99VEuWLNHMmTP14IMPatOmTVq9erXy8//3jaWMjAylpKRo0KBBuu2227Ro0SLV1tZqypQpLXuGAABAm9SsyFm2bJkk6a677vK5f+XKlZo8ebIk6eWXX1ZgYKDGjh2ruro6uVwuvfrqq9baoKAg5eXl6bHHHpPT6VT79u2VkpKiZ5991lrTo0cP5efna/r06Vq8eLG6du2q119/XS6Xy1ozbtw4nThxQllZWXK73RowYIAKCgou+DAyAAC4Ol3WdXLaOq6T0/ZxnRwAuPq0ynVyAAAAfq6IHAAAYCQiBwAAGInIAQAARiJyAACAkYgcAABgJCIHAAAYicgBAABGInIAAICRiBwAAGAkIgcAABiJyAEAAEYicgAAgJGIHAAAYCQiBwAAGInIAQAARiJyAACAkYgcAABgJCIHAAAYicgBAABGInIAAICRiBwAAGAkIgcAABiJyAEAAEYicgAAgJGIHAAAYCQiBwAAGInIAQAARiJyAACAkYgcAABgJCIHAAAYicgBAABGInIAAICRiBwAAGAkIgcAABiJyAEAAEYicgAAgJGIHAAAYCQiBwAAGInIAQAARiJyAACAkYgcAABgJCIHAAAYicgBAABGInIAAICRiBwAAGAkIgcAABiJyAEAAEYicgAAgJGIHAAAYCQiBwAAGInIAQAARiJyAACAkYgcAABgJCIHAAAYicgBAABGInIAAICRiBwAAGAkIgcAABiJyAEAAEZqduSUlJRozJgxio6OVkBAgNatW+ezf/LkyQoICPDZRo4c6bPm5MmTmjhxoux2u8LDw5WamqqamhqfNXv27NGdd96p0NBQxcTEaMGCBRfMsmbNGsXGxio0NFR9+/bV+++/39yXAwAADNXsyKmtrVX//v21dOnSi64ZOXKkjh07Zm1///vfffZPnDhR+/btU2FhofLy8lRSUqJHHnnE2u/xeDRixAh169ZNZWVleumllzRv3jytWLHCWrN161ZNmDBBqamp+vjjj5WcnKzk5GRVVFQ09yUBAAADBXi9Xu8lPzggQGvXrlVycrJ13+TJk3Xq1KkL3uFp9OmnnyouLk47d+7UoEGDJEkFBQUaPXq0vvzyS0VHR2vZsmV6+umn5Xa7ZbPZJEmzZ8/WunXrtH//fknSuHHjVFtbq7y8POvYQ4YM0YABA7R8+fKfNL/H45HD4VB1dbXsdvulnoYmdZ+d36LHQ9MOz0/y9wgAgFb2U39+X5HP5GzevFkRERHq1auXHnvsMX399dfWvtLSUoWHh1uBI0mJiYkKDAzU9u3brTXDhg2zAkeSXC6XDhw4oG+++cZak5iY6PO8LpdLpaWlF52rrq5OHo/HZwMAAGZq8cgZOXKk/va3v6moqEgvvviiiouLNWrUKNXX10uS3G63IiIifB4THBysDh06yO12W2siIyN91jTe/rE1jfubkp2dLYfDYW0xMTEt9KoBAMDPTXBLH3D8+PHWP/ft21f9+vXTTTfdpM2bN2v48OEt/XTNkpmZqYyMDOu2x+MhdAAAMNQV/wr5jTfeqE6dOungwYOSpKioKB0/ftxnzblz53Ty5ElFRUVZa6qqqnzWNN7+sTWN+5sSEhIiu93uswEAADNd8cj58ssv9fXXX6tLly6SJKfTqVOnTqmsrMxas2nTJjU0NCghIcFaU1JSorNnz1prCgsL1atXL11//fXWmqKiIp/nKiwslNPpvNIvCQAAtAHNjpyamhqVl5ervLxcknTo0CGVl5ersrJSNTU1mjFjhrZt26bDhw+rqKhI9913n3r27CmXyyVJ6t27t0aOHKmHH35YO3bs0EcffaT09HSNHz9e0dHRkqQHHnhANptNqamp2rdvn1atWqXFixf7/Krp8ccfV0FBgRYuXKj9+/dr3rx52rVrl9LT01vu7AAAgDar2ZGza9cuDRw4UAMHDpQkZWRkaODAgcrKylJQUJD27NmjX//617r55puVmpqq+Ph4/fOf/1RISIh1jDfffFOxsbEaPny4Ro8erTvuuMPnGjgOh0MbN27UoUOHFB8fryeeeEJZWVk+19K5/fbblZubqxUrVqh///56++23tW7dOvXp0+fyzwoAAGjzLus6OW0d18lp+7hODgBcffx6nRwAAAB/I3IAAICRiBwAAGAkIgcAABiJyAEAAEYicgAAgJGIHAAAYCQiBwAAGInIAQAARiJyAACAkYgcAABgJCIHAAAYicgBAABGInIAAICRiBwAAGAkIgcAABiJyAEAAEYicgAAgJGIHAAAYCQiBwAAGInIAQAARiJyAACAkYgcAABgJCIHAAAYicgBAABGInIAAICRiBwAAGAkIgcAABiJyAEAAEYicgAAgJGIHAAAYCQiBwAAGInIAQAARiJyAACAkYgcAABgJCIHAAAYicgBAABGInIAAICRiBwAAGAkIgcAABiJyAEAAEYicgAAgJGIHAAAYCQiBwAAGInIAQAARiJyAACAkYgcAABgJCIHAAAYicgBAABGInIAAICRiBwAAGAkIgcAABiJyAEAAEYicgAAgJGIHAAAYCQiBwAAGInIAQAARiJyAACAkYgcAABgJCIHAAAYqdmRU1JSojFjxig6OloBAQFat26dz36v16usrCx16dJF7dq1U2Jioj777DOfNSdPntTEiRNlt9sVHh6u1NRU1dTU+KzZs2eP7rzzToWGhiomJkYLFiy4YJY1a9YoNjZWoaGh6tu3r95///3mvhwAAGCoZkdObW2t+vfvr6VLlza5f8GCBXrllVe0fPlybd++Xe3bt5fL5dJ3331nrZk4caL27dunwsJC5eXlqaSkRI888oi13+PxaMSIEerWrZvKysr00ksvad68eVqxYoW1ZuvWrZowYYJSU1P18ccfKzk5WcnJyaqoqGj+WQAAAMYJ8Hq93kt+cECA1q5dq+TkZOn7d3Gio6P1xBNP6Mknn5QkVVdXKzIyUjk5ORo/frw+/fRTxcXFaefOnRo0aJAkqaCgQKNHj9aXX36p6OhoLVu2TE8//bTcbrdsNpskafbs2Vq3bp32798vSRo3bpxqa2uVl5dnzTNkyBANGDBAy5cv/0nzezweORwOVVdXy263X+ppaFL32fktejw07fD8JH+PAABoZT/153eLfibn0KFDcrvdSkxMtO5zOBxKSEhQaWmpJKm0tFTh4eFW4EhSYmKiAgMDtX37dmvNsGHDrMCRJJfLpQMHDuibb76x1pz/PI1rGp+nKXV1dfJ4PD4bAAAwU4tGjtvtliRFRkb63B8ZGWntc7vdioiI8NkfHBysDh06+Kxp6hjnP8fF1jTub0p2drYcDoe1xcTEXMarBQAAP2dX1berMjMzVV1dbW1Hjhzx90gAAOAKadHIiYqKkiRVVVX53F9VVWXti4qK0vHjx332nzt3TidPnvRZ09Qxzn+Oi61p3N+UkJAQ2e12nw0AAJipRSOnR48eioqKUlFRkXWfx+PR9u3b5XQ6JUlOp1OnTp1SWVmZtWbTpk1qaGhQQkKCtaakpERnz5611hQWFqpXr166/vrrrTXnP0/jmsbnAQAAV7dmR05NTY3Ky8tVXl4uff9h4/LyclVWViogIEDTpk3Tc889p3fffVd79+7VH//4R0VHR1vfwOrdu7dGjhyphx9+WDt27NBHH32k9PR0jR8/XtHR0ZKkBx54QDabTampqdq3b59WrVqlxYsXKyMjw5rj8ccfV0FBgRYuXKj9+/dr3rx52rVrl9LT01vu7AAAgDYruLkP2LVrl+6++27rdmN4pKSkKCcnRzNnzlRtba0eeeQRnTp1SnfccYcKCgoUGhpqPebNN99Uenq6hg8frsDAQI0dO1avvPKKtd/hcGjjxo1KS0tTfHy8OnXqpKysLJ9r6dx+++3Kzc3VnDlz9NRTT+mXv/yl1q1bpz59+lzO+QAAAIa4rOvktHVcJ6ft4zo5AHD18ct1cgAAAH4uiBwAAGAkIgcAABiJyAEAAEYicgAAgJGIHAAAYCQiBwAAGInIAQAARiJyAACAkYgcAABgJCIHAAAYicgBAABGInIAAICRiBwAAGAkIgcAABiJyAEAAEYicgAAgJGIHAAAYCQiBwAAGInIAQAARiJyAACAkYgcAABgJCIHAAAYicgBAABGInIAAICRiBwAAGAkIgcAABiJyAEAAEYicgAAgJGIHAAAYCQiBwAAGInIAQAARiJyAACAkYgcAABgJCIHAAAYicgBAABGInIAAICRiBwAAGAkIgcAABiJyAEAAEYicgAAgJGIHAAAYKRgfw8AXI7us/P9PcJV4/D8JH+PAADNwjs5AADASEQOAAAwEpEDAACMROQAAAAjETkAAMBIRA4AADASkQMAAIxE5AAAACMROQAAwEhEDgAAMBKRAwAAjETkAAAAIxE5AADASEQOAAAwEpEDAACMROQAAAAjETkAAMBILR458+bNU0BAgM8WGxtr7f/uu++Ulpamjh076rrrrtPYsWNVVVXlc4zKykolJSXp2muvVUREhGbMmKFz5875rNm8ebNuvfVWhYSEqGfPnsrJyWnplwIAANqwK/JOzi233KJjx45Z25YtW6x906dP13vvvac1a9aouLhYR48e1f3332/tr6+vV1JSks6cOaOtW7fqjTfeUE5OjrKysqw1hw4dUlJSku6++26Vl5dr2rRpeuihh7Rhw4Yr8XIAAEAbFHxFDhocrKioqAvur66u1l//+lfl5ubqnnvukSStXLlSvXv31rZt2zRkyBBt3LhRn3zyiT744ANFRkZqwIAB+vOf/6xZs2Zp3rx5stlsWr58uXr06KGFCxdKknr37q0tW7bo5ZdflsvluhIvCQAAtDFX5J2czz77TNHR0brxxhs1ceJEVVZWSpLKysp09uxZJSYmWmtjY2N1ww03qLS0VJJUWlqqvn37KjIy0lrjcrnk8Xi0b98+a835x2hc03iMi6mrq5PH4/HZAACAmVo8chISEpSTk6OCggItW7ZMhw4d0p133qnTp0/L7XbLZrMpPDzc5zGRkZFyu92SJLfb7RM4jfsb9/3QGo/Ho2+//fais2VnZ8vhcFhbTExMi71uAADw89Liv64aNWqU9c/9+vVTQkKCunXrptWrV6tdu3Yt/XTNkpmZqYyMDOu2x+MhdAAAMNQV/wp5eHi4br75Zh08eFBRUVE6c+aMTp065bOmqqrK+gxPVFTUBd+2arz9Y2vsdvsPhlRISIjsdrvPBgAAzHTFI6empkaff/65unTpovj4eF1zzTUqKiqy9h84cECVlZVyOp2SJKfTqb179+r48ePWmsLCQtntdsXFxVlrzj9G45rGYwAAALR45Dz55JMqLi7W4cOHtXXrVv3mN79RUFCQJkyYIIfDodTUVGVkZOjDDz9UWVmZpkyZIqfTqSFDhkiSRowYobi4OE2aNEn/+te/tGHDBs2ZM0dpaWkKCQmRJD366KP64osvNHPmTO3fv1+vvvqqVq9erenTp7f0ywEAAG1Ui38m58svv9SECRP09ddfq3Pnzrrjjju0bds2de7cWZL08ssvKzAwUGPHjlVdXZ1cLpdeffVV6/FBQUHKy8vTY489JqfTqfbt2yslJUXPPvustaZHjx7Kz8/X9OnTtXjxYnXt2lWvv/46Xx8HAACWAK/X6/X3EP7i8XjkcDhUXV3d4p/P6T47v0WPB/jb4flJ/h4BAKRm/Pzmb1cBAAAjETkAAMBIRA4AADASkQMAAIxE5AAAACMROQAAwEhEDgAAMBKRAwAAjETkAAAAIxE5AADASEQOAAAwEpEDAACMROQAAAAjETkAAMBIRA4AADASkQMAAIxE5AAAACMROQAAwEhEDgAAMBKRAwAAjETkAAAAIxE5AADASEQOAAAwEpEDAACMROQAAAAjETkAAMBIRA4AADASkQMAAIxE5AAAACMROQAAwEhEDgAAMBKRAwAAjETkAAAAIxE5AADASEQOAAAwEpEDAACMROQAAAAjETkAAMBIRA4AADASkQMAAIxE5AAAACMROQAAwEhEDgAAMBKRAwAAjETkAAAAIxE5AADASMH+HgBA29B9dr6/R7gqHJ6f5O8RAGPwTg4AADASkQMAAIxE5AAAACMROQAAwEhEDgAAMBKRAwAAjETkAAAAIxE5AADASEQOAAAwEpEDAACMROQAAAAjETkAAMBIRA4AADBSm4+cpUuXqnv37goNDVVCQoJ27Njh75EAAMDPQJuOnFWrVikjI0Nz587V7t271b9/f7lcLh0/ftzfowEAAD8L8Hq9Xn8PcakSEhI0ePBgLVmyRJLU0NCgmJgYTZ06VbNnz/7Rx3s8HjkcDlVXV8tut7fobN1n57fo8QAALefw/CR/j4DL8FN/fge36lQt6MyZMyorK1NmZqZ1X2BgoBITE1VaWtrkY+rq6lRXV2fdrq6ulr4/WS2toe6/LX5MAEDLuBL/v4/W0/jf34+9T9NmI+err75SfX29IiMjfe6PjIzU/v37m3xMdna2nnnmmQvuj4mJuWJzAgB+fhyL/D0BWsLp06flcDguur/NRs6lyMzMVEZGhnW7oaFBJ0+eVMeOHRUQENBiz+PxeBQTE6MjR460+K/B8D+c59bDuW4dnOfWwXluHVfyPHu9Xp0+fVrR0dE/uK7NRk6nTp0UFBSkqqoqn/urqqoUFRXV5GNCQkIUEhLic194ePgVm9Fut/M/oFbAeW49nOvWwXluHZzn1nGlzvMPvYPTqM1+u8pmsyk+Pl5FRUXWfQ0NDSoqKpLT6fTrbAAAwP/a7Ds5kpSRkaGUlBQNGjRIt912mxYtWqTa2lpNmTLF36MBAAA/a9ORM27cOJ04cUJZWVlyu90aMGCACgoKLvgwcmsLCQnR3LlzL/jVGFoW57n1cK5bB+e5dXCeW8fP4Ty36evkAAAAXEyb/UwOAADADyFyAACAkYgcAABgJCIHAAAYichpQSUlJRozZoyio6MVEBCgdevW+XskI2VnZ2vw4MEKCwtTRESEkpOTdeDAAX+PZZxly5apX79+1oW8nE6n1q9f7++xjDd//nwFBARo2rRp/h7FOPPmzVNAQIDPFhsb6++xjPSf//xHf/jDH9SxY0e1a9dOffv21a5du1p9DiKnBdXW1qp///5aunSpv0cxWnFxsdLS0rRt2zYVFhbq7NmzGjFihGpra/09mlG6du2q+fPnq6ysTLt27dI999yj++67T/v27fP3aMbauXOn/vKXv6hfv37+HsVYt9xyi44dO2ZtW7Zs8fdIxvnmm280dOhQXXPNNVq/fr0++eQTLVy4UNdff32rz9Kmr5PzczNq1CiNGjXK32MYr6CgwOd2Tk6OIiIiVFZWpmHDhvltLtOMGTPG5/bzzz+vZcuWadu2bbrlllv8NpepampqNHHiRL322mt67rnn/D2OsYKDgy/6p3/QMl588UXFxMRo5cqV1n09evTwyyy8k4M2r7q6WpLUoUMHf49irPr6er311luqra3lz6ZcIWlpaUpKSlJiYqK/RzHaZ599pujoaN14442aOHGiKisr/T2Scd59910NGjRIv/vd7xQREaGBAwfqtdde88ssvJODNq2hoUHTpk3T0KFD1adPH3+PY5y9e/fK6XTqu+++03XXXae1a9cqLi7O32MZ56233tLu3bu1c+dOf49itISEBOXk5KhXr146duyYnnnmGd15552qqKhQWFiYv8czxhdffKFly5YpIyNDTz31lHbu3Kk//elPstlsSklJadVZiBy0aWlpaaqoqOD36ldIr169VF5erurqar399ttKSUlRcXExodOCjhw5oscff1yFhYUKDQ319zhGO//jBP369VNCQoK6deum1atXKzU11a+zmaShoUGDBg3SCy+8IEkaOHCgKioqtHz58laPHH5dhTYrPT1deXl5+vDDD9W1a1d/j2Mkm82mnj17Kj4+XtnZ2erfv78WL17s77GMUlZWpuPHj+vWW29VcHCwgoODVVxcrFdeeUXBwcGqr6/394jGCg8P180336yDBw/6exSjdOnS5YJ/Eerdu7dffjXIOzloc7xer6ZOnaq1a9dq8+bNfvtA29WooaFBdXV1/h7DKMOHD9fevXt97psyZYpiY2M1a9YsBQUF+W0209XU1Ojzzz/XpEmT/D2KUYYOHXrBZT3+/e9/q1u3bq0+C5HTgmpqanz+jeDQoUMqLy9Xhw4ddMMNN/h1NpOkpaUpNzdX77zzjsLCwuR2uyVJDodD7dq18/d4xsjMzNSoUaN0ww036PTp08rNzdXmzZu1YcMGf49mlLCwsAs+T9a+fXt17NiRz5m1sCeffFJjxoxRt27ddPToUc2dO1dBQUGaMGGCv0czyvTp03X77bfrhRde0O9//3vt2LFDK1as0IoVK1p/GC9azIcffuiVdMGWkpLi79GM0tQ5luRduXKlv0czyoMPPujt1q2b12azeTt37uwdPny4d+PGjf4e66rwq1/9yvv444/7ewzjjBs3ztulSxevzWbz/uIXv/COGzfOe/DgQX+PZaT33nvP26dPH29ISIg3NjbWu2LFCr/MEeD9/z80AAAAjMIHjwEAgJGIHAAAYCQiBwAAGInIAQAARiJyAACAkYgcAABgJCIHAAAYicgBAABGInIAAICRiBwAAGAkIgcAABiJyAEAAEb6f8M11LhmDDKeAAAAAElFTkSuQmCC", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAk0AAAGwCAYAAAC0HlECAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAr+UlEQVR4nO3df1TVdZ7H8dcFBNTkkhogI/7KUsnfv5AsZ0zWa1E7bM6ultOgUa0uOCpTKpuD1tbQ2LG0NBm3nWh2c9M6oxUWiqiYipo0pFiw2ejRUtAyuEIJCnf/ML7TDauPDPK9yvNxzj3H+/1+uLzv93SG53zvly8Oj8fjEQAAAH6Qn90DAAAAXAmIJgAAAANEEwAAgAGiCQAAwADRBAAAYIBoAgAAMEA0AQAAGAiwe4CrRX19vY4fP64OHTrI4XDYPQ4AADDg8Xh05swZRUZGys/vh88lEU3N5Pjx44qKirJ7DAAA0ATHjh1T165df3AN0dRMOnToIH1z0ENCQuweBwAAGHC73YqKirJ+jv8QoqmZNHwkFxISQjQBAHCFMbm0hgvBAQAADBBNAAAABogmAAAAA0QTAACAAaIJAADAANEEAABggGgCAAAwQDQBAAAYsDWaMjIyNGLECHXo0EFhYWFKSEhQaWmp15qf/exncjgcXo/p06d7rTl69Kji4+PVrl07hYWF6ZFHHtH58+e91mzbtk1Dhw5VUFCQevfuraysrEbzrFixQj169FBwcLBiYmK0d+/ey/TOAQDAlcbWaMrPz1dycrJ2796t3NxcnTt3TuPHj1d1dbXXugcffFAnTpywHosXL7b21dXVKT4+XrW1tdq1a5defvllZWVlKT093Vpz+PBhxcfHa+zYsSoqKtLs2bP1wAMPaOPGjdaaNWvWKDU1VQsXLtT777+vQYMGyeVy6eTJky10NAAAgC9zeDwej91DNDh16pTCwsKUn5+vMWPGSN+caRo8eLCWLl160a955513dOedd+r48eMKDw+XJGVmZmrevHk6deqUAgMDNW/ePG3YsEHFxcXW102ePFkVFRXKycmRJMXExGjEiBFavny5JKm+vl5RUVGaOXOm5s+f3+j71tTUqKamxnre8LdrKisr+TMqAABcIdxut5xOp9HPb5+6pqmyslKS1LFjR6/tr7zyijp37qz+/fsrLS1NX331lbWvoKBAAwYMsIJJklwul9xutw4ePGitiYuL83pNl8ulgoICSVJtba0KCwu91vj5+SkuLs5a810ZGRlyOp3WIyoqqlmOAQAA8E0+8wd76+vrNXv2bI0ePVr9+/e3tt97773q3r27IiMjtX//fs2bN0+lpaX685//LEkqKyvzCiZJ1vOysrIfXON2u/X111/ryy+/VF1d3UXXlJSUXHTetLQ0paamWs8bzjQBAICrk89EU3JysoqLi7Vjxw6v7Q899JD17wEDBqhLly4aN26cPvnkE11//fU2THpBUFCQgoKCbPv+AACgZfnEx3MpKSnKzs7W1q1b1bVr1x9cGxMTI0k6dOiQJCkiIkLl5eVeaxqeR0RE/OCakJAQtW3bVp07d5a/v/9F1zS8BgAAaN1sPdPk8Xg0c+ZMrVu3Ttu2bVPPnj1/9GuKiookSV26dJEkxcbG6sknn9TJkycVFhYmScrNzVVISIiio6OtNW+//bbX6+Tm5io2NlaSFBgYqGHDhikvL08JCQnSNx8X5uXlKSUlpZnfNf4ePeZvsHuEZnfkqXi7RwAAGLA1mpKTk7V69Wq98cYb6tChg3UNktPpVNu2bfXJJ59o9erVuuOOO9SpUyft379fc+bM0ZgxYzRw4EBJ0vjx4xUdHa377rtPixcvVllZmRYsWKDk5GTr47Pp06dr+fLlmjt3ru6//35t2bJFa9eu1YYNf/sBnJqaqsTERA0fPlwjR47U0qVLVV1drWnTptl0dAAAgC+xNZpWrlwpfXNbgW976aWXNHXqVAUGBmrz5s1WwERFRWnixIlasGCBtdbf31/Z2dmaMWOGYmNj1b59eyUmJurxxx+31vTs2VMbNmzQnDlztGzZMnXt2lUvvviiXC6XtWbSpEk6deqU0tPTVVZWpsGDBysnJ6fRxeEAAKB18qn7NF3JLuU+D2g6Pp4DADSnK/Y+TQAAAL6KaAIAADBANAEAABggmgAAAAwQTQAAAAaIJgAAAANEEwAAgAGiCQAAwADRBAAAYIBoAgAAMEA0AQAAGCCaAAAADBBNAAAABogmAAAAA0QTAACAAaIJAADAANEEAABggGgCAAAwQDQBAAAYIJoAAAAMEE0AAAAGiCYAAAADRBMAAIABogkAAMAA0QQAAGCAaAIAADBANAEAABggmgAAAAwQTQAAAAaIJgAAAANEEwAAgAGiCQAAwADRBAAAYIBoAgAAMEA0AQAAGCCaAAAADBBNAAAABogmAAAAA0QTAACAAaIJAADAANEEAABggGgCAAAwQDQBAAAYIJoAAAAMEE0AAAAGiCYAAAADRBMAAIABogkAAMAA0QQAAGCAaAIAADBANAEAABggmgAAAAwQTQAAAAaIJgAAAANEEwAAgAGiCQAAwADRBAAAYIBoAgAAMEA0AQAAGCCaAAAADNgaTRkZGRoxYoQ6dOigsLAwJSQkqLS01GvN2bNnlZycrE6dOumaa67RxIkTVV5e7rXm6NGjio+PV7t27RQWFqZHHnlE58+f91qzbds2DR06VEFBQerdu7eysrIazbNixQr16NFDwcHBiomJ0d69ey/TOwcAAFcaW6MpPz9fycnJ2r17t3Jzc3Xu3DmNHz9e1dXV1po5c+borbfe0muvvab8/HwdP35cd999t7W/rq5O8fHxqq2t1a5du/Tyyy8rKytL6enp1prDhw8rPj5eY8eOVVFRkWbPnq0HHnhAGzdutNasWbNGqampWrhwod5//30NGjRILpdLJ0+ebMEjAgAAfJXD4/F47B6iwalTpxQWFqb8/HyNGTNGlZWVuu6667R69Wr94he/kCSVlJSoX79+Kigo0KhRo/TOO+/ozjvv1PHjxxUeHi5JyszM1Lx583Tq1CkFBgZq3rx52rBhg4qLi63vNXnyZFVUVCgnJ0eSFBMToxEjRmj58uWSpPr6ekVFRWnmzJmaP39+o1lrampUU1NjPXe73YqKilJlZaVCQkIu+7FqrXrM32D3CM3uyFPxdo8AAK2W2+2W0+k0+vntU9c0VVZWSpI6duwoSSosLNS5c+cUFxdnrenbt6+6deumgoICSVJBQYEGDBhgBZMkuVwuud1uHTx40Frz7ddoWNPwGrW1tSosLPRa4+fnp7i4OGvNd2VkZMjpdFqPqKioZjwSAADA1/hMNNXX12v27NkaPXq0+vfvL0kqKytTYGCgQkNDvdaGh4errKzMWvPtYGrY37Dvh9a43W59/fXX+vzzz1VXV3fRNQ2v8V1paWmqrKy0HseOHfu7jwEAAPBdAXYP0CA5OVnFxcXasWOH3aMYCQoKUlBQkN1jAACAFuITZ5pSUlKUnZ2trVu3qmvXrtb2iIgI1dbWqqKiwmt9eXm5IiIirDXf/W26huc/tiYkJERt27ZV586d5e/vf9E1Da8BAABaN1ujyePxKCUlRevWrdOWLVvUs2dPr/3Dhg1TmzZtlJeXZ20rLS3V0aNHFRsbK0mKjY3VgQMHvH7LLTc3VyEhIYqOjrbWfPs1GtY0vEZgYKCGDRvmtaa+vl55eXnWGgAA0LrZ+vFccnKyVq9erTfeeEMdOnSwrh9yOp1q27atnE6nkpKSlJqaqo4dOyokJEQzZ85UbGysRo0aJUkaP368oqOjdd9992nx4sUqKyvTggULlJycbH18Nn36dC1fvlxz587V/fffry1btmjt2rXasOFvv4mVmpqqxMREDR8+XCNHjtTSpUtVXV2tadOm2XR0AACAL7E1mlauXClJ+tnPfua1/aWXXtLUqVMlSc8++6z8/Pw0ceJE1dTUyOVy6YUXXrDW+vv7Kzs7WzNmzFBsbKzat2+vxMREPf7449aanj17asOGDZozZ46WLVumrl276sUXX5TL5bLWTJo0SadOnVJ6errKyso0ePBg5eTkNLo4HAAAtE4+dZ+mK9ml3OcBTcd9mgAAzemKvU8TAACAryKaAAAADBBNAAAABogmAAAAA0QTAACAAaIJAADAANEEAABggGgCAAAwQDQBAAAYIJoAAAAMEE0AAAAGiCYAAAADRBMAAIABogkAAMAA0QQAAGCAaAIAADBANAEAABggmgAAAAwQTQAAAAaIJgAAAANEEwAAgAGiCQAAwADRBAAAYIBoAgAAMEA0AQAAGCCaAAAADBBNAAAABogmAAAAA0QTAACAAaIJAADAANEEAABggGgCAAAwQDQBAAAYIJoAAAAMEE0AAAAGiCYAAAADRBMAAIABogkAAMAA0QQAAGCAaAIAADBANAEAABggmgAAAAwQTQAAAAaIJgAAAANEEwAAgAGiCQAAwADRBAAAYIBoAgAAMEA0AQAAGCCaAAAADBBNAAAABogmAAAAA0QTAACAAaIJAADAANEEAABggGgCAAAwQDQBAAAYIJoAAAAMEE0AAAAGiCYAAAADtkbT9u3bdddddykyMlIOh0Pr16/32j916lQ5HA6vx4QJE7zWnD59WlOmTFFISIhCQ0OVlJSkqqoqrzX79+/XrbfequDgYEVFRWnx4sWNZnnttdfUt29fBQcHa8CAAXr77bcv07sGAABXIlujqbq6WoMGDdKKFSu+d82ECRN04sQJ6/G///u/XvunTJmigwcPKjc3V9nZ2dq+fbseeugha7/b7db48ePVvXt3FRYW6umnn9aiRYu0atUqa82uXbt0zz33KCkpSX/5y1+UkJCghIQEFRcXX6Z3DgAArjQOj8fjsXsISXI4HFq3bp0SEhKsbVOnTlVFRUWjM1ANPvroI0VHR+u9997T8OHDJUk5OTm644479OmnnyoyMlIrV67Uo48+qrKyMgUGBkqS5s+fr/Xr16ukpESSNGnSJFVXVys7O9t67VGjRmnw4MHKzMy86PeuqalRTU2N9dztdisqKkqVlZUKCQlppqOC7+oxf4PdIzS7I0/F2z0CALRabrdbTqfT6Oe3z1/TtG3bNoWFhalPnz6aMWOGvvjiC2tfQUGBQkNDrWCSpLi4OPn5+WnPnj3WmjFjxljBJEkul0ulpaX68ssvrTVxcXFe39flcqmgoOB758rIyJDT6bQeUVFRzfq+AQCAb/HpaJowYYL+9Kc/KS8vT7///e+Vn5+v22+/XXV1dZKksrIyhYWFeX1NQECAOnbsqLKyMmtNeHi415qG5z+2pmH/xaSlpamystJ6HDt2rJneNQAA8EUBdg/wQyZPnmz9e8CAARo4cKCuv/56bdu2TePGjbN1tqCgIAUFBdk6AwAAaDk+fabpu3r16qXOnTvr0KFDkqSIiAidPHnSa8358+d1+vRpRUREWGvKy8u91jQ8/7E1DfsBAACuqGj69NNP9cUXX6hLly6SpNjYWFVUVKiwsNBas2XLFtXX1ysmJsZas337dp07d85ak5ubqz59+ujaa6+11uTl5Xl9r9zcXMXGxrbQOwMAAL7O1miqqqpSUVGRioqKJEmHDx9WUVGRjh49qqqqKj3yyCPavXu3jhw5ory8PP385z9X79695XK5JEn9+vXThAkT9OCDD2rv3r3auXOnUlJSNHnyZEVGRkqS7r33XgUGBiopKUkHDx7UmjVrtGzZMqWmplpzzJo1Szk5OVqyZIlKSkq0aNEi7du3TykpKTYdGQAA4GtsjaZ9+/ZpyJAhGjJkiCQpNTVVQ4YMUXp6uvz9/bV//3794z/+o2688UYlJSVp2LBhevfdd72uJXrllVfUt29fjRs3TnfccYduueUWr3swOZ1Obdq0SYcPH9awYcP0m9/8Runp6V73crr55pu1evVqrVq1SoMGDdLrr7+u9evXq3///i18RAAAgK/ymfs0Xeku5T4PaDru0wQAaE5X1X2aAAAAfAHRBAAAYIBoAgAAMNCkaOrVq5fXnzNpUFFRoV69ejXHXAAAAD6lSdF05MgR60+ZfFtNTY0+++yz5pgLAADAp1zSn1F58803rX9v3LhRTqfTel5XV6e8vDz16NGjeScEAADwAZcUTQkJCZIkh8OhxMREr31t2rRRjx49tGTJkuadEAAAwAdcUjTV19dLknr27Kn33ntPnTt3vlxzAQAA+JRLiqYGhw8fbv5JAAAAfFiTokmS8vLylJeXp5MnT1pnoBr88Y9/bI7ZAAAAfEaToumxxx7T448/ruHDh6tLly5yOBzNPxkAAIAPaVI0ZWZmKisrS/fdd1/zTwQAAOCDmnSfptraWt18883NPw0AAICPalI0PfDAA1q9enXzTwMAAOCjmvTx3NmzZ7Vq1Spt3rxZAwcOVJs2bbz2P/PMM801HwAAgE9oUjTt379fgwcPliQVFxd77eOicAAAcDVqUjRt3bq1+ScBAADwYU26pgkAAKC1adKZprFjx/7gx3Bbtmz5e2YCAADwOU2KpobrmRqcO3dORUVFKi4ubvSHfAEAAK4GTYqmZ5999qLbFy1apKqqqr93JgAAAJ/TrNc0/fKXv+TvzgEAgKtSs0ZTQUGBgoODm/MlAQAAfEKTPp67++67vZ57PB6dOHFC+/bt029/+9vmmg0AAMBnNCmanE6n13M/Pz/16dNHjz/+uMaPH99cswEAAPiMJkXTSy+91PyTAAAA+LAmRVODwsJCffTRR5Kkm266SUOGDGmuuQAAAHxKk6Lp5MmTmjx5srZt26bQ0FBJUkVFhcaOHatXX31V1113XXPPCQAAYKsm/fbczJkzdebMGR08eFCnT5/W6dOnVVxcLLfbrV//+tfNPyUAAIDNmnSmKScnR5s3b1a/fv2sbdHR0VqxYgUXggMAgKtSk8401dfXq02bNo22t2nTRvX19c0xFwAAgE9pUjTddtttmjVrlo4fP25t++yzzzRnzhyNGzeuOecDAADwCU2KpuXLl8vtdqtHjx66/vrrdf3116tnz55yu916/vnnm39KAAAAmzXpmqaoqCi9//772rx5s0pKSiRJ/fr1U1xcXHPPBwAA4BMu6UzTli1bFB0dLbfbLYfDoX/4h3/QzJkzNXPmTI0YMUI33XST3n333cs3LQAAgE0uKZqWLl2qBx98UCEhIY32OZ1O/eu//queeeaZ5pwPAADAJ1xSNH3wwQeaMGHC9+4fP368CgsLm2MuAAAAn3JJ0VReXn7RWw00CAgI0KlTp5pjLgAAAJ9ySdH0k5/8RMXFxd+7f//+/erSpUtzzAUAAOBTLima7rjjDv32t7/V2bNnG+37+uuvtXDhQt15553NOR8AAIBPuKRbDixYsEB//vOfdeONNyolJUV9+vSRJJWUlGjFihWqq6vTo48+erlmBQAAsM0lRVN4eLh27dqlGTNmKC0tTR6PR5LkcDjkcrm0YsUKhYeHX65ZAQAAbHPJN7fs3r273n77bX355Zc6dOiQPB6PbrjhBl177bWXZ0IAAAAf0KQ7gkvStddeqxEjRjTvNAAAAD6qSX97DgAAoLUhmgAAAAwQTQAAAAaIJgAAAANEEwAAgAGiCQAAwADRBAAAYIBoAgAAMEA0AQAAGCCaAAAADBBNAAAABogmAAAAA0QTAACAAaIJAADAANEEAABggGgCAAAwYGs0bd++XXfddZciIyPlcDi0fv16r/0ej0fp6enq0qWL2rZtq7i4OH388cdea06fPq0pU6YoJCREoaGhSkpKUlVVldea/fv369Zbb1VwcLCioqK0ePHiRrO89tpr6tu3r4KDgzVgwAC9/fbbl+ldAwCAK5Gt0VRdXa1BgwZpxYoVF92/ePFiPffcc8rMzNSePXvUvn17uVwunT171lozZcoUHTx4ULm5ucrOztb27dv10EMPWfvdbrfGjx+v7t27q7CwUE8//bQWLVqkVatWWWt27dqle+65R0lJSfrLX/6ihIQEJSQkqLi4+DIfAQAAcKVweDwej91DSJLD4dC6deuUkJAgfXOWKTIyUr/5zW/08MMPS5IqKysVHh6urKwsTZ48WR999JGio6P13nvvafjw4ZKknJwc3XHHHfr0008VGRmplStX6tFHH1VZWZkCAwMlSfPnz9f69etVUlIiSZo0aZKqq6uVnZ1tzTNq1CgNHjxYmZmZF523pqZGNTU11nO3262oqChVVlYqJCTkMh6p1q3H/A12j9DsjjwVb/cIANBqud1uOZ1Oo5/fPntN0+HDh1VWVqa4uDhrm9PpVExMjAoKCiRJBQUFCg0NtYJJkuLi4uTn56c9e/ZYa8aMGWMFkyS5XC6Vlpbqyy+/tNZ8+/s0rGn4PheTkZEhp9NpPaKioprx3QMAAF/js9FUVlYmSQoPD/faHh4ebu0rKytTWFiY1/6AgAB17NjRa83FXuPb3+P71jTsv5i0tDRVVlZaj2PHjv0d7xYAAPi6ALsHuFIFBQUpKCjI7jEAAEAL8dkzTREREZKk8vJyr+3l5eXWvoiICJ08edJr//nz53X69GmvNRd7jW9/j+9b07AfAADAZ6OpZ8+eioiIUF5enrXN7XZrz549io2NlSTFxsaqoqJChYWF1potW7aovr5eMTEx1prt27fr3Llz1prc3Fz16dNH1157rbXm29+nYU3D9wEAALA1mqqqqlRUVKSioiLpm4u/i4qKdPToUTkcDs2ePVtPPPGE3nzzTR04cEC/+tWvFBkZaf2GXb9+/TRhwgQ9+OCD2rt3r3bu3KmUlBRNnjxZkZGRkqR7771XgYGBSkpK0sGDB7VmzRotW7ZMqamp1hyzZs1STk6OlixZopKSEi1atEj79u1TSkqKTUcGAAD4Gluvadq3b5/Gjh1rPW8ImcTERGVlZWnu3Lmqrq7WQw89pIqKCt1yyy3KyclRcHCw9TWvvPKKUlJSNG7cOPn5+WnixIl67rnnrP1Op1ObNm1ScnKyhg0bps6dOys9Pd3rXk4333yzVq9erQULFujf//3fdcMNN2j9+vXq379/ix0LAADg23zmPk1Xuku5zwOajvs0AQCa01VxnyYAAABfQjQBAAAYIJoAAAAMEE0AAAAGiCYAAAADRBMAAIABogkAAMAA0QQAAGCAaAIAADBANAEAABggmgAAAAwQTQAAAAaIJgAAAANEEwAAgAGiCQAAwADRBAAAYIBoAgAAMEA0AQAAGCCaAAAADBBNAAAABogmAAAAA0QTAACAAaIJAADAANEEAABggGgCAAAwQDQBAAAYIJoAAAAMEE0AAAAGiCYAAAADRBMAAIABogkAAMAA0QQAAGCAaAIAADBANAEAABggmgAAAAwQTQAAAAaIJgAAAANEEwAAgAGiCQAAwADRBAAAYIBoAgAAMEA0AQAAGAiwewAAl67H/A12j9DsjjwVb/cIAPCDONMEAABggGgCAAAwQDQBAAAYIJoAAAAMEE0AAAAGiCYAAAADRBMAAIABogkAAMAA0QQAAGCAaAIAADBANAEAABggmgAAAAwQTQAAAAaIJgAAAANEEwAAgAGiCQAAwADRBAAAYMCno2nRokVyOBxej759+1r7z549q+TkZHXq1EnXXHONJk6cqPLycq/XOHr0qOLj49WuXTuFhYXpkUce0fnz573WbNu2TUOHDlVQUJB69+6trKysFnuPAADgyuDT0SRJN910k06cOGE9duzYYe2bM2eO3nrrLb322mvKz8/X8ePHdffdd1v76+rqFB8fr9raWu3atUsvv/yysrKylJ6ebq05fPiw4uPjNXbsWBUVFWn27Nl64IEHtHHjxhZ/rwAAwHcF2D3AjwkICFBERESj7ZWVlfqv//ovrV69Wrfddpsk6aWXXlK/fv20e/dujRo1Sps2bdKHH36ozZs3Kzw8XIMHD9Z//Md/aN68eVq0aJECAwOVmZmpnj17asmSJZKkfv36aceOHXr22Wflcrla/P0CAADf5PNnmj7++GNFRkaqV69emjJlio4ePSpJKiws1Llz5xQXF2et7du3r7p166aCggJJUkFBgQYMGKDw8HBrjcvlktvt1sGDB601336NhjUNr/F9ampq5Ha7vR4AAODq5dPRFBMTo6ysLOXk5GjlypU6fPiwbr31Vp05c0ZlZWUKDAxUaGio19eEh4errKxMklRWVuYVTA37G/b90Bq3262vv/76e2fLyMiQ0+m0HlFRUc32vgEAgO/x6Y/nbr/9duvfAwcOVExMjLp37661a9eqbdu2ts6Wlpam1NRU67nb7SacAAC4ivn0mabvCg0N1Y033qhDhw4pIiJCtbW1qqio8FpTXl5uXQMVERHR6LfpGp7/2JqQkJAfDLOgoCCFhIR4PQAAwNXrioqmqqoqffLJJ+rSpYuGDRumNm3aKC8vz9pfWlqqo0ePKjY2VpIUGxurAwcO6OTJk9aa3NxchYSEKDo62lrz7ddoWNPwGgAAAPL1aHr44YeVn5+vI0eOaNeuXfqnf/on+fv765577pHT6VRSUpJSU1O1detWFRYWatq0aYqNjdWoUaMkSePHj1d0dLTuu+8+ffDBB9q4caMWLFig5ORkBQUFSZKmT5+uv/71r5o7d65KSkr0wgsvaO3atZozZ47N7x4AAPgSn76m6dNPP9U999yjL774Qtddd51uueUW7d69W9ddd50k6dlnn5Wfn58mTpyompoauVwuvfDCC9bX+/v7Kzs7WzNmzFBsbKzat2+vxMREPf7449aanj17asOGDZozZ46WLVumrl276sUXX+R2AwAAwIvD4/F47B7iauB2u+V0OlVZWcn1TZdRj/kb7B6h2R15Kv6Sv4bjAADN41J+fvv0x3MAAAC+gmgCAAAwQDQBAAAYIJoAAAAMEE0AAAAGiCYAAAADRBMAAIABogkAAMAA0QQAAGCAaAIAADBANAEAABggmgAAAAwQTQAAAAaIJgAAAANEEwAAgAGiCQAAwADRBAAAYIBoAgAAMEA0AQAAGCCaAAAADBBNAAAABogmAAAAA0QTAACAAaIJAADAANEEAABggGgCAAAwQDQBAAAYIJoAAAAMEE0AAAAGiCYAAAADRBMAAIABogkAAMAA0QQAAGCAaAIAADBANAEAABggmgAAAAwQTQAAAAaIJgAAAANEEwAAgAGiCQAAwADRBAAAYIBoAgAAMEA0AQAAGCCaAAAADBBNAAAABogmAAAAAwF2DwAAf48e8zfYPUKzO/JUvN0jALgIzjQBAAAYIJoAAAAMEE0AAAAGiCYAAAADRBMAAIABogkAAMAA0QQAAGCAaAIAADBANAEAABggmgAAAAwQTQAAAAaIJgAAAANEEwAAgAGi6TtWrFihHj16KDg4WDExMdq7d6/dIwEAAB8QYPcAvmTNmjVKTU1VZmamYmJitHTpUrlcLpWWliosLMzu8QDge/WYv8HuEZrdkafi7R4B8MKZpm955pln9OCDD2ratGmKjo5WZmam2rVrpz/+8Y92jwYAAGzGmaZv1NbWqrCwUGlpadY2Pz8/xcXFqaCgoNH6mpoa1dTUWM8rKyslSW63u4Umbp3qa76ye4Rm15T/ZjgOf8OxuIDjcEH/hRsvyyx2Kn7MZfcIV7WG/848Hs+PriWavvH555+rrq5O4eHhXtvDw8NVUlLSaH1GRoYee+yxRtujoqIu65y4+jiX2j2Bb+A4/A3H4gKOwwUch5Zx5swZOZ3OH1xDNDVRWlqaUlNTref19fU6ffq0OnXqJIfDYetsTeV2uxUVFaVjx44pJCTE7nFsxbG4gONwAcfhAo7D33AsLrgajoPH49GZM2cUGRn5o2uJpm907txZ/v7+Ki8v99peXl6uiIiIRuuDgoIUFBTktS00NPSyz9kSQkJCrtj/+Jsbx+ICjsMFHIcLOA5/w7G44Eo/Dj92hqkBF4J/IzAwUMOGDVNeXp61rb6+Xnl5eYqNjbV1NgAAYD/ONH1LamqqEhMTNXz4cI0cOVJLly5VdXW1pk2bZvdoAADAZkTTt0yaNEmnTp1Senq6ysrKNHjwYOXk5DS6OPxqFRQUpIULFzb62LE14lhcwHG4gONwAcfhbzgWF7S24+DwmPyOHQAAQCvHNU0AAAAGiCYAAAADRBMAAIABogkAAMAA0QTLihUr1KNHDwUHBysmJkZ79+61e6QWt337dt11112KjIyUw+HQ+vXr7R6pxWVkZGjEiBHq0KGDwsLClJCQoNLSUrvHssXKlSs1cOBA68Z9sbGxeuedd+wey3ZPPfWUHA6HZs+ebfcoLWrRokVyOBxej759+9o9lm0+++wz/fKXv1SnTp3Utm1bDRgwQPv27bN7rMuKaIIkac2aNUpNTdXChQv1/vvva9CgQXK5XDp58qTdo7Wo6upqDRo0SCtWrLB7FNvk5+crOTlZu3fvVm5urs6dO6fx48erurra7tFaXNeuXfXUU0+psLBQ+/bt02233aaf//znOnjwoN2j2ea9997TH/7wBw0cONDuUWxx00036cSJE9Zjx44ddo9kiy+//FKjR49WmzZt9M477+jDDz/UkiVLdO2119o92mXFLQcgSYqJidGIESO0fPly6Zu7oUdFRWnmzJmaP3++3ePZwuFwaN26dUpISLB7FFudOnVKYWFhys/P15gxY+wex3YdO3bU008/raSkJLtHaXFVVVUaOnSoXnjhBT3xxBMaPHiwli5tPX9NdtGiRVq/fr2KiorsHsV28+fP186dO/Xuu+/aPUqL4kwTVFtbq8LCQsXFxVnb/Pz8FBcXp4KCAltng/0qKyulb2KhNaurq9Orr76q6urqVvunlZKTkxUfH+/1vxWtzccff6zIyEj16tVLU6ZM0dGjR+0eyRZvvvmmhg8frn/+539WWFiYhgwZov/8z/+0e6zLjmiCPv/8c9XV1TW683l4eLjKyspsmwv2q6+v1+zZszV69Gj179/f7nFsceDAAV1zzTUKCgrS9OnTtW7dOkVHR9s9Vot79dVX9f777ysjI8PuUWwTExOjrKws5eTkaOXKlTp8+LBuvfVWnTlzxu7RWtxf//pXrVy5UjfccIM2btyoGTNm6Ne//rVefvllu0e7rPgzKgC+V3JysoqLi1vtdRuS1KdPHxUVFamyslKvv/66EhMTlZ+f36rC6dixY5o1a5Zyc3MVHBxs9zi2uf32261/Dxw4UDExMerevbvWrl3b6j6ura+v1/Dhw/W73/1OkjRkyBAVFxcrMzNTiYmJdo932XCmCercubP8/f1VXl7utb28vFwRERG2zQV7paSkKDs7W1u3blXXrl3tHsc2gYGB6t27t4YNG6aMjAwNGjRIy5Yts3usFlVYWKiTJ09q6NChCggIUEBAgPLz8/Xcc88pICBAdXV1do9oi9DQUN144406dOiQ3aO0uC5dujT6Pw79+vW76j+uJJqgwMBADRs2THl5eda2+vp65eXltdprN1ozj8ejlJQUrVu3Tlu2bFHPnj3tHsmn1NfXq6amxu4xWtS4ceN04MABFRUVWY/hw4drypQpKioqkr+/v90j2qKqqkqffPKJunTpYvcoLW706NGNbkXyf//3f+revbttM7UEPp6DJCk1NVWJiYkaPny4Ro4cqaVLl6q6ulrTpk2ze7QWVVVV5fX/Gg8fPqyioiJ17NhR3bp1s3W2lpKcnKzVq1frjTfeUIcOHazr2pxOp9q2bWv3eC0qLS1Nt99+u7p166YzZ85o9erV2rZtmzZu3Gj3aC2qQ4cOja5pa9++vTp16tSqrnV7+OGHddddd6l79+46fvy4Fi5cKH9/f91zzz12j9bi5syZo5tvvlm/+93v9C//8i/au3evVq1apVWrVtk92uXlAb7x/PPPe7p16+YJDAz0jBw50rN79267R2pxW7du9Uhq9EhMTLR7tBZzsfcvyfPSSy/ZPVqLu//++z3du3f3BAYGeq677jrPuHHjPJs2bbJ7LJ/w05/+1DNr1iy7x2hRkyZN8nTp0sUTGBjo+clPfuKZNGmS59ChQ3aPZZu33nrL079/f09QUJCnb9++nlWrVtk90mXHfZoAAAAMcE0TAACAAaIJAADAANEEAABggGgCAAAwQDQBAAAYIJoAAAAMEE0AAAAGiCYAAAADRBMA/ICsrCyFhob+3a/jcDi0fv36ZpkJgD2IJgBXvalTpyohIcHuMQBc4YgmAAAAA0QTgFbtmWee0YABA9S+fXtFRUXp3/7t31RVVdVo3fr163XDDTcoODhYLpdLx44d89r/xhtvaOjQoQoODlavXr302GOP6fz58y34TgBcbkQTgFbNz89Pzz33nA4ePKiXX35ZW7Zs0dy5c73WfPXVV3ryySf1pz/9STt37lRFRYUmT55s7X/33Xf1q1/9SrNmzdKHH36oP/zhD8rKytKTTz5pwzsCcLk4PB6Px+4hAOBymjp1qioqKowuxH799dc1ffp0ff7559I3F4JPmzZNu3fvVkxMjCSppKRE/fr10549ezRy5EjFxcVp3LhxSktLs17nf/7nfzR37lwdP35c+uZC8HXr1nFtFXAFC7B7AACw0+bNm5WRkaGSkhK53W6dP39eZ8+e1VdffaV27dpJkgICAjRixAjra/r27avQ0FB99NFHGjlypD744APt3LnT68xSXV1do9cBcGUjmgC0WkeOHNGdd96pGTNm6Mknn1THjh21Y8cOJSUlqba21jh2qqqq9Nhjj+nuu+9utC84OPgyTA7ADkQTgFarsLBQ9fX1WrJkifz8LlziuXbt2kbrzp8/r3379mnkyJGSpNLSUlVUVKhfv36SpKFDh6q0tFS9e/du4XcAoCURTQBahcrKShUVFXlt69y5s86dO6fnn39ed911l3bu3KnMzMxGX9umTRvNnDlTzz33nAICApSSkqJRo0ZZEZWenq4777xT3bp10y9+8Qv5+fnpgw8+UHFxsZ544okWe48ALi9+ew5Aq7Bt2zYNGTLE6/Hf//3feuaZZ/T73/9e/fv31yuvvKKMjIxGX9uuXTvNmzdP9957r0aPHq1rrrlGa9assfa7XC5lZ2dr06ZNGjFihEaNGqVnn31W3bt3b+F3CeBy4rfnAAAADHCmCQAAwADRBAAAYIBoAgAAMEA0AQAAGCCaAAAADBBNAAAABogmAAAAA0QTAACAAaIJAADAANEEAABggGgCAAAw8P8EXnfR5pfcLAAAAABJRU5ErkJggg==", "text/plain": [ "<Figure size 640x480 with 1 Axes>" ] @@ -290,31 +291,31 @@ } ], "source": [ - "plt.hist(mkbatch(2**15, large=False, target=\"onpath\")[1].cpu(), bins=5)" + "bins = torch.bincount(mkbatch(2**15, large=False, target=\"onpath\")[1].to(torch.uint8).cpu())\n", + "plt.bar(range(len(bins)), bins)\n", + "plt.xlabel(\"Label\")\n", + "plt.ylabel(\"Count\")" ] }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 11, "execution_state": "idle", "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "(array([2.0379e+04, 7.8380e+03, 3.0080e+03, 1.0690e+03, 3.4600e+02,\n", - " 9.8000e+01, 2.5000e+01, 5.0000e+00]),\n", - " array([1., 2., 3., 4., 5., 6., 7., 8., 9.]),\n", - " <BarContainer object of 8 artists>)" + "Text(0, 0.5, 'Count')" ] }, - "execution_count": 13, + "execution_count": 11, "metadata": {}, "output_type": "execute_result" }, { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjkAAAGdCAYAAADwjmIIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAwsElEQVR4nO3de3BUZZ7/8U8T7AQwHa65DSEEcAiXAHIxtBeUJZOAGcasjCKioEQZ3KBAFCHKYICdCQMFyBS3ZVXCrCDIlkQFBUIQGIaoEIwQHDKCMNGFDs4IaYgaSHJ+/2zOz15ADXZs8vh+VZ2qnOf59unv01r0p06fPu2wLMsSAACAYZoEugEAAICGQMgBAABGIuQAAAAjEXIAAICRCDkAAMBIhBwAAGAkQg4AADASIQcAABipaaAbCKTa2lqdPHlSoaGhcjgcgW4HAAB8D5Zl6dy5c4qOjlaTJlc+X/OTDjknT55UTExMoNsAAABX4dNPP1X79u2vOP+TDjmhoaHS/75ILpcr0O0AAIDvwev1KiYmxn4fv5KfdMip+4jK5XIRcgAAaGS+61ITLjwGAABGIuQAAAAjEXIAAICRCDkAAMBIhBwAAGAkQg4AADASIQcAABiJkAMAAIxEyAEAAEYi5AAAACMRcgAAgJEIOQAAwEiEHAAAYCRCDgAAMFLTQDdgqo7TNwe6hQZ3Ym5qoFsAAOCKOJMDAACMRMgBAABGqlfIycnJ0YABAxQaGqrw8HClpaWptLTUp+brr79WRkaG2rRpo+uvv14jRoxQeXm5T01ZWZlSU1PVvHlzhYeHa+rUqaqurvap2blzp/r27avg4GB16dJFubm5l/SzdOlSdezYUSEhIUpMTNT7779fv9UDAABj1Svk7Nq1SxkZGXr33XeVn5+vixcvKjk5WZWVlXbNlClT9Oabb2rDhg3atWuXTp48qbvvvtuer6mpUWpqqi5cuKC9e/dq9erVys3N1cyZM+2a48ePKzU1VYMHD1ZxcbEmT56sRx55RFu3brVr1q9fr8zMTD333HM6cOCAevfurZSUFJ0+ffqHvyoAAKDRc1iWZV3tgz///HOFh4dr165dGjRokCoqKtSuXTutXbtWv/71ryVJR44cUbdu3VRYWKiBAwfq7bff1i9/+UudPHlSERERkqQVK1Zo2rRp+vzzz+V0OjVt2jRt3rxZJSUl9nPdd999Onv2rLZs2SJJSkxM1IABA7RkyRJJUm1trWJiYvT4449r+vTp36t/r9ersLAwVVRUyOVyXe3LcFlceAwAQMP4vu/fP+ianIqKCklS69atJUlFRUW6ePGikpKS7Jr4+Hh16NBBhYWFkqTCwkIlJCTYAUeSUlJS5PV6dfjwYbvmm8eoq6k7xoULF1RUVORT06RJEyUlJdk1l1NVVSWv1+uzAQAAM111yKmtrdXkyZN1yy23qGfPnpIkj8cjp9Opli1b+tRGRETI4/HYNd8MOHXzdXPfVuP1evXVV1/pH//4h2pqai5bU3eMy8nJyVFYWJi9xcTEXO3yAQDANe6qQ05GRoZKSkq0bt06/3bUgLKyslRRUWFvn376aaBbAgAADeSqbgY4ceJEbdq0Sbt371b79u3t8cjISF24cEFnz571OZtTXl6uyMhIu+b/fguq7ttX36z5v9/IKi8vl8vlUrNmzRQUFKSgoKDL1tQd43KCg4MVHBx8NUsGAACNTL3O5FiWpYkTJ2rjxo3asWOH4uLifOb79eun6667TgUFBfZYaWmpysrK5Ha7JUlut1uHDh3y+RZUfn6+XC6Xunfvbtd88xh1NXXHcDqd6tevn09NbW2tCgoK7BoAAPDTVq8zORkZGVq7dq1ef/11hYaG2te/hIWFqVmzZgoLC1N6eroyMzPVunVruVwuPf7443K73Ro4cKAkKTk5Wd27d9eDDz6oefPmyePxaMaMGcrIyLDPskyYMEFLlizR008/rXHjxmnHjh169dVXtXnz///GUmZmpsaOHav+/fvrpptu0vPPP6/Kyko9/PDD/n2FAABAo1SvkLN8+XJJ0h133OEzvmrVKj300EOSpEWLFqlJkyYaMWKEqqqqlJKSomXLltm1QUFB2rRpkx577DG53W61aNFCY8eO1ezZs+2auLg4bd68WVOmTNHixYvVvn17vfDCC0pJSbFrRo4cqc8//1wzZ86Ux+NRnz59tGXLlksuRgYAAD9NP+g+OY0d98n5YbhPDgAgEH6U++QAAABcqwg5AADASIQcAABgJEIOAAAwEiEHAAAYiZADAACMRMgBAABGIuQAAAAjEXIAAICRCDkAAMBIhBwAAGAkQg4AADASIQcAABiJkAMAAIxEyAEAAEYi5AAAACMRcgAAgJEIOQAAwEiEHAAAYCRCDgAAMBIhBwAAGImQAwAAjETIAQAARiLkAAAAIxFyAACAkQg5AADASIQcAABgJEIOAAAwEiEHAAAYiZADAACMRMgBAABGIuQAAAAj1Tvk7N69W8OHD1d0dLQcDofy8vJ85h0Ox2W3+fPn2zUdO3a8ZH7u3Lk+xzl48KBuu+02hYSEKCYmRvPmzbuklw0bNig+Pl4hISFKSEjQW2+9Vd/lAAAAQ9U75FRWVqp3795aunTpZedPnTrls7300ktyOBwaMWKET93s2bN96h5//HF7zuv1Kjk5WbGxsSoqKtL8+fOVnZ2tlStX2jV79+7VqFGjlJ6erg8++EBpaWlKS0tTSUlJfZcEAAAM1LS+Dxg2bJiGDRt2xfnIyEif/ddff12DBw9Wp06dfMZDQ0Mvqa2zZs0aXbhwQS+99JKcTqd69Oih4uJiLVy4UOPHj5ckLV68WEOHDtXUqVMlSXPmzFF+fr6WLFmiFStW1HdZAADAMA16TU55ebk2b96s9PT0S+bmzp2rNm3a6MYbb9T8+fNVXV1tzxUWFmrQoEFyOp32WEpKikpLS3XmzBm7JikpyeeYKSkpKiwsvGI/VVVV8nq9PhsAADBTvc/k1Mfq1asVGhqqu+++22f8iSeeUN++fdW6dWvt3btXWVlZOnXqlBYuXChJ8ng8iouL83lMRESEPdeqVSt5PB577Js1Ho/niv3k5ORo1qxZflwhAAC4VjVoyHnppZc0evRohYSE+IxnZmbaf/fq1UtOp1O/+c1vlJOTo+Dg4AbrJysry+e5vV6vYmJiGuz5AABA4DRYyPnzn/+s0tJSrV+//jtrExMTVV1drRMnTqhr166KjIxUeXm5T03dft11PFequdJ1PpIUHBzcoCEKAABcOxrsmpwXX3xR/fr1U+/evb+ztri4WE2aNFF4eLgkye12a/fu3bp48aJdk5+fr65du6pVq1Z2TUFBgc9x8vPz5Xa7/b4WAADQ+NQ75Jw/f17FxcUqLi6WJB0/flzFxcUqKyuza7xerzZs2KBHHnnkkscXFhbq+eef14cffqhPPvlEa9as0ZQpU/TAAw/YAeb++++X0+lUenq6Dh8+rPXr12vx4sU+HzVNmjRJW7Zs0YIFC3TkyBFlZ2dr//79mjhx4tW+FgAAwCD1/rhq//79Gjx4sL1fFzzGjh2r3NxcSdK6detkWZZGjRp1yeODg4O1bt06ZWdnq6qqSnFxcZoyZYpPgAkLC9O2bduUkZGhfv36qW3btpo5c6b99XFJuvnmm7V27VrNmDFDzzzzjG644Qbl5eWpZ8+e9X8VAACAcRyWZVmBbiJQvF6vwsLCVFFRIZfL5ddjd5y+2a/HuxadmJsa6BYAAD9B3/f9m9+uAgAARiLkAAAAIxFyAACAkQg5AADASIQcAABgJEIOAAAwEiEHAAAYiZADAACMRMgBAABGIuQAAAAjEXIAAICRCDkAAMBIhBwAAGAkQg4AADASIQcAABiJkAMAAIxEyAEAAEYi5AAAACMRcgAAgJEIOQAAwEiEHAAAYCRCDgAAMBIhBwAAGImQAwAAjETIAQAARiLkAAAAIxFyAACAkQg5AADASIQcAABgJEIOAAAwEiEHAAAYiZADAACMVO+Qs3v3bg0fPlzR0dFyOBzKy8vzmX/ooYfkcDh8tqFDh/rUfPHFFxo9erRcLpdatmyp9PR0nT9/3qfm4MGDuu222xQSEqKYmBjNmzfvkl42bNig+Ph4hYSEKCEhQW+99VZ9lwMAAAxV75BTWVmp3r17a+nSpVesGTp0qE6dOmVvr7zyis/86NGjdfjwYeXn52vTpk3avXu3xo8fb897vV4lJycrNjZWRUVFmj9/vrKzs7Vy5Uq7Zu/evRo1apTS09P1wQcfKC0tTWlpaSopKanvkgAAgIEclmVZV/1gh0MbN25UWlqaPfbQQw/p7Nmzl5zhqfPXv/5V3bt31759+9S/f39J0pYtW3TnnXfqs88+U3R0tJYvX65nn31WHo9HTqdTkjR9+nTl5eXpyJEjkqSRI0eqsrJSmzZtso89cOBA9enTRytWrPhe/Xu9XoWFhamiokIul+tqX4bL6jh9s1+Pdy06MTc10C0AAH6Cvu/7d4Nck7Nz506Fh4era9eueuyxx/TPf/7TnissLFTLli3tgCNJSUlJatKkid577z27ZtCgQXbAkaSUlBSVlpbqzJkzdk1SUpLP86akpKiwsPCKfVVVVcnr9fpsAADATH4POUOHDtWf/vQnFRQU6A9/+IN27dqlYcOGqaamRpLk8XgUHh7u85imTZuqdevW8ng8dk1ERIRPTd3+d9XUzV9OTk6OwsLC7C0mJsZPqwYAANeapv4+4H333Wf/nZCQoF69eqlz587auXOnhgwZ4u+nq5esrCxlZmba+16vl6ADAIChGvwr5J06dVLbtm119OhRSVJkZKROnz7tU1NdXa0vvvhCkZGRdk15eblPTd3+d9XUzV9OcHCwXC6XzwYAAMzU4CHns88+0z//+U9FRUVJktxut86ePauioiK7ZseOHaqtrVViYqJds3v3bl28eNGuyc/PV9euXdWqVSu7pqCgwOe58vPz5Xa7G3pJAACgEah3yDl//ryKi4tVXFwsSTp+/LiKi4tVVlam8+fPa+rUqXr33Xd14sQJFRQU6K677lKXLl2UkpIiSerWrZuGDh2qRx99VO+//77+8pe/aOLEibrvvvsUHR0tSbr//vvldDqVnp6uw4cPa/369Vq8eLHPR02TJk3Sli1btGDBAh05ckTZ2dnav3+/Jk6c6L9XBwAANFr1Djn79+/XjTfeqBtvvFGSlJmZqRtvvFEzZ85UUFCQDh48qF/96lf6+c9/rvT0dPXr109//vOfFRwcbB9jzZo1io+P15AhQ3TnnXfq1ltv9bkHTlhYmLZt26bjx4+rX79+evLJJzVz5kyfe+ncfPPNWrt2rVauXKnevXvrv//7v5WXl6eePXv+8FcFAAA0ej/oPjmNHffJ+WG4Tw4AIBACep8cAACAQCPkAAAAIxFyAACAkQg5AADASIQcAABgJEIOAAAwEiEHAAAYiZADAACMRMgBAABGIuQAAAAjEXIAAICRCDkAAMBIhBwAAGAkQg4AADASIQcAABiJkAMAAIxEyAEAAEYi5AAAACMRcgAAgJEIOQAAwEiEHAAAYCRCDgAAMBIhBwAAGImQAwAAjETIAQAARiLkAAAAIxFyAACAkQg5AADASIQcAABgJEIOAAAwEiEHAAAYiZADAACMVO+Qs3v3bg0fPlzR0dFyOBzKy8uz5y5evKhp06YpISFBLVq0UHR0tMaMGaOTJ0/6HKNjx45yOBw+29y5c31qDh48qNtuu00hISGKiYnRvHnzLullw4YNio+PV0hIiBISEvTWW2/VdzkAAMBQ9Q45lZWV6t27t5YuXXrJ3JdffqkDBw7ot7/9rQ4cOKDXXntNpaWl+tWvfnVJ7ezZs3Xq1Cl7e/zxx+05r9er5ORkxcbGqqioSPPnz1d2drZWrlxp1+zdu1ejRo1Senq6PvjgA6WlpSktLU0lJSX1XRIAADBQ0/o+YNiwYRo2bNhl58LCwpSfn+8ztmTJEt10000qKytThw4d7PHQ0FBFRkZe9jhr1qzRhQsX9NJLL8npdKpHjx4qLi7WwoULNX78eEnS4sWLNXToUE2dOlWSNGfOHOXn52vJkiVasWJFfZcFAAAM0+DX5FRUVMjhcKhly5Y+43PnzlWbNm104403av78+aqurrbnCgsLNWjQIDmdTnssJSVFpaWlOnPmjF2TlJTkc8yUlBQVFhZesZeqqip5vV6fDQAAmKneZ3Lq4+uvv9a0adM0atQouVwue/yJJ55Q37591bp1a+3du1dZWVk6deqUFi5cKEnyeDyKi4vzOVZERIQ916pVK3k8HnvsmzUej+eK/eTk5GjWrFl+XiUAALgWNVjIuXjxou69915ZlqXly5f7zGVmZtp/9+rVS06nU7/5zW+Uk5Oj4ODghmpJWVlZPs/t9XoVExPTYM8HAAACp0FCTl3A+fvf/64dO3b4nMW5nMTERFVXV+vEiRPq2rWrIiMjVV5e7lNTt193Hc+Vaq50nY8kBQcHN2iIAgAA1w6/X5NTF3A+/vhjbd++XW3atPnOxxQXF6tJkyYKDw+XJLndbu3evVsXL160a/Lz89W1a1e1atXKrikoKPA5Tn5+vtxut7+XBAAAGqF6n8k5f/68jh49au8fP35cxcXFat26taKiovTrX/9aBw4c0KZNm1RTU2NfI9O6dWs5nU4VFhbqvffe0+DBgxUaGqrCwkJNmTJFDzzwgB1g7r//fs2aNUvp6emaNm2aSkpKtHjxYi1atMh+3kmTJun222/XggULlJqaqnXr1mn//v0+XzMHAAA/XQ7Lsqz6PGDnzp0aPHjwJeNjx45Vdnb2JRcM13nnnXd0xx136MCBA/q3f/s3HTlyRFVVVYqLi9ODDz6ozMxMn4+SDh48qIyMDO3bt09t27bV448/rmnTpvkcc8OGDZoxY4ZOnDihG264QfPmzdOdd975vdfi9XoVFhamioqK7/xIrb46Tt/s1+Ndi07MTQ10CwCAn6Dv+/5d75BjEkLOD0PIAQAEwvd9/+a3qwAAgJEIOQAAwEiEHAAAYCRCDgAAMBIhBwAAGImQAwAAjETIAQAARiLkAAAAIxFyAACAkQg5AADASIQcAABgJEIOAAAwEiEHAAAYiZADAACMRMgBAABGIuQAAAAjEXIAAICRCDkAAMBIhBwAAGAkQg4AADASIQcAABiJkAMAAIxEyAEAAEYi5AAAACMRcgAAgJEIOQAAwEiEHAAAYCRCDgAAMBIhBwAAGImQAwAAjETIAQAARiLkAAAAI9U75OzevVvDhw9XdHS0HA6H8vLyfOYty9LMmTMVFRWlZs2aKSkpSR9//LFPzRdffKHRo0fL5XKpZcuWSk9P1/nz531qDh48qNtuu00hISGKiYnRvHnzLullw4YNio+PV0hIiBISEvTWW2/VdzkAAMBQ9Q45lZWV6t27t5YuXXrZ+Xnz5umPf/yjVqxYoffee08tWrRQSkqKvv76a7tm9OjROnz4sPLz87Vp0ybt3r1b48ePt+e9Xq+Sk5MVGxuroqIizZ8/X9nZ2Vq5cqVds3fvXo0aNUrp6en64IMPlJaWprS0NJWUlNT/VQAAAMZxWJZlXfWDHQ5t3LhRaWlp0v+exYmOjtaTTz6pp556SpJUUVGhiIgI5ebm6r777tNf//pXde/eXfv27VP//v0lSVu2bNGdd96pzz77TNHR0Vq+fLmeffZZeTweOZ1OSdL06dOVl5enI0eOSJJGjhypyspKbdq0ye5n4MCB6tOnj1asWPG9+vd6vQoLC1NFRYVcLtfVvgyX1XH6Zr8e71p0Ym5qoFsAAPwEfd/3b79ek3P8+HF5PB4lJSXZY2FhYUpMTFRhYaEkqbCwUC1btrQDjiQlJSWpSZMmeu+99+yaQYMG2QFHklJSUlRaWqozZ87YNd98nrqauucBAAA/bU39eTCPxyNJioiI8BmPiIiw5zwej8LDw32baNpUrVu39qmJi4u75Bh1c61atZLH4/nW57mcqqoqVVVV2fter/cqVwoAAK51P6lvV+Xk5CgsLMzeYmJiAt0SAABoIH4NOZGRkZKk8vJyn/Hy8nJ7LjIyUqdPn/aZr66u1hdffOFTc7ljfPM5rlRTN385WVlZqqiosLdPP/30B6wWAABcy/wacuLi4hQZGamCggJ7zOv16r333pPb7ZYkud1unT17VkVFRXbNjh07VFtbq8TERLtm9+7dunjxol2Tn5+vrl27qlWrVnbNN5+nrqbueS4nODhYLpfLZwMAAGaqd8g5f/68iouLVVxcLP3vxcbFxcUqKyuTw+HQ5MmT9e///u964403dOjQIY0ZM0bR0dH2N7C6deumoUOH6tFHH9X777+vv/zlL5o4caLuu+8+RUdHS5Luv/9+OZ1Opaen6/Dhw1q/fr0WL16szMxMu49JkyZpy5YtWrBggY4cOaLs7Gzt379fEydO9N+rAwAAGq16X3i8f/9+DR482N6vCx5jx45Vbm6unn76aVVWVmr8+PE6e/asbr31Vm3ZskUhISH2Y9asWaOJEydqyJAhatKkiUaMGKE//vGP9nxYWJi2bdumjIwM9evXT23bttXMmTN97qVz8803a+3atZoxY4aeeeYZ3XDDDcrLy1PPnj1/yOsBAAAM8YPuk9PYcZ+cH4b75AAAAiEg98kBAAC4VhByAACAkQg5AADASIQcAABgJEIOAAAwEiEHAAAYiZADAACMRMgBAABGIuQAAAAjEXIAAICRCDkAAMBIhBwAAGAkQg4AADASIQcAABiJkAMAAIxEyAEAAEYi5AAAACMRcgAAgJEIOQAAwEiEHAAAYCRCDgAAMBIhBwAAGImQAwAAjETIAQAARiLkAAAAIxFyAACAkQg5AADASIQcAABgJEIOAAAwEiEHAAAYiZADAACM1DTQDaDx6jh9c6BbaFAn5qYGugUAwA/AmRwAAGAkv4ecjh07yuFwXLJlZGRIku64445L5iZMmOBzjLKyMqWmpqp58+YKDw/X1KlTVV1d7VOzc+dO9e3bV8HBwerSpYtyc3P9vRQAANCI+f3jqn379qmmpsbeLykp0S9+8Qvdc8899tijjz6q2bNn2/vNmze3/66pqVFqaqoiIyO1d+9enTp1SmPGjNF1112n3//+95Kk48ePKzU1VRMmTNCaNWtUUFCgRx55RFFRUUpJSfH3kgAAQCPk95DTrl07n/25c+eqc+fOuv322+2x5s2bKzIy8rKP37Ztmz766CNt375dERER6tOnj+bMmaNp06YpOztbTqdTK1asUFxcnBYsWCBJ6tatm/bs2aNFixYRcgAAgNTQ1+RcuHBBL7/8ssaNGyeHw2GPr1mzRm3btlXPnj2VlZWlL7/80p4rLCxUQkKCIiIi7LGUlBR5vV4dPnzYrklKSvJ5rpSUFBUWFn5rP1VVVfJ6vT4bAAAwU4N+uyovL09nz57VQw89ZI/df//9io2NVXR0tA4ePKhp06aptLRUr732miTJ4/H4BBxJ9r7H4/nWGq/Xq6+++krNmjW7bD85OTmaNWuW39cJAACuPQ0acl588UUNGzZM0dHR9tj48ePtvxMSEhQVFaUhQ4bo2LFj6ty5c0O2o6ysLGVmZtr7Xq9XMTExDfqcAAAgMBos5Pz973/X9u3b7TM0V5KYmChJOnr0qDp37qzIyEi9//77PjXl5eWSZF/HExkZaY99s8blcl3xLI4kBQcHKzg4+KrXBAAAGo8GuyZn1apVCg8PV2rqt99Qrbi4WJIUFRUlSXK73Tp06JBOnz5t1+Tn58vlcql79+52TUFBgc9x8vPz5Xa7G2AlAACgMWqQkFNbW6tVq1Zp7Nixatr0/58sOnbsmObMmaOioiKdOHFCb7zxhsaMGaNBgwapV69ekqTk5GR1795dDz74oD788ENt3bpVM2bMUEZGhn0WZsKECfrkk0/09NNP68iRI1q2bJleffVVTZkypSGWAwAAGqEGCTnbt29XWVmZxo0b5zPudDq1fft2JScnKz4+Xk8++aRGjBihN998064JCgrSpk2bFBQUJLfbrQceeEBjxozxua9OXFycNm/erPz8fPXu3VsLFizQCy+8wNfHAQCAzWFZlhXoJgLF6/UqLCxMFRUVcrlcfj226b/r9FPAb1cBwLXp+75/89tVAADASIQcAABgJEIOAAAwEiEHAAAYiZADAACMRMgBAABGIuQAAAAjEXIAAICRCDkAAMBIhBwAAGAkQg4AADASIQcAABiJkAMAAIxEyAEAAEYi5AAAACMRcgAAgJEIOQAAwEiEHAAAYCRCDgAAMBIhBwAAGImQAwAAjETIAQAARiLkAAAAIxFyAACAkQg5AADASIQcAABgJEIOAAAwEiEHAAAYiZADAACMRMgBAABGIuQAAAAjEXIAAICR/B5ysrOz5XA4fLb4+Hh7/uuvv1ZGRobatGmj66+/XiNGjFB5ebnPMcrKypSamqrmzZsrPDxcU6dOVXV1tU/Nzp071bdvXwUHB6tLly7Kzc3191IAAEAj1iBncnr06KFTp07Z2549e+y5KVOm6M0339SGDRu0a9cunTx5Unfffbc9X1NTo9TUVF24cEF79+7V6tWrlZubq5kzZ9o1x48fV2pqqgYPHqzi4mJNnjxZjzzyiLZu3doQywEAAI1Q0wY5aNOmioyMvGS8oqJCL774otauXat/+Zd/kSStWrVK3bp107vvvquBAwdq27Zt+uijj7R9+3ZFRESoT58+mjNnjqZNm6bs7Gw5nU6tWLFCcXFxWrBggSSpW7du2rNnjxYtWqSUlJSGWBIAAGhkGuRMzscff6zo6Gh16tRJo0ePVllZmSSpqKhIFy9eVFJSkl0bHx+vDh06qLCwUJJUWFiohIQERURE2DUpKSnyer06fPiwXfPNY9TV1B3jSqqqquT1en02AABgJr+HnMTEROXm5mrLli1avny5jh8/rttuu03nzp2Tx+OR0+lUy5YtfR4TEREhj8cjSfJ4PD4Bp26+bu7barxer7766qsr9paTk6OwsDB7i4mJ8du6AQDAtcXvH1cNGzbM/rtXr15KTExUbGysXn31VTVr1szfT1cvWVlZyszMtPe9Xi9BBwAAQzX4V8hbtmypn//85zp69KgiIyN14cIFnT171qemvLzcvoYnMjLykm9b1e1/V43L5frWIBUcHCyXy+WzAQAAMzV4yDl//ryOHTumqKgo9evXT9ddd50KCgrs+dLSUpWVlcntdkuS3G63Dh06pNOnT9s1+fn5crlc6t69u13zzWPU1dQdAwAAwO8h56mnntKuXbt04sQJ7d27V//6r/+qoKAgjRo1SmFhYUpPT1dmZqbeeecdFRUV6eGHH5bb7dbAgQMlScnJyerevbsefPBBffjhh9q6datmzJihjIwMBQcHS5ImTJigTz75RE8//bSOHDmiZcuW6dVXX9WUKVP8vRwAANBI+f2anM8++0yjRo3SP//5T7Vr10633nqr3n33XbVr106StGjRIjVp0kQjRoxQVVWVUlJStGzZMvvxQUFB2rRpkx577DG53W61aNFCY8eO1ezZs+2auLg4bd68WVOmTNHixYvVvn17vfDCC3x9HAAA2ByWZVmBbiJQvF6vwsLCVFFR4ffrczpO3+zX4+HHd2JuaqBbAABcxvd9/+a3qwAAgJEIOQAAwEiEHAAAYCRCDgAAMBIhBwAAGImQAwAAjETIAQAARiLkAAAAIxFyAACAkQg5AADASIQcAABgJEIOAAAwEiEHAAAYiZADAACMRMgBAABGIuQAAAAjEXIAAICRCDkAAMBIhBwAAGAkQg4AADASIQcAABiJkAMAAIxEyAEAAEYi5AAAACMRcgAAgJEIOQAAwEiEHAAAYKSmgW4AuFZ1nL450C00qBNzUwPdAgA0KM7kAAAAIxFyAACAkQg5AADASIQcAABgJL+HnJycHA0YMEChoaEKDw9XWlqaSktLfWruuOMOORwOn23ChAk+NWVlZUpNTVXz5s0VHh6uqVOnqrq62qdm586d6tu3r4KDg9WlSxfl5ub6ezkAAKCR8nvI2bVrlzIyMvTuu+8qPz9fFy9eVHJysiorK33qHn30UZ06dcre5s2bZ8/V1NQoNTVVFy5c0N69e7V69Wrl5uZq5syZds3x48eVmpqqwYMHq7i4WJMnT9YjjzyirVu3+ntJAACgEfL7V8i3bNnis5+bm6vw8HAVFRVp0KBB9njz5s0VGRl52WNs27ZNH330kbZv366IiAj16dNHc+bM0bRp05SdnS2n06kVK1YoLi5OCxYskCR169ZNe/bs0aJFi5SSkuLvZQEAgEamwa/JqaiokCS1bt3aZ3zNmjVq27atevbsqaysLH355Zf2XGFhoRISEhQREWGPpaSkyOv16vDhw3ZNUlKSzzFTUlJUWFh4xV6qqqrk9Xp9NgAAYKYGvRlgbW2tJk+erFtuuUU9e/a0x++//37FxsYqOjpaBw8e1LRp01RaWqrXXntNkuTxeHwCjiR73+PxfGuN1+vVV199pWbNml3ST05OjmbNmtUgawUAANeWBg05GRkZKikp0Z49e3zGx48fb/+dkJCgqKgoDRkyRMeOHVPnzp0brJ+srCxlZmba+16vVzExMQ32fAAAIHAa7OOqiRMnatOmTXrnnXfUvn37b61NTEyUJB09elSSFBkZqfLycp+auv2663iuVONyuS57FkeSgoOD5XK5fDYAAGAmv4ccy7I0ceJEbdy4UTt27FBcXNx3Pqa4uFiSFBUVJUlyu906dOiQTp8+bdfk5+fL5XKpe/fudk1BQYHPcfLz8+V2u/28IgAA0Bj5PeRkZGTo5Zdf1tq1axUaGiqPxyOPx6OvvvpKknTs2DHNmTNHRUVFOnHihN544w2NGTNGgwYNUq9evSRJycnJ6t69ux588EF9+OGH2rp1q2bMmKGMjAwFBwdLkiZMmKBPPvlETz/9tI4cOaJly5bp1Vdf1ZQpU/y9JAAA0Aj5PeQsX75cFRUVuuOOOxQVFWVv69evlyQ5nU5t375dycnJio+P15NPPqkRI0bozTfftI8RFBSkTZs2KSgoSG63Ww888IDGjBmj2bNn2zVxcXHavHmz8vPz1bt3by1YsEAvvPACXx8HAACSJIdlWVagmwgUr9ersLAwVVRU+P36nI7TN/v1eIC/nZibGugWAOCqfN/3b367CgAAGImQAwAAjETIAQAARiLkAAAAIxFyAACAkQg5AADASIQcAABgJEIOAAAwEiEHAAAYiZADAACMRMgBAABGIuQAAAAjEXIAAICRCDkAAMBIhBwAAGAkQg4AADASIQcAABiJkAMAAIzUNNANAAiMjtM3B7qFBndibmqgWwAQQJzJAQAARiLkAAAAIxFyAACAkQg5AADASIQcAABgJEIOAAAwEiEHAAAYiZADAACMRMgBAABGIuQAAAAjEXIAAICR+O0qAMYy/fe5+G0u4NtxJgcAABip0YecpUuXqmPHjgoJCVFiYqLef//9QLcEAACuAY065Kxfv16ZmZl67rnndODAAfXu3VspKSk6ffp0oFsDAAAB1qhDzsKFC/Xoo4/q4YcfVvfu3bVixQo1b95cL730UqBbAwAAAdZoLzy+cOGCioqKlJWVZY81adJESUlJKiwsvOxjqqqqVFVVZe9XVFRIkrxer9/7q6360u/HBIBv6jBlQ6BbaFAls1IC3QKuUXXv25ZlfWtdow05//jHP1RTU6OIiAif8YiICB05cuSyj8nJydGsWbMuGY+JiWmwPgEAVyfs+UB3gGvduXPnFBYWdsX5RhtyrkZWVpYyMzPt/draWn3xxRdq06aNHA6H357H6/UqJiZGn376qVwul9+Oey0xfY2sr/EzfY2sr/EzfY0NuT7LsnTu3DlFR0d/a12jDTlt27ZVUFCQysvLfcbLy8sVGRl52ccEBwcrODjYZ6xly5YN1qPL5TLyf9xvMn2NrK/xM32NrK/xM32NDbW+bzuDU6fRXnjsdDrVr18/FRQU2GO1tbUqKCiQ2+0OaG8AACDwGu2ZHEnKzMzU2LFj1b9/f9100016/vnnVVlZqYcffjjQrQEAgABr1CFn5MiR+vzzzzVz5kx5PB716dNHW7ZsueRi5B9bcHCwnnvuuUs+GjOJ6WtkfY2f6WtkfY2f6Wu8FtbnsL7r+1cAAACNUKO9JgcAAODbEHIAAICRCDkAAMBIhBwAAGAkQo4f7d69W8OHD1d0dLQcDofy8vIC3ZJf5eTkaMCAAQoNDVV4eLjS0tJUWloa6Lb8avny5erVq5d98yq3262333470G01mLlz58rhcGjy5MmBbsUvsrOz5XA4fLb4+PhAt+V3//M//6MHHnhAbdq0UbNmzZSQkKD9+/cHui2/6Nix4yX/DR0OhzIyMgLdml/U1NTot7/9reLi4tSsWTN17txZc+bM+c7fYGpMzp07p8mTJys2NlbNmjXTzTffrH379gWkl0b9FfJrTWVlpXr37q1x48bp7rvvDnQ7frdr1y5lZGRowIABqq6u1jPPPKPk5GR99NFHatGiRaDb84v27dtr7ty5uuGGG2RZllavXq277rpLH3zwgXr06BHo9vxq3759+o//+A/16tUr0K34VY8ePbR9+3Z7v2lTs/6ZO3PmjG655RYNHjxYb7/9ttq1a6ePP/5YrVq1CnRrfrFv3z7V1NTY+yUlJfrFL36he+65J6B9+csf/vAHLV++XKtXr1aPHj20f/9+PfzwwwoLC9MTTzwR6Pb84pFHHlFJSYn+67/+S9HR0Xr55ZeVlJSkjz76SD/72c9+3GYsNAhJ1saNGwPdRoM6ffq0JcnatWtXoFtpUK1atbJeeOGFQLfhV+fOnbNuuOEGKz8/37r99tutSZMmBbolv3juuees3r17B7qNBjVt2jTr1ltvDXQbP5pJkyZZnTt3tmprawPdil+kpqZa48aN8xm7++67rdGjRwesJ3/68ssvraCgIGvTpk0+43379rWeffbZH70fPq7CVauoqJAktW7dOtCtNIiamhqtW7dOlZWVxv1USEZGhlJTU5WUlBToVvzu448/VnR0tDp16qTRo0errKws0C351RtvvKH+/fvrnnvuUXh4uG688Ub953/+Z6DbahAXLlzQyy+/rHHjxvn1R5QD6eabb1ZBQYH+9re/SZI+/PBD7dmzR8OGDQt0a35RXV2tmpoahYSE+Iw3a9ZMe/bs+dH7Mes8Ln40tbW1mjx5sm655Rb17Nkz0O341aFDh+R2u/X111/r+uuv18aNG9W9e/dAt+U369at04EDBwL2GXlDSkxMVG5urrp27apTp05p1qxZuu2221RSUqLQ0NBAt+cXn3zyiZYvX67MzEw988wz2rdvn5544gk5nU6NHTs20O35VV5ens6ePauHHnoo0K34zfTp0+X1ehUfH6+goCDV1NTod7/7nUaPHh3o1vwiNDRUbrdbc+bMUbdu3RQREaFXXnlFhYWF6tKly4/f0I9+7ugnwvSPqyZMmGDFxsZan376aaBb8buqqirr448/tvbv329Nnz7datu2rXX48OFAt+UXZWVlVnh4uPXhhx/aYyZ9XPV/nTlzxnK5XEZ93HjddddZbrfbZ+zxxx+3Bg4cGLCeGkpycrL1y1/+MtBt+NUrr7xitW/f3nrllVesgwcPWn/605+s1q1bW7m5uYFuzW+OHj1qDRo0yJJkBQUFWQMGDLBGjx5txcfH/+i9EHIaiMkhJyMjw2rfvr31ySefBLqVH8WQIUOs8ePHB7oNv9i4caP9D0/dJslyOBxWUFCQVV1dHegW/a5///7W9OnTA92G33To0MFKT0/3GVu2bJkVHR0dsJ4awokTJ6wmTZpYeXl5gW7Fr9q3b28tWbLEZ2zOnDlW165dA9ZTQzl//rx18uRJy7Is695777XuvPPOH70HrsnB92ZZliZOnKiNGzdqx44diouLC3RLP4ra2lpVVVUFug2/GDJkiA4dOqTi4mJ769+/v0aPHq3i4mIFBQUFukW/On/+vI4dO6aoqKhAt+I3t9xyyyW3bvjb3/6m2NjYgPXUEFatWqXw8HClpqYGuhW/+vLLL9Wkie9bb1BQkGprawPWU0Np0aKFoqKidObMGW3dulV33XXXj94D1+T40fnz53X06FF7//jx4youLlbr1q3VoUOHgPbmDxkZGVq7dq1ef/11hYaGyuPxSJLCwsLUrFmzQLfnF1lZWRo2bJg6dOigc+fOae3atdq5c6e2bt0a6Nb8IjQ09JJrqFq0aKE2bdoYcW3VU089peHDhys2NlYnT57Uc889p6CgII0aNSrQrfnNlClTdPPNN+v3v/+97r33Xr3//vtauXKlVq5cGejW/Ka2tlarVq3S2LFjjbsFwPDhw/W73/1OHTp0UI8ePfTBBx9o4cKFGjduXKBb85utW7fKsix17dpVR48e1dSpUxUfH6+HH374x2/mRz93ZLB33nnHknTJNnbs2EC35heXW5ska9WqVYFuzW/GjRtnxcbGWk6n02rXrp01ZMgQa9u2bYFuq0GZdE3OyJEjraioKMvpdFo/+9nPrJEjR1pHjx4NdFt+9+abb1o9e/a0goODrfj4eGvlypWBbsmvtm7dakmySktLA92K33m9XmvSpElWhw4drJCQEKtTp07Ws88+a1VVVQW6Nb9Zv3691alTJ8vpdFqRkZFWRkaGdfbs2YD04rBMus0iAADA/+KaHAAAYCRCDgAAMBIhBwAAGImQAwAAjETIAQAARiLkAAAAIxFyAACAkQg5AADASIQcAABgJEIOAAAwEiEHAAAYiZADAACM9P8AMCmCWHBic9EAAAAASUVORK5CYII=", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAk0AAAGwCAYAAAC0HlECAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA09klEQVR4nO3de1RVdf7/8dcBBdSB4y1uK1KyUlG8K1KToyPD0aj58s1m8lJqkY4NmErjhcYMtcJ04aXxwvjtQvNNv6nzTWvQURFTUigVRcOC6aKjpQebDE5qocD5/fGL/fWE1tYBzwGej7X2WuzP5332ee+dLV5rn8/ZWJxOp1MAAAD4UV7ubgAAAKAhIDQBAACYQGgCAAAwgdAEAABgAqEJAADABEITAACACYQmAAAAE5q5u4HGorq6WqdOnZK/v78sFou72wEAACY4nU598803Cg0NlZfXj99LIjTVkVOnTiksLMzdbQAAgOtw8uRJ3XzzzT9aQ2iqI/7+/tL3Fz0gIMDd7QAAABMcDofCwsKM3+M/htBUR2o+kgsICCA0AQDQwJhZWsNCcAAAABMITQAAACYQmgAAAEwgNAEAAJhAaAIAADCB0AQAAGACoQkAAMAEQhMAAIAJhCYAAAATCE0AAAAmEJoAAABMIDQBAACYQGgCAAAwgdAEAABggltDU1pamvr37y9/f38FBgYqPj5eJSUlLjXfffedEhMT1a5dO/3sZz/TiBEjVFpa6lJz4sQJxcXFqWXLlgoMDNT06dNVWVnpUrNr1y716dNHvr6+uu2225SZmVmrnxUrVqhjx47y8/NTVFSU9u3bV09nDgAAGppm7nzz3bt3KzExUf3791dlZaWeeuopxcbG6sMPP1SrVq0kSdOmTdPmzZu1YcMGWa1WJSUl6f7779fevXslSVVVVYqLi1NwcLDy8vJ0+vRpjR07Vs2bN9fzzz8vSTp27Jji4uI0adIkrVmzRjk5OXrssccUEhIim80mSVq3bp2Sk5OVkZGhqKgoLV26VDabTSUlJQoMDHTjVWrYOs7a7O4Wajm+IM7dLQAAGiCL0+l0uruJGl9++aUCAwO1e/duDRo0SOXl5brpppu0du1aPfDAA5Kk4uJide3aVfn5+Ro4cKD+/ve/695779WpU6cUFBQkScrIyNDMmTP15ZdfysfHRzNnztTmzZtVVFRkvNfIkSNVVlamrVu3SpKioqLUv39/LV++XJJUXV2tsLAwTZ48WbNmzfrJ3h0Oh6xWq8rLyxUQEFBPV6jhITQBADzZtfz+9qg1TeXl5ZKktm3bSpIKCgp06dIlxcTEGDVdunTRLbfcovz8fElSfn6+IiMjjcAkSTabTQ6HQ0ePHjVqLj9GTU3NMS5evKiCggKXGi8vL8XExBg1P1RRUSGHw+GyAQCAxstjQlN1dbWmTp2qu+66S927d5ck2e12+fj4qHXr1i61QUFBstvtRs3lgalmvmbux2ocDoe+/fZb/etf/1JVVdUVa2qO8UNpaWmyWq3GFhYW9m9fAwAA4Lk8JjQlJiaqqKhIb7zxhrtbMSUlJUXl5eXGdvLkSXe3BAAA6pFbF4LXSEpKUlZWlnJzc3XzzTcb48HBwbp48aLKyspc7jaVlpYqODjYqPnht9xqvl13ec0Pv3FXWlqqgIAAtWjRQt7e3vL29r5iTc0xfsjX11e+vr7/9rkDAICGwa13mpxOp5KSkrRx40bt3LlT4eHhLvN9+/ZV8+bNlZOTY4yVlJToxIkTio6OliRFR0frgw8+0JkzZ4ya7OxsBQQEKCIiwqi5/Bg1NTXH8PHxUd++fV1qqqurlZOTY9QAAICmza13mhITE7V27Vq99dZb8vf3N9YPWa1WtWjRQlarVQkJCUpOTlbbtm0VEBCgyZMnKzo6WgMHDpQkxcbGKiIiQg8//LAWLlwou92u2bNnKzEx0bgTNGnSJC1fvlwzZszQo48+qp07d2r9+vXavPn/vtmVnJyscePGqV+/fhowYICWLl2q8+fP65FHHnHT1QEAAJ7EraFp1apVkqTBgwe7jL/66qsaP368JGnJkiXy8vLSiBEjVFFRIZvNppUrVxq13t7eysrK0uOPP67o6Gi1atVK48aN07x584ya8PBwbd68WdOmTdOyZct0880366WXXjKe0SRJDz74oL788kvNmTNHdrtdvXr10tatW2stDgcAAE2TRz2nqSHjOU1XxnOaAACerME+pwkAAMBTEZoAAABMIDQBAACYQGgCAAAwgdAEAABgAqEJAADABEITAACACYQmAAAAEwhNAAAAJhCaAAAATCA0AQAAmEBoAgAAMIHQBAAAYAKhCQAAwARCEwAAgAmEJgAAABMITQAAACYQmgAAAEwgNAEAAJhAaAIAADCB0AQAAGACoQkAAMAEQhMAAIAJhCYAAAATCE0AAAAmEJoAAABMIDQBAACYQGgCAAAwgdAEAABgAqEJAADABEITAACACYQmAAAAEwhNAAAAJrg1NOXm5uq+++5TaGioLBaLNm3a5DJvsViuuC1atMio6dixY635BQsWuBznyJEjuvvuu+Xn56ewsDAtXLiwVi8bNmxQly5d5Ofnp8jISG3ZsqUezxwAADQ0bg1N58+fV8+ePbVixYorzp8+fdple+WVV2SxWDRixAiXunnz5rnUTZ482ZhzOByKjY1Vhw4dVFBQoEWLFik1NVWrV682avLy8jRq1CglJCTo0KFDio+PV3x8vIqKiurx7AEAQEPSzJ1vPnz4cA0fPvyq88HBwS77b731loYMGaJbb73VZdzf379WbY01a9bo4sWLeuWVV+Tj46Nu3bqpsLBQixcv1sSJEyVJy5Yt07BhwzR9+nRJ0vz585Wdna3ly5crIyOjDs4UAAA0dA1mTVNpaak2b96shISEWnMLFixQu3bt1Lt3by1atEiVlZXGXH5+vgYNGiQfHx9jzGazqaSkRF9//bVRExMT43JMm82m/Pz8q/ZTUVEhh8PhsgEAgMbLrXearsVrr70mf39/3X///S7jTzzxhPr06aO2bdsqLy9PKSkpOn36tBYvXixJstvtCg8Pd3lNUFCQMdemTRvZ7XZj7PIau91+1X7S0tI0d+7cOjxDAADgyRpMaHrllVc0ZswY+fn5uYwnJycbP/fo0UM+Pj763e9+p7S0NPn6+tZbPykpKS7v7XA4FBYWVm/vBwAA3KtBhKZ3331XJSUlWrdu3U/WRkVFqbKyUsePH1fnzp0VHBys0tJSl5qa/Zp1UFerudo6KUny9fWt11AGAAA8S4NY0/Tyyy+rb9++6tmz50/WFhYWysvLS4GBgZKk6Oho5ebm6tKlS0ZNdna2OnfurDZt2hg1OTk5LsfJzs5WdHR0nZ8LAABomNwams6dO6fCwkIVFhZKko4dO6bCwkKdOHHCqHE4HNqwYYMee+yxWq/Pz8/X0qVLdfjwYX322Wdas2aNpk2bpoceesgIRKNHj5aPj48SEhJ09OhRrVu3TsuWLXP5aG3KlCnaunWr0tPTVVxcrNTUVB04cEBJSUk35DoAAADP59aP5w4cOKAhQ4YY+zVBZty4ccrMzJQkvfHGG3I6nRo1alSt1/v6+uqNN95QamqqKioqFB4ermnTprkEIqvVqu3btysxMVF9+/ZV+/btNWfOHONxA5J05513au3atZo9e7aeeuop3X777dq0aZO6d+9ez1cAAAA0FBan0+l0dxONgcPhkNVqVXl5uQICAtzdjsfoOGuzu1uo5fiCOHe3AADwENfy+7tBrGkCAABwN0ITAACACYQmAAAAEwhNAAAAJhCaAAAATCA0AQAAmEBoAgAAMIHQBAAAYAKhCQAAwARCEwAAgAmEJgAAABMITQAAACYQmgAAAEwgNAEAAJhAaAIAADCB0AQAAGACoQkAAMAEQhMAAIAJhCYAAAATCE0AAAAmEJoAAABMIDQBAACYQGgCAAAwgdAEAABgAqEJAADABEITAACACYQmAAAAEwhNAAAAJhCaAAAATCA0AQAAmEBoAgAAMIHQBAAAYAKhCQAAwAS3hqbc3Fzdd999Cg0NlcVi0aZNm1zmx48fL4vF4rINGzbMpebs2bMaM2aMAgIC1Lp1ayUkJOjcuXMuNUeOHNHdd98tPz8/hYWFaeHChbV62bBhg7p06SI/Pz9FRkZqy5Yt9XTWAACgIXJraDp//rx69uypFStWXLVm2LBhOn36tLH9z//8j8v8mDFjdPToUWVnZysrK0u5ubmaOHGiMe9wOBQbG6sOHTqooKBAixYtUmpqqlavXm3U5OXladSoUUpISNChQ4cUHx+v+Ph4FRUV1dOZAwCAhsbidDqd7m5CkiwWizZu3Kj4+HhjbPz48SorK6t1B6rGRx99pIiICO3fv1/9+vWTJG3dulX33HOPPv/8c4WGhmrVqlX64x//KLvdLh8fH0nSrFmztGnTJhUXF0uSHnzwQZ0/f15ZWVnGsQcOHKhevXopIyPjiu9dUVGhiooKY9/hcCgsLEzl5eUKCAioo6vS8HWctdndLdRyfEGcu1sAAHgIh8Mhq9Vq6ve3x69p2rVrlwIDA9W5c2c9/vjj+uqrr4y5/Px8tW7d2ghMkhQTEyMvLy+9//77Rs2gQYOMwCRJNptNJSUl+vrrr42amJgYl/e12WzKz8+/al9paWmyWq3GFhYWVqfnDQAAPItHh6Zhw4bpL3/5i3JycvTCCy9o9+7dGj58uKqqqiRJdrtdgYGBLq9p1qyZ2rZtK7vdbtQEBQW51NTs/1RNzfyVpKSkqLy83NhOnjxZR2cNAAA8UTN3N/BjRo4cafwcGRmpHj16qFOnTtq1a5eGDh3q1t58fX3l6+vr1h4AAMCN49F3mn7o1ltvVfv27fXJJ59IkoKDg3XmzBmXmsrKSp09e1bBwcFGTWlpqUtNzf5P1dTMAwAANKjQ9Pnnn+urr75SSEiIJCk6OlplZWUqKCgwanbu3Knq6mpFRUUZNbm5ubp06ZJRk52drc6dO6tNmzZGTU5Ojst7ZWdnKzo6+gadGQAA8HRuDU3nzp1TYWGhCgsLJUnHjh1TYWGhTpw4oXPnzmn69Ol67733dPz4ceXk5Og//uM/dNttt8lms0mSunbtqmHDhmnChAnat2+f9u7dq6SkJI0cOVKhoaGSpNGjR8vHx0cJCQk6evSo1q1bp2XLlik5OdnoY8qUKdq6davS09NVXFys1NRUHThwQElJSW66MgAAwNO4NTQdOHBAvXv3Vu/evSVJycnJ6t27t+bMmSNvb28dOXJEv/71r3XHHXcoISFBffv21bvvvuuylmjNmjXq0qWLhg4dqnvuuUc///nPXZ7BZLVatX37dh07dkx9+/bVk08+qTlz5rg8y+nOO+/U2rVrtXr1avXs2VN//etftWnTJnXv3v0GXxEAAOCpPOY5TQ3dtTznoSnhOU0AAE/WqJ7TBAAA4AkITQAAACYQmgAAAEwgNAEAAJhAaAIAADCB0AQAAGACoQkAAMAEQhMAAIAJhCYAAAATCE0AAAAmEJoAAABMIDQBAACYQGgCAAAwgdAEAABgAqEJAADABEITAACACYQmAAAAEwhNAAAAJhCaAAAATCA0AQAAmEBoAgAAMIHQBAAAYAKhCQAAwARCEwAAgAmEJgAAABMITQAAACYQmgAAAEwgNAEAAJhAaAIAADCB0AQAAGACoQkAAMAEQhMAAIAJbg1Nubm5uu+++xQaGiqLxaJNmzYZc5cuXdLMmTMVGRmpVq1aKTQ0VGPHjtWpU6dcjtGxY0dZLBaXbcGCBS41R44c0d133y0/Pz+FhYVp4cKFtXrZsGGDunTpIj8/P0VGRmrLli31eOYAAKChcWtoOn/+vHr27KkVK1bUmrtw4YIOHjyop59+WgcPHtSbb76pkpIS/frXv65VO2/ePJ0+fdrYJk+ebMw5HA7FxsaqQ4cOKigo0KJFi5SamqrVq1cbNXl5eRo1apQSEhJ06NAhxcfHKz4+XkVFRfV49gAAoCFp5s43Hz58uIYPH37FOavVquzsbJex5cuXa8CAATpx4oRuueUWY9zf31/BwcFXPM6aNWt08eJFvfLKK/Lx8VG3bt1UWFioxYsXa+LEiZKkZcuWadiwYZo+fbokaf78+crOztby5cuVkZFRh2cMAAAaqga1pqm8vFwWi0WtW7d2GV+wYIHatWun3r17a9GiRaqsrDTm8vPzNWjQIPn4+BhjNptNJSUl+vrrr42amJgYl2PabDbl5+dftZeKigo5HA6XDQAANF5uvdN0Lb777jvNnDlTo0aNUkBAgDH+xBNPqE+fPmrbtq3y8vKUkpKi06dPa/HixZIku92u8PBwl2MFBQUZc23atJHdbjfGLq+x2+1X7SctLU1z586t47MEAACeqkGEpkuXLum3v/2tnE6nVq1a5TKXnJxs/NyjRw/5+Pjod7/7ndLS0uTr61tvPaWkpLi8t8PhUFhYWL29HwAAcC+PD001gemf//yndu7c6XKX6UqioqJUWVmp48ePq3PnzgoODlZpaalLTc1+zTqoq9VcbZ2UJPn6+tZrKAMAAJ7Fo9c01QSmjz/+WDt27FC7du1+8jWFhYXy8vJSYGCgJCk6Olq5ubm6dOmSUZOdna3OnTurTZs2Rk1OTo7LcbKzsxUdHV3n5wQAABomt95pOnfunD755BNj/9ixYyosLFTbtm0VEhKiBx54QAcPHlRWVpaqqqqMNUZt27aVj4+P8vPz9f7772vIkCHy9/dXfn6+pk2bpoceesgIRKNHj9bcuXOVkJCgmTNnqqioSMuWLdOSJUuM950yZYp+8YtfKD09XXFxcXrjjTd04MABl8cSAACAps3idDqd7nrzXbt2aciQIbXGx40bp9TU1FoLuGu88847Gjx4sA4ePKjf//73Ki4uVkVFhcLDw/Xwww8rOTnZ5aOzI0eOKDExUfv371f79u01efJkzZw50+WYGzZs0OzZs3X8+HHdfvvtWrhwoe655x7T5+JwOGS1WlVeXv6THyE2JR1nbXZ3C7UcXxDn7hYAAB7iWn5/uzU0NSaEpisjNAEAPNm1/P726DVNAAAAnoLQBAAAYAKhCQAAwARCEwAAgAmEJgAAABMITQAAACYQmgAAAEwgNAEAAJhwXaHp1ltv1VdffVVrvKysTLfeemtd9AUAAOBRris0HT9+XFVVVbXGKyoq9MUXX9RFXwAAAB7lmv5g79tvv238vG3bNlmtVmO/qqpKOTk56tixY912CAAA4AGuKTTFx8dLkiwWi8aNG+cy17x5c3Xs2FHp6el12yEAAIAHuKbQVF1dLUkKDw/X/v371b59+/rqCwAAwKNcU2iqcezYsbrvBAAAwINdV2iSpJycHOXk5OjMmTPGHagar7zySl30BgAA4DGuKzTNnTtX8+bNU79+/RQSEiKLxVL3nQEAAHiQ6wpNGRkZyszM1MMPP1z3HQEAAHig63pO08WLF3XnnXfWfTcAAAAe6rpC02OPPaa1a9fWfTcAAAAe6ro+nvvuu++0evVq7dixQz169FDz5s1d5hcvXlxX/QEAAHiE6wpNR44cUa9evSRJRUVFLnMsCgcAAI3RdYWmd955p+47AQAA8GDXtaYJAACgqbmuO01Dhgz50Y/hdu7c+e/0BAAA4HGuKzTVrGeqcenSJRUWFqqoqKjWH/IFAABoDK4rNC1ZsuSK46mpqTp37ty/2xMAAIDHqdM1TQ899BB/dw4AADRKdRqa8vPz5efnV5eHBAAA8AjX9fHc/fff77LvdDp1+vRpHThwQE8//XRd9QYAAOAxris0Wa1Wl30vLy917txZ8+bNU2xsbF31BgAA4DGuKzS9+uqrdd8JAACAB7uu0FSjoKBAH330kSSpW7du6t27d131BQAA4FGuKzSdOXNGI0eO1K5du9S6dWtJUllZmYYMGaI33nhDN910U133CQAA4FbX9e25yZMn65tvvtHRo0d19uxZnT17VkVFRXI4HHriiSdMHyc3N1f33XefQkNDZbFYtGnTJpd5p9OpOXPmKCQkRC1atFBMTIw+/vhjl5qzZ89qzJgxCggIUOvWrZWQkFDrWVFHjhzR3XffLT8/P4WFhWnhwoW1etmwYYO6dOkiPz8/RUZGasuWLdd8XQAAQON1XaFp69atWrlypbp27WqMRUREaMWKFfr73/9u+jjnz59Xz549tWLFiivOL1y4UC+++KIyMjL0/vvvq1WrVrLZbPruu++MmjFjxujo0aPKzs5WVlaWcnNzNXHiRGPe4XAoNjZWHTp0UEFBgRYtWqTU1FStXr3aqMnLy9OoUaOUkJCgQ4cOKT4+XvHx8SoqKrqOqwMAABoji9PpdF7ri/z9/fXuu+/W+nMqhw4d0i9+8Qs5HI5rb8Ri0caNGxUfHy99f5cpNDRUTz75pP7whz9IksrLyxUUFKTMzEyNHDlSH330kSIiIrR//37169dP+j7Q3XPPPfr8888VGhqqVatW6Y9//KPsdrt8fHwkSbNmzdKmTZtUXFwsSXrwwQd1/vx5ZWVlGf0MHDhQvXr1UkZGxhX7raioUEVFhbHvcDgUFham8vJyBQQEXPP5N1YdZ212dwu1HF8Q5+4WAAAewuFwyGq1mvr9fV13mn75y19qypQpOnXqlDH2xRdfaNq0aRo6dOj1HLKWY8eOyW63KyYmxhizWq2KiopSfn6+9P3DNFu3bm0EJkmKiYmRl5eX3n//faNm0KBBRmCSJJvNppKSEn399ddGzeXvU1NT8z5XkpaWJqvVamxhYWF1ct4AAMAzXVdoWr58uRwOhzp27KhOnTqpU6dOCg8Pl8Ph0J/+9Kc6acxut0uSgoKCXMaDgoKMObvdrsDAQJf5Zs2aqW3bti41VzrG5e9xtZqa+StJSUlReXm5sZ08efLfOFsAAODpruvbc2FhYTp48KB27NhhfMTVtWvXWndrGjNfX1/5+vq6uw0AAHCDXNOdpp07dyoiIkIOh0MWi0W/+tWvNHnyZE2ePFn9+/dXt27d9O6779ZJY8HBwZKk0tJSl/HS0lJjLjg4WGfOnHGZr6ys1NmzZ11qrnSMy9/jajU18wAAANcUmpYuXaoJEyZccaGU1WrV7373Oy1evLhOGgsPD1dwcLBycnKMMYfDoffff1/R0dGSpOjoaJWVlamgoMCo2blzp6qrqxUVFWXU5Obm6tKlS0ZNdna2OnfurDZt2hg1l79PTU3N+wAAAFxTaDp8+LCGDRt21fnY2FiXAPNTzp07p8LCQhUWFkrfL/4uLCzUiRMnZLFYNHXqVD377LN6++239cEHH2js2LEKDQ01vmHXtWtXDRs2TBMmTNC+ffu0d+9eJSUlaeTIkQoNDZUkjR49Wj4+PkpISNDRo0e1bt06LVu2TMnJyUYfU6ZM0datW5Wenq7i4mKlpqbqwIEDSkpKupbLAwAAGrFrWtNUWlqq5s2bX/1gzZrpyy+/NH28AwcOaMiQIcZ+TZAZN26cMjMzNWPGDJ0/f14TJ05UWVmZfv7zn2vr1q3y8/MzXrNmzRolJSVp6NCh8vLy0ogRI/Tiiy8a81arVdu3b1diYqL69u2r9u3ba86cOS7Pcrrzzju1du1azZ49W0899ZRuv/12bdq0Sd27d7+WywMAABqxa3pOU6dOnZSenm7c6fmhN998U3/4wx/02Wef1WWPDcK1POehKeE5TQAAT1Zvz2m655579PTTT7s8kbvGt99+q2eeeUb33nvvtXcMAADg4a7p47nZs2frzTff1B133KGkpCR17txZklRcXKwVK1aoqqpKf/zjH+urVwAAALe5ptAUFBSkvLw8Pf7440pJSVHNJ3sWi0U2m00rVqyo9ZBIAACAxuCaH27ZoUMHbdmyRV9//bU++eQTOZ1O3X777cbX9wEAABqj63oiuCS1adNG/fv3r9tuAAAAPNR1/e05AACApobQBAAAYAKhCQAAwARCEwAAgAmEJgAAABMITQAAACYQmgAAAEwgNAEAAJhAaAIAADCB0AQAAGACoQkAAMAEQhMAAIAJhCYAAAATCE0AAAAmEJoAAABMIDQBAACYQGgCAAAwgdAEAABgAqEJAADABEITAACACYQmAAAAEwhNAAAAJhCaAAAATGjm7gYAT9Rx1mZ3t1DL8QVx7m4BAJo07jQBAACYQGgCAAAwgdAEAABggseHpo4dO8pisdTaEhMTJUmDBw+uNTdp0iSXY5w4cUJxcXFq2bKlAgMDNX36dFVWVrrU7Nq1S3369JGvr69uu+02ZWZm3tDzBAAAns3jF4Lv379fVVVVxn5RUZF+9atf6Te/+Y0xNmHCBM2bN8/Yb9mypfFzVVWV4uLiFBwcrLy8PJ0+fVpjx45V8+bN9fzzz0uSjh07pri4OE2aNElr1qxRTk6OHnvsMYWEhMhms92wcwUAAJ7L40PTTTfd5LK/YMECderUSb/4xS+MsZYtWyo4OPiKr9++fbs+/PBD7dixQ0FBQerVq5fmz5+vmTNnKjU1VT4+PsrIyFB4eLjS09MlSV27dtWePXu0ZMkSQhMAAJAawsdzl7t48aJef/11Pfroo7JYLMb4mjVr1L59e3Xv3l0pKSm6cOGCMZefn6/IyEgFBQUZYzabTQ6HQ0ePHjVqYmJiXN7LZrMpPz//qr1UVFTI4XC4bAAAoPHy+DtNl9u0aZPKyso0fvx4Y2z06NHq0KGDQkNDdeTIEc2cOVMlJSV68803JUl2u90lMEky9u12+4/WOBwOffvtt2rRokWtXtLS0jR37tx6OU8AAOB5GlRoevnllzV8+HCFhoYaYxMnTjR+joyMVEhIiIYOHapPP/1UnTp1qrdeUlJSlJycbOw7HA6FhYXV2/sBAAD3ajCh6Z///Kd27Nhh3EG6mqioKEnSJ598ok6dOik4OFj79u1zqSktLZUkYx1UcHCwMXZ5TUBAwBXvMkmSr6+vfH19/61zAgAADUeDWdP06quvKjAwUHFxP/6nJAoLCyVJISEhkqTo6Gh98MEHOnPmjFGTnZ2tgIAARUREGDU5OTkux8nOzlZ0dHQ9nAkAAGiIGkRoqq6u1quvvqpx48apWbP/uzn26aefav78+SooKNDx48f19ttva+zYsRo0aJB69OghSYqNjVVERIQefvhhHT58WNu2bdPs2bOVmJho3CmaNGmSPvvsM82YMUPFxcVauXKl1q9fr2nTprntnAEAgGdpEKFpx44dOnHihB599FGXcR8fH+3YsUOxsbHq0qWLnnzySY0YMUJ/+9vfjBpvb29lZWXJ29tb0dHReuihhzR27FiX5zqFh4dr8+bNys7OVs+ePZWenq6XXnqJxw0AAABDg1jTFBsbK6fTWWs8LCxMu3fv/snXd+jQQVu2bPnRmsGDB+vQoUP/Vp8AAKDxahB3mgAAANyN0AQAAGACoQkAAMAEQhMAAIAJhCYAAAATCE0AAAAmEJoAAABMIDQBAACYQGgCAAAwgdAEAABgAqEJAADABEITAACACYQmAAAAEwhNAAAAJhCaAAAATCA0AQAAmEBoAgAAMIHQBAAAYAKhCQAAwARCEwAAgAmEJgAAABMITQAAACYQmgAAAEwgNAEAAJhAaAIAADCB0AQAAGACoQkAAMAEQhMAAIAJhCYAAAATCE0AAAAmEJoAAABMIDQBAACYQGgCAAAwwaNDU2pqqiwWi8vWpUsXY/67775TYmKi2rVrp5/97GcaMWKESktLXY5x4sQJxcXFqWXLlgoMDNT06dNVWVnpUrNr1y716dNHvr6+uu2225SZmXnDzhEAADQMHh2aJKlbt246ffq0se3Zs8eYmzZtmv72t79pw4YN2r17t06dOqX777/fmK+qqlJcXJwuXryovLw8vfbaa8rMzNScOXOMmmPHjikuLk5DhgxRYWGhpk6dqscee0zbtm274ecKAAA8VzN3N/BTmjVrpuDg4Frj5eXlevnll7V27Vr98pe/lCS9+uqr6tq1q9577z0NHDhQ27dv14cffqgdO3YoKChIvXr10vz58zVz5kylpqbKx8dHGRkZCg8PV3p6uiSpa9eu2rNnj5YsWSKbzXbDzxcAAHgmj7/T9PHHHys0NFS33nqrxowZoxMnTkiSCgoKdOnSJcXExBi1Xbp00S233KL8/HxJUn5+viIjIxUUFGTU2Gw2ORwOHT161Ki5/Bg1NTXHuJqKigo5HA6XDQAANF4eHZqioqKUmZmprVu3atWqVTp27JjuvvtuffPNN7Lb7fLx8VHr1q1dXhMUFCS73S5JstvtLoGpZr5m7sdqHA6Hvv3226v2lpaWJqvVamxhYWF1dt4AAMDzePTHc8OHDzd+7tGjh6KiotShQwetX79eLVq0cGtvKSkpSk5ONvYdDgfBCQCARsyj7zT9UOvWrXXHHXfok08+UXBwsC5evKiysjKXmtLSUmMNVHBwcK1v09Xs/1RNQEDAjwYzX19fBQQEuGwAAKDxalCh6dy5c/r0008VEhKivn37qnnz5srJyTHmS0pKdOLECUVHR0uSoqOj9cEHH+jMmTNGTXZ2tgICAhQREWHUXH6MmpqaYwAAAMjTQ9Mf/vAH7d69W8ePH1deXp7+8z//U97e3ho1apSsVqsSEhKUnJysd955RwUFBXrkkUcUHR2tgQMHSpJiY2MVERGhhx9+WIcPH9a2bds0e/ZsJSYmytfXV5I0adIkffbZZ5oxY4aKi4u1cuVKrV+/XtOmTXPz2QMAAE/i0WuaPv/8c40aNUpfffWVbrrpJv385z/Xe++9p5tuukmStGTJEnl5eWnEiBGqqKiQzWbTypUrjdd7e3srKytLjz/+uKKjo9WqVSuNGzdO8+bNM2rCw8O1efNmTZs2TcuWLdPNN9+sl156iccNAAAAFxan0+l0dxONgcPhkNVqVXl5OeubLtNx1mZ3t1DL8QVxP1nTUPsGAFyba/n97dEfzwEAAHgKQhMAAIAJhCYAAAATCE0AAAAmEJoAAABMIDQBAACYQGgCAAAwgdAEAABgAqEJAADABEITAACACYQmAAAAEwhNAAAAJhCaAAAATCA0AQAAmEBoAgAAMIHQBAAAYAKhCQAAwARCEwAAgAmEJgAAABMITQAAACYQmgAAAEwgNAEAAJhAaAIAADCB0AQAAGACoQkAAMCEZu5uAEDd6Thrs7tbqOX4gjh3twAAdYI7TQAAACYQmgAAAEwgNAEAAJhAaAIAADCB0AQAAGACoQkAAMAEQhMAAIAJHh2a0tLS1L9/f/n7+yswMFDx8fEqKSlxqRk8eLAsFovLNmnSJJeaEydOKC4uTi1btlRgYKCmT5+uyspKl5pdu3apT58+8vX11W233abMzMwbco4AAKBh8OjQtHv3biUmJuq9995Tdna2Ll26pNjYWJ0/f96lbsKECTp9+rSxLVy40JirqqpSXFycLl68qLy8PL322mvKzMzUnDlzjJpjx44pLi5OQ4YMUWFhoaZOnarHHntM27Ztu6HnCwAAPJdHPxF869atLvuZmZkKDAxUQUGBBg0aZIy3bNlSwcHBVzzG9u3b9eGHH2rHjh0KCgpSr169NH/+fM2cOVOpqany8fFRRkaGwsPDlZ6eLknq2rWr9uzZoyVLlshms13xuBUVFaqoqDD2HQ5HHZ01AADwRB59p+mHysvLJUlt27Z1GV+zZo3at2+v7t27KyUlRRcuXDDm8vPzFRkZqaCgIGPMZrPJ4XDo6NGjRk1MTIzLMW02m/Lz86/aS1pamqxWq7GFhYXV2XkCAADP49F3mi5XXV2tqVOn6q677lL37t2N8dGjR6tDhw4KDQ3VkSNHNHPmTJWUlOjNN9+UJNntdpfAJMnYt9vtP1rjcDj07bffqkWLFrX6SUlJUXJysrHvcDgITgAANGINJjQlJiaqqKhIe/bscRmfOHGi8XNkZKRCQkI0dOhQffrpp+rUqVO99ePr6ytfX996Oz4AAPAsDeLjuaSkJGVlZemdd97RzTff/KO1UVFRkqRPPvlEkhQcHKzS0lKXmpr9mnVQV6sJCAi44l0mAADQ9Hh0aHI6nUpKStLGjRu1c+dOhYeH/+RrCgsLJUkhISGSpOjoaH3wwQc6c+aMUZOdna2AgABFREQYNTk5OS7Hyc7OVnR0dB2fEQAAaKg8OjQlJibq9ddf19q1a+Xv7y+73S673a5vv/1WkvTpp59q/vz5Kigo0PHjx/X2229r7NixGjRokHr06CFJio2NVUREhB5++GEdPnxY27Zt0+zZs5WYmGh8vDZp0iR99tlnmjFjhoqLi7Vy5UqtX79e06ZNc+v5AwAAz+HRoWnVqlUqLy/X4MGDFRISYmzr1q2TJPn4+GjHjh2KjY1Vly5d9OSTT2rEiBH629/+ZhzD29tbWVlZ8vb2VnR0tB566CGNHTtW8+bNM2rCw8O1efNmZWdnq2fPnkpPT9dLL7101ccNAACApsejF4I7nc4fnQ8LC9Pu3bt/8jgdOnTQli1bfrRm8ODBOnTo0DX3CAAAmgaPvtMEAADgKQhNAAAAJhCaAAAATCA0AQAAmEBoAgAAMIHQBAAAYAKhCQAAwARCEwAAgAmEJgAAABMITQAAACYQmgAAAEwgNAEAAJhAaAIAADCB0AQAAGACoQkAAMAEQhMAAIAJhCYAAAATCE0AAAAmNHN3AwDQcdZmd7dQy/EFce5uAYCH4U4TAACACYQmAAAAEwhNAAAAJhCaAAAATCA0AQAAmEBoAgAAMIHQBAAAYAKhCQAAwARCEwAAgAmEJgAAABP4MyoAcJ348y9A08KdJgAAABMITQAAACYQmn5gxYoV6tixo/z8/BQVFaV9+/a5uyUAAOABCE2XWbdunZKTk/XMM8/o4MGD6tmzp2w2m86cOePu1gAAgJuxEPwyixcv1oQJE/TII49IkjIyMrR582a98sormjVrlrvbA4A6wQJ24PoQmr538eJFFRQUKCUlxRjz8vJSTEyM8vPza9VXVFSooqLC2C8vL5ckORyOG9Rxw1BdccHdLdRi5r8Rfdcd+r6xGnPf3Z/ZdkN6uRZFc23ubgH/ppp/e06n86eLnXA6nU7nF1984ZTkzMvLcxmfPn26c8CAAbXqn3nmGackNjY2NjY2tkawnTx58iezAnearlNKSoqSk5ON/erqap09e1bt2rWTxWJxa29X43A4FBYWppMnTyogIMDd7TR6XO8bi+t9Y3G9byyud/1xOp365ptvFBoa+pO1hKbvtW/fXt7e3iotLXUZLy0tVXBwcK16X19f+fr6uoy1bt263vusCwEBAfxPdwNxvW8srveNxfW+sbje9cNqtZqq49tz3/Px8VHfvn2Vk5NjjFVXVysnJ0fR0dFu7Q0AALgfd5ouk5ycrHHjxqlfv34aMGCAli5dqvPnzxvfpgMAAE0XoekyDz74oL788kvNmTNHdrtdvXr10tatWxUUFOTu1uqEr6+vnnnmmVofK6J+cL1vLK73jcX1vrG43p7B4jT1HTsAAICmjTVNAAAAJhCaAAAATCA0AQAAmEBoAgAAMIHQ1ISsWLFCHTt2lJ+fn6KiorRv3z53t9QopaWlqX///vL391dgYKDi4+NVUlLi7raahAULFshisWjq1KnubqVR++KLL/TQQw+pXbt2atGihSIjI3XgwAF3t9UoVVVV6emnn1Z4eLhatGihTp06af78+eb+ThrqHKGpiVi3bp2Sk5P1zDPP6ODBg+rZs6dsNpvOnDnj7tYand27dysxMVHvvfeesrOzdenSJcXGxur8+fPubq1R279/v/785z+rR48e7m6lUfv666911113qXnz5vr73/+uDz/8UOnp6WrTpo27W2uUXnjhBa1atUrLly/XRx99pBdeeEELFy7Un/70J3e31iTxyIEmIioqSv3799fy5cul7592HhYWpsmTJ2vWrFnubq9R+/LLLxUYGKjdu3dr0KBB7m6nUTp37pz69OmjlStX6tlnn1WvXr20dOlSd7fVKM2aNUt79+7Vu+++6+5WmoR7771XQUFBevnll42xESNGqEWLFnr99dfd2ltTxJ2mJuDixYsqKChQTEyMMebl5aWYmBjl5+e7tbemoLy8XJLUtm1bd7fSaCUmJiouLs7l3zjqx9tvv61+/frpN7/5jQIDA9W7d2/913/9l7vbarTuvPNO5eTk6B//+Ick6fDhw9qzZ4+GDx/u7taaJJ4I3gT861//UlVVVa0nmwcFBam4uNhtfTUF1dXVmjp1qu666y51797d3e00Sm+88YYOHjyo/fv3u7uVJuGzzz7TqlWrlJycrKeeekr79+/XE088IR8fH40bN87d7TU6s2bNksPhUJcuXeTt7a2qqio999xzGjNmjLtba5IITUA9SkxMVFFRkfbs2ePuVhqlkydPasqUKcrOzpafn5+722kSqqur1a9fPz3//POSpN69e6uoqEgZGRmEpnqwfv16rVmzRmvXrlW3bt1UWFioqVOnKjQ0lOvtBoSmJqB9+/by9vZWaWmpy3hpaamCg4Pd1ldjl5SUpKysLOXm5urmm292dzuNUkFBgc6cOaM+ffoYY1VVVcrNzdXy5ctVUVEhb29vt/bY2ISEhCgiIsJlrGvXrvrf//1ft/XUmE2fPl2zZs3SyJEjJUmRkZH65z//qbS0NEKTG7CmqQnw8fFR3759lZOTY4xVV1crJydH0dHRbu2tMXI6nUpKStLGjRu1c+dOhYeHu7ulRmvo0KH64IMPVFhYaGz9+vXTmDFjVFhYSGCqB3fddVetR2j84x//UIcOHdzWU2N24cIFeXm5/qr29vZWdXW123pqyrjT1EQkJydr3Lhx6tevnwYMGKClS5fq/PnzeuSRR9zdWqOTmJiotWvX6q233pK/v7/sdrskyWq1qkWLFu5ur1Hx9/evtVasVatWateuHWvI6sm0adN055136vnnn9dvf/tb7du3T6tXr9bq1avd3VqjdN999+m5557TLbfcom7duunQoUNavHixHn30UXe31iTxyIEmZPny5Vq0aJHsdrt69eqlF198UVFRUe5uq9GxWCxXHH/11Vc1fvz4G95PUzN48GAeOVDPsrKylJKSoo8//ljh4eFKTk7WhAkT3N1Wo/TNN9/o6aef1saNG3XmzBmFhoZq1KhRmjNnjnx8fNzdXpNDaAIAADCBNU0AAAAmEJoAAABMIDQBAACYQGgCAAAwgdAEAABgAqEJAADABEITAACACYQmAAAAEwhNAPAjMjMz1bp163/7OBaLRZs2baqTngC4B6EJQKM3fvx4xcfHu7sNAA0coQkAAMAEQhOAJm3x4sWKjIxUq1atFBYWpt///vc6d+5crbpNmzbp9ttvl5+fn2w2m06ePOky/9Zbb6lPnz7y8/PTrbfeqrlz56qysvIGngmA+kZoAtCkeXl56cUXX9TRo0f12muvaefOnZoxY4ZLzYULF/Tcc8/pL3/5i/bu3auysjKNHDnSmH/33Xc1duxYTZkyRR9++KH+/Oc/KzMzU88995wbzghAfbE4nU6nu5sAgPo0fvx4lZWVmVqI/de//lWTJk3Sv/71L+n7heCPPPKI3nvvPUVFRUmSiouL1bVrV73//vsaMGCAYmJiNHToUKWkpBjHef311zVjxgydOnVK+n4h+MaNG1lbBTRgzdzdAAC4044dO5SWlqbi4mI5HA5VVlbqu+++04ULF9SyZUtJUrNmzdS/f3/jNV26dFHr1q310UcfacCAATp8+LD27t3rcmepqqqq1nEANGyEJgBN1vHjx3Xvvffq8ccf13PPPae2bdtqz549SkhI0MWLF02HnXPnzmnu3Lm6//77a835+fnVQ+cA3IHQBKDJKigoUHV1tdLT0+Xl9f+XeK5fv75WXWVlpQ4cOKABAwZIkkpKSlRWVqauXbtKkvr06aOSkhLddtttN/gMANxIhCYATUJ5ebkKCwtdxtq3b69Lly7pT3/6k+677z7t3btXGRkZtV7bvHlzTZ48WS+++KKaNWumpKQkDRw40AhRc+bM0b333qtbbrlFDzzwgLy8vHT48GEVFRXp2WefvWHnCKB+8e05AE3Crl271Lt3b5ftv//7v7V48WK98MIL6t69u9asWaO0tLRar23ZsqVmzpyp0aNH66677tLPfvYzrVu3zpi32WzKysrS9u3b1b9/fw0cOFBLlixRhw4dbvBZAqhPfHsOAADABO40AQAAmEBoAgAAMIHQBAAAYAKhCQAAwARCEwAAgAmEJgAAABMITQAAACYQmgAAAEwgNAEAAJhAaAIAADCB0AQAAGDC/wOGFHlNUKJeowAAAABJRU5ErkJggg==", "text/plain": [ "<Figure size 640x480 with 1 Axes>" ] @@ -324,7 +325,10 @@ } ], "source": [ - "plt.hist(mkbatch(2**15, large=True, target=\"onpath\", largetarget=False)[1].cpu(), bins=8)" + "bins = torch.bincount(mkbatch(2**15, large=True, target=\"onpath\", largetarget=False)[1].to(torch.uint8).cpu())\n", + "plt.bar(range(len(bins)), bins)\n", + "plt.xlabel(\"Label\")\n", + "plt.ylabel(\"Count\")" ] }, { @@ -338,7 +342,7 @@ }, { "cell_type": "code", - "execution_count": 14, + "execution_count": 13, "execution_state": "idle", "metadata": { "id": "tLOWhg_CeWzH" @@ -375,7 +379,7 @@ }, { "cell_type": "code", - "execution_count": 16, + "execution_count": 14, "execution_state": "idle", "metadata": { "colab": { @@ -389,7 +393,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "Training data: 104857K\n", + "Training data: 209715K\n", "Trainable parameters in the model: 550433\n" ] } @@ -398,7 +402,7 @@ "# PARAMS\n", "VOCAB_SIZE = 1 + MAX_VTXS + 1 # pad plus max number of vertices plus target token\n", "MODEL_DIM = 64 # Dimension of model (embedding and transformer)\n", - "NEPOCHS = 100\n", + "NEPOCHS = 200\n", "BSZ = 2**15 # Batch size\n", "NHEADS = 2\n", "NLAYERS = 11\n", @@ -426,7 +430,7 @@ }, { "cell_type": "code", - "execution_count": 20, + "execution_count": 15, "execution_state": "idle", "metadata": {}, "outputs": [], @@ -470,17 +474,334 @@ "name": "stderr", "output_type": "stream", "text": [ + "/home/sipb/.venv/lib64/python3.12/site-packages/torch/nn/functional.py:6278: UserWarning: Memory Efficient attention on Navi31 GPU is still experimental. Enable it with TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1. (Triggered internally at ../aten/src/ATen/native/transformers/hip/sdp_utils.cpp:269.)\n", + " attn_output = scaled_dot_product_attention(\n", "/tmp/torchinductor_sipb/lc/clcqc3ufbzrethiy77dmsu54kurxdmh4eji2f3msm347rhmfpf4j.py:1078: UserWarning: Attempting to use hipBLASLt on an unsupported architecture! Overriding blas backend to hipblas (Triggered internally at ../aten/src/ATen/Context.cpp:296.)\n", " extern_kernels.mm(reinterpret_tensor(buf1, (524288, 64), (64, 1), 0), reinterpret_tensor(primals_5, (64, 192), (1, 64), 0), out=buf2)\n", "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:42<00:00, 1.33s/it]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:41<00:00, 1.31s/it]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 0/200 \t Train Err: 29.90892779827118 11.204990872880444 6.002899471088313 2.7772275000461377 119.98257398605347\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 1/200 \t Train Err: 24.833097636699677 7.996231320299557 10.17519548535347 4.934848785400391 95.85787630081177\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 2/200 \t Train Err: 20.868117928504944 0.06162079039495438 9.863971814513206 5.750443048775196 84.68168807029724\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 3/200 \t Train Err: 16.48520603775978 0.03265308297704905 2.906232228502631 4.825529254972935 70.63315153121948\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 4/200 \t Train Err: 12.373839050531387 0.012339260501903482 1.0458019012585282 5.470552921295166 52.21906542778015\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 5/200 \t Train Err: 9.265015065670013 0.020387541531817988 0.6235579419881105 4.304356303066015 38.14229780435562\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 6/200 \t Train Err: 6.847102478146553 0.00765054406656418 0.34211935847997665 3.4029701724648476 27.37058013677597\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 7/200 \t Train Err: 5.120877355337143 0.006255884032725589 0.24545069271698594 2.8054168857634068 19.56860715150833\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 8/200 \t Train Err: 3.8786706030368805 0.01005001129124139 0.19835278647951782 2.2635713517665863 14.35572400689125\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 9/200 \t Train Err: 3.386067658662796 0.06476508973355521 0.3309611896984279 2.269792005419731 11.835063099861145\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.07it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 10/200 \t Train Err: 2.6640610471367836 0.019311404510517605 0.15212045633234084 1.7280069142580032 9.439383015036583\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 11/200 \t Train Err: 2.271987594664097 0.004357099162007216 0.09561244037467986 1.4371714778244495 8.072030693292618\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 12/200 \t Train Err: 2.110176682472229 0.0037538947944995016 0.0762243423378095 1.3135331608355045 7.434565886855125\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 13/200 \t Train Err: 1.935215126723051 0.0025575611580279656 0.06788019218947738 1.1661983076483011 6.806286051869392\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 14/200 \t Train Err: 1.8060001619160175 0.0021859380358364433 0.05315158853773028 1.051108269020915 6.399133637547493\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.07it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 15/200 \t Train Err: 1.7425284087657928 0.003451302447501803 0.042916826030705124 0.9607762675732374 6.2206505835056305\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 16/200 \t Train Err: 1.649382572621107 0.0032472783022967633 0.039788190391846 0.9184560999274254 5.846587151288986\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 17/200 \t Train Err: 1.6013295575976372 0.0026137664608540945 0.03258192975772545 0.8397737592458725 5.709988132119179\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 18/200 \t Train Err: 1.5462302453815937 0.002207836727393442 0.028909130429383367 0.7771711964160204 5.481079399585724\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 19/200 \t Train Err: 1.4744272604584694 0.002453724335282459 0.027229167113546282 0.7396254185587168 5.192027851939201\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 20/200 \t Train Err: 1.405750896781683 0.0035180113754904596 0.025897870946209878 0.7007174119353294 4.946545287966728\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 0/100 \t Train Err: 48.97900426387787 12.91722442075843 7.231296321027912 3.5385852727340534 115.3402452468872\n" + "Epoch 21/200 \t Train Err: 1.37083425745368 0.005473145276482683 0.026675976172555238 0.6762280324473977 4.781045973300934\n" ] }, { @@ -488,14 +809,14 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:31<00:00, 1.03it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 1/100 \t Train Err: 41.59842586517334 23.066684544086456 14.473573058843613 7.877466633915901 84.72042870521545\n" + "Epoch 22/200 \t Train Err: 1.3181258291006088 0.003216923019863316 0.02211966845788993 0.6302088163793087 4.611788898706436\n" ] }, { @@ -503,14 +824,14 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:31<00:00, 1.03it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 2/100 \t Train Err: 40.415191769599915 29.482473254203796 19.633903205394745 11.77738669514656 73.48623991012573\n" + "Epoch 23/200 \t Train Err: 1.270566176623106 0.0022422261081374018 0.019996988266939297 0.5934071252122521 4.4385934472084045\n" ] }, { @@ -518,14 +839,44 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:31<00:00, 1.03it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 3/100 \t Train Err: 40.01692616939545 32.29490512609482 22.276952624320984 13.907412678003311 68.378258228302\n" + "Epoch 24/200 \t Train Err: 1.1932538747787476 0.004239499055984197 0.020018045412143692 0.552213310264051 4.133698016405106\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 25/200 \t Train Err: 1.173534195870161 0.00575368078352767 0.02177350874990225 0.5586085859686136 4.002377942204475\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 26/200 \t Train Err: 1.0854019112884998 0.003320175252156332 0.01748313583084382 0.5035764751955867 3.7186833322048187\n" ] }, { @@ -540,7 +891,97 @@ "name": "stdout", "output_type": "stream", "text": [ - "Epoch 4/100 \t Train Err: 31.416786193847656 2.3271487059355422 26.761382937431335 18.088589638471603 60.98844397068024\n" + "Epoch 27/200 \t Train Err: 1.0489223580807447 0.002155393432985875 0.014516596915200353 0.47800380270928144 3.589349642395973\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 28/200 \t Train Err: 1.0285752080380917 0.003717334211614798 0.01597064675297588 0.4826665371656418 3.4730969667434692\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 29/200 \t Train Err: 0.9720142241567373 0.004084829584826366 0.014119503815891221 0.43038015346974134 3.3248252645134926\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 30/200 \t Train Err: 0.9608706310391426 0.002451939526508795 0.011319156386889517 0.4139528265222907 3.2747417390346527\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 31/200 \t Train Err: 0.9179007802158594 0.002311610933247721 0.01079355452384334 0.3930521896108985 3.1119269728660583\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 32/200 \t Train Err: 0.8977434728294611 0.0021589329044218175 0.00950733212812338 0.3691776511259377 3.040615402162075\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 33/200 \t Train Err: 0.881706677377224 0.0020159517880529165 0.009437998596695252 0.35962088825181127 2.982758790254593\n" ] }, { @@ -555,7 +996,37 @@ "name": "stdout", "output_type": "stream", "text": [ - "Epoch 5/100 \t Train Err: 25.921700596809387 0.22728093068872113 19.57061032950878 15.32799781858921 52.15890157222748\n" + "Epoch 34/200 \t Train Err: 0.8403967656195164 0.001898838694614824 0.00869621682795696 0.3381345858797431 2.8688452169299126\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 35/200 \t Train Err: 0.8310248833149672 0.0013434199390758295 0.007526626766775735 0.32297050580382347 2.818771593272686\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 36/200 \t Train Err: 0.8045722767710686 0.0018811499339790316 0.0069304521166486666 0.30397533625364304 2.722321219742298\n" ] }, { @@ -570,7 +1041,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "Epoch 6/100 \t Train Err: 17.580547362565994 0.021982330930768512 4.470714939758182 19.619352281093597 36.34695905447006\n" + "Epoch 37/200 \t Train Err: 0.807910643517971 0.0022051451851439197 0.007862750302592758 0.30017248587682843 2.739243097603321\n" ] }, { @@ -578,14 +1049,14 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:31<00:00, 1.03it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:29<00:00, 1.07it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 7/100 \t Train Err: 12.400098770856857 0.013556713653088082 0.5100052966736257 23.03424423933029 23.36328774690628\n" + "Epoch 38/200 \t Train Err: 0.7853260803967714 0.005179399433473009 0.008613282887381501 0.2864332143217325 2.6513936147093773\n" ] }, { @@ -600,7 +1071,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "Epoch 8/100 \t Train Err: 9.38240310549736 0.007538945435953792 0.22025173716247082 19.04077085852623 16.662322163581848\n" + "Epoch 39/200 \t Train Err: 0.7792632356286049 0.004665168027713662 0.009170589764835313 0.277913779951632 2.6232368499040604\n" ] }, { @@ -608,14 +1079,14 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:31<00:00, 1.03it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 9/100 \t Train Err: 7.412262797355652 0.008171883615432307 0.1555994711816311 14.766773402690887 12.345462799072266\n" + "Epoch 40/200 \t Train Err: 0.7583489026874304 0.00292711379006505 0.007873743350501172 0.27461743634194136 2.522021509706974\n" ] }, { @@ -630,7 +1101,37 @@ "name": "stdout", "output_type": "stream", "text": [ - "Epoch 10/100 \t Train Err: 6.162406742572784 0.017439005838241428 0.14521901519037783 10.868462145328522 10.12831449508667\n" + "Epoch 41/200 \t Train Err: 0.7282119914889336 0.0017558942890900653 0.006133111986855511 0.23886349285021424 2.4373623058199883\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 42/200 \t Train Err: 0.7180026862770319 0.0023761796346661868 0.006810532162489835 0.23614320810884237 2.375364724546671\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 43/200 \t Train Err: 0.7019325699657202 0.005046901515015634 0.008944688888732344 0.22622147155925632 2.30900501832366\n" ] }, { @@ -645,7 +1146,127 @@ "name": "stdout", "output_type": "stream", "text": [ - "Epoch 11/100 \t Train Err: 5.102278828620911 0.006056587655621115 0.07073448912706226 7.790802486240864 8.321609899401665\n" + "Epoch 44/200 \t Train Err: 0.6817822623997927 0.0017093055594159523 0.005646904020977672 0.21380048524588346 2.223499771207571\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 45/200 \t Train Err: 0.6641208492219448 0.0038460963405668736 0.0067629652403411455 0.21014394168742 2.1612670943140984\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 46/200 \t Train Err: 0.6523595433682203 0.0018703954374359455 0.006019242704496719 0.20436032535508275 2.1252111680805683\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 47/200 \t Train Err: 0.617787566035986 0.0030503157668135827 0.0069770159825566225 0.1832384041044861 1.9817177206277847\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 48/200 \t Train Err: 0.600332060828805 0.002064528713162872 0.0051435344576020725 0.17475251341238618 1.9067389331758022\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 49/200 \t Train Err: 0.5904311630874872 0.004617106849764241 0.006938040380191524 0.16196146188303828 1.8893518187105656\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 50/200 \t Train Err: 0.5741921290755272 0.004996264638975845 0.007533758063800633 0.15988326235674322 1.8061449080705643\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 51/200 \t Train Err: 0.532125998288393 0.001918236821438768 0.0039053107029758394 0.1349696076940745 1.6755047999322414\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 52/200 \t Train Err: 0.5333235822618008 0.002287628563863109 0.005320303003827576 0.14349294803105295 1.6620671227574348\n" ] }, { @@ -653,14 +1274,14 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:31<00:00, 1.02it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 12/100 \t Train Err: 4.366904504597187 0.005133508995641023 0.059961416525766253 5.993938364088535 6.998718574643135\n" + "Epoch 53/200 \t Train Err: 0.4952954547479749 0.0010008449244196527 0.003929206512111705 0.1226266548037529 1.5398627035319805\n" ] }, { @@ -668,14 +1289,29 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:31<00:00, 1.03it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 13/100 \t Train Err: 3.831405833363533 0.003056721754546743 0.04472046362934634 4.653905652463436 6.165115922689438\n" + "Epoch 54/200 \t Train Err: 0.490776726976037 0.0021560709155892255 0.004821252721740166 0.12306429445743561 1.5102213434875011\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 55/200 \t Train Err: 0.4630659334361553 0.002020925041506416 0.0043996189124300145 0.11745399446226656 1.4124069288372993\n" ] }, { @@ -690,7 +1326,142 @@ "name": "stdout", "output_type": "stream", "text": [ - "Epoch 14/100 \t Train Err: 3.4279200956225395 0.004735801303468179 0.03825691540259868 3.906827114522457 5.425094351172447\n" + "Epoch 56/200 \t Train Err: 0.4430495109409094 0.0022180716405273415 0.004020084164949367 0.10757899098098278 1.3317870814353228\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 57/200 \t Train Err: 0.45779580902308226 0.007255890723172342 0.010569222271442413 0.1332057132385671 1.3446197882294655\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.07it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 58/200 \t Train Err: 0.40347473975270987 0.0013051820133114234 0.0031506957930105273 0.09731322806328535 1.1896578464657068\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 59/200 \t Train Err: 0.3944752989336848 0.0025091177740250714 0.005466303970024455 0.10591710731387138 1.1414270270615816\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 60/200 \t Train Err: 0.3850894244387746 0.0022618337816311396 0.00499681286237319 0.10177210683468729 1.1139366328716278\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 61/200 \t Train Err: 0.3536613713949919 0.0016294859724439448 0.003189665607351344 0.09225898724980652 1.0001246724277735\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 62/200 \t Train Err: 0.3431710870936513 0.0014996515783423092 0.0034722756390692666 0.09043321071658283 0.9589487668126822\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 63/200 \t Train Err: 0.3311467822641134 0.001135409189373604 0.0031531365275441203 0.0849773638183251 0.9113363474607468\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 64/200 \t Train Err: 0.31347283348441124 0.0010190900329689612 0.003212062452803366 0.08056736667640507 0.8582371436059475\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 65/200 \t Train Err: 0.2970640519633889 0.0010584184110484784 0.002994029131514253 0.07881279999855906 0.8098555449396372\n" ] }, { @@ -705,7 +1476,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "Epoch 15/100 \t Train Err: 3.138390600681305 0.005375595836085267 0.03724290645914152 3.326357141137123 4.948893174529076\n" + "Epoch 66/200 \t Train Err: 0.2887888355180621 0.0012073961215719464 0.0033224597755179275 0.0822362134931609 0.7632680460810661\n" ] }, { @@ -720,7 +1491,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "Epoch 16/100 \t Train Err: 2.899445064365864 0.006989890585828107 0.041147086303681135 2.920688170939684 4.558141328394413\n" + "Epoch 67/200 \t Train Err: 0.27321118395775557 0.0009283916206186404 0.0024455750972265378 0.07351583207491785 0.7209800574928522\n" ] }, { @@ -735,7 +1506,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "Epoch 17/100 \t Train Err: 2.6482545658946037 0.006691730450256728 0.03093722724588588 2.4344960935413837 4.143990509212017\n" + "Epoch 68/200 \t Train Err: 0.2642599456012249 0.0024096847391774645 0.004758599683555076 0.07438989775255322 0.6782895382493734\n" ] }, { @@ -743,14 +1514,14 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:31<00:00, 1.03it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 18/100 \t Train Err: 2.4724042862653732 0.013588363406597637 0.03676938998978585 2.2025532834231853 3.8459226489067078\n" + "Epoch 69/200 \t Train Err: 0.24411699920892715 0.0009143148763541831 0.002384097140748054 0.06588289514183998 0.637260627001524\n" ] }, { @@ -758,14 +1529,14 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:31<00:00, 1.03it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 19/100 \t Train Err: 2.2938634902238846 0.007989803227246739 0.031059849599841982 1.9167942628264427 3.582200199365616\n" + "Epoch 70/200 \t Train Err: 0.2355448892340064 0.0011828851866084733 0.00317692070530029 0.07014255912508816 0.5985564198344946\n" ] }, { @@ -773,14 +1544,14 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:31<00:00, 1.03it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 20/100 \t Train Err: 2.0970346182584763 0.004351407576905331 0.02517502213595435 1.652792677283287 3.244159609079361\n" + "Epoch 71/200 \t Train Err: 0.22664254251867533 0.0011206458884771564 0.002485872115357779 0.06011384085286409 0.5786449136212468\n" ] }, { @@ -788,14 +1559,14 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:31<00:00, 1.03it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 21/100 \t Train Err: 1.974123526364565 0.003171493917761836 0.02390849226503633 1.4561462700366974 3.0673259645700455\n" + "Epoch 72/200 \t Train Err: 0.21451926324516535 0.0008609092228653026 0.002164756482670782 0.0564098390750587 0.5476357256993651\n" ] }, { @@ -803,14 +1574,14 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:31<00:00, 1.03it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 22/100 \t Train Err: 1.8447572737932205 0.0030846048357489053 0.021993933914927766 1.3033475428819656 2.8325533121824265\n" + "Epoch 73/200 \t Train Err: 0.2044255300424993 0.0010747715696197702 0.00229402982222382 0.05511317937634885 0.5015372652560472\n" ] }, { @@ -818,14 +1589,14 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:31<00:00, 1.02it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 23/100 \t Train Err: 1.6970642134547234 0.002785886717902031 0.020618034002836794 1.1004324741661549 2.604609090834856\n" + "Epoch 74/200 \t Train Err: 0.200456322170794 0.001536047003355634 0.003252933354815468 0.05711311026243493 0.48181435000151396\n" ] }, { @@ -833,14 +1604,14 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:31<00:00, 1.02it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 24/100 \t Train Err: 1.5955667905509472 0.003940593182051089 0.02032866739318706 0.976159205660224 2.466688357293606\n" + "Epoch 75/200 \t Train Err: 0.19111737608909607 0.003518748955684714 0.004175829379164497 0.049318774370476604 0.45180629566311836\n" ] }, { @@ -848,14 +1619,29 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:31<00:00, 1.03it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 25/100 \t Train Err: 1.4617242440581322 0.002876215428841533 0.018645043048309162 0.8556559775024652 2.229958161711693\n" + "Epoch 76/200 \t Train Err: 0.1808590106666088 0.0013624421417262056 0.002329483479115879 0.04611049557570368 0.4279420841485262\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 77/200 \t Train Err: 0.16973466146737337 0.0008994456393338623 0.00207517536728119 0.04326868336647749 0.3952533109113574\n" ] }, { @@ -870,7 +1656,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "Epoch 26/100 \t Train Err: 1.3831324987113476 0.004849277267567231 0.02050616717315279 0.7726096417754889 2.099547818303108\n" + "Epoch 78/200 \t Train Err: 0.1702535841614008 0.0007078972075760248 0.0016872428823262453 0.04189401387702674 0.3984428942203522\n" ] }, { @@ -878,14 +1664,74 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:31<00:00, 1.03it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 27/100 \t Train Err: 1.311477318406105 0.0024031923712755088 0.01927993548451923 0.7051676390692592 1.976257335394621\n" + "Epoch 79/200 \t Train Err: 0.16377683635801077 0.0013382542465478764 0.0027396336699894164 0.04209184244973585 0.3769973455928266\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 80/200 \t Train Err: 0.1497581924777478 0.0016949967903201468 0.0021530323319893796 0.03568895586067811 0.33177076652646065\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 81/200 \t Train Err: 0.14450395992025733 0.001401656336383894 0.002270640025017201 0.03478072857251391 0.317978051956743\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 82/200 \t Train Err: 0.1389398977626115 0.0033198261371580884 0.00435145793017 0.0375910610309802 0.28218549815937877\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 83/200 \t Train Err: 0.12944017979316413 0.0013174645964681986 0.0020839970657107187 0.033180530241224915 0.2718075602315366\n" ] }, { @@ -900,7 +1746,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "Epoch 28/100 \t Train Err: 1.207214828580618 0.0035346322583791334 0.018441367952618748 0.6115700239315629 1.8264076933264732\n" + "Epoch 84/200 \t Train Err: 0.1269651602488011 0.001952557386175613 0.002797932973408024 0.030586290056817234 0.2562156552448869\n" ] }, { @@ -908,14 +1754,14 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:31<00:00, 1.03it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 29/100 \t Train Err: 1.1089427508413792 0.00329247322952142 0.0165127256186679 0.5199616495519876 1.6691240929067135\n" + "Epoch 85/200 \t Train Err: 0.1262973020784557 0.0034579528983158525 0.004187718592220335 0.03295240050647408 0.26100850058719516\n" ] }, { @@ -923,14 +1769,14 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.03it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 30/100 \t Train Err: 1.029175629839301 0.002865927770471899 0.01539153911289759 0.4984441949054599 1.5174495466053486\n" + "Epoch 86/200 \t Train Err: 0.11676973546855152 0.002271431191729789 0.0028272587187530007 0.029616059619002044 0.22773186769336462\n" ] }, { @@ -945,7 +1791,37 @@ "name": "stdout", "output_type": "stream", "text": [ - "Epoch 31/100 \t Train Err: 0.9538682177662849 0.0023230986480484717 0.01537516585085541 0.4138944335281849 1.406083919107914\n" + "Epoch 87/200 \t Train Err: 0.11645400826819241 0.003942164817090088 0.004389651330711786 0.02888663241174072 0.22617254313081503\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 88/200 \t Train Err: 0.1069438960403204 0.0007534859851148212 0.0015284864657587605 0.02535622369032353 0.20643022749572992\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 89/200 \t Train Err: 0.1019233949482441 0.0009516884665572434 0.001511421851319028 0.024724961316678673 0.18514555739238858\n" ] }, { @@ -960,7 +1836,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "Epoch 32/100 \t Train Err: 0.8980342578142881 0.0015845673569856444 0.014839567360468209 0.40191424917429686 1.3131387010216713\n" + "Epoch 90/200 \t Train Err: 0.09508711867965758 0.0008840842388053716 0.0013085861846775515 0.022952270082896575 0.17402319004759192\n" ] }, { @@ -968,14 +1844,14 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:31<00:00, 1.03it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 33/100 \t Train Err: 0.8195682223886251 0.0034502551516197855 0.014685693866340443 0.3501896969974041 1.191128795966506\n" + "Epoch 91/200 \t Train Err: 0.09273395943455398 0.0008308457795465074 0.0014873276431899285 0.020763451553648338 0.1700441548600793\n" ] }, { @@ -983,14 +1859,14 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:31<00:00, 1.03it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 34/100 \t Train Err: 0.7459018900990486 0.0021442237539304188 0.014179074845742434 0.31923429761081934 1.0604591444134712\n" + "Epoch 92/200 \t Train Err: 0.08730011875741184 0.000720000532510312 0.0011607171036303043 0.020305267593357712 0.15585350175388157\n" ] }, { @@ -998,14 +1874,14 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:31<00:00, 1.03it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 35/100 \t Train Err: 0.7101908139884472 0.0032578711288806517 0.014129422343103215 0.2923557236790657 1.0151417199522257\n" + "Epoch 93/200 \t Train Err: 0.08319679484702647 0.0007787930949234578 0.0012370077938612667 0.018640907626831904 0.14242116385139525\n" ] }, { @@ -1013,14 +1889,74 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.03it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 94/200 \t Train Err: 0.08445112314075232 0.002024608746069134 0.0023303976749957656 0.0193185547250323 0.14395680534653366\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 95/200 \t Train Err: 0.0859984909184277 0.001967401332422014 0.002160638517125335 0.02076483884593472 0.14143188181333244\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 96/200 \t Train Err: 0.07500915951095521 0.0011493155893731455 0.0014125802363196271 0.019469743536319584 0.12187899858690798\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 36/100 \t Train Err: 0.6644597053527832 0.0026260755184921436 0.012881889037089422 0.257190125528723 0.9443178754299879\n" + "Epoch 97/200 \t Train Err: 0.07068800239358097 0.0006672579925179889 0.0009655766079958994 0.01619866766850464 0.11232927581295371\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 98/200 \t Train Err: 0.06899663107469678 0.0029092533500261197 0.002759141156275291 0.01706777891376987 0.10129301541019231\n" ] }, { @@ -1035,7 +1971,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "Epoch 37/100 \t Train Err: 0.6029678452759981 0.0022337357859214535 0.013130559556884691 0.232837010640651 0.837149228900671\n" + "Epoch 99/200 \t Train Err: 0.06587723165284842 0.0010087788145938248 0.0015239717367876437 0.015540250373305753 0.10049017099663615\n" ] }, { @@ -1043,14 +1979,14 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:31<00:00, 1.03it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 38/100 \t Train Err: 0.5509444028139114 0.0019162936450811685 0.012294809013837948 0.2053068270906806 0.7741311714053154\n" + "Epoch 100/200 \t Train Err: 0.06758000492118299 0.0019963337617809884 0.0021085088956169784 0.018880482559325173 0.10054084449075162\n" ] }, { @@ -1058,14 +1994,104 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.03it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 39/100 \t Train Err: 0.5144137293100357 0.0014641922125520068 0.009304212289862335 0.18962949723936617 0.6982439709827304\n" + "Epoch 101/200 \t Train Err: 0.06307036883663386 0.0012753937157867767 0.0016400622589571867 0.015593403164530173 0.09352678328286856\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 102/200 \t Train Err: 0.060316542629152536 0.0006818027850385988 0.0010231798241875367 0.013433107393211685 0.09090642654336989\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 103/200 \t Train Err: 0.060421090107411146 0.0027040049144488876 0.0027294684268781566 0.01441186985175591 0.0847808476537466\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 104/200 \t Train Err: 0.05678413005080074 0.001134833364176302 0.001431436577149725 0.012649833428440616 0.0780830224393867\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 105/200 \t Train Err: 0.05805800168309361 0.003231210850572097 0.003441706239755149 0.015121193689992651 0.0752038118080236\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 106/200 \t Train Err: 0.0547659246949479 0.002454680496839501 0.002737534454809065 0.013384125923039392 0.07274198567029089\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 107/200 \t Train Err: 0.04927568801213056 0.0005825197017657047 0.0007837470261620183 0.010716092379880138 0.05965513514820486\n" ] }, { @@ -1080,7 +2106,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "Epoch 40/100 \t Train Err: 0.4911631550639868 0.0034183577554358635 0.011847828980535269 0.17764808260835707 0.6830073017627001\n" + "Epoch 108/200 \t Train Err: 0.05318018130492419 0.0013841798254361493 0.0015562751477773418 0.01211979822255671 0.07018060437985696\n" ] }, { @@ -1088,14 +2114,14 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:31<00:00, 1.02it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 41/100 \t Train Err: 0.44534157309681177 0.002719711333156738 0.011579871497815475 0.17093974631279707 0.5986328851431608\n" + "Epoch 109/200 \t Train Err: 0.051697830320335925 0.0010280100987074547 0.001098815353543614 0.010801961965626106 0.06942529141088016\n" ] }, { @@ -1103,14 +2129,14 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.03it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 42/100 \t Train Err: 0.4234167579561472 0.0020350335244074813 0.010087796414154582 0.1561200835276395 0.5679095359519124\n" + "Epoch 110/200 \t Train Err: 0.04692836094181985 0.0008446434776487877 0.0008702427021489711 0.01032250898424536 0.05933116714004427\n" ] }, { @@ -1118,14 +2144,14 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.03it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 43/100 \t Train Err: 0.38217254262417555 0.001672563766987878 0.008801718227914535 0.1451707179658115 0.5035076225176454\n" + "Epoch 111/200 \t Train Err: 0.044668186688795686 0.0011062668152135302 0.0010394338919468282 0.009862772989436053 0.054550442087929696\n" ] }, { @@ -1133,28 +2159,1228 @@ "output_type": "stream", "text": [ "\n", - "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.03it/s]" + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 112/200 \t Train Err: 0.0447915755212307 0.0014217505979559064 0.0016150277879205532 0.010183335049077868 0.05510500964010134\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 113/200 \t Train Err: 0.043250435031950474 0.002363460382412086 0.002828258522640681 0.00971244159154594 0.047139356975094415\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 114/200 \t Train Err: 0.045086685102432966 0.0011699077874709474 0.0013979203313283506 0.010937912025838159 0.0543683817377314\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Epoch 44/100 \t Train Err: 0.36583498027175665 0.0006311295665000216 0.0070743790856795385 0.13195386109873652 0.4824865907430649\n" + "Epoch 115/200 \t Train Err: 0.0411334311356768 0.0013191173275117762 0.0011784266635004315 0.00858650918235071 0.04524634237168357\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ - " 69%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊ | 22/32 [00:21<00:09, 1.06it/s]" + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 116/200 \t Train Err: 0.039555212075356394 0.0005507277148808498 0.0005802504801977193 0.007789377530571073 0.04690207863677642\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 117/200 \t Train Err: 0.03692283679265529 0.00043711723651540524 0.0005214224675000878 0.007687705321586691 0.03813074165373109\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 118/200 \t Train Err: 0.03628927277168259 0.00044126920215603604 0.0005619980265691993 0.007773650635499507 0.0384644401492551\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 119/200 \t Train Err: 0.03890394070185721 0.001729899499878229 0.001493205483257043 0.00798933092301013 0.041969388112192973\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 120/200 \t Train Err: 0.03712784848175943 0.0007071991701650404 0.0008030551111914974 0.007690385325986426 0.04134120314847678\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 121/200 \t Train Err: 0.03516055888030678 0.0004485433178160747 0.0006193339004312293 0.006722165890096221 0.03440287291596178\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 122/200 \t Train Err: 0.03431048977654427 0.0007996012898274785 0.000579838467274385 0.007459181011654437 0.03256209765095264\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 123/200 \t Train Err: 0.0351686873473227 0.0006591397341253469 0.0006555682361977233 0.006613661207666155 0.037333723666961305\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 124/200 \t Train Err: 0.03355215600458905 0.0002945618743979139 0.0004387611575111805 0.006839323941676412 0.034134260716200515\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 125/200 \t Train Err: 0.029283913841936737 0.00046266792719507066 0.00058449966218177 0.0056224113504868 0.024552044065785594\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 126/200 \t Train Err: 0.03119118546601385 0.0005014674925405416 0.0007099974022821698 0.0060376596375135705 0.02815229452244239\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 127/200 \t Train Err: 0.031841877149417996 0.00027776491560871364 0.00046666834350617137 0.005703783383069094 0.03038091241910479\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 128/200 \t Train Err: 0.030603406310547143 0.0006404905841463915 0.0006846741659956024 0.006221181378350593 0.025205713599461887\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 129/200 \t Train Err: 0.029333731392398477 0.0005602014119858723 0.000642365913336107 0.006284213814069517 0.02611703903221496\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 130/200 \t Train Err: 0.028047623636666685 0.0008307177447477443 0.0007543345300291548 0.006256156790186651 0.02223676520952722\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.07it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 131/200 \t Train Err: 0.02852347370935604 0.0005703339811589103 0.0009201887021390576 0.005433428774267668 0.025521833274979144\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 132/200 \t Train Err: 0.028580970887560397 0.0005810092270621681 0.0005398729952048598 0.005319414980476722 0.027794180728960782\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 133/200 \t Train Err: 0.02704069105675444 0.0003059067348658573 0.0003909187862518593 0.0050654686710913666 0.023506886311224662\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 134/200 \t Train Err: 0.024850417801644653 0.00024160909720194468 0.0003672063955946214 0.004596906874212436 0.01910146742920915\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 135/200 \t Train Err: 0.024681290611624718 0.00037400397729925317 0.000489667942019878 0.004580263695970643 0.019593202690884937\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 136/200 \t Train Err: 0.027638425526674837 0.0004977623525519448 0.0005427452181265835 0.005582943878835067 0.02646890448522754\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 137/200 \t Train Err: 0.024757418257649988 0.0003685475935526483 0.0003517182428822707 0.004618304148607422 0.018053345412681665\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 138/200 \t Train Err: 0.02439499576576054 0.00027227840394061786 0.00036000374461764295 0.004171678388956934 0.020061984119138287\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 139/200 \t Train Err: 0.0229199705645442 0.00046586399253101263 0.0005068485352239804 0.004258882479916792 0.01709363247573492\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 140/200 \t Train Err: 0.024649846891406924 0.0006087151427891513 0.0007250320297771395 0.004237969173118472 0.022251372053233354\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 141/200 \t Train Err: 0.023699990182649344 0.0006855616327356984 0.0008273822963928978 0.004291944787837565 0.01694076002422662\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 142/200 \t Train Err: 0.022072256018873304 0.00023243963084951247 0.0003913031405318179 0.0035956969295511954 0.019112591930934286\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 143/200 \t Train Err: 0.023525280528701842 0.00040764258983472246 0.0003686606121391378 0.00405594674157328 0.02093119484015915\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 144/200 \t Train Err: 0.02343721961369738 0.00033207295859938313 0.0005353521628421731 0.004094722855370492 0.020869585259788437\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 145/200 \t Train Err: 0.02064004127169028 0.00022399188014787796 0.0003062049111122178 0.004059467602928635 0.017480447175330482\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 146/200 \t Train Err: 0.020763279171660542 0.0004162326339383071 0.0003410943327253335 0.0032998187525663525 0.01631780587376852\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 147/200 \t Train Err: 0.020571442728396505 0.00044352215627441183 0.0005010924719499599 0.003667172095447313 0.014751498597888713\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 148/200 \t Train Err: 0.01997521542944014 0.00021835572067629982 0.000277048953648773 0.0034534092810645234 0.015434162895644477\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 149/200 \t Train Err: 0.021312509546987712 0.00030864444181588624 0.0005689966556019499 0.004085102085809922 0.01556764474753436\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 150/200 \t Train Err: 0.019479235576000065 0.0003387887146573121 0.000369657065903084 0.003481321306026075 0.01391503391016613\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 151/200 \t Train Err: 0.023493657063227147 0.0003406951947226844 0.0004765782725826284 0.003387280346942134 0.024297427552482986\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 152/200 \t Train Err: 0.020062169001903385 0.0001796720187030587 0.0002183982696806197 0.003409319913771469 0.013818501599416777\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 153/200 \t Train Err: 0.019131189765175804 0.00029574484346994723 0.0002947069822312187 0.002951359812868759 0.01664116706706409\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 154/200 \t Train Err: 0.017693023837637156 0.00024806520877973526 0.0002983832400786923 0.00312236421086709 0.010928787386546901\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.07it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 155/200 \t Train Err: 0.017173090571304783 0.00020240362800905132 0.0002837313263626129 0.0024838966492097825 0.01155280199873232\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 156/200 \t Train Err: 0.018064823234453797 0.00025287491257586225 0.00042023322453133005 0.0031510025983152445 0.013277582744422034\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 157/200 \t Train Err: 0.01679313532076776 0.00016895983264930692 0.00023369373479908973 0.0025660492974566296 0.013730080009736412\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 158/200 \t Train Err: 0.018779858626658097 0.0006825042137279524 0.00039098577053664485 0.0030517936720571015 0.014663536599073268\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 159/200 \t Train Err: 0.020110780431423336 0.000910040460212258 0.000834133242506141 0.004305297403334407 0.013905958928830842\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 160/200 \t Train Err: 0.019773374748183414 0.0009348806023581346 0.0008922487559175352 0.003975474533945089 0.01617197653695257\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 161/200 \t Train Err: 0.017475509172072634 0.00046213520988658274 0.0005404102516877174 0.003040906784008257 0.014273262689584953\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 162/200 \t Train Err: 0.015029630245408043 0.0001904235211895866 0.00021963075801068044 0.002168249424357782 0.010509091553103644\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 163/200 \t Train Err: 0.018164523935411125 0.0005775698982688482 0.0006741533361491747 0.0036203847994329408 0.011193782368536631\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 164/200 \t Train Err: 0.01632957032416016 0.0003306799113715897 0.0003712935063049372 0.0029763625025225338 0.009990961348194105\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 165/200 \t Train Err: 0.0153682763047982 0.0002632132137705412 0.0003610171986565547 0.0023970418515091296 0.012086256845577736\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 166/200 \t Train Err: 0.013866374094504863 0.00012185312010615235 0.00018107489950125455 0.0018877797901950544 0.008829483279441774\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 167/200 \t Train Err: 0.015403825411340222 0.00034605842233759176 0.00046738405558244267 0.002471095247528865 0.009661761823792858\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 168/200 \t Train Err: 0.015333578106947243 0.0007963078196553397 0.0006389631322463174 0.0029809060470142867 0.009118859635066201\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 169/200 \t Train Err: 0.01377101321122609 0.00028809423406528367 0.0002655940699014536 0.0022669341324217385 0.0077932964362616985\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 170/200 \t Train Err: 0.014377146551851183 0.0006095316629171066 0.000566291571203692 0.002504439868062036 0.007364666845887768\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 171/200 \t Train Err: 0.01476844641729258 0.0002849637266422178 0.00032093194135995873 0.002440014208332286 0.00954770603743782\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 172/200 \t Train Err: 0.014352218277053908 0.0001346213794590767 0.0002298443192785271 0.0027853190676978556 0.011637392487727993\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 173/200 \t Train Err: 0.013293113705003634 0.0002518077910735883 0.00021664196594883833 0.0016753806212363997 0.008185207087933577\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 174/200 \t Train Err: 0.01469786892994307 0.0005061904257672722 0.0006176612511126223 0.0025409646332263947 0.009795411139066346\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 175/200 \t Train Err: 0.01577709667617455 0.0002547664359440205 0.0003405914605991711 0.0020204293869028334 0.013386641423039691\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 176/200 \t Train Err: 0.014100697473622859 0.00036510653808363713 0.0004396585003405562 0.0022091059381637024 0.00978638644005514\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 177/200 \t Train Err: 0.014368639851454645 0.00044816699323746434 0.00045128344186196045 0.002814159254739934 0.008726180123630911\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 178/200 \t Train Err: 0.014014390297234058 0.0006206710049809772 0.00046763980253672344 0.002513490486308001 0.008434324339646082\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 179/200 \t Train Err: 0.011972312582656741 0.00018721630084428398 0.00029149899171443394 0.0016975146063487045 0.007282969920481719\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 180/200 \t Train Err: 0.01289494353113696 0.00020095304734013553 0.000251299350793488 0.0017014556851790985 0.010998782015121833\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 181/200 \t Train Err: 0.013175164320273325 0.000340911025944024 0.00035113496642225073 0.002221787386588403 0.008535892283987323\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 182/200 \t Train Err: 0.012816024391213432 0.0003103266114408143 0.00028728846336889546 0.0020579011925292434 0.00718217800522325\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 183/200 \t Train Err: 0.01449834939558059 0.0005017787769361348 0.0005829150882163958 0.002624640494104824 0.012894641169850729\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 184/200 \t Train Err: 0.01308623020304367 0.0003065749452275668 0.0002674301056231343 0.0018643831190274796 0.009087016163903172\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 185/200 \t Train Err: 0.011409497121348977 0.00029326116043648653 0.0003596148741280558 0.001658103732552263 0.007742758548488382\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 186/200 \t Train Err: 0.010975291283102706 0.00022339119993830536 0.00027696753636519134 0.0014451522856688825 0.0054481543454585335\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 187/200 \t Train Err: 0.010257220594212413 0.00023113110682970728 0.0002813607433154175 0.001320708272032789 0.005819132793249082\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.06it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 188/200 \t Train Err: 0.012054045568220317 0.000388129293298789 0.0004733345151635149 0.001878096192740486 0.00747043985700202\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 189/200 \t Train Err: 0.012783893820596859 0.00031030693571665324 0.00032841233428371197 0.0017014134382407065 0.006328187721919676\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.05it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 190/200 \t Train Err: 0.01047908267355524 0.000252351564199671 0.0003104653961827353 0.0015047969627630664 0.00647446524692441\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 191/200 \t Train Err: 0.01059192547108978 0.00032810493928536744 0.00036611615337278636 0.0015751951596030267 0.005902428454646724\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "00%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:30<00:00, 1.04it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 192/200 \t Train Err: 0.011238765117013827 0.0005425223790780365 0.0005242744022098123 0.0020095554464205634 0.006199448111232186\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " 75%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▎ | 24/32 [00:22<00:07, 1.03it/s]" ] } ], "source": [ "while epoch < NEPOCHS:\n", " model.train()\n", - " with open(f\"data/{epoch}.pickle\", \"rb\") as f:\n", + " with open(f\"data-new/{epoch}.pickle\", \"rb\") as f:\n", " pickled_stuff = pickle.load(f)\n", " data = pickled_stuff[\"data\"].to(device)\n", " label = pickled_stuff[\"labels\"].to(device).to(torch.float32)\n", @@ -1186,31 +3412,60 @@ }, { "cell_type": "code", - "execution_count": 16, + "execution_count": 12, + "execution_state": "idle", "metadata": {}, "outputs": [], "source": [ - "# \"\"\"\n", - "# Now let's figure out what it's doing. \n", - "\n", - "# step 1: figure out what people are attending to \n", - "# \"\"\"\n", - "\n", - "# example_graph, answer, padding = mkbatch(1)\n", - "# sentance_embeddings = model.full_embedding(example_graph)[0,:,:][example_graph.flatten() != 0]\n", - "# WQ,WK,WV = torch.split(model.transformer_encoder.layers[0].self_attn.in_proj_weight, (MODEL_DIM, MODEL_DIM, MODEL_DIM))\n", - "\n", - "# Q = sentance_embeddings@WQ\n", - "# K = sentance_embeddings@WK\n", - "\n", - "# raw_scores = Q @ K.T / sqrt(MODEL_DIM)\n", - "# soft = torch.softmax(raw_scores, dim=-1).detach().cpu().to(float).numpy()\n", - "# plt.imshow(soft)\n", - "# plt.show()\n", - "\n", - "# print(example_graph)\n", + "def evaluate(data):\n", + " model.eval()\n", + " test_loss = 0\n", + " with torch.no_grad():\n", + " batch_src, batch_labels, batch_padding_mask = map(lambda x: x.to(device), data)\n", + " output = model(batch_src, batch_padding_mask)\n", + " loss = criterion(output.squeeze(1), batch_labels)\n", + " return output.detach(), batch_labels.detach().to(torch.uint8), loss.item()\n", "\n", - "# print(Q)" + "def mkhist(data):\n", + " y, x, loss = evaluate(data)\n", + " print(loss)\n", + " cnts = torch.bincount(x)\n", + " weights = [1/cnts[i.item()].item() for i in x] # normalize by label count\n", + " fig, ax = plt.subplots()\n", + " h = ax.hist2d(x.cpu().numpy().flatten(), y.cpu().numpy().flatten(), weights=weights, bins=[15,50], norm=mpl.colors.LogNorm())\n", + " ax.set_xlabel(\"Label\")\n", + " ax.set_ylabel(\"Model Output\")\n", + " fig.colorbar(h[3], ax=ax)" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/home/sipb/.venv/lib64/python3.12/site-packages/torch/nn/modules/transformer.py:871: UserWarning: Memory Efficient attention on Navi31 GPU is still experimental. Enable it with TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1. (Triggered internally at ../aten/src/ATen/native/transformers/hip/sdp_utils.cpp:269.)\n", + " return torch._transformer_encoder_layer_fwd(\n", + "/home/sipb/.venv/lib64/python3.12/site-packages/torch/nn/modules/transformer.py:871: UserWarning: Attempting to use hipBLASLt on an unsupported architecture! Overriding blas backend to hipblas (Triggered internally at ../aten/src/ATen/Context.cpp:296.)\n", + " return torch._transformer_encoder_layer_fwd(\n" + ] + }, + { + "data": { + "text/plain": [ + "0.031184175983071327" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "evaluate(mkbatch(BSZ))[2]" ] }, { @@ -1219,99 +3474,57 @@ "metadata": {}, "outputs": [], "source": [ - "with open('training-loss') as f:\n", - " train_err = list(map(float, f.read().split()))\n", - " plt.suptitle('Log MSE vs Epochs')\n", - " plt.plot(torch.log(torch.tensor(train_err)[:500]), label='Train', color='blue')\n", - " plt.xlabel('Epochs')\n", - " plt.ylabel('Log MSE')\n", - " plt.show()\n", - "\n", - "plt.suptitle('Log MSE vs Epochs')\n", - "plt.plot(torch.log(torch.tensor(train_err)), label='Train', color='blue')\n", - "plt.plot(torch.log(torch.tensor(len1)).to(torch.float16), label='Len 1', color='red')\n", - "plt.plot(torch.log(torch.tensor(len2)).to(torch.float16), label='Len 2', color='green')\n", - "plt.plot(torch.log(torch.tensor(len3)).to(torch.float16), label='Len 3', color='yellow')\n", - "plt.plot(torch.log(torch.tensor(len15)).to(torch.float16), label='Len 15', color='magenta')\n", + "with open('loss') as f:\n", + " losses = torch.tensor([list(map(float, l.split())) for l in f.readlines()])\n", + "plt.plot(torch.log(losses[:, 0]), label='Train', color='blue')\n", "plt.xlabel('Epochs')\n", "plt.ylabel('Log MSE')\n", - "plt.legend()\n", - "plt.show()" + "with open(\"plots/train-loss.html\", \"w\") as f:\n", + " mpld3.save_html(plt.gcf(), f)" ] }, { "cell_type": "code", - "execution_count": 1, + "execution_count": 27, "metadata": { "id": "LoGEmM5lH7_A" }, "outputs": [ { - "ename": "NameError", - "evalue": "name 'mkbatch' is not defined", - "output_type": "error", - "traceback": [ - "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", - "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)", - "Cell \u001b[0;32mIn[1], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m batch_src, batch_labels, batch_padding_mask \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mmap\u001b[39m(\u001b[38;5;28;01mlambda\u001b[39;00m x: x\u001b[38;5;241m.\u001b[39mto(device), \u001b[43mmkbatch\u001b[49m(BSZ))\n\u001b[1;32m 2\u001b[0m model\u001b[38;5;241m.\u001b[39meval()\n\u001b[1;32m 3\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m torch\u001b[38;5;241m.\u001b[39mno_grad():\n", - "\u001b[0;31mNameError\u001b[0m: name 'mkbatch' is not defined" - ] + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhIAAAGlCAYAAACvGh/vAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAspElEQVR4nO3df3RU9Z3/8ddkYgakSTSwJAwmkrVUGsBEIfBF/NZQc8qmNJT1WLWHYhrPYmsHAeNBYHeBWn+k6JZGbRaKPUrbI4o9LdTiV1o2RaJbkPxoPHVb+bGNNIUmyFYzJJSAM/f7hzBr5II3c+/cmdw8H+d8jmfuzJ33+4qSdz4/fYZhGAIAAIhDWrITAAAAgxeFBAAAiBuFBAAAiBuFBAAAiBuFBAAAiBuFBAAAiBuFBAAAiBuFBAAAiBuFBAAAiBuFBAAAiBuFBAAAiBuFBAAAHrZ9+3ZdffXVGj9+vH7wgx84/v0+Du0CAMCb3n//fRUVFWnXrl3Kzs7WlClT9Jvf/EYjR450LAY9EgAAeNS+ffs0ceJEjR07Vp/4xCdUUVGhX/3qV47GoJAAACBFNTY2qrKyUsFgUD6fT9u2bTvvM/X19Ro3bpyGDRum6dOna9++fbH3jh49qrFjx8Zejx07VkeOHHE0RwoJAABSVG9vr4qLi1VfX2/6/pYtW1RTU6M1a9aotbVVxcXFmj17to4dO+ZajumuRbIoGo3q6NGjyszMlM/nS3Y6AIAUZhiGTpw4oWAwqLS0xP1ufOrUKZ0+fdr29xiGcd7PtkAgoEAgYPr5iooKVVRUXPD71q1bp4ULF6q6ulqStGHDBr300kt6+umntWLFCgWDwX49EEeOHNG0adNsP0c/Rorp6OgwJNFoNBqNZrl1dHQk7OfS3/72NyNvtN+RPD/xiU+cd23NmjWW8pBkbN26Nfa6r6/P8Pv9/a4ZhmHccccdxty5cw3DMIwzZ84Yn/zkJ40///nPxokTJ4xPfepTxvHjxx3995NyPRKZmZmSpI6ODmVlZSU7HQBAnKJd1yY8Rrgnqiuvezv2syMRTp8+rc5jEbW3XKmszPh7PcInoiqccvi8n28X6o34OMePH1ckElFubm6/67m5uXrrrbckSenp6frOd76jWbNmKRqN6v7773d0xYZScWjjXJdPVlYWhQQADGLRk37XYrkxFJ6VmWarkIh9j8s/3+bOnau5c+cm7PtTrpAAACAVRYyoIoa9+500atQo+f1+dXV19bve1dWlvLw8R2NdDKs2AACwICrDdnNSRkaGpkyZooaGhv/NMRpVQ0ODZsyY4Wisi6FHAgAAC6KKyk6fwrm7S0tL5ff7FQqFFAqFLnpPT0+PDh06FHvd3t6utrY25eTkqKCgQDU1NaqqqtLUqVM1bdo01dXVqbe3N7aKww0UEgAAuKipqcnyHInm5mbNmjUr9rqmpkaSVFVVpU2bNum2227TO++8o9WrV6uzs1MlJSXasWPHeRMwEynlztoIh8PKzs5Wd3c3ky0RU5G/xJU4L3c87kocAM5w42fGuRgdb421vWojf8IRz/18o0cCAAAL7M5zcHqORKpgsiUAAIgbhQQAABZEZShio53rkSgtLVVRUdEFz88YbBjaAADAAqeGNgYy2XIwoEcCAADEjR4JAAAsiBiGIjYWOtq5N5VRSGBQYFkmgGSLnm127vcihjYAAEDcKCQAALDAzoqNc02s2gAAYGiKGLJ5+ucH//Taqg0KCQAALGCOhDmGNgAAQNzokQAAwIKofIrIZ+t+L6KQAADAgqjxQbNzvxcNeGijsbFRlZWVCgaD8vl82rZt2wU/+/Wvf10+n091dXV28wQAAClowIVEb2+viouLP3bZytatW7V3714Fg0E7+QEAkBIiZ4c27DSx/FOqqKhQRUXFRT9z5MgR3XPPPfrlL3+pOXPm2MkPAICUELE5R+LcvV5b/un4qo1oNKoFCxZo2bJlmjhxotNfDwAAUojjky3Xrl2r9PR0LV682NLn+/r61NfXF3sdDoedTgkAANuihk9Rw8aqDRv3pjJHC4mWlhY9/vjjam1tlc9n7V9YbW2tHnjgASfTAADAcU4NbXiNo0Mbr776qo4dO6aCggKlp6crPT1dhw8f1n333adx48aZ3rNy5Up1d3fHWkdHh5MpAQCABHK0R2LBggUqLy/vd2327NlasGCBqqurTe8JBAIKBAJOpgHgQ8Y986grcd6uvt+VOECyRJSmiI3fvyOOZpM6BlxI9PT06NChQ7HX7e3tamtrU05OjgoKCjRy5Mh+n7/kkkuUl5enq6++2pmMAQBIAsPmHAmDORIfaG5u1qxZs2Kva2pqJElVVVXatGmTs9kBAJAimCNhbsCFRFlZmQzD+j6fb7/99kBDAADgWaWlpfL7/QqFQgqFQslOxzbO2gAAwIKIkaaIYWOOxNnfwb22IRWFBAAAFkTlU9TGZMuovHlql+M7WwIAgKGDHgnA41iWCTiDyZbmKCQAALDA/hwJhjYAAAD6oUcCAAALPphsaePQLoY2AAAYuqI2t8hm1QYAAMBHUEgAAGDBucmWdprO7mxZVFSk+vr6ZD+SIxjaAADAgqjSHNmQip0tAQAYgiKGTxEbJ3jauTeVMbQBAADiRo8EAAAWRGyu2oh4dNUGhQQAABZEjTRFbexsGWVnSwAAgP7okQAAwAKGNsxRSAAAYEHU5sqLqKPZpA6GNgAAQNzokQAAwAL7G1J583d3CgkAACz48DbX8d7vRd58KgAA4AoKCQAALIjKZ7uJQ7sAABianBra4NAuAACGIPv7SHhzEMCbTwUAAFxBjwQAABZEDZ+idjak8ugx4hQSAABYELU5tOHVfSS8+VQAAMAV9EgAAGCB/WPEvfm7O4UEAAAWRORTRPHPc7BzbyrzZnkEAABcQY8EAAAWMLRhjkICg0JF/hJX4rzc8bgrcQAMPhGbwxMRR7NJHd4sjwAAgCvokQAAwAKGNsxRSAAAYIFTh3Z5DYUEAAAWGB86Cjze+73Im+URAABwBT0SAABYwNCGOQoJDApeW5Y5ccV3XYvVe/UZV+K8XX2/K3GAZOH0T3PeLI8AAEhRpaWlKioqUn19fbJTccSAC4nGxkZVVlYqGAzK5/Np27ZtsffOnDmj5cuXa/LkyRoxYoSCwaDuuOMOHT161Om8AQBwVeTsMeJ2miQ1NTXp97//vUKhULIfyREDLiR6e3tVXFxsWkmdPHlSra2tWrVqlVpbW/Wzn/1M+/fv19y5c53KFwCApDg3tGGnedGA50hUVFSooqLC9L3s7Gzt3Lmz37Xvfe97mjZtmv70pz+poKAg/kwBAEDKSfhky+7ubvl8Pl122WWm7/f19amvry/2OhwOJzolAAAGLKo0RW1MLbRzbypL6FOdOnVKy5cv15e//GVlZWWZfqa2tlbZ2dmxlp+fn8iUAACIS8Tw2W5elLBC4syZM7r11ltlGIbWr19/wc+tXLlS3d3dsdbR0ZGolAAAgMMSMrRxrog4fPiwfv3rX1+wN0KSAoGAAoFAItIAUtZ/ffveZKcAYIDYR8Kc44XEuSLi4MGD2rVrl0aOHOl0CAAAXGfYPP3TYGfLD/T09OjQoUOx1+3t7Wpra1NOTo7GjBmjW265Ra2trdq+fbsikYg6OzslSTk5OcrIyHA2ewAAXBKRTxEbB2/ZuTeVDbiQaG5u1qxZs2Kva2pqJElVVVX65je/qRdffFGSVFJS0u++Xbt2qayszH7GAAAgZQy4kCgrK5NhGBd8/2LvAQAwWEUNe/Mcoh798cihXQAAWBC1OUfCzr2pzJtPBQAAXEGPBAAAFkTlU9TGhEk796YyCgkAACywuzslO1sCAAB8BD0SAABYwGRLcxQSAABYEJXNLbI9OkfCm+URAABwBT0SAABYYNhctWF4tEeCQgIAAAs4/dMchQTwITdWPuZKnN2/WOZKHADOYbKlOW8+FQAAcAU9EgAAWMDQhjkKCQAALGCLbHMMbQAAMIT94z/+oy6//HLdcsstcd1PIQEAgAXnhjbstFS0ZMkS/ehHP4r7fgoJAAAs8GohUVZWpszMzLjvZ44E8CFeXJY57plHXYnzdvX9rsQBhpLGxkY99thjamlp0V/+8hdt3bpV8+bN6/eZ+vp6PfbYY+rs7FRxcbGefPJJTZs2zbUcKSQAALAgGas2ent7VVxcrDvvvFM333zzee9v2bJFNTU12rBhg6ZPn666ujrNnj1b+/fv1+jRoyVJJSUlev/998+791e/+pWCwWCcT/O/KCQAALDAqUIiHA73ux4IBBQIBEzvqaioUEVFxQW/c926dVq4cKGqq6slSRs2bNBLL72kp59+WitWrJAktbW1xZ2zFcyRAADARfn5+crOzo612trauL7n9OnTamlpUXl5eexaWlqaysvLtWfPHgczvjh6JAAAsMCwuReEcfafHR0dysrKil2/UG/Exzl+/LgikYhyc3P7Xc/NzdVbb71l+XvKy8v1xhtvqLe3V1dccYV+8pOfaMaMGZbvp5AAAMACp4Y2srKy+hUSyfYf//Eftu6nkAAAwIJU2yJ71KhR8vv96urq6ne9q6tLeXl5jsa6GOZIAAAwCGVkZGjKlClqaGiIXYtGo2poaBjQ0IRd9EgAHsf+DoAznOqRKC0tld/vVygUUigUuug9PT09OnToUOx1e3u72tralJOTo4KCAtXU1KiqqkpTp07VtGnTVFdXp97e3tgqDjdQSAAAYIFThURTU5PlORLNzc2aNWtW7HVNTY0kqaqqSps2bdJtt92md955R6tXr1ZnZ6dKSkq0Y8eO8yZgJhKFBAAAKaqsrEyGYVz0M4sWLdKiRYtcy+mjKCQAALDAMHwybPRI2Lk3lVFIAABgQVQ+W/tI2Lk3lbFqAwAAF5WWlqqoqEj19fXJTsUR9EgAAGBBMiZbDgYUEhgUKvKXuBLn5Y7HXYnjJo4RB5zBHAlzDG0AAIC40SMBAIAFqbZFdqqgkAAAwAKGNswxtAEAgAXG2R6JeJvxoS2yWbUBAADiwqoNAACGIEPSx+xW/bH3exGFBAYFLy7LdAvLMgFnROWTj50tz8McCQAAEDd6JAAAsIBVG+YG3CPR2NioyspKBYNB+Xw+bdu2rd/7hmFo9erVGjNmjIYPH67y8nIdPHjQyZwBAHCdnRUbdvegSGUDLiR6e3tVXFx8wWUrjz76qJ544glt2LBBr7/+ukaMGKHZs2fr1KlTTuQLAMCgNuSXf1ZUVKiiosL0PcMwVFdXp3/913/VF7/4RUnSj370I+Xm5mrbtm26/fbb7WcMAEASGIbNVRtn7/Xa8k9HJ1u2t7ers7NT5eXlsWvZ2dmaPn269uzZY3pPX1+fwuFwvwYAQKo5N0fCTvMiRwuJzs5OSVJubm6/67m5ubH3Pqq2tlbZ2dmxlp+f72RKAAAggZK+amPlypWqqamJvQ6HwxQTSJobKx9zJc7xie79r9d79RlX4rBfBbyOVRvmHP3bLC8vT5LU1dWlMWPGxK53dXWppKTE9J5AIKBAIOBkGgAAOC5q+OTj9M/zODq0UVhYqLy8PDU0NMSuhcNhvf7665oxY4aToQAAcNW5yZZ2mhcNuEeip6dHhw4dir1ub29XW1ubcnJyVFBQoKVLl+qhhx7S+PHjVVhYqFWrVikYDGrevHlO5w4AAJJswIVEc3OzZs2aFXt9bn5DVVWVNm3apPvvv1+9vb2666679N577+mGG27Qjh07NGzYMGczBwDARR/0KtiZI+FoOiljwIVEWVmZjIv82/D5fPrWt76lb33rW3ZzAwAgZTg12bK0tFR+v1+hUEihUMjBDJMj6as2AAAYSry2IRWFBPAhu3+xzJU4V31nnStx5OKyzLf/PMbCp5wx7oq/uBYLOMc42+zc70UUEgAAWMA+EuYcXf4JAACGFnokAACwgrENUxQSAABYYffgLY8ObVBIAABggVPHiHsNcyQAAEDc6JEAAMACVm2Yo5AAPsStY8T/26X9KiRp1q/vcyXOrs+ytwM8zvDZm+fAzpYAAMAudrYEAGAIYrKlOQoJAACsYB8JU6zaAAAAcaNHAgAAC1i1YY5CAgAAqzw6PGEHhQTwIW4dIz7u+//mShxJygqOcC0WgKGHQgIAAAsY2jBHIQEAgBWs2jBFIQEAgCW+s83O/d7D8k8AABA3eiQAALCCoQ1T9EgAAGCF4UA7e2hXUVGR6uvrk/1EjqBHAoNCRf4SV+K83PG4K3FGtPtdiSNJ4ycfdyXOjw/+H1fiSNKC8XtdiwU4jUO7AAAYihw6RtxrKCQAALCA0z/NMUcCAADEjR4JAACsYNWGKQoJAACsYI6EKYY2AABA3OiRAADAAp/xQbNzvxdRSGBQcGt/h6n/tM6VOGlzu12JI0kH/2eUK3EWXM/eDvA45kiYopAAAMAK5kiYYo4EAACIGz0SAABYwdCGKQoJAACsoJAwxdAGAACIGz0SAABYQY+EKQoJDApuHSM+ouQKV+IUjjrmShxJisqbM8UB17FqwxRDGwAAIG70SAAAYAE7W5pzvEciEolo1apVKiws1PDhw3XVVVfpwQcflOHVg9gBAEOD4UCTVFpaqqKiItXX1yf7iRzheI/E2rVrtX79ev3whz/UxIkT1dzcrOrqamVnZ2vx4sVOhwMAYFBpampSVlZWstNwjOOFxG9+8xt98Ytf1Jw5cyRJ48aN03PPPad9+/Y5HQoAACSZ40Mb119/vRoaGnTgwAFJ0htvvKHXXntNFRUVTocCAMA1vg/Nk4irJfsBEsTxHokVK1YoHA5rwoQJ8vv9ikQievjhhzV//nzTz/f19amvry/2OhwOO50SAAD2sfzTlOOFxAsvvKBnn31Wmzdv1sSJE9XW1qalS5cqGAyqqqrqvM/X1tbqgQcecDoNeIxbx4h/8tHvuhJn4rATrsSRpMvTT7oWC8DQ43ghsWzZMq1YsUK33367JGny5Mk6fPiwamtrTQuJlStXqqamJvY6HA4rPz/f6bQAALCHnS1NOV5InDx5Umlp/ade+P1+RaNR088HAgEFAgGn0wAAwFkUEqYcLyQqKyv18MMPq6CgQBMnTtRvf/tbrVu3TnfeeafToQAAQJI5Xkg8+eSTWrVqlb7xjW/o2LFjCgaD+trXvqbVq1c7HQoAANews6U5xwuJzMxM1dXVqa6uzumvBgAgeRjaMMWhXQAAIG4c2oVBwa1jxN9/KOhKHDeXZP7meKFrsQBPo0fCFIUEAAAWMEfCHEMbAAAgbvRIAABgBVtkm6KQAADACuZImKKQAADAAuZImGOOBAAAiBs9EhgU/jbRnWWZ46/sdCVOdvrfXIkjSeMy/+paLMDTGNowRSEBAIAVNoc2vFpIMLQBAADiRo8EAABWMLRhikICAAArKCRMMbQBAADiRo8EAAAWsI+EOXokAAAYojo6OlRWVqaioiJdc801+slPfjLg76BHAnH7h+w7XYvVPX+yK3G+MPJtV+K8+j+fdCWOJP3i/z7pWiwAg0t6errq6upUUlKizs5OTZkyRZ///Oc1YsQI69+R0AwBAPAKD062HDNmjMaMGSNJysvL06hRo/TXv/51QIUEQxsAAFhwbo6EnTZQjY2NqqysVDAYlM/n07Zt2877TH19vcaNG6dhw4Zp+vTp2rdvX1zP19LSokgkovz8/AHdRyEBAIBVho0Wh97eXhUXF6u+vt70/S1btqimpkZr1qxRa2uriouLNXv2bB07diz2mZKSEk2aNOm8dvTo0dhn/vrXv+qOO+7Qxo0bB5wjQxsAALgoHA73ex0IBBQIBEw/W1FRoYqKigt+17p167Rw4UJVV1dLkjZs2KCXXnpJTz/9tFasWCFJamtru2g+fX19mjdvnlasWKHrr79+wM9DjwQAAFbY6Y34UK9Efn6+srOzY622tjaudE6fPq2WlhaVl5fHrqWlpam8vFx79uyx9kiGoa9+9av67Gc/qwULFsSVBz0SAABY4NQ+Eh0dHcrKyopdv1BvxMc5fvy4IpGIcnNz+13Pzc3VW2+9Zek7/vM//1NbtmzRNddcE5t/8eMf/1iTJ1tfKUchgbjt6H7atVjj/v3fXIkzJuM9V+K88M61rsQBkHqysrL6FRLJdMMNNygajdr6DgoJAACsSLHln6NGjZLf71dXV1e/611dXcrLy3M22EUwRwIAAAuSsfzzYjIyMjRlyhQ1NDTErkWjUTU0NGjGjBnOBrsIeiQAAHBRaWmp/H6/QqGQQqHQRT/b09OjQ4cOxV63t7erra1NOTk5KigoUE1NjaqqqjR16lRNmzZNdXV16u3tja3icAOFBAAAVjg0tNHU1GR5jkRzc7NmzZoVe11TUyNJqqqq0qZNm3TbbbfpnXfe0erVq9XZ2amSkhLt2LHjvAmYiUQhAQCAFUmYI1FWVibDuPiNixYt0qJFi+LPyybmSAAAgLjRI4G4uXn6Z94L7nTTHem73JU477d/wpU4AJzj1D4SXkOPBAAAVji0s2VpaamKiooueH7GYEOPBAAAViRhsuVgQI8EAACIGz0SAABYwBwJcxQSAABYkWJbZKcKhjYAAEDcKCQAALDAqbM2WLUBnBWZfJVrsT6T1+ZKnLf/NtKVOP99X40rcQA4iFUbpuiRAAAAcaNHAgAAK5hsaYpCAgAAC3xnm537vSghQxtHjhzRV77yFY0cOVLDhw/X5MmT1dzcnIhQAAAgiRzvkXj33Xc1c+ZMzZo1Sy+//LL+7u/+TgcPHtTll7tzGBIAAAnB0IYpxwuJtWvXKj8/X88880zsWmFhodNhAABwlVM7W5aWlsrv9ysUCikUCjmWX7I4Xki8+OKLmj17tr70pS9p9+7dGjt2rL7xjW9o4cKFpp/v6+tTX19f7HU4HHY6JSTIu1cPdy1WyYg/uRLnhbaprsTRdHfCAHAQyz9NOT5H4o9//KPWr1+v8ePH65e//KXuvvtuLV68WD/84Q9NP19bW6vs7OxYy8/PdzolAACQII4XEtFoVNddd50eeeQRXXvttbrrrru0cOFCbdiwwfTzK1euVHd3d6x1dHQ4nRIAAM4wbDSPcnxoY8yYMSoqKup37dOf/rR++tOfmn4+EAgoEAg4nQYAAI7i9E9zjvdIzJw5U/v37+937cCBA7ryyiudDgUAAJLM8ULi3nvv1d69e/XII4/o0KFD2rx5szZu3OiJmakAgCHMzrCGh4c3HC8kSktLtXXrVj333HOaNGmSHnzwQdXV1Wn+/PlOhwIAwDVOnf7pNQnZIvsLX/iCvvCFLyTiqwEAQArhrA3E7fiNp12LdfTMZa7EGfXqJa7EUZU7YQA4yKF9JNiQCgCAIcipVRtsSAUAAHAWPRIAAFjBoV2mKCQAALCCQsIUhQQAABaws6U55kgAAIC40SOBuH2u6A+uxdrz7lWuxGn+QY0rcQAMQgxtmKKQAADAAp9hyGfEXw3YuTeVMbQBAADiRiEBAIAVDh3aVVpaqqKiItXX1yf7iRzB0AYAABaws6U5eiQAAEDc6JEAAMAKVm2YopDwoIpCd5YwLtu115U4knT3j7/uTqDr3QkDYPBhQypzDG0AAIC40SMBAIAVDG2YopAAAMAChjbMUUgAAGAFPRKmmCMBAADiRo8EAAAWeXV4wg4KCQAArDCMD5qd+z2IQsKDjlbmuxLnYF+eK3Ekaf+qe12LBQCwjjkSAABYcG7Vhp0mDu0CAGCIcmjVBod2AQAAnEWPBAAAFviiHzQ793sRhQQAAFawIZUphjYAAEDc6JHwoO6r3ek/qz9woytxJOlrV7sWCgBMcdaGOQoJAACsYEMqUxQSAABYQI+EOeZIAACAuNEjAQCAFazaMEUhAQCABQxtmGNoAwAAxI0eCQAArGDVhikKCQ8qLm53Jc5bu65yJY4kqdK9UABghqENcwxtAACAuNEjAQCAFazaMEUhAQCABQxtmEv40Ma3v/1t+Xw+LV26NNGhAACAyxJaSDQ1Nen73/++rrnmmkSGAQAg8aKG/SaptLRURUVFqq+vT/YTOSJhQxs9PT2aP3++nnrqKT300EOJCgMAgDscmiPR1NSkrKwsp7JKuoQVEqFQSHPmzFF5eTmFhKR/uPyfXIv109/vdCXOTesXuxIHAFKBz+Y8B5+TyaSQhBQSzz//vFpbW9XU1PSxn+3r61NfX1/sdTgcTkRKAAAgARyfI9HR0aElS5bo2Wef1bBhwz7287W1tcrOzo61/Px8p1MCAMC+cztb2mke5Hgh0dLSomPHjum6665Tenq60tPTtXv3bj3xxBNKT09XJBLp9/mVK1equ7s71jo6OpxOCQAA284t/7TTvMjxoY2bbrpJv/vd7/pdq66u1oQJE7R8+XL5/f5+7wUCAQUCAafTAAAALnC8kMjMzNSkSZP6XRsxYoRGjhx53nUAAAYNdrY0xc6WAABY4DMM+WzMc7BzbypzpZB45ZVX3AiT0nyXubdm+PP/FHIlzmWv/ZcrcQAAqYseCQAArIiebXbu9yAKCQAALGBow1zCD+0CAADeRY8EAABWsGrDFIUEAABW2N2d0qNDGxQSAABYYHd3Sq/ubMkcCQAAEDd6JFzSWeHeYWSXH+yz8Cn7dnQ/7UocAEgJDG2YopAAAMACX/SDZud+L2JoAwAAxI0eCQAArGBowxSFBAAAVrCPhCmGNgAAQNzokQAAwALO2jBHIeGSltXrXYs1O1jsWiwAGDKYI2GKoQ0AABA3CgkAAKwwJEVttBTskHjvvfc0depUlZSUaNKkSXrqqacG/B0MbQAAYIEX50hkZmaqsbFRl156qXp7ezVp0iTdfPPNGjlypOXvoJAAAMAKw+Y8h9SrI+T3+3XppZdKkvr6+mQYhowBPiNDGwAApKjGxkZVVlYqGAzK5/Np27Zt532mvr5e48aN07BhwzR9+nTt27dvQDHee+89FRcX64orrtCyZcs0atSoAd1PIQEAgBXnVm3YaQPU29ur4uJi1dfXm76/ZcsW1dTUaM2aNWptbVVxcbFmz56tY8eOxT5zbv7DR9vRo0clSZdddpneeOMNtbe3a/Pmzerq6hpQjgxtAABgRVSSz+b9ksLhcL/LgUBAgUDA9JaKigpVVFRc8CvXrVunhQsXqrq6WpK0YcMGvfTSS3r66ae1YsUKSVJbW5ul9HJzc1VcXKxXX31Vt9xyi9WnopBwS1reAddi7fToCXMA4AX5+fn9Xq9Zs0bf/OY3B/w9p0+fVktLi1auXBm7lpaWpvLycu3Zs8fSd3R1denSSy9VZmamuru71djYqLvvvntAeVBIAABggVOrNjo6OpSVlRW7fqHeiI9z/PhxRSIR5ebm9ruem5urt956y9J3HD58WHfddVdskuU999yjyZMnDygPCgkAAKxwaGfLrKysfoVEMk2bNs3y0MeFMNkSAIBBaNSoUfL7/edNjuzq6lJeXp5reVBIAABgRRJWbVxMRkaGpkyZooaGhti1aDSqhoYGzZgxw9FYF8PQBgAAVjg0tFFaWiq/369QKKRQKHTRW3p6enTo0KHY6/b2drW1tSknJ0cFBQWqqalRVVWVpk6dqmnTpqmurk69vb2xVRxuoJAAAMBFTU1NludINDc3a9asWbHXNTU1kqSqqipt2rRJt912m9555x2tXr1anZ2dKikp0Y4dO86bgJlIFBIAAFjh0D4SA1FWVvaxW1YvWrRIixYtij8vmygkAACwwIuHdjmBQgIAACscmiPhNazaAADARaWlpSoqKrrg+RmDDT0SAABYETUkn41ehegH9w5ksuVgQCEBAIAVDG2YYmgDAADEbcj3SFR8arkrcV4+sNaVOACARLG7O6U3eySGfCEBAIAlDG2YYmgDAAAXsWoDAIChKGrYG55g1QYAAEOYEf2g2bnfgxjaAAAAcXO8kKitrVVpaakyMzM1evRozZs3T/v373c6DAAA7jo32dJO8yDHC4ndu3crFApp79692rlzp86cOaPPfe5z6u3tdToUAADuiRr2mwc5Pkdix44d/V5v2rRJo0ePVktLiz7zmc84Hc62lxq3uhSJfSQAYFBj+aephM+R6O7uliTl5OSYvt/X16dwONyvAQDgVSz/HIBoNKqlS5dq5syZmjRpkulnamtr9cADDyQyDQAA7DNs9iqcvdVryz8T2iMRCoX05ptv6vnnn7/gZ1auXKnu7u5Y6+joSGRKAADEh8mWphLWI7Fo0SJt375djY2NuuKKKy74uUAgoEAgkKg0AABAAjleSBiGoXvuuUdbt27VK6+8osLCQqdDAADgvmhUko1NpaLe3JDK8UIiFApp8+bN+vnPf67MzEx1dnZKkrKzszV8+HCnwwEA4A5WbZhyvJBYv369JKmsrKzf9WeeeUZf/epXnQ5nW1regWSnAADAoJWQoQ0AADyHHglTHNoFAIAVDp3+6TUc2gUAgIvYkAoAgCHIMKIybBwFfu5er21IRSEBAIAVhs2Dt5gjAQDAEGbYnCPh0UKCORIAACBu9EgAAGBFNCr5bOxOaWN+RSqjkAAAwAqGNkwxtAEAAOJGjwQAABYY0agMG0MbdpaOpjIKCQAArGBowxRDGwAAuIidLQEAGIqihuSz3yPBzpYuiXZdq+hJf8LjcIw4AMASw5BkZ/knQxsAAAD9pGyPBAAAqcSIGjJsDG0YHu2RoJAAAMAKI2pzaIPlnwAADFn0SJhjjgQAAIhbyvVInKvYwj3udAGlXRp2JQ4AwHnh8Ad/h7vx2/77Rp+t4Yn3dcbRfFJFyhUSJ06ckCRded3bLkXMdikOACBRTpw4oezsxPx9npGRoby8PL3W+f9sf1deXp4yMjIcyStV+IwUG7SJRqM6evSoMjMz5fP5XIkZDoeVn5+vjo4OT20Sco7Xn088oyd4/fnEMyaEYRg6ceKEgsGg0tISN1p/6tQpnT592vb3ZGRkaNiwYY7klCpSrkciLS1NV1xxRVJiZ2VlefZ/bg2B5xPP6Alefz7xjI5LVE/Ehw0bNsxzBYBTmGwJAADiRiEBAADiRiEhKRAIaM2aNQoEAslOJSG8/nziGT3B688nnhEelXKTLQEAwOBBjwQAAIgbhQQAAIgbhQQAAIgbhQQAAIjbkC0kamtrVVpaqszMTI0ePVrz5s3T/v37k51WQn3729+Wz+fT0qVLk52Ko44cOaKvfOUrGjlypIYPH67Jkyerubk52Wk5IhKJaNWqVSosLNTw4cN11VVX6cEHHxzUpwg2NjaqsrJSwWBQPp9P27Zt6/e+YRhavXq1xowZo+HDh6u8vFwHDx5MWr7xuNgznjlzRsuXL9fkyZM1YsQIBYNB3XHHHTp69GhScx6oj/tz/LCvf/3r8vl8qqurczVHuGPIFhK7d+9WKBTS3r17tXPnTp05c0af+9zn1Nvbm+zUEqKpqUnf//73dc011yQ7FUe9++67mjlzpi655BK9/PLL+v3vf6/vfOc7uvzyy5OdmiPWrl2r9evX63vf+57+8Ic/aO3atXr00Uf15JNPJju1uPX29qq4uFj19fWm7z/66KN64okntGHDBr3++usaMWKEZs+erVOnTrmea7wu9ownT55Ua2urVq1apdbWVv3sZz/T/v37NXfu3KTkGq+P+3M8Z+vWrdq7d6+CwaBrucFlBgzDMIxjx44Zkozdu3cnOxXHnThxwhg/fryxc+dO48YbbzSWLFmS7JQcs3z5cuOGG25IdhoJM2fOHOPOO+/sd+3mm2825s+fn7ScnCTJ2Lp1a+x1NBo18vLyjMceeyx27b333jMCgYDx3HPPJSlLez76jGb27dtnSDIOHz7sWl5OutAz/vnPfzbGjh1rvPnmm8aVV15pfPe7301KfkisIdsj8VHd3d2SpJycnGSn4rhQKKQ5c+aovLw82ak47sUXX9TUqVP1pS99SaNHj9a1116rp556KtlpOeb6669XQ0ODDhw4IEl644039Nprr6mioiLZqSVEe3u7Ojs7+/23mp2drenTp2vPnj1JzS2Ruru75fP5dNlllyU7FcdEo1EtWLBAy5Yt08SJE5OdDhIo5Q7tSoZoNKqlS5dq5syZmjRpUrLTcdTzzz+v1tZWNTU1JTuVhPjjH/+o9evXq6amRv/8z/+spqYmLV68WBkZGaqqqkp2eratWLFC4XBYEyZMkN/vVyQS0cMPP6z58+cnO7WE6OzslCTl5ub2u56bmxt7z2tOnTql5cuX68tf/rKnDvJau3at0tPTtXjx4mSnggSjkDj7G/ubb76p1157LdmpOKqjo0NLlizRzp07PXtqXTQa1dSpU/XII49Ikq699lq9+eab2rBhgycKiRdeeEHPPvusNm/erIkTJ6qtrU1Lly5VMBj0xPMNdWfOnNGtt94qwzC0fv36ZKfjmJaWFj3++ONqbW2Vz+dLdjpIsCE/tLFo0SJt375du3btStrx5YnS0tKiY8eO6brrrlN6errS09O1e/duPfHEE0pPT1ckEkl2iraNGTNGRUVF/a59+tOf1p/+9Kek5eSkZcuWacWKFbr99ts1efJkLViwQPfee69qa2uTnVpC5OXlSZK6urr6Xe/q6oq95xXniojDhw9r586dnuqNePXVV3Xs2DEVFBTE/u45fPiw7rvvPo0bNy7Z6cFhQ7ZHwjAM3XPPPdq6dateeeUVFRYWJjslx91000363e9+1+9adXW1JkyYoOXLl8vv9yctN6fMnDnzvGW7Bw4c0JVXXpm0nJx08uRJpaX1r/f9fr+i0WjSckqkwsJC5eXlqaGhQSUlJZKkcDis119/XXfffXey03PMuSLi4MGD2rVrl0aOHJnslBy1YMGC8+ZkzZ49WwsWLFB1dXXS8kJiDNlCIhQKafPmzfr5z3+uzMzM2Phrdna2hg8fnuz0HJGZmXnenI8RI0Zo5MiRnpkLcu+99+r666/XI488oltvvVX79u3Txo0btXHjxmSn5ojKyko9/PDDKigo0MSJE/Xb3/5W69at05133pns1OLW09OjQ4cOxV63t7erra1NOTk5Kigo0NKlS/XQQw9p/PjxKiws1KpVqxQMBjVv3ryk5j0QF3vGMWPG6JZbblFra6u2b9+uSCQS+/snJydHGRkZSczcuo/7c/xocXTJJZcoLy9PV199dRKyRUIle9lIskgybc8880yyU0sory3/NAzD+MUvfmFMmjTJCAQCxoQJE4yNGzcmOyXHhMNhY8mSJUZBQYExbNgw4+///u+Nf/mXfzH6+vqSnVrcdu3aZfr/XlVVlWGcXQK6atUqIzc31wgEAsZNN91k7N+/P9lpD8jFnrG9vf2Cf//s2rUr2alb9nF/jh/F8k/v4hhxAAAQtyE/2RIAAMSPQgIAAMSNQgIAAMSNQgIAAMSNQgIAAMSNQgIAAMSNQgIAAMSNQgIAAMSNQgIAAMSNQgIAAMSNQgIAAMSNQgIAAMTt/wMA0Egu4flACAAAAABJRU5ErkJggg==", + "text/plain": [ + "<Figure size 640x480 with 2 Axes>" + ] + }, + "metadata": {}, + "output_type": "display_data" } ], "source": [ - "batch_src, batch_labels, batch_padding_mask = map(lambda x: x.to(device), mkbatch(BSZ))\n", - "model.eval()\n", - "with torch.no_grad():\n", - " output = model(batch_src, batch_padding_mask)\n", - "x = batch_labels.detach().to(torch.uint8)\n", - "y = output.detach()\n", - "cnts = torch.bincount(x)\n", - "weights = [1/cnts[i.item()].item() for i in x] # normalize by label count\n", - "fig, ax = plt.subplots()\n", - "h = ax.hist2d(x.cpu().numpy().flatten(), y.to(torch.float16).cpu().numpy().flatten(), weights=weights, bins=[15,50], norm=mpl.colors.LogNorm())\n", - "fig.colorbar(h[3], ax=ax)\n", - "plt.show()" + "mkhist(mkbatch(BSZ))\n", + "plt.show()\n", + "with open(\"plots/train-hist.html\", \"w\") as f:\n", + " mpld3.save_html(plt.gcf(), f)" ] }, { "cell_type": "code", - "execution_count": 10, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/home/sipb/.venv/lib64/python3.12/site-packages/torch/nn/modules/transformer.py:871: UserWarning: Memory Efficient attention on Navi31 GPU is still experimental. Enable it with TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1. (Triggered internally at ../aten/src/ATen/native/transformers/hip/sdp_utils.cpp:269.)\n", - " return torch._transformer_encoder_layer_fwd(\n", - "/home/sipb/.venv/lib64/python3.12/site-packages/torch/nn/modules/transformer.py:871: UserWarning: Attempting to use hipBLASLt on an unsupported architecture! Overriding blas backend to hipblas (Triggered internally at ../aten/src/ATen/Context.cpp:296.)\n", - " return torch._transformer_encoder_layer_fwd(\n", - "/home/sipb/.venv/lib64/python3.12/site-packages/torch/_inductor/compile_fx.py:167: UserWarning: TensorFloat32 tensor cores for float32 matrix multiplication available but not enabled. Consider setting `torch.set_float32_matmul_precision('high')` for better performance.\n", - " warnings.warn(\n" - ] - }, - { - "data": { - "text/plain": [ - "0.0005554668023250997" - ] - }, - "execution_count": 10, - "metadata": {}, - "output_type": "execute_result" - } - ], + "outputs": [], "source": [ - "def evaluate():\n", - " model.eval()\n", - " test_loss = 0\n", - " with torch.no_grad():\n", - " batch_src, batch_labels, batch_padding_mask = map(lambda x: x.to(device), mkbatch(BSZ))\n", - " output = model(batch_src, batch_padding_mask)\n", - " loss = criterion(output.squeeze(1), batch_labels)\n", - " return loss.item()\n", - "\n", - "evaluate()" + "plt.suptitle('Log MSE vs Epochs')\n", + "plt.plot(torch.log(torch.tensor(losses[:, 0])), label='Train', color='blue')\n", + "plt.plot(torch.log(torch.tensor(losses[:, 1])), label='Len 1', color='red')\n", + "plt.plot(torch.log(torch.tensor(losses[:, 2])), label='Len 2', color='green')\n", + "plt.plot(torch.log(torch.tensor(losses[:, 3])), label='Len 3', color='yellow')\n", + "plt.plot(torch.log(torch.tensor(losses[:, 4])), label='Len 15', color='magenta')\n", + "plt.xlabel('Epochs')\n", + "plt.ylabel('Log MSE')\n", + "plt.legend()\n", + "with open(\"plots/train-lens.html\", \"w\") as f:\n", + " mpld3.save_html(plt.gcf(), f)" ] }, { @@ -1331,10 +3544,9 @@ "source": [ "N_TUNE_EPOCHS = 100\n", "TUNE_LR = 1e-5\n", - "TUNE_WD = 0 # 1e-5\n", "\n", "tune_criterion = nn.MSELoss()\n", - "tune_optimizer = torch.optim.Adam(model.parameters(), lr=TUNE_LR, weight_decay=TUNE_WD)\n", + "tune_optimizer = torch.optim.Adam(model.parameters(), lr=TUNE_LR)\n", "\n", "tune_train_err = []\n", "\n", @@ -1458,7 +3670,7 @@ "for epoch in range(N_TUNE_EPOCHS):\n", " model.train()\n", " train_loss = 0\n", - " batch_src, batch_labels, batch_padding_mask = map(lambda x: x.to(device), mktunebatch(BSZ))\n", + " batch_src, batch_labels, batch_padding_mask = map(lambda x: x.to(device), mkbatch(BSZ, large=False, target=\"onpath\"))\n", " optimizer.zero_grad()\n", " output = model(batch_src, batch_padding_mask)\n", " loss = criterion(output.squeeze(1), batch_labels)\n", @@ -1477,55 +3689,48 @@ }, { "cell_type": "code", - "execution_count": 17, + "execution_count": 19, "metadata": {}, "outputs": [ { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAHgCAYAAABZ+0ykAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABCC0lEQVR4nO3deXRU9f3/8dckYZJAFpaQQCCsoih7g0BABTTIJlaLgpRKxBVQxPLTKlpB7ZeCuBQFBaXiziJWqVLBYsQFyhoMoCKuLIoJIJCELYHM5/fHZSaMAUxgZm5y83ycc8/M3Ll35j23al79bNdljDECAABwiDC7CwAAAAgkwg0AAHAUwg0AAHAUwg0AAHAUwg0AAHAUwg0AAHAUwg0AAHAUwg0AAHAUwg0AAHAUwg0ABJHL5dIdd9xhdxlAlUK4ASqxl156SS6XSy6XS8uXLy/1vjFGKSkpcrlcuuKKK/zeO3DggCZMmKDWrVurRo0aqlOnjtq3b68xY8Zo586dvuMeeugh33ecbMvJyQnJbz2V09U2YsQIW2sDYI8IuwsAcPaioqI0Z84cXXTRRX77P/74Y/3444+KjIz023/06FFdcskl+uqrr5SRkaHRo0frwIED+uKLLzRnzhxdffXVSk5O9jtnxowZiomJKfXdNWvWDNKvKrtevXpp2LBhpfafe+65ttQDwF6EG8AB+vXrpwULFujpp59WRETJv9Zz5sxRamqq9uzZ43f8woUL9dlnn+n111/XH//4R7/3jhw5oqKiolLfcc011yghISGIv+LMnXvuufrTn/5kdxkAKgi6pQAHGDJkiH755RctXbrUt6+oqEhvvvlmqfAiSd99950kqVu3bqXei4qKUlxcXEDqat26tXr27Flqv8fjUYMGDXTNNdf49s2bN0+pqamKjY1VXFyc2rRpo6eeeiogdUhSjx491Lp1a2VlZalr166Kjo5W06ZNNXPmzFLH7tq1SzfddJOSkpIUFRWldu3a6eWXXz7p73jqqafUpk0bRUVFqW7duurTp4/WrVtX6tiFCxeqdevWioyMVKtWrbRkyRK/9wsKCnTXXXepSZMmioyMVGJionr16qX169cH7BoAVQXhBnCAJk2aKC0tTXPnzvXtW7x4sfLy8nTdddeVOr5x48aSpFdeeUXGmDJ9x969e7Vnzx6/bf/+/ac9Z/Dgwfrkk09KjctZvny5du7c6att6dKlGjJkiGrVqqVHH31UkydPVo8ePbRixYoy1XbkyJFSte3Zs6dUC9S+ffvUr18/paamasqUKWrYsKFGjhyp2bNn+445fPiwevTooVdffVVDhw7VY489pvj4eN1www2lwtZNN92ku+66SykpKXr00Ud13333KSoqSqtWrSr1e0eNGqXrrrtOU6ZM0ZEjRzRw4ED98ssvvmNGjBihGTNmaODAgXr22Wd19913Kzo6Wps3by7TNQBwAgOg0nrxxReNJLN27Vozffp0Exsbaw4dOmSMMebaa681PXv2NMYY07hxY9O/f3/feYcOHTLnnXeekWQaN25sbrjhBvPCCy+Y3NzcUt8xYcIEI+mk23nnnXfa+rZs2WIkmWnTpvntHzVqlImJifHVOmbMGBMXF2eOHTtW7mtwqtokmblz5/qO6969u5FknnjiCd++wsJC0759e5OYmGiKioqMMcZMnTrVSDKvvfaa77iioiKTlpZmYmJiTH5+vjHGmA8//NBIMnfeeWepmjwej199brfbfPvtt759GzZsKHVd4uPjze23317u3w+gNFpuAIcYNGiQDh8+rEWLFqmgoECLFi06aZeUJEVHR2v16tW65557pOOzrm666SbVr19fo0ePVmFhYalz/vWvf2np0qV+24svvnjams4991y1b99e8+fP9+0rLi7Wm2++qQEDBig6Olo6Pij54MGDft1q5fH73/++VG1Lly4t1SUWERGh2267zffa7Xbrtttu065du5SVlSVJeu+991SvXj0NGTLEd1y1atV055136sCBA/r4449918PlcmnChAml6nG5XH6v09PT1bx5c9/rtm3bKi4uTt9//71vX82aNbV69Wq/mWoAzgwDigGHqFu3rtLT0zVnzhwdOnRIxcXFfmNafi0+Pl5TpkzRlClTtG3bNmVmZurxxx/X9OnTFR8fr//7v//zO/6SSy45owHFgwcP1v3336+ffvpJDRo00EcffaRdu3Zp8ODBvmNGjRqlN954Q3379lWDBg10+eWXa9CgQerTp0+ZvqNhw4ZKT0//zeOSk5NVo0YNv33eGVVbt25Vly5dtG3bNrVo0UJhYf7/3+/888+XJG3btk06Pm4pOTlZtWvX/s3vbdSoUal9tWrV0r59+3yvp0yZooyMDKWkpCg1NVX9+vXTsGHD1KxZs9/8fAD+aLkBHOSPf/yjFi9erJkzZ6pv375lnqbduHFj3XjjjVqxYoVq1qyp119/PWA1DR48WMYYLViwQJL0xhtvKD4+3i+4JCYmKjs7W++8846uvPJKLVu2TH379lVGRkbA6rBTeHj4SfefON5p0KBB+v777zVt2jQlJyfrscceU6tWrbR48eIQVgo4A+EGcJCrr75aYWFhWrVq1Sm7pE6nVq1aat68uX7++eeA1dS0aVN16tRJ8+fP17Fjx/TWW2/pqquuKrX2jtvt1oABA/Tss8/qu+++02233aZXXnlF3377bcBq2blzpw4ePOi37+uvv5aOD8rW8aD3zTffyOPx+B331Vdf+d6XpObNm2vnzp3au3dvwOqrX7++Ro0apYULF+qHH35QnTp1NHHixIB9PlBVEG4AB4mJidGMGTP00EMPacCAAac8bsOGDaXWvtHxLpcvv/xS5513XkDrGjx4sFatWqXZs2drz549fl1SkvxmDUlSWFiY2rZtK0knHf9zpo4dO6bnnnvO97qoqEjPPfec6tatq9TUVOn4mkE5OTl+44SOHTumadOmKSYmRt27d5ckDRw4UMYYPfzww6W+p6wz0LyKi4uVl5fnty8xMVHJyckB/f1AVcGYG8BhytKVs3TpUk2YMEFXXnmlunTpopiYGH3//feaPXu2CgsL9dBDD5U658033zzpCsW9evVSUlLSab9v0KBBuvvuu3X33Xerdu3apcbH3Hzzzdq7d68uvfRSNWzYUNu2bdO0adPUvn1731iX0/n666/12muvldqflJSkXr16+V4nJyfr0Ucf1datW3Xuuedq/vz5ys7O1vPPP69q1apJkm699VY999xzuuGGG5SVlaUmTZrozTff1IoVKzR16lTFxsZKknr27Knrr79eTz/9tL755hv16dNHHo9Hn376qXr27Fmu+0kVFBSoYcOGuuaaa9SuXTvFxMTogw8+0Nq1a/XEE0+U+XMAHGf3dC0AZ+7EqeCn8+up4N9//70ZP3686dKli0lMTDQRERGmbt26pn///ubDDz/0O/d0U8ElmWXLlpWp1m7duhlJ5uabby713ptvvmkuv/xyk5iYaNxut2nUqJG57bbbzM8///ybn3u62rp37+47rnv37qZVq1Zm3bp1Ji0tzURFRZnGjRub6dOnl/rM3NxcM3z4cJOQkGDcbrdp06aNefHFF0sdd+zYMfPYY4+Zli1bGrfbberWrWv69u1rsrKy/Oo72RTvxo0bm4yMDGOOT0m/5557TLt27UxsbKypUaOGadeunXn22Wd/8/cDKM1lytt+CgCVUI8ePbRnzx59/vnndpcCIMgYcwMAAByFcAMAAByFcAMAAByFMTcAAMBRaLkBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOEmF3AaHm8Xi0c+dOxcbGyuVy2V0OAAAoA2OMCgoKlJycrLCw07fNVLlws3PnTqWkpNhdBgAAOAM7duxQw4YNT3tMlQs3sbGx0vGLExcXZ3c5AACgDPLz85WSkuL7O346VS7ceLui4uLiCDcAAFQyZRlSwoBiAADgKIQbAADgKIQbAADgKIQbAADgKIQbAADgKIQbAADgKIQbAADgKIQbAADgKIQbAADgKIQbAADgKIQbAADgKIQbAADgKFXuxpnBUlQk5eZKHo/UuLHd1QAAUHXRchMga9ZIjRpJvXrZXQkAAFUb4SZAoqKsxyNH7K4EAICqjXATIN5wU1hodyUAAFRthJsAoeUGAICKgXATIJGR1iPhBgAAexFuAsTbclNUZM2YAgAA9iDcBIg33IhxNwAA2IpwEyAnhhu6pgAAsA/hJkAiIqSw41eTlhsAAOxDuAkQl4sZUwAAVASEmwBixhQAAPYj3AQQLTcAANiPcBNAhBsAAOxHuAkgwg0AAPYj3AQQ95cCAMB+hJsAouUGAAD7EW4CiNlSAADYj3ATQLTcAABgP8JNABFuAACwH+EmgAg3AADYj3ATQIQbAADsR7gJIKaCAwBgP8JNANFyAwCA/Qg3AcRUcAAA7Ee4CSBabgAAsB/hJoAINwAA2I9wE0CEGwAA7Ee4CSBmSwEAYD/CTQDRcgMAgP0INwHEbCkAAOxHuAkgWm4AALAf4SaACDcAANiPcBNAhBsAAOxHuAkgZksBAGA/wk0A0XIDAID9CDcBxGwpAADsR7gJIFpuAACwH+EmgAg3AADYr0KEm2eeeUZNmjRRVFSUOnfurDVr1pTpvHnz5snlcumqq64Keo1l4Q03xcXSsWN2VwMAQNVke7iZP3++xo4dqwkTJmj9+vVq166devfurV27dp32vK1bt+ruu+/WxRdfHLJaf4s33IjWGwAAbGN7uHnyySd1yy23aPjw4brgggs0c+ZMVa9eXbNnzz7lOcXFxRo6dKgefvhhNWvWLKT1no53QLGYDg4AgG1sDTdFRUXKyspSenp6SUFhYUpPT9fKlStPed4jjzyixMRE3XTTTb/5HYWFhcrPz/fbgiU8XIqIsJ7TcgMAgD1sDTd79uxRcXGxkpKS/PYnJSUpJyfnpOcsX75cL7zwgmbNmlWm75g0aZLi4+N9W0pKSkBqPxUGFQMAYC/bu6XKo6CgQNdff71mzZqlhISEMp0zbtw45eXl+bYdO3YEtUbCDQAA9oqw88sTEhIUHh6u3Nxcv/25ubmqV69eqeO/++47bd26VQMGDPDt83g8kqSIiAht2bJFzZs39zsnMjJSkScOhgkywg0AAPayteXG7XYrNTVVmZmZvn0ej0eZmZlKS0srdXzLli21adMmZWdn+7Yrr7xSPXv2VHZ2dtC7nMqCcAMAgL1sbbmRpLFjxyojI0MdO3ZUp06dNHXqVB08eFDDhw+XJA0bNkwNGjTQpEmTFBUVpdatW/udX7NmTUkqtd8u3DwTAAB72R5uBg8erN27d2v8+PHKyclR+/bttWTJEt8g4+3btyssrPIMDaLlBgAAe7mMMcbuIkIpPz9f8fHxysvLU1xcXMA//6KLpBUrpH/9S/rDHwL+8QAAVEnl+ftdeZpEKglabgAAsBfhJsAINwAA2ItwE2CEGwAA7EW4CTBmSwEAYC/CTYDRcgMAgL0INwHmXQyZcAMAgD0INwFGyw0AAPYi3AQY4QYAAHsRbgKMcAMAgL0INwFGuAEAwF6EmwBjKjgAAPYi3AQYs6UAALAX4SbA6JYCAMBehJsAI9wAAGAvwk2AEW4AALAX4SbACDcAANiLcBNgzJYCAMBehJsAY7YUAAD2ItwEGN1SAADYi3ATYIQbAADsRbgJMMINAAD2ItwE2Inhxhi7qwEAoOoh3ASYN9xI0tGjdlYCAEDVRLgJsBPDDV1TAACEHuEmwNzukueEGwAAQo9wE2AuF2vdAABgJ8JNEDBjCgAA+xBugoBwAwCAfQg3QUC4AQDAPoSbIODmmQAA2IdwEwQMKAYAwD6EmyCgWwoAAPsQboKAcAMAgH0IN0FAuAEAwD6EmyAg3AAAYB/CTRAwWwoAAPsQboKA2VIAANiHcBMEdEsBAGAfwk0QEG4AALAP4SYICDcAANiHcBMEhBsAAOxDuAkCZksBAGAfwk0Q0HIDAIB9CDdBwFRwAADsQ7gJAlpuAACwD+EmCAg3AADYh3ATBIQbAADsQ7gJAsINAAD2IdwEAVPBAQCwD+EmCJgtBQCAfQg3QUC3FAAA9iHcBAHhBgAA+xBugoBwAwCAfQg3QUC4AQDAPoSbIPCGm6NHJY/H7moAAKhaCDdB4J0tJaaDAwAQcoSbIPC23IiuKQAAQo5wEwQREVLY8StLuAEAILQIN0HgcjGoGAAAuxBugoRwAwCAPQg3QUK4AQDAHoSbIPHOmGK2FAAAoUW4CRJabgAAsEeFCDfPPPOMmjRpoqioKHXu3Flr1qw55bFvvfWWOnbsqJo1a6pGjRpq3769Xn311ZDWWxaEGwAA7GF7uJk/f77Gjh2rCRMmaP369WrXrp169+6tXbt2nfT42rVr64EHHtDKlSu1ceNGDR8+XMOHD9f7778f8tpPh3ADAIA9bA83Tz75pG655RYNHz5cF1xwgWbOnKnq1atr9uzZJz2+R48euvrqq3X++eerefPmGjNmjNq2bavly5eHvPbTIdwAAGAPW8NNUVGRsrKylJ6eXlJQWJjS09O1cuXK3zzfGKPMzExt2bJFl1xyyUmPKSwsVH5+vt8WCoQbAADsYWu42bNnj4qLi5WUlOS3PykpSTk5Oac8Ly8vTzExMXK73erfv7+mTZumXr16nfTYSZMmKT4+3relpKQE/HecjDfcMFsKAIDQsr1b6kzExsYqOztba9eu1cSJEzV27Fh99NFHJz123LhxysvL8207duwISY3eqeC03AAAEFoRdn55QkKCwsPDlZub67c/NzdX9erVO+V5YWFhOueccyRJ7du31+bNmzVp0iT16NGj1LGRkZGKPPE23SFCtxQAAPawteXG7XYrNTVVmZmZvn0ej0eZmZlKS0sr8+d4PB4VVrD+H8INAAD2sLXlRpLGjh2rjIwMdezYUZ06ddLUqVN18OBBDR8+XJI0bNgwNWjQQJMmTZKOj6Hp2LGjmjdvrsLCQr333nt69dVXNWPGDJt/iT/CDQAA9rA93AwePFi7d+/W+PHjlZOTo/bt22vJkiW+Qcbbt29XWFhJA9PBgwc1atQo/fjjj4qOjlbLli312muvafDgwTb+itIINwAA2MNljDF2FxFK+fn5io+PV15enuLi4oL2PQ8/LD30kDRypPTss0H7GgAAqoTy/P2ulLOlKgNmSwEAYA/CTZDQLQUAgD0IN0FCuAEAwB6EmyAh3AAAYA/CTZAQbgAAsAfhJkgINwAA2INwEyTe2VIVbOFkAAAcj3ATJLTcAABgD8JNkBBuAACwB+EmSAg3AADYg3ATJIQbAADsQbgJEsINAAD2INwECbOlAACwB+EmSLwtN8XF0rFjdlcDAEDVQbgJEm+4EV1TAACEFOEmSLzdUiLcAAAQUoSbIAkPl6pVs54TbgAACB3CTRAxYwoAgNAj3ASRN9wwYwoAgNAh3ASRd9wNLTcAAIQO4SaI6JYCACD0CDdBRLgBACD0CDdBRLgBACD0CDdBRLgBACD0CDdBRLgBACD0CDdBxM0zAQAIPcJNENWoYT3u2mV3JQAAVB2EmyC65BLr8a237K4EAICqg3ATRNdea91jau1a6dtv7a4GAICqgXATRImJ0mWXWc/nzrW7GgAAqgbCTZD98Y/W49y5kjF2VwMAgPMRboLs6qutWVObN0sbN9pdDQAAzke4CbK4OKl/f+v5nDl2VwMAgPMRbkJgyBDrcd48yeOxuxoAAJyNcBMC/ftLsbHS9u3SypV2VwMAgLMRbkIgOtoaeyO6pgAACLpyhZspU6bo8OHDvtcrVqxQ4Qn3FigoKNCoUaMCW6FDeLumFiyQjh2zuxoAAJzLZUzZJyiHh4fr559/VmJioiQpLi5O2dnZatasmSQpNzdXycnJKi4uDl7FZyk/P1/x8fHKy8tTXFxcyL736FGpQQNp925pyRKpd++QfTUAAJVeef5+l6vl5tc5qBy5qMqrVs1asVgs6AcAQFAx5iaEvF1Tb79tteQAAIDAI9yEUNeuUp06Un6+tGaN3dUAAOBMEeU94Z///KdiYmIkSceOHdNLL72khIQE6fiAYpxaWJh1r6k33pCWLpW6dbO7IgAAnKdcA4qbNGkil8v1m8f98MMPZ1tX0Ng1oNjrn/+UbrnFCjbLl4f86wEAqJTK8/e7XC03W7duPdvaqrz0dOtx1Sqre8qGfAUAgKMx5ibEmjSRmjeXiouljz+2uxoAAJynXOFm5cqVWrRokd++V155RU2bNlViYqJuvfVWv0X9cHK9elmPH3xgdyUAADhPucLNI488oi+++ML3etOmTbrpppuUnp6u++67T++++64mTZoUjDodxds1tXSp3ZUAAOA85Qo32dnZuuyyy3yv582bp86dO2vWrFkaO3asnn76ab3xxhvBqNNRLr1UcrmkzZuln36yuxoAAJylXOFm3759SkpK8r3++OOP1bdvX9/rCy+8UDt27AhshQ5Uq5bUsaP1nK4pAAACq1zhJikpyTfNu6ioSOvXr1eXLl187xcUFKhatWqBr9KBGHcDAEBwlCvc9OvXT/fdd58+/fRTjRs3TtWrV9fFF1/se3/jxo1q3rx5MOp0HO+4mw8+kLhFFwAAgVOucPO3v/1NERER6t69u2bNmqXnn39ebrfb9/7s2bN1+eWXB6NOx+naVYqOlnJypBPGaAMAgLNUrkX8EhIS9MknnygvL08xMTEKDw/3e3/BggWKjY0NdI2OFBkpXXKJ9P77VutN69Z2VwQAgDOUK9zceOONZTpu9uzZZ1pPlZKeboWbpUulu+6yuxoAAJyhXOHmpZdeUuPGjdWhQweV45ZUOAXvoOKPP5aKiqQTevgAAMAZKle4GTlypObOnasffvhBw4cP15/+9CfVrl07eNU5XJs2Ut260u7d0urV0gljswEAwBkq14DiZ555Rj///LP+8pe/6N1331VKSooGDRqk999/n5acMxAWJnnXRGS1YgAAAqPcN86MjIzUkCFDtHTpUn355Zdq1aqVRo0apSZNmujAgQPBqdLBvOFm2TK7KwEAwBnO6q7gYWFhcrlcMsaouLg4cFVVIZdeaj2uWiUdPGh3NQAAVH7lDjeFhYWaO3euevXqpXPPPVebNm3S9OnTtX37dsXExASnSgdr2lRq3Fg6dkxavtzuagAAqPzKFW5GjRql+vXra/Lkybriiiu0Y8cOLViwQP369VNY2Fk1AlVZLpfUs6f1nK4pAADOnsuUYyRwWFiYGjVqpA4dOsjlcp3yuLfeeitQ9QVcfn6+4uPjlZeXp7i4OLvLkSS9+qo0bJh04YXSmjV2VwMAQMVTnr/f5ZoKPmzYsNOGGpwZb8tNVpaUlyfFx9tdEQAAlVe5F/ELhmeeeUaPPfaYcnJy1K5dO02bNk2dOnU66bGzZs3SK6+8os8//1ySlJqaqr///e+nPL4yaNhQatFC+uYb6ZNPpAED7K4IAIDKy/aBMvPnz9fYsWM1YcIErV+/Xu3atVPv3r21a9eukx7/0UcfaciQIVq2bJlWrlyplJQUXX755frpp59CXnsgeWdNffih3ZUAAFC5lWvMTTB07txZF154oaZPny5J8ng8SklJ0ejRo3Xffff95vnFxcWqVauWpk+frmHDhv3m8RVxzI0kzZ8vXXed1K6dlJ1tdzUAAFQs5fn7bWvLTVFRkbKyspSenl5SUFiY0tPTtXLlyjJ9xqFDh3T06NFT3gaisLBQ+fn5fltF1KOH9bhhg7Rnj93VAABQedkabvbs2aPi4mIlJSX57U9KSlJOTk6ZPuPee+9VcnKyX0A60aRJkxQfH+/bUlJSAlJ7oCUlSa1bW88/+sjuagAAqLxsH3NzNiZPnqx58+bp7bffVlRU1EmPGTdunPLy8nzbjh07Ql5nWbHeDQAAZ8/WcJOQkKDw8HDl5ub67c/NzVW9evVOe+7jjz+uyZMn67///a/atm17yuMiIyMVFxfnt1VU5RlU/NNP0rZtQS8JAIBKx9Zw43a7lZqaqszMTN8+j8ejzMxMpaWlnfK8KVOm6G9/+5uWLFmijh07hqja4Ove3Vqx+KuvpJ07/d8rLJQyM6V77pHatLGmj597rrRunV3VAgBQMdneLTV27FjNmjVLL7/8sjZv3qyRI0fq4MGDGj58uHR84cBx48b5jn/00Uf14IMPavbs2WrSpIlycnKUk5PjiDuS16oldehgPfeOu9m3T7r/fikhQUpPlx5/XDq+xI+KiqSRIyXuWQoAQIlyLeIXDIMHD9bu3bs1fvx45eTkqH379lqyZIlvkPH27dv97ls1Y8YMFRUV6ZprrvH7nAkTJuihhx4Kef2Bduml0vr10qJF0tat0pQp1qrFklSvntSnj7W1aSOlpVktN7NmSSNG2F05AAAVg+3r3IRaRV3nxmvxYqlfP/99rVtLEydaKxefePeLadOkO++UataUtmyREhNDXi4AACFRada5QWkXXSRFR1vPmzWTXnvNWtTvyiv9g41kdUl16CDt3y/de68t5QIAUOEQbiqY2Fjp/felefOkzZuloUOl8PCTHxsRIc2YYYWel16Sli8PdbUAAFQ8hJsK6OKLpcGDJbf7t4/t3Fm65Rbr+ciR0tGjQS8PAIAKjXDjAH//u1SnjjWL6vgtugAAqLIINw5Qp44VcCSrmwoAgKqMcOMQQ4ZI1apJ33wjff213dUAAGAfwo1DxMaW3Fn8P/+xuxoAAOxDuHGQ/v2tx0WL7K4EAAD7EG4c5IorrMdPPpHy8+2uBgAAexBuHKR5c+m886Rjx6T//tfuagAAsAfhxmG8rTeMuwEAVFWEG4fxjrt57z3J47G7GgAAQo9w4zAXXSTFxUm7dll3DAcAoKoh3DhMtWpS797Wc2ZNAQCqIsKNAzHuBgBQlRFuHKhvX+tO4evXSzt32l0NAAChRbhxoLp1rbuF6/jAYgAAqhLCjUOxWjEAoKoi3DiUd9zNBx9IR47YXQ0AAKFDuHGodu2kBg2kgwdZrRgAULUQbhzK5ZKuu856/txzdlcDAEDoEG4cbMQI63HxYun77+2uBgCA0CDcONg551gL+hkjzZxpdzUAAIQG4cbhbr/denzhBenwYburAQAg+Ag3Dtevn9SokbR3rzR/vt3VAAAQfIQbhwsPLxl78+yzdlcDAEDwEW6qgJtuktxuae1aayuP/fulKVOkyy+XPv00WBUCABA4hJsqIDFRuvZa63lZW2+2bZP+/GcpJUW6915p6VLrnlWrVgW1VAAAzhrhporwDiyeN0/65ZdTH3fggDR8uNS8uTR1qvW6VSspLc1aELBvXyk7O2RlAwBQboSbKqJLF6l9e+tWDC+8cPJjtm+XunWTXnpJKi6WLrvMWiNn0yar5aZbN6ubqlcvafPmUP8CAADKhnBTRbhcJa0348ZJt94q7dxZ8v7q1VKnTtLGjVJSkvTJJ9Z9qfr0sc6tUUP6z3+k1FRpzx4r+Hz3nW0/BwCAU3IZY4zdRYRSfn6+4uPjlZeXp7i4OLvLCanCQun666UFC6zX0dHWuJpzzpFGjrTeb9tWevdda/r4yfzyi9Sjh/T551LTptKGDVJsbEh/BgCgCirP329abqqQyEjpjTesWU9du1qL+v3979KNN1rBZsAAafnyUwcbSapTx+qiatxY+uEHaeLEUP4CAAB+G+GmCrroIivELFwotWxp7ft//096++2ytcLUqydNn249f/JJacuW4NYLAEB50C1VxRUXS7t3W4GlvK64whqH07u3NfDY5QpGhQAA0C2FcggPP7NgI1lTxd1u6f33pXfeCXRlAACcGcINztg550h33209v+subswJAKgYCDc4K/ffLzVsKG3dat2mAQAAuxFucFZq1JCeeMJ6PnmyFXIAALAT4QZn7dprpZ49rdWPBw2S8vPtrggAUJURbnDWXC5p5kxrDZy1a6X+/a37UAEAYAfCDQLi3HOl//5Xio+31tC56iqrJQcAgFAj3CBgfvc7a72bGjWs+1Jde61UVGR3VQCAqoZwg4BKS5MWLZKioqzHoUNpwQEAhBbhBgHXo4d1K4dq1aQ337Ru9/DDD3ZXBQCoKgg3CIo+faxbM9SpI2VlWV1W775rd1UAgKqAcIOg6dVL+uwzqUsXaf9+6corpfvuk44ds7syAICTEW4QVCkp0scfS2PGWK8ffdSaKn7ggN2VAQCcinCDoHO7rZtsvvGGVL26NWX80kulPXvsrgwA4ESEG4TMtddKH35YstjfRRdJ27bZXRUAwGkINwipzp2tRf5SUqQtW6Ru3aQvvrC7KgCAkxBuEHItW0r/+590wQXSTz9JF18sLVhgd1UAAKcg3MAWDRtKn34qde0q7dtn3XDz2mulXbvsrgwAUNkRbmCb2rWlZcuk8eOliAhrwb9WrQLXirNrl5STE5jPAgBUHoQb2Mrtlh5+WFqzRmrTxppBNWiQNHr02X3ukSPWwoFt2lhr7AAAqg7CDSqEDh2kdeukBx+UwsKk6dOtm3CeqQ8/tMbz7NkjvfVWICsFAFR0hBtUGG639Mgj0l13Wa9vu00qKDizz3rnnZLnc+YEpj4AQOVAuEGF88gjUpMm0o4d0l//Wv7zPR7/cLNsmfTzzwEtEQBQgRFuUOHUqCE995z1fNo0adWq8p2flWWFmZgYKTXVCjtvvBGUUgEAFRDhBhXS5ZdLGRmSMdLNN0tFRWU/99//th779JFuuMF6TtcUAFQdhBtUWE88IdWta61gPHly2c/zdkn9/vfW2jnh4dZsrG+/DVqpAIAKhHCDCqtOHenpp63n//d//uNoTuWHH6RNm6xA06+flJQkpadb782dG9x6AQAVA+EGFdrgwdKVV0pHj1otMQMHSj/+eOrjvQHo4outRQIl6Y9/tB7nzLG6uQAAzmZ7uHnmmWfUpEkTRUVFqXPnzlqzZs0pj/3iiy80cOBANWnSRC6XS1OnTg1prQg9l8tqcbn3Xqs15q23pPPPt1p0iotLH+8db/P735fsu+oqKSpK+uoracOG0NUOALCHreFm/vz5Gjt2rCZMmKD169erXbt26t27t3ad4gZDhw4dUrNmzTR58mTVq1cv5PXCHtWrW2Nu1q+XunSRDhyQxoyRevSQ8vNLjtu3T/rkE+v5lVeW7I+Lk664wnrOwGIAcD5bw82TTz6pW265RcOHD9cFF1ygmTNnqnr16po9e/ZJj7/wwgv12GOP6brrrlNkZGTI64W92raVVqyQZs60Asvy5daMKG/Aee89qzWndWupWTP/c71dU3PnWlPDAQDOZVu4KSoqUlZWltK9oz0lhYWFKT09XStXrgzY9xQWFio/P99vQ+UVFmatXLxsmVSrlrRypdS3r7WSsXe8zYmtNl59+0rx8dZ4neXLQ142ACCEbAs3e/bsUXFxsZKSkvz2JyUlKSeAt3KeNGmS4uPjfVtKSkrAPhv2+d3vpA8+kGrWlP73P6sFx3svqhPH23hFRVmDkSUW9AMAp7N9QHGwjRs3Tnl5eb5tx44ddpeEAPl1wCkokOrXlzp2PPnxffpYj+Vd8RgAULnYFm4SEhIUHh6u3Nxcv/25ubkBHSwcGRmpuLg4vw3OkZoqLV1qBRxJGjDA6ro6mQsvtB43bpQKC0NXIwAgtGwLN263W6mpqcrMzPTt83g8yszMVFpaml1loRLq2FH66CNp5EjpgQdOfVzjxtbCgEePMiUcAJzM1m6psWPHatasWXr55Ze1efNmjRw5UgcPHtTw4cMlScOGDdO4ceN8xxcVFSk7O1vZ2dkqKirSTz/9pOzsbH3LuvpVXrt20rPPSo0anfoYl6uk9WbdupCVBgAIsQg7v3zw4MHavXu3xo8fr5ycHLVv315LlizxDTLevn27wk7oY9i5c6c6dOjge/3444/r8ccfV/fu3fXRRx/Z8htQuXTsKC1ZIq1da3clAIBgcRlTtRakz8/PV3x8vPLy8hh/UwW98441m6pVK+nzz+2uBgBQVuX5++342VLAibwzqTZvtlY6BgA4D+EGVUpysrV5PNJnn9ldDQAgGAg3qHK8g4oZdwMAzkS4QZXDjCkAcDbCDaoc77gbWm4AwJkIN6hyvOHm22+lffvsrgYAEGiEG1Q5depIzZpZz7Oy7K4GABBohBtUSXRNAYBzEW5QJTFjCgCci3CDKokZUwDgXIQbVEm/+511I80dO6TcXLurAQAEEuEGVVJsrNSypfWcrikAcBbCDaosuqYAwJkIN6iymDEFAM5EuEGVdeKMKWPsrgYAECiEG1RZ7dpJERHS7t3Se+/ZXQ0AIFAIN6iyoqOlIUOs51dfLf3rX3ZXBAAIBMINqrR//lMaNEg6etR6fPlluysCAJwtwg2qNLdbmjNHuvFGyeORbrhBmj7d7qoAAGeDcIMqLzxcmjVLGjPGej16tDR1qt1VAQDOFOEGkBQWJv3jH9KDD1qv/9//k5Yts7sqAMCZINwAx7lc0sMPSxkZVhfVdddJO3faXRUAoLwIN8AJXC7p2Weltm2lXbusgHP0qN1VAQDKg3AD/Er16tKbb1r3n/r0U+n+++2uCABQHoQb4CRatJBeesl6/vjj0ttv210RAKCsCDfAKfzhD9LYsdbzG26wWnEAABUf4QY4jcmTpYsvlvLzpZ49pSef5D5UAFDREW6A06hWTVq82LpNQ3GxNUV88GCpoMDuygAAp0K4AX5DjRrS669L06ZZN9pcsEDq1En66iu7KwMAnAzhBigDl0u64w7pk0+k5GQr2PTsKf34o92VAQB+jXADlENamvTZZ1Lr1lJOjnU38cOH7a4KAHAiwg1QTomJ0jvvSHXqSOvWSTffzCBjAKhICDfAGWja1FroLyLCuqv4lCl2VwQA8CLcAGeoRw/pqaes5+PGSf/5j90VAQBEuAHOzsiR0m23Wd1SQ4ZY3VQAAHsRboCz4HJJTz8tXXKJtfZN9+7Sv/9td1UAULURboCz5HZL774r9e4tHTpkzaDydlcBAEKPcAMEQFycFXBuvdXqorrrLunOO61VjQEAoUW4AQKkWjVp5sySmVPTpknXXCMdPWp3ZQBQtRBugAByuaR77rFu0RAZKS1cKN10k+Tx2F0ZAFQdhBsgCK65RvrXv6TwcOnVV6W772ahPwAIFcINECT9+0svvmg9/8c/pMmT7a4IAKoGwg0QRNdfLz35pPX8/vulWbPsrggAnI9wAwTZn/9srWAsSSNGSK+/bndFAOBshBsgBCZOtKaJezxWa84zz9hdEQA4F+EGCAGXS5oxQxo92hpYfMcd0t/+xiBjAAgGwg0QImFh1srFEyZYr8ePt7qsmCYOAIFFuAFCyOWSHnqo5PYMTz1l3a5hyRKpqMju6gDAGQg3gA3uvFN65RVrHZx33pH69pXq1pWGDrXWx9m3z+4KAaDychlTtXr98/PzFR8fr7y8PMXFxdldDqq4tWuttXDeflvKyfF/74ILpIsukrp1k3r0kBo1sqtKALBfef5+E26ACsDjkVavlt56S1q0SPrqq9LHdO8u3XCDtfpxTIwdVQKAfQg3p0G4QWWwe7f0v/9Jy5db2+rVJTOrqle3Ak7PnlLLltZWs6bdFQNAcBFuToNwg8poxw7rHlUvvyx9/XXp9+vVk845R0pIkGrXlurUsR737ZO2bi3ZCgqkTp2sbq4ePaQuXaSoKDt+EQCUD+HmNAg3qMyMkVatkt54Q9q0Sdq8Wdq588w/LzLSugfWI49IrVoFslIACCzCzWkQbuA0+fnWGJ2tW6VffpH27i15jI+XmjQp2dxuacUK6aOPpGXLSgYxu1zWyskPP2wdBwAVDeHmNAg3gMUYaeNGa6Xkf/3L2letmnTzzVKbNlKNGiffYmKk2FiJf30AhBLh5jQIN0Bpa9ZYN/f88MOyn5OUJLVrV7Kdc44VeGJjrQAUEyNFRASzagBVCeHmNAg3wKl98IE0Z46UlycdPGhtBw6UPPe+Li4u2+edc47UsWPJ1qqVFXoiI62uMAAoK8LNaRBugLNjjBVyvvhC2rChZNu+3dpfUCAdPXr6z3C5rCnt0dElW1RUyeOvt6ZNpYsvtmZ3Va8eql8KoCIpz99vGo0BlIvLZbW+dO5sbSdTVGRNQ9+4UVq3rmTbvt163xuQDh4s33dHREipqdb3ut1WiPJuhw9bg6sLCqzHI0esVZ67drW29u2tcwA4Hy03AELGG0IOHSrZjhyx9nkfDx+WCgut10eOWMds3Ch9+qn0449n/t1RUSULHsbFWTPJataUkpOtW1t4t/r1rYHVACoWWm4AVEjVqlnbmfz/CmOkbdukTz6xwo7LVfJ51apZ4SUurmRQc3i49Nln1krPK1da0+Ozs8v2XWFh1rgg71atmtVqFB5uPVavLp17rtUydP751mPt2tZYJO8mlQSo8PDS3+ENerGxjD8CAo2WGwCOZ4z0zTfSd99ZXVZ5eda2f7/VGrR9u7Xt2PHb44XORFycFX7Cwkq6zgoLrffcbqu1qEEDa4uPt97zbkVF1nknBjnJv/Xr0CErcJ24OnV8fMmYJW9Ii4uz3vdutWvTVYfKo9K13DzzzDN67LHHlJOTo3bt2mnatGnq1KnTKY9fsGCBHnzwQW3dulUtWrTQo48+qn79+oW0ZgCVh8tltbSce+7pjysutkLPieGisFA6dsx/y8uzFk788ktrlegvv7TGD4WHW1tYmBWoDh+2Pjc/39pOpqjIapHati3wv7ss3O6S9Ytq1LBCkjcQeR+9rVZhYSW/8devo6Otc73biTPivI+Rkf6DyN1u63p6x00dO2bVkJAg1a1rPUZGWtcoL88ax7V/vxXw6tWzjjndcgPGWJ9bVGRtbrf1vSdrSYOz2B5u5s+fr7Fjx2rmzJnq3Lmzpk6dqt69e2vLli1KTEwsdfz//vc/DRkyRJMmTdIVV1yhOXPm6KqrrtL69evVunVrW34DAGcID7daM8piwIDfPqaoyPpjvG+ftWK0MVY3lHcRxKgo6yapP/1Ush044N8l5nZbd433hoBjx6zPOTFIREVZQWrv3pIVqvfv9w9oR45YAeGXX6xt3z7rc7x/+PftO+vLFxSRkSWtXL8WFmYFnLp1reviHbPlHbdVVHTy87zdmN5Qd+Lmdpd0P54Y4lwuawsLK725XP7X2RuIvf0i3sfoaP8FMd3u0l2Sxlj/e3s3HR9I793Cw0u6SSMiSlrzTvxnJjLSOvfXLYBSye9wufy7cuPirN9vjBXyPR7/blbva4/H//vcbuv1r39HZKQVQO1ie7dU586ddeGFF2r69OmSJI/Ho5SUFI0ePVr33XdfqeMHDx6sgwcPatGiRb59Xbp0Ufv27TVz5szf/D66pQCgpJXqxHWMDhwoGdx94qDuX/+BO9nmHfzt3byB5MQ/8IWFJeHj0CHrD673D7X38cABac8ea/v1ekregeBHj0q7dpX88UfFk5ZmjXcLpErTLVVUVKSsrCyNGzfOty8sLEzp6elauXLlSc9ZuXKlxo4d67evd+/eWrhw4UmPLywsVOEJsT//VG3DAFCFeFupytpSFWoej9X6lJ9f0rJwYhdUcbEVgHJyrNavatX810v6detXtWpWmDpxVt6Joa6gwHr0to6dGNy8rSm/fvSGPWP8u/G8g9C9XC7r2CNHrO87dMh6/HXLkjGlW4W8LSneLtGjR63X3jqPHbM+59ctR78eFH/i2CpjrO3IkZIu0/x86/e7XCfvdvS+DguzvvvEFqGTjVOLjAzGPxVlZ2u42bNnj4qLi5WUlOS3PykpSV999dVJz8nJyTnp8TneOwD+yqRJk/Twww8HsGoAQLCFhZ0+fIWHW7cA+dWfg9OqVs3qDoLzhdldQLCNGzdOeXl5vm3Hjh12lwQAAILI1pabhIQEhYeHKzc3129/bm6u6p1iJFK9evXKdXxkZKQi7W4fAwAAIWNry43b7VZqaqoyMzN9+zwejzIzM5WWlnbSc9LS0vyOl6SlS5ee8ngAAFC12D4VfOzYscrIyFDHjh3VqVMnTZ06VQcPHtTw4cMlScOGDVODBg00adIkSdKYMWPUvXt3PfHEE+rfv7/mzZundevW6fnnn7f5lwAAgIrA9nAzePBg7d69W+PHj1dOTo7at2+vJUuW+AYNb9++XWFhJQ1MXbt21Zw5c/TXv/5V999/v1q0aKGFCxeyxg0AAJAqwjo3ocY6NwAAVD7l+fvt+NlSAACgaiHcAAAARyHcAAAARyHcAAAARyHcAAAARyHcAAAARyHcAAAARyHcAAAAR7F9heJQ865ZmJ+fb3cpAACgjLx/t8uy9nCVCzcFBQWSpJSUFLtLAQAA5VRQUKD4+PjTHlPlbr/g8Xi0c+dOxcbGyuVyBfSz8/PzlZKSoh07dnBrhyDjWocO1zp0uNahw7UOnUBda2OMCgoKlJyc7HfPyZOpci03YWFhatiwYVC/Iy4ujn9ZQoRrHTpc69DhWocO1zp0AnGtf6vFxosBxQAAwFEINwAAwFEINwEUGRmpCRMmKDIy0u5SHI9rHTpc69DhWocO1zp07LjWVW5AMQAAcDZabgAAgKMQbgAAgKMQbgAAgKMQbgAAgKMQbgLkmWeeUZMmTRQVFaXOnTtrzZo1dpdU6U2aNEkXXnihYmNjlZiYqKuuukpbtmzxO+bIkSO6/fbbVadOHcXExGjgwIHKzc21rWanmDx5slwul+666y7fPq514Pz000/605/+pDp16ig6Olpt2rTRunXrfO8bYzR+/HjVr19f0dHRSk9P1zfffGNrzZVRcXGxHnzwQTVt2lTR0dFq3ry5/va3v/ndm4hrfeY++eQTDRgwQMnJyXK5XFq4cKHf+2W5tnv37tXQoUMVFxenmjVr6qabbtKBAwfOvjiDszZv3jzjdrvN7NmzzRdffGFuueUWU7NmTZObm2t3aZVa7969zYsvvmg+//xzk52dbfr162caNWpkDhw44DtmxIgRJiUlxWRmZpp169aZLl26mK5du9pad2W3Zs0a06RJE9O2bVszZswY336udWDs3bvXNG7c2Nxwww1m9erV5vvvvzfvv/+++fbbb33HTJ482cTHx5uFCxeaDRs2mCuvvNI0bdrUHD582NbaK5uJEyeaOnXqmEWLFpkffvjBLFiwwMTExJinnnrKdwzX+sy999575oEHHjBvvfWWkWTefvttv/fLcm379Olj2rVrZ1atWmU+/fRTc84555ghQ4acdW2EmwDo1KmTuf32232vi4uLTXJyspk0aZKtdTnNrl27jCTz8ccfG2OM2b9/v6lWrZpZsGCB75jNmzcbSWblypU2Vlp5FRQUmBYtWpilS5ea7t27+8IN1zpw7r33XnPRRRed8n2Px2Pq1atnHnvsMd++/fv3m8jISDN37twQVekM/fv3NzfeeKPfvj/84Q9m6NChxnCtA+rX4aYs1/bLL780kszatWt9xyxevNi4XC7z008/nVU9dEudpaKiImVlZSk9Pd23LywsTOnp6Vq5cqWttTlNXl6eJKl27dqSpKysLB09etTv2rds2VKNGjXi2p+h22+/Xf379/e7puJaB9Q777yjjh076tprr1ViYqI6dOigWbNm+d7/4YcflJOT43et4+Pj1blzZ651OXXt2lWZmZn6+uuvJUkbNmzQ8uXL1bdvX4lrHVRlubYrV65UzZo11bFjR98x6enpCgsL0+rVq8/q+6vcjTMDbc+ePSouLlZSUpLf/qSkJH311Ve21eU0Ho9Hd911l7p166bWrVtLknJycuR2u1WzZk2/Y5OSkpSTk2NTpZXXvHnztH79eq1du7bUe1zrwPn+++81Y8YMjR07Vvfff7/Wrl2rO++8U263WxkZGb7rebL/pnCty+e+++5Tfn6+WrZsqfDwcBUXF2vixIkaOnSodPyfa3Gtg6Is1zYnJ0eJiYl+70dERKh27dpnff0JN6gUbr/9dn3++edavny53aU40o4dOzRmzBgtXbpUUVFRdpfjaB6PRx07dtTf//53SVKHDh30+eefa+bMmcrIyLC7PEd544039Prrr2vOnDlq1aqVsrOzdddddyk5OZlr7XB0S52lhIQEhYeHl5o1kpubq3r16tlWl5PccccdWrRokZYtW6aGDRv69terV09FRUXav3+/3/Fc+/LLysrSrl279Lvf/U4RERGKiIjQxx9/rKeffloRERFKSkriWgdI/fr1dcEFF/jtO//887V9+3bp+D/XOn5tT8S1Lr977rlH9913n6677jq1adNG119/vf785z9r0qRJEtc6qMpybevVq6ddu3b5vX/s2DHt3bv3rK8/4eYsud1upaamKjMz07fP4/EoMzNTaWlpttZW2RljdMcdd+jtt9/Whx9+qKZNm/q9n5qaqmrVqvld+y1btmj79u1c+3K67LLLtGnTJmVnZ/u2jh07aujQob7nXOvA6NatW6klDb7++ms1btxYktS0aVPVq1fP71rn5+dr9erVXOtyOnTokMLC/P/MhYeHy+PxSFzroCrLtU1LS9P+/fuVlZXlO+bDDz+Ux+NR586dz66AsxqODGOOTwWPjIw0L730kvnyyy/NrbfeamrWrGlycnLsLq1SGzlypImPjzcfffSR+fnnn33boUOHfMeMGDHCNGrUyHz44Ydm3bp1Ji0tzaSlpdlat1OcOFvKcK0DZs2aNSYiIsJMnDjRfPPNN+b111831atXN6+99prvmMmTJ5uaNWuaf//732bjxo3m97//PdOTz0BGRoZp0KCBbyr4W2+9ZRISEsxf/vIX3zFc6zNXUFBgPvvsM/PZZ58ZSebJJ580n332mdm2bZsxZby2ffr0MR06dDCrV682y5cvNy1atGAqeEUybdo006hRI+N2u02nTp3MqlWr7C6p0pN00u3FF1/0HXP48GEzatQoU6tWLVO9enVz9dVXm59//tnWup3i1+GGax047777rmndurWJjIw0LVu2NM8//7zf+x6Pxzz44IMmKSnJREZGmssuu8xs2bLFtnorq/z8fDNmzBjTqFEjExUVZZo1a2YeeOABU1hY6DuGa33mli1bdtL/RmdkZBhTxmv7yy+/mCFDhpiYmBgTFxdnhg8fbgoKCs66Npc5calGAACASo4xNwAAwFEINwAAwFEINwAAwFEINwAAwFEINwAAwFEINwAAwFEINwAAwFEINwAAwFEINwCqJJfLpYULF9pdBoAgINwACLkbbrhBLper1NanTx+7SwPgABF2FwCgaurTp49efPFFv32RkZG21QPAOWi5AWCLyMhI1atXz2+rVauWdLzLaMaMGerbt6+io6PVrFkzvfnmm37nb9q0SZdeeqmio6NVp04d3XrrrTpw4IDfMbNnz1arVq0UGRmp+vXr64477vB7f8+ePbr66qtVvXp1tWjRQu+8847vvX379mno0KGqW7euoqOj1aJFi1JhDEDFRLgBUCE9+OCDGjhwoDZs2KChQ4fquuuu0+bNmyVJBw8eVO/evVWrVi2tXbtWCxYs0AcffOAXXmbMmKHbb79dt956qzZt2qR33nlH55xzjt93PPzwwxo0aJA2btyofv36aejQodq7d6/v+7/88kstXrxYmzdv1owZM5SQkBDiqwDgjJz1fcUBoJwyMjJMeHi4qVGjht82ceJEY4wxksyIESP8zuncubMZOXKkMcaY559/3tSqVcscOHDA9/5//vMfExYWZnJycowxxiQnJ5sHHnjglDVIMn/96199rw8cOGAkmcWLFxtjjBkwYIAZPnx4gH85gFBgzA0AW/Ts2VMzZszw21e7dm3f87S0NL/30tLSlJ2dLUnavHmz2rVrpxo1avje79atmzwej7Zs2SKXy6WdO3fqsssuO20Nbdu29T2vUaOG4uLitGvXLknSyJEjNXDgQK1fv16XX365rrrqKnXt2vUsfzWAUCDcALBFjRo1SnUTBUp0dHSZjqtWrZrfa5fLJY/HI0nq27evtm3bpvfee09Lly7VZZddpttvv12PP/54UGoGEDiMuQFQIa1atarU6/PPP1+SdP7552vDhg06ePCg7/0VK1YoLCxM5513nmJjY9WkSRNlZmaeVQ1169ZVRkaGXnvtNU2dOlXPP//8WX0egNCg5QaALQoLC5WTk+O3LyIiwjdod8GCBerYsaMuuugivf7661qzZo1eeOEFSdLQoUM1YcIEZWRk6KGHHtLu3bs1evRoXX/99UpKSpIkPfTQQxoxYoQSExPVt29fFRQUaMWKFRo9enSZ6hs/frxSU1PVqlUrFRYWatGiRb5wBaBiI9wAsMWSJUtUv359v33nnXeevvrqK+n4TKZ58+Zp1KhRql+/vubOnasLLrhAklS9enW9//77GjNmjC688EJVr15dAwcO1JNPPun7rIyMDB05ckT/+Mc/dPfddyshIUHXXHNNmetzu90aN26ctm7dqujoaF188cWaN29ewH4/gOBxGWvWAABUGC6XS2+//bauuuoqu0sBUAkx5gYAADgK4QYAADgKY24AVDj0lgM4G7TcAAAARyHcAAAARyHcAAAARyHcAAAARyHcAAAARyHcAAAARyHcAAAARyHcAAAAR/n/66M0wFecaLoAAAAASUVORK5CYII=", "text/plain": [ - "<Figure size 640x480 with 1 Axes>" + "0.001733972690999508" ] }, + "execution_count": 19, "metadata": {}, - "output_type": "display_data" + "output_type": "execute_result" } ], "source": [ - "plt.suptitle('MSE vs Epochs')\n", - "plt.plot(tune_train_err, label='Train', color='blue')\n", - "plt.xlabel('Epochs')\n", - "plt.ylabel('MSE')\n", - "plt.show()" + "evaluate(mkbatch(BSZ, large=False, target=\"onpath\"))[2]" ] }, { "cell_type": "code", - "execution_count": 19, + "execution_count": 17, "metadata": {}, "outputs": [ { "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAHgCAYAAABZ+0ykAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABCC0lEQVR4nO3deXRU9f3/8dckYZJAFpaQQCCsoih7g0BABTTIJlaLgpRKxBVQxPLTKlpB7ZeCuBQFBaXiziJWqVLBYsQFyhoMoCKuLIoJIJCELYHM5/fHZSaMAUxgZm5y83ycc8/M3Ll35j23al79bNdljDECAABwiDC7CwAAAAgkwg0AAHAUwg0AAHAUwg0AAHAUwg0AAHAUwg0AAHAUwg0AAHAUwg0AAHAUwg0AAHAUwg0ABJHL5dIdd9xhdxlAlUK4ASqxl156SS6XSy6XS8uXLy/1vjFGKSkpcrlcuuKKK/zeO3DggCZMmKDWrVurRo0aqlOnjtq3b68xY8Zo586dvuMeeugh33ecbMvJyQnJbz2V09U2YsQIW2sDYI8IuwsAcPaioqI0Z84cXXTRRX77P/74Y/3444+KjIz023/06FFdcskl+uqrr5SRkaHRo0frwIED+uKLLzRnzhxdffXVSk5O9jtnxowZiomJKfXdNWvWDNKvKrtevXpp2LBhpfafe+65ttQDwF6EG8AB+vXrpwULFujpp59WRETJv9Zz5sxRamqq9uzZ43f8woUL9dlnn+n111/XH//4R7/3jhw5oqKiolLfcc011yghISGIv+LMnXvuufrTn/5kdxkAKgi6pQAHGDJkiH755RctXbrUt6+oqEhvvvlmqfAiSd99950kqVu3bqXei4qKUlxcXEDqat26tXr27Flqv8fjUYMGDXTNNdf49s2bN0+pqamKjY1VXFyc2rRpo6eeeiogdUhSjx491Lp1a2VlZalr166Kjo5W06ZNNXPmzFLH7tq1SzfddJOSkpIUFRWldu3a6eWXXz7p73jqqafUpk0bRUVFqW7duurTp4/WrVtX6tiFCxeqdevWioyMVKtWrbRkyRK/9wsKCnTXXXepSZMmioyMVGJionr16qX169cH7BoAVQXhBnCAJk2aKC0tTXPnzvXtW7x4sfLy8nTdddeVOr5x48aSpFdeeUXGmDJ9x969e7Vnzx6/bf/+/ac9Z/Dgwfrkk09KjctZvny5du7c6att6dKlGjJkiGrVqqVHH31UkydPVo8ePbRixYoy1XbkyJFSte3Zs6dUC9S+ffvUr18/paamasqUKWrYsKFGjhyp2bNn+445fPiwevTooVdffVVDhw7VY489pvj4eN1www2lwtZNN92ku+66SykpKXr00Ud13333KSoqSqtWrSr1e0eNGqXrrrtOU6ZM0ZEjRzRw4ED98ssvvmNGjBihGTNmaODAgXr22Wd19913Kzo6Wps3by7TNQBwAgOg0nrxxReNJLN27Vozffp0Exsbaw4dOmSMMebaa681PXv2NMYY07hxY9O/f3/feYcOHTLnnXeekWQaN25sbrjhBvPCCy+Y3NzcUt8xYcIEI+mk23nnnXfa+rZs2WIkmWnTpvntHzVqlImJifHVOmbMGBMXF2eOHTtW7mtwqtokmblz5/qO6969u5FknnjiCd++wsJC0759e5OYmGiKioqMMcZMnTrVSDKvvfaa77iioiKTlpZmYmJiTH5+vjHGmA8//NBIMnfeeWepmjwej199brfbfPvtt759GzZsKHVd4uPjze23317u3w+gNFpuAIcYNGiQDh8+rEWLFqmgoECLFi06aZeUJEVHR2v16tW65557pOOzrm666SbVr19fo0ePVmFhYalz/vWvf2np0qV+24svvnjams4991y1b99e8+fP9+0rLi7Wm2++qQEDBig6Olo6Pij54MGDft1q5fH73/++VG1Lly4t1SUWERGh2267zffa7Xbrtttu065du5SVlSVJeu+991SvXj0NGTLEd1y1atV055136sCBA/r4449918PlcmnChAml6nG5XH6v09PT1bx5c9/rtm3bKi4uTt9//71vX82aNbV69Wq/mWoAzgwDigGHqFu3rtLT0zVnzhwdOnRIxcXFfmNafi0+Pl5TpkzRlClTtG3bNmVmZurxxx/X9OnTFR8fr//7v//zO/6SSy45owHFgwcP1v3336+ffvpJDRo00EcffaRdu3Zp8ODBvmNGjRqlN954Q3379lWDBg10+eWXa9CgQerTp0+ZvqNhw4ZKT0//zeOSk5NVo0YNv33eGVVbt25Vly5dtG3bNrVo0UJhYf7/3+/888+XJG3btk06Pm4pOTlZtWvX/s3vbdSoUal9tWrV0r59+3yvp0yZooyMDKWkpCg1NVX9+vXTsGHD1KxZs9/8fAD+aLkBHOSPf/yjFi9erJkzZ6pv375lnqbduHFj3XjjjVqxYoVq1qyp119/PWA1DR48WMYYLViwQJL0xhtvKD4+3i+4JCYmKjs7W++8846uvPJKLVu2TH379lVGRkbA6rBTeHj4SfefON5p0KBB+v777zVt2jQlJyfrscceU6tWrbR48eIQVgo4A+EGcJCrr75aYWFhWrVq1Sm7pE6nVq1aat68uX7++eeA1dS0aVN16tRJ8+fP17Fjx/TWW2/pqquuKrX2jtvt1oABA/Tss8/qu+++02233aZXXnlF3377bcBq2blzpw4ePOi37+uvv5aOD8rW8aD3zTffyOPx+B331Vdf+d6XpObNm2vnzp3au3dvwOqrX7++Ro0apYULF+qHH35QnTp1NHHixIB9PlBVEG4AB4mJidGMGTP00EMPacCAAac8bsOGDaXWvtHxLpcvv/xS5513XkDrGjx4sFatWqXZs2drz549fl1SkvxmDUlSWFiY2rZtK0knHf9zpo4dO6bnnnvO97qoqEjPPfec6tatq9TUVOn4mkE5OTl+44SOHTumadOmKSYmRt27d5ckDRw4UMYYPfzww6W+p6wz0LyKi4uVl5fnty8xMVHJyckB/f1AVcGYG8BhytKVs3TpUk2YMEFXXnmlunTpopiYGH3//feaPXu2CgsL9dBDD5U658033zzpCsW9evVSUlLSab9v0KBBuvvuu3X33Xerdu3apcbH3Hzzzdq7d68uvfRSNWzYUNu2bdO0adPUvn1731iX0/n666/12muvldqflJSkXr16+V4nJyfr0Ucf1datW3Xuuedq/vz5ys7O1vPPP69q1apJkm699VY999xzuuGGG5SVlaUmTZrozTff1IoVKzR16lTFxsZKknr27Knrr79eTz/9tL755hv16dNHHo9Hn376qXr27Fmu+0kVFBSoYcOGuuaaa9SuXTvFxMTogw8+0Nq1a/XEE0+U+XMAHGf3dC0AZ+7EqeCn8+up4N9//70ZP3686dKli0lMTDQRERGmbt26pn///ubDDz/0O/d0U8ElmWXLlpWp1m7duhlJ5uabby713ptvvmkuv/xyk5iYaNxut2nUqJG57bbbzM8///ybn3u62rp37+47rnv37qZVq1Zm3bp1Ji0tzURFRZnGjRub6dOnl/rM3NxcM3z4cJOQkGDcbrdp06aNefHFF0sdd+zYMfPYY4+Zli1bGrfbberWrWv69u1rsrKy/Oo72RTvxo0bm4yMDGOOT0m/5557TLt27UxsbKypUaOGadeunXn22Wd/8/cDKM1lytt+CgCVUI8ePbRnzx59/vnndpcCIMgYcwMAAByFcAMAAByFcAMAAByFMTcAAMBRaLkBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOQrgBAACOEmF3AaHm8Xi0c+dOxcbGyuVy2V0OAAAoA2OMCgoKlJycrLCw07fNVLlws3PnTqWkpNhdBgAAOAM7duxQw4YNT3tMlQs3sbGx0vGLExcXZ3c5AACgDPLz85WSkuL7O346VS7ceLui4uLiCDcAAFQyZRlSwoBiAADgKIQbAADgKIQbAADgKIQbAADgKIQbAADgKIQbAADgKIQbAADgKIQbAADgKIQbAADgKIQbAADgKIQbAADgKIQbAADgKFXuxpnBUlQk5eZKHo/UuLHd1QAAUHXRchMga9ZIjRpJvXrZXQkAAFUb4SZAoqKsxyNH7K4EAICqjXATIN5wU1hodyUAAFRthJsAoeUGAICKgXATIJGR1iPhBgAAexFuAsTbclNUZM2YAgAA9iDcBIg33IhxNwAA2IpwEyAnhhu6pgAAsA/hJkAiIqSw41eTlhsAAOxDuAkQl4sZUwAAVASEmwBixhQAAPYj3AQQLTcAANiPcBNAhBsAAOxHuAkgwg0AAPYj3AQQ95cCAMB+hJsAouUGAAD7EW4CiNlSAADYj3ATQLTcAABgP8JNABFuAACwH+EmgAg3AADYj3ATQIQbAADsR7gJIKaCAwBgP8JNANFyAwCA/Qg3AcRUcAAA7Ee4CSBabgAAsB/hJoAINwAA2I9wE0CEGwAA7Ee4CSBmSwEAYD/CTQDRcgMAgP0INwHEbCkAAOxHuAkgWm4AALAf4SaACDcAANiPcBNAhBsAAOxHuAkgZksBAGA/wk0A0XIDAID9CDcBxGwpAADsR7gJIFpuAACwH+EmgAg3AADYr0KEm2eeeUZNmjRRVFSUOnfurDVr1pTpvHnz5snlcumqq64Keo1l4Q03xcXSsWN2VwMAQNVke7iZP3++xo4dqwkTJmj9+vVq166devfurV27dp32vK1bt+ruu+/WxRdfHLJaf4s33IjWGwAAbGN7uHnyySd1yy23aPjw4brgggs0c+ZMVa9eXbNnzz7lOcXFxRo6dKgefvhhNWvWLKT1no53QLGYDg4AgG1sDTdFRUXKyspSenp6SUFhYUpPT9fKlStPed4jjzyixMRE3XTTTb/5HYWFhcrPz/fbgiU8XIqIsJ7TcgMAgD1sDTd79uxRcXGxkpKS/PYnJSUpJyfnpOcsX75cL7zwgmbNmlWm75g0aZLi4+N9W0pKSkBqPxUGFQMAYC/bu6XKo6CgQNdff71mzZqlhISEMp0zbtw45eXl+bYdO3YEtUbCDQAA9oqw88sTEhIUHh6u3Nxcv/25ubmqV69eqeO/++47bd26VQMGDPDt83g8kqSIiAht2bJFzZs39zsnMjJSkScOhgkywg0AAPayteXG7XYrNTVVmZmZvn0ej0eZmZlKS0srdXzLli21adMmZWdn+7Yrr7xSPXv2VHZ2dtC7nMqCcAMAgL1sbbmRpLFjxyojI0MdO3ZUp06dNHXqVB08eFDDhw+XJA0bNkwNGjTQpEmTFBUVpdatW/udX7NmTUkqtd8u3DwTAAB72R5uBg8erN27d2v8+PHKyclR+/bttWTJEt8g4+3btyssrPIMDaLlBgAAe7mMMcbuIkIpPz9f8fHxysvLU1xcXMA//6KLpBUrpH/9S/rDHwL+8QAAVEnl+ftdeZpEKglabgAAsBfhJsAINwAA2ItwE2CEGwAA7EW4CTBmSwEAYC/CTYDRcgMAgL0INwHmXQyZcAMAgD0INwFGyw0AAPYi3AQY4QYAAHsRbgKMcAMAgL0INwFGuAEAwF6EmwBjKjgAAPYi3AQYs6UAALAX4SbA6JYCAMBehJsAI9wAAGAvwk2AEW4AALAX4SbACDcAANiLcBNgzJYCAMBehJsAY7YUAAD2ItwEGN1SAADYi3ATYIQbAADsRbgJMMINAAD2ItwE2Inhxhi7qwEAoOoh3ASYN9xI0tGjdlYCAEDVRLgJsBPDDV1TAACEHuEmwNzukueEGwAAQo9wE2AuF2vdAABgJ8JNEDBjCgAA+xBugoBwAwCAfQg3QUC4AQDAPoSbIODmmQAA2IdwEwQMKAYAwD6EmyCgWwoAAPsQboKAcAMAgH0IN0FAuAEAwD6EmyAg3AAAYB/CTRAwWwoAAPsQboKA2VIAANiHcBMEdEsBAGAfwk0QEG4AALAP4SYICDcAANiHcBMEhBsAAOxDuAkCZksBAGAfwk0Q0HIDAIB9CDdBwFRwAADsQ7gJAlpuAACwD+EmCAg3AADYh3ATBIQbAADsQ7gJAsINAAD2IdwEAVPBAQCwD+EmCJgtBQCAfQg3QUC3FAAA9iHcBAHhBgAA+xBugoBwAwCAfQg3QUC4AQDAPoSbIPCGm6NHJY/H7moAAKhaCDdB4J0tJaaDAwAQcoSbIPC23IiuKQAAQo5wEwQREVLY8StLuAEAILQIN0HgcjGoGAAAuxBugoRwAwCAPQg3QUK4AQDAHoSbIPHOmGK2FAAAoUW4CRJabgAAsEeFCDfPPPOMmjRpoqioKHXu3Flr1qw55bFvvfWWOnbsqJo1a6pGjRpq3769Xn311ZDWWxaEGwAA7GF7uJk/f77Gjh2rCRMmaP369WrXrp169+6tXbt2nfT42rVr64EHHtDKlSu1ceNGDR8+XMOHD9f7778f8tpPh3ADAIA9bA83Tz75pG655RYNHz5cF1xwgWbOnKnq1atr9uzZJz2+R48euvrqq3X++eerefPmGjNmjNq2bavly5eHvPbTIdwAAGAPW8NNUVGRsrKylJ6eXlJQWJjS09O1cuXK3zzfGKPMzExt2bJFl1xyyUmPKSwsVH5+vt8WCoQbAADsYWu42bNnj4qLi5WUlOS3PykpSTk5Oac8Ly8vTzExMXK73erfv7+mTZumXr16nfTYSZMmKT4+3relpKQE/HecjDfcMFsKAIDQsr1b6kzExsYqOztba9eu1cSJEzV27Fh99NFHJz123LhxysvL8207duwISY3eqeC03AAAEFoRdn55QkKCwsPDlZub67c/NzdX9erVO+V5YWFhOueccyRJ7du31+bNmzVp0iT16NGj1LGRkZGKPPE23SFCtxQAAPawteXG7XYrNTVVmZmZvn0ej0eZmZlKS0sr8+d4PB4VVrD+H8INAAD2sLXlRpLGjh2rjIwMdezYUZ06ddLUqVN18OBBDR8+XJI0bNgwNWjQQJMmTZKOj6Hp2LGjmjdvrsLCQr333nt69dVXNWPGDJt/iT/CDQAA9rA93AwePFi7d+/W+PHjlZOTo/bt22vJkiW+Qcbbt29XWFhJA9PBgwc1atQo/fjjj4qOjlbLli312muvafDgwTb+itIINwAA2MNljDF2FxFK+fn5io+PV15enuLi4oL2PQ8/LD30kDRypPTss0H7GgAAqoTy/P2ulLOlKgNmSwEAYA/CTZDQLQUAgD0IN0FCuAEAwB6EmyAh3AAAYA/CTZAQbgAAsAfhJkgINwAA2INwEyTe2VIVbOFkAAAcj3ATJLTcAABgD8JNkBBuAACwB+EmSAg3AADYg3ATJIQbAADsQbgJEsINAAD2INwECbOlAACwB+EmSLwtN8XF0rFjdlcDAEDVQbgJEm+4EV1TAACEFOEmSLzdUiLcAAAQUoSbIAkPl6pVs54TbgAACB3CTRAxYwoAgNAj3ASRN9wwYwoAgNAh3ASRd9wNLTcAAIQO4SaI6JYCACD0CDdBRLgBACD0CDdBRLgBACD0CDdBRLgBACD0CDdBRLgBACD0CDdBxM0zAQAIPcJNENWoYT3u2mV3JQAAVB2EmyC65BLr8a237K4EAICqg3ATRNdea91jau1a6dtv7a4GAICqgXATRImJ0mWXWc/nzrW7GgAAqgbCTZD98Y/W49y5kjF2VwMAgPMRboLs6qutWVObN0sbN9pdDQAAzke4CbK4OKl/f+v5nDl2VwMAgPMRbkJgyBDrcd48yeOxuxoAAJyNcBMC/ftLsbHS9u3SypV2VwMAgLMRbkIgOtoaeyO6pgAACLpyhZspU6bo8OHDvtcrVqxQ4Qn3FigoKNCoUaMCW6FDeLumFiyQjh2zuxoAAJzLZUzZJyiHh4fr559/VmJioiQpLi5O2dnZatasmSQpNzdXycnJKi4uDl7FZyk/P1/x8fHKy8tTXFxcyL736FGpQQNp925pyRKpd++QfTUAAJVeef5+l6vl5tc5qBy5qMqrVs1asVgs6AcAQFAx5iaEvF1Tb79tteQAAIDAI9yEUNeuUp06Un6+tGaN3dUAAOBMEeU94Z///KdiYmIkSceOHdNLL72khIQE6fiAYpxaWJh1r6k33pCWLpW6dbO7IgAAnKdcA4qbNGkil8v1m8f98MMPZ1tX0Ng1oNjrn/+UbrnFCjbLl4f86wEAqJTK8/e7XC03W7duPdvaqrz0dOtx1Sqre8qGfAUAgKMx5ibEmjSRmjeXiouljz+2uxoAAJynXOFm5cqVWrRokd++V155RU2bNlViYqJuvfVWv0X9cHK9elmPH3xgdyUAADhPucLNI488oi+++ML3etOmTbrpppuUnp6u++67T++++64mTZoUjDodxds1tXSp3ZUAAOA85Qo32dnZuuyyy3yv582bp86dO2vWrFkaO3asnn76ab3xxhvBqNNRLr1UcrmkzZuln36yuxoAAJylXOFm3759SkpK8r3++OOP1bdvX9/rCy+8UDt27AhshQ5Uq5bUsaP1nK4pAAACq1zhJikpyTfNu6ioSOvXr1eXLl187xcUFKhatWqBr9KBGHcDAEBwlCvc9OvXT/fdd58+/fRTjRs3TtWrV9fFF1/se3/jxo1q3rx5MOp0HO+4mw8+kLhFFwAAgVOucPO3v/1NERER6t69u2bNmqXnn39ebrfb9/7s2bN1+eWXB6NOx+naVYqOlnJypBPGaAMAgLNUrkX8EhIS9MknnygvL08xMTEKDw/3e3/BggWKjY0NdI2OFBkpXXKJ9P77VutN69Z2VwQAgDOUK9zceOONZTpu9uzZZ1pPlZKeboWbpUulu+6yuxoAAJyhXOHmpZdeUuPGjdWhQweV45ZUOAXvoOKPP5aKiqQTevgAAMAZKle4GTlypObOnasffvhBw4cP15/+9CfVrl07eNU5XJs2Ut260u7d0urV0gljswEAwBkq14DiZ555Rj///LP+8pe/6N1331VKSooGDRqk999/n5acMxAWJnnXRGS1YgAAAqPcN86MjIzUkCFDtHTpUn355Zdq1aqVRo0apSZNmujAgQPBqdLBvOFm2TK7KwEAwBnO6q7gYWFhcrlcMsaouLg4cFVVIZdeaj2uWiUdPGh3NQAAVH7lDjeFhYWaO3euevXqpXPPPVebNm3S9OnTtX37dsXExASnSgdr2lRq3Fg6dkxavtzuagAAqPzKFW5GjRql+vXra/Lkybriiiu0Y8cOLViwQP369VNY2Fk1AlVZLpfUs6f1nK4pAADOnsuUYyRwWFiYGjVqpA4dOsjlcp3yuLfeeitQ9QVcfn6+4uPjlZeXp7i4OLvLkSS9+qo0bJh04YXSmjV2VwMAQMVTnr/f5ZoKPmzYsNOGGpwZb8tNVpaUlyfFx9tdEQAAlVe5F/ELhmeeeUaPPfaYcnJy1K5dO02bNk2dOnU66bGzZs3SK6+8os8//1ySlJqaqr///e+nPL4yaNhQatFC+uYb6ZNPpAED7K4IAIDKy/aBMvPnz9fYsWM1YcIErV+/Xu3atVPv3r21a9eukx7/0UcfaciQIVq2bJlWrlyplJQUXX755frpp59CXnsgeWdNffih3ZUAAFC5lWvMTTB07txZF154oaZPny5J8ng8SklJ0ejRo3Xffff95vnFxcWqVauWpk+frmHDhv3m8RVxzI0kzZ8vXXed1K6dlJ1tdzUAAFQs5fn7bWvLTVFRkbKyspSenl5SUFiY0tPTtXLlyjJ9xqFDh3T06NFT3gaisLBQ+fn5fltF1KOH9bhhg7Rnj93VAABQedkabvbs2aPi4mIlJSX57U9KSlJOTk6ZPuPee+9VcnKyX0A60aRJkxQfH+/bUlJSAlJ7oCUlSa1bW88/+sjuagAAqLxsH3NzNiZPnqx58+bp7bffVlRU1EmPGTdunPLy8nzbjh07Ql5nWbHeDQAAZ8/WcJOQkKDw8HDl5ub67c/NzVW9evVOe+7jjz+uyZMn67///a/atm17yuMiIyMVFxfnt1VU5RlU/NNP0rZtQS8JAIBKx9Zw43a7lZqaqszMTN8+j8ejzMxMpaWlnfK8KVOm6G9/+5uWLFmijh07hqja4Ove3Vqx+KuvpJ07/d8rLJQyM6V77pHatLGmj597rrRunV3VAgBQMdneLTV27FjNmjVLL7/8sjZv3qyRI0fq4MGDGj58uHR84cBx48b5jn/00Uf14IMPavbs2WrSpIlycnKUk5PjiDuS16oldehgPfeOu9m3T7r/fikhQUpPlx5/XDq+xI+KiqSRIyXuWQoAQIlyLeIXDIMHD9bu3bs1fvx45eTkqH379lqyZIlvkPH27dv97ls1Y8YMFRUV6ZprrvH7nAkTJuihhx4Kef2Bduml0vr10qJF0tat0pQp1qrFklSvntSnj7W1aSOlpVktN7NmSSNG2F05AAAVg+3r3IRaRV3nxmvxYqlfP/99rVtLEydaKxefePeLadOkO++UataUtmyREhNDXi4AACFRada5QWkXXSRFR1vPmzWTXnvNWtTvyiv9g41kdUl16CDt3y/de68t5QIAUOEQbiqY2Fjp/felefOkzZuloUOl8PCTHxsRIc2YYYWel16Sli8PdbUAAFQ8hJsK6OKLpcGDJbf7t4/t3Fm65Rbr+ciR0tGjQS8PAIAKjXDjAH//u1SnjjWL6vgtugAAqLIINw5Qp44VcCSrmwoAgKqMcOMQQ4ZI1apJ33wjff213dUAAGAfwo1DxMaW3Fn8P/+xuxoAAOxDuHGQ/v2tx0WL7K4EAAD7EG4c5IorrMdPPpHy8+2uBgAAexBuHKR5c+m886Rjx6T//tfuagAAsAfhxmG8rTeMuwEAVFWEG4fxjrt57z3J47G7GgAAQo9w4zAXXSTFxUm7dll3DAcAoKoh3DhMtWpS797Wc2ZNAQCqIsKNAzHuBgBQlRFuHKhvX+tO4evXSzt32l0NAAChRbhxoLp1rbuF6/jAYgAAqhLCjUOxWjEAoKoi3DiUd9zNBx9IR47YXQ0AAKFDuHGodu2kBg2kgwdZrRgAULUQbhzK5ZKuu856/txzdlcDAEDoEG4cbMQI63HxYun77+2uBgCA0CDcONg551gL+hkjzZxpdzUAAIQG4cbhbr/denzhBenwYburAQAg+Ag3Dtevn9SokbR3rzR/vt3VAAAQfIQbhwsPLxl78+yzdlcDAEDwEW6qgJtuktxuae1aayuP/fulKVOkyy+XPv00WBUCABA4hJsqIDFRuvZa63lZW2+2bZP+/GcpJUW6915p6VLrnlWrVgW1VAAAzhrhporwDiyeN0/65ZdTH3fggDR8uNS8uTR1qvW6VSspLc1aELBvXyk7O2RlAwBQboSbKqJLF6l9e+tWDC+8cPJjtm+XunWTXnpJKi6WLrvMWiNn0yar5aZbN6ubqlcvafPmUP8CAADKhnBTRbhcJa0348ZJt94q7dxZ8v7q1VKnTtLGjVJSkvTJJ9Z9qfr0sc6tUUP6z3+k1FRpzx4r+Hz3nW0/BwCAU3IZY4zdRYRSfn6+4uPjlZeXp7i4OLvLCanCQun666UFC6zX0dHWuJpzzpFGjrTeb9tWevdda/r4yfzyi9Sjh/T551LTptKGDVJsbEh/BgCgCirP329abqqQyEjpjTesWU9du1qL+v3979KNN1rBZsAAafnyUwcbSapTx+qiatxY+uEHaeLEUP4CAAB+G+GmCrroIivELFwotWxp7ft//096++2ytcLUqydNn249f/JJacuW4NYLAEB50C1VxRUXS7t3W4GlvK64whqH07u3NfDY5QpGhQAA0C2FcggPP7NgI1lTxd1u6f33pXfeCXRlAACcGcINztg550h33209v+subswJAKgYCDc4K/ffLzVsKG3dat2mAQAAuxFucFZq1JCeeMJ6PnmyFXIAALAT4QZn7dprpZ49rdWPBw2S8vPtrggAUJURbnDWXC5p5kxrDZy1a6X+/a37UAEAYAfCDQLi3HOl//5Xio+31tC56iqrJQcAgFAj3CBgfvc7a72bGjWs+1Jde61UVGR3VQCAqoZwg4BKS5MWLZKioqzHoUNpwQEAhBbhBgHXo4d1K4dq1aQ337Ru9/DDD3ZXBQCoKgg3CIo+faxbM9SpI2VlWV1W775rd1UAgKqAcIOg6dVL+uwzqUsXaf9+6corpfvuk44ds7syAICTEW4QVCkp0scfS2PGWK8ffdSaKn7ggN2VAQCcinCDoHO7rZtsvvGGVL26NWX80kulPXvsrgwA4ESEG4TMtddKH35YstjfRRdJ27bZXRUAwGkINwipzp2tRf5SUqQtW6Ru3aQvvrC7KgCAkxBuEHItW0r/+590wQXSTz9JF18sLVhgd1UAAKcg3MAWDRtKn34qde0q7dtn3XDz2mulXbvsrgwAUNkRbmCb2rWlZcuk8eOliAhrwb9WrQLXirNrl5STE5jPAgBUHoQb2Mrtlh5+WFqzRmrTxppBNWiQNHr02X3ukSPWwoFt2lhr7AAAqg7CDSqEDh2kdeukBx+UwsKk6dOtm3CeqQ8/tMbz7NkjvfVWICsFAFR0hBtUGG639Mgj0l13Wa9vu00qKDizz3rnnZLnc+YEpj4AQOVAuEGF88gjUpMm0o4d0l//Wv7zPR7/cLNsmfTzzwEtEQBQgRFuUOHUqCE995z1fNo0adWq8p2flWWFmZgYKTXVCjtvvBGUUgEAFRDhBhXS5ZdLGRmSMdLNN0tFRWU/99//th779JFuuMF6TtcUAFQdhBtUWE88IdWta61gPHly2c/zdkn9/vfW2jnh4dZsrG+/DVqpAIAKhHCDCqtOHenpp63n//d//uNoTuWHH6RNm6xA06+flJQkpadb782dG9x6AQAVA+EGFdrgwdKVV0pHj1otMQMHSj/+eOrjvQHo4outRQIl6Y9/tB7nzLG6uQAAzmZ7uHnmmWfUpEkTRUVFqXPnzlqzZs0pj/3iiy80cOBANWnSRC6XS1OnTg1prQg9l8tqcbn3Xqs15q23pPPPt1p0iotLH+8db/P735fsu+oqKSpK+uoracOG0NUOALCHreFm/vz5Gjt2rCZMmKD169erXbt26t27t3ad4gZDhw4dUrNmzTR58mTVq1cv5PXCHtWrW2Nu1q+XunSRDhyQxoyRevSQ8vNLjtu3T/rkE+v5lVeW7I+Lk664wnrOwGIAcD5bw82TTz6pW265RcOHD9cFF1ygmTNnqnr16po9e/ZJj7/wwgv12GOP6brrrlNkZGTI64W92raVVqyQZs60Asvy5daMKG/Aee89qzWndWupWTP/c71dU3PnWlPDAQDOZVu4KSoqUlZWltK9oz0lhYWFKT09XStXrgzY9xQWFio/P99vQ+UVFmatXLxsmVSrlrRypdS3r7WSsXe8zYmtNl59+0rx8dZ4neXLQ142ACCEbAs3e/bsUXFxsZKSkvz2JyUlKSeAt3KeNGmS4uPjfVtKSkrAPhv2+d3vpA8+kGrWlP73P6sFx3svqhPH23hFRVmDkSUW9AMAp7N9QHGwjRs3Tnl5eb5tx44ddpeEAPl1wCkokOrXlzp2PPnxffpYj+Vd8RgAULnYFm4SEhIUHh6u3Nxcv/25ubkBHSwcGRmpuLg4vw3OkZoqLV1qBRxJGjDA6ro6mQsvtB43bpQKC0NXIwAgtGwLN263W6mpqcrMzPTt83g8yszMVFpaml1loRLq2FH66CNp5EjpgQdOfVzjxtbCgEePMiUcAJzM1m6psWPHatasWXr55Ze1efNmjRw5UgcPHtTw4cMlScOGDdO4ceN8xxcVFSk7O1vZ2dkqKirSTz/9pOzsbH3LuvpVXrt20rPPSo0anfoYl6uk9WbdupCVBgAIsQg7v3zw4MHavXu3xo8fr5ycHLVv315LlizxDTLevn27wk7oY9i5c6c6dOjge/3444/r8ccfV/fu3fXRRx/Z8htQuXTsKC1ZIq1da3clAIBgcRlTtRakz8/PV3x8vPLy8hh/UwW98441m6pVK+nzz+2uBgBQVuX5++342VLAibwzqTZvtlY6BgA4D+EGVUpysrV5PNJnn9ldDQAgGAg3qHK8g4oZdwMAzkS4QZXDjCkAcDbCDaoc77gbWm4AwJkIN6hyvOHm22+lffvsrgYAEGiEG1Q5depIzZpZz7Oy7K4GABBohBtUSXRNAYBzEW5QJTFjCgCci3CDKokZUwDgXIQbVEm/+511I80dO6TcXLurAQAEEuEGVVJsrNSypfWcrikAcBbCDaosuqYAwJkIN6iymDEFAM5EuEGVdeKMKWPsrgYAECiEG1RZ7dpJERHS7t3Se+/ZXQ0AIFAIN6iyoqOlIUOs51dfLf3rX3ZXBAAIBMINqrR//lMaNEg6etR6fPlluysCAJwtwg2qNLdbmjNHuvFGyeORbrhBmj7d7qoAAGeDcIMqLzxcmjVLGjPGej16tDR1qt1VAQDOFOEGkBQWJv3jH9KDD1qv/9//k5Yts7sqAMCZINwAx7lc0sMPSxkZVhfVdddJO3faXRUAoLwIN8AJXC7p2Weltm2lXbusgHP0qN1VAQDKg3AD/Er16tKbb1r3n/r0U+n+++2uCABQHoQb4CRatJBeesl6/vjj0ttv210RAKCsCDfAKfzhD9LYsdbzG26wWnEAABUf4QY4jcmTpYsvlvLzpZ49pSef5D5UAFDREW6A06hWTVq82LpNQ3GxNUV88GCpoMDuygAAp0K4AX5DjRrS669L06ZZN9pcsEDq1En66iu7KwMAnAzhBigDl0u64w7pk0+k5GQr2PTsKf34o92VAQB+jXADlENamvTZZ1Lr1lJOjnU38cOH7a4KAHAiwg1QTomJ0jvvSHXqSOvWSTffzCBjAKhICDfAGWja1FroLyLCuqv4lCl2VwQA8CLcAGeoRw/pqaes5+PGSf/5j90VAQBEuAHOzsiR0m23Wd1SQ4ZY3VQAAHsRboCz4HJJTz8tXXKJtfZN9+7Sv/9td1UAULURboCz5HZL774r9e4tHTpkzaDydlcBAEKPcAMEQFycFXBuvdXqorrrLunOO61VjQEAoUW4AQKkWjVp5sySmVPTpknXXCMdPWp3ZQBQtRBugAByuaR77rFu0RAZKS1cKN10k+Tx2F0ZAFQdhBsgCK65RvrXv6TwcOnVV6W772ahPwAIFcINECT9+0svvmg9/8c/pMmT7a4IAKoGwg0QRNdfLz35pPX8/vulWbPsrggAnI9wAwTZn/9srWAsSSNGSK+/bndFAOBshBsgBCZOtKaJezxWa84zz9hdEQA4F+EGCAGXS5oxQxo92hpYfMcd0t/+xiBjAAgGwg0QImFh1srFEyZYr8ePt7qsmCYOAIFFuAFCyOWSHnqo5PYMTz1l3a5hyRKpqMju6gDAGQg3gA3uvFN65RVrHZx33pH69pXq1pWGDrXWx9m3z+4KAaDychlTtXr98/PzFR8fr7y8PMXFxdldDqq4tWuttXDeflvKyfF/74ILpIsukrp1k3r0kBo1sqtKALBfef5+E26ACsDjkVavlt56S1q0SPrqq9LHdO8u3XCDtfpxTIwdVQKAfQg3p0G4QWWwe7f0v/9Jy5db2+rVJTOrqle3Ak7PnlLLltZWs6bdFQNAcBFuToNwg8poxw7rHlUvvyx9/XXp9+vVk845R0pIkGrXlurUsR737ZO2bi3ZCgqkTp2sbq4ePaQuXaSoKDt+EQCUD+HmNAg3qMyMkVatkt54Q9q0Sdq8Wdq588w/LzLSugfWI49IrVoFslIACCzCzWkQbuA0+fnWGJ2tW6VffpH27i15jI+XmjQp2dxuacUK6aOPpGXLSgYxu1zWyskPP2wdBwAVDeHmNAg3gMUYaeNGa6Xkf/3L2letmnTzzVKbNlKNGiffYmKk2FiJf30AhBLh5jQIN0Bpa9ZYN/f88MOyn5OUJLVrV7Kdc44VeGJjrQAUEyNFRASzagBVCeHmNAg3wKl98IE0Z46UlycdPGhtBw6UPPe+Li4u2+edc47UsWPJ1qqVFXoiI62uMAAoK8LNaRBugLNjjBVyvvhC2rChZNu+3dpfUCAdPXr6z3C5rCnt0dElW1RUyeOvt6ZNpYsvtmZ3Va8eql8KoCIpz99vGo0BlIvLZbW+dO5sbSdTVGRNQ9+4UVq3rmTbvt163xuQDh4s33dHREipqdb3ut1WiPJuhw9bg6sLCqzHI0esVZ67drW29u2tcwA4Hy03AELGG0IOHSrZjhyx9nkfDx+WCgut10eOWMds3Ch9+qn0449n/t1RUSULHsbFWTPJataUkpOtW1t4t/r1rYHVACoWWm4AVEjVqlnbmfz/CmOkbdukTz6xwo7LVfJ51apZ4SUurmRQc3i49Nln1krPK1da0+Ozs8v2XWFh1rgg71atmtVqFB5uPVavLp17rtUydP751mPt2tZYJO8mlQSo8PDS3+ENerGxjD8CAo2WGwCOZ4z0zTfSd99ZXVZ5eda2f7/VGrR9u7Xt2PHb44XORFycFX7Cwkq6zgoLrffcbqu1qEEDa4uPt97zbkVF1nknBjnJv/Xr0CErcJ24OnV8fMmYJW9Ii4uz3vdutWvTVYfKo9K13DzzzDN67LHHlJOTo3bt2mnatGnq1KnTKY9fsGCBHnzwQW3dulUtWrTQo48+qn79+oW0ZgCVh8tltbSce+7pjysutkLPieGisFA6dsx/y8uzFk788ktrlegvv7TGD4WHW1tYmBWoDh+2Pjc/39pOpqjIapHati3wv7ss3O6S9Ytq1LBCkjcQeR+9rVZhYSW/8devo6Otc73biTPivI+Rkf6DyN1u63p6x00dO2bVkJAg1a1rPUZGWtcoL88ax7V/vxXw6tWzjjndcgPGWJ9bVGRtbrf1vSdrSYOz2B5u5s+fr7Fjx2rmzJnq3Lmzpk6dqt69e2vLli1KTEwsdfz//vc/DRkyRJMmTdIVV1yhOXPm6KqrrtL69evVunVrW34DAGcID7daM8piwIDfPqaoyPpjvG+ftWK0MVY3lHcRxKgo6yapP/1Ush044N8l5nZbd433hoBjx6zPOTFIREVZQWrv3pIVqvfv9w9oR45YAeGXX6xt3z7rc7x/+PftO+vLFxSRkSWtXL8WFmYFnLp1reviHbPlHbdVVHTy87zdmN5Qd+Lmdpd0P54Y4lwuawsLK725XP7X2RuIvf0i3sfoaP8FMd3u0l2Sxlj/e3s3HR9I793Cw0u6SSMiSlrzTvxnJjLSOvfXLYBSye9wufy7cuPirN9vjBXyPR7/blbva4/H//vcbuv1r39HZKQVQO1ie7dU586ddeGFF2r69OmSJI/Ho5SUFI0ePVr33XdfqeMHDx6sgwcPatGiRb59Xbp0Ufv27TVz5szf/D66pQCgpJXqxHWMDhwoGdx94qDuX/+BO9nmHfzt3byB5MQ/8IWFJeHj0CHrD673D7X38cABac8ea/v1ekregeBHj0q7dpX88UfFk5ZmjXcLpErTLVVUVKSsrCyNGzfOty8sLEzp6elauXLlSc9ZuXKlxo4d67evd+/eWrhw4UmPLywsVOEJsT//VG3DAFCFeFupytpSFWoej9X6lJ9f0rJwYhdUcbEVgHJyrNavatX810v6detXtWpWmDpxVt6Joa6gwHr0to6dGNy8rSm/fvSGPWP8u/G8g9C9XC7r2CNHrO87dMh6/HXLkjGlW4W8LSneLtGjR63X3jqPHbM+59ctR78eFH/i2CpjrO3IkZIu0/x86/e7XCfvdvS+DguzvvvEFqGTjVOLjAzGPxVlZ2u42bNnj4qLi5WUlOS3PykpSV999dVJz8nJyTnp8TneOwD+yqRJk/Twww8HsGoAQLCFhZ0+fIWHW7cA+dWfg9OqVs3qDoLzhdldQLCNGzdOeXl5vm3Hjh12lwQAAILI1pabhIQEhYeHKzc3129/bm6u6p1iJFK9evXKdXxkZKQi7W4fAwAAIWNry43b7VZqaqoyMzN9+zwejzIzM5WWlnbSc9LS0vyOl6SlS5ee8ngAAFC12D4VfOzYscrIyFDHjh3VqVMnTZ06VQcPHtTw4cMlScOGDVODBg00adIkSdKYMWPUvXt3PfHEE+rfv7/mzZundevW6fnnn7f5lwAAgIrA9nAzePBg7d69W+PHj1dOTo7at2+vJUuW+AYNb9++XWFhJQ1MXbt21Zw5c/TXv/5V999/v1q0aKGFCxeyxg0AAJAqwjo3ocY6NwAAVD7l+fvt+NlSAACgaiHcAAAARyHcAAAARyHcAAAARyHcAAAARyHcAAAARyHcAAAARyHcAAAAR7F9heJQ865ZmJ+fb3cpAACgjLx/t8uy9nCVCzcFBQWSpJSUFLtLAQAA5VRQUKD4+PjTHlPlbr/g8Xi0c+dOxcbGyuVyBfSz8/PzlZKSoh07dnBrhyDjWocO1zp0uNahw7UOnUBda2OMCgoKlJyc7HfPyZOpci03YWFhatiwYVC/Iy4ujn9ZQoRrHTpc69DhWocO1zp0AnGtf6vFxosBxQAAwFEINwAAwFEINwEUGRmpCRMmKDIy0u5SHI9rHTpc69DhWocO1zp07LjWVW5AMQAAcDZabgAAgKMQbgAAgKMQbgAAgKMQbgAAgKMQbgLkmWeeUZMmTRQVFaXOnTtrzZo1dpdU6U2aNEkXXnihYmNjlZiYqKuuukpbtmzxO+bIkSO6/fbbVadOHcXExGjgwIHKzc21rWanmDx5slwul+666y7fPq514Pz000/605/+pDp16ig6Olpt2rTRunXrfO8bYzR+/HjVr19f0dHRSk9P1zfffGNrzZVRcXGxHnzwQTVt2lTR0dFq3ry5/va3v/ndm4hrfeY++eQTDRgwQMnJyXK5XFq4cKHf+2W5tnv37tXQoUMVFxenmjVr6qabbtKBAwfOvjiDszZv3jzjdrvN7NmzzRdffGFuueUWU7NmTZObm2t3aZVa7969zYsvvmg+//xzk52dbfr162caNWpkDhw44DtmxIgRJiUlxWRmZpp169aZLl26mK5du9pad2W3Zs0a06RJE9O2bVszZswY336udWDs3bvXNG7c2Nxwww1m9erV5vvvvzfvv/+++fbbb33HTJ482cTHx5uFCxeaDRs2mCuvvNI0bdrUHD582NbaK5uJEyeaOnXqmEWLFpkffvjBLFiwwMTExJinnnrKdwzX+sy999575oEHHjBvvfWWkWTefvttv/fLcm379Olj2rVrZ1atWmU+/fRTc84555ghQ4acdW2EmwDo1KmTuf32232vi4uLTXJyspk0aZKtdTnNrl27jCTz8ccfG2OM2b9/v6lWrZpZsGCB75jNmzcbSWblypU2Vlp5FRQUmBYtWpilS5ea7t27+8IN1zpw7r33XnPRRRed8n2Px2Pq1atnHnvsMd++/fv3m8jISDN37twQVekM/fv3NzfeeKPfvj/84Q9m6NChxnCtA+rX4aYs1/bLL780kszatWt9xyxevNi4XC7z008/nVU9dEudpaKiImVlZSk9Pd23LywsTOnp6Vq5cqWttTlNXl6eJKl27dqSpKysLB09etTv2rds2VKNGjXi2p+h22+/Xf379/e7puJaB9Q777yjjh076tprr1ViYqI6dOigWbNm+d7/4YcflJOT43et4+Pj1blzZ651OXXt2lWZmZn6+uuvJUkbNmzQ8uXL1bdvX4lrHVRlubYrV65UzZo11bFjR98x6enpCgsL0+rVq8/q+6vcjTMDbc+ePSouLlZSUpLf/qSkJH311Ve21eU0Ho9Hd911l7p166bWrVtLknJycuR2u1WzZk2/Y5OSkpSTk2NTpZXXvHnztH79eq1du7bUe1zrwPn+++81Y8YMjR07Vvfff7/Wrl2rO++8U263WxkZGb7rebL/pnCty+e+++5Tfn6+WrZsqfDwcBUXF2vixIkaOnSodPyfa3Gtg6Is1zYnJ0eJiYl+70dERKh27dpnff0JN6gUbr/9dn3++edavny53aU40o4dOzRmzBgtXbpUUVFRdpfjaB6PRx07dtTf//53SVKHDh30+eefa+bMmcrIyLC7PEd544039Prrr2vOnDlq1aqVsrOzdddddyk5OZlr7XB0S52lhIQEhYeHl5o1kpubq3r16tlWl5PccccdWrRokZYtW6aGDRv69terV09FRUXav3+/3/Fc+/LLysrSrl279Lvf/U4RERGKiIjQxx9/rKeffloRERFKSkriWgdI/fr1dcEFF/jtO//887V9+3bp+D/XOn5tT8S1Lr977rlH9913n6677jq1adNG119/vf785z9r0qRJEtc6qMpybevVq6ddu3b5vX/s2DHt3bv3rK8/4eYsud1upaamKjMz07fP4/EoMzNTaWlpttZW2RljdMcdd+jtt9/Whx9+qKZNm/q9n5qaqmrVqvld+y1btmj79u1c+3K67LLLtGnTJmVnZ/u2jh07aujQob7nXOvA6NatW6klDb7++ms1btxYktS0aVPVq1fP71rn5+dr9erVXOtyOnTokMLC/P/MhYeHy+PxSFzroCrLtU1LS9P+/fuVlZXlO+bDDz+Ux+NR586dz66AsxqODGOOTwWPjIw0L730kvnyyy/NrbfeamrWrGlycnLsLq1SGzlypImPjzcfffSR+fnnn33boUOHfMeMGDHCNGrUyHz44Ydm3bp1Ji0tzaSlpdlat1OcOFvKcK0DZs2aNSYiIsJMnDjRfPPNN+b111831atXN6+99prvmMmTJ5uaNWuaf//732bjxo3m97//PdOTz0BGRoZp0KCBbyr4W2+9ZRISEsxf/vIX3zFc6zNXUFBgPvvsM/PZZ58ZSebJJ580n332mdm2bZsxZby2ffr0MR06dDCrV682y5cvNy1atGAqeEUybdo006hRI+N2u02nTp3MqlWr7C6p0pN00u3FF1/0HXP48GEzatQoU6tWLVO9enVz9dVXm59//tnWup3i1+GGax047777rmndurWJjIw0LVu2NM8//7zf+x6Pxzz44IMmKSnJREZGmssuu8xs2bLFtnorq/z8fDNmzBjTqFEjExUVZZo1a2YeeOABU1hY6DuGa33mli1bdtL/RmdkZBhTxmv7yy+/mCFDhpiYmBgTFxdnhg8fbgoKCs66Npc5calGAACASo4xNwAAwFEINwAAwFEINwAAwFEINwAAwFEINwAAwFEINwAAwFEINwAAwFEINwAAwFEINwCqJJfLpYULF9pdBoAgINwACLkbbrhBLper1NanTx+7SwPgABF2FwCgaurTp49efPFFv32RkZG21QPAOWi5AWCLyMhI1atXz2+rVauWdLzLaMaMGerbt6+io6PVrFkzvfnmm37nb9q0SZdeeqmio6NVp04d3XrrrTpw4IDfMbNnz1arVq0UGRmp+vXr64477vB7f8+ePbr66qtVvXp1tWjRQu+8847vvX379mno0KGqW7euoqOj1aJFi1JhDEDFRLgBUCE9+OCDGjhwoDZs2KChQ4fquuuu0+bNmyVJBw8eVO/evVWrVi2tXbtWCxYs0AcffOAXXmbMmKHbb79dt956qzZt2qR33nlH55xzjt93PPzwwxo0aJA2btyofv36aejQodq7d6/v+7/88kstXrxYmzdv1owZM5SQkBDiqwDgjJz1fcUBoJwyMjJMeHi4qVGjht82ceJEY4wxksyIESP8zuncubMZOXKkMcaY559/3tSqVcscOHDA9/5//vMfExYWZnJycowxxiQnJ5sHHnjglDVIMn/96199rw8cOGAkmcWLFxtjjBkwYIAZPnx4gH85gFBgzA0AW/Ts2VMzZszw21e7dm3f87S0NL/30tLSlJ2dLUnavHmz2rVrpxo1avje79atmzwej7Zs2SKXy6WdO3fqsssuO20Nbdu29T2vUaOG4uLitGvXLknSyJEjNXDgQK1fv16XX365rrrqKnXt2vUsfzWAUCDcALBFjRo1SnUTBUp0dHSZjqtWrZrfa5fLJY/HI0nq27evtm3bpvfee09Lly7VZZddpttvv12PP/54UGoGEDiMuQFQIa1atarU6/PPP1+SdP7552vDhg06ePCg7/0VK1YoLCxM5513nmJjY9WkSRNlZmaeVQ1169ZVRkaGXnvtNU2dOlXPP//8WX0egNCg5QaALQoLC5WTk+O3LyIiwjdod8GCBerYsaMuuugivf7661qzZo1eeOEFSdLQoUM1YcIEZWRk6KGHHtLu3bs1evRoXX/99UpKSpIkPfTQQxoxYoQSExPVt29fFRQUaMWKFRo9enSZ6hs/frxSU1PVqlUrFRYWatGiRb5wBaBiI9wAsMWSJUtUv359v33nnXeevvrqK+n4TKZ58+Zp1KhRql+/vubOnasLLrhAklS9enW9//77GjNmjC688EJVr15dAwcO1JNPPun7rIyMDB05ckT/+Mc/dPfddyshIUHXXHNNmetzu90aN26ctm7dqujoaF188cWaN29ewH4/gOBxGWvWAABUGC6XS2+//bauuuoqu0sBUAkx5gYAADgK4QYAADgKY24AVDj0lgM4G7TcAAAARyHcAAAARyHcAAAARyHcAAAARyHcAAAARyHcAAAARyHcAAAARyHcAAAAR/n/66M0wFecaLoAAAAASUVORK5CYII=", "text/plain": [ - "0.001733972690999508" + "<Figure size 640x480 with 1 Axes>" ] }, - "execution_count": 19, "metadata": {}, - "output_type": "execute_result" + "output_type": "display_data" } ], "source": [ - "def tune_evaluate():\n", - " model.eval()\n", - " test_loss = 0\n", - " with torch.no_grad():\n", - " batch_src, batch_labels, batch_padding_mask = map(lambda x: x.to(device), mktunebatch(BSZ))\n", - " output = model(batch_src, batch_padding_mask)\n", - " loss = criterion(output.squeeze(1), batch_labels)\n", - " return loss.item()\n", - "\n", - "tune_evaluate()" + "plt.suptitle('Log MSE vs Epochs')\n", + "plt.plot(torch.log(tune_train_err), label='Train', color='blue')\n", + "plt.xlabel('Epochs')\n", + "plt.ylabel('Log MSE')\n", + "plt.show()\n", + "with open(\"plots/tune-loss.html\", \"w\") as f:\n", + " mpld3.save_html(plt.gcf(), f)" ] }, { @@ -1581,13 +3786,9 @@ } ], "source": [ - "batch_src, batch_labels, batch_padding_mask = map(lambda x: x.to(device), mktunebatch(BSZ))\n", - "model.eval()\n", - "with torch.no_grad():\n", - " output = model(batch_src, batch_padding_mask)\n", - "x = batch_labels.detach().to(torch.float16).cpu().numpy().flatten()\n", - "y = output.detach().to(torch.float16).cpu().numpy().flatten()\n", - "plt.hist2d(x, y, bins=50, norm=mpl.colors.LogNorm())" + "mkhist(mkbatch(BSZ, large=False, target=\"onpath\"))\n", + "with open(\"plots/tune-hist.html\", \"w\") as f:\n", + " mpld3.save_html(plt.gcf(), f)" ] }, { @@ -1601,6 +3802,17 @@ }, { "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "mkhist(mkbatch(BSZ, large=True, target=\"onpath\", largetarget=False))\n", + "with open(\"plots/test-onpath-smalltarget.html\", \"w\") as f:\n", + " mpld3.save_html(plt.gcf(), f)" + ] + }, + { + "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ @@ -1660,14 +3872,42 @@ } ], "source": [ - "batch_src, batch_labels, batch_padding_mask = map(lambda x: x.to(device), mktunebatch(BSZ, test=True))\n", - "model.eval()\n", - "with torch.no_grad():\n", - " output = model(batch_src, batch_padding_mask)\n", - "print(criterion(output.squeeze(1), batch_labels).item())\n", - "x = batch_labels.detach().to(torch.float16).cpu().numpy().flatten()\n", - "y = output.detach().to(torch.float16).cpu().numpy().flatten()\n", - "plt.hist2d(x, y, bins=50, norm=mpl.colors.LogNorm())" + "mkhist(mkbatch(BSZ, large=True, target=\"onpath\", largetarget=True))\n", + "with open(\"plots/test-onpath-largetarget.html\", \"w\") as f:\n", + " mpld3.save_html(plt.gcf(), f)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "mkhist(mkbatch(BSZ, large=False, target=\"any\", largetarget=False))\n", + "with open(\"plots/test-small-any.html\", \"w\") as f:\n", + " mpld3.save_html(plt.gcf(), f)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "mkhist(mkbatch(BSZ, large=True, target=\"any\", largetarget=False))\n", + "with open(\"plots/test-large-any-smalltarget.html\", \"w\") as f:\n", + " mpld3.save_html(plt.gcf(), f)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "mkhist(mkbatch(BSZ, large=True, target=\"any\", largetarget=True))\n", + "with open(\"plots/test-large-any-largetarget.html\", \"w\") as f:\n", + " mpld3.save_html(plt.gcf(), f)" ] } ], |