Add latest version of notebooks

2 files changed, 784 insertions, 248 deletions

diff --git a/.ipynb_checkpoints/quickstart_tutorial-checkpoint.ipynb b/.ipynb_checkpoints/quickstart_tutorial-checkpoint.ipynb
index 95d56f7..3ea080e 100644
--- a/.ipynb_checkpoints/quickstart_tutorial-checkpoint.ipynb
+++ b/.ipynb_checkpoints/quickstart_tutorial-checkpoint.ipynb
@@ -1,283 +1,747 @@
 {
-  "cells": [
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "collapsed": false
-      },
-      "outputs": [],
-      "source": [
-        "%matplotlib inline"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "\n`Learn the Basics <intro.html>`_ ||\n**Quickstart** ||\n`Tensors <tensorqs_tutorial.html>`_ ||\n`Datasets & DataLoaders <data_tutorial.html>`_ ||\n`Transforms <transforms_tutorial.html>`_ ||\n`Build Model <buildmodel_tutorial.html>`_ ||\n`Autograd <autogradqs_tutorial.html>`_ ||\n`Optimization <optimization_tutorial.html>`_ ||\n`Save & Load Model <saveloadrun_tutorial.html>`_\n\nQuickstart\n===================\nThis section runs through the API for common tasks in machine learning. Refer to the links in each section to dive deeper.\n\nWorking with data\n-----------------\nPyTorch has two `primitives to work with data <https://pytorch.org/docs/stable/data.html>`_:\n``torch.utils.data.DataLoader`` and ``torch.utils.data.Dataset``.\n``Dataset`` stores the samples and their corresponding labels, and ``DataLoader`` wraps an iterable around\nthe ``Dataset``.\n\n\n"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "collapsed": false
-      },
-      "outputs": [],
-      "source": [
-        "import torch\nfrom torch import nn\nfrom torch.utils.data import DataLoader\nfrom torchvision import datasets\nfrom torchvision.transforms import ToTensor, Lambda, Compose\nimport matplotlib.pyplot as plt"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "PyTorch offers domain-specific libraries such as `TorchText <https://pytorch.org/text/stable/index.html>`_,\n`TorchVision <https://pytorch.org/vision/stable/index.html>`_, and `TorchAudio <https://pytorch.org/audio/stable/index.html>`_,\nall of which include datasets. For this tutorial, we  will be using a TorchVision dataset.\n\nThe ``torchvision.datasets`` module contains ``Dataset`` objects for many real-world vision data like\nCIFAR, COCO (`full list here <https://pytorch.org/vision/stable/datasets.html>`_). In this tutorial, we\nuse the FashionMNIST dataset. Every TorchVision ``Dataset`` includes two arguments: ``transform`` and\n``target_transform`` to modify the samples and labels respectively.\n\n"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "collapsed": false
-      },
-      "outputs": [],
-      "source": [
-        "# Download training data from open datasets.\ntraining_data = datasets.FashionMNIST(\n    root=\"data\",\n    train=True,\n    download=True,\n    transform=ToTensor(),\n)\n\n# Download test data from open datasets.\ntest_data = datasets.FashionMNIST(\n    root=\"data\",\n    train=False,\n    download=True,\n    transform=ToTensor(),\n)"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "We pass the ``Dataset`` as an argument to ``DataLoader``. This wraps an iterable over our dataset, and supports\nautomatic batching, sampling, shuffling and multiprocess data loading. Here we define a batch size of 64, i.e. each element\nin the dataloader iterable will return a batch of 64 features and labels.\n\n"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "collapsed": false
-      },
-      "outputs": [],
-      "source": [
-        "batch_size = 64\n\n# Create data loaders.\ntrain_dataloader = DataLoader(training_data, batch_size=batch_size)\ntest_dataloader = DataLoader(test_data, batch_size=batch_size)\n\nfor X, y in test_dataloader:\n    print(\"Shape of X [N, C, H, W]: \", X.shape)\n    print(\"Shape of y: \", y.shape, y.dtype)\n    break"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "Read more about `loading data in PyTorch <data_tutorial.html>`_.\n\n\n"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "--------------\n\n\n"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "Creating Models\n------------------\nTo define a neural network in PyTorch, we create a class that inherits\nfrom `nn.Module <https://pytorch.org/docs/stable/generated/torch.nn.Module.html>`_. We define the layers of the network\nin the ``__init__`` function and specify how data will pass through the network in the ``forward`` function. To accelerate\noperations in the neural network, we move it to the GPU if available.\n\n"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "collapsed": false
-      },
-      "outputs": [],
-      "source": [
-        "# Get cpu or gpu device for training.\ndevice = \"cuda\" if torch.cuda.is_available() else \"cpu\"\nprint(\"Using {} device\".format(device))\n\n# Define model\nclass NeuralNetwork(nn.Module):\n    def __init__(self):\n        super(NeuralNetwork, self).__init__()\n        self.flatten = nn.Flatten()\n        self.linear_relu_stack = nn.Sequential(\n            nn.Linear(28*28, 512),\n            nn.ReLU(),\n            nn.Linear(512, 512),\n            nn.ReLU(),\n            nn.Linear(512, 10),\n            nn.ReLU()\n        )\n\n    def forward(self, x):\n        x = self.flatten(x)\n        logits = self.linear_relu_stack(x)\n        return logits\n\nmodel = NeuralNetwork().to(device)\nprint(model)"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "Read more about `building neural networks in PyTorch <buildmodel_tutorial.html>`_.\n\n\n"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "--------------\n\n\n"
-      ]
-    },
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [],
+   "source": [
+    "%matplotlib inline"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "`Learn the Basics <intro.html>`_ ||\n",
+    "**Quickstart** ||\n",
+    "`Tensors <tensorqs_tutorial.html>`_ ||\n",
+    "`Datasets & DataLoaders <data_tutorial.html>`_ ||\n",
+    "`Transforms <transforms_tutorial.html>`_ ||\n",
+    "`Build Model <buildmodel_tutorial.html>`_ ||\n",
+    "`Autograd <autogradqs_tutorial.html>`_ ||\n",
+    "`Optimization <optimization_tutorial.html>`_ ||\n",
+    "`Save & Load Model <saveloadrun_tutorial.html>`_\n",
+    "\n",
+    "Quickstart\n",
+    "===================\n",
+    "This section runs through the API for common tasks in machine learning. Refer to the links in each section to dive deeper.\n",
+    "\n",
+    "Working with data\n",
+    "-----------------\n",
+    "PyTorch has two `primitives to work with data <https://pytorch.org/docs/stable/data.html>`_:\n",
+    "``torch.utils.data.DataLoader`` and ``torch.utils.data.Dataset``.\n",
+    "``Dataset`` stores the samples and their corresponding labels, and ``DataLoader`` wraps an iterable around\n",
+    "the ``Dataset``.\n",
+    "\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [],
+   "source": [
+    "import torch\n",
+    "from torch import nn\n",
+    "from torch.utils.data import DataLoader\n",
+    "from torchvision import datasets\n",
+    "from torchvision.transforms import ToTensor, Lambda, Compose\n",
+    "import matplotlib.pyplot as plt"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "PyTorch offers domain-specific libraries such as `TorchText <https://pytorch.org/text/stable/index.html>`_,\n",
+    "`TorchVision <https://pytorch.org/vision/stable/index.html>`_, and `TorchAudio <https://pytorch.org/audio/stable/index.html>`_,\n",
+    "all of which include datasets. For this tutorial, we  will be using a TorchVision dataset.\n",
+    "\n",
+    "The ``torchvision.datasets`` module contains ``Dataset`` objects for many real-world vision data like\n",
+    "CIFAR, COCO (`full list here <https://pytorch.org/vision/stable/datasets.html>`_). In this tutorial, we\n",
+    "use the FashionMNIST dataset. Every TorchVision ``Dataset`` includes two arguments: ``transform`` and\n",
+    "``target_transform`` to modify the samples and labels respectively.\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [
     {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "Optimizing the Model Parameters\n----------------------------------------\nTo train a model, we need a `loss function <https://pytorch.org/docs/stable/nn.html#loss-functions>`_\nand an `optimizer <https://pytorch.org/docs/stable/optim.html>`_.\n\n"
-      ]
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-images-idx3-ubyte.gz\n",
+      "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-images-idx3-ubyte.gz to data/FashionMNIST/raw/train-images-idx3-ubyte.gz\n"
+     ]
     },
     {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "collapsed": false
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "47918efb82854fc7a269ce73230391b0",
+       "version_major": 2,
+       "version_minor": 0
       },
-      "outputs": [],
-      "source": [
-        "loss_fn = nn.CrossEntropyLoss()\noptimizer = torch.optim.SGD(model.parameters(), lr=1e-3)"
+      "text/plain": [
+       "  0%|          | 0/26421880 [00:00<?, ?it/s]"
       ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
     },
     {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "In a single training loop, the model makes predictions on the training dataset (fed to it in batches), and\nbackpropagates the prediction error to adjust the model's parameters.\n\n"
-      ]
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Extracting data/FashionMNIST/raw/train-images-idx3-ubyte.gz to data/FashionMNIST/raw\n",
+      "\n",
+      "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-labels-idx1-ubyte.gz\n",
+      "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-labels-idx1-ubyte.gz to data/FashionMNIST/raw/train-labels-idx1-ubyte.gz\n"
+     ]
     },
     {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "collapsed": false
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "9abecd52d9144d53bd028f14a2cfd60b",
+       "version_major": 2,
+       "version_minor": 0
       },
-      "outputs": [],
-      "source": [
-        "def train(dataloader, model, loss_fn, optimizer):\n    size = len(dataloader.dataset)\n    for batch, (X, y) in enumerate(dataloader):\n        X, y = X.to(device), y.to(device)\n\n        # Compute prediction error\n        pred = model(X)\n        loss = loss_fn(pred, y)\n\n        # Backpropagation\n        optimizer.zero_grad()\n        loss.backward()\n        optimizer.step()\n\n        if batch % 100 == 0:\n            loss, current = loss.item(), batch * len(X)\n            print(f\"loss: {loss:>7f}  [{current:>5d}/{size:>5d}]\")"
+      "text/plain": [
+       "  0%|          | 0/29515 [00:00<?, ?it/s]"
       ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
     },
     {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "We also check the model's performance against the test dataset to ensure it is learning.\n\n"
-      ]
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Extracting data/FashionMNIST/raw/train-labels-idx1-ubyte.gz to data/FashionMNIST/raw\n",
+      "\n",
+      "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-images-idx3-ubyte.gz\n",
+      "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-images-idx3-ubyte.gz to data/FashionMNIST/raw/t10k-images-idx3-ubyte.gz\n"
+     ]
     },
     {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "collapsed": false
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "df61f428b0c44a818d2ab0f64420d9b3",
+       "version_major": 2,
+       "version_minor": 0
       },
-      "outputs": [],
-      "source": [
-        "def test(dataloader, model, loss_fn):\n    size = len(dataloader.dataset)\n    num_batches = len(dataloader)\n    model.eval()\n    test_loss, correct = 0, 0\n    with torch.no_grad():\n        for X, y in dataloader:\n            X, y = X.to(device), y.to(device)\n            pred = model(X)\n            test_loss += loss_fn(pred, y).item()\n            correct += (pred.argmax(1) == y).type(torch.float).sum().item()\n    test_loss /= num_batches\n    correct /= size\n    print(f\"Test Error: \\n Accuracy: {(100*correct):>0.1f}%, Avg loss: {test_loss:>8f} \\n\")"
+      "text/plain": [
+       "  0%|          | 0/4422102 [00:00<?, ?it/s]"
       ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
     },
     {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "The training process is conducted over several iterations (*epochs*). During each epoch, the model learns\nparameters to make better predictions. We print the model's accuracy and loss at each epoch; we'd like to see the\naccuracy increase and the loss decrease with every epoch.\n\n"
-      ]
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Extracting data/FashionMNIST/raw/t10k-images-idx3-ubyte.gz to data/FashionMNIST/raw\n",
+      "\n",
+      "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-labels-idx1-ubyte.gz\n",
+      "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-labels-idx1-ubyte.gz to data/FashionMNIST/raw/t10k-labels-idx1-ubyte.gz\n"
+     ]
     },
     {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "collapsed": false
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "418ca86b3df24c84979a54ca66cebe56",
+       "version_major": 2,
+       "version_minor": 0
       },
-      "outputs": [],
-      "source": [
-        "epochs = 5\nfor t in range(epochs):\n    print(f\"Epoch {t+1}\\n-------------------------------\")\n    train(train_dataloader, model, loss_fn, optimizer)\n    test(test_dataloader, model, loss_fn)\nprint(\"Done!\")"
+      "text/plain": [
+       "  0%|          | 0/5148 [00:00<?, ?it/s]"
       ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
     },
     {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "Read more about `Training your model <optimization_tutorial.html>`_.\n\n\n"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "--------------\n\n\n"
-      ]
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Extracting data/FashionMNIST/raw/t10k-labels-idx1-ubyte.gz to data/FashionMNIST/raw\n",
+      "\n"
+     ]
     },
     {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "Saving Models\n-------------\nA common way to save a model is to serialize the internal state dictionary (containing the model parameters).\n\n"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "collapsed": false
-      },
-      "outputs": [],
-      "source": [
-        "torch.save(model.state_dict(), \"model.pth\")\nprint(\"Saved PyTorch Model State to model.pth\")"
-      ]
-    },
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/home/ta180m/.local/lib/python3.9/site-packages/torchvision/datasets/mnist.py:498: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  /build/python-pytorch/src/pytorch-1.9.0-opt/torch/csrc/utils/tensor_numpy.cpp:174.)\n",
+      "  return torch.from_numpy(parsed.astype(m[2], copy=False)).view(*s)\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Download training data from open datasets.\n",
+    "training_data = datasets.FashionMNIST(\n",
+    "    root=\"data\",\n",
+    "    train=True,\n",
+    "    download=True,\n",
+    "    transform=ToTensor(),\n",
+    ")\n",
+    "\n",
+    "# Download test data from open datasets.\n",
+    "test_data = datasets.FashionMNIST(\n",
+    "    root=\"data\",\n",
+    "    train=False,\n",
+    "    download=True,\n",
+    "    transform=ToTensor(),\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We pass the ``Dataset`` as an argument to ``DataLoader``. This wraps an iterable over our dataset, and supports\n",
+    "automatic batching, sampling, shuffling and multiprocess data loading. Here we define a batch size of 64, i.e. each element\n",
+    "in the dataloader iterable will return a batch of 64 features and labels.\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [
     {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "Loading Models\n----------------------------\n\nThe process for loading a model includes re-creating the model structure and loading\nthe state dictionary into it.\n\n"
-      ]
-    },
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Shape of X [N, C, H, W]:  torch.Size([64, 1, 28, 28])\n",
+      "Shape of y:  torch.Size([64]) torch.int64\n"
+     ]
+    }
+   ],
+   "source": [
+    "batch_size = 64\n",
+    "\n",
+    "# Create data loaders.\n",
+    "train_dataloader = DataLoader(training_data, batch_size=batch_size)\n",
+    "test_dataloader = DataLoader(test_data, batch_size=batch_size)\n",
+    "\n",
+    "for X, y in test_dataloader:\n",
+    "    print(\"Shape of X [N, C, H, W]: \", X.shape)\n",
+    "    print(\"Shape of y: \", y.shape, y.dtype)\n",
+    "    break"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Read more about `loading data in PyTorch <data_tutorial.html>`_.\n",
+    "\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "--------------\n",
+    "\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Creating Models\n",
+    "------------------\n",
+    "To define a neural network in PyTorch, we create a class that inherits\n",
+    "from `nn.Module <https://pytorch.org/docs/stable/generated/torch.nn.Module.html>`_. We define the layers of the network\n",
+    "in the ``__init__`` function and specify how data will pass through the network in the ``forward`` function. To accelerate\n",
+    "operations in the neural network, we move it to the GPU if available.\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [
     {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "collapsed": false
-      },
-      "outputs": [],
-      "source": [
-        "model = NeuralNetwork()\nmodel.load_state_dict(torch.load(\"model.pth\"))"
-      ]
-    },
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Using cpu device\n",
+      "NeuralNetwork(\n",
+      "  (flatten): Flatten(start_dim=1, end_dim=-1)\n",
+      "  (linear_relu_stack): Sequential(\n",
+      "    (0): Linear(in_features=784, out_features=512, bias=True)\n",
+      "    (1): ReLU()\n",
+      "    (2): Linear(in_features=512, out_features=512, bias=True)\n",
+      "    (3): ReLU()\n",
+      "    (4): Linear(in_features=512, out_features=10, bias=True)\n",
+      "    (5): ReLU()\n",
+      "  )\n",
+      ")\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Get cpu or gpu device for training.\n",
+    "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
+    "print(\"Using {} device\".format(device))\n",
+    "\n",
+    "# Define model\n",
+    "class NeuralNetwork(nn.Module):\n",
+    "    def __init__(self):\n",
+    "        super(NeuralNetwork, self).__init__()\n",
+    "        self.flatten = nn.Flatten()\n",
+    "        self.linear_relu_stack = nn.Sequential(\n",
+    "            nn.Linear(28*28, 512),\n",
+    "            nn.ReLU(),\n",
+    "            nn.Linear(512, 512),\n",
+    "            nn.ReLU(),\n",
+    "            nn.Linear(512, 10),\n",
+    "            nn.ReLU()\n",
+    "        )\n",
+    "\n",
+    "    def forward(self, x):\n",
+    "        x = self.flatten(x)\n",
+    "        logits = self.linear_relu_stack(x)\n",
+    "        return logits\n",
+    "\n",
+    "model = NeuralNetwork().to(device)\n",
+    "print(model)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Read more about `building neural networks in PyTorch <buildmodel_tutorial.html>`_.\n",
+    "\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "--------------\n",
+    "\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Optimizing the Model Parameters\n",
+    "----------------------------------------\n",
+    "To train a model, we need a `loss function <https://pytorch.org/docs/stable/nn.html#loss-functions>`_\n",
+    "and an `optimizer <https://pytorch.org/docs/stable/optim.html>`_.\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [],
+   "source": [
+    "loss_fn = nn.CrossEntropyLoss()\n",
+    "optimizer = torch.optim.SGD(model.parameters(), lr=1e-3)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In a single training loop, the model makes predictions on the training dataset (fed to it in batches), and\n",
+    "backpropagates the prediction error to adjust the model's parameters.\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [],
+   "source": [
+    "def train(dataloader, model, loss_fn, optimizer):\n",
+    "    size = len(dataloader.dataset)\n",
+    "    for batch, (X, y) in enumerate(dataloader):\n",
+    "        X, y = X.to(device), y.to(device)\n",
+    "\n",
+    "        # Compute prediction error\n",
+    "        pred = model(X)\n",
+    "        loss = loss_fn(pred, y)\n",
+    "\n",
+    "        # Backpropagation\n",
+    "        optimizer.zero_grad()\n",
+    "        loss.backward()\n",
+    "        optimizer.step()\n",
+    "\n",
+    "        if batch % 100 == 0:\n",
+    "            loss, current = loss.item(), batch * len(X)\n",
+    "            print(f\"loss: {loss:>7f}  [{current:>5d}/{size:>5d}]\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We also check the model's performance against the test dataset to ensure it is learning.\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [],
+   "source": [
+    "def test(dataloader, model, loss_fn):\n",
+    "    size = len(dataloader.dataset)\n",
+    "    num_batches = len(dataloader)\n",
+    "    model.eval()\n",
+    "    test_loss, correct = 0, 0\n",
+    "    with torch.no_grad():\n",
+    "        for X, y in dataloader:\n",
+    "            X, y = X.to(device), y.to(device)\n",
+    "            pred = model(X)\n",
+    "            test_loss += loss_fn(pred, y).item()\n",
+    "            correct += (pred.argmax(1) == y).type(torch.float).sum().item()\n",
+    "    test_loss /= num_batches\n",
+    "    correct /= size\n",
+    "    print(f\"Test Error: \\n Accuracy: {(100*correct):>0.1f}%, Avg loss: {test_loss:>8f} \\n\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The training process is conducted over several iterations (*epochs*). During each epoch, the model learns\n",
+    "parameters to make better predictions. We print the model's accuracy and loss at each epoch; we'd like to see the\n",
+    "accuracy increase and the loss decrease with every epoch.\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [
     {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "This model can now be used to make predictions.\n\n"
-      ]
-    },
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 1\n",
+      "-------------------------------\n",
+      "loss: 1.758146  [    0/60000]\n",
+      "loss: 1.820034  [ 6400/60000]\n",
+      "loss: 1.846449  [12800/60000]\n",
+      "loss: 1.975245  [19200/60000]\n",
+      "loss: 1.612495  [25600/60000]\n",
+      "loss: 1.748993  [32000/60000]\n",
+      "loss: 1.628008  [38400/60000]\n",
+      "loss: 1.655061  [44800/60000]\n",
+      "loss: 1.770255  [51200/60000]\n",
+      "loss: 1.654287  [57600/60000]\n",
+      "Test Error: \n",
+      " Accuracy: 37.7%, Avg loss: 1.749445 \n",
+      "\n",
+      "Epoch 2\n",
+      "-------------------------------\n",
+      "loss: 1.670408  [    0/60000]\n",
+      "loss: 1.743051  [ 6400/60000]\n",
+      "loss: 1.773547  [12800/60000]\n",
+      "loss: 1.924395  [19200/60000]\n",
+      "loss: 1.529726  [25600/60000]\n",
+      "loss: 1.692361  [32000/60000]\n",
+      "loss: 1.559834  [38400/60000]\n",
+      "loss: 1.593531  [44800/60000]\n",
+      "loss: 1.712157  [51200/60000]\n",
+      "loss: 1.605115  [57600/60000]\n",
+      "Test Error: \n",
+      " Accuracy: 38.1%, Avg loss: 1.694516 \n",
+      "\n",
+      "Epoch 3\n",
+      "-------------------------------\n",
+      "loss: 1.607648  [    0/60000]\n",
+      "loss: 1.684907  [ 6400/60000]\n",
+      "loss: 1.716139  [12800/60000]\n",
+      "loss: 1.888849  [19200/60000]\n",
+      "loss: 1.474264  [25600/60000]\n",
+      "loss: 1.652733  [32000/60000]\n",
+      "loss: 1.514825  [38400/60000]\n",
+      "loss: 1.549373  [44800/60000]\n",
+      "loss: 1.670293  [51200/60000]\n",
+      "loss: 1.571395  [57600/60000]\n",
+      "Test Error: \n",
+      " Accuracy: 39.0%, Avg loss: 1.653676 \n",
+      "\n",
+      "Epoch 4\n",
+      "-------------------------------\n",
+      "loss: 1.561757  [    0/60000]\n",
+      "loss: 1.640771  [ 6400/60000]\n",
+      "loss: 1.669458  [12800/60000]\n",
+      "loss: 1.862879  [19200/60000]\n",
+      "loss: 1.435348  [25600/60000]\n",
+      "loss: 1.623189  [32000/60000]\n",
+      "loss: 1.482370  [38400/60000]\n",
+      "loss: 1.515045  [44800/60000]\n",
+      "loss: 1.638349  [51200/60000]\n",
+      "loss: 1.545919  [57600/60000]\n",
+      "Test Error: \n",
+      " Accuracy: 39.9%, Avg loss: 1.621615 \n",
+      "\n",
+      "Epoch 5\n",
+      "-------------------------------\n",
+      "loss: 1.525517  [    0/60000]\n",
+      "loss: 1.604991  [ 6400/60000]\n",
+      "loss: 1.630397  [12800/60000]\n",
+      "loss: 1.841878  [19200/60000]\n",
+      "loss: 1.406707  [25600/60000]\n",
+      "loss: 1.599460  [32000/60000]\n",
+      "loss: 1.456716  [38400/60000]\n",
+      "loss: 1.485950  [44800/60000]\n",
+      "loss: 1.612476  [51200/60000]\n",
+      "loss: 1.525381  [57600/60000]\n",
+      "Test Error: \n",
+      " Accuracy: 40.7%, Avg loss: 1.595456 \n",
+      "\n",
+      "Done!\n"
+     ]
+    }
+   ],
+   "source": [
+    "epochs = 5\n",
+    "for t in range(epochs):\n",
+    "    print(f\"Epoch {t+1}\\n-------------------------------\")\n",
+    "    train(train_dataloader, model, loss_fn, optimizer)\n",
+    "    test(test_dataloader, model, loss_fn)\n",
+    "print(\"Done!\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Read more about `Training your model <optimization_tutorial.html>`_.\n",
+    "\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "--------------\n",
+    "\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Saving Models\n",
+    "-------------\n",
+    "A common way to save a model is to serialize the internal state dictionary (containing the model parameters).\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [
     {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "collapsed": false
-      },
-      "outputs": [],
-      "source": [
-        "classes = [\n    \"T-shirt/top\",\n    \"Trouser\",\n    \"Pullover\",\n    \"Dress\",\n    \"Coat\",\n    \"Sandal\",\n    \"Shirt\",\n    \"Sneaker\",\n    \"Bag\",\n    \"Ankle boot\",\n]\n\nmodel.eval()\nx, y = test_data[0][0], test_data[0][1]\nwith torch.no_grad():\n    pred = model(x)\n    predicted, actual = classes[pred[0].argmax(0)], classes[y]\n    print(f'Predicted: \"{predicted}\", Actual: \"{actual}\"')"
-      ]
-    },
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Saved PyTorch Model State to model.pth\n"
+     ]
+    }
+   ],
+   "source": [
+    "torch.save(model.state_dict(), \"model.pth\")\n",
+    "print(\"Saved PyTorch Model State to model.pth\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Loading Models\n",
+    "----------------------------\n",
+    "\n",
+    "The process for loading a model includes re-creating the model structure and loading\n",
+    "the state dictionary into it.\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [
     {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "Read more about `Saving & Loading your model <saveloadrun_tutorial.html>`_.\n\n\n"
+     "data": {
+      "text/plain": [
+       "<All keys matched successfully>"
       ]
+     },
+     "execution_count": 13,
+     "metadata": {},
+     "output_type": "execute_result"
     }
-  ],
-  "metadata": {
-    "kernelspec": {
-      "display_name": "Python 3",
-      "language": "python",
-      "name": "python3"
-    },
-    "language_info": {
-      "codemirror_mode": {
-        "name": "ipython",
-        "version": 3
-      },
-      "file_extension": ".py",
-      "mimetype": "text/x-python",
-      "name": "python",
-      "nbconvert_exporter": "python",
-      "pygments_lexer": "ipython3",
-      "version": "3.6.13"
+   ],
+   "source": [
+    "model = NeuralNetwork()\n",
+    "model.load_state_dict(torch.load(\"model.pth\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This model can now be used to make predictions.\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
     }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Predicted: \"Ankle boot\", Actual: \"Ankle boot\"\n"
+     ]
+    }
+   ],
+   "source": [
+    "classes = [\n",
+    "    \"T-shirt/top\",\n",
+    "    \"Trouser\",\n",
+    "    \"Pullover\",\n",
+    "    \"Dress\",\n",
+    "    \"Coat\",\n",
+    "    \"Sandal\",\n",
+    "    \"Shirt\",\n",
+    "    \"Sneaker\",\n",
+    "    \"Bag\",\n",
+    "    \"Ankle boot\",\n",
+    "]\n",
+    "\n",
+    "model.eval()\n",
+    "x, y = test_data[0][0], test_data[0][1]\n",
+    "with torch.no_grad():\n",
+    "    pred = model(x)\n",
+    "    predicted, actual = classes[pred[0].argmax(0)], classes[y]\n",
+    "    print(f'Predicted: \"{predicted}\", Actual: \"{actual}\"')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Read more about `Saving & Loading your model <saveloadrun_tutorial.html>`_.\n",
+    "\n",
+    "\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
   },
-  "nbformat": 4,
-  "nbformat_minor": 0
-}
\ No newline at end of file
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
diff --git a/.ipynb_checkpoints/test-checkpoint.ipynb b/.ipynb_checkpoints/test-checkpoint.ipynb
index 363fcab..3312e79 100644
--- a/.ipynb_checkpoints/test-checkpoint.ipynb
+++ b/.ipynb_checkpoints/test-checkpoint.ipynb
@@ -1,6 +1,78 @@
 {
- "cells": [],
- "metadata": {},
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "5a2a1e37-f4db-4963-a0ac-7b39353f826f",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "tensor([[0.2226, 0.2227, 0.1347],\n",
+      "        [0.7975, 0.3823, 0.9395],\n",
+      "        [0.1675, 0.9185, 0.4175],\n",
+      "        [0.0476, 0.1244, 0.6878],\n",
+      "        [0.5177, 0.7245, 0.2723]])\n"
+     ]
+    }
+   ],
+   "source": [
+    "import torch\n",
+    "x = torch.rand(5, 3)\n",
+    "print(x)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "53469c52-9438-4d62-aa2a-e93fb3fef23b",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "False"
+      ]
+     },
+     "execution_count": 2,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "torch.cuda.is_available()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f13c7010-acb0-4870-aa6f-1294701cfc7e",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.6"
+  }
+ },
  "nbformat": 4,
  "nbformat_minor": 5
 }