updates

content/_index.md

@@ -5,6 +5,6 @@ draft: false
 
 # Nic Hoffs
 
-_Mostly_ deep learning stuff.
+_Mostly_ deep learning stuff. Contact me at <[email protected]>. Here's my [resume](/resume.pdf).
 
 Raw list of posts below. To see it organized categorically, check out [posts](/post/)

content/about.md

@@ -1,9 +1,9 @@
 ---
 title: "About"
-draft: false
 ---
 
 I'm a third-year student studying CS in the Engineering School at the University of Virginia.
 
-- Here's my [resume](https://google.com).
-- Here's my [linkedin](https://google.com).
+- <[email protected]>
+- Here's my [resume](/resume.pdf).
+- Here's my [linkedin](https://google.com) that I need to update.

content/post/mini_projects/xor.md

@@ -3,3 +3,129 @@ title: XOR
 type: post
 date: 2024-06-08
 ---
+
+The tiniest project -- teaching a neural net to XOR. 
+Obviously, I'm doing this in `tinygrad` because it's tiny.
+
+| A | B | A XOR B |
+|---|---|---------|
+| 0 | 0 |    0    |
+| 0 | 1 |    1    |
+| 1 | 0 |    1    |
+| 1 | 1 |    0    |
+
+First, imports.
+
+```python
+from tinygrad import Tensor, TinyJit
+from tinygrad.nn.optim import SGD
+import numpy as np
+from tqdm import tqdm, trange
+import matplotlib.pyplot as plt
+```
+
+Next, dataset. It's the same as the truth table.
+
+```python
+x = Tensor([[0,0],[0,1],[1,0],[1,1]])
+y = Tensor([[0],[1],[1],[0]])
+x.shape, y.shape
+# ((4, 2), (4, 1))
+```
+
+After that, let's define the network. It has two inputs, a hidden layer with two neurons(weight+bias), and an output. I'll also create a method `int_output` that will just return the outputs aas a list of integers.
+
+```python
+class XOR_Net:
+    def __init__(self):
+        self.W1 = Tensor.normal(2, 2, mean=0, std=1)
+        self.b1 = Tensor.randn(2)
+        self.W2 = Tensor.randn(2, 1)
+        self.b2 = Tensor.randn(1)
+
+    def __call__(self, x):
+        x = x.linear(self.W1, self.b1).sigmoid()
+        x = x.linear(self.W2, self.b2)
+        return x
+
+    def int_output(self, x):
+        return [int(i[0]) for i in self(x).round().numpy().tolist()]
+```
+
+Instantiate.
+
+```python
+model = XOR_Net()
+optim = SGD(get_parameters(model), lr=0.01, momentum=0.9)
+```
+
+Train.
+
+```python
+def train(model, x, y, optim, steps, allow_jit=True):
+
+  def train_step(x, y):
+    out = model(x)
+    loss = ((out - y) ** 2).mean()
+    optim.zero_grad()
+    loss.backward()
+    optim.step()
+    return loss.realize()
+
+  if allow_jit: train_step = TinyJit(train_step)
+
+  with Tensor.train():
+    losses, accuracies = [], []
+    for i in (t := trange(steps)):
+      loss = train_step(x, y).numpy()
+      losses.append(loss)
+      t.set_description("loss %.2f" % loss)
+  return losses
+```
+
+Time to train the model and see if we get to 0 loss as expected. We'll let it go for 7500 iterations.
+
+```python
+losses = train(model, x, y, optim, 7500)
+#loss 0.00: 100%|██████████| 7500/7500 [00:06<00:00, 1194.23it/s]
+```
+
+We can check if the model was correctly trained by running through all of x with `int_output` and seeing if it aligns with the truth table.
+
+```python
+model.int_output(x)
+#[0, 1, 1, 0]
+```
+
+Now let's observe the decision boundary of the model.
+
+```python
+def meshgrid(x, y):
+  grid_x = Tensor.cat(*[x[idx:idx+1].expand(y.shape).unsqueeze(0) for idx in range(x.shape[0])])
+  grid_y = Tensor.cat(*[y.unsqueeze(0)]*x.shape[0])
+  return Tensor.cat(grid_x.reshape(-1, 1), grid_y.reshape(-1, 1), dim=1)
+
+# Creating grid points in the input space
+x_grid = Tensor.arange(0,1,.01)
+y_grid = Tensor.arange(0,1,.01)
+grid_points = meshgrid(x_grid, y_grid)
+
+# Use the model to predict over the entire grid
+predictions = model.int_output(grid_points)
+
+# Convert the predictions to a numpy array and reshape for plotting
+predictions_array = np.array(predictions).reshape(100, 100)
+
+# Plotting
+plt.contourf(x_grid.numpy(), y_grid.numpy(), predictions_array, levels=[0, 0.5, 1], cmap='RdGy')
+plt.colorbar()
+plt.scatter([0, 1], [0, 1], color='red', label='Output 1')
+plt.scatter([0, 1], [1, 0], color='blue', label='Output 0')
+plt.title('XOR Decision Boundary')
+plt.xlabel('Input 1')
+plt.ylabel('Input 2')
+plt.legend()
+plt.show()
+```
+
+![boundary](/boundary.png)

hugo.toml

@@ -18,3 +18,6 @@ weight = 1
 name = "Posts"
 url ='/post/'
 weight = 2
+
+[params]
+  customCSS = ["css/custom.css"]

layouts/partials/foot_custom.html

@@ -2,3 +2,6 @@
 <script src="//cdn.jsdelivr.net/combine/npm/katex/dist/katex.min.js,npm/katex/dist/contrib/auto-render.min.js,npm/@xiee/utils/js/render-katex.js"
 	defer></script>
 <script src="//cdn.jsdelivr.net/npm/@xiee/utils/js/center-img.min.js" defer></script>
+<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/styles/default.min.css">
+<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/highlight.min.js"></script>
+<script>hljs.highlightAll();</script>

public/404.html

@@ -31,6 +31,9 @@
 <script src="//cdn.jsdelivr.net/combine/npm/katex/dist/katex.min.js,npm/katex/dist/contrib/auto-render.min.js,npm/@xiee/utils/js/render-katex.js"
 	defer></script>
 <script src="//cdn.jsdelivr.net/npm/@xiee/utils/js/center-img.min.js" defer></script>
+<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/styles/default.min.css">
+<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/highlight.min.js"></script>
+<script>hljs.highlightAll();</script>
 
 
   </footer>

public/about/index.html

@@ -32,8 +32,9 @@ <h1><span class="title">About</span></h1>
 <main>
 <p>I&rsquo;m a third-year student studying CS in the Engineering School at the University of Virginia.</p>
 <ul>
-<li>Here&rsquo;s my <a href="https://google.com">resume</a>.</li>
-<li>Here&rsquo;s my <a href="https://google.com">linkedin</a>.</li>
+<li><a href="mailto:[email protected]">[email protected]</a></li>
+<li>Here&rsquo;s my <a href="/resume.pdf">resume</a>.</li>
+<li>Here&rsquo;s my <a href="https://google.com">linkedin</a> that I need to update.</li>
 </ul>
 
 </main>
@@ -43,6 +44,9 @@ <h1><span class="title">About</span></h1>
 <script src="//cdn.jsdelivr.net/combine/npm/katex/dist/katex.min.js,npm/katex/dist/contrib/auto-render.min.js,npm/@xiee/utils/js/render-katex.js"
 	defer></script>
 <script src="//cdn.jsdelivr.net/npm/@xiee/utils/js/center-img.min.js" defer></script>
+<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/styles/default.min.css">
+<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/highlight.min.js"></script>
+<script>hljs.highlightAll();</script>
 
 
   </footer>

public/categories/index.html

@@ -35,6 +35,9 @@ <h1>Categories</h1>
 <script src="//cdn.jsdelivr.net/combine/npm/katex/dist/katex.min.js,npm/katex/dist/contrib/auto-render.min.js,npm/@xiee/utils/js/render-katex.js"
 	defer></script>
 <script src="//cdn.jsdelivr.net/npm/@xiee/utils/js/center-img.min.js" defer></script>
+<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/styles/default.min.css">
+<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/highlight.min.js"></script>
+<script>hljs.highlightAll();</script>
 
 
   </footer>

public/css/style.css

@@ -26,7 +26,6 @@ hr {
 pre {
   border: 1px solid #ddd;
   box-shadow: 5px 5px 5px #eee;
-  padding: 1em;
   overflow-x: auto;
 }
 code { background: #f9f9f9; }

public/index.html

@@ -28,7 +28,7 @@
 
 
 <h1 id="nic-hoffs">Nic Hoffs</h1>
-<p><em>Mostly</em> deep learning stuff.</p>
+<p><em>Mostly</em> deep learning stuff. Contact me at <a href="mailto:[email protected]">[email protected]</a>. Here&rsquo;s my <a href="/resume.pdf">resume</a>.</p>
 <p>Raw list of posts below. To see it organized categorically, check out <a href="/post/">posts</a></p>
 
 
@@ -56,12 +56,18 @@ <h1 id="nic-hoffs">Nic Hoffs</h1>
 <script src="//cdn.jsdelivr.net/combine/npm/katex/dist/katex.min.js,npm/katex/dist/contrib/auto-render.min.js,npm/@xiee/utils/js/render-katex.js"
 	defer></script>
 <script src="//cdn.jsdelivr.net/npm/@xiee/utils/js/center-img.min.js" defer></script>
+<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/styles/default.min.css">
+<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/highlight.min.js"></script>
+<script>hljs.highlightAll();</script>
 
   <footer>
   <link rel="stylesheet" href="//cdn.jsdelivr.net/npm/katex/dist/katex.min.css">
 <script src="//cdn.jsdelivr.net/combine/npm/katex/dist/katex.min.js,npm/katex/dist/contrib/auto-render.min.js,npm/@xiee/utils/js/render-katex.js"
 	defer></script>
 <script src="//cdn.jsdelivr.net/npm/@xiee/utils/js/center-img.min.js" defer></script>
+<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/styles/default.min.css">
+<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/highlight.min.js"></script>
+<script>hljs.highlightAll();</script>
 
 
   </footer>

public/index.xml

@@ -20,14 +20,14 @@
       <link>http://localhost:1313/post/mini_projects/xor/</link>
       <pubDate>Sat, 08 Jun 2024 00:00:00 +0000</pubDate>
       <guid>http://localhost:1313/post/mini_projects/xor/</guid>
-      <description></description>
+      <description>The tiniest project &amp;ndash; teaching a neural net to XOR. Obviously, I&amp;rsquo;m doing this in tinygrad because it&amp;rsquo;s tiny.&#xA;A B A XOR B 0 0 0 0 1 1 1 0 1 1 1 0 First, imports.&#xA;from tinygrad import Tensor, TinyJit from tinygrad.nn.optim import SGD import numpy as np from tqdm import tqdm, trange import matplotlib.pyplot as plt Next, dataset. It&amp;rsquo;s the same as the truth table.&#xA;x = Tensor([[0,0],[0,1],[1,0],[1,1]]) y = Tensor([[0],[1],[1],[0]]) x.</description>
     </item>
     <item>
       <title>About</title>
       <link>http://localhost:1313/about/</link>
       <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
       <guid>http://localhost:1313/about/</guid>
-      <description>I&amp;rsquo;m a third-year student studying CS in the Engineering School at the University of Virginia.&#xA;Here&amp;rsquo;s my resume. Here&amp;rsquo;s my linkedin. </description>
+      <description>I&amp;rsquo;m a third-year student studying CS in the Engineering School at the University of Virginia.&#xA;[email protected] Here&amp;rsquo;s my resume. Here&amp;rsquo;s my linkedin that I need to update. </description>
     </item>
   </channel>
 </rss>

public/post/grokking_squared/index.html

@@ -48,12 +48,18 @@ <h1>Grokking Squared</h1>
 <script src="//cdn.jsdelivr.net/combine/npm/katex/dist/katex.min.js,npm/katex/dist/contrib/auto-render.min.js,npm/@xiee/utils/js/render-katex.js"
 	defer></script>
 <script src="//cdn.jsdelivr.net/npm/@xiee/utils/js/center-img.min.js" defer></script>
+<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/styles/default.min.css">
+<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/highlight.min.js"></script>
+<script>hljs.highlightAll();</script>
 
   <footer>
   <link rel="stylesheet" href="//cdn.jsdelivr.net/npm/katex/dist/katex.min.css">
 <script src="//cdn.jsdelivr.net/combine/npm/katex/dist/katex.min.js,npm/katex/dist/contrib/auto-render.min.js,npm/@xiee/utils/js/render-katex.js"
 	defer></script>
 <script src="//cdn.jsdelivr.net/npm/@xiee/utils/js/center-img.min.js" defer></script>
+<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/styles/default.min.css">
+<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/highlight.min.js"></script>
+<script>hljs.highlightAll();</script>
 
 
   </footer>

public/post/grokking_squared/part1/index.html

@@ -39,6 +39,9 @@ <h2 class="date">2024/06/08</h2>
 <script src="//cdn.jsdelivr.net/combine/npm/katex/dist/katex.min.js,npm/katex/dist/contrib/auto-render.min.js,npm/@xiee/utils/js/render-katex.js"
 	defer></script>
 <script src="//cdn.jsdelivr.net/npm/@xiee/utils/js/center-img.min.js" defer></script>
+<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/styles/default.min.css">
+<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/highlight.min.js"></script>
+<script>hljs.highlightAll();</script>
 
 
   </footer>

public/post/index.html

@@ -50,12 +50,18 @@ <h1>Posts by Category</h1>
 <script src="//cdn.jsdelivr.net/combine/npm/katex/dist/katex.min.js,npm/katex/dist/contrib/auto-render.min.js,npm/@xiee/utils/js/render-katex.js"
 	defer></script>
 <script src="//cdn.jsdelivr.net/npm/@xiee/utils/js/center-img.min.js" defer></script>
+<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/styles/default.min.css">
+<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/highlight.min.js"></script>
+<script>hljs.highlightAll();</script>
 
   <footer>
   <link rel="stylesheet" href="//cdn.jsdelivr.net/npm/katex/dist/katex.min.css">
 <script src="//cdn.jsdelivr.net/combine/npm/katex/dist/katex.min.js,npm/katex/dist/contrib/auto-render.min.js,npm/@xiee/utils/js/render-katex.js"
 	defer></script>
 <script src="//cdn.jsdelivr.net/npm/@xiee/utils/js/center-img.min.js" defer></script>
+<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/styles/default.min.css">
+<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/highlight.min.js"></script>
+<script>hljs.highlightAll();</script>
 
 
   </footer>

public/post/mini_projects/index.html

@@ -48,12 +48,18 @@ <h1>Mini-Projects</h1>
 <script src="//cdn.jsdelivr.net/combine/npm/katex/dist/katex.min.js,npm/katex/dist/contrib/auto-render.min.js,npm/@xiee/utils/js/render-katex.js"
 	defer></script>
 <script src="//cdn.jsdelivr.net/npm/@xiee/utils/js/center-img.min.js" defer></script>
+<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/styles/default.min.css">
+<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/highlight.min.js"></script>
+<script>hljs.highlightAll();</script>
 
   <footer>
   <link rel="stylesheet" href="//cdn.jsdelivr.net/npm/katex/dist/katex.min.css">
 <script src="//cdn.jsdelivr.net/combine/npm/katex/dist/katex.min.js,npm/katex/dist/contrib/auto-render.min.js,npm/@xiee/utils/js/render-katex.js"
 	defer></script>
 <script src="//cdn.jsdelivr.net/npm/@xiee/utils/js/center-img.min.js" defer></script>
+<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/styles/default.min.css">
+<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/highlight.min.js"></script>
+<script>hljs.highlightAll();</script>
 
 
   </footer>

public/post/mini_projects/index.xml

@@ -13,7 +13,7 @@
       <link>http://localhost:1313/post/mini_projects/xor/</link>
       <pubDate>Sat, 08 Jun 2024 00:00:00 +0000</pubDate>
       <guid>http://localhost:1313/post/mini_projects/xor/</guid>
-      <description></description>
+      <description>The tiniest project &amp;ndash; teaching a neural net to XOR. Obviously, I&amp;rsquo;m doing this in tinygrad because it&amp;rsquo;s tiny.&#xA;A B A XOR B 0 0 0 0 1 1 1 0 1 1 1 0 First, imports.&#xA;from tinygrad import Tensor, TinyJit from tinygrad.nn.optim import SGD import numpy as np from tqdm import tqdm, trange import matplotlib.pyplot as plt Next, dataset. It&amp;rsquo;s the same as the truth table.&#xA;x = Tensor([[0,0],[0,1],[1,0],[1,1]]) y = Tensor([[0],[1],[1],[0]]) x.</description>
     </item>
   </channel>
 </rss>