From Transformers to GPT

Building GPT using the plain transformer as the central module.

Transformer Architecture

(Multi-head) attention + the basic transformer architecture.

Convolutional Neural Nets

Implementing a conv2d layer from scratch.

Autograd

How does autograd work?

Cross-Entropy Loss

Justifying cross-entropy loss.

Eigenvectors & Systems of ODEs

How to solve a system of ODEs with eigenvectors, and why eigenvectors are even relevant.

Simple huffman coding tree visualizer.

Neural Network Derivation

Forward and backprop in an FNN.

Coordinate Matrix

Finding the coordinate matrix of a linear map.

Rank-Nullity Theorem

Rank-Nullity through the lens of injectivity.

Nullspace

A more intuitive view on the nullspace.

Discrete Fourier Transform

Along with its applications in drawing images.

Try and fit functions (ℝ^2 -> ℝ) with your own hand-crafted neural networks.