In June 2025, I successfully defended my master thesis, which provides a mathematical description on Training Large Language Models. While most resources online omit gradient derivation, my thesis aims to provide a complete description. Alongside, a Python implementation is provided making use of AutoCompyute.

AutoCompyute is a deep learning library that provides automatic differentiation using only NumPy as the backend for computation (CuPy can be used as a drop-in replacement for NumPy). It is designed for simplicity and performance and enables you to train deep learning models with minimal dependencies while leveraging GPU acceleration.

An Implemenentation of the paper An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale in PyTorch.