Articles
By Vladislav KruglikovHere's a list of my articles:
- Flash Attention
- Critical Batch Size
- Diminishing Returns Law
- Fusing
- Memory Fragmentation
- Mixed precision
- Semantics of a Training Loop
- Activation checkpointing
- Parallelisms
- Architectures
- Positional Embeddings
- Curse of Dimensionality
- Gradient Clipping
- How To Debug Training
- Learning Rate as Loss Landscape Navigation
- Normalizations
- The Shape of Loss
- Weight Initialization
- How Much Should One Model Update Know
- Anatomy of GPU Performance
- Optimizers
- How to Work with Research Papers
- Tokenization
- How to Create Better Pull Requests
- How to Create Better Tickets
- Information Theory
- Knowledge Distillation
- Learning Rate Annealing
- Learning Rate Warmup
- PEFT
- Backpropagation
- Load Testing Language Models
- CUDA Software Stack
- Speculative Decoding
- Why Batch Size Changes Outputs
- KV Cache Pruning (Part 1)
- KV Cache Pruning (Part 2)