Interview questions

How Context Window Length Shapes LLM Quality, Cost and Latency

Companies: OpenAI

AI Product Management · Transformers & Attention

Last reported Apr 2026

Debug MiniGPT and Backpropagate Matmul

Companies: OpenAI

ML Coding · Transformers & Attention

Last reported Apr 2026

Debug a Broken Transformer

Companies: OpenAI

ML Coding · Transformers & Attention

Last reported Mar 2026

Debug Transformer and Add KV Cache

Companies: OpenAI

ML Coding · Transformers & Attention

Last reported Feb 2026

Debug a broken Transformer implementation

Companies: OpenAI

ML Coding · Transformers & Attention

Last reported Jan 2026

Explain KV cache in Transformer inference

Companies: OpenAI

LLM Inference & Serving · Transformers & Attention

Last reported Jan 2026

Debug and fix a PyTorch Transformer training loop

Companies: OpenAI

ML Coding · Transformers & Attention

Last reported Oct 2025

Implement and analyze custom attention

Companies: Anthropic

ML Coding · Transformers & Attention

Last reported Sep 2025

Implement KV cache for inference

Companies: Applied Intuition

LLM Inference & Serving · ML System Design · Transformers & Attention

Last reported Sep 2025

Analyze attention complexity and improvements

Companies: Amazon

Transformers & Attention

Answer an LLM Systems Quick-Fire Interview

Companies: Mistral AI

Fine-Tuning & Alignment Techniques · Transformers & Attention

Compare NLP tokenization and LLM recommendations

Companies: Google

LLM Applications · Transformers & Attention

Explain FlashAttention, KV cache, and RoPE

Companies: TikTok

LLM Inference & Serving · Transformers & Attention

Explain Layer Normalization in Transformers

Companies: Amazon

Neural Networks & Training · Transformers & Attention

Explain Transformer and Fine-Tuning Basics

Companies: Lyft

Fine-Tuning & Alignment Techniques · Transformers & Attention

Explain Transformer and MoE Fundamentals

Companies: Amazon

Transformers & Attention

Explain Transformer Attention Fundamentals

Companies: Adobe

LLM Inference & Serving · Transformers & Attention

Explain Transformer Internals and Implement Top-p Decoding

Companies: Mistral AI

ML Coding · Transformers & Attention

Implement a Transformer Decoder from Scratch Without Starter Code

Companies: ByteDance

ML Coding · Transformers & Attention

Implement Grouped-Query Attention (GQA)

Companies: Datadog

ML Coding · Transformers & Attention

Implement Masked Multi-Head Self-Attention

Companies: Apple

ML Coding · Transformers & Attention

Implement Multi-Head Attention from Scratch

Companies: ByteDance

ML Coding · Transformers & Attention