Software Engineer - GPU Kernels

Baseten powers mission-critical inference for AI companies like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma, and Writer. They recently raised a $1.5B Series F led by Altimeter Capital, Conviction Partners, and Spark Capital.

As a GPU Kernel Engineer on the Model Performance team, you will design and implement high-performance GPU kernels for key ML operations, including matrix multiplications, attention mechanisms, and mixture-of-experts routing. You will write and optimize code using CUDA, PTX assembly, and architecture-specific techniques, applying methods like memory coalescing, warp-level programming, tensor core acceleration, and compute/memory overlap. You will implement quantization (FP8/FP4), sparsity, and compute/communication overlap, and identify bottlenecks using Nsight Systems, Nsight Compute, and Torch Profiler. You will collaborate with research teams and contribute to internal and open-source GPU libraries, with opportunities to present at conferences like NVIDIA GTC and AWS re:Invent.

• Strong understanding of GPU architecture: memory hierarchy, thread/block/grid organization, synchronization, race condition mitigation
• Proficiency in C++ and GPU performance profiling tools
• Knowledge of CUDA C++ API, memory access patterns, numerical precision, quantization, and modern GPU features (tensor cores, async operations)
• Nice to have: experience with Transformer models, attention optimization (e.g., Flash Attention), GPU kernel libraries (Cutlass, Triton, Thrust, CUB), GEMM tuning, distributed/multi-GPU compute, open-source GPU projects, research publications

Salary range $180K – $360K, with equity offered. Benefits include 100% coverage of medical, dental, and vision insurance for employee and dependents, flexible PTO with company-wide Winter Break, paid parental leave, fertility and family-building stipend (U.S. only), company-facilitated 401(k), and exposure to ML startups.