Senior GPU Performance Software Engineer, AI/ML

Overview

In this role you help shape LLM training and serving performance at Google scale. You will define and maintain benchmarks reflecting production workloads, and collaborate with product teams to address performance bottlenecks in GPU-based ML systems. You’ll run simulations and roofline analyses to guide hardware and compiler optimizations, ensuring efficient training and inference for large-scale models. You will enable cross-team initiatives to push faster, more cost-efficient AI at scale, anchored in measurable performance improvements.

Responsibilities

• Identify and maintain LLM benchmarks representative of production and industry benchmarks
• Engage with product teams (e.g., DeepMind) to onboard new LLM models and optimize GPU-based training and serving
• Run architecture-level simulations on GPU designs and perform roofline analysis
• Analyze performance and efficiency metrics to identify bottlenecks and implement fleet-wide improvements
• Conduct performance benchmarks on GPU hardware using internal and external tools

Key requirements

• Bachelor’s degree or equivalent practical experience
• 5 years of experience with software development in one or more programming languages
• 5 years of experience with data structures and algorithms
• 5 years of experience with machine learning algorithms and tools, AI, deep learning, or NLP
• 3 years of experience testing, maintaining, or launching software products
• 1 year of experience with software design and architecture

• leadership qualities
• problem-solving
• cross-team collaboration
• CUDA
• Triton kernels
• GPU programming

Full job description from WhatJobs. Apply for this role →