Controllable Video Generation Researcher

Overview

In this role you advance multimodal, controllable video generation for games, collaborating with engine teams to define interfaces and optimize for low-latency, multi-player control. You contribute to research on high-fidelity generation with sparse signals and conditional control paradigms, shaping next-gen technologies for Lightspeed Studios. You will work across trajectory, camera, motion, and posture control to enable responsive, collaborative gameplay. This position offers a chance to impact how AI-driven visuals and control systems integrate with large-scale game development.

Responsibilities

• Design a multimodal controllable generation framework supporting inputs such as keyboard, controller, trajectory, and voice
• Research high-fidelity generation under sparse control signals and extend conditional control paradigms in video
• Decouple and combine trajectory, camera, motion, and posture control
• Develop dedicated control modules for different game genres
• Build video generation capabilities for multi-player collaborative control with low latency and inter-frame state retention
• Collaborate with Engine/Agent teams to define standardized controllable signal interfaces
• Drive joint optimization of controllable generation and Reinforcement Learning post-training

Key requirements

• Ph.D. in AI-related fields focusing on Computer Vision/Deep Learning
• Proficient in conditional generation models and multi-player/multi-channel control technologies; familiar with diffusion models
• Project experience in camera control, trajectory control, or posture control
• Understanding of game engine input/output logic and frame synchronization mechanisms
• Proficient in Python/PyTorch; publications in controllable video generation or conditional generation
• Experience in Game AI or game development is preferred
• Experience in real-time/interactive video generation projects is preferred

• collaboration with cross-functional teams
• strong problem-solving and experimental mindset
• clear technical communication
• conditional generation models
• diffusion models
• multi-player/multi-channel control technologies

Full job description from WhatJobs. Apply for this role →