Machine Learning Engineer, Zoox

• Master's degree or PhD in computer science or related field
• Experience with Vision Language Model or multi-modal 3D foundation model
• Experience with Python and/or C++
• Experience with model deployment with TensorRT
• Proficient in PyTorch model design, implementation, training and evaluation

Desirable

• The Perception team at Zoox is fundamental to our autonomous vehicle technology, creating the understanding of the world for our self-driving robots
• We enable safe and efficient navigation in complex environments through sophisticated detection, classification, and tracking systems
• As a member of the Detection, you'll be responsible for helping build sensor fusion models for detecting agents around the robot
• You will work on building the vision backbone of the detector by designing sensor fusion models for Zoox's sensor data
• You will work on incorporating SOTA language alignment strategies into the per-sensor modality backbones and help Zoox advance faster into new geofences by making the models more generalizable and adaptable to new circumstances
• Train and upgrade current image backbone by performing various studies into incorporating SOTA approaches to be multi-frame, even higher resolution and using large foundational, language-aligned feature encoder
• Deploy temporal image backbone into on-vehicle inference architecture and optimize runtime compute resource
• Incorporate temporal information into the the existing Zoox’s multi-sensor (Vision, Lidar and Radar) early fusion system to predict object velocity and scene flow
• Develop effective metrics and evaluating algorithm performance with these metrics to demonstrate improvement
• Collaborate with teams such as ML Optimization, Data Labeling, V&V/Metrics and Planner/Prediction to ensure project is successful for Zoox releases