Data Infrastructure Engineer, Anthropic

• 5+ YOE outside of internships
• Strong software engineering skills with experience building high-throughput fault-tolerant distributed systems
• Hands-on experience with distributed computing frameworks, particularly Apache Spark
• Excellent problem-solving skills and attention to detail
• Strong communication skills and ability to work in a collaborative environment
• Advanced degree in Computer Science or related field
• Experience with language model training infrastructure
• Background in Data Infrastructure, MLOps, or ML infrastructure
• Minimum education: Bachelor’s degree or an equivalent combination of education, training, and/or experience
• Required field of study: A field relevant to the role as demonstrated through coursework, training, or professional experience
• Minimum years of experience: Years of experience required will correlate with the internal job level requirements for the position
• We encourage you to apply even if you do not believe you meet every single qualification

Desirable

• Anthropic is at the forefront of AI research, dedicated to developing safe, ethical, and powerful artificial intelligence
• Our mission is to ensure that transformative AI systems are aligned with human interests
• We are seeking a Staff level Engineer to join our Pre-training team, responsible for developing the next generation of large language models
• In this role, you will work at the intersection of cutting-edge research and practical engineering, contributing to the development of safe, steerable, and trustworthy AI systems
• Design and implement data processing infrastructure for large language model training (highly performant, reproducible, traceable)
• Develop and maintain core processing primitives (e.g., tokenization, deduplication, chunking) with a focus on scalability
• Build robust systems for data quality assurance and validation at scale
• Collaborate with research teams to implement novel data processing architectures
• Build and operate end-to-end data pipelines that turn raw web-scale corpora into training-ready datasets
• Sample Projects:
• Designing and implementing distributed computing architecture for web-scale data processing
• Building scalable infrastructure for model training data preparation
• Developing fault-tolerant distributed processing systems
• Implementing new infrastructure components based on research requirements