Intermediate Machine Learning Engineer, GitLab
AI Framework
+ Equity Compensation + Employee Stock Purchase Plan
One DevOps platform
Be an early applicant
One DevOps platform
1001+ employees
Be an early applicant
+ Equity Compensation + Employee Stock Purchase Plan
1001+ employees
Company mission
To make it so that everyone can contribute. When everyone can contribute, users become contributors and we greatly increase the rate of innovation.
Role
Who you are
- Proven experience designing and implementing LLM evaluation systems
- Strong understanding of ML model architectures, including public vs. private implementations
- Expertise in ML evaluation metrics and dataset management
- Demonstrated ability to build production-grade ML infrastructure
- Practical experience with Python-based ML frameworks and evaluation tools (e.g., Langsmith, ELI5)
- Excellent problem-solving skills with an engineering mindset
- Ability to collaborate in an asynchronous, remote-first environment
- Familiarity with open-source development and contribution is a plus
What the job involves
- Are you passionate about building robust frameworks to evaluate and ensure the reliability of AI models?
- As a Machine Learning Engineer on GitLab’s AIF team, you’ll play a critical role in shaping the future of AI-powered features at GitLab. This is an exciting opportunity to work on impactful projects that directly influence the quality of GitLab’s AI capabilities
- You’ll help merge cutting-edge evaluation tools, optimize dataset management, and scale our validation infrastructure. Working closely with other AI feature teams, you’ll ensure that every AI feature we deliver is robust, reliable, and meets the highest quality standards
- Some challenges in this role include designing scalable solutions for LLM evaluation, consolidating disparate validation tools, and contributing to GitLab’s innovative AI roadmap
- Design and implement technical evaluators for LLM assessment
- Contribute to evaluation infrastructure consolidation efforts
- Build scalable evaluation pipelines and frameworks
- Develop and manage datasets and evaluation metrics
- Collaborate with feature teams to integrate validation solutions
- Optimize performance across ML evaluation systems
- Support improvements to GitLab’s AI-powered tools through validation
- Ensure all solutions align with GitLab’s infrastructure and security protocols
Our take
GitLab is an open-source collaboration platform for developers. It enables teams to build code quickly from anywhere without compromising on security or quality. GitLab already has an estimated 30M+ users from all over the world.
According to a 2021 GitLab report, 60% of developers can release code 2x faster with DevOps platforms that bring teams together and accelerate processes. It stands out by catering to programmers of all experiences. Gitlab's version control system helps streamline the process of creating, maintaining, and deploying code, whoever you are.
GitLab was in the right place to capitalize on the recent trend toward rapid digital transformation and remote working. This trend is expected to continue to grow, and with it, GitLab's popularity. With its successful 2021 IPO and the patronage of major brands such as Nasdaq and Jaguar Land Rover, GitLab is expected to solidify its position as the go-to platform for development teams.
Kirsty
Company Specialist at Welcome to the Jungle
Insights
Some candidates hear
back within 2 weeks
15% employee growth in 12 months
Company
Funding (last 2 of 8 rounds)
Sep 2019
$268m
SERIES E
Dec 2018
$20m
SERIES D
Total funding: $432.8m
Company benefits
- We offer benefits to manage your health, wealth, and well-being regardless of location
- Flexibility in schedule to be there for life’s important moments
- Equity compensation & Employee Stock Purchase Plan offered
- Generous Paid Time Off
Company values
- Collaboration - We prioritize things that help us work together effectively, such as assuming positive intent, saying “thanks” and “sorry,” and giving timely feedback.
- Results - We operate with a sense of urgency and bias for action, and we do what we promise — to each other, customers, users, and investors.
- Efficiency - From choosing boring solutions to documenting everything and being managers of one, we strive to make fast progress on the right things.
- Diversity, Inclusion, & Belonging - We work to ensure GitLab is a place where people from every background and circumstance feel like they belong and can thrive.
- Iteration - We aim to do the smallest viable and valuable thing, and get it out quickly for feedback.
- Transparency - Everything we do is public by default, from our company handbook to the issue trackers for our product.
Company HQ
Financial District, San Francisco, CA
Articles
Leadership
Sid Sijbrandij
(CEO)Studied Physics at the University of Twente. Interned at P&G and IBM. Was Lead Architect at the Netherlands' Ministry of Justice from 2009 to 2012. Founded app store Comcoaster in 2008, and then co-founded GitLab in 2012.
Diversity, Equity & Inclusion at GitLab
- Diversity, Inclusion, & Belonging is woven into our values and everything we do
- GitLab is home to many dedicated Team Member Resource Groups
- GitLab is a global, multi-cultural organization with team members all over the world
Share this job
View 46 more jobs at GitLab