Member of Technical Staff, Cohere
Pre-Training Data Engineer
More information about location
Natural language processing AI platform
Be an early applicant
Natural language processing AI platform
501-1000 employees
Be an early applicant
More information about location
501-1000 employees
Company mission
To build machines that understand the world, and to make them safely accessible to all.
Role
Who you are
- If you are passionate about transforming data into the foundation of AI systems, this role offers a unique opportunity to make a meaningful impact
- Strong software engineering skills, with proficiency in Python and experience building data pipelines
- Familiarity with data processing frameworks such as Apache Spark, Apache Beam, Pandas, or similar tools
- Experience working with large-scale datasets, including web data, code data, and multilingual corpora
- Knowledge of data quality assessment techniques and experimentation with data mixtures
- A passion for bridging research and engineering to solve complex data-related challenges in AI model training
- Bonus: paper at top-tier venues (such as NeurIPS, ICML, ICLR, AIStats, MLSys, JMLR, AAAI, Nature, COLING, ACL, EMNLP)
What the job involves
- As a Pre-Training Data Engineer, you will play a pivotal role in developing the data infrastructure that underpins Cohere’s advanced language models
- Your responsibilities will encompass the end-to-end management of training data, including ingestion, cleaning, filtering, and optimization, as well as data modeling to ensure datasets are structured and formatted for optimal model performance
- You will work with diverse data sources—such as web data, code data, multilingual corpora, and synthetic data—to ensure their quality, diversity, and reliability
- In this role, you will design and implement scalable, robust pipelines for data processing, conduct data ablations to evaluate quality, and experiment with data mixtures to enhance model performance
- By combining research and engineering, you will bridge the gap between raw data and cutting-edge AI models, directly contributing to improvements in critical training metrics like throughput and accelerator utilization
- Your work will be essential to Cohere’s mission of delivering efficient and reliable language understanding and generation capabilities, driving innovation in natural language processing
- Design and build scalable data pipelines to ingest, clean, filter, and optimize diverse datasets, including web data, code data, multilingual corpora, and synthetic data
- Conduct data ablations to assess data quality and experiment with data mixtures to enhance model performance
- Develop robust data modeling techniques to ensure datasets are structured and formatted for optimal training efficiency
- Research and implement innovative data curation methods, leveraging Cohere’s infrastructure to drive advancements in natural language processing
- Collaborate with cross-functional teams, including researchers and engineers, to ensure data pipelines meet the demands of cutting-edge language models
Our take
The AI industry biggest stakeholders insist that the next step in enhancing web experiences will be through natural language interactions. While this may be true, only companies with considerable resources can reasonably address the gap needed to bring NLP to the forefront of web integration. Cohere was founded to address this shortfall between NLP development, widespread usage, and the resources required.
Cohere works on building top-of-the-class NLP processing models and software which it makes available to customers through APIs and custom solutions. In a strategic move, the company has partnered with Google Cloud, giving them access to resources that will undoubtedly push the field of NLP forward. This allows Cohere to serve as middlemen by offering a wide range of customers a product built on unmatched technology.
AI is currently a worldwide talking point and so it is no surprise that Cohere has seen rapid development, and raised $270M in a 2023 funding round. The company plans to introduce a dialogue model that would resemble ChatGPT, however, in this case, its technology would be mainly accessible to developers and businesses.
Steph
Company Specialist at Welcome to the Jungle
Insights
192% employee growth in 12 months
Company
Funding (last 2 of 5 rounds)
Jul 2024
$500m
SERIES D
Mar 2023
$270m
SERIES C
Total funding: $940m
Company benefits
- We cover 100% of premiums across health, dental, vision & travel
- We offer RRSP, 401K, and Pension Scheme contributions
- We offer lunch and fitness studio credits and a quality time fund which can be spent on things like dog walking and laundry services
- We want everyone who contributes to our success to get a ‘piece of the pie’
- Work Remote & PTO
- We know that some people work better at home, so we are remote-first, and sure that all meetings, events, and opportunities are set up for our distributed team
- Every employee at Cohere gets 6 weeks of paid time off & US and Canadian federal holidays, and we provide unlimited sick days
- All new parents (including those who adopt or go through a surrogate journey) are eligible to receive 100% salary for 6 months in Canada, the US, and the UK
- The decision and journey to having kids is different for each person and family. Part of our commitment to diversity is being able to support the widest variety of these scenarios as we can so we offer financial support for egg freezing and IVF for those in the US, Canada and the UK
- Every employee at Cohere receives an annual $2,000 education fund to use as they please
Company HQ
Grange Park, Toronto, ON
Leadership
Aidan Gomez
(CEO)Has a PhD in Computer Science from Oxford University. Previously worked as a Researcher at Google (Brain) and FOR.ai
Ivan Zhang
(CTO)Previous experience working in research and engineering at For.ai, Cortex labs, Pressly and Ranomics.
Nick Frosst
(CTO)Previously worked as a Research Engineer at Google (Brain).
Salary benchmarks
We don't have enough data yet to provide salary benchmarks for this role.
Submit your salary to help other candidates with crowdsourced salary estimates.
Share this job
View 37 more jobs at Cohere