Senior Data Engineer, Magnite
Scala
$150-170k
+ Equity and Employee Stock Purchase Plan
More information about location
3+ days a week in office
Sell-side omnichannel advertisement platform
Be an early applicant
Sell-side omnichannel advertisement platform
501-1000 employees
Be an early applicant
$150-170k
+ Equity and Employee Stock Purchase Plan
More information about location
3+ days a week in office
501-1000 employees
Company mission
Magnite's mission is to help the world’s leading advertisement agencies and brands reach millions of consumers efficiently and safely through its omnichannel sell-side advertisement platform.
Role
Who you are
- Proficiency in Scala and Spark to build and optimize large-scale batch and streaming data pipelines
- With 5+ years of software development experience with a BS/MS Computer Science or equivalent work experience
- Who wants to take ownership. We are responsible for the entire software development lifecycle, from requirements gathering to production support
- To engage in critical and creative thinking, and constructive brainstorming. We expect engineers to present and discuss tradeoffs to solve the problems we face
- With either existing experience designing and building systems that work with large-scale data volumes and data ingestion systems at scale in a cloud-first setting, OR established technical excellence with a desire to learn and crush the data engineering world
What the job involves
- We are seeking someone with proficiency in Scala and Spark to build and optimize large-scale batch and streaming data pipelines
- The Core Data Engineering team sits within the data engineering group and maintains technical involvement across the applications and infrastructure that make up Magnite's data pipelines
- Our pipelines collectively handle 400+ billion events per day and generate terabytes of data per hour across both cloud-based and on-prem infrastructure
- This data underpins the various business lines - including client reporting, internal data science, account managers, and product + business teams - and as such we need to build systems that remain scalable and efficient at this volume of data while also ensuring data consistency and reliability
- We value communication, discussion, and sharing of ideas to come to the best technical solutions to our large-scale data challenges
- We are looking for people who want to get things done and value open collaboration (including constructive feedback when brainstorming)
- The team's mandate is technical development across the three platform-specific data engineering teams in the group
- As such they get exposure to all aspects of our data engineering infrastructure and applications including Spark jobs, Java-based real-time event processing, and large-scale data warehousing
- Get to work on handling internet-scale data problems
- Help architect and build systems to process our data volume to empower all consumers
- Have hands-on involvement across the group's various data pipelines and related systems for data delivery
- Be a part of and promote our culture of collaboration and mentorship
- Typical challenges we face in this role include:
- Highly-scalable infrastructure: Our traffic has patterns with peak- and off-peak times and seasonal shifts, and our data infrastructure needs to be able to respond accordingly
- Cost optimization: Improving profit margins by lowering the infrastructure cost basis through building efficient systems
- Technical architecture considerations given various SLAs for data delivery
- Our current tech stack:
- Spark (batch + streaming) data pipelines, currently based both on Databricks and on-prem Spark infrastructure, primarily using Scala
- Streaming event processing data pipeline, using Java
- Terraform, Docker, Jenkins for CI/CD / infra / application deployment
- Airflow for job orchestration
- AWS-based cloud infrastructure including RDS, EC2, S3, Kinesis, ECS
Our take
Due to the rise in mobile and smart devices in recent years, the way in which advertisement publishers must format their ads is changing. A key example of this is TV, advertisements that present immediate calls to action within a Connected TV user interface, but publishing ads in such a format is currently inaccessible to most publishers.
Magnite provides a sell-side omnichannel advertisement platform for publishers, focused on facilitating the selling of advertising on all formats including CTV, mobile, desktop, and audio. In addition to facilitating the buying and selling of omnichannel ads, the company also provides a real-time ad performance dashboard, data protection and compliance tools, and insights into buying and selling patterns.
Whilst other omnichannel advertisement platforms present significant competition to the company, Magnite stands out by providing an end-to-end marketplace for the buying, selling, management, and analysis of ads. Furthermore, by focussing its efforts on cutting-edge formats such as CTV, the company shows great potential to futureproof itself within the advertisement industry.
Kirsty
Company Specialist at Welcome to the Jungle
Insights
5% employee growth in 12 months
Company
Company benefits
- Equity and Employee Stock Purchase Plan
- Pension and Retirement Savings Plan in Several Countries
- Comprehensive Healthcare Benefits for You and Your Family
- Generous Time Off, Holiday Breaks and Summer Fridays
- Family-Focused Leave Benefits
- Cell Phone Subsidy
- Performance Management, and Investment in Diversity Initiatives
- Bonusly Peer-to-Peer Recognition Program, Turning Recognition into Tangible Perks and Magnite Swag
- Community Service Events
- Wellness Coach—Meditate and Recharge with an Unlimited User Account for You and a Plus One
Company HQ
Westchester, Los Angeles, CA
Leadership
Michael Barrett
(CEO)Experience as CEO of Rubicon Project from 2017 to 2020. President and Founder of Ichabod Farm Ventures LLC since 2012.
Salary benchmarks
We don't have enough data yet to provide salary benchmarks for this role.
Submit your salary to help other candidates with crowdsourced salary estimates.
Share this job
View 15 more jobs at Magnite