Core & ML Ops Team Lead, Zyte
Data scraping platform
Job no longer available
Data scraping platform
201-500 employees
Job no longer available
201-500 employees
Company mission
To give organizations faster, easier access to accurate web data that drives better business decisions.
Role
Who you are
- 5+ years experience building distributed systems; 3+ years in MLOps/ML platform engineering (or equivalent impact)
- Knowledge of Linux/OS internals (process model, cgroups/namespaces), networking (TCP/IP, HTTP/2), concurrency, and performance profiling
- Deep understanding of Kubernetes (bonus: Mesos)
- Proficiency developing high-performance services in Java, Rust, Go or C++ (bonus: familiarity with vert.x and Netty frameworks); strong Python skills
- Experience with GPU infrastructure (scheduling, containerization, optimization)
- Track record of designing and operating model platforms (registry, training, serving, monitoring) in production
- Demonstrated success leading technical teams and implementing organization-wide platform solutions
Desirable
- Streaming & workflows: Kafka plus Argo/Temporal/Airflow or equivalents
- EBPF‑based observability, perf tooling, or io_uring experience
- Cost optimization for ML/AI; multi‑tenant quotas and fairness
- Hands‑on experience authoring Golden Paths (service chassis/templates, CI/CD blueprints, CLI scaffolds)
- SRE practices (SLIs/SLOs, incident management)
What the job involves
- Zyte is seeking an experienced Team Lead to manage our Core & MLOps Squad, responsible for "Building the bedrock infrastructure that powers Zyte at scale."
- This hands-on technical leadership role requires expertise across MLOps, systems programming, and orchestration to lead a cross-functional team in designing and maintaining the scalable foundation that enables all Zyte teams to build and run their services with confidence
- Design and evolve the core platform (Kubernetes, Mesos, GPU scheduling/autoscaling, distributed compute)
- Own the model platform: registry, experiment tracking, training orchestration, evaluation, serving, and monitoring
- Build the Golden Path: reference repos, a scaffold CLI, opinionated CI/CD pipelines, runtime contracts (health/metrics/tracing/SLOs), high-performance clients, circuit breakers and other production‑ready defaults
- Operate a secure, multi‑tenant model registry and training platform with standardized experiment/evaluation harnesses
- Provide turnkey serving patterns (online + batch), drift/quality monitoring, and rollback playbooks
- Integrate public/open‑source AI capabilities as managed platform services with cost and data‑governance guardrails
- Run the squad: roadmap/prioritization, delivery, mentoring, and high engineering standards
- Partner with product engineering (Zyte API, Scrapy Cloud), Prod Ops, and Security on adoption and rollout plans
- Mentor the team and foster a platform-thinking mindset
- Container orchestration (Kubernetes/Knative), GPU provisioning & autoscaling, environment & secret management
- Operators, sidecars, and internal SDKs/libraries (Go/Rust/Python/Java) that enforce the golden path contract
- Model platform: registry, experiment tracking, training orchestration, evaluation framework, serving infra, model monitoring
- Observability: logging/metrics/tracing pipelines
- Billing pipeline: metering/events/cost tracking abstractions
- Golden Path: Java, Python, ML templates + CI/CD blueprints + docs + scaffold CLI
- Reliability enablement (SRE practices), cost governance, supply‑chain security (SBOM, image signing)
Our take
Data is an invaluable resource for businesses, giving them the information they need to produce actionable insights that will improve various aspects of their operations. However, the necessary process of data scraping can be complicated to set up and depends on high-spec infrastructure and proxies, ultimately making data less accessible and more resource-intensive.
Zyte, formerly ScrapingHub, provides a data scraping platform for businesses, offering users data extraction services for any project size, whilst also automatically handling infrastructure and proxies. The platform's services are priced differently depending on the complexity of data types, helping to provide a fairer pricing system that smaller businesses can afford.
While data scraping services are nothing new, Zyte has recognised that existing solutions generally still require manual proxy configuration, which stunts data scraping workflows. Through its automated proxy system and “pay for what you use” approach to data complexity, the company shows great promise to disrupt the data scraping market with its uniquely accessible and easy-to-use platform.
Kirsty
Company Specialist at Welcome to the Jungle
Insights
Company
Company benefits
- Flexibility - As a remote company we understand and encourage flexibility
- PTO - Take your time, relax, enjoy recharge. It's important
- Wellness - Talk to the experts, we value mental health, take advantage of our great programs
- L&D - Learn, develop, grow. We are here to support you and your career
- Hangouts and events - We are all remote, but we can still hang out. Participate in great remote events, talks and activities
- Family Friendly - Every family is different. We accept them all. From dogs on calls to maternity leave, we've got it all covered
Company HQ
Carrigrohane, Ireland
Leadership
Shane Evans
(Co-Founder & CEO)Experience as Head of Engineering and Senior Development for lastminute.com from 1999 to 2004. Head of Development for mydeco from 2007 to 2010.
Pablo Hoffman
(Co-Founder & Director)Experience as Systems Administrator, Software Engineer and CIO for Ikatu from 2000 to 2007. Director and Co-founder of Insophia from 2007 to 2012.
Share this job
View 1 more job at Zyte