Software Engineer, NewsBreak
Web (New Initiatives)
$165-250k
3+ days a week in office (Mountain View, CA)
Local breaking news app
Be an early applicant
Local breaking news app
201-500 employees
Be an early applicant
$165-250k
3+ days a week in office (Mountain View, CA)
201-500 employees
Company mission
To connect and empower local users, local content creators, and local businesses at scale, with the goal of helping people everywhere live safer, more vibrant, more truly connected lives.
Role
Who you are
- Bachelor's degree or higher in Computer Science, Engineering, or a related technical field
- 5+ years of proven experience designing and operating large-scale web crawling and indexing infrastructure at major technology companies or innovative startups
- Extensive experience with distributed systems, crawler frontier design, real-time URL prioritization, and high-QPS crawling infrastructure
- Strong system-level coding skills in Python, Go, or C++
- Demonstrated ability to integrate web crawling systems with downstream indexing, search engines, or NLP/AI pipelines
- Solid understanding of web technologies, web protocols (HTTP/HTTPS), JavaScript rendering, and anti-scraping countermeasures
- Experience building responsive crawling systems driven by real-time user signals or query logs is a strong plus
- Excellent problem-solving, analytical, and communication skills, with a proactive attitude towards system improvement and innovation
What the job involves
- We're seeking a founding engineer to lead the design and development of our next-generation web crawling and dynamic indexing infrastructure
- Your mission will be to create an adaptive, real-time crawling system that not only integrates seamlessly with external search providers but rapidly evolves based on user queries and interactions, continuously expanding and refining NewsBreak’s proprietary content index and recommendation knowledge graph
- You’ll architect and implement sophisticated crawling strategies informed directly by real user search patterns, enabling our AI agents to provide fresh, accurate, and hyper-localized responses
- You will build infrastructure capable of dynamically responding to user queries, proactively crawling and indexing content within minutes, rather than days or weeks
- Your work will directly empower our AI-driven question-answering and recommendation systems, creating a closed-loop feedback mechanism where user queries trigger real-time crawling and indexing tasks, continuously improving our content quality and comprehensiveness
- Design, develop, and deploy a real-time, adaptive web crawling and indexing infrastructure capable of proactively responding to user-generated queries and external search results integration
- Architect dynamic crawling strategies that rapidly prioritize, fetch, parse, and index web pages based on real-time demand signals
- Implement scalable crawling systems supporting millions of URLs per day with low latency (minutes-level) from discovery to indexing
- Collaborate closely with AI, search, and recommendation teams to build a tightly coupled feedback loop between user queries, crawling decisions, and content indexing
- Own the full lifecycle of the crawler infrastructure, from discovery algorithms, URL state management, garbage collection, deduplication, and storage optimization, to downstream indexing integration
- Optimize crawler performance, reliability, and resource utilization through rigorous profiling, monitoring, and tooling
- Mentor junior engineers and help build out a high-performing infrastructure team with deep expertise in intelligent web crawling systems
Our take
News aggregation apps often focus on big global stories, but local news can easily get buried. NewsBreak exists to change this situation, using an AI-powered aggregator to connect readers with content creators who report on matters a little closer to home.
NewsBreak stands out with its focus on local news, which is beneficial for users who are looking for more relevant content than can be served by national and global publishers, and with smaller scale press companies which have been suffering financially for decades.
This focus seems to have paid off: NewsBreak has over 50 million users across the US and its app has topped the US Apple App Store and Google Play repeatedly. If uptake continues as it has to date, the company stands to grow substantially while helping reinvigorate a struggling and vital industry.
Steph
Company Specialist at Welcome to the Jungle
Insights
10% employee growth in 12 months
Company
Funding (last 2 of 3 rounds)
Jan 2021
$115m
SERIES C
Sep 2019
$14.8m
SERIES B
Total funding: $139.8m
Company benefits
- Work from home opportunities
- Health, dental, and vision care for you and your family
- Top-tier 401(K) plan with company matching
- Paid time off and paid holidays
- Paid parental leave
- FSA and commuter benefit programs
- Team activity budget
Company HQ
Old Mountain View, Mountain View, CA
Leadership
Jeff Zheng
(CEO)Having undertaken a PhD at the University at Buffalo, Jeff spent over 7 years at Yahoo!, prior to founding Yidianzixun, where he worked as CEO. He left this role in April 2015 to found NewsBreak, serving as CEO since.
Salary benchmarks
We don't have enough data yet to provide salary benchmarks for this role.
Submit your salary to help other candidates with crowdsourced salary estimates.
Share this job
View 26 more jobs at NewsBreak