Senior Software Engineer, Horizon3

• 5+ years building production software, with strong Python
• Hands-on experience building LLM-powered applications or agents, tool use / function calling, structured outputs, multi-step orchestration, and the glue that makes it all hold together
• A track record of making LLMs reliable in production, you've wrestled nondeterminism, designed around model limitations, and shipped something that worked when it mattered
• Real experience with evaluation: you've built or owned the harness that tells you whether a model or agent change is an improvement, not just a vibe
• Strong instincts for prompt and context engineering, and the judgment to keep the model's job small and well-scoped
• Solid software fundamentals — testing, observability, and the discipline to keep a complex agent debuggable
• Ownership mentality, comfortable owning a critical, fast-moving subsystem end to end
• You've shipped an autonomous agent that did real, valuable work unattended in production, and you have scar tissue from making it trustworthy
• You've designed evaluation systems that actually drove improvement, closed the loop between "we changed something" and "it measurably got better."
• You pair an offensive-security mindset (CTF, bug bounty, pentesting, or research background) with the engineering chops to turn that intuition into a reliable system
• You have hands-on experience with agent fine-tuning or RL (SFT, GRPO, reward design for tool-using agents) and a grounded view of when it's worth it versus improving the harness
• You've published or spoken on agent reliability, evaluation, or autonomous security tooling

Desirable

• We're building an autonomous, black-box web application penetration tester
• It crawls and attacks real production websites the way a skilled human pentester would, finding broken access control, injection, XSS, SSRF, SSTI, and more, under a strict production-safe, no-false-positives mandate
• We have deep offensive expertise on this team: people who know exactly how to find and exploit these vulnerabilities by hand
• What we need is an engineer who can turn that expertise into autonomous agent capability, the reasoning, orchestration, tooling, and evaluation that lets an LLM-driven agent do this work reliably, at scale, and unattended
• You'd own and evolve the attack-agent layer: the part of the system that decides what to probe, forms and tests hypotheses, exploits, and verifies, without false positives and without touching anything it shouldn't
• This is a build role, not a research role
• We use models surgically, deterministic-first, LLM-as-scalpel, and the hard problems are in engineering reliability, not chasing benchmarks
• Build and evolve the agent harness and orchestration that turns an LLM into a reliable autonomous pentester, the loop that reasons over an application, forms attack hypotheses, acts, and verifies results
• Design the tools and tool-shaped feedback the agent uses to probe and exploit, and the structured-output and validation layers that keep it reliable (e.g., hook-enforced mandatory validation, schema-constrained outputs)
• Translate the team's offensive expertise into repeatable agent capabilities — partnering directly with our attackers to encode how they think into something the agent can do consistently
• Own and grow our evaluation infrastructure: benchmark suites, a failure-mode taxonomy across the pipeline (discovery → hypothesis → exploitation → verification), and regression detection, so we actually know whether the agent is getting better
• Manage LLM inference in production: model selection, prompt and context engineering, and keeping cost and latency under control (we run on AWS Bedrock with centralized cost tracking)
• Hold the line on production-safety and no-false-positives, every finding the agent reports has to be real and reproducible