Principal AI Engineer, LogicMonitor

$215.2-282.4k

Kubernetes

Python

GraphQL

Java

Kafka

Semantic

JSON

REST API

FastAPI

Pydantic

WebSocket

Senior and Expert level

San Francisco Bay Area

Hybrid observability powered by AI

Be an early applicant

Hybrid observability powered by AI

1001+ employees

B2BArtificial IntelligenceAnalyticsSaaS

Be an early applicant

$215.2-282.4k

Kubernetes

Python

GraphQL

Java

Kafka

Semantic

JSON

REST API

FastAPI

Pydantic

WebSocket

Senior and Expert level

San Francisco Bay Area

1001+ employees

B2BArtificial IntelligenceAnalyticsSaaS

Company mission

To expand what’s possible for businesses by advancing their technology.

Job

Company

Role

Who you are

8+ years backend/data platform engineering
3 - 5+ years shipping production AI systems (RAG/agents)
Strong Python (or Go/Java) and distributed systems: queues, concurrency, idempotency, backpressure
Hands-on with FastAPI/gRPC, GraphQL, OpenAPI/Pydantic, and schema governance
Practical MCP experience (servers/clients, tool schemas, versioning, auth)
Retrieval depth: embeddings, hybrid search, vector/graph DBs, chunking, evals
Kubernetes/Helm, IaC, autoscaling, secrets, blue/green & canaries for platform services
Security/tenancy: OIDC/OAuth2, RBAC/ABAC, audit, DLP/PII, per-tenant isolation

What the job involves

We are looking for a Principal AI Engineer- Multi Agent Systems that will own the multi-tenant Data platform (ingestion, semantics, governance, APIs) and the MCP-enabled multi-agent runtime (planning, routing, memory, safety, evaluation)
You’ll define the north-star architecture and ship a reliable, cost-aware platform that lets teams compose production agents over trusted data
Connectors & Ingestion: ERP/CRM/DB/files; Kafka/Pulsar; schema evolution & backfills
Semantic Layer & Contracts: Pydantic/JSON Schema/OpenAPI; business semantics, field descriptions, policy enforcement points
APIs & Access: REST/GraphQL + MCP tool endpoints with consistent pagination, filtering, row/column-level security
Retrieval Layer: embeddings service, hybrid dense/sparse, re-rankers, graph joins; deterministic citation/attribution
Multi-Agent Runtime: supervisor-decomposer, router-selector, tool-using workers, reflection/verifier/judge agents; capability negotiation; blackboard/event bus patterns
State & Checkpointing: step store, durable checkpoints, resume/replay; concurrency controls and backpressure
HITL: interruptible workflows, approval gates, timeouts/escalations, and UX hooks (SSE/WebSocket) for human feedback

Application process

Anticipated Application Close Date: 12/19/25

Our take

As businesses shift from reactive to proactive monitoring, there is a need for tools which empower them to make predictions and solve problems before they start. This is what LogicMonitor supplies, through an observability platform for hybrid environments.

The company's extensive product suite encompasses monitoring of all sorts, including but not limited to network, virtual machine, database, container and SaaS. Each service supplies real-time insights and automated monitoring, with agentless architecture.

LogicMonitor is held in high regard by the industry, evidenced by its being named in the Visionary quadrant of the Gartner Magic Quadrant 2024. It has also seen significant funding which will support further development and growth for the business.

Freddie

Company Specialist at Welcome to the Jungle

Company

Funding (last 2 of 5 rounds)

Nov 2024

$400m

GROWTH EQUITY VC

Jun 2016

$130m

GROWTH EQUITY VC

Total funding: $543m

Company HQ

Santa Barbara Downtown, Santa Barbara, CA

Leadership

Christina Crawford Kosmowski

(CEO)

Board Member at Rapid7 and Founding LP at Operator Collective. Was also a Board Member at Poema Global, and CCO, VP and Global Head of Customer Success at Slack.

Salary benchmarks

We don't have enough data yet to provide salary benchmarks for this role.

Submit your salary to help other candidates with crowdsourced salary estimates.

Share this job

View 19 more jobs at LogicMonitor