Staff Backend Engineer - Second Horizon
Grafana Labs- Base salary
- $186k–$224k Published base salary range
- Location
- Remote - US or Canada (not eligible in Quebec) Remote eligibility
- Employment
- Full-time Staff / Principal
About the job
Company
Grafana Labs is the company behind the open observability cloud, Grafana Cloud, a fully managed observability platform. We are a 100% remote company with 1,600+ team members across 40+ countries, backed by leading investors. Our mission is to help users understand, respond to, and improve their systems regardless of scale, complexity, or tech stack.
Role
We are looking for a Staff-level Backend Engineer to join a new skunkworks initiative, Second Horizon, focused on building an AI-native data intelligence system that gives agents reliable, governed access to enterprise context. You will design and ship the core backend architecture powering context ingestion, storage, retrieval APIs, and agent-facing integrations. This is an early-stage role on a high-autonomy team, requiring comfort with ambiguity and pragmatic architectural decisions.
Responsibilities
- Design, implement, test, and operate the first services for context ingestion, indexing, retrieval orchestration, API access, and system administration.
- Build a scalable multi-tenant SaaS foundation including tenant isolation, usage tracking, quotas, audit logs, background jobs, and reliable service boundaries.
- Build APIs and service interfaces for AI agents, MCP tools, CLIs, and internal applications to retrieve context, provenance, confidence signals, and warnings.
- Partner with the team to balance fast experimentation and long-term reliability as the project moves from prototype to production.
- Instrument services with metrics, logs, traces, alerts, and dashboards using observability tools.
- Contribute to technical direction: architecture, service boundaries, storage choices, API contracts, deployment patterns, and engineering practices.
Qualifications
- Experience with LLMs, prompt engineering, and building applications powered by GenAI.
- Proven track record of delivering production software actively used by users.
- Exposure to cloud-native environments (e.g., AWS, GCP, Azure).
- Experience using observability tools to understand and troubleshoot system behavior.
- Nice to have: agent frameworks, data platforms (Looker, Tableau, PowerBI, Snowflake, DataBricks), data engineering tools.
Compensation & Benefits
Base compensation range for Canada: CAD $186,368 - $223,642. Benefits include equity, bonus (if applicable), and other benefits. We operate a global annual leave policy of 30 days per annum.
Application
Apply via the Greenhouse link provided.
Skills & tags
Compare the essentials before you leave: pay, remote scope, employment type, source, and the employer apply destination.