Site Reliability Engineer - Observability

We are hiring an SRE focused on observability, automation, and runtime reliability for AI platforms and internal agentic systems. This is not a generic SOC role. It is an engineering role for someone who builds telemetry, automates findings-to-fix loops, improves production readiness, and keeps AI systems measurable, resilient, and controllable in production.

Tech stack

• Python for automation and workflow integration
• Observability tooling: metrics, logs, traces, OpenTelemetry, Datadog or adjacent stacks
• AWS logging, telemetry, IAM-aware diagnostics, and infrastructure scripting
• CI/CD integration for runtime checks, rollback drills, and policy validation
• Nice to have: Wiz, CrowdStrike, Orca, GuardDuty, WAF / RASP-style controls, MCP / agent telemetry

Responsibilities

• Design and operate the telemetry and observability layer for AI platforms, including audit trails, tool-call logs, correlation IDs, traces, and runtime visibility across service boundaries.
• Build automated findings-to-fix loops for AI and cloud platforms, integrating signals from tooling such as Wiz, Astrix, or future AI security products into pragmatic remediation workflows.
• Implement reliability and hardening controls for internal AI systems, including alerting, health checks, rollback drills, kill-switch validation, rate limiting, and drift detection.
• Codify detections, policies, and operational checks as code where they reduce toil, prevent regressions, and improve platform control.
• Review platform and AI-application changes from a reliability and application-hardening perspective, especially around secrets, telemetry, external calls, risky MCP usage, and production readiness.
• Own AI-platform-specific operational readiness and partner with central IT / EAS / SOC teams for escalations, handoffs, and shared incident workflows when needed.
• Continuously improve production readiness through automation, post-incident learning, and repeatable playbooks for AI runtime issues.