(Senior/Junior AI Inference Reliability Engineer, Compute Infra

Together AI·San Francisco·Posted 27 October 2026

Apply with a tailored CV View on Together AI's site

About the role

About the Role We're looking for a software engineer, reliability-minded but still a builder, to own what happens in production to the Kubernetes-native control plane that provisions and runs our GPU inference fleet. The platform team builds the manifest-driven API and controllers that let the inference team declare what they need, a cluster, a model deployment, a capacity change, and get it reconciled underneath. Your job is to keep that promise true in the real world: triaging InfiniBand issues when prefill and decode nodes stall exchanging KV cache over RDMA, diagnosing hero-model and LoRA-serving faults that don't show up in generic health checks, and extending the automated self-healing system so each new failure mode becomes a permanent fix instead of a recurring page. The core value we're after is keeping the decoupling real even when things break, and keeping the fleet efficient without that efficiency becoming a new source of instability. You'll own SLOs and error budgets for inference serving, lead incident response, and turn what postmortems teach you into actual changes in the controllers and automation, not just updated runbooks. You'll also work the reliability side…

This is a summary of the advert. The full description and the application are on Together AI's careers site.

Apply faster with Prepper

A version of your CV tailored to this (Senior/Junior AI Inference Reliability Engineer, Compute Infra role.
A cover letter written from your CV and this advert.
Prepper can fill in Together AI's application form and send it once you approve.
The application saved in your tracker, with reminders to follow up.

Start free

More roles at Together AI

Prepper is independent and not affiliated with Together AI. Roles are listed from the employer's public careers system and removed when they close.