Senior Inference Reliability Engineer

The posting, as Parasail wrote it

Parasail is redefining AI infrastructure by enabling seamless deployment across a distributed network of GPUs, optimizing for cost, performance, and flexibility. Our mission is to empower AI developers with a fast, cost-efficient, and scalable cloud experience—free from vendor lock-in and designed for the next generation of AI workloads.

The Senior/Staff Inference Reliability Engineer will own the end-to-end reliability and production performance of customer inference workloads. This role sits at the intersection of inference platform engineering, LLM performance, and infrastructure reliability.

You will ensure that customer endpoints meet expectations for availability, latency, throughput, quality, and cost. When an endpoint degrades, you will follow the problem across the entire serving path—from APIs, routing, scheduling, and autoscaling through model servers, GPUs, networking, and underlying infrastructure—and drive it through resolution.

This is not a traditional DevOps role focused only on clusters and deployments. It is a production systems role for an engineer who enjoys investigating ambiguous performance problems, building diagnostic tooling, and turning recurring incidents into durable platform improvements.

Prior LLM-inference experience is valuable but not required. We are looking for someone with deep production systems experience who can quickly learn inference-specific technologies and metrics.

WHAT YOU WILL OWN

END-TO-END INFERENCE RELIABILITY

Own the production health of customer inference workloads, including availability, request success, time to first token, inter-token latency, throughput, and operational efficiency.

Establish clear service-level indicators, objectives, performance baselines, and escalation paths for production endpoints.

DETECTION AND OBSERVABILITY

Build the telemetry, dashboards, alerts, and automated diagnostics needed to detect meaningful endpoint degradation before customers report it.

Create visibility across the full inference-serving path, including request queues, routing, scheduling, model servers, GPU utilization, networking, storage, and provider infrastructure.

PRODUCTION INVESTIGATION

Lead the investigation of complex latency, throughput, capacity, and reliability regressions.

Determine whether an issue originates in customer traffic patterns, platform services, inference-engine configuration, GPU hardware, networking, storage, or an external infrastructure provider.

Remain accountable for the customer outcome while partnering with the appropriate engineering teams to implement the fix.

INCIDENT RESPONSE AND PREVENTION

Help lead customer-impacting incidents and establish effective operational practices for acknowledgement, diagnosis, recovery, and communication.

Convert significant incidents into automated tests, safeguards, runbooks, capacity controls, anomaly detection, and platform improvements.

PERFORMANCE AND CAPACITY

Partner with the LLM Performance team t
[…]
omation, diagnostic tooling, or production safeguards.

Strong communication and technical leadership skills, including the ability to coordinate incident resolution across engineering teams.

Experience with Kubernetes, Linux, networking, and cloud-native infrastructure.

While prior LLM-inference experience is not required, a demonstrated ability to learn unfamiliar systems and develop deep technical expertise is essential.

WHAT SUCCESS LOOKS LIKE

Within your first six months, you will help establish clear end-to-end ownership and observability for Parasail’s production inference workloads.

Success means

• Material endpoint regressions are increasingly detected before customers report them.
• Engineers can quickly determine which layer of the system is responsible for a production issue.
• Customer-impacting incidents are acknowledged, diagnosed, and resolved faster.
• Production endpoints consistently meet defined availability, latency, throughput, and efficiency objectives.
• Recurring failure modes are converted into automated detection, safeguards, and durable platform improvements.
• Infrastructure and inference capacity are used efficiently while preserving customer performance and model quality.

WHAT YOU BRING TO THE TABLE

This role is pivotal in building a new operational discipline for AI infrastructure.

You may come from GPU infrastructure, ML platforms, databases, streaming systems, search, low-latency services, distributed storage, or large-scale production engineering. You do not need to have previously held the title “Inference Reliability Engineer.”

What matters is that you have owned important production systems, can investigate problems that span organizational and technical boundaries, and continue following an issue until the customer experience is restored.

If you are passionate about production systems, performance, reliability, and learning the cutting edge of AI infrastructure, we are excited to welcome you aboard.