Senior Software Development Engineer in Test (SDET), QA - AI Systems

The posting, as Nexxa wrote it

ABOUT NEXXA

Nexxa is building the best AI systems for heavy industries, enabling machines, systems and operations to think, decide and act autonomously across manufacturing, large-scale infrastructure, logistics and legacy environments.

Our mission is to translate deep technical breakthroughs into operational reality, solving some of the hardest systems-level problems in industry.

ROLE OVERVIEW

We’re looking for a Senior / Staff SDET to own quality for Nexxa’s AI agent systems, combining deep test automation with hands-on manual and exploratory testing. Our agents plan, call tools, and take multi-step actions autonomously in industrial environments. You’ll write production-grade code to build evaluation harnesses, simulation environments, and CI/CD quality gates, and you’ll also get hands-on with the agents yourself: probing behavior, reviewing traces, and validating releases where automation alone can’t judge whether the agent did the right thing.

You’ll work closely with ML engineers, backend engineers, and Forward Deployed Engineers to define what “good” looks like for an agent operating in high-stakes industrial settings, then build and run the systems and processes that measure it continuously.

KEY RESPONSIBILITIES

Automation and Infrastructure

• Design, build, and maintain test frameworks, evaluation harnesses, and regression suites for LLM-based agents, covering reasoning quality, tool-call correctness, task completion, and multi-turn coherence.
• Build automated pipelines that run evals and test suites on every model, prompt, or tool-integration change, and integrate them into CI/CD as enforceable quality gates.
• Develop simulators, mocks, and sandbox environments for industrial systems and tool APIs so agents can be tested safely, repeatably, and at scale.
• Implement API, contract, integration, and end-to-end tests for the services, tools, and orchestration layers that agents depend on.
• Develop performance, load, and reliability tests for agent workflows, including failure injection and resilience testing.

Manual and Exploratory Testing

• Perform hands-on exploratory testing of agent behavior, including new capabilities, ambiguous or underspecified requests, long multi-turn sessions, and unusual operational scenarios that scripted tests don’t anticipate.
• Manually review agent traces and outputs to judge reasoning quality, appropriateness of actions, and safety, and use those findings to improve automated checks.
• Execute manual regression and release verification passes in staging and customer-like environments before new agent capabilities ship.
• Conduct manual red-teaming sessions (prompt injection, jailbreaks, tool misuse, unsafe actions) in partnership with security teams, then convert successful attacks into automated test cases.
• Support user acceptance testing and pilot rollouts with Forward Deployed Engineers, capturing real-world failures and feeding them back into test suites.

[…]
ting tools (e.g., k6, Locust) and chaos or fault-injection testing.

• Background in ML/data science sufficient to read model evals and statistical significance.
• Experience red-teaming or doing adversarial/security testing on ML systems.
• Familiarity with observability and tracing tools for LLM applications (e.g., LangSmith, Arize, Langfuse, Weights & Biases) and general observability stacks (OpenTelemetry, Grafana, Datadog).
• Experience testing systems in industrial, IoT, or operational technology (OT) environments, including industrial protocols (OPC UA, MQTT, Modbus) or simulation and hardware-in-the-loop setups.
• Experience supporting UAT, pilots, or customer-facing validation of complex products.
• Experience setting up test and eval infrastructure from scratch at a startup or fast-moving team.

WHAT WE’RE LOOKING FOR

• An engineer who is equally comfortable writing a test framework and sitting down to break an agent by hand.
• Someone who can build rigorous evaluation systems for problems that don’t have a single right answer, and who knows when human judgment is the right tool.
• A systems thinker who enjoys turning ambiguous agent behavior into measurable, trustworthy signals.
• A strong collaborator who partners well with ML engineers, backend engineers, and Forward Deployed teams.

WHY JOIN NEXXA.AI http://Nexxa.AI?

• Innovative Environment: Play a critical role in transforming heavy industries through groundbreaking AI and automation technologies.
• Collaborative Culture: Be part of a team that values innovation, discipline, and continuous improvement.
• Professional Growth: Benefit from significant opportunities for career development and advancement.
• Competitive Compensation: Enjoy a comprehensive salary and equity package reflective of your expertise and contributions.

If you’re passionate about AI quality and eager to help define what trustworthy autonomous systems look like in heavy industry, we’d love to connect.