Software Engineer, Infrastructure & Platform

Accountabilities:

• • Design and build sandboxed evaluation environments that allow AI models to safely execute code, interact with tools and services, and perform complex tasks.
  • Develop backend services and infrastructure that support large-scale, repeatable AI and agentic evaluations.
  • Build agent scaffolding and evaluation harnesses covering tool-use loops, context management, retries, state management, token budgets, and multi-agent or subagent workflows.
  • Provision and orchestrate isolated environments using technologies such as Docker, Kubernetes, virtual machines, and cloud infrastructure.
  • Design secure approaches to networking, permissions, credentials, secrets management, and resource isolation for model-driven environments.
  • Develop APIs, internal tools, and automation that enable researchers, engineers, and subject-matter experts to efficiently create and execute evaluations.
  • Improve evaluation reliability and reproducibility through logging, observability, snapshotting, debugging capabilities, and automated testing.
  • Build infrastructure capable of running thousands of evaluation tasks reliably while capturing the artifacts and telemetry required to analyze model behavior.
  • Partner with analysts, red teamers, and technical experts to translate sophisticated evaluation concepts into dependable engineering systems.
  • Investigate failures across application, infrastructure, networking, and evaluation layers, distinguishing model limitations from problems with the underlying environment or harness.
  • Continuously improve platform scalability, security, resilience, and developer experience as evaluation requirements evolve.

  Requirements

  • 3–5+ years of professional software engineering experience, particularly in backend, infrastructure, platform, SRE, or distributed systems engineering.
  • Strong programming skills in Python and experience developing production-quality software.
  • Proven experience designing and operating backend services, APIs, or distributed systems.
  • Hands-on experience with Docker, Kubernetes, virtual machines, or comparable container and orchestration technologies.
  • Experience working with AWS, GCP, or similar cloud infrastructure platforms.
  • Strong understanding of Linux systems, networking, authentication, permissions, and infrastructure security.
  • Experience with Infrastructure as Code and automation tools such as Terraform.
  • Excellent debugging and troubleshooting abilities across application, infrastructure, and networking layers, including complex agentic workflows.
  • Ability to build systems that are reproducible, observable, scalable, reliable, and secure.
  • Comfort working through ambiguous technical challenges where requirements and architecture may change rapidly.
  • Strong collaboration and communication skills, with the ability to work effectively with researchers, engineers, analysts, and technical specialists.
  • Genuine interest in AI systems, agentic workflows, AI security, or model evaluations; previous professional AI experience is beneficial but not mandatory.
  • Experience with developer platforms, CI/CD systems, test infrastructure, sandboxes, or ephemeral compute environments is a plus.
  • Familiarity with LLM APIs, agent frameworks, tool-calling systems, or AI evaluation infrastructure is advantageous.
  • Experience designing secure execution environments for untrusted or semi-trusted code is highly desirable.
  • Background in SRE, platform engineering, cloud infrastructure, cybersecurity, or developer tooling is a strong plus.
  • Knowledge of distributed task execution, queues, workflow orchestration, or large-scale automated testing is beneficial.
  • Familiarity with AI safety, adversarial testing, model evaluations, autonomous agents, or agentic AI concepts—including Model Context Protocol, agent benchmarks, and AI-agent security risks—is advantageous.
  • Candidates must be based in the United States and able to meet applicable work authorization requirements.

  Benefits

  • Competitive salary: $110,000–$160,000 annually, depending on experience and location.
  • Performance bonus: Annual performance-based bonus opportunity.
  • Fully remote: Work remotely from anywhere in the United States.
  • Health coverage: Comprehensive health, dental, and vision benefits.
  • Paid time off: Generous PTO and paid holiday schedule.
  • Retirement: 401(k) plan.
  • Professional development: Support for conferences, continuing education, and leadership training.
  • Impactful work: Build infrastructure supporting advanced AI evaluation, safety, security, and research initiatives.