Senior AI DevOps / LLMOps

Key Responsibilities

• Automation of Build-to-Production
  • Design and implement robust CI/CD pipelines tailored for AI, covering model weights, dataset versioning, and application code.
  • Develop specialized workflows for PromptOps, ensuring that system prompts are version-controlled, tested for regressions, and deployed with the same rigor as traditional code.
  • Automate the deployment of Agentic workflows, managing the complexities of stateful AI interactions and multi-agent handoffs.
• AI Infrastructure as Code (IaC)
  • Provision and manage high-performance compute environments (GPU clusters, TPU pods) using Terraform, Pulumi, or Ansible.
  • Define and enforce Policy-as-Code for AI endpoints to ensure compliance with security, cost-usage limits, and data residency requirements.
  • Maintain a consistent environment across Hybrid Infrastructure, ensuring seamless parity between On-Premises development and Cloud production.
• Safe Experimentation & Controlled Releases
  • Architect Progressive Delivery strategies for AI, including Canary releases, Blue-Green deployments, and Shadowing (where new models run in parallel with production to compare outputs).
  • Build "Evaluation-in-the-Loop" gates within the pipeline to automatically test for bias, hallucination, and performance degradation before a release.
  • Implement A/B testing frameworks specifically designed for LLM outputs and agentic behavior.
• Monitoring & Observability
  • Establish deep observability into Inference Endpoints, tracking metrics like tokens-per-second, latency, and drift in model accuracy.
  • Integrate feedback loops that capture production "edge cases" to feed back into the training and fine-tuning pipelines.