AI/MLOps Engineer

Responsibilities

• Design and maintain monitoring and observability solutions for AI applications and ML pipelines
• Track logs, metrics, and traces using tools such as CloudWatch, Datadog, or similar platforms
• Develop evaluation and testing frameworks for prompts, models, and AI workflows
• Perform regression testing and quality validation for LLM-based systems
• Manage prompt experimentation, versioning, and A/B testing processes
• Debug AI workflows, including model outputs, orchestration pipelines, and infrastructure failures
• Support deployment, scaling, and maintenance of AI/ML infrastructure in production environments
• Collaborate with engineering and product teams to improve system reliability and performance
• Analyze production data and user feedback to drive continuous improvement of AI systems
• Contribute to operational best practices, documentation, and incident response processes