AI Platform and Harness Engineer

LTS is seeking an AI Platform and Harness Engineer to develop and maintain the infrastructure, tooling, and evaluation frameworks that power enterprise AI solutions.

What You'll Do

• Design, build, and maintain enterprise AI platform capabilities supporting Large Language Models (LLMs), AI agents, RAG, and Generative AI applications.
• Develop reusable AI harnesses to automate testing, prompt evaluation, model benchmarking, regression testing, and quality assurance.
• Build AI evaluation frameworks to measure model accuracy, retrieval quality, hallucination detection, latency, throughput, cost, and overall application performance.
• Implement observability and monitoring solutions for AI applications, including telemetry, tracing, logging, dashboards, and operational metrics.
• Build and maintain LLMOps pipelines supporting model deployment, versioning, evaluation, experimentation, rollback, and continuous improvement.
• Design automated workflows for prompt testing, retrieval evaluation, AI system validation, and performance benchmarking.
• Develop internal tools for prompt management, model experimentation, AI performance optimization, and developer productivity.
• Build scalable backend services and APIs supporting AI platforms and enterprise AI integrations.
• Collaborate with AI architects and engineering teams to integrate LLMs, RAG pipelines, vector databases, and agentic AI solutions into enterprise applications.
• Support deployment of AI services across AWS, Azure, or Google Cloud using containerized and cloud-native architectures.
• Implement CI/CD pipelines and infrastructure automation supporting enterprise AI development and deployment.
• Apply security, governance, and Responsible AI controls throughout the AI development lifecycle.
• Evaluate emerging AI frameworks, LLMOps technologies, evaluation methodologies, and automation tools to improve engineering productivity.
• Troubleshoot production AI issues and continuously improve platform reliability, scalability, security, and user experience.
• Document engineering standards, AI platform architecture, evaluation methodologies, and operational best practices.

What We're Looking For

• Bachelor's degree in Computer Science, Software Engineering, Artificial Intelligence, Data Science, or a related technical field.
• 5+ years of experience in software engineering, platform engineering, backend engineering, DevOps, cloud engineering, or infrastructure engineering.
• 2+ years building or supporting Generative AI, Large Language Model (LLM), or machine learning applications.
• Strong programming experience in Python.
• Experience developing APIs, backend services, and distributed systems.
• Experience with cloud platforms including AWS, Azure, or Google Cloud Platform.
• Experience deploying applications using Docker and Kubernetes.
• Experience working with Git, CI/CD pipelines, Infrastructure as Code (IaC), and infrastructure automation.