Staff Machine Learning Engineer

About the role:

Machine Learning Engineers at Cresta work across several high-impact AI initiatives. Final team placement is determined based on experience, strengths, and business needs.

Current focus areas include:

• Agentic Assist: Lead and build next-generation agentic AI systems that augment contact center agents in real time. This track requires strong pre-LLM ML foundations, deep expertise in LLMs and modern prompting techniques, a rapid prototyping mindset, and a proven ability to translate cutting-edge research into scalable, production-grade systems.
• Agent & System Quality: Design evaluation frameworks and improve the reliability, robustness, and performance of LLM-powered agents. This includes diagnosing and mitigating failure modes such as hallucinations, retrieval errors, tool misuse, context drift, prompt brittleness, and multi-step reasoning breakdowns, while defining measurable quality metrics (e.g., accuracy, faithfulness, task completion, latency, and cost) for complex, non-deterministic systems.
• Insights: Architect and scale LLM and retrieval-augmented generation pipelines that ground models in enterprise data. This track focuses on building high-performance ML systems that process complex data, extract structured insights, and deliver real-time, actionable intelligence at scale.

Responsibilities:

• Define and lead the technical vision for Cresta’s next-generation Agentic AI systems, including Agentic Assist and enterprise AI Agents.
• Architect scalable, production-grade LLM systems that integrate reasoning, retrieval, planning, tool use, and real-time decision-making into cohesive, intelligent workflows.
• Design and evolve multi-agent orchestration frameworks that combine RAG, structured knowledge, domain-adapted models, and automated actions.
• Establish best practices for building robust, reliable, and cost-efficient LLM-powered systems in high-scale production environments.
• Own evaluation strategy for complex, non-deterministic AI systems, including offline benchmarking, online experimentation, LLM-as-a-judge methodologies, and systematic failure analysis.
• Proactively identify and mitigate agent failure modes such as hallucinations, tool misuse, retrieval errors, prompt brittleness, context drift, and multi-step reasoning breakdowns.
• Define measurable quality standards (accuracy, faithfulness, task completion, latency, cost efficiency, robustness) and drive continuous improvement.