Senior QA Automation Engineer (Python | Selenium | Pytest)

About 3Pillar Global

3Pillar is an AI transformation partner on a mission to help enterprises build the AI-native products and intelligent agents that will define the next era of business. With teams across North America, Europe, Latin America, and Asia, we work with the most ambitious companies in financial services, healthcare, media, and technology — helping them move faster, modernize boldly, and compete on their own terms. Our HelixAI platform and Helix Pods delivery model put our engineers at the center of real agentic transformation — doing work that is open, portable, and built to last. We are building the future of enterprise AI.

Senior QA Automation Engineer

You'll own test architecture and semantic validation for a mission-critical AI-powered system where consistency guarantees are non-negotiable. This isn't traditional QA; you're designing test harnesses that prove AI outputs are correct, not just crash-free.

Your core responsibility: build a golden-set validation harness that measures whether the system's LLM stage produces the right decisions at the right confidence levels. You'll design automated tests across unit, integration, E2E, and access-control layers. You'll also establish CI/CD quality gates that block bad builds—schema validation failures, cross-tenant isolation defects, missing floor controls, semantic inconsistencies—before they reach production.

This is front-loaded work (weeks 1–5), then you support the team through delivery. You'll mentor a junior test engineer and set the standard for test coverage in a compressed timeline.

Desired Capabilities

• Leadership skills
• Mentoring and coaching abilities
• Strategic thinking and decision-making skills
• Project management skills
• Stakeholder management and collaboration skills
• Ability to motivate and inspire team members
• Continuous improvement mindset
• Familiarity with DevOps processes and tools
• Proficiency in programming languages such as Java, Python, or Ruby

Key Responsibilities

• Design comprehensive test architecture – plan unit, integration, E2E, and semantic validation test suites that cover the full pipeline from profile intake through roadmap generation; define coverage targets and acceptance criteria
• Build semantic validation harnesses for AI outputs – design golden-set test cases (5–8 CISO-approved reference profiles) that measure AI stage precision/recall, rigor-tier correctness, floor-control invariants, traceability completeness, and run-to-run stability; a semantic regression blocks merge
• Implement access-control testing – design automated probes at API and database layers that verify cross-tenant isolation; simulate compromised sessions and verify RLS policies enforce tenant boundaries
• Design and maintain CI/CD quality gates – build automated test runners that execute on every commit: unit tests, integration tests, schema validation, migration guards, image scanning, access-control probes; red builds block merge
• Lead test automation practices – expand automation frameworks and practices; identify opportunities for improving test processes; recommend tools and technologies for testing efficiency
• Collaborate across the team – serve as the QA voice in technical design reviews; flag testability issues early; work with the AI/LLM Lead to define what "correct" means for each pipeline stage
• Mentor junior test engineers – guide the team on test design, semantic validation, and CI/CD best practices; share knowledge on testing AI systems where reproducibility is critical
• Own test documentation and reporting – maintain clear test plans, results, and traceability matrices; report test coverage and quality metrics to leadership; communicate blockers and risks transparently

Minimum Qualifications

• 5+ years of hands-on software QA with a track record of:
  • Test strategy, planning, execution, and maintenance
  • Comprehensive test documentation and traceability