Software Test Automation Engineering Lead (Bangalore)

This role also provides practice leadership across agent development squads by establishing QA standards, improving LLM and agent testing approaches, mentoring through technical example, and driving consistent test framework design, automation quality, and continuous improvement without direct people management.

What You Will Do

The essential functions include, but are not limited to the following:

• Platform Team Quality Assurance
• Own test automation for Platform Team services — multi-tenant data isolation, IAM and OAuth 2.0 flows, event-driven backbone integrity, API gateway contract testing, billing and metering accuracy, and agent marketplace functionality.
• Design and maintain test strategies for complex SaaS platform concerns — tenant boundary validation, cross-tenant security isolation, event ordering and delivery guarantees, and immutable audit log integrity.
• Build and maintain Pytest-based test suites for Platform Team Python services, validating API contracts, event schemas, and service-level behaviour under multi-tenant conditions.
• Collaborate with the Platform Software Architect and Senior QA Engineer on test coverage strategy — ensuring platform capabilities are testable by design and quality gates are integrated into the Platform Team’s CI/CD pipelines.
• Own audit trail validation for platform services — immutable log structure, correlation ID chains, and event emission correctness that underpin compliance requirements.
• QA Practice Leadership Across Agent Stream
• Define and own QA standards across all Agent Stream pods — test framework patterns, Pytest and Playwright conventions, CI/CD quality gate requirements, and test code quality bar that all pod QA engineers follow.
• Standardise LLM and agent testing approaches across pods — define how non-deterministic outputs are handled, what constitutes a regression, and how agent workflow validation is structured consistently across squads.
• Review and challenge test strategies proposed by pod QA engineers — raise the bar on coverage, framework design, and evaluation rigour across the stream.
• Drive LLM testing capability building across the stream — introduce and embed Langfuse trace validation, eval harness patterns, and prompt regression testing as shared practices, not squad-level experiments.
• Contribute to QA hiring — define the technical bar for QA engineers across the stream, participate in interviews, and ensure new hires are assessed consistently.
• Quality Standards & Engineering Craft
• Set the test code quality bar for the AI Platform — clean, reusable, well-structured Pytest and Playwright implementations that others reference as the standard.
• Identify and address systemic quality gaps proactively — gaps in coverage, inconsistent practices across pods, or framework debt that is slowing squad delivery.
• Integrate test suites into Azure DevOps CI/CD pipelines as automated quality gates — ensuring consistent pipeline standards across Platform Team and Agent Stream.

Who You Are

• 7+ years of software testing and test automation experience, with strong experience in SaaS platform, backend services, or enterprise application testing.
• Strong experience designing, building, and maintaining automated test frameworks using Pytest, Playwright, Selenium, or similar tools, including framework architecture, reusable components, and test code quality standards.
• Strong Python experience for test framework development, custom assertions, API validation, and automated testing support.
• Experience with REST API testing, contract testing, schema validation, and tools such as Postman, HTTPX, or similar platforms.
• Experience integrating automated test suites into CI/CD pipelines, including test reporting, pipeline definition, and automated quality gate enforcement using Azure DevOps or similar tools.
• Experience with SQL and NoSQL databases for test data management, teardown, and database state validation.
• Experience testing multi-tenant SaaS systems, including tenant isolation, cross-tenant security boundaries, and scoped data access.
• Experience testing event-driven systems, including message ordering, delivery guarantees, idempotency, and event schema validation.
• Experience with API gateway testing, OpenAPI schema validation, versioning, rate limiting, and inter-service contract testing.
• Experience testing IAM and authentication workflows, including OAuth 2.0, token lifecycle validation, and identity propagation across services.
• Experience validating audit trails, immutable logs, correlation ID chains, and event emissions for compliance-grade platform services.
• Experience defining QA standards, test framework conventions, coverage expectations, or quality gate patterns across multiple teams or squads.
• Demonstrated ability to raise QA capability through mentoring, framework contributions, standards documentation, or introduction of improved testing practices.
• Understanding of LLM and agent testing concepts, including non-deterministic outputs, multi-step agent workflows, confidence-based validation, and evolving AI testing practices.
• Familiarity with Langfuse or equivalent LLM observability tools and their use in agent validation.
• Strong communication, collaboration, technical leadership, problem-solving, and continuous improvement skills.
• Hands-on experience with Langfuse, including trace validation, evaluation pipeline setup, or confidence score assertion preferred.
• Experience with LLM evaluation frameworks, such as Ragas or equivalent tools for RAG output quality testing preferred.
• Experience with performance or load testing for API or LLM inference endpoints preferred.
• Experience with Azure Monitor, Application Insights, or similar tools for observability-assisted debugging preferred.
• Contributions to open-source test tooling or automation frameworks preferred.
• Prior formal or informal QA leadership experience, such as tech lead, QA chapter lead, or equivalent preferred.

At our core, Trintechers stand committed to fostering a culture rooted in our core values – Humble, Empowered, Reliable, and Open. Together, these values guide our actions, define our identity, and inspire us to continuously strive for excellence in everything we do.

All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin or disability.