Senior Quality Assurance Engineer

Key Responsibilities

• Own the verification stage that runs after a build — provision the environment, deploy the artifact under test, run the suite against it, and produce the result a release decision rests on.
• Test backend services as a black box across their APIs, covering request routing and provider failover, streaming integrity and early termination, retries and backoff, timeout semantics, rate-limit handling, error mapping, and usage accounting.
• Build API compatibility coverage against each service's published surface, so that a breaking schema change is caught before the applications consuming it are.
• Build end-to-end coverage with Playwright against deployed environments: single sign-on, streamed responses, long-running generation, and verification of the files and documents the system produces.
• Gate releases of shared services and libraries against the applications that depend on them, proving an upgrade is safe before it ships rather than after.
• Own the test environments — containerized dependencies, seeded data, isolation between runs, and teardown — so that a suite result means the same thing every time it runs.
• Add performance and load coverage for streaming endpoints, concurrent sessions, long-running jobs, and document conversion.
• Test security-relevant behavior at the service boundary: authentication and session handling, cross-site request forgery protection, role- and resource-level access control, tenant isolation, and the safe handling of generated content.
• Run quality measurement of model output on a schedule and report how it moves over time — engineering defines what good looks like, and you make it run, gate where appropriate, and surface the trend.
• Keep the suite trustworthy: flake detection and quarantine, clear failure diagnostics, and reporting the team acts on rather than ignores.
• Triage defects found from the outside — reproduce them, narrow them, and hand them back with a failing case the owning engineer can run.
• Set the verification standard that later services and applications inherit, and document it well enough that another team can adopt it without you.

Required Qualifications

• 5+ years in software quality engineering, test automation, or software engineering with a testing focus, including 3+ years building automated test suites in Python.
• Strong API testing experience against running HTTP services — authentication, streaming and server-sent events, pagination, error and timeout behavior — and fluency asserting at the wire level.
• Deep, practical pytest experience — fixtures, parametrization, async tests, plugins, and structuring a large suite so that it stays fast and comprehensible.
• Strong Playwright skills for browser automation, including authentication state reuse, network interception, testing streamed or progressively rendered interfaces, and diagnosing failures from traces.
• Experience testing deployed or packaged software from the outside, rather than only from inside the repository that builds it.
• Experience with API compatibility and schema-drift testing — detecting a breaking change to a published interface before its consumers do.
• CI/CD engineering experience, ideally with GitHub Actions — pipeline design, post-build verification stages, release gates, caching, parallelization, and keeping a suite trustworthy enough to leave blocking.
• Experience provisioning containerized environments for testing, using Testcontainers, Docker Compose, or an equivalent, including databases and backing services.
• Has tested systems that do not have a single correct answer — machine learning output, search or recommendation relevance, data quality, OCR or speech pipelines, or genuinely nondeterministic distributed systems — and can explain how you kept assertions meaningful and managed flakiness.
• Able to read an unfamiliar production codebase and form your own judgment about where the real risk sits, rather than testing what is easiest to test.
• Strong debugging judgment for asynchronous and distributed systems, including failures that only appear in deployed environments.
• Clear written communication — test failures that explain themselves, defect reports engineers can act on, and strategy documents that get adopted.
• Has worked alongside the engineers who own the code under test, and can make the case for testability changes without friction.

Preferred Qualifications

• Robot Framework, particularly with the Browser library, for business-readable acceptance suites.
• Experience testing LLM-backed or agentic applications — scripted model responses, golden datasets, and regression gates on quality scores.
• Experience testing an API gateway, proxy, or similar pass-through service, including multi-provider routing and failover behavior.
• Familiarity with observability and tracing tooling for AI or distributed systems.
• Performance and load testing with k6, Locust, or similar, against streaming or long-running endpoints.
• Security testing background — OWASP, session and cookie handling, content security policy, and multi-tenant isolation.
• Experience verifying generated documents programmatically, including presentation, PDF, or HTML output.
• Working knowledge of React and TypeScript, enough to add stable test hooks and write component-level tests.
• Visual regression testing, including the practicalities of keeping it stable over time.
• Treats static analysis and type checking as part of the quality picture — linters, type checkers, and architectural or import rules enforced in CI.
• Experience as the first dedicated quality hire on a team, establishing standards from scratch.
• Exposure to vector or graph databases in a testing context.