QA Engineer – RAG-Enabled Test Automation

Perform

  • São Paulo - SP
  • Permanente
  • Período integral
  • Há 23 dias
We’re looking for a hands-on QA Engineer to own functional quality as we integrate a Retrieval-Augmented Generation (RAG) assistant into our existing Java/Selenium/RestAssured automation stack. You will create high-value manual test cases, validate and refine autoscripted outputs, and help improve the feedback loop that makes our RAG engine smarter over time.What You’ll Do
  • Author and curate requirement tickets and manual test cases using structured formats (GIVEN/WHEN/THEN + data, edge cases, and links).
  • Validate autoscripted tests generated by the RAG pipeline: compile, run, triage, and refactor them following Screenplay best practices; log precision/recall metrics into LangChain evaluators with guidance from the Automation Agent Engineer.
  • Collaborate with SDETs to refactor duplicated Java/Cucumber steps, increase API-layer test coverage, and adopt code snippets surfaced by the retriever.
  • Perform exploratory and regression testing on the new onboarding UI MVP; file defects, verify fixes, and maintain traceable acceptance criteria in Qtest.
What You Bring
  • 3+ years hands-on QA automation experience with Java, Cucumber (BDD), Selenium/WebDriver, and RestAssured.
  • Comfortable writing structured requirement tickets and decomposing stories into atomic acceptance criteria.
  • Intermediate SQL skills for multi-database validation.
  • Familiarity with Jenkins pipelines and artifact triage processes.
  • Clear and precise communication skills, with the ability to translate fuzzy requirements into crisp GIVEN/WHEN/THEN steps.
It is an asset if you have
  • Exposure to the Screenplay pattern (Serenity BDD) and Serenity reporting.
  • Experience reviewing or prompting AI-generated code (e.g., GitHub Copilot, Cursor).
  • Knowledge of LangChain evaluators or RAGAS for retrieval-quality scoring.
Since 2005, Perform's engineers have been helping companies scale their apps and their teams. We were near-shoring before it was even a term and have worked with 100s of clients along the way.

Perform