Senior Quality Assurance Engineer

Location

Salary

Searchability

The Role

Key Responsibilities

• Design and execute testing strategies for conversational AI and generative AI products.
• Develop and maintain automated testing frameworks using JavaScript and Python.
• Create and execute chatbot testing scenarios covering user intent, conversation flows, edge cases and behavioural consistency.
• Design LLM testing approaches including prompt validation, grounding validation, golden datasets and non-deterministic testing methodologies.
• Perform AI safety and quality testing, including hallucination detection, bias assessment, toxicity testing and PII leakage validation.
• Validate structured and unstructured data outputs to ensure accuracy, consistency and compliance.
• Establish testing approaches for real-time and streaming data systems.
• Conduct API testing using both manual and automated techniques.
• Collaborate with Machine Learning and NLP teams to define evaluation frameworks and performance metrics.
• Leverage AI-assisted development tools such as GitHub Copilot, Claude and ChatGPT to enhance testing, documentation and defect analysis processes.
• Drive QA best practices across automation, data quality and AI product validation.

Required Experience

• 8+ years' experience in QA Engineering with a strong focus on automation testing.
• 5+ years' experience testing data platforms, data products or data-intensive systems.
• Strong automation experience using JavaScript and/or Python.
• Proven experience testing AI-powered applications and leveraging AI tools within QA workflows.
• Strong understanding of API testing methodologies and tooling.
• Excellent analytical and problem-solving skills with exceptional attention to detail.
• Experience identifying subtle inconsistencies in data quality, system behaviour and AI outputs.

Desirable Experience

• Experience testing conversational AI, chatbots or virtual assistants.
• Familiarity with Large Language Models (LLMs) and Generative AI systems.
• Experience with prompt engineering validation, grounding techniques and AI evaluation frameworks.
• Knowledge of AI safety testing, including hallucination detection, bias testing and PII validation.
• Experience with tools such as PyTest, Postman, dbt Tests and modern automation frameworks.
• Exposure to machine learning, NLP or data science environments.

Technology & Tools

• JavaScript
• Python
• PyTest
• Postman
• API Testing
• dbt Tests

• AI-Assisted Development Tools (Copilot, Claude, ChatGPT)

• What's on Offer?
• Opportunity to work on cutting-edge AI and conversational agent technologies.
• Exposure to modern LLM evaluation and AI testing frameworks.
• Collaborative environment working alongside ML, NLP and Data Engineering specialists.
• Significant influence over QA strategy and testing standards.
• Career growth within a rapidly evolving AI-focused organisation.
• If you're passionate about quality engineering, AI systems and automation-first testing practices, we'd love to hear from you.