Senior Quality Assurance Engineer

Lattice AI is an independent evaluation, red-teaming, and assurance firm dedicated to supporting organisations that ship AI systems.

Headquartered in the UAE and operating globally, the company focuses on being the independent “third signature” that validates whether an AI system is ready for deployment and scale.

Lattice AI does not build models or agents; instead, it rigorously evaluates and stress-tests what others have built.

Its services span evaluation, red-teaming, and governance, mapped to standards such as SOC 2, ISO 42001, the EU AI Act, and the UAE AI Charter, serving clients from solo developers to enterprises and government programs.

The company emphasises evidence-based assessments and transparent, signed, and auditable scorecards and reports.

We are hiring: QA Engineer (LLM Evals + Quality Engineering), Client Location - Abu Dhabi

We are building AI systems that teams actually depend on, and we need someone who can prove they work.

This role sits at the intersection of large language models and rigorous quality engineering.

What you will own:

Automated frameworks that test non-deterministic LLM outputs for hallucination, consistency, and factual accuracy against gold-standard datasets

Evaluation metrics (RAGAS, faithfulness, answer relevance) that validate RAG pipelines and the citations behind them

Prompt regression suites that catch drift when underlying models or system instructions change

Integration tests that keep AI agents honest against SAAS enterprise systems

Performance benchmarks (Locust, JMeter, K6) and CI/CD quality gates in GitLab

What we are looking for:

5+ years in QA automation, with at least 2 years on ML models, data-heavy applications, or AI agents

Hands-on with LLM evaluation frameworks (DeepEval, TruLens, or custom evaluators) and ground-truth dataset creation

SQL and data validation (Great Expectations), vector databases, and modern QE practice (shift left, test pyramid, mono-repo)

Comfort defining pass/fail criteria for probabilistic systems and communicating confidence levels to engineering leadership

If you can tell the difference between a model that looks right and one that is right, we want to talk.

Send us a message if your profile matches the above expectation; only traditional quality engineering roles should not apply.