AI QA & Evaluation Lead
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
About the Role
Jodayn is seeking an AI QA & Evaluation Lead to establish and manage enterprise-grade quality assurance, validation, and evaluation processes for AI-powered systems and intelligent automation platforms.
Key Skills for This Role
Full Job Posting
Overview
Jodayn is seeking an AI QA & Evaluation Lead to establish and manage enterprise-grade quality assurance, validation, and evaluation processes for AI-powered systems and intelligent automation platforms.
The successful candidate will be responsible for ensuring the reliability, consistency, security, explainability, and governance alignment of AI-generated outputs and AI-driven workflows.
This role is critical in maintaining enterprise trust, operational quality, and compliance standards across AI initiatives delivered to government and enterprise clients.
The ideal candidate must possess strong experience in enterprise quality assurance, automation testing, AI evaluation methodologies, and governance-oriented validation frameworks.
AI Evaluation & Validation
- Design and maintain AI testing and evaluation frameworks
- Develop benchmarking datasets and output validation methodologies
- Evaluate AI-generated responses for consistency, accuracy, and reliability
- Define confidence scoring and quality measurement approaches
AI Quality Assurance
- Implement automated regression testing for AI workflows
- Monitor prompt performance and behavioural consistency
- Detect model drift and output degradation
- Validate multilingual AI outputs including Arabic and English content
Governance & Compliance
- Ensure AI outputs align with enterprise governance standards
- Support auditability and traceability of AI-generated results
- Implement review and approval workflows for AI systems
- Support compliance validation activities for enterprise deployments
Security & Risk Testing
- Conduct AI security and prompt injection testing
- Validate protections against unsafe outputs and misuse scenarios
- Support secure AI deployment and operational practices
Reporting & Continuous Improvement
- Produce quality assurance reports and evaluation summaries
- Recommend improvements to prompts, workflows, and AI models
- Support continuous optimisation of AI solution quality
Education
- Bachelor's degree in Computer Science, Information Systems, Software Engineering, or related field
Experience
- Minimum 5 years experience in enterprise quality assurance or software testing
- Experience with automation testing frameworks
- Experience evaluating AI-powered systems preferred
- Experience in enterprise or regulated environments preferred
Quality Assurance
- Enterprise QA methodologies
- Automation testing frameworks
- API testing
- Regression testing methodologies
- Test case development
AI Evaluation
- Large language model evaluation
- Prompt engineering concepts
- AI benchmarking methodologies
- AI governance and validation concepts
Security & Governance
- Risk assessment concepts
- AI security testing
- Auditability and traceability principles
Preferred Skills
- Experience in enterprise AI initiatives
- Exposure to regulated industries
- Experience with bilingual AI validation
- Familiarity with enterprise governance frameworks
Core Competencies
- Exceptional attention to detail
- Strong analytical thinking
- Structured problem-solving capability
- High quality and governance awareness
- Strong documentation and reporting capability
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at JODAYN | جودين
Senior AI Product Engineer
Riyadh, KSA
Jodayn is seeking a highly skilled Senior AI Product Engineer to lead the design, development, and deployment of enterprise-grade artificial intelligence solutions and intelligent automation platforms. The successful can
AI Solutions Architect
Riyadh, KSA
Jodayn is seeking an experienced AI Solutions Architect to lead the architecture, integration, and enterprise deployment design of AI-powered platforms and intelligent automation solutions. The successful candidate will
Senior Software Quality
Riyadh, KSA
We are seeking a Senior Software Quality professional responsible for ensuring software products meet quality standards and business requirements through effective testing, quality assurance practices, and continuous imp
Innovation Consultant
Riyadh, KSA
Candidates should have strong capabilities in innovation management and experience working in consulting environments, particularly in projects related to Digital Government Authority (DGA) - Qiyas innovation requirement
Senior Atlassian Consultant (Full-Stack Jira Expert)
Riyadh, KSA
Partnership Management & Service Portfolio Development Build and manage the service portfolio for the partnership with Atlassian. Prepare technical and commercial proposals. Plan and execute projects. Develop executi
Software Quality Engineer
Riyadh, KSA
Job Summary The Software Quality Engineer is responsible for executing testing activities, ensuring product quality, and supporting the improvement of QA processes. This role works closely with development teams to ident
Senior AI Product Engineer
Riyadh, KSA
AI Solutions Architect
Riyadh, KSA
Senior Software Quality
Riyadh, KSA
Innovation Consultant
Riyadh, KSA
Senior Atlassian Consultant (Full-Stack Jira Expert)
Riyadh, KSA
Software Quality Engineer
Riyadh, KSA