AI Application Test Enginee (m/f/d)
Skills
About This Role
Overview
We are looking for an experienced
AI Application Test Engineer
to join our growing team.
You will focus on testing AI-powered applications, including conversational AI and RAG-based systems, ensuring they meet quality standards and perform reliably at scale.
Key Responsibilities
- Design and execute test cases for AI/ML applications, with emphasis on conversational AI and RAG architectures
- Perform end-to-end testing of AI applications across development, staging, and production environments
- Test LLM-based applications for accuracy, consistency, and appropriate responses
- Validate retrieval systems, vector search functionality, and document ingestion pipelines
- Conduct functional, regression, and integration testing for AI components
- Identify and document bugs, edge cases, and model behavior issues
- Collaborate with cross-functional teams to ensure quality throughout the AI development lifecycle
- Contribute to the development of AI testing best practices and standards
Required Technical Skills
- AI/ML Testing: 3+ years of hands-on experience testing AI applications, chatbots, or RAG-based solutions
- Azure Platform: Strong experience with Azure AI services including Azure OpenAI, Azure ML, Cognitive Services, and Azure AI Search
- Test Automation: Proficiency in automation frameworks and tools (Selenium, Playwright, PyTest, etc.)
- Programming: Good coding skills in Python for test automation
- AI Concepts: Understanding of LLM behavior, prompt testing, and conversational AI patterns
- API Testing: Experience testing REST APIs, especially AI model endpoints
- Cloud Testing: Familiarity with testing cloud-native applications on Azure
- Version Control: Experience with Git and collaborative development workflows
Preferred Qualifications
- Experience with Azure DevOps and CI/CD pipelines
- Knowledge of data quality testing and validation techniques
- Familiarity with vector databases (e.g., Pinecone, Weaviate, Azure AI Search)
- Understanding of AI model evaluation metrics
- Experience in Agile environments using Jira or similar tools
- Exposure to MLOps practices
Halian Group
With over 28 years of experience, we have come to understand that innovation is the only way to provide agile, practical solutions that transform businesses and careers.
Our resourcing and smart services help you to realize tomorrow’s potential.
Discover the amazing things possible when you bring the right people and the right technologies together.
At Halian, we recognize that diversity, equity, and inclusion (DEI) are essential to building high-performing teams for our clients.
We are committed to connecting organizations with top talent from all backgrounds, ensuring that every individual feels valued, respected, and empowered to contribute their unique perspectives.
We encourage applications from all qualified candidates, regardless of race, gender, disability, or any other characteristic that makes them unique.
By fostering diverse and inclusive workplaces, we help our clients drive innovation, enhance collaboration, and better reflect the communities they serve.
AI Application Test Enginee in Abu Dhabi, United Arab Emirates
Your resume, rewritten
for this exact role.
Sign up free — Base Career tailors your CV to this job description in 60 seconds.
01 / 05
Resume Tailored to This Job
Your keywords, structure, and story — rewritten to match this exact role and pass ATS filters.
Free · No card · 60 seconds
02 / 05
Cover Letter for This Role, Done
Job-specific cover letters written in Gulf professional tone — ready in seconds, not hours.
Free · No card · 60 seconds
03 / 05
See How Well You Fit This Role
AI match score with clear reasons — know your fit before investing time in the application.
Free · No card · 60 seconds
04 / 05
Apply in One Click
Autofill any application form on Workday, LinkedIn, Bayt, Greenhouse — with your tailored content.
Free · No card · 60 seconds
05 / 05
Track It. Follow Up at the Right Time.
Visual pipeline for every application with AI-timed follow-up reminders so nothing slips.
Free · No card · 60 seconds
Similar Jobs
Full-stack Engineer, AI Applications
Technology Innovation Institute · Abu Dhabi
The Institute Technology Innovation Institute (TII) a publicly funded research institute based in Abu Dhabi, the United Arab Emirates. It is home to a diverse community of leading scientists, engineers, mathematicians, a
Skills
5 days ago
Apply Now↗Apply Now ↗Blockchain Security AI Application Support Engineer
Crypto.com · Dubai
Ensure production application stability, manage incidents, automate support tasks using AI, and possess strong blockchain technical skills and communication.
Skills
2 weeks ago
Apply Now↗Apply Now ↗Product Manager of AI Applications, Global Public Sector
Scale AI · Dubai
Scale is growing rapidly, and joining the Global Public Sector team is an opportunity to work on one of the most exciting and quickly expanding teams at Scale. This team is responsible for generating, executing, and fost
Skills
2 weeks ago
Apply Now↗Apply Now ↗Blockchain Security AI Application Support Engineer
CRYPTO.com · Dubai
At Crypto.com, our dedication to user security is led by our highly experienced Security Team. Comprising an international roster of seasoned cybersecurity experts, our team leads the company's Security, Privacy, and Sec
Skills
2 weeks ago
Apply Now↗Apply Now ↗Blockchain Security AI Application Support Engineer
Crypto.com · Dubai
At Crypto.com, our dedication to user security is led by our highly experienced Security Team. Comprising an international roster of seasoned cybersecurity experts, our team leads the company's Security, Privacy, and Sec
Skills
2 weeks ago
Apply Now↗Apply Now ↗QA Engineer (AI Applications) (Remote)
Jobs Ai ·
Job Title: QA Engineer (AI Applications) (Remote) Location : Remote * Work Mode: Fully Remote Role Overview Help design and evaluate autonomous AI agents across multiple LLMs, spanning health, education, daily life, and
Skills
3 weeks ago
Apply Now↗Apply Now ↗AI Application Engineer (Remote)
Jobs Ai ·
Job Title: AI Application Engineer (Remote) Location : Remote * Work Mode: Fully Remote Role Overview Help design and evaluate autonomous AI agents across multiple LLMs, spanning health, education, daily life, and other
Skills
3 weeks ago
Apply Now↗Apply Now ↗AI Application Engineer (Remote)
Jobs Ai ·
Job Title: AI Application Engineer (Remote) Location : Remote * Work Mode: Fully Remote Role Overview Help design and evaluate autonomous AI agents across multiple LLMs, spanning health, education, daily life, and other
Skills
3 weeks ago
Apply Now↗Apply Now ↗QA Engineer (AI Applications) (Remote)
Jobs Ai ·
Job Title: QA Engineer (AI Applications) (Remote) Location : Remote * Work Mode: Fully Remote Role Overview Help design and evaluate autonomous AI agents across multiple LLMs, spanning health, education, daily life, and
Skills
3 weeks ago
Apply Now↗Apply Now ↗2.2K+
Cover Letters & Follow-ups
1.8K+
Resumes Tailored
190.5K+
Jobs Tracked
Trusted by professionals at
Stop applying blindly.
Start getting hired.
Base Career automates the hardest parts of job searching — apply smarter, not harder.
AI Resume in 60s
Your resume rewritten for this exact role using the job description as the brief.
ATS-Optimized
Get past automated screening filters with the right keywords matched to each job.
Application Tracker
Track every job, follow-up, and interview in one visual kanban board.
Free plan · No credit card required