Medical AI Evaluator (Remote, Hourly Contractor)
Role Overview
In this remote, hourly contractor role, you will evaluate AI-generated medical content and develop cases that test clinical reasoning accuracy. Your work directly improves how leading AI models handle healthcare information, making them more accurate, reliable, and safe. Tasks may include:
Evaluating AI-generated medical responses for clinical accuracy, reasoning quality, and patient safety implications
Identifying errors in clinical methodology, unsafe assumptions, missing contraindications, and misinterpretation of diagnostic data
Writing clear, precise feedback explaining corrections and reasoning gaps
Developing prompts and test cases that probe AI accuracy across clinical scenarios
Rating and comparing AI responses based on correctness, internal consistency, and contextual appropriateness
Fact-checking medical content against reliable sources using consistent reasoning
Full Job Posting
Position Summary
In this remote, hourly contractor role, you will evaluate AI-generated medical content and develop cases that test clinical reasoning accuracy. Your work directly improves how leading AI models handle healthcare information, making them more accurate, reliable, and safe. Tasks may include:
Evaluating AI-generated medical responses for clinical accuracy, reasoning quality, and patient safety implications
Identifying errors in clinical methodology, unsafe assumptions, missing contraindications, and misinterpretation of diagnostic data
Writing clear, precise feedback explaining corrections and reasoning gaps
Developing prompts and test cases that probe AI accuracy across clinical scenarios
Rating and comparing AI responses based on correctness, internal consistency, and contextual appropriateness
Fact-checking medical content against reliable sources using consistent reasoning
Profile Requirements
- Bachelor's degree or higher in Medicine (MD/DO), Nursing, Public Health, Health Sciences, or Allied Health
- 5+ years of professional experience in a relevant healthcare discipline
- Strong clinical reasoning skills: differential diagnosis, risk stratification, red-flag recognition
- Solid grounding in disease processes, patient care, public health principles, and medical terminology
- Full professional English proficiency
- Exceptional attention to detail and ability to explain corrections clearly in writing
- Reliable and self-directed, with consistent output quality in a remote, asynchronous workflow
Preferred Experience
Experience in clinical documentation review, utilization review, or healthcare editorial QA
Prior experience with AI data training or annotation
About CNTXT AI
CNTXT AI builds artificial intelligence products and data solutions with a focus on making AI accurate, safe, and globally relevant for impact. Our work spans data services, custom AI solutions, and proprietary AI products, with deep expertise in Arabic-native and secure, sovereign solutions.
About CNTXT AI
UAE-based Arabic-first AI company helping governments and enterprises build and scale sovereign AI solutions.
Visit company websiteJobs and hiring trendsApply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at CNTXT AI
Voice Over Artist - Saudi Dialect
, KSA
CNTXT AI is seeking a Saudi-dialect Voice Over Artist to support voice cloning and AI training data work. The role requires strong Saudi dialect diction and control, prior voice over experience, and a professional home s
Senior QA Engineer
Abu Dhabi, UAE
CNTXT AI is seeking a Senior QA Engineer to test web, mobile, API, and AI-enabled applications, with particular attention to Arabic-language content and interfaces. The role requires five or more years of QA experience,
C++ Coding Expert – AI Training (Remote, Hourly Contractor)
Abu Dhabi, UAE
CNTXT AI seeks a C++ coding expert to evaluate AI-generated code and create prompts and test cases that assess coding accuracy and reasoning. Applicants need modern C++ proficiency, professional English, and strong code-
Arabic/English Bilingual AI Data Annotator & Evaluator (Remote, Hourly Contractor)
Riyadh, KSA
CNTXT AI is seeking an Arabic/English bilingual annotator and evaluator for remote AI language projects on a flexible, project-based hourly contractor basis. Applicants need native or near-native Arabic, C1 English, and
Senior Marketing Executive - Tech
Dubai, UAE
CNTXT AI is seeking a Senior Marketing Executive with technology or AI marketing experience to grow awareness, qualified leads, and adoption of its products. The role combines B2B campaigns, product messaging, content, a
Python Coding Expert – AI Training (Remote, Hourly Contractor)
Abu Dhabi, UAE
CNTXT AI seeks a Python coding expert to evaluate AI-generated code and create prompts and test cases for AI training. The work includes assessing correctness, efficiency, readability, and reasoning quality. Strong Pytho
Lead Security Engineer
Abu Dhabi, UAE
CNTXT seeks a hands-on security lead to establish and implement cybersecurity across its applications, cloud infrastructure, endpoints, identity systems, and AI/data products. Candidates need 7–10+ years in security engi
Voice Over Artist Saudi Dialect
, KSA
Voice Over Artist - Saudi Dialect
, KSA
Senior QA Engineer
Abu Dhabi, UAE
C++ Coding Expert – AI Training (Remote, Hourly Contractor)
Abu Dhabi, UAE
Arabic/English Bilingual AI Data Annotator & Evaluator (Remote, Hourly Contractor)
Riyadh, KSA
Senior Marketing Executive - Tech
Dubai, UAE
Python Coding Expert – AI Training (Remote, Hourly Contractor)
Abu Dhabi, UAE
Lead Security Engineer
Abu Dhabi, UAE