Language Model Evaluator - Fully Remote | Upto $20/hr Part-time
About The Job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.
Skills
About This Role
Mercor
connects elite creative and technical talent with leading AI research labs.
Headquartered in San Francisco, our investors include
Benchmark
,
General Catalyst
,
Peter Thiel
,
Adam D'Angelo
,
Larry Summers
, and
Jack Dorsey
.
Compensation
$15–$20/hour
Role Responsibilities
- Conduct fact-checking using trusted public sources and external tools.
- Generate high-quality human evaluation data by identifying response strengths, areas for improvement, and factual inaccuracies.
- Assess reasoning quality, clarity, tone, and completeness of responses.
- Ensure model responses align with expected conversational behavior and system guidelines.
- Work independently and asynchronously to meet deadlines while improving AI model performance.
Must-Have
- Bachelor's degree
- Native speaker in Urdu
- Significant experience using large language models (LLMs)
- Excellent writing skills in English
- Strong attention to detail
- Background or experience in domains requiring structured analytical thinking
Preferred
- Experience with RLHF, model evaluation, or data annotation work
- Experience writing or editing high-quality written content
- Experience comparing multiple outputs and making fine-grained qualitative judgments
- Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome
- For any help or support, reach out to: support@mercor.com
- *PS: Our team reviews applications daily.
- Please complete your AI interview and application steps to be considered for this opportunity.*
Your resume, rewritten
for this exact role.
Sign up free — Base Career tailors your CV to this job description in 60 seconds.
01 / 05
Resume Tailored to This Job
Your keywords, structure, and story — rewritten to match this exact role and pass ATS filters.
Free · No card · 60 seconds
02 / 05
Cover Letter for This Role, Done
Job-specific cover letters written in Gulf professional tone — ready in seconds, not hours.
Free · No card · 60 seconds
03 / 05
See How Well You Fit This Role
AI match score with clear reasons — know your fit before investing time in the application.
Free · No card · 60 seconds
04 / 05
Use Autofill When You Apply
Autofill any application form on Workday, LinkedIn, Bayt, Greenhouse — with your tailored content.
Free · No card · 60 seconds
05 / 05
Track It. Follow Up at the Right Time.
Visual pipeline for every application with AI-timed follow-up reminders so nothing slips.
Free · No card · 60 seconds
Similar Jobs
Large Language Model Trainer (Remote)
Hired · Abu Dhabi
Role: Large Language Model Trainer (Remote) Location: Remote (Work from Anywhere) * Payout: Competitive Role Overview: We are hiring for one of our clients, seeking a LLM Trainer - Agent Function Call to work on a Full-
Skills
Principal AI – Large Language Models Expert
UNEY · Dubai
We are seeking a Principal AI – Large Language Model (LLM) Expert to lead the design, training, optimization, and deployment of language models across our privacy-first security platform. This role combines deep research
Skills
5 days ago
Tailor Resume↗Tailor Resume ↗Large Language Model Trainer (Remote)
Hired · Abu Dhabi
Role: Large Language Model Trainer (Remote) Location: Remote (Work from Anywhere) * Payout: Competitive Role Overview: We are hiring for one of our clients, seeking a LLM Trainer - Agent Function Call to work on a Full-
Skills
5 days ago
Tailor Resume↗Tailor Resume ↗Language Model Analyst - Fully Remote | Upto $20/hr Part-time
Mercor · Dubai
About The Job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Su
Skills
6 days ago
Tailor Resume↗Tailor Resume ↗Language Model Analyst - Fully Remote | Upto $20/hr Part-time
Mercor · Dubai
About The Job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Su
Skills
1 weeks ago
Tailor Resume↗Tailor Resume ↗Language Model Evaluator - Fully Remote | Upto $20/hr Part-time
Mercor · Dubai
About The Job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Su
Skills
1 weeks ago
Tailor Resume↗Tailor Resume ↗Language Model Analyst - Fully Remote | Upto $20/hr Part-time
Mercor · Abu Dhabi
About The Job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Su
Skills
1 weeks ago
Tailor Resume↗Tailor Resume ↗Language Model Evaluator - Fully Remote | Upto $20/hr Part-time
Mercor · Abu Dhabi
About The Job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Su
Skills
1 weeks ago
Tailor Resume↗Tailor Resume ↗2.2K+
Cover Letters & Follow-ups
1.8K+
Resumes Tailored
190.5K+
Jobs Tracked
Trusted by professionals at
Stop applying blindly.
Start getting hired.
Base Career automates the hardest parts of job searching — apply smarter, not harder.
AI Resume in 60s
Your resume rewritten for this exact role using the job description as the brief.
ATS-Optimized
Get past automated screening filters with the right keywords matched to each job.
Application Tracker
Track every job, follow-up, and interview in one visual kanban board.
Free plan · No credit card required