Senior Machine Learning Engineer - AI-Assisted Data Annotation
Role Overview
We are seeking a Senior Machine Learning Engineer – AI-Assisted Data Annotation to own the automated annotation track within ABBYY’s Document AI Data team.
This role sits at the intersection of large model capabilities and production data engineering, leveraging LLMs and vision-language models to generate high-quality training data at scale. You will design and build AI-assisted annotation pipelines, ensuring outputs are accurate, measurable, and reliable for downstream model training.
This is an ideal role for engineers who combine deep model expertise with strong system-building instincts and thrive in fast-moving, experimental environments.
Key Responsibilities
Technical Development & Innovation
Design and implement AI-powered annotation pipelines using large models to generate ground truth labels at scale
Develop and refine prompting strategies, few-shot examples, and fine-tuning approaches to improve accuracy and consistency
Build systems for label verification, confidence scoring, and quality validation
Evaluate which tasks are suitable for automated annotation vs. human review, and define decision criteria
Create evaluation frameworks to benchmark automated annotations against human-labeled data
Continuously improve annotation quality using feedback from human review workflows
Project Ownership & Leadership
Own the automated annotation track end-to-end, from architecture through production monitoring
Drive technical decisions across model selection, pipeline design, and validation strategies
Define integration points with platform infrastructure and model serving systems
Collaborate with Data Operations to design human-in-the-loop workflows for efficient review
Contribute to roadmap planning with Principal-level technical leadership
Infrastructure & Scale
Build and optimize large-scale inference pipelines for processing millions of documents
Implement monitoring and alerting for quality degradation and system failures
Design batching, caching, and fallback mechanisms to balance cost, throughput, and accuracy
Collaborate with Platform teams on model serving, APIs, and infrastructure scaling
Maintain clear documentation of annotation strategies, metrics, and known limitations
Key Skills for This Role
Full Job Posting
About the Role
We are seeking a Senior Machine Learning Engineer – AI-Assisted Data Annotation to own the automated annotation track within ABBYY’s Document AI Data team.
This role sits at the intersection of large model capabilities and production data engineering, leveraging LLMs and vision-language models to generate high-quality training data at scale. You will design and build AI-assisted annotation pipelines, ensuring outputs are accurate, measurable, and reliable for downstream model training.
This is an ideal role for engineers who combine deep model expertise with strong system-building instincts and thrive in fast-moving, experimental environments.
Key Responsibilities
Technical Development & Innovation
Design and implement AI-powered annotation pipelines using large models to generate ground truth labels at scale
Develop and refine prompting strategies, few-shot examples, and fine-tuning approaches to improve accuracy and consistency
Build systems for label verification, confidence scoring, and quality validation
Evaluate which tasks are suitable for automated annotation vs. human review, and define decision criteria
Create evaluation frameworks to benchmark automated annotations against human-labeled data
Continuously improve annotation quality using feedback from human review workflows
Project Ownership & Leadership
Own the automated annotation track end-to-end, from architecture through production monitoring
Drive technical decisions across model selection, pipeline design, and validation strategies
Define integration points with platform infrastructure and model serving systems
Collaborate with Data Operations to design human-in-the-loop workflows for efficient review
Contribute to roadmap planning with Principal-level technical leadership
Infrastructure & Scale
Build and optimize large-scale inference pipelines for processing millions of documents
Implement monitoring and alerting for quality degradation and system failures
Design batching, caching, and fallback mechanisms to balance cost, throughput, and accuracy
Collaborate with Platform teams on model serving, APIs, and infrastructure scaling
Maintain clear documentation of annotation strategies, metrics, and known limitations
Qualifications
- Education & Experience
- MS or PhD in Computer Science, Engineering, Mathematics, or related field
- 5+ years of experience in Machine Learning / AI, with focus on:
- Large Language Models (LLMs)
- Vision-Language Models (VLMs)
- Data annotation or labeling systems
- Demonstrated success using large AI models to automate annotation at production scale
- Strong background in evaluation design and quality measurement
- Technical Expertise
- Deep expertise in LLMs and VLMs, including prompting, instruction tuning, and output evaluation
- Strong understanding of document understanding tasks (classification, extraction, layout analysis, semantic parsing)
- Experience designing label quality metrics, confidence scoring, and agreement analysis
- Strong programming skills in Python and proficiency with PyTorch or similar frameworks
- Experience with large-scale inference pipelines and model serving systems
- Familiarity with human-in-the-loop annotation systems and automation trade-offs
- Leadership & Communication
- Proven ability to independently own complex technical workstreams
- Strong collaboration with data operations, platform, and modeling teams
- Ability to clearly communicate quality trade-offs and system behavior to diverse stakeholders
- Rigorous, data-driven problem-solving approach
- Here are some of our local benefits:
- Comprehensive medical, accidental, and life insurance
- Weekly wellness sessions to support your physical and mental well-being
- A generous paid time off policy
- Join ABBYY, and you will:
- Love how you work
- We provide remote and hybrid working options to fit all lifestyles.
- We use flexible hours across most of our teams to allow you to find your own definition of balance.
- Encouraging a culture of giving, we provide two paid volunteering days off every year so you can take time to contribute to the causes you care about.
- To ensure your family is cared for, we offer paid parental leave in all our locations.
- Love whom you work with
- We are a global team of 600+ colleagues, spread across 15 countries on four continents.
- With colleagues representing 30+ nationalities, our workforce reflects the world.
- Innovation and excellence run through our veins. Our teams gather the expertise which has garnered ABBYY more than 140 technology patents.
- We are guided by the values of respect, transparency, and simplicity.
- "Team Environment" is in the top three highest-scoring drivers of engagement across all of our departments.
- Love what you work on
- We are a company with more than 35 years of experience in the technology market;
- Over 10,000 customers trust ABBYY, including many Fortune 500 ones, with names such as DHL, Johnson & Johnson, FDA, DMV, PwC, KeyBank, Spotify, and H&R BLOCK;
- We have modernized the capture market by creating the first low-code/no-code IDP platform.
- Our Machine Learning, Natural Language Processing, Computer Vision Technologies, and a marketplace built with AI, can transform any document in any process;
- Top Analyst firms recognize ABBYY's market leadership, including Gartner, Everest PEAK Matrix ® Assessment, ISG Intelligent Automation Lens, and NelsonHall, amongst others.
- ABBYY is an Equal Employment Opportunity employer that values the strength that diversity brings to the workplace. To learn more about our commitment to Diversity and Inclusion, check out the careers section on our website.
About ABBYY
ABBYY is an enterprise AI company founded in 1989 that develops intelligent document processing, process mining, and task mining solutions used by over 10,000 customers including many Fortune 500 companies.
Visit company websiteJobs and hiring trendsApply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at ABBYY
Technical Success Manager
, IND
ABBYY is seeking a Technical Success Manager to guide enterprise customers through intelligent document processing deployments and help them realize technical and business value. Candidates need 5+ years in a technical c
Senior Machine Learning Engineer, Synthetic Data & Document Understanding
Bengaluru, IND
Solution Development Representative (SaaS/OCR/IDP) – Italian Speaking
London, GBR
Principal Machine Learning Engineer, Document AI Data (Tech Lead Manager)
Bengaluru, IND
Demand Generation and Field Marketing Specialist (6 -8 months contract)
Bengaluru, IND
Technical Success Manager
, IND
Senior Machine Learning Engineer, Synthetic Data & Document Understanding
Bengaluru, IND
Solution Development Representative (SaaS/OCR/IDP) – Italian Speaking
London, GBR
Technical Success Manager
Bengaluru, IND
Principal Machine Learning Engineer, Document AI Data (Tech Lead Manager)
Bengaluru, IND
Demand Generation and Field Marketing Specialist (6 -8 months contract)
Bengaluru, IND
