workable
Senior SWE - Python / TypeScript
Weekday
Remote, USA
Contract
Senior · 5+ years experience
Remote
Discovered 2 weeks ago
javascriptpythontypescript
Key Skills for This Role
javascriptpythontypescript
Full Job Posting
Requirements
- Key Responsibilities
- Rubric Development: Design, refine, and iterate on rubrics, evaluation frameworks, and quality criteria for preference data.
- Data Review and Quality Assurance: Review, assess, and label data while identifying subtle inconsistencies, quality issues, and opportunities for improvement.
- Engineering Infrastructure: Build, maintain, and optimize pipelines and infrastructure supporting data generation, collection, processing, and evaluation workflows.
- Technical Documentation: Synthesize findings from data analysis and evaluation activities into concise reports, updates, recommendations, and actionable summaries.
- Cross-Functional Collaboration: Work closely with researchers and engineers to translate qualitative judgments into structured, repeatable, and scalable processes.
- Codebase Navigation: Understand, contribute to, and maintain complex software repositories while ensuring code quality, reliability, and performance.
- Continuous Improvement: Identify workflow inefficiencies and implement practical improvements to data quality, evaluation consistency, and engineering processes.
- Required Skills and Qualifications
- At least 5 years of professional software engineering experience with Python, particularly in building and maintaining pipelines and infrastructure, or with TypeScript, JavaScript, or Go.
- Hands-on experience with preference data collection, rubric creation, or developing evaluation criteria.
- Strong understanding of software engineering principles, code quality, debugging, and maintainable system design.
- Excellent written and verbal communication skills, with the ability to explain complex technical concepts clearly and concisely.
- Strong analytical skills and the ability to make consistent, well-reasoned quality judgments.
- Demonstrated ability to explain why a particular solution, example, or implementation meets a high quality standard.
- Comfort working independently in ambiguous environments while prioritizing execution and timely delivery.
- Preferred Qualifications
- Experience contributing to or navigating large-scale, complex open-source codebases, such as VS Code.
- Experience building data pipelines, developer infrastructure, or automated evaluation systems.
- Background in AI model evaluation, preference data workflows, data annotation, or quality assessment.
- Experience collaborating with AI researchers, machine learning engineers, or multidisciplinary technical teams.
- Education from a leading university or professional experience at a highly regarded technology company.
- Evaluation and Selection Process
- Candidates will undergo the following evaluation stages:
- Application and Screening: Submit an application and complete the required screening questions.
- AI Interview: Complete an AI-based interview.
- Coding Challenge: Demonstrate technical proficiency through a coding assessment.
- Technical and Cultural Interview: Participate in an approximately 30-minute discussion covering technical expertise, communication, judgment, and collaboration.
- Final Review: Complete the final candidate evaluation and selection process.
- Additional Requirements
- A LinkedIn profile is mandatory and must be included in the candidate's resume.
- Candidates from leading technology companies and top-tier universities are strongly preferred.
- Candidates must not currently be employed by Microsoft.
- Applicants should be able to commit to the required weekly hours and PST overlap for the full contract duration.
- Required: 5+ years
Preferred Qualifications
- Experience contributing to or navigating large-scale, complex open-source codebases, such as VS Code.
- Experience building data pipelines, developer infrastructure, or automated evaluation systems.
- Background in AI model evaluation, preference data workflows, data annotation, or quality assessment.
- Experience collaborating with AI researchers, machine learning engineers, or multidisciplinary technical teams.
- Education from a leading university or professional experience at a highly regarded technology company.
- Evaluation and Selection Process
- Application and Screening: Submit an application and complete the required screening questions.
- AI Interview: Complete an AI-based interview.
- Coding Challenge: Demonstrate technical proficiency through a coding assessment.
- Technical and Cultural Interview: Participate in an approximately 30-minute discussion covering technical expertise, communication, judgment, and collaboration.
- Final Review: Complete the final candidate evaluation and selection process.
- Candidates from leading technology companies and top-tier universities are strongly preferred.
Responsibilities
- Rubric Development: Design, refine, and iterate on rubrics, evaluation frameworks, and quality criteria for preference data.
- Data Review and Quality Assurance: Review, assess, and label data while identifying subtle inconsistencies, quality issues, and opportunities for improvement.
- Engineering Infrastructure: Build, maintain, and optimize pipelines and infrastructure supporting data generation, collection, processing, and evaluation workflows.
- Technical Documentation: Synthesize findings from data analysis and evaluation activities into concise reports, updates, recommendations, and actionable summaries.
- Cross-Functional Collaboration: Work closely with researchers and engineers to translate qualitative judgments into structured, repeatable, and scalable processes.
- Codebase Navigation: Understand, contribute to, and maintain complex software repositories while ensuring code quality, reliability, and performance.
- Continuous Improvement: Identify workflow inefficiencies and implement practical improvements to data quality, evaluation consistency, and engineering processes.
- Required Skills and Qualifications
- At least 5 years of professional software engineering experience with Python, particularly in building and maintaining pipelines and infrastructure, or with TypeScript, JavaScript, or Go.
- Hands-on experience with preference data collection, rubric creation, or developing evaluation criteria.
- Strong understanding of software engineering principles, code quality, debugging, and maintainable system design.
- Excellent written and verbal communication skills, with the ability to explain complex technical concepts clearly and concisely.
- Strong analytical skills and the ability to make consistent, well-reasoned quality judgments.
- Demonstrated ability to explain why a particular solution, example, or implementation meets a high quality standard.
- Comfort working independently in ambiguous environments while prioritizing execution and timely delivery.
About Weekday
Founded 2021
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Weekday
Manager - Direct Tax
Mumbai, IND
Discovered TodayFull-time
Guidewire Integration Developer
Delhi, IND
Discovered YesterdayFull-time
Digital Risk Consultant
Bengaluru, IND
Discovered YesterdayFull-time
Key Accounts Manager
Bengaluru, IND
Discovered YesterdayFull-time
Regional Credit Manager
Delhi, IND
Discovered YesterdayFull-time
ServiceNow Developer
Gurugram, IND
Discovered YesterdayFull-time
Enterprise ERP and Cloud Solutions Sales
Mumbai, IND
Discovered YesterdayFull-time
Team Manager Benefits
Bengaluru, IND
Discovered YesterdayFull-time