{bc}
Apply
workable

Software Engineer, Python - Codebase QA

Weekday
Remote, USA
Contract
Mid · 3+ years experience
Remote
$130
Discovered 2 days ago
oktapython

Full Job Posting

Key Responsibilities

  • Select tasks from a pre-validated pool of Python repositories, each pinned to a specific git commit and arriving with an engineering question, positive and negative rubrics, foils, and a golden solution
  • Explore the repository until you genuinely understand the subsystem in question, including the relevant PR or issue history
  • Rewrite the task's question so it defeats frontier coding agents while remaining well-defined and fairly answerable
  • Adjust rubrics and foils so they reward correct reasoning and reject plausible-but-wrong answers
  • Run the full validation loop in Studio — Check, Golden, Run, Analyze — and iterate until every stage passes
  • Document your work in the task README and submit for expert review, addressing reviewer feedback on returned tasks
  • Ideal Qualifications
  • 3+ years of professional software engineering experience, with substantial production Python
  • Demonstrated ability to navigate a large, unfamiliar Python codebase and explain how it behaves at runtime — not just what the source says
  • Fluency in Python internals that matter in real systems: import machinery, descriptors and metaclasses, async and event loops, packaging, and C-extension boundaries
  • Experience with web frameworks, developer tooling, type checkers, or scientific/numeric libraries is highly relevant — the task mix is weighted toward architecture and system design (41%) and code onboarding (31%)
  • Precise written English: the questions and rubrics you write are the product
  • Comfortable with git at commit level, containerized environments, and command-line tooling
  • Working Details
  • Fully remote and asynchronous, with expert team leads across US, India, and Nigeria time zones
  • Work is performed in our Studio; time is tracked with Insightful, and platform access is provisioned through Okta
  • Onboarding requires identity verification, a background check, signed Terms of Work and CIIAA, a tax form, and passing a short calibration quiz before task access is granted
  • Payment is per approved task; every task is reviewed by a second engineer and a super reviewer before acceptance
  • Application Process
  • Submit your resume or relevant technical background to get started
  • Qualified applicants may be asked to complete a brief technical assessment or provide additional information
  • We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.

Qualifications

  • Key Responsibilities
  • Select tasks from a pre-validated pool of Python repositories, each pinned to a specific git commit and arriving with an engineering question, positive and negative rubrics, foils, and a golden solution
  • Explore the repository until you genuinely understand the subsystem in question, including the relevant PR or issue history
  • Rewrite the task's question so it defeats frontier coding agents while remaining well-defined and fairly answerable
  • Adjust rubrics and foils so they reward correct reasoning and reject plausible-but-wrong answers
  • Run the full validation loop in Studio — Check, Golden, Run, Analyze — and iterate until every stage passes
  • Document your work in the task README and submit for expert review, addressing reviewer feedback on returned tasks
  • Ideal Qualifications
  • 3+ years of professional software engineering experience, with substantial production Python
  • Demonstrated ability to navigate a large, unfamiliar Python codebase and explain how it behaves at runtime — not just what the source says
  • Fluency in Python internals that matter in real systems: import machinery, descriptors and metaclasses, async and event loops, packaging, and C-extension boundaries
  • Experience with web frameworks, developer tooling, type checkers, or scientific/numeric libraries is highly relevant — the task mix is weighted toward architecture and system design (41%) and code onboarding (31%)
  • Precise written English: the questions and rubrics you write are the product
  • Comfortable with git at commit level, containerized environments, and command-line tooling
  • Working Details
  • Fully remote and asynchronous, with expert team leads across US, India, and Nigeria time zones
  • Work is performed in our Studio; time is tracked with Insightful, and platform access is provisioned through Okta
  • Onboarding requires identity verification, a background check, signed Terms of Work and CIIAA, a tax form, and passing a short calibration quiz before task access is granted
  • Payment is per approved task; every task is reviewed by a second engineer and a super reviewer before acceptance
  • Application Process
  • Submit your resume or relevant technical background to get started
  • Qualified applicants may be asked to complete a brief technical assessment or provide additional information
  • We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.

Apply for this job in 1 click

Skip the repetitive application forms

Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.

Sarah M.James T.Maya R.

Trusted by over 500,000 job seekers on Base Career

Start Free Today

More from this employer

More jobs at Weekday

Senior Engagement Manager

Bengaluru, IND

Discovered TodayFull-time

Finance Professional

, USA

Discovered TodayFull-time

Head of HR

Bengaluru, IND

Discovered TodayFull-time

Specialist Data Analytics Alteryx

Hyderabad, IND

Discovered TodayFull-time

Data Scientist

, USA

Discovered TodayFull-time

Management Consultant

, USA

Discovered TodayFull-time

Demand Generation Manager

Bengaluru, IND

Discovered TodayFull-time

Sr Manager Customer sucess Enterprise

Bengaluru, IND

Discovered TodayFull-time