{bc}
Apply
greenhouse

Principal AI Evaluations Platform Engineer

Microsoft
London, GBR
Full-time
Senior · 4+ years experience
Hybrid
CHF 183800-309700 yearly / year
Discovered 3 weeks ago
PythonRustRayKubernetesSlurmDatadog

Full Job Posting

Responsibilities

  • Serve as a primary engineer for the evaluation platform during European hours, providing dedicated coverage for the evaluation stack supporting large-scale model training.
  • Develop and extend core evaluation platform capabilities, including benchmark configurations, problem sets, graders, and evaluation runners.
  • Improve evaluation throughput and scheduling efficiency.
  • Build and maintain monitoring, alerting, and dashboards to proactively detect evaluation degradation.
  • Partner closely with researchers and training teams across European offices to define evaluation requirements, coordinate launches, and unblock experiments.
  • Contribute to technical design and code reviews across the evaluation platform stack, and maintain high-quality operational documentation and on-call runbooks.
  • Embody our culture and values.

Required Qualifications

  • Bachelor's Degree in Computer Science, Engineering, Mathematics, or a related field AND 4+ years of technical engineering experience with coding in languages including, but not limited to, Python, C++, Go, or Rust;
  • OR Master's Degree in Computer Science, Engineering, Mathematics, or a related field AND 2+ years of technical engineering experience with coding in languages including, but not limited to, Python, C++, Go, or Rust;
  • OR equivalent experience.
  • Demonstrated experience operating production distributed systems, including participation in a formal on-call rotation.
  • Strong proficiency in Python, including experience working in a large, shared, typed codebase.
  • Experience debugging distributed system failures across the stack, including scheduling, resource exhaustion, networking, and process-level faults.
  • Willingness and availability to participate in a scheduled on-call rotation covering European business hours.

Preferred Qualifications

  • Experience with GPU-based workloads, distributed training, or large-scale inference infrastructure.
  • Experience with distributed compute frameworks and schedulers, such as Ray, Kubernetes, Slurm, or comparable systems.
  • Experience with observability and alerting tooling, such as Datadog, Prometheus, or Grafana.
  • Familiarity with LLM evaluation, benchmarking, or reinforcement learning training infrastructure.
  • Track record of reducing operational burden through systematic reliability improvements, automation, or tooling.
  • Ability to work effectively in a fast-paced environment, manage multiple priorities, and adapt to changing requirements and deadlines.
  • The typical base pay range for this role across Switzerland is CHF 183,800.00 - CHF 309,700.00 per year. Certain roles may be eligible for benefits and other compensation.
  • Find additional benefits and pay information here: https://careers.microsoft.com/v2/global/en/corporate-pay/switzerland-corporate-pay
  • Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay
  • Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.

Apply for this job in 1 click

Skip the repetitive application forms

Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.

Sarah M.James T.Maya R.

Trusted by over 500,000 job seekers on Base Career

Start Free Today

More from this employer

More jobs at Microsoft

Senior Go-to-Market Manager

Redmond, USA

Discovered TodayFull-time

Principal Software Engineer

Redmond, USA

Discovered TodayFull-time

Software Engineer, Teamwork Experiences Internship Opportunities

Redmond, USA

Discovered TodayFull-time

EHS Campus Manager

Phoenix, USA

Discovered TodayFull-time

Critical Environment Technician Manager (CETM)

, GBR

Discovered TodayFull-time

Senior Software Engineer

Redmond, USA

Discovered TodayFull-time

Product Manager II

Redmond, USA

Discovered TodayFull-time

Cloud and AI Solution Specialist

Atlanta, USA

Discovered TodayFull-time