Principal Software Engineer

Role overview

Responsibilities

• Set platform technical direction and partner across Copilot to define rigorous, reproducible evaluation scenarios, metrics, and scorecards.
• Lead architecture of an AI-first, agentic-first platform with services, pipelines, and tooling designed for autonomous-agent operation.
• Design observability, guardrails, graceful degradation, intelligent retries, and dependency-aware gating for reliable operation.
• Define end-to-end agentic evaluation across multi-turn trajectories, tool selection, task outcomes, UX, and voice; build simulation, scraping, and scoring capabilities.
• Mentor engineers, modernize the evaluation runtime, and scale extensible systems while improving cost efficiency and latency.
• Share emerging practices in agentic systems, large-scale AI evaluation, and autonomous operations across the organization.

Required qualifications

• Bachelor’s degree in computer science or a related technical field and six or more years of technical engineering experience coding in languages such as C, C++, C#, Java, JavaScript, or Python, or equivalent experience.
• Must meet applicable Microsoft, customer, or government security screening requirements, including the Microsoft Cloud Background Check upon hire or transfer and every two years thereafter.

Preferred qualifications

• Master’s degree and eight or more years of technical engineering experience, or bachelor’s degree and twelve or more years, or equivalent experience.
• Deep experience with large-scale distributed systems and reliability engineering across interacting dependencies under heavy or unusual load.
• Experience building AI-first or agentic systems and designing systems, APIs, and tooling for autonomous agents, including observability, guardrails, and safety mechanisms.
• Experience with AI/ML evaluation, measurement, or benchmarking, such as LLM-based grading, metric design, or large-scale data and quality pipelines.

Compensation

• Typical base pay range across Canada is CAD $142,400–$257,500 per year for Software Engineering IC5.

About Microsoft

Application

• The position will be open for at least five days, with applications accepted on an ongoing basis until it is filled.