Principal Applied Scientist - Copilot Quality

Build trusted measures of quality Develop online and offline metrics, representative evaluation datasets, human and AI-assisted labeling approaches, and benchmarks grounded in real customer tasks. Validate that quality measures are statistically sound, sensitive to meaningful changes, and predictive of customer outcomes. Improve quality through hill-climbing approaches such as post-training, retrieval and ranking, prompting, orchestration, and product changes. Design experiments that distinguish promising offline results from improvements customers actually experience. Establish rigorous A/B experimentation, quality gates, regression protection, and rollout criteria that connect offline gains with improvements in satisfaction, task success, engagement, retention, latency, reliability, safety, and cost. Bachelor's Degree in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 8+ years related experience (e.g., statistics, predictive analytics, research) OR Master's Degree in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 6+ years related experience (e.g., statistics, predictive analytics, research) OR Doctorate in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 5+ years related experience (e.g., statistics, predictive analytics, research) OR equivalent experience. Advanced degree (MS or PhD) in a relevant technical discipline.

improving large language models, AI assistants, agents, or other large-scale AI products.

Ability to operate in a fast‐paced, research‐driven environment while maintaining high technical standards.

Passion for advancing human‐aligned AI and staying at the forefront of emerging model‐training technologies.

Demonstrated commitment to inclusive leadership, talent development, and fostering a culture where teams do the best work of their careers.

Significant experience building, evaluating, or improving production ML or AI systems.

Demonstrated expertise in at least one relevant area, such as large-model evaluation, deep learning, post-training, reward modeling, reinforcement learning, retrieval, or ranking.

designing rigorous metrics or experiments and using results to guide technical and product decisions.