workday
Sr Analyst Model Risk & AI, Gen AI Model Validation
The Hartford
Hartford, USA
Full-time
Senior · 3+ years experience
Remote
Discovered 2 weeks ago
awsgcpgithuggingfacelangchainopenai
Key Skills for This Role
awsgcpgithuggingfacelangchainopenai
Full Job Posting
Responsibilities:
- Perform model validations on GenAI and agentic AI model use cases across The Hartford’s functional areas and lines of business to ensure models are performing effectively and efficiently
- − Ensure model calculations, algorithms, and methods are accurate and appropriate for intended use
- − Design and build challenger solutions and/or testing methods for tasks such as summarization, question answering, search, data synthesis, LLM-as-a-judge etc.
- − Review and assess quantitative and qualitative testing techniques to ensure model accuracy, robustness, and reliability
- − Assess key data inputs, assumptions, prompt engineering, and context engineering for accuracy and appropriateness
- − Design and execute testing approaches appropriate for non-deterministic AI systems, including variability analysis, confidence intervals, and statistical evaluation methodologies.
- − Deliver effective challenge to key modeling elements such as inputs, calculations, outputs, conceptual soundness, monitoring & controls, documentation, etc.
- − Identify findings and recommendations, including impact analysis, to mitigate model risk and compile clear and concise model validation reports
- Perform governance accountabilities related to findings tracking, remediation testing, and validation
- Assist in enhancing the existing GenAI model validation framework to include standardized evaluation metrics for performance and reliability, deployment of model validation tools for increased efficiency, and ensure continued alignment with regulatory standards
- Strengthen partnerships with Data Science teams to keep model risk practices aligned with the proliferation and sophistication of modeling, promote proactive risk management, and share best practices.
- Pro-actively stay informed with advancements in AI/ML, GenAI, agentic systems, and regulatory expectations for emerging technologies and of department initiatives, deliverables, and reporting
- Assist with the evaluation and testing of cutting-edge tools, such as VertexAI/Google Agent Development Kit (ADK), LangChain/LangGraph, RAG frameworks, HuggingFace, OpenAI APIs, etc.
- Assess solution alignment with internal AI Governance standards, Model Risk Management policies, and emerging regulatory expectations.
Qualifications:
- Advanced degree (M.S. or Ph.D.) in a relevant field e.g. Artificial Intelligence, Machine Learning, Computational Science, Engineering, Statistics, Applied Mathematics, Actuarial Science, Computer Science, Quantitative Economics.
- 3+ years of industry experience in machine learning or data science with 1+ years focused on GenAI and/or agentic AI.
- P&C, Group, Life or related insurance product experience is a plus
- Strong programming and technology experience across languages/ platforms including Python, SQL, and Git.
- Solid understanding of GenAI concepts including prompt and context engineering, retrieval-augmented generation (RAG), LLM internals, LLM evaluations, and multi-agent systems.
- Experience in modern cloud and AI tools such as GCP/Vertex AI, Google uAgent Development Kit (ADK), AWS/SageMaker, LangChain/LangGraph, HuggingFace, OpenAI APIs.
- Ability to act independently with proactive self-directed accountability and demonstrated experience and consistency in meeting deadlines while adapting to shifting priorities
- Strong analytical, critical, and investigative thinking skills
- Demonstrated commitment to lifelong learning with a strong desire for continuous development to keep pace with evolving modeling techniques and AI technologies.
- Solution oriented creativity, innovative thinking, and challenging the status quo.
- Excellent communication and collaboration skills, with the ability to explain complex technical concepts to non-technical stakeholders across the enterprise.
- Compensation
- The listed annualized base pay range is primarily based on analysis of similar positions in the external market. Actual base pay could vary and may be above or below the listed range based on factors including but not limited to performance, proficiency and demonstration of competencies required for the role. The base pay is just one component of The Hartford’s total compensation package for employees. Other rewards may include short-term or annual bonuses, long-term incentives, and on-the-spot recognition. The annualized base pay range for this role is:
- $108,000 - $162,000
- Equal Opportunity Employer/Sex/Race/Color/Veterans/Disability/Sexual Orientation/Gender Identity or Expression/Religion/Age
- About Us | Our Culture | What It’s Like to Work Here | Perks & Benefits
Preferred Qualifications
- P&C, Group, Life or related insurance product experience is a plus
About The Hartford
Insurance19200 employeesFounded 1810
Provider of property and casualty insurance and employee benefits.
Visit company websiteJobs and hiring trendsApply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at The Hartford
Senior Software Engineer / Data
, IND
Discovered TodayFull-time
IND Staff Engineer, Infrastructure
, IND
Discovered 3 days agoFull-time
IND Red Team Engineer, Security
, IND
Discovered 3 days agoFull-time
IND Associate Software Engineer - Red Team Engineer
, IND
Discovered 3 days agoFull-time
General Liability Claim Consultant/Specialist - Personal and Advertising Injury Liability
Hartford, USA
Discovered 3 days agoFull-time
Manager Learning Measurement and Impact
Hartford, USA
Discovered 4 days agoFull-time
Workers’ Compensation Senior Claim Representative
Hartford, USA
Discovered 4 days agoFull-time
Inland Marine Underwriter
Charlotte, USA
Discovered 5 days agoFull-time