Associate, AI Model Validation, RQA

Key Responsibilities:

• Conduct independent validation of AI usage in workflows, including testing for accuracy, efficiency of guardrails, orchestration, memory management and token usage.
• Design and implement comprehensive validation strategies and test plans tailored to each AI use case. Define clear success criteria and quality benchmarks to measure whether the AI solution meets all requirements and standards.
• Understand tool architecture, potential vulnerabilities and weaknesses.
• Assess conceptual soundness, methodology, and alignment with intended business use.
• Review data quality, preprocessing steps and engineering decisions for AI use cases.
• Execute functional and performance tests.
• Evaluate performance metrics, robustness, and limitations under various scenarios.
• Document validation findings, provide recommendations
• Stay current on emerging AI technologies, validation techniques, and regulatory developments.
• Communicate validation conclusions to relevant stakeholders, including escalating identified findings, risks or gaps to senior leadership and approval committees
• Collaborate with AI Engineering, Data Science, and Risk teams to enhance AI transparency and trustworthiness.
• Experience, Skills and Qualification:
• With 3-7 years’ works experience in the IT or financial industry.
• Degree or Advanced degree (MS, PhD) in Computer Science, Data Science, Statistics, Applied Mathematics, or related field.
• Strong understanding of generative AI technologies, including large language models (LLMs), retrieval-augmented generation (RAG), agentic systems, prompt engineering, tool use, and modern AI application architectures.
• Experience with Python and common AI development libraries and frameworks (PyTorch, TensorFlow or similar).
• Excellent analytical, communication, and documentation skills.
• Ability to work independently and challenge stakeholders constructively.
• Experience with one or more of the following:
• Validating usage of large language models (LLMs) in business context
• Evaluation methodologies, benchmark development, ground-truth creation, and AI quality assessment
• AI observability, monitoring, performance measurement, and operational controls
• AI orchestration frameworks, agent architectures, prompt engineering, memory management, and context optimization
• Exposure to cloud-based AI platforms and scalable model deployment
• MCP, APIs, connectors and integrations of AI solutions with enterprise systems and data sources
• Responsible AI practices, including fairness, explainability, bias assessment, transparency, and model governance
• Fine-tuning or model customization techniques
• Security testing and risk assessment of AI systems, including prompt injection, jailbreak, and data leakage testing
• Our benefits To help you stay energized, engaged and inspired, we offer a wide range of benefits including a strong retirement plan, tuition reimbursement, comprehensive healthcare, support for working parents and Flexible Time Off (FTO) so you can relax, recharge and be there for the people you care about.
• Our hybrid work model
• BlackRock’s hybrid work model is designed to enable a culture of collaboration and apprenticeship that enriches the experience of our employees, while supporting flexibility for all. Employees are currently required to work at least 4 days in the office per week, with the flexibility to work from home 1 day a week. Some business groups may require more time in the office due to their roles and responsibilities. We remain focused on increasing the impactful moments that arise when we work together in person – aligned with our commitment to performance and innovation. As a new joiner, you can count on this hybrid model to accelerate your learning and onboarding experience here at BlackRock.
• Guidance on AI use for candidates
• At BlackRock, AI has long been part of how we work – enhancing decision-making, improving operations, and helping us deliver better outcomes for clients. We encourage candidates to use AI thoughtfully to learn, prepare, and work more effectively; but during our interview process, we want to focus on getting to know you through your own experiences, thinking, and judgment. To support you, we’ve provided guidance on when and how to use AI during our hiring process so you can approach each step with confidence and showcase your best self.

About BlackRock

Qualifications

• Experience with Python and common AI development libraries and frameworks (PyTorch, TensorFlow or similar).