Principal Software Engineer - IT
Key Skills for This Role
Full Job Posting
AI Evaluation Engineer
The Security Resiliency Organization builds the AI-native capabilities that are replacing manual security processes across Dell, and every one of those capabilities has to earn its release through evidence, not demonstration. This team owns the quality bar for that shift: defining what "good enough" means for each workflow and building the harnesses that prove it, before and after release.
Join us as an AI Evaluation Engineer on our Security Resiliency team in Bangalore, India to do the best work of your career and make a profound social impact.
What you’ll achieve
- You will own the release gate for AI-native security capabilities, deciding what evidence is required before a workflow can go live.
- You will build evaluation harnesses that measure model and agent behaviour objectively, not by sampling outputs and forming an impression.
- You will turn evaluation into a repeatable, measurable analytics practice so that every workflow after the first costs less to evaluate than the one before it.
You will:
- Define and record acceptance criteria for each capability with the functional owner before evaluation begins, and specify the evidence, ground truth and coverage required to support the claim.
- Build and maintain evaluation harnesses in Python covering dataset construction, test orchestration, scoring and reporting, and integrate them as automated quality gates in CI/CD.
- Design measurable analytics for models and agentic tools – quantifiable metrics for accuracy, consistency across repeated runs, confidence-versus-accuracy calibration, latency and cost – using tooling such as LangFuse for trace-level evaluation and embedding-based similarity scoring for output comparison.
- Own the release gate: apply the criteria to the evidence, record the result, and block releases that do not clear the threshold, with thresholds set proportionate to the consequence of an incorrect agent action.
- Run ongoing regression evaluation after release to detect behavioural drift from model, prompt, data or upstream changes, and report results to engineering and functional stakeholders in decision-ready terms.
Take the first step towards your dream career
Every Dell Technologies team member brings something unique to the table. Here’s what we are looking for with this role:
Essential Requirements
- 8+ years in software engineering, quality engineering or machine learning evaluation, with systems shipped to production.
- Hands-on construction of evaluation harnesses in Python: dataset design, test orchestration, scoring implementation and result reporting for non-deterministic systems.
- Ability to build measurable analytics of models and agentic tools, using LangFuse (or equivalent LLM observability tooling) and embedding-based comparison techniques.
- Statistical literacy: sample size, variance across runs, and the difference between a real change in behaviour and noise, applied with Python data tooling at volume.
- Experience owning a quality gate or release decision, including declining a release and holding that position under schedule pressure.
Desirable Requirements
- Experience evaluating AI systems within a security domain, and familiarity with agent orchestration frameworks such as LangGraph to understand what is being evaluated.
- Familiarity with the OWASP Top 10 for LLM Applications and the NIST AI Risk Management Framework, ideallyin an audited or regulated environment.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Dell Technologies
Principal Consultant - AI Infrastructure Services
Dubai, UAE
Dell Technologies is seeking a Principal Consultant to lead AI infrastructure and cloud transformation engagements, including discovery, architecture, migration planning, and value realization. Candidates need expert clo
Senior Product Management – Team Lead, Core+ Launch
Round Rock, USA
Senior Principal Mechanical Engineer (Modular Data Center Architecture)
Hopkinton, USA
Deployment Engineer - Federal Customer
, USA
Principal Consultant - AI Infrastructure Services
Dubai, UAE
Senior Product Management – Team Lead, Core+ Launch
Round Rock, USA
Senior Principal Mechanical Engineer (Modular Data Center Architecture)
Hopkinton, USA
Long Range Server Planning
Round Rock, USA
Software Principal Engineer
Round Rock, USA
Software Senior Engineer
Round Rock, USA