{bc}
Apply
Company Site

Machine Learning Engineer / SME (LLM Eval & RAG Testing) - SME | TS/SCI

Falls Technology
McLean, USA
Full-time
Mid
Onsite
Discovered 1 weeks ago
flaskscikit-learntableau

Full Job Posting

What you'll do

  • Test, evaluate and validate AI systems with a focus on LLM-based systems (including RAG testing)
  • Design and run experiments that benchmark LLMs against defined measures
  • Tune neural networks / LLMs on custom datasets and apply results to concrete use cases
  • Build AI/ML models and ensembles in Python or R (library selection, preprocessing, model assessment)
  • Use deep learning frameworks such as PyTorch, TensorFlow or Keras
  • Leverage GPUs for accelerated training and evaluation
  • Write solid SQL (CTEs, set operations, aggregates, nested subqueries) against analytical datasets
  • Work in GitHub and Jenkins-style CI for reproducible experiments
  • Communicate methodological choices and model results to technical and non-technical stakeholders
  • What you bring (required)
  • Active TS/SCI
  • Hands-on RAG testing experience (must-have)
  • Tuning neural nets / LLMs on custom datasets for real use cases
  • Building AI/ML models and ensembles in Python or R
  • PyTorch, TensorFlow or Keras (key)
  • GPU-accelerated computing (key)
  • Testing, evaluating or validating LLM-based AI systems (key)
  • Experiment design to benchmark LLMs (key)
  • SQL including CTEs, set ops, aggregates and nested subqueries
  • GitHub and Jenkins (or equivalent) version control / CI

Nice to have

  • Hugging Face Transformers library and hub
  • Building benchmarks that measure LLM performance
  • Cloud computing development and architecture
  • Flask or similar front-end/web frameworks
  • Text classification and topic modeling (scikit-learn or deep learning)
  • Semantic search applications
  • Tableau dashboards
  • Verification/validation test benches, Explainable AI (XAI), ONNX
  • Why Falls
  • A 9% retirement contribution, 100% of family medical premiums paid, and 4 weeks of PTO. We're a small business that's been delivering since 2019, so your work is visible and your voice counts.
  • Falls Technology is an Equal Opportunity Employer. All qualified applicants will receive consideration without regard to race, color, religion, sex, national origin, disability, or protected veteran status.

Apply for this job in 1 click

Skip the repetitive application forms

Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.

Sarah M.James T.Maya R.

Trusted by over 500,000 job seekers on Base Career

Start Free Today

More from this employer

More jobs at Falls Technology

Data Architect | TS/SCI

Chantilly, USA

Discovered 6 days agoFull-time

Automated Test Engineer (SDET) | TS/SCI w/ Poly

Chantilly, USA

Discovered 6 days agoFull-time

Cyber Security Analyst / Cyber Security Project Engineer | TS/SCI w/ Poly

Chantilly, USA

Discovered 6 days agoFull-time

Systems Administrator | TS/SCI w/ Poly

Reston, USA

Discovered 6 days agoFull-time

Salesforce Developer - Mid-Level | On-Site, No Clearance Required

Herndon, USA

Discovered 3 weeks agoFull-time

E-Learning / Multimedia Developer - Expert | TS/SCI w/ Poly

Chantilly, USA

Discovered 3 weeks agoFull-time

Data Scientist / Data Modeler (ECL/HPCC) - SME | TS/SCI w/ Poly

McLean, USA

Discovered 3 weeks agoFull-time