{bc}
Apply
greenhouse

AI Engineer, AI & Applications

Firmus Technologies
Sydney, AUS
Mid · 3+ years experience
Onsite
Discovered 1 weeks ago
mlflow

Full Job Posting

Key Responsibilities

  • Design, develop, implement and maintain end-to-end MLOps workflows: training → evaluation → registry → deployment → monitoring → retraining/retirement in dev/staging/production environments, with clear standards and ownership boundaries.
  • Own the model registry and promotion lifecycle using MLflow, including stage/alias strategy, approvals, environment separation, access control, rollback readiness and integration with software release pipelines.
  • Establish reproducibility and lineage across the model lifecycle, including versioned code/configuration, artifact traceability, dataset/version references and repeatable evaluation runs.
  • Design, develop and implement automated model quality gates for production, including accuracy, latency, cost, safety and reliability requirements.
  • Define SLOs/SLIs for all AI platform features, including training job success rate, inference latency p99, RAG retrieval accuracy, availability and cost metrics.
  • Develop and maintain production monitoring dashboards to track model performance, data drift and operational health, and integrate monitoring outputs with alerting tools such as PagerDuty and Slack.
  • Create on-call runbooks and triage procedures for AI service incidents and lead postmortem-driven improvements to improve software reliability and maintainability.
  • Instrument AI services for debugging, including request traces, GPU metrics per model, retrieval performance and communication bottlenecks.
  • Integrate evaluation frameworks, including benchmarking, RAGAS and LLM-as-judge, into CI/CD pipelines to support automated testing, deployment and release management.

Skills & Experience

  • Bachelor’s degree or higher qualification in Software Engineering, Computer Science, Information Technology or a closely related field.
  • 3 + years in MLOps, ML platform, software engineering or production engineering roles with hands-on ownership of production ML delivery pipelines.
  • Deep understanding of the ML software lifecycle, including model versioning, promotion strategies, evaluation automation, governance, deployment strategies, monitoring and drift detection.
  • Hands-on experience with MLflow Model Registry workflows, including stages/aliases, approvals, traceability and integration of registry actions into software release pipelines.
  • Experience designing, developing and operationalising model evaluation systems, including metrics standards, orchestration, logging and reproducibility.
  • Strong software engineering and production fundamentals, including metrics, logs, traces, alert design, incident response and reliability practices.
  • Familiarity with CI/CD pipelines, model packaging, deployment automation and software testing, with the ability to collaborate across ML engineers, platform/SRE and application teams to turn requirements into robust workflows.
  • Understanding of distributed systems, resource management and failure modes in training and inferencing environments.
  • Key Competencies
  • Production Ownership: comfortable owning services in production; proactive about monitoring, alerting, and preventing issues.
  • Reliability Engineering: can define SLOs, error budgets, and blameless postmortem culture.
  • Cross-Functional Leadership: works with ML engineers, data scientists, and platform teams; unblocks reliably.
  • Incident Response: triage skills, root cause analysis, systemic thinking (not just fighting fires).
  • Programmatic automation reduces toil and makes the right path with a balanced rigor with speed.
  • Communication: explains complex ML/systems issues clearly to both technical and non-technical stakeholders.
  • Location & Reporting
  • Location Sydney, Australia
  • Reporting to Head of AI & Applications
  • Employment Basis
  • Full-time
  • Diversity
  • At Firmus, we are committed to building a diverse and inclusive workplace. We encourage applications from candidates of all backgrounds who are passionate about creating a more sustainable future through innovative engineering solutions.
  • Join us in our mission to revolutionize the AI industry through sustainable practices and cutting-edge engineering. Apply now to be part of shaping the future of sustainable AI infrastructure.

Qualifications

  • Bachelor’s degree or higher qualification in Software Engineering, Computer Science, Information Technology or a closely related field.
  • 3 + years in MLOps, ML platform, software engineering or production engineering roles with hands-on ownership of production ML delivery pipelines.
  • Deep understanding of the ML software lifecycle, including model versioning, promotion strategies, evaluation automation, governance, deployment strategies, monitoring and drift detection.
  • Hands-on experience with MLflow Model Registry workflows, including stages/aliases, approvals, traceability and integration of registry actions into software release pipelines.
  • Experience designing, developing and operationalising model evaluation systems, including metrics standards, orchestration, logging and reproducibility.
  • Strong software engineering and production fundamentals, including metrics, logs, traces, alert design, incident response and reliability practices.
  • Familiarity with CI/CD pipelines, model packaging, deployment automation and software testing, with the ability to collaborate across ML engineers, platform/SRE and application teams to turn requirements into robust workflows.
  • Understanding of distributed systems, resource management and failure modes in training and inferencing environments.
  • Key Competencies
  • Production Ownership: comfortable owning services in production; proactive about monitoring, alerting, and preventing issues.
  • Reliability Engineering: can define SLOs, error budgets, and blameless postmortem culture.
  • Cross-Functional Leadership: works with ML engineers, data scientists, and platform teams; unblocks reliably.
  • Incident Response: triage skills, root cause analysis, systemic thinking (not just fighting fires).
  • Programmatic automation reduces toil and makes the right path with a balanced rigor with speed.
  • Communication: explains complex ML/systems issues clearly to both technical and non-technical stakeholders.
  • Location & Reporting
  • Location Sydney, Australia
  • Reporting to Head of AI & Applications
  • Employment Basis
  • Full-time
  • Diversity
  • At Firmus, we are committed to building a diverse and inclusive workplace. We encourage applications from candidates of all backgrounds who are passionate about creating a more sustainable future through innovative engineering solutions.
  • Join us in our mission to revolutionize the AI industry through sustainable practices and cutting-edge engineering. Apply now to be part of shaping the future of sustainable AI infrastructure.

Apply for this job in 1 click

Skip the repetitive application forms

Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.

Sarah M.James T.Maya R.

Trusted by over 500,000 job seekers on Base Career

Start Free Today

More from this employer

More jobs at Firmus Technologies

Project Manager - FIFO to Tasmania

Sydney, AUS

Discovered TodayFull-time

Network Engineer

Sydney, AUS

Discovered 2 days ago

Senior AI Engineer, Agents & Applications

Sydney, AUS

Discovered 2 days ago

Project Manager - FIFO to Tasmania

Sydney, AUS

Discovered 5 days ago

Principal Engineer, Platform

Sydney, AUS

Discovered 1 weeks agoFull-time

Senior Kubernetes Platform Engineer

Sydney, AUS

Discovered 1 weeks ago

Project Manager - Data Centre Build

Tailem Bend, AUS

Discovered 1 weeks ago

Principal Engineer, AI Cloud Software

Sydney, AUS

Discovered 1 weeks ago