{bc}
Apply
greenhouse

Senior Kubernetes Platform Engineer

Firmus Technologies
Sydney, AUS
Senior · 8+ years experience
Onsite
Discovered 1 weeks ago
ansibleargobashkubernetespythonterraform

Full Job Posting

Key Responsibilities

  • Responsible for the reliable operation, automation and continuous improvement of the multi-tenant Kubernetes platform that Firmus' products and tenants run on, spanning every site in the estate.
  • Build the operational tooling, guarded remediation and orchestration that automate operations at fleet scale, and contribute operator and controller requirements, and code where agreed, to AI Infrastructure's platform backlog with the production evidence behind them.
  • Execute the Kubernetes cluster lifecycle across the fleet, including provisioning, patching, upgrade and decommissioning, running the deployment and upgrade mechanisms built by AI Infrastructure through the agreed staged or canary path, and holding estate version compliance and retirement coordination.
  • Operate and recover Kubernetes control planes carrying live tenant workload, including etcd state, certificate rotation, failed upgrades and corrupted resources.
  • Operate the virtual cluster platform and multi-tenant isolation patterns that tenants are provisioned onto, and the automated tenant onboarding and release pipeline that lands new tenants safely and repeatably.
  • Operate the GPU integration layer for Kubernetes (for example the NVIDIA GPU Operator), including device plugins, GPU scheduling and driver coordination.
  • Diagnose and resolve scheduling failures, CNI and CSI faults, admission rejections and resource contention from first principles, and drive continuous improvement in cluster validation, CI/CD automation, and provisioning and testing frameworks.
  • Run the Kubernetes baselines in production carrying the admission policy, workload identity and network policy content set by the Senior Platform Security Engineer, and hold the operational acceptance requirements those baselines have to meet before they enter production.
  • Provide the deepest technical expertise for Kubernetes faults across the estate, diagnosing the faults that require internals-level knowledge to root cause, and driving the permanent fix to closure through AI Infrastructure, and mentor the engineers who carry frontline diagnosis, documenting operational procedures, runbooks and performance results.
  • Lead technical recovery during major Kubernetes incidents under the incident commander, drive the changes that remove repeat causes through the problem record, and share the after-hours escalation roster for the Kubernetes estate.

Skills & Experience

  • Required Skills
  • Strong skills in platform and infrastructure engineering, with 8+ years of experience overall and substantial ownership of production Kubernetes platforms in a 24/7 environment.
  • Deep experience operating Kubernetes at fleet scale, including cluster lifecycle, upgrades and multi-cluster management.
  • Strong experience with Kubernetes control plane internals, including etcd, the API server, controllers, schedulers, and certificate and credential rotation.
  • Strong experience writing Kubernetes controllers, operators or admission logic in a production setting.
  • Experience with multi-tenant or virtual cluster patterns (for example vCluster or equivalent), including tenant isolation at the Kubernetes layer.
  • Experience operating GPU-enabled Kubernetes, including device plugins, GPU scheduling and driver coordination (for example the NVIDIA GPU Operator).
  • Strong skills in infrastructure automation, infrastructure-as-code and GitOps practices (for example OpenTofu or Terraform, Ansible, Argo CD), with change delivered through peer review, automated testing and progressive rollout.
  • Strong experience with scripting or programming for operational automation and tooling, such as Go, Python or Bash.
  • Proven ability to act as a senior escalation point in production, including major incident response, on-call participation, post-incident review, and the production of runbooks that others can execute successfully.
  • Solid understanding of Kubernetes security fundamentals, including admission control, workload identity and network policy.
  • Clear technical judgement and communication skills, with the ability to produce documentation, design notes and escalations that other engineers can act on.
  • Preferred Experience
  • Experience operating Kubernetes for GPU or HPC workloads at scale.
  • Experience with automated tenant or customer onboarding pipelines in a multi-tenant platform.
  • Experience with vendor Kubernetes distributions or reference architectures for accelerated computing.
  • Familiarity with DPU or SmartNIC-based networking as it relates to Kubernetes CNI design.
  • Experience contributing to open-source Kubernetes ecosystem projects.
  • A Bachelor's degree in computer science, engineering or a related discipline, or an equivalent combination of relevant experience and training.
  • Expected Outcomes
  • Fleet-wide cluster upgrades executed through the staged path with no unplanned tenant-visible outage, and version compliance held across the estate.
  • Tenant onboarding running as a routine operation rather than a project.
  • Control plane recovery tested against live-equivalent conditions and executable by an engineer who did not write the procedure.
  • Kubernetes escalations falling as operational automation and guarded remediation take on the common faults.
  • Platform defects and operability gaps evidenced into the platform engineering backlog and closed permanently, with repeat causes falling.
  • Location & Reporting
  • Location: Based in Australia or Singapore, with travel to Australian AI Factory sites as required.
  • On-call: The function runs 24/7. First line monitoring and first response sit with the operations centre. This role shares the after-hours escalation roster for its domain with the other senior engineers in the function.

Qualifications

  • Strong skills in platform and infrastructure engineering, with 8+ years of experience overall and substantial ownership of production Kubernetes platforms in a 24/7 environment.
  • Deep experience operating Kubernetes at fleet scale, including cluster lifecycle, upgrades and multi-cluster management.
  • Strong experience with Kubernetes control plane internals, including etcd, the API server, controllers, schedulers, and certificate and credential rotation.
  • Strong experience writing Kubernetes controllers, operators or admission logic in a production setting.
  • Experience with multi-tenant or virtual cluster patterns (for example vCluster or equivalent), including tenant isolation at the Kubernetes layer.
  • Experience operating GPU-enabled Kubernetes, including device plugins, GPU scheduling and driver coordination (for example the NVIDIA GPU Operator).
  • Strong skills in infrastructure automation, infrastructure-as-code and GitOps practices (for example OpenTofu or Terraform, Ansible, Argo CD), with change delivered through peer review, automated testing and progressive rollout.
  • Strong experience with scripting or programming for operational automation and tooling, such as Go, Python or Bash.
  • Proven ability to act as a senior escalation point in production, including major incident response, on-call participation, post-incident review, and the production of runbooks that others can execute successfully.
  • Solid understanding of Kubernetes security fundamentals, including admission control, workload identity and network policy.
  • Clear technical judgement and communication skills, with the ability to produce documentation, design notes and escalations that other engineers can act on.
  • Experience operating Kubernetes for GPU or HPC workloads at scale.
  • Experience with automated tenant or customer onboarding pipelines in a multi-tenant platform.
  • Experience with vendor Kubernetes distributions or reference architectures for accelerated computing.
  • Familiarity with DPU or SmartNIC-based networking as it relates to Kubernetes CNI design.
  • Experience contributing to open-source Kubernetes ecosystem projects.
  • A Bachelor's degree in computer science, engineering or a related discipline, or an equivalent combination of relevant experience and training.
  • Expected Outcomes
  • Fleet-wide cluster upgrades executed through the staged path with no unplanned tenant-visible outage, and version compliance held across the estate.
  • Tenant onboarding running as a routine operation rather than a project.
  • Control plane recovery tested against live-equivalent conditions and executable by an engineer who did not write the procedure.
  • Kubernetes escalations falling as operational automation and guarded remediation take on the common faults.
  • Platform defects and operability gaps evidenced into the platform engineering backlog and closed permanently, with repeat causes falling.
  • Location & Reporting
  • Location: Based in Australia or Singapore, with travel to Australian AI Factory sites as required.
  • On-call: The function runs 24/7. First line monitoring and first response sit with the operations centre. This role shares the after-hours escalation roster for its domain with the other senior engineers in the function.

Preferred Qualifications

  • Preferred Experience

Apply for this job in 1 click

Skip the repetitive application forms

Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.

Sarah M.James T.Maya R.

Trusted by over 500,000 job seekers on Base Career

Start Free Today

More from this employer

More jobs at Firmus Technologies

Network Engineer

Sydney, AUS

Discovered 2 days ago

Senior AI Engineer, Agents & Applications

Sydney, AUS

Discovered 2 days ago

Project Manager - FIFO to Tasmania

Sydney, AUS

Discovered 4 days ago

Principal Engineer, Platform

Sydney, AUS

Discovered 1 weeks agoFull-time

Project Manager - Data Centre Build

Tailem Bend, AUS

Discovered 1 weeks ago

AI Engineer, AI & Applications

Sydney, AUS

Discovered 1 weeks ago

Principal Engineer, AI Cloud Software

Sydney, AUS

Discovered 1 weeks ago

Community Engagement Partner

Launceston, AUS

Discovered 2 weeks ago