{bc}
ashby

Engineer, Supercomputing & Distributed Systems

Krea
San Francisco, USA
Full-time
Mid
Onsite
Discovered 1 months ago
PythonKubernetesTorchPyArrowDuckDBArrow

Full Job Posting

About Krea

At Krea, we are building next-generation AI creative tools.

We are dedicated to making AI intuitive and controllable for creatives. Our mission is to build tools that empower human creativity, not replace it.

We believe AI is a new medium that allows us to express ourselves through various formats—text, images, video, sound, and even 3D. We're building better, smarter, and more controllable tools to harness this medium.

Supercomputing / AI Infra at Krea

We build and operate the infrastructure for Krea's research and inference. Distributed training, 1000+ K8s GPU clusters, petabyte scale data pipelines, etc. We build a lot of this from scratch — custom distributed datastores, job orchestration systems, and streaming pipelines that replace tools like Kafka and Ray for modern AI workloads at scale.

Example projects

Distributed data systems

Design multi-stage pipelines that turn petabytes of raw data into clean, annotated datasets

Run classification models on billions of images

Deploy and combine LLMs to caption massive multimedia data

GPU infrastructure

Manage distributed training and inference on 1000+ GPU Kubernetes clusters

Solve orchestration and scaling for large-scale GPU job processing

Scale workloads and research between clusters in multiple datacenters

Distributed training

Profile and optimize dataloaders streaming thousands of images per second

Profile and debug InfiniBand networking on huge training runs

Build fault tolerance systems for large-scale pretraining

Collaborate with researchers on evolving RL infrastructure

Applied ML pipelines

Find clean scenes in millions of videos using distributed shot-boundary detection

Customize and train models to filter billions of images for questions like "is this a screenshot?"

Build the systems that bridge raw cluster capacity and research output

Who we're looking for

Systems people. If you've read a blog post about InfiniBand debugging or building a custom distributed database and thought "I want to do that" — this is that team.

You'll spend your time working heavily with Python, Kubernetes, Torch, and data tools like DuckDB, Arrow, etc. It's OK if you don't have K8s or ML experience — the main thing we hire for is an intuition for distributed systems, and a great mental model of how systems interact and function under different conditions.

Strong candidates may have experience with…

Python, PyArrow, DuckDB, SQL, massive relational databases, PyTorch, Pandas, NumPy…

Kubernetes

Designing and implementing large-scale ETL systems

Fundamental knowledge of containerization, operating systems, file-systems, and networking

Distributed systems design

Distributed training systems (NCCL, InfiniBand, RDMA)

Streaming and event processing systems (Kafka, Pulsar, or similar)

PyTorch internals, custom dataloaders, and training infrastructure

Apply for this job in 1 click

Skip the repetitive application forms

Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.

Sarah M.James T.Maya R.

Trusted by over 500,000 job seekers on Base Career

Start Free Today

More from this employer

More jobs at Krea