AI Research Engineer (Kernel & Inference Optimization) - 100% Remote Worldwide

About Tether

Role overview

Responsibilities

• Design and deploy model-serving architectures for high throughput, low latency, and efficient memory use on constrained devices and edge platforms.
• Set performance targets for latency, token response, and memory footprint.
• Run and monitor inference tests in simulated and live production environments, tracking latency, throughput, memory use, and errors.
• Document iterative test results and compare them with benchmarks across platforms.
• Prepare high-quality datasets and simulation scenarios for real-world, low-resource deployment conditions.
• Diagnose computational and memory bottlenecks, including batch-processing inefficiencies and network delays.
• Integrate optimized serving and inference frameworks into edge and on-device production pipelines with cross-functional teams.
• Define success metrics and monitor and refine production systems.

Requirements

• A degree in Computer Science or a related field is required; a PhD in NLP, Machine Learning, or a related field is described as ideal.
• A strong AI R&D track record is expected, with publications at A* conferences described as ideal.
• Must know Metal Shading Language (MSL) and be comfortable writing custom compute shaders from scratch.
• Low-level kernel optimization and mobile inference optimization experience with measurable improvements are essential.
• Deep expertise in model-serving architectures, inference optimization, performance, and memory management is required.
• Strong GPU kernel expertise for smartphones and knowledge of serving frameworks and engines are required.
• Practical end-to-end inference pipeline development and deployment experience, including resource-constrained devices, is required.
• Research-driven evaluation and optimization skills for latency, computational bottlenecks, and memory constraints are required.
• Understanding of diffusion models and Vision Transformers is required.
• Understanding of pruning, quantization, Flash Attention, KV cache, and speculative decoding (Eagle) is required.
• Excellent English communication skills are expected.

Technical areas

• Distributed inference systems may use tensor, pipeline, and expert parallelism to run large models on GPU clusters.
• The posting names pruning, quantization, Flash Attention, KV cache, and speculative decoding (Eagle).

Workplace

• The title states 100% remote worldwide; the company also describes its team as remote.