Backend Engineer (LLM inference)

About Aleroy

About the Role

What You Will Own

• Design, build, and maintain backend services in Go and TypeScript/JavaScript (Node.js), plus Python where it fits.
• Build and scale distributed systems that stay reliable as traffic and complexity grow.
• Own features end to end, from design docs and implementation through testing and release.
• Identify tech debt, bottlenecks, and reliability risks before they become problems.
• Design and maintain OpenAI-compatible APIs for chat completions, streaming, and tool calls.
• Build token streaming over SSE/WebSockets with cancellation, disconnect handling, and partial responses.
• Build request routing and load balancing across inference backends by model, capacity, latency, and region.
• Implement backpressure, queueing, and admission control for variable-cost LLM workloads.
• Design REST and gRPC interfaces between internal services.
• Design and optimize data models in PostgreSQL and use Redis/Valkey for caching and state.
• Build authentication and authorization flows with Auth0 and JWT, and token-based rate limiting and usage metering.
• Build reliable third-party integrations, including Stripe payments, with retries, idempotency, and failure handling.

About You

• 5+ years of professional backend engineering experience.
• Strong commercial experience with both Go and TypeScript/JavaScript (Node.js); Python is a plus.
• Proven experience building and operating distributed systems with consistency, fault tolerance, and service-to-service communication.
• Deep understanding of concurrency models including Go goroutines, channels, scheduler, and Node.js event loop and async/await.
• Strong database knowledge: schema design, indexing, query optimization, transactions, isolation levels, and locking, primarily in PostgreSQL.
• Experience with Redis, Valkey, or similar in-memory stores.
• Experience building backend systems that serve LLM inference, or strong understanding of request lifecycle, token streaming, and long-running requests.
• Experience building streaming APIs (SSE/WebSockets), including cancellation and client disconnect handling.
• Understanding of OpenAI-compatible API design and its streaming and error semantics.
• Experience with request routing, load balancing, backpressure, and admission control for variable-cost workloads.
• Hands-on experience designing REST and/or gRPC APIs.
• Experience building auth systems with JWT, Auth0, or similar.

Nice to Have

• Familiarity with inference engines like vLLM, SGLang, or TGI, and concepts like continuous batching and KV cache.
• Experience with fair scheduling and tenant isolation on shared inference capacity.
• Experience with failover and retries for inference requests without duplicating billing or output.
• Experience with usage metering and billing for AI or developer APIs.
• Stripe or other payment infrastructure experience.
• S3-compatible storage or Cloudflare R2 experience.
• Building developer-facing APIs or platform products.

What We Offer

• Real ownership of the backend that powers the Aleroy inference platform.
• Influence over backend architecture and product direction.
• Hands-on work on hard problems in distributed systems, real-time streaming, and LLM serving.
• A small, senior team with fast execution and low bureaucracy.
• Competitive salary with performance-based incentives.
• 24 days annual leave plus public holidays.
• Health insurance.
• Modern office in Yas Creative Hub.