Must-haves

• Strong engineering skills in Python and PyTorch.
• Hands-on experience deploying or optimizing LLM, VLM, or high-throughput transformer inference.
• Practical knowledge of at least one modern inference stack, such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, NVIDIA Dynamo, Ray Serve, KServe, or an equivalent internal system.
• A strong understanding of transformer inference bottlenecks involving KV cache, attention, memory bandwidth, batching, parallelism, and long-context serving.
• Ability to quantify trade-offs among latency, throughput, quality, utilization, and cost.
• Experience designing or optimizing distributed inference system architecture, with hands-on work in one or more areas such as request routing, distributed scheduling, PDD, multi-node inference, or expert parallelism.
• Strong communication and collaboration skills across research, kernel, infrastructure, product, and customer teams.

Nice-to-haves

• Experience with quantization-aware training, post-training quantization, FP8, INT8, INT4, NVFP4, MXFP4, AWQ, GPTQ, or SmoothQuant.
• Experience with distillation, speculative decoding, EAGLE, Medusa, multi-token prediction, or related inference acceleration methods.
• Experience supporting agentic workloads involving tool calling, structured outputs, streaming APIs, high concurrency, and multi-step orchestration.
• Familiarity with CUDA or Triton; the role does not require kernel engineering to be the candidate’s primary specialization.
• Contributions to open-source projects such as vLLM, SGLang, TensorRT-LLM, FlashInfer, LMCache, PyTorch, Triton, Ray, or KServe.
• Implementation experience with cache-aware request routing, PDD, or WideEP, including diagnosing communication bottlenecks and load imbalance.
• Familiarity with GPU communication libraries and interconnects, such as NCCL, NVLink, InfiniBand, or RoCE.

Benefits & Perks:

• Competitive compensation
• Career growth and learning opportunities
• Flexibility and ownership
• Collaborative and innovative culture
• Opportunity to work on impactful AI projects
• International environment and talented teams

What's it like to work at Nebius:

Fast moving - Bold thinking - Constant growth - Meaningful impact - Trust and real ownership - Opportunity to shape the future of AI

Equal Opportunity Statement:

Nebius is an equal opportunity employer. We are committed to fostering an inclusive and diverse workplace and to providing equal employment opportunities in all aspects of employment. We do not discriminate on the basis of race, color, religion, sex (including pregnancy), national origin, ancestry, age, disability, genetic information, marital status, veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by applicable law.

Applicants must be authorized to work in the country in which they apply and will be required to provide proof of employment eligibility as a condition of hire.

If you need accommodations during the application process, please let us know.