Responsibilities:

• Design, deploy and operate GPU clusters across AWS, GCP, and other hyperscalers/neoclouds/on-premise data centers, including custom Kubernetes and Slurm operators and the metadata services behind them.
• Build scheduling that keeps accelerators busy: gang scheduling for distributed jobs, bin-packing to reduce fragmentation, and quotas and fair share for multi-tenant teams.
• Run distributed training infrastructure on PyTorch Distributed, Ray, or Megatron, with fast checkpointing and model loading.
• Deploy and tune high-throughput LLM serving with vLLM or similar: tensor and model/expert parallelism, prefill/decode disaggregation, KV cache offload, FP8/FP4 quantization.
• Build benchmarking and traffic-replay tools that measure TTFT, TPOT and latency percentiles, and set per-config performance baselines.
• Build health checks across hardware, network and software layers, and automated remediation that handles fault storms without human intervention.
• Instrument fleet utilization with DCGM, Prometheus and Grafana. Write recording rules and alerts that teams rely on.
• Ingest utilization, occupancy and billing data into a warehouse, attribute spend to teams and workloads, and support capacity planning.
• Drive autoscaling, rightsizing and idle-capacity recovery programs with measurable cost results.
• Write clear runbooks and documentation, and explain infrastructure tradeoffs to researchers, finance and leadership.
• Build internal tools or dashboards that researchers and non-engineers use daily.

You may be a good fit if you

• Have 5+ years building and operating production distributed systems or ML infrastructure
• Write production-quality Python plus Go or C++, and are fluent in shell scripting for system administration.
• Have deep hands-on Kubernetes and container experience, plus working knowledge of Slurm or another HPC resource manager.
• Have run GPU workloads at scale, including scheduling, orchestration and troubleshooting.
• Have hands-on experience with at least one distributed training or serving stack: PyTorch Distributed, Ray, vLLM or Megatron-LM.
• Have deep experience with at least one major cloud provider.
• Know observability tools like DataDog, Prometheus, PromQL and Grafana, and are comfortable on-call for critical systems.
• Own projects end to end and gather your own requirements with little direction.

Strong candidates may also have experience with (or a strong desire to learn deeply)

• LLM inference optimization: KV cache systems, disaggregated serving, quantization, speculative decoding
• High-performance networking: RDMA, InfiniBand, NCCL, NVLink
• GPU profiling with Nsight or similar, and CUDA or Triton kernel work
• Accelerators beyond NVIDIA: AMD MI-series, TPU, Trainium
• Capacity planning, billing normalization or cost attribution across providers
• Automated fault diagnosis, log mining or AIOps at fleet scale
• Configuration management (Ansible, Puppet) and secure or air-gapped deployments

Benefits

• Bi-annual performance bonus structure
• Generous equity grant vested over 4 years
• Up to $15k Relocation bonus
• $10K housing bonus (if you live within 0.5 miles of our office)
• $1.5K monthly stipend for meals
• Free Equinox membership
• $200 monthly laundry reimbursement
• $200 monthly personal wellness reimbursement
• Health, Dental, Vision insurance

Compensation: $200K - $400K