Post Job Free
Sign in

Senior Machine Learning Engineer, Inference

Company:
Head-Huntress
Location:
United States
Pay:
$250K–$300K + equity
Posted:
November 06, 2026
Apply

Posted By Premium Recruiter

Description:

Remote · Full-time / Direct hire · $250K–$300K base + equity

Open to candidates located in the US, Canada, UK, Germany, Switzerland or Serbia. Office in Mountain View, CA available for those nearby.

Head-Huntress is hiring on behalf of a fast-growing real-time AI company building sub-second multimodal inference at scale. 2–3 openings.

WHO WE'RE LOOKING FOR

A year ago, reliably working agentic systems and sub-second multimodal inference at scale barely existed, so nobody has a decade of experience here. We're not screening for a resume template. We want strong people from varied backgrounds who learn fast, thrive in ambiguity, and can show what they've built, broken and understood.

EXPERIENCE WE FIND USEFUL (you don't need all of it, but enough to make a case)

- Inference optimization: deep understanding of modern serving frameworks such as vLLM or TRT-LLM

- Model acceleration: quantization, distillation, caching strategies, continuous batching, paged attention, speculative decoding

- High-performance systems: C++, CUDA, Rust or highly optimized Python; profiling and squeezing performance out of NVIDIA GPUs

- Distributed systems and scaling: Kubernetes, Ray, custom load balancing, multi-GPU/multi-node inference, thousands of concurrent connections

- Public work: non-trivial systems projects, open-source contributions to major inference engines, or deep technical write-ups

- Full-cycle ownership: take a model from research, containerize it, optimize its serving and run it reliably in production

- Background: PhD in CS, Physics or Math, or equivalent practical experience building backend or ML systems

MUST-HAVES

- Senior-level experience (Google L5 / Meta E5 equivalent)

- Hands-on ML inference optimization with vLLM or TRT-LLM

- 5+ years as an ML inference or systems engineer

- Experience building backend or ML systems and serving models in production

- Systems programming and GPU optimization (C++, CUDA, Rust or optimized Python)

- Distributed systems and scaling experience

- Located in the US, Canada, UK, Germany, Switzerland or Serbia

NICE TO HAVE

- Promotions / internal progression within the same company

- Open-source or public technical work on inference

- PhD or top CS program

- Able to commute to Mountain View, CA

WHO THRIVES HERE

- You don't need a roadmap to start walking; you pick a direction and build the map as you go

- Engineering isn't finished until it's shipped and stable; you have a bias for impact

- You obsess over the why and question an architecture when there's a better way to solve latency or throughput

- You want deep context and the logic behind every decision

COMPENSATION AND BENEFITS

- Base $250K–$300K USD, adjusted by location (top of range for SF Bay Area/NYC), plus competitive equity

- Visa transfers supported (OPT, H-1B transfers)

- Relocation assistance to the SF Bay Area

INTERVIEW PROCESS

Recruiter screen (30 min) ML technical exercise on a real production problem, not LeetCode (90 min) Hiring manager (30 min) ML system design (45 min) CTO (45 min) CEO (30 min)

HOW TO APPLY

Apply through this posting with your resume. Head-Huntress reviews every application and will reach out to qualified candidates.

Apply