Machine Learning Engineer (Inference and Optimization)

Job Description

About Zyphra

At Zyphra, we're building Maia—a cutting-edge multimodal agent system that combines advanced research in neural network architectures, long-term memory, and reinforcement learning. Based in Palo Alto, our team brings together talent from leading AI organizations including Google DeepMind, Anthropic, StabilityAI, Qualcomm, Neuralink, Nvidia, and Apple.

Machine Learning Engineer (Inference and Optimization)

About the role:

You will be tasked with improving the efficiency of our training and inference stacks. You will interface very closely with our pretraining and architecture research teams, as well as our inference backend. You’ll work across:

Large-scale training runs and model parallelization

Performance optimization of our pretraining stack

Performance optimization and porting of models to our inference engine

Writing and optimizing novel kernels to efficiently implement our architectural innovations

While deep expertise in all areas isn't required, you should be enthusiastic about learning each aspect as needed to drive our models forward. We also strongly look for velocity over position. However, having deep expertise in at least one of the above is helpful to begin contributing immediately.

Requirements:

Strong engineering aptitude for rapidly implementing reliable and robust systems

Has a strong aptitude for and interest in understanding the low-level end-to-end details of our pipelines and a knack for finding inefficiencies

Can rapidly learn new fields and are excited to implement new ideas

Excellent communication and collaboration skills and can work effectively on both research and engineering implementation at scale.

Good to have:

Experience with profiling and performance optimization, ideally of machine learning pipelines

Experience with low-level GPU programming such as CUDA or Triton

Experience with optimizing communications and libraries such as NCCL and techniques such as overlapping communication and computation

Experience with non-NVIDIA hardware such as AMD MI300X series of GPUs or other AI accelerators

Enjoy pushing hardware to its limits

High proficiency with Pytorch and Python

Strong ability to jump into large pre-existing codebases and rapidly get up to speed and become productive

Previously published machine learning research in well-respected venues

Postgraduate degree in scientific subject (Computer Science, EE/EECS, Math, Physics)

Culture:

Our research methodology is to make grounded, methodical steps toward ambitious goals. Both deep research and engineering excellence are equally valued

We strongly value new and crazy ideas and are very willing to bet big on new ideas

We move as quickly as we can; we aim to make the bar to impact as low as possible

We all enjoy what we do and love discussing AI

Benefits and Perks:

Medical, dental, vision and FSA plans

Competitive salary, equity and 401(k)

Relocation and immigration support on a case-by-case basis

On-site meals prepared by a dedicated culinary team; Thursday Happy Hours

General requirements:

Willing to be in-person in our office in Palo Alto

US authorization to work. We will consider O1 visa sponsorship for the right candidate.

Full-time