Inference Stack Engineer

Inference Stack Engineer

(AI Systems / Compiler & Runtime)

We are building a next-generation AI inference stack designed for high-performance execution on modern and custom compute architectures. Our mission is to deliver industry-leading low-latency and high-throughput AI systems by designing and optimizing the full execution path — from model representation to hardware-level execution.

This is a deeply technical role at the intersection of compiler systems, AI runtimes, and high-performance computing .

You will work on core infrastructure that defines how modern AI models are executed efficiently at scale.

What you will do

• Design and build components of an AI inference stack , from high-level model representation to low-level execution

• Develop and extend a Python-based DSL for expressing AI workloads and kernels

• Work on compiler infrastructure including:

  • IR design and transformation pipelines

  • graph lowering and optimization passes

  • backend code generation for target execution environments

• Optimize model execution for:

  • latency

  • throughput

  • memory efficiency

  • numerical stability

• Contribute to runtime systems responsible for model execution and scheduling

• Profile and analyze inference workloads to identify system bottlenecks

• Collaborate closely with hardware and systems engineers on execution efficiency

• Influence architecture decisions for next-generation AI execution platforms

What we are looking for

• Strong software engineering background (C++ and Python)

• Experience with performance-critical systems or compiler-related work

• Understanding of AI model execution (especially transformers / LLMs)

• Familiarity with compute graphs, tensor operations, or execution frameworks

• Ability to analyze complex systems end-to-end (model → runtime → hardware)

• Experience working with large codebases and system-level debugging

• Strong communication skills and ability to work in cross-functional teams

Nice to have

• Experience with compiler frameworks such as:

  • LLVM

  • MLIR

  • Triton

  • TVM

  • XLA

• Experience contributing to deep learning frameworks (PyTorch, TensorFlow, JAX)

• Understanding of GPU or accelerator execution models

• Experience with kernel optimization or operator-level performance tuning

• Knowledge of distributed inference systems (e.g. NCCL, RPC-based serving)

• Familiarity with hardware-aware optimizations (memory hierarchy, vectorization, scheduling)

What we offer

• Work on the core execution layer of modern AI systems

• Direct impact on inference performance of large-scale AI workloads

• Collaboration with experts in compilers, systems, and AI infrastructure

• Highly technical environment with strong engineering autonomy

• Opportunity to shape the architecture of a next-generation inference stack

• Competitive compensation and flexible working model

Why this role is different

This is not a typical ML engineering or application role.

You will not be training models.

You will be working on how models actually run efficiently , at scale, across compute systems, shaping the performance layer that sits between AI models and hardware.