AI Inference Engineer

Responsibilities (some of the following)

• Design, develop, and maintain components of the deployment stack and software kernels for AI compute platforms
• Optimize and implement core ML operators (e.g., GEMMs, convolutions, BLAS routines, SIMD kernels)
• Translate computational graphs from ML frameworks onto the underlying hardware
• Contribute to compiler infrastructure together with compiler and hardware teams
• Investigate and resolve issues through system-level debugging and performance analysis
• Deliver scalable software solutions under ambitious development schedules
• Define and apply practices for testing, deployment, and scaling AI systems

• Bachelor’s degree in Computer Science, Engineering, Mathematics, or related discipline, with 3+ years of professional software development experience
• Solid knowledge of computer architecture, system software, data structures
• Strong programming skills in C/C++ or Python in Linux environments using common development tools
• Hands-on experience implementing algorithms in high-level languages (C/C++/Python)
• Exposure to specialized hardware (GPUs, FPGAs, DSPs, AI accelerators) and frameworks such as OpenCL or CUDA
• Experience designing or working with high-performance software systems
• Solid knowledge of ML fundamentals
• Motivated team player with a strong sense of responsibility

You are a great fit if you have experience in at least one of the following areas:

• Model serving frameworks (e.g., Triton Inference Server, DeepSpeed Inference, vLLM)
• Deep learning frameworks (e.g., PyTorch, TensorFlow)
• ML runtimes (e.g., ONNX Runtime, TVM, IREE, XLA)
• Distributed collectives (e.g., Gloo, MPI)
• Software testing and validation methodologies
• Deploying ML workloads (LLMs, VLMs, NLP, etc.) across distributed systems
• Implementation of ML operators and kernels (e.g., SIMD routines, Activation functions, Pooling layers, Quantization layers)
• Hardware-aware optimizations and performance tuning
• 2+ years of experience developing software targeting AI hardware