Headline

AI Inference Engineer · Applied AI Engineer | LLM Serving & Optimization · vLLM · Quantization

About

AI engineer focused on LLM inference systems and production AI products. I work on LLM serving, optimization, and deployment across Trainium and NVIDIA GPUs. My main open-source contribution is AWS Neuron NxDI PR #173, where I brought up Qwen 3.6-27B hybrid (DeltaNet + GQA) serving on AWS Trainium with custom NKI kernels, hybrid recurrent + KV cache, selective FP8, 256K context, and vLLM/OpenAI-compatible serving. At Cultus, I build and operate production AI systems: video interview analysis, adaptive learning, RAG-based assessments, automated grading, structured-output validation, background jobs, and real-time AI flows. I also self-host Qwen 3.6 models on NVIDIA GPUs with vLLM, NVFP4 quantization, use-case-based routing, and Kubernetes. I'm currently focused on inference systems - vLLM, NKI, quantization, KV/prefix caching, TensorRT-LLM, SGLang - while keeping AI engineering roles open. Ramping CUDA/Triton.

Location

Bengaluru

AI stack

claude code

codex

deepseek

hermes agent

hugging face

opencode

Work experience

AI Inference Engineer at GyanSys Inc., Sep 2026 - Present · 1 mo, Bengaluru, Advanced AI Systems (Client Deployment — Global Semiconductor R&D) Focusing on distributed LLM serving, inference optimization, and high-performance GPU infrastructure across enterprise cloud and secure on-premises environments.

AI Engineer at Cultus Education & Technology Services, May 2024 - Present · 2 yrs 4 mos, Bengaluru, Karnataka, India, What I Built: Cultus Skills Center Solo-architected and developed an AI-powered education platform from scratch. Built LLM systems for video interview analysis and adaptive learning. Key Work: Real time AI Mock-Video interview analysis system processing 100+ interviews/week Multi-stage AI pipeline with structured output validation Complete ownership: architecture — code — deployment — maintenance Self-hosted LLM inference on NVIDIA GPUs: deployed Qwen 3.6 models with vLLM, NVFP4 quantization, use-case-based routing, and Kubernetes serving.

Software Engineer at I-RAY IT SOLUTIONS, Nov 2022 - Apr 2024 · 1 yr 6 mos, Hyderabad, Telangana, India

Software Engineer at i5 Systems Private Limited, May 2022 - Aug 2022 · 4 mos, Hyderabad, Telangana, India

Education

Bachelor of Technology - BTech, Computer Science, JNTUH College of Engineering Hyderabad

Public links

github: https://github.com/m-deepankar-singh