Headline

Making ML models run fast on real hardware | ONNX · TensorRT · INT8/FP16 Quantization · llama.cpp/MLX On-Device Inference | AI Engineer @ Griphic

About

I make deep-learning models run in real time — on GPUs and on-device hardware. My work is everything after training: PyTorch → ONNX → TensorRT export, INT8/FP16 quantization, runtime profiling, and serving. It has shipped in two very different products — a multi-agent LLM service in production, which I own as its sole engineer, and an X-ray enhancement product taken through clinical trials to release — with a granted Indian patent and two more pending along the way(hopefully!), and a stretch leading an ML team. The details and numbers are in the roles below. The models I have shipped see (X-ray imaging) and read and write (LLMs). The next ones I want listening and watching — on-device. In my own time I benchmark LLM inference engines. I built a reproducible harness that holds mlx-lm and llama.cpp to one fairness contract on an M4 Pro, and measured two things their docs do not say: decode is memory-bandwidth-bound (~273 GB/s ceiling), and K isolated server instances return +9% throughput for 3× the memory. I also built ipic, a fully offline multimodal search engine that indexes a 350k-file drive at ~2,500 files/s. I contribute upstream where I work: a merged fix in ONNX Runtime's graph optimizer (#33091), and open PRs in PyTorch's ONNX exporter (#200180) and mlx-lm's server (#1944). M.Tech AI/ML @ BITS Pilani, 2027. Write-ups: anubhavagr.github.io. If your team is working on something interesting, I'd love to hear from you: anubhavagr.mail@gmail.com.

Location

Bengaluru

AI stack

claude code

hugging face

kilocode

openai

llama.cpp

vLLM

Work experience

AI Engineer - 2 at Griphic, Jan 2026 - Present · 11 mos, Delhi, India, Working on multi-agent LLM service in production (Python/FastAPI) — designed, built, shipped, and maintained by one person. I cut per-request generation latency cost by 60% by implementing token tracing, prompt caching, and bounding parallel fan-out. Outputs are safe to ship unread: 99.5% go out with zero human review, and a two-stage guardrail — a batched judge, then an independent adjudicator — stops prompt injection and credential leaks for at most 2 extra LLM calls per clean request. Every merge clears unit + BDD suites, SonarQube, and Trivy scans.

Machine Learning Engineer at Innvolution Healthcare Private Limited, Sep 2023 - Dec 2025 · 2 yrs 4 mos, Bengaluru, Healthcare ML with real stakes: images from live cardiac procedures, clinical trials, regulators watching. I owned AIMAG — X-ray super-resolution and denoising, $200K+ ARR — from dataset curation through release after clinical trials. The part I actually loved was the part nobody posts about: dragging PyTorch models through ONNX export (dynamic shapes, unsupported ops, opset mismatches) into TensorRT with INT8/FP16 quantization, landing at sub-5 ms per frame for 4× upsampling on an RTX 4090. Also fine-tuned U-Net++ (ResNet50 + attention) for coronary segmentation running in under 4 seconds in the cath lab, co-invented a granted patent (No. 604176) and two pending applications in X-ray enhancement, and led a team of engineers— roadmap, code review, deployment runbooks.

Image Quality Engineer (Machine Learning) Intern at Innvolution Healthcare Private Limited, Dec 2022 - May 2023 · 6 mos, Bangalore, Motion-compensated image stitching and denoising for Digital Subtraction Angiography (DSA) images

Computer Vision Intern at enord, Sep 2022 - Dec 2022 · 4 mos, Delhi, India

Research Intern at ISRO - Indian Space Research Organization, Jun 2022 - May 2023 · 1 yr, Remote, Time-series forecasting (XGBoost/CatBoost/LightGBM) on NDVI (Normalized Difference Vegetation Index) data from AWiFS, MODIS and Sentinel satellites.

Education

Master of Technology - MTech, Artificial Intelligence and Machine Learning, BITS Pilani Work Integrated Learning Programmes

Bachelor of Technology - BTech, Computer Science and Engineering, Bhilai Institute of Technology (BIT), Durg

Public links

github: https://github.com/anubhavagr

anubhavagr.github.io: https://anubhavagr.github.io/