Headline

AI Infrastructure Engineer | LLM Inference & Serving | Distributed Systems | vLLM Contributor | Microsoft, Ex-Amazon

About

I build the platform layer between infrastructure and applications — control planes, scheduling, and developer tooling — most recently applied to ML serving and inference. My core thesis: the bottleneck in AI isn't always the model; it's the runtime. Backed by 7+ years of distributed systems engineering at Microsoft and Amazon, I take infrastructure ideas from research → production → open source. ⚙️ Production Infrastructure • Microsoft: Built distributed metadata infrastructure supporting 17,000+ microservices, owning a 99.99% availability SLA. Focused on event-driven systems, adaptive concurrency, observability, and developer tooling. • Amazon: Engineered scheduling/orchestration for Alexa device infrastructure. Built resilient fleet management with self-healing automation, achieving 98.4% scheduler uptime across 50+ physical device categories. 🚀 AI Infrastructure & LLM Serving • Adaptive Compute Orchestrator (ACO): Custom K8s scheduler extender on GKE (Terraform, multi-tenant isolation, Prometheus/Grafana). Applies ant-colony optimisation to cluster scheduling, improving placement efficiency by ~70-80%. • Clairvoyant: Drop-in scheduling proxy for serial LLM backends (Ollama, llama.cpp). Uses a lightweight ONNX predictor (0.029ms P99 overhead) to reorder requests, reducing short-request P50 latency by 70–76% under burst traffic. (arXiv:2606.07248) • vLLM Contributions: Two open PRs in vLLM's core scheduler, demonstrating hands-on experience with production-grade, parallelised LLM inference serving internals. • ServiceScope v2: Agentic dependency mapping combining AST parsing with local LLM inference (gemma3:4b). Orchestrated with FastAPI, Celery, Redis, Postgres, Neo4j. 🎯 What I'm Looking For: A role bridging research and production in AI infrastructure — building and operating platforms that make AI serving reliable, efficient, and scalable. Focused on inference serving, scheduling systems, and multi-tenant GPU platforms. Core tech: Python • Go • C++ • C# • Kubernetes • FastAPI • Azure • AWS • Redis • Cosmos DB • PostgreSQL • Neo4j • ONNX • XGBoost • Prometheus • Grafana

Location

Chennai

AI stack

antigravity

claude code

hugging face

claudecode

Cohere

Synthesia

Work experience

Founder / ML infrastructure Engineer at Anavind labs, Mar 2024 - Present · 2 y 7 mo, Hyderabad, Independent Research Practice · Hyderabad, India • Independent systems research on LLM inference, GPU memory, disaggregated serving and training correctness; prototypes benchmarked on rented NVIDIA GPUs and GKE, results labelled measured, modeled or synthetic; published Clairvoyant (arXiv:2606.07248); upstream vLLM and SGLang fixes

Software Engineer at Microsoft IDC, Aug 2021 - Nov 2023 · 2 y 3 mo, Hyderabad, Service Inventory & Metadata Platform (Internal Production Platform) Designed and evolved Microsoft’s distributed metadata platform responsible for maintaining deployment truth across 17,000+ microservices, enabling reliable service registration, validation, and recovery through event-driven infrastructure. Key Contributions • Architected a distributed validation platform using Azure Service Bus, Cosmos DB, Azure Functions, and Redis to process service metadata at enterprise scale. • Designed adaptive concurrency control using Redis (95% optimistic / 5% pessimistic locking), eliminating cascading retry storms during production-scale outages. • Built Azure-native active-passive BCDR workflows with idempotent event processing, achieving zero-data-loss recovery objectives and sub-minute failover. • Developed Windows ARM64 Validation Provisioner using distributed worker nodes, reducing validation time from 60+ minutes to ~20 minutes. • Built a Test-in-Production telemetry framework using lightweight C++ runtime instrumentation, adopted across 10 engineering groups to improve production observability. • Automated Windows validation workflows using Power Automate and RPA, reducing UI-related release defects by approximately 40%. Technologies Azure Service Bus • Cosmos DB • Redis • Azure Functions • C# • C++ • Distributed Systems • Event-driven Architecture • Observability

Software Engineer in Test Infrastructure at Amazon, Sep 2017 - Aug 2021 · 4 yrs, Greater Chennai Area, Alexa Device Infrastructure Platform Designed distributed scheduling and orchestration infrastructure powering automated validation across Alexa hardware fleets. Key Contributions • Engineered a deterministic scheduling platform orchestrating continuous workloads across 50+ physical Alexa devices. • Designed resilient fleet management through automated recovery, ADB keepalive monitoring, and self-healing device orchestration. • Eliminated non-deterministic hardware failures through automated state management, Linux network isolation, and runtime health monitoring. • Built Python-based diagnostic tooling aggregating multi-device crash logs, saving over 1,000 engineering hours annually. • Automated firmware promotion and validation workflows across multiple Alexa hardware families. Technologies Java • Python • AWS • Device Orchestration • Scheduling • Distributed Infrastructure • CI/CD • Linux

Technical Lead at OMG LABS, Apr 2017 - Sep 2017 · 6 mos, Chennai Area, India, • Spearheaded customer acquisition through a subscription-based model, boosting customer traffic by 15–20% in key market segments. • Led end-to-end project delivery — from strategy and design to technical implementation and deployment. • Managed vendor relationships and integrated e-commerce tools such as site search and email marketing solutions, improving digital presence by 25–35%. • Played a key role in achieving Amazon Launchpad selection, positioning OMG Labs among top emerging startups for international expansion. • Coordinated across product, marketing, and tech teams to ensure agile execution and sustainable growth infrastructure.

Education

Bachelor of Technology (B.Tech.), Information Technology, Anna University Chennai

HSC, Computer Science, Velammal Matriculation Hr.Sec.School

Public links

github: https://github.com/aravind0403

anavindlabs.com: https://www.anavindlabs.com/