Headline

AI Engineering Leader | RAG, AI Search & Agentic Systems in Production | Director of Engineering @ InstantMarkets | LLMs · Python · Elasticsearch · MLOps

About

I build AI systems that go into production and stay there — and I lead the teams that keep them running. For the last 11 years I've been at InstantMarkets, moving from software engineer to Director of Engineering. Today I own architecture, roadmap, and delivery for a public procurement intelligence platform that ingests millions of documents from thousands of external sources and makes them searchable and answerable. I plan directly with the CEO. I still write code. What I actually do: → AI search & RAG — Designed enterprise search and conversational retrieval over a document corpus in the millions. Hybrid retrieval, relevance tuning, and grounded answers on messy real-world procurement data. → Agentic automation — LLM-driven workflows that take manual document triage and classification steps out of the pipeline entirely. → ML platform & MLOps — Experimentation, evaluation, and monitoring pipelines so model and retrieval changes ship against measured baselines, not guesswork. → Applied research that pays for itself — Wrote an adaptive crawl scheduler in Rust (Thompson Sampling with Beta priors, Whittle Index) after finding ~98% of crawl resources were being spent re-fetching unchanged pages. Simulation showed ~80% crawl volume reduction. → Leading engineers — 12–15 people across backend, frontend, AI/ML, DevOps, and QA. Hiring, architecture review, delivery, reliability, and cloud cost. Alongside my core role I've shipped for other teams: enterprise AI/ML monitoring at Vianai Systems — drift, data quality, and bias monitoring across tabular, image, and large language models — and a computer vision platform for Clove Dental. Stack I reach for: Python, LLMs and RAG frameworks, Elasticsearch, gRPC/REST, AWS and Azure, Docker/Kubernetes, Rust when latency matters. Happy to talk retrieval quality, evaluation, or what it actually takes to run AI in production. → prem6667@outlook.com

Location

Noida

AI stack

gemini

hermes agent

langchain

langgraph

TruLens

vLLM

Work experience

Director of Engineering at Instant Systems, Inc., Oct 2019 - Present · 7 yrs, Hybrid, • Own architecture, roadmap, and delivery for a procurement intelligence platform processing millions of documents from thousands of external sources; plan technical strategy directly with the CEO. • Lead 12–15 engineers across backend, frontend, AI/ML, DevOps, and QA — hiring, technical direction, and delivery accountability. • Architected enterprise AI search and conversational retrieval (RAG) across the full corpus, moving users from keyword search to grounded, answerable results. • Built agentic automation into document ingestion and classification, removing manual triage from the pipeline. • Established MLOps workflows for experimentation, evaluation, and monitoring so retrieval and model changes ship against measured baselines. • Designed an adaptive crawl scheduler in Rust (Thompson Sampling with Beta priors, Whittle Index) after diagnosing ~98% of crawl spend going to unchanged content; simulation showed ~80% crawl volume reduction. • Delivered secure REST and gRPC services with production monitoring, and drove cloud cost optimization across the platform.

AI/ML Engineer (Contract) at Vianai Systems, Inc., Oct 2022 - May 2024 · 1 yr 8 mos, India, Contract engagement supporting Vianai's enterprise AI platform work alongside my role at Instant Systems. • Contributed to Hila, Vianai's enterprise AI/ML platform, and VIANOPS, its high-scale model monitoring product. • Built monitoring for feature and prediction drift, data quality, and bias/fairness across tabular, unstructured/image, and large language models. • Worked on real-time root-cause analysis surfacing the high-risk segments driving model degradation. • Delivered Python and full-stack services integrating the platform across multiple clouds and data sources.

Software Engineer at Instant Systems, Inc., Oct 2015 - Oct 2019 · 4 yrs 1 mo, On-site, • Led the migration from Apache Solr to Elasticsearch, rebuilding indexing and relevance for the platform's core search. • Built web-acquisition frameworks (PhantomJS, CasperJS, Puppeteer) to collect and normalize procurement data from thousands of heterogeneous public sources. • Engineered document-intelligence and NLP pipelines with Apache Tika and Python for extraction, parsing, and classification at scale. • Managed a team of 4–5 developers while owning AWS infrastructure and CI/CD across production environments.

Education

B.Tech / B.E., Computer Engineering, Bundelkhand Institute of Engineering and Technology, Jhansi