Shubham Kumar profile photo 1 on Tal

Shubham Kumar

Senior Backend Engineer | Building Scalable Distributed Systems | Python · Golang · Java · Scala | Kubernetes · Spark · System Design · Cloud

India

5+ years of work experience

Nielsen
May 20251 yr 5 mosPresent

• Built multimodal video search pipeline using Gemini Multimodal API + Vertex AI — parsed video clips into PyTorch embeddings enabling semantic clip-level search and segment-to-text retrieval across 50M+ media assets at sub-200ms latency. • Architected FAISS + ApertureDB semantic search platform with consistent-hashing sharding across 20+ shards serving 1,200 QPS peak — reduced retrieval latency 65% and eliminated 40% cross-shard queries with fault-isolated horizontal scaling. • Building AI agent–powered web crawler using MCP protocol and LangChain — automates media metadata ingestion from external sources with LLM-driven extraction and DeepEval-based output quality validation. • Designed Kubernetes media ingestion platform (100K+ req/day, burst 300 QPS) via Golang and Python microservices on Kafka; re-architected Spark ETL DAGs cutting 100M+ records/day runtime 30% and EMR costs 20%.
AdvaRisk
Jun 20232 yrsMay 2025

India

• Built end-to-end AI-integrated data pipeline ingesting loan applicant records (farmers, property owners) — scraped government land registry websites at scale, parsed and normalized structured data via Django, and routed records through an ML document parsing service for automated field extraction before persisting to MySQL. • Engineered a large-scale property database aggregating land records from multiple state government portals — automated scraping, deduplication, and enrichment pipeline reducing manual ops workload by 70%. • Implemented OTP-based 2FA and secure authentication system (OAuth 2.0 + RBAC) for regulated financial data access; system passed SOC 2 audit serving 10K+ enterprise users at 99.95% SLA. • Cut query latency 40% at 5K RPM via PostgreSQL schema redesign with compound and partial indexes; reduced deployment time 50% via GitHub Actions + ArgoCD blue-green pipeline.
Melorra
Jul 20221 yrJun 2023

Bengaluru

• Built event-driven order processing and inventory management backend for a jewellery e-commerce platform — Kafka pipelines processing 1M+ events/day with exactly-once semantics and dead-letter queue handling for 99.99% reliability. • Designed Redis write-through caching layer reducing DB load 60% and p99 latency from 200ms to 80ms; built API gateway handling 50K+ req/min with token-bucket rate limiting and JWT auth.
Prodapt Solutions
Apr 20211 yr 4 mosJul 2022

Bengaluru

• Built real-time network telemetry collection system using SNMP and Riverbed collector agents — streamed metrics via Kafka into Prometheus for live visualization on Grafana dashboards for enterprise network clients. • Reduced MTTD from 35 min to under 5 min across 100+ distributed services via configurable PagerDuty SLA breach alerting integrated with the observability stack. • Improved p99 REST API latency 45% at 50K+ req/min via async processing, connection pooling, and payload serialization optimization.

Education

Uttarakhand Technical University

Bachelor of Technology, Computer Science and Engineering

Uttarakhand Technical University

20164 yrs2020

Join Shubham and 10k+ other professionals on

Tal Boss

Tal is the app where bosses meet engineers & builders directly, without middlemen

Download Tal Boss

Shubham Kumar on Tal

Headline

Senior Backend Engineer | Building Scalable Distributed Systems | Python · Golang · Java · Scala | Kubernetes · Spark · System Design · Cloud

About

Senior Backend Engineer specializing in AI infrastructure, multimodal search, and distributed systems. 5+ years building production LLM pipelines, vector search engines, and event-driven platforms at scale. What I build: • Multimodal video search across 50M+ media assets at sub-200ms latency (Gemini Multimodal API + Vertex AI + PyTorch + FAISS + ApertureDB) • AI-agent–powered crawling systems using MCP protocol + LangChain + Claude, with DeepEval-based output validation • Kubernetes ingestion platforms handling 100K+ req/day and Spark ETL DAGs processing 100M+ records/day • FAISS + ApertureDB semantic search with consistent-hashing sharding across 20+ shards (1,200 QPS peak) Tech: Python · Golang · Java · Scala · Kafka · Spark · Kubernetes · AWS (EKS/S3/EMR) · DynamoDB · Redis · LangChain · FAISS · Vertex AI · Claude · Gemini System design strengths: sharding, distributed consistency, fault tolerance, event-driven architecture, multi-tenant isolation, LLM cost optimization.

Location

India

Work experience

Senior Software Engineer at Nielsen, May 2025 - Present · 1 yr 5 mos, • Built multimodal video search pipeline using Gemini Multimodal API + Vertex AI — parsed video clips into PyTorch embeddings enabling semantic clip-level search and segment-to-text retrieval across 50M+ media assets at sub-200ms latency. • Architected FAISS + ApertureDB semantic search platform with consistent-hashing sharding across 20+ shards serving 1,200 QPS peak — reduced retrieval latency 65% and eliminated 40% cross-shard queries with fault-isolated horizontal scaling. • Building AI agent–powered web crawler using MCP protocol and LangChain — automates media metadata ingestion from external sources with LLM-driven extraction and DeepEval-based output quality validation. • Designed Kubernetes media ingestion platform (100K+ req/day, burst 300 QPS) via Golang and Python microservices on Kafka; re-architected Spark ETL DAGs cutting 100M+ records/day runtime 30% and EMR costs 20%.

Software Engineer at AdvaRisk, Jun 2023 - May 2025 · 2 yrs, India, • Built end-to-end AI-integrated data pipeline ingesting loan applicant records (farmers, property owners) — scraped government land registry websites at scale, parsed and normalized structured data via Django, and routed records through an ML document parsing service for automated field extraction before persisting to MySQL. • Engineered a large-scale property database aggregating land records from multiple state government portals — automated scraping, deduplication, and enrichment pipeline reducing manual ops workload by 70%. • Implemented OTP-based 2FA and secure authentication system (OAuth 2.0 + RBAC) for regulated financial data access; system passed SOC 2 audit serving 10K+ enterprise users at 99.95% SLA. • Cut query latency 40% at 5K RPM via PostgreSQL schema redesign with compound and partial indexes; reduced deployment time 50% via GitHub Actions + ArgoCD blue-green pipeline.

Software Engineer at Melorra, Jul 2022 - Jun 2023 · 1 yr, Bengaluru, • Built event-driven order processing and inventory management backend for a jewellery e-commerce platform — Kafka pipelines processing 1M+ events/day with exactly-once semantics and dead-letter queue handling for 99.99% reliability. • Designed Redis write-through caching layer reducing DB load 60% and p99 latency from 200ms to 80ms; built API gateway handling 50K+ req/min with token-bucket rate limiting and JWT auth.

Associate Software Engineer at Prodapt Solutions, Apr 2021 - Jul 2022 · 1 yr 4 mos, Bengaluru, • Built real-time network telemetry collection system using SNMP and Riverbed collector agents — streamed metrics via Kafka into Prometheus for live visualization on Grafana dashboards for enterprise network clients. • Reduced MTTD from 35 min to under 5 min across 100+ distributed services via configurable PagerDuty SLA breach alerting integrated with the observability stack. • Improved p99 REST API latency 45% at 50K+ req/min via async processing, connection pooling, and payload serialization optimization.

Education

Bachelor of Technology, Computer Science and Engineering, Uttarakhand Technical University