---
title: "Vaibhav Agarwal - Lead Software Engineer in Greater Bengaluru Area | Tal"
description: "Meet Vaibhav Agarwal, Lead Backend & Platform Engineer @ LeadSquared (Ex-HPE) | Go, Kafka, ClickHouse, K8s | Scaled Multi-Tenant Streaming (~100× Lag Reduction, 40% Infra Cost Savings) based in Greater Bengaluru Area. AI stack includes antigravity, claude code, and claude. Explore public work links..."
image: "https://tal.club/meet/vaibhav-agarwal-7/share-image"
---

# Vaibhav Agarwal on Tal

## Headline

Lead Backend & Platform Engineer @ LeadSquared \(Ex\-HPE\) \| Go, Kafka, ClickHouse, K8s \| Scaled Multi\-Tenant Streaming \(\~100× Lag Reduction, 40% Infra Cost Savings\)

## About

I am a Lead Software Engineer and Distributed Systems Architect with \~8 years of experience building high\-throughput streaming pipelines, analytical query engines, and cloud\-native infrastructure across LeadSquared and Hewlett Packard Enterprise \(HPE\) Over the past five years at LeadSquared, I have served as a core architect and major contributor for an enterprise multi\-tenant analytics and data platform serving \~2,000 organizations and processing \~2 million daily Kafka streaming events\. I specialize in taking bottlenecked, high\-scale production systems and re\-architecting them for throughput, resilience, and cost efficiency: • \~100× Streaming Throughput & Lag Optimization: Re\-architected a synchronous Kafka to ClickHouse write pipeline by designing a windowed batch\-accumulation and in\-memory deduplication engine, eliminating single\-record round\-trip bottlenecks while enforcing strict offset ordering and flush\-barrier synchronization across interleaved schema and data streams\. • 40% Cloud Compute Cost Reduction: Designed event\-driven Kubernetes consumer autoscaling using KEDA and AWS MSK, simplifying cluster topology by replacing a custom master\-worker pod model with automated scale\-to\-zero policies, level\-triggered self\-healing reconciliation, and per\-workload resource tuning\. • 65% Memory & 70% Storage Optimization: Diagnosed and resolved critical production memory leaks to cut service memory footprint by 65%, and restructured configuration storage schemas to reduce cloud storage spend by 70%\. • Analytical Query Compiler & Engine Design: Scaled and stabilized the distributed SQL query generator supporting multi\-dataset Common Table Expression \(CTE\) joins, dynamic multi\-dimensional pivot tables, and runtime\-parametric filter evaluation\. • Core Reporting APIs & Incremental ETL: Designed and shipped customer\-facing analytical REST APIs, role\-based configuration services, audit logging, and incremental ETL data synchronization pipelines across Go, MongoDB, and ClickHouse\. Beyond architecture and hands\-on implementation, I drive a strong written engineering culture \(authoring Architecture Decision Records, root\-cause analyses, and operational runbooks\), serve as the primary technical lead coordinating data contracts across 5 partner engineering teams, mentor junior engineers, and leverage AI\-assisted development tools \(Claude Code\) daily to accelerate architecture exploration and delivery\. Core Specialties: Go, Python, Apache Kafka, ClickHouse, Kubernetes, KEDA, MongoDB, Redis, AWS, Prometheus, Distributed Systems & Cloud FinOps\.

## Location

Greater Bengaluru Area

## AI stack

antigravity

claude code

claude

gemini

notebooklm

ollama

## Work experience

Lead Software Engineer at LeadSquared, Apr 2025 \- Present · 1 yr 7 mos, On\-site, \- \~100× Streaming Lag Reduction & Back\-Pressure Control: Owned end\-to\-end re\-architecture of the core Kafka\-to\-ClickHouse ingestion pipeline processing \~2M daily events across \~1,000 enterprise tenants, replacing synchronous single\-record writes with a windowed batch\-accumulation and in\-memory deduplication engine to reduce consumer lag by \~100× while preserving strict offset ordering and schema consistency\. \- 40% Cloud Infrastructure Cost Reduction: Designed and deployed workload\-driven Kubernetes autoscaling using KEDA and AWS MSK, simplifying cluster topology by replacing a custom master\-worker pod model with automated scale\-to\-zero policies, self\-healing reconciliation, and per\-workload resource tuning\. \- Pipeline Fault Isolation & Data Integrity: Overhauled event\-driven failure recovery by deprecating legacy delay\-retry queues in favor of strongly typed error classification \(segregating transient infrastructure faults from malformed payload packets\), deterministic Dead\-Letter Queue \(DLQ\) routing, and strict schema\-validation guardrails\. \- Analytical Query Engine Extensibility: Designed and shipped a runtime\-parametric filter evaluation engine within the analytical query compiler, enabling dynamic dashboard filtering at query execution time while maintaining full backward compatibility across core database interfaces\. \- Cross\-Functional Technical Leadership: Served as the primary technical authority coordinating data contracts, schema evolution, and pipeline integrations across 5 external engineering teams; authored formal Architecture Decision Records \(ADRs\) and operational runbooks, established Prometheus lag observability and feature\-flagged canary rollouts, enforced code review standards, and mentored junior engineers\.

Senior Software Development Engineer \(SDE3\) at LeadSquared, Apr 2024 \- Mar 2025 · 1 yr, On\-site, \- 65% Memory amp; 70% Storage Cost Reduction: Diagnosed and resolved critical production memory leaks to cut service memory consumption by 65%, and re\-architected configuration storage models to reduce cloud storage costs by 70%\. \- Distributed SQL Query Compiler amp; Cross\-Dataset Joins: Scaled and stabilized the analytical query generation engine supporting complex multi\-dataset joins and Common Table Expressions \(CTEs\), eliminating systemic query\-planning and data\-hierarchy bottlenecks\. \- Multi\-Dimensional Pivot & Aggregation Engine: Led end\-to\-end delivery and production stabilization of dynamic pivot\-table reporting across high\-volume datasets, optimizing expression\-tree compilation for complex filtering, sorting, and drilldown analytics\. \- Distributed Scheduler Reliability: Spearheaded written root\-cause analyses and architectural fixes for complex UTC timezone normalization and recurring job execution edge cases across the distributed report scheduling platform\.

Software Engineer \(SDE 2\) at LeadSquared, Dec 2021 \- Mar 2024 · 2 yrs 4 mos, Bengaluru, \- Zero\-Downtime Legacy Platform Migration: Engineered automated CLI migration tooling, data\-transformation pipelines, and MongoDB indexing utilities to seamlessly transition enterprise customers from legacy reporting architectures to a modern multi\-tenant schema\. \- Core Reporting APIs & Incremental ETL: Designed and shipped customer\-facing analytical REST APIs, role\-based configuration services, audit logging, and incremental ETL data synchronization pipelines across Go, MongoDB, and ClickHouse\. \- Conducted technical interviews and mentored junior developers\.

Software Engineer at Hewlett Packard Enterprise, Jul 2019 \- Dec 2021 · 2 yrs 6 mos, Bengaluru Area, India, HPE Storage: Cloud Data Services; Software Engineer \(March 2020 – December 2021\) o Backend developer on the Functions, Analytics and GraphQL modules → Adding and modifying functionalities and Bug fixes → Involved in design discussions and worked on most effective and efficient implementation → Writing Unit Tests and maintaining proper documentation o Worked on the new Rest API Framework → Investigating tools for effective validation and proper documentation for the REST framework → OpenApi documentation and framework for request and response validation → Bug fixes o A POC on an Automated Support Chat Bot, earning first prize in HPE Storage Hackathon o Tools: GoLang, Kubernetes, Docker, Python, Shell Script, GraphQL, SQL HPE Servers: Synergy ImageStreamer; Systems Software Engineer \(Jul 2019 \- March 2020\) o i3s System Performance Monitoring and Fault Prediction Model using ML concepts → Performance analysis and Effective visualization based on key parameters \(like response times, data sent, errors, etc\.\)\. → Fault prediction model using ML concepts, like clustering, classification, and neural networks\. → Developed an easy to use application for using the model\. → Model was developed in Python, and application using HTML/CSS\. Initial work on extending to use ELK stack\. → Model can be used by ATC/ERT/Engineering teams for providing better customer experience via optimizing resources to suit customer needs\. o i3s system software support o cloud\-init \(a multi\-platform cloud instance initialization tool\) integration → In\-depth Study, Prototyping and testing of the cloud\-init tool, for integration with i3s and using it for OS personalization and deployment\. → This helped standardize OS configuration script irrespective of Linux variant, thereby saving efforts in developing and maintaining separate configuration scripts for different Linux variants\. o Tools: Java, Python, Shell Script, SQL

Research And Development Intern at Hewlett Packard Enterprise, Jan 2019 \- Jul 2019 · 7 mos, Bengaluru Area, India, Built a tool for automated creation of OS configuration scripts in guestfish format using python and shell script\. Tool helps to reduce the time required to develop and test artifacts for any new OS for end users of i3s and worked on log analysis\.

Research Intern at ISRO \- Indian Space Research Organization, Aug 2018 \- Feb 2019 · 7 mos, Bangalore, Karnataka, \- Did research in the fields of Deep Learning, Machine Learning, Statistical Modelling and Image Processing for Remote Sensing at RRSC \- Bangalore\. \- Co\-author of a research paper titled "DEEP LEARNING AND STATISTICAL MODELS FOR DETECTION OF WHITE STEM BORER DISEASE IN ARABICA COFFEE"\. The paper was published in ISPRS Archives in February, 2019\. \- Was invited to present my paper at the ISPRS\-GEOGLAM\-ISRS International Workshop on Earth Observations for Agricultural Monitoring on 19th February, 2019 at the Indian Agricultural Research Institute \(IARI\), New Delhi\. \- Tools: Tools: Keras, Tensorflow, Scikit\-learn, Jupyter Notebook, Pandas, Python, Shell Scripts

Research And Development Intern at IIT Mandi, May 2018 \- Jul 2018 · 3 mos, Himachal Pradesh, India, Metrics such as number of requests, response time and bytes transferred were mined from internet access logs\. Statistical time series analysis methods such as stationarity tests, change point detection, long range dependency tests were carried out on the data\. Autoencoders and convolutional autoencoders were used to build a classifier to classify the data as normal or abnormal\. Recurrent Neural Networks \(LSTMs\) were used for predicting the network usage\. Tools: Keras, Tensorflow, Scikit\-learn, NLTK, Jupyter Notebook, Pandas, Python, Shell Scripts

Research And Development Intern at Indian Institute of Technology, Mandi, May 2017 \- Aug 2017 · 4 mos, Himachal Pradesh, India, This project involved data analytics on the ping rate data of the IIT Mandi university network\. The goal of the project was to incorporate a time series prediction model with contextual information to give better predictions\. K\-means clustering was used to group together similar data points and inference with respect to timings were made\. Different statistical tests were applied to test the stationarity of data and to check for long range dependence in data\. During the analysis phase it was found that there was a surge in ping rates during exams, major sport events, online sales, major webcasts, etc\. Amongst ARIMA modelling, Artificial Neural Networks, NARX nets and Recurrent Neural Networks\(LSTMs\), NARX nets and LSTMs performed better for time series prediction\. Contextual information \(such as major news events, sports matches, etc\) was scraped off the web and the university calendar\. Weights were assigned to the events and the final output was a function of the output of the time series model and the context\. The hybrid model showed better results than a basic time series prediction model\. Tools: Keras, Tensorflow, Scikit\-learn, NLTK, Jupyter Notebook, Pandas, Python, Shell Scripts

## Education

Bachelor of Technology, Computer Science, PES University

Individual Courses, Computer Science, Georgia Institute of Technology

## Public links

- vaibhavagarwal\.vercel\.app: https://vaibhavagarwal\.vercel\.app/

[View the original profile](https://tal.club/meet/vaibhav-agarwal-7)

```json
{
  "@context": "https://schema.org",
  "@graph": [
    {
      "@type": "Person",
      "@id": "https://tal.club/meet/vaibhav-agarwal-7#person",
      "name": "Vaibhav Agarwal",
      "url": "https://tal.club/meet/vaibhav-agarwal-7",
      "image": "https://tal.club/meet/vaibhav-agarwal-7/profile-photo.jpg",
      "description": "Meet Vaibhav Agarwal, Lead Backend & Platform Engineer @ LeadSquared (Ex-HPE) | Go, Kafka, ClickHouse, K8s | Scaled Multi-Tenant Streaming (~100× Lag Reduction, 40% Infra Cost Savings) based in Greater Bengaluru Area. AI stack includes antigravity, claude code, and claude. Explore public work links...",
      "jobTitle": "Lead Software Engineer",
      "worksFor": {
        "@type": "Organization",
        "name": "LeadSquared"
      },
      "hasOccupation": {
        "@type": "Occupation",
        "name": "Lead Software Engineer"
      },
      "homeLocation": {
        "@type": "Place",
        "name": "Greater Bengaluru Area"
      },
      "alumniOf": [
        {
          "@type": "EducationalOrganization",
          "name": "PES University"
        },
        {
          "@type": "EducationalOrganization",
          "name": "Georgia Institute of Technology"
        }
      ],
      "knowsAbout": [
        "antigravity",
        "claude code",
        "claude",
        "gemini",
        "notebooklm",
        "ollama"
      ],
      "sameAs": [
        "https://vaibhavagarwal.vercel.app/"
      ],
      "mainEntityOfPage": {
        "@id": "https://tal.club/meet/vaibhav-agarwal-7#webpage"
      }
    },
    {
      "@type": "ProfilePage",
      "@id": "https://tal.club/meet/vaibhav-agarwal-7#webpage",
      "url": "https://tal.club/meet/vaibhav-agarwal-7",
      "name": "Vaibhav Agarwal on Tal",
      "description": "Meet Vaibhav Agarwal, Lead Backend & Platform Engineer @ LeadSquared (Ex-HPE) | Go, Kafka, ClickHouse, K8s | Scaled Multi-Tenant Streaming (~100× Lag Reduction, 40% Infra Cost Savings) based in Greater Bengaluru Area. AI stack includes antigravity, claude code, and claude. Explore public work links...",
      "dateModified": "2026-10-21T05:09:26.416Z",
      "isPartOf": {
        "@id": "https://tal.club/#website"
      },
      "mainEntity": {
        "@id": "https://tal.club/meet/vaibhav-agarwal-7#person"
      },
      "primaryImageOfPage": {
        "@type": "ImageObject",
        "url": "https://tal.club/meet/vaibhav-agarwal-7/profile-photo.jpg",
        "contentUrl": "https://tal.club/meet/vaibhav-agarwal-7/profile-photo.jpg",
        "caption": "Vaibhav Agarwal profile photo on Tal"
      },
      "inLanguage": "en-IN"
    }
  ]
}
```