---
title: "Tanmay gupta - Senior Data Engineer in Bengaluru | Tal"
description: "Meet Tanmay gupta, Senior Data Engineer @ iCIMS (TL) | Scalable Data Pipelines | PySpark, Spark SQL, Airflow | Big Data & ML Systems | ex-synopsys based in Bengaluru. AI stack includes antigravity, claude code, and cursor."
image: "https://tal.club/meet/tanmay-gupta-8/share-image"
---

# Tanmay gupta on Tal

## Headline

Senior Data Engineer @ iCIMS \(TL\) \| Scalable Data Pipelines \| PySpark, Spark SQL, Airflow \| Big Data & ML Systems \| ex\-synopsys

## About

I am a Senior R&D Data Engineer with 6\+ years of experience designing and building scalable, production\-grade data systems\. I specialize in end\-to\-end big data pipelines – from data ingestion and preprocessing to feature engineering and analytics\-ready datasets – with a strong focus on performance, scalability, and data quality\. I have built systems that process massive Electronic Design Automation \(EDA\) metrics and tool\-flow data to empower chip design teams with faster, data\-driven insights\. At Synopsys, I’ve led projects on ML\-driven analytics platforms for chip design optimization\. My work has included developing PySpark\-based data processing workflows, creating predictive models to spot and mitigate design congestion issues early, and implementing graph database solutions \(using Neo4j and ArangoDB\) to analyze complex upstream/downstream dependencies\. These efforts have enabled faster analytics turnaround for R&D teams and have directly supported better design decisions\. My core strengths are in Python and the Hadoop ecosystem \(Spark SQL, Hive, HDFS\), as well as orchestration tools like Airflow\. I enjoy collaborating cross\-functionally to translate complex engineering problems into robust data solutions\. I’m passionate about solving challenging data problems and building reliable, scalable platforms that drive faster, smarter decision\-making in real\-world production environments\.

## Location

Bengaluru

## AI stack

antigravity

claude code

cursor

github copilot

langchain

langgraph

## Work experience

Senior Data Engineer at iCIMS, Apr 2026 \- Present · 8 mos, Designed and developed scalable data models and ETL pipelines for analytics and reporting across multiple business data sources\. Built and optimized SQL\-based data transformations in Amazon Redshift, integrating data from multiple source systems\. Developed Looker data models and dashboards to support business analytics and operational reporting\. Worked on data integration, data quality, and performance optimization, ensuring reliable and consistent datasets for downstream analytics\.

Sr Research and Development Engineer at Synopsys Inc, Feb 2024 \- Jan 2026 · 2 yrs, Bengaluru, Karnataka, India, \* Lead design and implementation of scalable data pipelines for high\-volume EDA metrics, enabling faster analytics and decision\-making for design implementation teams\. \* Build ML\-driven data platforms to optimize chip design processes, accelerating data\-driven decision\-making across R&D\. \* Spearhead development of PySpark workflows for data preprocessing, feature engineering, and statistical modeling, emphasizing high performance and reliability at scale\. \* Architect data models and orchestrate pipelines \(using Airflow\) to ensure platform scalability, resilience, and efficient ETL processes\. \* Collaborate with cross\-functional R&D teams to define data requirements and translate complex engineering challenges into robust, data\-centric solutions\.

Senior Data Engineer at Synopsys Inc, Jul 2022 \- Jan 2026 · 3 yrs 7 mos, Designed and delivered two variants of a Data Lakehouse using Medallion Architecture \(Bronze/Silver/Gold\) with Delta Lake — AWS \(S3, Glue, Lambda\) and on\-premises S3\-compatible MinIO; implemented ACID storage, schema enforcement, SCD Type 2 models, and time\-travel analytics\. Built a reusable metadata\-driven ETL/ELT framework supporting RDBMS, flat files, APIs, and Kafka; standardized transformation, validation, and reconciliation logic across 50\+ production\-grade PySpark pipelines, reducing pipeline development time by 40%\. Reduced pipeline processing time by 60% through PySpark optimization including broadcast joins, AQE, dynamic partitioning, memory tuning, and shuffle optimization\. Implemented Delta Lake schema evolution and SCD Type 2 tracking via PostgreSQL metastore with lineage, schema versioning, audit trails, and time\-travel queries\. Orchestrated batch ingestion, transformation, and data\-quality workflows using Apache Airflow DAGs with scheduling, dependency management, and monitoring\. Built Kafka and Spark Streaming pipelines enabling near\-real\-time PPA metric visibility alongside batch workflows\. Integrated AWS Lambda with S3 input events to automatically trigger downstream ingestion pipelines\. Implemented automated reconciliation, anomaly alerts, row\-level auditing, CI/CD, automated testing, and Docker/Kubernetes deployments\. Defined data platform architecture standards and mentored 4–5 junior engineers on PySpark, data modeling, and pipeline best practices\.

Research And Development Engineer II at Synopsys Inc, Jul 2022 \- Feb 2024 · 1 yr 8 mos, Bengaluru, Karnataka, India, Contributed to the development of DesignDash, the industry’s first ML\-driven analytics platform for design implementation\. Built and maintained end\-to\-end PySpark pipelines to ingest, transform, and analyze high\-volume EDA data\. Developed predictive models to identify and optimize design congestion, helping mitigate power, performance, and congestion issues early in the design cycle\. Designed and implemented graph\-based data workflows using Neo4j and ArangoDB to analyze downstream and upstream dependencies\. Wrote complex queries and analytics logic to uncover insights from highly connected datasets\. Improved data accessibility and reduced analysis turnaround time by enabling analytics\-ready datasets for downstream consumers\.

Software Engineer at CitiusTech, Sep 2019 \- Jul 2022 · 2 yrs 11 mos, Mumbai, Maharashtra, India, Worked as a Software Engineer in the Big Data domain, developing and maintaining scalable data pipelines using Python, Spark, Hive, and Hadoop\. Contributed to data processing workflows, data transformations, and performance optimization for large datasets in production environments\.

Big Data Developer at Citiustech HealthCare Technology, Sep 2019 \- Jul 2022 · 2 yrs 11 mos, Developed an end\-to\-end healthcare data integration platform using Apache NiFi and Apache Spark, ingesting structured and unstructured data from 50\+ payers and providers across HL7, FHIR, CSV, and XML formats; processed 100M\+ healthcare records monthly\. Built a reusable metadata\-driven ETL/ELT framework with pluggable data\-quality checks, standardization rules, and reconciliation logic\. Designed Spark SQL transformations for healthcare normalization using Star Schema, SCD Type 2 dimensions, and multi\-level aggregations for claims analytics\. Performed Spark performance tuning across memory, shuffle, executor, and partition configuration, improving pipeline SLA compliance from 75% to 98%\. Established Git\-based CI/CD workflows, automated testing, and Docker\-based containerized deployments\.

## Education

PG \- Diploma, Big \- Data Analytics, cdac

Bachelor of Technology, Electrical, Electronics and Communications Engineering, Guru Gobind Singh Indraprastha University \(GGSIPU\), Delhi

[View the original profile](https://tal.club/meet/tanmay-gupta-8)

```json
{
  "@context": "https://schema.org",
  "@graph": [
    {
      "@type": "Person",
      "@id": "https://tal.club/meet/tanmay-gupta-8#person",
      "name": "Tanmay gupta",
      "url": "https://tal.club/meet/tanmay-gupta-8",
      "image": "https://tal.club/meet/tanmay-gupta-8/profile-photo.jpg",
      "description": "Meet Tanmay gupta, Senior Data Engineer @ iCIMS (TL) | Scalable Data Pipelines | PySpark, Spark SQL, Airflow | Big Data & ML Systems | ex-synopsys based in Bengaluru. AI stack includes antigravity, claude code, and cursor.",
      "jobTitle": "Senior Data Engineer",
      "worksFor": {
        "@type": "Organization",
        "name": "iCIMS"
      },
      "hasOccupation": {
        "@type": "Occupation",
        "name": "Senior Data Engineer"
      },
      "homeLocation": {
        "@type": "Place",
        "name": "Bengaluru"
      },
      "alumniOf": [
        {
          "@type": "EducationalOrganization",
          "name": "cdac"
        },
        {
          "@type": "EducationalOrganization",
          "name": "Guru Gobind Singh Indraprastha University (GGSIPU), Delhi"
        }
      ],
      "knowsAbout": [
        "antigravity",
        "claude code",
        "cursor",
        "github copilot",
        "langchain",
        "langgraph"
      ],
      "mainEntityOfPage": {
        "@id": "https://tal.club/meet/tanmay-gupta-8#webpage"
      }
    },
    {
      "@type": "ProfilePage",
      "@id": "https://tal.club/meet/tanmay-gupta-8#webpage",
      "url": "https://tal.club/meet/tanmay-gupta-8",
      "name": "Tanmay gupta on Tal",
      "description": "Meet Tanmay gupta, Senior Data Engineer @ iCIMS (TL) | Scalable Data Pipelines | PySpark, Spark SQL, Airflow | Big Data & ML Systems | ex-synopsys based in Bengaluru. AI stack includes antigravity, claude code, and cursor.",
      "dateModified": "2026-11-13T11:40:30.735Z",
      "isPartOf": {
        "@id": "https://tal.club/#website"
      },
      "mainEntity": {
        "@id": "https://tal.club/meet/tanmay-gupta-8#person"
      },
      "primaryImageOfPage": {
        "@type": "ImageObject",
        "url": "https://tal.club/meet/tanmay-gupta-8/profile-photo.jpg",
        "contentUrl": "https://tal.club/meet/tanmay-gupta-8/profile-photo.jpg",
        "caption": "Tanmay gupta profile photo on Tal"
      },
      "inLanguage": "en-IN"
    }
  ]
}
```