Hemanth kumar N profile photo 1 on Tal

Hemanth kumar N

Data Engineer | Azure Databricks | PySpark | Building Scalable Data Pipelines | Ex-Microsoft Project (Infosys)

Bengaluru

3+ years of work experience

Infosys
Apr 20242 yrs 2 mosPresent

Bengaluru

Building Azure-native data platforms that power analytics for one of the world's largest tech companies. ■ Engineered 10+ ADF + Databricks ETL pipelines ingesting from 5+ sources into ADLS Gen2 — reduced data latency to reporting layer by 35% ■ Built PySpark transformation modules processing 50M+ records per run with zero SLA breaches over 12 months ■ Cut average query execution time by 40% through partition pruning and composite indexing on warehouse schemas ■ Reduced report generation from 8 min → under 2 min by rewriting 15+ stored procedures and eliminating full table scans ■ Achieved <5-min mean time to detect pipeline failures using structured Azure Monitor logging and alerting
Pacecom Technologies Pvt Ltd
Feb 202311 mosDec 2023

Bengaluru

Built the AWS data infrastructure that centralized and automated ingestion for structured business datasets. ■ Delivered 6 end-to-end AWS Glue ETL workflows, eliminating ~12 hours/week of manual analyst data prep ■ Standardized 8 disparate data sources into a unified schema using Python + SQL — cut ad-hoc data requests by 60% ■ Implemented automated data quality checks (nulls, schema validation, referential integrity) catching 95%+ of anomalies before reporting ■ Reduced nightly batch load times by 28% via Parquet formatting, snappy compression, and optimized Redshift COPY commands
Pacecom Technologies Pvt Ltd
Feb 202311 mosDec 2023

Built 6 end-to-end ADF-based ETL pipelines automating ingestion of structured datasets into ADLS Gen2-backed centralised repositories, replacing manual uploads and saving ~12 hours/week of analyst effort. Developed Python + SQL data processing jobs that standardised 8 disparate data sources into a unified schema, reducing ad-hoc data-prep requests to the engineering team by ~60%. Implemented automated data quality checks (null checks, referential integrity, schema validation) across ingestion pipelines, catching 95%+ of data anomalies before they reached reporting layers. Optimised ADF copy activities and ADLS Gen2 data layouts (columnar Parquet + partitioning), reducing average load times by 28% for daily batch jobs.

Hemanth’s AI Stack

gemini
gemini
github copilot
github copilot
ollama
ollama
openai
openai

Education

PES college of Engineering, Mandya

Bachelor of Engineering, Civil Engineering

PES college of Engineering, Mandya

Join Hemanth and 10k+ other professionals on

Tal Boss

Tal is the app where bosses meet engineers & builders directly, without middlemen

Download Tal Boss

Hemanth kumar N on Tal

Headline

Data Engineer | Azure Databricks | PySpark | Building Scalable Data Pipelines | Ex-Microsoft Project (Infosys)

About

I’m a Data Engineer with 3 years of experience designing and building scalable data pipelines on Azure. I specialize in Azure Data Factory, Databricks, and PySpark, working with large-scale datasets (50M+ records) to deliver reliable and optimized data solutions. In my current role, I work on Microsoft client projects where I design ETL pipelines, optimize SQL queries, and improve data platform performance by up to 40%. I’m passionate about solving data engineering challenges, improving pipeline efficiency, and building robust data systems. Skills: Azure Data Factory | Databricks | PySpark | SQL | ADLS Gen2 | Data Modeling

Location

Bengaluru

AI stack

gemini

github copilot

ollama

openai

Work experience

Data Engineer at Infosys, Apr 2024 - Present · 2 yrs 2 mos, Bengaluru, Building Azure-native data platforms that power analytics for one of the world's largest tech companies. ■ Engineered 10+ ADF + Databricks ETL pipelines ingesting from 5+ sources into ADLS Gen2 — reduced data latency to reporting layer by 35% ■ Built PySpark transformation modules processing 50M+ records per run with zero SLA breaches over 12 months ■ Cut average query execution time by 40% through partition pruning and composite indexing on warehouse schemas ■ Reduced report generation from 8 min → under 2 min by rewriting 15+ stored procedures and eliminating full table scans ■ Achieved <5-min mean time to detect pipeline failures using structured Azure Monitor logging and alerting

Traniee Engineer at Pacecom Technologies Pvt Ltd, Feb 2023 - Dec 2023 · 11 mos, Bengaluru, Built the AWS data infrastructure that centralized and automated ingestion for structured business datasets. ■ Delivered 6 end-to-end AWS Glue ETL workflows, eliminating ~12 hours/week of manual analyst data prep ■ Standardized 8 disparate data sources into a unified schema using Python + SQL — cut ad-hoc data requests by 60% ■ Implemented automated data quality checks (nulls, schema validation, referential integrity) catching 95%+ of anomalies before reporting ■ Reduced nightly batch load times by 28% via Parquet formatting, snappy compression, and optimized Redshift COPY commands

Data Engineer at Pacecom Technologies Pvt Ltd, Feb 2023 - Dec 2023 · 11 mos, Built 6 end-to-end ADF-based ETL pipelines automating ingestion of structured datasets into ADLS Gen2-backed centralised repositories, replacing manual uploads and saving ~12 hours/week of analyst effort. Developed Python + SQL data processing jobs that standardised 8 disparate data sources into a unified schema, reducing ad-hoc data-prep requests to the engineering team by ~60%. Implemented automated data quality checks (null checks, referential integrity, schema validation) across ingestion pipelines, catching 95%+ of data anomalies before they reached reporting layers. Optimised ADF copy activities and ADLS Gen2 data layouts (columnar Parquet + partitioning), reducing average load times by 28% for daily batch jobs.

Education

Bachelor of Engineering, Civil Engineering, PES college of Engineering, Mandya