Shivam Chowdhary
Data & AI Engineer | AWS • Databricks • GenAI
Gurugram
6+ years of work experience
Nov 20232 yrs 7 mosPresent
Gurugram, Haryana, India
Managed and optimized Databricks pipelines for processing large-scale
structured and unstructured datasets utilizing Spark, Python, and R.
Engineered Delta Lake architectures to enhance query performance, data
reliability, and overall data quality.
Automated data ingestion, validation, and workflow orchestration employing
Databricks Jobs, notebooks, and REST APIs.
Fine-tuned Spark configurations and cluster resources to minimize compute
costs while concurrently increasing processing throughput.
Developed cloud-native services on AWS (Lambda, DynamoDB,
OpenSearch) with integrated monitoring through Grafana and CloudWatch.
Sep 20233 mosNov 2023
Bengaluru, Karnataka, India
Jul 20233 mosSep 2023
South Delhi, Delhi, India
Jan 20221 yr 5 mosMay 2023
Remote
Cluster firefighter 🧑🚒
Containerising web-services for visibility of messages across kafka pipelines
GraphQL APIs
Oct 20214 mosJan 2022
Data Science on ELK for DAU > 5M
Dashboards’ Gatekeeper
Oct 20192 yrsSep 2021
Dubai, United Arab Emirates
Automated Reports and pipelines
Education
PG Diploma, Data science
International Institute of Information Technology Bangalore
20171 yr2018
Bachelor of Technology (B.Tech.), Electrical, Electronics and Communications Engineering
SRM IST Chennai
20124 yrs2016
Join Shivam and 10k+ other professionals on
Tal is the app where bosses meet engineers & builders directly, without middlemen
Download Tal Boss