Associate Engineer @KPIT | PySpark | Azure | SQL

Bengaluru, Karnataka, India

Bengaluru, Karnataka, India

Remote

Bachelor of Technology - BTech, Electronics and Communications Engineering
samrat ashok technological institute
12th, PCM
Ram Krishna Vidhya Mandir
Join Ankit and 10k+ other professionals on
Tal is the app where bosses meet engineers & builders directly, without middlemen
Download Tal BossAssociate Engineer @KPIT | PySpark | Azure | SQL
Data professional with 2+ year of experience building and optimizing data pipelines for reliable reporting and downstream use cases. In my current role, I design and implement ETL/ELT workflows using Python and SQL, develop robust data validation frameworks, and optimize processing latency across ingestion, transformation, and validation stages. I work extensively with Spark and PySpark for large-scale data processing and leverage Azure cloud services to build scalable and production-grade data pipelines. I further expanded into workflow orchestration using Apache Airflow for managing task dependencies and scheduling, and Docker for containerizing pipeline components to ensure consistent deployments across environments. Tech stack: Python, SQL, PySpark, ETL/ELT Pipeline, ADF, ADLS, Azure Synapse, Databricks, AWS S3, Apache Airflow, Docker(Basics), JavaScript, C++,REST APIs, Gi
Bengaluru
Associate Engineer at KPIT, Aug 2024 - Present · 1 yr 9 mos, Bengaluru, Karnataka, India, - Optimized data ingestion and storage using Azure Data Lake and PySpark, increasing team productivity by 60% while reducing data processing time by 50% and improving data accuracy. - Deployed and managed data pipeline infrastructure on Azure, enabling scalable compute and storage for workloads processing. - Designed and maintained scalable data models to support analytical reporting, reducing average query execution time by ~40%. - Implemented data validation checks to improve the data redundancy and data processing to near ~80%. - Developed Python Automation executable scripts to automate the manual process to increase the teams productivity by 80%. - Worked with tools such as Databricks for data processing and transformations.
Trainee at KPIT, Nov 2023 - Jul 2024 · 9 mos, Bengaluru, Karnataka, India
Web Development Intern at Exposys Data Labs, Jul 2022 - Aug 2022 · 2 mos, Remote, During the internship period (July 2022 - August 2022), I had the following responsibilities: - Create user-friendly web pages using ReactJs and TailwindCss
Bachelor of Technology - BTech, Electronics and Communications Engineering, samrat ashok technological institute
12th, PCM, Ram Krishna Vidhya Mandir
github: https://github.com/ankitbisen12