Data Engineer | Pyspark | AWS | Python | SQL


Hyderabad, Telangana, India

Hyderabad, Telangana, India


Hyderabad, Telangana, India


Mysore, Karnataka, India
Greater Hyderabad Area




Bachelor of Technology, Information Technology
Hindustan University

Science, Computer Science
Delhi public school
Join Muskan and 10k+ other professionals on
Tal is the app where bosses meet engineers & builders directly, without middlemen
Download Tal BossData Engineer | Pyspark | AWS | Python | SQL
Experienced Big Data Developer with a demonstrated history of working in the information technology and services industry. Skilled in Python (Programming Language),SQL, PySpark, Databricks and AWS.
Greater Hyderabad Area
claude
github copilot
openai
Senior Software Engineer at EPAM Systems, Mar 2026 - Present · 8 mos, Developed and maintained scalable ETL pipelines for a mainframe-to-AWS cloud migration project using AWS Glue, PySpark, Amazon S3, and AWS Step Functions. Supported the migration from the legacy mainframe environment to AWS by validating cloud-generated outputs and resolving data discrepancies identified during testing. Designed Spark-based ETL jobs to ingest data from Amazon S3, perform business transformations using PySpark, and generate processed files for downstream pipelines. Optimized Spark ETL jobs using techniques such as Adaptive Query Execution (AQE), skew join handling, predicate pushdown, and dynamic resource allocation to improve processing performance. Implemented robust logging, exception handling, and monitoring to improve the reliability and maintainability of production ETL pipelines. Worked with reusable utilities for file handling, validation, and logging to support consistent processing across ETL workflows.
Software Engineer II at EPAM Systems, Jun 2024 - Present · 2 yrs 5 mos, Hyderabad, Telangana, India
Data Engineer at Tata Consultancy Services, Jul 2022 - Jun 2024 · 2 yrs, Hyderabad, Telangana, India
Software Engineer at Tata Consultancy Services, Jul 2022 - Jun 2024 · 2 yrs, Improved overall pipeline performance by ˜25–30% using Spark optimizations such as broadcast joins, repartition/coalesce, caching, predicate pushdown, and shuffle tuning. Processed large volumes of fixed-width, CSV, Parquet, and text files from Amazon S3 using PySpark DateTime functions, complex joins, and window operations. Delivered reusable ETL enhancements across multiple user stories by building PySpark logic and validation rules, reducing manual effort. Designed and executed end-to-end ingestion pipelines writing transformed data into Aurora DB and producing downstream-ready datasets. Automated configuration handling using Python + boto3 for complex JSON structures (Struct Array types), reducing manual updates by 40%. Built modular ETL workflows using AWS Glue Jobs, Lambda, and Step Functions, improving team productivity and deployment efficiency.
Senior System Engineer at Infosys, Nov 2021 - Jul 2022 · 9 mos, Hyderabad, Telangana, India
Software Engineer at Infosys, Nov 2019 - 2022 · 2 yrs 3 mos, Maintained 99.8% uptime for data pipelines ingesting transactional data using PySpark and Databricks. Processed data from SQL and API sources to build analytical views for BI tools such as Power BI. Performed Spark optimization and developed efficient PySpark DataFrames, Spark SQL, and Databricks workflows. Designed and maintained UI components using React, Angular, and JavaScript.
System Engineer at Infosys, Nov 2019 - Oct 2021 · 2 yrs, Mysore, Karnataka, India
Software Engineer Intern at Customer Experience Labs, Aug 2019 - Oct 2019 · 3 mos, Greater Hyderabad Area, Worked on genesys platform
Bachelor of Technology, Information Technology, Hindustan University
Science, Computer Science, Delhi public school