Headline

Data Engineer at LTIMindtree

About

Data Engineer with 3 years of experience in designing, developing, and optimizing scalable data pipelines and ETL workflows using PySpark and SQL. Experienced in processing large datasets, implementing transformation logic, optimizing queries, and building reliable data solutions to support analytics and business reporting requirements. Hands-on experience with Databricks, Python, Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), and cloud-based data ecosystems for data ingestion, transformation, orchestration, and pipeline development. Worked on developing batch processing workflows, improving data quality, and enhancing overall pipeline efficiency and maintainability. Contributed to performance optimization initiatives by improving data processing workflows and SQL execution efficiency, resulting in reduced execution time and improved processing throughput. Involved in building scalable ETL frameworks, handling high-volume datasets, automating data workflows, and supporting downstream reporting and analytics use cases. Possess foundational exposure to Generative AI concepts, including basic understanding of LLMs and LangChain with practical experimentation and prototyping experience. Focused on delivering scalable, efficient, and business-driven data solutions while continuously expanding expertise in big data and emerging technologies.

Location

Pune Division

Work experience

Data Engineer at LTIMindtree, Dec 2023 - Present · 2 yrs 7 mos, Nagpur, Maharashtra, India, • Designed and built scalable Azure data pipelines using Azure Data Factory (ADF) and Azure Data Lake Storage (ADLS) following the Medallion Architecture (Bronze–Silver–Gold). • Improved batch pipeline efficiency and data processing performance by ~40% through optimized orchestration, incremental loads, and Spark-based transformations. • Developed robust batch transformation logic using PySpark on Databricks, enabling reliable and scalable data processing for analytical workloads. • Worked in the Center of Excellence (CoE) to develop an internal code migration and modernization framework to convert legacy analytics workloads into PySpark-based implementations. • Built a Databricks-based migration tool that: Parsed and converted legacy code into PySpark • Stored converted code, execution chunks, and lineage in Unity Catalog volumes Maintained metadata and governance using Databricks Unity Catalog • Implemented a metadata-driven architecture by storing pipeline and code metadata in catalog tables and embeddings in a vector store. • Enabled cross-code and metadata querying using Retrieval-Augmented Generation (RAG), allowing intelligent search and contextual analysis across migrated workloads. • Collaborated with CoE teams to ensure scalability, governance, and reusability of data engineering and migration solutions.

Graduate Engineering Trainee at LTIMindtree, Sep 2023 - Dec 2023 · 4 mos, Navi Mumbai, Maharashtra, India, Graduate Engineer Trainee – Azure Data Engineering (Batch Processing) • Completed intensive training as a Graduate Engineer Trainee specializing in Azure batch data processing pipelines. • Designed and executed batch workflows using Azure Data Factory, Azure Data Lake, and PySpark on Spark. • Processed large datasets using distributed batch processing techniques and optimized transformations for performance. • Used SQL for data validation, aggregation, and analytical reporting. • Applied cloud data engineering principles including scalable storage, orchestration, and fault-tolerant batch execution.

Intern at LTIMindtree, May 2023 - Jul 2023 · 3 mos, India, Ignite Program, LTIMindtree • Successfully completed enterprise-level Ignite training at LTIMindtree with a focus on batch data processing pipelines. • Developed and executed batch jobs using Python, SQL, Apache Spark, and PySpark. • Processed large-scale datasets using Spark DataFrames and PySpark transformations and actions. • Designed SQL-based data validation and aggregation logic to support batch analytics. •Applied distributed processing concepts to optimize performance and handle high-volume data.

Education

Bachelor of Engineering - BE, Mechanical Engineering, Sinhgad college of engineering

HSC, Science, Horizon English Medium School And Junior College