Headline

Data Engineer | PySpark, Azure, AWS, SQL| Building scalable data pipelines and AI solutions

About

Currently contributing as a Data Engineer at Thorogood, focusing on designing and executing data pipelines using Spark / PySpark and Azure Data Factory / Synapse . Works closely with global clients and business teams to translate requirements into scalable technical solutions that optimize data workflows. Holds an M.Sc Integrated degree in Information Technology from the College of Engineering, Guindy. Proficient in data engineering, data architecture, and exploring new concepts. Certified in Azure Fundamentals and Azure Data Fundamentals, showcasing a commitment to continuous learning and professional development.

Location

Bengaluru

AI stack

claude

gemini

openai

perplexity

Work experience

Data Engineer at Thorogood, Dec 2022 - Present · 3 yrs 10 mos, Bengaluru, Karnataka, India, • Collaborated directly with global clients and business teams to gather requirements, propose solutions, and optimize data workflows, translating business needs into scalable, technical implementations. • Designed and implemented end-to-end data pipelines using PySpark and Azure Data Factory (ADF) to automate ETL processes, enabling seamless integration of structured and unstructured data sources into cloud data lakes and warehouses. • Built and optimized Synapse Pipelines and leveraged Serverless Synapse SQL to process and query large-scale datasets with high performance, significantly reducing manual effort and query execution time. • Explored and applied AWS cloud services such as EMR, IAM, S3, and Lambda through internal POCs and learning projects, strengthening cross-cloud capabilities and expanding solution architecture expertise. • Created a production-ready static web application integrated with AWS Bedrock for GenAI interaction, Lambda for backend logic, S3 for static hosting, and Transcribe for real-time audio processing deployed using AWS Amplify. • Adopted DevOps best practices by managing GitLab CI/CD pipelines, version-controlling codebases, and implementing automated testing processes to maintain code quality and accelerate delivery cycles.

Business Intelligence and Analytics Consultant at Thorogood, Jun 2021 - Jan 2022 · 8 mos, Bengaluru, Karnataka, India

Education

M.Sc Integrated, Information Technology, College of Engineering, Guindy

HSC, Mathematics and Computer Science, DAV Matric. Hr. Sec. School