Headline

Senior Data Engineer @ Visa | Big Data | Scala | Apache Spark | Kafka | GCP | Apache Iceberg

About

Big Data Engineer with 5+ years of experience building scalable data platforms using Scala, Apache Spark, Spark SQL, Hive, Apache Iceberg, Kafka, and GCP. Currently working as a Senior Data Engineer at Visa, delivering high-performance data solutions for enterprise-scale analytics and reporting. Highlights: - Optimized Spark applications, reducing processing time by 81% and increasing throughput by 12.5x. - Led migration of analytics tables from Hive to Apache Iceberg, improving scalability, governance, and maintainability. - Built scalable ETL pipelines and enterprise reporting solutions for large-scale data platforms. - Automated CI/CD workflows to streamline deployments and improve engineering efficiency. - Passionate about distributed systems, performance optimization, and building reliable, scalable data architectures. Always eager to learn emerging technologies, solve complex data engineering challenges, and contribute to impactful, data-driven solutions.

Location

Bengaluru

AI stack

claudecode

Work experience

Senior Data Engineer at Visa, Feb 2025 - Present · 1 yr 7 mos, Bangalore Urban, Senior Data Engineer at Visa working on large-scale data platforms using Spark and Scala. Designed and implemented campaign resubscription workflows with secure data handling, staged validations, and Hive/Iceberg integrations. Led migration of analytics workloads from Hive to Iceberg, enabling ACID-compliant operations, schema evolution, and improved data governance. Built scalable reporting frameworks and automated pipelines to generate client-ready engagement metrics with efficient data processing and delivery.

Software Engineer - III at Walmart Global Tech India, Jul 2023 - Feb 2025 · 1 yr 8 mos, Bengaluru, Karnataka, India

Software Engineer -II at Walmart Global Tech India, Jul 2021 - Jul 2023 · 2 yrs 1 mo

Research And Development Intern at samsung R&D, Jan 2021 - May 2021 · 5 mos, Noida, Uttar Pradesh, India, Developed and tested Proof-of-Concept (PoC) solutions leveraging data and AI at Samsung R&D. Deep Sleep Detection with Smartwatch Integration: brainstormed and developed a PoC for deep sleep detection using a Samsung smartwatch. The solution analyzed audio data (snoring vs. non-snoring) to create a spectrogram and utilized a Convolutional Neural Network (CNN) for sleep classification. Done to enhance existing sleep tracking features in Samsung wearables. Food Recommendation System based on User Data: designed and prototyped a food recommendation system that utilizes user data collected through the Samsung Health app. This PoC aimed to personalize food suggestions based on individual health goals and dietary preferences.

Artificial Intelligence Intern at TheSmartBridge, May 2020 - Jun 2020 · 2 mos, Jaipur, Rajasthan, India, Developed a Virtual Assistant using IBM Watson Cloud Services In this internship at TheSmartBridge, I played a key role in building a virtual assistant to reduce reliance on human interaction. Here's what I accomplished: Leveraged IBM Watson Assistant: Designed and implemented the virtual assistant using IBM Watson Assistant, enabling natural language interactions between users and the system. Integrated IBM Discovery Services: Integrated IBM Discovery Services to empower the virtual assistant with access to a vast knowledge base, allowing it to provide accurate and insightful responses to user queries. Reduced Human Interference : Streamlined processes by enabling the virtual assistant to handle routine tasks and answer basic questions, freeing up human resources for more complex inquiries.

Education

Electronics and Computer engineering, Computer Science, Thapar Institute of Engineering & Technology

PCM, 12th, Jayshree Periwal High School - Jaipur