Headline

Actively looking for Data Engineer & Analyst | Python, SQL, R | ETL & Database Management | Tableau & Power BI | Big Data Technologies | Agile Project Management

About

I am a Data Engineer with 6+ years of experience building scalable data platforms, ETL pipelines, and cloud based data solutions across Azure and AWS. My core expertise includes Python, SQL, PySpark, Apache Spark, Databricks, Azure Data Factory, AWS, Hadoop, Kafka, and data warehousing. I have worked extensively with large scale structured and semi structured data, batch and real time processing, API integrations, and data transformation. In my current role, I work on enterprise data engineering and AI initiatives, including RAG systems that enable business users to interact with organizational data using natural language. I have also built data reconciliation pipelines and LLM powered analytics solutions to improve data quality and make business insights more accessible. Throughout my career, I have focused on building reliable, automated, and high performance data pipelines while continuously improving processing efficiency and data quality. Key areas: Data Engineering | PySpark | Databricks | Python | SQL | Azure | AWS | Big Data | ETL/ELT | Data Warehousing | RAG | LLM | Data Analytics | Kafka | Cloud Data Platforms I enjoy solving complex data problems and building scalable solutions that create measurable business value.

Location

United States

AI stack

antigravity

claude code

claude

gemini

langgraph

langsmith

Work experience

Data engineer at Landmark Group, Dec 2024 - Present · 1 y 9 mo, Bengaluru, Karnataka, India

Data Engineer at Excellus BCBS, Jan 2024 - Dec 2024 · 1 yr, California, United States, • Utilized Python for advanced data analysis and visualization, developing scripts and algorithms with libraries like Matplotlib, Pandas, and NumPy to uncover insights, predict trends, and support data-driven decisions. Gained experience in creating interactive visuals and presenting complex analytical findings to stakeholders. • Implemented data ingestion and processing solutions using Big Data technologies, handling over 1.5 million data records monthly with technologies like Hadoop, MapReduce, HBase, Hive, and Sqoop. • Transformed over 90 complex Hive/SQL queries into Spark transformations monthly using Spark RDD, Scala, and Python, achieving a 25% increase in data processing efficiency. • Enhanced data infrastructure and pipelines, increasing data processing efficiency. Focused on optimizing data collection, storage, and processing for datasets. • Managed and processed data from Azure Data Lake Storage into Spark Data Frames monthly, performing vital data transformations and actions. • Demonstrated proficiency in Microsoft Azure Services, managing data across services like Azure VM, Azure Blob Storage, Azure SQL Database, Azure Synapse Analytics, Azure Data Factory, HDInsight, and Power BI. • Enhanced data processing with Hive by implementing partitioning and bucketing, generating data cubes monthly for advanced data visualization. • Worked with data engineering teams to improve data workflows, leading to an improvement in data quality and a reduction in data integration issues. • Regularly resolved data discrepancies for business users by optimizing complex SQL queries, ensuring data accuracy and reliability. • Developed Azure Functions to aggregate data from numerous event hubs, storing processed data in Azure Cosmos DB.

Student Assistant at California State University, Fullerton, Aug 2023 - Jun 2024 · 11 mos

Data Engineer at Capgemini, Sep 2019 - Aug 2022 · 3 yrs, India, • Revitalized a 10GB+ warehouse infrastructure, elevating automation to 60%. This overhaul minimized downtime, ensuring maximized data accessibility and operational continuity. • Utilized Spark’s in memory capabilities to handle large datasets on S3 Data Lake. Loaded data into S3 buckets, then filtered and loaded into Hive external tables. • Strong Hands on experience in creating and modifying SQL stored procedures, functions, views, indexes, and triggers. • Performed ETL operations using Python, Sparks, SQL , S3 and Redshift on terabytes of data to obtain customer insights • Involved heavily in setting up the CI/CD pipeline using Jenkins, Terraform and AWS • Performed end- to-end Architecture & implementation assessment of various AWS services like Amazon EMR, Redshift, S3 • Used AWS EMR to transform and move large amounts of data into and out of other AWS data stores and databases, such as Amazon Simple Storage Service (Amazon S3) and Amazon DynamoDB • Good Understanding of other AWS services like S3, EC2 IAM, RDS Experience with Orchestration and Data Pipeline like AWS Step functions/Data Pipeline/Glue. • Transformed the data using AWS Glue dynamic frames with PySpark; cataloged the transformed the data using Crawlers and scheduled the job and crawler using workflow feature • Designed and managed public/private cloud infrastructures using Confidential Web Services (AWS) which include EC2, S3, Cloud Front, Elastic File System, and IAM which allowed automated operations. • Leveraged SQL to query, extract, and manipulate large datasets, enabling in-depth analysis

Data Analyst at KPMG, Jan 2019 - Aug 2019 · 8 mos, India, • Leveraged R's integration with big data tools (like SparkR, RHIPE) for analyzing massive datasets, resulting in a 20% improvement in handling large-scale data and a 15% speed increase in data processing. • Expert in Python with a focus on Pandas, NumPy, and SciKit-learn for efficient data analysis. Achieved a 30% boost in data processing efficiency through optimized code. • Skilled in managing MySQL databases, including data storage, retrieval, and complex query formulation. Ensured data accuracy and integrity, resulting in a 20% improvement in database performance. • Experienced in creating dynamic and insightful visualizations using Power BI. Effectively enhanced data storytelling and analysis, leading to a 15% improvement in data presentation and understanding. • Managed extensive data read/write operations to CSV and Excel files using Pandas, handling about 100,000 records per operation. • Implemented Collibra for enhanced data governance and management, leading to a 20% improvement in data quality and a 15% increase in compliance with data standards. • Designed comprehensive Power BI reports and dashboards, analyzing datasets over 1TB, to facilitate strategic business decision-making.s

Education

Master's degree, Computer Science, California State University, Fullerton

Bachelor of Engineering - BE, Information Technology, Rajiv Gandhi Prodyogiki Vishwavidyalaya