Data Engineer at IBM | Azure | Databricks | PySpark | SQL | FMCG | ETL & Data Pipelines | Data Lakehouse & Governance

Mumbai, Maharashtra, India

Bachelor of Technology - BTech, Computer Science
Vellore Institute of Technology
Join Abhishek and 10k+ other professionals on
Tal is the app where bosses meet engineers & builders directly, without middlemen
Download Tal BossData Engineer at IBM | Azure | Databricks | PySpark | SQL | FMCG | ETL & Data Pipelines | Data Lakehouse & Governance
I am a Data Engineer at IBM with a Databricks certification and hands-on experience in Azure Services. I have developed a solid foundation in Python, SQL, Spark and other Data Engineering skills. I am passionate about continuously enhancing my technical skills and keeping abreast of industry advancements in the field. I am eager to explore opportunities that will further my professional growth and allow me to tackle challenging projects in Data Engineering. Programming Languages:Python, SQL, PySpark, Java Data Warehouse Platforms and Databases: Databricks, Azure SQL Server, SAP GUI Data Processing and Orchestration: Databricks Workflows, Spark Cloud Platforms: Azure, AWS, GCP Version Control and Tools: GitHub, Git, Azure DevOps
Mumbai
Data Engineer at IBM, Jul 2023 - Present · 2 yrs 10 mos, Mumbai, Maharashtra, India, Data Engineer – FMCG Client Project: Data Lakehouse & Supply Chain Optimization - Developed a scalable data lakehouse on Microsoft Azure, improving data processing speed by 75% and reducing storage costs by 40%, handling 1M+ records daily. - Built and optimized 15+ data pipelines using Medallion Architecture, ensuring data integrity and efficient transformations. - Designed 10+ views for real-time KPI tracking in Power BI, leading to better inventory management and 15% fewer stock-outs. - Automated workflows with custom scripts, reducing processing time by 80% and increasing efficiency. - Integrated Databricks with SAP ECC & SAP BTP, enabling seamless data transfer and stakeholder collaboration. - Partnered with business analysts and clients to translate business needs into scalable data solutions. - Implemented data governance and security measures, improving data compliance by 30%. Data Engineer – ETL Migration Project Project: IBM DataStage to PySpark Migration - Migrated an ETL asset from IBM DataStage to PySpark, boosting processing speed and scalability by 40%, leading to better client adoption. - Conducted rigorous validation and testing of PySpark code to ensure data accuracy and consistency. - Created technical documentation for legacy processes, improving developer productivity by 20%.
Bachelor of Technology - BTech, Computer Science, Vellore Institute of Technology