Headline

Data Engineer | 10+ years of Experience | Building Scalable Data Platforms | Azure | Databricks | Python

About

• Result-oriented Azure Data Engineer with expertise in big data platforms such as Azure Databricks, Azure Stream Analytics, Azure Synapse Analytics, and Apache Spark. • Proficient in designing and implementing end-to-end data pipelines using Azure Data Factory, transforming raw data into actionable insights through advanced data wrangling techniques. • Known for delivering high-quality solutions efficiently, with strong problem-solving skills and a collaborative mindset. • Adaptable and skilled at leveraging cutting-edge technologies to develop innovative, scalable data solutions across diverse environments

Location

Bhubaneswar

Work experience

Data Engineer at Accenture, Sep 2023 - Present · 2 yrs 11 mos, Bhubaneswar, Odisha, India, 1. Built an AI agent using agentic AI libraries like Langchain, ChatDatabricks that finds the root cause behind data transformation failure and reducing debugging efforts by 50%. 2. Built an automated monitoring and reporting notebook that consolidated ingestion, data quality, and transformation statuses across REST API feeds, delivering real-time pipeline health summaries via Logic App notifications. 3. Automated REST API ingestion through a metadata-driven framework with integrated post-load quality checks using Databricks DQX framework and audit-based alerting, ensuring end-to-end data accuracy and visibility while reducing manual monitoring effort by ~80%. 4. Built a metadata-driven Databricks transformation framework that merged bronze data through multi-stage Spark SQL joins and columnar transformations, automating the end-to-end flow into a unified silver table and improving processing efficiency. 5. Orchestrate and automate azure data factory pipelines to extract documents from Veeva vault. 6. Design notebooks using pyspark to make API call for document extraction, apply file naming convention to files in staging server and copy them into data lake storage gen2. 7. Follow medallion architecture to load data from API source into end tables and create data products for downstream consumers. 8. Validate document metadata entries and create XML files to get it processed by proprietary system for archive. Finally copy the files from vault system to azure fileshare. 9. Built an automation script to grant fine-grained access to Databricks groups and setup data governance across Databricks assets in all the environments.

Packaged App Development Specialist at Accenture, Sep 2023 - Present · 2 yrs 11 mos, India

Senior Data Engineer at IBM, Sep 2022 - Sep 2023 · 1 yr 1 mo, Bengaluru, Karnataka, India, • Enhanced data quality from diverse source systems by implementing a common semantic model prior to ingestion. • Utilized CI/CD tools for efficient testing, building, and deployment of Python modules into applications. • Developed comprehensive data mappings and filtered unstructured data for seamless integration into target systems.

Information Technology Analyst at Tata Consultancy Services, Feb 2019 - Aug 2022 · 3 yrs 7 mos, Optimized Informatica data pipelines to load data from text files into the target Oracle tables, resulting in 50% increased data load efficiency. Directed a large-scale data migration from on-premises to cloud, completing within 3 months with 100% accuracy through robust validation and reconciliation processes.

IT Analyst at Tata Consultancy Services, Jan 2019 - Aug 2022 · 3 yrs 8 mos, Bhubaneshwar, Odisha, India, Worked as a Quality Analyst for a Canadian bank client in corporate centre of technology domain. 1. Developed a script to perform structure, count and data validation between Oracle source tables and hive target tables before launching them in product environment. 2. Created an automation script to validate structure and count between hive source and impala target tables. 3. Developed an automation script to validate data between source and target table. I have designed automation scripts based on customer requirements to meet business logics(like view name validation as per business logic, table names versus mapping document validation). 4. Created an automation script to validate data between source table versus target view with and without PII columns among Impala tables.

System Engineer at Tata Consultancy Services, Jun 2019 - Mar 2020 · 10 mos, Bhubaneswar, Odisha, India, Worked as a middleware application developer in wholesale banking for a Canadian bank. 1. Responsible for supporting wholesale banking application and generating sales trading reports. 2. Receive development requirements from client and write PL/SQL stored procedures for meeting business requirements. 3. Conducted various testing activities like unit testing, system integration testing to make any change go live to production. I had performed an automation activity to validate count and data between 2 different oracle databases. 4. Created mappings and workflows to incrementally load data into target oracle tables using ETL tool Informatica.

Senior Associate at WIPRO TECHNOLOGIES, Aug 2015 - Jan 2019 · 3 yrs 6 mos, Bhubaneswar, Odisha, India, 1. Played a key role in 60+ incident resolution and support activities, ensuring system stability and minimizing downtime. 2. To allocate $7.8 billion across all COBs for an international oil company through cost allocation tool every month. 3. I am responsible for troubleshooting errors/developing codes for client applications using T-SQL language. 4. I take the ownership for deploying code changes to add enhancement to client applications that requires T-SQL and Unix Shell scripting. 5. Demonstrated leadership in addressing and communicating critical issues through leading On-Time Delivery (OTD) and incident status calls, specifically for Real-Time Visibility and Analysis (RTVA).

Education

Bachelor of Technology (B.Tech.), CSE, KIIT - Kalinga Institute of Industrial Technology

10+2 Intermediate, ODM Public School