Headline

Data Engineer III @ American Express | Building Scalable Data Pipelines & Driving Insights with Spark, Kafka, GCP & AWS | ETL | Big Data | ML Ops

About

At American Express, the team optimizes big data applications by enhancing non-functional requirements, consolidating Java services into streamlined Python solutions, and automating DevOps support activities with Python-based tools. Collaboration with business and scrum teams has driven operational efficiency and improved service maintainability. A Bachelor of Technology graduate from Guru Gobind Singh Indraprastha University, the focus has been on leveraging skills in large-scale data processing, operational efficiency, and team management. With a strategic approach to runtime engineering, contributions include building resilient systems and advancing automation to align with organizational goals.

Location

Delhi

AI stack

antigravity

opencode

Work experience

Data Engineer III at American Express, Oct 2023 - Present · 2 yrs 11 mos, Gurugram, Haryana, India · Hybrid, • Partnered with business and scrum teams to enhance the resilience and efficiency of big data applications by optimizing non-functional requirements. • Consolidated four Java services into a single Python service using FastAPI, enhancing code maintainability, teamwork, and reducing technical debt. • Enhanced production efficiency through the implementation of strategic runtime engineering, supported by comprehensive Root Cause Analyses and operational runbooks. • Led the development of a monthly reporting portal to streamline communication of key metrics and achievements with development teams and product owners. • Automated DevOps support activities using a Python-based SlackBOT, facilitating self-service capabilities across business, development, and SRE teams. • Containerized and deployed the SlackBOT on OpenShift using CI/CD pipelines to ensure system scalability and reliability. • Designed and deployed a REST API for the ServiceNow tool using Python FastAPI, improving accessibility and performance by hosting it on OpenShift.

Engineer II at American Express, Jun 2022 - Oct 2023 · 1 yr 5 mos, Gurugram, Haryana, India · Hybrid, • Designed and implemented automations with intelligent Slack BOT in python automating DevOps support activities by enabling self-service capabilities for Business, Dev and SRE members. • Dockerized the BOT and deployed it as container on an Openshift cloud using CI/CD. • Designed and implemented REST API interface for ServiceNow tool using Python FastAPI and deployed as container on cloud running on Openshift. • Developed tool for auto creation of incident using python by monitoring Big Data job statuses from BigData Job Orchestrator.

Data Engineer I at ZS, Feb 2021 - Jun 2022 · 1 yr 5 mos, New Delhi, • Managed the full development and configuration lifecycle of applications tailored for sales and marketing operations across a diverse client portfolio. • Devised solutions to integrate data from multiple sources, handling both structured and semi-structured data types. • Developed robust data pipelines to support advanced analytics and reporting, ensuring high data integrity and accuracy. • Implemented strict data standardization and quality checks, thereby ensuring superior data quality and reliability. • Applied business rules on staged data to create targeted datasets for in-depth business reporting.

Software Engineer at Amdocs, Aug 2019 - Feb 2021 · 1 yr 7 mos, Gurugram, • Acquired extensive expertise in cloud technologies (Azure, AWS), obtaining certifications and successfully applying Azure solutions for 3UK client applications. • Implemented SRE methodologies to automate repetitive tasks, reducing time and effort by over 80%. • Redesigned and enhanced .Net web/window-based applications using the MERN stack, focusing on quality and security. • Developed a web-based application from the ground up using MongoDB, Express, NodeJS, and ReactJS, integrating Google Fitness V1 API for additional functionality. • Consistently delivered high-quality software solutions for web and windows platforms, adhering to strict deadlines.

Education

Bachelor of Technology, Computer Engineering, Guru Gobind Singh Indraprastha University