Headline

Senior AI Engineer at Confidential

About

Data Architect and Engineer with proven expertise in designing and scaling enterprise-grade data platforms, modern data warehouses, and cloud analytics ecosystems that drive measurable business impact. Experienced in building high-performance data engineering and data architecture solutions across healthcare, retail, and enterprise domains using Microsoft Fabric, Azure, Databricks, Snowflake, dbt, PySpark, BigQuery, AWS, and modern cloud-native technologies. Strong background in: • Enterprise Data Architecture & Modern Data Platform Design • Data Warehousing & Lakehouse Architecture • ETL/ELT Pipeline Development & Optimization • Distributed Data Processing using PySpark & Databricks • Real-Time & Batch Data Processing & Streaming using AWS Kinesis & Firehose • Serverless Data Processing using AWS Lambda • Scalable Data Integration using AWS Glue • Data Modeling (Star/Snowflake Schema) • Master Data Management (MDM) • Medallion Architecture (Bronze, Silver, Gold Layers) • Slowly Changing Dimensions • Semantic Modeling & BI Enablement • Data Governance & Data Quality Optimization • CI/CD Automation & DevOps for Data Platforms • Cloud Data Engineering across Azure, AWS & GCP Played a key role in developing enterprise analytics and data platforms supporting business operations, contributing to $3B+ in revenue through scalable reporting systems and data-driven decision-making solutions. Proven track record of: • Designing scalable enterprise data warehouses and lakehouse solutions • Building reliable ingestion frameworks from APIs, GCP BigQuery, SQL Server, MySQL, SharePoint, AWS services, and cloud-native systems • Optimizing pipelines to reduce failures and improve processing efficiency • Implementing highly available production-grade data systems with 99.9% uptime • Automating deployments using Azure DevOps, Git, and CI/CD pipelines • Collaborating with Data Analysts, Data Scientists, and business teams to deliver actionable insights at scale Passionate about solving complex data challenges, building modern scalable architectures. Tech Stack: • Microsoft Fabric | Azure | Databricks | Snowflake | AWS | GCP • AWS Glue | AWS Lambda | AWS Kinesis | AWS Firehose • Python | PySpark | SQL | dbt | Airflow • BigQuery | SQL Server | MySQL | Synapse Analytics • Data Factory | Fivetran | Azure DevOps | Git | CI/CD • CluedIn | Profisee | MDM | Semantic Modeling Certifications: • Microsoft Certified: Azure Data Engineer Associate (DP-203) • Microsoft Certified: Fabric Analytics Engineer Associate (DP-600) Mail: devsumitdebnath@gmail.com Best, Sumit

Location

India

AI stack

anthropic apis

antigravity

claude code

claude

codex

gemini

Work experience

Senior AI Data Engineer at Confidential, Mar 2026 - Present · 7 mos

Senior Data Engineer II at Accordion, Jan 2025 - Mar 2026 · 1 yr 3 mos, Hybrid

Senior Data Engineer I at Accordion, Nov 2023 - Dec 2024 · 1 yr 2 mos, On-site, - Accumulated extensive expertise in data ingestion, modeling, and transformation, specializing in ETL and ELT processes. - Orchestrated the complete data ingestion process from various sources, constructing pipelines for comprehensive ingestion, data modeling, and implementing logic for data modification. - Played a pivotal role in daily connect meetings, contributing to project progress discussions, providing status updates, and establishing ETA for sprint items. - Fostered direct communication with clients, ensuring a deep understanding of project requirements and addressing concerns effectively. - Recognized with a spot award for exceptional dedication, consistently delivering high-quality work. - Applied proficiency in data modeling, module analysis, and solution development within Microsoft Fabric, SQL Server and Azure technologies for database solution development. - Spearheaded a 94% increase in the efficiency of an existing backend solution by transitioning from a Microsoft Flow-based process to a more streamlined SQL and Azure Data Factory pipeline. - Engineered data warehouse solutions, normalizing source data tables, implementing SCD type 2, and ensuring data availability for downstream users. - Acknowledged for achieving a substantial enhancement in the effectiveness of the existing backend solution, showcasing expertise in ETL and ELT processes. - Proficient in Python, contributing to CICD (Continuous Integration and Continuous Deployment) pipelines, further enhancing data warehousing practices. - Skills: problem-solving, communication, data modeling, dbt, SQL Server, Azure technologies, pipeline development, module analysis, solution development, Microsoft Fabric, Azure DevOps, DBMS, Microsoft SQL Server, Azure Data Factory, software development, GIT, and team leadership.

Data Engineer at Deltatech Gaming Limited, Nov 2022 - Nov 2023 · 1 y, Gurugram, Haryana, India, • Collaborated with stakeholders to design data solutions, contributing to a 15% increase in data-driven strategies. Executed SQL queries to extract, transform, and analyze data from various data warehouses i.e., SnowFlake. MongoDB, PostgreSQL, BigQuery. • Successfully designed and executed user acquisition campaigns targeting specific user cohorts, with continuous performance monitoring. Achieved a noteworthy 12% improvement in user acquisition rate. • Engineered in-app promotions aimed at boosting the participation of high cash rollers, enhancing customer retention, and increasing player engagement. Improved Player participation by 15% and Retention by 20%. • Developed ETL processes, cutting data processing time by 25% and reducing operational costs. Also built scalable data pipelines, resulting in a 20% increase in real-time data processing efficiency. Utilized Pyspark & Python for data manipulation • Employed Apache Airflow for workflow orchestration, scheduling pipelines, and automating email reports. decreasing errors by 25%. Managed large-scale data on AWS S3, Athena, and Amazon Relational Database Service (RDS), ensuring cost-efficient processing. Task automation using Bash Scripting. • Created interactive dashboards in Holistics & Looker, driving a 15% increase in informed decisions. • Being the SPOC (Single Point of Contact) for Marketing, Product, Compliance, CRM, and Acquisition Successfully managed and coordinated diverse functions to ensure smooth operations and organizational alignment.

Team Leader at Youth India Foundation, May 2017 - Aug 2017 · 4 mos, Pune Division, Maharashtra, India, • Worked with India’s fastest-growing E-Education company under the internal design branch team. • Analysed my team, let them use their strengths, provide clear direction and establish a clear schedule. • Structured user flows, wireframes, prototypes, and mockups using Adobe XD and Figma. • Designed UI elements such as complete page design, input controls, and informational components. • Worked closely with the leadership team to devise data-driven strategies and improve process efficiency.

Freelance at Manasvi Counselling Services, May 2017 - May 2017 · 1 mo, Pune Division, Maharashtra, India, Here in Manasvi Counselling Services I have designed all the page from scratch. and redesign on all the elements and give it a very beautiful look unacceptable format that my client appreciate it very much.

Frontend Developer at Youth India Foundation, Jan 2017 - Apr 2017 · 4 mos, Pune Division, Maharashtra, India, • Developed new user-facing features using React.js • Built reusable components and front-end libraries for the future using Material UI. • Optimized components for maximum performance across a vast array of web-capable devices and browsers. • Executed a flexible and well-structured architecture, along with the APIs to support it. • Work and collaborate with the rest of the engineering team.

Education

Bachelor of Technology - BTech, Information Technology, J.C. Bose University of Science and Technology, YMCA

Public links

github: https://github.com/sumitdnath