Somchandra Pandey profile photo 1 on Tal

Somchandra Pandey

Microsoft & Databricks Certified Data Engineer

Hyderabad

6+ years of work experience

Accordion India
Jul 20242 yrs 2 mosPresent

Hyderabad

• Designed and implemented scalable ETL pipelines using Azure Data Factory and Databricks (PySpark, Spark SQL) to ingest data from 7+ Practice Management Systems (PMS) and ActiveCampaign CRM into Azure SQL Data Warehouse and Snowflake. • Built and standardized Common Data Models (CDM) for 3 PMS systems using SQL, enabling consistent schema alignment and seamless downstream analytics and CRM integrations. • Developed data cleansing and transformation workflows using PySpark, SQL, and Azure Logic Apps, removing irrelevant contact and deal records from ActiveCampaign CRM and improving data quality and usability by 60%. • Implemented robust exception handling, validation checks, and data quality rules across ADF pipelines, Databricks jobs, and SQL stored procedures to ensure accurate Patient Status, Appointment Status, and Referral mappings. Performance Optimization, Reliability, and Observability • Applied SRE-aligned best practices by optimizing SQL stored procedures and PySpark transformations, improving pipeline reliability, observability, and execution performance while reducing manual intervention and lowering client costs by $50000. • Identified and resolved performance bottlenecks in distributed data processing jobs and relational workloads, improving end to-end pipeline efficiency and scalability. • Enabled CI/CD for data pipelines using Azure DevOps supporting controlled deployments. • Worked with Apache Airflow for workflow orchestration concepts and Apache Kafka for event-driven ingestion patterns and producer–consumer fundamentals.
SAE IITR
Mar 20213 yrs 3 mosMay 2024

Team KNOx IIT Roorkee
Mar 20203 yrs 5 mosJul 2023

GitHub
@pandeysom921

Somchandra’s AI Stack

codex
codex
opencode
opencode
claudecode
claudecode

Education

Indian Institute of Technology, Roorkee

Mechanical engineering, Mechanical Engineering

Indian Institute of Technology, Roorkee

20204 yrs2024

Join Somchandra and 10k+ other professionals on

Tal Boss

Tal is the app where bosses meet engineers & builders directly, without middlemen

Download Tal Boss

Somchandra Pandey on Tal

Headline

Microsoft & Databricks Certified Data Engineer

About

Data Engineer 2 at Accordion India with hands-on experience in building scalable data solutions using Azure and Databricks. I specialize in Big Data processing, ETL pipeline development, and data governance, enabling reliable and high-performance data systems for business decision-making. In my current role, I design and optimize data pipelines, streamline data workflows, and ensure data quality across platforms using tools like Azure Data Factory, PySpark, and SQL. I actively contribute to improving system efficiency and delivering robust, production-ready data solutions. I have earned certifications in Azure Data Fundamentals and Databricks Associate. I continuously explore advancements in Generative AI, LLMs, and emerging Agentic AI concepts, aiming to integrate intelligent capabilities into modern data engineering workflows. I also have experience in data analysis & visualization tools like PowerBI, Tableau & Advanced Excel. I’m particularly interested in opportunities at the intersection of Data Engineering, AI, and Analytics, where I can build scalable, future-ready data systems and drive data-driven innovation.

Location

Hyderabad

AI stack

codex

opencode

claudecode

Work experience

Data Engineer 2 at Accordion India, Jul 2024 - Present · 2 yrs 2 mos, Hyderabad, • Designed and implemented scalable ETL pipelines using Azure Data Factory and Databricks (PySpark, Spark SQL) to ingest data from 7+ Practice Management Systems (PMS) and ActiveCampaign CRM into Azure SQL Data Warehouse and Snowflake. • Built and standardized Common Data Models (CDM) for 3 PMS systems using SQL, enabling consistent schema alignment and seamless downstream analytics and CRM integrations. • Developed data cleansing and transformation workflows using PySpark, SQL, and Azure Logic Apps, removing irrelevant contact and deal records from ActiveCampaign CRM and improving data quality and usability by 60%. • Implemented robust exception handling, validation checks, and data quality rules across ADF pipelines, Databricks jobs, and SQL stored procedures to ensure accurate Patient Status, Appointment Status, and Referral mappings. Performance Optimization, Reliability, and Observability • Applied SRE-aligned best practices by optimizing SQL stored procedures and PySpark transformations, improving pipeline reliability, observability, and execution performance while reducing manual intervention and lowering client costs by $50000. • Identified and resolved performance bottlenecks in distributed data processing jobs and relational workloads, improving end to-end pipeline efficiency and scalability. • Enabled CI/CD for data pipelines using Azure DevOps supporting controlled deployments. • Worked with Apache Airflow for workflow orchestration concepts and Apache Kafka for event-driven ingestion patterns and producer–consumer fundamentals.

Senior Executive member at SAE IITR, Mar 2021 - May 2024 · 3 yrs 3 mos

Executive member at Team KNOx IIT Roorkee, Mar 2020 - Jul 2023 · 3 yrs 5 mos

Education

Mechanical engineering, Mechanical Engineering, Indian Institute of Technology, Roorkee

Public links

github: https://github.com/pandeysom921