Arka Sarkar profile photo 1 on Tal

Arka Sarkar

Data Engineer @ Kfintech | SQL • Python • AWS • PySpark • Airflow

Hyderabad

3+ years of work experience

KFin Technologies Ltd.
Sep 20241 yr 8 mosPresent

Orchestrated High-Volume Data Processing: Managed processing of 35M+ folio-level records daily using cron jobs via EventBridge, leveraging Athena for data querying and deploying as Docker images on ECS for scalable execution. Built Automated Compliance Workflows: Developed end-to-end workflows using Step Functions, Lambda, Athena (via boto3), and SNS to automate regulatory data processing and enable compliance reporting via automated email delivery. Implemented Data Validation Framework: Developed automated data validation system ensuring end-to-end integrity between MSSQL source systems and production data warehouse with real-time discrepancy detection . Automated ETL Pipeline Orchestration: Built and optimized various data workflows utilizing AWS Glue workflows to automate PySpark jobs, reducing manual intervention and improving processing reliability.
KFin Technologies Ltd.
Sep 20231 yrAug 2024

Worked on data management and SQL querying using AWS Athena, migrated on-premises data to cloud using DMS for table migrations. Automated business reports using AWS Lambda and Step Functions for improved operational efficiency .
Celebal Technologies
Jun 20232 mosJul 2023

Jaipur, Rajasthan, India

Developed complex SQL queries using Microsoft SSMS, performed data analysis, and optimized large-scale SQL query performance for business intelligence reports.
HighRadius
May 20232 mosJun 2023

Bhubaneswar, Odisha, India

Built an AI-enabled Fintech B2B Invoice Management Application. Designed user-friendly UI with ReactJS for a seamless user experience. Implemented robust backend functionality in Java for efficient server-side operations.

Arka’s AI Stack

gemini
gemini
github copilot
github copilot
ollama
ollama
openai
openai

Education

Kalinga Institute of Industrial Technology, Bhubaneswar

Bachelor's degree, Information Technology

Kalinga Institute of Industrial Technology, Bhubaneswar

20204 yrs2024

Join Arka and 10k+ other professionals on

Tal Boss

Tal is the app where bosses meet engineers & builders directly, without middlemen

Download Tal Boss

Arka Sarkar on Tal

Headline

Data Engineer @ Kfintech | SQL • Python • AWS • PySpark • Airflow

About

Data Engineer with 2 years of experience building scalable data pipelines and leading cloud migration initiatives on AWS. Skilled in utilizing AWS services including Lambda, S3, Glue, Kinesis, Redshift, and Athena and programming languages including Python and Scala.

Location

Hyderabad

AI stack

gemini

github copilot

ollama

openai

Work experience

Software Engineer at KFin Technologies Ltd., Sep 2024 - Present · 1 yr 8 mos, Orchestrated High-Volume Data Processing: Managed processing of 35M+ folio-level records daily using cron jobs via EventBridge, leveraging Athena for data querying and deploying as Docker images on ECS for scalable execution. Built Automated Compliance Workflows: Developed end-to-end workflows using Step Functions, Lambda, Athena (via boto3), and SNS to automate regulatory data processing and enable compliance reporting via automated email delivery. Implemented Data Validation Framework: Developed automated data validation system ensuring end-to-end integrity between MSSQL source systems and production data warehouse with real-time discrepancy detection . Automated ETL Pipeline Orchestration: Built and optimized various data workflows utilizing AWS Glue workflows to automate PySpark jobs, reducing manual intervention and improving processing reliability.

Intern at KFin Technologies Ltd., Sep 2023 - Aug 2024 · 1 yr, Worked on data management and SQL querying using AWS Athena, migrated on-premises data to cloud using DMS for table migrations. Automated business reports using AWS Lambda and Step Functions for improved operational efficiency .

SQL Developer at Celebal Technologies, Jun 2023 - Jul 2023 · 2 mos, Jaipur, Rajasthan, India, Developed complex SQL queries using Microsoft SSMS, performed data analysis, and optimized large-scale SQL query performance for business intelligence reports.

Product Engineering Intern at HighRadius, May 2023 - Jun 2023 · 2 mos, Bhubaneswar, Odisha, India, Built an AI-enabled Fintech B2B Invoice Management Application. Designed user-friendly UI with ReactJS for a seamless user experience. Implemented robust backend functionality in Java for efficient server-side operations.

Education

Bachelor's degree, Information Technology, Kalinga Institute of Industrial Technology, Bhubaneswar

Arka Sarkar - Software Engineer in Hyderabad | Tal