Abdul Kalam profile photo 1 on Tal

Abdul Kalam

Senior Data Engineer @ Eli Lilly and Company | AWS Data Engineer, Python, SQL, Databricks, Gen AI, LLM, MCP Server

Greater Bengaluru Area

6+ years of work experience

Eli Lilly and Company
Mar 20269 mosPresent

Hybrid

Eli Lilly and Company
Oct 20241 yr 5 mosFeb 2026

On-site

Eli Lilly and Company
Mar 20231 yr 8 mosOct 2024

On-site

• Framework Development for Asynchronous Job Monitoring: Designed and developed a comprehensive framework to capture the status of asynchronous jobs using API Gateway, AWS Step Functions, Lambda, and DynamoDB, enabling real-time monitoring and streamlined processing. • Team Leadership in AWS CDK Implementation: Led the team in designing and implementing an AWS CDK framework to provision critical services, including Lambda, Glue, Redshift, IAM, Cognito, and API Gateway, in alignment with Infrastructure as Code (IaC) best practices. • Unified AWS Account Framework: Developed an innovative framework that consolidated three separate AWS accounts into a unified repository. This initiative significantly improved management and deployment processes, boosting collaboration, security, and operational consistency across the organization. • API Security Enhancement: Engineered a robust API that retrieves data from third-party sources and implemented a Cognito user pool using AWS CDK to validate JWT tokens, thereby strengthening API security. • Infrastructure as Code Implementation: Led the setup of AWS deployment infrastructure within a GitHub repository, ensuring streamlined and repeatable deployments. • Data Transformation and Ingestion: Developed PySpark scripts to read data from data lakes, perform complex transformations, and ingest the processed data into Amazon Redshift. • Orchestration Automation: Developed AWS Step Functions to orchestrate complex workflows, ensuring efficient and reliable process automation. • Role and Permission Management: Created OIDC roles, federated roles, and service roles, meticulously defining and assigning the necessary permissions to ensure secure and compliant access control. • Automated Input File Quality Control: Spearheaded the development of an automated QC process for input files, which reduced manual effort by approximately 50% by eliminating the need to load files into Hive and manually execute SQL queries.
Deloitte India (Offices of the US)
Feb 20212 yrs 1 moFeb 2023

Bengaluru, Karnataka, India

• Automated the auditing framework for data ingestion using AWS, Python and SQL to reduce the manual efforts in auditing the data. • Implemented the Kinesis stream data ingestion pipeline which supports near real time ingestion. • Designed the pipeline for ingestion of data from external S3 to the target systems and incorporated into the existing framework aligned to the business requirement. • Implemented the SNS and Logger functionality in the code for better managing and alerting of process. • Implemented the Kinesis Stream to collect data with near real-time frequency and flattening the json files. • Created script for updating the records in DynamoDB table • Development of python script to load the data into s3 from API. • Processing the files across the cloud platform and ingest it into the redshift • Developed Pyspark scripts to read the data from Oracle, perform transformations and ingest into redshift. • Developed CFT template to create the glue job with job level parameters. • Developed automation script in python to run the jobs automatically without any manual intervention.
Cognizant
Dec 20191 yr 3 mosFeb 2021

Chennai, Tamil Nadu, India

• Creating ETL jobs and providing support to ensure delivery of data within the SLA. • Developed SQL logic for process data validation. • Involved in analysis and defect review meetings with the customer • Debug and processing the job according to the required output. • Ingesting the bulk data into the IDP tool and process the data. • Involved in unit testing to test end-to-end application functionalities using DICE Framework. • Coordinate and communicate with Onsite team and preparing technical design documents. • Analyse the customer change requests, come up with estimation of work, and provide walkthrough to the Product owners.
GitHub
@Abdulkalam98

Abdul’s AI Stack

antigravity
antigravity
claude code
claude code
claude
claude
github copilot
github copilot
hugging face
hugging face
langchain
langchain

Education

PSG College of Technology

Bachelor of Engineering - BE, Electronics and Communications Engineering

PSG College of Technology

20163 yrs2019
udemy

Data Science

udemy

Join Abdul and 10k+ other professionals on

Tal Boss

Tal is the app where bosses meet engineers & builders directly, without middlemen

Download Tal Boss

Abdul Kalam on Tal

Headline

Senior Data Engineer @ Eli Lilly and Company | AWS Data Engineer, Python, SQL, Databricks, Gen AI, LLM, MCP Server

About

6 years of extensive experience in developing and implementing AWS Cloud services and tools using Python, Pyspark, and also have experience across development, Data Engineering, Data visualization, Data Analytics and AWS. • Primary Skills: ---- AWS | Pyspark | Python | Databricks | Cloud Architecture | Data Engineering | Glue | SQS | EMR | SQL | • Secondary Skills: ---- | Spark | Git | Data Science | Machine Learning | • Hobbies/Interest: ---- Kaggle’ing | Programming | Github ---- TT | PC Games | Traveling | Badminton | Cricket | Football • Strengths: ---- Ownership | time management | Self-Learner | analytical skills | Self-Confident | • Weaknesses: ---- 'NO' to 'No' | Lack of prioritization skills | Spontaneous Git: https://github.com/Abdulkalam98

Location

Greater Bengaluru Area

AI stack

antigravity

claude code

claude

github copilot

hugging face

langchain

Work experience

Senior Data Engineer - Data Engineering at Eli Lilly and Company, Mar 2026 - Present · 9 mos, Hybrid

Senior Data Engineer II at Eli Lilly and Company, Oct 2024 - Feb 2026 · 1 yr 5 mos, On-site

AWS Data Engineer at Eli Lilly and Company, Mar 2023 - Oct 2024 · 1 yr 8 mos, On-site, • Framework Development for Asynchronous Job Monitoring: Designed and developed a comprehensive framework to capture the status of asynchronous jobs using API Gateway, AWS Step Functions, Lambda, and DynamoDB, enabling real-time monitoring and streamlined processing. • Team Leadership in AWS CDK Implementation: Led the team in designing and implementing an AWS CDK framework to provision critical services, including Lambda, Glue, Redshift, IAM, Cognito, and API Gateway, in alignment with Infrastructure as Code (IaC) best practices. • Unified AWS Account Framework: Developed an innovative framework that consolidated three separate AWS accounts into a unified repository. This initiative significantly improved management and deployment processes, boosting collaboration, security, and operational consistency across the organization. • API Security Enhancement: Engineered a robust API that retrieves data from third-party sources and implemented a Cognito user pool using AWS CDK to validate JWT tokens, thereby strengthening API security. • Infrastructure as Code Implementation: Led the setup of AWS deployment infrastructure within a GitHub repository, ensuring streamlined and repeatable deployments. • Data Transformation and Ingestion: Developed PySpark scripts to read data from data lakes, perform complex transformations, and ingest the processed data into Amazon Redshift. • Orchestration Automation: Developed AWS Step Functions to orchestrate complex workflows, ensuring efficient and reliable process automation. • Role and Permission Management: Created OIDC roles, federated roles, and service roles, meticulously defining and assigning the necessary permissions to ensure secure and compliant access control. • Automated Input File Quality Control: Spearheaded the development of an automated QC process for input files, which reduced manual effort by approximately 50% by eliminating the need to load files into Hive and manually execute SQL queries.

Data Engineer at Deloitte India (Offices of the US), Feb 2021 - Feb 2023 · 2 yrs 1 mo, Bengaluru, Karnataka, India, • Automated the auditing framework for data ingestion using AWS, Python and SQL to reduce the manual efforts in auditing the data. • Implemented the Kinesis stream data ingestion pipeline which supports near real time ingestion. • Designed the pipeline for ingestion of data from external S3 to the target systems and incorporated into the existing framework aligned to the business requirement. • Implemented the SNS and Logger functionality in the code for better managing and alerting of process. • Implemented the Kinesis Stream to collect data with near real-time frequency and flattening the json files. • Created script for updating the records in DynamoDB table • Development of python script to load the data into s3 from API. • Processing the files across the cloud platform and ingest it into the redshift • Developed Pyspark scripts to read the data from Oracle, perform transformations and ingest into redshift. • Developed CFT template to create the glue job with job level parameters. • Developed automation script in python to run the jobs automatically without any manual intervention.

Data Engineer at Cognizant, Dec 2019 - Feb 2021 · 1 yr 3 mos, Chennai, Tamil Nadu, India, • Creating ETL jobs and providing support to ensure delivery of data within the SLA. • Developed SQL logic for process data validation. • Involved in analysis and defect review meetings with the customer • Debug and processing the job according to the required output. • Ingesting the bulk data into the IDP tool and process the data. • Involved in unit testing to test end-to-end application functionalities using DICE Framework. • Coordinate and communicate with Onsite team and preparing technical design documents. • Analyse the customer change requests, come up with estimation of work, and provide walkthrough to the Product owners.

Education

Bachelor of Engineering - BE, Electronics and Communications Engineering, PSG College of Technology

Data Science, udemy

Public links

github: https://github.com/Abdulkalam98