Headline

Data Engineer | Databricks • PySpark • Delta Lake | Building Scalable Data Systems & Self-Service Data Tools | BI & AI/ML

About

I am a Data Engineer with a focus on developing scalable data systems and self-service data utilities utilizing Databricks, PySpark, and Delta Lake. In my present position, I have addressed a significant issue in data platforms—manual and inconsistent metadata extraction. I have designed and constructed reusable, parameterized utilities that automate the retrieval of schema and metadata across Delta tables, allowing data engineers and reporting teams to efficiently access essential information. These solutions have greatly diminished manual effort, enhanced standardization, and improved accessibility among teams. My primary strengths include: - Databricks ecosystem (DBFS, Delta Lake, workflows) - PySpark-based data transformations - Data modeling and schema management - Performance optimization of data workloads In addition to data engineering, I also engage with Business Intelligence tools and am actively investigating AI/ML to create intelligent, data-driven systems. I have a particular interest in positions where I can aid in the development of robust data platforms, optimize extensive data workflows, and facilitate improved decision-making through data.

Location

Bhubaneswar

AI stack

ollama

openai

ChatGPT

Work experience

System Engineer at Tata Consultancy Services, Oct 2024 - Present · 1 yr 11 mos, Bhubaneswar, Odisha, India, Working as IT Support Analyst on the AMCB ETL Support Project using Azure (ADF & Azure Databricks), SQL Server Management Studio, TIBCO Mailbox, Confluence, and IT Support Management for TD Bank Canada.

Career transition at Career Break, Aug 2024 - Oct 2024 · 3 mos, Bhubaneswar, Odisha, India, Working on transition in Data Analyst & Data Engineering along with preparation for competitive exams for Central & State Government. I am also actively looking for new opportunities that upgrade my career in the field of Data Analytics and Visualization with ML & Data Engineering.

Founder at SAMBANDHSETU, Apr 2023 - Sep 2024 · 1 yr 6 mos, Bhubaneswar, Odisha, India, Established and led a community aimed at bridging the communication gap between alumni and current students. Implemented social media strategies to boost member engagement and build alumni profiles.

Software Engineer at CSM Technologies, Dec 2021 - Aug 2024 · 2 yrs 9 mos, Bhubaneswar, Odisha, India, Developed and deployed Python applications, ensuring efficiency and scalability. Utilized MySQL for data storage and retrieval, enhancing data management processes. Conducted comprehensive analysis and visualization of data using Pandas and SQLAlchemy. Created intuitive and shareable Tableau dashboards, integrating visual data and user-friendly interfaces. Leveraged BeautifulSoup for web scraping to gather and analyze live data. Documented code and analysis processes using Jupyter Notebooks for transparency and ease of understanding.

Education

Bachelor of Technology, Computer Science and Engineering, Gandhi Institute for Technological Advancement

Public links

github: https://github.com/abhayvikramnayak98

abhayvikramnayak98.github.io: https://abhayvikramnayak98.github.io/