Vamsikrishna Kayyala
Lead Data Engineer
Chennai,TN
Summary
Insightful, result-driven Data Engineering professional with extensive experience in designing, building, and optimizing large-scale data pipelines using Spark, Scala, Kafka, AWS, Airflow and Delta Lake. Proven expertise in ingesting and transforming high-volume event/CDC data, architecting scalable data platforms, and enabling analytics and business intelligence through reliable, high-quality data solutions. Adept at leading engineering initiatives, driving best practices, and delivering robust, production-grade data systems in fast-paced and dynamic environments. Seeking opportunities within a challenging organization that values innovation, technical leadership, and meaningful contribution to data-driven decision making.
Overview
12
12
years of professional experience
6
6
years of post-secondary education
5
5
Certifications
4
4
Languages
Work History
Lead Data Engineer
Chargebee
02.2022 - Current
- Managing a portfolio of data engineering initiatives, and leading a team of engineers to ensure high-quality, timely deliverables.
- Performing feasibility assessments for new data ingestion requirements, and defining optimal architecture using Spark, Kafka, and AWS services.
- Designing and developing scalable data ingestion pipelines to process high-volume events data from distributed source systems.
- Implemented a highly scalable custom dynamic partitioning logic in the Delta Lake to solve a scaling problem; it reduced the streaming batch time from 60 minutes to 20 minutes with fewer cluster resources than before.
- Well-versed in creating complex and scalable orchestration DAGs to schedule the Spark jobs in EMR.
- Designing and maintaining Delta Lake–based data lakes with efficient partitioning, schema evolution, and ACID-compliant operations.
- Building reusable data processing modules, ingestion components, and framework-level utilities to accelerate pipeline development.
- Managing end-to-end deployment, orchestration, and monitoring of data pipelines using AWS EMR, Glue, Lambda, and CloudWatch.
- Preparing and maintaining test datasets to validate ETL logic, transformations, and incremental loads.
- Benchmarking and optimizing existing Spark workloads to improve performance, reduce costs, and enhance cluster utilization.
Test Engineering Associate Manager
Accenture
10.2017 - 02.2022
- Leading 10 members team including both functional and automation resources
- Led account level initiatives to bring more awareness towards innovation and automation in general.
- Have direct interaction with client and identify new areas for opportunities to staff people accordingly in both automation and functional roles
- Automation Test Planning, Estimation and Automation Feasibility analysis for various projects.
- Developed Hybrid Automation Framework for various applications in this Project with Selenium and UFT.
- Worked extensively in improving existing BDD framework
- Guide scrum engineering teams to create meaningful feature files in Gherkin syntax and later convert them to automated scenarios using cucumber framework.
- Identify potential automation candidates within project and address gaps by implementing process improvement tools.
- Involved in In-Sprint and Regression automation on Agile projects.
- Involved in training new resources from functional team as part of QA to QE transformation.
- Develop Excel macros, utilities for various non-conventional automation requirements across projects.
- Created plans and communicated deadlines to ensure projects were completed on time.
- Developed and maintained courteous and effective working relationships.
Associate
Cognizant Technology Solutions
01.2014 - 09.2017
- Performing feasibility studies on AUT by assessing the compatibility of the testing tool and the Proof of Concept.
- Design and development of a hybrid automation framework as per business functionalities.
- Design and development of reusable functions and modules.
- Script development for individual test cases.
- Test data preparation for dry runs and trial executions of developed scripts.
- Maximized productivity by keeping detailed records of daily progress, and identifying and rectifying areas for improvement. Automation, Estimation, and Project Planning.
- Developing utilities or macros for any unconventional automation.
- Educated new hires on company policies and procedures by designing and developing a training program.
- Mentored and coached other team members to meet automation goals.
Education
B.Tech - Electronics and communication engineering
MeRITS College
Udayagiri, AP 06.2009 - 04.2013
Higher Secondary -
Srivema Junior College
Nayudupet, AP 06.2007 - 03.2009
SSC -
Navodaya High School
Nayudupet, AP 06.2006 - 03.2007
Skills
Spark
Scala
Kafka
AWS
Airflow
PySpark
Java
Python
GitHub Actions
Selenium
SQL
Accomplishments
- Received ACE award in accenture twice for innovation in automation
- Received “Pillar of the Month” award in recognition of consistent high performance and contribution towards HealthCare - QE & A for the month of February 2016
Certification
Introduction to Managed Healthcare (AHM 250 CCP) awarded by Cognizant Technology Solutions
Experience
More than 12 years of experience in IT Industry, working as a Lead Data Engineer in Chargebee.
Timeline
Lead Data Engineer
Chargebee
02.2022 - Current
IBM: Data Science Orientation
11-2021
Spring Framework Master Class - Java Spring the Modern Way
02-2021
Microsoft Certified: Azure Fundamentals
07-2020
Test Engineering Associate Manager
Accenture
10.2017 - 02.2022
Cognizant certified product consultant on HP QTP
02-2016
Introduction to Managed Healthcare (AHM 250 CCP) awarded by Cognizant Technology Solutions
06-2015
Associate
Cognizant Technology Solutions
01.2014 - 09.2017
B.Tech - Electronics and communication engineering
MeRITS College
06.2009 - 04.2013
Higher Secondary -
Srivema Junior College
06.2007 - 03.2009
SSC -
Navodaya High School
06.2006 - 03.2007