Ajin Rajan
Data Engineer
Marthandam
Summary
Driven Data Engineer with a proven track record at Infosys, adept in migrating and optimizing data processes using SQL, Apache Airflow, and Python. Excelled in enhancing pipeline reliability and data security, showcasing strong analytical skills and a commitment to efficiency. Demonstrates exceptional problem-solving abilities and teamwork, ensuring seamless data operations.
Overview
3
3
years of professional experience
4
4
years of post-secondary education
Work History
Data Engineer
Infosys
07.2023 - Current
- Worked on migrating legacy SQL Server to Amazon Redshift and converting SSIS packages into SQL scripts, then scheduling them using Airflow DAGs
- Review existing SSIS packages and converted the SSIS package logic into SQL scripts that can be executed in Redshift.
- Created 50+ DAGs and configured AWS SNS to send alert messages in case of DAG failures, ensuring timely issue resolution and improved pipeline reliability.
- Developed Python scripts to automate data transfer from Amazon S3 to Amazon Redshift, ensuring efficient and reliable data ingestion.
- Implemented Python functions to securely access credentials from AWS Secrets Manager, enhancing the security of data operations.
- Successfully migrated Excel reports from SSAS to Tableau by directly connecting to Amazon Redshift tables.
Data Engineer
Infosys
10.2022 - 06.2023
- Worked on migrating 75+ DAGs and configurations from GCP Cloud Composer 1 to Cloud Composer 2.
- Ensured that the DAGs are compatible with Cloud Composer 2 by replacing the depreciated operators.
- Implemented logic to identify and segregate bad records with extra delimiters into a separate table while loading the flat files from Google Cloud Storage (GCS) to BigQuery for further analysis.
- Ensured the DAGs are working as expected before deploying them into the Cloud Composer 2 production environment
Data Engineer
Infosys
10.2021 - 09.2022
- Involved in monitoring the ETL pipeline that extracts the data from source system ,writes it to staging area, transforms it based on requirements, and then writes it to Teradata target system using the Tivoli job monitoring tool
- Involved in identifying the root cause of the Teradata Mload failure and the data discrepancy between source and target system
- Performed data cleansing by identifying the bad records and duplicates records from the source system using complex queries
- Involved in optimizing the complex Sql queries
- Identified problems and applied quick fixes to the Sql script by removing the duplicates and null values before loading the data into Target tables
Education
B.E - Mechanical Engineering
Jeppiaar Maamallan Engineering College
Chennai, India 08.2015 - 04.2019
Skills
ETLs
SQL (Redshift, Big Query)
Apache Airflow
Python
AWS (s3, sns, secrets manager)
Tableau
Linux
Git
Timeline
Data Engineer
Infosys
07.2023 - Current
Data Engineer
Infosys
10.2022 - 06.2023
Data Engineer
Infosys
10.2021 - 09.2022
B.E - Mechanical Engineering
Jeppiaar Maamallan Engineering College
08.2015 - 04.2019
Similar Profiles
Ved ShuklaVed Shukla
Senior System Engineer at InfosysSenior System Engineer at Infosys
ASMITA KULKARNIASMITA KULKARNI
Consultant at InfosysConsultant at Infosys
Akshat YadavAkshat Yadav
Technology Lead-US at InfosysTechnology Lead-US at Infosys