Saranya Manoharan
Cloud Data Engineer
Chennai
Summary
Results-driven Data Engineer with over four years of experience in the Big Data domain, specializing in technologies such as Spark and the Databricks Platform. Proven expertise in data extraction, transformation, cleansing, and loading processes, ensuring high-quality data management. Skilled in identifying and resolving performance bottlenecks across various stages of data workflows, leveraging Databricks to optimize efficiency and enhance overall system performance. Committed to delivering innovative data solutions that drive business insights and support strategic decision-making.
Overview
6
6
years of professional experience
Work History
Azure Cloud Engineer
Orennia
05.2024 - 03.2025
- Designed and implemented data ingestion pipelines in Azure Data Factory (ADF) to bring in raw energy sector datasets.
- Developed PySpark and Spark SQL transformations in Azure Databricks to process large-scale energy production, asset, and market datasets.
- Built Delta Lake tables with optimized partitioning and Z-Ordering to improve query performance and support incremental loads.
- Reduced ETL processing time from 40–50 minutes to 10–15 minutes through parallel API calls, optimized joins, and caching strategies.
- Proactively addressed potential bottlenecks in the ETL process through regular monitoring, enabling seamless workflow operations.
- Increased operational efficiency by automating repetitive tasks using Python scripts, allowing focus on higher-priority projects.
Big Data Engineer
Tech Mahindra
08.2022 - 01.2023
- Designed parameterized notebooks to read metadata and create Delta tables dynamically.
- Integrated Databricks with Azure Synapse Analytics for reporting and analytics.
- Automated pipeline orchestration and monitoring using Azure Data Factory.
- Optimized data processing by implementing Hadoop and Spark frameworks for big data management.
- Conducted thorough performance testing to optimize system configurations and maximize resource utilization.
- Improved collaboration between teams by creating comprehensive documentation detailing technical aspects of various big data solutions.
- Migrated legacy systems to modern cloud-based platforms for increased efficiency and scalability.
- Designed scalable ETL pipelines for improved data ingestion, processing, and storage.
- Increased operational efficiency by automating repetitive tasks using Python scripts, allowing focus on higher-priority projects.
Associate Data Engineer
Capgemini
07.2021 - 07.2022
- Migrated data from Hive to Oracle database and performed data enrichment tasks such as filtering, format modelling, reporting, and aggregation using Azure Databricks.
- Explored new technologies such as Hadoop or Spark to process large volumes of unstructured data effectively.
- Participated in code reviews and provided constructive feedback to peers, fostering a culture of continuous improvement within the team.
- Developed custom scripts for automating repetitive tasks, freeing up valuable time for team members to focus on more strategic initiatives.
- Compiled, cleaned and manipulated data for proper handling.
- Collaborated on ETL (Extract, Transform, Load) tasks, maintaining data integrity and verifying pipeline stability.
Junior Data Engineer
M2 Data Studio
03.2019 - 07.2021
- Developed Big Data pipelines for extracting, transforming, and loading large datasets to support business insights.
- Ingested publishing-related data from multiple RDBMS tables using Sqoop into Data Lake.
- Used Spark DataFrames to process product, retailer, contract, royalty, and sales information.
- Categorized sales status (completed, pending, processing) at 3-hour intervals.
- Stored processed data in HBase daily and generated monthly reports.
- Enabled sales and royalty data visualization using Power BI.
- Optimized data processing by implementing efficient ETL pipelines and data transformation techniques.
Education
PG Diploma - Reporting System and Database Development
Conestoga College
Kitchener, Canada08-2023
MBA - Human Resources
University of Madras
Chennai, India06-2011
B.Sc. - Chemistry
Thiru.Vi.Ka. College of Arts And Science
Thiruvarur, India04-2006
Skills
Big Data Framework
Databricks
Spark
Data Lake
Hadoop/Hive (bigdata ecosystem)
Snowflake
Azure Data Factory
undefinedPlatform/Tools/Technologies Used
- Databricks
- Azure Data Factory
- Azure data lake storage
- Power BI
- Snowflake
Timeline
Azure Cloud Engineer
Orennia
05.2024 - 03.2025
Big Data Engineer
Tech Mahindra
08.2022 - 01.2023
Associate Data Engineer
Capgemini
07.2021 - 07.2022
Junior Data Engineer
M2 Data Studio
03.2019 - 07.2021
PG Diploma - Reporting System and Database Development
Conestoga College
MBA - Human Resources
University of Madras
B.Sc. - Chemistry
Thiru.Vi.Ka. College of Arts And Science
Similar Profiles
Zaheer Syed.SZaheer Syed.S
Team Manager at GSR Global ServicesTeam Manager at GSR Global Services
PIYUSH RANJAN DASPIYUSH RANJAN DAS
Associate at BNY IndiaAssociate at BNY India
MANOJ MMANOJ M
Senior Software Engineer (Managing Position) at LTIMindtreeSenior Software Engineer (Managing Position) at LTIMindtree
Manikandan SekarManikandan Sekar
JDE World 9.4 Production Support at LennarJDE World 9.4 Production Support at Lennar
Suman UnnikatSuman Unnikat
Senior Systems Engineer (Azure Cloud Engineer) at Infosys LimitedSenior Systems Engineer (Azure Cloud Engineer) at Infosys Limited