Vanmathi Varadharaj

Over 4 years of experience within the hospital sector, showcasing a dynamic blend of software development and analytics process to enhance operational efficiency.Have collaborated extensively with cross-functional teams and functional heads for creating essential daily reports for higher management, medical practitioners financial stakeholders.This data-driven approach has consistently supported financial decision-making and insightful lab analyses.

Client: Narayana Health Hospital | Health City Cayman Islands | Aarthi Scans

Role: Data Engineer

Responsibilities:

• Developed spark applications using spark-sql in databricks for data extraction,transformation, aggregation of multiple file formats for analysing and transforming the data to insights by adhering to clients requirement.
• Experience in managing incremental and historical data loads to efficiently handle incremental updates and historical data.
• Designed and implemented automated email notifications with PySpark data processing pipelines to keep stakeholders informed of job statuses and data processing milestones.
• Developed Spark jobs to efficiently parse JSON data from Elasticsearch indices using PySpark.Utilized Spark SQL to handle and process structured data extracted from JSON documents, applying transformations and aggregations as required.
• Proficient in leveraging Microsoft Azure services, including Azure Data Factory, Databricks, Storage Accounts, Key Vault, and Logic Apps, to architect and implement end-to-end data solutions.
• Extensive experience in Azure Data Factory, designing and deploying ADF pipelines for data extraction from diverse sources, including PostgreSQL, Oracle, and Azure SQL Database, using linked services and secure data source connections with Azure Key Vault secrets.Skilled in setting up triggers and scheduling automated job runs in Azure Data Factory for seamless data processing and orchestration, ensuring timely and reliable execution of data workflows.
• Designed and implemented data engineering solutions leveraging AWS Glue and Databricks platforms for efficient data processing and integrated AWS SES (Simple Email Service) into data processing pipelines to send automated email notifications for job statuses, error alerts.
• Led a team of 2 members in the migration of ETL processes from legacy SSIS framework to Azure Databricks, ensuring seamless transition and minimal disruption to operations.

Requirement was to Extract data from different sources [Pi server/Oracle/SAP] to staging using the transformation business logic as required and loading from staging tables to datawarehouse for Unconventional energy data using the interfaces created.

Client: Royal Dutch Shell

Role: ETL Developer

Responsibilities:

• Collaborate with onshore/offshore Functional Analyst to gather Business requirements,
translate and apply business rules for data transformations for ETL functions.
• Developed ETL’s as Enhanced Request, Change Request, New development as scoped by
business, deployed, optimized and maintained the data integrity.
• Excellent knowledge in doing Root Cause Analysis, debugging and fixing the defects raised by
business.
• Experience in creating RFC, Config files for deploying in various environment [UAT/INT/PROD].
• In depth knowledge in writing T-SQL Queries, stored procedures, CTE’s, triggers, Views.
• Experience in providing logging, Error handling by using Event handlers, custom logging for
SSIS packages and Exception data handling.
• Involved in preparing unit test cases, performing Code reviews, provided application
production support, monitored running of critical packages.
• Created complex ETL packages using SSIS to extract data from staging tables to data
warehouse with Incremental loads, Batch processing, Versioning tables, Skeleton approach
and defining data quality columns.
• Been able to deliver large amounts of changes in very little time and with great quality.
• Experienced in using Breakpoints, Containers -For loop, Sequence container, For each loop.
Data Flow Transformations -Lookups, Pivot task, Script task, Execute Sequel Task, Merge, Join,
Union, Conditional split, Derived column and loading data from Excel files, Flat files sources to SQL Server.
• Experience in handling production data for business and performed operations like Production
Build verification, Data reconciliation, validation and error handling after extracting data into
SQL server.