Lead Data Engineer with hands-on experience in cloud, data engineering, and DevOps, specializing in designing and delivering scalable AWS-based data platforms using Spark, Scala, Python, Docker, and Kubernetes.
Quantexa 2023-02-06 —
Quantexa is a global data and analytics software company that provides a Decision Intelligence platform to help organizations make trusted operational decisions by turning disparate data into a single, contextual, and meaningful view.
Highlights
- Spearheaded the end-to-end implementation and successful delivery of a high-impact Quantexa financial crime detection system—processing over 12 million capital market trades to identify money laundering activities,while demonstrating strong technical leadership and effective management in a complex banking environment..
- Applied advanced technical skills including entity resolution, scoring, graph scripting, ETL development, and task loading using Spark and Scala to architect a robust, contextual decision intelligence solution.
- Managed a 25 person cross-functional team, collaborating closely with product leadership to influence strategic roadmap decisions and project prioritization.
- Actively contributed to drafting best practices and comprehensive documentation for scoring methodologies and pipeline design, enabling other use cases to seamlessly adopt and reuse standardized implementation guidelines.
- Not only implemented the Quantexa solution but also configured and set up the client’s infrastructure using Kubernetes and Docker, ensuring scalable, secure, and resilient deployment across environments.
Change Healthcare | Innova Solutions 2021-07-02 — 2023-01-31
A healthcare technology company with a focus on calculating risk analytics of individuals for insurance companies in the US.
Highlights
- Successfully upgraded their existing commerical health insurance line of business that was previously hosted on-premise to AWS. This is primarily being done so their operational costs reduce as well as they are able to scale faster. Implemented the solution using Spark, Scala, and AWS services such as Lambdas, Step Functions, SQS, SNS, Glue, and Athena.
- Optimised pipelines by 7 times, their initial processing times were around 3 weeks, which is now close to 3 to 4 days. This was done by rewriting most of the systems to support concurrency.
- Improved accessibility to data that previously lived in one data mart, which now is structured into multiple data lakes. The read queries were quicker which contributed to the faster pipelines.
Principal Global Services 2018-10-18 — 2021-07-01
Principal Global Services is a global delivery center for Principal Financial Group which is a global financial investment management and insurance company.
Highlights
- Benefit Event Prediction and Propensity Models - Automated prediction models using Python to predict retirement and job change of a customer with Principal.This helped save 19 billion dollars for Principal.
- Campaign Metrics - Creating an omni channel marketing dashboard using Power BI, Spark, Hive and Scala on AWS for measuring marketing performance measurement across various channels like web, social media, email, banners etc.Earlier marketing teams had to manually check for these metrices in different systems. This integration helped save 60 working hours each quarter.
- Email Sentiment Analysis - Developed an application to analyse sentiments on emails between employers and Principal. This helped marketing teams to classify employer sentiments which helped employer retention with Principal.
- Advisor Lead Scoring - Automated scoring logic using Scala for scoring advisors based of which Principal can take decisions on future engagements. With this Principal was able to increase the advisor network by 3 times.
- Log Analysis and Storage Prediction - Built an application using ELK stack and Filebeat which predicts the disk space,query metrics,time and CPU usage. This helped the DBAs to flag resource crunches on time, helping avoid incidents in production.
- Tech Connects - Conducted 22 sessions to train 60 people on the Cloud, Python and other data engineering concepts.
Vietnamobile Telecommunications Ltd - Onsite (Tech Mahindra) 2017-11-01 — 2018-10-10
Vietnamobile is a vietnamese mobile network operator,My Team helped them build their data platform
Highlights
- Acted as a hadoop designer - Designed solutions to address issues pertaining to marketing, finance and BI teams. Educated teams on big data technologies, Hadoop infrastructure maintenance, Hadoop operations and the ETL stack on Hadoop. Also designed hadoop integration with other tools like Splunk,R and Tableau.
- Automated a new process to accommodate mobile number changes of the entire country as per government norms, using CDC in hadoop datalake. This design helped us to make changes to around 12 million subscribers in a week's time, without any impact.
Tech Mahindra Ltd. 2016-01-01 — 2017-10-31
Tech Mahindra is a services firm that helps clients deliver software better and faster.
Highlights
- Provided enterprise solutions to a leading telecom provider of Vietnam by creating a visualization environment using Tableau to aid them in viewing network traffic, revenue trends,subscriber categorizations, RGS and ARPU reports for taking business decisions.
- Built and deployed two Hadoop clusters using Hortonworks distribution.These clusters could help process around 1 to 2 TB of data daily.
- Designed and implemented a datalake in Hadoop which helped analyse data from 10 different source systems like Single View, Voucher Management Systems, Campaign Management Systems and Siebel CRM systems(these systems are commonly used in the telecom domain).
- Created transformation logic and implemented optimizations on Hive, Sqoop, and YARN to process 1–2 TB of data per day, including scheduling and workflow forking through Oozie.
Jawaharlal Nehru Technological University, Kakinada 2011-03-01 — 2015-05-01
St Agnes Convent, Kharagpur 2009-03-01 — 2011-01-01
St Agnes Convent, Kharagpur 1998-03-01 — 2009-01-01
Ace Performer
Winner
Programming
- Scala
- Python
- SQL
- Unix Shell Scripting
Tools
- Quantexa
- Tableau
- Power BI
- Talend
- Qlikview
- Denodo
Technologies
- Spark
- Quantexa
- Airflow
- Hadoop
- Kafka
- Kerberos
Devops
- AWS
- Docker
- Kubernetes
- Terraform
- Bamboo
- Github Actions
Domains
- Healthcare
- Aerospace
- Finance
- Telecom
- Marketing
Software Delivery Practices
- Agile
- Dojo
- SAFE