Data engineer with a mathematics background and experience designing reliable, scalable data systems across healthtech, energy, and sustainability. Strong in Python, SQL, ETL/ELT, data modelling, orchestration, and performance optimization, with hands-on experience across AWS, GCP, and Spark-based data platforms. Works closely with technical and non-technical stakeholders to translate business requirements into robust production solutions.
Experience
Senior member of a small team building and maintaining the data ingestion system that migrates client data onto the Wawa platform during clinic onboarding. Given the sensitivity of patient health data, the role carries a strong emphasis on correctness, governance, and auditability, alongside direct client engagement.
- Design and maintain a scalable data ingestion and transformation pipeline used to migrate client data from legacy systems onto the Wawa platform.
- Build maintainable data solutions with a strong emphasis on correctness, governance, auditability, and reproducibility in a regulated data environment.
- Introduced a partially AI-driven approach to the migration process while respecting data privacy and regulatory requirements, delivering substantial speed improvements.
- Built the pipeline using Polars and DuckDB, integrating it with an AWS-hosted Ruby on Rails application.
- Work directly with clients in sales meetings and data validation sessions, translating requirements and technical constraints into practical solutions.
- Handle data extraction and secure interfacing with a range of external systems and data formats, including non-standard/exotic formats.
- Contribute to the infrastructure supporting the pipeline, provisioned and managed with Terraform on AWS.
Core member of the Grid modelling and methodology team, responsible for ingesting, cleaning, and applying business logic to electricity production, weather, and price data. Enabling forecasting and product teams through scalable data infrastructure.
- Migrated legacy codebase to best-in-class data engineering tools on GCP, implementing Airflow for orchestration, BigQuery for analytics, and PostgreSQL for API and map data.
- Expanded geographical coverage from regional to global scale, significantly increasing product offering and market reach.
- Optimized PostgreSQL data models following best practices, achieving order-of-magnitude improvements in query performance while reducing storage costs.
- Modernized data pipelines by migrating from Apache Beam to faster frameworks like Polars, achieving up to 90% performance improvements in critical workflows.
- Introduced higher granularity data offerings, enabling 5-minute and 15-minute resolution across all zones beyond initial hourly data.
- Expanded data signals to include day-ahead price and location marginal price data, enriching product capabilities for customers.
- Led a revamp of the emission factor calculation methodology, uncovering and resolving numerous business logic bugs that had been affecting the core product's accuracy.
- Fed data into a FastAPI-based backend application, occasionally contributing directly to its codebase.
Progressed from designing custom data solutions for climate accounting dashboards to core product development, scaling and standardizing the company's offerings while ensuring correctness of business logic and data sources.
- Led the company's largest and most complex implementation project, a greenfield solution that later evolved into the Product Carbon Footprint offering — which went on to drive the majority of sales in 2024.
- Built and orchestrated Apache Spark-based data transformation pipelines on Palantir Foundry, the primary platform for processing and standardizing client data.
- Interfaced with a GCP-hosted Golang-based backend application, bridging data engineering pipelines with core product services.
- Collaborated with sustainability experts, product, tech, and customer success teams to translate business and regulatory requirements into code and ensure efficient delivery.
- Contributed to team growth through code reviews and mentoring, including onboarding two senior data engineers, in an agile team setup.
Developed and deployed ML models for ADHD detection in patients based on EEG recordings.
- Developed CNN model with over 80% accuracy in detecting ADHD in children under 14.
- Collaborated with neuroscientists to implement custom mental health metrics.
- Applied numerical techniques for outlier detection and bootstrapping.
Education
Awards
Santa 2022: a modified travelling salesman problem. Applied the LKH heuristic solver; placed 55th out of 874.
Volunteer
Contributed to open source projects, mainly pandas and scikit-learn.
- Expanded coverage of
.attrsmetadata feature for pandas DataFrames. - Expanded coverage of copy-on-write behaviour for pandas DataFrames.