×
Shaughn Finnerty

Shaughn Finnerty

Senior Data & AI Platform Engineer

Toronto, Ontario, CA
(613) 585-8175

Background


About

About

Senior Data & AI Platform Engineer with experience building self-service platforms, distributed systems, and operational tooling that enable Analytics, Data Engineering, Data Science, Applied AI, and Product Engineering teams to deploy intelligent systems at scale. Experienced in AI platform architecture, machine learning infrastructure, event-driven systems, data platforms, and observability. Proven track record of designing reliable, scalable services that accelerate delivery velocity while maintaining governance, security, and operational excellence. Recipient of the University of Ottawa Faculty of Engineering Plaque in Computer Science, awarded to the graduating student with the highest academic standing in the program.

Work Experience

Work Experience

  • Senior Data & AI Platform Engineer, 

    Dec, 2024 - Present

    Building self-service AI, machine learning, and data platforms that enable Analytics, Applied AI, Data Science, Product Engineering, and Business Operations teams to deploy and operate intelligent systems at scale.

    • Architected and launched Homebase's first production LLM platform, enabling Applied AI and Data Science teams to deploy agentic workflows behind scalable gRPC endpoints consumable by product engineering teams.

    • Designed the platform to be fully self-service, allowing teams to independently develop, validate, deploy, and operate LLM workflows while platform infrastructure handled scaling, monitoring, security, cost controls, and operational reliability.

    • Built reusable deployment patterns and infrastructure for machine learning inference services, accelerating the productionization of predictive models and AI-powered product features.

    • Designed and implemented one of Homebase's first event-driven platform services, enabling near real-time synchronization of operational data into Salesforce and establishing foundational patterns for future event-driven architectures across the organization.

    • Created self-service tooling that enabled Business Operations teams to onboard and instrument new data flows without dedicated Data Engineering involvement while maintaining governance and reliability standards.

    • Led the migration of critical Airflow workloads to ECS-based execution environments after identifying systemic zombie-task and worker reliability issues, significantly improving orchestration stability and reducing operational incidents.

    • Led adoption of Databricks Serverless Compute, improving execution times of long-running data pipelines while reducing platform management overhead.

    • Established platform-wide standards for monitoring, observability, alerting, and deployment across gRPC services, Celery workers, ML inference services, and Streamlit applications, improving operational visibility and accelerating incident detection and resolution.

    • Designed and validated a governed data activation platform enabling teams across the organization to securely move warehouse data to external systems and partners, reducing Data Engineering bottlenecks and increasing delivery velocity.

  • Data Platform Engineering Lead, 

    Dec, 2022 - Nov, 20241 year 11 months

    Data Platform Engineering Delivery Lead

    • Spearheaded one of the first moves towards micro-services in the engineering organization by creating a fully-monitored gRPC API Python service that would provide scores and recommendations generated by the Data Science team for use by the Product Engineering team in-app

    • Worked cross-functionally to tune Redshift performance by identifying high-risk queries (disk and time-based) from all users and mitigating each query with their appropriate stakeholder. Designed alerting and a runbook for diagnosis to prevent future degradation to Redshift performance.

    • Diagnosed & executed various improvements to the in-house Airflow deployment to ensure reliable running nightly ETL processes with a high degree of up-time from Airflow workers

    • Coordinated upgrades to legacy Spark code Data Transformations so that they could be migrated to the most up-to-date available Databricks clusters from Amazon EMR

    • Implemented a large refactor to a legacy monolithic Airflow DAG responsible all core nightly ETL processes, resulting in minimal disruption to SLAs and improved clarity/control over all the different Data Team pipeline

    • Strengthened SLAs of datasets used by the Biz Ops team internally, to the product for powering key financial products like Cash Out lending, and data provided to key partners/integrations of the business

    • Advocated for increased monitoring & alerting of ETLs, API services, Redshift clusters with the use of tools like Datadog, Cloudwatch & Pagerduty.

    • Worked to significantly decrease pages of Data Team processes and ensure the Data Team could focus more on planned work and less on fire-fighting

    • Led planning, grooming, and other agile meetings for the Data Platform team and Data Features team when required

    • Part of the 2023 AI Hackathon Winning team. Implemented a POC of locally running LLM (using Ollama) that could ingest internal Confluence documents, Airflow logs, and Slack conversations, then provide answers via a Slack chatbot to internal users asking questions about the product, ETL processes, and data we have stored in our warehouse.

  • Senior Engineering Technical Lead - Data & Analytics Team, 

    May, 2020 - Nov, 20222 years 6 months

    Senior Engineering Technical Lead for Lever's Data & Analytics team of 6 Engineers, 2 BI Developers

    • Built a last-mile ETL pipeline to move data from Postgres to Snowflake using Kafka Connect with Debezium & Snowflake Sink Connectors.

    • Defined, and lead the team to build additional monitoring and observability for this pipeline.

    • Implemented custom transformation/merge Snowflake Tasks using Snowflake Streams to consolidate change record data arriving in Snowflake.

    • Tuned RDS to enable more efficient logical replication

    • Troubleshooted & identified resource-intensive Postgres queries which required indexes or better performing implementations.

  • Engineering Technical Lead - Data & Analytics Team, 

    Oct, 2020 - May, 20221 year 6 months

    Technical Lead for Lever's Data & Analytics team of 6 Engineers, 2 BI Developers

    • Deliver continued improvements, new data & features for the reporting ETL pipeline, self-healing service, and Data Warehouse Sync features. We also maintain the legacy reporting pipeline and solution built using MongoBI and Tableau.

    • Primarily working on an in-app Analytics product that allows customers to perform data driven recruiting with embedded Looker dashboards, export curated data sets using Looker's API, and share dashboards using Looker's API, with over 100,000 quarterly visits and growing as it becomes the new primary in-app reporting solution for our application.

    • Added the ability to Bookmark dashboards with unique filters selected for customers to easily navigate and access their favourite dashboards from within the app.

    • Designed a solution for rendering PDFs and images of dashboards using Looker's API that could be shared (directly or via a user-defined schedule) with other members of the organization that might not have immediate access to in-app analytics.

    • Added the ability to allow customers that have purchased the Data Warehouse features to provision it themselves in the application, eliminating the need for Product Support to intervene and provide access keys.

    • Monitor, debug and tune the Postgres to Snowflake solution that we have implemented for ensuring scalability so that our Looker instance used within our embedded dashboards can serve many queries very quickly using Snowflake.

    • As lead, I also own implementation specs for these projects, which involves flushing out implementation details, coordinating security reviews & peer reviews, cutting JIRA tickets, pointing them with the team, and ensuring scoped work can be delivered on time.

    • I work with product, design, and the engineering team to define requirements and understand scope as well as provide detailed estimates and technical feasibility on potential projects.

  • Software Engineer - Data & Analytics Team, 

    Apr, 2020 - Oct, 20205 months

    Powering Lever's brand-new ETL pipeline and the dashboard/BI features that utilize it.

    • Improved and maintained the brand new analytics ETL pipeline that leveraged Mongo Change Streams, Kafka & NodeJS services to transform raw application data stored in Mongo into relational data stored in a Postgres database

    • Coordinated the addition of new columns to the Postgres ETL database (designing constraints, applying required migrations) and implemented new mapping logic in the transformation services so that new data was available to be reported on as required by customers and the product. Totalling over 120 tables, 980 columns of reportable data to be used by customers to extract/export and for in-app reporting dashboards.

    • Designed a self-healing service for the analytics ETL pipeline that also used Kafka and NodeJS to 'retry' failed data transformations in the pipeline.

    • Implemented CI pipelines with Concourse for the service to run linting, automated tests, and create deployment images.

    • Defined Kubernetes deployments for the service and successfully delivered it into a production environment, healing over 70 Million documents of data in the first month and 100 Million since inception.

    • Improved to a greater than 99.25% successful transformation rate in the overall ETL pipeline since the self-healing service was introduced.

    • Exposed a UI for interacting with this service in Lever's administrative tooling front-end.

    • Wrote LookML in Looker for defining datasets that we want to provide customers for exporting.

    • Debugged and solved various issues for a Data Warehouse Sync feature that uses the Postgres ETL database to extract customer-specific data to each of their own queryable data stores. This involved lots of interaction with AWS CloudFormation, AWS Glue, AWS Athena, IAM for managing each customer's access to their parquet files in S3/Athena, as well as CloudFormation's CDK for automating the data store provisioning for each Data Warehouse customer.

  • Software Engineer - API & Integrations Team, 

    Dec, 2019 - Apr, 20204 months

    Building integrations with third-party services

    • Implemented Lever's OAuth Authentication with DocuSign to allow virtual offer/document signing through the application.

    • Tracked key success metrics of the implementation (e.g. % of accounts switching from Basic Auth to OAuth, # of errors occurring during the OAuth Flow) using Segment & Metabase with alerting/monitoring output to relevant stakeholders in Slack

    • Improved Zoom interview scheduling in Lever by adding support to include individual Zoom links for each panel of the interview in the calendar invite description.

  • Software Developer, 

    May, 2015 - Aug, 20194 years 3 months

    Klipfolio is an online dashboard platform for building powerful real-time business dashboards for your team or your clients.

    • Built component-based UIs with React, using both AMD and Common JS modular design

    • Designed a distributed, message-passing Akka microservice API for retrieving cloud user data via OAuth, JDBC, SFTP

    • Implemented an encryption solution to protect users’ third-party passwords, OAuth tokens, database and SFTP credentials

    • Created custom OAuth integrations with third party APIs like Basecamp, Shopify, iFormBuilder

    • Created a helper syntax to add dynamic hash-based authentication parameters to API requests that users define to retrieve their data

    • Participated in bi-weekly demos of new features to support, sales, marketing, QA

  • Software Developer Intern, 

    Sep, 2014 - Dec, 20143 months

    Communications Security Establishment Canada's mission is to provide and protect information of national interest through leading-edge technology.

    • Developed a mapping web app using Java, Esri JavaScript API, Dojo, jQuery

    • Redesigned Java-based Oracle database syncing/transfer services with Python, lxml, PyCrypto

    • Prototyped a native Android app using Esri Android SDK and Java

  • Software Developer Intern, 

    May, 2014 - Aug, 20143 months

    NetFore Systems is a software development and technology consulting group combining deep technology experience with a high-touch approach to help their customers reach their business goals faster.

    • Built the functionality to list, filter, and sort voters for a nationwide campaigning application used by one of Canada's major political parties with Bootstrap, FooTables, Grails

    • Developed a geocoding service with Google Maps & Address Doctor APIs that added geographic coordinates to a voter's address entered by a campaigner into the system

  • Software Developer Intern, 

    May, 2013 - Aug, 20133 months

    Versaterm delivers Records, Computer Aided Dispatch, and Mobile systems to public safety clients

    • Developed a web app for public safety official to view a summary of ongoing police calls, call status, response time, and more information using CSS, JavaScript/JQuery, Four Js Genero

    • Planned & presented technical demo for app summary to 50+ colleagues

    • Worked primarily using Vim editor on Red Hat Linux Server

  • Student, 

    Jan, 2015 - Present

    Proposed and validated a theory that user-based collaborative filtering algorithms in recommender systems can be improved by incorporating the geographic positions of users into the algorithm.

    • Transformed data of 1200 users and their top 50 music artist preferences into a canonical form that could be used in a K-Nearest Neighbours algorithm with Scikit-learn.

    • Defined a weight termed 'geographic similarity' that used geographic coordinates of two users to determine how close they were and incorporated weight to their overall similarity score.

    • Compared the algorithm augmented with geographic similarity to the original using common machine learning validation methods to show that it was 5.6% more accurate with its recommendations.

  • Creator, 

    Oct, 2014 - Jan, 20161 year 3 months

    A classroom and behavior management web application for teachers. This is a personal project I started to practice new web technologies.

    • Created a Node.js RESTful API using the Sails.js MVC framework with Access Control List policies and user authentic.ation via Passport.js. Deployed to Heroku.

    • Designed a front-end Angular.js single page application to list students, classroom incidents, and contact information. Deployed to GitHub Pages.

    • Scheduled a Python script that used Twitter Search API to generate daily leads upon launch (i.e. teachers).

  • Open Source Contributor, 

    Sep, 2015 - Present

    MarkUs (pronounced "mark us") is an open-source tool which recreates the ease and flexibility of grading assignments with pen on paper, within a web application.

    • Contributor to the open source repository on GitHub for the Ruby on Rails app

    • Wrote Ruby scripts to check for consistent code style when committing

    • Added visualizations using Chart.js and ActiveRecord Queries to provide useful statistics on Grader progress (i.e. Distribution of marks assigned by each grader, assignments completed, average annotations per marked assignment)

    • Wrote Rake scripts to bootstrap data into the application for developers to have a better sense of how a live production deployment would look

    • Participated in a code sprint at UBC in Vancouver with other contributors

Skills

Skills

  • Languages & Service Development

    Python

    TypeScript

    Node.js

    SQL

    gRPC

    Celery

    Streamlit

  • AI & Machine Learning Platforms

    LLM Platforms

    Agentic Workflows

    Machine Learning Inference

    Self-Service AI Infrastructure

    Scalable Model Serving

    AI Governance & Cost Controls

  • Data Platforms & Distributed Systems

    Apache Airflow

    Databricks

    Apache Spark

    Kafka

    Event-Driven Architecture

    ETL/ELT Pipelines

    Data Activation

    Redshift

    Snowflake

    Postgres

  • Cloud Infrastructure & DevOps

    AWS (ECS, EC2, RDS, MSK, S3, Athena, Glue, IAM, CloudFormation)

    Terraform

    Kubernetes

    Infrastructure as Code

    CI/CD

    Containerized Services

  • Observability & Reliability

    Datadog

    Amazon CloudWatch

    PagerDuty

    Monitoring & Alerting

    Incident Response

    Production Reliability

    Performance Tuning

Education

Education

  • Computer Science, Graduate Courses, Continuing Education, University of Ottawa

    Jan, 2016 - Present

    9.7/10

    CSI5149 - Probabilistic Graphical Models

    CSI5387 - Data Mining & Concept Learning

    CSI5389 - Electronic Commerce Technologies

    CSI5118 - Automated Verification & Validation of Software

    CSI7163 - Advanced Topics in Computer Systems (Multimedia Communications)

  • Computer Science, Bachelor's Degree, University of Ottawa

    Sep, 2011 - Dec, 2015

    9.9/10 Diploma Grade Point Average, 9.4/10 Cumulative GPA
Awards

Awards

  • Faculty of Engineering Plaque in Computer Science , Univeristy of Ottawa

    Awarded on: Jun 01, 2016

    Recognition of Excellence - The Faculty of Engineering plaque for the single highest standing in the bachelor in computer science.

  • Chancellor Scholarship Finalist , Univeristy of Ottawa

    Awarded on: Sep 05, 2011

    Students with greater than 92% entering average demonstrating leadership qualities and a commitment to academic and extracurricular activities.

  • Nortel Networks Admission Scholarship , Univeristy of Ottawa

    Awarded on: Sep 05, 2011

    Greater than 95% entering average.

  • Merit scholarship , Univeristy of Ottawa

    Awarded on: Sep 05, 2012

    Outstanding academic success in previous academic year. Received every year of undergrad.

  • Governor General’s Academic Bronze Medal , The Governor General of Canada

    Awarded on: Jun 15, 2011

    Awarded to the student who achieves the highest average upon graduation from a secondary school.

Volunteer Work

Volunteer Work

  • Laborer, Oneness

    Jan, 2010 - May, 2012

    Oneness is an organization of Canadians and Guatemalans working together to improve educational opportunities.

    • Organized fundraising initiatives to bring basic items to families in Guatamela

    • Laid cement floors in houses with dirt floors

    • Assisted in building a local feeding/community centre

    • Presented at various schools in my own hometown to share my experiences