Software Development Engineer, Amazon Web Services (Aurora Serverless team)
Mar, 2021 - May, 20221 year 1 month
Aurora Serverless provides managed replicated MySQL and PostgreSQL services that can vertically autoscale, restart, and patch, all without dropping connections. I worked on the Control Plane team.
Enhancement, maintenance, and operation of a system managing hundreds of thousands of MySQL and PostgreSQL databases. It is a distributed system with five major service node types, in addition to many AWS services.
Contributed to design of the next generation "heat management" optimization for Aurora Serverless V2, which is the system that decides where to place Aurora instances on the multi tenant oversubscribed hosts, and what live migrations to do in order to alleviate overloaded hosts. It is a combination of ML for predicting instance resource needs (CPU, memory, network IO) and stochastic/robust mixed integer programming for decisions (with work on a RL agent on the horizon).
CI/CD, Unit and integration testing, design docs, code reviews
Java for main application. React, Python, shell scripting for dev and ops tools. Ruby for infrastructure-as-code.
Regular use of many AWS technologies including Lambda, S3, DynamoDB, CloudWatch/CloudFormation/CodeDeploy, various RDS services, SWF, Step Functions.