avatar

Ankush Singh
Senior Data Engineer

About

Senior Spark Big Data Developer with 6 years of experience in Kafka, Spark / Scala, Java and Airflow with knowledge of HDFS, Hive, Python, PostgreSQL. Seeking Roles in Tech Lead (Big Data)

Work Experience

The Walt Disney Company
https://thewaltdisneycompany.com/
December 2018 – Present
Senior Data Engineer
The project's objective was to measure the viewership increase/drop across India for Star Network and its competitors. We worked in a team of 3, to develop and setup ETL pipeline scalable and optimised to ingested raw events of user viewing pattern. Proper analytics queries was developed along with tables and excel reports were shared to Business Team.
Highlights
  • ELT Pipeline. Airflow, Spark, Kafka
  • Spark Job
  • SQL for business analytics
  • AWS(Amazon Web Services)
  • Kubernetes
  • Jenkins, Github Pipeline
  • Git
Happiest Minds Technologies
https://www.happiestminds.com/
October 2016 – November 2018
Big Data Developer
The project's objective was to evalute/detect anamoly in e-commerce product data. We developed serveral algorithms to find anomaly/inconsistency in data which include text as well as image. 6 algorithms were developed by me.
Highlights
  • ELT Pipeline. Oozie + Spark
  • Spark Jobs with Image processing
  • Scikit + Pillow + OpenCV for Image Processing
  • ML for text analysis
  • Oozie

Awards

  • July 2018

    The Happiest Minds

    2nd Spot Award

    Recognition for solving problems in Computer Vision & Image Recognition area and proposing a complete solution.

  • December 2016

    The Happiest Minds

    1st Spot Award

    Recognition for solving problems in Computer Vision & Image Recognition area.

  • October 2017

    The Happiest Minds

    Runner Up in Blitz 2016

    Tech Competition with organisation

Projects

  • Redis Cache Management System: Cache Management System with support to fetch cache from multiple source, in seamless way.
  • Athena Query Scheduler: Athena Query Scheduler was developed, to schedule production query on priority basis and uat and dev queries were scheduled. This project was developed on pub-sub design and wighted scheduling was adopted.
  • User Access Management: A sync system was developed in Akka Streaming and Akka Http, which used to get all the delta changes of new/old employees and refreshed the snapshot maintained for an application called PRISM.

Contact

Bangalore, Karnataka 560093 IN
ankush.singh.2250@gmail.com
https://registry.jsonresume.org/ghstrider
LinkedIn
ankush-singh-20
GitHub
ghstrider

Education

  • 2012 2016

    Centurion University of Technology and Management

    http://cutm.in/

    Bachelor

    Computer Science and Engineering

    Grade: 9.1

    Courses
    • Basic SQL
    • Operating Systems
    • Discrete Mathematics

Skills

Spark Senior
Scala Python Spark ML
Airflow Master
Python
Scala Senior
Functional Programming Cats Cats-Effect Tapir Http4s ScalaJS Scalanative Typelevel Scala Library Multithreaded Programming
Python Senior
Flask FastAPI Swagger Boto, Awswrangler Airflow plugin

Interests

Functional Programming
Category Theory Monads Free Monads Domain Driven Design Property Based Testing