gproto
/
/post
Ivan Ega Pratama
Excel on make things swift and efficient
Personal Info
  • ivan[dot]ega[dot]p[at]gmail[dot]com
  • <Contact Me>
  • <Contact Me>
Profiles
Experiences
  • Senior Data Engineer
    2022 Jan - Present
    Tokopedia
    https://tokopedia.com/
    • Design, oversee, and develop internal framework that manage the data ingestion process from and to various data sources and sinks for thousands of data resources that integrates within minutes
    • Build a custom framework on top of Apache Airflow to generate and manage the DAG automatically for various tasks e.g. RDBMS to BigQuery, BigQuery to BigQuery, BigQuery to BigTable, custom Docker based tasks, etc.
    • Develop high throughput streaming pipelines for Clickstream, Change Data Capture (CDC), internal service events, etc. that can handle hundred of thousands requests per second with low cost using custom data encoding format
    • Build custom data quality framework that runs on BigQuery
    • Design and plan various custom data related solutions with its availability goal (SLO), migration plan, etc.
    Airflow
    Beam
    BigQuery
    Debezium
    Docker
    Flink
    GCP
    Go
    Java
    K8S
    Python
    Spark
  • Tech Advisor | Individual Contributor
    2020 Aug - Present
    MUSA Green
    https://musagreen.com/
    • Creating Go-based IoT Firmware for data acquisition using Modbus protocol and send the data to Cloud IoT Core, used by B2B users
    • Creating cost-effective streaming data ingestion process using Cloud Run
    • Creating auto email notification for data reporting in case there are missing data and the measurement value exceeds standard threshold
    BigQuery
    Cloud Run
    Firebase
    GCP
    Go
    IoT
    Modbus
  • Data Team Lead
    2021 Apr - 2023 Jul
    Lembaga Kerjasama Fakultas Teknik (LKFT) UGM
    https://lk.ft.ugm.ac.id/
    • Leading an AI project with the state-owned insurance company
    • Creating data architecture/pipeline diagram
    • Researching technical stacks that suitable for the project use case
    • Creating proof of concept and production grade microservices
    • Handling microservices deployment related tasks to Kubernetes cluster using Helm
    AI
    Airflow
    BigQuery
    Docker
    GCP
    Go
    K8S
    Python
    Spark
  • Data Engineer
    2020 Jun - 2021 Dec
    Mars Indonesia Digital
    https://marsdigital.id
    • Integrating client data across many platforms by employing web scraper in Python or Go, REST API call in Python, copy files from FTP server, getting data from Google Sheets, and custom database dump program in Go
    • Creating ETL in Python, Dataprep, and Spark for data transformation
    • Inventing fuzzy data matcher to integrate data across platforms with non-existing primary keys using available personally identifiable information
    • Optimizing the operational cost by setting up an auto-scaled Kubernetes cluster (GKE) that can scale down to zero
    • Deploying the data schedule pipeline and create custom operators for Apache Airflow
    • Optimizing the workflow by setting up CI/CD for auto Docker image creation and deployment
    • Inventing circuit breaker script to handle data schema changes and notify the stakeholder if only some direct action needed
    • Implementing slowly changing dimension script to be executed in BigQuery, so that all of the historical data is preserved
    • Handling deployment and Cloud Ops related processes such as setting up the VPN and configuring Tableau Server
    • Doing ad-hoc exploration and analysis based on new project leads requirements
    Airflow
    BigQuery
    Dataprep
    Docker
    GCP
    Go
    Kubernetes
    Python
    Spark
Achievements
  • 11th Place of 2019 American Astronautical Society CanSat Competition
    2019 Jun
    American Astronautical Society (Stephenville, Texas, USA)
    http://cansatcompetition.com/
    • Leading the team under the flag of Gadjah Mada Aerospace Team in doing research about designing and manufacturing rocket payload based on competition requirement of NASA backed aerospace competition
    • Designing and manufacturing high-performance antenna to solve the data communication problem
    • Doing quality assurance for the rocket payload manufactured to make sure it complies with competition requirements
    Arduino
    Docker
    Matplotlib
    Node.js
    Python
    Raspberry Pi
    Xbee
  • 3rd Place of 2017 Komurindo-Kombat
    2017 Aug
    Lembaga Penerbangan dan Antariksa Nasional (Garut, West Java, Indonesia)
    https://komurindo-kombat.lapan.go.id/
    • Acting as telecommunication engineer in Gadjah Mada Aerospace Team
    • Designing and manufacturing high-performance antenna to solve the data communication problem
    • Creating image chunking and image storage algorithm that can be deployed on memory limited microcontroller
    Arduino
    Electron.js
    Node.js
    Xbee
Education
  • Universitas Gadjah Mada
    2015 Aug - 2019 Oct
    Engineering Physics, Faculty of Engineering
    GPA 3.64/4.00