창언 정

5 Followers

Goal is Fullstack Data Engineer I have been data engineer within Telecoms, Finance, E-commerce and have experience of various Data Warehouse tools like ETL, BI and now mainly focus on ETL system on various System(Legacy Database, Newly adopted Hadoop/Cloud System) and Data(RDB, Files(WebLog), APIs) I have built ETL, BI system from the beginning within few people(2~3) and managed bigger team doing data processing, BI, analysis. Specialties : SQL(procedure), ETL(Talend, SSIS, DataStage, Informatica), BI/Report(MSTR, Cognos, Tableau, SSRS), Databases(Oracle, PDA(Netezza), MySQL, Hive), Programming(Java, C, Scala) and I have specialties in E-commerce Data Warehouse. more than 8 years of wor...

data platform kubernetes eks spark python spark thrift server spark on kubernetes discovery data k8s aws metadata amundsen emr data pipeline 직방 etl 쿠버네티스 glue airflow

Activity
About

창언 정

Presentations

Data platform data pipeline(Airflow, Kubernetes)

data platform on kubernetes

Transfer to kubernetes data platform from EMR

Data discovery & metadata management (amundsen installation)

Lessons learned from operating Data Platform on Kubernetes(EKS)

spark database Service

Likes

Change Data Capture to Data Lakes Using Apache Pulsar and Apache Hudi - Pulsar Summit NA 2021

Pulsar in the Lakehouse: Overview of Apache Pulsar and Delta Lake Connector - Pulsar Summit Asia 2021

Elastic Stack 을 이용한 게임 서비스 통합 로깅 플랫폼 - elastic{on} 2019 Seoul

Democratizing Data at Airbnb

Airflow Summit 2020 - Migrating airflow based spark jobs to kubernetes - the native way

data platform on kubernetes

Apache Airflow in Production

Airflow at lyft

Apache Ignite vs Alluxio: Memory Speed Big Data Analytics

OLAP for Big Data (Druid vs Apache Kylin vs Apache Lens)

Netflix Data Engineering @ Uber Engineering Meetup

Hive Bucketing in Apache Spark

2017 AWS DB Day | Amazon Athena 서비스 최신 기능 소개

Handlebars

Docker란 무엇인가? : Docker 기본 사용법

Spark Job Server and Spark as a Query Engine (Spark Meetup 5/14)

Apache Spark Architecture

Stanford CS347 Guest Lecture: Apache Spark

DMBOK 2.0 and other frameworks including TOGAF & COBIT - keynote from DAMA Australia November 2013

Introduction to Apache Sqoop