5. http://clickhouse.yandex
• Developed by Yandex for Yandex.Metrica
• Yandex (NASDAQ: YNDX) – “Russian Google” (50% market share in
search, 50+ b2b and b2c products)
• Yandex.Metrica – world 2nd largest web analytics platform
• Open Source since June 2016 (Apache 2.0 license)
• 200+ companies using in production today
• Several hundred experimenting, doing POC etc.
• Dozens of contributors to the source code
14. Time Series Benchmarks
• https://github.com/timescale/tsbs
• Benchmark suite to automate testing
• Loads 103M rows, 10 metrics per row
• Runs 15 queries, 1000 runs each in 8 parallel threads
• Supports TimescaleDB, InfluxDB, Cassandra, MongoDB and
ClickHouse (Altinity PR is submitted)
25. with (select groupArray(C) from C) as Ca
select id,
groupArray(S) Sa, groupArray(V) Va, groupArray(D) Da, groupArray(P) Pa,
arrayMap(c -> arrayFirstIndex(s -> s > c, Sa)-1, Ca) Ka,
arrayMap((c,k) -> Va[k] + (Va[k+1] - Va[k])/(Sa[k+1] - Sa[k])*(c-
Sa[k]),Ca,Ka) Ta,
arrayMap(s -> arrayFirstIndex(c -> c>s, Ca)>0 ? arrayFirstIndex(c ->
c>s, Ca)-1 : toInt32(length(Ca)), Sa) Ja,
arrayMap(i -> Ta[i], Ja) Ra,
arrayMap((v,r) -> v - r, Va, Ra) ARa,
arraySum((x,y,z) -> x*y*z, ARa, Da, Pa) result
from T group by id
26. What’s new in 2019
… Alexey Milovidov will disclose latest coolest features in 15 minutes
27. More user friendly than ever!
• GDPR compliance – thanks to UPDATE/DELETE
• Easier BI integration – thanks to SQL compatibility changes and
improvements in ODBC driver
• Easier cluster operation – thanks to clickhouse-copier, distributed DDL and
upcoming Altinity ClickHouse operator for Kubernetes
• Easier integration with other systems. Thanks to:
• HTTP/TCP protocols
• Table functions to access external data
• Kafka storage engine
• Logs integration with Logstash, ClickTail and other tools
• Integration wth MySQL, PostgreSQL
28. Integrates with MySQL
• mysql() table function
• MySQL table engine
• MySQL external dictionaries
• ProxySQL
• Binary log replication with clickhouse-mysql
29. mysql() table function
select * from mysql('host:port', database, 'table', 'user', 'password');
https://www.altinity.com/blog/2018/2/12/aggregate-mysql-data-at-high-speed-with-clickhouse
• Easiest and fastest way to get data from MySQL
• Load to CH table and run queries much faster
30. MySQL table engine
CREATE TABLE …
Engine = MySQL('host:port', 'database', 'table', 'user',
'password'[, replace_query, 'on_duplicate_clause']);
• SELECTs and INSERTs!
• No caching, data is queried from th remote server
https://clickhouse.yandex/docs/en/operations/table_engines/mysql/
31. MySQL external dictionaries
• Makes data from mysql database accessible in ClickHouse queries
• Stores in memory
• Updates when the source data changes
SELECT dictGet(‘dim_geo’, ‘country_name’, geo_key)
country_name,sum(imps)
FROM T
GROUP BY country_name;
32. Binary log replication from MySQL
to ClickHouse
MySQL
clickhouse-mysql
Queries
Source Data
See details at:
https://www.altinity.com/blog/2018/6/30/realtime-mysql-clickhouse-replication-in-practice
33. Integrates with PostgreSQL
• odbc() table function
• ODBC engine
• ODBC dictionaries
• PostgreSQL foreign data wrapper – clickhouse_fwd by Percona
• pg2ch -- binary log replication, Murat Kibilov will present later today
35. $ kubectl -n test apply -f docs/examples/hello-kubernetes.yaml
clickhouseinstallation.clickhouse.altinity.com/hello-kubernetes created
$ kubectl -n test get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
AGE
chi-a9cffb-347e-0-0 ClusterIP None <none>
8123/TCP,9000/TCP,9009/TCP 7s
chi-a9cffb-347e-1-0 ClusterIP None <none>
8123/TCP,9000/TCP,9009/TCP 7s
chi-a9cffb-347e-2-0 ClusterIP None <none>
8123/TCP,9000/TCP,9009/TCP 7s
clickhouse-example-02 LoadBalancer 10.98.156.78 <pending>
8123:30703/TCP,9000:30348/TCP 7s
36. $ docker run -it yandex/clickhouse-client clickhouse-client -h 10.98.156.78
ClickHouse client version 19.1.14.
Connecting to 10.98.156.78:9000.
Connected to ClickHouse server version 19.1.14 revision 54413.
chi-a9cffb-347e-1-0-0.chi-a9cffb-347e-1-0.test.svc.cluster.local :)
39. …coming soon
• Managing persistent volumes to be used for ClickHouse data
• Configuring pod deployment (pod templates, affinity rules and so on)
• Creating replicated tables
• Managing users/profiles configuration
• Exporting ClickHouse metrics to Prometheus
• Handling ClickHouse version upgrades
… and more
40. ClickHouse Today
• Mature Analytic DBMS. Proven by many companies
• Almost 3+ years in Open Source
• Constantly improves
• Solid community
• Emerging eco-system
• Support from Altinity