Duo Zhang and Liangliang He (Xiaomi)
In this session, we’ll discuss the various practices around HBase in use at Xiaomi, including those relating to HA, tiered compaction, multi-tenancy, and failover across data centers.
AI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
Apache HBase Improvements and Practices at Xiaomi
1. Some improvements and practices of
HBase at Xiaomi
Duo Zhang, Liangliang He
{zhangduo, heliangliang}@xiaomi.com
........ ..... ................. ................. ................. .... .... . .... ........ .
11. .....
.
....
.
....
.
.....
.
....
.
....
.
....
.
.....
.
....
.
....
.
....
.
.....
.
....
.
....
.
....
.
.....
.
....
.
.....
.
....
.
....
.
Per-CF Flush
▶ Why we must flush all families?
▶ Our sequence id accounting is per region.
▶ Can not know the lowest unflushed sequence id.
▶ Track sequence id per store, i.e., per family
▶ Map<RegionName, SequenceId> to
Map<RegionName, Map<FamilyName, SequenceId>>
▶ SequenceId map in WAL implementation
▶ FlushedSequenceId in ServerManager at master
▶ Report a Map of flushed sequence id to master(Thanks protobuf for
compatibility)
▶ Skip WAL cells per store when replaying
▶ FlushPolicy
▶ FlushAllStoresPolicy
▶ FlushLargeStoresPolicy
10 / 38