HBase In Action - Chapter 04: HBase table design Learning HBase, Real-time Access to Your Big Data, Data Manipulation at Scale, Big Data, Text Mining, HBase, Deploying HBase

1. CHAPTER 04: HBASE TABLE DESIGN
HBase IN ACTION
by Nick Dimiduk et. al.

2. Overview: HBase table design
 HBase schema design concepts
 Mapping relational modeling knowledge to the
HBase world
 Advanced table definition parameters
 HBase Filters to optimize read performance

3. 4.1 How to approach schema design
 When we say schema, we include the following
considerations:
 How many column families should the table have?
 What data goes into what column family?
 How many columns should be in each column family?
 What should the column names be?
 What information should go into the cells?
 How many versions should be stored for each cell?
 What should the rowkey structure be, and what should it
contain?

4. Hbase Course
 Data Manipulation at Scale: Systems and
Algorithms
 Using HBase for Real-time Access to Your Big
Data

5. 4.1.1 Modeling for the questions
 A table store data about what users a particular user
follows, support
 read the entire list of users,
 and query for the presence of a specific user in that list

6. 4.1.1 Modeling for the questions (cont.)

7. 4.1.1 Modeling for the questions (cont.)
 Thinking further along those lines, you can come up
with the following questions:
1. Whom does TheFakeMT follow?
2. Does TheFakeMT follow TheRealMT?
3. Who follows TheFakeMT?
4. Does TheRealMT follow TheFakeMT?

8. 4.1.2 Defining requirements: more work up front
always pays
 From the perspective of TwitBase, you expect data to
be written to HBase when the following things
happen:
 A user follows someone
 A user unfollows someone they were following

9. 4.1.2 Defining requirements: more work up front
always pays (cont.)

10. 4.1.2 Defining requirements: more work up front
always pays (cont.)
 What is different from design tables in relational
systems and tables in HBase?

11. 4.1.3 Modeling for even distribution of data and
load

12. 4.1.3 Modeling for even distribution of data and
load (cont.)

13. 4.1.4 Targeted data access
 Only the keys are indexed in HBase tables.
 There are two ways to retrieve data from a table: Get and
Scan.
 HBase tables are flexible, and you can store anything in
the form of byte[].
 Store everything with similar access patterns in the same
column family.
 Indexing is done on the Key portion of the KeyValue
objects, consisting of the rowkey, qualifier, and
timestamp in that order.
 Tall tables can potentially allow you to move toward O(1)
operations, but you trade atomicity

14. 4.1.4 Targeted data access (cont.)
 De-normalizing is the way to go when designing HBase
schemas.
 Think how you can accomplish your access patterns in
single API calls rather than multiple API calls.
 Hashing allows for fixed-length keys and better
distribution but takes away ordering.
 Column qualifiers can be used to store data, just like
cells.
 The length of column qualifiers impacts the storage
footprint because you can put data in them.
 The length of the column family name impacts the size of
data sent over the wire to the client (in KeyValue
objects).

15. 4.2 De-normalization is the word in HBase land
 One of the key concepts when designing HBase
tables is de-normalization.

16. 4.3 Heterogeneous data in the same table
 HBase schemas are flexible, and you’ll use that
flexibility now to avoid doing scans every time you
want a list of followers for a given user.
 Isolate different access patterns as much as possible.
 The way to improve the load distribution in this case
is to have separate tables for the two types of
relationships you want to store.

17. 4.4 Rowkey design strategies
 In designing HBase tables, the rowkey is the single
most important thing.
 Your rowkeys determine the performance you get
while interacting with HBase tables.
 Unlike relational databases, where you can index on
multiple columns, Hbase indexes only on the key;

18. 4.5 I/O considerations
 The sorted nature of HBase tables can turn out to be
a great thing for your application—or not
 Optimized for writes
 HASHING
 SALTING
 Optimized for reads
 Cardinality and rowkey structure

19. 4.6 From relational to non-relational
 There is no simple way to map your relational
database knowledge to HBase. It’s a different
paradigm of thinking
 Things don’t necessarily map 1:1, and these concepts
are evolving and being defined as the adoption of
NoSQL systems increases.

20. 4.6.1 Some basic concepts
 ENTITIES
 These map to tables.
 In both relational databases and HBase, the default container
for an entity is a table, and each row in the table should
represent one instance of that entity.
 ATTRIBUTES
 These map to columns.
 Identifying attribute: This is the attribute that uniquely
identifies exactly one instance of an entity (that is, one row).
 Non-identifying attribute: Non-identifying attributes are
easier to map.

21. 4.6.1 Some basic concepts (cont.)
 RELATIONSHIPS
 These map to foreign-key relationships.
 There is no direct mapping of these in HBase, and often it
comes down to denormalizing the data.
 HBase, not having any built-in joins or constraints, has little
use for explicit relationships.

22. 4.6.2 Nested entities
 In Hbase, the columns (also known as column
qualifiers) aren’t predefined at design time.

23. 4.6.2 Nested entities (cont.)
 it’s possible to model it in HBase
as a single row.
 There are some limitations to
this
 this technique only works to one
level deep: your nested entities can’t
themselves have nested entities.
 it’s not as efficient to access an
individual value stored as a nested
column qualifier inside a row

24. 4.6.3 Some things don’t map
 COLUMN FAMILIES
 (LACK OF) INDEXES
 VERSIONING

25. 4.7 Advanced column family configurations
 HBase has a few advanced features that you can use
when designing your tables.
 Configurable block size
 hbase(main):002:0> create 'mytable', {NAME => 'colfam1',
BLOCKSIZE => '65536'}
 Block cache
 hbase(main):002:0> create 'mytable', {NAME => 'colfam1',
BLOCKCACHE => 'false’}
 Aggressive caching
 hbase(main):002:0> create 'mytable', {NAME => 'colfam1',
IN_MEMORY => 'true'}

26. 4.7 Advanced column family configurations
(cont.)
 Bloom filters
 hbase(main):007:0> create 'mytable', {NAME => 'colfam1',
BLOOMFILTER => 'ROWCOL'}
 The default value for the BLOOMFILTER parameter is NONE.
 TTL
 hbase(main):002:0> create 'mytable', {NAME => 'colfam1', TTL
=> '18000'}
 Compression
 hbase(main):002:0> create 'mytable', {NAME => 'colfam1',
COMPRESSION => 'SNAPPY'}
 Cell versioning
 hbase(main):002:0> create 'mytable', {NAME => 'colfam1',
VERSIONS => 1}

27. 4.8 Filtering data
 Filters are a powerful feature that can come in handy
in such cases.
 HBase provides an API you can use to implement
custom filters.

28. 4.8.1 Implementing a filter
 Implement custom filter by extending FilterBase
abstract class
 The filtering logic goes in the filterKeyValue(..) method
 To install custom filters
 have to compile them into a JAR and put them in the HBase
classpath so they get picked up by the RegionServers at startup
time.
 To compile the JAR, in the top-level directory of the project, do
the following:
 mvn install
 cp target/twitbase-1.0.0.jar /my/folder/

29. 4.8.2 Prebundled filters
 ROWFILTER
 PREFIXFILTER
 QUALIFIERFILTER
 VALUEFILTER
 TIMESTAMPFILTER
 FILTERLIST

30. Hbase Course
 Data Manipulation at Scale: Systems and
Algorithms
 Using HBase for Real-time Access to Your Big
Data

31. 4.9 Summary
 It’s about the questions, not the relationships.
 Design is never finished.
 Scale is a first-class entity.
 Every dimension is an opportunity.