Slides for the my Droidcon NYC 2015 talk about Realm. Why a new database is needed on mobile platforms and how we build it.

1. Realm
Building a mobile database
Christian Melchior
@chrmelchior

2. Why a new database?

3. Once upon a time

4. Once upon a time

5. BCNF
Once upon a time

6. Once upon a time

7. m:n?
Once upon a time

8. Once upon a time

9. SELECT owner.name, dog.name, city.name
FROM owner
INNER JOIN dog ON owner.dog_id = dog.id
INNER JOIN city ON owner.city_id = city.id
WHERE owner.name = 'Frank'
Once upon a time

10. String query = "SELECT " + Owner.NAME + ", " + Dog.NAME + ", " + City.NAME
+ " FROM " + Owner.TABLE_NAME
+ " INNER JOIN " + Dog.TABLE_NAME + " ON " + Owner.DOG_ID + " = " + Dog.ID
+ " INNER JOIN " + City.TABLE_NAME + " ON " + Owner.CITY_ID + " = " + City.ID
+ " WHERE " + Owner.NAME = "'" + escape(queryName) + "'";
Once upon a time

12. “Lets use an ORM”
Abstract the problem away

13. –Joel Spolsky
“All non-trivial abstractions, to some degree,
are leaky.”

14. Why a new database?
• ActiveRecord
• Androrm
• Blurb
• Cupboard
• DBFlow
• DBQuery
• DBTools
• EasyliteOrm
• GreenDAO
• Ollie
• Orman
• OrmLite
• Persistence
• RushORM
• Shillelagh
• Sprinkles
• SquiDB
• SugarORM

15. What is Realm?

16. Zero-copy object store
Designed for mobile devices

17. Object Store
A
B C
D E F
G
VS

18. A
B C
D E F
G
Realm.getA().getC().getF()
Object Store references

19. x
z
y
x y z
SELECT table1.x, table2.y, table3.z
FROM table1
INNER JOIN table2 ON table1.table2_id = table1.id
INNER JOIN table3 ON table1.table3_id = table3.id
References in SQL

20. What is zero-copy?
• CursorAdapter
• FlatBuffers
public class MyCursorAdapter extends CursorAdapter {
// ...
@Override
public void bindView(View view, Context context, Cursor cursor) {
ViewHolder holder = (ViewHolder) view.getTag();
String foo = cursor.getString(cursor.getColumnIndexOrThrow("foo"));
int bar = cursor.getInt(cursor.getColumnIndexOrThrow("bar"));
holder.fooView.setText(foo);
holder.barView.setText(Integer.toString(bar));
}
}

21. What is zero-copy?
• CursorAdapter
• FlatBuffers
public long id() {
int o = __offset(4);
return o != 0 ? bb.getLong(o + bb_pos) : 0;
}
public String name() {
int o = __offset(6);
return o != 0 ? __string(o + bb_pos) : null;
}
ByteBuffer bb = ByteBuffer.wrap(bytes);
MyObject obj = MyObject.getRootAsMyObject(bb);
long id = obj.id();
String name = obj.name();

22. Zero-copy in Realm
Person
{
• name
=
Tommy
• age
=
8
• dog
=
{
• name
=
Lassie
}
}
Person
{
• name
=
Tommy
• age
=
8
• dog
=
{
• name
=
Lassie
}
}
Person
{
• name
=
Tommy
• age
=
8
• dog
=
{
• name
=
Lassie
}
}
PersonProxy
{
• name
• age
• dog
}
PersonProxy
{
• name
• age
• dog
}
Person
{
• name
=
Tommy
• age
=
8
• dog
=
{
• name
=
Lassie
}
}
Realm
ORM Realm
SQLite

23. Zero-copy in Realm
// Objects
Person javaPerson = new Person(); // Standard Java POJO
Person proxyPerson = realm.copyToRealm(javaPerson); // Convert POJO to Proxy
Person proxyPerson = realm.createObject(Person.class); // Proxy
// Query results are lazy
RealmResults<Person> queryResults = realm.allObjects(Person.class);
queryResults.get(0) != queryResults.get(0); // Different Java objects
queryResults.get(0).equals(queryResults.get(0)); // Same data

24. Designed for mobile
• Memory-mapped files
• C++ core
• B+ trees
• Column-oriented
• Bit-packing
• Minimize I/O overhead
• Cross-platform
• Faster reads and writes
• Queries > writes
• Smaller file size

25. Benchmarks
http://static.realm.io/downloads/java/android-benchmark.zip
1.09%
2.26%
3.79%
4.55%
22.85%
13.37%
1% 1% 1% 1% 1% 1%
0%
5%
10%
15%
20%
25%
BatchWrite% SimpleWrite% SimpleQuery% FullScan% Sum% Count%
Speedup&vs.&SQLite&
Tests&/&1000&objects&
Realm%
SQLite%

26. Implementing
Realm Android

27. Architecture
JNI
Realm Internal API
Realm Core
Realm Object Store API

28. Zero copy
public class Pojo {
private String foo;
private int bar;
public Pojo() {
}
public String getFoo() {
return foo;
}
public void setFoo(String foo) {
this.foo = foo;
}
public int getBar() {
return bar;
}
public void setBar(int bar) {
this.bar = bar;
}
}

29. Annotation processor
public class Pojo extends RealmObject {
private String foo;
private int bar;
public Pojo(String foo, int bar) {
this.foo = foo;
this.bar = bar;
}
public String getFoo() {
return foo;
}
public void setFoo(String foo) {
this.foo = foo;
}
public int getBar() {
return bar;
}
public void setBar(int bar) {
this.bar = bar;
}
}
@RealmClass
public abstract class RealmObject {
protected Row row;
protected Realm realm;
// ...
}
+ =

30. Proxy classes
public class PojoRealmProxy extends Pojo
implements RealmObjectProxy {
private static long INDEX_FOO;
private static long INDEX_BAR;
@Override
public String getFoo() {
realm.checkIfValid();
return (java.lang.String) row.getString(INDEX_FOO);
}
@Override
public void setFoo(String value) {
realm.checkIfValid();
row.setString(INDEX_FOO, (String) value);
}
@Override
public int getBar() {
realm.checkIfValid();
return (int) row.getLong(INDEX_BAR);
}
@Override
public void setBar(int value) {
realm.checkIfValid();
row.setLong(INDEX_BAR, (long) value);
}
@Override
public String toString() {
// ..
}
@Override
public int hashCode() {
// ...
}
@Override
public boolean equals(Object o) {
// ...
}
// Static helper methods ...
}

31. public class NormalObject extends RealmObject {
public String foo;
public int bar;
@Override
public String toString() {
return "[" + foo + ";" + bar + "]";
}
}
NormalObject obj = new NormalObject();
obj.foo = "DroidCon";
Standard Java objects

32. Byte code manipulation
• Custom Gradle plugin
• Morpheus
• JavaAssist
Compile *.java
Create proxy classes
Bytecode: Replace field
access with accessors
Succes!
Dex

33. public class Pojo extends RealmObject {
public String foo;
public int bar;
@Override
public String toString() {
return "[" + foo + ";" + bar + "]";
}
public String realm$$getFoo() {
return foo;
}
public void realm$$setFoo(String s) {
foo = s;
}
public int realm$$getBar() {
return bar;
}
public int realm$$setBar(int i) {
return bar = i;
}
}
Byte code manipulation

34. public class Pojo extends RealmObject {
public String foo;
public int bar;
@Override
public String toString() {
return "[" + realm$$getFoo() + ";" + realm$$getBar() + "]";
}
public String realm$$getFoo() {
return foo;
}
public void realm$$setFoo(String s) {
foo = s;
}
public int realm$$getBar() {
return bar;
}
public int realm$$setBar(int i) {
return bar = i;
}
}
Byte code manipulation
Pojo pojo = new Pojo();
pojo.realm$$setFoo(“DroidCon”);
Join points

35. Method count
• Realm: ~1750
• Model class: +2x no. of fields
• Proxy class: + 2x no. of fields + 11
• Total method count: “it depends” (~2000)
Picasso: ~600
GSON: ~950
OrmLite: ~2400
RxJava: ~3500
Support library: ~12300

36. What does consistency
mean?
• Database/ACID: The consistency property
ensures that any transaction will bring the
database from one valid state to another
• UI consistency: Visible data should represent one
valid state.

37. UI Consistency
Fragment 1
Fragment 2
Fragment 1
Fragment 2
☺ ☹

38. Consistency today
Fragment 1
Fragment 2
• Loader
• Content URI
• ContentObserver
• ContentProvider

39. Consistency today
Fragment 1
Fragment 2
ContentProvider
1
2
2
3

41. • Multiversion concurrency control
• Multiple read transactions
• Data is versioned
• One write transaction
R
A B
C D E
MVCC
R
A B
X D E
v1
v2

42. Consistency in Realm
Background threadUI thread
queue.next();
queue.next();
thread.start();

43. Consistency in Realm
Background threadUI thread
queue.next();
queue.next();
thread.start();
Realm realm = Realm.getDefaultInstance();
Realm realm = Realm.getDefaultInstance();

44. Consistency in Realm
Background threadUI thread
queue.next();
queue.next();
thread.start();
realm.close();
realm.close();

45. realm.executeTransaction(new Realm.Transaction() {
@Override
public void execute(Realm realm) {
//...
}
});
Consistency in Realm
Background threadUI thread
queue.next();
queue.next();
thread.start();

46. Consistency in Realm
Background threadUI thread
queue.next();
queue.next();
thread.start();
@OnClick
public void buttonClicked() {
Person p = realm.where(Person.class).findFirst();
int ager = p.getAge();
String name = p.getName();
}
☹

47. Consistency in Realm
Background threadUI thread
queue.next(); thread.start();
new RealmChangeListener() {
@Override
public void onChange() {
//...
}
}
handler.sendEmptyMessage(REALM_CHANGED);

48. Consistency with Realm
Fragment 1
Fragment 2
Fragment 1
Fragment 2
new RealmChangeListener() {
@Override
public void onChange() {
invalidateViews();
}
}

49. Current status

50. • Moving towards 1.0
• Used in well more than 500M installations.
• Still need to solve hard problems: Interprocess
cursors, move objects across threads, Parcelable.
Realm today

51. Questions?
@chrmelchior
cm@realm.io
We are hiring
https://realm.io/jobs/