170517 damien gérard framework facebook

Introduction Server-side scripting Client-side scripting Thrift Databases
Facebook’s open-source contributions
Geeks Anonymes
Damien Gerard
May 17th, 2017

Once a pony time. . .
The“face book”was the official printed book or web directory
consisting of individuals’ photographs and names, given to students.
On October 23, 2003 Harvard student Zuckerberg launches
Facemash.com: rate other people’s attractiveness. Shut down within
days. Zuckerberg faced expulsion for breaching security, violating
copyrights and individual privacy because he stole the student pictures
from the face book to populate Facemash.
On February 4, 2004 Zuckerberg and fellow Harvard students launch
TheFacebook.com, an unofficial online face book of Harvard
university students, with no material stolen from Harvard.
Users could search for other people and had privacy options to
prevent e.g. their name from being searched, to prevent student
outrage as for Facemash.
After one month, half of the Harvard undergraduate students had a
profile.

Once a pony time. . .
TheFacebook was promptly extended to other Boston universities and
eventually all US universities.
It dropped the“The”and became“Facebook”in August 2005.
Worldwide universities spread.
In September 2006, network extented beyond educational institutions
to anyone aged 13 with an email address.
On May 4th, 2017 Facebook reaches 2bn users.

Server-side scripting language – the choice of PHP
FB uses PHP as the server-side scripting language to generate HTML.
Despite its ﬂaws and slowness, PHP is still by far the most used
among server-side scripting languages.
Nowadays, viable alternatives exist, but at the time, PHP was a good
choice for FB.
When FB faced scaling issues, their PHP codebase was already
several millions lines of code long.
FB kept and improved PHP instead of trying to rewrite their whole
codebase into another language.

HPHPc – PHP-to-C++ transpiler
The default PHP execution engine is Zend, an interpreter.
In November 2007, FB starts experiments to convert (transpile) PHP
code to C++, to compile it with a regular C++ compiler.
Called HPHPc, in 2009 it became FB’s sole execution engine
powering FB’s web servers.
FB adds HPHPd to allow developers to use break points, watches etc.
In early 2010, HPHPc open sourced as“Hip Hop for PHP”.
HPHPc was performant, but FB still needed something faster.
In 2010, FB started working on JIT compilation called HHVM.
HHVM took over FB’s entire web ﬂeet in early 2013.
HPHPc is not deployed anywhere anymore.

HHVM – JIT compilation
HHVM stands for Hip Hop Virtual Machine (pretty much like what JVM is
for Java).
Why a Virtual Machine? This is unrelated with VMWare, VirtualBox and
the rest, which perform hardware virtualization.
The JVM compiles a Java source code to an intermediate language called
the Java Bytecode. A CPU cannot execute bytecode (it is not machine
code). The JVM translates the Java Bytecode to machine code. The JVM
actually deﬁnes an abstract CPU, or virtualizes the CPU.
The HHVM compiles PHP source code into Hip Hop Bytecode (HHBC).
Why an intermediate step? JIT (Just-in-Time) compilation of bytecode
allows adaptive optimization which can yield faster execution than static
compilation.

Hack
FB developed Hack, an evolution of PHP, tailored for the HHVM. Hack
mainly allows to use both dynamic and static typing:
function division(float $a, int $b) : float {
return $a / $b;
}
FB does not rewrite its PHP codebase into Hack. Old PHP code is kept.
New code is written in Hack. Hack code can call PHP code (and vice
versa). The HHVM runs both PHP and Hack. Hack deployed in
production in June 2012.
Hack’s code check tool was coined the“typechecker”(command
hh_client file.hh).
Hack is not a superset of PHP. Most bad PHP features are purposely
forbidden in Hack.

Hack – static styping
PHP 5.x actually allowed typehints, but for class types only:
function enroll(Student $student, School $school) {
echo ’Enrolling ’ . $student->name . ’ in ’ . $school->name;
}
enroll(new Student("Damien"), new Program("Medicare"));
// Argument 2 passed to enroll() must be an instance of School.
Typehints of primitive types were not allowed in PHP 5.x:
function stringTest(string $str) { echo $str; }
stringTest(’definitely a string’); // Argument 1 passed to
// stringTest() must be an instance of string, string given.
Hack supports all typehints, for function parameters and return type.

Hack – static styping
Static typing for class properties:
class C {
public static int $loggingLevel_ = 2;
private string $name_;
}
However, the type of local variables cannot be speciﬁed:
$x = 10;
$x = ’ten’; // valid
In Hack, local variables do not have types; local variables hold values,
which have types. At any point in the program, the typechecker knows
which type of value each variable holds. If a new value is assigned to a
variable, the typechecker will update its knownledge.

Hack – multiple possible types
How to deal with
if(some_condition())
$x = 10; // int
else
$x = ’ten’; // string
The typechecker remembers that $x can be either an int or a string. The
following operations must support both int and string:
echo $x; // ok: can echo an int or a string
echo $x+20; // error: incompatible with a string

Hack – type inference
Type inference is type guessing when there is no annotation (no type
speciﬁed). Type inference is only function-local. When analyzing a
function and performing type inference, the typechecker never propagates
its guesses to called functions or caller functions.
function f($str) {
return ’Here is a string:’ . $str;
}
function main() {
echo f(’boo!’);
}
main();
We can infer the return type of f to be only a string. The typechecker will
not check main’s code with the known returned type of f.

Hack – type inference
1 An inferred parameter or return type may impact numerous other
functions. This is not aﬀordable considering FB’s large codebase. The
typechecker must run quickly. An inferred type is only used to check
the code of the current function, not called of caller functions.
2 If the types are speciﬁed, the typechecker only needs to check at most
the directly called and caller functions, not the whole tree of called
and caller functions, which puts a fairly low cap on the work required.

Hack – number of parameters
PHP allowed
function f($a, $b) { }
f(1,2,3);
$a receives 1 and $b receives 2, third argument is forgotten. This is
accepted by PHP although the developer obviously made a mistake, which
may be tough to ﬁnd. Hack forbids this.

Hack – default values
Hack allows default values for arguments:
function f(string $str = ”, int $i = 0) { }
However, Hack forbids a null default value, whereas PHP allowed it:
function f(SomeClass $obj = null) { }
null is not really a default value. It depicts the lack of value. Hack forces
to declare the parameter nullable with
function f(?SomeClass $obj = null) { }
f();

Hack – default values
Example with a nullable string:
function takes_string(string $str) { }
function takes_nullable_string(?string $str) {
if($str !== null) {
takes_string($str);
}
}
takes_nullable_string(null);
takes_nullable_string(’hello’);
!is_null($str) may also be used instead of $str !== null.
The if is mandatory. Inside the if, the typechecker sees that $str is not
null, so takes_string can be called.

Hack – generics, arrays, tuples
function max_and_index(array<float> $items) : (float, int) {
$max = -INF;
$maxIndex = -1;
foreach($items as $index => $value) {
if($value > $max) {
$max = $value;
$maxIndex = $index;
}
}
return tuple($max, $maxIndex);
}
Tuples are actually a restricted version of arrays.
Hack also introduced new collections (Vector, Set and Map) on top of
PHP’s array.

Hack – Callables
Pass function to a parameter:
function do_work(array<float> $items,
(function(int,float):string) $fct)
{
foreach($items as $index => $value) {
echo $fct($index, $value);
}
}
In PHP, a function passed as a parameter was a callable. Under the
hood, $fct is just an object. What can be done in Hack can be done in
PHP. Hack just gives ways to better express programs.
R

Hack – Enumeration
PHP has no enums. Class constants or global constants had to be used.
enum CardSuit : int {
SPADES = 0; HEARTS = 1; CLUBS = 2; DIAMONDS = 3;
}
function suit_symbol(CardSuit $suit) : string {
switch($suit) {
case CardSuit::SPADES :
return "xe2x99xa4";
case CardSuit::CLUBS :
return "xe2x99xa7";
}
}
The typechecker reports“Switch statement nonexhaustive; the following
cases are missing: HEARTS, DIAMONDS”. default can also be used.

Hack – Switch statement
In switch statements, PHP allows multiple default statements.
Forbidden in Hack.
The developer must not forget break; at the end of a case. Hack
returns an error if no break; speciﬁed, unless the case is empty. If
fallthrough is intentional,“// FALLTHROUGH”must be added.

Hack – Shape
In PHP, most of the time, to represent a user with an id and a name, we do
not use a class with two properties, but an array with keys“id”and“name”.
Arrays shapes are meant to tell the Hack typechecker about the structure
of such an array:
type user = shape(’id’ => int, ’name’ => string);
function make_user(int $id, string $name) : user {
return shape(’id’ => $id, ’name’ => $name);
}
function make_user_2(int $id, string $name) : user {
$user = shape();
$user[’id’] = $id;
$user[’name’] = $name;
return $user;
}

Hack – Closures and lambdas
Closure with array_map (PHP and Hack):
$id_to_user = [1=>’Mr1’, 2=>’Mr2’, 3=>’Mr3’, 4=>’Mr4’];
$user_ids = [2, 4];
$user_names = array_map(
function($id) use ($id_to_user) { return $id_to_user[$id]; },
$user_ids
);
The closure can be rewritten as a lambda expression in Hack:
$user_names = array_map($id ==> id_to_user[$id], $user_ids);
R

Hack – Typechecker modes
PHP code is started with <?php. In Hack, it is <?hh with the choice of
mode.
1 Declarative mode: <?hh // decl
2 Partial mode: <?hh
3 Strict mode: <?hh // strict
R

Hack – Declarative mode
Declarative mode : <?hh // decl
The typechercker does not check the static types. The typechecker only
checks for invalid syntax.
The declarative mode is a transition aid if one is migrating an existing
PHP codebase to Hack.
The HHVM/Hack package also provides tools to help migrate PHP code
to Hack, and Hack to PHP (if some users want to write in Hack but
cannot use HHVM for example).
R

Hack – Partial mode
Partial mode : <?hh
The typechercker does all the typechecking it can, but does not require
typehints.
Top–level code is allowed but not typechecked. This is because top-level
code requires globals. Code should be wrapped in a function and have only
one top-level statement be a call to that function.
function main() {
// ...
}
main(); // top-level statement
R

Hack – Strict mode
Strict mode : <?hh // strict
Anywhere there can be a type annotation, it must be speciﬁed, with a few
exceptions: constructors and destructors (no return type) and closures.
Strict mode cannot call PHP, whereas partial and declarative modes can.
But strict mode can call partial-mode or decl-mode Hack code.
In all web apps, there has to be at least one top-level statement to serve
as an entry point. So there will always be at least one partial-mode ﬁle.
If the code is 100% strict, it should be impossible to incur a type error at
runtime in the HHVM.
R

Hack – Accessing superglobals
Strict mode does not allow superglobals. Accessor must be written in a
partial-mode ﬁle, called from strict-mode ﬁles.
function get_params() : array { return $_GET; }
But this does not contribute to any type safety. We should write instead
function get_param(arraykey $key) : ?string {
if(!array_key_exists($key, $_GET)) { return null; }
$value = $_GET[$key];
return is_string($value) ? $value : null;
}
Better: replace the return statement with
invariant(is_string($value), ’get_param: Expected string’);
return $value;
R

Hack – Special attributes
There are several attributes (decorators) which can be written in front of
methods, e.g. <<__Override>> to report an error if there is no function
override.
R

Common implementation to cache the result of a time-consuming
computation:
function factorize_impl($num) {
// some factorization algorithm
}
function factorize($num) {
static $cache = [];
if(!isset($cache[$num]))
$cache[$num] = factorize_impl($num);
return $cache[$num];
}

HHVM handles the caching of functions results declared with the
<<__Memoize>> special attribute:
<<__Memoize>>
function factorize($num) {
// some factorization algorithm
}
HHVM makes no garantee that all results are kept forever. HHVM may
delete entries from the cache to free up memory.

Memcached
Memcached is an open-source (not developed by FB) distributed memory
caching system, to exploit unused RAM of multiple computers.
Facebook has around 300TB of data in Memcached processes. Memcached
is also used by YouTube, Reddit, Twitter, Tumblr, Wikipedia etc.
Most cloud providers sell Memcached services at part of their Platform as
a Service (PaaS), including Google App Engine, Amazon ElastiCache, IBM
Bluemix and Microsoft Azure.
Memcached can be used through Hack’s async capabilities.

Hack – async
Hack’s async capabilities oﬀer a way to use cooperative multitasking, in
which tasks voluntarily and explicitely cede the CPU to one another.
async function do_work() : Awaitable<string> {
// time-consuming computation
return ’ok’;
}
async function main() {
$msg = await do_work();
echo $msg;
}
await tells the HHVM to wait for the function result to be available.
await yields to another suspended async function, if there is any.
do_work() does not return a string. It returns an object that represents
a string which may or may not be available.

Hack – async
HHVM provides await extensions for four kinds of operations:
1 queries to MySQL databases;
2 queries to Memcached;
3 reads and writes of stream resources;
4 cURL requests.

Hack – MySQL query
async function fetch_db(int $id) : Awaitable<string> {
$conn = await AsyncMysqlClient::connect(localhost, 3306,
’dbname’,’admin’,’pass’);
$result = await $conn->queryf(
’SELECT name FROM users WHERE id=%d’, $id);
return $result->mapRows()[0][’name’];
}
queryf automatically performs appropriate escaping, to prevent SQL
injection attacks.

Hack – cURL request
async function fetch_web() : Awaitable<string> {
return await HHAsiocurl_exec(’https://www.example.com’);
}
Await multiple asynchronous operations and resume when all the results
are available:
async function fetch_all() : Awaitable<string> {
list($web, $db) = await HHAsiov(
array(fetch_web(), fetch_db(2)));
return $web . $db;
}

Hack – Memcached
Memcached is used through MCRouter, an open-source project developed
by FB to aid in scaling a Memcached deployment.
async function fetch_name(MCRouter $mcr, int $id) :
Awaitable<string>
{
$cachedResult = await $mcr->get($id);
if($cachedResult->isSucceeded()) {
return $cachedResult->getResult();
}
else {
$name = await fetch_db($id);
await $mcr->set($id, $name);
return $name;
}
}

XHP
XHP: represent an HTML tree as PHP/Hack objects, by means of
embedded XML-like syntax.
function build_paragraph(string $text, string $style): :div {
return <div style={$style}>
<p>{$text}</p>
</div>;
}
If a tag is misspelled or not closed, HHVM triggers a runtime error, which
may have gone unnoticed without XHP.
XHP enforces HTLM5 rules.

XHP
XHP prevents the cross-site scripting (XSS) vulnerability.
$user_name = $_REQUEST[’name’];
echo ’<html><head><title>Welcome</title></head>’;
echo ’<body>Welcome ’ . $user_name . ’</body></html>’;
If a user submits a string containing HTML markup, it will be interpreted
by the browser as part of the document object model (DOM). To avoid:
$user_name = htmlspecialchars($_REQUEST[’name’]);
Applying it exactly once is tough with large codebase, something XHP
automatically takes care of.
XHP dates back to 2009. Since 2015, 100% of FB’s web frontend code
uses XHP to generate HTML.

React
React is a JavaScript library for building user interfaces (UI) rendered
as HTML.
React only provides the“V”, the View layer, in the MVC template.
React was built at FB, was released in March 2013, open-sourced on
May 29, 2013.

React – Virtual DOM
The browser has its own Document Object Model (DOM). The key feature
of React is that it keeps an in-memory copy of the DOM, called the virtual
DOM.
The developer does not directly update the brower’s DOM to
add/modify/remove UI elements. Instead, the developer updates the
virtual DOM. React takes care of comparing the browser’s DOM and its
own virtual DOM, and updates the browser’s DOM as eﬃciently as
possible. If there are e.g. tabs on a page, React will only update the visible
tab, not the hidden ones.
Ember’s Glimmer engine is heavily inspired by React’s virtual DOM.

React – Create a simple element
<html><body>
<div id="app"></div>
<script src="https://fb.me/react-15.1.0.js"</script>
<script src="https://fb.me/react-dom-15.1.0.js"</script>
<script>
var title = React.createElement("h1",
{id:"titleId", className:"titleClass"}, "Hello World!");
ReactDOM.render(title, document.getElementById("app"));
</script>
</body></html>
which outputs
<div id="app">
<h1 id="titleId" class="titleClass">Hello World!</h1>
</div>

React – Create an element with children
A tag content may also be some children tags:
var obj = React.createElement("ul", null,
React.createElement("li", null, "1 lb Salmon"),
React.createElement("li", null, "1 cup Pine Nuts"),
React.createElement("li", null, "2 cups Butter Lettuce")
);
ReactDOM.render(obj, document.getElementById("app"));
which outputs
<div id="app">
<ul>
<li>1 lb Salmon</li><li>1 cup Pine Nuts</li>
<li>2 cups Butter Lettuce</li>
</ul>
</div>

React – Create elements with map
A li element can be created for each element in an array, with map:
var items = ["1 lb Salmon", "1 cup Pine Nuts",
"2 cups Butter Lettuce"];
var obj = React.createElement("ul", null,
items.map((ingredient, i) =>
React.createElement("li", { key: i }, ingredient)
)
);
ReactDOM.render(obj, document.getElementById("app"));
However, we need to write the speciﬁc array name (items). We would like
to receive the array as an input parameter.

React – Factory
const IngredientsList = ({ list }) =>
React.createElement("ul", null,
list.map((ingredient, i) =>
React.createElement("li", {key: i}, ingredient)
)
);
const IngredientsFactory = React.createFactory(IngredientsList);
const ingredients = ["1 lb pizza", "1 cup Pine Nuts",
ReactDOM.render(IngredientsFactory({ingredients}),
document.getElementById("app"));

React – JSX
JSX for React is what XHP is for PHP/Hack:
const Ingredients = ({ name, list }) =>
<h1>{name}</h1>
<ul className="ingredients">
{list.map((ingredient, i) =>
<li key={i}>{ingredient}</li>
)}
</ul>;
var ingredients = ["1 lb Salmon", "1 cup Pine Nuts",
ReactDOM.render(
<Ingredients name="Delicious recipe" list={ingredients} />,
document.getElementById("app"));

Flux
Flux is FB’s application architecture pattern for building user interfaces.
Example : input text area and button to add items to a todo list.
1 The View receives a click event.
2 The View uses the Dispatcher to broadcast the message (called an
action)“one wants to add an item to the todo list”.
3 The Store keeps the data, namely all the todo list items (as an array).
4 On startup, the Store told the Dispatcher that it wanted to be
informed about dispatched actions.
5 The Store handles the received action and adds the new item to its
array.
6 The Store tells the View to refresh, based on the Store’s data.

Flux example – HTML
<input type="text" id="new_todo" />
<button onclick="buttonClicked()">Add!</button>
<div id="div_todo_list">
</div>

Flux example – Store’s data
var todoStore = {
items: []
};

Flux example – Click event and Dispatcher
var appDispatcher = new Flux.Dispatcher();
function buttonClicked() {
var newText = document.getElementById("new_todo").value;
var action = {
actionName: ’add-to-todo’,
newTodoText: newText
};
appDispatcher.dispatch(action);
}

Flux example – Store receives and handles the action
appDispatcher.register(function(action) {
if (action.actionName === ’add-to-todo’) {
todoStore.items.push(action.newTodoText);
}
});

Flux example – Store asks View to refresh
var todoStoreEmitter = new EventEmitter();
appDispatcher.register(function(action) {
if (action.actionName === ’add-to-todo’) {
todoStore.items.push(action.newTodoText);
todoStoreEmitter.emit(’change’);
}
});

Flux example – View receives signal and refreshes
todoStoreEmitter.on(’change’, refreshView);
function refreshView() {
ReactDOM.render(
<TodoList todoItems={todoStore.items} />,
document.getElementById("div_todo_list"));
}

Flux example – View’s React
const TodoList = ({ todoItems }) =>
<ul>
{todoItems.map((itemText, i) =>
<li key={i}>{itemText}</li>
)}
</ul>;
ReactDOM.render(
<TodoList todoItems={[]} />,
document.getElementById("div_todo_list"));

Flux – Notes
There is only one Dispatcher in the application.
There may be multiple Stores and multiple Views.
The Views completely re-render upon change in the Stores’ data.
The re-rendering is on React’s virtual DOM, then React makes sure
to update the browser’s DOM very eﬃciently.
Flux is only a pattern, implemented in many frameworks, including
FB’s Flux, Reﬂux, Redux, Alt.
FB also developed and open sourced Flow, a static type checker for
JavaScript.

GraphQL
const FriendList = ({ friends }) =>
<div id="div_friend_list">
{friends.map(f => <Friend friend={f} />)}
</div>;
const Friend = ({ friend }) =>
<div id="div_friend">
<FriendImg urlImg={friend.urlImg} />
<span>{friend.name}</span>
<span>Mutual friends: {friend.numMutualFriends}</span>
</div>;
const FriendImg = ({ urlImg }) => <img src={urlImg} />;
What if we want to add an icon on each friend’s image to tell which
friends are veriﬁed users?

GraphQL
Each React component can be accompanied by a short GraphQL code
listing its data dependencies, independently of the other React
components.
Each React component could use e.g. Ajax to request its data to the
server. However, this would imply too many server round-trips.

Relay
Relay aggregates all the required data dependencies to reduce the number
of queries to the server.
Relay is an evolution of Flux. For instance, Relay uses only one Store.
Some FB projects use React/Flux, whereas some others use
React/Relay/GraphQL.

React Native
In March 2015 FB open sourced React Native, a framework for building
cross-platform applications which render natively on iOS and Android.
Netﬂix modiﬁed React to render on many televisions and DVD players.
Views in React Native are also written in JavaScript/HTML/CSS.
On April 18, 2017 FB announced React Fiber and Relay Modern.

Thrift
FB developed and open sourced Thrift, which is meant to build
cross-platform RPC services that can connect applications written in
diﬀerent languages, over a network.

Databases
Cassandra (open sourced in 2008) is a NoSQL database designed to
handle large amounts of data across many servers, initially to power
FB’s Inbox Search. FB abandoned Cassandra in late 2010 for HBase.
Cassandra is still used by Apple, AppScale, Digg, Netﬂix, Reddit. . .
RocksDB is an embedded database, optimized to exploit many CPUs
and use eﬃciently SSDs. RocksDB is used by FB, Yahoo!, LinkedIn.
MyRocks is a RocksDB storage engine for MySQL.
FB developed and open sourced Hive and Presto which are SQL query
engines for analytics at scale. In 2013 FB gave up Hive/MapReduce
for Presto, to perform interactive queries against their 300PB data
warehouse.

170517 damien gérard framework facebook

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to 170517 damien gérard framework facebook

Similar to 170517 damien gérard framework facebook (20)

More from Geeks Anonymes

More from Geeks Anonymes (20)

Recently uploaded

Recently uploaded (11)

170517 damien gérard framework facebook