Face ID is a facial recognition system designed and developed by Apple which was released on November 2017 (as part of the launch of the first iPhone X). Many iOS apps currently use biometrics (Face ID and Touch ID), especially the banking and financial sectors. The talk will focus on gotchas and practical considerations for including biometrics in an application. The talk will consist of: - A brief overview of biometrics - How to log a user into your app with Face ID or Touch ID using LocalAuthentication - Protecting keychain items with biometric authentication. - Roll your own facial recognition apps using Core ML

1. Implementing Biometric Authentication
& Features in iOS Apps
Jean-Luc David
iOS Developer
MyPlanet,Inc

2. What are Biometrics?
• A biometric is a unique, measurable characteristic of a human being
that can be used to automatically recognize an individual or verify an
individual‟s identity. Biometrics can measure both physiological and
behavioral characteristics. Physiological biometrics (based on
measurements and data derived from direct measurement of a part of
the human body) include:
• Finger-scan
• Facial Recognition
• Iris-scan
• Retina-scan
• Hand-scan

3. Facial Recognition circa 2012

4. Open Computer Vision

5. History of Biometrics (1960-2017)
• Woodrow Wilson Bledsoe - Manual Measurements (1960) Rand tablet
• Goldstein, Harmon, and Lesk - 21 Facial Markers (1970)
• Sirovich and Kirby - EigenFaces (1998) Low-dimensional representation of facial
images.
• Turk and Pentland - Face Recognition in images (1991)
• DARPA - FacE REcognition Technology (FERET) (1993-2000)
• National Institute of Standards and Technology (NIST) - Face Recognition Vendor
Tests (FRVT) (2000)
• Social Media (2000)
• Pinellas County Sherriff’s Office - Law Enforcement Forensic DB (2009)
• Panama Airport - FaceFirst’s facial recognition platform (2011)
• U.S. Military - Osama Bin Laden Identified (2011)
• Automated Regional Justice Information System (ARJIS) (2014)
• Retail Loss Prevention (2017)
• iPhone X (2017)
• FaceFirst Watchlist as a Service (2017)

6. Eigenfaces

7. Touch ID
• Introduced in iOS 7
• iPhone5s / iPadAir2 / iPadMini3
• Unlock device (Passcode)
• iTunes/AppStore purchases
• Pay

8. iOS 8
• Touch ID Security APIs
• Local Authentication
• LAContext
• secItem
• App needs to be foreground

10. Info.plist - NSFaceIDUsageDescription
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>NSFaceIDUsageDescription</key>
<string>This app requires Face ID.</string>
</dict>
</plist>

11. Detecting Biometric Capabilities
import LocalAuthentication
let context = LAContext()
var error: NSError?
if context.canEvaluatePolicy(.deviceOwnerAuthenticationWithBiometrics,
error: &error) {
if #available(iOS 11.0, *) {
if context.biometryType == LABiometryType.faceID {
print("Device supports Face ID.")
} else if context.biometryType == LABiometryType.touchID {
print("Device supports Touch ID.")
} else {
print("Device does not support Biometrics.")
}
}
}

12. Detecting Biometry Changes
import LocalAuthentication
let context = LAContext()
var error: NSError?
if context.canEvaluatePolicy(.deviceOwnerAuthenticationWithBiometrics,
error: nil) {
if let domainState = context.evaluatedPolicyDomainState {
let bData = domainState.base64EncodedData()
if let decodedString = String(data: bData, encoding: .utf8) {
print("Decoded Value: (decodedString)")
}
}
}
}
• If the biometric databases changes significantly (like upgrading to an
iPhone X), the evaluatedPolicyDomainState will also change.

13. iOS 11 Face ID - iPhone X

16. Keychain

17. Architecture

18. Face Recognition with ARKit

19. Popular Perceptions

20. Popular Perceptions

21. Popular Perceptions

22. Nomenclature

23. Application of Machine Learning

24. Silicon Valley - Jian-Yang's Not Hotdog App

25. Fiction Becomes Reality

26. Types of Machine Learning

27. Core ML
• Announced at WWDC 17.
• Works only on iOS 11
• Inference engine
• Does not support training models.
• Heterogenous compute architecture - CPU & GPU.
• Open Source Python coremltool for converting
to coremlmodel file.

28. But I want to train models!

29. Runs On Device
User Privacy Data Cost Server Cost Available

30. How Does a Model Work?

31. Machine Learning Model Types

32. Demo - Face Detection & Recognition

33. Types of Machine Learning
Offline
+ Labels

34. Task - Find a Rose

35. Inference is Difficult

36. Inference is Difficult

37. Core ML Model
• Single Document
• Public Format
• Reduced Size
• Improved Accuracy
• Decreased Prediction
Time

38. Core ML Architecture
• Unified Fine Tuned Inference Engine
• Xcode Integration
• Built on Accelerate and Metal
• Public Model Format
• Support for Popular Training Libraries

39. Core ML Model
• Function learned from data
• Observed inputs
• Predicts outputs
• Single Document
• Public Format
• Ready to Use
• Task Specific

40. Sample Models
https://developer.apple.com/machine-learning/

41. Converting to Core ML
Core ML
Tools
Open Source

42. Converting to Core ML
• Download a trained .caffemodel and
a .prototxt of the dataset, as well as a .txt list
of names related to the model.
• Install Python 2.7 and pip
• Install, create, then activate the virtualenv.
• Write and run a Python script to convert
the .caffemodel to a Core ML model,
using coremltools
• Add the generated .mlmodel to the Xcode project

43. Demo - Converting a Model to MLModel

44. Xcode Integration

45. Swift ML Code
Generates this code

46. Model as Code
• Quick Initialization
• Optimized Prediction
Xcode Your App

47. Demo - FlowerClassifier

48. Resources
• https://www.slideshare.net/jldavid
• https://github.com/jldavid/FaceDetection
• https://github.com/jldavid/FlowerClassifier

49. Thank You

50. Questions?
jldavid@gmail.com