Tutorial on how to use Snap Lens Studio to create mobile Augmented Reality applications. Given on June 23rd by Mark Billinghurst from the University of Auckland.
4. 2016 Tacohead
• Taco Bell mobile AR campaign
• 224 million views in one day
• $750K USD to develop
• Snapchat sponsored lens
https://www.youtube.com/watch?v=Wkysq4gvRK4
5. Snapchat
• Founded in 2011
• Picture based social media application
• Over 280 million daily users (5/21)
• >4 billion pictures send everyday (2020)
• 82% of US teens use Snapchat once a month
• $2.5 Billion revenue in 2020
6. Snapchat Lenses
• Introduced in 2015 (Snap acquired Looksery)
• Add AR content to live camera view
• Use face detection/tracking
• In 2017 AR “World Lenses” introduced
• Virtual content fixed in space
9. Making Snap Lenses
• Lens Studio (2018)
• Visual authoring tool
• https://lensstudio.snapchat.com/
10. Setting Up
• Go to https://www.snapchat.com/
• Install Snapchat on mobile phone
• Create an account
• Download and install Lens Studio
• https://lensstudio.snapchat.com/
14. Process
1. Add taco image to resources
2. Add Face Image to scene objects
3. Set face image texture to taco
4. Resize taco image
5. Add Face Inset for mouth
6. Add 2 Face Insets for eyes
7. Publish lens
8. Done
15. 1. Add Resource
• Find Taco image with transparent background
• Download to local machine
• Drag into the resources area
16.
17. 2. Add Face Image to Scene
• Hit ‘+’ to add object to scene
• Search for Face Image
• Add to scene
• Select Head Binding
• View inspector
• Attach to point Face Center
18. 3. Set Face Image to Taco
• Select Image 1
• Open inspector
• Click and drag taco image from
resources into texture slot
19.
20. 4. Resize Taco Image
• Select Image1
• Open Transform in inspector
• Set x, y scale values to 120.0, 120.0
21. 5. Add Mouth Face Inset
• Add face inset to scene
• Select Face Inset 1
• In inspector make sure Face
Region is set to Mouth
• Select Face Inset Binding 1
• In inspector, in Head Binding set
Attach To Point to Mouth Center
• Rename Face Inset Binding to
Mouth Face Inset
22. 6. Add Left and Right Eye Insets
• Add Face Inset to scene
• Select Face Inset
• Set Face Region in Face inset to Left Eye
• Select Face Inset Binding
• Set Head Binding Attach to Point to Left Eyeball
• Rename Face inset binding to Left Eyeball
• Repeat for Right Eyeball
27. Methods for Adding Interaction
• Drag and drop
• Simple behaviours
• Using object components
• Visual Scripting
• Node based
• Coding
• Javascript
28. Adding Taco Behaviour
• When mouth open !Taco! text appears
• Key Elements
• Taco Text
• Text positioning
• Text material/colour
• Mouth open detection
• Mouth close detection
29. Key Steps
1. Create text object
• Attach to face, set font size, colour
2. Create Scene Manager object
3. Create Visual Script and add to Scene Manager
4. Add nodes to visual script
• Mouth open/closed, text input, set text nodes
5. Done
30. 1. Create Text Object
• Create text object in Object list
• Drag to be under head binding object
• Attaches to head video
• Open text object inspector
• Type in sample text
• Set text properties
• Font size, colour, drop shadow, etc.
• Position text
31. 2. Add Object Manger
• Add Scene Object to object list
• Place at top level
• Rename as ObjectManager
32. 3. Add Visual Script
• Add Visual Script to Resources
• Add script component to ObjectManager
• Set VisualScript to ObjectManager script
33. 4. Creating Visual Script
• Node based visual programming
• Basic workflow
• Open script graph editor
• Double click on Visual Script
• Load up needed nodes
• Wire nodes together
• Set parameters
• Test logic
34. Load Nodes
• Right click in script graph editor, or hit tab
• Search for key terms (e.g. Mouth)
• Needed Nodes
• MouthOpenEvent – detect when mouth open
• MouthCloseEvent – delect when mouth closed
• InputText – set the text object value
• SetText – set text string value
• Important: Delete text in TacoTex object
36. Specifying InputText Object
• To associate TacoText object with InputText node
• Select Manager Object script and check inspector
• Drag TacoText object into empty Text input field
37. Arrange Nodes in Visual Script
• Duplicate SetText Node
• Set SetText values
• Select node, enter text value in variables view
• “!TACO!” in one node, “ “ in the other
38. Wire Nodes Together
• Output from MouthEvent nodes
to trigger of setText nodes
• setText nodes to InputText node
• Note: impossible to connect
incompatible nodes