The normal interaction with computers is with keyboard and a mouse. For display a rectangular somewhat small screen is used with 2D windowing systems. The mouse was invented more the 40 years ago and has been for 20 years dominant input. Now we are seeing new types of input devices. Multi-touch adds new dimensions and new applications. Natural user interfaces or gesture interfaces where people point to drag objects. Computers are also beginning to recognize facial expressions of people, so it knows if you are smiling. Voice and natural language understanding is getting to a usable stage. All this calls all types of new applications. Displays are getting bigger. What if any surface was a screen? If you could spray the wall with screen? Or have you phone project images to the wall. This lectures explores some of these new types of interactions with computers and software. It makes the old mouse look old.

1. LECTURE L22
AUGMENTED AND VIRTUAL REALITY

2. Evolution of Computers

3. 1960 1970 1980 1990 2000 2010 2020 2030

4. Mainframes Mini
Computers
PC
Time-sharing
1960 1970 1980 1990 2000 2010 2020 2030

5. 1960 1970 1980 1990 2000 2010 2020 2030
Mainframes Mini
Computers
PC
Clients
Time-sharing
Servers
Laptops
Server
rooms
Smartphones
Tablets
Music
players
Consumer Products
Data
centers
Mobile
phones
Convergence
Divergence
DecentralisationCentralisation
Mobility
eReaders
Wearables
IoT
New Business Models
Nearables
Internet of Me
Flyables
Utilities
AI
Enterprise

6. 1960 1970 1980 1990 2000 2010 2020 2030
Mainframes Mini
Computers
PC
Clients
Time-sharing
Servers
Laptops
Server
rooms
Smartphones
Tablets
Music
players
Consumer Products
Data
centers
Mobile
phones
Convergence
Divergence
DecentralisationCentralisation
Mobility
eReaders
Wearables
IoT
New Business Models
Nearables
Internet of Me
Flyables
Utilities
AI
Enterprise
What
next?

7. World fastest computer?

8. 200 petaflops, or 200,000 trillion
calculations per second
one petaflop is one quadrillion floating-point calculations per second
DoE and IBM Summit

9. The Internet cloud
More programs and data is stored on network
servers
The Personal Computer becomes one of the
form factors to access the network
Examples
Amazon API
Google Apps
Facework Platform API
The Network is the Computer

10. Tom Watson was wrong – there
is not room for five computers.
It’s only one
The Network is the Computer

11. What about Moore’s Law?
Shift to multicore started in
2005 – new dimension

12. Move to cloud-core
Amazon has over 2.24
million machines in
some 87 data centers
(2014 data)
Google has about 1
million
What about Moore’s Law?

14. USDbillions
$241
billion in
2020
33,000+ Data
Centers in the
world

15. Data Centers are
now a Software Problem

16. Small devices that have computer power
Wireless capabilities
Dedicated devices
Enough computer power for limited functionality
Examples
RFID
UAV – Unmanned Arial Vehicles
Internet of things
Medical - nano robots
Miniature Computers

17. How long will the Hard Disk Drive last?
Solid state memory is getting bigger
Terabit Flash Memory
Computer architecture will change
More and more devices are
using Flash memory
Driving prices down
The Future of the PC

18. New form factors

19. Wearables, flyables, drivables, scannables…

20. What is the future of Personal Computers?

21. PC Evolution
1975 1980 1985 1990 1995 2000 2005
Hardware era
PC, Mac
Software OS era
Windows, Office, MacOS
Internet
Hardware Connects
IBM
Microsoft
Apple
2010
Software web era
Web 2.0, Social
2015
Internet of things
2020
Apple
Google
Smartphone
Wearable
Startups
Facebook
AmazonMicrosoft
Twitter

22. Consumer Devices
The computer and the
Internet
disappear

23. Any significantly
important technology
will disappear
Consumer Devices

24. Future of PC
Cloud Computing and home Serves
Everything will be connected: computer, TV, Phone, etc.
Simpler specialised operating systems
Utility computing
Wireless
Ubiquitous network access
Computers will be smaller and integrated into
things
New input methodsBetter interfaces
Big Brian, Small Brain
Better friction of devices
Consumer Devices
AI

25. Future of the Desktop Metaphor

26. Early Computers
Goal:
Data processing Read
data, process, write or
print

27. Early Computers
Result:
Dramatic Increase in
productivity
Automation

28. Xerox Early GUIs
Goal:
Emulate the
office environment
in a computer

29. Macintosh 1984
First commercial
GUI system

30. Macintosh 1984
Result:
More User Friendly
Graphics, fonts, WYSIWYG
Laserprinter

31. Windows 3.0
Business software
migrates to GUIs

32. Windows 95
First consumer targeted
OS

33. Windows 95
Result:
More people, connection
to the Internet

34. Ubuntu
Mac OS X
Windows
More choices, less important
Android
Operating Systems Today

35. 35
Is the desktop metaphor
dead?

36. 36

37. Basically file systems
Concept of files, loading and saving
Multistep install process (parts made by lawyers)
Then with the Internet, the concepts of uploading and
downloading
These concepts are disappearing…
Traditional Operating Systems

38. 38
Forgot to save
your file?

39. What do people do with computes? today?
Communication - talk, messages
Reading
Take/View Photos
Watch Movies
Shopping
Browse the Internet
Read news
Write documents, blogs
Financial transactions
Learning
Desktop, laptop, tablet, smartphone

40. The Next Device

41. The world in 2010

42. Device on your arm

43. The Smartwatch market

44. Digital World
SOFTWARE
WORK
FOCUS
APPS
VIEW/BROWSE
ENTERTAIN
APPS
CHECK
NOW!
VIDEO
WATCHING
RELAXED
APPS
GLANCE
NOTIFY

45. Bluetooth Devices
Lots of devices - communicating
Less friction - just works
Simple and easy to use with
bad security

46. Okey — If it look ugly
on this guy,
it is ugly on you
„Bluedouche“

48. http://wersm.com/facebook-wants-to-be-the-leading-second-screen-app/
87% TV viewers are using a device when watching TV
The traditional TV model
of scheduled TV does not
work anymore
The only thing that works in
TV is live TV and premieres
of major shows - due to Twitter
Digital World

49. Many objects around us will become
“smartobject”

50. Smart everything

51. Fitness trackers

53. Smart Leggings
Uses electrodes to
monitor the electricity
in the muscles
Measure muscle
fatigue

54. New Reality
“I'm not crazy about reality, but it's still
the only place to get a decent meal.”
― Groucho Marx

55. Clay Bavor, VP Virtual Reality
VR and AR at Google (Google I/O '17)

57. Real world
Computer-generated
Real
Reality
Augmented
Reality
Virtual
Reality
Immersive Computing Spectrum
Mixed Reality

58. Augmented Reality

59. The blurring line between reality and virtual reality

60. Augmented Reality
Augmented reality (AR) is a direct or indirect live view of a
physical, real-world environment whose elements are
"augmented" by computer-generated perceptual information,
ideally across multiple sensory modalities, including visual,
auditory, haptic, somatosensory, and olfactory

61. Adding digital to the real world

62. 62
How can Augmented Reality improve
our life and work?

63. Google translate

64. Aparna Chennapragada
AR at Google (Google I/O '18)

66. Facebook
What is the vision for Facebook of the Future?

68. Wearable Augmented Reality

69. 69
Glasses

71. What happened to Google Glasses?

72. Product was primitive, low quality, very expensive
And not enough people wanted it, social issues
Marketing failure

73. Adjacent Possible
It was not the time - technology not ready

74. Will it come back?
Yes, in some form

76. Microsoft HoloLens

78. Hololens
Microsoft is targeting
enterprise users
Are not targeting the
consumer market yet
Price point needs to be
$200 for massive
consumer adoption to
be possible

79. Virtual Reality

81. Virtal Reality
Virtual reality (VR) is a computer-generated scenario that
simulates a realistic experience
The immersive environment can be similar to the real world in
order to create a lifelike experience grounded in reality or sci-fi

82. Learning

83. Virtual Reality Headsets
Low End
Simple and cheap
No wires, require high
end phone
Hidden costs
High End
Big and expensive
Wires, require high end
PC
Hidden costs

84. Computing is escaping the form

85. Game consoles with new controllers

87. Microsoft Kinect

91. Voice Recognition

93. Mobile, tablets, smartphones,
wearables, PDAs, smart everything
Desktop, laptops
BEFORE NOW

94. Desktop metaphor, mouse
keyboard
Gesture, audio, touch
BEFORE NOW

96. Robotics and Drones
NEXT:
Research Paper