N-Screens Problem is a set of challenges you come across when you develop an App for multiple connected devices. How HTML5 WebApp solution was the most promising solution for the developers. My session was around the 7 Key implementation challenges you will face when developing a Web App solution for multiple devices with different form factors. The 7 key challenges were, Feature Detection, Layout detection, Using CSS3, Choice of Animation between SVG, Canvas, CSS3 Animation, and WebGL, Audio Video related problems and their work-arounds, and how to use advanced HTML5 API with a brief on Polyfills and Shims. Here is the online demo of my presentation. Source code is also available at the demo page. Detect your N-device HTML5 features and APIs at http://widgets-gadgets.com/html5 ----------------------------------------------------- Everything you need to know about developing HTML5 Apps on multiple devices is at W3C conference. http://www.w3.org/conf/ ----------------------------------------------------- Redmond, Nov 15-16, 2011, amazing conference, first time all the vendors together, instead of talks about individual device/ browsers, all the sessions were focused on the common part, the HTML5 part. If you missed the conference check videos here http://www.w3.org/conf/#presentations

1. THE N-SCREENS PROBLEM
Building Apps in the world of TV & Mobile
Rajesh Lal
Senior Engineer

2. Agenda 7 Key Challenges
WebApp
Develop
Design
N-Screens

3. The N-Screens

4. Mobile PC
Tablet TV

5. 1. Design Problem: N Screens

6. On the Go Mobile
Screen size 2-4”
Distance 1’
Connection 3G
Not reliable
Highly Interruptible
Finger Input

7. Screen 7-14” Tablet
Casual Environment
Distance 1’
Wi-Fi Loosely connected
Interruptible
Input Touch

8. Screen size 14-27”
Computer Desk
Distance 2’
Reliable LAN
PC Focused
Keyboard Mouse

9. Screen size 25-65” Couch Distance 10’ LAN Fast
TV connection Total Emersion D-Pad Remote

10. 2. Developing Apps for N Screens
Native vs. WebApp

11. Native: Developing Apps for N-Screens
Pros
• Each “Screen” with an SDK
• Great for THAT device
• Robust
• Device APIs

12. Native: Developing for N-Screens
Cons
Objective C
• Learning curve for each SDK
Java
• Multiple Programing Languages C# C++
• Time and Effort for every device Qt, QML
XAML
• Maintenance
• Versions

13. 3. The WebApp Solution
Web Application 1.0

14. WebApp: Developing for N-Screens
A WebApp is an application created using
HTML, CSS and JavaScript.
• App in the Web browser (Full screen)
• WebApp in a native container (Hybrid)

15. History
“Old" HTML
Web RIAs HTML5 WebApp
Applications
Basic Server Script Media Advanced Touch
Table, Img Client-Server Players Canvas Camera
Cookies Thin Client Plug-ins Media Contacts
Sandbox Validation Extensions Storage Calendar
CSS Object Tag Web db Notification
JavaScript Pvt. Storage Geo location Battery
Audio video Offline Orientation
XPCOM Drag Drop Acceleromet
Threads
CSS3

16. WebApp: Developing Apps for N-Screens
Pros
• One “SDK”
• Easy to Develop
• Powerful HTML5 APIs
• CSS3

17. WebApp: Developing for N-Screens
Cons
• Work in Progress
• Challenging when porting to many Screens

18. 4. Seven Key Challenges
And How to Work Around Them

19. 1/7 Feature Detection
Which Feature Your Screen Support ?

20. Feature Detection
Feature
Browser Detection
Detection
Old method New method

21. Feature Detection
In the code

22. Feature Detection
Use libraries
• Modernizr
• jQuery.support

23. Feature Detection
code
<script src="modernizr-1.7.js“/>
Modernizr.canvas ? "Canvas" : “No canvas";
<script src="jquery-latest.js“/>
jQuery.support.ajax ? "Ajax" : “No Ajax";

24. Feature Detection
Modernizr
Canvas SVG Cross Window Msg
Audio InlineSVG Web Workers
Video WebGL Web SQL database
Animation Websockets Touch
Transformation Geolocation
Drag and Drop Local Storage

25. Feature Detection
jQuery.Support
Ajax
Cross Domain Resource Sharing (CORS)
Fixed Position
CSS Opacity
CSS Float
htmlNormalize

26. 2/7 Layout Detection
How to adapt the layout to
the device form factor?

27. Layout Detection
Media
Orientati Queries
Viewport on
JavaScript

28. Layout Detection
Media-queries enable style sheets tailored for
different ’width’, ‘height’ and ‘color’

29. Media Queries: Layout Detection
<link rel="stylesheet" href="handheld.css"
media="only screen and (max-width: 480px), handheld" />
<link rel="stylesheet" href="default.css" media="screen" />
<link rel="stylesheet" href="wider.css"
media="only screen and (min-width: 1200px)" />

30. Media Queries: Layout Detection
handheld Default screen Wider screen
http://www.lancs.ac.uk/

31. Media Queries: Layout Detection
jQuery.mediaqueries
The script adds basic Media Query-Support (min-
width and max-width in px units ) to all browsers
<script src="jquery.mediaqueries.js“></script>

32. Layout
Media Query for device-aspect-ratio
@media screen and (device-aspect-ratio: 16/9)
@media screen and (device-aspect-ratio: 32/18)
@media screen and (device-aspect-ratio: 1280/720)
@media screen and (device-aspect-ratio: 2560/1440)

33. Orientation
Media Query for Orientation*
@media screen and (orientation:portrait) {
/* Portrait styles */
}
@media screen and (orientation:landscape) {
/* Landscape styles */
}
*DeviceOrientation Events specification at W3C

34. Viewport
Viewable area on the screen
<meta name="viewport" content="width=device-width,
initial-scale=1, maximum-scale=1">

35. 3/7 Advanced Styles
How to utilize CSS3 for
Advanced Graphics ?

36. Advanced Styles with CSS3
• Background
• Border Radius
• Drag and Drop
• Opacity
• Transformation
• …

37. Challenge: Advanced Styles with CSS3
Prefix
• -moz-
• -ms-
• -o-
• -webkit-

38. Workaround: Advanced Styles with CSS3
.style
{
-moz-border-radius: 1em;
-ms-border-radius: 1em;
-o-border-radius: 1em;
-webkit-border-radius: 1em;
border-radius: 1em;
}

39. 4/7 Animation
When to Use
SVG, Canvas, CSS3 and
WebGL Animation?

40. Animation: Challenge
WebGL
CSS3
Canvas & Animation
SVG & JavaScript
JavaScript

41. Animation: SVG
• 2D vector graphics using XML
• Object retained in the memory
• Full DOM support
• SVG elements can be styled
• Check Modernizr.svg / Modernizr.inlinesvg
Perform better when smaller
number of elements and large surface

42. Animation: SVG
<svg id="svgElement"
width="800px" height="600px"
viewBox="0 0 800 600">
<rect x="0" y="0" width="100%"
height="100%" rx="10" ry="10"
style="fill: white; stroke: black;" />
<circle id="circle0" cx="40" cy="40"
r="40" style="fill: orange; stroke: black;
stroke-width: 1;" />
</svg>

43. Animation: Canvas: Context 2D
• Bitmap drawing area
• Rectangles, lines, arcs, paths
• Stateless
• No DOM support, single element
• Check Modernizr.canvas
Perform better when large number of objects
and surface is small

44. Animation: CSS3
• Styles for Individual elements
• Use CSS3 animation if available
• Better than JavaScript based animation
• Check Modernizr.csstransitions
Can perform better with GPU acceleration

45. Animation: CSS3
var elem = document.getElementById('transelem');
elem.addEventListener('click',
function loop() { elem.style.left = „100px';}, false);
#transelement
{
…
-xxx-transition: opacity 1s ease;
transition: opacity 1s ease;
}

46. Animation: WebGL (Canvas Context: webgl)
• Canvas based 3D implementation
• Use GPU acceleration
• Supported only in a few browsers
• Check Modernizr.webgl

47. 5/7 Media
How to use Audio-Video?

48. Audio: Big Challenge
Open AL
Audio Web
RIA Audio Data - Audio -
Firefox Webkit
HTML5 Flash/
Audio Silverlight
Not
suitable
for Sfx

49. 6/7 Ajax
How to use Ajax and do
Cross Domain call?

50. Ajax: XMLHttpRequest
CORS
Write a Cross
JSONP Proxy Origin
Resource
Hybrid Not sharing
App Standard

51. Ajax: XMLHttpRequest
• Cors Server code
self.response.headers.add_header
("Access-Control-Allow-Origin", "*")
• Check jQuery.Support.Ajax
jQuery.Support.cors

52. 7/7 HTML5 APIs
How to use HTML5 advanced APIs
in all Screens?
&
What are PolyFills and Shims?

53. HTML5 API: Challenge
HTML5 API
Polyfills and
Shims
JavaScript
Code

54. HTML5 API: Polyfills and Shims?
Provide support for missing features.
• Polyfills: Replicate standard feature API
• Shims: Own API with useful features

55. W3C Working Groups
And other resources

56. Where to look for more information
W3C Working Groups
http://www.w3.org/Mobile/
http://www.w3.org/TR/css3-mediaqueries/
http://www.w3.org/2011/audio/
http://www.w3.org/testing/browser/
http://www.w3.org/Graphics/SVG/WG/wiki/Main_Page
http://www.w3.org/TR/cors/
http://dev.w3.org/geo/api/spec-source-orientation.html
Other Resources
http://www.modernizr.com/
http://api.jquery.com/jQuery.support/
http://Html5rocks.com
http://mediaqueri.es/

57. The future is already here—it’s just not very
evenly distributed.
- William Gibson
Thank You
@rajeshlalnokia
rajesh.lal@nokia.com