WebGL and COLLADA were discussed as technologies for 3D rendering and asset interchange on the web. The presentation covered the history and capabilities of both standards. It also described approaches for loading COLLADA assets into WebGL, such as preprocessing COLLADA into JSON or loading XML directly and parsing it with JavaScript. Optimizing COLLADA assets for WebGL rendering through techniques like quantization and compression was also mentioned.

1. COLLADA& WebGLWeb3D 2011(Paris) Rémi Arnaud Screampoint Inc

2. Agenda Déja-vu? (thx to Tony Parisi for the slides) COLLADA WebGL Going from COLLADA to WebGL

3. “History repeats itself, first as tragedy, second as farce.” --Karl Marx

4. Web 3D: A Brief History Intermediate Format + XML, Skinning, Programmable Pipeline, Multitexturing, … In-Browser Rendering + Animation, Scripting, Interactivity Cautious Optimism Built In GPU @mpesce #ZOMG #Winning! Model + Hyperlink in a Page Raging Indifference Competing Standards Proprietary Platforms “Virtual Worlds” “Gaming” Redefined More important stuff: Web 2.0 Mass Disillusionment No Broadband No GPUs No Killer App Warring Factions More important stuff: Web 1.0 Eager Enthusiasm @mpesce #WTF? #Winning!

5. http://www.gameenginegems.net/geg2.php

6. WebGL and other technologies 3D with Java (HW accelerated) OpenGL binding (JSR-231), OpenGL ES (JSR-184) Did not prevail 3D with Flash Papervision3D, Sandy3D, Away3D and now HW acceleration with Molehill (available) http://labs.adobe.com/technologies/flashplatformruntimes/incubator/features/molehill.html Between 1 and 2 Billions installations of flash player 3D with plug-in Unity extremely popular. +500,000 SDK downloads, +50 Millions of plug-in download Cross platform (iOS, Android, Xbox, Wii, PS3, …) Splendid world editing tool (includes scripting) Google Native Client Direct access (with a protection layer) to OpenGL ES HW acceleration Other native apps… smartphones 600% growth, more and more users see internet only through their phone/tablet device.

7. COLLADA XML encoding for 3D assets Geometry, Materials, Shaders, Animation, Skin/Bones, Physics, B-Rep*, IK* Intermediate format – for the tool chain(including conditioning / optimizer) Tool interchange format COLLADA available for most if not all modeling tools COLLADA & X3D white paper http://www.khronos.org/collada/presentations/Developing_Web_Applications_with_COLLADA_and_X3D.pdf * COLLADA 1.5

9. COLLADA is everywhere Max, Maya, Softimage (Autodesk dominance) through fbx sub-standard support opencollada.org plug-ins Cinema4D, Lightwave, Blender, Modo ….. Google warehouse, 3dvia Google Earth, Sketchup Microstation, AutoCAD, Catia … But not very well implemented Not a single adopter (conformance test)

10. http://downloads.akpeters.com/product.asp?ProdCode=2876 COLLADA reference manuscript

11. libraries Data elements are grouped by feature. A document may contain one or more libraries. User is free to organize documents/collections. geometries materials, effects images animations animation clips controllers cameras lights nodes scenes physics (models, materials, scenes)

12. COLLADA reference card page 1

13. assets COLLADA defines where assets can be defined The <COLLADA> element <asset> is mandatory, optional everywhere else Assets are defined in their own coordinate system COLLADA reference card page 1

14. scenes A COLLADA document can contain several scenes, or none. Can contain physic scene Only one ‘visual_scene’ reference the specific scene to be visualized by default. This is because 99% of modelers do not know how to handle multiple scenes.

15. nodes Nodes define a local coordinate system, hierarchy transform Nodes have no semantic, they do not convey a specific action. (i.e. no LOD, switch… nodes) Maybe they should have been called ‘places’ Transforms in a node are to applied in the same order they appear.

16. transform hierarchy <node> <rotate> <translate> <instance_geometry> .. Is same transform as .. <node> <rotate> <node> <translate> <instance_geometry> COLLADA reference card page 2

17. id and names id are used to reference an element, when instancing for example. They have to be UNIQUE in a document name are used to store human readable information. It’s like a <extra> tag. names are not unique, the type has been relaxed in 1.5 (xs:NCName has too many restrictions) COLLADA reference card page 2

18. Example 1: documents are tiles tile_1.dae tile_2.dae Id = A Id = A tile_1.dae#A tile_2.dae#A

19. Example 2: using name to group tile_1.dae id = A name = arrow id = B name = arrow arrow = tile_1.dae#A + tile_1.dae#B

20. instances Instances are basic building block of COLLADA nodes define local coordinate system where geometry is instanced nodes, with their attached geometry can be instanced. They can be stored in the node library rather than enumerated in the scene Instanced elements can be modified at binding point e.g. material bind is done when instancing a geometry

21. Notice the <extra> element to store application specific data. COLLADA reference card page 2

22. techniques technique_common is mandatory. Default rendering mode other rendering techniques are optional, there can be many (glsl, hlsl, mobile, PS3…)

23. Common rendering techniques COLLADA reference card page 4

24. Binding materials (daenotepad)

25. COLLADA reference card page 6

26. Geometry – e.g. triangles COLLADA reference card page 2

27. dataflow COLLADA reference card page 3

28. http://www.crcpress.com/product/isbn/9781568814322 e.g. daenotepad https://github.com/RemiArnaud/DAE-notepad

29. WebGL WebGL is OpenGL ES 2.0 for javascript Rendering with OpenGL ES 2.0 requires the use of shaders. Shadersmust be loaded with a source string (shaderSource), compiled (compileShader), and attached to a program (attachShader) which must be linked (linkProgram) and then used (useProgram).

30. WebGL Reference Card page 2

31. InitWebGL (J3DI.js) function initWebGL(canvasName, vshader, fshader, attribs, clearColor, clearDepth) { var canvas = document.getElementById(canvasName); vargl = canvas.getContext("webgl"); if (!gl) { alert("No WebGL context found"); return null; } // create our shaders varvertexShader = loadShader(gl, vshader); *** see ‘Load Shader’ slide varfragmentShader = loadShader(gl, fshader); *** see ‘Load Shader’ slide if (!vertexShader || !fragmentShader) return null; // Create the program object gl.program = gl.createProgram(); if (!gl.program) return null; // Attach our two shaders to the program gl.attachShader (gl.program, vertexShader); gl.attachShader (gl.program, fragmentShader);

32. // Bind attributes for (var i in attribs) gl.bindAttribLocation (gl.program, i, attribs[i]); // Link the program gl.linkProgram(gl.program); // Check the link status var linked = gl.getProgramParameter(gl.program, gl.LINK_STATUS); if (!linked) { // something went wrong with the link var error = gl.getProgramInfoLog (gl.program); gl.console.log("Error in program linking:"+error); gl.deleteProgram(gl.program); gl.deleteProgram(fragmentShader); gl.deleteProgram(vertexShader); return null; } gl.useProgram(gl.program); gl.clearColor(clearColor[0], clearColor[1], clearColor[2], clearColor[3]); gl.clearDepth(clearDepth); gl.enable(gl.DEPTH_TEST); gl.enable(gl.BLEND); gl.blendFunc(gl.SRC_ALPHA, gl.ONE_MINUS_SRC_ALPHA); return gl; }

33. Load Shader function loadShader(ctx, shaderId) { varshaderScript = document.getElementById(shaderId); if (!shaderScript) { ctx.console.log("*** Error: shader script '"+shaderId+"' not found"); return null; } if (shaderScript.type == "x-shader/x-vertex") varshaderType = ctx.VERTEX_SHADER; else if (shaderScript.type == "x-shader/x-fragment") varshaderType = ctx.FRAGMENT_SHADER; else { ctx.console.log("*** Error: shader script '"+shaderId+"' of undefined type '"+shaderScript.type+"'"); return null; }

34. // Create the shader object varshader = ctx.createShader(shaderType); if (shader == null) { ctx.console.log("*** Error: unable to create shader '"+shaderId+"'"); return null; } // Load the shader source ctx.shaderSource(shader, shaderScript.text); // Compile the shader ctx.compileShader(shader); // Check the compile status var compiled = ctx.getShaderParameter(shader, ctx.COMPILE_STATUS); if (!compiled) { // Something went wrong during compilation; get the error var error = ctx.getShaderInfoLog(shader); ctx.console.log("*** Error compiling shader '"+shaderId+"':"+error); ctx.deleteShader(shader); return null; } return shader; }

35. Array Buffers function makeBox(ctx) { // vertex coords array var vertices = new Float32Array( [ 1, 1, 1, -1, 1, 1, -1,-1, 1, 1,-1, 1, ….. ); // normal array varnormals = new Float32Array( [ 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, …. // texCoord array vartexCoords = new Float32Array( [ 1, 1, 0, 1, 0, 0, 1, 0 ,…. // index array var indices = new Uint8Array( [ 0, 1, 2, 0, 2, 3, …. ); varretval = { }; retval.normalObject = ctx.createBuffer(); ctx.bindBuffer(ctx.ARRAY_BUFFER, retval.normalObject); ctx.bufferData(ctx.ARRAY_BUFFER, normals, ctx.STATIC_DRAW); retval.texCoordObject= ctx.createBuffer(); ctx.bindBuffer(ctx.ARRAY_BUFFER, retval.texCoordObject); ctx.bufferData(ctx.ARRAY_BUFFER, texCoords, ctx.STATIC_DRAW); retval.vertexObject = ctx.createBuffer(); ctx.bindBuffer(ctx.ARRAY_BUFFER, retval.vertexObject); ctx.bufferData(ctx.ARRAY_BUFFER, vertices, ctx.STATIC_DRAW); ctx.bindBuffer(ctx.ARRAY_BUFFER, null); retval.indexObject = ctx.createBuffer(); ctx.bindBuffer(ctx.ELEMENT_ARRAY_BUFFER, retval.indexObject); ctx.bufferData(ctx.ELEMENT_ARRAY_BUFFER, indices, ctx.STATIC_DRAW); ctx.bindBuffer(ctx.ELEMENT_ARRAY_BUFFER, null); retval.numIndices = indices.length; return retval; }

36. Buffer Objects WebGL reference card – page 1

37. Typed Array http://www.khronos.org/registry/typedarray/specs/latest/

38. Texture // // loadImageTexture // // Load the image at the passed url, place it in a new WebGLTexture object and return the WebGLTexture. // function loadImageTexture(ctx, url) { var texture = ctx.createTexture(); texture.image = new Image(); texture.image.onload = function() { doLoadImageTexture(ctx, texture.image, texture) } texture.image.src = url; return texture; } function doLoadImageTexture(ctx, image, texture) { ctx.bindTexture(ctx.TEXTURE_2D, texture); ctx.texImage2D(ctx.TEXTURE_2D, 0, ctx.RGBA, ctx.RGBA, ctx.UNSIGNED_BYTE, image); ctx.texParameteri(ctx.TEXTURE_2D, ctx.TEXTURE_MAG_FILTER, ctx.LINEAR); ctx.texParameteri(ctx.TEXTURE_2D, ctx.TEXTURE_MIN_FILTER, ctx.LINEAR); ctx.texParameteri(ctx.TEXTURE_2D, ctx.TEXTURE_WRAP_S, ctx.CLAMP_TO_EDGE); ctx.texParameteri(ctx.TEXTURE_2D, ctx.TEXTURE_WRAP_T, ctx.CLAMP_TO_EDGE); ctx.bindTexture(ctx.TEXTURE_2D, null); }

39. WebGL reference card page 2

40. Pointers http://www.khronos.org/webgl/ Public email list http://www.khronos.org/developers/library/siggraph2006/OpenKODE_Course/03_GLSL-for-OpenGL-ES.pdf http://learningwebgl.com http://planet-webgl.org/ http://www.webglcamp.com ….

41. Middleware https://github.com/greggman/tdl http://scenejs.org/ https://github.com/mrdoob/three.js/ http://alteredqualia.com/ http://code.google.com/p/o3d/ http://senchalabs.github.com/philogl/ Lessons http://www.everyday3d.com/j3d/ …..

42. Going from COLLADA to WebGL

43. Loading COLLADA (or X3D) Spore COLLADA viewer (client XML parser) xml.evaluate(XPATH) Javascript XML cheat sheet http://weblogs.asp.net/rrobbins/archive/2008/09/10/javascript-xml-cheat-sheet.aspx Google O3D – preprocessed to json http://code.google.com/p/o3d/wiki/ColladaConverter O3D mod – (client XML parser) 2010 http://www.colladawebviewer.com/ 2011 http://capstone.azurenet.net/web3d/demo/ Scene js – (server preprocessed) http://scenejs.wikispaces.com/scenejs-pycollada (server side python) http://pycollada.github.com/ (PyCollada) X3dom – (client DOM) X3D integrated in the Xhtml page http://www.x3dom.org/

44. Typical (COLLADA) xml ‘model’ http://scenejsorg.ipage.com/dist/curr/extr/examples/tron-tank/extras/tron-tank-model.dae

45. XML & XHTML varfloat_array=textValue.split(/+/).map(parseFloat); DOM (jquery) allow for fast tree parsing/modifications Direct XML export (POST/PUT) to server

46. Typical json ‘model’ https://github.com/mrdoob/three.js/blob/master/examples/obj/Suzanne.js myData = JSON.parse(text, function (key, value) { if (value && typeof value === 'object') {… return value; }}); Need convention for attributes vs value, not easy to go back to XML

47. Typed Array loading function onLoad(){ var canvas = document.getElementById("lesson01-canvas"); initGL(canvas); varxmlhttp = new XMLHttpRequest(); xmlhttp.open("GET", "cube.bm", true); xmlhttp.responseType = "arraybuffer"; xmlhttp.onload = function() { var buffer = xmlhttp.response; gl.bindBuffer(gl.ARRAY_BUFFER, buffer); // Reading Data var v1 = new Float32Array(buffer); alert(v[0]); // This works fine. } xmlhttp.send(); }

49. Google Body Javascript / WebGL / tdl library Meshes grouped by texture – 63 draw calls, 1800+ elements Index and attribute array merged (65K max) 16-bits fixed point for pos, normal, textcoords Watch http://www.youtube.com/watch?v=vsHHNClzJPg

50. Simple, Fast, Compact Meshes for WebGL Won Chun (@won3d on Twitter) Google Web Search Infrastructure JSON / XML :-( ASCII v. Binary :-) Quantization :-) Compression :-)

51. What is REST ? Roy Fielding dissertation (2000) http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm Representational State Transfer (REST) is an architecture, not a standard. REST is based on http, xml, uristandards REST is everywhere already twitter API http://dev.twitter.com/doc Dropbox API https://www.dropbox.com/developers/docs Google MAP API Web services http://code.google.com/apis/maps/documentation/webservices/index.html ….

53. Stateless Client context is not stored on the server between requests. Each request from any client contains all of the information necessary to service the request, and any session state is held in the client. The server can be stateful; this constraint merely requires that server-side state be addressable by URL as a resource

54. Cacheable As on the World Wide Web, clients are able to cache responses. Responses must therefore, implicitly or explicitly, define themselves as cacheable, or not, to prevent clients reusing stale or inappropriate data in response to further requests. Well-managed caching partially or completely eliminates some client–server interactions, further improving scalability and performance.

55. Layered system A client cannot ordinarily tell whether it is connected directly to the end server, or to an intermediary along the way. Intermediary servers may improve system scalability by enabling load balancing and by providing shared caches. They may also enforce security policies.

56. Code on demand Servers are able to temporarily extend or customize the functionality of a client by transferring logic to it that it can execute. Examples of this may include client-side scripts such as JavaScript.

57. Uniform interface The uniform interface between clients and servers simplifies and decouples the architecture, which enables each part to evolve independently. The four guiding principles of this interface are:Identification of resources (e.g. URI), Manipulation of resources, Self-descriptive messages, Hypermedia as the engine of application statehttp://http://en.wikipedia.org/wiki/Representational_State_Transfer

58. methods PUT and DELETE are defined to be idempotent - multiple identical requests should have the same effect as a single request POST is not necessarily idempotent, and therefore sending an identical POST request multiple times may further affect state or cause further side effects See also HEAD, OPTIONS and TRACE methods Potential use: http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html

59. What is REST 3D ? 3D as a web service http://rest3d request Web Server Client application Response document Cloud Storage

60. Multiple services – same API Web Server Web Server Web Server Client application Cloud Storage Cloud Storage Cloud Storage

61. Multiple applications Client application Web Server Web Server Web Server Client application Cloud Storage Cloud Storage Cloud Storage Client application

62. rest 3d methods Potential use model: http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html

63. Use case 1 – content gathering Program interface to content repositories. Current state requires human in the loop, and content creation tool to explore the model. http://sketchup.google.com/3dwarehouse/ http://www.3dvia.com/ http://www.3dcadbrowser.com/ http://www.3d02.com/ http://www.turbosquid.com/ …

65. browse – Avoid downloading all the 3D models that matches the search, this feature provides information about the models such as number of polygons, date, tool used to create the model, …

66. traverse – explore the model hierarchy, and sub parts or/and categories.

67. dependencies - list all the dependencies, for instance the list of textures, shaders, or other parts of the model

68. filter – select only the parts needed by the client. Allows for texture/3D LOD selection, paging…

70. scene – Create a scene and position the models in the scene

71. viewer – extend the client with a viewer application (<embed>) of the created scene.

72. collaborate – enable multiple applications/users to interact with the scene

73. store – store the created scene for others to discover

75. update – modify an existing model

76. version – server list all the different versions available, with metadata

77. diff / merge – provide tools to show the difference between versions

78. process – provide tools to process the models, mesh optimization, texture atlas,

80. Leaving the REST to YOU? Official web page www.rest3d.org Join the discussion: http://groups.google.com/group/3d-rest