University of Crete

Department of Computer Science

Δημήτρης Καμηλέρης (kamileri@csd.uoc.gr)

Ηράκλειο 22/8/2012

Επιβλέπων καθηγητής: Γεώργιος Παπαγιαννάκης

Dimitris Kamileris

Heraklion 22/8/2012

Instructor: George Papagiannakis

Diploma thesis:

Real-time shader-based Per-fragment lighting for dynamic objects in WebGL

2

Περίληψη

H WebGL είναι μια νέα γλώσσα προγαμματισμού που ειδικεύεται στην ανάπτυξη τρισδιάστατων

εφαρμογών για μοντέρνους φυλλομετρητές (web browsers). Σε αυτήν την εργασία εξετάζονται

τα βασικά σημεία αυτής της γλώσσας και αναλύονται κάποιοι από τους μηχανισμούς λειτουργίας

της. Αυτό γίνεται υλοποιόντας το ανα-εικονοτεμαχίδιο (per-fragment) μοντέλο φωτισμού Phong.

Abstract

WebGL is a new programming language oriented in developing 3D applications for modern web

browsers. This thesis goes over the basic points of WebGL and explains some of the

mechanisms. This is done by implementing the per-fragment Phong reflection model on a

dynamic 3D object.

3

Table of Contents

Introduction ...................................................................................................................... 4

WebGL compatible browsers ............................................................................................... 5

WebGL application structure ............................................................................................... 5

Development of a WebGL application ................................................................................... 5

The WebGLRenderingContext.............................................................................................. 5

The WebGL rendering pipeline ............................................................................................ 6

The WebGL buffers ............................................................................................................ 8

Constructing the buffers ............................................................................................... 8

Rendering the buffers .................................................................................................. 9

Shaders and Lighting .........................................................................................................11

Initializing the shaders ................................................................................................11

Attributes and Uniforms ..............................................................................................12

Attributes ...............................................................................................................12

Uniforms ................................................................................................................13

Normals .................................................................................................................13

Lighting: the Phong reflection model ............................................................................14

Per-Fragment Phong implementation ...........................................................................15

The viewing transformations ..............................................................................................17

Movement ........................................................................................................................19

Textures ...........................................................................................................................20

Loading a Collada 3D object ...............................................................................................23

Conclusion - Future plans ..................................................................................................26

4

Introduction

The need for high performance that dominates every branch of computer science could

not have let uninfluenced webpage developers who need fast hardware accelerated interactive

3D graphics. So far the only way of rendering graphics on the web was by loading separate

applications as plug-ins. Plug-ins are often problematic and need a user-side installation.

Applications what are executed over plug-ins are usually developed in high level languages

which tend to have worse performance than low level languages. In order to cover the needs of

3D web applications, Khronos™ group has developed WebGL™.

WebGL is a new technology integrated into web standards (e.g. HTML, DOM) and allows

rendering hardware accelerated interactive 2D and 3D graphics within any major modern web

browser without the use of plug-ins. To be more specific WebGL is a low-level JavaScript API

based on OpenGL ES 2.0 exposed through the HTML 5 canvas element. The HTML

CanvasElement is a new feature adopted since HTML 5 and allows for dynamic rendering of 2D

or 3D scenes via scripting languages (e.g. JavaScript). Like the OpenGL ES 2.0, WebGL is

Shader-based using the GLSL shading language. –Shaders are small programs that are

executed by the Graphics Processing Unit (GPU) –. WebGL is cross-platform and royalty-free the

only restriction of using it is that the user must have a compatible web browser. Luckily most

major web browsers Safari, Chrome, Firefox and Opera support WebGL on their newest

distributions.

Another issue of modern 3D application development is the vast majority of different 3D

object file formats. Each modeling software (Maya, 3D Studio Max, Blender etc) uses its own

file format and this creates compatibility problems and makes sharing and transporting difficult

among 3D applications. To solve this problem the Collada file format has been introduced.

Collada introduces a new XML (Extensible Markup Language) schema to make it easy to

transport 3D assets between applications without loss of information or compatibility problems.

In this thesis I explain how WebGL works in detail and how it renders 3D objects. I also

explain how a Collada 3D object can be loaded into a WebGL application and how it can be

illuminated using GLSL shaders.

5

WebGL compatible browsers

As mentioned earlier WebGL is based completely plug-in free so the only restriction in

order to run and start developing WebGL applications is to have a compatible browser. Mozilla

Firefox supports WebGL since version 4.0, Google Chrome since version 9, Opera 11 and 12

also support WebGL. Safari supports WebGL since version 5.1 but it is disabled by default so it

needs to be enabled in order to view WebGL content. Finally Microsoft Internet Explorer does

not support WebGL and so far Microsoft has not shown any intention of supporting it in the

future.

WebGL application structure

A typical WebGL application consists of the following components:

Canvas: the canvas consists of a drawable region defined in HTML code with height

and width attributes and is accessible through JavaScript code. Canvas is the

placeholder where the scene will be rendered.

Objects: WebGL buffers contain all the data necessary in order to render the 3D

objects that make up the scene.

Lighting: Lighting makes the 3D scene visible. WebGL uses GLSL shaders in order to

illuminate the scene created by the buffers and to add a more realistic tone.

Development of a WebGL application

The WebGLRenderingContext

A WebGL context is a JavaScript object through which we can access all the WebGL functions and attributes this means WebGL's Application Program Interface (API). As mentioned in the introduction WebGL is exposed through the canvas element so in order to

develop a WebGL application the first step is to create a canvas.

<canvas id="Canvas1" width="640" height="480"></canvas>

This line of code is the only HTML code really necessary inside the HTML body. The rest of the

application is written in JavaScript. After creating the canvas it is time to initialize the WebGL

context. In order to do it the canvas created in the HTML body is needed. The way to get the

canvas into the script is to use JavaScript’s method getElementById. This method will find

6

any HTML element with the specified id on the current page or as it is referred in the HTML

“document” and return it.

var canvas = document.getElementById("Canvas1");

The WebGL context now can be created by the canvas

var gl = canvas.getContext("experimental-webgl");

The context name "experimental-webgl" is temporary and will be replaced by "webgl"

soon in the future.

At this point a WebGLRenderingContext object has been created and the WebGL API can be

accessed by the variable gl. For example the function clearColor that WebGL uses to

clear the canvas is called by this manner: gl.clearColor(0.0, 0.0, 0.0, 1.0); The

four float values passed as parameters are the RGBa values of the desirable color.

The WebGL rendering pipeline

Here follows a simplified diagram of WegGL’s rendering pipeline and a simple description of

each segment.

Buffers Attributes

Vertex Shader

Uniform Variables

JavaScript

Fragment Shader

Varying Variables

Frame Buffer

7

1. Buffers: The buffers contain the data that WebGL needs in order to render anything

from a simple 2D shape to a full 3D complicated scene. There are several data objects

that buffers can contain such as vertex coordinates, vertex normals and texture

coordinates.

2. Vertex Shader: The vertex shader operates on each vertex and handles per-vertex data

such as vertex and texture coordinates colors and vertex normals. This data is

represented by attributes inside the vertex shader. Each attribute points to the buffer

from where it reads the data.

3. Fragment Shader: Every three vertices form up a triangle, the surface of this triangle

consists of elements called fragments fragment shader’s job to calculate the color of

each fragment.

4. Attributes: are input variables used in the vertex shader. Because the vertex shader is

called upon each vertex, the attributes will be different every time the shader is called.

5. Uniforms: are input variables for the vertex and also the fragment shader. Uniforms do

not change throughout the rendering cycle.

6. Varyings: are used to pass data from the vertex to the fragment shader.

7. Framebuffer: The framebuffer is the buffer that contains the fragments already

processed by the fragment shader. Once the fragment shader has completed its job the

framebuffer is imprinted on the screen.

8

The WebGL buffers

Constructing the buffers

WebGL buffers are constructed from JavaScript arrays because the API does not provide

methods to pass independent vertices to the rendering pipeline like OpenGL does. In order to

create 3D objects there are used two kinds of buffers the vertex and the index buffers. The

vertex buffer that contains the coordinates of all the vertices those make up the scene, and the

index buffer. The index buffer contains numbers that correspond in vertices in the vertex buffer.

Indices show how the vertices connect in order to form up triangles.

For example the vertex array for the square in Figure 1 is

x y z x y z x y z x y z

[0,0,0, 1.0,0,0, 0,1.0,0, 1.0,1.0,0]

vertex0 vertex1 vertex2 vertex3

and the index array is

[0,1,2,1,2,3]

The index array indicates that the first surface is created by the vertices 0, 1 and 2 and the

other by the vertices 1, 2 and 3.

Once the vertex and the index arrays are created, the respective buffers can be created

as well.

var vertex = [0,0,0, 1.0,0,0, 0,1.0,0, 1.0,1.0,0]; //the arrays as

var index = [0,1,2,1,2,3]; //created previously

var mesh = gl.createBuffer(); //the variable gl holds the WebGL context

The variable mesh is initialized as a WebGL buffer. Because of the fact that buffers are part of

the graphics card memory and not the main memory (RAM), accessing them is not as simple.

In order to perform any operation at a given buffer it must be set as the current buffer. This

means that after setting a buffer as the current buffer any buffer-related operation will be

executed on this buffer.

gl.bindBuffer(gl.ARRAY_BUFFER, mesh);

The first parameter is the type of buffer. There are two types of buffers using the one or the

other depends on the type of data this buffer will hold. The gl.ARRAY_BUFFER holds vertex

data (i.e. vertex coordinates, vertex colors, texture coordinates, and vertex normals). The

gl.ELEMENT_ARRAY_BUFFER is used to hold index data.

Since the buffer mesh is the current buffer the contents can be passed.

Figure 1

9

gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(vertex), gl.STATIC_DRAW);

vertex is the JavaScript array created previously holding the vertex coordinates. The

Float32Array is a WebGL data type called typed array and it is used instead of the

JavaScript array because it is in binary form. The WebGL typed arrays are Int8Array, Uint8Array, Int16Array,Uint16Array, Int32Array, UInt32Array,

Float32Array, and Float64Array.

At this point the JavaScript array containing the vertex coordinates is passed in to the respective buffer and it is time to do the same for the index array. var indexbuf = gl.createBuffer(); gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, indexbuf);

gl.bufferData(gl. ELEMENT_ARRAY_BUFFER, new Uint16Array (index),

gl.STATIC_DRAW);

Rendering the buffers

In principle the buffers can now be rendered on the screen. Of course without the shaders nothing would be visible but since the shaders will be analyzed a bit further down and the rendering method remains the same there is no harm in showing how the buffers are rendered here.

There are two functions that can render the buffers, the drawArrays and the

drawElements. The first uses the ARRAY_BUFFER in order to render the meshes, the

second uses the ELEMENT_ARRAY_BUFFER in order to access the vertex buffer and then

render on the screen. The drawArrays function is used when the mesh is simple enough for

example a triangle or a square this means that for the more complicated meshed the

drawElements function is used and there is a good reason for it. drawArrays renders

every vertex in the order found in the buffer this means that in order to draw more complicated objects the same vertices need to be more than once in the buffers. The first and obvious problem is the increased buffer size. The second problem is that because there are more vertices the vertex shader will be called more times spoiling the performance. On the other hand when using indices each vertex needs to be in the buffers only once and the vertex shader is called only once upon each vertex. By using the element array buffer for rendering there is a significant improvement of the performance and a great reduction of the required memory. gl.drawArrays(Mode, First, Count);

Mode: The mode specifies what kind of primitives are constructed. Accepted values

are: gl.POINTS, gl.LINES, gl.LINE_STRIP, gl.LINE_LOOP, gl.TRIANGLES, gl.TRIANGLE_STRIP, gl.TRIANGLE_FAN.

First: specifies the first to be rendered element.

Count: specifies the number of elements to be rendered.

gl.drawElements(Mode, Count, Type, Offset);

10

Mode: The mode specifies what kind of primitives are constructed. Acceptable

values are: gl.POINTS, gl.LINES, gl.LINE_STRIP, gl.LINE_LOOP, gl.TRIANGLES, gl.TRIANGLE_STRIP, gl.TRIANGLE_FAN.

Count: specifies the number of elements to be rendered.

Type: indicates the data type of the values in the index buffer. Acceptable values

are: UNSIGNED_BYTE UNSIGNED_SHORT

Offset: Specifies the first to be rendered element

As mentioned earlier no operation can be executed to a buffer unless it is set as the current

buffer. Rendering it is no exception so before calling either the gl.drawArrays or the

gl.drawElements, the gl.bindBuffer must be called.

11

Shaders and Lighting

Initializing the shaders

The shaders are small programs that are executed by the graphics card and are usually used to model effects such as lights reflections and shades. The shaders are independent from the WebGL application they are even written in a different programming language. The GLSL (OpenGL Shading Language) is used for this purpose. In order for the shaders to be used they need to be compiled and linked to the WebGL application. This process is executed by the WebGL application itself. To be more precise shaders are loaded as string, compiled and attached to a program (the fragment and vertex shaders together are called a program) and then linked to the WebGL application and finally the program is used by the WebGL application. var shaderScript = document.getElementById(shaderid);

var str = "";

var k = shaderScript.firstChild;

while (k) {

if (k.nodeType == 3) {

str += k.textContent;

}

k = k.nextSibling;

}

The piece of code above loads a shader as a string using the DOM tree. Once the var str

contains the loaded source code it’s time to create a shader containing that source code.

WebGL provides the createShader function which creates either a fragment or a vertex

shader. var shader;

if (shaderScript.type == "x-shader/x-fragment") {

shader = gl.createShader(gl.FRAGMENT_SHADER);

} else if (shaderScript.type == "x-shader/x-vertex") {

shader = gl.createShader(gl.VERTEX_SHADER);

} else {

return null;

}

gl.shaderSource(shader, str);

And now compile the shader gl.compileShader(shader);

if (!gl.getShaderParameter(shader, gl.COMPILE_STATUS)) {

alert(gl.getShaderInfoLog(shader));

return null;

}

12

This process must be performed for both the vertex and the fragment shader. After the compilation has been successfully completed the shaders need to be attached to a program and then the program needs to be linked shaderProgram = gl.createProgram();

gl.attachShader(shaderProgram, vertexShader);

gl.attachShader(shaderProgram, fragmentShader);

gl.linkProgram(shaderProgram);

if (!gl.getProgramParameter(shaderProgram, gl.LINK_STATUS)) {

alert("Could not initialise shaders");

}

Finally use the program gl.useProgram(shaderProgram);

At this point the GLSL shaders are part of the rendering pipeline. Attributes and Uniforms

As mentioned earlier attributes and uniforms are input variables to the vertex and

fragment shaders. Input can be any information related to the shaders’ job for example the vertex coordinates for the vertex shader or coloring data for the fragment shader.

Attributes

The buffers can feed their data to the vertex shader by being associated with an

attribute. Each buffer is associated with one and only one attribute. In order to point an attribute to a buffer first of all the buffer must become the current buffer then WebGL’s

function vertexAttribPointer points the attribute to the buffer.

gl.vertexAttribPointer(Index,Size,Type,Norm,Stride,Offset);

Index: refers to the attribute that is going to be associated with the current buffer.

Size: refers to the size of the current buffer.

Type: refers to the data type of the values stored in the current buffer. Acceptable values

are: FIXED, BYTE, UNSIGNED_BYTE, FLOAT, SHORT, or UNSIGNED_SHORT.

Norm: (boolean)Specifies whether fixed-point data values should be normalized or

converted directly as fixed-point values when they are accessed.

Stride: specifies the byte offset between consecutive generic vertex attributes. If

stride is 0, the generic vertex attributes are understood to be tightly packed in the array. The initial value is 0.

Offset: specifies the position in the buffer from which we will start reading

values for the corresponding attribute.

13

The most common attributes and the ones used in the context of this thesis are: The

vertexPositionAttribute: “aVertexPosition” bound to the buffer that contains the

vertex coordinates, the vertexNormalAttribute: “aVertexNormal” bound to the

normals buffer (will be explained shortly) and finally the textureCoordAttribute:

“aTextureCoord” bound to the buffer that contains the texture coordinates.

Of course before associating the buffers with the attributes they need to be enabled. shaderProgram.vertexPositionAttribute = gl.getAttribLocation(shaderProgram,

"aVertexPosition");

gl.enableVertexAttribArray(shaderProgram.vertexPositionAttribute);

Uniforms

Uniform variables follow the same principles with the attributes but because they remain

constant throughout the rendering cycle (consecutive calls of drawElements) their values

need to be set whenever they change and not in every cycle. Initialization is done like this:

shaderProgram.ambientColorUniform = gl.getUniformLocation(shaderProgram,

"uAmbientColor");

And pass values like this: gl.uniform3f(shaderProgram.ambientColorUniform,0.2,0.2,0.2);

Normals

A normal is a vector perpendicular to a given surface. A normal vector is used to

calculate the effect of a light source to a given surface because the vector defines an orientation for the surface. In simple words a normal vector can tell were the surface is faced. As expected surfaces that are faced towards the light source, will be brighter than ones that face in other directions.

In order to calculate the normals vector, the cross product is used because the cross product of two vectors A and B will give a vector perpendicular to both vectors A and B.

14

Lighting: the Phong reflection model

Lighting is a very important part of a 3D scene because it adds realism. In the context of

this thesis the Phong reflection model is analyzed and by using shaders, applied to a WebGL scene. Real world objects are visible because they reflect light. This comes from light that is either reflected from other objects or surfaces or comes directly from a light source. Each object reacts differently to the incident light depending on the material of the object. Rough surfaces scatter the light in every direction (diffuse reflection) while smooth ones tend to reflect it in a mirror-like fashion (specular reflection). The Phong reflection model is a combination of the diffuse reflection, the specular reflection and also an effect of the ambient light. The Phong reflection model is very accurately represented by the following image.

The Ambient reflection comes from light scattered around the scene. This type of reflection is independent from any light source and its effect is the same for each fragment of the illuminated object.

The diffuse reflection effect is light source dependent which means that each fragment diffuses different amount of light depending on its position compared to the light source. Assuming that the light source is a vector the amount of diffused light is calculated by the cosine of the angle between the vector of the light source and the normal of this fragment. Instead of calculating the cosine, which is a very time consuming operation, if both vectors have a length of one, the dot product of the two vectors can be used. Luckily even if a vector has not length of one it can be converted to such a vector via an operation called normalizing. Normalizing a vector involves two steps: 1 calculate its length, then, 2 divide each of its (xy or xyz) components by its length. Given vector a its xyz components are calculated as follows,

x = ax/|a| y = ay/|a|

15

z = az/|a|

The specular reflection is mirror-like reflection of light from a surface, in which light from a single incoming direction is reflected into a single outgoing direction. The intensity of the reflected light that is visible depends on the surface it reflects on, and the relative position between the light source and the observer. When light hits a mirror the angle of the reflected light is the same as the angle of the incident light. Like the specular light the amount of the reflected light is proportional to the cosine of two vectors at this case the one is the vector of a ray that was reflected of the surface like it hit a mirror (from now on it will be called R) and the other is the vector pointing in the observer’s direction (from now on called V). Of course in terms with calculations economy the dot product is used in this case again. So this far the observer’s point of view and the position of the light source have played their part in the calculation the only remaining is the material. In order to take under consideration this factor the dot product of (normalized) R and V is exponentiated to a constant describing the material’s shininess α

Per-Fragment Phong implementation

The per-fragment term means that the light effect is calculated for each pixel separately. This means that the calculations are done by the fragment shader. Here follows an implementation of the per-fragment Phong reflection model fragment shader. <script id="shader-fs" type="x-shader/x-fragment">

precision mediump float;

varying vec2 vTextureCoord;

varying vec3 vTransformedNormal;

varying vec4 vPosition;

uniform float uMaterialShininess;

uniform bool uUseTextures;

uniform vec3 uAmbientColor;

uniform vec3 uPointLightingLocation;

uniform vec3 uPointLightingSpecularColor;

uniform vec3 uPointLightingDiffuseColor;

uniform sampler2D uSampler;

16

void main(void) {

vec3 lightWeighting;

//Calculate the light source’s direction

vec3 lightDirection = normalize(uPointLightingLocation -

vPosition.xyz);

vec3 normal = normalize(vTransformedNormal);

//Calculate how much specular reflection the viewers eye can see

float specularLightWeighting = 0.0;

vec3 eyeDirection = normalize(-vPosition.xyz);

vec3 reflectionDirection = reflect(-lightDirection, normal);

specularLightWeighting = pow(max(dot(reflectionDirection,

eyeDirection), 0.0), uMaterialShininess);

//Calculate the diffuse light

float diffuseLightWeighting = max(dot(normal, lightDirection),

0.0);

//Calculate the aggregate effect

lightWeighting = uAmbientColor

+ uPointLightingSpecularColor * specularLightWeighting

+ uPointLightingDiffuseColor * diffuseLightWeighting;

vec4 fragmentColor;

//Find out the color of the fragment before it is illuminated

fragmentColor = texture2D(uSampler, vec2(vTextureCoord.s,

vTextureCoord.t));

//Apply the effect of the light on the fragment

gl_FragColor = vec4(fragmentColor.rgb * lightWeighting,

fragmentColor.a);

}

</script>

17

The viewing transformations

In order to draw an object contained in the buffers to the screen it needs to go through a series of conversions. These conversions translate the 3D coordinates contained in the buffers to the 2D world that is the screen. Until computer screens are replaced by 3D holograms this procedure is essential so that the viewer can perceive the essence of a 3D scene through a 2D means. The process of coordinate conversion begins by converting the world coordinate system to the eye coordinate system. This is done by the model and view transformations. The first defines the object’s position in the scene and the other transforms the orientation of the world to the orientation of the viewer’s eye. The second step of the conversion is to transform the eye coordinate system to the canonical screen space this is implemented by the projection transformation. The projection transformation has two different versions the perspective and the orthographic projection. The first takes into consideration the fact that objects in a distance are smaller than objects closer to the viewer’s eye while the second does not. This property makes the perspective projection suitable for drawing 3D scenes where depth maters such as a room, and the orthographic projection suitable to show profile, or precise measurements of a 3D object such as architectural plans. The final step of the conversion is to transform the canonical screen space to the viewport which in WebGL’s case is the canvas. This means that the canonical screen space is set to fit to the exact size that the canvas was set to during initialization.

In WebGL, the matrices that perform the transformations are the model-view matrix and the

projection matrix. The viewport transformation is done by the viewport function.

World Coordinate System

Eye Coordinate System

Canonical Screen Space

Model and View

Transformations

Projection

Transformation

Viewport

Transformation

Canvas

18

gl.viewport(minX, minY, width, height);

Because WebGL has no implementation of array as a native data type, JavaScript arrays are

used and because JavaScript does not have methods implementing the linear algebra

operations needed for the transformation, in this thesis I used a third-party matrix library

Brandon Jones‘s, glMatrix. glMatrix offers a method that applies the perspective projection

transformation

mat4.perspective(45, gl.viewportWidth / gl.viewportHeight, 0.1,

1000.0, pMatrix);

The first argument is the angle of the vertical field of view, the second is the aspect ratio of the

canvas the third is the nearest bound of the viewport and the fourth the far end. The last is the

destination matrix.

In every rendering cycle the model-view and the projection matrices need to be uploaded to the

shaders

function setMatrixUniforms() {

gl.uniformMatrix4fv(shaderProgram.pMatrixUniform, false,

pMatrix);

gl.uniformMatrix4fv(shaderProgram.mvMatrixUniform, false,

mvMatrix);

var normalMatrix = mat3.create();

mat4.toInverseMat3(mvMatrix, normalMatrix);

mat3.transpose(normalMatrix);

gl.uniformMatrix3fv(shaderProgram.nMatrixUniform, false,

normalMatrix);

}

The first two lines pass as uniform variables the mvMatrix and the pMatrix. When vertices are

transformed, normal vectors should also be transformed in order to remain perpendicular. The

correct Normal matrix is obtained by transposing the inverse of the Model-View matrix. This

Normal matrix is used (in the vertex shader) to multiply the normal vectors in order to remain

perpendicular to the surface during a transformation.

vTransformedNormal = uNMatrix * aVertexNormal;

19

Movement

Moving objects around is what gives action to a scene. Moving objects can be achieved

by mathematical transformations on the model-view matrix. The problem is that the model-view

matrix contains all the objects of a scene and in order to move only a subset of them matrix

stacks must be used to make sure that any modification affect only the object that needs to be

moved and not anything else.

Moving an object requires three steps. The first step is to save (pushMatrix) the current

model-view matrix to a stack. The second step is to apply the transformation, by multiplying the

original model-view matrix by the transformation matrix that represents the respective

operation (rotation, translation, scale). The new model-view matrix needs to be passed to the

program (shaders) and then draw it using either the drawArrays or the drawElements

functions. The third step is to recover (popMatrix) the original model-view matrix from the

stack.

Here follows the implementation of pushMatrix and popMatrix using the glMatrix library

function mvPushMatrix() {

var copy = mat4.create();

mat4.set(mvMatrix, copy);

mvMatrixStack.push(copy);

}

function mvPopMatrix() {

if (mvMatrixStack.length == 0) {

throw "Invalid popMatrix!";

}

mvMatrix = mvMatrixStack.pop();

}

The procedure described above moves only one object and performs only one movement

for example it can rotate an object for some degrees. In order to see motion this procedure

needs to be called several times every second. Just like animated films the scene is drawn every

time with the object that needs to be moved, in a slightly different position. If this happens in a

rate greater than thirty scenes every second, the human eye cannot distinct each different

scene an sees a smoothly moving object.

In WebGL the way to achieve rendering a scene in such a frame rate is to use a loop that executes an iteration every given time interval. During an iteration there is time to calculate the position of each object and render the scene. In order to control the time interval of each

iteration, JavaScript offers the requestAnimFrame function. Because so far each browser has

a different name for this function here follows an elegant solution by Google.

20

requestAnimFrame = (function() {

return window.requestAnimationFrame ||

window.webkitRequestAnimationFrame ||

window.mozRequestAnimationFrame ||

window.oRequestAnimationFrame ||

window.msRequestAnimationFrame ||

function(/* function FrameRequestCallback */ callback, /* DOMElement

Element */ element) {

window.setTimeout(callback, 1000/60);

};

})();

This function is called every 1000/60 milliseconds providing a very good frame rate for the loop.

function mainLoop() {

requestAnimFrame(mainLoop);

drawScene();

animate();

}

Textures

Textures are bitmap images that are applied to the surface of a 2D or 3D object in order to add

detail.

In order to use an image as a texture in WebGL it needs to be loaded

Texture = gl.createTexture();

Texture.image = new Image();

Texture.image.onload = function ()

{

handleLoadedTexture(Texture)

}

Texture.image.src = "texture.jpg";

At this point the variable Texture has been initialized as a WebGL texture and an additional

attribute is added to this variable that holds the image. After the image has been loaded, is

time to initialize the texture so it can be used. First of all the texture needs to be flipped

vertically because the texture’s coordinate system and the image’s coordinate system are

opposite on the vertical axis. Texture coordinates follow the Cartesian system, the y parameter

increases as the y axis moves away from the x axis while images are encoded in the opposite

way.

gl.pixelStorei(gl.UNPACK_FLIP_Y_WEBGL, true);

21

In WebGL textures are manipulated like the buffers. Just as in order to perform any operation

on a buffer it needs to be set as the current buffer, textures need to be set as the current

texture in order to use them.

gl.bindTexture(gl.TEXTURE_2D, texture);

As expected textures need to be uploaded to the graphics card. This is done by the following

function call

gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE,

texture.image);

The first parameter defines what kind of image is uploaded, the second is the level of detail

level 0 is the base image level. The two following parameters specify the format in which the

texture is loaded in the graphics card. The next specifies the data type of the data and the last

the image itself.

The following step is to specify some scaling parameters of the texture. These parameters

configure how the texture is scaled up or scaled down when this is necessary.

gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);

gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER,

gl.LINEAR_MIPMAP_NEAREST);

The gl.TEXTURE_MAG_FILTER argument sets the texture magnification method to either

gl.NEAREST or gl.LINEAR. The first returns the value of the texture element that is nearest

to the center of the pixel being textured. The second returns the weighted average of the four

texture elements that are closest to the center of the pixel being textured. Respectively the

gl.TEXTURE_MIN_FILTER argument sets the texture’s minifying method. There are six

methods the first two are the gl.NEAREST and gl.LINEAR which work exactly as mentioned

above. The other four use mipmaps. A mipmap is an ordered set of arrays representing the

same image at progressively lower resolutions.

gl.NEAREST_MIPMAP_NEAREST Chooses the

mipmap that most closely matches the size of

the pixel being textured and uses the NEAREST

criterion to produce a texture value.

gl.LINEAR_MIPMAP_NEAREST Chooses the

mipmap that most closely matches the size of

the pixel being textured and uses the LINEAR

criterion to produce a texture value.

gl.NEAREST_MIPMAP_LINEAR Chooses the

22

two mipmaps that most closely match the size of the pixel being textured and uses the

NEAREST criterion to produce a texture value from each mipmap. The final texture value is a

weighted average of those two values.

gl.LINEAR_MIPMAP_LINEAR Chooses the two mipmaps that most closely match the size of

the pixel being textured and uses the LINEAR criterion to produce a texture value from each

mipmap. The final texture value is a weighted average of those two values.

In order to map a texture to a 3D object, coordinates are required. Every vertex in a

polygon is assigned a texture coordinate. Texture coordinates are usually automatically

generated with the 3D object by appropriate software and a little further down they will be

loaded from a Collada object so I won’t mention further about UV mapping.

Textures are applied on a surface by the shaders so the texture coordinates need to be

uploaded to the program. Texture coordinates are uploaded to the vertex shader by an

attribute attribute vec2 aVertexTextureCoords; Except the coordinates the program

needs the texture itself. Shaders access texture data by a data type called sampler2D. The

sampler is used by the fragment shader so it is a uniform uniform sampler2D uSampler;

WebGL can handle up to 32 textures every rendering cycle and are numbers from

TEXTURE0 to TEXTURE31 so the active texture needs to be specified. Also the sampler needs

to be associated with the current texture.

gl.activeTexture(gl.TEXTURE0);

gl.bindTexture(gl.TEXTURE_2D, texture);

gl.uniform1i(Program.uSampler, 0);

At this point the shaders can use the textures to define the color of each fragment

fragmentColor = texture2D(uSampler, vec2(vTextureCoord.s, vTextureCoord.t));

Because the texture coordinates are passed as attribute in the vertex shader the

vTextureCoord is passed as a varying variable from the vertex to the fragment shader.

23

Loading a Collada 3D object

Collada is an XML-based format for interchanging 3D data between applications. Collada’s structure is huge and very complicated and making an XML parser to load a 3D Collada object is a very demanding task. For this reason I used the Collada loader from the KickJS game engine. KickJS is a WebGL based game engine build for modern web browsers such as the most recent version of Chrome and Firefox. The source code is Open Source (BSD New License) and the source code is available on GitHub. At this point I should mention that despite the fact that KickJS is a complete game engine, I only used the Collada loader and not any other part related to rendering or lighting. In order to use the KickJS the following steps need to be followed

1. Download the source code from GitHub 2. Build the project. Windows users need to download Cygwin in order to be able execute

Unix shell commands from their system. After downloading and installing Cygwin locate the Cygwin directory (usually C:\cygwin) inside this directory Cygwin is able to execute Unix shell commands nowhere else in the windows file system. For this reason copy the KickJS source code in the Cygwin/home/user directory, then download and install nodejs also in this folder as well as the Google Clojure Compiler. After that move into the folder that contains the KickJS source code and make sure that there is the build.sh script and then open the Cygwin console and move in this directory. Then run the following command sh build.sh ../nodejs/node . ../GoogleClojure/compiler.jar

../nodejs/node

If the build was successful there must be a folder named “build” and inside it two JavaScript files kick-min-0.4.0 and Kick-debug-0.4.0 one folder named “pre” containing multiple JavaScript files.

3. In order to embed the KickJS project to another project copy the folder created in step 2

and paste it wherever in the project’s folder is necessary 4. Include the kick-min-0.4.0.js or the kick-debug-0.4.0.js as long as every JavaScript file

necessary from the “pre” folder (I recommend to include them all because there are many dependencies among them.) <script type="text/javascript" src="src/kick-min-

0.4.0.js"></script>

<script src="src/pre/chunk.js"></script>

<script src="src/pre/collada.js"></script>

<script src="src/pre/constants.js"></script>

<script src="src/pre/core.js"></script>

<script src="src/pre/glslconstants.js"></script>

24

<script src="src/pre/material.js"></script>

<script src="src/pre/math.js"></script>

<script src="src/pre/mesh.js"></script>

<script src="src/pre/meshfactory.js"></script>

<script src="src/pre/obj.js"></script>

<script src="src/pre/resource.js"></script>

<script src="src/pre/scene.js"></script>

<script src="src/pre/texture.js"></script>

At this point the KickJS components can be used and of course the Collada loader. As mentioned earlier Collada is an XML schema so in order to parse XML content from HTML, an XMLHttpRequest is necessary. The XMLHttpRequest loads the XML content so that it is available from the DOM interface. function loadCollada(url)

{

var oReq = new XMLHttpRequest();

oReq.open("GET", url, false);

oReq.onreadystatechange = handler;

oReq.send();

function handler()

{

if (oReq.readyState == 4 /* complete */)

{

console.log( oReq.status );

var xmlDom = oReq.responseXML;

Load(xmlDom,url,false);

}

}

}

The oReq variable is the request, the open method states that a GET request will be executed

on the given url (url is the general term in this case a path is used.) the false argument states

that the loading should be done synchronously. After the loading is done the Load function is

called. function Load(content,url,rotateAroundX){

var core = KICK.namespace("KICK.core"),

importer = KICK.namespace("KICK.importer");

var engine = new

KICK.core.Engine(document.getElementById("canvas"),{enableDebugContext

: false});

createdObject =

importer.ColladaImporter.import(content,engine,engine.activeScene,rota

teAroundX);

25

Because KickJS uses a way of simulating namespaces in JavaScript, in order to use the respective engine components they need to be used in this manner. The important part here is the import function which is KickJS’s Collada loader. This function returns a JavaScript object that contains the following attributes.

Attribute Name Data Type Description

vertex Float Vec3 The vertex coordinates

normal Float Vec3 The normal of each vertex

tangent Float Vec4 The tangent of each vertex

uv1 Float Vec2 The primary texture coordinates

uv2 Float Vec2 The secondary texture coordinates

int1 … int4 Integer Vec1-4 Generic integer fields of size (1,2,3,4)

color Float Vec4 Vertex colour

Using the respective fields the WebGL buffers are populated during initialization. vertices = createdObject.mesh[0].meshData.vertex;

Mesh = gl.createBuffer();

gl.bindBuffer(gl.ARRAY_BUFFER, Mesh);

gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(vertices),

gl.STATIC_DRAW);

26

Conclusion - Future plans

This thesis is showing only some of WebGL’s basic components. WebGL is capable for much more and since JavaScript is getting faster by the day the road is open for large scale applications to be developed. 3D games, Augmented or Virtual Reality applications so far could not be imagined to run through a web browser but with WebGL this is not an unlikely scenario.

27

References 1. WebGL beginner’s guide Diego Cantor, Brandon Jones Pact Publishing 2012 2. www.learningwebgl.com 3. Interactive Computer Graphics - A Top-Down Appr. 6th ed. - E. Angel, et al., (Pearson,

2012) BBS 4. https://www.khronos.org/registry/webgl/specs/1.0/ 5. http://en.wikipedia.org/wiki/WebGL 6. http://en.wikipedia.org/wiki/Canvas_element 7. http://en.wikipedia.org/wiki/Texture_mapping 8. http://en.wikipedia.org/wiki/UV_mapping 9. http://en.wikipedia.org/wiki/Shaders 10. http://www.khronos.org/registry/webgl/specs/latest/ 11. http://en.wikipedia.org/wiki/XML 12. http://www.opengl.org/sdk/docs/man/xhtml/glTexParameter.xml 13. http://www.kickjs.org/ 14.

http://mortennobel.files.wordpress.com/2012/03/webgl-based-game-engine1.pdf