three-gpu-pathtracer

npm version lgtm code quality build github twitter sponsors

Path tracing project using three-mesh-bvh and WebGL 2 to accelerate high quality, physically based rendering on the GPU. Features include support for GGX surface model, material information, textures, normal maps, emission, environment maps, tiled rendering, and more!

More features and capabilities in progress!

Examples

Beauty Demos

Physically Based Materials

Lego Models

Interior Scene

Depth of Field

Features

Skinned Geometry Support

Morph Target Support

Test Scenes

Material Test Orb

Transmission Preset Orb

Light Baking

Ambient Occlusion Material

Use

Basic Renderer

import * as THREE from 'three';
import { FullScreenQuad } from 'three/examples/jsm/postprocessing/Pass.js';
import {
	PathTracingSceneGenerator,
	PathTracingRenderer,
	PhysicalPathTracingMaterial,
} from 'three-gpu-pathtracer';

// init scene, renderer, camera, controls, etc

// initialize the path tracing material and renderer
const ptMaterial = new PhysicalPathTracingMaterial();
const ptRenderer = new PathTracingRenderer( renderer );
ptRenderer.setSize( window.innerWidth, window.innerHeight );
ptRenderer.camera = camera;
ptRenderer.material = ptMaterial;

// if rendering transparent background
ptRenderer.alpha = true;

// init quad for rendering to the canvas
const fsQuad = new FullScreenQuad( new THREE.MeshBasicMaterial( {
	map: ptRenderer.target.texture,
	
	// if rendering transparent background
	blending: THREE.CustomBlending,
} ) );

// ensure scene matrices are up to date
scene.updateMatrixWorld();

// initialize the scene and update the material properties with the bvh, materials, etc
const generator = new PathTracingSceneGenerator();
const { bvh, textures, materials } = generator.generate( scene );

// update bvh and geometry attribute textures
ptMaterial.bvh.updateFrom( bvh );
ptMaterial.normalAttribute.updateFrom( geometry.attributes.normal );
ptMaterial.tangentAttribute.updateFrom( geometry.attributes.tangent );
ptMaterial.uvAttribute.updateFrom( geometry.attributes.uv );

// update materials and texture arrays
ptMaterial.materialIndexAttribute.updateFrom( geometry.attributes.materialIndex );
ptMaterial.textures.setTextures( renderer, 2048, 2048, textures );
ptMaterial.materials.updateFrom( materials, textures );

// set the environment map
const texture = await new RGBELoader().loadAsync( envMapUrl );
ptRenderer.material.envMapInfo.updateFrom( texture );

animate();

// ...

function animate() {

	// if the camera position changes call "ptRenderer.reset()"

	// update the camera and render one sample
	camera.updateMatrixWorld();
	ptRenderer.update();

	// copy the current state of the path tracer to canvas to display
	renderer.autoClear = false;
	fsQuad.material.map = ptRenderer.target.texture;
	fsQuad.render( renderer );
	renderer.autoClear = true;

}

Blurred Environment Map

Using a pre blurred envioronment map can help improve frame convergence time at the cost of sharp environment reflections. If performance is concern then multiple importance sampling can be disabled and blurred environment map used.

import { BlurredEnvMapGenerator } from 'three-gpu-pathtracer';

// ...

const envMap = await new RGBELoader().loadAsync( envMapUrl );
const generator = new BlurredEnvMapGenerator( renderer );
const blurredEnvMap = generator.generate( envMap, 0.35 );

// render!

Dynamic Scenes

Using the dynamic scene generator the same, frequently updated scene can be converted into a single reusable geometry multiple times and BVH refit which greatly improves subsequent scene updates. See DynamicPathTracingSceneGenerator docs for more info.

import { DynamicPathTracingSceneGenerator } from 'three-gpu-pathtracer';

// ... initialize scene etc

const generator = new DynamicPathTracingSceneGenerator( scene );
const { bvh, textures, materials } = generator.generate( scene );

// ... update path tracer and render

Asynchronous Scene Generation

NOTE WebWorker syntax is inconsistently supported across bundlers and sometimes not supported at all so the PathTracingSceneWorker class is not exported from the package root. If needed the code from src/worker can be copied and modified to accomodate a particular build process.

import { PathTracingSceneWorker } from 'three-gpu-pathtracer/src/workers/PathTracingSceneWorker.js';

// ...

// initialize the scene and update the material properties with the bvh, materials, etc
const generator = new PathTracingSceneWorker();
const { bvh, textures, materials } = await generator.generate( scene );

// ...

Exports

PathTracingRenderer

Utility class for tracking and rendering a path traced scene to a render target.

.samples

readonly samples : Number

Number of samples per pixel that have been rendered to the target.

.target

readonly target : WebGLRenderTarget

The target being rendered to. The size of the target is updated with setSize and is initialized to a FloatType texture.

.camera

camera = null : Camera

The camera to render with. The view offset of the camera will be updated every sample to enable anti aliasing.

.material

material = null : ShaderMaterial

The Path Tracing material to render. This is expected to be a full screen quad material that respects the “opacity” field for every pixel so samples can be accumulated over time. The material is also expected to have cameraWorldMatrix and invProjectionMatrix fields of type Matrix4.

.tiles

tiles = ( 1, 1 ) : Vector2

Number of tiles on x and y to render to. Can be used to improve the responsiveness of a page while still rendering a high resolution target.

.stableNoise

stableNoise = false : Boolean

Whether to reset the random seed to 0 when restarting the render. If true then a consistent random sample pattern will appear when moving the camera, for example.

.alpha

alpha = false : Boolean

Whether to support rendering scenes with transparent backgrounds. When transparent backgrounds are used two extra render targets are used, custom blending is performed, and PathTracingRenderer.target will change on every completed sample.

constructor

constructor( renderer : WebGLRenderer )

.setSize

setSize( size : Vector2 ) : void

Sets the size of the target to render to.

.update

update()

Renders a single sample to the target.

.reset

reset() : void

Resets and restarts the render from scratch.

PathTracingSceneGenerator

Utility class for generating the set of data required for initializing the path tracing material with a bvh, geometry, materials, and textures.

.generate

generate( scene : Object3D, options = {} : Object ) : {
	bvh : MeshBVH,
	materials : Array<Material>,
	textures : Array<Texture>
}

Merges the geometry in the given scene with an additional “materialIndex” attribute that references the associated material array. Also produces a set of textures referenced by the scene materials.

PathTracingSceneWorker

extends PathTracingSceneGenerator

See note in Asyncronous Generation use snippet.

.generate

async generate( scene : Object3D, options = {} : Object ) : {
	bvh : MeshBVH,
	materials : Array<Material>,
	textures : Array<Texture>
}

.dispose

dispose() : void

PhysicalCamera

extends THREE.PerspectiveCamera

An extension of the three.js PerspectiveCamera with some other parameters associated with depth of field. These parameters otherwise do not affect the camera behavior are are for convenience of use with the PhysicalCameraUniform and pathtracer.

.focusDistance

focusDistance = 25 : Number

The distance from the camera in meters that everything is is perfect focus.

.fStop

fStop = 1.4 : Number

The fstop value of the camera. If this is changed then the bokehSize field is implicitly updated.

.bokehSize

bokehSize : Number

The bokeh size as derived from the fStop and focal length in millimeters. If this is set then the fStop is implicitly updated.

.apertureBlades

apertureBlades = 0 : Number

The number of sides / blades on the aperture.

.apertureRotation

apertureRotation = 0 : Number

The rotation of the aperture shape in radians.

.anamorphicRatio

anamorphicRatio = 1 : Number

The anamorphic ratio of the lens. A higher value will stretch the bokeh effect horizontally.

DynamicPathTracingSceneGenerator

A variation of the path tracing scene generator intended for quickly regenerating a scene BVH representation that updates frequently. Ie those with animated objects or animated skinned geometry.

In order to quickly update a dynamic scene the same BVH is reused across updates by refitting rather than regenerating. This is significantly faster but also results in a less optimal BVH after significant changes.

If geometry or materials are added or removed from the scene then reset must be called.

constructor

constructor( scene : Object3D )

Takes the scene to convert.

.generate

generate() : {
	bvh : MeshBVH,
	materials : Array<Material>,
	textures : Array<Texture>
}

Generates and refits the bvh to the current scene state. The same bvh, materials, and textures objects are returns after the initial call until reset is called.

.reset

reset() : void

Resets the generator so a new BVH is generated. This must be called when geometry, objects, or materials are added or removed from the scene.

BlurredEnvMapGenerator

Utility for generating a PMREM blurred environment map that can be used with the path tracer.

constructor

constructor( renderer : WebGLRenderer )

.generate

generate( texture : Texture, blur : Number ) : DataTexture

Takes a texture to blur and the amount to blur it. Returns a new DataTexture that has been PMREM blurred environment map that can have distribution data generated for importance sampling.

.dispose

dispose() : void

Disposes of the temporary files and textures for generation.

MaterialBase

extends THREE.ShaderMaterial

Convenience base class that adds additional functions and implicitly adds object definitions for all uniforms of the shader to the object.

.setDefine

setDefine( name : string, value = undefined : any ) : void

Sets the define of the given name to the provided value. If the value is set to null or undefined then it is deleted from the defines of the material. If the define changed from the previous value then Material.needsUpdate is set to true.

PhysicalPathTracingMaterial

extends MaterialBase

Uniforms

{
	// The number of ray bounces to test. Higher is better quality but slower performance.
	bounces = 3 : Number,

	// The physical camera parameters to use
	physicalCamera : PhysicalCameraUniform,

	// Geometry and BVH information
	bvh: MeshBVHUniformStruct,
	normalAttribute: FloatVertexAttributeTexture,
	tangentAttribute: FloatVertexAttributeTexture,
	uvAttribute: FloatVertexAttributeTexture,
	materialIndexAttribute: UIntVertexAttributeTexture,
	materials: MaterialsTexture,
	textures: RenderTarget2DArray,

	// Environment Map information
	envMapInfo: EquirectHdrInfoUniform,
	environmentRotation: Matrix3,
	environmentIntensity = 1: Number,

	// background blur
	backgroundBlur = 0: Number,

	// Factor for alleviating bright pixels from rays that hit diffuse surfaces then
	// specular surfaces. Setting this higher alleviates fireflies but will remove some
	// specular caustics.
	filterGlossyFactor = 0: Number,

	// The colors to use for the gradient background when enabled.
	bgGradientTop: Color,
	bgGradientBottom: Color,

	// The transparency to render the background with. Note that the "alpha" option
	// must be set to true on PathTracingRenderer for this field to work properly.
	backgroundAlpha: 1.0,
}

Defines

{

	// Whether to use multiple importance sampling to help the image converge more quickly
	FEATURE_MIS = 1 : Number,

	// Whether to use the "bg" gradient fields to sample for the background
	FEATURE_GRADIENT_BG = 0 : Number

	// The number of transparent pixels to allow on top of existing bounces for object transparency.
	TRANSPARENT_TRAVERSALS = 5 : Number,


}

RenderTarget2DArray

extends WebGLArrayRenderTarget

A convenience extension from WebGLArrayRenderTarget that affords easily creating a uniform texture array from an array of textures.

.setTextures

setTextures(
	renderer : WebGLRenderer,
	width : Number,
	height : Number,
	textures : Array<Texture>
) : void

Takes the rendering context to updateh the target for, the target dimensions of the texture array, and the array of textures to render into the 2D texture array. Every texture is stretched to the dimensions of the texture array at the same index they are provided in.

PhysicalCameraUniform

Uniform for storing the camera parameters for use with the shader.

.updateFrom

updateFrom( camera : PerspectiveCamera | PhysicalCamera ) : void

Copies all fields from the passed PhysicalCamera if available otherwise the defaults are used.

MaterialsTexture

extends DataTexture

Helper texture uniform for encoding materials as texture data.

.setSide

setSide( index : Number, side : FrontSide | BackSide | DoubleSide ) : void

Sets the side to render for the given material.

.setMatte

setMatte( index : Number, matte : Boolean ) : void

Sets whether or not the material of the given index is matte or not. When “true” the background is rendered in place of the material.

.updateFrom

updateFrom( materials : Array<Material>, textures : Array<Texture> ) : void

Updates the size and values of the texture to align with the provided set of materials and textures.

The “matte” and “side” values must be updated explicitly.

EquirectHdrInfoUniform

Stores the environment map contents along with the intensity distribution information to support multiple importance sampling.

.updateFrom

updateFrom( environmentMap : Texture ) : void

Takes an environment map to process into something usable by the path tracer. Is expected to be a DataTexture so the data can be read.

Functions

mergeMeshes

mergeMeshes( meshes : Array<Mesh> ) : {
	materials : Array<Material>,
	textures : Array<Textures>,
	geometry : BufferGeometry
}

Merges the set of meshes into a single geometry with a materialIndex vertex attribute included on the geometry identifying the associated material in the returned materials array.

Shader Chunks

shaderMaterialSampling

Set of functions for performing material scatter and PDF sampling. See the implementation for full list of functions.

shaderStructs

Material struct definition for use with uniforms. See the implementation for full list of functions.

shaderUtils

Set of randomness and other light transmport utilities for use in a shader. See the implementation for full list of functions.

Gotchas

Screenshots

Sample materials

"SD Macross City Standoff Diorama" scene by tipatat

"Interior Scene" model by Allay Design

Perseverance Rover, Ingenuity Helicopter models by NASA / JPL-Caltech

Gelatinous Cube model by glenatron

Lego models courtesy of the LDraw Official Model Repository

Octopus Tea model by AzTiZ

Resources

Raytracing in One Weekend Book

PBR Book

knightcrawler25/GLSL-PathTracer