Vuer

vuer.schemas.scene_components

SceneElement

python
class SceneElement(BlockElement)

Source

Inherited members

Frustum

python
class Frustum(SceneElement)

Source

Camera Frustum

:param position: An optional tuple of three numbers representing the position. :type position: tuple[float, float, float] :param rotation: An optional tuple of three numbers representing the rotation. :type rotation: tuple[float, float, float] :param matrix: An optional tuple of sixteen numbers representing the matrix. :type matrix: tuple[float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float] :param aspect: An optional number representing the aspect. :type aspect: float :param focus: An optional number representing the focus. :type focus: float :param fov: An optional number representing the field of view. :type fov: float :param near: An optional number representing the near field. :type near: float :param far: An optional number representing the far field. :type far: float :param scale: An optional number representing the scale. :type scale: float :param upScale: An optional number representing the up scale. :type upScale: float :param focalLength: An optional number representing the focal length. :type focalLength: float :param showUp: An optional boolean indicating whether to show up. :type showUp: bool :param showFrustum: An optional boolean indicating whether to show the frustum. :type showFrustum: bool :param showFocalPlane: An optional boolean indicating whether to show the focal plane. :type showFocalPlane: bool :param showImagePlane: An optional boolean indicating whether to show the image plane. :type showImagePlane: bool :param src: An optional string representing the source. :type src: str :param colorOrigin: An optional ColorRepresentation for the origin color. :type colorOrigin: ColorRepresentation :param colorFrustum: An optional ColorRepresentation for the frustum color. :type colorFrustum: ColorRepresentation :param colorCone: An optional ColorRepresentation for the cone color. :type colorCone: ColorRepresentation :param colorFocalPlane: An optional ColorRepresentation for the focal plane color. :type colorFocalPlane: ColorRepresentation :param colorUp: An optional ColorRepresentation for the up color. :type colorUp: ColorRepresentation :param colorTarget: An optional ColorRepresentation for the target color. :type colorTarget: ColorRepresentation :param colorCross: An optional ColorRepresentation for the cross color. :type colorCross: ColorRepresentation

Inherited members

python
tag = 'Frustum'

CameraHelper

python
class CameraHelper(SceneElement)

Source

Inherited members

python
tag = 'CameraHelper'

group

python
class group(SceneElement)

Source

Inherited members

python
tag = 'group'
python
children = []

mesh

python
class mesh(SceneElement)

Source

Inherited members

python
tag = 'mesh'
python
children = []

TriMesh

python
class TriMesh(SceneElement)

Source

python
vertices: NDArray[np.float16] = None
python
faces: NDArray[np.uint32] = None
python
colors: NDArray[np.uint8] = None
python
uv: NDArray[np.float16] = None

Inherited members

python
tag = 'TriMesh'
python
children = []

PointCloud

python
class PointCloud(SceneElement)

Source

PointCould element, highly optimized for payload size and speed.

:param vertices: An optional numpy array of shape (N, 3) containing the vertices of the pointcloud. :type vertices: NDArray[np.float16] :param colors: An optional numpy array of shape (N, 3) containing the colors of the point cloud. :type color: NDArray[np.uint8] :param size: An optional float that sets the size of the points. :type size: float :param key: str An optional string that sets the key of the element. :type key: str

Usage::

sess.upsert @ PointCloud( vertices=np.random.rand(1000, 3), colors=np.random.rand(1000, 3), size=0.01, key="pointcloud", )

python
tag: str = 'PointCloud'
python
vertices: NDArray[np.float16] = None
python
colors: NDArray[np.uint8] = None

Inherited members

python
children = []

Box

python
class Box(SceneElement)

Source

Inherited members

python
tag = 'Box'

Capsule

python
class Capsule(SceneElement)

Source

Inherited members

python
tag = 'Capsule'

Cone

python
class Cone(SceneElement)

Source

Inherited members

python
tag = 'Cone'

Circle

python
class Circle(SceneElement)

Source

Inherited members

python
tag = 'Circle'

Cylinder

python
class Cylinder(SceneElement)

Source

Inherited members

python
tag = 'Cylinder'

Dodecahedron

python
class Dodecahedron(SceneElement)

Source

Inherited members

python
tag = 'Dodecahedron'

Edges

python
class Edges(SceneElement)

Source

Inherited members

python
tag = 'Edges'

Extrude

python
class Extrude(SceneElement)

Source

Inherited members

python
tag = 'Extrude'

Icosahedron

python
class Icosahedron(SceneElement)

Source

Inherited members

python
tag = 'Icosahedron'

Lathe

python
class Lathe(SceneElement)

Source

Inherited members

python
tag = 'Lathe'

Octahedron

python
class Octahedron(SceneElement)

Source

Inherited members

python
tag = 'Octahedron'

Plane

python
class Plane(SceneElement)

Source

Inherited members

python
tag = 'Plane'

Polyhedron

python
class Polyhedron(SceneElement)

Source

Inherited members

python
tag = 'Polyhedron'

Ring

python
class Ring(SceneElement)

Source

Inherited members

python
tag = 'Ring'

Shape

python
class Shape(SceneElement)

Source

Inherited members

python
tag = 'Shape'

Sphere

python
class Sphere(SceneElement)

Source

Inherited members

python
tag = 'Sphere'

Tetrahedron

python
class Tetrahedron(SceneElement)

Source

Inherited members

python
tag = 'Tetrahedron'

Torus

python
class Torus(SceneElement)

Source

Inherited members

python
tag = 'Torus'

TorusKnot

python
class TorusKnot(SceneElement)

Source

Inherited members

python
tag = 'TorusKnot'

Tube

python
class Tube(SceneElement)

Source

Inherited members

python
tag = 'Tube'

Fog

python
class Fog(SceneElement)

Source

Fog is a scene element that adds fog to the scene. This can be used to approximate depth.

Args: color: The color of the fog. near: The distance to the near plane. far: The distance to the far plane.

Example Usage:

Fog(color="green", near=3, far=7)

Fog.init

python
def __init__(self, *, children=None, color=None, near=None, far=None, **kwargs)

Source

Inherited members

python
tag = 'fog'

Wireframe

python
class Wireframe(SceneElement)

Source

Inherited members

python
tag = 'Wireframe'

Splat

python
class Splat(SceneElement)

Source

Inherited members

python
tag = 'Splat'

LumaSplats

python
class LumaSplats(SceneElement)

Source

Inherited members

python
tag = 'Splats'

Pcd

python
class Pcd(SceneElement)

Source

Inherited members

python
tag = 'Pcd'

CameraView

python
class CameraView(SceneElement)

Source

CameraView for rendering from arbitrary camera poses.

:param fov: The vertical field of view of the camera. Defaults to 50. :type fov: float, optional :param width: The width of the camera image. Defaults to 320. :type width: int, optional :param height: The height of the camera image. Defaults to 240. :type height: int, optional :param key: The key of the camera view. Defaults to "ego". :type key: str, optional :param position: The position of the camera. Defaults to [0, 0, 0]. :type position: List[float], optional :param rotation: The rotation of the camera. Defaults to [0, 0, 0] :type rotation: List[float], optional :param stream: The stream of the camera. Defaults to "ondemand". :type stream: str, optional :param fps: The frames per second of the camera. Defaults to 30. :type fps: int, optional :param near: The near field of the camera. Defaults to 0.1. :type near: float, optional :param far: The far field of the camera. Defaults to 20. :type far: float, optional :param renderDepth: Whether to render depth. If set, returns value["depthFrame"]. Defaults to True. :type renderDepth: bool, optional :param showFrustum: Whether to show the frustum. Defaults to True. :type showFrustum: bool, optional :param downsample: The downsample rate. Defaults to 1. :type downsample: int, optional :param distanceToCamera: The distance to the camera. Defaults to 2. :type distanceToCamera: float, optional

Inherited members

python
tag = 'CameraView'

SceneBackground

python
class SceneBackground(Image, SceneElement)

Source

Sets the background of the scene to a static image. Does not work well with high frame rates. For displaying movies, use the ImageBackground element.

Inherited members

python
tag = 'SceneBackground'

ImageBackground

python
class ImageBackground(Image, SceneElement)

Source

Sets the background of the scene to an image, Supports high frame rates.

We use a plane that is always facing the camera to display the image.

Inherited members

python
tag = 'ImageBackground'

HUDPlane

python
class HUDPlane(Image, SceneElement)

Source

A Head-up display (HUD) plane that is always facing the camera. Requires mounting a material.

Inherited members

python
tag = 'HUDPlane'

VideoMaterial

python
class VideoMaterial(Image, SceneElement)

Source

A Video Material for loading from a file hosted at a url.

Inherited members

python
tag = 'VideoMaterial'

WebRTCVideoMaterial

python
class WebRTCVideoMaterial(Image, SceneElement)

Source

A Video Material for loading from a media stream.

Inherited members

python
tag = 'WebRTCVideoMaterial'

VideoPlane

python
class VideoPlane(Image, SceneElement)

Source

A Head-up display (HUD) plane that is always facing the camera. Requires mounting a material.

Inherited members

python
tag = 'VideoPlane'

WebRTCVideoPlane

python
class WebRTCVideoPlane(SceneElement)

Source

A Head-up display (HUD) plane that is always facing the camera. Requires mounting a material.

Inherited members

python
tag = 'WebRTCVideoPlane'

StereoVideoPlane

python
class StereoVideoPlane(Image, SceneElement)

Source

A Head-up display (HUD) plane that is always facing the camera. Requires mounting a material.

Inherited members

python
tag = 'StereoVideoPlane'

WebRTCStereoVideoPlane

python
class WebRTCStereoVideoPlane(SceneElement)

Source

A Head-up display (HUD) plane that is always facing the camera. Requires mounting a material.

Inherited members

python
tag = 'WebRTCStereoVideoPlane'

Group

python
class Group(SceneElement)

Source

Inherited members

python
tag = 'VuerGroup'

HemisphereLight

python
class HemisphereLight(SceneElement)

Source

Inherited members

python
tag = 'HemisphereLight'

RectAreaLight

python
class RectAreaLight(SceneElement)

Source

Inherited members

python
tag = 'RectAreaLight'

Stage

python
class Stage(SceneElement)

Source

Inherited members

python
tag = 'Stage'

Gamepads

python
class Gamepads(SceneElement)

Source

Inherited members

python
tag = 'Gamepads'

DirectionalLight

python
class DirectionalLight(SceneElement)

Source

Inherited members

python
tag = 'DirectionalLight'

PointLight

python
class PointLight(SceneElement)

Source

Inherited members

python
tag = 'PointLight'

SpotLight

python
class SpotLight(SceneElement)

Source

Inherited members

python
tag = 'SpotLight'

AmbientLight

python
class AmbientLight(SceneElement)

Source

Inherited members

python
tag = 'AmbientLight'

Html

python
class Html(SceneElement)

Source

Usage::

as='div' // Wrapping element (default: 'div') wrapperClass // The className of the wrapping element (default: undefined) prepend // Project content behind the canvas (default: false) center // Adds a -50%/-50% css transform (default: false) [ignored in transform mode] fullscreen // Aligns to the upper-left corner, fills the screen (default:false) [ignored in transform mode] distanceFactor={10} // If set (default: undefined), children will be scaled by this factor, and also by distance to a PerspectiveCamera / zoom by a OrthographicCamera. zIndexRange={[100, 0]} // Z-order range (default=[16777271, 0]) portal={domnodeRef} // Reference to target container (default=undefined) transform // If true, applies matrix3d transformations (default=false) sprite // Renders as sprite, but only in transform mode (default=false) calculatePosition={(el: Object3D, camera: Camera, size: { width: number; height: number }) => number[]} // Override default positioning function. (default=undefined) [ignored in transform mode] occlude={[ref]} // Can be true or a Ref<Object3D>[], true occludes the entire scene (default: undefined) onOcclude={(visible) => null} // Callback when the visibility changes (default: undefined) {...groupProps} // All THREE.Group props are valid {...divProps} // All HTMLDivElement props are valid

Inherited members

python
tag = 'Html'

Pivot

python
class Pivot(SceneElement)

Source

Inherited members

python
tag = 'Pivot'

Movable

python
class Movable(SceneElement)

Source

Inherited members

python
tag = 'Movable'

Hands

python
class Hands(SceneElement)

Source

.. admonition:: tip Setting stream to True

Important: You need to set the stream option to True to start streaming the hand movement.

The Hand component offers a way to stream the current pose of the hand to the server. You can get the full pose of the hands by listening to the HAND_MOVE event. You can add flags left and right to specify which hand you want to track. The returned data looks like the following:

.. code-block:: typescript

/**

  • Left and right pose are relative to the wrist transformations. */ export type HandsData = { left?: Float32Array; // 25 * 16 values. right?: Float32Array; // 25 * 16 values. leftState: HandState; rightState: HandState; };

export type HandState = { pinch: boolean; squeeze: boolean; tap: boolean;

pinchValue: number; squeezeValue: number; tapValue: number; }

Matrix format

All 4x4 transform matrices used in WebGL are stored in 16-element Float32Arrays. The values are stored in the array in column-major order; that is, each column is written into the array top-down before moving to the next column to the right and writing it into the array. Therefore, for the array [a0, a1, a2, …, a13, a14, a15], the matrix looks like this:

.. code-block:: ⌈ a0 a4 a8 a12 ⌉ | a1 a5 a9 a13 | | a2 a6 a10 a14 | ⌊ a3 a7 a11 a15 ⌋

For details, refer to the MDN documentation on XR Rigid Body Transformation

Hand Landmarks

We follow the XR Hand conventions, and return the landmarks in a single array of 25 * 16 values in the following order:

.. list-table:: :widths: 40 10 40 10 :header-rows: 1

    • Hand joint
    • Index
    • Hand joint (continue)
    • Index
    • wrist
    • 0
    • middle-finger-phalanx-distal
    • 13
    • thumb-metacarpal
    • 1
    • middle-finger-tip
    • 14
    • thumb-phalanx-proximal
    • 2
    • ring-finger-metacarpal
    • 15
    • thumb-phalanx-distal
    • 3
    • ring-finger-phalanx-proximal
    • 16
    • thumb-tip
    • 4
    • ring-finger-phalanx-intermediate
    • 17
    • index-finger-metacarpal
    • 5
    • ring-finger-phalanx-distal
    • 18
    • index-finger-phalanx-proximal
    • 6
    • ring-finger-tip
    • 19
    • index-finger-phalanx-intermediate
    • 7
    • pinky-finger-metacarpal
    • 20
    • index-finger-phalanx-distal
    • 8
    • pinky-finger-phalanx-proximal
    • 21
    • index-finger-tip
    • 9
    • pinky-finger-phalanx-intermediate
    • 22
    • middle-finger-metacarpal
    • 10
    • pinky-finger-phalanx-distal
    • 23
    • middle-finger-phalanx-proximal
    • 11
    • pinky-finger-tip
    • 24
    • middle-finger-phalanx-intermediate
    • 12

Usage Example:

.. code-block:: python

from vuer import Vuer, VuerSession from vuer.schemas import Hands from asyncio import sleep

app = Vuer()

@app.add_handler("HAND_MOVE") async def handler(event, session): print(f"Movement Event: key-{event.key}", event.value)

@app.spawn(start=True) async def main(session: VuerSession):

Important: You need to set the stream option to True to start

streaming the hand movement.

session.upsert @ Hands(stream=True, key="hands")

while True: await sleep(1)

Hands.init

python
def __init__(self, key='hands', stream=True, disableLeft=False, disableRight=False, hideLeft=False, hideRight=False, **kwargs)

Source

Inherited members

python
tag = 'Hands'

MotionControllers

python
class MotionControllers(SceneElement)

Source

MotionController for tracking XR controller poses and button states.

:param key: Unique identifier for the controller. :type key: str, optional :param eventTypes: A tuple or list of events to track (e.g., "trigger", "squeeze"). :type eventTypes: tuple or list, optional :param stream: Whether to enable streaming of controller data. :type stream: bool, optional :param left: Boolean indicating if the left controller should be tracked. :type left: bool, optional :param right: Boolean indicating if the right controller should be tracked. :type right: bool, optional :param showLeft: Boolean indicating if the left controller visualization should be shown. :type showLeft: bool, optional :param showRight: Boolean indicating if the right controller visualization should be shown. :type showRight: bool, optional

MotionControllers.init

python
def __init__(self, key='motionControllers', eventTypes=('trigger', 'squeeze'), stream=True, left=None, right=None, showLeft=True, showRight=True, **kwargs)

Source

Inherited members

python
tag = 'MotionControllers'

Obj

python
class Obj(SceneElement)

Source

Obj.init

python
def __init__(self, src=None, mtl=None, text=None, buff=None, materials=None, **kwargs)

Source

:param src: The source of the obj file. Can be a url or a local file. :type src: str :param mtl: The source of the mtl file. Can be a url or a local file. :type mtl: str :param text: The text content of the obj file, allow one to load a scene from a string. :type text: str :param buff: The binary content of the obj file. This is the most efficient, because you are sending binaries.. :type buff: bytes :param materials: A list of materials to be used for the obj file. :type materials: List[String]

todo: In the future we probably want to enable the loading of multiple material files.

Inherited members

python
tag = 'Obj'

CoordsMarker

python
class CoordsMarker(SceneElement)

Source

Coordinates Marker Component.

Args: position: A list of 3 numbers representing the position. rotation: A list of 3 numbers representing the rotation. matrix: A list of 16 numbers representing the matrix. Overrides position and rotation. scale: 1.0 headScale: 1.0 lod: Level of detail. The number of segments for the cone and the stem.

Inherited members

python
tag = 'CoordsMarker'

Arrow

python
class Arrow(SceneElement)

Source

Coordinates Marker Component.

Args: matrix: A list of 16 numbers representing the matrix. Overrides position and rotation. position: A list of 3 numbers representing the position. rotation: A list of 3 numbers representing the rotation. scale: 1.0 headScale: 1.0 lod: Level of detail. The number of segments for the cone and the stem.

Inherited members

python
tag = 'Arrow'

Ply

python
class Ply(SceneElement)

Source

Inherited members

python
tag = 'Ply'

Glb

python
class Glb(SceneElement)

Source

Glb Component

this follows the material type

:param materialType: Literal["basic", ...] :param material: { side=0: inside, side=1: outsie, side=2: both. }

Inherited members

python
tag = 'Glb'

Urdf

python
class Urdf(SceneElement)

Source

Inherited members

python
tag = 'Urdf'

Gripper

python
class Gripper(SceneElement)

Source

Inherited members

python
tag = 'Gripper'

SkeletalGripper

python
class SkeletalGripper(SceneElement)

Source

Inherited members

python
tag = 'SkeletalGripper'

Grid

python
class Grid(SceneElement)

Source

Inherited members

python
tag = 'Grid'

GrabRender

python
class GrabRender(SceneElement)

Source

Inherited members

python
tag = 'GrabRender'
python
key = 'DEFAULT'

TimelineControls

python
class TimelineControls(SceneElement)

Source

Inherited members

python
tag = 'TimelineControls'

PointerControls

python
class PointerControls(SceneElement)

Source

Inherited members

python
tag = 'PointerControls'

Scene

python
class Scene(BlockElement)

Source

Scene.init

python
def __init__(self, *children, rawChildren=None, htmlChildren=None, bgChildren: List[Element]=None, up=[0, 1, 0], background=None, grid=True, toneMapping: str=None, toneMappingExposure: float=None, frameloop: Literal['always', 'demand']=None, enableOrbitControl: bool=None, camPosition: List[float]=None, camRotation: List[float]=None, **kwargs)

Source

Scene.serialize

python
def serialize(self)

Source

python
tag = 'Scene'

DefaultScene

python
class DefaultScene(Scene)

Source

Default Scene that includes a basic setup of ambient lights.

:param children: list of children elements to be rendered in the scene. :type children: SceneElement, ... :param rawChildren: list of children elements to be rendered in the scene. :param htmlChildren: list of children elements to be rendered in the scene. :param bgChildren: list of children elements to be rendered in the scene. :param show_helper: list of children elements to be rendered in the scene. :param startStep: list of children elements to be rendered in the scene. :param endStep: list of children elements to be rendered in the scene. :param up: list of children elements to be rendered in the scene. :param kwargs: list of children elements to be rendered in the scene.

Example Usage::

DefaultScene(

Ambient Light does not have helper because it is ambient.

AmbientLight(intensity=1.0, key="default_ambient_light"), DirectionalLight( intensity=1, key="default_directional_light", helper=show_helper ), *children, rawChildren=rawChildren, htmlChildren=htmlChildren, bgChildren=[ GrabRender(), *[

we use a key here so that we can replace the timeline controls via update

TimelineControls(start=startStep, end=endStep, key="timeline") if endStep else None, ], PointerControls(), Grid(), *bgChildren, ], up=up, **kwargs, )

DefaultScene.init

python
def __init__(self, *children: SceneElement, rawChildren: List[SceneElement]=None, htmlChildren: List[Element]=None, bgChildren: List[SceneElement]=[], show_helper=True, startStep=0, endStep=None, up=[0, 1, 0], grid=True, **kwargs)

Source

Inherited members

Public imports

These symbols are available from this module. Their definitions are documented in the linked modules.

  • BlockElement — vuer.schemas.html_components.BlockElement
  • Element — vuer.schemas.html_components.Element
  • Image — vuer.schemas.html_components.Image