Skip to content

feat: Adding a webXR primer tutorial #444

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/static.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ jobs:
mkdir -p public/examples/hit-testing
mkdir -p public/examples/uikit
mkdir -p public/examples/portal
mkdir -p public/examples/webxr-primer
cp -r ./examples/minecraft/dist/* ./public/examples/minecraft
cp -r ./examples/pingpong/dist/* ./public/examples/pingpong
cp -r ./examples/rag-doll/dist/* ./public/examples/rag-doll
Expand All @@ -67,6 +68,7 @@ jobs:
cp -r ./examples/hit-testing/dist/* ./public/examples/hit-testing
cp -r ./examples/uikit/dist/* ./public/examples/uikit
cp -r ./examples/portal/dist/* ./public/examples/portal
cp -r ./examples/webxr-primer/dist/* ./public/examples/webxr-primer

- name: Upload Artifact
uses: actions/upload-artifact@v4
Expand Down
2 changes: 1 addition & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

// Run the equivalent of `eslint --fix` each time you save
"editor.codeActionsOnSave": {
"source.fixAll.eslint": true
"source.fixAll.eslint": "explicit"
},

// Disable the built‑in “Organize Imports” so it doesn’t fight ESLint
Expand Down
2 changes: 1 addition & 1 deletion docs/advanced/performance.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Performance
description: Important considerations for building performant immersive web applications with react-three/xr
nav: 22
nav: 23
---

All performance optimizations for non-immersive 3D web applications are also applicable for immersive XR web applications. Relevant guides on the topic of performance for 3D web applications are the [R3F performance guide](https://docs.pmnd.rs/react-three-fiber/advanced/scaling-performance), [R3F performance pitfalls](https://docs.pmnd.rs/react-three-fiber/advanced/pitfalls), and the [Threejs tips and tricks](https://discoverthreejs.com/tips-and-tricks/#performance). In general, it is good to check if your web application's performance is GPU- or CPU-bound to select the correct optimization techniques.
Expand Down
2 changes: 1 addition & 1 deletion docs/handles/handle-component.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Handle Component
description: The Handle and HandleTarget components and their properties
nav: 24
nav: 25
---

The `Handle` component is the core component of the `@react-three/handle` library, which is built on the `HandleStore`. This store provides developers with more control over the current state and user interactions.
Expand Down
2 changes: 1 addition & 1 deletion docs/handles/introduction.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Handles
description: Easily build 3D interactions using the concept of handles
nav: 23
nav: 24
---

Handles are everywhere, from scrollbar thumbs to window bars to door handles.
Expand Down
2 changes: 1 addition & 1 deletion docs/handles/prebuild-handles.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Prebuilt Handles for Editor Use Cases
description: The Handle and HandleTarget components and their properties
nav: 25
nav: 26
---

The `Handle` component allows for the orchestration of multiple handles together to achieve interactions typically found in 3D editors, such as the `TransformControls` offered by Three.js. Since these prebuilt handles are commonly used, `@react-three/handles` includes two of them: `TransformHandles` (based on TransformControls from Three.js) and `PivotHandles` (based on PivotControls from @react-three/drei). These prebuilt handles can be used in XR and non-XR environments, are highly configurable for use cases including multi-user editing, can be utilized through virtual screens, and respect the event system of the scene, preventing accidental interactions with multiple objects at once.
Expand Down
2 changes: 1 addition & 1 deletion docs/handles/screen-handle-components.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Screen Handles
description: Orbit and Map Handles as replacements for Orbit and Map controls
nav: 26
nav: 27
---

Screen handles are available for screen-based devices like smartphones and PCs and allow users to move the camera by dragging, swiping, and scrolling on the screen. Three.js directly offers `OrbitControls` and `MapControls`, which are built for this purpose, and we are building on top of their success. The main difference is that the `OrbitHandles` and `MapHandles` we provide use the event system of the scene, which means that interactions with objects in the scene prevent dragging the camera. Furthermore, the event system can forward the screen inputs on a virtual screen to a virtual camera inside a virtual scene, which is showcased in the [editor example](https://pmndrs.github.io/xr/examples/editor/).
Expand Down
2 changes: 1 addition & 1 deletion docs/migration/from-natuerlich.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: from Natuerlich
description: Migrate your application from natuerlich
nav: 24
nav: 25
---

@react-three/xr is inspired by natuerlich, and therefore, many things are similar, especially the way interactions are handled. However, a few things have been changed and renamed.
Expand Down
2 changes: 1 addition & 1 deletion docs/migration/from-react-three-xr-5.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: from @react-three/xr v5
description: Migrate your application from @react-three/xr v5
nav: 23
nav: 24
---

The goal of @react-three/xr v6 is to align this library closer to the react-three ecosystem. We, therefore, focussed on supporting the react-three/fiber event handlers. Another focus of v6 is to reduce boilerplate and provide more defaults while also giving developers more access to the lower-level WebXR primitives. In combination, these changes allow developers to build XR experiences that interoperate with the whole react-three ecosystem using only a few lines of code.
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorials/anchors.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Anchors
description: How to create and manage anchors in your AR experience?
nav: 17
nav: 18
---

Anchors allow to anchor virtual objects into the physical world in AR experiences. `react-three/xr` offers a multitude of ways to create and manage anchors. A simple solution is `useXRAnchor`, which works similarly to `useState` as it returns the current anchor and a function to request a new anchor as a tuple.
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorials/custom-inputs.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Custom Hands/Controllers/...
description: Customize interactions and style of inputs such as hands, controllers, and more
nav: 16
nav: 17
---

@react-three/xr provides a set of default hands, controllers, transient pointers, gazes, and screen input that can be configured and completely exchanged with your own implementation. The following example shows how to configure the ray color of the ray pointer in the users hand.
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorials/dom-overlay.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Dom Overlay
description: How to add HTML elements for hand-held AR experiences with Dom overlay?
nav: 18
nav: 19
---

For hand-held AR experiences, such as those using a Smartphone, WebXR offers the dom overlay capability, allowing developers to use HTML code overlayed over the experience. In case scene 3D overlays or overlays in non-handheld AR/VR experiences are needed, check out [pmndrs/uikit](https://github.com/pmndrs/uikit).
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorials/gamepad.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Gamepad
description: How to use the XRControllers gamepad?
nav: 13
nav: 14
---

All XR controllers are part of the state inside the xr store. The existing controllers can be read using the `useXR` hook. Alternatively, a specific xr controller can be retrived using `useXRInputSourceState("controller", "left")`.
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorials/guards.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Guards
description: Render and show parts of your application conditionally using guards
nav: 20
nav: 21
---

Guards allow to conditionally display or include content. For instance, the `IfInSessionMode` guard allows only displaying a background when the session is not an AR session. The `IfInSessionMode` can receive either a list of `allow` session modes or a list of `deny` session modes.
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorials/hit-test.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Hit Test
description: How to add hit testing capabilities to your AR experiences?
nav: 19
nav: 20
---

Hit testing allows to check intersections with real-world geometry in AR experiences. `@react-three/xr` provides various hooks and components for setting up hit testing.
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorials/layers.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Layers
description: How to use display images, videos, and custom renders at high quality on quad, cylinder, and equirect shapes?
nav: 15
nav: 16
---

Layers allow to render videos, images, and complete scenes with higher performance and higher quality while preserving battery life and latency for quad, cylinder, and equirect shapes using the WebXR Layer API. Layers are perfect for use cases that display flat, high-quality content, such as videos, images, and user interfaces. The following example illustrates how to create a layer that renders a video.
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorials/secondary-input-sources.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Secondary Input Sources
description: How to use primary and secondary input sources (multiple controllers and hands) simultaneously?
nav: 14
nav: 15
---

Most standalone XR headsets support hand and controller tracking. While typical XR experiences often support both input methods, they only use the primary inputs, which refers to one input per hand and limits the inputs to `2`. However, the headset often also tracks the secondary input sources. By enabling the `secondaryInputSources` flag when creating an xr store, we can access the secondary input sources and use them to track real-world objects, for example.
Expand Down
15 changes: 15 additions & 0 deletions docs/tutorials/webxr-spaces-primer.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
title: WebXR Spaces
description: A short experience to help understand how spaces work in WebXR
nav: 13
---

Spaces are core to any WebXR experience, and understanding which spaces are available, when to use them, and how they interact with each other can be a huge benefit when building XR experiences. The codesandbox below is a simple experience to show how various spaces work together in a WebXR environment.


<iframe src="https://pmndrs.github.io/xr/examples/webxr-primer/"
style={{width:"100%", height: "500px", border:0, borderRadius: "4px", overflow:"hidden"}}
title="WebXRSpacesStarter"
allow="accelerometer; ambient-light-sensor; camera; encrypted-media; geolocation; gyroscope; hid; microphone; midi; payment; usb; vr; xr-spatial-tracking"
sandbox="allow-forms allow-modals allow-popups allow-presentation allow-same-origin allow-scripts"
/>
4 changes: 2 additions & 2 deletions examples/watch/src/Hand.tsx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { signal, computed } from '@preact/signals-core'
import { computed, signal } from '@preact/signals-core'
import { useGLTF } from '@react-three/drei'
import { useFrame, useThree } from '@react-three/fiber'
import { Root, Text, Container } from '@react-three/uikit'
import { Container, Root, Text } from '@react-three/uikit'
import { FootprintsIcon, GoalIcon, PlayIcon } from '@react-three/uikit-lucide'
import { XRHandModel } from '@react-three/xr'
import { Suspense, useMemo, useRef, useState } from 'react'
Expand Down
1 change: 1 addition & 0 deletions examples/webxr-primer/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
dist
12 changes: 12 additions & 0 deletions examples/webxr-primer/index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Document</title>
<script async type="module" src="./index.tsx"></script>
</head>
<body style="touch-action: none; margin: 0; position: relative; width: 100dvw; height: 100dvh; overflow: hidden;">
<div id="root" style="position: absolute; inset: 0; display: flex; flex-direction: column;"></div>
</body>
</html>
9 changes: 9 additions & 0 deletions examples/webxr-primer/index.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
import { StrictMode } from 'react'
import { createRoot } from 'react-dom/client'
import App from './src/App.js'

createRoot(document.getElementById('root')!).render(
<StrictMode>
<App />
</StrictMode>,
)
10 changes: 10 additions & 0 deletions examples/webxr-primer/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"dependencies": {
"@react-three/xr": "workspace:~"
},
"scripts": {
"dev": "vite --host",
"check:eslint": "eslint \"*.{ts,tsx}\"",
"fix:eslint": "eslint \"*.{ts,tsx}\" --fix"
}
}
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
56 changes: 56 additions & 0 deletions examples/webxr-primer/src/App.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
import { OrbitControls } from '@react-three/drei'
import { Canvas } from '@react-three/fiber'
import { createXRStore, XR } from '@react-three/xr'
import { Suspense, useState } from 'react'
import * as THREE from 'three'
import { InstructionsPage } from './InstructionsPage.js'
import { Scene } from './Scene.js'
import { SelectableComponents } from './SelectableComponents.enum'
import './styles.css'

const store = createXRStore()

const axisColor = new THREE.Color('#9d3d4a')
const gridColor = new THREE.Color('#4f4f4f')

export default function App() {
const [selectedElement, setSelectedElement] = useState(SelectableComponents.default)

const onNextClick = () => {
setSelectedElement(SelectableComponents.getNextValue(selectedElement))
}

const onPrevClick = () => {
setSelectedElement(SelectableComponents.getPrevValue(selectedElement))
}

return (
<div className="App">
<Canvas
camera={{
position: [3.0, 2.1, 4.0],
}}
>
<color attach={'background'} args={['#3f3f3f']} />
<fog attach={'fog'} args={['#3f3f3f', 5, 30]} />
<ambientLight />
<directionalLight position={[0, 5, 5]} />
<gridHelper args={[50, 50, axisColor, gridColor]} />
<Suspense fallback={null}>
<XR store={store}>
<Scene
selectedElement={selectedElement}
isPositionedObjectSelected={SelectableComponents.isPositionedObjectSelected(selectedElement)}
setSelectedElement={setSelectedElement}
/>
</XR>
</Suspense>
<OrbitControls target={[1, 1, 0]} />
</Canvas>
<div className="titleText" onClick={() => setSelectedElement(SelectableComponents.default)}>
{'WebXR Spaces'}
</div>
<InstructionsPage selectedElement={selectedElement} onNextClick={onNextClick} onPrevClick={onPrevClick} />
</div>
)
}
8 changes: 8 additions & 0 deletions examples/webxr-primer/src/CodeBlock.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
import { ReactNode } from 'react'

interface CodeBlockProps {
children: ReactNode
}
export const CodeBlock = ({ children }: CodeBlockProps) => {
return <div className="codeBlock">{children}</div>
}
65 changes: 65 additions & 0 deletions examples/webxr-primer/src/Controllers.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
/*
Auto-generated by: https://github.com/pmndrs/gltfjsx
Command: npx [email protected] ./Controllers.glb --transform -t
Files: ./Controllers.glb [32.27KB] > /Users/kaileanokeefe/Desktop/TutorialExports/Controllers-transformed.glb [5.12KB] (84%)
*/

import { Outlines, useGLTF } from '@react-three/drei'
import React from 'react'
import * as THREE from 'three'
import { GLTF } from 'three/examples/jsm/loaders/GLTFLoader.js'
import { DashedLine } from './DashedLine.js'
import { Gimbal } from './Gimbal.js'
import { HighlightAndSelectableElement, SelectableComponents } from './SelectableComponents.enum'

type GLTFResult = GLTF & {
nodes: {
Cylinder: THREE.Mesh
Cylinder_1: THREE.Mesh
}
materials: {
ControllerWhite: THREE.MeshStandardMaterial
ControllerBlack: THREE.MeshStandardMaterial
}
}

export function DemoControllers(props: HighlightAndSelectableElement) {
const [isHovered, setIsHovered] = React.useState(false)

const showOutlines = props.isSelected || isHovered

const { nodes, materials } = useGLTF('/Controllers-transformed.glb') as unknown as GLTFResult
return (
<group
{...props}
dispose={null}
onPointerOver={(e) => {
setIsHovered(true)
e.stopPropagation()
}}
onPointerLeave={(e) => {
setIsHovered(false)
e.stopPropagation()
}}
onClick={(e) => {
props.setIsSelected?.(SelectableComponents.controllers)
e.stopPropagation()
}}
>
<mesh geometry={nodes.Cylinder.geometry} material={materials.ControllerWhite}>
{showOutlines && <Outlines thickness={7} color={'green'} opacity={1} />}
</mesh>
<mesh geometry={nodes.Cylinder_1.geometry} material={materials.ControllerBlack}></mesh>
{props.isSelected && (
<>
<Gimbal colorX={'#98fb98'} colorY={'#0b6623'} colorZ={'#023020'} length={0.5} position={[-0.5, 1.5, 1.2]} />
<Gimbal colorX={'#98fb98'} colorY={'#0b6623'} colorZ={'#023020'} length={0.5} position={[0.5, 1.5, 1.2]} />
<DashedLine start={[0.5, 1.5, 1.2]} end={[0, 0, 0]} thickness={4} color={'green'} />
<DashedLine start={[-0.5, 1.5, 1.2]} end={[0, 0, 0]} thickness={4} color={'green'} />
</>
)}
</group>
)
}

useGLTF.preload('/Controllers-transformed.glb')
Loading