To complete this workshop you will need:
Hardware:
Languages used:
AFrame is a HTML-based framework to develop virtual reality experiences and not only. Behind the scene, it is Three.js doing all the heavy lifting. A Hello World scene in AFrame requires just few HTML lines to run
Create a new index.html
<!DOCTYPE html>
<html>
<head>
<!--AFrame version 1.3.0-->
<script src="https://aframe.io/releases/1.3.0/aframe.min.js"></script>
</head>
<body>
<a-scene>
<a-box position="-1 0.5 -3" rotation="0 45 0" color="#4CC3D9"></a-box>
<a-entity geometry="primitive: box; width: 1; height: 1; depth: 1"
material="color: red; shader: standard;"
rotation="0 45 0" position="-1 1.7 -3"></a-entity>
<a-sphere position="0 1.25 -5" radius="1.25" color="#EF2D5E"></a-sphere>
<a-cylinder position="1 0.75 -3" radius="0.5" height="1.5" color="#FFC65D"></a-cylinder>
<a-plane position="0 0 -4" rotation="-90 0 0" width="4" height="4" color="#7BC8A4"></a-plane>
<a-sky color="#ECECEC"></a-sky>
</a-scene>
</body>
</html>
At the top of the index.html
file, the AFrame library is loaded. In the body
of the page, the HTML tag a-scene
defines the AFrame scene. AFrame is based on a entity-component system: the scene contains a series of objects, called entitiesa-entity
. They could be generic or based on primitives (e.g. box, sphere, cylinder, plane and sky).
An a-entity
by itself does not display anything. It is just a placeholder, a general-purpose object that needs one or more components to define:
For example, the primitive a-box
can be written also as a-entity geometry="primitive: box"
.
In addition to the standard console, it is possible to access an additional debug interface by pressing Ctrl+Shift+Alt+i
. This custom interface can be used to control the scene, however, to save the values and update the code the external add-on aframe-watcher is needed.
HTML attributes (or components in AFrame) are used to extend AFrame functionalities. For example, by adding the inbuilt component stats
to the a-scene
it is possible to overlay some useful information about the scene, such as frames per second, the number of textures and triangles and other performance values. The component vr-mode-ui="enabled: false"
is used to disable the VR button on the bottom right corner of the scene.
<a-scene stats vr-mode-ui="enabled: false"></a-scene>
An AFrame scene does not need to be full screen. It can be embedded in any web page using the component embedded
. We just need to set the size of the a-scene
element in the CSS of the page (between the style
element in the head
of the page or in an external CSS)
<style>
a-scene {
height: 60vh;
width: 50vw;
}
</style>
The embedded
component needs to be added to the a-scene
. It is possible to wrap the entire a-scene
in a DIV element if we need more control over the composition of the web page
<body>
<div id="myEmbeddedScene">
<a-scene embedded stats vr-mode-ui="enabled: false" >
<!-- ... -->
</a-scene>
</div>
</body>
To control the FOV (field of view) and height of the point of view we need to define a a-camera
element.
a-scene
add a a-camera
fov=60
position="0 2 0
<!-- ... -->
<a-camera fov="60" position="0 2 0"></a-camera>
<!-- ... -->
Any AFrame scene can be modified just using HTML. To add a blue sphere on the top of the existing one:
a-sphere
within the a-scene
position="0 3.5 -5"
radius="1"
color="blue"
<a-scene embedded stats vr-mode-ui="enabled: false">
<a-sphere
position="0 3.5 -5"
radius="1"
color="blue">
</a-sphere>
<a-box position="-1 0.5 -3" rotation="0 45 0" color="#4CC3D9"></a-box>
It is also possible to import external assets, like images and digital models. A good practice is to add the models to the Asset Management Systema-assets
of the scene. The Asset Management System is used to preload 3D models, sounds, images, videos etc. that will be used in the scene. Download the panoramic image and place it in the project folder (e.g. ./resources/imgs/
)
a-scene
elementa-assets
tag after the a-scene
tagimg id="sky" src="./resources/imgs/quad.webp"
a-sky
entity a-sky radius="1.6" src="#sky"
, the radius must be the same height of the camera used to take the panorama imagea-camera
entity a-camera position="0 0 1"
, by default AFrame set the camera height to 1.6<a-scene>
<a-asset>
<img id="sky" src="./resources/imgs/quad.webp" />
</a-asset>
<a-sky radius="1.6" src="#sky"></a-sky>
<a-camera wasd-controls-enabled="false" position="0 0 1"></a-camera>
</a-scene>
More complex geometries can be added to the scene using GLTF model
Plant
, display
and table
, to a folder of the project (e.g. ./resources/models/
), and add them to the Asset Management Systema-asset
<a-asset>
<img id="sky" src="./resources/imgs/quad.webp" />
<a-asset-item id="plant" src="resources/models/Plant.glb"> </a-asset-item>
<a-asset-item id="display" src="resources/models/display.glb"> </a-asset-item>
<a-asset-item id="table" src="resources/models/table.glb"> </a-asset-item>
</a-asset>
a-scene
using an empty a-entity
with attribute gltf-model
pointing at the unique ID of glb file<a-entity
id="_table"
gltf-model="#table"
scale="1 1 1"
position="0 -1.6 0"
rotation="0 0 0"
></a-entity>
<a-entity
id="_plant"
gltf-model="#plant"
scale="1 1 1"
position="0 -0.51 0.1"
rotation="0 0 0"
></a-entity>
<a-entity
id="_display"
gltf-model="#display"
scale="0.1 0.1 0.1"
position="-0.22 -0.3 0.1"
rotation="0 0 0"
></a-entity>
It is possible to organise the a-entity tags using the HTML tree structure. For example, to add some text to the display model we can use the a-troika-text
entity within the a-entity
of the display.
<a-entity
id="_display"
gltf-model="#display"
scale="0.1 0.1 0.1"
position="-0.22 -0.3 0.1"
rotation="0 0 0"
>
<a-troika-text
id="moisture"
value="Moisture: "
width="4"
anchor="left"
position="-0.9 0.5 0.02"
></a-troika-text>
<a-troika-text
id="temperature"
value="Temperature: "
width="4"
anchor="left"
position="-0.9 0.1 0.02"
></a-troika-text>
<a-troika-text
id="humidity"
value="Humidity: "
width="4"
anchor="left"
position="-0.9 -0.3 0.02"
></a-troika-text>
</a-entity>
a-scene
has an attribute renderer that can be use to improve the final aspect of the scene, however, certain settings (e.g. antialias or physicallyCorrectLights) might be too demanding on older hardware
<a-scene
renderer="antialias: true;
colorManagement: true;
sortObjects: true;
physicallyCorrectLights: true;
"
></a-scene>
The current camera does not allow us to properly see the added models, however, moving the camera to a new position will solve just half of the problem. A better solution is to use a different camera, like the ThreeJSOrbitControls. A ready-made component for AFrame can be used to add this camera in the scene aframe-orbit-controls
<head>
<!--AFrame version 1.3.0-->
<script src="https://aframe.io/releases/1.3.0/aframe.min.js"></script>
<!-- AFrame Troika Text -->
<script src="https://unpkg.com/aframe-troika-text/dist/aframe-troika-text.min.js"></script>
<!-- AFrame Orbit Control -->
<script src="https://unpkg.com/aframe-orbit-controls@1.0.0/dist/aframe-orbit-controls.min.js"></script>
</head>
orbit-controls
<a-entity
camera
look-controls
orbit-controls="target: 0 0 0; minDistance: 0; maxDistance: 0.8; initialPosition: 0 0 0.8; dampingFactor:0.05"
></a-entity>
As we have seen, AFrame components are used to further extend the functionalities of the scene (e.g. orbit-controls
). It is possible to create custom components in JavaScript. Before its use, a component needs to be registered using a specific syntax
AFRAME.registerComponent("name-of-the-component", {
init: function () {
myFunction;
},
});
The custom component will be used to interact with the scene using the mouse cursor or touch gestures. Before creating the actual component, we need to add two boxes to the display (they are going to be the interactive buttons) just after the third a-troika-text
entity
<a-entity id="buttonsGroup">
<a-entity
class="collidable"
id="connect-button"
geometry="primitive: box; width:0.4; height: 0.4; depth: 0.1;"
material="color: green; emissive:#b9ff19; emissiveIntensity:0.1"
position="-0.5 -0.65 0.02"
>
</a-entity>
<a-entity
class="collidable"
id="disconnect-button"
geometry="primitive: box; width:0.4; height: 0.4; depth: 0.1;"
material="color: red; emissive:#ff1100; emissiveIntensity:0.1"
position="0.5 -0.65 0.02"
>
</a-entity>
</a-entity>
We need also to inform the a-scene
that we are going to use the raycaster component only on the entities of class .collidable
and with origin from the mouse cursor
<a-scene
cursor="rayOrigin: mouse"
raycaster="objects: .collidable"
renderer="antialias: true;
colorManagement: true;
sortObjects: true;
physicallyCorrectLights: true;
"
></a-scene>
Create a new file custom-components.js
in the ./resources/js/
folder
AFRAME.registerComponent("controller", {
schema: {
connect: { type: "selector" },
disconnect: { type: "selector" },
},
init: function () {
this.data.connect.addEventListener("click", (event) => {
console.log(event.target.getAttribute("material").color);
this.data.connect.setAttribute("material", { emissiveIntensity: "0.8" });
this.data.disconnect.setAttribute("material", {
emissiveIntensity: "0.1",
});
});
this.data.disconnect.addEventListener("click", (event) => {
console.log(event.target.getAttribute("material").color);
this.data.connect.setAttribute("material", { emissiveIntensity: "0.1" });
this.data.disconnect.setAttribute("material", {
emissiveIntensity: "0.8",
});
});
},
});
Save the file and add the script in the head
of the index.html
<script src="./resources/js/custom-components.js"></script>
Finally, add the component to the the empty a-entity
that holds the two boxes
<a-entity
id="buttonsGroup"
controller="connect:#connect-button; disconnect:#disconnect-button"
>
<a-entity
class="collidable"
id="connect-button"
geometry="primitive: box; width:0.4; height: 0.4; depth: 0.1;"
material="color: green; emissive:#b9ff19; emissiveIntensity:0.1"
position="-0.5 -0.65 0.02"
>
</a-entity>
<a-entity
class="collidable"
id="disconnect-button"
geometry="primitive: box; width:0.4; height: 0.4; depth: 0.1;"
material="color: red; emissive:#ff1100; emissiveIntensity:0.1"
position="0.5 -0.65 0.02"
>
</a-entity>
</a-entity>
A custom component can also be used to interface with other JavaScript libraries, for example MQTT.js.
To start, add the MQTT.js script in the head of the index.html
<!--AFrame version 1.3.0-->
<script src="https://aframe.io/releases/1.3.0/aframe.min.js"></script>
<!-- AFrame Troika Text -->
<script src="https://unpkg.com/aframe-troika-text/dist/aframe-troika-text.min.js"></script>
<!-- AFrame Orbit Control -->
<script src="https://unpkg.com/aframe-orbit-controls@1.0.0/dist/aframe-orbit-controls.min.js"></script>
<!--MQTT.js 4.3.7-->
<script src="https://unpkg.com/mqtt@4.3.7/dist/mqtt.min.js"></script>
<script src="./resources/js/custom-components.js"></script>
Then, in the custom-components.js
, create two new methods in the controller
component, after the init: function(){[...]},
, and setup the topic to subscribe to
startMqtt: function (client) {
client.subscribe('TOPIC/TO/SUBSCRIBE/moisture');
client.subscribe('TOPIC/TO/SUBSCRIBE/temperature');
client.subscribe('TOPIC/TO/SUBSCRIBE/humidity')
client.on('message', (topic, message) => {
//Called each time a message is received
console.log('Received message:', topic, message.toString());
if (topic.includes('moisture')) {
document.querySelector('#moisture').setAttribute('value', 'Moisture: ' + parseFloat(message.toString()).toFixed(2).toString() + '%');
}
if (topic.includes('temperature')) { //temperature
document.querySelector('#temperature').setAttribute('value', 'Temperature: ' + parseFloat(message.toString()).toFixed(2).toString() + '°C');
}
if (topic.includes('humidity')) { //humidity
document.querySelector('#humidity').setAttribute('value', 'Humidity: ' + parseFloat(message.toString()).toFixed(2).toString() + '%');
}
})
},
stopMqtt: function (client) {
client.end(true);
console.log('connection closed');
}
In the init
function add a new variable let client
, just before the addEventListener
.
Inside the this.data.connect.addEventListener
function add the MQTT client with the WebSocket to connect to, the onConnect event and the onError event
//Create a new client with ID of the marker
client = mqtt.connect("ws://ADDRESS.MQTT.BROKER:8080");
client.on("connect", () => {
console.log("Connected");
this.startMqtt(client); //if connected, subscribe to the topics
});
client.on("error", (error) => {
console.log(error);
});
Inside the this.data.disconnect.addEventListener
function call the function this.stopMqtt(client)
to disconnect the client.
Run the live server and click on the green button to connect the client and visualise MQTT feed in real-time on the display
In this example we are adding a new function colorScaleFunction
from the D3js Library to dynamically change the colours of the 3D model using a defined colour ramp. Therefore we need to add it to the headindex.html
just before the MQTT one
<!--D3.js-->
<script src="https://d3js.org/d3.v7.min.js"></script>
In the custom-components.js
, after the stopMqtt
function, add a new function named colourLeaf
stopMqtt: function (client) {
client.end(true);
console.log("connection closed");
},
colourLeaf: function (msg) {
const colorScaleFunction = d3
.scaleThreshold()
.domain([0, 40])
.range([
[105 / 255, 179 / 255, 76 / 255], //0-13
[250 / 255, 183 / 255, 51 / 255], //14-26
[255 / 255, 13 / 255, 13 / 255], //26-40
]);
let modelEl = document.querySelector("#_plant");
console.log(modelEl.object3D.children);
let leaf = modelEl.object3D.children[0].children.find(
(el) => el.name == "leaf003"
);
leaf.material.color.fromArray(colorScaleFunction(parseInt(msg)));
},
});
this function will be trigged by the MQTT message received under the topic moisture
if (topic.includes("moisture")) {
document
.querySelector("#moisture")
.setAttribute(
"value",
"Moisture: " + parseFloat(message.toString()).toFixed(2).toString() + "%"
);
this.colourLeaf(message);
}
At the core of the WebAR experience is the library AR.Js. Based on the open source library ARToolkit, AR.Js can be used with Three.js or AFrame and, like the last one, its Hello World scene requires just few lines of HTML to run.
Since version 3.0.0, AR.js has two different sources: one using NFT Image tracking, and a second one using Marker Based Tracking. The latter one is used in this workshop.
<!--AFrame version 1.3.0-->
<script src="https://aframe.io/releases/1.3.0/aframe.min.js"></script>
<!--AR.js version 3.4.0-->
<script
type="text/javascript"
src="https://raw.githack.com/AR-js-org/AR.js/3.4.0/aframe/build/aframe-ar.js"
></script>
Add in the body
of the webpage the tag a-scene
with the following components:
embedded
arjs
with the properties:debugUIEnabled: false;
sourceType: webcam;
trackingMethod: best;
color-space="sRGB"
vr-mode-ui="enabled: false;"
to remove the VR buttonrenderer
of the scene, with the properties:logarithmicDepthBuffer: true;
physicallyCorrectLights: true;
colorManagement: true;
<a-scene
embedded
arjs="debugUIEnabled: false; sourceType: webcam; trackingMethod: best;"
color-space="sRGB"
vr-mode-ui="enabled: false"
renderer="logarithmicDepthBuffer: true; physicallyCorrectLights: true; colorManagement: true; "
>
Add a camera entity with a unique ID
<a-entity id="userCamera" camera> </a-entity>
and the a-marker
tag with the attribute preset="hiro"
<a-marker preset="hiro"> </a-marker>
AR.js contains the pattern of two markers: Hiro, the image can be download here, and Kanji, the image can be download here.
When AR.js detects the marker, the entities within <a-marker> </a-marker>
are set to visible.
<a-marker preset="hiro">
<a-box position="0 0 0" material="color: blue;"></a-box>
</a-marker>
Here the complete index.html for the Hello WebAR scene
<!DOCTYPE html>
<html lang="en">
<head>
<title>WebAR</title>
<!--AFrame version 1.3.0-->
<script src="https://aframe.io/releases/1.3.0/aframe.min.js"></script>
<!--AR.js version 3.4.0-->
<script
type="text/javascript"
src="https://raw.githack.com/AR-js-org/AR.js/3.4.0/aframe/build/aframe-ar.js"
></script>
</head>
<body>
<a-scene
embedded
arjs="debugUIEnabled: false; sourceType: webcam; trackingMethod: best;"
color-space="sRGB"
vr-mode-ui="enabled: false"
renderer="logarithmicDepthBuffer: true; physicallyCorrectLights: true; colorManagement: true; "
>
<a-entity id="userCamera" camera> </a-entity>
<a-marker preset="hiro">
<a-box position="0 0 0" material="color: blue;"></a-box>
</a-marker>
</a-scene>
</body>
</html>
To test your code using the inbuilt webcam of your laptop, or an external webcam, in VSCode run the live server (restart VSCode if the live server is not available).
To test your code using a smartphone, run the Node.JS command ws --https
in the folder of your project (you can open a terminal directly in VSCode from the top menu).
Point your camera or smartphone to the Hiro marker to display the blue cube
The renderer
component in the a-scene
tag is used to control the final appearance of the AR cube: e.g. without physically correct lighting...
...and with physically correct lighting
It is possible to create custom markers using theonline Marker Training tool.
Choose the right image is a process of trial and error, here are some minimum requirements:
ar.js
component of the a-scene
tag, therefore, if multiple markers are used in the same AR experience, they will need to have the same pattern ratio to be correctly detectedPrepare two images (below two samples) and upload them, one at a time, to online Marker Training tool. Set the pattern ratio to 85%
and press Download Marker to download the PATT
file (the pattern recognised by the source camera), Download Image to download the image with the added black border.
a-marker
from preset="hiro"
to type="pattern"
and add the location of the .patt
file just downloadedFrom:
<a-marker preset="hiro">
<a-box position="0 0.5 0" material="color: blue;"></a-box>
</a-marker>
To:
<a-marker id="marker1" type="pattern" url="resources/patt/pattern-image1.patt">
<a-box position='0 0 0' material='color: blue;'></a-box>
</a-marker>
<a-marker id="marker2" type="pattern" url="./resources/patt/pattern-image2.patt">
<a-box position='0 0 0' material='color: red;'></a-box>
</a-marker>
arjs
the patternRatio
(e.g. 85% pattern ratio, 15% black border => 0.85).<a-scene
embedded
arjs="debugUIEnabled: false; sourceType: webcam; trackingMethod: best; patternRatio: 0.85"
color-space="sRGB"
vr-mode-ui="enabled: false"
renderer="logarithmicDepthBuffer: true; physicallyCorrectLights: true; colorManagement: true;"
></a-scene>
The markers can be used to display any kind of AFrame entity (e.g. primitive geometries) as well as external assets (e.g. images, videos, 3d models).
Plant
, display
to a folder of the project (e.g. ./resources/models/
), and add them to the Asset Management Systema-asset
<a-asset>
<a-asset-item id="plant" src="resources/models/Plant.glb"> </a-asset-item>
<a-asset-item id="display" src="resources/models/display.glb"> </a-asset-item>
</a-asset>
a-scene
using an empty a-entity
with attribute gltf-model
pointing at the unique ID of glb file <a-entity id="_display" gltf-model="#display" scale="0.1 0.1 0.1"
a-marker
, as well as the other a-entity
can have nested elements.<a-marker
id="marker1"
type="pattern"
url="./resources/patt/pattern-image1.patt"
>
<!--<a-box position='0 0 0' material='color: blue;'></a-box>-->
<a-entity
id="platPot"
gltf-model="#plant"
scale="1 1 1"
position="0 0 0"
rotation="0 0 0"
>
</a-entity>
</a-marker>
<a-marker
id="marker2"
type="pattern"
url="./resources/patt/pattern-image2.patt"
>
<!--<a-box position='0 0 0' material='color: red;'></a-box>-->
<a-entity
id="plantDisplay"
gltf-model="#display"
scale="0.1 0.1 0.1"
position="0 0 0"
rotation="0 0 0"
>
</a-entity>
</a-marker>
It is possible to create a complex AR experience just using HTML, however, each new marker needs to be added manually to the scene. It is possible to automate this process using custom Aframe components
We are going to use an external json file to store the information about our markers
markers.json
file in the resources folder{
"info": {
"version": "1.0",
"title": "WebAR-MultiMarkers",
"markers": 2
},
"content": [
{
"markerName": "pattern-image1",
"topic": "1",
"textContent": "ONE"
},
{
"markerName": "pattern-image2",
"topic": "2",
"textContent": "TWO"
}
]
}
The markerName field need to match the name of the patt file without file extension
markerManager.js
file and create a component registerevents
. This component will listen to the markerFound
and markerLost
eventsAFRAME.registerComponent("registerevents", {
init: function () {
const handler = this.el;
handler.addEventListener("markerFound", (event) => {
let markerId = event.target.id;
console.log("Marker Found: ", markerId);
});
handler.addEventListener("markerLost", (event) => {
let markerId = event.target.id;
console.log("Marker Lost: ", markerId);
});
},
});
registerevents
to the a-scene
tag. Once a marker is detected, the event
fired by the component will contain the name of the marker itself.markerManager.js
file, another component markers_start_json
//component to create the marker a-entity from the content/json
AFRAME.registerComponent("markers_start_json", {
init: function () {},
});
init
function we are going to create a variable to reference the a-scene
and parse the .json
file using the fetch() functioninit: function () {
console.log('Add markers to the scene');
let sceneEl = document.querySelector('a-scene');
//index.json contains the list of markers and content
fetch("./resources/markers.json")
.then(response => response.json())
.then(json => {
console.log(json.content);
})
index.html
add a new script tag just after the libraries<script src="./resources/js/markerManager.js"></script>
and attach also this component markers_start_json
Component to the a-scene
<a-scene
embedded
arjs="debugUIEnabled: false; sourceType: webcam; trackingMethod: best; patternRatio: 0.85"
color-space="sRGB"
vr-mode-ui="enabled: false"
renderer="logarithmicDepthBuffer: true; physicallyCorrectLights: true; colorManagement: true; "
registerevents
markers_start_json
></a-scene>
Array [ {...}, {...} ]
0: Object { markerName: "pattern-image1", topic: "1", textContent: "ONE", ... }
1: Object { markerName: "pattern-image1", topic: "2", textContent: "TWO", ... }
and a log every time a marker is detected
a-entity
tags we will need. The pseudo-structure of the a-marker
will be the following<a-marker>
<a-entity plantcontainer>
<a-entity plant 3D model>
<a-entity display 3Dmodel></a-entity>
</a-entity>
<a-entity text></a-entity>
</a-entity>
</a-marker>
Instead of using the default a-text
entity, it is possible to use the more flexible and efficient package based on Troika.js
<!--Text component based on TroikaJS -->
<script src="https://unpkg.com/aframe-troika-text/dist/aframe-troika-text.min.js"></script>
markerManager.js
and add the forEach
loop after the console.log(json.content)
in the fetch functionconsole.log(json.content);
json.content.forEach((el) => {});
Inside the loop, for each element el
(i.e. the two markers) the script needs to create and setup the following variables
//0. createa a string that contain the URL of the PATT file
let markerURL = "./resources/patt/" + el.markerName + ".patt";
//1. Create and add a a-marker to scene
let markerEl = document.createElement("a-marker");
markerEl.setAttribute("type", "pattern");
markerEl.setAttribute("url", markerURL);
markerEl.setAttribute("id", el.topic); //the topic from the json file
sceneEl.appendChild(markerEl); //Add the marker to the scene, this was declare outside the loop
//2. Add a text entity to each marker
let textEl = document.createElement("a-entity");
textEl.setAttribute("id", "text" + el.textContent); //the text from the json file
textEl.setAttribute("troika-text", {
value: el.textContent,
fontSize: 0.25,
align: "center",
color: "red",
});
textEl.object3D.position.set(-0.0, 0.1, 0.5);
textEl.setAttribute("rotation", { x: -90, y: 0, z: 0 });
markerEl.appendChild(textEl); //add the text to the marker
//Create the Plant Model and Panel for the data
//3. The main container
let plantRT = document.createElement("a-entity");
plantRT.setAttribute("id", "planRT_" + el.topic);
plantRT.setAttribute("rotation", { x: -90, y: 0, z: 0 });
plantRT.object3D.position.set(0, 0, 0.2);
plantRT.object3D.scale.set(1.5, 1.5, 1.5);
markerEl.appendChild(plantRT);
//4. the 3D model of the Display.glb
let screen = document.createElement("a-entity");
screen.setAttribute("id", "screen_" + el.topic);
screen.setAttribute("gltf-model", "#display");
screen.object3D.position.set(0.3, 0.41, -0.01);
screen.object3D.scale.set(0.2, 0.2, 1);
screen.setAttribute("rotation", { x: 0, y: 0, z: 0 });
plantRT.appendChild(screen);
//5. the 3D model of the plant
let modelplant = document.createElement("a-entity");
modelplant.setAttribute("id", "modelplant_" + el.topic);
modelplant.setAttribute("gltf-model", "#plant");
modelplant.object3D.position.set(-0.2, 0, 0);
modelplant.object3D.scale.set(1, 1, 1);
modelplant.setAttribute("rotation", { x: 0, y: 180, z: 0 });
plantRT.appendChild(modelplant);
We can now add new plants just by adding the patter file to the patt folder and new entries in the json file
We can now add the text on the virtual display, first let's add a clock on the top right corner of the display.
timenow
//Component to visualise Time on a a-text entity
AFRAME.registerComponent("timenow", {
init: function () {
// Set up the tick throttling. Slow down to 500ms
this.tick = AFRAME.utils.throttleTick(this.tick, 500, this);
},
tick: function () {
this.el.setAttribute("troika-text", { value: this.getTime() });
},
getTime: function () {
var d = new Date();
return d.toLocaleTimeString();
},
});
This component is going to be attached to a new a-entity
text we are going to add to the markers_start_json
component. After the model plant element
//5. the 3D model of the plant
let modelplant = document.createElement("a-entity");
modelplant.setAttribute("id", "modelplant_" + el.topic);
modelplant.setAttribute("gltf-model", "#plant");
modelplant.object3D.position.set(-0.2, 0, 0);
modelplant.object3D.scale.set(1, 1, 1);
modelplant.setAttribute("rotation", { x: 0, y: 180, z: 0 });
plantRT.appendChild(modelplant);
//6. Date.time component
let timenowText = document.createElement("a-entity");
timenowText.setAttribute("id", "timenowText_" + el.topic);
timenowText.setAttribute("timenow", "");
timenowText.setAttribute("troika-text", {
value: "t",
fontSize: 0.2,
anchor: "left",
});
timenowText.object3D.position.set(0.11, 0.85, 0.02);
screen.appendChild(timenowText); //add the text to the screen
Let's now create the labels and placeholder texts use to visualise the MQTT plant sensor values. In the markers_start_json
component add after the timenowText element
//7. Sensor Moisture
let moistureText = document.createElement("a-text");
moistureText.setAttribute("id", "moisture-text_" + el.topic);
moistureText.setAttribute("troika-text", {
value: "Moisture: ",
fontSize: 0.2,
anchor: "left",
});
moistureText.object3D.position.set(-0.9, 0.5, 0.02);
screen.appendChild(moistureText); //add the text to the screen
//8. Sensor Temperature
let temperatureText = document.createElement("a-text");
temperatureText.setAttribute("id", "temperature-text_" + el.topic);
temperatureText.setAttribute("troika-text", {
value: "Temperature: ",
fontSize: 0.2,
anchor: "left",
});
temperatureText.object3D.position.set(-0.9, 0.1, 0.02);
screen.appendChild(temperatureText); //add the text to the screen
//9. Sensor Humidity
let humidityText = document.createElement("a-text");
humidityText.setAttribute("id", "humidity-text_" + el.topic);
humidityText.setAttribute("troika-text", {
value: "Humidity: ",
fontSize: 0.2,
anchor: "left",
});
humidityText.object3D.position.set(-0.9, -0.3, 0.02);
screen.appendChild(humidityText); //add the text to the screen
Finally, we can add the MQTT.js library to connect the AR experience with the plant sensors
Add the mqtt.js library in the index.html, just after the Aframe and AR.js library
<!--MQTT.js-->
<script
type="text/javascript"
src="https://unpkg.com/mqtt@4.3.7/dist/mqtt.min.js"
></script>
We are going to update the registerevents
by adding the MQTT client connection to the broker and two sub-function startMqtt
(to subscribe to a specific topic) and stopMqtt
(to disconnect from the broker) when the markers are detected or lost
//Listen to the markers
AFRAME.registerComponent("registerevents", {
init: function () {
let clients = []; //a list of all MQTT clients
const handler = this.el;
handler.addEventListener("markerFound", (event) => {
let markerId = event.target.id;
console.log("Marker Found: ", markerId);
const client = mqtt.connect("ws://BROKER.ADDRESS:8080", {
clientId: event.target.id,
});
client.on("connect", () => {
console.log("Connected");
clients.push(client); //add the client to the Array
this.startMqtt(event.target, client);
});
client.on("error", (error) => {
console.log(error);
});
});
handler.addEventListener("markerLost", (event) => {
let markerId = event.target.id;
console.log("Marker Lost: ", markerId);
//Find the client according to its ID
let clientToStop = clients.find(
(client) => client.options.clientId === event.target.id
);
//remove the client from the Array
let index = clients.indexOf(clientToStop);
clients.splice(index, 1);
//pass the client to stopMQTT to end the connection
this.stopMqtt(clientToStop);
});
},
startMqtt: function (marker, client) {
if (!client.connected) {
client.reconnect();
}
client.subscribe("TOPIC/TO/SUBSCRIBE/Moisture");
client.subscribe("TOPIC/TO/SUBSCRIBE/Temperature");
client.subscribe("TOPIC/TO/SUBSCRIBE/Humidity");
client.on("message", function (topic, message) {
//Called each time a message is received
console.log("Received message:", topic, message.toString());
if (topic.includes("Moisture")) {
marker
.querySelector("#moisture-text_" + marker.id)
.setAttribute("troika-text", {
value: "Moisture: " + message.toString() + "%",
});
}
if (topic.includes("Temperature")) {
//temperature
marker
.querySelector("#temperature-text_" + marker.id)
.setAttribute("troika-text", {
value:
"Temperature: " +
parseFloat(message.toString()).toFixed(1).toString() +
"°C",
});
}
if (topic.includes("Humidity")) {
//humidity
marker
.querySelector("#humidity-text_" + marker.id)
.setAttribute("troika-text", {
value: "Humidity: " + message.toString() + "%",
});
}
});
},
stopMqtt: function (client) {
client.end(true);
console.log("connection closed");
},
});