> For the complete documentation index, see [llms.txt](https://ms-kb.msd.unimelb.edu.au/next-lab/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ms-kb.msd.unimelb.edu.au/next-lab/augmented-reality/guides/fologram/extended-functionality/interactivity.md). # Interactivity Official Tutorials: {% embed url="" %} ## Overview While [**Syncing Parameters**](https://msd-makerspaces.gitbook.io/next-lab/augmented-reality/guides/fologram/fologram-grasshopper#syncing-parameters) is the easiest way to build control into the Fologram experience, for more immersive experiences or where accessing the Parameters menu inside Fologram is not desired, is article is to start you off with ways to interact with the scene more directly through things like holographic buttons. ## Tracking Use the \[Connected Devices] Component to access the connected Hololens devices properties, this gives Grasshopper an awareness of connected devices and how they are interacting with the scene, returning metadata such as device details, or positional data. These allow you to use this to manipulate data flows and the scene itself. ![Basic setup: use \[Connected Devices\] to begin tracking Hololens meta and positional data.](/files/-MhHaNN8tPq3UjXs5P95) {% hint style="info" %} Just like any other Grasshopper component, the right click context menus and list management options are also available for some components to extend their functionality. {% endhint %} ![](/files/-MhM2QasDCvoCOPuJir9) ## Triggers Triggers is the concept of how you intend a user to interact to with the scene that can 'trigger' some interactive aspect. There are two main ways provided by the Fologram for Grasshopper library. ### Taps Taps refers to the tapping gesture of the Hololens, allowing you to track if objects have been tapped.\ This can be tracked through the \[Track Taps] component, which gives you the device that initiated the tap, and the item that was tapped. \ \ **This needs to be enabled through the right-click menu of a Synced Object.** ![](/files/-MhH_mkE2N0W1AGRuRvq) ### Collision Collision can be conceptualised as when objects touch 'physically' touch. Such as your hand touching a 'button' geometry. This can be tracked using any of the components that returns positional data, for example:\ \&#xNAN;**`[Track Device]`** gives the transform of the Hololens device itself\ \&#xNAN;**`[Track Hands]`** gives the position of the hand model vertices tracked by the camera ### Finger Keys Use finger gesture combinations to trigger behaviours. The provided script is a bare example: {% file src="/files/mEC0X2GmGC3OKN83IWuB" %} {% embed url="" %} Gaze Use your gaze (the position of the Hololens device) to trigger. The provided script is a bare example: {% file src="/files/hfgNW1WwENvZ8qWiEXfJ" %} ## Behaviours With the triggers set up, you can then do basic data manipulation and flows to program a variety of effects. ### Showing Rhino Layers This is a basic setup to demonstrate how to use **`[Track Taps]`** to interact with Rhino layers, while you can go through the menus through Fologram to do the same thing, this can be a more intuitive or immersive method. More generally, this shows how a **`[Data Recorder]`** combined with **`[List Length]`** and **`[Modulus]`** can be used to create a toggling behaviour. This uses HUMAN plugin components to interact with Rhino layers.
### Showing Selected Elements \[GH] This is a basic setup to demonstrate how to use **`[Track Taps]`**'s returned selected index to show an item corresponding to a button. As long as your list orders are maintained across the two sets (buttons and geometry-to-show), then **`[List Item]`** can used to selectively stream from the list.\ \ Note that any geometry that is intended to be tap-able will need to be made so via the **`[Sync Object]`**'s Right Click Menu > Clickable. ![](/files/-MhHnwqO533_nlf8Gp2v) ### Incremental Slider This is a basic setup to demonstrate how **`[Track Tap]`** can be used in conjunction with **`[State Gate]`** to mimic a slider using a rocker and reset button. **`[Counter]`** is used to keep track of the clicks. **`[Counter]`** increments every time its *Trigger* input is updated, we can take advantage of this by using the **`[State Gate]`** to toggle these inputs. ![](/files/-MhLv9mu8hZbp6lCu55-) ### Collision Buttons This is a basic setup to demonstrate the use of **`[Track Hands]`** to test for collision with an object to simulate a 'button' interaction by averaging the hand position to use with **`[Point in Brep]`**. ![](/files/-MhLyrzZOatzGXMh4sJQ) ### Air Swipes This is an example grasshopper script that converts **`[Track Hands]`** into data that can then be used in various ways. The script provided is the bare working essentials. Some ways that they may be used might be to inspect an object in 3D or to change a value in an immersive way. {% file src="/files/8da9R9Qq3V37CDsL8aEE" %} {% embed url="" %} {% embed url="" %} ## Animation Animation can be achieved through the **`[Assign Transform]`** component. This is used to parametrically update the transform of an already synced object - for example, it can be used to parametrically place an object, such as having it follow the user, or for animation. Animation can be thought of as providing a series of transforms over a period of time to give the illusion of changes in time. While this could be done by syncing an object over time, using Assign Transform is more efficient as it does not need to send new geometry every transform. This also means that animation is limited to basic transformations and you should be wary about large meshes that may be too taxing on your device. The ***general setup*** is as follows: 1. Setup how the object should transform, you do not need to transform the object but you can use the Transform output of these components (move, rotate, orient, etc.) to get the transform. 2. Setup the list of transforms to go through, in this case, Heteroptera's timer-based streaming component is used to scale the transform between 0 and 1 at an increment of 0.001. This could be any list of transform data though. 3. Use Grasshopper's trigger component as a timer.
--- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://ms-kb.msd.unimelb.edu.au/next-lab/augmented-reality/guides/fologram/extended-functionality/interactivity.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.