> For the complete documentation index, see [llms.txt](https://ms-kb.msd.unimelb.edu.au/next-lab/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ms-kb.msd.unimelb.edu.au/next-lab/3d-scanning/guides/photogrammetry/realityscan.md).

# RealityScan

## Taking the Photos

If you do not have your photo-set ready for processing, please refer back to Overview, Process and the Benchmark:

{% content-ref url="/pages/-M20rqsMHKGe1hSgMoH9" %}
[Principles of 3D Scanning / Digital Reconstruction](/next-lab/3d-scanning/guides/digital-reconstruction-guide.md)
{% endcontent-ref %}

{% content-ref url="/pages/-M20qXkNtyBvaCgIVA4g" %}
[Photogrammetry](/next-lab/3d-scanning/guides/photogrammetry.md)
{% endcontent-ref %}

{% content-ref url="/pages/-M2I8P2O1dltXE5tjGCJ" %}
[Photogrammetry Benchmark](/next-lab/3d-scanning/guides/photogrammetry/photogrammetry-benchmark.md)
{% endcontent-ref %}

## Main Workflow

### Interface

1. Photogrammetry workflow along this main tab.
2. Pick your UI layout window layout, the 1x1x1 one is a good place to start.
3. When you are in certain windows, you will get additional controls

<figure><img src="/files/0rGrOOrPrDjEMTSzX4tg" alt=""><figcaption></figcaption></figure>

You can use the selection menu in the top-right of any window to change its view mode.

<table data-header-hidden><thead><tr><th width="374"></th><th></th></tr></thead><tbody><tr><td>1Ds - project structure, properties and settings. You should always have a 1Ds view.</td><td></td></tr><tr><td>2D - single photo view</td><td>2Ds - gallery view.</td></tr><tr><td>3Ds - main 3D view</td><td>4Ds - animation controls</td></tr></tbody></table>

<figure><img src="/files/sESz0NRZys2NUSv2484Y" alt=""><figcaption></figcaption></figure>

***

### Import Data

Import data using the *Workflow* tab or drag and drop your photoset in.

### Align Data

The *Alignment* tab holds all the relevant functionality - alignment refers to the algorithms finds each photo's orientation and location via 'tie points' - landmark features that are triangulated across the set.

1. **Adjust settings in** ***Registration > Settings***\
   The main settings to look at are:
   1. **Feature Detection Quality**: High can help in tricky datasets
   2. **Max features**: Increase up to 2x to look for more landmark features (tie points)
   3. **Image Downscale**: Lower number = better result.
2. **Align in&#x20;*****Registration > Align Images***
3. Visually inspect this Sparse Point Cloud (or tie points) for any errors, you should be able to spot if there are points where there should not be any. If there are problems, read further down the page for correcting errors.

{% hint style="info" %}
RS projects organise models into '**Components**'
{% endhint %}

<figure><img src="/files/6SXfbeOgU8XrXAGqmIin" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}
At this point, you might want to define the scale of the model - photogrammetry does not inherently have any sense of scale or orientation.
{% endhint %}

{% hint style="danger" %}
In an ideal world, your photography set is perfect and this works like a charm.\
However, if a lot of images do not align (check the number registered amount), or if you get multiple components or small components - check **Secondary Workflows** further down.
{% endhint %}

***

### Meshing

The *Mesh & Color* tab is for converting your dataset into a 3D mesh model.&#x20;

1. Define where it meshes via ***Reconstruction Region > Set Reconstruction Region***
2. Define settings ***Create Mesh > Settings***

   You can define Normal and Preview level settings respectively (but not high)\
   The main settings to look at are:

   1. Image depth map calculation > (Normal) > **Image downscale:**\
      Lowest number (1) for best result.
   2. Mesh Calculation > **Miniminal Dist:** Use this to set a specific resolution. \
      (Requires you to have defined the scale first, find in *Secondary Workflows* below)
3. Generate the mesh using ***Create Mesh > Normal Detail***

{% hint style="info" %}
Models are created per Component.
{% endhint %}

<figure><img src="/files/GoJddJ0HdgWb6FNnnyPj" alt=""><figcaption></figcaption></figure>

### Colour/Texture

Still in the Mesh & Color tab, there are two options for getting the colour data back on to the mesh, *Colouring* refers uses Vertex Colours (each mesh vertex is assigned a colour, colour detail is limited as it is dependent on resolution) or *Texturing* (the model has the photos mapped onto the surface). Texturing is preferred for superior colour reproduction.

{% hint style="info" %}
Use the Scene 3D View tab to change display options.
{% endhint %}

#### Colouring

1. Colour via **Color & Texture** > **Colorize** - Done!

#### **Texturing**

1. Unwrap the model using **Color & Texture > Unwrap**\
   This maps the model's various surfaces into a 2D texture - so the photo details can be remapped on.
2. Find the settings in your 1Ds panel:
   1. Change the max resolution and max textures to account for the size of the object:\
      More detail wanted = more textures and higher resolution.
   2. Press the 'Unwrap' at the bottom of this panel to commit.
3. Texture via **Color & Texture > Texture**

<figure><img src="/files/54LwcUHu8rJNQbprKCGx" alt=""><figcaption></figcaption></figure>

***

### Clean Up & Export

In the Scene 3D tab > Tools > Model & Texture, you will find a variety of clean up tools for cleaning up the mesh. \
\
Select parts manually using **Lasso** or **Rect,** **Advanced** allows for selecting geometry by the actual size of the triangles. **Filter Selection** to remove them.

Another helpful tool is to **Cut by Box** (using the Reconstruction Region) for quickly keeping a region.

#### **Export**

Export via Scene 3D Tools into mesh (.obj or .ply), or point cloud formats (.las).

***

## Secondary Workflows

### Additional Concepts

#### Selective Editing

Specific photos can be enabled/disabled for any step in the process of alignment, meshing and texturing. If working with multiple components, or you do not want a photo to contribute to colour etc., turn it off where appropriate.&#x20;

You can bulk select etc. to make this easier.

<figure><img src="/files/akS8Xq5A7nP76aWf4QO9" alt="" width="274"><figcaption><p>Enable/disable found in each input's settings.</p></figcaption></figure>

#### Aligning Refinement

RealityScan's alignment algorithm is not very transparent in what it does - depending on when you use it / what state the model is in - it'll either align or refine.

For example, if some parts are looking off, try running alignment again.

Or, if only a few cameras are off, use Selective Editing (above) to only align select cameras, and then align again to further refine the model.

#### Using Control Points

Alignment Tab > Constraints - is where to find a variety of ways to manually intervene in many alignment processes.

Control Points (F3) are the main way to stitch parts together or correct errors.\
Hover over a single image, or ideally, the 3d model or point cloud to place these.

Once placed, you must press the <mark style="background-color:$success;">**green +**</mark> to commit them. They can still be moved around in a 2D view per image. Use the error rates to inform your judgement.

<mark style="color:$primary;">**Each control point needs to be seen by at least two inputs (per component).**</mark>\ <mark style="color:$primary;">**Minimum 3 control points- 4-5 is preferred. These control points cannot be in a single line**</mark><mark style="color:$primary;">.</mark>

<figure><img src="/files/AjqBkLSEtme4Wz370SKR" alt=""><figcaption></figcaption></figure>

***

### Combining Parts Together

If the project splits into multiple components, you can force them together with **Control Points.** \
If you are capturing an object from multiple sides, like a top and bottom - continue reading.

Use **Control Points** to identify same landmarks across multiple components.\
Use **Merge Components** to only combine the components as they are - this will not adjust the alignment in any way.\
OR, use **Align** to re-run with the Control Points to constrain components together.

<figure><img src="/files/DOSl2uS5Np4NzTNaMFnO" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}
If multiple parts need to be combined, it is recommended to focus on two components at a time - remember to disable any photos that are not being currently used.
{% endhint %}

**Combine Datasets Using Masks**

If combining a good dataset from multiple sides - refer to this technique:

{% embed url="<https://www.youtube.com/watch?v=4qFl4k37dDc>" %}

***

### Correcting Errors

A selective example of the Aligning Refinment above.

Most of the time, you should be able to identify errors in the sparse point\
A easy way to find cameras that are off is in the Scene 3D Tools tab.

1. First use the **Point Lasso** to select errant tie points.
2. **Find Cameras** to select the cameras that are displaying errors.<br>

   <figure><img src="/files/xHNGShxPxnDM9z73UdoK" alt=""><figcaption></figcaption></figure>
3. You may try selective aligning to fix this, but a more ensured method is to use **Control Points**.\
   Place as many as needed to fix errant images - the error rates will naturally be really high as the software thinks they were in the right location - you use **Control Points** to correct it.<br>

   <figure><img src="/files/sQtEYdXkilMFZTXdMkXi" alt=""><figcaption></figcaption></figure>
4. Once adequate **Control Points** have been added, Align again to correct.

<figure><img src="/files/7O2uGUM6Su4dyEYgVBBM" alt=""><figcaption></figcaption></figure>

***

### Defining Scale

Alignment tab > Define Distance will guide you in setting up two Control Points that you have a measurement for.\
You may set up as many of these constraints as required.

In the 1Ds, enter in your measured distance in the Defined Distance.

<figure><img src="/files/Mi6gInZ5wTjUNidR3Tfy" alt=""><figcaption><p>Default photogrammetry scale is wrong, here it reports this distance as almost 10m!</p></figcaption></figure>

Align, or Update, to apply this scale.

<figure><img src="/files/QHtC7WF1PtI4NmjPtXtI" alt=""><figcaption><p>Define the distance and then allow it to recompute.</p></figcaption></figure>