Operator Presentation

What

The Operator Presentation Module turns operator-side feedback into output. It may update a video surface, draw a predicted path, animate a timeline, play audio, or drive a haptic device. The paper treats these as presentation choices: the module determines what the operator sees or feels, not how the remote robot moved or how communication was delayed.

It receives output from Operator-side Assistance (downlink) when that module is selected. With no downlink assistance, it receives released feedback directly from Downlink Communication.

Why

Presentation needs its own module because a study may change the operator interface without changing the feedback it receives. A delayed camera frame can be shown full-screen, placed beside a map, or combined with a haptic cue. A predicted path can be rendered as a line or an uncertainty region. Keeping those choices here makes the interface part of the study configuration rather than hidden inside transport or assistance code.

How

Operator Presentation receives released or assisted feedback, maps each package to a local source, and updates those sources for the operator.

The presentation module queues packages and processes them in release order. It then updates local presentation sources once per kernel step. A package handler should route or apply the package; it should not add communication delay or consult authoritative remote state.

• Package arrival. PresentPackage gives the implementation each released or assisted package.
• Source update. UpdatePresentation advances time-based output such as animations, throttled mesh refreshes, and delayed video surfaces.
• Source registry. Each OperatorPresentationSource has a unique id and owns its local camera, texture, mesh, audio, or haptic target.

Boundary

Contract

The base module owns the package queue and validates the configured source registry. Your implementation provides the two hooks that receive packages and update the local output.

public abstract class OperatorPresentationModule
    : TeleroboticsModule, IPackageConsumer
{
    protected abstract void PresentPackage(
        Package package,
        double timestampSeconds);

    protected virtual void UpdatePresentation(
        double timestampSeconds)
    {
    }
}

• PresentPackage handles a released package or passes it to a source.
• UpdatePresentation runs every presentation step, including ticks with no new package.
• OperatorPresentationSource is the extension point for a concrete output target.
• The module should preserve package meaning; display transformations belong in the selected presentation source.

Examples

The four conditions keep delayed video fixed while changing the timeline, path, and envelope overlays.

public sealed partial class RoverViewPresentation : OperatorPresentationModule
{
    private void PresentReconstructedVideo()
    {
        ViewFramePackage view = stateReconstruction.LatestViewFrame;
        if (view == null || !view.Frame)
            return;

        operatorVideoImage.texture = view.Frame;
        operatorVideoImage.color = Color.white;
        predictionCamera.worldToCameraMatrix =
            view.WorldToCameraMatrix;
        predictionCamera.projectionMatrix =
            view.ProjectionMatrix;
    }
}

public sealed class NetworkTimelineVisualisation : EveryMoveVisualisation
{
    public override bool TryPresent(Package package, double timestampSeconds)
    {
        if (!(package.Payload is NetworkTimelinePackage timeline) ||
            timeline.Directions == null)
            return false;

        for (int index = 0;
             index < timeline.Directions.Length;
             index++)
        {
            SpawnTimelineNode(
                (int)timeline.Directions[index],
                timeline.CommandTimestampSeconds,
                timeline.RoundTripDelaySeconds);
        }
        return true;
    }
}

public sealed class PathRibbonVisualisation : EveryMoveVisualisation
{
    public override bool TryPresent(Package package, double timestampSeconds)
    {
        if (!(package.Payload is PathAssistancePackage path))
            return false;

        pending = path;
        return true;

    }

    public override void UpdateVisual(double timestampSeconds)
    {
        DrawRibbon(centreMesh, pending.Centre);
        DrawRibbon(leftMesh, pending.Left);
        DrawRibbon(rightMesh, pending.Right);
    }
}

public sealed class EnvelopeRegionVisualisation : EveryMoveVisualisation
{
    public override bool TryPresent(Package package, double timestampSeconds)
    {
        if (!(package.Payload is EnvelopeAssistancePackage envelope))
            return false;

        pending = envelope;
        return true;

    }

    public override void UpdateVisual(double timestampSeconds)
    {
        TrajectoryMeshing.BuildStudyEnvelopeMesh(
            pending.Centre,
            pending.Left,
            pending.Right,
            pending.ExtremaLeft,
            pending.ExtremaRight,
            terrain,
            regionMesh);
    }
}

Config

1. Implement OperatorPresentationModule in the study's Runtime assembly.
2. Attach one presentation implementation to the TeleroboticsAgent and assign its OperatorPresentationSource components.
3. Give every source a unique id. The base module validates the source registry before the trial starts.
4. Keep presentation targets local to the source: cameras, canvases, meshes, textures, audio outputs, or haptic devices.
5. Do not read the live Vehicle / Robot Model from presentation code. Use Operator-side State Reconstruction or the released package data that the study has made available.