←LXXVII
Theatrum mechanicum

Afterglow

A live-streaming setup for one person, with a real-time 3D avatar on camera instead of the streamer.

Status
Live on stream
Engine
Unreal Engine 5
Source
Closed
Crew
One person
01

Overview

Afterglow replaces the streamer on camera with a real-time 3D avatar. One person runs the whole stream, with the kind of production that usually needs a crew.

Hands one to one

The avatar's fingers have the same lengths as the streamer's, so no fitting is needed.

LED face

An LED-matrix face shows emotions read from the streamer's face.

Two voice paths

The stream gets the voice with a mood mask. The game gets the clean voice.

Supervised services

A supervisor restarts any service that stops. The streamer controls everything from a phone.

02

Architecture

Picture, sound and viewer input run in parallel. They meet in the compositor, which combines the scene, the game and the overlays into the broadcast.

InputCamerasPhone: both handsDepth: the faceTrackingLifted to 3DOn the desk plane,expression rulesUnreal Engine 5SceneAvatar, one to oneLED face and wallCompositeCompositorScene, game,overlaysOutBroadcastThe live streamInMicrophoneDenoise, EQ, gateVoice chainMood maskStream branchUntouchedClean branchVoice chat, gameGaming PCChat line, musicOverlaysgameCompanion siteViewers order tracks and spend pointsorders flow back into the chat line
Fig. 2Block level only. The source is closed.
03

Anatomy of the rig

The same system, drawn as a room. The desk and cameras are in front, the machines are on the right, and behind them is the stage that the scene renders. Hover over a part to read what it does.

UE5
Fig. 1The stageThe scene renders the avatar at its desk, an LED-matrix face that shows its emotions, and a large LED wall behind it.
04

One to one

Most avatar setups use a stock hand and a solver that bends it until it looks close. Afterglow sets the avatar's finger bones to the lengths of the streamer's fingers, so the joint angles and fingertips match without fitting. The engine reports the final joint positions back, which makes the error measurable.

The handMeasureThe streamer's ownfinger lengthsThe bonesRe-lengthenAvatar fingers setto those lengthsThe anglesDriveJoint angles copied,nothing fittedThe engineReportSends its finaljoints backThe matchCompareMeasured in pixelsand millimetres
Fig. 3From the streamer's hand to a measured match.
05

Services

Small services run around the scene. The supervisor restarts any of them that stops.

ServiceWhat it does
OverlaysGraphics over the stream, including the chat line
MusicTracks, including the ones viewers order
Video wallThe giant LED wall behind the avatar
LED faceThe avatar's emotions, from the face tracking
DashboardControls the stream from a phone
Viewer siteShows the stream in a CRT style. Viewers order tracks and spend points there
1 : 1finger bones to the streamer's hand
2voice branches, stream and clean
~12services kept alive
1person runs the stream
06

Where it stands

Live on stream. The full stack runs today, and measuring hand accuracy comes next.

Works today

  • The full stream stack
  • The avatar scene
  • The voice chain
  • The LED face and the video wall
  • Overlays, viewer site and dashboard

Next

  • Measured verification of hand accuracy
  • Three-camera body tracking
  • Finer lighting