LS009 — Session persistence & mouth gate

Release: RMHLipSyncDemo Shipping — build stamp Apr 22 2026 01:13 (see debug overlay header row once loaded) Audience: Mindlabs backend / DevOps / CI-CD maintainers Purpose: One required CI/CD change, zero protocol changes. This document explains what’s new in the avatar build you’ll be receiving, what you need to adjust in your deployment pipeline, and how to verify the update is working.

1. TL;DR

Change you must make	Why	Effort
Launch `Run-Cloud.bat` instead of `BasicPX.exe` on your EC2 instances	Applies cloud-optimal flags (headless rendering, fixed resolution, muted local audio) and forwards your runtime arguments correctly	1-line change in your CI/CD launch command

Everything else — TCP protocol, JSON command schema, audio format, PixelStreaming signaling — is unchanged. No code changes on your side are required.

2. Required CI/CD change

Before

Your pipeline most likely launches:

H:\path\to\package\Windows\BasicPX.exe -PixelStreamingURL=ws://signaling:80 <other args>

After

Replace BasicPX.exe with Run-Cloud.bat:

H:\path\to\package\Windows\Run-Cloud.bat -PixelStreamingURL=ws://signaling:80 <other args>

That’s it. Your extra arguments are forwarded through the wrapper (%* passthrough). If the wrapper file is missing, the launch fails with exit code 2 and the message [Run-Cloud] ERROR: BasicPX.exe not found — use that as your CI/CD smoke signal.

What the wrapper applies on your behalf

-RenderOffScreen — renders to a GPU texture without needing an attached display. Required on headless EC2; avoids the common “no desktop” startup failure.
-ForceRes -ResX=540 -ResY=960 — half-portrait HD, fixed. Matches the baked PixelStreaming bitrate budget.
-NoSound — skips local audio hardware (EC2 has none); audio is carried over WebRTC as it always was.
-AvatarPort=4500 — default TCP port where your Director connects. Override with -AvatarPort=<N> in your CI/CD args if you use a different port.

What’s NOT in the wrapper

-PixelStreamingURL=... — per-deployment value, must come from your CI/CD.
Any custom flags you already pass (they’re forwarded verbatim).

3. What changed in the avatar engine (and why you care)

3.1 Lip-sync is now resilient to rapid emotion commands

Previous behavior: if your backend sent several emotion commands in close succession while the avatar was speaking (e.g. under 500 ms apart), the lip-sync pipeline would visibly stutter — mouth opens, then hangs slightly open longer than expected before closing. Non-deterministic, correlated with emotion volume.

New behavior: the lip-sync pipeline is decoupled from emotion events during speech. Rapid-fire emotion commands are handled as pure state transitions with no pipeline disruption. The emotion face (brows, eyes, cheeks) still applies; the mouth remains fully owned by the ONNX lip-sync model while speech is active.

Backend impact: none. You can keep sending emotions at whatever cadence you want. Timing-sensitive work on your side is no longer required.

3.2 Mouth-zone curve isolation

During active speech, the avatar now strictly prevents emotion-driven mouth articulator curves (jaw open, jaw forward, mouth close, mouth pucker, mouth funnel, etc.) from stacking on top of the lip-sync model’s output. Structural emotion signals (smile, frown, press) still apply and remain visible.

Backend impact: none. Emotions look slightly less “exaggerated at the mouth” during speech than before, but the lip-sync itself is visibly cleaner and more accurate.

3.3 Cloud-tuned defaults baked in

The following values are now built into the Shipping package and require no runtime overrides from your side:

Setting	Baked value	Rationale
Render FPS cap	30	Safe for multi-avatar co-hosting on a single EC2 GPU
WebRTC target FPS	30	Matches render cap
WebRTC start bitrate	4 Mbps	Sensible initial value for half-portrait HD
WebRTC min bitrate	0.5 Mbps	Floor during congestion
WebRTC max bitrate	8 Mbps	Ceiling for the chosen resolution
Keyframe interval	60 frames (= 2 s at 30 FPS)	Faster recovery from packet loss

If you want to override any of these per-deployment, your CI/CD can still pass the standard UE command-line arguments — your overrides take priority.

3.4 Runtime diagnostic warnings (new, opt-in)

The avatar now accumulates a ring buffer of runtime warnings visible on the debug overlay. You don’t need to enable this for normal operation, but it’s useful when QA or support investigates a specific issue.

Debug overlay default: OFF (production deployments never show diagnostic UI to end users). Three ways to enable:

Launch-time (recommended for QA sessions): add -DebugOverlay to the exe command line, e.g.:
```
Run-Cloud.bat -PixelStreamingURL=ws://... -DebugOverlay
```
Runtime (TCP, JSON): {"type":"config","key":"debug_overlay","value":1}
Blueprint: UAvatarDebugOverlaySpawner::ShowDebugOverlay() (editor / custom builds only).

Warning categories you may see:

Category	Meaning	Typical response
`[Emotion]`	An emotion command arrived during active speech. System is handling it correctly; this is informational.	None — just confirms the event happened.
`[LipSync]`	Lip-sync ONNX generator was rebuilt. Normal on first utterance; suspicious mid-session.	If you see this mid-flow, capture logs and share.
`[Audio]`	Audio queue depth exceeded ~60 ms — ONNX inference is running behind the audio stream.	If sustained, the EC2 instance is CPU-starved (multi-tenant symptom). Scale up or reduce co-tenants.

Warnings appear at the bottom of the debug overlay in amber text. The ring buffer holds the last 20.

4. Verification checklist for your test deployment

Once you’ve deployed a build with Run-Cloud.bat in the launch line:

Process starts without a visible window. Confirm via process manager that BasicPX-Win64-Shipping.exe is running. If a window pops up on an EC2 desktop, -RenderOffScreen is not taking effect — check the CI/CD is actually invoking Run-Cloud.bat not the raw exe.
TCP port 4500 is bound. Your Director should be able to connect as before.
WebRTC stream is 30 FPS. Easy to verify client-side: the received stream should advertise 30 FPS; if still 60, your deployment is overriding the baked default somewhere.
Lip-sync smoothness under emotion load. Stress-test: fire emotion commands at 2-5 Hz while audio is playing. Mouth should close fully between phonemes, no “held open” artifact. If you still see the old lag, verify you’re running the new build (debug overlay shows a build stamp in the header row).
Warnings panel populates. Enable debug overlay; you should see [LipSync] Generator rebuilt … as the first warning on the very first utterance.

5. Known behaviors that aren’t bugs

1 ms Speaking → Emoting → Speaking state flips appear in the logs when an emotion fires during active speech. This is the new correct behavior — the emotion event is registered in the state machine but the lip-sync pipeline is not interrupted.
First utterance has a small delay (~1-2 s) when a brand-new ONNX generator is loaded. This is the cost of model initialization; subsequent utterances reuse the generator and start instantly. The [LipSync] Generator rebuilt … warning marks this moment.
Debug overlay is off by default. Has to be enabled per-session via -DebugOverlay launch flag or the TCP config command above. Never visible to end users unless you opt in.
t.MaxFPS=30 is set as a console variable in Shipping. This is the intended cap. Local-dev tests on a 60 Hz monitor may observe effective 60 FPS due to vsync — this does not affect cloud deployments where there is no vsync.

6. Rollback

If the new build causes a blocker you can’t work around:

In your CI/CD, change back to BasicPX.exe (or your previous launch line).
Re-deploy the previous Shipping build artifact (the one you were using before this update).

The new build is a drop-in replacement with strictly additive fixes — there’s no schema change to roll back. Your backend code continues to work against either build.

7. Questions / support

Log captures are always more useful than descriptions. If you hit an issue, please include:

Build stamp shown on the debug overlay header row (e.g. Apr 22 2026 01:13)
Contents of %LOCALAPPDATA%\RMHLipSyncDemo\Saved\Logs\RMHLipSyncDemo.log (on Windows EC2) from the affected session
Reproduction steps if the issue is deterministic

For full bug-report / feature-request / log-submission templates and the contact channel matrix, see Support. The contract-specific channels (P0 escalation, dedicated Slack) take priority over the placeholders on that page.