Skip to content

OSC Face Tracking

TL;DR: Every OSC float value sent to /avatar/parameters/<Name> is enqueued into both the avatar animator parameter queue and the face-tracking parameter queue. Declaring an animator parameter with one of the names below is enough to drive that face-tracking shape from OSC — no extra setup. int/bool OSC values only update the animator parameter, not the face-tracking buffer. Eye and mouth modules time out after 5 s with no incoming packet.

OSCAvatarModule.HandleIncoming parses the address and value type:

if (address.StartsWith("/avatar/parameters/"))
{
string name = address.Substring("/avatar/parameters/".Length);
if (arg is int i) _avatarIntParamQueue.Enqueue(...);
if (arg is float f)
{
_avatarFloatParamQueue.Enqueue(...);
_faceTrackingParamQueue.Enqueue(new FaceTrackingParameterPayload(name, f));
}
if (arg is bool b) _avatarBoolParamQueue.Enqueue(...);
if (arg is null) _avatarNullParamQueue.Enqueue(...); // dispatched as 1.0
}

The face-tracking payload’s hash is computed from the last /-separated segment of the path:

ParameterHash = Animator.StringToHash(
name.LastIndexOf('/') == -1 ? name : name.Substring(name.LastIndexOf('/') + 1));

So /avatar/parameters/EyeLidRight and /avatar/parameters/v2/EyeLidRight both resolve to the same face-tracking hash. The avatar animator queue uses the full segment (v2/EyeLidRight) as the parameter name, so prefixed VRC-style names won’t collide with non-prefixed animator parameters unless your avatar declares a parameter literally called v2/Foo.

The avatar parameter dispatch always routes through PlayerSetup.ChangeAnimatorParam(name, value, ParameterChangeSource.OSC) — see Parameter Setters.

Module: OSCEyeTrackingModule._parameterEntries. Each name maps to one UnifiedEyeExpression:

ParameterMaps toNotes
EyeLidRightEyeWideRight + blinkRightBlink derived as 1 - InverseLerp(0, 0.75, value). Wide is InverseLerp(0.75, 1, value).
EyeLidLeftEyeWideLeft + blinkLeftSame dual mapping.
EyeSquintRight / EyeSquintLeftEyeSquintRight / EyeSquintLeft
BrowPinchRight / BrowPinchLeftBrowPinchRight / BrowPinchLeft
BrowLowererRight / BrowLowererLeftBrowLowererRight / BrowLowererLeft
BrowInnerUpRight / BrowInnerUpLeftBrowInnerUpRight / BrowInnerUpLeft
BrowOuterUpRight / BrowOuterUpLeftBrowOuterUpRight / BrowOuterUpLeft

OSCEyeTrackingModule._specialParameterEntries handles a few that are not single-shape one-to-ones:

ParameterEffect
EyeLeftX, EyeLeftYStored in _eyeLeft.x/y; combined with right to derive gazePoint.
EyeRightX, EyeRightYStored in _eyeRight.x/y.
PupilDilationDual-mapped: dilation is InverseLerp(0.5, 1, value) (active above 0.5); constriction is InverseLerp(0.5, 0, value) (active below 0.5). The result is written to both eyes — indices 4 and 5 for dilation, 6 and 7 for constriction in _eyeData.unifiedEyeExpressions.

OSCQuery surfaces these special names additionally under the v2/ prefix (v2/EyeLeftX, etc.) so VRC-style senders work without remapping.

Module: OSCMouthTrackingModule._parameterEntries. Every Unified Expressions v2 name routes to a UnifiedMouthExpression index. Names recognised:

NoseSneerRight NoseSneerLeft
NasalDilationRight NasalDilationLeft
NasalConstrictRight NasalConstrictLeft
CheekSquintRight CheekSquintLeft
JawOpen MouthClosed
JawClench JawMandibleRaise
LipSuckUpperRight LipSuckUpperLeft
LipSuckLowerRight LipSuckLowerLeft
LipSuckCornerRight LipSuckCornerLeft
LipFunnelUpperRight LipFunnelUpperLeft
LipFunnelLowerRight LipFunnelLowerLeft
LipPuckerUpperRight LipPuckerUpperLeft
LipPuckerLowerRight LipPuckerLowerLeft
MouthUpperUpRight MouthUpperUpLeft
MouthLowerDownRight MouthLowerDownLeft
MouthUpperDeepenRight MouthUpperDeepenLeft
MouthCornerPullRight MouthCornerPullLeft
MouthCornerSlantRight MouthCornerSlantLeft
MouthDimpleRight MouthDimpleLeft
MouthFrownRight MouthFrownLeft
MouthStretchRight MouthStretchLeft
MouthRaiserUpper MouthRaiserLower
MouthPressRight MouthPressLeft
MouthTightenerRight MouthTightenerLeft
TongueOut TongueRoll
TongueTwistRight TongueTwistLeft

OSCMouthTrackingModule._specialParameterEntries splits these single floats into pairs of unified expression slots based on sign:

ParameterPositive value writesNegative value writes
CheekPuffSuckRightpuff rightsuck right
CheekPuffSuckLeftpuff leftsuck left
JawXjaw rightjaw left
JawZjaw forwardjaw back
MouthUpperXupper-rightupper-left
MouthLowerXlower-rightlower-left
TongueXtongue righttongue left
TongueYtongue uptongue down
TongueArchYarch uparch down
TongueShapeshape positiveshape negative

The animator parameter still receives the raw signed float; the splitting only happens on the face-tracking side.

Both modules track Time.time - _lastOscTime and set _timeout = true after 5 s with no recognised incoming parameter. While timed out, IsDataAvailable() returns false and the eye/mouth data is held without being recomputed.

When an OSC server is bound, OSCAvatarModule.SetupOSCQuery exposes every parameter on the avatar under /avatar/parameters/.... If the active eye module is OSCEyeTrackingModule, the eye specials (v2/EyeLeftX, etc.) and module parameters (v2/EyeSquintRight, etc.) are added — and likewise for the mouth module — even when those names are not declared on the avatar’s animator. This lets a remote OSCQuery client send face-tracking data through the standard /avatar/parameters/ channel without the avatar author having to declare every shape.