---
library_name: onnx
pipeline_tag: image-to-image
tags:
  - onnx
  - webgpu
  - liveportrait
  - portrait-animation
  - image-to-image
license: mit
---

# LP-Distilled0.1 ONNX

`LP-Distilled0.1.onnx` is an experimental/proof-of-concept browser-oriented distillation of a
LivePortrait-style portrait animation pipeline. It takes a static portrait crop,
a LivePortrait appearance feature volume, normalized head-pose controls, and
normalized expression controls, then renders a `256x256` RGB animation frame.

This release is intended for interactive portrait-animation research and demos.
It is not a full replacement for the original LivePortrait pipeline and should not be used in production.

## Model Details

| Field | Value |
| --- | --- |
| Model file | `LP-Distilled0.1.onnx` |
| Format | ONNX |
| ONNX opset | 17 |
| ONNX IR version | 8 |
| Size | `74,573,121` bytes, about `71.1 MiB` |
| SHA-256 | `335F98B44E5FAB3577B15C3EC53EDDD39BE228F072D98D7CE11232BCF1C42AFD` |
| Output resolution | `256x256` |
| Teacher | Internal LivePortrait-based teacher pipeline |
| Runtime target | Browser ONNX Runtime Web / WebGPU, with WASM fallback where needed |

The released ONNX graph is a fused two-stage model:

```text
Stage1: source portrait + feature_3d + pose controls -> posed neutral RGB
Stage2: Stage1 RGB + same source feature_3d + expression controls -> final RGB
```

The fused graph routes the Stage1 `rgb` output into Stage2's image input. The
external caller only has to feed the original aligned portrait, the source
`feature_3d`, and the two control vectors.

## Inputs And Outputs

| Name | Shape | Type | Description |
| --- | --- | --- | --- |
| `feature_3d` | `[1, 32, 16, 64, 64]` | `float32` | LivePortrait appearance feature volume extracted once from the aligned source portrait. |
| `pose_control_vector` | `[1, 3]` | `float32` | Normalized `[yaw, pitch, roll]` controls. |
| `input_rgb` | `[1, 3, 256, 256]` | `float32` | Aligned source portrait crop in CHW RGB format. |
| `expression_control_vector` | `[1, 30]` | `float32` | Normalized expression, blink, eye-drift, mood, and viseme controls. |
| `rgb` | `[1, 3, 256, 256]` | `float32` | Rendered output frame in CHW RGB format. |

The model does not include face detection, crop/alignment, or appearance feature
extraction. A browser demo can prepare those inputs with compatible
LivePortrait-derived ONNX source-preparation models:

```text
retinaface_det_static.onnx
face_2dpose_106_static.onnx
appearance_feature_extractor.onnx
```

The fused model does not require `motion_extractor.onnx`.

## Control Contract

All controls are normalized numeric values. They are not image-space landmark
coordinates.

Pose controls:

```text
pose.yaw
pose.pitch
pose.roll
```

Expression controls:

```text
kp1.x
kp1.y
kp1.z
kp2.x
kp2.y
kp2.z
kp3.y
kp5.x
kp5.y
kp5.z
kp6.y
kp6.z
kp11.x
kp11.y
kp13.y
kp14.x
kp14.y
kp15.x
kp15.y
kp16.x
kp16.y
kp17.x
kp17.y
kp17.z
kp19.x
kp19.y
kp19.z
kp20.x
kp20.y
kp20.z
```

The control schema deliberately excludes many LivePortrait keypoint axes.
Runtime code should use the exact schema order above rather than assuming
`keypoint * 3 + axis` layout.

Keypoint expression controls are normalized deltas, not absolute keypoints. The
training convention was:

```text
raw_delta = clamp(raw_delta, -0.08, 0.08)
normalized_delta = raw_delta / 0.08
```

Pose controls are also normalized to `[-1, 1]`. The training convention was:

```text
yaw_norm   = yaw_degrees / 20
pitch_norm = pitch_degrees / 14
roll_norm  = roll_degrees / 14
```

Those ranges are part of the model contract. Sending much larger values can
produce degraded or unstable output.

## Pose Vocabulary

The pose model was trained primarily around a finite vocabulary of Catmull-Rom
head-pose trajectories, plus single-axis paths. This is intentional: the runtime
motion system is expected to follow, stop, reverse, or blend from learned
trajectories rather than freely sampling every possible yaw/pitch/roll
combination.

Inputs outside the trained trajectory vocabulary may still work, but quality can
degrade, especially for extreme rotations or unusual pose/expression
combinations.

## Limitations

- Output is fixed at `256x256`.
- This proof-of-concept model was trained on only a few hundred portrait style 
  images. In adhoc testing it can generalize some, but might not look great, 
  depending on the specific source image.
- Quality depends heavily on source crop/alignment quality.
- The model can blur or distort hair, facial details, accessories, and unusual
  backgrounds.
- Extreme poses can be softer than neutral or moderate poses.
- Controls are semantic approximations learned from a teacher; they are not a
  physically meaningful face rig.
- The model card does not assert compatibility with all upstream licenses. Check
  the relevant LivePortrait and dataset licenses before redistribution or
  commercial use.

## Ethical Considerations

Portrait animation models can be misused to create misleading or non-consensual
media. Applications should clearly disclose generated animation, obtain consent
for likeness use, and avoid representing outputs as authentic footage.

## Example Runtime Flow

```text
1. Load a portrait image.
2. Detect and align the face to a 256x256 RGB crop.
3. Run a compatible LivePortrait appearance feature extractor once:
   input_rgb -> feature_3d
4. For each frame:
   feature_3d
   pose_control_vector
   input_rgb
   expression_control_vector
     -> LP-Distilled0.1.onnx
     -> rgb
5. Draw rgb to a canvas or video frame.
```

For lip-sync, map phonemes or visemes to the 30-wide expression control vector,
then smooth the control curve over time. Directly snapping between discrete
viseme vectors will look jerky.

## Release Notes

This is an early research release. The current graph is useful for interactive
demos, but quality is still evolving. Future releases may change the control
schema, model capacity, training data mix, or source-prep assumptions.