gaforum

Field guide · Livestreaming

How to Set Up a VTuber Avatar for Streaming

A VTuber avatar setup has four parts: a model file, a tracking source that reads your face, a program that animates the model from that data, and a capture step that places the result in your streaming software with a transparent background. Live2D models go through a program such as VTube Studio, while 3D VRM models go through a program such as VSeeFace.

This guide walks the chain in order and flags the places where setups usually break.

Key takeaways

  • Pick the model type first: a Live2D (2D) model and a VRM (3D) model need different animation programs.
  • Webcam tracking is enough to start; an iPhone sending blendshape data over the local network improves detail and offloads work from the PC.
  • In OBS, a Game Capture with Allow transparency removes the need for chroma keying with VSeeFace.
  • Face tracking competes with your game for GPU and CPU time, so cap frame rates before you blame the avatar program.
  • Test the full scene with the game running, not with the avatar alone.

Start with the model format, not the software

Every later decision follows from what kind of model you own or plan to commission. A 2D model built in Live2D Cubism is a layered illustration that bends and shifts; a 3D model is a mesh with a skeleton and facial blendshapes.

They are not interchangeable, and the animation program you choose has to read the format you have.

VSeeFace states plainly that it handles VRM and VSFAvatar models and that Live2D models are not supported; it points Live2D users to VTube Studio instead. It also supports only the VRM0 standard rather than VRM 1.0, so when you export from VRoid Studio or another editor, select the VRM0 export format.

A model exported in the wrong version is the most common reason a freshly commissioned avatar will not load.

Ask the artist or the marketplace listing three questions before you pay: which format is delivered, whether facial expressions are included as blendshapes or Live2D parameters, and what the license says about streaming and commercial use. Our ranking of VTuber software compares the main programs, including Warudo, VNyan, VMagicMirror, Animaze, 3Tene, Veadotube mini and KalidoFace, so you can see which ones accept your format before you commit.

Choose a tracking source you can live with

Tracking is the part that reads your expressions and head movement. VSeeFace describes its tracking as done through a regular webcam, covering eye gaze, blink, eyebrow and mouth, with optional hand tracking that needs a Leap Motion device.

That is a workable starting point, and it costs nothing extra.

Lighting matters more than camera resolution. VSeeFace's FAQ notes that insufficient light makes webcams quietly lower their frame rate, giving one example camera that delivers only 15 fps when set to 30 unless the room is bright.

Light your face from the front, and if you wear glasses, move the lamp so it does not reflect into the lens.

Do not shop for a 60 fps webcam on the assumption that smoother tracking will follow. By default VSeeFace caps camera tracking at 30 fps, and its documentation says that raising it to 60 makes only a very small visual difference while doubling the CPU use of the tracking process.

Use an iPhone as the face tracker

If you have an iPhone or iPad, you can send its tracking to the PC. VTube Studio's wiki says the iOS version of the app can send blendshape tracking data to VSeeFace over the local network, that this function is free in the iOS version, and that you do not need VTube Studio on the PC for it.

The wiki lists the steps below.

Two practical notes from the same page: USB is not supported for this link, and the iPhone and PC must be on the same local network. VTube Studio also warns at startup about potential lag on a 2.4 GHz WiFi connection, so use a 5 GHz network where your router offers one.

  1. Set up your 3D model in VSeeFace as normal; no special setup is needed on that side.
  2. On the iPhone, open VTube Studio and tap the camera preview at least once after installing, otherwise tracking is not initialized.
  3. Open the app settings, scroll to the VSeeFace section at the bottom of the first tab, and turn on Activate.
  4. Press Show IP list to find the IP address the PC should connect to.
  5. In VSeeFace, open General settings, choose VTube Studio in the tracking app dropdown, enter the iPhone's IPv4 address and port 21412.
  6. Tick the options for receiving facial features, head movement and eye gaze, then adjust sensitivity sliders.

If the two devices will not connect, the wiki points to a firewall or security program blocking VSeeFace, the devices sitting on different WiFi networks, or the PC network profile not being set to Private.

Set up a Live2D model in VTube Studio

For 2D models, VTube Studio reads a model folder from its data directory. According to its Getting Started page, the folder should at minimum contain the model3.json index file, the .moc3 data file, the physics3.json file and the texture folder.

It also suggests a 512 by 512 pixel icon so the model is easy to find in the selector.

When you load a model for the first time, the app offers an Auto-Setup that assumes default Live2D parameter names and value ranges. If the artist used custom parameter IDs, Auto-Setup will miss them and you will configure the mapping by hand in Model Settings, or copy the setup from another model.

Calibrate with a neutral face at a normal distance from the camera. On iOS the calibration only resets face direction, while on Android and webcam it is more sensitive to how you sit.

Once the model mirrors you convincingly, close the camera preview on a phone; the wiki says there is no reason to keep it open.

Bring the avatar into OBS with a clean background

For VSeeFace, the documented method avoids a green screen entirely. Its FAQ says the program does not support chroma keying, and instead tells you to capture it in OBS with a Game Capture source and enable Allow transparency.

Then press the small button in the lower right corner of the VSeeFace window to hide its interface; the background turns transparent in OBS.

If edges look rough, the FAQ suggests right-clicking the capture source, choosing Scale Filtering and then Bilinear. If a custom shader makes parts of the model look translucent, it advises setting the alpha blending operation to Max in the shader's blending options.

A virtual camera is the fallback. VSeeFace can expose its output as a webcam, at a fixed resolution of 1280 by 720, which is useful for Discord calls or for laptops where game capture is slow.

The page notes that the virtual camera is disabled again when the program closes, for performance reasons, so enable it each session. Our ranking of streaming software for gamers covers the capture programs that can host the avatar scene.

Protect your game frame rate

The avatar is a second real-time application running beside your game, and both ask for the GPU at once. VSeeFace's FAQ describes the typical symptom: the model freezes or lags when its window is in the background and a game is running, usually because the system gives all GPU or CPU time to the game.

Its documented remedies are specific, and you can apply them one at a time. Keep the list short and change one thing per test so you know what worked.

  • Update the graphics driver and VSeeFace, and make sure hardware-accelerated GPU scheduling is enabled (one reported case found the opposite worked, so test both).
  • Run VSeeFace as administrator, and OBS too if you use Game Capture rather than Spout2.
  • Cap VSeeFace at 60 fps and cap the game at a similar rate; the page advises matching frame rates, and matching refresh rates across multiple screens.
  • Try the Spout2 capture option instead of Game Capture.
  • Turn off Steam overlay for the game, and switch the game to borderless windowed mode.
  • Disable any game booster feature in antivirus or graphics driver software.

If the PC is simply too busy, the documentation describes running the face tracker on a second PC over the network and keeping VSeeFace on the capturing machine, which moves the tracking load off the gaming PC.

Add expressions and hotkeys after the basics work

Resist the urge to configure fifty expressions on day one. First confirm that blink, mouth and head turn behave over a thirty-minute session.

Then add a small set of triggers.

VSeeFace offers two expression detection modes. The simple mode handles Fun, Angry and Surprised; the experimental mode adds Joy and Sorrow but needs calibration, where the program reports an accuracy between 0 and 1.

The documentation suggests aiming for roughly 0.95 to 0.98, and says that a value of exactly 1 is not a good sign, because it usually indicates that you need to record more data. Setting up only a few expressions usually improves detection accuracy, but always include Neutral.

VTube Studio has its own item scenes and hotkeys that toggle props, which suit stream moments such as a hat change or a reaction overlay. For outfit changes in VSeeFace, its FAQ advises putting all outfits on one model and toggling them rather than loading separate model files, since loading a model is too slow to use as a hotkey.

A pre-stream test that catches most problems

Run this once per session type, such as solo gaming or a collab call, and again after any driver or program update. VSeeFace's own notes recommend keeping the previous version of the program around when updating, since it is beta software and releases can change behavior.

  1. Start the game, the avatar program and OBS in the same order you will use live.
  2. Record two minutes with the game in a busy scene and check the avatar for freezes in the recording.
  3. Watch the OBS stats panel for dropped or skipped frames while the model is moving.
  4. Hide the avatar program's interface and confirm the background is transparent, including any warning bars near the edge of the camera view.
  5. Check the model's mouth against a few spoken sentences, and your eye gaze against the game screen.
  6. Look at the file on a phone, since small screens reveal tracking jitter that a large monitor hides.

Where this setup connects to the rest of your stream

An avatar is one source in a larger scene. Alerts, chat and clips sit beside it, and each browser-based overlay adds load.

Our guides on setting up stream overlays and alerts and recording gameplay without losing frame rate cover those neighboring decisions, and the ranking of voice changer and streaming audio software is the place to look if you want the voice to match the character.

Frequently asked questions

Can I use a Live2D model in VSeeFace?

No. VSeeFace's website says Live2D models are not supported and points to VTube Studio or PrprLive for those. VSeeFace works with VRM and VSFAvatar 3D models, and only the VRM0 standard rather than VRM 1.0, so choose the VRM0 export format in your model editor.

Do I need a green screen for a VTuber avatar?

Not with VSeeFace. Its documentation recommends an OBS Game Capture with Allow transparency enabled and the program's interface hidden, which gives a transparent background without chroma keying. Chroma key through the virtual camera is possible but the page says it may give less clean results.

Is an iPhone required for good tracking?

No. A regular webcam covers eye gaze, blink, eyebrows and mouth. An iPhone adds detailed blendshape data, and the iOS version of VTube Studio can send it to VSeeFace over your local network at no extra cost, which also reduces the tracking work done on the PC.

Why does my avatar freeze while I play a game?

The game is probably taking nearly all GPU or CPU time. Capping the game and VSeeFace frame rates, running the avatar program as administrator, turning off game boosters and trying Spout2 capture are the steps the VSeeFace FAQ lists. Moving face tracking to a second PC is the larger fix.

How many expressions should I set up first?

Start with a few. VSeeFace's documentation says setting up only some expressions usually improves detection accuracy, and Neutral should always be one of them. Add more once the basic tracking holds up over a full session.

Sources