|On this page|
The Voice Sync CHOP detects phonemes in an audio channel given some audio phoneme samples and produces lip control channels.
In order to set up a lip-synching system based on phoneme detection, you need to model different mouth shapes for each viseme (visually similar phonemes) and create a “phoneme library” of sample phonemes.
steps are complete, the Voice Sync CHOP matches the phoneme library to
the audio voice track in input 1. The Voice Sync CHOP outputs a channel
for each phoneme (or viseme). Each channel contains the occurrences of
the phoneme in the voice track (as
1 when detected,
over its full length.
Allows you to trade off between performance and accuracy.
Voice Detection will only be performed on the input audio if its average volume for a given segment is higher than this value. It should be adjusted so that background noise is not interpreted as voice.
Silence Chan Name
Creates a channel which acts as a 'silence phoneme'.
Vocal Range (Hz)
The approximate vocal range of the voice actor. The defaults are for an average male speaker. This parameter is only used by Voiced Phonemes.
Max Pitch Shift
The maximum number of octaves that a phoneme in the voice track can differ in pitch from the sample phoneme.
Noise Filter (Hz)
Controls the size of the noise filter to smooth out non-voiced phonemes.
Setup Phoneme Library
This button build the phoneme library from the second and third inputs. It is only used in time slice mode. Before starting the realtime synching, initialize the phoneme library by clicking this button.
Convert Scores into On/Off States
Produces binary on/off state channels for each viseme into On/Off States rather than outputing the raw scores.
The viseme with the highest score is chosen to be the match. However, if its score is not above the minimum score, then no visemes will be chosen. This parameter helps to eliminate poor matches.
Voiced and Non-Voiced phonemes are detected using the different methods. If you find that either the Voiced or Non-Voiced phonemes are dominating the output, you can shift the bias to balance them. Zero bias doesn’t favor either method, +1 completely favors Voiced phonemes and -1 completely favors Non-Voiced phonemes. Normally values should remain close to zero.
The sample rate of the output channels. This allow partially determines how many matches are done on the input audio.
How many comparisons are done per output sample. A higher super sample will increase the matching accuracy, but will affect performance greatly.
Some of these parameters may not be available on all CHOP nodes.
To determine the channels that are affected, some CHOPs have a scope string. Patterns can be used in Scope, for example
* (match all), and
? (match single character).
The following are examples of possible channel name matching options:
Matches a single channel name.
chan3 tx ty tz
Matches four channel names, separated by spaces.
Matches each channel that starts with
Matches each channel that has
foot in it.
? matches a single character.
t? matches two-character channels starting with t.
Matches number ranges, giving
[xyz]matches three characters, giving channels
Sample Rate Match
The Sample Rate Match options handle cases where multiple input CHOPs’ sample rates are different.
Resample At First Input’s Rate
Use the rate of the first input to resample the others.
Resample At Maximum Rate
Resample to the highest sample rate.
Resample At Minimum Rate
Resample to the lowest sample rate.
Error if Rates Differ
Does not accept conflicting sample rates.
The units of the time parameters.
For example, you can specify the amount of time a lag should last for in seconds (default), frames (at the Houdini FPS), or samples (in the CHOP’s sample rate).
When you change the Units parameter, the existing parameters are not converted to the new units.
Time slicing is a feature that boosts cooking performance and reduces memory usage. Traditionally, CHOPs calculate the channel over its entire frame range. If the channel needs to be evaluated every frame, then cooking the entire range of the channel is unnecessary. It is more efficient to calculate only the fraction of the channel that is needed. This fraction is the Time Slice.
Causes the memory consumed by a CHOP to be released after it is cooked, and the data passed to the next CHOP.
The Export Prefix is prepended to CHOP channel names to determine where to export to.
For example, if the CHOP channel was named
geo1:tx, and the prefix was
/obj, the channel would be exported to
You can leave the Export Prefix blank, but then your CHOP track names need to be absolute paths, such as
Every CHOP has this option. Each CHOP gets a default color assigned to it for display in the graph, but you can override the color with the Graph Color. There are 36 RGB color combinations in the palette.
Graph Color Step
When the graph displays the animation curves, and a CHOP has two or more channels, this defines the difference in color from one channel to the next, giving a rainbow spectrum of colors.