Saturday, April 27, 2013

Lip Sync with Blender and Papagayo


Papagayo is an open source and freely redistributable lip sync tool by Lost Marble. Lip syncing is the most boring thing you can do by hand when animating... I have created python script to take .dat files exported from Papagayo and import them as position keyframes on a control object which drives (using drivers ;) the morph targets of a face animation rig.

The linked zip file has the script, a slightly modified version of Papagayo (with my face rig images instead of the stock distro version) as well as a blend file with my full (simple and low poly) facial rig. See the included readme.txt for a step by step how to use this script.

The version here is slightly modified only by the addition of a new set of default 'face shape' images to match my Blender face rig.

The included Blend file: Face Rig - rev004.blend has the facial animation rig I use on all my characters, applied here to a slightly modified Suzanne monkey head. The face rig is obviously low poly and simple (I like to avoid the 'uncanny valley' effect) however this rig and script could be adapted to a more complex face rig if you were so inclined.

  • Step 1: Install the Papagayo application 
  • Step 2: Learn how to use Papagayo to sync up text to a sound .wav file Export a .dat file from Papagayo Step 3: Copy the script to blender's add-ons directory 
  • Step 4: Activate the script in the Addons tab of the user preferences window. You'll see a little control box appear in the 3D view tool shelf ("T" to bring this up) 
  • Step 5: Load the Face Rig .blend file Select the "Plus" control called "face.mouth.ctrl" 
  • Step 6: Pick a .dat file (saved from papagayo above) and click on "Process Input File"
  • Step 7: A bunch of keyframes should appear in the timeline. Scrub back and forth to see the keyframes in action You can load in your wav file into blender at this point to hear the audio synced to the mouth (but this isn't necessary for the script to function)