Lip Syncing with Fuse/Mixamo Avatars

Programs Needed:

Fuse (available for free if you have a Creative Cloud membership, or on Steam)

3DSMax

Cal3D Exporter (available to download from the WorldViz website)

1. Create your model in Fuse

 

2. Go to “File- Animate with Mixamo” or click on “Send to Mixamo”

 

3. Make sure “Facial Blend Shapes” is enabled and click “Finish”, then “Use this Character”

 

4. Download as an FBX model

 

5. Also, at this time you may want to select and download any animations you will wish to use

6. Open up 3DS Max and import your FBX model you just downloaded

 

7.  Carefully follow the steps for exporting using the Cal3D exporter, outlined in the Vizard documentation http://docs.worldviz.com/vizard/index.htm#cal3d_introduction.htm%3FTocPath%3DContent%20Creation%7C3D%20Models%7CCal3D%20Workflow%7C_____1

8. Create a new folder and name it whatever you want to call your avatar

9. In 3DS Max go to File- Export- Cal3D skeleton file

10. Make sure your units are set to meters in the export window, name your file and save it in the folder you created.

11. Next, select a mesh and click on the “material editor” and change the mode to “compact”

12. Rename each material so that it has sequential ID numbers in square brackets, e.g. body[0]

13. Now export your meshes, following the steps from the Cal3D exporter documentation

14. Next, select your head mesh, go to “Modify” and select “Morpher”

15. Click on where it says “Morpher” and you will see a list of all of your morphs show up. We will be using the one titled “Jaw_Down” for the lip syncing

16. Enter “100” in the value next to “Jaw_Down”

 

17. Now, export your head mesh once more and name this one something like “mSkel_m021_head_talk.CMF”

18. Next, export your materials using File- Export- Cal3D Material File

19. For the animations you will have to import each animation fbx model separately, then export using the Cal3D exporter and make sure to select the same skeleton you have been using. Save all your animations to the same folder as the other avatar files as well.

20. The next step is to create your .cfg file. Open up Vizard and select File- New Text file. Save your file as a .cfg file and give it a name. Make sure to save it in the same folder as all your other files.

21. Follow the steps in the documentation for setting up your .cfg file

22. You can test your Avatar’s morph and animations in Inspector by right clicking on your cfg file, select “Open with” and choose “Vizard 6 Inspector”

23. Once everything is setup, you can now add your Avatar to your scene, set its position, set its state, and then add the speech command by first importing the vizact module and using the code: speech = vizact.speak('jfk.wav') where “jfk.wav” is the name of any wave file you choose. You can now map this action to a keypress using vizact.onkeydown(' ', male.addAction, speech)

24. For a sample scene using a .cfg avatar and lip syncing click here . In the folder, open the FBX_lip_syncing.py in Vizard
If the lip syncing causes the head to distort in an unusual way, you may need to adjust the scale by adding vizact.speak(‘jfk.wav’, scale= [enter some value here])

 

 

How to have tracked FBX Avatars in vizconnect using 3DSMax and Mixamo’s Auto Rigging

The Autobiped script was created by Ofer Zelichover and Dan Babcock and it is freely released to Mixamo customers

The script will convert any character rigged using Mixamo auto-rigger into a Biped system in 3dsMax

  • If going from Fuse just send to Mixamo and Auto-rig

    • Go to mixamo.com

    • Select “Upload Character”

  • If coming out of other programs (such as Character Creator)

    • Bring avatar into 3DSMax and delete the skeleton (for some models may also have to merge meshes)

    • Export the mesh as an fbx

    • Upload to Mixamo and auto-rig then download

  • Import downloaded avatar into 3DSMax

  • Check size of avatar (Under Utilities- Measure)

  • May have to uncheck “Always Deform” on the mesh (except for feet) - Under Modify- Advanced Properties. This is if you get undesirable results on how the skeleton attaches to the mesh.

  • Add and run this script (AutoBiped) to convert to 3DSMax biped mesh object (Scripting- Run script)

  • Click on "Create Biped"

  • Rename all bones from Bip001*  to Bip01* in 3ds Max (Tools > Rename Objects). Expand all bones in the skeleton and select them all first.

  • Export with Cal3D

  • Use as an imported Complete Character in vizconnect

    • Start Vizconnect by going to Tools- Vizconnect in Vizard

    • Setup your trackers/ inputs/ etc. For help with vizconnect see the Vizard documentation. You can also choose from one of the common presets.

    • Click on “Avatars” tab and remove the current avatar if there is one there.

    • Add new avatar

    • Choose “imported Complete Characters”

    • In Settings under file, choose your exported .cfg file

    • Make sure to then go to the “Animator” tab under "Avatars" and map whatever trackers you are using to the avatar model

    • Drag the main camera under the head component in the scene graph, as well as any tools you may be using under the hand components.

    • Lastly, add gestures and map them to your input device

  • Afterwards you will want to manually edit the vizconnect file where the avatar is added to change the lines to use vizfx instead of viz.add :

  • import vizfx

  • avatar = vizfx.addChild(filename)

  • If wanting to change the avatar, you can just change the path in the code.

Moving Unity Assets to Vizard and Vizible with the Sketchfab glTF exporter

This is a method for transferring your Unity assets into Vizible or Vizard using Sketchfab.

Go to the asset store in Unity and search Sketchfab. Import the “Sketchfab for Unity” asset

 

 

Once imported, you need to press “Play” in your Unity scene to have it show up.

You then choose “Sketchfab- Publish to Sketchfab”

 

Login to your Sketchfab account (if you don’t have one, you’ll need to create one), name your scene and click “Upload to sketchfab”

You’ll then need to make your model downloadable in Sketchfab (Under “Manage this model”)

 

Once it’s been made downloadable, download the gltf, unzip it and add it to your Vizible or Vizard scene.

 

Note: for Vizible, you’ll need to convert to a glb or osgb using either this tool to convert to glb, or using Inspector to convert to osgb.

Note: There may be some issues with models that have transparency

 

 

FBX Avatar to Vizard

In earlier versions of Vizard, Cal3D exports out of 3ds Max were the only option for a character workflow. As of version 6, Vizard includes the ability to use any FBX avatar. These can come from a variety of sources. In this case, we'll be using a .FBX avatar exported from a program called “Character Creator”.

Importing and Preview Lighting
First, import your avatar into Inspector. This can be done by using Open With... and selecting Vizard 6 Inspector from the list, or by running Vizard 6 Inspector and dragging the avatar's FBX file into the scene.


Read more

PBR Workflow: Tools, Resources and Additional Reading

Banner Image

An Industry-Wide Revolution

The addition of physically based rendering to WorldViz products brings us into the fold of an industry-wide revolution in artist workflows. Physically based rendering has a few new concepts and methods compared to older methods, as well as some amazing new capabilities that simplify artist workflows an make sure that once an asset has been made once, it can be used anywhere without having to retouch the source files. This is a list of tools and resources to help you on your way.

Krzysztof Narkowicz's Interactive PBR Example

The most intuitive way to get a feel for the capabilities of a shader is often to simply mess around with some sliders and see how it responds. Krzysztof Narkowicz's embedded below from Shadertoy does just that, and even includes a realtime display of what's going on at the microsurface level.


Permalink: https://www.shadertoy.com/view/4sSfzK
Read more

PBR Workflow: Processing HDRs for Image based Lighting With Lys

Article Banner

A Tool for Processing HDR Images for Image Based Lighting

Image based lighting can use 8 bit images, but requires true HDRI cube map to get the best results. Once you have on in hand, you'll need a way to process it.

First you'll need a copy of Lys from Knald Technologies. Freeware tools also exist, but many of them are either buggy or are very outdated. Lys can be tried out for free with LDR watermarked images for free, but the paid version is fairly cheap and the reliability and features make it easily worth the price.
Lys Logo

Where to get HDR Panoramas and Cube Maps

Next you'll need an HDRI panorama or cube map. A great source for free HDRI sky panoramas is HDRI Haven The website is run by CG artist Greg Zaal, who generously puts them out with a CC0 license - free for both personal and commercial use, no catch. Shooting HDRs yourself can be done with any 360 capable camera that supports exposure bracketing, though higher resolutions give better backgrounds.

You can also capture HDRI cubemaps from virtual scenes within Inspector. These currently need to be reorganized into a cross format before use in Lys. This can be done through with AMD's classic editing tool, CubeMapGen. If you're using the modified version of CubeMapGen designed for making quick ambient maps, you'll want to temporarily disable multi-threading first before saving out to cross format.

HDRI Haven - a free source for high quality HDRI images
Read more

PBR Workflow: Saving HDR Cube Maps From Inspector

Article Header Image

Welcome to Inspector R6

If you've been working with WorldViz products for a while, you may already be aware of our visual editor, Inspector. Inspector allows for the inspection and manipulation of scenes, setting up realtime lighting, particle effects, Level of Detail nodes, and various other options.

New to Inspector in Vizard 6 is the ability to render out High Dynamic Range cube maps that can be processed in external programs for Image Based Lighting in a Physically Based Rendering workflow, such as in commercial tools like Lys, or freeware like IBLBaker.

Inspector is included with all installations of Vizard, and is unrestricted with all licenses including the free mode. If you don't already have a copy of Vizard installed and would like to try Inspector, download the free edition of Vizard here.

Start Inspector and Load the File

To start off, open up inspector from the start menu, then load your environment model using File > Open.
Read more

Baking Lighting in 3ds Max: A General Overview, Part 2

Header_Baking_20171128

Baking in Max can be a daunting process. Last time, in Part 1 we talked about what you need to get started, options for renderers, a few of the different kinds of bakes you can do, plus how and why to do test renders.

Part 2

This time, in Part 2, we go into the approach for Unwrapping, the details of Render to Texture, and the actual baking.

Part 1 | Part 2

Initial Bake

Baking_Seq_3_Baking

Unwrap Model
Bakes work by recording the lighting intensity for each point on a model's surface into a texture. This means that, under normal circumstances, you don't want any overlaps or tiling used in the UV channel that you bake to otherwise you may have some very strange results in your baked lighting. While unwrapping by hand produces the best results, automatic unwraps are usually more than enough. Bake UVs are usually unwrapped to channel 3.

There are a few common options for creating lightmap UVs for large numbers of objects. One is to use the automatic unwrap feature of Render to Texture, though this always groups matching material IDs together resulting in a lot of wasted UV space. Another option is to use a scripted tool written in-house called SteamRoller, which does a similar function but isolated from any other dialog and without requiring the material ID option. Paid plugins also exists that can do unwrapping, some of which also have their own special baking options.

UnwrapExample_20171205_short
Read more

Baking Lighting in 3ds Max: A General Overview, Part 1

Header_Baking_20171128

Baking in Max can be a daunting process. There are numerous choices, and it may be unclear what is going to give the ideal results. This guide gives a general overview of the process to take your work from an unbaked model to final baked scene.

Part 1

In this first section, we talk about what you need to get started, options for renderers, a few of the different kinds of bakes you can do, plus how and why to do test renders, and what mesh issues to look out for.

Part 1 | Part 2

Starting Out

Baking_Seq_1_Start

Ready to Go
Before you're ready to bake, you'll ideally have a scene that's realtime ready. This usually means a low-ish polycount (under 2 million), Scanline:Standard materials, and bitmap based textures instead of procedural ones. While it's possible to bake high polycount objects, such as unoptimized architecture scenes, they may take an exceptionally long time to unwrap, and may also have trouble running once exported.

You'll also want to have some familiarity with the OSG Exporter, but if not you can also start off by loading up the sample preset file from the Storing and Loading OSG Settings tutorial.

Baking_house Read more

Quick Previews with the Max OSG Exporter

OSG-Quick-Preview

The OpensceneGraph exporter's toolbar is the best and quickest way to export and preview a Vizard/Visible scene in Inspector when working in 3ds Max. Adding the toolbar, and setting it up, is quick and easy.

If you'd like to quickly set up the rest of your OSG Exporter's settings to match what we use here at WorldViz check out the tutorial on Storing and Loading Your OSG Export Settings, which also includes a downloadable preset file.

PreviewScene_20171205

Let's get started.

Read more

Next Page »