digest
Short Summary
Main recommendation: Be critical of what you see in videos and images, because realistic fake videos of real people are now possible and can cause harm before they are verified. Supasorn Suwajanakorn describes research that builds controllable 3D models of people from existing photos and videos, and uses audio to drive their mouth movements, with a countermeasure tool called Reality Defender in development.
Key Points
- The talk begins with a Holocaust-testimony hologram project (New Dimensions in Testimony) as the inspiration, showing how personal interactive recordings can feel more connecting than books or lectures.
- The core method reconstructs a high-detail 3D face model from any image without a 3D scan, by analyzing a large collection of the person's existing photos (e.g., searched from Google for George W. Bush).
- A new blending technique produces sharp facial textures and colors, so the model can show wrinkles that come and go with different expressions.
- Mouth movements can be synthesized from audio alone: the system was trained on 14 hours of Barack Obama's speeches, and a neural network converts input audio into mouth points.
- The speaker acknowledges misuse risk and says the work is paired with countermeasure research, including a machine-learning and human-moderator effort at the AI Foundation.
Details Worth Noting
- Concrete instructions: The transcript gives no step-by-step instructions for making or spotting fakes. The only actionable guidance is to make people aware of what is currently possible so viewers can hold the right assumptions and be critical about what they see. Reality Defender is described as a web browser plug-in that can flag potentially fake content automatically in the browser, and it is noted as a planned release.
- Data requirement: The speaker says the photo collection can come from typical photos, and that the taken-on context or expression does not matter; what matters is that there are a lot of them.
- Pipeline for audio-driven mouth: audio is converted to mouth points by a neural network, then texture and teeth detail are synthesized and blended into the head and background of a source video.
- Scope of the demonstrated audio approach: only the mouth region is synthesized in that example.
- Speaker's own caveat: the results are realistic and intriguing but also frightening; the stated goal is accuracy, not misrepresentation, and the speaker says the technology is not yet ready to model individuals truly or ensure safety.
Bottom Line
Realistic synthetic video of real people is already feasible from public photos and recordings, so viewers should treat unverified footage with skepticism while detection tools are still being developed.
