Hello readers, in this issue we cover:

• Researchers advance generalized deepfake video detection

• Ships, including warships, can be designed with AI

• Google DeepMind finds that smaller LLMs, once finetuned, are better synthetic data generators

• New technique combats model collapse with diffusion models and synthetic data

👨🏼‍🦲 Generalizing Deepfake Video Detection

This paper addresses 3 key challenges in deepfake video detection

1. Deepfake detection struggles with diverse changes over time. Finding common temporal patterns can improve model accuracy

2. Models often focus too much on either frame details (spatial) or changes over time (temporal). They need to learn both effectively.

3. Analyzing videos is resource-intensive. The challenge is to maintain efficiency without sacrificing accuracy.

The authors introduce Video-level Blending (VB) to help models detect a new forgery pattern called Facial Feature Drift (FFD). VB blends original and warped video frames to teach the model better.

They also develop a Spatiotemporal Adapter (StA) to help existing models capture both spatial and temporal features efficiently using two-stream 3D convolutions.

Read the research paper

🚢 Novel ship designs, brought to you by AI

Designing the hull shape is the most important part of a new ship and accounts for 70% of the total cost. Existing methods require traditional engineering and architecture approaches. This paper builds novel hull design with a model based of the Gaussian Mixture Model architecture and trained on a dataset of 30,000 hull forms.

Building a complex destroyer ship takes 7-8 years with traditional methods, and gen AI could rapidly create new hull designs to accelerate ship building.

Read the research paper

🐹 Google DeepMind finds that smaller LLMs are better synthetic data generators

A common approach to generating synthetic data is to leverage large, stronger LLMs for generating. Researchers at Google Deepmind challenge that approach and evaluate whether it’s compute optimal. The researchers found that smaller, weaker LLMs are better at generating synthetic data after finetuning. Utilizing smaller models could yield cost benefits.

Read the research paper

🌅 Combating model collapse in diffusion models with synthetic data

Researchers at Rice University and Adobe created a new training technique called SIMS (Self-IMprovement Diffusion Models with Synthetic Data). It provides models that use self-synthesized data to provide negative guidance during the generation process to steer a model’s generative process away from non-ideal synthetic data, which can leads to model collapse.

Read the research paper

🤯 Today I Learned

Every issue, we highlight new AI concepts and terminology to help educate our readers. This issue we learned about:

Facial Feature Drift

Facial Feature Drift (FFD) refers to a subtle but detectable inconsistency in facial features across frames in a deepfake video. This artifact occurs when the deepfake generation process fails to maintain consistency in the appearance, position, or alignment of facial features (like eyes, nose, mouth, etc.) from one frame to the next. As a result, facial elements may appear to "drift" slightly over time, creating unnatural movements or deformations.

Gaussian Mixture Model

A Gaussian Mixture Model (GMM) is a probabilistic model used to represent a distribution of data points by assuming that they are generated from a mixture of several Gaussian (normal) distributions with unknown parameters. GMMs are widely used in statistics, machine learning, and data analysis for clustering, density estimation, and pattern recognition.

Model Autophagy Disorder (MAD)

Model Autophagy Disorder, also known as model collapse, is a concept in machine learning that refers to a phenomenon where a model trained on data containing outputs from other models suffers a degradation in performance, originality, or generalization capabilities.

Temporal Features

Temporal features refer to characteristics or patterns in data that change or evolve over time. In machine learning, data science, and signal processing, temporal features are used to capture the dynamic aspects of data that are dependent on time.