SAM Audio

SAM-Audio is a foundation model for isolating any sound in audio using text, visual, or temporal prompts. It can separate specific sounds from complex audio mixtures based on natural language descriptions, visual cues from video, or time spans.

SAM-Audio supports three types of prompting: text, visual, and span. Each method allows you to specify which sounds to isolate in different ways.

1. Text Prompting: Use natural language descriptions to isolate sounds.
2. Visual Prompting: Isolate sounds associated with specific visual objects in a video using masked video frames.
3. Span Prompting (Temporal Anchors): Specify time ranges where the target sound occurs or doesn't occur. This provides a specific example to the model of what to isolate.