SSD-Edit: Instruction-Based Image and Video Editing with Mamba-2 Diffusion Backbones

Key Points

• The study aims to introduce SSD-Edit, a novel framework for instruction-based editing of images and videos using Mamba-2 diffusion.
• Developed an instruction-based image and video editing framework called SSD-Edit.
• Introduced Source-Instruction Dual-Stream SSD for handling edits.
• Employed Selective Edit Masking via State Divergence for automating edit localization.
• Implemented Temporal Edit Propagation for maintaining consistency in video edits.
• SSD-Edit effectively integrates instruction-based editing for images and videos.
• Demonstrated improved localization of edits compared to previous methods.
• Achieved consistent video editing outcomes through temporal propagation of edits.