Riffusion

What is Riffusion?

Riffusion began in late 2022 as a hackathon project by two former Facebook engineers, Seth Forsgren and Hayk Martiros. Their trick was clever: they fine-tuned Stable Diffusion, an image generation model, on spectrograms, which are visual representations of sound. The model generated pictures of audio that, when converted back, produced surprisingly musical clips, complete with melodies, drums and bass.

The open-source experiment went viral, and the team turned it into a company. The modern Riffusion app keeps the generative core but wraps it in a real music studio: you prompt for a style, get a full song, then use controls to change the vibe, swap instruments, extend sections and remix the result live in the browser.

Riffusion stands out for interactivity. Instead of a one-shot generation, you can steer the music while it plays, which makes it feel more like jamming with a robot bandmate than using a search engine for songs.

Key Features

• Text-to-song generation from style and genre prompts
• Live steering: change the mood and style of music as it plays
• Remix and extend generated songs in the browser
• Simple timeline editor for arranging sections
• Vocal and lyric generation in newer versions
• Works fully in the browser with no install
• Community library of shared prompts and songs

How to get started

Open riffusion.com and you are immediately in the studio. Type a prompt like 'lo-fi hip hop with vinyl crackle' or pick a starter style, hit generate, and a song starts building within seconds. You can keep tweaking while it plays: adjust the prompt, change the vibe slider, or regenerate sections.

When you like what you hear, save the song to your library, where you can come back to edit, extend or export it. The free tier is enough to explore; Pro adds more generations, longer songs and extra features. Sign-in is required to save your work, so make an account first if you plan to keep your creations.

• Create a free account at riffusion.com
• Enter a style prompt or pick a starter preset
• Generate a song and steer the vibe as it plays
• Remix sections and extend the track in the editor
• Save your songs to the library and export the final version

Use cases

• Generating background music for videos, streams and games
• Rapid prototyping of musical ideas and moods
• Creative jamming and improvisation with AI
• Learning about prompt engineering for music
• Making original tracks for social media content

Pricing and licensing

Riffusion uses a freemium model with a free tier for casual creation and a paid Pro plan for heavier use. The exact limits, features and prices change as the product evolves, so check the official website for current details.

Music you create is licensed for your own use, which covers most personal and content-creation scenarios, but read the terms before using generated tracks in large commercial projects. Riffusion's own training model is proprietary today, though its earlier research code remains available on GitHub for the curious.

Pros and cons

Riffusion is one of the most fun AI music tools to use. The live steering makes generation feel creative rather than like pulling a slot machine, and the browser-based studio keeps the barrier to entry at zero. It is great for sparking ideas and making usable background music fast.

The trade-offs are control and consistency. Songs can drift in quality, vocals are not always convincing, and you have less fine-grained production control than in a real DAW. For serious, polished releases you will likely finish the track in another tool.

Alternatives

• Suno: full AI songs with strong vocals and arrangements
• Mubert: endless royalty-free generative streams and API
• MusicGen: open-source text-to-music you can run locally
• Stable Audio: commercial generative music from Stability AI
• Boomy: quick song creation with streaming distribution