Gen-2

Overview

Gen-2 is a multimodal AI system developed by Runway Research that generates novel videos by combining text, images, and video clips as input modalities. It was introduced in February 2023 by Anastasis Germanidis, and is designed to 'realistically and consistently synthesize new videos' — either by applying the composition and style of an image or a text prompt to the structure of a source video (Video to Video), or by using nothing but words to create completely new footage (Text to Video). This paradigm-shifting approach eliminates the need for cameras, lighting, or physical sets, allowing creators to produce compelling moving images directly from their imagination. With support for a wide range of visual styles and narrative contexts, Gen-2 serves as a powerful bridge between conceptual ideas and finished cinematic output.

The system is structured around eight key modes, each addressing a different stage of creative production. Text to Video synthesizes videos in any style using nothing but a text prompt, making it possible to describe a scene and instantly see it animated. Text + Image to Video generates a video by using a driving image as a structural reference while a text prompt guides the artistic direction. Image to Video (Variations Mode) takes a single driving image and produces a video sequence from it, ideal for exploring variations. Stylization transfers the style of any image or prompt onto every frame of an input video, enabling consistent aesthetic transformation. Storyboard converts rough mockups into fully stylized and animated renders, turning early-stage designs into dynamic presentations. Mask applies stylization selectively to specific regions of a video, allowing localized changes without altering the rest of the frame. Render transforms untextured 3D renders into realistic outputs by applying an input image or prompt, bridging the gap between CGI and live-action realism. Finally, Customization lets users fine-tune Gen-2 for even higher fidelity results, adapting the model to proprietary datasets or niche visual languages.

Evaluation results highlight Gen-2's superiority in generative video quality. In user studies, Gen-2 outputs were preferred over existing image-to-image and video-to-video translation methods, with 73.53% preference over Stable Diffusion 1.5 and 88.24% preference over Text2Live. These numbers reflect Gen-2's ability to maintain structural coherence while applying sophisticated stylistic changes. The model is a product of Runway Research, which is dedicated to building the multimodal AI systems that will enable new forms of creativity. Gen-2 represents a pivotal step toward this mission, offering a platform that is both accessible to beginners and deep enough for professional workflows. By supporting customization, it also provides a path for organizations to deploy generative video models tailored to their specific needs. Its availability through the Runway creative platform allows seamless integration with other AI-powered production tools, making it a practical choice for both independent creators and large teams.

Gen-2 is applicable across many industries and creative practices. Filmmakers can use it for pre-visualization and conceptual exploration before committing to physical production. Content creators can rapidly generate social media videos or visual effects without expensive equipment. Designers can turn storyboards or concept art into animated product demonstrations. Researchers can experiment with state-of-the-art video generation, while educators can bring abstract ideas to life in the classroom. With its user-friendly interface and powerful capabilities, Gen-2 stands as a milestone in generative media, making it easier than ever to transform text, images, and video clips into rich, dynamic, and original films.

Core capabilities

Text to Video

Synthesize videos in any style using nothing but a text prompt.

Text + Image to Video

Generate a video by combining a driving image with a text prompt for guided artistic direction.

Image to Video

Create a video sequence from a single driving image, also known as Variations Mode.

Stylization

Transfer the style of any image or text prompt to every frame of an input video.

Storyboard

Convert rough mockups into fully stylized and animated renders.

Mask

Apply stylization selectively to specific regions of a video while leaving the rest unchanged.

Render

Turn untextured 3D renders into realistic outputs using an input image or prompt.

Customization

Fine-tune Gen-2 for higher fidelity results, adapting the model to proprietary datasets or niche visual languages.

Use cases

Pre-visualizing a film scene from a text prompt

Generating a video from a single still image for creative exploration

Transferring the artistic style of an image to an entire video

Converting storyboard mockups into animated rendered sequences

Applying selective stylization to masked regions in a video

Turning untextured 3D renders into photorealistic outputs

Rapidly prototyping social media content with AI-generated video