At its annual Adobe MAX conference, the company has revealed a new suite of experimental AI projects that showcase its commitment to pushing the boundaries of creative technology. These “Sneaks” offer a glimpse into the future of content creation, demonstrating capabilities ranging from transforming 2D images into explorable 3D environments to designing AI characters through conversation.
Project Screen Test: Conversational AI Character Creation
One of the most intriguing projects, dubbed Project Screen Test, aims to simplify the creation of AI-generated characters for video content. Unlike traditional methods that rely heavily on text prompts, this tool allows users to interact directly with AI characters in a virtual audition setting. Users begin by defining basic personality traits like ‘anxious,’ ‘shy,’ or ‘clumsy,’ and assigning a voice. The voice can be selected from presets, described via text prompts, or even based on an uploaded audio sample for the AI to reference.
Following this initial setup, users can engage in a conversational dialogue with the AI character, similar to a video call. This interaction allows for further refinement of the character’s persona, including their background, motivations, flaws, fears, and unique mannerisms. Once the character is fleshed out, users can describe the desired environment or scenario, generating a dynamic audition video. While currently a standalone experiment, there’s potential for Project Screen Test to be integrated into Adobe’s existing AI video tools. Its significance lies in creating AI characters with more distinct and coherent personalities compared to many current AI animations, addressing the challenge of imbuing AI-generated figures with the conscious design choices inherent in traditional animation.
Project Sight Span: Image to Interactive 3D Environment
Project Sight Span offers a powerful way to transform flat images into immersive, explorable 3D spaces. This technology can generate objects and backgrounds beyond the original frame, effectively expanding a 2D picture into a full 3D environment. Even when users change their viewpoint to see areas previously hidden behind subjects, the generated content seamlessly integrates with the original image. While browsing in 3D, elements might appear pixelated, but the system restores full resolution once a final perspective is chosen.
Sight Span supports various image types, including photos, paintings, and illustrations, while preserving the original artistic style. Remarkably, it maintains the recognizability of details like tattoos or facial features even when modified in the 3D environment, staying highly consistent with the reference image. This technology can be seen as an enhanced version of Apple’s Spatial Reframing, allowing for more significant adjustments and complete changes in camera angles, though extensively modified results may move beyond the realm of traditional photography.
Project Sound Bender: Intuitive AI Audio Mixing
Addressing audio manipulation, Project Sound Bender is an AI-powered mixing tool that lets users adjust sound effects in videos with intuitive controls, akin to moving a cursor on a color wheel. Users upload a video and use text prompts to request specific sound effects, such as “the sound of a bubbling sci-fi alchemy pot.” Sound Bender generates the appropriate audio and synchronizes it with the video.
A unique feature is the square control panel, where the four corners automatically populate with different AI-generated sound types – for instance, bubbling and popping sounds, roaring rock-boiling noises, pitch-shifting electronic tones, and crackling sparks. Users can drag a cursor around this panel to emphasize different sounds or assign distinct effects to each corner, offering granular control over the audio landscape in real-time.
Project Rinse Repeat: Effortless Editing Replication
Finally, Project Rinse Repeat introduces an intelligent way to replicate editing operations across an image. Users can perform a complex edit on one element, such as retouching skin on a person in a photo, and Rinse Repeat can then automatically apply similar adjustments to other people or objects in the same image, based on the specific modifications made. This capability extends to commercial applications, like adding consistent shadow effects to multiple products, even if their shapes and sizes vary.
Built on Photoshop, Rinse Repeat’s edits are fully editable because the tool leverages native Photoshop functions to recreate the modifications rather than simply overlaying a processed image. This ensures flexibility and maintainability of the edits.
The Future of Creative AI
It’s important to note that these “Sneaks” are currently technical previews, and their eventual integration into Adobe Creative Cloud or the Firefly platform is not guaranteed. However, given Adobe’s strategic focus on generative AI, there’s a high probability that at least some of these experimental features will see a formal release.
Collectively, these projects signal a significant trend: Adobe is actively working to lower the barrier to entry for complex creative workflows in photo editing, video production, and audio mixing, making them more intuitive and interactive. As Gautam Mysore, Senior Principal Scientist at Adobe, explained in an interview with The Verge, the goal is to “make these interactions more intuitive, and also to give experienced creators more precise control.” Tools like Sound Bender and Sight Span allow professionals to use their judgment for more direct refinement of final results, highlighting Adobe’s exploration into better aligning creative intent with actual control.
