At Adobe MAX, the annual creative event of Adobe, the global software (SW) giant known for Photoshop, the session that draws the most attention after the keynote is Sneaks. Sneaks is where Adobe introduces future technologies it is currently developing. Technologies unveiled here are highly likely to be commercialized later in Adobe's official products.
Adobe showcased 10 new technologies at Sneaks held on the 29th, the second day of MAX, local time. Like the keynote, the technologies announced at this year's Sneaks focused on image, video, and audio editing based on Generative AI. In particular, a new voice technology that naturally changes a speaker's intonation and tone drew praise, and a background generation feature developed with participation from an Adobe summer intern drew attention.
Adobe evangelist Paul Trani said, "Sneaks is where we show the cutting-edge technologies we are developing internally," adding, "Harmonizer, an AI-based compositing correction feature introduced in Photoshop last year, was also unveiled at Sneaks a year earlier." Trani and U.S. comedian Jessica Williams served as co-hosts and livened up the atmosphere.
① Project Motion Map
Motion Map is a technology that advances the prompt-based editing Adobe unveiled at MAX this year. It lets users select fine-grained parts within an image or video and apply changes by entering prompts. For example, when a dialog box appears over the eyes of a smiling-face logo, you can add animation by instructing, "Make the eyes blink."
② Project Clean Take
This is a new voice technology that lets you select and edit desired parts of audio. Researcher Lee Brimlow, who demonstrated the technology, took an audio file of a woman saying, "It has been 5 years since I started working at Adobe," changed "5 years" to "4 years," and cleanly removed a sudden pitch jump—commonly called a squeak—drawing applause and cheers from the audience. The woman's voice was natural enough in pitch, intonation, and timbre to make it hard to tell real from synthetic. Through "voiceover adjustment," it is also possible to change a monotonous tone to a confident one or modify it to a whisper.
③ Project Surface Swap
This technology selects specific elements in an image—such as only the surface of a car in a photo—and changes the color or texture.
④ Project New Depths
A technology that recognizes the environment in a photo in 3D and lets you add or remove new elements to fit the composition. For example, when adding a tree to a bushy environment, AI automatically recognizes that the base of the tree should be hidden in the bushes and places it accordingly.
⑤ Project Light Touch
A technology that lets you freely adjust lighting within a photo. In the demonstration, applying the "ignite" feature to a light that was off turned it on naturally. Users can set the brightness of the light as they wish.
⑥ Project Scene It
Developed with participation from Adobe summer intern Oindrila Saha, this technology automatically creates a background (scene) that suits a specific element. After first choosing elements to place in the image—such as shoes or a vase—and arranging them, you enter the desired background via prompt and it is generated.
⑦ Project Trace Erase
This technology naturally removes elements you want to erase in a photo. If you remove a specific person from a photo, not only does the person disappear, but the person's reflection in a window frame, shadows, and lighting vanish seamlessly as well. In backlit photos, sunlight can also be removed.
⑧ Project Frame Forward
A technology for editing videos with Generative AI that can remove selected people or objects from footage. After capturing the first frame where the video begins and removing specific elements using the image removal feature there, AI analyzes the footage and removes those elements throughout the entire video. Conversely, it is also possible to add desired elements to the video.
⑨ Project Sound Stage
AI analyzes the content of a video and inserts appropriate sound effects. People do not need to find each section and scene one by one to add effects. The AI-generated effects can be adjusted as much as you want via prompts.
⑩ Project Turn Style
A feature that turns specific elements in an image into 3D so they can be rotated from multiple angles. It enables more flexible image correction.