A napkin sketch is only the beginning of an idea. Or it used to be. Because Adobe's latest AI research project does something pretty incredible: It turns a 2D pencil drawing into a richly outlined and shaded cartoon—one that's big enough to stick on a billboard, if you like.

This is Project Sunshine, Adobe's latest "Sneak." The company's Sneaks are research and development demos that leverage Adobe's AI technology to create experimental new ways to manipulate images and videos. Sneaks aren't polished enough to be ready for products. But if people are excited enough about them, Adobe will often continue development on sneaks and bake them into software like Illustrator or Photoshop. (For instance, the 2017 Project Cloak became Content-Aware Fill for After Effects in 2019.)

[Image: Adobe]

With Sunshine, software can automatically analyze a black-and-white sketch, then transform it into a vector image. (Vector images are digital drawings, capable of being scaled infinitely without getting pixelated.) That step alone is handy, but Sunshine goes further by automatically coloring your image in.

[Image: Adobe]

During a product demo, I watched as software colored in a monster—complete with horns, teeth, and fur. It doesn't generate one possibility, but dozens of options to choose from. And only when you really analyze the work do you realize just how impressive the demo is.

[Image: Adobe]

These monsters aren't just colored in randomly, or to a particular color palette. The machine knows that all teeth should be the same color, and that when coloring in solid objects like the horns with more than one color, those colors need to be similar enough that the horn isn't visually jarring. Adobe used a novel approach to machine learning to build Project Sunshine. Instead of using a typical visual AI, it built the system off a natural language processing AI—the sort of AI a speech assistant would use. That means Project Sunshine can label and organize elements like horns or teeth, almost like they're words. From there, it's paint by numbers.

Once the image is colorized, Project Sunshine also lets you add shadows with a simple dial where you can set the angle of the light source. The effect turns a flat 2D drawing into a richer piece of work, more akin to the depth you get in comic books and animated shows like The Simpsons.

Who might use this? For a comic book creator, it could do a lot to automate the production of scenes. But for graphic designers, Project Sunshine could save hours in the laborious process of redrawing anything in vector with this quicker, automated conversion.

[Image: Adobe]

Project Make It Pop

The second Sneak that Adobe shared is called Project Make It Pop. Like Project Sunshine, it creates a vector cartoon. But instead of starting with a sketch, you can start with photos and video.

[Image: Adobe]

I watched as a photo of a skateboarder transformed into a simple, vector cartoon figure. But it doesn't stop there. Project Make It Pop recognizes the human figure and many objects, allowing you to actually reposition the skateboarder's arm or leg.

[Image: Adobe]

But what if you see another skateboarder, in a cooler pose that you'd like for your own cartoon? You can actually take the new photo, analyze it, and your character can copy the pose perfectly. It's wild to behold, especially because the technology works for more than just photography. You can actually replace someone or something in full motion video with a cartoon avatar. It's basically an animator's dream come true. Imagine creating something along the lines of Who Framed Roger Rabbit, without ever learning a thing about animation.