MIT Creates Image Editing AI That Makes Compositing & Background Swops
MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) has created an AI-assisted image editing tool that sounds like a godsend for designers.
When it comes to compositing and background replacements, creating a realistic-looking final product becomes even more challenging when intricate details such as fine human hairs are involved.
Even though existing image editing softwares such as Photoshopalready has the ‘Magnetic Lasso’ and ‘Magic Lasso’ tools, these isn’t entirely foolproof and still rely on user input for context—that is, manually selecting or tracing the object and catching fine details.
“The tricky thing about these images is that not every pixel solely belongs to one object,” explains Yagiz Aksoy—a visiting researcher at MIT’s CSAIL—inside a press release. “In many cases it can be hard to determine which pixels are part of the background and which are part of a specific person.”
With this ‘Semantic Soft Segmentation’ (SSS) system, MIT CSAIL automates object selection through machine learning, in aims of streamlining the compositing and background substitution processes, so it becomes accessible even for casual users.
The graphic is broken down by the AI into a set of distinct layers that are separated by a collection of “soft transitions” between layers. “Once these soft segments are computed, the user doesn’t have to manually change transitions or make individual modifications to the appearance of a specific layer of an image,” explained Aksoy.
“The vision is to get to a point where it just takes a single click for editors to combine images to create these full-blown, realistic fantasy worlds.”
While the ‘SSS’ system is currently being developed for static graphics, the team is optimistic for video and film applications in future, which could greatly ease editing burdens when it comes to making realistic-looking CGI that’s become increasingly common in today’s movies.
Check out MIT CSAIL’s video about the image editing AI below.
[via MIT CSAIL, video via MITCSAIL]