Remove any object from any image (Patreon)

A few days ago I came across a new AI model named SAM (segment anything model) a model created by the Meta AI team (Facebook research).

Included on their site is a demo which can be accessed without the need for an account or logging it for free. I gave it a try and was impressed with the results of the current model.

Removing objects from images has been done for many years through numerous ways. There is apps on the AppStore that's whole business models are based off this. Photo editors like Illustrator and Photoshop also provide a way to do this.

I remember looking for something similar when Apple released their dept effect on iOS 16. Was one of the "breakthrough features" that many updated their devices for. If you aren't familiar it takes a wallpaper and extracts a piece of the image out to overlay it on the time/date view. This gives the lock screen more depth but was limited to a very specific type of photo.

Example of iOS depth effect.

Image Credit: idownloadblog https://www.idownloadblog.com/2022/10/16/depth-effect-wallpapers-iphone/ 

This new AI model can use any image and they even show it working in videos. Very cool! So I gave it a try and with every image I sent it I was able to extract what I needed. It's still not perfect, but everything I tried gave me a result I could work with.

Here is one example of how I used it with my tweak LockPlus.

The image I used

The piece I extracted

I put them both together in LockPlus to create this lock screen. Steps: Apply wallpaper to the lockscreen, add time elements, then add an image element that I set above the time elements. 

This was done quickly, but I hope you can see with the right images you could use this to create a LockPlus theme where the time would feel part of the wallpaper. Example if this wall had a cutout or sign slightly behind the person. Adding a time to that area with a color that matches, the depth effect of layering an image on top of it, and a font to match the scene would make this much better. I'm still looking for a better design, but thought I would share this demo with you.

Link to the SAM model, demo, and paper.
https://segment-anything.com/  

Have a good day!