Meta claims that SAM or Segment Anything Model is an AI tool that can be used to segment and identify objects within a picture or video, even if the AI has not been trained to identify said object(s).
Meta, the parent company of Facebook unveiled its latest AI tool, SAM on Wednesday, April 5. The company’s research division said it released the Segment Anything Model (SAM), and its corresponding dataset to convert research into foundation models for cognition and computer understanding.
Imagine ChatGPT but for object recognition. Yes! Now users can type in prompts for the AI which it will then use to detect said objects. Users can also select objects by clicking on them. In a demonstration by Meta, SAM was successfully able to draw ‘boxes’ around multiple cats in response to a written prompt.
SAM has several complex features and uses which make it different from the conventional AI, such as :
- SAM is promptable, which allows it to segment objects of the users choice using boxes or points. SAM can generate masks for faces or objects of choice. It can segment multiple objects too, if provided with multiple prompts. It can also detect and segment objects in complex environments that have reflections and other disturbances
- Trained on a dataset of 11 million images and 1.1 billion masks, SAM becomes the largest segmentation dataset till date. Animals, plants, vehicles, furniture, food and a lot more objects and materials come under the dataset of SAM. It’s ability to generalize Objects allows it to segment objects that it has not come across till that point in time.
- SAM has zero-shot performance, which means that it can segment Objects without additional training or ‘fine tuning’ such as hair, clothes, hand and clothes. It can also segment objects in different imagining types such as infrared images and even depth maps.
Automatic object detection and masking make it a very usable asset for security imaging in the future too. When there is uncertainty in the segmented image, the SAM can produce several valid masks which is a crucial skill for AI to combat segmentation tasks in-real-life.
Other generative AI is also under development by Meta.
SAM is developed by Meta AI research and is available to the public on GitHub. One can try the demo online or download the dataset of 11 million images and 1.1 billion masks (SA-1B).
