Facebook removed envelopes from a project called Learn from videos. It uses artificial intelligence to understand and learn the audio, textual and visual representations in the videos of public users on the social network.
Learning from Videos has a number of purposes, such as improving Facebook’s AI systems related to content recommendations and policy enforcement. The project is in its early stages, but it is already bearing fruit. Facebook says it has already harnessed the technology to improve Instagram Reels’ recommendations, such as showing videos of people doing the same dance to the same music. The system also shows better results when it comes to speech recognition errors, which could reinforce automatic captioning features and make it easier to detect hate speech in videos.
Facebook says the project will help AI researchers avoid having to rely on labeled data and is part of efforts to create systems that learn the same way humans do. As such, Learning from Videos “will allow entirely new experiences”. The company didn’t give many details on these, except for a possible feature that would allow AI to find digital memories, including those captured by augmented reality glasses. You could, for example, ask such a system to show you “every time we sang to grandma,” and it could surface those clips. Facebook, in particular, is .
The company says the project is watching videos in hundreds of languages and from almost every country. This aspect of the project will make AI systems more precise and allow them to “adapt to our rapidly changing world and recognize nuances and visual cues across different cultures and regions”.
Facebook says it keeps privacy in mind when it comes to learning from videos. “We build and maintain a strong privacy foundation that uses automated solutions to enforce privacy at scale,” writes in a blog post. “By integrating this work at the infrastructure level, we can consistently enforce privacy requirements across all of our systems and support efforts such as AI. This includes the implementation of technical guarantees throughout the data lifecycle.
Agreement what happens in the videos can be an extremely difficult task for AI systems. They can include obstacles such as background noise that make it difficult to understand speech and language switching. Yet, less than a year after launching the Learning from Videos project, Facebook is taking what the system has learned and putting it into practice in other areas.