For the past few months, there have been plenty of rumors and reports about Apple’s plans to release AI-enabled wearables. Currently, it looks like Apple’s direct competitors to the Meta Ray-Bans will be launched around 2027, alongside AirPods with cameras, which will offer their own set of AI-enabled features.While it might be too early to know what exactly they will look like, Apple has just offered a peek at how their AI might work.In 2023, Apple’s Machine Learning Research team released MLX, its own open ML framework specifically designed for Apple Silicon.
In a nutshell, MLX offers a lightweight way to train and run models locally on Apple devices, while remaining familiar to developers who are used to frameworks and languages more traditionally associated with AI development.Apple’s new visual model is FAST Now, Apple has released FastVLM: a Visual Language Model (VLM) that leverages MLX to offer near-instant high-resolution image processing, while demanding significantly less compute than similar models.As Apple puts it: At the core of FastVLM is an encoder called FastViTHD.
This encoder was “specifically designed for efficient VLM performance on high-resolution images”.It is up to 3.2 times faster and 3.6 times smaller than similar models.That’s a big deal if you want your device to process information locally, without relying on the cloud to generate a response about what the user just asked (or is looking at).
On top of that, FastVLM was designed to output fewer tokens, which is also key during inference, the step when the model interprets the data and generates a response.According to Apple, its model has an 85 times faster time-to-first-token than similar models, which is the time it takes for the user to send in the first prompt and get the first token of the answer back.Fewer tokens on a faster and lighter model means speedier processing.
FastVLM is available on GitHub, while the report can be found on arXiv.It is not an easy read, but it is definitely worth checking out if you’re interested in the more technical aspects of Apple’s AI projects. You’re reading 9to5Mac — experts who break news about Apple and its surrounding ecosystem, day after day.
Be sure to check out our homepage for all the latest news, and follow 9to5Mac on Twitter, Facebook, and LinkedIn to stay in the loop.Don’t know where to start? Check out our exclusive stories, reviews, how-tos, and subscribe to our YouTube channel
