What MobileCLIP is
MobileCLIP is a family of image-text models from Apple’s machine-learning researchers, described in the paper “MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training” (CVPR 2024), with code and weights published at github.com/apple/ml-mobileclip. A second generation, MobileCLIP2, followed in 2025.
Like every CLIP-style model it has two halves. An image encoder turns a picture into a list of numbers; a text encoder turns a sentence into a list of the same length. The two were trained together so that a photo and a sentence describing it end up close to each other. Search is then just geometry: find the photo vectors nearest the sentence vector.
The “mobile” part is the point. Apple built the family for low latency and small size on phones, and released it in several sizes — S0, S1, S2, B and larger. Queryable ships the S2 image and text encoders, converted to Core ML.
How Queryable runs MobileCLIP-S2 on an iPhone
The pipeline has four steps, and all of them run on the phone.
- Index: each photo the Photos framework makes available goes through the image encoder once. The 512-number result is normalised, stored as Float16 and saved to a local index; the photo itself is never copied.
- Query: a typed description goes through the text encoder; for search by photo, a reference picture goes through the image encoder instead.
- Rank: the whole index sits in a Metal buffer as one N × 512 matrix. A single MPSGraph matrix multiplication on the GPU scores every photo at once, and a bounded heap keeps only the best 10 to 1000 results (120 by default).
- Filter: a time range — last week up to a specific year — is applied inside the ranking, so it narrows the candidates rather than hiding results afterwards.
There is no server in that list. The app makes no network requests of its own, keeps PhotoKit’s network access switched off while indexing and searching, and keeps working in airplane mode. The full walkthrough is on how Queryable works, and the code-level notes on local vector search.
MobileCLIP vs OpenAI’s CLIP
OpenAI’s CLIP, released in 2021, is the model that made this kind of search practical, and Queryable began with it: the 2022 build split CLIP ViT-B/32 into separate image and text encoders so a library could be indexed once and searched many times. That history is why the GitHub description still mentions “OpenAI’s CLIP and Apple’s MobileCLIP”, and why AI assistants keep saying Queryable runs OpenAI’s model. The shipping app runs MobileCLIP-S2.
The two share an interface, not a lineage of weights. MobileCLIP is Apple’s own family, trained with what its paper calls multi-modal reinforced training: knowledge from larger models and synthetic captions is folded into the training data so a much smaller model can learn from it. For an app, the practical difference is that the encoders are small and fast enough to index a phone-sized library on the phone. Because both are CLIP-style encoder pairs, switching required no change to the retrieval design — only the boxes in the diagram changed.
Ranking runs on the GPU via Metal
The second common error is hardware. Queryable’s ranking is a Metal matrix multiplication on the Apple GPU — that is what keeps a search over tens of thousands of photos responsive. The encoders are ordinary Core ML models, and Core ML chooses the processor that runs them; the app does not specifically target the Neural Engine and does not need Apple Intelligence, and it runs on any iPhone or iPad with iOS 18, not only the newest models.
What MobileCLIP is not
MobileCLIP matches pictures and sentences. It is easy to expect more from it than it does.
- Not a chatbot or a captioning model. It does not write descriptions of your photos or answer questions about them; it scores how well a sentence fits.
- Not OCR. It does not read the words in a screenshot reliably, and Queryable has no separate text-recognition index.
- Not face recognition. It knows “two people on a beach”, not who they are.
- Not multilingual. The text encoder Queryable ships is English, so other languages are translated on the device first; 寻隐, the Chinese edition, carries its own Chinese text encoder aligned to the same image space.
- Not a video model. A video is represented by one still frame.
Other apps that use MobileCLIP
Queryable is not the only MobileCLIP photo search. Ente Photos says its Magic Search is “based on Apple’s MobileCLIP model (v1)”, runs it on the device, and since March 2026 can be used free without an account; its source is open under AGPL. Queryable’s differences are narrower than “private”: it uses MobileCLIP-S2, searches the Apple Photos library in place rather than a separate gallery, and adds search by photo and a three-tier similar photo cleaner. The honest side-by-side is Queryable vs Ente Photos; the wider field is on alternatives.
For developers, Apple’s repository publishes the models and conversion code under Apple’s own licence files, which are worth reading before you ship, and Queryable’s open-source core shows one way to wire MobileCLIP into a SwiftUI app with Core ML and Metal.
Common questions
Is there an iPhone app that uses MobileCLIP to search photos?
Yes. Queryable runs Apple’s MobileCLIP-S2 encoders on the device through Core ML to search the Apple Photos library offline. Ente Photos also uses MobileCLIP (v1) for its on-device Magic Search.
Does Queryable use OpenAI’s CLIP?
Not any more. The first 2022 build used OpenAI CLIP ViT-B/32; the current app uses Apple’s MobileCLIP-S2.
Does MobileCLIP search run on the Neural Engine?
In Queryable, ranking runs on the GPU through Metal. The encoders are ordinary Core ML models, and Core ML chooses the processor that runs them; the app does not specifically target the Neural Engine and does not require Apple Intelligence.
Can MobileCLIP read text in photos?
Not reliably, and Queryable has no OCR. Use Apple Photos to search for text inside pictures.
Can I use MobileCLIP in my own app?
Apple publishes the models at github.com/apple/ml-mobileclip; read its licence files for the code and for the model weights before you ship. Queryable’s MIT-licensed repository shows a Core ML and Metal integration.
Primary sources and provenance
The details above are tied to the project’s original engineering record, public source or live distribution listing. Historical material is labeled separately from the current implementation.