Multimodal embedding

A numeric representation designed so related content from different media—such as text and images—can be compared in one vector space.

Queryable uses separate local encoders for photos and text. Their outputs share the same 512-dimensional space, allowing a sentence vector to be ranked against image vectors.

Queryable architecture

Photo embedding

A compact vector that represents visual features of a photo for similarity matching.

An embedding is not the original image, a thumbnail or a complete written caption. It preserves relationships useful to a model: visually or conceptually related items tend to be nearer each other in the embedding space.

How the local vector index works

Joint embedding space

The coordinate space in which image and text embeddings from a multimodal model are placed and compared.

The word “joint” matters because vectors from both encoders must be aligned. A random image vector and an unrelated text-vector system cannot be meaningfully compared just because their arrays have the same length.

Cosine similarity

A measure of how closely two vectors point in the same direction, commonly used to rank embedding matches.

When vectors are normalized, their dot product gives the cosine-similarity score. In photo retrieval it is a relative ranking signal, not a probability that a caption is factually true and not proof that two files are duplicates.

GPU similarity ranking

Top-k retrieval

Returning the k highest-ranked candidates from a larger set of similarity scores.

If a library contains 40,000 indexed photos and the interface needs 200 results, the ranking path can retain the best 200 rather than sorting and displaying every candidate. The value of k is a product and performance choice, not a confidence threshold.

On-device AI

Machine-learning inference that runs on the user’s device instead of sending the input to an app-operated cloud model.

For Queryable, the photo encoder, text encoder, embedding index and similarity ranking run on the iPhone. Apple services such as iCloud Photos remain outside that narrow boundary and may still use a network connection according to the user’s system settings.

Offline search and its boundary

Local photo index

On-device data that lets an app retrieve photos without re-encoding the entire library for every query.

Queryable stores photo identifiers and their embeddings locally, then updates that data as the library changes. The index accelerates repeat search; it does not replace the originals in Apple Photos.

Indexing setup guide

Image-to-image search

Using a reference image as the query to rank visually related images in a collection.

This is sometimes called reverse image search, but scope matters. Queryable searches the permitted personal photo library on the device; it does not look for public copies across the web.

Search by photo guide

iCloud-only original

A full-resolution photo or video stored in iCloud when the device currently holds only an optimized local representation or placeholder.

Queryable’s current index request keeps network access disabled. Make the item available locally through Apple Photos first; that Apple transfer is different from uploading the library to Queryable for cloud search.

What works offline

Limited Photos access

An iOS permission mode in which an app can access only the photos the user has selected.

With limited access, items outside the permitted selection cannot be indexed or returned by Queryable. A missing result may therefore be a permission boundary rather than a search-quality failure.

Prepare photo access

Duplicate photo vs. similar photo

A duplicate represents the same underlying content; a similar photo merely shares visual features, composition or subject matter.

Bursts, edits, Live Photos and repeated scenes make the boundary nuanced. Similarity should create a review queue, not an automatic deletion verdict.

Safer duplicate cleanup