Skip to content

Conversation

@zacharydenton
Copy link
Contributor

@zacharydenton zacharydenton commented Dec 18, 2025

This is a port of https://github.com/timesler/facenet-pytorch/blob/master/models/inception_resnet_v1.py.

I'm using this to generate facenet512 embeddings for face recognition:

{:ok, facenet_model_info} =
  Bumblebee.load_model({:hf, "zacharydenton/facenet-vggface2"}, module: Bumblebee.Vision.InceptionResnetV1))

{:ok, facenet_featurizer} =
  Bumblebee.load_featurizer({:hf, "zacharydenton/facenet-vggface2"})

facenet_serving =
  Bumblebee.Vision.image_embedding(facenet_model_info, facenet_featurizer,
    embedding_processor: :l2_norm,
    defn_options: [compiler: EXLA],
  )

# use face detection (e.g. yunet from evision) to get x/y/w/h
face_img = extract_face!(img, x, y, w, h)
{:ok, facenet512} = Nx.Serving.batched_run(FacenetServing, face_img)

I've tested this implementation and compared to the embeddings generated by the facenet-pytorch library. The embeddings match the pytorch implementation with 0.9995 average cosine similarity (with latest Axon - see elixir-nx/axon#622 for details).

n.b. I want to add unit tests but I didn't find any suitable weights in https://huggingface.co/hf-internal-testing - open to advice!


%{
pooled_state: pooled_state,
hidden_states: Axon.container({input, pooled_state})
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what to do with hidden_state and hidden_states

@zacharydenton
Copy link
Contributor Author

This model isn't part of the transformers library so there's no valid architecture for config.json on huggingface. Thus I'm not sure if it makes sense to include this in Bumblebee or if it should be in a separate library?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant