Behavior of same batch negative sampling ?

[Kublai-Jing]

From

PyTorch-BigGraph/torchbiggraph/model.py

Line 548 in a11ff0e

neg_embs = pos_embs

It seems that the same batch negatives are just using the positive embedding as negative directly. However, my understanding is that it should at least shuffle them to avoid the case where positive edge has exactly the same edge as its negative, which defeats the purpose of learning.

Am I missing something ? Thanks!

And from the code (

PyTorch-BigGraph/torchbiggraph/model.py

Line 557 in a11ff0e

neg_embs = torch.cat(

) it seems that if we set both num_batch_neg and num_uniform_neg to be > 0 it will perform both same batch negative and uniform negative and concatenate the two as the final negative embeddings ? Is this correct ?

[jayadev]

+1 to that question, was going over the code and was a bit confused myself on the exact usage of num_batch_neg and num_uniform_neg (read the documentation not sure it is consistent with the code currently). Does num_batch_neg pick the negatives from each "chunk" or "batch". And does uniform pick from negatives from the entire partition ?

Is the typical recommendation to use num_batch_neg=50, batch=2000 for large graphs, given that uniform sampling can be subject to popularity bias on the graph ?

Thanks