Thanks for creating these Mathematica packages.
When I run NMF on a GPT-2 embedding matrix, I get warnings from LinearSolve.
I tried the following RegularizationParameters: 0.01, 0.1, 1, 2.
I also tried different PrecisionGoals: 8, 16, 32, 64.
Any suggestions?
gpt2Model = NetModel["GPT2 Transformer Trained on WebText Data"]
embeddings =
NetExtract[gpt2Model, {"embedding", "embeddingtokens", "Weights"}]
{wMat, hMat} =
ResourceFunction["NonNegativeMatrixFactorization"][
Normal[embeddings], 256, "RegularizationParameter" -> 2,
MaxSteps -> 300, PrecisionGoal -> 64];
