Question about tokeniser codebook lengths #60
Description
Hi! Is there a table of tokeniser codebook lengths per modality? A clean table would help others who want to build off of the pre-trained AION tokenisers
{{ message }}
Hi! Is there a table of tokeniser codebook lengths per modality? A clean table would help others who want to build off of the pre-trained AION tokenisers