feat(hangul_benchmark): Seq2Seq comparison of Hangul Factorizer vs Three-Hot Tokenizer - #902

Open

kahye wants to merge 6 commits into

ReaLLMASIC:masterfrom

kahye:nochar-hybrid-pos-tokenizer

Commits on Aug 11, 2026

Add Ko-HellaSwag evaluation benchmarks, POS multicontext loss weights, and resilience experiment report
kahye
committed
dcf109b
feat(korean_pos): expand dataset, add pos loss weighting, and 4-capability evaluation suite
kahye
committed
842c30f

Commits on Aug 16, 2026

feat(korean_pos): add companion stream 256-byte fallback, full/coarse POS tagsets, and benchmark support
kahye
committed
dec8b07

Commits on Aug 24, 2026

feat(korean_pos): add OOV stress-test prompt suite, benchmark runner, and tokenizer byte-fallback improvements
kahye
committed
7a9944e

Commits on Sep 2, 2026