Nucleus Sampling

anchor

Findings

• IC-1199GPT-2 next-token distributions contain correctable tail errors from the softmax bottleneck that degrade generation quality under low-entropy sampling, with basis-aware threshold sampling improving MAUVE across all four sizes [compared-to]
• IC-1199GPT-2 next-token distributions contain correctable tail errors from the softmax bottleneck that degrade generation quality under low-entropy sampling, with basis-aware threshold sampling improving MAUVE across all four sizes [context]
• IC-1200GPT-2-XL's untruncated next-token log-probability matrix has rank saturating at its hidden dimensionality of 1600, while truncation sampling produces post-truncation distributions whose estimated rank grows far beyond 1600 [compared-to]
• IC-1231On APPS, released code generation models span pass@1 from 0.20 (GPT-3 175B) to 6.20 (CodeRL), with value-based and policy-based RL methods outperforming supervised baselines [eval]