We find that conditional training of large models (LMs), which learns the distribution over tokens based on human preference scores, reduces undesirable content while maintaining downstream task performance. Pre-training LMs with human feedback leads to better preference satisfaction than traditional LM pre-training followed by feedback-based finetuning.
February 15, 2023
Date Range
