Angelica Chen

We introduce Imitation Learning from Language Feedback (ILF), demonstrate that large language models accurately incorporate natural language feedback and that finetuning with ILF scales well with the dataset size, even outperforming finetuning on human summaries.

We introduce Imitation Learning from Language Feedback (ILF) to improve code generation, demonstrating that a small amount of natural language feedback during training can lead to significant performance gains on program synthesis benchmarks.

We find that conditional training of large models (LMs), which learns the distribution over tokens based on human preference scores, reduces undesirable content while maintaining downstream task performance. Pre-training LMs with human feedback leads to better preference satisfaction than traditional LM pre-training followed by feedback-based finetuning.

We propose a three-step learning algorithm to learn from natural language feedback, which conveys more information per human evaluation than comparisons.