Introducing SelFee, an LLM that Uses Self-Feedback Generation to Improve Itself Over Time

If you’re a language model enthusiast, you’re familiar with the challenges that come with generating high-quality responses. But what if we told you that SelFee, a natural language feedback model developed by KAIST researchers, might be the solution to all your problems?

Diving into their research, the team of experts explains that SelFee is a language model that specializes in self-feedback and self-revision generation, resulting in improved responses compared to previous models. They’ve achieved this by using a fine-tuned LLaMA-based instruction-following model, generating a response, and self-feedback sequence by analyzing the content of the generated feedback.

Their approach doesn’t require external, significant language or task-specific models, making it cost-effective and efficient. To increase the number of instances of feedback and revision, the researchers have augmented their dataset using teacher models called ChatGPT.

With training done using OpenAI API calls, SelFee generates the answer and feedback chain, including revisions. The team observed that increasing the minimum required revisions during the inference process improved answer quality.

While SelFee showed comparable performance to ChatGPT in the evaluation setting, it faired less well in specific areas like math, reasoning, and coding. Nonetheless, SelFee represents a step forward in improving the quality of language model responses.

Overall, the research highlights the importance of iterative revision when enhancing response quality and suggests that increasing the inference computation of a model may be more effective than merely increasing its size. Check out the project blog, demo, and Github link for more information, or join their 22k+ Machine Learning SubReddit and Discord channel for regular updates.

Leave a comment

Your email address will not be published. Required fields are marked *