🧑🏫 Instruction Tuning Vol. 1

The third advantage of imitation is that it allows the student (be it human or machine) to learn to do things that are hard to describe.

Principles of the "stream of activities" approach 1. Activities help people practice making the decisions that they make on the job, in a realistic context. 2. Activities target common mistakes and support behaviors that will achieve the business goal. 3. Activities target specific job roles and levels of expertise. They aren't expected t

One obvious step toward intentional language models would be for the creators of these models to be more intentional about what goes into their training data. For example, (Du et al., 2022) have demonstrated that a language model trained on a smaller dataset of curated data outperforms on standard benchmarks a model trained on a larger but less cur