跳到正文- Synthetic Data Generation
- Source: WizardLM / Evol-Instruct (Xu et al., 2023)
- Source: Will We Run Out of Data? (Villalobos et al., Epoch AI, 2024)
- Source: The Llama 3 Herd of Models (Meta, 2024)
- Source: The FineWeb Datasets (Penedo et al., Hugging Face, 2024)
- Source: The False Promise of Imitating Proprietary LLMs
- Source: The Curse of Recursion / Model Collapse (Shumailov et al.)
- Source: Textbooks Are All You Need (phi-1)
- Source: Self-Instruct (Wang et al., 2022)
- Source: Scaling Data-Constrained Language Models (Muennighoff et al., 2023)
- Source: OpenAI Terms of Use
- Source: Nemotron-4 340B Technical Report (NVIDIA, 2024)
- Source: Magpie (Xu et al., 2024)
- Source: Anthropic Commercial Terms of Service
- Source: 2 OLMo 2 Furious (Team OLMo, Ai2, 2025)