Using LLMs to generate training data sounds circular but works surprisingly well. You can use a powerful model like GPT- to generate data for fine-tuning a smaller model.
The key is careful prompting. Specify exactly what you want. Include examples. Request diversity.
Synthetic data bootstraps when you have nothing else. It's not a replacement for real data but a starting point.