Dropout randomly zeros activations during training, forcing the network to be resilient. Most pre-trained LLMs have dropout disabled by default.
For fine-tuning on small datasets, adding light dropout (-) can help prevent overfitting. For large datasets, dropout is usually unnecessary.
Experiment on your specific case. Start without dropout and add it if you see overfitting.