Here are some resources I've found useful in learning how to fine-tune GPT-2.
These posts by Max Woolf are the best place to start for beginners:
His gpt-2-simple library is a great building block for fine-tuning GPT-2:
Fine-tuning the 1558M model (GPT-2 XL) can be done for free on Google Colab using TPUs. However, the terrain is a bit rockier.
This blog post by Svilen Todorov is a good starting point:
Shawn Presser has a Colab notebook that's also a good reference.