Fly.io now offers GPUs to all users, which are well-documented and easy to use. They can scale to zero and have been successfully used for running Whisper and experimenting with Llama 2 LLMs. Lower-cost GPUs will also be available soon.
•1m read time• From simonwillison.net
Sort: