https://huggingface.co/blog/faster-transformers
Looks like OpenAI did move the local ecosystem forward, not by their model itself, but with the tricks they used to run it