>>106557154
>>106557312
>model they host without also changing the model available to download on HuggingFace
My subjective experience is that DS changes values on the hosting side. I've not only observed model shift, I've noted shifts from session to session, as if I'm being handled between different inference servers at DS that have slightly different settings.
If I were DS, I would totally do that sort of A/B testing to tune inference. It would be easy to set up and monitor, and allow them to tune performance.
>>106549016
/aicg/ came through with this on the prefilling providers:
https://rentry.org/or-prefill#providers