<- Back
Comments (45)
- swiftcoderIt would certainly be interesting to have a highly respected EU-native inference provider, if only to make the regulatory gods happy
- perelinWhats definitely missing: a solid (non Mistral) GDPR compliant coding plan / subscription. All offerings are either US or China based. With the newest open weights models this became really interesting imo.
- ano-therGood to see more developments in this space. I quite like this service, which is a little further than Hetzner and has several models to choose from: https://www.infomaniak.com/en/hosting/ai-services
- mark_l_watsonThis seems like a smart move, given their ability to host efficiently. I approve of efforts to make the cost of inference for smaller useful models slowly approach 'close to zero' and there are many good paths for getting there. It is useful for companies to get fast hosting for the class of smaller models they may end up hosting in house.
- embedding-shape> The enable_thinking option is worth mentioning. Without it, the model can spend a surprising amount of the completion budget reasoning before it returns a visible answer.Straight up the opposite, which the name makes abundantly clear, with the option it does reasoning, without it it doesn't...
- HavocInteresting. I could see them perhaps coming in competitive for models that fit into single cards? Less so playing in the big model serving league...climbing into that esp right now would be madness
- rebeldeHetzner is very efficient hosting serversWill this be the new division of labor?Americans - best proprietary modelsChinese - best open weight modelsEuropeans - best / most efficient inference service
- nubgPotentially interesting article ruined by AI slop hallucinations like> For now, the API is fast, free, and fun to try. The next hardware announcement will tell us much more than another small model would.
- scoriiu[flagged]