#Tag
Posts tagged "ai"
Apps should dial the switchboard, never the model
Wiring each application straight at a model server's URL works on the first try, and from then on every change to the model layer is a change to every consumer. One OpenAI-compatible gateway in front turns backends into lines that can be added, drained or replaced without editing the consumers that dial it, provided you are honest about what the gateway does and does not carry across.
The model is an artefact, not a service
An open-weights model is a file you pull, pin and put on a volume, so it obeys the artefact rules you already run rather than a new discipline called AI operations. The only new budget is VRAM, and the fix for that is unloading the tool the moment nobody is asking it questions.
A new model is a new hire, not a new file
A model pulled from the hub is not just weights - it ships tokeniser code, an executable chat template, loaders gated by trust_remote_code, sometimes its own runtime, and the ecosystem's default is to run all of it beside your credentials. So every new model starts on probation: a hardware-isolated microVM with its own guest kernel, a default-deny network, zero credentials, and promotion to the standard serving tier only when the observed record earns it.