Notes
Weekly notes
A short roundup each week: new models, infrastructure and papers worth knowing about, each with a number checked against its primary source.
Week of October 9: loading a KV cache from disk instead of recomputing it
A 50-million-token test reads saved KV blocks back from NVMe faster than a GPU can rebuild them, a placement scheme stretches the life of high-bandwidth flash, two MoE papers cut the cost of moving tokens between experts, and a forgetting study points at one setting on the output layer.
Week of October 7: shipping weight diffs instead of checkpoints
A trillion-parameter refit drops to 150 seconds by sending only the weights that changed, vLLM adds two ways to keep an engine warm across restarts, Aleph Alpha ships Kolibri-1 with FP8 weights, and a 975B model runs across eleven PCs.