Distributed ML OS
I am working on developing a distributed operating system for managing machine learning resources (the entire memory hierarchy, networking and compute hierarchy). I worked in O/S development early in my career, now retired and ML is a hobby I've invested in over the past few years. I'm curious about: - how to "upcycle" end-of-life NVIDIA GPUs, like K80, M40, P100, etc. - how to offload MoE expert calculations to MCUs with NPUs, possibly FPGAs - how to manage a 10G LAN-attached network of older Xeon servers with the above GPUs and USB-attached MCUs as a machine learning "fabric" - do ideas from IBM's ACP/TPF O/S (now Z/TPF) carry over? - do ideas from IBM's Workload Manager (business priorities managed resources) carry over? - can I bypass end-of-life CUDA driver support for older NVIDIA hardware by using Rust/Vulcan? - can I leverage the architecture of Freetoken (MoE caching/routing, Engram, KV-cache compression) and other approaches like Baby Dragon Hatchlings, Hierarchical Reasoning Models, Tiny Recursive Models, Recurisve Language Models, Multi-token predicion, etc? - is anyone else intersted in actually building something like this? I've explored many of these parts individually, now combining them...