“We will need much faster interconnects because as models get bigger, and more compute and data is needed to train them, it’s necessary to scale out to many racks worth of machines. Interconnect bandwidth during scale-out is critical to preventing the accelerators from stalling while waiting on network transfers for …

"The idea is to move the interconnects from the front panel directly into the same package, reducing the cost of the electronics. In this way, you simplify all the conditioning electronics, you reduce power consumption, and you reduce signal loss. You can also increase the bandwidth density per square centimeter. …