System architecture changes were also a big part of the conversation at Hot Chips, said Jim McGregor, principal analyst at Tirias Research, who attended the show. Most presentations focused on eliminating bottlenecks in data movement, advanced chip designs and computing closer to or inside memory, he said.
Enterprise AI is likely to become increasingly heterogeneous, Sopko said. As a result, IT decision-makers should design AI architectures so they can move between GPUs, CPUs and specialized chips as the economics of AI change, he said.
“The strategic risk isn’t choosing the wrong chip in 2026. It’s building an AI architecture that prevents you from taking advantage of a much cheaper or better one in 2028,” Sopko said.
