Now that AMD launched its Instinct MI455X Helios scale-up rack (72 chips, 31 TB of HBM4 capacity per rack, similar to Nvidia Rubin Oberon) and secured some relevant tenancy commitments, it’s not implausible they might get a meaningful enough share of the market in 2028+ for their systems to start enabling flagship models, if Nvidia falters. SemiAnalysis claims Kyber is delayed; if this is correct, it’s probably only coming out a year later with Feynman rather than Rubin Ultra, and so the 2028 buildout might end up missing a generally available 110 TB HBM4E scale-up system (HBM4E is fast enough to make 4-stage pipelines practical, and thus 240T total param models feasible). Nvidia’s Rubin Ultra 2x Oberon systems (which might also be canceled) weren’t as relevant (especially given Kyber), but Helios has 1.5x more stacks of HBM per rack, so if like Rubin Ultra they also upgrade to 12-Hi HBM4E (4 GB per DRAM die) next year, HBM capacity will be 41.5 TB per rack. If they additionally make a 2-rack scale-up system, a 4-stage pipeline of 2-rack scale-up systems will have 332 TB of HBM, borderline sufficient for a 240T model even in FP8 (and more centrally sufficient either in FP4 or for a 1.5x smaller model, with 20x sparsity rather than 30x).
Another reason to take a closer look at AMD Instinct chips is that they might offer clues about Nvidia Feynman. Feynman was announced to use TSMC A16 logic (I’m not quite sure, but this seems surprising for reticle-sized dies in 2028), and also some unspecified SoIC thing (hybrid bonding). If Feynman goes with chip layouts similar to MI355X and MI455X, using a more mature logic node (than A16) for reticle-sized base dies, and then putting 4 quarter-sized A16 compute dies on top of each base die (with hybrid bonding), that helps make sense of both of these details. Also, if Feynman (at least in the second year, launched in 2029, with most of the buildout over 2030) changes the layout to only connect HBM stacks to a logic (base) die on one side rather than two, the Nvidia precedent for Blackwell and Rubin is to put 2 HBM stacks per side of a reticle-sized compute die (it was 3 in Hopper, but then it was just 1 compute die per chip). MI455X (unlike MI355X) demonstrates that a modern chip with 2 reticle-sized logic dies can connect to 3 stacks of HBM per side, which makes it more plausible that Feynman gets 3 stacks of HBM per (base) reticle-sized logic die rather than 2 (if it does move to the layout with HBM stacks on only one side of the logic dies). If this is the case, a second-year Feynman 8x Kyber system will end up with 1.5x more HBM capacity (1.1 PB rather than 740 TB), or alternatively it won’t suffer too much if 16-Hi HBM5 gets delayed and only 12-Hi is available (then it gets 830 TB per scale-up system).
For performance, MI355X is an interesting anchor for predicting Feynman, since it has exactly the same power and performance per chip as B300 (5e15 FP8 FLOP/s, 8 stacks of HBM3E, 1.4 kW TDP), but it uses 2 layers of hybrid bonded logic, with TSMC N6 base dies and TSMC N3P compute (XCD) dies on top, somehow ending up with only 185B transistors (one factor is that the total area of the XCD dies is smaller than that of the base dies). Compare this to Blackwell’s 1 layer of TSMC N4P (208B transistors). Thus MI355X has on average 2x lower power per layer, fewer transistors, a newer process in half of them, and ends up with about the same performance. This predicts that a hypothetical Feynman chip with (say) N3P reticle-sized base dies and (multiple smaller) A16 hybrid bonded compute dies on top might have performance similar to that of a 1-layer N2P chip at the same TDP, but also that higher TDP (than for a 1-layer N2P chip) would be feasible if the cooling solutions keep up (and the total power per rack remains feasible).
Now that AMD launched its Instinct MI455X Helios scale-up rack (72 chips, 31 TB of HBM4 capacity per rack, similar to Nvidia Rubin Oberon) and secured some relevant tenancy commitments, it’s not implausible they might get a meaningful enough share of the market in 2028+ for their systems to start enabling flagship models, if Nvidia falters. SemiAnalysis claims Kyber is delayed; if this is correct, it’s probably only coming out a year later with Feynman rather than Rubin Ultra, and so the 2028 buildout might end up missing a generally available 110 TB HBM4E scale-up system (HBM4E is fast enough to make 4-stage pipelines practical, and thus 240T total param models feasible). Nvidia’s Rubin Ultra 2x Oberon systems (which might also be canceled) weren’t as relevant (especially given Kyber), but Helios has 1.5x more stacks of HBM per rack, so if like Rubin Ultra they also upgrade to 12-Hi HBM4E (4 GB per DRAM die) next year, HBM capacity will be 41.5 TB per rack. If they additionally make a 2-rack scale-up system, a 4-stage pipeline of 2-rack scale-up systems will have 332 TB of HBM, borderline sufficient for a 240T model even in FP8 (and more centrally sufficient either in FP4 or for a 1.5x smaller model, with 20x sparsity rather than 30x).
Another reason to take a closer look at AMD Instinct chips is that they might offer clues about Nvidia Feynman. Feynman was announced to use TSMC A16 logic (I’m not quite sure, but this seems surprising for reticle-sized dies in 2028), and also some unspecified SoIC thing (hybrid bonding). If Feynman goes with chip layouts similar to MI355X and MI455X, using a more mature logic node (than A16) for reticle-sized base dies, and then putting 4 quarter-sized A16 compute dies on top of each base die (with hybrid bonding), that helps make sense of both of these details. Also, if Feynman (at least in the second year, launched in 2029, with most of the buildout over 2030) changes the layout to only connect HBM stacks to a logic (base) die on one side rather than two, the Nvidia precedent for Blackwell and Rubin is to put 2 HBM stacks per side of a reticle-sized compute die (it was 3 in Hopper, but then it was just 1 compute die per chip). MI455X (unlike MI355X) demonstrates that a modern chip with 2 reticle-sized logic dies can connect to 3 stacks of HBM per side, which makes it more plausible that Feynman gets 3 stacks of HBM per (base) reticle-sized logic die rather than 2 (if it does move to the layout with HBM stacks on only one side of the logic dies). If this is the case, a second-year Feynman 8x Kyber system will end up with 1.5x more HBM capacity (1.1 PB rather than 740 TB), or alternatively it won’t suffer too much if 16-Hi HBM5 gets delayed and only 12-Hi is available (then it gets 830 TB per scale-up system).
For performance, MI355X is an interesting anchor for predicting Feynman, since it has exactly the same power and performance per chip as B300 (5e15 FP8 FLOP/s, 8 stacks of HBM3E, 1.4 kW TDP), but it uses 2 layers of hybrid bonded logic, with TSMC N6 base dies and TSMC N3P compute (XCD) dies on top, somehow ending up with only 185B transistors (one factor is that the total area of the XCD dies is smaller than that of the base dies). Compare this to Blackwell’s 1 layer of TSMC N4P (208B transistors). Thus MI355X has on average 2x lower power per layer, fewer transistors, a newer process in half of them, and ends up with about the same performance. This predicts that a hypothetical Feynman chip with (say) N3P reticle-sized base dies and (multiple smaller) A16 hybrid bonded compute dies on top might have performance similar to that of a 1-layer N2P chip at the same TDP, but also that higher TDP (than for a 1-layer N2P chip) would be feasible if the cooling solutions keep up (and the total power per rack remains feasible).