Scaling the Memory Wall: Behind Sandisk’s High Bandwidth Flash for AI Inferencing

In the past couple weeks, Sandisk announced four significant developments around its High Bandwidth Flash (HBF) technology: a win in the Future of Memory and Storage (FMS) Best of Show, Most Innovative Technology awards; the formation of a technical advisory board; the signing of a memorandum of understanding with SK hynix; and an outlined path for HBF products to move into customer sampling, targeted for 2026. The announcements demonstrate both industry recognition and the company's concrete momentum in the development of HBF at a time when the AI industry is ramping up its data use to unprecedented rates.

While Sandisk first debuted the technology at an investor event in February, the recent announcements reveal a quiet innovation engine that has been running for nearly two years. Engineering groups from NAND design, ASIC design, and packaging had already been developing the platform to address what Sandisk positions as a solution targeted at AI inferencing bottlenecks.

What is HBF?

At its core, HBF—a new form of NAND—is aimed at addressing the AI "memory wall" , a longstanding concept that has taken on new urgency in the AI space. The memory wall describes a growing mismatch between AI's insatiable appetite for memory bandwidth and what current architectures can deliver as GPUs continue to scale compute performance at a much higher pace than DRAM.

Alper Ilkbahar, Sandisk Executive Vice President, Chief Technology Officer, and HBF Technical Advisory Board member, presented a keynote on HBF at the Future or Memory and Storage (FMS) 2025 and addressed this challenge openly.

The industry, he says, AI operations have an insatiable appetite for data capacity and high bandwidth memory. AI inference models, specifically, require a significant amount of fast interconnect between GPUs. And the industry is currently addressing this voracious appetite by introducing greater amounts of compute in the form of GPUs paired with High-Bandwidth Memory (HBM). According to Ilkbahar, this approach is not adequate due to fundamental scalability limitations.

"The real value in AI inference has historically been in the compute engine," he said in an interview. "But when you start looking at it, when it comes to running these large inference models, you also need a lot of memory. Specifically, you need a lot of memory bandwidth to bring data to where calculations are conducted, but you cannot fit an entire model into a GPU because you don't have sufficient memory."

In other words, Sandisk is aiming to fundamentally change what was once a compute-centric issue in AI inferencing to a memory-centric one.

Scaling beyond the memory wall

As various high-profile AI models are already using tens or even hundreds of billions of active parameters, the pace at which these models scale is only accelerating. The industry currently relies on HBM (based on DRAM technology), which provides high-memory bandwidth, but faces fundamental constraints in capacity and cost.

HBF's advantage for this critical gap is specifically in its capacity to scale, primarily because it's based on NAND technology. Sandisk states in a published fact sheet (PDF) that HBF is capable of closely matching HBM's bandwidth while delivering 8-16x the capacity of HBM at a similar cost. HBF's simulated performance on a Llama 3.1 405B parameter model for reading pretrained weights is within a 2.2% delta of a hypothetically unlimited-capacity HBM¹. It also closely matches HBM4's physical footprint, power profile, and stack height, but offers 512GB² total capacity per 16 die stack. These feats are possible thanks to Sandisk's CBA technology (CMOS directly Bonded to Array) and its leveraging of NAND.

"NAND is one of the most scalable technologies out there," Ilkbahar said in an interview. "One of the insights we had about inference workloads [during the exploration stage of HBF] was that it is a very different beast than, say, training. It is assumed you need DRAM because you need the speed. But you can create bandwidth in different ways, and it's bandwidth that matters, not latency."

This insight proved pivotal. By using NAND's superior density to create massive parallel data paths, HBF can help deliver the high bandwidth that AI inference models need without relying solely on DRAM.

It sounds great, in theory, but translating promising technology into production-ready solutions requires more than theories and engineering prowess alone, which is why Sandisk also assembled a deeply experienced technical advisory board, announced back in July.

Building strong alliances

Two of the members, in addition to Ilkbahar, are Professor David Patterson—Pardee Professor of Computer Science, Emeritus at the University of California at Berkeley, and a Google distinguished engineer—and Raja Koduri—Founder and CEO of Oxmiq Labs, and a computer engineer and business executive renowned for leading graphics architecture.

Ilkbahar made clear that to pull off a disruptive technology like HBF, it's crucial to have the right people by your side: "[Patterson and Koduri] know what it takes to implement a new architecture," he said." There are a lot of ideas that look wonderful on paper but then fail to materialize for one reason or another. Having this level of experienced people to guide us is important because it will help us succeed, while also helping avoid some of the pitfalls."

Both Patterson and Koduri expressed their excitement for HBF's promise in a press release announcing the advisory board, saying it will play a "pivotal role" in providing high bandwidth and revolutionary memory capacity. Koduri highlighted an additional significant opportunity that extends HBF's impact beyond just the AI data center: "HBF is set to revolutionize edge AI by equipping devices with memory capacity and bandwidth capabilities that will support sophisticated models running locally in real time," said Koduri. "This advancement will unlock a new era of intelligent edge applications, helping to fundamentally change how and where AI inference is performed."

The announced Memorandum of Understanding with SK hynix will facilitate practical collaboration on technical specification standards, die requirements, and other necessary work needed to prepare the market for HBF adoption.

Dr. Hyun Ahn, SK hynix President and Chief Development Officer (CDO), had this to say in a release about the MOU: "Through our work with Sandisk to standardize the High Bandwidth Flash specification, we are actively contributing to the commercialization of this innovative technology, which we believe is key to unlocking the full potential of AI and next-generation data workloads."

Looking forward

In his keynote, Ilkbahar made an important announcement that Sandisk is targeting delivery of first samples of its HBF memory in the second half of calendar 2026 and expects samples of the first AI-inference devices with HBF to be available in early 2027.

From BiCS technology and CBA wafer bonding to proprietary stacking designs, ultra-low warpage designs, and industry-leading NAND scaling, Sandisk has systematically built the technological foundation for HBF. With AI models approaching trillion-parameter scales, these innovations position Sandisk to deliver the memory breakthrough that could help solve a critical challenge in AI scaling.

To find out more about HBF, check out the Fact Sheet (PDF) and press releases in the Sandisk Newsroom.

Disclosures

1. Based on internal testing and simulation. This is simulated for reading 8-bit pretrained weights on Llama 3.1 405B parameter model. One kernel is executed on the xPU performance model at a time. Actual results may vary depending on specific circumstances and contextual factors. The comparison does not reflect the capacity advantage of HBF that can fit the full model as HBM capacity is assumed to be unlimited for modeling purposes. This demonstrates that with the higher latency and larger page size of HBF compared to HBM, the system level performance is still comparable and HBF can work for AI inference workloads.
2. 1GB = 1,000,000,000 bytes and 1TB = 1,000,000,000,000 bytes. Actual user capacity less.

Forward-Looking Statements