NVIDIA Neural Network Sparsity Patent Granted Apr 21

← USPTO Patent Grants

Pruning and accelerating neural networks with hierarchical fine-grained structured sparsity

Grant US12608612B2 Kind: B2 Apr 21, 2026

Assignee

NVIDIA Corporation

Inventors

Yannan Wu, Po-An Tsai, Saurav Muralidharan, Joel Springer Emer

Abstract

Hierarchical structured sparse parameter pruning and processing improves runtime performance and energy efficiency of neural networks. In contrast with conventional (non-structured) pruning which allows for any distribution of the non-zero values within a matrix that achieves the desired sparsity degree (e.g., 50%) and is consequently difficult to accelerate, structured hierarchical sparsity requires each multi-element unit at the coarsest granularity of the hierarchy to be pruned to the desired sparsity degree. The global desired sparsity degree is a function of the per-level sparsity degrees. Distribution of non-zero values within each multi-element unit is constrained according to the per-level sparsity degree at the particular level of the hierarchy. Each level of the hierarchy may be associated with a hardware (e.g., logic or circuit) structure that can be enabled or disabled according to the per-level sparsity. Hierarchical sparsity provides performance improvements for a greater variety of sparsity patterns, granularity, and sparsity degrees.

CPC Classifications

G06N 3/082 G06N 3/0464 G06N 3/063 G06N 3/084 G06N 3/09

Filing Date

2022-02-28

Application No.

17681967

Claims

15

View original document →