Key Takeaways

• AI workloads require high bandwidth, ultra-low latency, and energy-efficient data movement, making the Network-on-Chip (NoC) crucial for system performance and scalability.
• Understanding AI communication patterns, such as tensor flow and memory reuse, is essential for effective mapping of compute, memory, and accelerator elements in SoC design.
• NoC construction strategies must consider real-world factors like die size and chip aspect ratios, with tree, mesh, and hybrid topologies evaluated for their physical awareness.
• The complexity of multi-die integration requires robust memory-coherency models and careful management of dynamic voltage and frequency scaling, impacting NoC behavior.
• Effective NoC partitioning and restructuring can simplify routing and improve predictability, essential for achieving timing closure in large AI-focused SoCs.

The explosive growth of AI and accelerated computing is placing unprecedented demands on system-on-chip (SoC) design. Modern AI workloads require extremely high bandwidth, ultra-low latency, and energy-efficient data movement across increasingly heterogeneous architectures. As SoCs scale to incorporate clusters of CPUs, GPUs, NPUs, memory subsystems, and domain-specific accelerators, the Network-on-Chip (NoC) becomes the backbone that determines system performance, power efficiency, and overall scalability. This presentation, featuring Arteris and Aion Silicon, explores the key considerations for architecting next-generation NoCs optimized for AI-driven designs.

REGISTER NOW

We begin with a deep look at AI communication patterns. Unlike traditional compute pipelines, AI workloads exhibit bursty, data-parallel behavior, with strong dependencies between compute engines and shared memory resources. Understanding tensor flow, on-chip bandwidth hotspots, and memory reuse patterns is essential for mapping compute, memory, and accelerator elements onto the SoC floorplan. This analysis directly influences NoC topology, routing strategies, QoS configuration, and buffer sizing.

NoC construction strategies form the core of the discussion. Tree, mesh, and hybrid topologies are evaluated not in isolation but through the lens of physical awareness—how real-world floorplans, die size, and chip aspect ratios dictate the most practical choices. AI-oriented SoCs often employ tiling strategies that replicate compute-memory clusters across the die. The NoC must scale with this modularity while maintaining predictable performance across thousands of concurrent data flows. The tradeoffs between bandwidth, power efficiency, and area become especially acute at advanced nodes, making architectural decisions increasingly consequential.

Another major theme is system complexity. Multi-die integration, enabled by 2.5D and 3D packaging, introduces new layers of NoC design challenges. Cross-die communication must handle longer physical paths, varying thermal conditions, and partitioned power domains. Distributed AI processing elevates the need for robust memory-coherency models—balancing performance with the overhead of maintaining shared state across heterogeneous compute engines. Designers must also account for dynamic voltage and frequency scaling (DVFS), power gating, and isolation across multiple domains, all of which impact NoC behavior.

To manage this complexity, performance simulation and KPI-driven evaluation are essential. The session discusses practical methodologies for modeling throughput, latency, congestion, and power consumption using a combination of transaction-level simulation, trace-based modeling, and workload-driven analysis. These tools allow architects to quantify tradeoffs early, before committing to RTL or physical implementation, ensuring the NoC meets system performance targets.

Finally, the presentation highlights the strategic role of NoC partitioning and restructuring. As designs approach physical limits, achieving timing closure becomes increasingly challenging. Partitioning the NoC into hierarchical or modular segments can simplify routing, reduce wire length, and improve predictability, especially in large AI-focused SoCs.

REGISTER NOW

With insights from Andy Nightingale, VP of Product Management and Marketing at Arteris, and Piyush Singh, Principal Digital SoC Architect at Aion Silicon, attendees will gain a practical, experience-driven perspective on designing scalable, high-performance NoCs. Their combined expertise—spanning system IP, NoC products, performance modeling, and complex heterogeneous SoC architecture—provides a grounded framework for building the next generation of AI-optimized silicon.

Also Read:

Architecting Your Next SoC: Join the Live Discussion on Tradeoffs, IP, and Ecosystem Realities

The Sondrel transformation to Aion SIlicon!

2025 Outlook with Oliver Jones of Sondrel

Share this post via:

Post navigation