AI will not replace chip designers anytime soon. In fact, AI has made chip design even harder in many ways. This post will illustrate why from first principles by covering the following topics:

• High Level Overview of VLSI Design Process

• Vibe Coding RTL – Systolic Array example

  • Benefits and Risks

• Why Chip Design Takes So Long: Cost of Error due to Fab Bottlenecks

• Key Bottlenecks that Limit AI use in Chip Design to Quality-of-Life Enhancements

  • 🔒Functional Verification

  • 🔒Physical Co-design: Modularity and Thermal/PDN/SI/Packaging Constraints

  • 🔒AI-Driven Design Space Exploration

  • 🔒The Hidden “Memory Wall” - Compute Resource Bottlenecks

I went to the floor of the Design Automation Conference on July 26-29 in Long Beach, CA. This is conference #5 of a broader conference tour this year (ISSCC, DesignCon, APEC, and ECTC) where I’ve worked my way up in abstraction levels to better understand the complete AI datacenter stack and educate you on it.

I’ve noticed a lot of prominent writers who don’t know much about chip design speculate about how Agentic AI will impact it. Most people assume that AI will have the same disruptive impact in the SW and chip industries since software and RTL code can easily be generated with LLMs, creating a lot of FUD.

I’ll highlight the benefits of AI with a vibe coding tutorial of RTL and how chip designers can benefit. Then I’ll show the critical flow differences in chip design that need to be scrutinized and cause veteran engineers to not completely trust AI.

Please note that I have more experience in analog / mixed signal design, not digital design. I have interfaced with digital / verification people and written my share of VerilogAMS to have a working understanding of the VLSI design flow. I reference the book “VLSI Physical Design: From Graph Partitioning to Timing Closure” for more detailed flow information.

As always, if you’re an expert in this area and notice a mistake, please reach out to me so I can have it promptly corrected.

High Level Overview of VLSI Design Flow

Chips have become much more complex over the years due to a well known phenomenon of Moore’s law. Moore’s law enables more transistors to be placed and doubling compute density about every two years. Very Large Scale Integration (VLSI) is the umbrella term used to refer to the tight integration density of the various classes of chips (SoC, GPU, ASIC, FPGA).

Automation is necessary for chip design because of the sheer design complexity and errors in chip designs are much harder and costly to fix that board-level errors. As a result, the entire EDA industry revolves around robust automation tools to perform a lot of complicated steps automatically. Only low-level JP and customized analog blocks tend to be hand drawn and instantiated by the tool at higher lever assemblies.

At a DesignCon keynote “Agentic AI for Chip Design”, Mark Ren, founder and CEO of AGENTRYS, notes how EDA tools flow raised the abstraction level that humans operate at. AI enables an even higher level of abstraction by allowing engineers to specify design intent in plain language rather than a specific programming language.

The goal of the VLSI design process is to produce a fabricable, error free mask that implements the desired system specification within the constraints of the fabrication Process Development Kit (PDK). The VLSI design process evolved over time to incorporate the following major steps:

• System Specification. The overall high level goals of the system are defined collectively amongst chip architects, product marketers, and ASIC engineers

• Architectural Design. An architecture must be defined to achieve the high level goals of the system. These include:

  • Analog/Mixed-Signal Blocks - ADCs/DACs, regulators

  • Digital Blocks - DSPs, and types of computational blocks

  • Interfaces - I2C, UCIe, PCIe

  • Hard / Soft IP

  • Packaging

  • Clocking Architecture

  • Process Node

• Functional Design and Logic Design - Logic design is performed at the RTL level using a HDL such as Verilog or VHDL.

• Custom Circuit Design - A handful of critical blocks such as ESD protection, I/O, analog circuits, and RAM blocks have to be hand-designed.

  • Companies can develop these IP in house with library / IP teams across major projects or license IP from specific vendors.

  • In some cases of one-time jobs, such as customized analog blocks, companies can utilize contract labor to complete the work if there is not a steady workload for a W2 employee.

• Physical Design - The RTL is synthesized and billions of design components get instantiated with their geometric representations using highly sophisticated Place-and-Route Tools. Physical design is highly iterative and includes the following steps:

  • Partitioning

  • Floorplanning

  • Placement

  • Clock tree Synthesis

  • Routing

  • Timing / Power Closure

• Physical Verification and Signoff. The layout needs to be checked against different design rules defined by the fab (DRC, LVS)

  • The design needs to be officially “signed off” by specific tools to keep a paper trail should errors arise down the line. These tools often have valuable information in the logfiles that can pinpoint sources of errors. Design waivers can be made to in special circumstances to overcome tool constraints and are agreed upon in design reviews, but are generally avoided at all costs.

• Fabrication - The final layout is streamed out as a GDSII Stream format and is sent to a fab.

• Packaging and Testing - The chips are packaged using appropriate package types such as DIPs, PGA, BPA, flip chip, or WLCSP.

The design flow is iterative, with engineers making design choices at each step based on simulation feedback before handing off the work to the next step.

Chip design has historically been siloed with well-defined specialties and tool flows developed within each specialty to maintain consistent flows along process node improvements. This approach has worked to scale chip performance for a number of years, but the slowing down of Moore’s law and multiphysics constraints had led to a paradigm shift toward cross-domain engineering. This approach requires a shift in engineering mindset that is not always easy to transition to due to entrenched organizational incentives. This substack aims to facilitate that mindset shift by making cross-domain information easier to access.

Vibe Coding RTL – Systolic Array example

Before I talk about the challenges of AI for chip design, I’ll highlight a positive example of how AI can benefit chip designers: code generation of RTL.

At the beginning of DAC, Steven Truitt, Nan Wu, and Liu Liu gave a very good tutorial “From Vibe Coding to Silicon: Engineering Agentic Systems for Hardware Design” using Microsoft Discovery. Their GitHub is public at the following link that you can check out:

https://github.com/stevetru1/DAC2026_Vibes

https://github.com/ORCA-lab/DAC26_VibeCoding_Runs

What is Vibe Engineering

Vibe engineering is the hottest trend in SW engineering that enables non-SW engineers to code SW and offers productivity enhancements for junior engineer-level tasks. Vibe engineering can also apply to the SW portions of chip design, including RTL coding.

Vibe engineering involves thinking an abstraction level higher than traditional engineering. You rigorously describe your intent in terms of the desired outcome, rather than the programming language, for your agent to perform. The design is iterated on through conversation and feedback loops based on what the agent produces.

Here is an example prompt template where the desired outcomes are rigorously specified for the agent.

One important consideration in vibe engineering is defining the definition of done. Task organization up front is extremely important to properly decompose, organize tasks, and build the harness flow for the agent to execute in. The test and evaluation suite is also very important to evaluate agent performance properly.

Vibe engineering places more emphasis on judgement in execution rather than execution expertise. The challenge is figuring out the right thing to build, instead of building faster. As a result, the human layer carries more weight.

Example: Systolic Array

One key block in AI accelerated ML applications is the systolic array. Systolic arrays are often used for AI acceleration and are typically co-designed to the specific data needs. I’ve written on Systolic arrays previously in my other posts.

A description a vibe coder might give an agent can look something like this:

Now I’ll admit: I hate writing RTL code, even though I have taken a Verilog and VHDL class, so I find this vibe engineering approach such a breath of fresh air.

More detailed information can be found in the slide decks and tutorials; I’ll leave it up to you to check those out.

Why Chip Design Takes So Long: Cost of Error due to Fab Bottlenecks

Chip Design is one of the most brutally unforgiving industries because chip design is very expensive, time consuming, and carries a high cost of error.

In board-level design, any design errors can be easily fixed after fabrication because the device is more interpretable and technicians can workaround issues such as adding jumpers and lifting off pins. As a result, there is less scrutiny during board design because components with datasheets can be hooked up with passives in between and simulated in SPICE at a high level.

Chip design has way more components than boards and does not offer that deep of a debug capability to probe deep inside the circuit. Test modes and circuits for DFT must be inserted such as scan-chains and test mode registers. It takes 3 - 4 months for a chip to be fabricated because each individual fabrication step can take days due to manufacturing queue times and tool bottlenecks.

First Pass Silicon Success

Generally speaking, there is a low pass rate of first pass success, around 14% nowadays. This might come at a surprise for outsiders, but not unexpected for engineers as schedules typically account for this. For most practical purposes, the first silicon revision is used to identify bugs that weren't caught early on, and the second revision is used to fix them. Two revisions are generally considered standard, with some CEOs like Intel’s Lip-Bu Tan threatening to fire engineers who exceed this.

There are a few ways to minimize time delays and errors:

• Pre-Silicon Emulation. The design is loaded on an FPGA or hardware array to make sure the design is bug-free. Emulation is much faster than computation-heavy simulation and typically run at lower clock speeds to test specific test cases and interactions within blocks.

• Pre-Silicon Prototyping. Not only does the chip has to work as designed, it also has to work as intended in the broader system context. Prototyping serves an important function to ensure you build the system you intended, not the bug free one you built.

• IP Reuse. Almost all new chip designs have a starting point with known good IP blocks. Low risk IP can be incorporated in a new design, and new IP to implement the performance requirements needs to be modified or created out of scratch. The overall design risk is assessed based on the risk of existing and new IP blocks themselves.

• Wafer Staging. During fabrication, several wafers are “staged” where only the time-consuming FEOL layers are formed. In case of bugs identified early that can be fixed in metal-only, this shortens the fab time to get wafers back without having to go through the entire PnR flow all over again.

  • Companies generally name revisions A0, A1, B0, where A0 is the first revision, A1 represents a metal change, and B0 represents a whole new silicon revision with a new mask set.

  • The fabrication cost and time is largely dictated by the # of masks since each tool loads a specific mask.

First pass silicon success is really only feasible when either the chip is simple enough, there is enough existing IP that is leveraged, or there are heavy emulation/prototyping efforts. Some companies who want to specifically integrate specific circuits want to see a test chip done first, key data taken, and risk evaluated before committing the chip to be integrated in their product in the next revision. General market products can be a bit more lax.

Scar Tissue

The long fab time and cost of error leads to a less discussed phenomenon in the chip industry: scar tissue, which is a form of lessons learned painfully through delays and expenses.

Scar tissue is one big reason why experienced silicon veterans often push back against claims that AI will “autonomously design chips” because it doesn’t carry accountability. This lack of accountability for AI tools means that it won’t replace human designers anytime soon.

Key Bottlenecks That Limit AI use in Chip Design

I’ll discuss four key bottlenecks that several panelists at DAC raised and promising solutions within them.

Functional Verification

Most people outside of the semiconductor industry don’t know that verification, not design, has historically been the most labor and time intensive task. Let me give a bit of context to what verification should be, and how its implemented in practice.

Share

Read more

Editors Note: (7/22/26): I trimmed the paywall to only paywall the scaling challenges with Analog CiM. This way, you can come away with a mental framework for the components of the Analog CiM.

Compute-in-memory has emerged as a promising technology to overcome the Von-Neumann bottlenecks of conventional digital computing. Many venture-backed startups have entered this space with promising solutions to enhance compute efficiency.

In this post I’ll cover a broad overview of the AI accelerator landscape and do a deep dive into the challenges with scaling Analog Compute-in-memory. The outline of this post is as follows:

• A Survey of the AI Accelerator landscape

  • Application Spaces - from VLP to Data Center Systems

  • Scaling Trends - Co-Design and Specialized Architecture

  • Hardware Classifications of Compute in/near memory AI Accelerators

    • Parallel Threads (NVIDIA GPU)

    • Tensor Array (Google TPU, Groq)

    • Manycore Accelerators (Cerebras, Tenstorrent)

  • Digital Compute-in-Memory

• The Building Blocks of Analog Compute-in-Memory

  • SRAM Arrays

  • Input Encoding

  • Charge / Current Domain Devices

• 🔒Scaling Challenges with Analog CiM

  • 🔒Array Nonidealities that degrade SNR

  • 🔒Area Inefficiency

  • 🔒Modality Conversion Overhead

  • 🔒Example CiM startup: Encharge AI

• 🔒BONUS: The Challenges of Optical Compute for AI

Back in February 2026 at ISSCC in San Francisco, CA, I personally sat in the tutorial “Fundamentals of Compute-in-Memory: From Circuits to Systems” by Prof. Jaydeep P. Kulkarni at the beginning and an all-day forum “Analog for AI and AI for Analog” at the end. Several heavyweights in the analog community such as Prof. Naveen Verma and EDA community such as Prof. David Pan were present to discuss trends in Analog CiM and Automation for AMS/RFIC Signal Design, the latter of which I will eventually cover after DAC.

Personally, analog compute-in-memory is one of my favorite topics because it almost resembled the way the brain processes data. Analog CiM is perhaps the most interdisciplinary topics among semiconductor specialties because it combines knowledge from digital, mixed signal, logic fabrication, and memory domains. I think the circuits are inherently power efficient for repetitive computations.

However, the CiM space is filled with a lot of dense academic research and marketing claims of VC-backed companies. During those ISSCC sessions and while researching CiM papers, I initially found CiM difficult to digest because there aren’t universally agreed on standards to classify related architectures. The terminology is also inconsistent; some authors use compute-in memory, others use processing-in-memory, and other add subcategories of their own. Moreover, most papers tend to jump right into the block diagrams of specific applications known among academics in that space. I’m sure many of you also feel the same way trying to understand the space.

In my opinion, CiM is a space where broad system benefits can be lost in the vast chasm between marketing claims and actual technical challenges. I believe that researchers are candid about the real engineering challenges. However, claims of performance benefits tend to narrowly focus on the energy savings of ONE part of the whole computational energy budget. These claims tend to get picked up and hyped up by non-technical people without their proper nuances, which, for some companies, is a marketing strategy all in itself.

Having sat in those rooms and felt the confusion myself, I aim to bring clarity to the space. Here are some of my previous posts I’ve written that lay a foundation for this material from both the software and hardware side:

A Survey of the AI Accelerator Landscape

GenAI and Agentic AI models have exploded in the past few years with a combination of gigantic training sets and powerful models to handle a variety of AI applications. This creates a huge demand for tokens for Agentic AI that is primarily satisfied by high power, general-purpose data center GPUs in data centers.

The underlying hardware is becoming more powerful to satisfy this demand. However, the logic is scaling much faster than the memory and interconnect by approximately ~2x. This means that the compute bottleneck is not in the compute itself, but the data movement. As models become more powerful, memory transfer operations are consuming more and more energy and latency.

As a result in the explosion of compute power, co-packaged optics and vertical power delivery are emerging and widely discussed trends to satisfy these demands. I have written extensively on these topics on my website.

One of the main arguments for CiM is to minimize the power overhead in compute-memory transfers. Here is a prominent slide given in 2014 with ballpark energy consumption of various computational operations and memory accesses. Notice the order of magnitude difference between the energy used in cache and DRAM accesses, and the higher energy consumption for higher precision operations. This chart has driven a trend toward lower precision computations in the digital domain and memory efficient architectures.

Application Space - from VLP to Data Center Systems

AI accelerators are not just limited to centralized data center GPUs, but also embedded applications with many different form factors and use cases.

Here we notice a range of publicly announced accelerators that range in compute efficiency from 0.1 - 100 TOPs/W. Notice how there is a cluster primarily centered around 1 TeraOps / W for general purpose GPUs data centers. Also notice how at lower power, accelerators specialized for embedded applications tend to have higher TOPs/W because they are tailored for the specific power / performance / form factor requirements of the system they are in. In those cases, it doesn’t make sense to stream data to a centralized data center to run inference on.

General purpose GPUs are built to support both training and inference across a broad range of AI models targeted for different applications, depending on the tasks you throw at it. Two common applications include:

• CNNs: Uses deep neural networks to extract lower level features and are used for image / speech classification

• LLMs: Uses attention layers to calculate relations between words using Query, Key, and Value vectors and require large KV cache storage for long sequences

Scaling Trends - Co-design and Specialized Architecture

Massive scale-up of AI clusters are driving a huge need for compute efficiency. Large networks of parallel general-purpose GPUs move a lot of data around, consuming vast amounts of power and generating a lot of heat in the process.

To address this need for more compute efficiency, the industry is shifting from general purpose to hardware-software Co-Design (or co-optimization) of both DNN Models and Hardware.

This involves customizing the HW to be as efficient at computing its specific task. Some optimization examples include:

• Memory: Higher bandwidth and more tightly integrated memory

• On the SW Side:

  • More aggressive quantization / lower precision architectures

  • Sparse weights

• HW architecture: Stationary architectures that “fix” one value in an array and stream in the other (weight, output, and input)

In general, all else being equal, there is a tradeoff between compute efficiency and how specialized the hardware is. Specialized hardware can augment general purpose GPUs but run the risk of being considered “one trick ponies” if data needs change in the future.

I personally think what most people describe as “co-design” should extend beyond the HW-SW interface into the broader high speed communication, power, and packaging subsystem. That is what my website aims to achieve.

A Taxonomy of the Compute-in-Memory Landscape

In classical computer architectures, main memory is stored on a huge chunk of off-chip DRAM (HBM) and chunks of data are read onto on-chip caches (typically SRAM) when requested. When MAC operations occur, intermediate results needs to be stored in registers until the final result is computed on.

Depending on how much you read/write data to/from memory, up to 30-70% (or even higher) of the compute energy can from memory transfers, not the compute itself. This makes efforts to optimize / trim data transfers between compute and memory such an attractive option to scale compute efficiency along with performance.

There are three major architectural classifications that classify how close the memory is to the bit cell:

• Compute with large memory

• Compute near memory

• Compute-in-memory

We’ll go through examples within each classification.

Compute with Large Memory

Compute with large memory is the easiest solution to scale in existing paradigms with a large GPU and HBM side by side.

There is a wide range of chip architectures within this compute fabric that vary in specificity and programmability. Most AI accelerators fall in between two conventional categories:

• CPUs: Very flexible compute-wise but not optimized for highly parallel inference and training computations.

• FPGAs: Used off-the-shelf and easily repurposable for specific embedded applications. FPGAs are preferred for faster time-to-market over custom ASICs since they can be modified with RTL. However, for large models AI, FPGAs are not “programmable” in the SW sense because the hardware effectively changes every time the FPGA is programmed.

The bulk of compute-efficient AI accelerator products lie in-between these two categories to enable software, hardware, and algorithm co-optimization. Two such categories in between fall in the “compute with large memory” paradigm:

• Parallel thread accelerators

  • Use a temporal architecture with traditional ALUs computed in a SIMT (Single Instruction, Multiple Threads / Data ) paradigm

    • Typically, one CUDA-thread processes one element of the target data set

    • All threads in a process share the same memory heap

  • Key Features

    • Warp Scheduler - controls a large number of ALU’s by broadcasting a single instruction to all of them to execute in lockstep

    • Memory Coalescing - combines multiple memory requests from parallel threads into a single, highly efficient memory transaction from the HBM

      • Memory coalescing is critically important for making data transfer efficient from the HBM

  • Advantages

    • Quick to program

    • The most flexible to handle a wide range of AI applications

    • Massive parallelism

  • Disadvantages

    • Low compute efficiency

    • Needs sophisticated control circuitry to coordinately data movement across ALUs and memory

      

  • Examples

    • NVIDIA H100

    • AMD Instinct MI250X

    • Intel Arc 770

    Generally speaking, powerful GPUs are used for heavier, general purpose tasks that require large complex models.

  

• Tensor Array

  • Tensors are mathematical objects that generalizes scalars, vectors, and matrices to higher dimensions

  • Tensor Arrays use Systolic Arrays or static spatial pipelines. A systolic array is a 2D grid of tightly integrated Processing Elements (PEs) which are a more specialized form of ALUs tailored for MAC operations.

  • A classic example of one is Eyeriss which I described in more detail in my previous post.

  • Key Features

    • Static scheduling - preload model parameters into accelerator before executing inference/training on them

  • Advantages

    • Data reuse - data doesn’t have to be stored in registers and can flow between ALUs that perform either Multiply or Accumulate operations

    • Dense Compute Density

    • Latency is deterministic - no complexity overhead from unpredictable scheduling

  • Disadvantages

    • Rigid, inflexible structure. The array size must be tailored to the specific application

    • Inefficient at handling sparsity. The sparse data still has to flow through the array

    • SW compiler complexity

  • Examples

    • Google TPU - showcased the first TPU in 2017

    • Groq LPU

    • Intel Gaudi

    In short, Tensor arrays have the potential to be computationally efficient, but only for specific array sizes.

Compute Near Memory - Manycore

Manycore is an example of compute-near-memory. This architecture decentralizes both compute and memory arrays them along a grid. This shortens the data path from the local SRAM memory to the compute units.

Prominent examples of manycore include:

• Cerebras

  • Perhaps the most well-known example of manycore with cores arranged on a wafer scale engine (WSE) with 850k cores

  • The memory is fully distributed, with the SRAM using 50% of the die area that provides full memory bandwidth to all data paths at full performance, removing the need for data reuse

  • Their WSE is efficient for sparse matrix operations

• Tenstorrent

  • Tenstorrent has a very similar architecture to Cerebras, which fuses a NoC interface to move data to local SRAM to be computed on in each Tensix core.

Advantages

• Memory path is short between compute and memory

Disadvantages

• Additional software overhead to coordinate data movement to individual cores

Other accelerator options

There are other accelerator options available:

• Neural Processing Unit (NPU): NPUs are built specifically to accelerate AI tasks directly on devices like smartphones and laptops

• RISC-V processors: RISC-V is a open source ISA widely used among academia and emerging startups. Its modular nature allows easier integration capabilities with embedded applications.

Within the CiM space there are two major categories: Digital, and Analog.

Digital Compute-in-Memory

In standard digital PE-based accelerators, the partial sums of the matrix operations are stored in registers and accumulated with PEs. In Digital CiM, these intermediate registers are eliminated and uses digital adder trees after the digital multiply to perform the accumulation of partial sums.

Digital CiM offers the potential to improve energy efficiency MAC operation through bit serial / parallel operations as well as sophisticated control circuits to better control data movement.

Most efficiency analysis of DCIM is done by examining the intricacies of floating point operations and accounting for the energy consumption of individual operations, which is beyond the scope of this post. In general, with data accumulation trees, the floating point operations need to be synchronized, with a few optimization choices including alignment-first and multiply-first.

D-Matrix is one of the most prominent examples that leverages their in-SRAM computing techniques. Here they optimize data movement by building SRAM directly into the compute units and use block floating-point math to assign a single shared exponent to a group of numbers.

Share

One issue with DCIM is that logic is incredibly bloated compared to memory. In the example above, 70% of the area is taken up by the MAC and 11% taken up by the memory.

When compared against a standard, state-of-the-art digital processing element (PE) on the exact same technology node, we see that there appears to be a big efficiency gain. However, most of the area efficiency gain comes from custom DCIM hard macros that can be generally made much denser, but adds to physical design time.

As a result, some people amongst the analog CiM community argue that DCIM reverts to standard digital PE acceleration with little efficiency gain when compared apples-to-apples. Here we see another slide with a side by side comparison of TOPs performance of digital accelerators vs digital IMC. Claims and counterclaims like this is partly why analyzing the performance benefits of IMC is so confusing because there are a lot of nuances.

So, yes, in theory, DCIM can be potentially compute dense if you hire an army of custom layout engineers to route the wires, but most companies don’t do that due to the time-to-market implications.

So in short, digital CiM offers marginal efficiency advantage for the effort involved, but it hasn’t stopped startups such as D-matrix from trying.

The Building Blocks of Analog Compute in Memory

SRAM Array Skeleton

Most work done in Analog CIM involves modifying a conventional SRAM array with either modulated inputs or alternative devices. In an conventional SRAM array, SRAM unit cells are arranged in rows of wordlines and columns of bit lines.

When SRAM arrays are modified for Analog CiM, peripheral circuits are modified to handle parallel analog operations. These include the following components:

• Input

  • Decoder - asserts multiple wordlines simultaneously with modulated input values. You can’t normally do this in SRAM memories, but this is essential for parallel matrix operations.

• Output

  • Bit Line Processing - Performs SIMD analog processing for each column (such as a S/H or integrator)

  • Cross bitline processor - Connects processing elements across different columns. This block performs analog averaging to clean up array non-idealities and common mode noise to enhance SNR

  • Residual digital unit - The ADC that converts the result to digital domain

Within this SRAM array the compute operation itself requires a few modifications.

Input Encoding

The input can be either an analog voltage or pulse width based, with the duty cycle representing the effective analog value to be either integrated or accumulated on.

Weight Encoding.

There are two primary classifications of devices used to encode the weights that this input computes on.

• Charge Domain

  • Charge Domains devices perform parallel MAC operations by injecting charge across a target capacitance through PWM, PAM, or a direct analog voltage. The total accumulated charge creates a voltage shift that maps to the inner product output.

  

  • Examples of charge domain devices include:

    • Conventional SRAM - uses either pulse amplitude or width modulation to charge the bitline voltage and reading the accumulated charged at fixed intervals

      • This bit line must be pre-charged to a high voltage before every compute cycle

      • Then, a wordline pulse allows the bitline to discharge, lowering the voltage on the line.

      • Note that additional switches are typically added to make the 6T SRAM either 8T or 10T

    • DRAM - relies on charge sharing between the internal cell capacitance and the bitline capacitance

      • Like with conventional DRAM, this can be potentially compute-dense, but required some modification to read out values non-destructively

    • Flash - Accumulating charge along a single bit line through non-volatile cells, effectively computing the entire parallel MAC operation natively.

      • This is perhaps the most “brain like” structure, but is very sensitive to device variation and array size as line parasitics matter more.

      • Note that flash looks like a current domain device below, but can be made charge domain by turning on the transistor with a highly precise, strict time window with one of the inputs I described.

• Current Domain

  

• Also know as resistance based memory, this memory exploits the resistance characteristics of non volatile memory to act as analog variable resistor.

• The resistance are programmed using sophisticated methods well beyond the scope of this post.

• When a voltage is applied to these lines, a current accumulates at the output, which is then processed and converted to the digital domain.

• Embedded non volatile memory can potentially be a high density option, but the memory devices themselves are subject to inconsistent resistance across devices that affect SNR, as well as typical memory nonidealities such as write endurance and speed.

  

• Examples of such devices include:

  

  • RRAM - consists of a a top and bottom electrode with a thin oxide layer between. This devices exploits oxygen vacancies to create conductive filaments.

    

  • Phase Change Memories (PCM) - One of the most extensively researched technologies, PCMs stores data by passing precise electrical current pulses through a chalcogenide material (almost like a flexible glass) to alter its atomic structure between a Crystalline Phase and Amorphous Phase.

    

  • MRAM - Magnetic RAM consists of two ferromagnetic layers with a fixed and a free layer. The orientation between the two layers affects the resistance.

Scaling Challenges with Analog CiM

Read more

While a lot of attention is being drawn to the current HBM4 shortage, this post will focused on the practical technical challenges in scaling HBM5 performance.

I will discuss the following signal integrity and co-design challenges in HBM5 based on SK Hynix and KAIST’s ECTC 2026 papers:

• Introduction to Signal Integrity trends from HBM1 to HBM5

• Jitter component tree and descriptions of key sources of jitter for short, high-speed interconnect:

  • Echo

  • SSC / SSN

  • PSIJ

• How SK Hynix quantifies transmission line effects in high-speed HBM5 data lines

  • Fundamentals of Transmission Lines

  • Proposal for two operating regions: RC and LC dominated regimes

  • 🔒Design of Experiments results for echo jitter across interconnect lengths and dimensions

• 🔒KAISTs SI/PI co-analysis framework for Chiplet (UCIe)-based GPU-HBM Interconnect for PSIJ

  • 🔒Shoreline density limitations for a 4 layer wide-IO and chiplet interconnect scheme

  • 🔒Calculating power-supply induced jitter (PSIJ) from SI, PI, SSC, and jitter sensitivity / amplification

• 🔒BONUS: “The Brain” - 3D Custom HBM and current status among TSMC, SK Hynix, NVIDIA, and Samsung from ECTC 2026

This post is heavily influenced by the ECTC session 11, “Signal Integrity Design for High-Speed Interfaces” as well as my experience at DesignCon 2026 I was personally in attendance for.

This will be more of an advanced deep dive, but grounded in fundamentals. I structured the post to start with baseline EE concepts and building a framework to understand the complexity of multi-layer SI/PI co-design. I have a post that covers the fundamentals of signal integrity that forms the foundation of the issues I discuss:

I encourage you to read through my backlog of knowledge and challenge you to take away one new in-depth concept. Even if the technical material doesn’t stick, I still hope you come away with an appreciation of the challenges that keep HBM and SI engineers up at night.

Note that I am not a signal integrity person. However, I immersed myself into the SI world by attending DesignCon 2026 with an RF / Microwave lens upon which to analyze high-speed signal integrity effects. It was clear upon reading these papers that memory and SI engineers do not speak the same language as RF people do, but they describe the same effects from their point of view.

I personally think that at increasingly faster data rates, there will be a convergence of the knowledge among RF / microwave and signal integrity domains. I aim to bridge this gap of understanding.

SK Hynix - High speed HBM roadmaps

High bandwidth memory (HBM) is a critical component for AI training workloads that stores everything involved in AI computation from model weights, gradients, optimizer states, and activations. HBM connects multiple DRAM dies vertically using TSVs to maximize memory density in the given footprint. These DRAM dies are placed as close to the GPU as possible to maximize the data throughput to overcome the Von-Neumann bottleneck.

In this SK Hynix roadmap presented at ECTC 2026, there are a few performance scaling trends along HBM generations:

• Between HBM1 and HBM4, the datarate per DQ increased linearly from 1Gb/s to 11.7Gb/s

• Along those generations, the signal rise time and unit interval (UI) scaled inversely proportional to the total data rate per DQ

• Between HBM3E and HBM4, there is a stairstep jump in the the total bandwidth by 2.5x, primarily driven by doubling I/O per cube from 1024 to 2048.

To keep with up AI workload demands, HBM5 expects a 20-30 Gbps per DQ range using state-of-the art interposer technologies such as the Chip-on-Wafer-on Substrate (CoWoS-L and CoWoS-R).

At those data rates, HBM is running into co-design challenges along the signal integrity, power integrity, and thermal domains. At ECTC, there are three high-level trends I noticed amongst three papers:

Jitter due to Transmission Line Effects

SK Hynix notes that significant transmission line effects were not observed for previous HBM generations with a data rate of under 10 Gbps per DQ and approximately 6-mm-long interposer interconnects.

However, at 30Gbps rates, advanced packaging interconnect technologies exhibit distinct lossy transmission line characteristics and must be accounted for.

Normally, transmission lines are terminated with equivalent impedances to avoid reflections. However, termination resistors cannot be reasonably employed for terminating 1000+ I/O in HBM as this would lead to high thermal penalties and static power dissipation. This leads to additional signal integrity challenges that will be discussed.

From “Wide-and-slow” to UCIe

Another challenge is edge density limitations of I/O. Doubling the number of I/O pins results in excessively large physical layer (PHY) footprint on the silicon. The number I/O is fundamentally constrained by the metal pitch, # of layers, and the area dedicated to ground rails to control crosstalk.

To scale the number of I/O, KAIST is investigating the performance impact of going from wide-and-slow I/O toward higher-speed SerDes lanes in Universal Chiplet Express (UCIe) standards.

KAIST notes that in chiplet-based GPU-HBM based on UCIe, eight D2D modules are utilized, with each module consisting of 64 Tx and Rx at 32Gb/s each. A total of 512 Tx and Rx supports 2 TB/s for read and write directions.

With a faster, more compact PHY, more I/O them can fit in the HBM shoreline to increase the data throughput.

PDN for 3D stack

Another co-design challenge is how the TSVs in the HBM stack affect thermal characteristics.

This paper evaluates the thermal and IR drop characteristics of different TSV configurations. Here we see how each component is modelled as an array of unit cells in a grid structure with an equivalent RLC circuit model.

Though I think this paper is important for effective co-design, especially when it comes to modelling PDNs for PSIJ, I have left the analysis out of this post for now to focus on signal integrity challenges from the first two papers.

Key sources of jitter in high-speed, low-voltage parallel lines

Jitter is a key constraint of high speed signal integrity and must be accounted and minimized as much as possible.

When high speed data is sent to and from the HBM, each bit is received within a given time slot refers to as the “Unit Interval” (UI) or Bit-period. Jitter and slew rate constraints reduce this to an effective aperture “window” where data can be received. Throughout the post, all aperture results are normalized to a UI of 1.

There are many difference sources of jitter as shown in this comprehensive jitter tree from Dr. Mike Peng Li’s DesignCon 2026 tutorial slides.

Jitter is broken down into two main statistical categories: deterministic and random jitter. I describe these in more detail, along with equalization techniques, in the following post:

Under deterministic jitter, there are three primary classes of jitter relevant to high speed, massively parallel, non-terminated lines:

• Echo Jitter - Jitter caused by impedance mismatches and reflections

  • When a signal propagates down the line and encounters an impedance mismatch, part of the signal reflects backward and forwards, resulting in a stored “memory” and overshoot that affects future bit measurements. This can lead to ISI.

• Bounded Uncorrelated Jitter - Jitter that is bound but uncorrelated to the signal

  • This includes crosstalk across massively parallel lines where there are many different possible combinations of crosstalk interactions, but the overall jitter impact is “bounded” to worst case conditions.

• Periodic Jitter - Jitter where the timing variations repeat in a cyclical pattern over time at a specific frequency

  • This includes simultaneous switching noise that occurs every switching period.

One source of jitter that is particularly problematic at increasing data rates is power-supply induced jitter (PSIJ) which is a form of power-noise-induced timing degradation within circuits themselves. This is a challenge to model because it combines jitter from a few sources.

The rise / fall time and propagation delay of a circuit depends on how quickly FET caps can get charged by the PDN. When gates are switched, they generate simultaneous switching current (SSC) that go through the PDN and cause simultaneous switching noise (SSN) on the line. Noise on the PDN can affect the current your caps charge at, introducing timing uncertainty.

PSIJ becomes more problematic as you stack and cram more HBM on the same PDN and the PDN line becomes noisier. With increasing data rate, thousands of I/O drivers switching simultaneously, and a low supply voltage, a marginal increase in PSIJ can eat into the jitter budget and lead to signal failures. A co-design approach to modelling PSIJ will be discussed in more detail behind the paywall.

How SK Hynix must incorporate transmission line effects in high speed HBM data lines

Now lets discuss some of the high-speed SI challenges SK Hynix is facing from transmission line effects. I’ll start with a broad introduction of transmission effects from an RF point of view, the nuances as it relates to SI, then jump right into SK Hynix’s analysis after the paywall.

From Lumped to Distributed Abstraction

Most circuit analysis (like V = IR) is initially taught in the “lumped” abstraction where component values are “lumped” or simplified into devices consisting of resistors, capacitors, and inductors. Lumped abstractions form the basis of most traditional circuit analyses like nodal, mesh, etc.

The important thing about a lumped abstraction is that the propagation characteristics of the physical connections between lumped components can be reasonably neglected for most practical applications if the frequency is low enough.
When the frequency is low enough, the signal level will be virtually the same across all lengths of the wire since the capacitors distributed throughout the line charge nearly instantaneously.

However, at high frequencies, the signals have to physically propagate down the wires to get from point A to B, so the physical characteristics of the interconnects are important. This is referred to as the “distributed” abstraction where the electromagnetic wave propagation characteristics along a transmission line matter. Notice how in the figure above, when a high frequency wave is sent down a line, its voltage rises at different time points.

When deciding on whether to model circuits with lumped or distributed elements, the rule of thumb I was taught in RF classes is that:

For interconnects that are approximately 6mm and an er of 4, this corresponds to a frequency of 2.5 GHz. Its clear that HBM data rates are running into this regime.

In transmission line theory, when a wave is sent down a line with a characteristic impedance Zo, there needs to be an equal impedance at the termination for the signal to “match” to, or else the wave will reflect backwards. The reflection coefficient quantifies this effect and its formula is given by:

When the impedance is NOT matched, the reflection coefficient is not 0. Depending on the extend of impedance mismatch, a portion of the wave that makes it way to the load is then reflected backwards and forwards along the line in a time delay T_d, decaying over time.

For an open ended or high impedance circuit, Z_L is infinite and the reflection coefficient is 1. This means that for high frequency RF signals, all of the propagated signal gets sent backwards as there is nowhere it can terminate. This is shown as the “stairstep” function in figure 8.

From the source POV, the effective impedance looking into the line depends on the length of the line between the source and termination. An impedance on a smith chart can be rotated depending on the integer multiple of wavelengths to find the equivalent impedance “looking” into the line. A special case (commonly asked in interviews) is λ / 4 lines that “transform” a open ended impedance to a short circuit looking into the line.

Though RF has a highly developed body of broadband analysis techniques such as smith charts, there are a few caveats as it applies to signal integrity:

• The data is digital data are ideally square waves with rise/fall times and odd mode harmonics, not broadband sine waves. When a signal goes high at the source, its value stays high at a fixed level until the next time period.

• The interconnect resistance is fairly high (1’s to 10’s of ohms) compared to the characteristic impedance. In most lossy RF transmission lines with enough board space, the resistance will usually not be this high. The only reason its high for HBM is for the tiny wires they need for the massive parallelism.

• In RF theory, the characteristic impedance is commonly standardized to 50 ohms and the source impedance matches this. However, matching the source impedance to the HBM interconnect impedance is not practical because changing Ron can fix one problem but create others:

  • Increasing Ron to impedance match will cut the launched voltage is half and charge the line slower, decreasing slew rate

  • Decreasing Ron would require large width transistors that may cut into edge density

  • Therefore, SK Hynix selects a Ron of 14 ohms for a characteristic impedance of 34 ohm to balance the tradeoffs of layout area and slew rate. This results in a reflection coefficient of -0.41 at the source end.

In short, for 30Gb/s data rate per DQ HBM5, SK Hynix is dealing with a tricky combination of high channel resistance and transmission line effects that can’t be easily analyzed using standard microwave techniques, and thus have to define their own set of evaluation criteria for high speed signal integrity. These evaluation criteria involve classifying the mode the operation into two distinct regions:

• RC Dominated

• LC Dominated

Share

After the paywall I will discuss this methodology along with KAISTs co-design framework in more depth. In the meantime, checkout a few of my other posts.

Side note: I saw a crazy idea at ISSCC 2026 where the data between GPU and HBM was…

Read more

• Motivation behind Optical Interconnects

• An Overview of GeSi Electro-Absorption Modulator (EAM)

  • Franz-Keldysh effect

  • Comparison with other external modulation options

• Why Component Level Test Vehicles are Needed to Reduce Overall Integration Risk of 3D Heterogenous Integrated Packages

• An Introduction to Marvell’s Photonic Link: A Fully Integrated Solution for XPU, CPO, and HBM

  • 🔒Marvell / Celestial OMIB: Overcoming Edge Density Constraints

  • 🔒Photonic Fabric Test Vehicle of EIC, PIC, and FAU w/ measurement results

  • 🔒Example 3D Packaged Application: Photonic Fabric Memory Module

• 🔒Key Co-design Integration Challenges

  • 🔒Thermal Management

    • 🔒EAM Thermal Stability

    • 🔒Package Thermal Steady State Characteristics

    • 🔒Package Thermal Transient Characteristics

  • 🔒Power Delivery Challenges

• 🔒How NVIDIA DWDM and Marvell’s EAM technologies can be powerful together

The research from this post comes from various Marvell’s papers from ECTC and their ISSCC forum presentation. I also incorporate relevant fundamental material from ISSCC short courses as well when developing the theory.

This post is aimed at more advanced understanding of some of the cross domain integration challenges that EAM is facing. Fortunately, I provide several references to several of my posts on advanced packaging, optical communications, and power architecture for all knowledge levels to understand the underlying fundamentals of Marvell’s Photonic Fabric Technology.

Before writing this, I wrote a similar deep dive on NVIDIA DWDM summarizing the challenges from both their ISSCC and ECTC papers, having attended both conferences myself. These posts come at a timely moment when NVIDIA recently announced they invested $2B into Marvell and Jensen announced at Computex that Marvell will be the next $1T company, causing the stock to soar.

I recommend you read my other NVIDIA post to put GeSi into its proper context and to read between the lines behind that stance:

Motivation behind Optical Interconnects

Scaling AI compute has many different bottlenecks: power delivery, thermal, and high speed interconnect density. Industry standard interconnects such as PCIe have limitations for the workloads AI compute demands. As a result, there is demand for high bandwidth, long reach interconnects that makes optical communications such an attractive option.

Within the optical domain, lasers themselves can be directly modulated (such as VCSELs) or light coming from lasers can be externally modulated. There are three primarily ways to externally modulate a light in the optical domain: Mach-Zehnder Interferometers (MZI), Ring, and Electro-Absorption. MZI modulates the phase of light travelling through two arms to cause destructive interference when recombined, and ring modulators act as band-stop filters for specific wavelengths of light.

I discuss these options in the following post:

GeSi Electro-Absorption modulators

EAMs operate by optical absorption of light through an electrically-controlled PIN junction at a single wavelength. A PIN Diode is a PN junction diode with an undoped intrinsic region to alter the PN diodes characteristics for the desired application, such as:

• High-frequency RF switches / attenuators (Si, GaAs, InP),

• High voltage tolerance by lowering the capacitance and thus increases breakdown voltage (GaN, SiC),

• Photodetectors by converting optical signals to electric current through generation of Electron-Hole pairs (InGaAs, Ge)

In optical communications, EAMs device have a high bandwidth and a fairly compact footprint for CPO applications.

EAMs have a property called the Franz-Keldysh effect which is where the optical absorption in a semiconductor changes when an electric field is applied. This is shown as the black curve above shifting to the red curve when a bias voltage is applied. This allows a fixed wavelength light to be externally modulated at high speeds by quickly switching the absorption curve at that wavelength to affect how much light goes through.

As shown in the equations above, the extinction ratio can be calculated as the ratio of the exponentials of the absorption factors of the two states, and the insertion loss can be calculated as the exponential of the absorption at 0V. Higher extinction ratio and lower insertion loss is desired.

GeSi is a common material used for the intrinsic region. The GeSi is epitaxially grown on top of a flat surface that is polished with CMP. Light is routed through the GeSi which has a higher refractive index than the surrounding SiO2, thus causing total internal reflection through the device.

One common FoM is the the transmitter penalty (TP) which is a measure of the loss due to the modulation itself and combines both the IL and ER:

Here we see some example characteristics of Marvell’s EAM IL over wavelength over bias voltage and different temperatures. High temperature causes the IL to shift up and affect the absorption that each signal level sees.

This means that temperature stability and thermal management is crucial to ensuring that EAM devices can operating consistently within its intended environment. Sophisticated mixed circuit control circuitry is necessary to anticipate and correct for rapid temperature swings in the GPU and package.

While ring modulators face challenges with precise thermal control and MZI are quite large, EAMs offer a promising “middle range” alterative in high performance CPO applications. Advantages include:

• Relatively small size

• Good thermal stability

• High speed

• Low power

• High optical bandwidth

However, there are a few downsides:

• EAMs struggle from power handling because they absorb light, not phase shift it. There is an almost guaranteed 3dB loss from light passing through the EAM that must be accounted for in the link budget.

  • I believe the link budget is an extremely important concept that anyone investing in or designing optics needs to know because it affects which components in the supply chain can be successfully integrated in real systems. I dive more into link budgets here in the free section:

• EAM devices are non-linear that, if uncorrected, affects the size of the individual eyes for PAM4 where the noise margin is governed by the smallest one.

  • To combat this, pre-distortion circuits are needed to pre-distort the four voltage levels so that the signal can be sent through the channel and recovered as linearly as possible.

• EAM devices are difficult to reliably manufacture at scale. Growing the GeSi epitaxially requires a fairly flat surface and sophisticated means to control the crystal growth. High volume testing is needed to screen for KGD.

In short, GeSi offers several promising benefits for CPO, but does suffer from inherent device level challenges as well as circuit complexity challenges to control the characteristics of the PAM4 signal.

Why Component Level Test Vehicles are Needed to Reduce Overall Integration Risk of 3D Heterogenous Integrated Packages

There are several excellent research works being performed across the industry and academia on components that comprise integrated 3D packaging solutions:

• Different substrates: organic, silicon, glass

• Different bridge option: LSI, embedded bridge

• Interconnect options: C2, C4, and Hybrid bonding

• Different integration options: 2.1D, 2.3D, 2.5D 3D, 3.3D, 3.5D.

• Different optical component integration and waveguide routing options in each substrate

Ideally, each component (whether that be chip-level IP or mechanical components) should have clearly defined datasheets (such as loss over frequency, CTE, etc) with validation/yield data backing it up to minimize final assembly risk. When system integrator components buy parts, they do a competitive analysis by reading multiple datasheets, selecting a part, and expecting the device to perform within those given operating conditions. Hybrid Bonding is an example where that is NOT the case, and is only used in specialized cases behind NDAs.

However, the current AI market is demanding innovative higher performance components that don’t have defined datasheets yet. Its not always possible for promising solutions to be fully vetted before they can be used, because by then the next higher performing part is right around the corner.

For tightly integrated AI Compute packages, innovative yet unproven components can cause integration risk. Parallel optimization efforts of innovative solutions are needed to make sure that block performance improves along with the whole system.

One major issue are the cross-domain interactions across interfaces. EAM have thermal stability challenges between the optical and thermal interface, and high speed signals can suffer from Power-supply induced Jitter (PSIJ) between the power and signal integrity interface. These cross domains interactions can be tricky to anticipate if each component is optimized in isolation from each other and not caught early on.

To minimize final integration risk, new IP is proven out on test chips or demonstration vehicles on smaller subsystems. These IP provide system integrator companies valuable data to assess integration risk and provide several options to choose from. Major tech companies often identify several “fallback” options of sockets within their product, whether that be consumer electronics or hyperscalers, so that they can protect their supply chain from one bad component and minimize of risk of integrating new ones.

I say this because I have many war stories of scaling superconducting computing from standard cells to VLSI integration, where tapeouts were mainly structured around integration risk reduction: before using newly invented logic gates in higher level P&R flows, they need to be taped out on test chips with data to show that they are at least functional. They might not be fully optimized to “gate” usage at high level assemblies, but optimization can be done in parallel alongside tool development.

Though it sounds exciting from the outside, its not always a fun process on the inside as it can feel chaotic. In hindsight, you do learn a lot in the process.

Marvell’s Photonic Link: A Fully Integrated Solution for XPU, CPO and HBM

Now I will comprehensively review Marvell’s/Celestial AI’s approach to CPO.

Marvell is trending toward 3D integration with the EIC stacked on top of the PIC with a silicon bridge connecting XPU and PIC. Their Photonic Fabric Approach has the following three components:

• PFLink™: a chiplet optical link that supports supporting UCIe-A interfaces with 14.4 Tbps BW and reach > 50 meters.

• PFSwitch™: a system-in-package switch enabling all-to-all XPU connectivity.

• OMIB™: an optical multichip interconnect for large package integration.

However, this approach presents several co-design challenges along several domains:

• Packaging Real Estate: One issue with CPO is the edge density (or Beachfront) of the ASIC. In AI Compute heavy GPUs, HBM, CPO, and the power delivery network want to be as close to the ASIC as possible to minimize loss in high-speed and high-power signal. This creates a real estate tradeoff of which components should remain close to the ASIC depending on what is being optimized. GPUs need HBM as close as possible for training AI models, while CPUs and other ASICs may not have as strict memory requirements.

• Power Integrity: Power needs to be delivered to the I/Os in the die interior. This requires enough power density and a low enough impedance over the frequency range of expected transients. Vertical power delivery is proposed to support this, but there needs to be enough space for TSVs and decoupling capacitors close enough to the ASIC to provide the power density at a low impedance at the transient frequency.

• Thermal Stability: Steady state and transient thermal simulations are needed to quantify the thermal proximity of the characteristics of the EAM, as well as how the EAM characteristic shifts in response.

After the paywall, I will dive deeper into Marvell’s 3D Photonic Fabric implementation and solutions to these three challenges from their ISSCC forum and ECTC papers. I’ll also cover a slide that caught my eye in how NVIDIA DWDM and Marvell EAM can potentially synergize with each other.

Share

Here are several of my other posts that give you a fundamental understanding of the key tradeoffs in each of these areas:

Marvell / Celestial OMIB: Overcoming Edge Density Constraints

Read more

This post will be an advanced deep dive of a state-of-the-art co-packaged optical system based on NVIDIA’s Dense-Wave Division Multiplexing (DWDM).

This post is organized in the following sections:

• What is Dense-Wave Division Multiplexing?

• A complete architecture breakdown of NVIDIAs DWDM with key physics constraints:

  • PIC - Ring modulator physics

    • Ring coupling theory - splitter and phasor theory

    • Key ring properties

    • How PN junctions shift the resonant frequency by changing the refractive index

    • Nonidealities - Thermal sensitivity and crosstalk

  • EIC - Clock forwarding and jitter analysis

    • Conventional clocking techniques - Forward clocking and embedded clocking

    • NVIDIA’s clocking architecture - half rate DWDM link architecture

  • 🔒 A Breakdown of the DWDM CW-DFB Laser Array and performance results

  • 🔒 Overlay of two critical spectrums that illustrate precise alignment challenges:

    • 🔒 NVIDIAs MRM transmission spectrum from the ISSCC paper

    • 🔒 NVIDIAs DFB Laser Array power spectrum from the ECTC paper

  • 🔒 A brief high-level overview of key CPO packaging features with TSMC COUPE

DWDM is fascinating because it blends together the knowledge amongst different specialties who all work together to create a successful product:

• The on-chip mixed signal SerDes designers designing the high speed TX and RX circuits

• The material scientists/solid state physicists quantifying the behavior of PN junctions and SiPh materials

• The packaging engineers who integrate everything together

The primary sources of this post are two NVIDIA papers on the circuit architecture and packaging aspects of DWDM from conferences I attended the sessions for in-person. These are:

• ISSCC 2026: “A 32Gb/s/λ 256Gb/s/Fiber Half-Rate Bandpass-Filtered Clock-Forwarding DWDM Optical Link in a 3D-Stacked 7nm EIC/65nm PIC Technology”

• ECTC 2026: “Design and Packaging of a DWDM CW-DFB Laser Array for Co-Integrated Optical Interconnects”

I also integrate relevant material from relevant ISSCC forums and short courses as well.

I find that one of the best ways to learn more advanced concepts is to work backwards and figure out what simpler prerequisite concepts you need to learn first. The free half covers a huge technical breadth of material with links to several of my fundamental posts I wrote leading up to this. This way, people of all skill levels can work backwards and figure out what they need to learn to understand more complex concepts.

I want to challenge you to try to learn one new concept in depth from this post or my body of work, even if everything else doesn’t make sense.

Please note that I do not consider myself a photonics expert. I wrote this post using the sources listed above as sources-of-truth to base my commentary off of. If you’re an expert familiar with this material and notice a mistake in this post, please reach out to me so I can have this promptly corrected.

For a broad overview of optical communications, including VCSELs, optical modes, and key impairments, I recommend reading this post where I dive into these fundamentals in the free section:

What is Dense-Wave Division Multiplexing (DWDM)?

Optical interconnects in co-packaged optics help overcome the energy dissipation and frequency scaling challenges inherent in copper interconnects. Co-packaged Optics moves the optical engine as close to the chip as possible to minimize the length the electrical signal travels before being converted to the optical domain.

Optical communications enable low-energy and low-latency data transmission. However, optical communications is limited by a few constraints:

• The physical edge density. The optical signals need to be propagated through waveguides into a fiber optic connector. How and where the signal flows from the laser source to the electrical devices matters a lot.

• Frequency constraints of individual devices. The electrical as well as the optical bandwidths needs to be fast enough to handle the data throughputs needed.

Several companies propose solutions that push all available scaling dimensions. One of the most promising solutions to scale data speed involves densely multiplexing light signals of different wavelengths on the same waveguide.

Dense-Wave Division Multiplexing consists of individual optical channels of equally spaced wavelengths. Electrical signals are frequency multiplexed in a waveguide through a series array of micro-ring modulators. Ring modulators act like on-off switches for each of their respective channels. Ring modulators are ultra-compact, enabling many channels to be packed into a common waveguide.

Pros and Cons of DWDM

Compared to other modulation options, DWDM has several benefits:

• Scalability. DWDM performance offers scaling potential with the # and size of devices.

• Low latency for XSR due to no DSP

• High energy efficiency

• High bandwidth density

However, there are several challenges with DWDM that involve the material physics aspect of these devices:

• Temperature sensitivity. Lasers and MRMs are inherently sensitive to temperature and must be well controlled

• Design Complexity - DWDM moves a lot of the complexity from the electrical domain to the optical domain, and imposes unique clocking and jitter tracking challenges at high speed

• Laser Integration - DWDM needs a multi-wavelength laser source that has to be brought onto the chip

  Silicon Co-Design is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Architecture Breakdown - EIC, PIC, Laser Array, and Packaging

Next I’ll discuss NVIDIAs DWDM multiplexing architecture. I’m going to describe the physics of the MRM on the PIC first to show how it works at a fundamental level. I’ll then discuss the clocking scheme of the EIC and show how the PIC requirements influence the EIC requirements.

At an architecture level, the job of an optical transceiver is to make sure that the data sent through an optical laser is received by a photodetector on the other end with acceptable BER. When selecting components, link budgeting is performed to take into account all sources of losses and optimize the performance of the entire system. I discuss link budgeting in the free section of this post:

PIC - Ring Resonator Physics

In a nutshell, microring modulators (MRM) accept or reject specific wavelengths of light going through a nearby waveguide. MRMs couple evanescently to nearby optical waveguides and act like band-stop/pass optical filters, depending on the way you look at it. Light is normally guided through an optical waveguide and passes right through, but the MRM acts like a low impedance for light at specific wavelengths.

MRMs consist of the following layers:

• A circular optical waveguide (grey) that is “sandwiched” between two regions:

  • p-doped region (blue)

  • n-doped region (red)

• A heater (green) that surrounds the MRM to regulate the temperature

A voltage source is attached to the n-doped region (yellow) to switch the optical signal going through the waveguide by shifting the resonant frequency over.

Lets dive into each of these elements in more detail.

Ring Coupling - the Splitter

The ring can be thought of as an splitter with two input and two output terminals. . Think of the ring like a “race track” that the coupled signal enters, runs laps around, and returns back onto itself until it ultimately decays due to losses.

In this splitter network, there are two constants that define the input-output characteristics:

• t is the transmission coefficient and represents how much of the signal gets through. Ideally, t should alternate between 0 and 1 at specific wavelengths.

• k is the coupling coefficient and is affected by how closely spaced the ring is to the optical line.

By conservation of energy, t^2 + k^2 = 1.

Once the optical signal enters the loop, it experiences loss and phase shift represented by a(λ). If the ring is unwound, the path of the light is represented by the phasor:

where the “a” term factors in the loss factor α and the propagation constant β that represents the wavelength-dependent phase shift:

The transmission formula is given by the ratio of the light that makes it through the optical waveguide vs the light coupled into the ring across wavelengths, which is given by:

Through substitution, as well as simplifying the phasor into a cosine, we arrive the following:

This formula lends itself to the following properties:

Rings Resonance Frequencies

In a microring modulator, the ring resonates when the diameter is equal to an integer number of wavelengths. These wavelengths is given by

where

• n_eff is the effective refractive index

  m = integer modes (1,2,3…)

  r = ring radius

Due to the periodic nature of the ring, the resonances are contained within a free-spectral range that limits the wavelength window signals can couple into it. Any signal outside of this window will alias into it.

One important property of the ring is the Full Width at Half Maximum (FWHM). This is the “width” of the resonance measured at half the max power (or 3dB on a log scale) at both edges of the resonances. FWHM is a useful qualitative reference to measure how “wide” the resonance is. This is given by:

The quality (Q) factor describes how narrow the resonance width is compared to the frequency it is centered at. This is given by:

where

• c is the speed of light in vacuum

  τ is the cavity photon lifetime that lumps in the above parameters

In NVIDIA’s implementation, the TX ring modulator quality factor Q is ~4.5K, and RX filter Q is ~4K.

In order to switch electrical signals into this ring, the ring bandwidth needs to be wide enough. The ring bandwidth is given by

where

• R is the ring junction resistance

  C is the ring junction capacitance

NVIDIA switches into each ring at 32Gbps, corresponding to a switching frequency of 16GHz.

PN Junction shifts the frequency by changing the refractive index

PN junctions normally act as electronic diodes that, electrically speaking, act as one way current flow with a small built-in voltage drop (~0.7V).

In conventional PN junctions, the p side is doped with type III elements (such as boron, gallium, or indium) with free holes in the valence band, and the n side is doped with type V (such as phosphorus or arsenic) atoms with free electrons in the conduction band. When a p and n material are sandwiched together, it creates a depletion region of “built-in” electrical potential due to the flow of carriers between the materials.

However, when PN junctions are used in SiPh, we care about the solid state physics properties that affect the index of refraction. In optical, these junctions are biased in reverse bias. As more voltage is applied to the n-doped region, the depletion region widens. This lowers the carrier concentration, causing the refractive index of the silicon to spike upward. This shifts the resonant frequency and moves it just enough to allow a light source to pass through. Think of this operation like a door opening and closing really fast.

In fact, these PN structures can be altered and routed in series with the optical signal itself. Distributed PN structures are used in Mach-Zehnder modulators that modulate travelling waves along long lengths and unique structures such as p-i-n are used in electro-optic modulators. Each structural variation has their own unique characteristics and tradeoffs. I discuss these options behind the paywall in this post:

Nonidealities

I’ll discuss two nonidealities of MRMs: Thermal sensitivity and crosstalk.

Thermal Sensitivity and Tuning Loop

Ring modulators are sensitive to temperature primarily because of the thermo-optic effect. The thermo-optic coefficient of silicon is ~1.8 x 10⁻⁴ K⁻¹ and changes in temperature affects the index of refraction.

Here, the thermal tuning loop measures the current at both TX and RX through:

• A drop-port photodetector (PD) at the TX to sense how much optical power is leaking out of the ring

• The TIA DC-loop at the RX

The PD current value is first digitized with an 8 bit SAR ADC, then processed digitally to produce a pulse density modulated signal to drive an integrated metal heater.

The paper doesn’t give too much detail beyond this, but I would imagine the ADC/DAC precision and loop bandwidth needs to be acceptable to correct for the expected temperature fluctuations with an acceptable wavelength error. I also imagine each individual ring can be tuned for the specific resonance offset.

I dive into depth regarding the SAR ADC in this post:

Crosstalk

When multiple microrings are in close proximity to each other on the same substrate, cross talk is a key first-order phenomenon. The frequency characteristics of microrings tend to have long tails that poorly suppress sidebands and leak into other rings as well.

When deciding on what the Q factor should be, there is a tradeoff between crosstalk and the bandwidth:

• A higher Q lowers crosstalk and bandwidth

• A lower Q increases crosstalk and bandwidth

Modelling crosstalk is not that intuitive since there are many crosstalk paths among different rings that occur at different wavelengths.

Summary

Ring modulators appear simple at first but are highly affected by material properties and their tightly coupled nature. They are subject to device-to-device variation, so appropriate tuning circuits and control loops are needed to account for this, adding additional design complexity.

EIC - NVIDIA’s half rate DWDM link architecture

Now I will talk about the electrical side, more specifically the clocking architecture.

Of all the topics I’ve written about, I found that clocking is perhaps one of the trickiest subjects to explain in depth. As a simple analogy, think of clock synchronization like an orchestra conductor making sure all the instruments in the orchestra are in sync with minimal variation in when notes are played. There are many outside sources that can affect each individual musicians rhythm.

Each ring modulator is driven through their own SerDes electrical circuit. Compared to standalone high speed SerDes, each ring’s SerDes is slower and doesn’t have the DSP due to the XSR nature, easing some complexity. However, parallelizing 8 data lines requires synchronization and jitter control that adds additional complexity to the clock network.

Lets break down NVIDIA’s clocking architecture in more detail and see if we can make sense of this.

Conventional Clocking Techniques

When digital data is transmitted through a channel, there needs to be some way to clock the data at the RX to recover the data. There are two conventional ways of doing this:

• In a forward clocking scheme, the clock is sent alongside the data in a separate channel.

• In an embedded clocking scheme, the clock is “integrated” into the data stream. The data stream is serialized in a way that ensures enough voltage transitions (edges) occur to represent the timing information. This clock is then recovered at the receiver side through a clock data recovery (CDR) circuit.

I discuss the conventional circuits used in CDR circuits (such as bang-bang phase detectors) after the paywall in the following post:

No matter which scheme is used, each channel will couple in noise that appears as clock jitter, which is a deviation of the clock edge from its ideal location. There are two types of jitter:

• Correlated jitter - this is jitter that is generated from device operation such as TXPLL jitter and supply noise induced jitter. This jitter is identical lane to lane.

• Uncorrelated jitter - this is jitter that is random in nature, such as thermal, flicker, shot, etc, and is mostly generated through devices. This jitter is different from lane to lane.

Jitter show up as “tails” in the frequency content of a high speed signal. If jitter is high enough, it can degrade BER from the value being outside the threshold used to read 0 or 1. I discuss more about jitter in this post:

When clocking parallel channels, it is important to track the jitter and filter out unnecessary sources of it. In general, it is easier to track correlated jitter than uncorrelated jitter, and filter out low frequency sources of jitter.

There are a few pros and cons of each scheme:

• Forward clocking has excellent jitter tracking bandwidth, but requires a separate clock line. Additionally, uncorrelated jitter can accumulate.

• Embedded Clocking saves on IO, but CDR circuits have the potential to be exposed to jitter beyond its limited bandwidth.

DWDM has the unique benefit of embedding the clock in as a separate wavelength along the data lines, but is does come with a a few unique challenges:

• The data at the TX needs to enter the rings synchronously

• When data is sent down through the optical fiber, there will be delay mismatches due to various non-idealities such as chromatic dispersion and polarization mode dispersion.

• The thermal noise-induced jitter is dominated by the transimpedance amplifier (TIA) at the RX

Lets discuss what NVIDIA does to overcome these challenges.

NVIDIAs clocking architecture - half rate DWDM link architecture

NVIDIA uses a half-rate FC DWDM link architecture that embeds the clock inline with the data. In this architecture, there are 9 rings on each of the TX and RX that combines for a total data throughput of 256Gbps:

• 8 rings are for data at 32Gbps each

• 1 ring is for FWDCLK at 16GHz

On both the TX and RX side, the signal will traverse different lengths depending on which ring it coupled into. As a result, the TX clock needs to be de-skewed to account for known path length misalignment.

At the RX end, the transimpedance amplifier (TIA) adds a lot of thermal-noise induced uncorrelated random jitter. This TIA noise primarily comes from the feedback resistor (circled above) which is necessary to provide a low impedance input from a current sourced photodetector into the TIA.

To overcome these jitter challenges, NVIDIA uses an Injection locked oscillator (ILO) - based bandpass filter to bandpass filter the clock with BW ≈ 1-2GHz. This filters out jitter beyond 1GHz to remove most of the uncorrelated jitter and tracks most of the correlated jitter.

This clock is then distributed to the RX as illustrated by one example setting of the RX CLK distribution. Each RX has a local ILO that acts as another jitter filter. The clock distribution is segmented into two distinct spinal wires CLKA and CLKB that cut the capacitive load per wire in half. The RXPLL acts as a on-chip frequency synthesizer that provides the local timing reference upon startup until the system is locked to the main timing source.

I write about phase-locked loops in more depth:

DWDM Laser Source and Packaging of EIC and PIC

After the paywall, I will cover the packaging part that represents a major constraint in the physical realization of this system. I will cover the following:

• A breakdown of the DWDM CW-DFB Laser Array and performance results

• A brief high-level overview of key CPO packaging features with TSMC COUPE

In between these two discussions, I will overlay NVIDIAs measurements of the ring spectrum from the ISSCC paper with the DFB Laser Array from the ECTC paper, and discuss the challenges with aligning both of these. Alignment between these two spectrums is crucial across multiple devices because misaligned channels don’t transmit data.

Here are several other posts I’ve written that cover the fundamentals of advanced packaging in-depth:

Share

Read more

At ECTC 2026, TSMC noted that many package designers are being too aggressive with their package designs and that a lot of communication is needed to balance the design performance with the manufacturing and reliability constraints of the package.

This post helps reduce that communication overhead.

In this post I’ll be diving into the key mechanical and thermal factors needed for co-designing advanced packaging for high power. The outline of this post is as follows:

• I will explain the current I-shaped paradigm in most technology companies and why engineers will need to embrace T-shaped skillset to effectively co-design tightly integrated systems

• I’ll discuss the key challenges that GPUs are placing on both the package and circuit side

• Then I’ll discuss two fundamental concepts and associated material properties for successful co-design of sub-components with advanced packaging:

  • Thermodynamics, including the three sources of heat transfer and thermal resistance

  • Stress-strain and associated material properties such as CTE

• Then I’ll describe the:

  • 🔒Three main sources of warpage

  • 🔒Four common failure modes

  • 🔒Challenges with modelling cross coupling failure modes

• 🔒Last, I’ll discuss a few warpage mitigation options

The research from this post comes from both from my conference experience at ECTC and the book “Hybrid Bonding, Advanced Substrates, Failure Mechanisms, and Thermal Management for Chiplets and Heterogeneous Integration” (2025). I found this book to be a useful mix of both high-level fundamental overviews of advanced packaging and current in-the-weeds academic research on key areas related to packaging.

I previously wrote a masterclass post on the fundamentals of advanced packaging that covers what packaging “is”. This post builds off that by covering the physics dimension when it comes to CTE, warpage, and failure modes. I recommend you read that post along side this one to build a more complete view of advanced packaging.

First I want to describe what I believe effective hardware co-design requires.

A New Approach for Hardware Co-Design: The T Shaped Engineer

Currently, most engineers at major tech companies operate in the “I” shaped paradigm where engineers are highly specialized in their field of choice, and interface with other engineers through requirement specifications and handoff at interfaces.

As systems start become more tightly coupled across domains, many leaders are calling for more cross domain awareness and co-design. However, I believe that most engineers in the “I” shaped paradigm treat co-design as more frequent and earlier meetings.

I believe that effective co-design requires you to become more ‘T” shaped, that is, depth in one domain and breadth in others. This requires you to build a semi-independent knowledge base of high level concepts in other domains to be able to intuitively assess how factors from your domain affect others. This allows you to anticipate issues and ask key questions from domain experts early on.

Becoming T shaped is actually quite difficult to do in an I shaped paradigm, both cognitively and organizationally. Vikram Sekar described these challenges best when he gave career advice because often times, it might be possible to “step” on adjacent teams who are CTA.

I totally understand this, and I’m not saying you should ignore your orgs politics. In any case, here is my blueprint for people who want to become more “T” shaped:

Ground yourself in fundamental tradeoffs in silos adjacent to your domain of expertise

Effective co-design across multiple domains requires you to first ground yourself in the fundamentals of domains that relate to each other in some way. This requires deep work to analyze the technical fundamentals in domains adjacent to your own.

For me, the bulk of my co-design posts revolve around four interconnected domains in hardware:

• On-chip mixed signal (SerDes architecture, ADC, PLL)

• High speed communications (Optical, high speed wireline, and signal integrity)

• Power integrity / management

• Advanced packaging

Each specialty has their own unique set of tradeoffs, such as:

• Gain vs bandwidth in Analog

• Interconnect distance vs BER in SI

• Switching frequency vs ripple vs loss in Power

• Substrate area vs bump stress in Packaging

Understanding the fundamentals in each domains is not an easy process because it requires to you sort through domain specific knowledge from experts and try to extract out the important insights. I found that the tutorials at the beginning of the conferences are excellent at providing a framework to understand more complex conference material that build upon the tutorial material; these are what the tutorials are specifically made for. That way, when I read dense technical papers or conference digests, I can efficiently filter through information that doesn’t matter. I find that the bulk of the writing in conference papers is really dancing around the nuances of previous works while making claims of proposed solutions.

I’m not saying you should become an expert in domains outside of your own; that is overkill for practical purposes. However, I do think you should work backwards to figure out what baseline knowledge you need to understand more domain-specific knowledge from other people.

Understand how adjacent domains couple together

Then, after understanding tradeoffs in each domain, try to understand how effects from each domain “couple” to other domains. Based on my previous writing, these are some interactions I can think of off the top of my head:

• Power supply induced jitter and crosstalk lower BER of the signal path

• Fast current transients in AI workloads cause localized joule heating that can worsen reliability

• Ring modulators in optical communications are sensitive to thermals that need to be corrected with heater feedback loops

Understanding interactions is quite tricky that I admittedly don’t really understand fully quite yet and am trying to understand better myself.

My advice: start small and teach others

As a start, I recommend people first ground themselves in ONE field. Power and packaging are excellent interdisciplinary fields to start for anyone involved in semiconductors in anyways. I have several posts that provide a good starting point.

Then, to really test if you have the knowledge, try to teach it to others, whether that be through writing or a seminar. This way, others can mutually benefit from the insight in your own process of becoming more T shaped. It’s quite easy to fool yourself in thinking you know something from reading someone’s article until you go and sit down and explain that topic to another person.

The Co-Design Problem: High Power Density Stresses the Package

GPU roadmaps are projecting a huge increase in computational power that comes with higher performance requirements, including:

• High power usage. GPUs draw huge current density that is lost in the PDN and dissipates heat.

• High transient response. The critical frequency band of AI workload transients is between ~0.1-10 MHz, and the PDN needs to provide low enough impedance at these frequencies to meet the on-chip droop specs. This is typically done by placing localized decoupling capacitors as close to the chip as possible.

• Large Area. More GPUs per package increase the package area that increases the Distance to the Neutral Point (DNP).

To meet these performance demands, innovations are both needed on the package and circuit design side.

On the circuits side, much work has been done on optimizing specialized power conversion topologies such as multi-phase buck/TLVRs at the PoL and LLC DCX for the IBC. I dive deep into these fundamentals of the circuit architectures in more detail:

On the packaging side, there are several well established failure mechanisms and integration techniques. I briefly touched upon these in these posts:

The issue with high power is that they stress the package, enhance the already established failure mechanisms, and opens up possibilities for new ones. Large BGAs with up to 2000 - 6000 balls face mechanical challenges with heavy heat sinks and continuous vibration.

Lets dive deeper into important packaging considerations.

Fundamental Thermal and Mechanical Concepts: Thermodynamics and Stress-Strain

Co-design of the high power with the package involves knowledge of fundamental concepts in thermodynamics and stress-strain. These are pretty well known to people in those domains and I realize there’s a lot more depth underneath these equations I am simplifying.

As an EE myself who is more specialized on the circuit side, I’m framing these as high level mental models for people outside of these domains to have a first order understanding of the mechanical and thermal effects of packaging.

Likewise, if you are very familiar with these but not familiar with those concepts on the electrical side like signal integrity, power electronics, or the intricacies of high speed SerDes, well, lucky for you, I have mental models on those on my Substack as well that are linked throughout this post.

The Three Sources of Heat Transfer

Conduction

Conduction is heat transfer within a stationary solid, liquid or gas due to temperature differences. Energy is “diffused” from more energetic regions to less energetic particles.

Conduction is governed by Fourier’s law:

where

• q′′ is the heat transfer per unit area (W/m² )

  k is the thermal conductivity of the material (W/m K)

  dT/dx is the temperature gradient (K/m)

Common values for thermal conduction range from:

• 0.024 (air),

• ~0.2 - 0.35 (for epoxy, polymide, and FR4)

• 120 (silicon)

• 390 (copper)

Conduction matters when dissipating heat from:

• The GPU - Rubin generates 1,800W to 2,300W per GPU

• Joule heating in the interconnects - this is self-heating due to IR drop that increases as current density increases

Convection

Convection refers to heat transfer between a surface and a moving fluid at different temperatures

Convection is governed by Newtons laws of cooling:

where

• q′′ is the heat transfer per unit area (W/m²)

  h is the convective heat transfer coefficient (W/m² K)

  Ts is the surface temperature of the solid (K or ⁰C)

  Tinf is the temperature of the surrounding fluid far form the surface (K or ⁰C)

Convection can be classified either as free (such as hot air rising from a heater) or forced (such as cooling a CPU with a fan).

Common values for convective heat transfer coefficient include:

• Free: 2 - 25 for gasses and 1000 for liquids

• Forced: 25 - 250 for gasses and 100 - 20,000 for liquids

Convection cooling can be facilitated through the use of heat sinks, liquid cooling or immersion cooling with single and two phase variants.

Radiation

Radiation is the transfer of thermal energy through EM waves without a need for a medium. It is governed by:

where

• E is heat transfer rate of emission (W)

• ε is the emissivity of the surface (between 0 to 1)

• σ is the Stefan-Boltzmann constant (5.67e-8 W/m² K⁴)

• A is the surface area of the radiating body

• T_s is the absolute temperature of the surface (K or ⁰C)

Compared to conduction and convection, radiation is not a primary heat transfer mechanism for ground based AI data centers, but could potentially be a dominant heat mechanism for space based AI data centers. We’ll see how that plays out.

Note that radiation also includes the absorption of incident radiation, so the net heat transfer depends on the balance between the inflow and outflow of radiation.

Conduction and convection must be in balance for ground based AI data centers

Fundamentally, for a GPU to maintain a thermal equilibrium below a threshold temperature, the conduction and convection equations must be in balance with each other for the cooling system to handle the heat dissipation of the GPU and subsystems.

Of course, there are many other factors such as boundary conditions, unique material properties, and CPU loads that are important and must be taken into account. If, for whatever reason, the heat removal becomes a bottleneck of GPU performance, then tricks need to be played on the SW side to manage the workload until the cooling or packaging system is able to catch up.

Thermal Resistance

Thermal resistance quantifies the opposition to heat flow at any two points where there is a temperature difference. It is given by

where

• R_th is the thermal resistance (K/W or ⁰C/W)

  ∆T is the temperature difference between two points (K or ⁰C)

  q is the heat transfer rate (W)

Thermal resistance is a convenient first order approximation because it almost resembles an Ohms law analogy to heat transfer, with analogous parameters:

• Electrical current is the heat transfer rate

• Electrical resistance/conductivity is thermal resistance/conductivity

• Electrical potential is the temperature

Heat sink fins can be modelled approximately using this model to estimate heat transfer. Of course, this approximation is valid if the thermal “circuits” are contained within a lumped approximation, where the heat flow is uniformly distributed in confined materials. Anything that is distributed or highly coupled will require sophisticated modelling.

Stress-Strain

All materials with a PCB stack experience strain / material deformation under some kind of loading. There are two major categories of materials:

• Ductile material - A material that initially deforms linearly as more stress is applied, then after a certain threshold, it undergoes plastic deformation over a wide range until it ultimately fails. This is the dynamic when you pull hard taffy or gum. Most metals fall into this category.

• Brittle materials - Brittle materials show a steep linear curve with a sharp fracture at the top. Glass falls into this category.

For ductile material, the left most region is governed by Hookes law which describes material deformation in the elastic region. It is given by

where

• σ is the average normal stress across a cross section area of a material

• ε is the average normal strain, give by (L - L0) / L0

• E is the modulus of elasticity (or Youngs modulus)

On the right side of the linear region, the material starts deforming and ultimately fails to a fracture. Of course, we hope that packaging materials don’t enter this region, but performance requirements can make it difficult to design around this.

Another form of strain, shearing, changes the angle between two normally perpendicular lines of a material. It is given by:

Bumps in between package layers experience both shearing and compressive / tensive forces.

There are a few material properties that matter:

• Youngs modulus - Describes the ability for a material to resist deformation when subjected to stress. Typical values include:

  • 6 for underfill epoxy

  • 8 - 22 for FR4

  • 131 for silicon

  • 132 for copper

• Poisson’s ratio - the ratio of the strains in the axial direction and the lateral direction. The max value is 0.5. Typical values include:

  • 0.1 - 0.14 for FR4

  • 0.28 Silicon

  • 0.35 for copper

• Creep - a time dependent permanent deformation of a material under a constant load or stress at a constant temperature. Creep is common at elevated temperatures.

• Stress relaxation - a gradual decrease in stress by a material under a constant strain over time.

• Viscoelasticity - a nonlinear, hysteretic curve to loading and unloading where a material recovers its shape over time after unloading

• Viscoplasticity - when a material suffers permanent, rate-dependent plastic deformation that does not recover

  • Viscoplasticity is important when describing the behavior of solder since it is subject to strain and creep deformation.

  • One popular viscoplasticity model of solder is the Anand model that governs the rate of change of the inelastic strain. It takes multiple effects such as strain rate sensitivity and deformation resistance into account.

• Coefficient of Thermal Expansion (CTE)

  • CTE is perhaps one of the most important and most talked about material properties for packages because it describes material behavior at the interface of thermal and mechanical properties.

  • CTE is a form of material “strain” where an increase in temperature causes the material to expand, and a decrease causes it to contract. It is given by:

    

    where

    • εth is the strain due to temperature with no mechanical load

      α is the coefficient of thermal expansion

      ∆T is the change in temperature

  • Common CTE values include:

    • 2.6 for silicon

    • 17 for copper

    • 20, 70 for epoxy

    • FR4:

      • 10, 15 in plane

      • 65, 180 in z-axis

  • For any given package size, material with a higher distance from the neutral point will experience higher deformation since it elongates / contracts more, thus increasing the stress on solder joints in that area.

Warpage

Now that I've established some thermal and mechanical fundamentals, we can discuss how these play a role in warpage, and failure modes after the paywall.

Share

Read more

In this post I’ll build intuition of the key tradeoffs and constraints surrounding advanced packaging.

• I’ll define each of the classifications at a high level (from 2D to 3.5D)

• Then I’ll do a breakdown of each of the key options of a TSMC 2.5D CHIP on WAFER on SUBSTRATE (CoWoS). We will examine this package architecture through the following lens:

  • TSV Fabrication

  • RDL Fabrication (with BEOL Process for Si)

  • Interconnect Options (C4 Bump, C2 Cu Pillar)

  • 🔒Substrate Options

    • 🔒Glass Substrate and the unique challenges: SeWaRe and bump reliability

    • 🔒Panel or Wafer - Why we didn’t do panels in the first place

    • 🔒Embedded bridge in Organic Substrate

      • 🔒TSMC CoWoS-L vs Intel EMIB-T

  • 🔒The Process Steps for Hybrid bonding and why it isn’t widely adopted

The reason I write this post is to establish some fundamentals of advanced packaging and communicate what a packaging engineer would like other engineers to know who are interfacing with packaging. My goal is to build on top of the knowledge on this post as well as my existing posts to be able to understand how to properly co-design electronics with the package.

Please note that I do not consider myself an expert in advanced packaging by any means. I reference John Lau’s slides from his seminar at ECTC 2026 and my overall conference experience to better understand the interactions with advanced packaging and other domains.

If you’re an expert and notice something incorrect, please reach out to me so I can have it promptly corrected.

Before I wrote this, I touched upon high level packaging issues regarding reliability, multiphysics simulation, and co-packaged optics in this post I wrote shortly after I attended ECTC 2026:

Advanced Packaging is like City Planning

Advanced Packaging is a lot like city planning:

• In rural areas, houses and buildings are distributed across a large plot of land (2D)

• In suburbs, houses and mid-sized buildings are bundled closer together and connected (2.1, 2.3, 2.5D)

• In cities, floors are stacked both above and below with means to access them (3D, 3.3D, 3.5D)

City planning is all about managing the flow of people to and from desired destinations.

Advanced packaging follows the same logic: signals need to get to and from compute devices. However, there is an an order of magnitude difference between the dimensions of the on-chip interconnect and the PCB/package interconnect. As speed and power requirements increase, more sophisticated and elegant means of package interconnect are needed to bridge this order of magnitude difference.

Just like any building, as you start stacking floors, the structural stability needs to be considered with appropriate safety factors designed in. This logic also applies to advanced packaging as factors like reliability, thermal, and mechanical characteristics become more important.

High Level Classifications: From 2D to 3D

There are several structural variants of advanced packaging. All of these build the system on top of a build-up package substrate on top of a PCB.

Why do you need a buildup package substrate? There are a few reasons:

• Package substrates help “bridge” the dimensional order-of-magnitude gap with the chip level interconnects and the board level traces. This is done through “fan out connections” where the package solder balls spread the signals out from underneath a chip.

• Mechanical Buffer. The package acts as a buffer to protect the chip against stress, vibration and thermal effects. The coefficient of thermal expansion is an important consideration. We’ll see how, quite paradoxically, the low CTE of glass can have a negative impact on reliability.

• Local interconnect amongst amongst chiplets. This is done through a combination of Re-distribution layers (RDLs) and Back-end-of-line (BEOL) metallization on Si Wafers

2D Integration

2D packaging represents the historical and architectural “baseline” packaging technology that more advanced variants build off of. In 2D we have the following classifications:

• 2D - the chip is bonded directly on top of a build-up package substrate.

• 2.1D - the RDLs (Redistribution Layers) are built as thin film layers directly on top of the organic substrate itself.

• 2.3D - The RDLs are fabricated separately on an organic interposer on top of a build-up package substrate. This is also known as the “hybrid approach”.

• 2.5D - The chips themselves sit directly on top of a massive, monolithic Silicon Core (Silicon Interposer) or a Glass Core (Glass Interposer)

In 2.1D and 2.3D, two popular materials used in the dielectric films are ABF and PID.

• Ajinomoto Build-Up Film (ABF) is a resin-based epoxy film embedded with silica fillers.

  • ABF has exceptional mechanical stability, excellent thermal resistance, and a low Coefficient of Thermal Expansion (CTE)

• Photoimageable Dielectric (PID) is almost chemically similar to photoresist; it is a liquid polymer or dry film that is chemically engineered to react directly to light.

  • PID can be made much finer than ABF, but is traditionally mechanically weaker than ABF.

There is a core mechanical stability vs feature size tradeoff between both films. ABF tends to be the dominant choice of packaging due to its ability to handle many metal layers.

2.1D / 2.3D are considered to be low cost alternatives to 2.5D, which is considered the standard for AI/HPC.

3D Integration

3D enables more computational density to be stacked in the vertical direction. The higher order classifications in 3D are still emerging and contested, but this is what Lau preliminarily defines:

• 3D - stacking chiplets directly on top of each other vertically on an active interposer (compared to 2.5D which is passive). These active interposers can contain IVRs, NoCs, BIST, or other peripheral circuitry that complement the functions of the SoC.

• 3.3D - consists of HBM on top of an SoC without a TSV interposer

• 3.5D - Involve hybrid bonding chiplets before integrating heterogeneously all components onto a single substrate.

As packages becoming increasingly integrated with the chip themselves, this leads to one of the biggest packaging challenges: whose responsibly should be for final assembly of the packaging: the fab, or the OSAT? Historically, both fab and OSAT specialized in different parts of the semiconductor production flow. The fab specialized in short pitch, finer interconnects and the OSAT specialized in higher pitch, coarser interconnects.

When hybrid technologies such as bridges and LSI in organic substrates come along, who will assume responsibility for final package integration? This is a question that is still being fiercely debated among the fab and packaging community and will be left for another time.

A Package Teardown: 2.5D

In the next section we will do a deep dive into the fabrication process of 2.5D and examine the fabrication process for each part. 2.5D is considered the package of choice for HPC due to its form factor and performance. For example, the NVIDIA H100 “Hopper” GPU uses this package and a cross section shows key features between the build-up package substrate and logic.

TSV Fabrication

A through-silicon via (TSV) transports signals and power completely through the thick structural floor (the raw Silicon Substrate). In CoWoS, the diameter is ~ 5-10um and the pitch is ~40-50um.

Lets go through the process steps:

• First, we start with standard lithography to add SiO2 on the Si surface that acts as the “hard mask” for TSV:

  • Deposit SiO2 by thermal oxidation of Plasma-Enhanced Chemical Vapor Deposition (PECVD)

  • Apply Photoresist

  • Pattern with lithography through a photo mask

  • Etch the SiO2

• After the SiO2 is deposited, the via hole is formed using Deep Reactive Ion Etching (DRIE). The most commonly used process is the Bosch process.

• After this hole is formed, it is filled with copper:

  • An SiO2 layer is added with PECVD/Sub-Atmospheric CVD to act an an electrical insulator against Si and Copper

  • Then, a barrier (Ti/Ta) and seed (Cu) added to prevent copper poisoning

  • Then, the hole is filled with copper, annealed to stabilize the grain structure, and polished with CMP.

Important thing to note: The more power TSVs you add to feed a power-hungry GPU, the more you “choke” the horizontal routing channels available for the RDLs to connect the GPU to its HBM memory.

RDL Fabrication

Then the RDLs are fabricated on top of the TSV using Standard BEOL Processes using dual damascene. Dual damescene refers to etching both the via and horizontal line trench before adding any metal. Heres how it works:

• Form the via and M1 trenches with standard lithography. This is the same process as with TSVs: Deposit SiO2 with PECVD + photoresist on top, etch the photoresist then the oxide

• Line trenches with a thin Ti/Cu seed layer with PVD

• Overfill the whole trench with electroplated copper

• Polish the excess copper on the surface with CMP to give you a planar surface

Compared to standard packaging methods that limit pitch to 2-5um, this method can get down to 0.4um (at the time of writing this paper).

A passive silicon interposer typically features 4 to 6 BEOL metal layers on the front-side (top), and 2 to 4 coarser packaging RDL layers on the back-side (bottom).

As more layers are added, film stress is added. With HBM4, foundries like TSMC and UMC are actively extending front-side BEOL capabilities to 8, 9, or 10 layers to accommodate the massive explosion of traces.

Interconnect Options

Bumps are formed by first forming an under/bump metallization with both a thin barrier layer (Titanium, tungsten, chromium) and seed layer (Copper/nickel). There are two major categories of bumps:

• C4 (Controlled Collapse Chip Connection):

  • C4 bumps are great for macro power delivery and board-level packaging

  • However, the pitch of C4 bumps are limited to 50-150um because they can cause “solder bridging” if they are too close

• C2 Pillar

  • The actual solder is just a tiny cap sitting on top of a rigid, electroplated copper that does not melt during reflow

  • These allow you to jam interconnects much closer together, but are much more expensive, primarily due to plating time of the Cu pillars

The other major option is Hybrid Bonding, which I’ll discuss at the end.

At higher current densities, both bump resistance and CTE affect reliability. I noticed a few posters at ECTC that demonstrate how a few small effects can magnify:

• Bumps further from the neutral point of a chip experience more deformation as substrate/chip area get larger

• The direction current flows into and out of the bump affects the equivalent resistance of the bump by a few percentage points

• The elemental composition at the bond interface can affect current density as well.

At higher current densities, these effects can lead to localized joule heating that can create “hotspots” and can increase the probability of failures.

Substrate Options

Silicon substrates are great for fabricating the fine pitch interconnect needed in between chiplets. However, silicon substrates are running into a few problems:

• They are expensive

• Fabrication is limited to major IDMs

• Substrates are running into photolithography reticle size limits

• Silicon everywhere is frankly overkill when the fine pitched interconnect is limited to specific areas in between chips.

After the paywall, I’ll discuss the following:

• Two alterative options to silicon: Glass and organic with bridges.

• Panels vs wafers, and why we didn’t do panels in the first place

• A brief history of hybrid bonding, the mechanism, and why its not widely adopted yet

Share

Here are several of my other posts I’ve written that provide the technology fundamentals of power, high speed communications, and AI compute that packaging is the physical container for.

Read more

In this post I discuss the following topics:

• Clear definitions of five buzzwords used in advanced packaging from John Lau’s tutorial

• How Reliability is one of the biggest concerns for engineers on the floor

• 🔒The Wild West of Co-packaged optics - Different CPO Integration Options

• 🔒Why Multiphysics simulation is not that easy in practice

Hardware is becoming the new bottleneck in hyperscalers.

That was was the underlying message at ECTC 2026, the flagship electronics packaging conference, as stated in a plenary session by Dr. Tien Wu, CEO of Advanced Semiconductor Engineering, the largest OSAT in the world.

I attended ECTC 2026, not as a pure packaging engineer, but as a Analog/Mixed signal system architect who is curious about how advanced packaging is interrelated with other elements.

Before I came to this conference, I went in knowing a bit about advanced packaging from other writers. I studied mixed signal architectures in high-speed communications and power systems through self-directed conference travel to ISSCC, DesignCon, and APEC. I wrote comprehensive architecture breakdowns specifically to bring awareness to key constraints so that engineers can ask more informed questions earlier in the design process. I wrote about why I am doing this in my post:

As an EE myself, we sometimes take packaging for granted. However, ECTC made me realize the sheer amount of depth and mechanical considerations that go into packaging. The performance demands of GPUs are placing a lot of stress on the packaging itself and shifts the bottleneck toward hardware, not software, as the limiting factor in scaling AI compute performance.

In this post I’ll give a very broad high level overview of my experience at ECTC and the key challenges engineers on the floor are facing.

John Lau Tutorial on Advanced Packaging

At the beginning of the conference, I acquainted myself with advanced packaging trends through a tutorial given by John Lau. He is considered a legend in the packaging community because he wrote virtually every major book on advanced packaging and heterogeneous integration. He is very practical and as close to the manufacturing reality. His lecture was an experience; he tends to “beat” concepts into your head through repetition; I am not joking. I felt like I passed through a rite of passage.

He broke down the classifications (2D, 2.1D, 2.3D, 2.5D, 3D, 3.5D) as shown in the figure above, as well as described key issues in hybrid bonding and glass substrates. These topics are covered quite extensively by other writers and I will not go into depth here.

He notes that several terms in advanced packaging are often misused. He clearly defines these terms as the following:

He is also well versed in the second-order effects of manufacturability. He mentioned how larger panel sizes (600x600mm) that pack more parts per panel are not always better because it takes longer for the pick and place machine to traverse the distance to move the parts between panel and product.

After the seminar, I personally ordered up a book myself “Hybrid Bonding, Advanced Substrates, Failure Mechanisms, and Thermal Management for Chiplets and Heterogeneous Integration” (2025) to understand the interactions between advanced packaging and adjacent domains. I anticipate this book will strike a balance in describing the interactions with advanced packaging with other domains and will be a valuable resource. I’ll use the material in this book to inform my future writing.

In my view, there is a ton of noise and hype around advanced packaging. At the end of this post, I’ll also list some advanced packaging books by John Lau so that if you’re deciding to write or understand this space better, you have a source of truth to reference.

Two of the major ones high volume advanced packaging techniques are TSMC CoWoS and Intel EMIB, and is clear that Intel is becoming popular as supply chain bottlenecks for CoWoS are coming into play.

Reliability is one of the biggest concerns for engineers on the floor

Most people out there hyperfocus on the raw performance metrics of the GPUs, CPOs, the potential of hybrid bonding and glass substrates, or what Jensen is doing; they are clearly some of the flashiest subjects.

However, as GPU power increases demand more power and signal speed, these effects put more stress the package and “magnify” the existing failure mechanisms! Reliability is especially a concern as package sizes get bigger; they get heavier and introduce more points of failure.

Reliability is something historically thrown over the wall in the semiconductor industry as basically a cost center with a “line” to hit.

I was quite surprised by how much reliability was discussed among the conference among special sessions and technical sessions. There were a new notable observations amongst panelists in the special sessions:

• Tsun-yen Wu (in place of Kathy Yan), Department Manager at TSMC, pointed out two things:

  • First, there is a tradeoff between four domains in packaging:

    • Routability

    • Crosstalk (NOTE: This doesn’t appear correct; to first order, when spacing goes down, crosstalk goes up since there there is higher capacitance between the lines. However, lower line width decreases the fringing capacitance as well, so overall crosstalk depends on the dimensional scaling factors and also ground shielding used)

    • Structure stress/tension

    • Process difficulty

  • He also notes that many package designers tend to design their packages aggressively and not taking manufacturability into consideration early in the design, such as stacking as many vias as possible. This requires constant back and forth to ensure the package is manufacturable. This wastes communication time, especially across time zones, if a working knowledge of reliability is not known by packaging designers beforehand.

• Dr. Richard Rao, Distinguished Engineer at Marvell, notes that transient switching can cause stress, and that a change from a stress strain approach to fracture mechanics approach is needed.

• Gang Duan, EVP, Samsung, points out that embedding components in substrate is aggressive and can affect yield and performance

• Dr. Choong Un Kim, Professor, Material Science at UT Arlington, points out that joule heating leads to local stress which leads to cracking

I also learned of several failure mechanisms in advanced packaging. Here are a few examples:

Much like how we personally take reliability into account when we buy a car, investors and engineers should also take reliability of systems into account. You can have a well performing part or tightly integrated system that generates buzz, but if it causes you reliability problems down the line, it might not be the best long-term solution.

I do think reliability engineering will become much more important than ever, or at the very least, a topic board level engineers MUST know as systems become more tightly integrated. I think that reliability training will go a LONG way to help minimize the cost of constant back and forth as systems become more tightly integrated.

I dive into more depth regarding the fundamental structures and physics behind these failure modes:

The Wild West of Co-packaged optics

Co packaged optics is one of the hottest, yet diverse, topics of the conference. CPO consists of three major blocks:

• Photonic IC (PIC) - This consists of the photonic IC such as photodiode, laser, etc.

• Electrical IC (EIC) - This consists of the electrical circuits such as the transimpedance amplifier (TIA), laser driver, etc.

• Application-specific IC (ASIC) Switch

There are a number of clear performance advantages of CPO that make it such an attractive option compared to pluggable transceivers, such as lower length, reducing energy to drive the signal, and lower latency. Many content creators have written exhaustively on this.

Though the physics of devices themselves can be a bit tricky, the architecture behind optical communications has a simple mental model:

• You have a laser source that is routed through a PIC.

• This laser is generated from a VCSEL, micro led, or other laser source.

• Then, this laser source is modulated externally with either a MZM, micro-ring, or electro absorption modulator from a high frequency signal from an EIC.

• This signal is sent through the optical channel at a particular “band” (850n, O,C,L), detected with a photodiode, and amplified with a trans-impedance amplifier.

I cover these in more detail in my other posts:

What makes co-packaged optics so interesting and yet confusing are the vast amount of packaging techniques involved. Each system has a own unique set of requirements for high speed data rates, system constraints, and a power/ BER link budget to handle.

Share

For instance, here are a few options for 2D CPO structures with bridging and bumping techniques:

Read more

Editors Note (5/31/26): I wrapped up ECTC, so this post reflects my experience when I originally wrote this before ECTC.

I added details about key substack articles I wrote along the way, and a link to an Asianometry video on superconducting electronics to get a flavor of what I worked on.

I’m currently attending ECTC, the flagship electronics packaging conference, from May 26-29 in Orlando, FL, to learn more about advanced packaging. Packaging represents the “container” that the chip sits within and interfaces with most HW domains.

I’ll be writing more on advanced packaging after this conference, but first, I want to take a moment in this post to reflect on the past three conferences I’ve attended, why I attended these, what I learned, and the advice I would give to engineers thinking of doing the same.

Why ISSCC, DesignCon, and APEC?

I recently had the unique opportunity to attend three major technical conferences back-to-back: ISSCC, DesignCon, and APEC.

I selected these specific conferences because, together, they develop a holistic, cross-functional view of the critical hardware challenges facing today’s AI data centers. Specifically, I wanted to understand the abstraction levels of the mixed-signal portions of the data center, which bridge two traditionally isolated domains: high-speed communications and power electronics.

But, there is a deeper reason why I attended these conferences. I anticipate the biggest technology risk in advanced electronic systems is not understanding the interactions amongst domains as domains become more tightly integrated. Many leaders throughout my conference travels urge the need for more cross functional awareness and that silos will no longer cut it. Many call for a “digital twin” as a simulation model that models all of these interactions. This is much needed, but just like any simulation, its important to develop a solid theory of these interactions to be able to interpret the simulation results and propose fixes.

To be completely honest, taking this on conference gauntlet was brutal—at least at first. But it got easier over time.

Not many people get the opportunity to do this. Companies are often hesitant to send engineers to conferences due to cost, inertia, or fears of intellectual property leaks. Because my circumstances were unique, I had to self-fund this journey.

Obtaining Interdiscplinary breadth: From Design, Applications, and Validation

The catalyst for this journey came after I was let go from my previous job. However, I build up a rich set of experiences leading up to it.

I gained a ton of interdisciplinary breadth while attempting to scale up superconducting computing from low level IP to EDA integration. Check out this video by Asianometry for a more in depth look into this technology.

One thing I noticed in any novel deep tech is that there is a HUGE disconnect between the potential for a technology from the financiers and its actual technological readiness from the bottom. I have my fair share of war stories dealing with the integration challenges when taking thinking from CMOS specialties and applying that to scale the EDA flow of a low impedance, highly coupled technology with unique first principles constraints.

In addition to superconducting electronics, I also have pretty extensive experience doing IC design for mixed signal, board design for power, and audio applications. Its quite a well-balanced range of activities that gave me a wholistic sense of the design process.

However, I was unexpectedly let go at my last job. I won’t go through all the details, but in short, it turned out not to be a good role fit for my skillset they brought me in for.

Since then, I’ve taken the time to go on a year long sabbatical to not only expand my own knowledge, but also to share it with others. I also wanted to provide a mental map for other engineers to become more T shaped or directors who want to bridge domains and need a technical reference point to understand engineers concerns across domains. The industry is screaming for cross-functional awareness, and I decided to cultivate it in myself.

The Strategy and Preparation

Before diving into the conferences, I took a few months for personal travel around the world in places in Europe and Asia, including key countries in the semi supply chain like Germany, Japan, Taiwan, and Singapore. I did cultural tours in each place to gain a sense of the historical and cultural influences, such as how key events such as the 228 incident influenced Taiwans politics today. It was a much-needed mental break from work, but it isn’t a lifestyle I’d want forever—the logistics and constant travel become a hassle.

When I returned, I committed to building my Substack. Over the previous year, I had accumulated a wealth of insights from technical textbooks and business books, including books on strategy, business, VC, and innovation. Check out some high level overviews for the some of the books I read and highly recommend:

Before I attended the technical conferences, I first started writing foundational, subsystem-level posts covering ADCs, PLLs, bandgaps, and AI accelerators from textbooks and key papers.

Building this technical baseline gave me the mental models I needed to absorb the conference material, starting with the toughest one of all.

1. ISSCC (International Solid-State Circuits Conference)

ISSCC is considered the “Olympics of chip design.” The overarching theme was “Advancing AI with IC and SoC Innovations,” focusing heavily on high-speed optical and wireline systems and associated high speed subblocks. It is heavily academic, dominated by professors and PhDs from top-tier universities.

Holding a Master’s degree myself, I’ll be honest: I would NOT recommend ISSCC for a first-timer. It was rough and felt like I was drinking from a firehose!

The plenary sessions covered a broad range of topics, including a talk given by Hope Giles, VP of PM at Apple, on why systems thinking and cross functional thinking are critical skills. Bernhard Wicht won a “Best book award” for his book “Design of Power Management Integrated Circuits” which is one of the best analog design books out there for not just PMIC design but also circuits that share the same blocks.

I purchased an All-Access Education Pass, which granted me entry to 10 tutorials and 6 forums/short courses, in addition to the technical sessions. While my previous writing on PLLs and ADCs helped me follow some of the technical sessions, there was still a massive gap in technical depth that made it hard to follow the advanced papers for long.

However, the tutorials and short courses were incredibly valuable. I attended deep dives on biomedical circuits, logic and memory, AI accelerators, and clocking/CDR techniques for wireline transceivers. The forums comprehensively covered both the power and high speed optical communications side of AI data centers, as well as a short course on optical communications. I also caught a fascinating panel on brain-machine interfaces featuring Jan Rabaey and the CEO of Neuralink.

As a result of this conference, I synthesized this firehose of information to write primers on optical communications and high-speed SerDes—creating the exact frameworks required to understand complex ISSCC papers.

2. DesignCon

DesignCon is the premier high-speed communications conference, famously themed around “where the chip meets the board.” While ISSCC is more posh and focuses on on-chip architectures, DesignCon is more laid back and focuses on the board-level implementation of high speed communications. It was a bit more industry-focused and accessible, though still a challenge coming the following week after ISSCC.

The event kicked off with an excellent tutorial by Dr. Mike Peng Li, a foundational figure in jitter and signal integrity. he has written extensively on jitter. Check out this chart of his that highlights all the possible sources of jitter:

The core technical sessions revolved around the practical challenges of handling 224 Gbps signaling and the scaling paths toward 448 Gbps. There was a huge variety of topics from co-packaged optics, power integrity, EMI mitigation, and AI applications in signal integrity. The massive expo hall was particularly enlightening, showcasing real-world in-rack and interconnect solutions. I learned about the existence of many lesser known companies in the high speed communications value chain such as Molex and Samtec.

The result of this experience allowed me to write my deep dives on high-speed signal integrity and silicon photonics. I don’t think every engineer needs to know every single source of jitter, just the fundamental concepts of DJ and RJ.

3. APEC (Applied Power Electronics Conference)

APEC is the flagship conference for power electronics. I kicked things off by attending three tutorials specifically focused on power delivery for AI data centers, as well as digital controllers for power electronics.

A massive portion of the conference focused on optimizing the data center power chain. Sessions broke down every major conversion step: 800V to 48V, 48V to 12V, 12V to 1V, and various intermediate bus voltage permutations. To be fair, the material felt a bit repetitive after a while; once you understand the underlying power architecture, many papers are just permutations of the same fundamental topologies. There was also a heavy focus on wide-bandgap semiconductors like GaN and SiC, as well as a focus on optimizing magnetics.

Check out this image from STMicroelectronic’s Booth on the power architecture:

I really appreciate the power domain because the board structure is a tightly coupled system. It forces designers to have a holistic sense of how every single component interacts. It’s an area you can nerd out on for an entire career because there is always a creative optimization trick to find.

This conference led to my article, “AI Data Center Power System: A Complete Overview.” It maps out the intricate tradeoffs between converter topology, footprint, and power delivery network (PDN) variables, explaining not just what the architecture is, but why it has to be that way.

4. ECTC (Electronic Component Technology Conference)

ECTC is the flagship conference for electronic packaging. This conference really came at a good time because packaging represents the physical container that contains the AI processor.

I kicked off the conference with a seminar given by none other by John Lau, who has authored several major electronic packaging books. His seminar was quite an experience; his diagrams are so incredible detailed and rich with insights I will never forget.

I attended a variety of talks including those on issues adjacent to the areas I’ve explored, including optical interposers and high speed signal integrity for HBM.

After this conference, I wrote several deep dives that go into depth regarding the fundamentals of advanced packaging and interactions with high speed and high power hardware

5. DAC (Design Automation Conference)

DAC represents the advancement in the automated tools that are responsible for creating the compute chips we all enjoy today. I came in with one question in mind: will AI fully automate chip design?

And the short answer I discovered at the conference is, no, because there are even bigger issues that humans are facing when designing such chips.

Many issued were raised by panelists, including physical co-design, compute bottlenecks, and verification.

After DAC, I wrote this post summarizing that experience:

Advice for Engineers

Navigating these conferences as an independent engineer taught me several lessons about career development, corporate dynamics, and learning strategies:

1. Try to get your company to pay first (and how to pitch it)

If you want to attend a conference, ask your company to sponsor you. Even if its for your own benefit, you should frame the request entirely around how the knowledge benefits the company. Sometimes management is simply unaware of the existence of these conferences, or might not see it as a big priority in your day job. Offer to give an internal presentation to your team afterward. This shows you want to distribute the value, rather than “hoarding” knowledge for your own resume.

2. Navigate corporate sensitivities carefully

Remember, your company’s name is printed on your badge. Have a clear understanding of who your competitors are before walking in. When networking, err heavily on the side of caution and silence. It is incredibly easy to let an innocuous statement about your current project slip—especially when alcohol is being served at networking events. Competitors can easily piece these details together into competitive intelligence.

3. If you self-fund, prepare in advance

If your company won’t pay and you decide to fund it yourself, make sure your finances are secure (I relied on index fund savings and 10 years of industry experience to comfortably fund this).

However, attending a few conferences is not something worth quitting your job over. If your company refuses to pay, it might be worth using vacation time and a bit of your own money. This money should be viewed as an investment in your own education.

Self-paying also has the added benefit of forcing you to take the conference more seriously. I found myself more actively engaged in the conferences, sitting near the front of most sessions, diligently taking notes. Its not the same experience as when a company pays for you to go because you have more skin in the game.

4. Consider if you’ll gain more value cheaper from purchasing and studying technical books

Conferences are excellent sources of truth for what is going on in the industry. With that said, conferences are probably not the first thing you should do until you have good fundamentals of the background technical material surrounding the conference. You might be better off going to an engineering library, picking up a few excellent technical textbooks, and thoroughly outlining them. I spent a year doing exactly that before I ever stepped foot in these conferences. That helped to close the knowledge gap with the conference material that builds up top of the fundamental textbook material.

Check out my post “Learning Analog/ Mixed Signal Design the efficient way” for my methodology that can apply to other technical areas.

5. Aim to become “T-Shaped”

As a near term career goal, try to attend one conference inside your core domain, and one that is interdisciplinary or adjacent to it. This builds a “T-shaped” skill set (deep expertise in one area, broad literacy across others). If you are looking for excellent interdisciplinary conferences, check out the ones I mentioned, but also consider:

• VLSI Symposium (excellent to pair with a vacation to Japan or Hawaii)

• DAC (Design Automation Conference)

• SEMICON (various locations)

• IEDM - IEEE International Electron Devices Meeting

6. Lean into the haze

Don’t be discouraged if you feel overwhelmed or hazy during the technical sessions. Even though I prepared quite diligently and my published posts look polished, I really felt incredibly out of my depth during the events at first. I needed weeks of breathing room afterward to digest the material. Take full advantage of the tutorials at the start of the week, and remember that you get access to the conference digest (papers and slide decks) afterward. Don’t put too much pressure on yourself to fully understand the technical sessions.

However, it does get easier over time because you’ll learn how to prepare a game plan to frame the material in the conference better.

Real growth happens when you sit slightly outside your comfort zone and process the gaps in your knowledge after the chaos subsides.

Conclusion

Its unfortunately very common that many companies nowadays are very execution focused and push the burden of responsibility on you to develop your own career. When I ran into organizational barriers trying to add value inside my company and let go, I decided to use the unemployment time as a “sabbatical” and add value to a broader audience.

Ultimately, the best path is to figure out how to add value to your current company in a way that is mutually beneficial. If you find yourself in a mismatched role, don’t just check out. Try everything in your power to add value to other around out first.

With that said, you’re ultimately responsible for investing in your own education. I, and many others, can provide this information, but cannot make you learn it if you don’t seek it out. Take coursera courses. Take advantage of engineering libraries. I would literally go to an engineering library and just casually browse through books and make notes on ones to dive deeper into later.

But just don’t hoard the insights for yourself. Find ways to share those insights with others so they might benefit as well. As they say, you learn 90% of the things you teach others.

Share

Here are some of the articles I was able to produce as a result of these conference travels, including the methodology I used to self-study:

But I wonder to myself, with all the risks of AI, why do people love AI so much? I contend that we are deep in the throes of cognitive offloading. We are outsourcing our thinking to agents to reclaim time and money, especially as companies face financial pressures to adopt it.

But as AI moves into the “reasoning era,” we have to ask: Will this offloading eventually atrophy human cognition?

My theory is that our intellectual survival depends on how much intellectual agency we maintain over quick dopamine hits and the comfort of not learning.

The Danger of Cognitive offloading

Cognitive offloading isn’t inherently “bad.” Humans have always outsourced cognition—we use calculators for math, GPS for navigation, and specialists (doctors, lawyers) for expertise we can’t master ourselves. There is simply too much information in the world for one brain to hold.

However, offloading to AI carries unique risks to human agency.

Overconfidence

There is a massive gap between interfacing with AI and understanding AI.

This is a vitally important point because AI has caused people to be overconfident about their knowledge of AI. When people claim to know AI, they are knowledgeable about the latest tools used to interface and control AI. These tools also have a very low barrier to entry to them. Anyone can learn Python and ML techniques in a reasonable amount of time. They might also be familiar with basic concepts such as neural networks.

Knowing AI requires understanding the mathematical scaffolding—linear algebra for parallelization, calculus for derivatives, and discrete mathematics. It also requires you to have knowledge of key technical papers and books that lay the foundation for human cognition.

Take, for instance, the Bellman Equation, a cornerstone of reinforcement learning:

When we treat the output as magic without acknowledging the math, we develop an unearned confidence. We mistake the tool’s competence for our own.

Hollow thinking

In software engineering, there is a phenomenon called "tutorial hell"—you follow a guide, feel like a genius, but realize you can't build anything from scratch once the instructions disappear.

AI creates a similar trap. Because LLMs have a low barrier to entry, we often skip the "hard work" of forming mental models. If you ask an AI to write your strategy memo, you aren't doing the heavy lifting of synthesizing information. You end up with a hollow body of knowledge: you have the result, but not the neural pathways required to defend or evolve it.

Cognitive Surrender

The ultimate risk is cognitive surrender. We already see this with "algorithmic feeding" on TikTok or Netflix, designed to exploit our neurochemistry. When we stop choosing what to consume and let an agent decide for us, we trade our agency for dopamine.

The Solution: Reclaiming Intellectual Agency

The solution to this is to embrace curiosity, expand your mental bandwidth, and embrace the discomfort of learning.

Prioritize Curiosity over Ambition

Most self-help content out there promotes developing motivation and ambition. This is good for people starting, but can quickly become a “tutorial” hell where people consume videos to feel good rather than doing something about it.

Ambition is important, but a narrow focus on ambition often narrows your focus to surface-level tactics to project it. It can also lead to “hustle-type” behavior that don’t create long term value.

Curiosity does the opposite: it forces you to be foundationally sound. It allows your brain to ask yourself how do things in the world work, admit to your self what you don’t know, and lets you seek out meaningful information.

Curiously can look like going to a library or bookstore and skimming books and seeing which books best satisfy one curiosity to know how the world works or how to operate in a specific niche. It allows you build mental models of how the world actually works.

Curiosity allows the knowledge accumulate so that when interfacing with AI, you can question why the AI gave that answer, rather than just hitting "copy-paste."

Expand Your “Mental Bandwidth”

Agency is the capacity to act independently, but it is easily eroded by stress and financial pressure. When you are stuck in a cycle of “survival mode,” you are more likely to surrender your thinking to the easiest path (the algorithm).

One way to practice “negative bandwidth” protection is to control your spending and limit mindless consumption. Most marketing is designed to tap into your insecurities; by opting out, you reclaim the mental energy needed for deep thought.

Embrace Cognitive Friction

Cognitive friction is that uncomfortable “blank” feeling when you encounter information that doesn’t fit your current mental model. In the AI era, we try to actively avoid this feeling.

Instead, embrace this discomfort. That friction is new neural connections forming. As an adult, this can feel embarrassing—like we “should” already know these things. But whether it’s learning geopolitics or advanced physics, leaning into the discomfort is the only way to ensure the knowledge actually “sticks.”

How to Learn Efficiently Alongside AI

I’d say that self-directed learning is the most valuable skill of the 21st century because knowledge of how to use new technologies is not quickly feeding back into education and academia to inform the next generation of workers properly. Here is how I suggest balancing AI utility with human growth:

Learn foundational material from the bottom up

Don't just learn the "how," learn the "why." If you’re using AI for data science, go back and brush up on the underlying calculus or statistics. Work backward from what you don't understand.

Stretch Your Cognition

Put yourself in situations where the knowledge is just out of reach—a new role, a technical conference, or a difficult book. But make sure the information is not completely “over your head” - ensure that the difficulty level is slightly higher so that you can raise your level gradually.

Use AI As a “Critic” against your thinking

Instead of asking AI to write your explanation, write it yourself first. Then, feed your thoughts into the AI and ask it to critique your logic or find gaps in your mental model.

Conclusion

I’d argue that the world will eventually belong to the generalists—those who can use AI effectively to bridge gaps — but possess the foundational curiosity to know when the AI is hallucinating or oversimplifying.

If you want to learn more about the technical (such as transformer architectures) and corporate forces (such as DeepMind vs OpenAI), check out my other post here:

If you want to learn more about the Hardware Side of AI Accelerators (such as DRAM/SRAM, MAC operations, and a foundational architecture, Eyeriss), check out my post here:

Silicon photonics is one of the hottest and fastest moving markets to address high speed data demands in co-packaged optics (CPO) in AI data centers.

In this post I’ll cover a few important high-level details of silicon photonic circuits. These include:

• An Overview of the Link Budget and important considerations:

  • Laser Power

  • Connector Losses

  • Receiver Figures of Merit

• 🔒Key Figures of Merit for eye quality

• 🔒Validation Challenges (wafer screening and thermal stability)

  Inside the Silicon Machine is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Link Budget

Understanding link budgeting is important to develop a system level understanding of a SiPh transceiver. Consider a typical electro-optical link with an external modulator:

At a high level, the job of a transmitter is to transmit the data at the specified data rate (224GPBS) with as low BER as possible in the given specification. Key system level performance specs include:

• Power efficiency: The ratio of energy consumed to the amount of data transmitted. This is usually measured in pJ/bit.

• BER: Bit error rate. Noise, loss and other effects degrade bit error rate. Requirements range between 1e-4 and 1e-6 depending on the standard.

A laser source (whether that be generated on board or coupled in externally) provides the power and is sent through a lossy modulator, connectors, and waveguides.

This power is then sent through the optical channel, converted back to a weak electrical signal through a photodetector, and amplified by a transimpedance amplifier. The signal at the RX need to be high enough compared to the noise in order to be able to detect the signal properly.

All of the sources of loss in between the TX and RX must be quantified for in a link budget in the optical chain. Lets break down each part.

Lasers

Lasers are either integrated within the PIC or coupled in externally. These external lasers are coupled into the chip through edge couplers or grating couplers.

There are a few important specifications:

• Power: The amount of coupled power is limited by two-photon absorption and other non ideal effects. Shown is a sweep of Pin vs Pout where Pout saturates and a high enough Pin can cause passivation melting:

  

• Relative intensity noise (RIN) - Rin refers to the relative intensity fluctuations of the optical signal of a laser source. This is a critically important parameter in PAM4 schemes because PAM4 has much lower noise margin than NRZ. A high RIN raises the noise floor causing a higher BER. RIN is given by:

Companies with lasers in the value chain include Lumentum, Broadcom, Coherent, Sivers, Scintil, Xscape, and Quintessent.

Losses in Connectors

There are several loss sources in the connectors themselves. These include:

• Modulator loss - due to intrinsic optical absorption and scattering

• Multipath Interference (MPI) - Optical reflections from multiple connectors / discontinuities

• Polarization Dependent Loss (PDL) - Measures loss due to change in signal polarization

Receiver

On the receive side, the signal is detected by a photodetector. This weak signal plus a noise signal is converted to a voltage through a transimpedance amplifier (TIA) that converts current to voltage.

A few key specs here are important:

• Responsivity - Measures how effective the photodetector converts photons to electrons. This is typically measured in A/W.

• Dark current - Measures the current flow when no light is present. Defects can cause high dark current that adds noise to RX and reduces sensitivity.

• Input-referred noise - assesses the noise floor of the TIA.

• Transimpedance Gain - measures the electrical strength of the receiver. TIA gain is very important because if TIA gain is too low, output voltage swing will not be high enough to compensate for noise. It is given by:

• Receiver sensitivity - the minimum optical power required to achieve a specific BER target. This is determined by plotting RX BER vs OMA. BER goes down when received optical power goes up.

• THD (Total Harmonic Distortion) - quantifies the non-linearity of the TIA. This affects PAM4 because it affects how even spaced out each of the four levels are after optical to electrical conversion.

Eye Quality

After the paywall I’ll go into key parameters that define the eye quality, practical challenges when scaling production of SiPh, and NVIDIA’s approach to thermal tuning design.

Share

If you’re curious to learn more, check out my other posts here:

Read more

However, leaders all suffer from human biases and tendencies, including:

• The tendency for linear thinking and templates in planning (SWOT, vision, mission, values)

• Tendency to reward people similar to themselves in business culture

• A disconnect between how high level business decisions affect people with boots on the ground

To establish a broad understanding on how engineers can lead, I’ve selected five books that cover a large breadth of challenges in the organizational space.

High Output Management

Written by the legendary former CEO of Intel, this is considered the “Bible” of Silicon Valley operations. Andy Grove treats management as a production process, using the “Breakfast Factory” as a metaphor for high-stakes manufacturing.

He establishes a few key responsibilities of managers in high-tech organizations:

• The Concept of Leverage: A manager’s output is the output of their organization plus the neighboring organizations they influence. A managers job is to focus on “High Leverage” activities—those where a small amount of their time produces a large amount of results.

• Task-Relevant Maturity (TRM): There is no one-size-fits-all management style. Depending on the TRM, new people might require hand on training, whereas more experienced people don’t need to be handheld as much. It is the managers job to ensure that people are adequately trained for the job they are in.

Grove observed that all large organizations with a common business purpose eventually become hybrid/matrix organizations with dual reporting to both functional and project managers. This structure allows a huge amount of flexibility to match talent to business needs, but this requires a strong corporate culture to manage dual reporting.

Good Strategy Bad Strategy

Rumelt argues that most “strategy” is actually just “bad strategy”—a collection of buzzwords, goals, and ambition, and the use of templates with vision, mission, and value statements. It is what leaders tend to revert to when they don’t want to deal with tricky fundamentals and risk tradeoffs.

Bad strategy can manifest at higher level positions because of things we tend to perceive and reward in business culture - sales ability, visibility, or pure business knowledge without much technical depth.

“Good strategy” is more intellectual in nature, and involves a “kernel” that consists of three components:

1. A Diagnosis: Defining the nature of the challenge.

2. A Guiding Policy: An overall approach to deal with the obstacles identified.

3. Coherent Actions: Steps coordinated with one another to support the guiding policy.

Good strategy involves viewing the organization as a “chain link” of interconnected functions that are hard to replace. An organization is only as strong as its weakest subunit. In semis, if your EDA tools are world-class but your packaging tech is mediocre, the entire product fails.

To increase performance, you must integrate components more tightly, not just improve them in isolation, so good strategy can involve strategic acquisitions that enhance this chain link.

Good strategy is hard because organizations are resource limited and it can involve choosing between a number of lucrative opportunities. To concentrate on one pivotal objective requires the courage to ignore a dozen other “important” things.

Innovators Solution

Speaking of bad strategy, there are no buzzwords more frequently abused than the words “disruptive” and “innovation” because everyone wants the benefit of these, but few people want to bear the risk involved.

If The Innovator’s Dilemma explained why large companies fail, The Innovator’s Solution explains how companies structure themselves to overcome this dilemma. This books highlights the key difference between conventional business processes which are deliberate, vs innovation processes which are emergent.

It advocated spitting business into into execution and R&D teams:

• Execution teams focus on core revenue generating business with low risk

• R&D teams incubating new ideas and technologies with high risk

On the product side, one core theme as it applies to complex engineering products is the concept of Modular vs. Interdependent Architectures: Early in a product’s life, when performance isn’t “good enough,” interdependent (proprietary) architectures win because they allow for performance optimization and control of all the components. Once performance exceeds market needs, the industry shifts to modular architectures that involves “outsourcing” components with industry-standard interfaces. This can lead to commoditization.

One major topic Christiansen advocates is avoiding the bias to favor “Right stuff Criteria” when evaluating managers for new growth businesses. These are mental heuristics used to evaluate people in corporate contexts among a large body of candidates, such as prestige of the company, the school they went to, whether they are a good communicator, results-oriented, decisive, and have good people skills. However, the downside of “right stuff criteria” is that managers can succeed for reasons not of their own making and the tendency to favor success in low-risk contexts. Instead, potential managers should be asked what problems they have wrestled with in the past.

Christiansen also talks about “good” and “bad” money involved between core revenue generating businesses and innovative businesses. This refers to the risk vs return expectations tied to the money that the shareholder or investor gives. Most corporate finance literature advocate for minimizing risk in the core business since shareholder value is largely tied to it. However, innovation requires risk distributed among a number of “bets”. When a core business sags, “good money” can turn into “bad money” when corporations demand that new ventures become “big, fast”. This “impatient capital” forces startups to bypass the learning phase, often leading to failure.

Thinking in Systems

Humans tend to feel a comfort in linear thinking - of thinking in terms of cause and effect. Frederick Taylor, one of the pioneers of scientific management, strongly advocated for breaking down complex tasks into smaller, specialized roles with rigorously defined processes for each.

However, this type of thinking can narrow ones view to complex interdependencies involved. Semiconductors are perhaps the world’s most complex system of feedback loops not only in the circuit themselves, but within the organization and supply chain.

Donella Meadows provides the tools to understand non-linear cause-and-effect with a few key concepts:

• Stocks, Flows, and Feedback: A system consists of elements (stocks), interconnections (flows), and a purpose. In our industry, the “stock” might be engineering talent or wafer inventory, and the “flow” is the rate of hiring or production.

• Feedback Delays: Because of long lead times in fabs, by the time a problem becomes apparent, it is often too late to solve easily. Leaders must “get the beat” of the system and look for “leverage points” where small shifts in structure cause big changes in behavior.

• The Trap of Sub optimization: When a subsystem (like a specific design team) optimizes for its own goals at the expense of the total system (the final chip’s power/performance), the whole organization suffers.

Reframing Organizations

“Reframing Organizations” by Bolman and Deal is one of the most influential leadership books that is commonly read in top MBA programs.

This book establishes four possible “frames” for leaders to view an organization through. These frames provide a framework for leaders to balance the coordination of the tasks involved with the psychological tendencies and needs of the humans that operate in organizations, including the desire for power and meaning. These four frames include:

1. The Structural Frame (The Factory): Focuses on goals, roles, and technology. It assumes the organization is a machine designed to achieve a specific task.

2. The Human Resource Frame (The Family): Focuses on the relationship between the organization and the people. It assumes that if people feel competent and supported, they are more willing to give more to the company.

3. The Political Frame (The Jungle): Sees the organization as an arena for competition. It recognizes that resources are scarce and power is the primary currency. Leaders here must be negotiators and coalition-builders.

4. The Symbolic Frame (The Theater): Focuses on meaning, ritual, and culture. It recognizes that in a world of ambiguity, people need a “vision” and symbols to give their work purpose.

Having knowledge of these frames allows leaders to be flexible in creating a culture that enables production. It also allow leaders to “reframing” to turn around failing organizations - that is, when organizations are facing issues in, say, structure and politics, leaders can emphasize the human resource and symbolic side to generate enough buy in amongst workers.

Conclusion: The Holistic Leader

Engineering leadership requires having a broad organizational knowledge of how peoples efforts are organized to achieve a goal. They must make decisions and set standards that balance the market needs and the psychology of the people within the organization.

All of the books I mentioned provide a foundational overview into the type of thinking leaders must face. Leaders might be concerned with how projects are being managed from within, the overall business strategy, how innovation is being handled, and whether all the parts of the organizations are being optimized properly.

Top leaders might start their morning in the structural frame (analyzing fab yields), move to the political frame (negotiating with a geopolitical partner), then the human resource frame (understand what motivates your workers to achieve), and end the day in the symbolic frame (rallying the team around a breakthrough).

To lead in this space, semiconductor marketers must possess a rare combination of skills: an understand of what key performance metrics customers are willing to pay a premium for, the ability to translate complex capabilities into business value, a time-to-market instinct, and a holistic grasp of the supply chain involved.

To understand why the industry is shaped the way it is, I have mapped out five foundational books and three key themes from each. Together, they form a cohesive foundation for understanding the move from disruptive innovation to the brutal realities of global economies-of-scale.

This post is designed for readers who want to gain a feel for to “why” the semi value chain is the way it is due to the complex interdependencies. I’ll keep a high level discussion here; refer to the books for more detail.

Innovators Dilemma

Innovation is a trap. Christensen’s core thesis is that “great” companies fail not because they are poorly managed, but because they are too responsive to their most profitable customers. He hits upon three key themes:

• Sustaining vs. Disruptive Innovation: Large incumbents excel at sustaining innovation—incremental improvements for high-margin tiers. However, they often ignore disruptive innovation—low-end, lower-performing tech that eventually scales to intersect and destroy the incumbent’s market.

• The Resource Allocation Trap: In semiconductors, where gross margins of 50%+ are the gold standard, executives are incentivized to “eat their way up” the value chain. This leaves the “bottom” of the market open for new entrants who can scale faster than the incumbents can pivot.

• The “Right Stuff” Bias: Organizations often use “heuristic shortcuts” for hiring and promotion, favoring managers who maximize short-term ROI. However, these are often the wrong people to lead a disruptive unit that needs to focus on learning rather than just execution.

Crossing the Chasm

If Christensen explains why firms fail to innovate, Geoffrey Moore explains why innovations fail to sell. The “Chasm” is the massive psychological gap between Early Adopters (who see the performance benefits) and the Early Majority (who want to follow the market leader). For products to cross this chasm, Moore proposes three key marketing strategies:

• The Beachhead Strategy: In a mainstream market, the early majority delays buying until they see a clear leader. To win, market share, companies must secure a “beachhead”—a specific, referenceable pragmatist customer base that allows them to “own” a niche before expanding.

• The Whole Product Concept: Marketing in semis often fails when engineers tout specs (FLOPs/TOPs) instead of solutions. Pragmatist customers buy the “whole product”—the silicon, the software stack, the documentation, and the support.

• Positioning as a Heuristic: Positioning isn’t about what the product is; it’s about creating a mental shortcut for the buyer that differentiates your product relative to the competition.

  • Example: NVIDIA’s early success wasn’t just about GPU architecture; it was about positioning themselves as the “only way” to play Doom or run professional CAD, creating a marketplace where they were the only reasonable choice.

Fabless: The Transformation of the Semiconductor Industry

This book tracks the “raw” history of how the industry moved from Integrated Device Manufacturers (IDMs) to the specialized, horizontal ecosystem we see today.

• The EDA & IP Revolution: The rise of Electronic Design Automation (Cadence, Synopsys) and reusable IP (ARM) decoupled design from manufacturing. This allowed “Fabless” companies to focus on architecture while outsourcing the capital-intensive fabrication.

• The Rise of “Coopetition”: As technology advances, no single company can do it all. We see increasing interdependencies: Samsung is Apple’s fiercest rival in handsets but its most critical partner in the foundry/supply chain.

• The System-in-a-Package (SiP) Era: We are moving past the “single chip” era. Modern high-end silicon (like FinFET-class chips with 3D memory stacks) requires a massive ecosystem of specialist companies to collaborate just to get a single product to market.

Chip War: The Fight for the world most critical technology

Strategy does not happen in a vacuum. Miller maps the industry through the lens of “Geopolitical Chokepoints”—specific nodes in the supply chain where a single entity holds disproportionate power.

• Mapping the Nodes: The supply chain is concentrated in specific geographies: EUV lithography in the Netherlands (ASML), manufacturing in Taiwan (TSMC), and EDA tools in the U.S.

• The Silicon Shield: Taiwan’s dominance in high-end manufacturing creates a “shield” where the global economy is too dependent on their output to allow for instability.

• Sovereign Silicon: We are seeing a shift where “marketing” a semiconductor company now involves navigating national security interests, as countries race to reverse-engineer, replicate, or replace foreign technology.

Apple in China: The Capture of the Worlds Greatest Company

Apple is the ultimate case study of all the previous concepts clashing in real-time. It is a masterclass in managing the “value network” across geopolitical lines while fighting the “innovator’s dilemma.”

• Zero-Defect Mentality: Apple’s success is built on a “zero-defect” mindset and a brutal Custom Silicon Management (CSM) process. They train suppliers to scale quality, then “pit” rivals against each other to drive down margins.

• The China Challenge: Apple highlights the tension of being a Western company deeply integrated into the Chinese ecosystem. While they benefit from the scale of partners like Foxconn, they face “reverse-engineering” threats from local giants like Huawei.

• Serialization as Control: To maintain their brand and deter bad actors, Apple serializes almost every component. This is a marketing and supply chain tactic to “own” the customer experience—even if it creates friction with right-to-repair advocates.

Conclusion

Understanding the semiconductor market isn’t just about knowing the “what” (the specs); it’s about knowing the “why” and how all the elements (marketing, geopolitics and technology) are interconnected. We see a map of an industry that is simultaneously the most globalized and yet the most protective on earth.

Leave a comment

Demis Hassabis is perhaps one of the most important influences in Modern AI that isn’t a household name.

Modern AI is not just a story of better models. It was shaped by the interaction between scientific ambition, commercial urgency, safety concerns, and the compute power available to model it.

Every AI enthusiast should read “The Infinity Machine” to better understand the development of modern AI. It tells the story of Demis Hassabis, CEO of Google DeepMind, from his origins, to his founding of DeepMind and its acquisition by Google.

I’ll highlight a brief history of AI through the formative years of Demis Hassabis, and cover the key technical developments and organizational challenges along the path to modern AI, particularly around AI safety.

I’ll also dive into the key papers addressed in the book that influence modern AI today such as deep reinforcement learning, transformer architecture, and chain-of-thought prompting that approximate human intuition.

Demis Hassabis

From his early days Demis Hassabis is a modern example of a modern day Polymath, with broad-ranging interests from sci-fi, philosophy, computer science, and neuroscience.

His entire career has been dedicated to developing artificial general intelligence (AGI), a hypothetical type of AI that matches or exceeds human cognitive abilities across a wide range of tasks. His dedication to this mission has him referred to as an “authentic entrepreneur” by VCs because he is genuinely dedicated to a cause, as opposed to entrepreneurs fishing for ideas to strike it rich.

We’ll see through his formative years in video game development about how he distinguishes between early machine vs human intelligence:

• Early machine intelligence uses deductive, first order reasoning to reason through problems

• Human intelligence generally uses induction or high level pattern recognition to identify shapes, sounds, and reason through sentences

Formative Years

Demis had a fairly modest upbringing in North London. When he was young he was highly influenced by classic Sci-Fi: “Enders Game”, “Foundation”, “Godel, Escher, Bach” and Ian Banks “Culture” series were some of his early influences. He was a chess genius and video game enthusiast in his early days, frequently ruminating on what algorithms are needed to program a machine to play chess.

He won admission at University of Cambridge to study CS, but he was too young at the age of 16. During this time gap, he worked on video game development at Bullfrog with Peter Molyneux to develop video games, including Theme Park.

After he graduated, he founded Elixir Studios in 1998, a London-based video games developer, and work on developing video games that focused on complex decisions on the user’s part, including Republic: The Revolution and Evil Genius.

Elixir Studios ultimately failed in April 2005, at which point he decided to pursue his PhD in neuroscience at University College London (UCL) from 2005 to 2009. His studies focused on the field of imagination, memory, and amnesia.

After he obtained his PhD, Demis Hassabis founded DeepMind Technologies in London in 2010, a ML AI startup with Shane Legg and Mustafa Suleyman. The goal is to combine insights from neuroscience with ML and computing hardware to create new algorithms to advance toward AGI. His company was successful at training a Deep Q-Network to play Atari games in 2013. [1]

In 2014, Google purchased DeepMind for $400 million where it remained a separate, siloed entity from Google. Since being purchased, it racked up a list of accomplishments from 2014 to 2020:

• It created a clinical alert system, Streams, for acute kidney injury in the UK. While well intended, it was controversial because of concerns over patient data governance.

• It created AlphaGo, a program that played Go. It defeated world champion Lee Sedol in 2016 and several other high level players with moves that humans had very little intuition for.

• It created AlphaFold and AlphaFold 2 to take on the CASP protein-structure prediction competitions. AlphaFold 2 achieved a breakthrough at CASP14 in 2020. In 2021, DeepMind and EMBL-EBI launched the AlphaFold Protein Structure Database, and by 2022 it included predictions for more than 200 million protein structures. Hassabis later won a Nobel Prize in Chemistry for his contribution, along with John Jumper and David Baker.

AI Safety: Commercial time-to-market vs Research Discovery

While DeepMind racked up an impressive list of accomplishments in its early years, it entered a ferocious competition to develop LLMs against OpenAI, led by Sam Altman.

Much of the story that follows revolves around two concepts:

• AI safety

• Research vs Commercial Environments

I’ll cover the groundwork for each concept, then dive into the high level narrative of the competition between OpenAI and DeepMind.

AI Safety

Long before AI safety became a technical field, Sci-Fi gave the public a meaningful way to think about machine intelligence and control. “I, Robot” by Isaac Asimov in 1950, proposed the “the Three Laws of Robotics” that dictate that “robots must never harm humans, must obey orders unless they conflict with the first law, and must protect their own existence”. Additionally, classic Sci-Fi highlights the potential dangers of misuse of AI such as Terminator, Blade Runner, and Cyberpunk genres.

These examples influenced the development of AI where humans ensure that AI is safe and aligned with human intentions. Humans need to ensure that AI doesn’t deceive users by providing harmful / misleading information.

One way to test for AI safety is through adversarial testing, or “red teaming”, where human testers act as adversaries to intentionally break, deceive, or trigger harmful outputs from AI systems. Red teams simulate real-world attacks to identify safety, security, and ethical flaws, such as prompt injections, bias, or data leakage, before malicious actors exploit them.

As AI models scaled in complexity, they became harder to interpret. Inputs could be fed in during training and outputs could be evaluated during inference, but the internal representations became increasingly difficult to explain. AI can produce human-aligned outputs 99.9% of the time, but identifying the 0.1% of cases where it acts against human intentions can be difficult.

Red teaming alone does not guarantee total safety, but is one tool in a broad range of strategies to ensure that AI is aligned with human intentions.

Research vs Commercial incentives

In large tech corporations such as Google, there are two main categories of groups that contribute to business revenues:

• Revenue-generating core business - Engineers who create products to take to market or maintain/update existing infrastructure. Here, work is considered to be low risk and execution on set priorities is important.

• Research and development - Researchers who work on high-risk, long-term, and ambitious ventures. In R&D environments, ideas are incubated to either be published or folded into products to take to market.

Organizations consist of groups that range from commercial readiness vs research focus that help sustain the revenue of the core business as well as bet on “moonshot” ideas that can generate revenue in the future.

In 2015, Google restructured under the umbrella holding company “Alphabet”, separating its core businesses from longer-term moonshot bets such as autonomous driving, fiber, and life sciences. This move positioned Google to overcome the “innovators dilemma” by incubating capabilities to enhance its core business or act as additional revenue generating streams in the future should outside disrupters arrive and threaten it.

DeepMind, HQed in London, was acquired by Google because its capabilities were potentially useful to the core business. However, it was heavily siloed from the main HQ in California at first. This siloing gave the researchers to environment to produce many of the key innovations but caused it to hesitate entering the market when commercial pressure arose.

OpenAI, on the other hand, is HQed in San Francisco, California, with a highly influential leader, Sam Altman, and the entire network and talent pool of the Valley nearby.

OpenAI - Fast to Market, with Safety Governance Under Pressure

OpenAI was founded in 2015 by Sam Altman, Elon Musk, Greg Brockman, and others as a non-profit with the primary goal of ensuring that artificial general intelligence (AGI) benefits all of humanity.

In 2020, OpenAI published “Language Models are Few-Shot Learners,” introducing GPT-3, a 175-billion-parameter autoregressive language model, building off of the “Attention is all you Need” paper published in 2017. This was one of the first truly major, large-scale Large Language Models (LLMs).

However, as models were becoming more complex, AI safety became a concern. Former OpenAI employees including Dario and Daniela Amodei defected and founded Anthropic in 2021 to prioritize AI safety, alignment, and research.

To improve AI safety, OpenAI released a paper in 2022, “Training language models to follow instructions with human feedback” which used Reinforcement Learning with Human Feedback (RLHF). It demonstrated that humans are able to “teach” AI outputs that aligns with humans intention so that the AI can be trained more effectively. This paper demonstrated improvements in how humans can “guide” AI to produce output that aligns with its intentions.

On November 30, 2022, ChatGPT was officially released by OpenAI. It was intended to be introduced as a “research preview” and fast tracked to not let Anthropic get ahead. It quickly went viral, reaching one million users within a few days of its launch, the fastest growing consumer application ever at the time (Threads would later surpass this).

About a year later, the board was concerned about Sam Altman’s handling of AI safety and other allegations. The board abruptly fired Altman on November 17, 2023, citing concerns about his candor. But the move triggered a rapid employee and investor backlash: nearly all OpenAI employees threatened to leave unless he was reinstated, and Altman returned as CEO on November 22, 2023 with a restructured board.

Google DeepMind - Prioritizing AI safety, but Caught in the Innovators Dilemma

Google saw commercial potential in AI when it first acquired DeepMind because it recognized the strategic importance of AI in its core business.

At first, DeepMind’s culture leaned toward research depth, scientific credibility, and safety caution. It saw too much existential risk to let commercial priorities drive AI’s deployment, which made it slower to convert frontier models into consumer products. Demis wanted a path to spinout to a semi-independent Alphabet company, but ultimately didn’t due to the legal effort involved.

When OpenAI released GPT-3 in 2020, DeepMind did their own work on LLMs and published a few key papers, including Flamingo, Gato, and Chinchilla in 2022. In Sept 2022, it worked on Sparrow, DeepMind’s version of ChatGPT, which used RLHF. However, it was ultimately not released to the general public.

When OpenAI released ChatGPT, Google was briefly trapped by the innovators dilemma where it would put its brand at risk if it released an AI model too early and it hallucinated. During this time, a lot of DeepMind researchers were being poached by OpenAI who were frustrated by lack of progress by Google releasing a product.

On April 20, 2023, Google Brain (aimed at applied machine learning for Google products like TensorFlow) and DeepMind merged to form a single unit, Google DeepMind, to become one comprehensive unit.

Google DeepMind ultimately launched Gemini in Dec 2023 and upgraded versions that followed. While it is widely used nowadays and offers many benefits over ChatGPT (such as much larger context window for large documents), it ultimately suffered from the first mover advantage OpenAI had.

Key Technical Developments in modern AI

With that institutional backdrop in place, the technical story becomes easier to understand.

Next I’ll dive deeper into the key technical developments along the way to modern AI referenced in the book that approximate human intuitions. This is by no means a complete list since this book does not cover the more recent “reasoning era” Agentic AI.

Deep Reinforcement Learning (2013-2015)

Reinforcement learning is a fairly intuitive, but math-heavy, ML method where an agent interacts with the world and periodically receives rewards / reinforcements that reflect how well its doing. Reinforcement learning is especially popular in environments where there are small number of training examples compared to the state space the agent exists in (like chess) that supervised learning would struggle in.

Each agent exists in an environment that resembles a Markov decision process, such as a maze, with other states that the agent can transition to. The agent has a policy that determines whether it will explore the environment or exploit an action with known outcomes. Each agent goes through several trials of decisions, determines which actions likely led to the desired outcomes, and updates its policies with what it learned.

One popular method of reinforcement learning is Q-learning (or Quality) where the agent estimates the max expected future reward for each action in each state. While Q learning is fairly primitive, it forms the basis for more sophisticated algorithms.

In a 2013 paper, “Playing Atari with Deep Reinforcement Learning”, Hassabis attempted to combine deep and reinforcement learning to create a single NN agent that is able to learn to play as many of the games as possible without knowing the rules in advance [1]. However, combining deep learning and reinforcement learning presented some challenges:

• Deep learning applications require large amounts of hand-labelled training data (at the time). It also assumes samples to be independent.

• RL algorithms, on the other hand, learn from a scalar reward signal that is frequently sparse, noisy and delayed. It also frequently encounters highly correlated states.

DeepMind used Deep Q-Networks (DQN) to play Atari games. They use a method of “experience replay” where the agent’s experiences are stored at each time step, then are sampled uniformly when performing updates. Randomizing samples helps break up correlations and allows the DNN to learn more effectively.

They demonstrate this approach on seven Atari Games (Beam Rider, Breakout, Enduro, Pong, Q*bert, Seaquest, and Space Invaders) and showed that it generalizes to a variety of games without specific information about each game.

Transformer (2017)

The Transformer architecture was introduced in the June 2017 seminal paper "Attention Is All You Need" by researchers from Google Brain and Google Research. It is widely considered one of the most important, consequential, and influential AI papers of the 21st century because it influences the architecture of Large Language Models we use today.

Prior to the transformer model, the standard approach for language training were Recurrent Neural Networks (RNNs) and sequence-to-sequence models with Long Short-Term Memory (LSTM). A RNN/LSTM processes words sequentially. This is useful in simple, condensed sentences structures such as subject-verb-object pairs. However, it suffers from modern sentences especially when meaning between words (both local and global context) are far away from each other in time.

The proposed “Attention” mechanism allows each word context into what other related words are doing. To get an idea for how the Attention mechanism works, lets consider real sentences.

Real sentences can have multiple parts of speech, with nouns (things), adjectives (modifiers to nouns), verbs (actions) and other parts of speech. Nouns can take on multiple meanings depending on the “context” is it in. Take the word “apple” in the following sentences:

• Apple will release an iPhone next year.

• I ate an apple and it was delicious.

In each sentence, its clear to us what “Apple” refers to: a company in the first sentence because of the world “IPhone” that provided context, and a fruit in the second because of the word, “delicious”. We humans used context clues to determine what “apple” means based on supporting words around it.

This is roughly what “Attention” enables mathematically in the Transformer architecture.

In the original Transformer, words are processed in parallel through attention layers rather than one step at a time as in RNNs. These words are broken down into several units called “tokens”.

Then, each token is converted to a high dimensional vector called an embedding that captures the meaning of each token. Think of this vector like the entries of a thesaurus where each tokens’ values corresponds to how strong it correlates to each entry in the thesaurus. A word like apple can map well on the the position that represents companies (such as Tesla, Exxon) and another position that specific fruits (oranges, pears). These embedding values are found during training on billions of words since each embedding has multiple “dimensions” it sits on.

Then, through matrix operations, every word in a sentence calculates how much “attention” it should pay to every other word in that same sentence. This allows it to figure out relationship between words based on context clues.

To accomplish this, each embedding vector is decomposed into three vectors : Query (Q), Key (K), and Value (V). The query vector for each word is then multiplied by the key vectors for all the other words to come up with an “Attention” matrix that represents how related words are to each other. This attention value is given by the following formula:

To predict the next word in the sequence, it uses a decoder to look at words in context to be able to generate the sequence of words that map well to the original meaning of the sentence.

AlphaFold 2 also showed how attention-like architectures could move beyond language: it used attention over sequence and pair representations to reason about relationships between amino acids, including residues far apart in the 1D sequence but close in 3D structure.

The transformer is significant because it is scalable along memory and compute:

• The context window represents the “short term memory” as the maximum number of tokens (words) the model can “see” in one single look.

• The main attention calculation is a matrix multiplication, so modern GPUs can parallelize these matrix multiplications to higher embedding dimensions.

Sentiment Neuron (2017)

The paper “Learning to Generate Reviews and Discovering Sentiment” was released by OpenAI on April 5, 2017. This paper was consequential because it became an early and memorable example of a neural network learning an internal representation that researchers did not explicitly program.

Originally, the researchers at OpenAI trained a specific type of RNN called an mLSTM (Multiplicative LSTM) on 82 million Amazon reviews with unsupervised language modeling. They found that one single neuron had essentially "volunteered" to track sentiment of whether parts of the review were positive, neutral, or negative.

This paper is considered highly influential to AI safety because it gave insight to researchers that certain neurons can be “inspected” to find big picture behavior and debug potentially dangerous behavior.

Few-shot and in-context learning in GPT-3 (2020)

Since the transformer paper was published, the standard approach for LLMs was to pretrain the models on a ton of data and fine tune the model for each specific task, such as reading comprehension and answering questions.

In Apr 2020, when OpenAI released GPT-3, it published a paper, “Language Models are Few-Shot Learners”. This paper presents “Few-shot learning” where a model learns to perform new tasks or recognize patterns simply by providing examples in the prompt, a capability known as in-context learning. It demonstrated that a generalized LLM model could be taught to solve new tasks using only a very small amount of training data. Other settings such as zero-shot and one-shot were shown with some accuracy loss compared to few-shot at the time.

Together, the Transformer architecture and GPT-3 helped make scaling a central paradigm in AI: as model size, data, and compute increased, capabilities improved in increasingly predictable ways.

Reinforcement Learning with Human Feedback (2022)

When LLMs began scaling in complexity, they sometimes generated outputs that were not aligned with their users, such as toxic or biased statements. The paper “Training language models to follow instructions with human feedback” used human-guided Reinforcement learning to “teach” the model what outputs are acceptable to humans.

OpenAI hired 40 contractors and trained a model, InstructGPT, with human feedback. They evaluated its responses vs GPT-3 and found that labelers significantly prefer InstructGPT outputs over outputs from GPT-3.

This procedure showed improvements in truthfulness and reducing toxicity. However, RLHF is hard to scale because human-provided feedback on AI is harder as AI generated more sophisticated outputs.

Chain-of-thought Prompting (2022)

The concept of Chain-of-thought prompting was introduced in the 2022 paper, “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models”. CoT was a major demonstration that providing intermediate reasoning examples to pretrained LLMs can improve performance on arithmetic, commonsense, and symbolic reasoning tasks in sufficiently large models.

CoT also made model outputs more interpretable at the surface level, because humans could inspect the generated intermediate steps, though those steps should not be treated as a guaranteed faithful transcript of the model’s internal computation.

This approach is similar to how humans learn by analogy/example. It is typical for humans to decompose the problem into intermediate steps and solve each before giving the final answer. Most of us were taught to “show your work” when solving complex problems and were rewarded partial credit even though our answer might be wrong, since the line of reasoning can be valid up to a certain point.

Anthropic - Interpretability (2021-)

Anthropic was founded in 2021 by former OpenAI executives Dario and Daniela Amodei and other colleagues. It was established as a public benefit corporation (PBC) dedicated to AI safety, focusing on creating reliable, interpretable, and safe AI systems (like Claude) following disagreements over the rapid commercialization and safety approach at OpenAI.

Anthropic’s safety research pushed hard on mechanistic interpretability: trying to identify features, circuits, and internal representations that help explain why a model produced a given output.

This matters because interpretability is one possible path toward moving AI safety beyond trial-and-error behavior testing. Red teaming can reveal when a model fails from the outside; interpretability tries to understand why the model behaves that way from the inside.

Conclusion

Modern AI did not emerge from a single breakthrough, company, or personality. It came from the convergence of several technical ideas: deep reinforcement learning, transformers, scaling laws, RLHF, chain-of-thought prompting, and interpretability — with a set of organizational tensions that shaped how quickly those ideas moved from research labs into the hands of the public.

Demis Hassabis and Sam Altman represent two different archetypes in this story.

• Hassabis reflects the research-driven, long-horizon pursuit of intelligence as a scientific problem.

• Altman reflects the commercial, political, and go-to-market instincts required to turn frontier research into a mass-market product.

Both approaches have strengths and failure modes, and that tension is the real story of modern AI.

The next era of AI will likely be shaped not only by better models, but by how well institutions can balance those forces while building systems that are powerful, useful, and aligned with human intent.

If you want to continue expanding your knowledge on the HW side of AI, check out my other post:

References

[1] Mnih, V, et al. “Playing Atari with Deep Reinforcement Learning” https://arxiv.org/pdf/1312.5602

[2] Sutskever, I, Vinyals, O, Le, Q. “Sequence to Sequence Learning with Neural Networks” https://arxiv.org/pdf/1409.3215

[3] Vaswani, A, et al. “Attention Is All You Need” https://arxiv.org/pdf/1706.03762

[4] Radford, A, Jozefowicz, R, Sutskever, I. “Learning to Generate Reviews and Discovering Sentiment” https://arxiv.org/pdf/1704.01444

[5] Brown, T, et al. “Language Models are Few-Shot Learners” https://arxiv.org/pdf/2005.14165

[6] Ouyang, L, et al. “Training language models to follow instructions with human feedback” https://arxiv.org/pdf/2203.02155

[7] Wei, J, et al. “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models” https://arxiv.org/pdf/2201.11903

I attended ISSCC in 2026, which is the “Olympics” of chip design and one of the most highly specialized conferences in the world. During a plenary talk, Hope Giles, VP HW PM at Apple, emphasized that system thinking and cross domain awareness will become critical skills as semiconductors and technology become more complex.

And I absolutely agree.

However, the advanced nature of technology work nowadays makes it difficult to develop this type of thinking in practice.

I’m writing this post because I notice a huge gap amongst the way job roles and defined nowadays, and the type of cross-domain thinking that is actually a premium as systems become more tightly integrated.

The Generational Career Gap

I’d argue that there is a perspective gap amongst leaders/more senior engineers and junior/mid level engineers nowadays. As companies have gotten bigger and more integrated, the former group have scaled their experience along with the development of the technology. This allowed them to get a much broader perspective of entire systems earlier in their career when they were less complex, and build their career capital through incremental ownership of complex systems.

Now, as systems have become more complex, most job descriptions are specifically tailored toward specific subsystem ownership of more complex systems, not broad ownership of less complex systems. This is causing gaps understanding amongst engineers where they have trouble describing the impacts their role had on the bigger picture.

Why Cross Domain Thinking is Easy to Say but Difficult in Practice

Due to structural and psychological factors, cross-functional awareness is hard to develop in complex technology organizations because it tends to reward specialized subsystem ownership with handoffs at interfaces.

To coordinate peoples efforts in designing a complex product, organizations are set up to flow down requirements to subsystem executors, and later integrated at the end. This is an unfortunate consequence of how we must coordinate work to be able to create complex products that meet performance demands of the market.

Some organizations are especially notorious for keeping information under tightly guarded wraps on a need-to-know basis. For some organizations, this is to prevent IP leaks, while for others, its for national security reasons.

The consequence of this is that big picture assumptions often do not make it to subsystem executors, and can lead to inefficient work being done as information is thrown over the wall. There have been a few times where I’ve asked “can I better understand why you need this?” or “can I sit in a meeting to better understand customer needs” and either told no or gotten spoon-fed information.

• For instance, the Manhattan Project was one of the most heavily compartmentalized efforts in American history. Approximately 130k people worked on the project not knowing they were building a bomb. Only about 10 people knew of the projects purpose. This causes quite a stir amongst physicists like Richard Feynman, who thought it would make people “stupid and incompetent”. Compartmentalization even caused serious risks when workers handed Uranium without knowing what is was intended for. When these workers were read on and understood the purpose, they were able to better handle the uranium.

Cross domain thinking is also difficult psychologically because it requires you to admit to yourself what you don’t know. When you are an expert in a specific area, it often feels like a blow to the ego to not understand material in adjacent domains and to ask someone in that field to explain something simple to them. This requires a huge amount of psychological safety amongst members of an organization, which may not always be present. Not all organizations and teams really emphasize psychological safety or rather treat it as “fluff”.

Of course, businesses need to make money, execution on set priorities is important and information does need to be controlled to some extent.

How to develop Cross Domain Awareness

It can be very difficult and frustrating to develop cross-functional awareness in such environments that rely on sole handoffs at interfaces and deadline pressure. Trust me, I’ve had my experiences.

However, the two most powerful ways to develop cross-functional awareness within such organizations is by cultivating your own curiosity and by teaching others in small ways. Let me explain.

In my experience, which I teach something, whether as a TA in undergrad, mentoring others, or writing dense technical Substack articles, it forces me to consider what the learner doesn’t know. There’s also a risk of me being wrong when I put information out there. This forces me to do a ton of research and filter out complex information to convey what is important for them. In the process, it helps me sharpen my own mental models.

Teaching is more about the process of synthesizing information on your own so that both you and the learner mutually benefit from the information exchange. This is the reasoning behind the old adage, “You remember 90% of the things you teach others”.

Why we often mistake curiosity for consumption

However, when it comes to tech, the real risk is mistaking curiosity with consumption of information. What do I mean by this?

• Consumption is passive and mindless. When we consume information, such as YouTube videos or Substack articles, we tend to do so because the algorithm effectively lowers the resistance to seek out such information. You might read a nice article describing a complex system and thinking you gained something of value, and you might have indeed if you really focused. But unless you successfully explain to someone like they are 5, you haven’t really learned it.

• Real curiosity, however, is deliberate and requires focus. It forces you to admit to yourself what you don’t know, and force you to seek information to satisfy that curiosity.

When out jobs make us stressed, we tend to resort to consumption and escape at the end of the work day. This is understandable.

However, to develop a broad base of knowledge, we need to actively carve out small chunks of time as “deep work” times to let new material soak in.

How to Cultivate Curiosity

There are several little ways to cultivate your own curiosity. These include:

• Taking advantage of organizational resources.

• Going to the local/university library and look for technical and business resources. Read small units of chapters at a time.

• Attending internal seminars and asking the speaker questions afterwards

• Attending external conferences and reporting back on what you learned

• Asking questions of more senior people like “How did you get to your position”. (In my experience, people enjoy imparting their wisdom, but only if you ask them for it.)

• Reach out to people in other domains in the company and ask them what they do.

One thing that actually helped me cultivate real curiosity outside of work is Brazilian Jiu Jitsu. BJJ is a sport that humbles even the strongest people because in most cases, success is more about leverage and technique than pure strength. As a white belt you might be learning something new every class but not able to use it right away against more skilled opponents. Rolls feel frustrating. However, by continually showing up, you slowly accumulate knowledge little by little and a few months later, you know more than the new guy who just came in.

I personally BJJ is a perfect analogy of how real insight is developed: not through chasing quick trends, but slowly accumulating hard-won insight that compounds over time. When new people show up, you have a platform to share those insights.

Another useful thing that helps is integrating a mindfulness practice in your daily routine to help regulate your nervous system. Even if it helps a bit, it can open up small windows that all you to develop the curiosity needed to let new material soak in.

How to Teach Others

Once you’ve gathered enough insight, there are small ways you can teach others in corporate environments:

• Find ways to offer interesting insight to people during hallway conversations and handoffs at interfaces

• Propose a seminar in your organization about a past problem or conference you’ve been in

• Create internal “lessons learned” and forum posts on things you think would benefit others in the organization

• Find impressionable people, such as early career engineers, and offer to mentor them

You don’t have to go out of your way; just find little moments here and there to add value to others. Run little micro-experiments with people, see what works and doesn’t.

Eventually, these little efforts now can lead to much bigger efforts down the road such as organizing seminars and writing that you and other greatly benefit from. These also lead to visibility that helps during promotion decisions.

In some cases, teaching others may not improve your visibility within the organization. Even in these cases, you still benefit from the preparation that went into knowledge accumulation that can’t be taken away, unlike titles and status. It also helps future proofs your skillset in case the industry can and will take a turn. Trust me, you will want to ensure your skillset is sharp because you never know what will happen.

Conclusion

In my opinion, cultivating curiosity and teaching others around you is more beneficial for both your career and others around you in the longer run. Organizational execution and gaining visibility for promotions is also important, but should not be the sole focus for career success, even when it seems like everyone else is doing it.

Share

• I will introduce the conventional IM-DD optical approach with VCSELs used for short reach

• I will then highlight challenges with scaling conventional IM-DD to higher data rates

• Then I’ll explain techniques to improve data bandwidth:

  • 🔒 Co-packaged optics

  • 🔒 Silicon photonics

  • 🔒 External modulation

  • 🔒 Coherent optics

High speed optical interconnects have become one of the hottest markets in AI data centers. As bandwidth demand rises, the challenge is no longer just computing on data faster, but moving it across packages, boards, racks, and clusters efficiently.

The optical space has exploded over the past few years with many companies and startups proposing solutions to tackle data demands.

But I want to step back and ask ourselves: what are the first principles challenges with scaling high-speed optical communications?

The workhorse for short reach - IM-DD & Pluggable Optical transceiver

The workhorse for short reach optical communications today are IM-DD with pluggable optical transceivers. Lets take a looks at what each consists of.

Intensity Modulation - Direct Detection

IM-DD is the workhorse of short reach optics. IM-DD encodes information in optical intensity (whether that be 0/1 or higher levels in PAM4) and recovers it through direct detection.

This intensity modulation can be performed in one of two ways:

• Direct modulation - the laser is turned up/down by the electrical signal

• External modulation - a driver modulates a steady optical carrier

Pluggable Optical Transceivers

These IM-DD are typically packaged into pluggable optical transceiver modules that sit far away from the chip. These are are match-box sized modules that take in electrical (client) side data and convert it to optical (line side). Each block consists of following:

• Serdes + DSP - For serializing / desterilizing the data from/to host and perform equalization

• EIC - Electrical integrated circuit - consists of the electrical circuits fabricated in electrical processes (drivers, mixed signal conditioning, etc)

• PIC - Photonic integrated Circuit - consists of the optical circuits fabricated in optical processes (Si Photonics, VCSELs, etc)

Optical transceiver modules are a very mature technology that has scaled in data throughput for many years. However, with AI workloads demanding 224+ GPBS throughput capability, the performance of the optical transceiver itself needs to adapt, as well as the signal integrity between host die and optical engine.

System architecture breakdown of a IM-DD

Next I will break down the architecture of an IM-DD in each of its components:

• Transmitter

• Fiber

• Receiver

I will discuss several options for each and important design considerations.

TX Driver - Direct Modulation

Laser (Light amplification by stimulated emission of radiation)

Lasers are powered by a large pump current stimulating emission. Photons with energy close to the material bandgap interact with electron - hole pairs and cause identical photons to be emitted. This causes an “optical gain” of photons within the active region.

This photons is then confined using two mirrors: one opaque, and other that is semitransparent to allow the light to shine through.

Lasing occurs when the pump current is high enough such that optical gain > optical losses.

Then, either a cavity structure or frequency selective bragg reflectors selects which optical modes are sustained.

In a Distributed Feedback laser (DFB) laser, a periodic grating of Bragg reflectors provides wavelength - selective optical feedback, favoring a narrow lasing wavelength. These Bragg Reflectors are highly tunable depending on the index of refractions and the lengths and # of sections.

VCSEL (Vertical-Cavity Surface-Emitting Laser)

VCSELS are lasers that emit light vertically in a circular cross section. The active region consists of multiple quantum wells that enhances optical gain. Distributed bragg reflectors are stacked on the top and bottom of the active region with alternating refractive indexes. These layers are fabricated using epitaxial growth on a substrate.

The light is then confined with an oxide and current is pumped in through pads. The light is shown vertically above and redirected toward the optical fiber with optics.

The Pout/I of VCSELs almost resembles a linear diode. The VCSEL begins lasing above a particular threshold current and exhibit a linear relationship between Pout / Iin over a useful operating region. Lasers require a DC bias to set the device in the linear region.

One important parameter is the extinction ratio, which is the ratio between P1 and P0. A higher extinction ratio is preferred as it improves BER margin to detect teh signal at the other end, but higher extinction ratios require higher currents, and therefore power, to drive the device.

The two major materials used in lasers are GaAs or InP:

• 850 nm GaAs VCSELs are common for short-reach multimode links.

• 1310/1550 nm InP-based lasers/modulators are common where single-mode fiber, longer reach, or telecom/datacom bands matter.

VCSELs are very popular in short reach communications for a numbers of reasons:

• Low cost manufacturability - they are easy to test on-wafer across multiple samples

• Scalability - they can be formed in “arrays” to drive multiple optical lanes

However, VCSELs do have some drawbacks:

• Reliability - issues due to oxide confinement and high currents degradating mean-time-to-failure

• Signal switching nonidealities, including:

  • Turn on delay- a delay in the turn on of the signal that adds to latency

  • Jitter - uncertainty in the signal edge that degrades BER

  • Relaxation oscillation - an “underdamped” response in the turn-on characteristic that can affect BER and signal quality

The main tradeoff in VCSELs is between BW and reliability. Higher current densities allow for higher BWs, but it also decreases the mean time to failure, a common metric used when evaluating reliability.

Driving the Laser

Here are a few DC coupled, common anode circuits used to to drive the laser directly using either fully differential or pseudodifferential to convert a voltage to a current to drive the VCSEL. There are other methods of driving such as common-cathode, BJTs, and push-pull implementations depending on the requirements of the system.

In general, the more VCSEL current that is needed, the slower the driver will be due to increasing output capacitance.

Fibers

Multimode fiber optic cables are primarily constructed from silica glass (silicon dioxide). There are two major categories of fiber for high speed communications:

• Multimode - Uses a larger core (50–62.5 um) to transmit multiple light paths

  • There are five classifications, OM1-5, that grade the quality of the fiber.

  • Pros: Cost-effective for short-distance, high-speed applications

  • Cons: Limited distance due to higher signal modal dispersion

• Single Mode - Uses a smaller core (8-10um) to transmit a single mode of light

  • Pros: Low signal loss, avoids modal dispersion

  • Cons: Higher cost of laser components and more complex installation requirements

Generally speaking, multi-mode fibers are used for short reach interconnect due to its low cost and complexity. However, single mode fiber is being evaluated to “future proof” the data communications since it can support the higher BWs, though it adds to cost and integration complexity.

There are also four wavelength bands used in the fiber:

• 850nm band - used for short reach links

• O-Band - 1310nm

• C-band - 1550nm

• L-Band - Extends C band to allow 2x network capacity

There are four major impairments:

• Loss. Losses cause pulse attenuation after propagation from a fiber. These are caused by intrinsic sources, including

  • Rayleigh Scattering

  • Absorption

  • Manufacturing Stress

• Chromatic Dispersion - pulse spreading after propagation over the fiber. This is caused by fiber refractive index dependence on wavelength

• Modal dispersion - distortion due to different propagation delays of different optical modes. Modal dispersion matters in multimode fiber.

  

• Polarization Mode dispersion (PMD) - dispersion cause by pulse spreading of different polarizations due to asymmetries in the fiber core.

Receiver - Photodetector

A photodetector consists of a Reverse biased PIN junction that generates current proportional to the detected light. The transimpedance amplifier then amplifies this current to an output voltage at the other end.

There are some important specs of this system:

• PIN characteristics: Responsivity, capacitance, noise, dark current

• Receiver Q-factor: The received signal level relative to total receiver noise

• BER: BER can be derived from this Q factor: BER = 1/2*erfc(Q/sqrt(2))

A higher extinction ratio is desired to ease the burden on the receiver, but it does come with a higher power usage.

Why Scaling IM-DD is hard

Scaling IM-DD to higher data throughputs is hard due to several unfavorable tradeoffs between BW, reach, noise, power, SNR, and cost. These include effects in both optical and electrical domain themselves:

Optical

• Chromatic dispersion

• Polarization Mode dispersion (PMD)

• Nonlinear and interference effects

Electrical

• Noise penalties at Higher BW

• Tighter jitter, skew, and ISI

• BW scaling limits in front ends

You can have the fastest optical transceiver, fiber, but its pointless if the signal can’t get to it fast enough. Not only the optical receiver itself needs to improve, but the signal integrity from the electrical interconnect is running into challenges shuttling high speed data back and forth. Solutions need to ensure that speed “bottlenecks” within the system scale with the system demands.

This leads to four major trends that generally increase complexity:

• Co-packaged optics - embedding optical components within the package itself

• Silicon Photonics - routing light through silicon waveguides and modulating it

• External modulation - modulating a continuously running laser for higher BW

• Coherent Optics - modulation amplitude and phase in the optical domain

After the paywall I’ll discuss these in further detail.

Share

If you want to learn more about how optical communications interfaces with adjacent domains, consider checking out these other posts:

Read more

In this post I cover the following topics:

• Core Electrical Constraints of AI Loads

• Architectural Goals of AI Power Delivery

• Converting below 48V: The many possibilities

• 🔒Possibilities of 800V to 0.8V Conversion

• 🔒The Conventional PoL Approach: Lateral Multiphase Buck Converters

• 🔒Why PDN Constraints Push Regulation Closer to the Chip

• 🔒Why Vertical Power Delivery Is Emerging as the Endgame

AI compute is scaling so quickly that power delivery is becoming a first-order system bottleneck. Traditional 12V server racks were not designed for AI accelerators that operate at extremely low voltage, draw enormous current, and swing their load profile rapidly.

Over the past few years, AI power architecture has evolved quickly, with power demands projected to exceed 1 MW per rack and companies proposing solutions across every layer of the stack. Among those solutions, one direction stands out: vertical power delivery.

Vertical power delivery is emerging as the natural endpoint of a system that must deliver massive current into increasingly power-dense silicon. But that raises a few interesting questions:

• How did the architecture arrive there?

• What tradeoffs in power distribution make vertical power delivery an increasingly attractive solution at the point of load?

In this post, I build a holistic overview of AI data-center power architecture from first principles to explain why it is evolving in this direction. My aim is to keep the discussion high level enough to provide a map of how the different subsystems influence one another.

Much of the architectures in AI data centers derive themselves from the principles of buck converters. Buck converters are step down DC-DC converters and are the workhorse of most power electronics to bridge the gap between distribution voltage and the voltage that electronics require.

Core Electrical Constraints of AI Loads

Modern AI accelerators are among the most challenging electrical loads because they combine three properties that make power delivery unusually difficult: low supply voltage, high current density, and extremely fast transients.

High power density

GPUs consume massive amount of power due to low supply voltage and very high current:

• GPUs operate at very low supply voltage, typically around 0.6 V to 1.1 V. The supply voltage wants to be as low as possible because these devices switch an enormous number of transistors at high frequency, and a lower voltage reduces dynamic switching loss through the familiar P=CV^2*f relationship. Advanced CMOS devices also impose tighter voltage limits because thinner oxides leave less reliability margin. The downside is that at such low voltages, even a small amount of droop becomes significant.

• At the same time, GPUs can draw on the order of 1000’s of amps to support massive parallel computation. As compute density increases, the current required to support it rises as well. Since conduction losses scale with I^2*R, every bit of resistance in the path adds significantly more to loss.

Extremely fast transients (up to 2000A/us)

AI loads also exhibit extremely fast transients, sometimes on the order of 2000 A/µs. AI loads can change more quickly than a distant regulator can respond because much of AI computation is performed on-demand through large bursts of multiply-accumulate computations. That forces the regulator and the PDN to work together across a broad bandwidth, often with additional system-level support mechanisms.

Architectural Goals of AI Power Delivery

These electrical constraints force the AI power delivery architecture to optimize around two goals: efficiency and reliability.

Efficiency

Efficiency matters because even a 1% improvement can translate into major savings in heat, electricity, and cooling when multiplied across racks, clusters, and full data centers. That means losses must be evaluated across the entire chain, not just at the converter level.

• The converters contribute loss from conduction loss, switching loss, diode-related loss, and CV^2 loss

• The PDN contributes additional loss through planes, traces, vias, connectors, package parasitics, and capacitor ESR.

The interconnectedness of the power delivery system is what makes apples-to-apples comparisons across vendors and research papers so difficult. A converter may look excellent on paper, but that does not mean it is the best system-level solution once power density, component count, and PDN burden are considered.

Reliability

Reliability matters just as much. Data centers make money when they stay online and productive, so the power system must be designed with fault tolerance, serviceability, and safety in mind. As higher distribution voltages such as 800 V enter the architecture, maintainability and safe access become even more important.

Design Goals for the Architecture

The rack architecture for 1 MW-class systems exists to manage current over distance. At a high level, AI power delivery is trying to satisfy two goals at once:

• Transport energy efficiently from the power source to the compute load

• Regulate it precisely at the point of load.

This is why AI power delivery is fundamentally a distribution-and-distance problem. A data center contains large amounts of compute spread across multiple racks and shelves, and the goal is to move power to each GPU with minimal loss.

That naturally leads to a hierarchical architecture: distribute power at high voltage across longer distances, then step it down progressively as the delivery path shortens and the burden shifts toward precise near-load regulation.

These goals have led to the conventional modular stack: 800V → 48V → 12V → 1V

• High voltage minimizes current during long-distance transport amongst racks

• 48 V provides a practical intermediate distribution rail and minimizes PDN losses within the rack

• Lower voltages are introduced only as the distance to the load gets shorter

In short, the architecture tries to keep very high current confined to the shortest possible path near the load itself.

Converting below 48V: The many possibilities

Below 48 V, the design space opens up dramatically because there are many possible ways to reach the roughly 0.6 V to 1.1 V rail required by the GPU.

Broadly speaking, these approaches fall into either single-stage or two-stage architectures.

• In a single-stage approach, the converter attempts to step 48 V directly down to the load voltage.

• In a two-stage architecture, an intermediate bus converter first steps 48 V down to a lower bus voltage, which is then converted to the final load voltage by a PoL regulator.

The intermediate bus architecture is typically employed because it separates the conversion problem into two more manageable stages in the IBC and PoL. The IBC can focus on efficient transformation, while the PoL focuses on tight local regulation.

Within this architecture there are also commonly employed topologies for both the IBC and PoL and options for the bus voltage:

• Convert from 48V to 12V and then from 12V to 0.8V. 12V has a very robust server power infrastructure that can be leveraged, so that most engineering effort can be put in the 48V to 12V IBC conversion. This way, researchers can focus on just optimizing on either the 48V → 12V conversion or the 12V→ 0.8V conversion.

  

  

• Lower the bus voltage even further from 12V to 6/5/3V. Lowering the bus voltage eases the burden on the PoL since the Vout/Vin ratio is smaller. However, a lower bus voltage also increases the current between the first and second stage, and therefore, the PDN loss between those two stages. This solution pushes the IBC closer to the GPU to minimize PDN resistance, and this in turn pushes the PoL as close to the GPU as possible.

  

• Convert directly from 48V to 1V. Many research proposals attempt to directly convert 48V to 1V conversion, which typically employ two stage hybrid topologies. This option is very attractive and has a lot of potential, but current solutions don’t always scale to the require current densities and transient responses required.

Possibilities of 800V to 0.8V Conversion

After the paywall I’ll describe the following:

• A high level chart possibilities of 800V to 0.8V conversion and PoL delivery options with efficiency parameters

• The Conventional PoL Approach: Lateral Multiphase Buck Converters and TLVRs

• Why PDN Constraints Push Regulation Closer to the Chip

• Why Vertical Power Delivery Is Emerging as the Endgame

Share

If you want to continue expanding your knowledge of the subsystems that underlie power systems, I have a post on the tradeoffs inherent in PMIC design using the buck converter as a design case study.

Read more

As data speeds climb from 112Gbps to 224Gbps and 448Gbps, data starting running into physics constraints and transceivers become more complex to compensate for these effects.

In this post I’ll be doing a system architecture breakdown of a modern DSP-based transceiver. I’ll explain the underlying physics constraints and the major blocks data passes through. Then, I’ll do a deep dive into the high-speed building blocks (shown in yellow) on the TX and RX side. I’ll be covering the following:

• The real problem: increasing data throughput under physical limits

• System Datapath overview

• TX Architecture:

  • Serializer / Mux tree

  • DAC - 6-7 Bit current steering DAC

• RX Side - Time-Interleaved ADC

• 🔒Clocking Architecture

  • 🔒Clock Recovery: Bang Bang Phase Detector

  • 🔒Clock Generation: LC-VCO Based PLL

  • 🔒Multi-phase clock generation: The phase Interpolator

  • 🔒Clock Distribution: Repeater, transmission line, and resonator-based

Most of the figures in these slides are from the ISSCC tutorials as well as the papers in the references.

The real problem: increasing data throughput under physical limits

As data rates climb, we need to ask ourselves: what fundamentally constraints the data speed from first principles?

Data speed is now limited by physics and device parameters on the chip, which include, but not limited to, the following:

• Channel Loss: Loss scales with signal frequency, and high frequency attenuation rounds out the sharp edges of digital signals

• High impedance nodes in analog circuits: Active loads have a high input resistance, which is good for gain, but also creates dominant poles that lower overall BW.

• On-Chip Capacitance: Bigger transistors have bigger C that can affect rise/fall times of signals

• Noise / Jitter: The randomness in clock / data edges affect how fast those signals can be relative to the clock period. This impacts reliability of high speed digital circuits and time-interleaved switches

What tricks do designers have to increase speed? I contend there are two major paradigm shifts:

• Current steering circuits: These circuits steer constant currents through switches to a resistive load. The main advance is that it minimizes charging/discharging time of parasitic capacitors. However, current steering circuits tend to consume more power than equivalent charged-based topologies.

• Time interleaving: Instead of making a single component faster, place many in parallel and distribute data through “ranks” of S/H and T/H switches. Time interleaving introduces added complexity in clocks to coordinate parallel operations

System Datapath overview

Next lets look at the major blocks in the datapath.

The function of a transceiver is quite simple: transmit a large amount of data from the TX and RX with minimal errors across a medium as fast as possible.

For short reach, low speed operation, the architecture relied mostly on analog blocks. However, due to recent data demands, one of the big paradigm shifts is a move to digital for the processing. Digital is great at performing parallel processing of data at a moderate clock (1-10’s GHz) frequency.

However, when it comes to data transmission, edge density becomes the constraint. You can only pack so much circuitry to drive the parallel channels on the edge of a chip and copper cables in the system.

As a result, data transmission favors serialized data at a high speed to pack as much data density as possible and transmit on a single cable.

These constraints influence the architecture of modern wireline transceivers.

On the TX Side, the main high speed blocks include:

• Forward Error Encoding (FEC) - Encodes the data with parity bits that correct for channel error at the RX side

• Feed Forward Equalization (FFE) - Pre-distorts the signal using a FIR filter to compensate for channel losses

• Serializer - Serializes the digital data from multiple channels and raises the effective output rate. 112 Gb/s PAM4 corresponds to a 56 GBd symbol rate, though practical serializers often use half-rate or quarter-rate architectures that are time-interleaved

• High-speed DAC - turns digital symbols into an analog waveform

• Driver - amplitudes the signal before it is transmitted on the channel

In the Channel:

• Matching network - matches the TX and RX to the characteristic impedance to avoid reflections and transmission line effects

• Channel - attenuates and distorts the transmitted signal

In the RX:

• Receiver front-end - conditions the signal by amplifying it (variable gain amplifier) and providing equalization (CTLE)

• TI-ADC - converts the analog value to digital values

• Deserializer - converts the digitized data from the transmission rate to the digital frequency

• DSP - processes and recovers data

As a result of the added digital complexity, high speed and parallel processing blocks require a robust clock architecture to synchronize the data properly.

In the following sections, we’ll step through each of the high speed blocks, with the principle of operation, the key impairments, and how they affect the system level.

TX Architecture

Serializer / Mux tree

The mux tree is used for serializing / deserializing data. Here, stages of 2-1 muxes are arranged in ranks. This ensure each interconnect between muxes aren’t heavily loaded that can slow down operation.

The final rank operates with the highest-rate clock, while earlier ranks use divided versions of that clock so that each stage runs at a rate compatible with its local data rate.

A common high-speed mux cell is the current steering mux cell. This cell uses two differential pairs for each of the inputs. To select each input, the SEL signal, synchronized to the CLK, steers the current between the two input branches.

Impairments

• Kickback noise from clock feedthrough. Clock feedthrough and switching transients can disturb internal common-mode nodes and degrade timing margin, especially in the final high-speed stages.

• Skew between SEL and clk.

• Skew between stages in the MUX tree. The accumulated clock delay associated with the driver and the mux itself can cause ranks to fail to clock the data fast enough.

DAC - 6-7 Bit current steering DAC

The high speed DAC commonly employs a segmented current steering architecture. This architecture uses scaled current sources representing the “weight” of the binary bits. The steered currents are summed and converted to an output voltage across a load impedance.

The current steering cell consist of two main branches and a selector with a flip flop and driver. This current source is generated from the bottom two transistors. The driver cell “steers” this current between two branches: the left branch to enable the bit and right branch to disable it. The switch control is retimed so current steering occurs with precise timing at the symbol rate.

Current continuity is often preferred for high speed because it minimizes voltage swings from constantly charging / discharging capacitive nodes. However, it also consumes static power even when the bit is disabled.

This current cell is then scaled to a segmented architecture, which uses both unary and binary DACs. Unary (or thermometer code) uses unit-sized current sources arrayed in a 2x2 grid in layout to achieve better matching of current sources.

In many wireline transmitters, DAC resolution is kept modest, often around 6–7 bits, because channel loss, jitter, and front-end noise dominate before very fine quantization does.

Impairments

There are several impairments that cause both timing and amplitude errors:

• Time delay

• Voltage glitches

• Clock Skew

• Finite Input impedance

• EMI at all sources

  

These impairments introduce timing and amplitude error at the system level. They translates to jitter in the x domain and noise in the y dimension. These effects directly affect the size of the “eye” at the RX side, so it is important to account for these and spec these appropriately.

RX Side - Time-Interleaved ADC

A common modern RX approach uses time-interleaved asynchronous SAR ADCs operating in a self-clocked manner.

In normal asynchronous SARs, it takes N clock cycles to complete an N bit conversion. In asynchronous circuits, self-clocking allows the completion of one comparison to automatically trigger the next.

Self-clocking removes the need for a very high external clock, but it makes the conversion timing dependent on timing delays and handshake robustness.

Asynchronous SARs are commonly used due to low comparator count, simple digital logic, and capacitive DACs for energy efficient operation at typical 6-7 bit resolution.

These SAR ADCs are time interleaved. The high speed data is “fanned out” to each of the sub-ADCs in order for their analog values to be digitalized. This is accomplished through a two-rank NxM interleaving scheme. This architecture performs sampling in two ranks:

• Rank 1 - Track and hold. The first rank tracks and holds the signal. The performance of these switches are critical as they sample the full BW signal.

• Rank 2 - Sample and hold buffer. These buffer the T/H signals to drive the parallel ADCs.

In modern PAM-4 112Gbps, the interleave factors range from 36-128 sub-ADCs depending on the layout area, with 64 being common. [1]

Impairments: There are several impairments:

• Skew and Jitter of clock

• Mismatches between sub-ADCs

These impairments translate timing mismatches into sampled voltage errors, often in a signal-dependent way, and can create both noise-like degradation and spurious tones.

Clocking Architecture

Perhaps the most important part of a high speed chip is the one everyone takes for granted: The clock.

The clock distribution becomes an increasingly important constraint as dimensions and speeds have scaled down. There are several challenges the clock faces:

• The clock needs to be recovered from the asynchronous RX data to ensure that the clock is in “sync” with the data to clock it appropriately

• The VCO needs to generate a reference clock with low jitter

• The time-interleaved blocks need some form of multi-phase clock to synchronize operations across parallel processing elements cleanly

• The parallel clock distribution needs minimal skew to minimize mismatches

After the pay wall I’ll describe the clocking architecture in more detail.

Share

If you want to get a flavor for the adjacent domains involved, I have several other posts to check out as well:

Read more

In this post I cover the following topics:

• SerDes - The Workhorse of chip to chip communication

• The Root Causes of Signal-Integrity Degradation at High Speed

• How Channel loss and Jitter degrade Bit Error Rate

• Major System Trends:

  • PAM-4

  • DSP-based Architectures

• 🔒Error Reduction Techniques

  • 🔒Forward Error Correction

  • 🔒Equalization

    • 🔒TX - Pre-emphasis / FFE

    • 🔒RX - CTLE and DFE

• 🔒How to Achieve Higher Data Rate

Everyone talks about the importance of chip-to-chip communication, but few people appreciate the complexity hiding underneath it.

Signal integrity and SerDes architecture are often discussed as if they belong to different worlds: one in the lab and on the board, the other inside the chip. In reality, they are deeply coupled.

My goal in this post is to show how signal-integrity constraints at the channel and board level—loss, jitter—affect performance through BER and ultimately drive complexity back into the transceiver itself, shaping choices in equalization, clocking, and data conversion. Here I’ll focus on the signal-integrity foundations from first principles, then build on that in a follow-up post on chip architecture.

Much of the content of this post is material consolidated from a Tutorial given by Dr. Mike Peng Li from DesignCon 2026, the premier high-speed communications conference, as well as from ISSCC, academic papers and SI books. Figures are reproduced for commentary and educational purposes.

SerDes - The Workhorse of chip to chip communication

A serializer-deserializer (SerDes) converts parallel digital data into a high-speed serial stream for transmission, then reconstructs it at the receiver. This allows systems to move large amounts of data between chips and systems without requiring a huge number of parallel wires.

SerDes and optical links are the “workhorses” of communication within data centers in “scale up” and “scale out” networks:

• Chip to chip

• Modular top chip copper links on a PCN and an optical module

• Between a CPU and network interface

• Backplane and midplane links over backplane PCBs or passive copper cables

SerDes is what enables parallel computing across many CPUs and GPUs as a means of data distribution.

Historically, many SerDes architectures have been analog-based. SerDes speeds have scaled with PAM-2 (NRZ) signals over time to meet the growing needs of traditional computing.

Then, AI and chiplets came in and started demanding more data.

These two technological trends put a lot of pressure on chip-to-chip communication to get faster and reach farther due to the massive amounts of data movement:

• AI workloads involve massive data movement: repeated memory access, movement of intermediate results, and parallel processing across many GPUs in a rack.

• Chiplets disaggregate a system into multiple smaller dies, which must then be tied back together through high-speed chip-to-chip interfaces.

However, as data demands push speeds higher and links longer, fundamental physics begins constraining performance:

• Channel loss due to higher attenuation at higher frequencies

• Jitter budgets getting tighter

• Transmission line effects that cause distortions

• Noise that can cause errors

These add complexity to the SerDes to correct for. Unfortunately, the added complexity makes it more challenging to understand how one part of the system affects the other, such as:

• How jitter in the VCO eats into the timing margin needed to meet the target BER

• How receiver clock skew and TI-ADC area constraints affect sampling margin, data throughput, and calibration burden

In the following sections I’ll be focusing on signal integrity fundamentals from first principles. The goal is to ask:

• What are the fundamental physics constraints limiting performance?

• What complexity in the chip architecture is needed to scale speeds beyond 112.5Gbps to 224Gbps and eventually 448Gbps?