While a lot of attention is being drawn to the current HBM4 shortage, this post will focused on the practical technical challenges in scaling HBM5 performance.

I will discuss the following signal integrity and co-design challenges in HBM5 based on SK Hynix and KAIST’s ECTC 2026 papers:

• Introduction to Signal Integrity trends from HBM1 to HBM5

• Jitter component tree and descriptions of key sources of jitter for short, high-speed interconnect:

  • Echo

  • SSC / SSN

  • PSIJ

• How SK Hynix quantifies transmission line effects in high-speed HBM5 data lines

  • Fundamentals of Transmission Lines

  • Proposal for two operating regions: RC and LC dominated regimes

  • 🔒Design of Experiments results for echo jitter across interconnect lengths and dimensions

• 🔒KAISTs SI/PI co-analysis framework for Chiplet (UCIe)-based GPU-HBM Interconnect for PSIJ

  • 🔒Shoreline density limitations for a 4 layer wide-IO and chiplet interconnect scheme

  • 🔒Calculating power-supply induced jitter (PSIJ) from SI, PI, SSC, and jitter sensitivity / amplification

• 🔒BONUS: “The Brain” - 3D Custom HBM and current status among TSMC, SK Hynix, NVIDIA, and Samsung from ECTC 2026

This post is heavily influenced by the ECTC session 11, “Signal Integrity Design for High-Speed Interfaces” as well as my experience at DesignCon 2026 I was personally in attendance for.

This will be more of an advanced deep dive, but grounded in fundamentals. I structured the post to start with baseline EE concepts and building a framework to understand the complexity of multi-layer SI/PI co-design. I have a post that covers the fundamentals of signal integrity that forms the foundation of the issues I discuss:

I encourage you to read through my backlog of knowledge and challenge you to take away one new in-depth concept. Even if the technical material doesn’t stick, I still hope you come away with an appreciation of the challenges that keep HBM and SI engineers up at night.

Note that I am not a signal integrity person. However, I immersed myself into the SI world by attending DesignCon 2026 with an RF / Microwave lens upon which to analyze high-speed signal integrity effects. It was clear upon reading these papers that memory and SI engineers do not speak the same language as RF people do, but they describe the same effects from their point of view.

I personally think that at increasingly faster data rates, there will be a convergence of the knowledge among RF / microwave and signal integrity domains. I aim to bridge this gap of understanding.

SK Hynix - High speed HBM roadmaps

High bandwidth memory (HBM) is a critical component for AI training workloads that stores everything involved in AI computation from model weights, gradients, optimizer states, and activations. HBM connects multiple DRAM dies vertically using TSVs to maximize memory density in the given footprint. These DRAM dies are placed as close to the GPU as possible to maximize the data throughput to overcome the Von-Neumann bottleneck.

In this SK Hynix roadmap presented at ECTC 2026, there are a few performance scaling trends along HBM generations:

• Between HBM1 and HBM4, the datarate per DQ increased linearly from 1Gb/s to 11.7Gb/s

• Along those generations, the signal rise time and unit interval (UI) scaled inversely proportional to the total data rate per DQ

• Between HBM3E and HBM4, there is a stairstep jump in the the total bandwidth by 2.5x, primarily driven by doubling I/O per cube from 1024 to 2048.

To keep with up AI workload demands, HBM5 expects a 20-30 Gbps per DQ range using state-of-the art interposer technologies such as the Chip-on-Wafer-on Substrate (CoWoS-L and CoWoS-R).

At those data rates, HBM is running into co-design challenges along the signal integrity, power integrity, and thermal domains. At ECTC, there are three high-level trends I noticed amongst three papers:

Jitter due to Transmission Line Effects

SK Hynix notes that significant transmission line effects were not observed for previous HBM generations with a data rate of under 10 Gbps per DQ and approximately 6-mm-long interposer interconnects.

However, at 30Gbps rates, advanced packaging interconnect technologies exhibit distinct lossy transmission line characteristics and must be accounted for.

Normally, transmission lines are terminated with equivalent impedances to avoid reflections. However, termination resistors cannot be reasonably employed for terminating 1000+ I/O in HBM as this would lead to high thermal penalties and static power dissipation. This leads to additional signal integrity challenges that will be discussed.

From “Wide-and-slow” to UCIe

Another challenge is edge density limitations of I/O. Doubling the number of I/O pins results in excessively large physical layer (PHY) footprint on the silicon. The number I/O is fundamentally constrained by the metal pitch, # of layers, and the area dedicated to ground rails to control crosstalk.

To scale the number of I/O, KAIST is investigating the performance impact of going from wide-and-slow I/O toward higher-speed SerDes lanes in Universal Chiplet Express (UCIe) standards.

KAIST notes that in chiplet-based GPU-HBM based on UCIe, eight D2D modules are utilized, with each module consisting of 64 Tx and Rx at 32Gb/s each. A total of 512 Tx and Rx supports 2 TB/s for read and write directions.

With a faster, more compact PHY, more I/O them can fit in the HBM shoreline to increase the data throughput.

PDN for 3D stack

Another co-design challenge is how the TSVs in the HBM stack affect thermal characteristics.

This paper evaluates the thermal and IR drop characteristics of different TSV configurations. Here we see how each component is modelled as an array of unit cells in a grid structure with an equivalent RLC circuit model.

Though I think this paper is important for effective co-design, especially when it comes to modelling PDNs for PSIJ, I have left the analysis out of this post for now to focus on signal integrity challenges from the first two papers.

Key sources of jitter in high-speed, low-voltage parallel lines

Jitter is a key constraint of high speed signal integrity and must be accounted and minimized as much as possible.

When high speed data is sent to and from the HBM, each bit is received within a given time slot refers to as the “Unit Interval” (UI) or Bit-period. Jitter and slew rate constraints reduce this to an effective aperture “window” where data can be received. Throughout the post, all aperture results are normalized to a UI of 1.

There are many difference sources of jitter as shown in this comprehensive jitter tree from Dr. Mike Peng Li’s DesignCon 2026 tutorial slides.

Jitter is broken down into two main statistical categories: deterministic and random jitter. I describe these in more detail, along with equalization techniques, in the following post:

Under deterministic jitter, there are three primary classes of jitter relevant to high speed, massively parallel, non-terminated lines:

• Echo Jitter - Jitter caused by impedance mismatches and reflections

  • When a signal propagates down the line and encounters an impedance mismatch, part of the signal reflects backward and forwards, resulting in a stored “memory” and overshoot that affects future bit measurements. This can lead to ISI.

• Bounded Uncorrelated Jitter - Jitter that is bound but uncorrelated to the signal

  • This includes crosstalk across massively parallel lines where there are many different possible combinations of crosstalk interactions, but the overall jitter impact is “bounded” to worst case conditions.

• Periodic Jitter - Jitter where the timing variations repeat in a cyclical pattern over time at a specific frequency

  • This includes simultaneous switching noise that occurs every switching period.

One source of jitter that is particularly problematic at increasing data rates is power-supply induced jitter (PSIJ) which is a form of power-noise-induced timing degradation within circuits themselves. This is a challenge to model because it combines jitter from a few sources.

The rise / fall time and propagation delay of a circuit depends on how quickly FET caps can get charged by the PDN. When gates are switched, they generate simultaneous switching current (SSC) that go through the PDN and cause simultaneous switching noise (SSN) on the line. Noise on the PDN can affect the current your caps charge at, introducing timing uncertainty.

PSIJ becomes more problematic as you stack and cram more HBM on the same PDN and the PDN line becomes noisier. With increasing data rate, thousands of I/O drivers switching simultaneously, and a low supply voltage, a marginal increase in PSIJ can eat into the jitter budget and lead to signal failures. A co-design approach to modelling PSIJ will be discussed in more detail behind the paywall.

How SK Hynix must incorporate transmission line effects in high speed HBM data lines

Now lets discuss some of the high-speed SI challenges SK Hynix is facing from transmission line effects. I’ll start with a broad introduction of transmission effects from an RF point of view, the nuances as it relates to SI, then jump right into SK Hynix’s analysis after the paywall.

From Lumped to Distributed Abstraction

Most circuit analysis (like V = IR) is initially taught in the “lumped” abstraction where component values are “lumped” or simplified into devices consisting of resistors, capacitors, and inductors. Lumped abstractions form the basis of most traditional circuit analyses like nodal, mesh, etc.

The important thing about a lumped abstraction is that the propagation characteristics of the physical connections between lumped components can be reasonably neglected for most practical applications if the frequency is low enough.
When the frequency is low enough, the signal level will be virtually the same across all lengths of the wire since the capacitors distributed throughout the line charge nearly instantaneously.

However, at high frequencies, the signals have to physically propagate down the wires to get from point A to B, so the physical characteristics of the interconnects are important. This is referred to as the “distributed” abstraction where the electromagnetic wave propagation characteristics along a transmission line matter. Notice how in the figure above, when a high frequency wave is sent down a line, its voltage rises at different time points.

When deciding on whether to model circuits with lumped or distributed elements, the rule of thumb I was taught in RF classes is that:

For interconnects that are approximately 6mm and an er of 4, this corresponds to a frequency of 2.5 GHz. Its clear that HBM data rates are running into this regime.

In transmission line theory, when a wave is sent down a line with a characteristic impedance Zo, there needs to be an equal impedance at the termination for the signal to “match” to, or else the wave will reflect backwards. The reflection coefficient quantifies this effect and its formula is given by:

When the impedance is NOT matched, the reflection coefficient is not 0. Depending on the extend of impedance mismatch, a portion of the wave that makes it way to the load is then reflected backwards and forwards along the line in a time delay T_d, decaying over time.

For an open ended or high impedance circuit, Z_L is infinite and the reflection coefficient is 1. This means that for high frequency RF signals, all of the propagated signal gets sent backwards as there is nowhere it can terminate. This is shown as the “stairstep” function in figure 8.

From the source POV, the effective impedance looking into the line depends on the length of the line between the source and termination. An impedance on a smith chart can be rotated depending on the integer multiple of wavelengths to find the equivalent impedance “looking” into the line. A special case (commonly asked in interviews) is λ / 4 lines that “transform” a open ended impedance to a short circuit looking into the line.

Though RF has a highly developed body of broadband analysis techniques such as smith charts, there are a few caveats as it applies to signal integrity:

• The data is digital data are ideally square waves with rise/fall times and odd mode harmonics, not broadband sine waves. When a signal goes high at the source, its value stays high at a fixed level until the next time period.

• The interconnect resistance is fairly high (1’s to 10’s of ohms) compared to the characteristic impedance. In most lossy RF transmission lines with enough board space, the resistance will usually not be this high. The only reason its high for HBM is for the tiny wires they need for the massive parallelism.

• In RF theory, the characteristic impedance is commonly standardized to 50 ohms and the source impedance matches this. However, matching the source impedance to the HBM interconnect impedance is not practical because changing Ron can fix one problem but create others:

  • Increasing Ron to impedance match will cut the launched voltage is half and charge the line slower, decreasing slew rate

  • Decreasing Ron would require large width transistors that may cut into edge density

  • Therefore, SK Hynix selects a Ron of 14 ohms for a characteristic impedance of 34 ohm to balance the tradeoffs of layout area and slew rate. This results in a reflection coefficient of -0.41 at the source end.

In short, for 30Gb/s data rate per DQ HBM5, SK Hynix is dealing with a tricky combination of high channel resistance and transmission line effects that can’t be easily analyzed using standard microwave techniques, and thus have to define their own set of evaluation criteria for high speed signal integrity. These evaluation criteria involve classifying the mode the operation into two distinct regions:

• RC Dominated

• LC Dominated

Share

After the paywall I will discuss this methodology along with KAISTs co-design framework in more depth. In the meantime, checkout a few of my other posts.

Side note: I saw a crazy idea at ISSCC 2026 where the data between GPU and HBM was…

Read more

• Motivation behind Optical Interconnects

• An Overview of GeSi Electro-Absorption Modulator (EAM)

  • Franz-Keldysh effect

  • Comparison with other external modulation options

• Why Component Level Test Vehicles are Needed to Reduce Overall Integration Risk of 3D Heterogenous Integrated Packages

• An Introduction to Marvell’s Photonic Link: A Fully Integrated Solution for XPU, CPO, and HBM

  • 🔒Marvell / Celestial OMIB: Overcoming Edge Density Constraints

  • 🔒Photonic Fabric Test Vehicle of EIC, PIC, and FAU w/ measurement results

  • 🔒Example 3D Packaged Application: Photonic Fabric Memory Module

• 🔒Key Co-design Integration Challenges

  • 🔒Thermal Management

    • 🔒EAM Thermal Stability

    • 🔒Package Thermal Steady State Characteristics

    • 🔒Package Thermal Transient Characteristics

  • 🔒Power Delivery Challenges

• 🔒How NVIDIA DWDM and Marvell’s EAM technologies can be powerful together

The research from this post comes from various Marvell’s papers from ECTC and their ISSCC forum presentation. I also incorporate relevant fundamental material from ISSCC short courses as well when developing the theory.

This post is aimed at more advanced understanding of some of the cross domain integration challenges that EAM is facing. Fortunately, I provide several references to several of my posts on advanced packaging, optical communications, and power architecture for all knowledge levels to understand the underlying fundamentals of Marvell’s Photonic Fabric Technology.

Before writing this, I wrote a similar deep dive on NVIDIA DWDM summarizing the challenges from both their ISSCC and ECTC papers, having attended both conferences myself. These posts come at a timely moment when NVIDIA recently announced they invested $2B into Marvell and Jensen announced at Computex that Marvell will be the next $1T company, causing the stock to soar.

I recommend you read my other NVIDIA post to put GeSi into its proper context and to read between the lines behind that stance:

Motivation behind Optical Interconnects

Scaling AI compute has many different bottlenecks: power delivery, thermal, and high speed interconnect density. Industry standard interconnects such as PCIe have limitations for the workloads AI compute demands. As a result, there is demand for high bandwidth, long reach interconnects that makes optical communications such an attractive option.

Within the optical domain, lasers themselves can be directly modulated (such as VCSELs) or light coming from lasers can be externally modulated. There are three primarily ways to externally modulate a light in the optical domain: Mach-Zehnder Interferometers (MZI), Ring, and Electro-Absorption. MZI modulates the phase of light travelling through two arms to cause destructive interference when recombined, and ring modulators act as band-stop filters for specific wavelengths of light.

I discuss these options in the following post:

GeSi Electro-Absorption modulators

EAMs operate by optical absorption of light through an electrically-controlled PIN junction at a single wavelength. A PIN Diode is a PN junction diode with an undoped intrinsic region to alter the PN diodes characteristics for the desired application, such as:

• High-frequency RF switches / attenuators (Si, GaAs, InP),

• High voltage tolerance by lowering the capacitance and thus increases breakdown voltage (GaN, SiC),

• Photodetectors by converting optical signals to electric current through generation of Electron-Hole pairs (InGaAs, Ge)

In optical communications, EAMs device have a high bandwidth and a fairly compact footprint for CPO applications.

EAMs have a property called the Franz-Keldysh effect which is where the optical absorption in a semiconductor changes when an electric field is applied. This is shown as the black curve above shifting to the red curve when a bias voltage is applied. This allows a fixed wavelength light to be externally modulated at high speeds by quickly switching the absorption curve at that wavelength to affect how much light goes through.

As shown in the equations above, the extinction ratio can be calculated as the ratio of the exponentials of the absorption factors of the two states, and the insertion loss can be calculated as the exponential of the absorption at 0V. Higher extinction ratio and lower insertion loss is desired.

GeSi is a common material used for the intrinsic region. The GeSi is epitaxially grown on top of a flat surface that is polished with CMP. Light is routed through the GeSi which has a higher refractive index than the surrounding SiO2, thus causing total internal reflection through the device.

One common FoM is the the transmitter penalty (TP) which is a measure of the loss due to the modulation itself and combines both the IL and ER:

Here we see some example characteristics of Marvell’s EAM IL over wavelength over bias voltage and different temperatures. High temperature causes the IL to shift up and affect the absorption that each signal level sees.

This means that temperature stability and thermal management is crucial to ensuring that EAM devices can operating consistently within its intended environment. Sophisticated mixed circuit control circuitry is necessary to anticipate and correct for rapid temperature swings in the GPU and package.

While ring modulators face challenges with precise thermal control and MZI are quite large, EAMs offer a promising “middle range” alterative in high performance CPO applications. Advantages include:

• Relatively small size

• Good thermal stability

• High speed

• Low power

• High optical bandwidth

However, there are a few downsides:

• EAMs struggle from power handling because they absorb light, not phase shift it. There is an almost guaranteed 3dB loss from light passing through the EAM that must be accounted for in the link budget.

  • I believe the link budget is an extremely important concept that anyone investing in or designing optics needs to know because it affects which components in the supply chain can be successfully integrated in real systems. I dive more into link budgets here in the free section:

• EAM devices are non-linear that, if uncorrected, affects the size of the individual eyes for PAM4 where the noise margin is governed by the smallest one.

  • To combat this, pre-distortion circuits are needed to pre-distort the four voltage levels so that the signal can be sent through the channel and recovered as linearly as possible.

• EAM devices are difficult to reliably manufacture at scale. Growing the GeSi epitaxially requires a fairly flat surface and sophisticated means to control the crystal growth. High volume testing is needed to screen for KGD.

In short, GeSi offers several promising benefits for CPO, but does suffer from inherent device level challenges as well as circuit complexity challenges to control the characteristics of the PAM4 signal.

Why Component Level Test Vehicles are Needed to Reduce Overall Integration Risk of 3D Heterogenous Integrated Packages

There are several excellent research works being performed across the industry and academia on components that comprise integrated 3D packaging solutions:

• Different substrates: organic, silicon, glass

• Different bridge option: LSI, embedded bridge

• Interconnect options: C2, C4, and Hybrid bonding

• Different integration options: 2.1D, 2.3D, 2.5D 3D, 3.3D, 3.5D.

• Different optical component integration and waveguide routing options in each substrate

Ideally, each component (whether that be chip-level IP or mechanical components) should have clearly defined datasheets (such as loss over frequency, CTE, etc) with validation/yield data backing it up to minimize final assembly risk. When system integrator components buy parts, they do a competitive analysis by reading multiple datasheets, selecting a part, and expecting the device to perform within those given operating conditions. Hybrid Bonding is an example where that is NOT the case, and is only used in specialized cases behind NDAs.

However, the current AI market is demanding innovative higher performance components that don’t have defined datasheets yet. Its not always possible for promising solutions to be fully vetted before they can be used, because by then the next higher performing part is right around the corner.

For tightly integrated AI Compute packages, innovative yet unproven components can cause integration risk. Parallel optimization efforts of innovative solutions are needed to make sure that block performance improves along with the whole system.

One major issue are the cross-domain interactions across interfaces. EAM have thermal stability challenges between the optical and thermal interface, and high speed signals can suffer from Power-supply induced Jitter (PSIJ) between the power and signal integrity interface. These cross domains interactions can be tricky to anticipate if each component is optimized in isolation from each other and not caught early on.

To minimize final integration risk, new IP is proven out on test chips or demonstration vehicles on smaller subsystems. These IP provide system integrator companies valuable data to assess integration risk and provide several options to choose from. Major tech companies often identify several “fallback” options of sockets within their product, whether that be consumer electronics or hyperscalers, so that they can protect their supply chain from one bad component and minimize of risk of integrating new ones.

I say this because I have many war stories of scaling superconducting computing from standard cells to VLSI integration, where tapeouts were mainly structured around integration risk reduction: before using newly invented logic gates in higher level P&R flows, they need to be taped out on test chips with data to show that they are at least functional. They might not be fully optimized to “gate” usage at high level assemblies, but optimization can be done in parallel alongside tool development.

Though it sounds exciting from the outside, its not always a fun process on the inside as it can feel chaotic. In hindsight, you do learn a lot in the process.

Marvell’s Photonic Link: A Fully Integrated Solution for XPU, CPO and HBM

Now I will comprehensively review Marvell’s/Celestial AI’s approach to CPO.

Marvell is trending toward 3D integration with the EIC stacked on top of the PIC with a silicon bridge connecting XPU and PIC. Their Photonic Fabric Approach has the following three components:

• PFLink™: a chiplet optical link that supports supporting UCIe-A interfaces with 14.4 Tbps BW and reach > 50 meters.

• PFSwitch™: a system-in-package switch enabling all-to-all XPU connectivity.

• OMIB™: an optical multichip interconnect for large package integration.

However, this approach presents several co-design challenges along several domains:

• Packaging Real Estate: One issue with CPO is the edge density (or Beachfront) of the ASIC. In AI Compute heavy GPUs, HBM, CPO, and the power delivery network want to be as close to the ASIC as possible to minimize loss in high-speed and high-power signal. This creates a real estate tradeoff of which components should remain close to the ASIC depending on what is being optimized. GPUs need HBM as close as possible for training AI models, while CPUs and other ASICs may not have as strict memory requirements.

• Power Integrity: Power needs to be delivered to the I/Os in the die interior. This requires enough power density and a low enough impedance over the frequency range of expected transients. Vertical power delivery is proposed to support this, but there needs to be enough space for TSVs and decoupling capacitors close enough to the ASIC to provide the power density at a low impedance at the transient frequency.

• Thermal Stability: Steady state and transient thermal simulations are needed to quantify the thermal proximity of the characteristics of the EAM, as well as how the EAM characteristic shifts in response.

After the paywall, I will dive deeper into Marvell’s 3D Photonic Fabric implementation and solutions to these three challenges from their ISSCC forum and ECTC papers. I’ll also cover a slide that caught my eye in how NVIDIA DWDM and Marvell EAM can potentially synergize with each other.

Share

Here are several of my other posts that give you a fundamental understanding of the key tradeoffs in each of these areas:

Marvell / Celestial OMIB: Overcoming Edge Density Constraints

Read more

This post will be an advanced deep dive of a state-of-the-art co-packaged optical system based on NVIDIA’s Dense-Wave Division Multiplexing (DWDM).

This post is organized in the following sections:

• What is Dense-Wave Division Multiplexing?

• A complete architecture breakdown of NVIDIAs DWDM with key physics constraints:

  • PIC - Ring modulator physics

    • Ring coupling theory - splitter and phasor theory

    • Key ring properties

    • How PN junctions shift the resonant frequency by changing the refractive index

    • Nonidealities - Thermal sensitivity and crosstalk

  • EIC - Clock forwarding and jitter analysis

    • Conventional clocking techniques - Forward clocking and embedded clocking

    • NVIDIA’s clocking architecture - half rate DWDM link architecture

  • 🔒 A Breakdown of the DWDM CW-DFB Laser Array and performance results

  • 🔒 Overlay of two critical spectrums that illustrate precise alignment challenges:

    • 🔒 NVIDIAs MRM transmission spectrum from the ISSCC paper

    • 🔒 NVIDIAs DFB Laser Array power spectrum from the ECTC paper

  • 🔒 A brief high-level overview of key CPO packaging features with TSMC COUPE

DWDM is fascinating because it blends together the knowledge amongst different specialties who all work together to create a successful product:

• The on-chip mixed signal SerDes designers designing the high speed TX and RX circuits

• The material scientists/solid state physicists quantifying the behavior of PN junctions and SiPh materials

• The packaging engineers who integrate everything together

The primary sources of this post are two NVIDIA papers on the circuit architecture and packaging aspects of DWDM from conferences I attended the sessions for in-person. These are:

• ISSCC 2026: “A 32Gb/s/λ 256Gb/s/Fiber Half-Rate Bandpass-Filtered Clock-Forwarding DWDM Optical Link in a 3D-Stacked 7nm EIC/65nm PIC Technology”

• ECTC 2026: “Design and Packaging of a DWDM CW-DFB Laser Array for Co-Integrated Optical Interconnects”

I also integrate relevant material from relevant ISSCC forums and short courses as well.

I find that one of the best ways to learn more advanced concepts is to work backwards and figure out what simpler prerequisite concepts you need to learn first. The free half covers a huge technical breadth of material with links to several of my fundamental posts I wrote leading up to this. This way, people of all skill levels can work backwards and figure out what they need to learn to understand more complex concepts.

I want to challenge you to try to learn one new concept in depth from this post or my body of work, even if everything else doesn’t make sense.

Please note that I do not consider myself a photonics expert. I wrote this post using the sources listed above as sources-of-truth to base my commentary off of. If you’re an expert familiar with this material and notice a mistake in this post, please reach out to me so I can have this promptly corrected.

For a broad overview of optical communications, including VCSELs, optical modes, and key impairments, I recommend reading this post where I dive into these fundamentals in the free section:

What is Dense-Wave Division Multiplexing (DWDM)?

Optical interconnects in co-packaged optics help overcome the energy dissipation and frequency scaling challenges inherent in copper interconnects. Co-packaged Optics moves the optical engine as close to the chip as possible to minimize the length the electrical signal travels before being converted to the optical domain.

Optical communications enable low-energy and low-latency data transmission. However, optical communications is limited by a few constraints:

• The physical edge density. The optical signals need to be propagated through waveguides into a fiber optic connector. How and where the signal flows from the laser source to the electrical devices matters a lot.

• Frequency constraints of individual devices. The electrical as well as the optical bandwidths needs to be fast enough to handle the data throughputs needed.

Several companies propose solutions that push all available scaling dimensions. One of the most promising solutions to scale data speed involves densely multiplexing light signals of different wavelengths on the same waveguide.

Dense-Wave Division Multiplexing consists of individual optical channels of equally spaced wavelengths. Electrical signals are frequency multiplexed in a waveguide through a series array of micro-ring modulators. Ring modulators act like on-off switches for each of their respective channels. Ring modulators are ultra-compact, enabling many channels to be packed into a common waveguide.

Pros and Cons of DWDM

Compared to other modulation options, DWDM has several benefits:

• Scalability. DWDM performance offers scaling potential with the # and size of devices.

• Low latency for XSR due to no DSP

• High energy efficiency

• High bandwidth density

However, there are several challenges with DWDM that involve the material physics aspect of these devices:

• Temperature sensitivity. Lasers and MRMs are inherently sensitive to temperature and must be well controlled

• Design Complexity - DWDM moves a lot of the complexity from the electrical domain to the optical domain, and imposes unique clocking and jitter tracking challenges at high speed

• Laser Integration - DWDM needs a multi-wavelength laser source that has to be brought onto the chip

  Silicon Co-Design is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Architecture Breakdown - EIC, PIC, Laser Array, and Packaging

Next I’ll discuss NVIDIAs DWDM multiplexing architecture. I’m going to describe the physics of the MRM on the PIC first to show how it works at a fundamental level. I’ll then discuss the clocking scheme of the EIC and show how the PIC requirements influence the EIC requirements.

At an architecture level, the job of an optical transceiver is to make sure that the data sent through an optical laser is received by a photodetector on the other end with acceptable BER. When selecting components, link budgeting is performed to take into account all sources of losses and optimize the performance of the entire system. I discuss link budgeting in the free section of this post:

PIC - Ring Resonator Physics

In a nutshell, microring modulators (MRM) accept or reject specific wavelengths of light going through a nearby waveguide. MRMs couple evanescently to nearby optical waveguides and act like band-stop/pass optical filters, depending on the way you look at it. Light is normally guided through an optical waveguide and passes right through, but the MRM acts like a low impedance for light at specific wavelengths.

MRMs consist of the following layers:

• A circular optical waveguide (grey) that is “sandwiched” between two regions:

  • p-doped region (blue)

  • n-doped region (red)

• A heater (green) that surrounds the MRM to regulate the temperature

A voltage source is attached to the n-doped region (yellow) to switch the optical signal going through the waveguide by shifting the resonant frequency over.

Lets dive into each of these elements in more detail.

Ring Coupling - the Splitter

The ring can be thought of as an splitter with two input and two output terminals. . Think of the ring like a “race track” that the coupled signal enters, runs laps around, and returns back onto itself until it ultimately decays due to losses.

In this splitter network, there are two constants that define the input-output characteristics:

• t is the transmission coefficient and represents how much of the signal gets through. Ideally, t should alternate between 0 and 1 at specific wavelengths.

• k is the coupling coefficient and is affected by how closely spaced the ring is to the optical line.

By conservation of energy, t^2 + k^2 = 1.

Once the optical signal enters the loop, it experiences loss and phase shift represented by a(λ). If the ring is unwound, the path of the light is represented by the phasor:

where the “a” term factors in the loss factor α and the propagation constant β that represents the wavelength-dependent phase shift:

The transmission formula is given by the ratio of the light that makes it through the optical waveguide vs the light coupled into the ring across wavelengths, which is given by:

Through substitution, as well as simplifying the phasor into a cosine, we arrive the following:

This formula lends itself to the following properties:

Rings Resonance Frequencies

In a microring modulator, the ring resonates when the diameter is equal to an integer number of wavelengths. These wavelengths is given by

where

• n_eff is the effective refractive index

  m = integer modes (1,2,3…)

  r = ring radius

Due to the periodic nature of the ring, the resonances are contained within a free-spectral range that limits the wavelength window signals can couple into it. Any signal outside of this window will alias into it.

One important property of the ring is the Full Width at Half Maximum (FWHM). This is the “width” of the resonance measured at half the max power (or 3dB on a log scale) at both edges of the resonances. FWHM is a useful qualitative reference to measure how “wide” the resonance is. This is given by:

The quality (Q) factor describes how narrow the resonance width is compared to the frequency it is centered at. This is given by:

where

• c is the speed of light in vacuum

  τ is the cavity photon lifetime that lumps in the above parameters

In NVIDIA’s implementation, the TX ring modulator quality factor Q is ~4.5K, and RX filter Q is ~4K.

In order to switch electrical signals into this ring, the ring bandwidth needs to be wide enough. The ring bandwidth is given by

where

• R is the ring junction resistance

  C is the ring junction capacitance

NVIDIA switches into each ring at 32Gbps, corresponding to a switching frequency of 16GHz.

PN Junction shifts the frequency by changing the refractive index

PN junctions normally act as electronic diodes that, electrically speaking, act as one way current flow with a small built-in voltage drop (~0.7V).

In conventional PN junctions, the p side is doped with type III elements (such as boron, gallium, or indium) with free holes in the valence band, and the n side is doped with type V (such as phosphorus or arsenic) atoms with free electrons in the conduction band. When a p and n material are sandwiched together, it creates a depletion region of “built-in” electrical potential due to the flow of carriers between the materials.

However, when PN junctions are used in SiPh, we care about the solid state physics properties that affect the index of refraction. In optical, these junctions are biased in reverse bias. As more voltage is applied to the n-doped region, the depletion region widens. This lowers the carrier concentration, causing the refractive index of the silicon to spike upward. This shifts the resonant frequency and moves it just enough to allow a light source to pass through. Think of this operation like a door opening and closing really fast.

In fact, these PN structures can be altered and routed in series with the optical signal itself. Distributed PN structures are used in Mach-Zehnder modulators that modulate travelling waves along long lengths and unique structures such as p-i-n are used in electro-optic modulators. Each structural variation has their own unique characteristics and tradeoffs. I discuss these options behind the paywall in this post:

Nonidealities

I’ll discuss two nonidealities of MRMs: Thermal sensitivity and crosstalk.

Thermal Sensitivity and Tuning Loop

Ring modulators are sensitive to temperature primarily because of the thermo-optic effect. The thermo-optic coefficient of silicon is ~1.8 x 10⁻⁴ K⁻¹ and changes in temperature affects the index of refraction.

Here, the thermal tuning loop measures the current at both TX and RX through:

• A drop-port photodetector (PD) at the TX to sense how much optical power is leaking out of the ring

• The TIA DC-loop at the RX

The PD current value is first digitized with an 8 bit SAR ADC, then processed digitally to produce a pulse density modulated signal to drive an integrated metal heater.

The paper doesn’t give too much detail beyond this, but I would imagine the ADC/DAC precision and loop bandwidth needs to be acceptable to correct for the expected temperature fluctuations with an acceptable wavelength error. I also imagine each individual ring can be tuned for the specific resonance offset.

I dive into depth regarding the SAR ADC in this post:

Crosstalk

When multiple microrings are in close proximity to each other on the same substrate, cross talk is a key first-order phenomenon. The frequency characteristics of microrings tend to have long tails that poorly suppress sidebands and leak into other rings as well.

When deciding on what the Q factor should be, there is a tradeoff between crosstalk and the bandwidth:

• A higher Q lowers crosstalk and bandwidth

• A lower Q increases crosstalk and bandwidth

Modelling crosstalk is not that intuitive since there are many crosstalk paths among different rings that occur at different wavelengths.

Summary

Ring modulators appear simple at first but are highly affected by material properties and their tightly coupled nature. They are subject to device-to-device variation, so appropriate tuning circuits and control loops are needed to account for this, adding additional design complexity.

EIC - NVIDIA’s half rate DWDM link architecture

Now I will talk about the electrical side, more specifically the clocking architecture.

Of all the topics I’ve written about, I found that clocking is perhaps one of the trickiest subjects to explain in depth. As a simple analogy, think of clock synchronization like an orchestra conductor making sure all the instruments in the orchestra are in sync with minimal variation in when notes are played. There are many outside sources that can affect each individual musicians rhythm.

Each ring modulator is driven through their own SerDes electrical circuit. Compared to standalone high speed SerDes, each ring’s SerDes is slower and doesn’t have the DSP due to the XSR nature, easing some complexity. However, parallelizing 8 data lines requires synchronization and jitter control that adds additional complexity to the clock network.

Lets break down NVIDIA’s clocking architecture in more detail and see if we can make sense of this.

Conventional Clocking Techniques

When digital data is transmitted through a channel, there needs to be some way to clock the data at the RX to recover the data. There are two conventional ways of doing this:

• In a forward clocking scheme, the clock is sent alongside the data in a separate channel.

• In an embedded clocking scheme, the clock is “integrated” into the data stream. The data stream is serialized in a way that ensures enough voltage transitions (edges) occur to represent the timing information. This clock is then recovered at the receiver side through a clock data recovery (CDR) circuit.

I discuss the conventional circuits used in CDR circuits (such as bang-bang phase detectors) after the paywall in the following post:

No matter which scheme is used, each channel will couple in noise that appears as clock jitter, which is a deviation of the clock edge from its ideal location. There are two types of jitter:

• Correlated jitter - this is jitter that is generated from device operation such as TXPLL jitter and supply noise induced jitter. This jitter is identical lane to lane.

• Uncorrelated jitter - this is jitter that is random in nature, such as thermal, flicker, shot, etc, and is mostly generated through devices. This jitter is different from lane to lane.

Jitter show up as “tails” in the frequency content of a high speed signal. If jitter is high enough, it can degrade BER from the value being outside the threshold used to read 0 or 1. I discuss more about jitter in this post:

When clocking parallel channels, it is important to track the jitter and filter out unnecessary sources of it. In general, it is easier to track correlated jitter than uncorrelated jitter, and filter out low frequency sources of jitter.

There are a few pros and cons of each scheme:

• Forward clocking has excellent jitter tracking bandwidth, but requires a separate clock line. Additionally, uncorrelated jitter can accumulate.

• Embedded Clocking saves on IO, but CDR circuits have the potential to be exposed to jitter beyond its limited bandwidth.

DWDM has the unique benefit of embedding the clock in as a separate wavelength along the data lines, but is does come with a a few unique challenges:

• The data at the TX needs to enter the rings synchronously

• When data is sent down through the optical fiber, there will be delay mismatches due to various non-idealities such as chromatic dispersion and polarization mode dispersion.

• The thermal noise-induced jitter is dominated by the transimpedance amplifier (TIA) at the RX

Lets discuss what NVIDIA does to overcome these challenges.

NVIDIAs clocking architecture - half rate DWDM link architecture

NVIDIA uses a half-rate FC DWDM link architecture that embeds the clock inline with the data. In this architecture, there are 9 rings on each of the TX and RX that combines for a total data throughput of 256Gbps:

• 8 rings are for data at 32Gbps each

• 1 ring is for FWDCLK at 16GHz

On both the TX and RX side, the signal will traverse different lengths depending on which ring it coupled into. As a result, the TX clock needs to be de-skewed to account for known path length misalignment.

At the RX end, the transimpedance amplifier (TIA) adds a lot of thermal-noise induced uncorrelated random jitter. This TIA noise primarily comes from the feedback resistor (circled above) which is necessary to provide a low impedance input from a current sourced photodetector into the TIA.

To overcome these jitter challenges, NVIDIA uses an Injection locked oscillator (ILO) - based bandpass filter to bandpass filter the clock with BW ≈ 1-2GHz. This filters out jitter beyond 1GHz to remove most of the uncorrelated jitter and tracks most of the correlated jitter.

This clock is then distributed to the RX as illustrated by one example setting of the RX CLK distribution. Each RX has a local ILO that acts as another jitter filter. The clock distribution is segmented into two distinct spinal wires CLKA and CLKB that cut the capacitive load per wire in half. The RXPLL acts as a on-chip frequency synthesizer that provides the local timing reference upon startup until the system is locked to the main timing source.

I write about phase-locked loops in more depth:

DWDM Laser Source and Packaging of EIC and PIC

After the paywall, I will cover the packaging part that represents a major constraint in the physical realization of this system. I will cover the following:

• A breakdown of the DWDM CW-DFB Laser Array and performance results

• A brief high-level overview of key CPO packaging features with TSMC COUPE

In between these two discussions, I will overlay NVIDIAs measurements of the ring spectrum from the ISSCC paper with the DFB Laser Array from the ECTC paper, and discuss the challenges with aligning both of these. Alignment between these two spectrums is crucial across multiple devices because misaligned channels don’t transmit data.

Here are several other posts I’ve written that cover the fundamentals of advanced packaging in-depth:

Share

Read more

Silicon photonics is one of the hottest and fastest moving markets to address high speed data demands in co-packaged optics (CPO) in AI data centers.

In this post I’ll cover a few important high-level details of silicon photonic circuits. These include:

• An Overview of the Link Budget and important considerations:

  • Laser Power

  • Connector Losses

  • Receiver Figures of Merit

• 🔒Key Figures of Merit for eye quality

• 🔒Validation Challenges (wafer screening and thermal stability)

  Inside the Silicon Machine is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Link Budget

Understanding link budgeting is important to develop a system level understanding of a SiPh transceiver. Consider a typical electro-optical link with an external modulator:

At a high level, the job of a transmitter is to transmit the data at the specified data rate (224GPBS) with as low BER as possible in the given specification. Key system level performance specs include:

• Power efficiency: The ratio of energy consumed to the amount of data transmitted. This is usually measured in pJ/bit.

• BER: Bit error rate. Noise, loss and other effects degrade bit error rate. Requirements range between 1e-4 and 1e-6 depending on the standard.

A laser source (whether that be generated on board or coupled in externally) provides the power and is sent through a lossy modulator, connectors, and waveguides.

This power is then sent through the optical channel, converted back to a weak electrical signal through a photodetector, and amplified by a transimpedance amplifier. The signal at the RX need to be high enough compared to the noise in order to be able to detect the signal properly.

All of the sources of loss in between the TX and RX must be quantified for in a link budget in the optical chain. Lets break down each part.

Lasers

Lasers are either integrated within the PIC or coupled in externally. These external lasers are coupled into the chip through edge couplers or grating couplers.

There are a few important specifications:

• Power: The amount of coupled power is limited by two-photon absorption and other non ideal effects. Shown is a sweep of Pin vs Pout where Pout saturates and a high enough Pin can cause passivation melting:

  

• Relative intensity noise (RIN) - Rin refers to the relative intensity fluctuations of the optical signal of a laser source. This is a critically important parameter in PAM4 schemes because PAM4 has much lower noise margin than NRZ. A high RIN raises the noise floor causing a higher BER. RIN is given by:

Companies with lasers in the value chain include Lumentum, Broadcom, Coherent, Sivers, Scintil, Xscape, and Quintessent.

Losses in Connectors

There are several loss sources in the connectors themselves. These include:

• Modulator loss - due to intrinsic optical absorption and scattering

• Multipath Interference (MPI) - Optical reflections from multiple connectors / discontinuities

• Polarization Dependent Loss (PDL) - Measures loss due to change in signal polarization

Receiver

On the receive side, the signal is detected by a photodetector. This weak signal plus a noise signal is converted to a voltage through a transimpedance amplifier (TIA) that converts current to voltage.

A few key specs here are important:

• Responsivity - Measures how effective the photodetector converts photons to electrons. This is typically measured in A/W.

• Dark current - Measures the current flow when no light is present. Defects can cause high dark current that adds noise to RX and reduces sensitivity.

• Input-referred noise - assesses the noise floor of the TIA.

• Transimpedance Gain - measures the electrical strength of the receiver. TIA gain is very important because if TIA gain is too low, output voltage swing will not be high enough to compensate for noise. It is given by:

• Receiver sensitivity - the minimum optical power required to achieve a specific BER target. This is determined by plotting RX BER vs OMA. BER goes down when received optical power goes up.

• THD (Total Harmonic Distortion) - quantifies the non-linearity of the TIA. This affects PAM4 because it affects how even spaced out each of the four levels are after optical to electrical conversion.

Eye Quality

After the paywall I’ll go into key parameters that define the eye quality, practical challenges when scaling production of SiPh, and NVIDIA’s approach to thermal tuning design.

Share

If you’re curious to learn more, check out my other posts here:

Read more

• I will introduce the conventional IM-DD optical approach with VCSELs used for short reach

• I will then highlight challenges with scaling conventional IM-DD to higher data rates

• Then I’ll explain techniques to improve data bandwidth:

  • 🔒 Co-packaged optics

  • 🔒 Silicon photonics

  • 🔒 External modulation

  • 🔒 Coherent optics

High speed optical interconnects have become one of the hottest markets in AI data centers. As bandwidth demand rises, the challenge is no longer just computing on data faster, but moving it across packages, boards, racks, and clusters efficiently.

The optical space has exploded over the past few years with many companies and startups proposing solutions to tackle data demands.

But I want to step back and ask ourselves: what are the first principles challenges with scaling high-speed optical communications?

The workhorse for short reach - IM-DD & Pluggable Optical transceiver

The workhorse for short reach optical communications today are IM-DD with pluggable optical transceivers. Lets take a looks at what each consists of.

Intensity Modulation - Direct Detection

IM-DD is the workhorse of short reach optics. IM-DD encodes information in optical intensity (whether that be 0/1 or higher levels in PAM4) and recovers it through direct detection.

This intensity modulation can be performed in one of two ways:

• Direct modulation - the laser is turned up/down by the electrical signal

• External modulation - a driver modulates a steady optical carrier

Pluggable Optical Transceivers

These IM-DD are typically packaged into pluggable optical transceiver modules that sit far away from the chip. These are are match-box sized modules that take in electrical (client) side data and convert it to optical (line side). Each block consists of following:

• Serdes + DSP - For serializing / desterilizing the data from/to host and perform equalization

• EIC - Electrical integrated circuit - consists of the electrical circuits fabricated in electrical processes (drivers, mixed signal conditioning, etc)

• PIC - Photonic integrated Circuit - consists of the optical circuits fabricated in optical processes (Si Photonics, VCSELs, etc)

Optical transceiver modules are a very mature technology that has scaled in data throughput for many years. However, with AI workloads demanding 224+ GPBS throughput capability, the performance of the optical transceiver itself needs to adapt, as well as the signal integrity between host die and optical engine.

System architecture breakdown of a IM-DD

Next I will break down the architecture of an IM-DD in each of its components:

• Transmitter

• Fiber

• Receiver

I will discuss several options for each and important design considerations.

TX Driver - Direct Modulation

Laser (Light amplification by stimulated emission of radiation)

Lasers are powered by a large pump current stimulating emission. Photons with energy close to the material bandgap interact with electron - hole pairs and cause identical photons to be emitted. This causes an “optical gain” of photons within the active region.

This photons is then confined using two mirrors: one opaque, and other that is semitransparent to allow the light to shine through.

Lasing occurs when the pump current is high enough such that optical gain > optical losses.

Then, either a cavity structure or frequency selective bragg reflectors selects which optical modes are sustained.

In a Distributed Feedback laser (DFB) laser, a periodic grating of Bragg reflectors provides wavelength - selective optical feedback, favoring a narrow lasing wavelength. These Bragg Reflectors are highly tunable depending on the index of refractions and the lengths and # of sections.

VCSEL (Vertical-Cavity Surface-Emitting Laser)

VCSELS are lasers that emit light vertically in a circular cross section. The active region consists of multiple quantum wells that enhances optical gain. Distributed bragg reflectors are stacked on the top and bottom of the active region with alternating refractive indexes. These layers are fabricated using epitaxial growth on a substrate.

The light is then confined with an oxide and current is pumped in through pads. The light is shown vertically above and redirected toward the optical fiber with optics.

The Pout/I of VCSELs almost resembles a linear diode. The VCSEL begins lasing above a particular threshold current and exhibit a linear relationship between Pout / Iin over a useful operating region. Lasers require a DC bias to set the device in the linear region.

One important parameter is the extinction ratio, which is the ratio between P1 and P0. A higher extinction ratio is preferred as it improves BER margin to detect teh signal at the other end, but higher extinction ratios require higher currents, and therefore power, to drive the device.

The two major materials used in lasers are GaAs or InP:

• 850 nm GaAs VCSELs are common for short-reach multimode links.

• 1310/1550 nm InP-based lasers/modulators are common where single-mode fiber, longer reach, or telecom/datacom bands matter.

VCSELs are very popular in short reach communications for a numbers of reasons:

• Low cost manufacturability - they are easy to test on-wafer across multiple samples

• Scalability - they can be formed in “arrays” to drive multiple optical lanes

However, VCSELs do have some drawbacks:

• Reliability - issues due to oxide confinement and high currents degradating mean-time-to-failure

• Signal switching nonidealities, including:

  • Turn on delay- a delay in the turn on of the signal that adds to latency

  • Jitter - uncertainty in the signal edge that degrades BER

  • Relaxation oscillation - an “underdamped” response in the turn-on characteristic that can affect BER and signal quality

The main tradeoff in VCSELs is between BW and reliability. Higher current densities allow for higher BWs, but it also decreases the mean time to failure, a common metric used when evaluating reliability.

Driving the Laser

Here are a few DC coupled, common anode circuits used to to drive the laser directly using either fully differential or pseudodifferential to convert a voltage to a current to drive the VCSEL. There are other methods of driving such as common-cathode, BJTs, and push-pull implementations depending on the requirements of the system.

In general, the more VCSEL current that is needed, the slower the driver will be due to increasing output capacitance.

Fibers

Multimode fiber optic cables are primarily constructed from silica glass (silicon dioxide). There are two major categories of fiber for high speed communications:

• Multimode - Uses a larger core (50–62.5 um) to transmit multiple light paths

  • There are five classifications, OM1-5, that grade the quality of the fiber.

  • Pros: Cost-effective for short-distance, high-speed applications

  • Cons: Limited distance due to higher signal modal dispersion

• Single Mode - Uses a smaller core (8-10um) to transmit a single mode of light

  • Pros: Low signal loss, avoids modal dispersion

  • Cons: Higher cost of laser components and more complex installation requirements

Generally speaking, multi-mode fibers are used for short reach interconnect due to its low cost and complexity. However, single mode fiber is being evaluated to “future proof” the data communications since it can support the higher BWs, though it adds to cost and integration complexity.

There are also four wavelength bands used in the fiber:

• 850nm band - used for short reach links

• O-Band - 1310nm

• C-band - 1550nm

• L-Band - Extends C band to allow 2x network capacity

There are four major impairments:

• Loss. Losses cause pulse attenuation after propagation from a fiber. These are caused by intrinsic sources, including

  • Rayleigh Scattering

  • Absorption

  • Manufacturing Stress

• Chromatic Dispersion - pulse spreading after propagation over the fiber. This is caused by fiber refractive index dependence on wavelength

• Modal dispersion - distortion due to different propagation delays of different optical modes. Modal dispersion matters in multimode fiber.

  

• Polarization Mode dispersion (PMD) - dispersion cause by pulse spreading of different polarizations due to asymmetries in the fiber core.

Receiver - Photodetector

A photodetector consists of a Reverse biased PIN junction that generates current proportional to the detected light. The transimpedance amplifier then amplifies this current to an output voltage at the other end.

There are some important specs of this system:

• PIN characteristics: Responsivity, capacitance, noise, dark current

• Receiver Q-factor: The received signal level relative to total receiver noise

• BER: BER can be derived from this Q factor: BER = 1/2*erfc(Q/sqrt(2))

A higher extinction ratio is desired to ease the burden on the receiver, but it does come with a higher power usage.

Why Scaling IM-DD is hard

Scaling IM-DD to higher data throughputs is hard due to several unfavorable tradeoffs between BW, reach, noise, power, SNR, and cost. These include effects in both optical and electrical domain themselves:

Optical

• Chromatic dispersion

• Polarization Mode dispersion (PMD)

• Nonlinear and interference effects

Electrical

• Noise penalties at Higher BW

• Tighter jitter, skew, and ISI

• BW scaling limits in front ends

You can have the fastest optical transceiver, fiber, but its pointless if the signal can’t get to it fast enough. Not only the optical receiver itself needs to improve, but the signal integrity from the electrical interconnect is running into challenges shuttling high speed data back and forth. Solutions need to ensure that speed “bottlenecks” within the system scale with the system demands.

This leads to four major trends that generally increase complexity:

• Co-packaged optics - embedding optical components within the package itself

• Silicon Photonics - routing light through silicon waveguides and modulating it

• External modulation - modulating a continuously running laser for higher BW

• Coherent Optics - modulation amplitude and phase in the optical domain

After the paywall I’ll discuss these in further detail.

Share

If you want to learn more about how optical communications interfaces with adjacent domains, consider checking out these other posts:

Read more

In this post I cover the following topics:

• SerDes - The Workhorse of chip to chip communication

• The Root Causes of Signal-Integrity Degradation at High Speed

• How Channel loss and Jitter degrade Bit Error Rate

• Major System Trends:

  • PAM-4

  • DSP-based Architectures

• 🔒Error Reduction Techniques

  • 🔒Forward Error Correction

  • 🔒Equalization

    • 🔒TX - Pre-emphasis / FFE

    • 🔒RX - CTLE and DFE

• 🔒How to Achieve Higher Data Rate

Everyone talks about the importance of chip-to-chip communication, but few people appreciate the complexity hiding underneath it.

Signal integrity and SerDes architecture are often discussed as if they belong to different worlds: one in the lab and on the board, the other inside the chip. In reality, they are deeply coupled.

My goal in this post is to show how signal-integrity constraints at the channel and board level—loss, jitter—affect performance through BER and ultimately drive complexity back into the transceiver itself, shaping choices in equalization, clocking, and data conversion. Here I’ll focus on the signal-integrity foundations from first principles, then build on that in a follow-up post on chip architecture.

Much of the content of this post is material consolidated from a Tutorial given by Dr. Mike Peng Li from DesignCon 2026, the premier high-speed communications conference, as well as from ISSCC, academic papers and SI books. Figures are reproduced for commentary and educational purposes.

SerDes - The Workhorse of chip to chip communication

A serializer-deserializer (SerDes) converts parallel digital data into a high-speed serial stream for transmission, then reconstructs it at the receiver. This allows systems to move large amounts of data between chips and systems without requiring a huge number of parallel wires.

SerDes and optical links are the “workhorses” of communication within data centers in “scale up” and “scale out” networks:

• Chip to chip

• Modular top chip copper links on a PCN and an optical module

• Between a CPU and network interface

• Backplane and midplane links over backplane PCBs or passive copper cables

SerDes is what enables parallel computing across many CPUs and GPUs as a means of data distribution.

Historically, many SerDes architectures have been analog-based. SerDes speeds have scaled with PAM-2 (NRZ) signals over time to meet the growing needs of traditional computing.

Then, AI and chiplets came in and started demanding more data.

These two technological trends put a lot of pressure on chip-to-chip communication to get faster and reach farther due to the massive amounts of data movement:

• AI workloads involve massive data movement: repeated memory access, movement of intermediate results, and parallel processing across many GPUs in a rack.

• Chiplets disaggregate a system into multiple smaller dies, which must then be tied back together through high-speed chip-to-chip interfaces.

However, as data demands push speeds higher and links longer, fundamental physics begins constraining performance:

• Channel loss due to higher attenuation at higher frequencies

• Jitter budgets getting tighter

• Transmission line effects that cause distortions

• Noise that can cause errors

These add complexity to the SerDes to correct for. Unfortunately, the added complexity makes it more challenging to understand how one part of the system affects the other, such as:

• How jitter in the VCO eats into the timing margin needed to meet the target BER

• How receiver clock skew and TI-ADC area constraints affect sampling margin, data throughput, and calibration burden

In the following sections I’ll be focusing on signal integrity fundamentals from first principles. The goal is to ask:

• What are the fundamental physics constraints limiting performance?

• What complexity in the chip architecture is needed to scale speeds beyond 112.5Gbps to 224Gbps and eventually 448Gbps?

The Root Causes of Signal-Integrity Degradation at High Speed

In this section, I’ll focus on describing two primary causes of signal-integrity degradation at high speeds: channel loss, and jitter. I’ll then show how they affect BER.

Channel loss in Copper interconnects

Copper is widely used as the interconnect of choice to transmit signals between chips due to its high electrical conductivity and durability.

Copper has been used for many years at low data rates where the losses were much less. However, the loss increases at higher signal frequencies. There are two major loss mechanisms:

Dielectric loss. Dielectric loss occurs due to heat energy dissipation of dipoles at high frequencies. A quickly alternating electric field in the dielectric causes polarization in dipoles in the dielectric (i.e. the positive and negative parts of the atom move apart) and results in energy dissipation through heat.

Dielectric loss scales with frequency and is generally the dominant loss mechanism at the data rates that modern SerDes nowadays run at.

Skin effect. Skin effect occurs when time-varying electromagnetic fields push current toward the edges of the conductor. This results in an effective lowering of the cross-sectional area that the signal actually “sees”.

Skin-effect loss scales roughly with the square root of frequency and has historically been a major source of channel loss until dielectric loss becomes more dominant at higher frequencies.

In both cases, signal attenuation increases with frequency. This means that signals with multiple frequency components like square waves will have higher order frequencies attenuated more strongly. This will result in a “rounding” of the signal at the other end with degraded amplitude.

Modern 112G PAM4 systems typically target 25 dB to 35 dB of channel loss.

Jitter

Jitter is defined as the time variation in when a signal edge actually transitions compared to its ideal transition point. Jitter is a significant limiter of performance in high-speed applications because it affects how much of a time ”window” you have to accurately sample the digital data.

Jitter is broadly broken down into two major categories: Random and Deterministic:

Random Jitter comes from noise-like mechanisms that are fundamentally statistical in nature, such as thermal noise.

This is always uncorrelated and typically modelled as a Gaussian distribution due to the central limit theorem that states that summing up a variety of PDFs from uncorrelated sources tend toward a Gaussian distribution.

Deterministic Jitter is jitter from repeatable structure in the system such as data movement and reflections.

This jitter is typically bounded with a max and min timing shift - the edge shifts in a repeatable way. It can also be correlated or uncorrelated. Examples include:

• Data dependent jitter (ISI) - Symbols from previous transitions spill into the next time window

• Periodic Jitter

• Bounded uncorrelated jitter

As link complexity has grown, many other jitter sources must be accounted for during design and measured in silicon.

How Channel loss and Jitter degrade Bit Error Rate

Both losses distort the signal in complementary ways:

• Channel loss distorts the waveform itself by attenuating and smearing its higher-frequency content

• Jitter introduces uncertainty in when transitions occur.

Together, these (and other effects such as reflections) reduce eye opening and increase the Bit error rate (BER) at your sampling edge.

Bit error rate is not easy to measure directly because of the amount of time it takes to observe enough samples (sometimes trillions) to get a statically significant measurement of BER. It is usually easy to measure a BER of 10^-6 which is 1 error in 1 million bits with a BERT (Bit error rate tester), but some systems require BERs of 10^-12 which is one error in one-trillion bits!

Often, lower BERs are often extrapolated from higher BER curves. The Dual-Dirac model is a widely accepted “worst case model” used to extrapolate this. To show how BER is driven from jitter, consider the two major classes of jitter:

• RJ spreads the signal edge according to a Gaussian distribution

• DJ shifts the signal edge with two “bounds”. In this model, the worst cases are represented as two worst case impulse ”spikes” at either bound.

  (Note that this is not the actual PDF of the DJ as it varies in uniformity within the bounds. However, this works fine because we only care about the “tails” of the RJ curve convolved at the bounds)

These two sources combine in a “convolution” format, so the gaussians of the RJ map onto the bounds of the DJ to obtain the Total Jitter (TJ). The BER is then estimated from the tails of the total-jitter distribution: as the distribution spreads further into the sampling margin, the probability of error increases.

You can then use this to map contours of constant BER in the center of the eye that decrease the more the signal is sampled at the center of the eye:

Major System Trends: PAM-4 and DSP-based Architectures

PAM-4: 4 level pulse amplitude modulation

PAM-4 uses four levels of modulation instead of two. This allows you to transmit two bits per symbol while halving the required symbol rate for a given bit rate.

However, added signal levels make PAM-4 more sensitive to noise, nonlinearity, ISI, and timing jitter. It also requires DACs to generate the four analog levels. There are also more transition combinations to account for.

DSP-Based Transceivers

When data speeds were slower, most of the processing came from analog circuits that were sufficient to perform the signal conditioning needed to achieve good BERs. Figure 9 shows a typical short-reach, analog-based transceiver with common building blocks.

As speed and throughput increased, most architectures added a lot of DSP along with the analog blocks. Digital allows a lot more programmability to handle data processing with FIR/IIR filters. However, digital also adds a large amount of peripheral complexity to handle the speed it needs, including:

• The ADC/DACs needed to convert between the analog and digital domains

• The clocking architecture to clock the circuits

Error Reduction Techniques

After the paywall, I’ll cover key error reduction techniques, including equalization, FEC, CTLE, and DFE.

If you want to continue expanding your knowledge related to signal integrity, including why modern SerDes is so complex, check out my other posts:

Read more