Algorithms and Architectures for Training-Based Channel Characterization - Cooke, Mitchell (My University, 2025)
Datacenter connectivity and performance rely on high-speed data movement over baseband wireline channels. In these channels, adaptive equalizers compensate for channel-induced distortion and are critical to achieving high bandwidths. While signal fidelity may be improved by designing novel equalizers, maximizing the performance of existing equalizer designs by designing optimal calibration algorithms has distinct challenges that necessitate embedded hardware architectures. Systems for adaptation must learn how the channel distorts the transmitted signal with few hardware resources, high accuracy, and low latency. Stochastic gradient descent algorithms are widely used for equalizer adaptation. This class of algorithms learns from random channel error information to compensate for channel filtering, which is often a function of the channel impulse response. However, these conventional algorithms can struggle to adapt non-linear equalizers, which are often employed in wireline receivers, and stochastic behavior cannot make latency guarantees, which can be valuable to ensure compliance in some transceiver standards. Instead of stochastic adaptation, if the pulse response of a communication channel can be identified, an effective equalizer can be configured to directly compensate for known channel distortion. Referred to as training-based channel characterization, this can provide timing guarantees by measuring the channel output from training sequences to form linear systems that solve for the channel response. However, in practice, this requires real-time matrix inversion, which is often considered too numerically complex and thereby unable to be performed with both low latency and few hardware resources in wireline transceiver designs. To address the equalization requirements of state-of-the-art wireline Pulse Amplitude Modulation (PAM) transceivers, we co-design novel algorithms and low resource digital architectures for low-latency training-based channel characterization in the presence of additive Gaussian and pattern-dependent noise. We provide a series of contributions that demonstrate channel characterization is feasible in-system with high accuracy by defining novel classes of training sequences and designing efficient algorithms and architectures that obtain inverse matrices. We first approach the challenge of Decision Feedback Equalizer (DFE) calibration in DDR5 memory interfaces. Although improved interface bandwidths in DDR5 interfaces can address the memory bottleneck for processors, the inclusion of a DFE to enhance signal fidelity presents a distinct challenge to real-time in-system calibration with few hardware resources whilst operating within the constraints of the DDR5 specification. We design a novel adaptive algorithm to calibrate DDR5 DFEs, then we design a novel channel characterization system that directly illustrates latency benefits and implementation feasibility. After demonstrating that channel characterization can satisfy the need for deterministic, low-latency, low-resource, and high-accuracy DFE calibration in DDR5 interfaces, we investigate both how to scale beyond polar Non Return to Zero (NRZ) modulation and to longer symbol lengths. We first investigate longer channel response lengths by designing sets of polar NRZ training sequences and a hardware architecture to leverage the properties of these training sequences for channel characterization whilst further reducing latency and hardware resources. Considering that many emerging standards are adopting higher-order PAM, we then investigate how training sequences, algorithms, and architectures can be designed in a systematic manner to generalize characterization to higher-order PAM channels. This is facilitated by several key innovations, which first include an algorithm and architecture that uses the Number Theoretic Transform (NTT) alongside finite field arithmetic to invert the circulant and skew-circulant matrices formed by training sequences in-system in real-time without a loss in precision. Alongside this, we analyze unique relationships between training sequences in higher-order PAM signaling to further reduce latency when multiple channel estimates are used to average the channel response. The algorithm that runs on the low-resource architecture also achieves high-accuracy channel characterization in the presence of further additive Gaussian and pattern-dependent noise. To summarize, in this thesis, we have revisited channel characterization in-system in real-time. We have demonstrated that by designing new training sequences alongside algorithms and low-resource architectures, channel characterization may be achieved in form-factor-constrained environments.
Keywords: Channel Estimation; Channel Characterization; Equalization; Wireline; Digital Design; Signal Processing; Channel Identification; Ethernet; DDR5; PCIe