A Custom RTL Domain-Specific GEMM Accelerator on Cyclone V SoC: Dynamic Precision Selection and Tiling, from RTL Design to Linux Bring-Up for Edge AI Inference and Predictive Control

Abstract

General-purpose GPUs deliver high peak throughput for large-scale matrix workloads. Their deeply pipelined microarchitecture, however, incurs disproportionate overhead when executing the small, latency-sensitive GEMM operations typical of edge AI inference and Model Predictive Control (MPC). In this thesis, I quantify that overhead by combining cycle-accurate GPU simulation (GPGPU-Sim 4.2.0, SM75 RTX 2060) with physical benchmarking on an NVIDIA RTX 4090 (SM89 Ada Lovelace). I then eliminate the overhead by designing, implementing, and evaluating a custom RTL systolic-array GEMM accelerator on the Intel Cyclone V SoC FPGA (5CSXFC6D6F31C6). I present five measured contributions. First, I assess runtime precision selection. A floating-point-capable processing element requires 12.8x the area and 7.5x the dynamic power of an integer element, although it uses half the hard-multiplier resource. Two independent tools locate the expense in the per-element floating-point accumulator rather than the multiplier. An array in which every element carries floating point requires 108% to 114% of the target device, so the heterogeneous 4x4 corner inside an 8x8 grid is forced rather than preferred. Second, block floating point placed on the feed path avoids almost all of that cost: BFP16 converts to fixed point before the array, reaches all 64 elements rather than the 16 in the corner, and matches INT16 cycle for cycle to within 0.014%. The conversion costs 514 logic cells, zero DSP slices, and roughly 1% of accelerator power. I measure its accuracy crossover against IEEE BF16: block float is the more accurate format below an exponent spread of 8 and the less accurate above it. Third, a two-stage DMA fast path widens the A-tile drain to one bus beat per cycle and fans the B-tile beat across all M bank write ports in parallel. Together with operand residency and hardware tile-row batching, the fast path lifts a 512x512x512 INT8 launch from 0.327 to 3.136 hardware GOPS at a fixed 50 MHz fabric. Fourth, the accelerator runs end-to-end under Linux on the dual-A9 hard processor system through a custom kernel, device-tree overlay, and userspace UIO driver that I built for this platform. The shipping bitstream supports seven operand precisions (INT8, INT16, INT32, FP16, BF16, FP32, and block-float BFP16) with no per-precision resynthesis, and occupies all 112 DSP slices of the target part. Finally, I characterize the accelerator against the predictive-control workload that motivates this work, not against inference alone. On the SoCKit board, at the condensed-Hessian shapes a re-linearizing quadrotor controller produces (32x32x300 to 128x128x1,152), the INT8 fabric returns bit-exact results 2.09x to 2.80x faster than the same kernel compiled natively for the on-chip A9. The margin widens as the horizon lengthens. A 96-step-horizon controller sustains 1,111 Hz on the fabric against 414 Hz on the A9, moving a 1 kHz update rate from unreachable to reachable at that horizon. The characterization identifies independent output-column count, not total operation count, as the property that decides whether a systolic array can serve a controller at all. That distinction separates re-linearizing formulations that issue genuine matrix products from precomputed-gain formulations that issue only a single column and leave the array idle. That same characterization fixes the operating regime in which a 28 nm FPGA accelerator inside a 5 W envelope is the appropriate part, measured against a 4 nm application-specific GPU with 16,384 cores and a 450 W envelope. Sustained throughput per watt at large shapes tracks the process node and the arithmetic-unit budget. I confirm with a bandwidth and area model that the boundary belongs to the platform rather than the architecture: the accelerator's ceiling with unlimited memory bandwidth and unlimited fabric is 24.4 GOPS/W, set by per-element energy on 28 nm and unmoved by clock or array size. Energy per solve at a fixed control deadline is the regime I built the accelerator for. Both parts stay powered in a control loop, and each must finish one solve per tick, so the comparison reduces to average system power. The GPU's measured 33 W to 37 W idle floor alone exceeds the entire SoCKit board. Measured that way, the accelerator delivers 5.8x to 7.4x the efficiency.