Reinforcement learning for hardware acceleration and system-level optimization on FPGA and integrated-circuit platforms: A survey

Introduction

Reinforcement learning (RL) has become a cornerstone for autonomous decision‑making in edge, embedded, and heterogeneous computing environments. To meet the stringent latency, power, and flexibility requirements of these domains, researchers increasingly turn to specialized hardware such as field‑programmable gate arrays (FPGAs) and custom integrated circuits (ICs). The surveyed literature—more than one hundred papers—offers a comprehensive view of how RL algorithms are mapped onto reconfigurable and fixed silicon, what performance gains are achieved, and where the field still lacks mature solutions.

Mapping RL Algorithms to Hardware

The survey categorises implementations by the RL family they target:

• Model‑based Markov decision processes (MDPs) – often realized with dedicated state‑transition engines that exploit the parallelism of FPGA lookup tables.
• Value‑based methods (e.g., Q‑learning, DQN) – typically implemented using pipelined arithmetic units for rapid Bellman updates, with on‑chip memory storing Q‑tables or neural‑network weights.
• Deep RL and actor‑critic architectures – leverage high‑throughput matrix‑multiply engines and systolic arrays, sometimes combined with quantisation techniques to fit within limited DSP resources.
• Policy‑gradient and multi‑agent approaches – benefit from flexible data‑flow graphs that can be reconfigured at runtime to accommodate varying numbers of agents or policy networks.

Across these families, the survey notes recurring hardware‑algorithm co‑design patterns: weight‑sharing across multiple processing elements, on‑chip reinforcement‑learning loops that minimise off‑chip communication, and dynamic reconfiguration to adapt precision or network depth based on workload.

Performance Evaluation and Trade‑offs

The surveyed works report a consistent set of metrics:

• Resource utilization – measured in lookup tables (LUTs), flip‑flops, DSP blocks, and on‑chip memory. Efficient designs often prune network layers or employ binary/ternary quantisation to stay within tight budgets.
• Latency and throughput – FPGA pipelines can achieve sub‑microsecond decision latencies, while ASIC implementations push throughput into the gigasample‑per‑second range for high‑frequency control tasks.
• Power consumption – energy‑aware designs exploit clock‑gating, voltage scaling, and approximate computing to keep power under a few watts, a crucial factor for edge devices.
• Speed‑up over software baselines – reported gains range from 5× to over 200× compared with CPU or GPU implementations, depending on algorithm complexity and hardware specialization.

The survey highlights that while ASICs deliver the highest raw performance, FPGAs provide a valuable middle ground: they enable rapid prototyping, runtime reconfiguration, and lower non‑recurring engineering costs.

Practical Limitations and Open Challenges

Despite impressive progress, several hurdles remain:

1. Design automation – Current toolchains still require substantial manual RTL coding and hardware‑aware algorithm tuning. End‑to‑end compilers that translate high‑level RL specifications into optimized hardware are scarce.
2. Scalability of deep models – As RL networks grow deeper, fitting them onto modest FPGA fabrics becomes difficult without aggressive quantisation, which can degrade learning stability.
3. Robustness to hardware variability – Process variations and temperature fluctuations affect fixed‑point arithmetic and can lead to policy drift; mitigation strategies are still in early stages.
4. Multi‑agent coordination – Efficiently sharing resources among many agents while preserving low latency is an open systems‑level problem.

Conclusion

The survey consolidates a fragmented body of work into a unified framework, showing that RL can be successfully accelerated on both reconfigurable and custom silicon. It underscores the importance of co‑design—aligning algorithmic choices with hardware constraints—to achieve low‑latency, energy‑efficient decision making. By cataloguing architectural patterns, performance results, and recurring bottlenecks, the review equips researchers with a clear roadmap: advance design automation, explore robust quantisation schemes, and develop system‑level strategies for multi‑agent RL on specialized hardware.