Interpreting Reservoir Computing Through the Equivalent Visualization of its Loss Landscape

Abstract

Reservoir computing (RC) has been recognized as a promising ultra-lightweight model, but its black-box nature renders its predictions less interpretable.As a result, how to interpret the underlying prediction mechanism of RC attracts increasing attention.Among the interpretability methods, visualization serves as an intuitive approach for interpreting RC, enabling even novices to observe RC loss landscape and its dependency on the parameters directly.However, existing visualization methods depict the high-dimensional loss landscape of RC only through low-dimensional 2D or 3D plots, without sufficient explanatory context.It can naturally raise concerns about the reliability and practical utility of such visualization plots.To address this issue, we propose an equivalent visualization method to depict the RC loss landscape while maintaining high confidence.Specifically, the number of parameter orderings is first introduced to quantify the representativeness of RC parameter candidates, as the parameter ordering is experimentally identified as a predominant factor influencing RC predictions.Then, the 1D or 2D periodic interpolation approach is presented to generate RC parameter candidates.These candidates are proven to form a symmetric group encompassing all parameter orderings, ensuring that the resulting 2D or 3D visualization plots are equivalent to original loss landscape in terms of parameter ordering.On this basis, how the loss landscape of RC relates to its trainability and key components (e.g., the hyperparameter) are interpreted visually.Furthermore, the reliability of the obtained visualization explanations is substantiated by the inherent memory property of RC.Finally, the proposed visualization method is designed to be model-agnostic, allowing its extension to portray the loss landscape of ResNet-56.

Keywords: Reservoir computing, Loss landscape visualization, Interpretability analysis, Symmetric group, Memory capacity.