The Higher Order Statistics, such as the Higher Order Moments, Cumulants and Polyspectra, have been recognized as important tools in modem time series analysis since they overcome well-known limitations of the autocorrelation/power spectrum second order methods. The systematic synthesis of parallel algorithms and architectures for the real-time estimation of moments up to any desirable maximal order k > 3 is presented. First, a design methodology is developed which can take into account the desirable characteristics of the targeted parallel architecture and used to construct an optimal locally recursive form of the algorithm amenable to efficient parallelization. The design methodology is then used to synthesize a family of algorithms and minimum latency, low granularity, processor array architectures that can compute all lags of Higher Order Moments, from the samples of the incoming data sequence in real-time.