Click on the image to see a PDF version (for zooming in)

Fig. 14.2. Sparse, redundancy-reduced coding with self-organized lateral connections. In (a), an example test input consisting of two multi-segment contours is shown. The V1 initially responds to this pattern with multiple large patches of activation (b), but lateral interactions focus the response into the most active neurons (d). This process results in a sparse code, as shown by the increased kurtosis values beneath each plot. The settling reduces redundancy but does not lose information; the input pattern can be reconstructed from both the initial response (c) and the settled response (e) equally well. When the lateral interactions are replaced with isotropic patterns, such as a sum of two Gaussians (f), a sparse code with a similar kurtosis results. However, crucial information about the input is lost in this process. All active neurons inhibit each other, and occasionally a crucial component of the representation is turned off. For example, the activity patch at the center represents the rightmost element of the left contour. It disappears in the settling process of the SoG network, and consequently the reconstruction image is missing this element as well (g). Self-organized patchy lateral connections are therefore crucial in forming a sparse redundancy-reduced coding of the visual input.