Complex cells and topography

Aapo Hyvärinen, Patrik Hoyer, Mika Inki, Urs Köster

We have introduced modifications of the basic ICA model that describe further statistical structure of natural images and explain further aspects of the primary visual cortex. The modifications use a linear decomposition as in the basic ICA case described above, but the components are not assumed to be all independent.

The first model in this direction is independent subspace analysis, in which the components are divided into groups or subspaces so that components in different subspaces are independent, but components in the same subspace are not. In particular, the distribution of the components in a subspace is assumed to depend only on the norm of the projection on that subspace. Typically, this implies that the components of a subspace tend to be active simultaneously.

When estimated from natural image data, the model shows emergence of complex-cell properties, in particular phase and translation invariance (insensitivity), together with orientation and frequency selectivity. Here are the estimated basis vectors, grouped according to the subspace structure.

Furthermore, we have generalized the independent subspace model so that it models more general dependency structures. The point is to define a topographic order (spatial organization) using the higher-order correlations of the components. Basically, we use correlations of energies, i.e. squares, of the components. Thus we order the basis vectors so that component that are near-by in the topographic representation tend to be active, i.e. non-zero at the same time. This can be considered as a generalization of independent subspaces, so that every neighbourhood corresponds to one subspace. Thus we obtain a linear representation in which the coefficients and the basis vectors have a topographic organization that gives us information on the statistical higher-order structure of the data.

When estimated from natural image data, the model shows simultaneous emergence of topography (i.e. spatial organization on the cortical surface) and complex-cell properties. This is because every neighbourhood corresponds to one feature subspace as in independent subspace analysis, i.e. one complex-cell.

Our most recent work shows how to estimate even the second pooling layer from natural images (Köster and Hyvärinen, 2007).

References

• A. Hyvärinen and P.O. Hoyer, "Emergence of phase and shift-invariant features by decomposition of natural images into independent feature subspaces", Neural Computation, vol. 12, no. 7, pp. 1705-1720, 2000.
• A. Hyvärinen, P.O. Hoyer and M. Inki, "Topographic independent component analysis", Neural Computation, vol. 13, no. 7, pp. 1527-1558, 2001.
• A. Hyvärinen and P.O. Hoyer, "A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images", Vision Research, vol. 41, no. 18, pp. 2413-2423, 2001.
• U. Köster and A. Hyvärinen. A two-layer ICA-like model estimated by Score Matching. Proc. Int. Conf. on Artificial Neural Networks (ICANN2007), Porto, Portugal, pp. 798-807, 2007.

Links

• imageica - MATLAB package for performing ICA, ISA, and TICA on image data