Enriching ConvNets with pre-cortical processing enhances alignment with human brain responses

Convolutional Neural Networks (ConvNets) are the current state-of-the-art formodelling human visual processing whilst also performing tasks on a human per-formance level. Convolutional features can be seen as analogous to visual recep-tive fields and thus render them biologically plausible. However, spatially-uniformsampling and reuse of features across the entire visual field do not accurately rep-resent structural properties of the human visual system. Here, we present empir-ical findings of incorporating functional and structural properties of the humanretina into ConvNets on their alignment with human brain activity. We showthat predictions of human EEG data using ConvNets features improve by usingfoveated stimuli and that differential spatial sampling in ConvNets explains sev-eral qualities of EEG recordings. We also find that color and contrast filtering ofinputs in turn do not yield improved predictions. Overall, our results suggest thatincorporating biologically plausible spatial sampling is important for increasingrepresentational alignment between ConvNets and humans.