Retinocortical in-sensor neuromorphic vision platform for NIR-augmented artificial vision

In-sensor neuromorphic vision platforms with near-infrared (NIR) sensitivity are essential for intelligent imaging in low-light and multispectral environments. Here, a retinocortical dual-mode platform based on evolved-synaptic transistor (Evo-SynT) devices using upconversion nanoparticles (UCNP) and their integration into evolved-retina optical synapse (EROS) arrays is introduced. Evo-SynT devices exhibit key synaptic features, including paired-pulse facilitation indices exceeding 183.93% and 136.36% at a 0.5 s interval under 808 nm and 940 nm illumination, respectively, and analog weight modulation across 512 conductance states. A 12×12 EROS array enables dual-mode operation: retinal-like in-sensor preprocessing and cortical-like in-memory classification. The EROS array improves pedestrian detection accuracy under low-light conditions from 0.7806 to 0.8481 (808 nm) and 0.9071 (940 nm), and achieves classification accuracies of 77.19% and 79.40%, respectively. These results highlight the EROS platform as a scalable solution for integrated NIR-sensitive neuromorphic vision systems. Current artificial vision systems are less effective beyond the visible spectrum. Here, An et al. report an oxide-semiconductor-based vision platform that enables direct near-infrared signal detection, reliable image enhancement and subsequent image processing under low-light conditions.