The brain's faculty to assimilate and retain information, continually updating its memory while limiting the loss of valuable past information, remains largely a mystery. We address this challenge related to continuous learning in the context of associative memory networks, where the iterative storage of correlated patterns traditionally requires non-local learning rules or external memory systems. Our work demonstrates how incorporating biologically-inspired inhibitory plasticity enables networks to autonomously explore their attractor landscape through local dynamics alone. The autonomous recovery of stored patterns enables continuous addition of new memories by allowing them to be incorporated with previously stored information while limiting its degradation, drawing parallels with memory consolidation during sleep-like states in biological systems. The resulting framework provides insights into how neural circuits might maintain memories through purely local interactions, while suggesting new approaches for building more biologically plausible associative memory systems.