Harel Lidar

Paper summary · Oct 2026

Continual LR with Neuroevolution

Summary of the paper by Eleni Nisioti, Andrea Cossu, Kathrin Korte, Sebastian Risi · paper ↗

TL;DR

1. Problem Statement

2. Methodology / Architecture

Evolutionary Algorithms Refresher

Neuroevolution optimises the weights θ∈Rd\theta \in \mathbb{R}^d of a policy network πθ\pi_\theta with an evolutionary algorithm (Risi et al., 2025; Stanley et al., 2019). Each generation it evaluates PP candidate networks, using the episodic return RτR_\tau as the fitness of each. A genetic algorithm (GA) keeps an archive of NN networks. Each generation it perturbs parents drawn uniformly from the archive with Gaussian noise of scale σ\sigma and keeps the fittest NN of archive and offspring (Such et al., 2018). Evolution strategies (ES) keep a single network θ\theta. Each generation they evaluate PP perturbations θ+σϵi\theta + \sigma\epsilon_i, ϵi∼N(0,I)\epsilon_i \sim \mathcal{N}(0, I), and move θ\theta along their fitness-weighted sum, a Monte Carlo estimate of the gradient of the expected return under N(θ,σ2I)\mathcal{N}(\theta, \sigma^2 I) (Wierstra et al., 2008; Salimans et al., 2017).

3. Results & Performance

Prior Work

Evaluations

Metrics are computed from the return RτR_\tau (Section 2.1), averaged over several episodes over multiple environment seeds and deterministic actions.

Learning accuracy (LA), the return on each task at the end of training on it Cumulative return (Cum.) on the current task over training, which also captures how quickly a method recovers after a switch. Forgetting (F), the return lost on a task between the end of training on it and the end of the run. stability-plasticity trade-off LA −- F, as is common in continual learning Zero-shot transfer (ZT) is the return on the next task at a switch, before any training on it

Neighborhood width The neighborhood of a solution is the region of weight space around it in which the policy still solves its task. We say that a policy solves a task when its rescaled return is at least 0.5. To measure the size of the neighborhood we follow Lehman et al. (2018). We take the centroid at the end of a task and perturb the weights of each layer by a random Gaussian direction whose length is a fraction ϵ\epsilon of that layer’s weight norm, so that the measure does not depend on the scale of the weights (Li et al., 2018). The neighborhood width is the radius ϵ\epsilon at which half of the perturbed policies no longer solve the task. A wide neighborhood ensures that a method solves the new task immediately if it also contains solutions of the new task. The shared neighborhood measures this: we perturb the centroid in the same way at a single radius, ϵ=0.1\epsilon = 0.1, and report the fraction of perturbed policies that solve both the current and the next task.

Comparison between methods

Analysis

4. Takeaways

Limitations

Future Work

Questions