History matching is a crucial process that enables the calibration of uncertain parameters of the numerical model to obtain an acceptable match between simulated and observed historical data. However, the implementation of the history-matching algorithm is usually based on iteration, which is a computationally expensive process due to the numerous runs of the simulation. To address this challenge, we propose a surrogate model for simulation based on an autoregressive model combined with a convolutional gated recurrent unit (ConvGRU). The proposed ConvGRU-based autoregressive neural network (ConvGRU-AR-Net) can accurately predict state maps (such as saturation maps) based on spatial and vector data (such as permeability and relative permeability, respectively) in an end-to-end fashion. Furthermore, history matching must be performed multiple times throughout the production cycle of the reservoir to fit the most recent production observations, making continual learning crucial. To enable the surrogate model to quickly learn recent data by transferring experience from previous tasks, an ensemble-based continual learning strategy is used. Together with the proposed neural network–based surrogate model, the randomized maximum likelihood (RML) is used to calibrate uncertain parameters. The proposed method is evaluated using 2D and 3D reservoir models. For both cases, the surrogate inversion framework successfully achieves a reasonable posterior distribution of reservoir parameters and provides a reliable assessment of the reservoir’s behaviors.