TRAINING METHOD FOR CONTINUAL LEARNING MODEL AND NON-TRANSITORY COMPUTER-READABLE MEDIUM

Abstract

A training method for continual learning model and a non-transitory computer-readable medium are proposed. The method includes: training the encoder and self-attention layer in the essence generation procedure according to the raw data of a task when the current training process is the first task in continual learning; otherwise, freezing the parameters of the encoder and self-attention layer, performing the essence generation procedure to convert the raw data into a data essence, and adding the data essence into the essence memory. The training process is repeated until the continual learning model converges. The training process includes: obtaining a training batch from the raw data, updating the replay memory according to the training batch, training the continual learning model according to the replay memory and the essence memory, and updating the data essence in the essence memory when the current training process is the first task.

CPC Classifications