Data driven caching strategy

Abstract

A computer-implemented method includes receiving an input for a model from a data stream, computing an output from the model, and storing the input and the output as an element of a cache. The method also includes using an algorithm to determine a set of parameters associated with the cache; the algorithm optimizes a function including a time taken by the model to generate outputs from a set of inputs sampled from the data stream. The method further includes calculating a caching score associated with each cache element, based on the set of parameters and the time taken by the model to generate the output, a usage of the element expressed as a number of iterations over which the element has been retained in the cache, and a frequency of usage of the element. The method also includes subsequently removing from the cache the element having the lowest caching score.

CPC Classifications