Giant-scale fashions are routinely educated on a mix of various information sources.
Completely different information mixtures yield very totally different downstream performances.
We suggest a novel structure that may instantiate one mannequin for every information combination with out having to re-train the mannequin.
Our structure consists of a financial institution of knowledgeable weights, that are linearly mixed to instantiate one mannequin.
We be taught the linear mixture coefficients as a operate of the enter histogram.
To coach this structure, we pattern random histograms, instantiate the corresponding mannequin, and backprop by means of one batch of information sampled from the corresponding histogram.
We reveal the promise of our strategy to shortly acquire small specialised fashions on a number of datasets.
