Move fashions parameterized as time-dependent velocity fields can generate information from noise by integrating an ODE. These fashions are sometimes skilled utilizing move matching, i.e. by sampling random pairs of noise and goal factors and guaranteeing that the speed subject is aligned, on common, with when evaluated alongside a phase linking to . Whereas these pairs are sampled independently by default, they will also be chosen extra rigorously by matching batches of noise to goal factors utilizing an optimum transport (OT) solver. Though promising in principle, the OT move matching (OT-FM) method shouldn’t be extensively utilized in apply. Zhang et al. (2025) identified just lately that OT-FM really begins paying off when the batch measurement grows considerably, which solely a multi-GPU implementation of the Sinkhorn algorithm can deal with. Sadly, the prices of operating Sinkhorn can rapidly balloon, requiring operations for each pairs used to suit the speed subject, the place is a regularization parameter that ought to be usually small to yield higher outcomes. To meet the theoretical guarantees of OT-FM, we suggest to maneuver away from batch-OT and rely as a substitute on a semidiscrete formulation that leverages the truth that the goal dataset distribution is often of finite measurement . The SD-OT drawback is solved by estimating a twin potential vector utilizing SGD; utilizing that vector, freshly sampled noise vectors at prepare time can then be matched with information factors at the price of a most inside product search (MIPS). Semidiscrete FM (SD-FM) removes the quadratic dependency on that bottlenecks OT-FM. SD-FM beats each FM and OT-FM on all coaching metrics and inference funds constraints, throughout a number of datasets, on unconditional/conditional technology, or when utilizing mean-flow fashions.
- ** Work achieved whereas at Apple
