Apple researchers advance AI and ML via elementary analysis, and to help the broader analysis group and assist speed up progress on this subject, we share a lot of this work via publications and engagement at conferences.
Subsequent month, the thirty ninth annual Convention on Neural Info Processing Techniques (NeurIPS), shall be held in San Diego, California, with a satellite tv for pc occasion additionally going down in Mexico Metropolis, Mexico. Apple is proud to as soon as once more to take part on this essential occasion for the group and to help it with our sponsorship.
On the most important convention and related workshops, Apple researchers will current many papers throughout a wide range of subjects in ML. As highlighted beneath, this consists of new works advancing privacy-preserving ML, understanding the strengths and limitations of reasoning fashions, sharing progressive approaches to generative AI, and detailing a principled strategy to figuring out coaching knowledge mixtures.
NeurIPS attendees will be capable to expertise demonstrations of Apple’s ML analysis in our sales space # 1103, throughout exhibition hours. Apple can also be sponsoring and collaborating in a variety of affinity group-hosted occasions that help underrepresented teams within the ML group. A complete overview of Apple’s participation in and contributions to NeurIPS 2025 will be discovered right here, and a choice of highlights observe beneath.
Advancing Privateness-Preserving ML
At Apple, we consider privateness is a elementary human proper, and advancing privacy-preserving strategies in AI and ML is a crucial space of ongoing analysis. The work Apple researchers will current at NeurIPS this yr consists of a number of papers sharing progress on this space.
Precisely estimating a discrete distribution from samples is a elementary job in statistical ML. Measuring accuracy by the Kullback-Leibler (KL) divergence error is beneficial for selling range and smoothness within the estimated distribution, and is essential in a variety of contexts, together with knowledge compression, speech recognition, and language modeling. Within the Highlight paper, Occasion-Optimality for Non-public KL Distribution Estimation, Apple researchers discover learn how to estimate chance distributions precisely whereas defending privateness. The work focuses on instance-optimality – designing algorithms that adapt to every particular dataset and carry out practically in addition to the absolute best methodology for that case. The paper shares new algorithms that obtain this steadiness each with and with out differential privateness, exhibiting that distributions will be estimated precisely below KL error, whereas mathematically guaranteeing that no single particular person’s knowledge will be inferred.
In differential privateness, randomizing which knowledge factors are utilized in computations can amplify privateness, making it harder to attach knowledge to a person. Within the Highlight paper, Privateness Amplification by Random Allocation, Apple researchers analyze a brand new sampling technique known as random allocation. On this sampling scheme a consumer’s knowledge is utilized in okay steps chosen randomly and uniformly from a sequence (or set) of t steps. The paper gives first theoretical ensures and numerical estimation algorithms for this scheme. This permits for higher privateness analyses (and therefore higher privacy-utility tradeoffs) for a bunch of essential algorithms similar to in style variants of differentially non-public SGD and algorithms for environment friendly safe aggregation, similar to these introduced in PREAMBLE: Non-public and Environment friendly Aggregation by way of Block Sparse Vectors, one other paper that Apple researchers will current at NeurIPS this yr.
Understanding the Strengths and Limitations of Reasoning Fashions
Reasoning is a crucial functionality for AI, enabling techniques to perform advanced aims that require planning and a number of steps – similar to fixing math and coding issues, in addition to duties for robots and digital assistants. Whereas the sector has made vital progress in growing reasoning fashions, elementary analysis that rigorously investigates the strengths and limitations of present approaches is crucial to additional advancing this functionality for the longer term.
At NeurIPS, Apple researchers will current The Phantasm of Pondering: Understanding the Strengths and Limitations of Reasoning Fashions by way of the Lens of Drawback Complexity, which explores how present AI fashions deal with advanced reasoning duties. With controllable puzzle environments, the work systematically exams how these fashions’ efficiency adjustments as issues enhance in complexity (see Determine 1). The paper exhibits that the accuracy of frontier Giant Reasoning Fashions (LRMs) collapses past sure complexities, and finds that LRMs’ reasoning effort will increase together with the complexity of a problem – up to a degree – after which it declines, regardless of having a enough token finances. The work additionally compares the efficiency of Giant Reasoning Fashions (LRMs) and LLMs with equal inference compute, discovering that LLMs outperform LRMs for low-complexity duties, LRMs present a bonus in medium-complexity duties, and each varieties fail for high-complexity duties. The paper gives perception into LRMs’ strengths and limitations, elevating essential questions on these fashions’ reasoning capabilities as we speak, which can finally illuminate alternatives to make LRMs extra succesful sooner or later.
One of many authors of the above paper can even ship an Expo Speak on the subject of reasoning on Tuesday, December 2, at 8:30am PST within the Higher Degree Ballroom 20AB. The speak will present a important evaluation of reasoning in language fashions, spotlight why present evaluations will be deceptive, and emphasize that reasoning isn’t just about “what” fashions reply, however “how” they resolve issues.
Revolutionary Approaches to Generative AI
The trade has made spectacular progress in high-resolution picture era fashions, however the dominant approaches even have undesirable traits. Diffusion fashions are computationally costly in each coaching and inference, autoregressive generative fashions will be costly at inference and require quantization that may adversely have an effect on their output’s constancy, and hybrid fashions that apply autoregressive strategies immediately in steady house are advanced.
Within the NeurIPS Highlight paper, STARFlow: Scaling Latent Normalizing Flows for Excessive-resolution Picture Synthesis, Apple researchers share a scalable strategy that generates comparable high quality high-resolution pictures (see Determine 2), with out the computational value and complexity of prior strategies. This methodology builds on the Transformer Autoregressive Circulation (TARFlow), which mixes normalizing flows (NF) and the autoregressive transformer structure. STARFlow produces pictures at resolutions and high quality ranges beforehand thought unreachable for NF fashions, rivaling high diffusion and autoregressive strategies whereas sustaining actual chance modeling and quicker inference. This work is the primary profitable demonstration of normalizing flows at this scale and determination, and it exhibits that normalizing flows are a strong various to diffusion fashions for AI picture era.
As generative AI fashions grow to be more and more broadly used, environment friendly strategies to manage their generations – for instance to make sure they produce protected content material or present customers with the flexibility to discover model adjustments – have gotten more and more essential. Ideally, these strategies ought to preserve output high quality, and never require a considerable amount of knowledge or computational value at coaching or inference time.
Apple researchers have beforehand demonstrated that an efficient and environment friendly strategy to this problem is intervening completely on mannequin activations, with the purpose of correcting distributional variations between activations seen when utilizing prompts from a supply vs. a goal set (e.g. poisonous and non-toxic sentences). At NeurIPS, Apple researchers will current LinEAS: Finish-to-end Studying of Activation Steering with a Distributional Loss,which describes linear end-to-end activation steering (LinEAS), an strategy skilled with a world loss that accounts concurrently for all layer-wise distributional shifts (see Determine 3). LinEAS solely requires a handful of unpaired samples to be efficient, and beats related baselines on toxicity mitigation in language fashions. Its world optimization permits together with a sparsity regularization, leading to extra exact and focused interventions which are efficient whereas preserving the bottom mannequin fluency. This methodology is modality-agnostic is proven to outperform present activation-steering strategies at mitigating and together with new ideas on the output of single-step text-to-image era fashions.
A Principled Strategy to Figuring out Coaching Information Mixtures
Giant basis fashions are sometimes skilled on knowledge from a number of domains, and the info combination – the proportion of every area utilized in coaching – performs a important function in mannequin efficiency. The usual strategy to choosing this combination depends on trial and error, which turns into impractical for large-scale pretraining.
At NeurIPS, Apple researchers will current Scaling Legal guidelines for Optimum Information Mixtures, which gives a greater strategy to this elementary problem. The paper shares a scientific methodology to find out the optimum knowledge combination for any goal area utilizing scaling legal guidelines (see Determine 4). The scaling legal guidelines predicts the lack of a mannequin of dimension N skilled with D tokens with a combination h . The paper exhibits that these scaling legal guidelines are common, and demonstrates their predictive energy for large-scale pretraining of huge language fashions (LLMs), native multimodal fashions (NMMs), and enormous imaginative and prescient fashions (LVMs). It additionally exhibits that these scaling legal guidelines can extrapolate to new knowledge mixtures and throughout scales: their parameters will be precisely estimated utilizing just a few small-scale coaching runs, and used to estimate the efficiency at bigger scales and unseen area weights. The scaling legal guidelines permit practitioners to derive the optimum area weights for any goal area below a given coaching finances (N, D), offering a principled various to pricey trial-and-error strategies.
Demonstrating ML Analysis within the Apple Sales space
Throughout exhibition hours, NeurIPS attendees will be capable to work together with stay demos of Apple ML analysis in sales space # 1103. These embody:
-
MLX – an open supply array framework designed for Apple silicon that permits quick and versatile ML and scientific computing on Apple {hardware}. The framework is optimized for Apple silicon’s unified reminiscence structure and leverages each the CPU and GPU. Guests will be capable to expertise two MLX demos:
- Picture era with a big diffusion mannequin on an iPad Professional with M5 chip
- Distributed compute with MLX and Apple silicon: Guests will be capable to discover textual content and code era with a 1 trillion-parameter mannequin operating in Xcode on a cluster of 4 Mac Studios outfitted with M3 Extremely chips, every working with 512 GBs of unified reminiscence.
-
FastVLM – a household of mobile-friendly imaginative and prescient language fashions, constructed utilizing MLX. These fashions use a mixture of CNN and Transformer architectures for imaginative and prescient encoding designed particularly for processing high-resolution pictures. Collectively, they exhibit a powerful strategy that achieves an optimum steadiness between accuracy and velocity. Guests will get to expertise a real-time visible question-and-answer demo on iPhone 17 Professional Max.
Supporting the ML Analysis Neighborhood
Apple is dedicated to supporting underrepresented teams within the ML group, and we’re proud to once more sponsor a number of affinity teams internet hosting occasions onsite at NeurIPS 2025 in San Diego, together with Girls in Machine Studying (WiML) (workshop on December 2), LatinX in AI (workshop on December 2), and Queer in AI (workshop and night social on December 4). Along with supporting these workshops with sponsorship, Apple staff can even be collaborating at every of those, in addition to different occasions going down in the course of the convention.
Be taught Extra about Apple ML Analysis at NeurIPS 2025
This put up highlights only a handful of the works Apple ML researchers will current at NeurIPS 2025, and a complete overview and schedule of our participation will be discovered right here.
