DiScoFormer: One transformer for density and score, across distributions
DiScoFormer is a model that estimates both the density and score of a distribution from a finite sample in a single forward pass, using a transformer architecture with cross-attention. It leverages the mathematical relationship between density and score, and adapts to out-of-distribution inputs without requiring ground-truth density or score. The model outperforms kernel density estimation (KDE) in both density and score estimation, especially in high dimensions, and maintains accuracy even when moving outside the training data. It is promising for applications in generative modeling, Bayesian inference, and scientific computing due to its improved accuracy and adaptability.
More from Huggingface
LFM2.5-VL-3B for Better and Faster Vision Capabilities for the Edge
Thinking of ACE? We Can Do It with Fewer Tokens
Meta is back with Muse Glimmer: local, agentic, multimodal, and open source
