From b17e31a3316d6a975ab0faa00727d583d5a28556 Mon Sep 17 00:00:00 2001 From: josemanuel22 Date: Fri, 2 Aug 2024 18:20:49 +0200 Subject: [PATCH] change readme --- paper.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/paper.md b/paper.md index 58fa169..2e95d49 100644 --- a/paper.md +++ b/paper.md @@ -36,7 +36,7 @@ Generative adversarial networks (GANs) [goodfellow2014generative], $f$-GANs [now Approximation of 1-dimensional (1D) parametric distributions is a seemingly naive problem for which the above-mentioned models can perform below expectations. In [zaheer2017gan], the authors report that various types of GANs struggle to approximate relatively simple distributions from samples, emerging with MMD-GAN as the most promising technique. However, the latter implements a kernelized extension of a moment-matching criterion defined over a reproducing kernel Hilbert space, and consequently, the objective function is expensive to compute. -In this work, we introduce a novel approach to train univariate implicit models that relies on a fundamental property of rank statistics. Let $r_1 < r_2 < \cdots < r_k$ be a ranked (ordered) sequence of independent and identically distributed (i.i.d.) samples from the generative model with probability density function (pdf) $tilde{p}$, and let $y$ be a random sample from a pdf $p$. If $\tilde{p} = p$, then $\mathbb{P}(r_{i-1} \leq y < r_{i}) = \frac{1}{K}$ for every $i = 1, \ldots, K+1$, with the convention that $r_0=-\infty$ and $r_{K+1}=\infty$ (see, e.g., [rosenblatt1952remarks] or [elvira2016adapting] for a short explicit proof). This invariant property holds for any continuously distributed data, i.e., for any data with a pdf $p$. Consequently, even if $p$ is unknown, we can leverage this invariance to construct an objective (loss) function. This objective function eliminates the need for a discriminator, directly measuring the discrepancy of the transformed samples with respect to (w.r.t.) the uniform distribution. The computational cost of evaluating this loss increases linearly with both $K$ and $N$, allowing for low-complexity mini-batch updates. Moreover, the proposed criterion is invariant across true data distributions, hence we refer to the resulting objective function as invariant statistical loss (ISL). Because of this property, the ISL can be exploited to learn multiple modes in mixture models and different time steps when learning temporal processes. Additionally, considering the marginal distributions independently, it is straightforward to extend ISL to the multivariate case. +In this work, we introduce a novel approach to train univariate implicit models that relies on a fundamental property of rank statistics. Let $r_1 < r_2 < \cdots < r_k$ be a ranked (ordered) sequence of independent and identically distributed (i.i.d.) samples from the generative model with probability density function (pdf) $\tilde{p}$, and let $y$ be a random sample from a pdf $p$. If $\tilde{p} = p$, then $\mathbb{P}(r_{i-1} \leq y < r_{i}) = \frac{1}{K}$ for every $i = 1, \ldots, K+1$, with the convention that $r_0=-\infty$ and $r_{K+1}=\infty$ (see, e.g., [rosenblatt1952remarks] or [elvira2016adapting] for a short explicit proof). This invariant property holds for any continuously distributed data, i.e., for any data with a pdf $p$. Consequently, even if $p$ is unknown, we can leverage this invariance to construct an objective (loss) function. This objective function eliminates the need for a discriminator, directly measuring the discrepancy of the transformed samples with respect to (w.r.t.) the uniform distribution. The computational cost of evaluating this loss increases linearly with both $K$ and $N$, allowing for low-complexity mini-batch updates. Moreover, the proposed criterion is invariant across true data distributions, hence we refer to the resulting objective function as invariant statistical loss (ISL). Because of this property, the ISL can be exploited to learn multiple modes in mixture models and different time steps when learning temporal processes. Additionally, considering the marginal distributions independently, it is straightforward to extend ISL to the multivariate case. # Software Description @@ -67,6 +67,6 @@ examples and tutorials on how to use the package. # Acknowledgements -This work has been supported by the the Office of Naval Research (award N00014-22-1-2647) and Spain's Agencia Estatal de Investigación (refs. PID2021-125159NB-I00 TYCHE and PID2021-123182OB-I00 EPiCENTER). Pablo M. Olmos also acknowledges the support by Comunidad de Madrid under grants IND2022/TIC-23550 and ELLIS Unit Madrid. +This work has been supported by the the Office of Naval Research (award N00014-22-1-2647) and Spain's Agencia Estatal de Investigación (refs. PID2021-125159NB-I00 TYCHE and PID2021-123182OB-I00 EPiCENTER). # References \ No newline at end of file