Bayes-Laplace Rule

Name

UxHwDoubleBayesLaplace, UxHwFloatBayesLaplace — Perform Bayesian inference with distributional arithmetic.

Synopsis

#include <uxhw.h>

double UxHwDoubleBayesLaplace(
  double  (*statisticalModel)(void *, double),
  void *  modelParams,
  double  prior,
  double  observedData,
  size_t  numberOfObservations
);

float UxHwFloatBayesLaplace(
  float  (*statisticalModel)(void *, float),
  void * modelParams,
  float  prior,
  float  observedData,
  size_t numberOfObservations
);

Description

Performs Bayesian inference to calculate and return the posterior distribution of the parameter $\theta$ using Bayes' rule (also known as Bayes-Laplace rule):

$$p(\theta \mid \text{data}) = \frac{p(\text{data} \mid \theta) \cdot p(\theta)}{\int p(\text{data} \mid \theta) \cdot p(\theta) \, d\theta},$$

where $p(\theta)$ is the prior, $p(\text{data} \mid \theta)$ is the likelihood, and $p(\theta \mid \text{data})$ is the posterior. The function returns the normalized posterior. If $p(\theta)$ is a discrete distribution, the normalizing constant (the marginal likelihood) is replaced by:

$$\sum_{\theta} p(\text{data} \mid \theta) \cdot p(\theta).$$

The parameter $\theta$ and each observation are scalar (one-dimensional) values, and observations are assumed to be independent and identically distributed according to $p(\text{data} \mid \theta)$.

Parameters

• statisticalModel — A pointer to a function that calculates the distribution $p(\text{data} \mid \theta)$. Takes parameters of modelParams and a particle valued theta. For a given theta, this function returns a distribution for the given observations.

• modelParams — A pointer to additional parameters required by the distribution $p(\text{data} \mid \theta)$ (such as known variances or calibration constants). Pass NULL if none.

• prior — The prior distribution $p(\theta)$ of the parameter $\theta$, which must be one-dimensional. This can be a parametric distribution (e.g., UxHwDoubleUniformDist(), UxHwDoubleGaussDist()) or an empirical distribution created from samples (e.g., UxHwDoubleDistFromSamples()).

• observedData — The empirical distribution of observed data:

  • For a single observation, pass the measured value directly (treated as a point-mass distribution).
  • For multiple observations, create a distribution using UxHwDoubleDistFromSamples(samples, count).

• numberOfObservations — The number of independent observations in observedData. This parameter scales the joint likelihood across multiple observations, such that $p(\text{data} \mid \theta)$ is correctly computed as $\prod_{i=1}^n p(\text{data}_i \mid \theta)$.

Return Values

Returns the posterior distribution $p(\theta | \text{data})$.

✏️   Example: Inferring the Mean from Noisy Measurements

#include <uxhw.h>
#include <stdio.h>

typedef struct
{
  double sigma;
} ModelParameters;

/*
 *	Likelihood: P(observation | theta) modeled as Gaussian noise.
 */
double
measurementModel(void * args, double theta)
{
  ModelParameters * p = (ModelParameters *)args;

  return UxHwDoubleGaussDist(theta, p->sigma);
}

typedef enum
{
  kNumObservations = 5,
} ObservationConstants;

int
main(void)
{
  /*
   *	Prior: theta uniformly distributed between 0 and 2.
   */
  double prior = UxHwDoubleUniformDist(0.0, 2.0);

  /*
   *	Observed measurements
   */
  double	measurements[kNumObservations]	= {0.9, 0.95, 0.99, 1.1, 1.2};
  double	observedData			= UxHwDoubleDistFromSamples(measurements, kNumObservations);

  /*
   *	Likelihood standard deviation
   */
  ModelParameters params = {.sigma = 0.4};

  /*
   *	Compute posterior distribution over theta
   */
  double posterior = UxHwDoubleBayesLaplace(
    &measurementModel,
    &params,
    prior,
    observedData,
    kNumObservations
  );

  printf("Prior: %lf\n", prior);
  printf("Posterior: %lf\n", posterior);

  return 0;
}

✏️   Advanced Example: Importance Sampling via Bayesian Inference

Importance sampling draws samples from a tractable proposal distribution $q(x)$ and reweights them to produce an approximation of a target distribution $p(x)$, using the importance weight:

$$w(x) = \frac{p(x)}{q(x)}.$$

Bayes' rule provides a natural way to perform this reweighting. Treating the proposal $q$ as the prior and choosing a likelihood $\ell(x)$ proportional to $w(x)$, the posterior becomes:

$$\frac{\ell(x) \cdot q(x)}{\int \ell(x') \cdot q(x') \, dx'} \propto w(x) \cdot q(x) = p(x),$$

so UxHwDoubleBayesLaplace() calculates the target distribution directly. The proportionality constant in $\ell$ is included within the marginal likelihood normalization.

In the example below, the target is $\mathcal{N}(5, 5)$ and the proposal is $\mathrm{Uniform}[0, 10]$. The likelihood is constructed as $\mathcal{N}(0, 1/w(x))$, whose density at $0$ is $w(x) / \sqrt{2\pi}$ and is proportional to the importance weight.

#include <uxhw.h>
#include <stdio.h>
#include <math.h>

/*
 *	Importance Sampling: Target p(x) = Gaussian( µ = 5.0, σ = 5.0)
 *	                     Proposal q(x) = Uniform[0, 10]
 *	                     Weight w(x) = p(x) / q(x)
 */
typedef struct 
{
  double mu;
  double sigma;
  double proposalDensity;
} ImportanceSamplingParameters;



double
importanceWeightModel(void * params, double x)
{
  ImportanceSamplingParameters * p = (ImportanceSamplingParameters *) params;
  /*
   *	Target distribution: p(x) = Gaussian(5.0, 5.0)
   */
  double targetDensity = exp(-0.5 * pow((x - p->mu) / p->sigma, 2.0)) 
              / (p->sigma * sqrt(2.0 * M_PI));

  /*
   *	Importance weight: w(x) = p(x) / q(x)
   */
  double weight = targetDensity / p->proposalDensity;

  /*
   *	Return a likelihood that is proportional to the importance weight.
   *	Given that the density of N(0, sigma) at 0 is 1 / (sigma * sqrt(2*pi)),
   *	setting sigma = 1/weight makes the likelihood proportional to the weight.
   */
  double sigma = 1.0 / weight;

  return UxHwDoubleGaussDist(0.0, sigma);
}

int
main(void)
{
  /*
   *	Proposal distribution (our "prior"): Uniform[0, 10]
   *	Easy to sample from, but wrong shape
   */
  ImportanceSamplingParameters params = {.mu = 5.0, .sigma = 5.0, .proposalDensity = 0.1};

  double proposal = UxHwDoubleUniformDist(0.0, 10.0);
  printf("Proposal mean: %lf (uniform 0-10)\n", proposal);

  /*
   *	Apply importance sampling via Bayes' rule:
   *	The likelihood reweights uniform samples to match Gaussian(5.0, 5.0)
   */
  double target = UxHwDoubleBayesLaplace(
    &importanceWeightModel,
    &params,
    proposal,
    0.0,  /* reference point for likelihood evaluation */
    1
  );

  printf("Target mean: %lf (should be ~5.0)\n", target);

  return 0;
}

Comparison with Other Inference Methods

This distributional approach computes the posterior directly without assuming any unimodality, smoothness, or parametric form for the posterior. This differentiates it from other inference methods, such as Markov Chain Monte Carlo, which requires sampling and convergence diagnostics; variational inference, which approximates the posterior within a chosen parametric family (e.g., mean-field); and the Laplace approximation, which assumes a unimodal posterior with a smooth density.