A Guide to Prior Selection

In Bayesian modeling, a prior distribution (or simply “prior”) represents our beliefs about a parameter before we have seen the data. Choosing appropriate priors is a key part of the modeling process. This guide explains the rationale behind the prior choices in the ConversionFlow library.

The Role of Priors

Priors are a way to incorporate domain knowledge into the model. They can be:

  • Informative: A narrow distribution that strongly pulls the parameter towards a certain value.

  • Weakly Informative: A wider distribution that gently guides the model, preventing it from exploring completely unrealistic parameter values.

  • Uninformative (Flat): A very wide distribution that provides little to no information to the model.

In ConversionFlow, we primarily use weakly informative priors. This is a best practice in modern Bayesian modeling, as it helps to regularize the model (preventing overfitting) without overly biasing the results.

Priors in config.yml

The priors for the beta coefficients (the strength of the relationships between nodes) are defined in the priors.beta_distributions section of the config.yml file.

HalfCauchy

The HalfCauchy distribution is often used as a weakly informative prior for scale parameters (parameters that must be non-negative). In our case, we use it for beta coefficients where we have a strong prior belief that the effect should be positive.

For example, we expect that session_start can only have a positive or zero effect on download. A user cannot be less likely to download a brochure simply because they started a session.

session_start:
  download: { distribution: HalfCauchy, sigma: 5 }

The sigma parameter controls the width of the distribution. A larger sigma allows for larger effects.

StudentT

The StudentT distribution is a good choice for a weakly informative prior for regression coefficients. It is similar to the Normal distribution but has “heavier tails,” which means it is more robust to outliers.

car_configuration:
  configdetailsevent: { distribution: StudentT, nu: 3, sigma: 2 }

  • nu: The degrees of freedom. A lower nu results in heavier tails. nu=3 is a common choice.

  • sigma: The scale parameter, similar to the standard deviation in a Normal distribution.

Normal

The Normal distribution is a good default choice for a weakly informative prior when you believe the effect is likely to be centered around a certain value (usually 0).

feature_engagement:
  add_to_cart: { distribution: Normal, mu: 0, sigma: 2.5 }

  • mu: The mean of the distribution (the center of your prior belief).

  • sigma: The standard deviation (how much you expect the parameter to vary).

Choosing Your Own Priors

When customizing the model, think about the relationships you are modeling:

  • If you have a strong belief that an effect can only be positive, a HalfCauchy or HalfNormal prior is a good choice.

  • If you believe an effect could be positive or negative, but is likely to be small, a Normal or StudentT distribution centered at 0 is a good choice.

  • If you have strong domain knowledge that suggests a different center or scale for your prior, you can adjust the mu and sigma parameters accordingly.