Frequently Asked Questions (FAQ)

This section provides answers to common questions and solutions to common problems.

General

Q: What is the main purpose of the ConversionFlow library?

A: The ConversionFlow library is designed to analyze user conversion funnels using a two-stage Bayesian network model. It helps identify key factors influencing conversion and provides recommendations for optimization.

Q: Why were Bayesian Networks chosen over other network or sequence models like Markov Chains or Recurrent Neural Networks (RNNs)?

A: That’s a great question. While other models can analyze sequences of events, Bayesian Networks offer a unique combination of features that make them particularly well-suited for understanding and optimizing customer journeys:

  1. Explicit Causality: Unlike models that only capture sequential patterns (e.g., Markov Chains), Bayesian Networks allow us to define a causal graph based on domain knowledge. This means we are not just modeling that “B often follows A,” but rather that “A has a causal influence on B.” This is critical for the optimization stage, where we need to understand which touchpoints drive conversions, not just which ones precede them.

  2. Handling Non-Sequential Behavior: Customer journeys are often not linear. A user might explore different touchpoints in various orders. Bayesian Networks model the relationships between touchpoints regardless of the order in which they occur, which is a more realistic representation of user behavior than a strict sequential model like a basic Markov Chain or RNN.

  3. Quantifying Uncertainty: Bayesian Networks excel at quantifying uncertainty. Instead of providing a single point estimate for the influence of a touchpoint, they provide a full probability distribution. This allows us to understand not just the likely effect, but also our level of certainty about that effect, which is crucial for making robust business decisions.

  4. Interpretability: The graphical structure of a Bayesian Network is highly interpretable. It provides a clear visual representation of the assumed causal relationships in the customer journey, making the model’s assumptions transparent and easy to discuss with stakeholders.

Here’s a more relevant comparison table:

Method

Strengths

Weaknesses for this Problem

Bayesian Networks

Causal inference, uncertainty quantification, interpretability

Can be computationally intensive, requires defining a causal structure

Markov Chains

Simple, good for modeling sequences

Assumes memorylessness (the next step only depends on the current one), does not model causality

Recurrent Neural Networks (RNNs)

Powerful for complex sequences, can capture long-term dependencies

“Black box” model (not easily interpretable), requires large amounts of data, does not explicitly model causality

In summary, while other methods could predict the next step in a journey, Bayesian Networks are the superior choice for the ultimate goal of this project: to understand the causal drivers of conversion and use that understanding to make optimal, uncertainty-aware decisions.

Q: What is the difference between the “Estimation” and “Optimization” stages?

A: The “Estimation” stage is about learning from the data to build a probabilistic model of the customer journey. The “Optimization” stage then uses this model to find the best allocation of a marketing budget to maximize conversions.

Installation and Setup

Q: I’m having trouble installing the dependencies. What should I do?

A: Make sure you are using a compatible Python version (e.g., Python 3.8 or higher). Try creating a fresh virtual environment and installing the dependencies listed in requirements.txt using pip install -r requirements.txt.

Running the Pipeline

Q: The pipeline is running very slowly. How can I speed it up?

A: The Bayesian network estimation can be computationally intensive. You can try the following:

  • Use a smaller dataset for initial testing.

  • Reduce the number of MCMC samples in the configuration file (config.yml).

  • Simplify the model by reducing the number of nodes or edges.

Q: I’m getting an error about a missing configuration file.

A: Make sure you have a config.yml file in the root directory of the project. You can use assets/config.yml as a template.

Model Diagnostics

Q: How can I check if the MCMC simulation has converged?

A: You should check the Gelman-Rubin statistic (R-hat) and the effective sample size (ESS) in the convergence_summary.csv file. An R-hat value close to 1.0 (e.g., < 1.01) and an ESS of at least 400 for each parameter are good indicators of convergence.

Q: How do I perform posterior predictive checks to validate the model?

A: The purchase_outcome_ppc.png visualization shows the results of a posterior predictive check. This plot compares the distribution of the observed data to the distribution of data simulated from the model. If the model is a good fit, the simulated data should look similar to the observed data.

Model Specification

Q: What is a “prior” and why is it important in this model?

A: A prior represents our beliefs about a model parameter before we have seen the data. In this library, we use “weakly informative priors” to help guide the model and prevent it from finding extreme, unrealistic parameter values (a phenomenon known as overfitting). For more details, see the Prior Selection Guide.

Q: How do I choose between HalfCauchy, StudentT, and Normal distributions for my priors?

A: As a general guideline:

  • Use HalfCauchy or HalfNormal if you believe an effect can only be positive.

  • Use StudentT or Normal if you believe an effect could be positive or negative, but is likely to be small. The StudentT distribution is generally more robust to outliers than the Normal distribution.

Q: What do the mu, sigma, and nu parameters in the config.yml file mean?

A: These parameters define the shape of your prior distributions:

  • mu: The mean of the distribution (the center of your prior belief).

  • sigma: The scale parameter, similar to the standard deviation. It controls how much the parameter can vary from the mean.

  • nu: The degrees of freedom for the StudentT distribution. A lower nu results in “heavier tails,” making the prior less sensitive to outliers.

Interpretation of Results

Q: How should I interpret the credible intervals in the parameter summary?

A: The 94% Highest Posterior Density Interval (HPDI) represents the range where the true parameter value lies with 94% probability, given the data and the model. A wider interval indicates more uncertainty about the parameter’s value.

Q: What do the posterior_effects.png plots show?

A: These plots visualize the marginal effect of each touchpoint on the probability of conversion. They allow you to compare the relative importance of different touchpoints in driving conversions.

Model Comparison

Q: How can I compare the performance of two different models?

A: You can use the Leave-One-Out Cross-Validation (LOO-CV) information in the parameter_comparison_loo.csv file. The elpd_loo value (expected log predictive density) is a good metric for comparing models. A higher elpd_loo indicates a better out-of-sample predictive accuracy.