convergenceDFM: Dynamic Factor Models for Economic Convergence Analysis

Description

Tools to analyze economic convergence using Dynamic Factor Models (DFM) and Factor Ornstein-Uhlenbeck (OU) processes. Includes data preparation, PLS-based factor selection, DFM estimation with robust inference, OU estimation (Stan or fallback), formal convergence tests, robustness checks, rotation/coupling tests, and visualization utilities.

Main functions

• run_complete_factor_analysis_robust: End-to-end pipeline.

• estimate_DFM: Estimate a VAR-based Dynamic Factor Model.

• estimate_factor_OU: Estimate Factor OU model (Stan or fallback).

• run_convergence_robustness_tests: Robustness test suite.

• visualize_factor_dynamics: Visualization helpers.

Author(s)

Maintainer: José Mauricio Gómez Julián isadorenabi@pm.me

See Also

vignette("convergence-analysis")

Select optimal VAR lag order with multiple criteria

Description

Determines the optimal lag order for a Vector Autoregression model using information criteria, stability checks, serial correlation tests, and optional out-of-sample validation.

Usage

choose_var_lag(
  F_combined,
  lag.max = 4,
  type = "const",
  p_pref = c("SC(n)", "HQ(n)"),
  alpha = 0.05,
  oos_eval = TRUE,
  oos_start = 0.7,
  verbose = TRUE
)

Arguments

Details

The function combines multiple selection criteria: (1) information criteria (AIC, BIC, HQ), (2) VAR stability (eigenvalue modulus < 1), (3) Portmanteau test for serial correlation, and (4) out-of-sample forecast performance. Returns the model that best balances these considerations.

Value

List with components:

p

Selected optimal lag order.

fit

Fitted VAR model object of class varest.

roots_ok

Logical indicating if stability condition is satisfied.

serial_ok

Logical indicating if serial correlation test passed.

oos_mse

Out-of-sample mean squared error (if oos_eval = TRUE).

Incremental R-squared from X in OU model

Description

Computes the incremental explanatory power (delta R-squared) contributed by X factors in predicting Y factors, both in-sample and out-of-sample.

Usage

deltaR2_ou(
  results_robust,
  lag = 1,
  oos = TRUE,
  seed = 123,
  ridge = 1e-08,
  verbose = TRUE
)

Arguments

Value

List with components:

in_sample

Data frame with R2_full, R2_baseline, and deltaR2.

per_equation

Vector of deltaR2 for each Y factor equation.

OOS

List with out-of-sample RMSE and deltaR2 (if oos = TRUE).

Diagnose and prepare data matrices

Description

Performs data validation, missing value imputation, and variance checks on input matrices to prepare them for factor analysis. Handles dimension compatibility, NA values, and zero-variance columns.

Usage

diagnose_data(X_matrix, Y_matrix, verbose = TRUE)

Arguments

Details

The function:

• Converts to matrix format if needed

• Validates dimensional compatibility

• Imputes missing values via interpolation (using zoo if available)

• Adds minimal noise to zero-variance columns

• Reports diagnostic information

Value

List with components:

X_matrix

Cleaned and prepared X matrix.

Y_matrix

Cleaned and prepared Y matrix.

Estimate Dynamic Factor Model with VAR dynamics

Description

Estimates a Dynamic Factor Model by extracting factors via PLS and modeling their dynamics with a Vector Autoregression. Includes automatic lag selection, robust inference, and optional out-of-sample evaluation.

Usage

estimate_DFM(
  factors_data,
  p = 2,
  compute_oos = TRUE,
  hc_type = "HC3",
  verbose = TRUE
)

Arguments

Details

This function models the joint dynamics of X and Y factors using a VAR. It performs stability checks, tests for serial correlation, computes robust standard errors, and optionally evaluates forecast performance out-of-sample.

Value

List with components:

var_fit

Fitted VAR model on combined factors.

p_used

VAR lag order used.

robust_se

Matrices of robust standard errors.

diagnostics

List of diagnostic tests (stability, serial correlation).

oos_metrics

Out-of-sample forecast evaluation (if requested).

Estimate Factor Ornstein-Uhlenbeck model (Stan if available)

Description

Estimates a multivariate OU with cross-equation coupling Y_{t} depending on lagged X_{t-1} via a \beta matrix. Uses cmdstanr when available, otherwise rstan, with a discrete AR(1) fallback.

Usage

estimate_factor_OU(
  factors_data,
  data_prep = NULL,
  chains = 4,
  iter = 2000,
  seed = 1234,
  adapt1 = 0.98,
  adapt2 = 0.999,
  mtd1 = 12,
  mtd2 = 15,
  verbose = TRUE
)

Arguments

Value

A list with posterior medians (\phi, \mu, \beta), half-lives, coupling strength, pseudo-R2, and the fitted Stan object.

Examples

  # Create toy factor data
  set.seed(123)
  n <- 50
  X_scores <- matrix(rnorm(n * 2), n, 2)
  Y_scores <- matrix(rnorm(n * 2), n, 2)
  factors_data <- list(scores_X = X_scores, scores_Y = Y_scores)
  
  # Estimate OU model (reduce iterations for speed)
  ou_result <- estimate_factor_OU(factors_data, chains = 2, iter = 500, 
                                   verbose = FALSE)
  
  # Check half-lives
  print(ou_result$half_lives_Y)

Extract X innovations from VAR model

Description

Computes structural shocks (innovations) for X factors by fitting a VAR model and extracting residuals.

Usage

make_X_innovations(results, p = NULL)

Arguments

Value

List with components:

shocks

Matrix (T x Kx) of structural shocks with NA padding for initial lags.

var_fit

Fitted VAR model object.

lag_used

Integer lag order used.

Plot error correction panel

Description

Creates a two-panel plot showing (1) error correction terms over time and (2) their estimated half-lives, providing visual assessment of convergence speed.

Usage

plot_error_correction_panel(
  results_robust,
  use_contemporaneous_X = TRUE,
  main_left = expression(u[t] == Y[t] - hat(mu)[Y](X[t])),
  main_right = "Half-life de u[t]"
)

Arguments

Value

Invisibly returns a list with components:

U

Matrix of error correction terms (T x Ky).

half_lives

Vector of estimated half-lives for each Y factor.

Read CPI data from Excel file

Description

Reads Consumer Price Index data from an Excel file with robust error handling and data validation.

Usage

read_cpi(path_cpi)

Arguments

Value

Data frame with CPI data. Returns NULL if file not found or read operation fails.

Rescue short-run channel test

Description

Evaluates the short-run causal relationship from X to Y factors using Granger causality tests and out-of-sample forecast comparison.

Usage

rescue_short_run_channel(
  results_robust,
  lag = 1,
  B = 1000,
  seed = NULL,
  ridge = 0.001,
  oos_start = 0.6,
  verbose = TRUE
)

Arguments

Details

testing. Default is 0.2.

Value

List with components:

granger_p

P-values from Granger causality tests.

OOS

List with out-of-sample RMSE comparison.

p_values

Bootstrap p-values for RMSE differences.

Rotation null hypothesis test for factor coupling

Description

Tests whether the observed correlation structure between X and Y factor spaces is significantly stronger than would be expected under random orthogonal rotations.

Usage

rotation_null_test(
  scores_X,
  scores_Y,
  lag = 1,
  B = 1000,
  seed = 123,
  compute = c("procrustes", "cca", "principal", "dynbeta"),
  progress = TRUE,
  rotate = "Y"
)

Arguments

Details

(contemporaneous). "spearman", or "kendall".

Value

List with components:

observed

Observed correlation statistics.

null_distribution

Matrix of statistics under null rotations.

p_values

One-sided p-values for each statistic.

significant

Logical indicating significance at alpha = 0.05.

Normalize matrix rows to sum to one

Description

Scales each row of a matrix so that its elements sum to 1. Handles zero-sum rows by leaving them unchanged (avoiding division by zero).

Usage

row_norm1(M)

Arguments

Value

Matrix of the same dimensions as M with each row summing to 1 (except rows that originally summed to zero).

Examples

M <- matrix(c(1, 2, 3, 4, 5, 6), nrow = 2)
M_norm <- row_norm1(M)
rowSums(M_norm)  # Should be c(1, 1)

Complete factor-OU convergence analysis pipeline

Description

Executes the end-to-end analysis workflow: data preparation, PLS-based factor extraction, DFM estimation, Factor-OU estimation, convergence tests, and robustness checks. This is the main user-facing function.

Usage

run_complete_factor_analysis_robust(
  X_matrix,
  Y_matrix,
  TMG = NULL,
  COM_matrix = NULL,
  SPVR_matrix = NULL,
  CA = NULL,
  sector_names = NULL,
  max_comp = 3,
  dfm_lags = 1,
  ou_chains = 10,
  ou_iter = 10000,
  skip_ou = FALSE,
  run_convergence_tests = TRUE,
  path_cpi = NULL,
  path_weights = NULL,
  verbose = TRUE
)

Arguments

Details

uses column names of X_matrix. and reweighting tests). Can be NULL.

This function orchestrates the complete analysis:

1. Data validation and diagnostics

2. Bayesian CPI disaggregation (if applicable)

3. PLS-based factor extraction with optimal component selection

4. Dynamic Factor Model estimation via VAR

5. Factor Ornstein-Uhlenbeck mean-reversion model

6. Formal convergence tests (stationarity, cointegration, speed)

7. Robustness tests (permutation, reweighting, jackknife)

Value

List with components:

factors

List containing PLS-extracted factors (scores_X, scores_Y) and related objects (pls_X, pls_Y, ncomp_X, ncomp_Y).

dfm

List with DFM estimation results including VAR fit, lag order, diagnostics, and optional impulse responses.

factor_ou

List with Factor-OU model estimates: beta, lambda, sigma, half_life, method used, and optional Stan fit object.

convergence_tests

List with formal convergence test results (if run_convergence_tests = TRUE).

robustness_tests

List with robustness test results (if run_robustness_tests = TRUE).

diagnostics

List with data diagnostics (multicollinearity, stationarity, structural breaks).

bayesian_cpi

List with Bayesian disaggregation results (if CPI paths provided).

metadata

List with analysis metadata (timestamp, versions, parameters).

See Also

estimate_DFM, estimate_factor_OU, run_convergence_robustness_tests, visualize_factor_dynamics

Examples

# Basic usage with simulated data
set.seed(123)
X <- matrix(rnorm(100 * 10), 100, 10)
Y <- X + matrix(rnorm(100 * 10, 0, 0.5), 100, 10)

results <- run_complete_factor_analysis_robust(
  X_matrix = X,
  Y_matrix = Y,
  max_comp = 3,
  dfm_lags = 1,
  ou_chains = 4,
  ou_iter = 2000,
  verbose = FALSE
)

# View convergence summary
summary(results$convergence_tests)

# Visualize results
visualize_factor_dynamics(
  dfm_result = results$dfm,
  ou_result = results$factor_ou,
  factors_data = results$factors
)

Run comprehensive robustness test suite

Description

Executes all available robustness tests (permutation, reweighting, jackknife) and synthesizes results into an integrated assessment.

Usage

run_convergence_robustness_tests(
  results_robust,
  X_matrix,
  Y_matrix,
  path_cpi = NULL,
  path_weights = NULL,
  sector_names = NULL,
  run_permutation = TRUE,
  run_reweighting = FALSE,
  run_jackknife = TRUE,
  run_leadlag = FALSE,
  run_common_factor = FALSE,
  sensitivity_analysis = TRUE,
  verbose = TRUE
)

Arguments

Value

List with components:

permutation

Results from permutation test.

reweighting

Results from reweighting test (if applicable).

jackknife

Results from jackknife test (if requested).

summary

Data frame summarizing all tests.

overall_robust

Logical indicating if convergence is robust across all tests.

Run rotation null test on complete analysis results

Description

Wrapper function that extracts factors from a complete analysis object and runs the rotation null test.

Usage

run_rotation_null_on_results(
  results_robust,
  lag = 1,
  B = 1000,
  seed = 42,
  rotate = "Y",
  compute = c("procrustes", "cca", "principal", "dynbeta")
)

Arguments

Value

List. Output from rotation_null_test.

Select optimal number of PLS components with cross-validation

Description

Determines the optimal number of Partial Least Squares components using leave-one-out or k-fold cross-validation, with robust error handling.

Usage

select_optimal_components_safe(X, Y, max_comp = 15, verbose = TRUE)

Arguments

Details

default) or "CV" (k-fold cross-validation). validation = "LOO".

The function fits PLS models with 1 to max_comp components, evaluates each via cross-validation, and selects the number that minimizes prediction error while avoiding overfitting.

Value

List with components:

ncomp_optimal

Optimal number of components selected.

RMSEP

Root Mean Squared Error of Prediction for each component.

R2

R-squared values for each component.

validation_method

Validation method used.

pls_fit

Fitted PLS model object with optimal components.

Classical cointegration control (Johansen trace or eigen)

Description

Runs Johansen's cointegration test on the first min(2, ncol(X), ncol(Y)) factors from scores_X and scores_Y, and counts the number of cointegrating relations at the 5% level.

Usage

test_cointegration_control(
  factors_data,
  max_lag = 4,
  type = "trace",
  ecdet = "const",
  verbose = TRUE
)

Arguments

Details

This function requires the optional package urca (declared in Suggests). It does not attempt to install packages at runtime; if urca is unavailable, an informative error is thrown.

The 5% critical values are taken from the "5pct" column of the cval slot returned by urca::ca.jo().

Value

A list with:

• test: the urca::ca.jo fitted object,

• n_coint: integer number of cointegrating relations at 5%,

• vectors: matrix of cointegrating vectors (or NULL if none).

References

Johansen, S. (1991). Estimation and Hypothesis Testing of Cointegration Vectors in Gaussian Vector Autoregressive Models. Econometrica, 59(6), 1551-1580.

Johansen, S. (1995). Likelihood-Based Inference in Cointegrated Vector Autoregressive Models. Oxford University Press.

See Also

ca.jo

Examples

if (requireNamespace("urca", quietly = TRUE)) {
  set.seed(1)
  T <- 120
  X <- cbind(cumsum(rnorm(T)), cumsum(rnorm(T)))
  Y <- cbind(cumsum(rnorm(T)), cumsum(rnorm(T)))
  fd <- list(scores_X = X, scores_Y = Y)
  out <- test_cointegration_control(fd, max_lag = 2, verbose = FALSE)
  str(out)
}

Jackknife robustness test by sector

Description

Assesses the influence of individual sectors by systematically dropping each sector and re-estimating convergence parameters. Identifies influential sectors and checks stability.

Usage

test_jackknife_sectors(
  X_matrix,
  Y_matrix,
  sector_names = NULL,
  k_exclude = 3,
  max_comp = 3,
  verbose = TRUE
)

Arguments

Details

column names.

Value

List with components:

jackknife_estimates

Matrix of lambda estimates (iterations x factors).

original_estimate

Original lambda from full sample.

bias

Estimated jackknife bias.

se

Jackknife standard errors.

influential_sectors

Character vector of highly influential sectors.

dfbetas

Matrix of influence measures (DFBETAS).

Permutation-based robustness test

Description

Tests the robustness of factor-OU convergence findings by randomly permuting the Y factor space and re-estimating the model. Generates empirical null distribution for convergence statistics.

Usage

test_permutation_robustness(
  factors_data,
  data_prep,
  n_perms = 100,
  seed = 123,
  use_stan = TRUE,
  chains = 4,
  iter = 2000,
  verbose = TRUE
)

Arguments

Details

(too slow for many iterations).

Value

List with components:

observed_lambda

Original mean-reversion speeds.

null_distribution

Matrix of permutation-based lambda values.

p_values

One-sided p-values for each factor.

significant

Logical vector indicating significance at alpha = 0.05.

effect_size

Standardized effect sizes (z-scores).

Reweighting-based robustness test

Description

Tests sensitivity of convergence results to alternative sectoral weighting schemes by re-running Bayesian disaggregation and full pipeline with perturbed weights.

Usage

test_reweighting_robustness(
  path_cpi,
  path_weights,
  X_matrix,
  max_comp = 3,
  verbose = TRUE
)

Arguments

Value

List with components:

lambda_distribution

Matrix of lambda estimates across schemes.

original_lambda

Baseline lambda from original weights.

robust_factors

Logical vector indicating robust convergence.

sensitivity_metrics

Summary statistics of sensitivity.

Convert European number format to numeric

Description

Converts character strings in European number format (using comma as decimal separator and period as thousands separator) to numeric values. Already numeric inputs are returned unchanged.

Usage

to_num_commas(x)

Arguments

Value

Numeric value. Returns NA if conversion fails.

Examples

to_num_commas("1.234,56")
to_num_commas("1234.56")
to_num_commas(1234.56)

Visualize factor dynamics comprehensively

Description

Creates a multi-panel visualization summarizing all aspects of the factor model: scores over time, loadings, correlations, OU dynamics, and convergence patterns.

Usage

visualize_factor_dynamics(
  dfm_result,
  ou_result,
  factors_data,
  save_plot = FALSE,
  plot_file = NULL,
  use_device = "default",
  verbose = TRUE
)

Arguments

Details

Default is NULL (display only).

Value

Invisibly returns NULL. Called for side effect of creating plots.

Simple factor dynamics visualization

Description

Creates a simplified multi-panel plot of factor scores over time with minimal dependencies. Useful for quick diagnostics.

Usage

visualize_factor_dynamics_simple(
  factors_data,
  output_file = NULL,
  verbose = TRUE
)

Arguments

Value

Invisibly returns NULL.

Examples

data <- list(scores_X = matrix(rnorm(100), ncol = 2),
             scores_Y = matrix(rnorm(100), ncol = 2))
tmp_file <- file.path(tempdir(), "test_plot.pdf")
visualize_factor_dynamics_simple(data, output_file = tmp_file)