Package {riskutility}

Title:	Disclosure Risk and Data Utility Metrics for Synthetic and Anonymized Data
Version:	0.1.0
Description:	Provides comprehensive methods to measure disclosure risk and data utility for anonymized and synthetic data. Implements attribution-based risk metrics including Correct Attribution Probability (CAP), Targeted CAP (TCAP), Within Equivalence Class Attribution Probability (WEAP), and RAPID (Risk of Attribute Prediction-Induced Disclosure). Also provides distance-based privacy metrics such as Distance to Closest Record (DCR), Nearest Neighbor Distance Ratio (NNDR), and Identical Match Share (IMS). Utility assessment includes propensity score analysis, distribution comparisons, and various statistical tests. Methods are based on Taub et al. (2018) <doi:10.1007/978-3-319-99771-1_9> and related literature. Designed for integration with 'simPop' S4 classes.
Depends:	R (≥ 4.1.0)
Imports:	ggplot2, data.table, MASS, VIM, randomForest, ranger, reshape2, stats, graphics, utils
Suggests:	simPop, synthpop, caretEnsemble, kernelshap, uwot, Rtsne, dbscan, vip, rpart, xgboost, partykit, testthat (≥ 3.0.0), torch, knitr, rmarkdown, plotly, misc3d, sdcMicro, caret, robustbase, gridExtra, Hmisc, robCompositions, clue
License:	GPL-3
URL:	https://github.com/matthias-da/riskutility
BugReports:	https://github.com/matthias-da/riskutility/issues
Encoding:	UTF-8
RoxygenNote:	7.3.3
VignetteBuilder:	knitr
Config/testthat/edition:	3
NeedsCompilation:	no
Packaged:	2026-06-17 11:54:46 UTC; matthias
Author:	Matthias Templ [aut, cre], Oscar Thees [ctb]
Maintainer:	Matthias Templ <matthias.templ@gmail.com>
Repository:	CRAN
Date/Publication:	2026-06-22 15:40:09 UTC

riskutility: Disclosure Risk and Data Utility for Anonymized and Synthetic Data

Description

Provides a comprehensive toolkit for measuring disclosure risk and data utility in anonymized and synthetic datasets. The package supports multivariate Risk-Utility (R-U) evaluation following the framework described in "Beyond the Trade-off Curve" (Thees, Mueller, Templ 2026).

Details

The package covers the following metric families:

Attribution-based risk: CAP-family measures (dcap, tcap, weap, disco) that assess whether adversaries can infer sensitive attributes from quasi-identifiers.

ML-based risk: The rapid metric uses machine learning models to predict sensitive attributes, capturing complex non-linear relationships that CAP methods may miss.

Distance-based risk: Holdout-based metrics (dcr, nndr, ims) detect memorization by comparing distances to training vs. holdout data.

Utility measures: Functions for assessing how well synthetic data preserves the statistical properties of the original, including pMSE, specks, hellinger, energy_distance, ci_proximity, mmd, tstr, copula_fidelity, tail_fidelity, subgroup_utility, regression_fidelity, contingency_fidelity, propscore, gower, and mqs.

Privacy models: Classical privacy model assessments including kanonymity, ldiversity, individual_risk, and suda.

Extended privacy risk: Additional risk models including tcloseness, nnaa, singling_out, linkability, population_uniqueness, epsilon_identifiability, delta_presence, hitting_rate, attacker_risk, drisk, domias, and mia_classifier.

Information-theoretic measures: Entropy and divergence measures (KLDiv, JSDiv, RenyiEntropy, mutualInformation, information_surprisal, max_info_leakage) for quantifying information leakage.

Comparison functions: Direct comparison tools for original and synthetic data including visualization, statistical tests, dimensionality reduction, and predictive model comparisons.

R-U mapping: The rumap function provides comprehensive multivariate Risk-Utility evaluation with Pareto frontier identification.

Visualization Functions

compare_histograms()

Compare distributions using histograms (weighted and unweighted, faceted).

compare_boxplots()

Compare numeric variable distributions using weighted/unweighted boxplots.

Statistical Tests

compare_ks_test()

Kolmogorov-Smirnov test for comparing distributions.

compare_chisq_gof()

Chi-square Goodness-of-Fit test with multi-dimensional frequency tables (weighted/unweighted).

Distributional Comparisons

compare_wasserstein()

Calculate Wasserstein distances between numeric or nominal variables.

compare_multivariate_distribution()

Compare joint distributions using Mahalanobis distance or mutual information.

Summary Statistics

compare_means_frequencies()

Compare means, medians, robust statistics, skewness, kurtosis for numeric data, and frequencies for categorical data.

compare_correlation_matrices()

Compare correlations: Pearson, Spearman, robust correlations for numeric, categorical, and mixed data types.

compare_multivariate_summary_statistics()

Calculate joint summary statistics for numeric and categorical data.

Dimensionality Reduction

compare_pca()

Principal Component Analysis comparison with biplot options.

compare_embedding()

t-SNE, UMAP, or Sammon's Mapping (MDS) visualizations for embedding comparisons.

Machine Learning and Predictive Modeling

compare_model_performance()

Evaluate predictive model performance (accuracy, precision, recall, F1-score, AUC, R-squared, RMSE) using cross-validation.

compare_feature_importance()

Evaluate stability of feature importance measures from Random Forests, Decision Trees, Gradient Boosting, Permutation Importance, and SHAP values.

Dependencies

The package builds on ggplot2 and data.table; uses VIM, MASS, randomForest, and ranger for core computations; and optionally integrates with simPop, synthpop, Rtsne, uwot, caret, and robustbase when installed.

Missing-value handling

Two parameter conventions are used deliberately, according to how much choice a function offers. A logical na.rm (the standard R idiom) appears where the only decision is whether to drop incomplete cases before computing a statistic. A character selector appears where several treatments are supported: na ("remove", "impute", "stop", and in pMSE additionally "indicator") in propscore, pMSE, and mqs; and na_strategy ("constant", "drop", "median") in rapid, which acts specifically on the sensitive target attribute.

Author(s)

Matthias Templ

See Also

ggplot, data.table

Examples

# Create example datasets
set.seed(123)
X <- data.frame(
  income = rnorm(100, mean = 50000, sd = 10000),
  age = rnorm(100, mean = 40, sd = 10),
  gender = sample(c("Male", "Female"), 100, replace = TRUE),
  weight = runif(100, 0.5, 1.5)
)
Y <- data.frame(
  income = rnorm(100, mean = 48000, sd = 12000),
  age = rnorm(100, mean = 42, sd = 11),
  gender = sample(c("Male", "Female"), 100, replace = TRUE),
  weight = runif(100, 0.5, 1.5)
)

# Example for histogram comparison
compare_histograms(X, Y, num_var = "income", cat_vars = c("gender"),
                   weight_X = "weight", weight_Y = "weight")

# Example for PCA comparison
X_pca <- data.frame(
  income = rnorm(100, mean = 50000, sd = 10000),
  age = rnorm(100, mean = 40, sd = 10),
  expenditure = rnorm(100, mean = 2000, sd = 500)
)
Y_pca <- data.frame(
  income = rnorm(100, mean = 48000, sd = 12000),
  age = rnorm(100, mean = 42, sd = 11),
  expenditure = rnorm(100, mean = 2200, sd = 600)
)
res_pca <- compare_pca(X_pca, Y_pca,
                       vars = c("income", "age", "expenditure"),
                       biplot = TRUE)

Compute normalized Euclidean distances in latent space

Description

Computes pairwise Euclidean distances between query and search embeddings, normalized to [0, 1] using the 97.5th percentile of within-query distances (with fallback to max for small datasets, n < 40).

Usage

.ae_distance(emb_query, emb_search, threshold = NULL)

Arguments

Value

list with:

dist_mat

numeric matrix (n_q x n_s) of normalized distances

threshold

numeric, normalization threshold used

Encode data through trained autoencoder

Description

Passes data through the encoder portion of the trained autoencoder, returning an n x latent_dim embedding matrix.

Usage

.ae_encode(model, data, prep, key, type = NULL)

Arguments

Value

numeric matrix (n x latent_dim)

Define autoencoder nn_module for mixed data

Description

Creates a torch nn_module with entity embeddings for categorical variables and a 2-layer encoder (input -> hidden -> latent) / decoder (latent -> hidden -> output).

Usage

.ae_model(prep, latent_dim)

Arguments

Value

instantiated torch nn_module

Preprocess data for autoencoder embedding

Description

Min-max scales numeric columns to [0, 1], builds 1-based category-to-index mappings for categorical columns, and computes metadata for model construction.

Usage

.ae_preprocess(data, key, type, cat_maps = NULL, num_ranges = NULL)

Arguments

Value

list with:

num_mat

numeric matrix of scaled numerics (n x p_num), or NULL

cat_idx

list of integer vectors, one per categorical variable, each of length n with 1-based indices (unseen levels get n_levels + 1)

cat_maps

named list: variable -> named integer vector mapping level -> index (1-based)

num_ranges

named list: variable -> c(min, max)

num_keys

character vector of numeric key names

cat_keys

character vector of categorical key names

n_levels

named integer vector: n_levels per categorical var (training levels only, excluding UNK)

emb_dims

named integer vector: embedding dimension per categorical var

input_dim

integer, total input dimension after entity embeddings

Train autoencoder on data

Description

Manual training loop with Adam optimizer, composite loss (MSE for numerics + cross-entropy for categoricals), and early stopping on a 20% validation holdout.

Usage

.ae_train(
  data,
  key,
  type = NULL,
  latent_dim = NULL,
  epochs = 50L,
  batch_size = 32L,
  lr = 0.001,
  patience = 5L,
  val_fraction = 0.2
)

Arguments

Value

list with:

model

trained torch nn_module

prep

preprocessing metadata from .ae_preprocess()

latent_dim

integer, latent dimension used

Permutation-based variable importance in latent space

Description

For each variable, shuffles its values in the data, re-encodes, and measures the mean Euclidean shift in the latent space compared to the original embeddings. Normalized to sum to 1.

Usage

.ae_var_importance(model, data, prep, key, type = NULL, emb_original = NULL)

Arguments

Value

named numeric vector of importance scores summing to 1 (all-zero if n <= 1 or no variable changes the embedding)

Build a predicate from a synthetic record

Description

Build a predicate from a synthetic record

Usage

.build_predicate(record, cols, medians, col_types)

Arguments

Value

list of conditions, each with elements: col, op, value

Compute Shannon entropy for a variable

Description

For numeric variables, values are discretized into bins before computing entropy. For categorical/factor variables, entropy is computed directly from category frequencies.

Usage

.compute_entropy(x, n_bins = 20L)

Arguments

Value

numeric scalar, the Shannon entropy (natural log)

Compute hitting rates at several standard thresholds

Description

Compute hitting rates at several standard thresholds

Usage

.compute_rates_at_thresholds(min_distances)

Arguments

Value

named numeric vector of rates at standard thresholds

Shared holdout preparation for distance-risk functions

Description

Validates inputs, intersects variables, handles NAs, and splits holdout. Used by dcr(), nndr(), and rf_privacy().

Usage

.distance_risk_prepare(
  X,
  Y,
  holdout = NULL,
  holdout_fraction = 0.5,
  vars = NULL,
  na.rm = TRUE,
  seed = NULL,
  min_holdout = 1L
)

Arguments

Value

list with train, synthetic, holdout, vars, was_split

Orchestrate embedding-based linkage for one block

Description

Preprocesses, trains autoencoder, encodes query and search data, computes normalized distances, and variable importance.

Usage

.embedding_linkage_block(
  query,
  search,
  key,
  type = NULL,
  latent_dim = NULL,
  epochs = 50L
)

Arguments

Value

list with:

dist_mat

numeric matrix (n_q x n_s) of normalized distances

var_importance

named numeric vector

threshold

numeric, distance normalization threshold

latent_dim

integer, latent dimension used

Euclidean distance matrix

Description

Computes pairwise Euclidean distances between rows of A and B.

Usage

.euclidean_dist(A, B)

Arguments

Value

nrow(A) x nrow(B) distance matrix

Evaluate a predicate against a dataset

Description

Evaluate a predicate against a dataset

Usage

.evaluate_predicate(predicate, data)

Arguments

Value

integer count of matching rows

Run linkability attack: find NNs via aux and secret columns, check for links

Description

Run linkability attack: find NNs via aux and secret columns, check for links

Usage

.linkability_attack(Y_attack, ref_data, aux_cols, secret_cols, n_neighbors)

Arguments

Value

logical vector of length nrow(Y_attack), TRUE if link succeeded

Compute Mahalanobis-Based Distance from One Record to Candidates

Description

For numeric/ordinal variables, computes Mahalanobis distance using the pre-computed inverse covariance from .mahal_prepare(), normalized by the chi-squared threshold to produce values in approximately [0, 1]. For nominal variables, computes Gower-style exact match (0/1). Combined as weighted average based on proportion of each type.

Usage

.mahal_dist(x_row, candidates, prep, type, na_anon = "ignore")

Arguments

Value

numeric vector of distances (one per candidate row).

Prepare Mahalanobis Distance Components

Description

Estimates a (robust) covariance matrix from the data's numeric/ordinal variables and pre-computes its inverse. Also identifies the split between numeric and nominal key variables for the combined distance.

Usage

.mahal_prepare(data, key, type, robust = TRUE, alpha = 0.025)

Arguments

Value

list with cov_inv, center (diagnostic only; not used in distance computation), numeric_keys, nominal_keys, alpha (proportion numeric), robust, chi_sq_threshold, p.

Train a probability random forest and return P(class=1) on test

Description

Internal helper for mia_classifier.

Usage

.mia_backend_rf(X_train, y_train, X_test, num_threads = 1L, num_trees = 300L)

Arguments

Value

Numeric vector of length nrow(X_test) with P(class = 1).

Joint min-max normalization for multiple data frames

Description

Combines multiple numeric data frames, applies per-column min-max normalization over the union, and returns each subset as a numeric matrix. Used by euclidean-distance branches in nnaa(), hitting_rate(), etc.

Usage

.normalize_and_split(...)

Arguments

Value

A list of numeric matrices in the same order as the inputs, each min-max normalized using the joint column ranges.

Min-max normalization

Description

Min-max normalization

Usage

.normalize_minmax(x)

Arguments

Value

normalized vector in [0, 1]

Perturbation-aware expected PRAM risk for a single record

Description

Holding all other released records at their realized values, integrate record i's re-identification risk over its own PRAM transition distribution P(x_i \to \cdot). With S the realized transition mass on the non-true candidates, the risk of a hypothetical output o is P(x_i \to o) / (S + P(x_i \to o)), so the expectation is \sum_o P(x_i \to o)^2 / (S + P(x_i \to o)) (exact) or its Monte-Carlo estimate E_{o \sim P}[P(o)/(S+P(o))] when the support is large. Assumes per-variable independent PRAM (forward direction, truth="row").

Usage

.pram_expected_risk(
  x_row,
  key,
  pram_matrix,
  S,
  max_support = 1000L,
  n_mc = 500L
)

Arguments

Value

numeric scalar in [0,1] (or NA if a value is absent from the matrix).

Compute proximity submatrix from terminal nodes

Description

Uses a tree-by-tree accumulator: for each tree, groups records by terminal node ID, then increments the proximity counter only for pairs that share a node. This is efficient when trees have many terminal nodes (each with few records), avoiding the full O(n2 * n1 * n_trees) scan.

Usage

.proximity_from_nodes(terminal_nodes, idx1, idx2, progress = FALSE)

Arguments

Value

numeric matrix of dim length(idx2) x length(idx1)

Compute proximity between new data and reference records

Description

Pushes newdata through a trained forest to get terminal nodes, then computes proximity to reference records.

Usage

.proximity_from_nodes_newdata(
  forest,
  newdata,
  terminal_nodes_ref,
  idx_ref,
  progress = FALSE
)

Arguments

Value

matrix of dim nrow(newdata) x length(idx_ref)

Pitman model for population uniqueness

Description

Pitman model for population uniqueness

Usage

.pu_pitman(record_freq, freq_vec, n, pi_frac)

Arguments

Value

list with risk_per_record, global_risk, n_population_uniques_est, alpha

SNB (Sample-Negative-Binomial) model for population uniqueness

Description

SNB (Sample-Negative-Binomial) model for population uniqueness

Usage

.pu_snb(record_freq, freq_vec, n, pi_frac)

Arguments

Value

list with risk_per_record, global_risk, n_population_uniques_est, r, q

Zayatz model for population uniqueness

Description

Zayatz model for population uniqueness

Usage

.pu_zayatz(record_freq, n, pi_frac)

Arguments

Value

list with risk_per_record, global_risk, n_population_uniques_est

Compute residual risk from attack and control Wilson scores

Description

Shared helper for singling_out and linkability. Returns list with risk, risk_ci, risk_attack, risk_attack_ci, risk_control, risk_control_ci, privacy_pass.

Usage

.residual_risk(
  n_success,
  n_control_success,
  n_attacks,
  confidence_level = 0.95,
  privacy_threshold = 0.1
)

Arguments

Value

list with risk components

RF linkage matching for one block

Description

Trains RF on query + candidate records, computes cross-proximity, returns per-record matches.

Usage

.rf_linkage_block(query, candidate, key, n_trees = 500L, ...)

Arguments

Value

list with risk, match_idx, cross_prox, var_importance

RF Proximity Internal Engine

Description

Trains a supervised RF on combined data and extracts terminal node matrix.

Usage

.rf_proximity(
  data1,
  data2,
  vars = NULL,
  n_trees = 500L,
  mtry = NULL,
  importance = TRUE,
  seed = NULL,
  ...
)

Arguments

Value

list with forest, terminal_nodes, n1, n2, importance, oob_error

Sinkhorn Optimal Transport Solver

Description

Computes the entropy-regularized optimal transport plan between two discrete distributions using the Sinkhorn-Knopp algorithm (Cuturi, 2013).

Usage

.sinkhorn(C, epsilon = NULL, max_iter = 100L, tol = 1e-08)

Arguments

Details

Given a cost matrix C (n_orig x n_anon) and regularization \varepsilon > 0, the transport plan P minimizes

\sum_{ij} P_{ij} C_{ij} + \varepsilon \sum_{ij} P_{ij} \log P_{ij}

subject to marginal constraints (rows sum to 1/n_{orig}, columns sum to 1/n_{anon}). With \varepsilon \to 0, the solution approaches the Hungarian (LSAP) assignment.

Supports rectangular matrices (unequal dataset sizes). Marginals default to uniform: r_i = 1/n_{orig}, c_j = 1/n_{anon}.

Value

numeric matrix. Transport plan P of same dimensions as C. Entry P_{ij} is the transport weight (fraction of record i's mass shipped to record j; higher weight = stronger association). Row sums = 1/n_orig, column sums = 1/n_anon.

See Also

.solve_ot, recordLinkage

Solve bijective (one-to-one) assignment via the Hungarian algorithm

Description

Builds a cost matrix per blocking group and solves the LSAP so that each query record is assigned to at most one search record. Risk is binary: 1 if assigned to the true match, 0 otherwise.

Usage

.solve_bijective(score_cache, true_idx, n_query, split_search, blk_query)

Arguments

Value

list with assigned_search (integer) and risk (numeric).

Solve Optimal Transport Matching from Score Cache

Description

Post-loop handler parallel to .solve_bijective(). Builds cost matrices from score_cache, runs Sinkhorn per block, and extracts continuous risk scores from the transport plan.

Usage

.solve_ot(
  score_cache,
  true_idx,
  n_query,
  split_search,
  blk_query,
  epsilon = NULL,
  max_iter = 100L
)

Arguments

Details

Risk for record i is defined as:

risk_i = P_{i, \text{true}} \cdot n_{\text{query}}

where P_{i, \text{true}} is the transport weight from query record i to its true match in the search data. Under uniform random matching, this gives risk = 1/n_search (same baseline as independent matching). When n_{query} > n_{search}, the raw product can exceed 1 and is capped.

Value

list with risk (numeric), d_rank (integer), transport_plans (list of matrices per block).

See Also

.sinkhorn, .solve_bijective

Risk of correctly identifying the true match within a candidate set

Description

Shared back-end for the per-record re-identification risk: given the per- candidate scores for one query record, build the attacker's guess set and return the (weighted) probability mass on the true match. This is the exact logic the deterministic / probabilistic / pram / predictive / rbrl / mahalanobis methods apply inline; the rf and embedding methods route their scores through it so that all eight methods share one risk definition.

Usage

.true_match_risk(
  scores,
  cand,
  true_pos,
  maximize,
  strategy,
  risk_weighting,
  k = NULL,
  threshold = NULL,
  kappa = NULL,
  bandwidth = NULL,
  kernel = "gaussian"
)

Arguments

Value

list with risk, cand_n, true_in_set, guess.

Shared input validation for two-dataset (X vs Y) risk measures

Description

Validates data frames, intersects variables, checks types, removes NAs, and optionally splits holdout. Used by nnaa(), singling_out(), linkability(), domias(), hitting_rate(), and epsilon_identifiability().

Usage

.validate_pair_inputs(
  X,
  Y,
  holdout = NULL,
  holdout_fraction = NULL,
  vars = NULL,
  na.rm = TRUE,
  seed = NULL,
  check_types = TRUE,
  min_vars = 1L
)

Arguments

Value

list with X, Y, vars (character), and optionally train, holdout (if holdout splitting was performed)

Wilson score confidence interval

Description

Wilson score confidence interval

Usage

.wilson_score(n_success, n_total, confidence_level = 0.95)

Arguments

Value

list with estimate, ci_lower, ci_upper

Entropy measures

Description

Kullback-Leibler and Jensen-Shannon divergence

Usage

KLDiv(A, B)

KLDiv_bayes(A, B)

JSDiv(A, B)

JSDiv_bayes(A, B)

CrossEntropy(A, B)

Arguments

Details

The Kullback-Leibler divergence is defined as:

KLD(X,Y) = \sum X \times \log\left(\frac{X}{Y}\right)

If the data are multivariate, a Kullback-Leibler inspired divergence is calculated through

KLD(X,Y) = \sum_{i,j}{X_{i,j} \left( \log\left(\frac{X_{i,j}}{Y_{i,j}}\right) + \log(Y) - \log(X) \right)}

with

X = \sum{X_{i,j}}

and

Y = \sum{Y_{i,j}}

The compositional versions of the Kullback-Leibler divergence is given in the reference below.

Value

The divergence measure

Author(s)

Matthias Templ

References

J.A. Martin-Fernandez, M. Bren, C. Barcelo-Vidal, V. Pawlowski-Glahn (1999). A measure of difference for compositional data based on measures of divergence. In S. Lippard, A. Nass, and R. Sinding-Larsen (Eds.), Proceedings of IAMG'99, Volume 1, Trondheim (Norway), pp. 211-215.

See Also

Other information-theory: EntropyMeasures, information_surprisal(), max_info_leakage(), mutualInformation(), positive_information_disclosure(), privacy_score(), systemAnonymityLevel()

Examples

# Kullback-Leibler divergence between two probability vectors
p <- c(0.4, 0.3, 0.2, 0.1)
q <- c(0.25, 0.25, 0.25, 0.25)
KLDiv(p, q)

# Bayesian KL divergence
KLDiv_bayes(p, q)

# Jensen-Shannon divergence (symmetric)
JSDiv(p, q)

# Bayesian Jensen-Shannon divergence
JSDiv_bayes(p, q)

# Cross-entropy
CrossEntropy(p, q)

Additional Entropy-Based Privacy Measures

Description

This set of functions implements various entropy-based measures used in privacy and information theory. All functions assume that the input vectors p or matrices are valid probability distributions.

Usage

RenyiEntropy(p, alpha = 2)

MaxEntropy(p)

MinEntropy(p)

NormalizedEntropy(p)

ConditionalEntropy(joint)

CumulativeEntropy(x)

Arguments

Details

RenyiEntropy(p, alpha = 2)

Computes Rényi entropy of order alpha for a probability vector p.

MaxEntropy(p)

Returns the Hartley (max) entropy, defined as the logarithm of the number of non-zero categories.

MinEntropy(p)

Computes the min-entropy, defined as the negative logarithm of the maximum probability.

NormalizedEntropy(p)

Returns the Shannon entropy normalized by the maximum entropy, resulting in a value between 0 and 1.

ConditionalEntropy(joint)

Computes the conditional entropy H(Y|X), given a joint probability matrix (rows = X, columns = Y).

CumulativeEntropy(x)

Estimates cumulative entropy based on the empirical cumulative distribution function of a numeric vector x.

Value

A numeric value representing the corresponding entropy or risk measure.

Author(s)

Matthias Templ

References

Rényi, A. (1961). On measures of entropy and information. In Proceedings of the 4th Berkeley Symposium on Mathematical Statistics and Probability (Vol. 1, pp. 547–561).

Shannon, C.E. (1948). A Mathematical Theory of Communication. Bell System Technical Journal, 27(3), 379–423.

Bender, S., Haas, A., & Klose, C. (2001). The IAB Employment Sample. Journal of Applied Social Science Studies, 121(2), 183–190.

Ichim, D. (2009). Local neighbourhood-based record linkage and its application to the Canadian census. In Privacy in Statistical Databases (pp. 207–221). Springer.

See Also

Other information-theory: Entropy, information_surprisal(), max_info_leakage(), mutualInformation(), positive_information_disclosure(), privacy_score(), systemAnonymityLevel()

Examples

# Renyi entropy of order 2
p <- c(0.4, 0.3, 0.2, 0.1)
RenyiEntropy(p, alpha = 2)

# Maximum (Hartley) entropy
MaxEntropy(p)

# Min-entropy
MinEntropy(p)

# Normalized Shannon entropy (0 to 1)
NormalizedEntropy(p)

# Conditional entropy H(Y|X) from a joint probability matrix
joint <- matrix(c(0.1, 0.2, 0.3, 0.4), nrow = 2)
ConditionalEntropy(joint)

# Cumulative entropy of a numeric vector
x <- c(1.2, 3.4, 2.1, 5.0, 4.3)
CumulativeEntropy(x)

Attacker Risk Models (Prosecutor/Journalist/Marketer)

Description

Computes re-identification risk under three canonical attacker models from Statistical Disclosure Control (SDC). Each model makes different assumptions about the attacker's background knowledge, leading to different risk quantifications based on quasi-identifier (QI) frequencies.

Usage

attacker_risk(X, ...)

## S3 method for class 'synth_pair'
attacker_risk(X, data = c("synthetic", "original"), ...)

## Default S3 method:
attacker_risk(
  X,
  key_vars,
  sampling_fraction = 0.01,
  model = c("all", "prosecutor", "journalist", "marketer"),
  na.rm = TRUE,
  ...
)

Arguments

Details

The three attacker models differ in their assumptions:

Prosecutor Model: The attacker knows a specific individual is in the dataset and attempts to re-identify them. For each record in equivalence class k with frequency f_k, the individual risk is:

risk_i = 1 / f_k

The global prosecutor risk is the mean over all records.

Journalist Model: The attacker does not know whether the target is in the dataset. This requires estimating the population frequency F_k from the sample frequency f_k using the sampling fraction:

F_k \approx f_k / sampling\_fraction

risk_i = 1 / F_k = sampling\_fraction / f_k

This typically yields lower risks than the prosecutor model because population frequencies are larger than sample frequencies.

Marketer Model: The attacker does not target any specific individual but wants to re-identify as many records as possible. The global risk is the expected number of successful re-identifications divided by the total number of records:

risk = (1/n) \sum_{i=1}^{n} 1/f_k(i)

Note that for the marketer model, the global risk equals the prosecutor global risk (both are mean 1/f_k).

Interpretation:

• Risk close to 1: High re-identification risk (many unique records)

• Risk > 0.1: Elevated risk, privacy protection may be insufficient

• Risk < 0.05: Generally acceptable

• Risk close to 0: Low re-identification risk (large equivalence classes)

Value

An object of class "attacker_risk" containing:

• risk_per_record: data.frame with columns: prosecutor, journalist (if applicable), freq (f_k)

• global_risk: named list with prosecutor (mean 1/f_k), journalist (mean 1/F_k), marketer (1/n * sum(1/f_k))

• n_uniques: count of records with f_k = 1

• pct_uniques: fraction of unique records

• freq_table: table of QI combination frequencies

• model: model(s) used

• key_vars: quasi-identifier variables used

• sampling_fraction: sampling fraction used

• n_records: number of records assessed

• privacy_pass: logical, prosecutor global risk <= 0.1

Author(s)

Matthias Templ

References

Hundepool, A., Domingo-Ferrer, J., Franconi, L., Giessing, S., Schulte Nordholt, E., Spicer, K. & de Wolf, P.-P. (2012). Statistical Disclosure Control. John Wiley & Sons. doi:10.1002/9781118348239

Elliot, M. (2006). A Computational Algorithm for Handling the Special Uniques Problem. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 14(Supp. 1), 45-59.

Templ, M. (2017). Statistical Disclosure Control for Microdata: Methods and Applications in R. Springer. doi:10.1007/978-3-319-50272-4

See Also

kanonymity for k-anonymity assessment, individual_risk for model-based individual risk, suda for special uniques detection

Other privacy-models: delta_presence(), disclosure_report(), domias(), drisk(), epsilon_identifiability(), individual_risk(), inspect_record(), kanonymity(), ldiversity(), linkability(), merge_per_record(), mia_classifier(), population_uniqueness(), recordLinkage(), risk_by_group(), singling_out(), suda(), tcloseness(), top_at_risk()

Examples

# Create example data
set.seed(123)
data <- data.frame(
  age = sample(c("young", "middle", "old"), 200, replace = TRUE),
  gender = sample(c("M", "F"), 200, replace = TRUE),
  region = sample(c("N", "S", "E", "W"), 200, replace = TRUE),
  income = sample(c("low", "medium", "high"), 200, replace = TRUE)
)

# Compute all three attacker models
result <- attacker_risk(data, key_vars = c("age", "gender", "region"))
print(result)
summary(result)


plot(result)

# Prosecutor model only
result_p <- attacker_risk(data, key_vars = c("age", "gender", "region"),
                          model = "prosecutor")
print(result_p)

# Journalist model with different sampling fraction
result_j <- attacker_risk(data, key_vars = c("age", "gender"),
                          model = "journalist", sampling_fraction = 0.05)
print(result_j)

Chi-Square Utility Measures

Description

Computes chi-square based utility measures comparing original and synthetic data frequency tables. Implements VW (Voas-Williamson), FT (Freeman-Tukey), G (log-likelihood), and JSD (Jensen-Shannon Divergence) statistics.

Usage

chisq_utility(X, ...)

## S3 method for class 'synth_pair'
chisq_utility(X, vars = NULL, max_cells = 10000, ...)

## Default S3 method:
chisq_utility(
  X,
  Y,
  vars,
  max_cells = 10000,
  weight_X = NULL,
  weight_Y = NULL,
  ...
)

Arguments

Details

These measures compare frequency tables from original and synthetic data.

Chi-Square (\chi^2): The standard Pearson chi-square statistic:

\chi^2 = \sum \frac{(O - E)^2}{E}

where O is observed (synthetic) and E is expected (original proportions scaled).

Voas-Williamson (VW): A normalized chi-square measure (Voas & Williamson, 2001):

VW = \frac{\chi^2 - df}{N}

where df is degrees of freedom and N is sample size. Lower is better; 0 indicates perfect replication.

Freeman-Tukey (FT): Uses a variance-stabilizing transformation:

FT = 4 \sum (\sqrt{O} - \sqrt{E})^2

G-Test (Log-Likelihood Ratio):

G = 2 \sum O \log(O/E)

Jensen-Shannon Divergence (JSD): A symmetric, bounded divergence measure:

JSD = \frac{1}{2}[KL(P||M) + KL(Q||M)]

where M = (P + Q)/2, and KL is Kullback-Leibler divergence.

Interpretation:

• VW close to 0: excellent utility

• VW < 0.01: good utility

• VW > 0.05: potential utility concerns

• JSD ranges 0-1: 0 = identical, 1 = completely different

Value

An object of class "chisq_utility" containing:

• chi2: standard chi-square statistic

• df: degrees of freedom

• p_value: p-value for chi-square test

• VW: Voas-Williamson statistic (normalized chi-square)

• FT: Freeman-Tukey statistic

• G: G-test (log-likelihood ratio) statistic

• JSD: Jensen-Shannon Divergence

• n_cells: number of cells in the table

• n_empty_orig: empty cells in original

• n_empty_synth: empty cells in synthetic

• pct_utility: overall utility percentage (based on VW)

• table_orig: frequency table for original data

• table_synth: frequency table for synthetic data

Author(s)

Matthias Templ

References

Voas, D., & Williamson, P. (2001). Evaluating goodness-of-fit measures for synthetic microdata. Geographical and Environmental Modelling, 5(2), 177-200.

Freeman, M. F., & Tukey, J. W. (1950). Transformations related to the angular and the square root. Annals of Mathematical Statistics, 21, 607-611.

See Also

propscore for propensity score utility, specks for SPECKS utility measure

Other utility: ci_proximity(), contingency_fidelity(), copula_fidelity(), energy_distance(), gower(), hellinger(), mmd(), mqs(), pMSE(), plot.rumap(), propscore(), regression_fidelity(), rumap(), specks(), subgroup_utility(), tail_fidelity(), tstr()

Examples

# Create example data
set.seed(123)
orig <- data.frame(
  age = sample(c("young", "middle", "old"), 500, replace = TRUE),
  gender = sample(c("M", "F"), 500, replace = TRUE),
  region = sample(c("N", "S", "E", "W"), 500, replace = TRUE)
)

# Synthetic data with some differences
synth <- data.frame(
  age = sample(c("young", "middle", "old"), 500, replace = TRUE,
               prob = c(0.35, 0.4, 0.25)),
  gender = sample(c("M", "F"), 500, replace = TRUE),
  region = sample(c("N", "S", "E", "W"), 500, replace = TRUE)
)

# Compute chi-square utility
result <- chisq_utility(orig, synth, vars = c("age", "gender"))
print(result)
summary(result)
plot(result)

Comparison of confidence intervals

Description

Given a sample, how much higher is the change compared to the sampling error.

Usage

ci_overlap(x, y, estimator = median, R = 1000)

Arguments

Value

A list with the following elements:

m_x

The point estimate of the function provided in function argument estimator for the vector x.

m_y

The point estimate of the function provided in function argument estimator for the vector x.

ci_x

The confidence interval of x.

ci_y

The confidence interval of y.

overlap

Overlap in confidence intervals.

overlap_perc

Overlap in confidence intervals in percentage

ratioSE

How much differ is the point estimate of y compared to the half of the confidence interval.

Author(s)

Matthias Templ

See Also

Other comparison: compare_boxplots(), compare_chisq_gof(), compare_correlation_matrices(), compare_distributions_cont(), compare_embedding(), compare_feature_importance(), compare_histograms(), compare_ks_test(), compare_means_frequencies(), compare_missing_values(), compare_model_performance(), compare_multivariate_distribution(), compare_multivariate_summary_statistics(), compare_outliers(), compare_pca(), compare_wasserstein(), densitydiff_1d_num(), densitydiff_kl_num(), densitydiff_pca()

Examples

x <- rnorm(100, 10)
y <- x + runif(100, 0, 1)

ci_overlap(x, y, R = 1000)

x <- factor(sample(1:2, size = 100, replace = TRUE))
y <- factor(sample(1:2, size = 100, replace = TRUE))

my_func <- function(x){
  table(x)[1]
}

ci_overlap(x, y, estimator = my_func)

Confidence Interval Proximity

Description

Computes the proximity of confidence intervals for numeric variables between original and synthetic datasets. This utility measure assesses how well the synthetic data preserves point estimates and their uncertainty.

Usage

ci_proximity(X, ...)

## S3 method for class 'synth_pair'
ci_proximity(X, ...)

## Default S3 method:
ci_proximity(
  X,
  Y,
  vars = NULL,
  conf.level = 0.95,
  weight_X = NULL,
  weight_Y = NULL,
  na.rm = TRUE,
  ...
)

Arguments

Details

For each numeric variable, this function computes:

• CI Overlap: Proportion of the CIs that overlap, ranging from 0 (no overlap) to 1 (complete overlap or one contains the other)

• Relative Error: |mean_Y - mean_X| / |mean_X|, measuring how close the synthetic mean is to the original mean

• Proximity Score: Combined measure accounting for both mean accuracy and CI overlap

The CI overlap coefficient is computed as:

Overlap = \frac{\max(0, \min(U_X, U_Y) - \max(L_X, L_Y))}{\max(U_X, U_Y) - \min(L_X, L_Y)}

where L and U are lower and upper CI bounds.

For utility assessment, higher proximity scores indicate better preservation of statistical properties in synthetic data.

Value

An object of class "ci_proximity" containing:

• per_variable: data.frame with CI comparison for each variable

• proximity_mean: mean proximity score across all variables

• overlap_mean: mean overlap coefficient across all variables

• relative_error_mean: mean relative error of synthetic means

• n_vars: number of variables compared

• vars: variable names used

• conf.level: confidence level used

Author(s)

Matthias Templ

See Also

ci_overlap for basic CI overlap calculation, compare_ks_test for distribution comparison

Other utility: chisq_utility(), contingency_fidelity(), copula_fidelity(), energy_distance(), gower(), hellinger(), mmd(), mqs(), pMSE(), plot.rumap(), propscore(), regression_fidelity(), rumap(), specks(), subgroup_utility(), tail_fidelity(), tstr()

Examples

set.seed(123)
# Original data
X <- data.frame(
  income = rnorm(500, mean = 50000, sd = 10000),
  age = rnorm(500, mean = 40, sd = 10),
  score = rnorm(500, mean = 100, sd = 15)
)

# Good synthetic data (similar means)
Y_good <- data.frame(
  income = rnorm(500, mean = 50000, sd = 10000),
  age = rnorm(500, mean = 40, sd = 10),
  score = rnorm(500, mean = 100, sd = 15)
)

# Poor synthetic data (shifted means)
Y_poor <- data.frame(
  income = rnorm(500, mean = 55000, sd = 10000),
  age = rnorm(500, mean = 45, sd = 10),
  score = rnorm(500, mean = 90, sd = 15)
)

result_good <- ci_proximity(X, Y_good)
print(result_good)
summary(result_good)

result_poor <- ci_proximity(X, Y_poor)
print(result_poor)

Compare Boxplots of a Numeric Variable in Two Datasets

Description

This function compares the distribution of a numeric variable between two datasets (e.g., an original dataset and an anonymized/synthetic version) using boxplots. It allows conditional faceting on categorical variables and supports different visualization styles, including side-by-side, overlapping, and separate boxplots. Sampling weights can be considered to compute weighted medians and quartiles.

Usage

compare_boxplots(X, ...)

## S3 method for class 'synth_pair'
compare_boxplots(X, ...)

## Default S3 method:
compare_boxplots(
  X,
  Y,
  num_var,
  cat_vars,
  weight_X = NULL,
  weight_Y = NULL,
  facet_type = "wrap",
  plot_type = "side",
  ...
)

Arguments

Value

A ggplot2 boxplot visualization comparing the distributions.

Author(s)

Matthias Templ

See Also

Other comparison: ci_overlap(), compare_chisq_gof(), compare_correlation_matrices(), compare_distributions_cont(), compare_embedding(), compare_feature_importance(), compare_histograms(), compare_ks_test(), compare_means_frequencies(), compare_missing_values(), compare_model_performance(), compare_multivariate_distribution(), compare_multivariate_summary_statistics(), compare_outliers(), compare_pca(), compare_wasserstein(), densitydiff_1d_num(), densitydiff_kl_num(), densitydiff_pca()

Examples

set.seed(123)
X <- data.frame(
  age = sample(20:80, 500, replace = TRUE),
  gender = sample(c("Male", "Female"), 500, replace = TRUE),
  income = rnorm(500, mean = 50000, sd = 10000),
  weight = runif(500, 0.5, 1.5)
)

Y <- data.frame(
  age = sample(20:80, 1000, replace = TRUE),
  gender = sample(c("Male", "Female"), 1000, replace = TRUE),
  income = rnorm(1000, mean = 48000, sd = 12000),
  weight = runif(1000, 0.5, 1.5)
)

compare_boxplots(X, Y, num_var = "income", cat_vars = c("gender"),
                 weight_X = "weight", weight_Y = "weight",
                 facet_type = "wrap", plot_type = "side")

compare_boxplots(X, Y, num_var = "income", cat_vars = c("gender"),
                 weight_X = "weight", weight_Y = "weight",
                 facet_type = "wrap", plot_type = "separate")

Compare Frequencies using Chi-square Goodness-of-Fit Test with Optional Grouping, Weights, and Simulated p-values

Description

This function performs a Chi-square goodness-of-fit test to compare the observed frequencies in an anonymized/synthetic dataset (Y) with the expected frequencies from an original dataset (X), based on one or more categorical variables (cat_vars). If sampling weights are provided via weight_X and weight_Y, weighted frequency tables are computed using xtabs with a weight formula; otherwise, unweighted counts are computed. Additionally, if group_vars is provided, the Chi-square test is computed separately for each unique group defined by the grouping variables. For each group (or overall if no grouping is provided), the expected frequencies from X are scaled to match the total count in Y before performing the test.

Usage

compare_chisq_gof(X, ...)

## S3 method for class 'synth_pair'
compare_chisq_gof(X, ...)

## Default S3 method:
compare_chisq_gof(
  X,
  Y,
  cat_vars,
  group_vars = NULL,
  weight_X = NULL,
  weight_Y = NULL,
  simulate_p = TRUE,
  B = 2000,
  ...
)

Arguments

Details

The function computes frequency tables for the variables in cat_vars for both datasets X and Y. If sampling weights are provided, xtabs is used with a formula of the form "weight ~ var1 + var2 + ..."; otherwise, unweighted counts are computed using xtabs. A complete grid of all possible combinations of factor levels is created to ensure that missing cells are filled with zeros. The expected frequencies from X are scaled to match the total count in Y before performing the Chi-square goodness-of-fit test. When group_vars is provided, the function iterates over each unique combination of grouping variables and performs the test on each subset. The simulation of p-values (when simulate_p is TRUE) helps to handle cases with small expected frequencies.

Value

A data.table containing the grouping variables (if provided), the Chi-square statistic, degrees of freedom, and p-value for each group (or one overall row if group_vars is NULL).

Author(s)

Matthias Templ

See Also

Other comparison: ci_overlap(), compare_boxplots(), compare_correlation_matrices(), compare_distributions_cont(), compare_embedding(), compare_feature_importance(), compare_histograms(), compare_ks_test(), compare_means_frequencies(), compare_missing_values(), compare_model_performance(), compare_multivariate_distribution(), compare_multivariate_summary_statistics(), compare_outliers(), compare_pca(), compare_wasserstein(), densitydiff_1d_num(), densitydiff_kl_num(), densitydiff_pca()

Examples

set.seed(123)
X <- data.frame(
  gender = factor(sample(c("Male", "Female"), 500, replace = TRUE)),
  region = factor(sample(c("North", "South", "East", "West"), 500, replace = TRUE)),
  occupation = factor(sample(c("Engineer", "Doctor", "Artist", "Teacher"), 500, replace = TRUE)),
  weight = runif(500, 0.5, 1.5)
)

Y <- data.frame(
  gender = factor(sample(c("Male", "Female"), 1000, replace = TRUE)),
  region = factor(sample(c("North", "South", "East", "West"), 1000, replace = TRUE)),
  occupation = factor(sample(c("Engineer", "Doctor", "Artist", "Teacher"), 500, replace = TRUE)),
  weight = runif(1000, 0.5, 1.5)
)

# Overall test (weighted) with simulated p-values
result_overall <- compare_chisq_gof(X, Y, cat_vars = c("gender", "region"),
                                    weight_X = "weight", weight_Y = "weight",
                                    simulate_p = FALSE, B = 2000)
print(result_overall)

# Test conditionally by grouping variables (e.g., "region" and "gender")
result_by_group <- compare_chisq_gof(X, Y,
  cat_vars = c("gender"),
  group_vars = c("region", "occupation"),
  weight_X = "weight", weight_Y = "weight",
  simulate_p = TRUE, B = 2000)
print(result_by_group)

Compare Correlation Matrices between Two Datasets

Description

This function computes and compares the correlation matrices for a specified set of variables between two datasets (X and Y), such as an original dataset and an anonymized/synthetic version. For continuous variables, the user can choose Pearson, Spearman, or robust correlation methods (using either MM‐estimation or the MCD estimator). For categorical variables, the function computes Cramér’s V. For variable pairs with one continuous and one nominal variable, if the nominal variable is binary, the point‐biserial correlation is computed; otherwise, the correlation ratio (eta squared) is used and transformed into a correlation‐like measure.

Usage

compare_correlation_matrices(X, ...)

## S3 method for class 'synth_pair'
compare_correlation_matrices(X, ...)

## Default S3 method:
compare_correlation_matrices(
  X,
  Y,
  vars,
  method = "pearson",
  mixed_method = "point_biserial",
  ...
)

Arguments

Details

The function determines the type of each variable (continuous or categorical) based on its class. Continuous variables are correlated using the specified method. For robust methods, the MCD estimator from the robustbase package or MM-estimation via MASS::cov.rob is used. For categorical variables, Cramér’s V is computed from the chi-square statistic. For mixed pairs, if one variable is continuous and the other nominal, point-biserial correlation is used when the nominal variable has two levels, and the correlation ratio (eta squared) is computed otherwise.

Value

A list with three elements:

corr_X

The correlation matrix computed from dataset X.

corr_Y

The correlation matrix computed from dataset Y.

diff

The absolute difference matrix between the two correlation matrices.

Author(s)

Matthias Templ

See Also

Other comparison: ci_overlap(), compare_boxplots(), compare_chisq_gof(), compare_distributions_cont(), compare_embedding(), compare_feature_importance(), compare_histograms(), compare_ks_test(), compare_means_frequencies(), compare_missing_values(), compare_model_performance(), compare_multivariate_distribution(), compare_multivariate_summary_statistics(), compare_outliers(), compare_pca(), compare_wasserstein(), densitydiff_1d_num(), densitydiff_kl_num(), densitydiff_pca()

Examples

set.seed(123)
X <- data.frame(
  income = rnorm(500, mean = 50000, sd = 10000),
  age = rnorm(500, mean = 40, sd = 10),
  gender = factor(sample(c("Male", "Female"), 500, replace = TRUE)),
  region = factor(sample(c("North", "South", "East", "West"), 500, replace = TRUE))
)

Y <- data.frame(
  income = rnorm(1000, mean = 48000, sd = 12000),
  age = rnorm(1000, mean = 42, sd = 11),
  gender = factor(sample(c("Male", "Female"), 1000, replace = TRUE)),
  region = factor(sample(c("North", "South", "East", "West"), 1000, replace = TRUE))
)

# Continuous correlation using Pearson
res1 <- compare_correlation_matrices(X, Y, vars = c("income", "age"), method = "pearson")
print(res1$corr_X)
print(res1$corr_Y)
print(res1$diff)

# Categorical correlation using Cramér's V (computed automatically)
res2 <- compare_correlation_matrices(X, Y, vars = c("gender", "region"), method = "pearson")
print(res2$corr_X)
print(res2$corr_Y)

# Mixed example: income (continuous) vs gender (nominal); point-biserial or eta-squared is computed.
res3 <- compare_correlation_matrices(X, Y,
  vars = c("income", "gender"),
  method = "pearson",
  mixed_method = "point_biserial")
print(res3$corr_X)
print(res3$corr_Y)

Compare continuous distributions conditionally

Description

This function calculates and compares the empirical cumulative distribution functions (ECDF) of a sample (X) and a population (Y) for specified variables. It can handle weighted samples and provides options for conditional CDFs, and approximation.

Usage

compare_distributions_cont(X, ...)

## S3 method for class 'synth_pair'
compare_distributions_cont(X, ...)

## Default S3 method:
compare_distributions_cont(
  X,
  Y,
  vars = NULL,
  kind = NULL,
  conditional = NULL,
  weights = NULL,
  approx = c(FALSE, TRUE),
  n_approx = 10000,
  bounds = TRUE,
  ...
)

Arguments

Details

The following default calculations are taken:

If vars links to one or two numeric variables: the (weighted) ECDF is calculated for both data sets. With kind one can change to "density", "density_bayes", "density_ratio" and "density_ratio_bayes".

If vars links to one, two or three categorical variables: barcharts and mosaic plots are made.

If vars links to one numeric and one categorical variable: boxplots statistics are calculated.

If weights are provided, the weighted versions are computed. The function supports conditional estimates, which are computed by conditioning on the specified variable.

If the approx argument is set to TRUE, the ECDFs are approximated using the specified number of points (n_approx).

Value

A list with class "compare" containing the following components:

formula

A formula representing the CDF comparison.

ecdf

A data frame with the ECDF values.

kind

A character string indicating the type of comparison ("numeric").

Author(s)

Matthias Templ

See Also

Other comparison: ci_overlap(), compare_boxplots(), compare_chisq_gof(), compare_correlation_matrices(), compare_embedding(), compare_feature_importance(), compare_histograms(), compare_ks_test(), compare_means_frequencies(), compare_missing_values(), compare_model_performance(), compare_multivariate_distribution(), compare_multivariate_summary_statistics(), compare_outliers(), compare_pca(), compare_wasserstein(), densitydiff_1d_num(), densitydiff_kl_num(), densitydiff_pca()

Examples

# Example data 1: Complex sample and population data
S <- data.frame(
  age = sample(20:80, 100, replace = TRUE),
  income = rnorm(100, mean = 50000, sd = 10000),
  weights = runif(100, 0.5, 2)
)
U <- data.frame(
  age = sample(20:80, 10000, replace = TRUE),
  income = rnorm(10000, mean = 50000, sd = 10000)
)

# Compare ECDFs for age and income
result <- compare_distributions_cont(S, U, vars = c("age", "income"), weights = "weights")

# View the result
print(result$formula)
head(result$ecdf)
plot(result)
plot(result, which = "density")
plot(result, which = "density_bayes")

# Example data 2: Complex sample and complex sample data
X <- data.frame(
  age = sample(20:80, 100, replace = TRUE),
  income = rnorm(100, mean = 50000, sd = 10000),
  weights = runif(100, 0.5, 2)
)
Y <- data.frame(
  age = sample(20:80, 100, replace = TRUE),
  income = rnorm(10000, mean = 50000, sd = 10000),
  weights = runif(100, 0.5, 2)
)

# Compare ECDFs for age and income
result <- compare_distributions_cont(X, Y, vars = c("age", "income"), weights = "weights")

# View the result
print(result$formula)
head(result$ecdf)
plot(result)
plot(result, which = "density")
plot(result, which = "density_bayes")
plot(result, which = "density_ratio")

# Example data 3: Sample and sample data
X <- data.frame(
  age = sample(20:80, 100, replace = TRUE),
  income = rnorm(100, mean = 50000, sd = 10000)
)
Y <- data.frame(
  age = sample(20:80, 100, replace = TRUE),
  income = rnorm(100, mean = 50000, sd = 10000)
)

# Compare ECDFs for age and income
result <- compare_distributions_cont(X, Y, vars = c("age", "income"), n_approx = 100)

# View the result
print(result$formula)
head(result$ecdf)
plot(result)
plot(result, which = "density")
plot(result, which = "density_bayes")

Compare Dimensionality Reduction Methods (t-SNE, UMAP, MDS-Sammon) for Two Datasets

Description

This function compares two datasets using dimensionality reduction methods: t-distributed Stochastic Neighbor Embedding (t-SNE), Uniform Manifold Approximation and Projection (UMAP), and Multidimensional Scaling (MDS) with Sammon's mapping. It allows visualizing complex data structures, such as clusters or outliers, in two-dimensional space.

Usage

compare_embedding(X, ...)

## S3 method for class 'synth_pair'
compare_embedding(X, ...)

## Default S3 method:
compare_embedding(
  X,
  Y,
  vars,
  method = "tsne",
  perplexity = 30,
  n_neighbors = 15,
  side_by_side = FALSE,
  seed = NULL,
  ...
)

Arguments

Value

A list containing the embeddings and a ggplot2 visualization.

Author(s)

Matthias Templ

See Also

Other comparison: ci_overlap(), compare_boxplots(), compare_chisq_gof(), compare_correlation_matrices(), compare_distributions_cont(), compare_feature_importance(), compare_histograms(), compare_ks_test(), compare_means_frequencies(), compare_missing_values(), compare_model_performance(), compare_multivariate_distribution(), compare_multivariate_summary_statistics(), compare_outliers(), compare_pca(), compare_wasserstein(), densitydiff_1d_num(), densitydiff_kl_num(), densitydiff_pca()

Examples

# MDS (Sammon) embedding - doesn't require optional packages
set.seed(123)
X <- data.frame(
  var1 = rnorm(100, 50, 10),
  var2 = rnorm(100, 40, 8),
  var3 = rnorm(100, 30, 5)
)
Y <- data.frame(
  var1 = rnorm(100, 48, 12),
  var2 = rnorm(100, 42, 9),
  var3 = rnorm(100, 28, 6)
)
res_mds <- compare_embedding(X, Y, vars = c("var1", "var2", "var3"),
                             method = 'mds')


# t-SNE (requires Rtsne package)
if (requireNamespace("Rtsne", quietly = TRUE)) {
  res_tsne <- compare_embedding(X, Y, vars = c("var1", "var2", "var3"),
                                method = 'tsne')
}

# UMAP (requires uwot package)
if (requireNamespace("uwot", quietly = TRUE)) {
  res_umap <- compare_embedding(X, Y, vars = c("var1", "var2", "var3"),
                                method = 'umap')
}

Compare Feature Importance between Real and Synthetic Data

Description

Compares feature importance rankings from models trained on real (X) and synthetic/anonymized (Y) datasets.

Usage

compare_feature_importance(X, ...)

## S3 method for class 'synth_pair'
compare_feature_importance(X, ...)

## Default S3 method:
compare_feature_importance(
  X,
  Y,
  formula,
  method,
  importance_type = c("model", "permutation", "shap"),
  pred_wrapper = NULL,
  metric = "Accuracy",
  nsim = 5,
  ...
)

Arguments

Details

This function supports three types of feature importance measures:

• Model-based: Uses caret::varImp.

• Permutation-based: Uses vip::vi_permute. A prediction wrapper is required.

• SHAP-based: Computes mean absolute Shapley values using the kernelshap package.

• If the target variable in X is a factor, the target variable in Y is coerced to a factor with the same levels.

• For SHAP importance, the function uses the kernelshap package. The returned importance is the mean absolute SHAP value per feature.

• For classification tasks in the SHAP branch, if no pred_wrapper is provided, predictions are taken as the probability of the first class.

Value

A list containing:

Author(s)

Matthias Templ

See Also

Other comparison: ci_overlap(), compare_boxplots(), compare_chisq_gof(), compare_correlation_matrices(), compare_distributions_cont(), compare_embedding(), compare_histograms(), compare_ks_test(), compare_means_frequencies(), compare_missing_values(), compare_model_performance(), compare_multivariate_distribution(), compare_multivariate_summary_statistics(), compare_outliers(), compare_pca(), compare_wasserstein(), densitydiff_1d_num(), densitydiff_kl_num(), densitydiff_pca()

Examples

# Model-based importance (Decision Tree) - basic example
set.seed(123)
X <- data.frame(
  age = rnorm(100, 40, 10),
  income = 50000 + rnorm(100, 0, 10000),
  outcome = factor(sample(c("Yes", "No"), 100, replace = TRUE))
)
Y <- data.frame(
  age = rnorm(100, 42, 12),
  income = 48000 + rnorm(100, 0, 12000),
  outcome = factor(sample(c("Yes", "No"), 100, replace = TRUE))
)
res_tree <- compare_feature_importance(X, Y, outcome ~ .,
                                       method = "rpart",
                                       importance_type = "model")


# Permutation importance (requires vip package)
if (requireNamespace("vip", quietly = TRUE)) {
  pred_fn <- function(object, newdata) predict(object, newdata, type = "raw")
  res_perm <- compare_feature_importance(
    X, Y, outcome ~ ., method = "rf",
    importance_type = "permutation",
    pred_wrapper = pred_fn, metric = "Accuracy", nsim = 5
  )
}

Compare Histograms of a Numeric Variable in Two Datasets

Description

This function compares the distribution of a numeric variable between two datasets (e.g., an original dataset and an anonymized/synthetic version) using histograms. It allows conditional faceting on categorical variables and supports different visualization styles, including overlapping, side-by-side, stacked, and separate histograms.

Usage

compare_histograms(X, ...)

## S3 method for class 'synth_pair'
compare_histograms(X, ...)

## Default S3 method:
compare_histograms(
  X,
  Y,
  num_var,
  cat_vars,
  weight_X = NULL,
  weight_Y = NULL,
  facet_type = "wrap",
  bins = 30,
  transparency = 0.4,
  plot_type = "overlap",
  ...
)

Arguments

Value

A ggplot2 histogram visualization comparing the distributions.

Author(s)

Matthias Templ

See Also

Other comparison: ci_overlap(), compare_boxplots(), compare_chisq_gof(), compare_correlation_matrices(), compare_distributions_cont(), compare_embedding(), compare_feature_importance(), compare_ks_test(), compare_means_frequencies(), compare_missing_values(), compare_model_performance(), compare_multivariate_distribution(), compare_multivariate_summary_statistics(), compare_outliers(), compare_pca(), compare_wasserstein(), densitydiff_1d_num(), densitydiff_kl_num(), densitydiff_pca()

Examples

set.seed(123)
X <- data.frame(
  age = sample(20:80, 500, replace = TRUE),
  gender = sample(c("Male", "Female"), 500, replace = TRUE),
  income = rnorm(500, mean = 50000, sd = 10000),
  weight = runif(500, 0.5, 1.5)
)

Y <- data.frame(
  age = sample(20:80, 1000, replace = TRUE),
  gender = sample(c("Male", "Female"), 1000, replace = TRUE),
  income = rnorm(1000, mean = 48000, sd = 12000),
  weight = runif(1000, 0.5, 1.5)
)

# Overlapping histograms with transparency
compare_histograms(X, Y, num_var = "income", cat_vars = c("gender"),
                   weight_X = "weight", weight_Y = "weight",
                   facet_type = "wrap", plot_type = "overlap")
# Side-by-side histograms
compare_histograms(X, Y, num_var = "income", cat_vars = c("gender"),
                   weight_X = "weight", weight_Y = "weight",
                   facet_type = "wrap", plot_type = "side")

# Stacked histograms
compare_histograms(X, Y, num_var = "income", cat_vars = c("gender"),
                   weight_X = "weight", weight_Y = "weight",
                   facet_type = "wrap", plot_type = "stack")

# Separate histograms
compare_histograms(X, Y, num_var = "income", cat_vars = c("gender"),
                   weight_X = "weight", weight_Y = "weight",
                   facet_type = "wrap", plot_type = "separate")

Compare Distributions using the Kolmogorov-Smirnov Test

Description

This function performs a non-parametric Kolmogorov-Smirnov (KS) test to compare the cumulative distribution functions of a numeric variable between an original dataset (X) and an anonymized/synthetic dataset (Y). The test measures the maximum deviation between the two empirical cumulative distribution functions. The function can also perform the test separately for groups defined by one or more categorical variables.

Usage

compare_ks_test(X, ...)

## S3 method for class 'synth_pair'
compare_ks_test(X, ...)

## Default S3 method:
compare_ks_test(X, Y, num_var, cat_vars = NULL, ...)

Arguments

Value

A data.table with columns for the grouping variables (if provided), the KS test statistic, and the corresponding p-value. If no grouping is provided, a data.table with one row is returned.

Author(s)

Matthias Templ

See Also

Other comparison: ci_overlap(), compare_boxplots(), compare_chisq_gof(), compare_correlation_matrices(), compare_distributions_cont(), compare_embedding(), compare_feature_importance(), compare_histograms(), compare_means_frequencies(), compare_missing_values(), compare_model_performance(), compare_multivariate_distribution(), compare_multivariate_summary_statistics(), compare_outliers(), compare_pca(), compare_wasserstein(), densitydiff_1d_num(), densitydiff_kl_num(), densitydiff_pca()

Examples

set.seed(123)
X <- data.frame(
  age = sample(20:80, 500, replace = TRUE),
  gender = sample(c("Male", "Female"), 500, replace = TRUE),
  region = sample(c("North", "South", "East", "West"), 500, replace = TRUE),
  income = rnorm(500, mean = 50000, sd = 10000)
)

Y <- data.frame(
  age = sample(20:80, 1000, replace = TRUE),
  gender = sample(c("Male", "Female"), 1000, replace = TRUE),
  region = sample(c("North", "South", "East", "West"), 500, replace = TRUE),
  income = rnorm(1000, mean = 48000, sd = 12000)
)

# Perform KS test on the entire datasets
compare_ks_test(X, Y, num_var = "income")

# Perform KS test within groups defined by gender
compare_ks_test(X, Y, num_var = "income", cat_vars = c("gender"))
compare_ks_test(X, Y, num_var = "income", cat_vars = c("gender","region"))

Compare Means and Frequencies between Two Datasets

Description

This function compares the central tendencies of continuous variables and the frequency distributions of categorical variables between two datasets (e.g., an original dataset and an anonymized/synthetic dataset). For continuous variables, the function computes a set of summary statistics for each variable, including arithmetic mean, median, Huber mean, standard deviation, interquartile range (IQR), median absolute deviation (MAD), skewness, and kurtosis. Weighted estimates are computed if sampling weights are provided. In addition, conditional summaries can be produced if grouping variables are specified. For categorical variables, frequency and relative frequency tables are computed, using weights if available.

Usage

compare_means_frequencies(X, ...)

## S3 method for class 'synth_pair'
compare_means_frequencies(X, ...)

## Default S3 method:
compare_means_frequencies(
  X,
  Y,
  cont_vars,
  cat_vars,
  group_vars = NULL,
  weight_X = NULL,
  weight_Y = NULL,
  stats = c("mean", "median", "sd", "IQR", "mad", "huber", "skewness", "kurtosis"),
  ...
)

Arguments

Details

For continuous variables, if sampling weights are provided the function computes weighted estimates. The arithmetic mean and standard deviation are computed using weighted.mean and a weighted variance formula. The median is computed using Hmisc::wtd.quantile when weights are provided. The Huber mean is computed via MASS::huber (weighted Huber is not implemented). The IQR and MAD are computed using base functions. For skewness and kurtosis, if weights are provided, the function computes them using custom formulas; otherwise, it computes the unweighted versions.

For categorical variables, frequencies are computed as the sum of weights (if provided) or as raw counts, with relative frequencies calculated accordingly.

Value

A list containing three elements:

continuous_summary

A data.table with overall summary statistics for each continuous variable in X and Y.

conditional_summary

A data.table with conditional summary statistics computed by the grouping variables (if provided); otherwise, NULL.

categorical_summary

A data.table with frequency and relative frequency tables for each categorical variable in X and Y.

Author(s)

Matthias Templ

See Also

Other comparison: ci_overlap(), compare_boxplots(), compare_chisq_gof(), compare_correlation_matrices(), compare_distributions_cont(), compare_embedding(), compare_feature_importance(), compare_histograms(), compare_ks_test(), compare_missing_values(), compare_model_performance(), compare_multivariate_distribution(), compare_multivariate_summary_statistics(), compare_outliers(), compare_pca(), compare_wasserstein(), densitydiff_1d_num(), densitydiff_kl_num(), densitydiff_pca()

Examples

set.seed(123)
X <- data.frame(
  age = sample(20:80, 500, replace = TRUE),
  income = rnorm(500, mean = 50000, sd = 10000),
  gender = sample(c("Male", "Female"), 500, replace = TRUE),
  region = sample(c("North", "South", "East", "West"), 500, replace = TRUE),
  weight = runif(500, 0.5, 1.5)
)

Y <- data.frame(
  age = sample(20:80, 1000, replace = TRUE),
  income = rnorm(1000, mean = 48000, sd = 12000),
  gender = sample(c("Male", "Female"), 1000, replace = TRUE),
  region = sample(c("North", "South", "East", "West"), 1000, replace = TRUE),
  weight = runif(1000, 0.5, 1.5)
)

result <- compare_means_frequencies(X, Y,
             cont_vars = c("age", "income"),
             cat_vars = c("gender", "region"),
             group_vars = c("region"),
             weight_X = "weight", weight_Y = "weight",
             stats = c("mean", "median", "sd", "IQR", "mad", "huber", "skewness", "kurtosis"))

print(result$continuous_summary)
print(result$conditional_summary)
print(result$categorical_summary)

Compare Missing Value Patterns Between Original and Synthetic Data

Description

This function compares missing value patterns in an original dataset (X) and a synthetic/anonymized dataset (Y). It assesses whether the synthetic data mimics the same percentage and distribution of missingness across variables.

Usage

compare_missing_values(X, ...)

## S3 method for class 'synth_pair'
compare_missing_values(X, ...)

## Default S3 method:
compare_missing_values(X, Y, method = "percentage", ...)

Arguments

Details

Supported methods include:

• "percentage": Computes and compares missing value percentages for each variable.

• "pattern": Summarizes the frequency of missingness patterns across rows.

Both X and Y should be numeric data frames or matrices with the same structure.

Value

An object of class "missingCompare":

• For "percentage": a list with a data frame (summary) reporting per-variable missing counts and percentages.

• For "pattern": a list with two tables (patterns_X and patterns_Y) reporting the frequency of missingness patterns.

Author(s)

Matthias Templ

See Also

Other comparison: ci_overlap(), compare_boxplots(), compare_chisq_gof(), compare_correlation_matrices(), compare_distributions_cont(), compare_embedding(), compare_feature_importance(), compare_histograms(), compare_ks_test(), compare_means_frequencies(), compare_model_performance(), compare_multivariate_distribution(), compare_multivariate_summary_statistics(), compare_outliers(), compare_pca(), compare_wasserstein(), densitydiff_1d_num(), densitydiff_kl_num(), densitydiff_pca()

Examples

set.seed(123)
X <- data.frame(a = c(rnorm(95), rep(NA, 5)), b = c(rnorm(90), rep(NA, 10)))
Y <- data.frame(a = c(rnorm(95), rep(NA, 5)), b = c(rnorm(90), rep(NA, 10)))

# Compare missing value percentages
res_pct <- compare_missing_values(X, Y, method = "percentage")
print(res_pct)
summary(res_pct)
plot(res_pct)

# Compare missing value patterns
res_pat <- compare_missing_values(X, Y, method = "pattern")
print(res_pat)
summary(res_pat)
plot(res_pat)

Compare Predictive Model Performance between Two Datasets

Description

This function evaluates and compares predictive model performance between an original dataset (X) and a synthetic/anonymized dataset (Y). The model is trained separately within each dataset using cross-validation, and various performance metrics are computed.

Usage

compare_model_performance(X, ...)

## S3 method for class 'synth_pair'
compare_model_performance(X, ...)

## Default S3 method:
compare_model_performance(
  X,
  Y,
  formula,
  method = "rpart",
  metric = ifelse(is.factor(X[[as.character(formula[[2]])]]), "Accuracy", "RMSE"),
  trControl = NULL,
  ...
)

Arguments

Value

List containing trained models, performance metrics for X and Y, and comparison results.

Author(s)

Matthias Templ

See Also

Other comparison: ci_overlap(), compare_boxplots(), compare_chisq_gof(), compare_correlation_matrices(), compare_distributions_cont(), compare_embedding(), compare_feature_importance(), compare_histograms(), compare_ks_test(), compare_means_frequencies(), compare_missing_values(), compare_multivariate_distribution(), compare_multivariate_summary_statistics(), compare_outliers(), compare_pca(), compare_wasserstein(), densitydiff_1d_num(), densitydiff_kl_num(), densitydiff_pca()

Examples

if (requireNamespace("caret", quietly = TRUE)) {
library(caret)
set.seed(123)
X <- data.frame(income = rnorm(500, 50000, 10000),
                age = rnorm(500, 40, 10),
                gender = factor(sample(c("M", "F"), 500, replace = TRUE)),
                target = factor(sample(c("Yes", "No"), 500, replace = TRUE)))

Y <- data.frame(income = rnorm(500, 48000, 12000),
                age = rnorm(500, 42, 11),
                gender = factor(sample(c("M", "F"), 500, replace = TRUE)),
                target = factor(sample(c("Yes", "No"), 500, replace = TRUE)))

result <- compare_model_performance(X, Y,
           formula = target ~ income + age + gender,
           method = 'rpart',
           metric = 'Accuracy')

print(result$comparison)
}

Compare Multivariate Distributions using Mahalanobis Distance or Jensen-Shannon Divergence

Description

This function compares the joint distribution of a set of variables between two datasets (X and Y) using one of two methods. For continuous variables, it computes the weighted (if weights are provided) Mahalanobis distance between the mean vectors, using a pooled covariance matrix (or its generalized inverse). For nominal variables, it discretizes the data (if necessary) and computes a divergence based on the Jensen-Shannon divergence of the joint frequency distributions, which serves as a symmetric measure of distributional difference.

Usage

compare_multivariate_distribution(X, ...)

## S3 method for class 'synth_pair'
compare_multivariate_distribution(X, ...)

## Default S3 method:
compare_multivariate_distribution(
  X,
  Y,
  vars,
  method = "mahalanobis",
  weight_X = NULL,
  weight_Y = NULL,
  n_bins = 10,
  ...
)

Arguments

Details

For the "mahalanobis" method, if sampling weights are provided, the function computes the weighted mean vector for each dataset and a weighted pooled covariance matrix. The Mahalanobis distance is then computed as

D_M = \sqrt{(\mu_X - \mu_Y)^T S^{-1} (\mu_X - \mu_Y)}

where S is the pooled covariance matrix (or its generalized inverse if singular). For the "mutual_information" method, numeric variables are discretized into n_bins bins using cut(), and joint frequency tables are constructed. The tables are normalized into probability vectors, and the Jensen-Shannon divergence is computed as

JS(p,q) = \frac{1}{2} KL(p||m) + \frac{1}{2} KL(q||m)

where m = \frac{1}{2}(p+q) and KL is the Kullback-Leibler divergence.

Value

A list containing:

distance

The computed distance measure (Mahalanobis distance or Jensen-Shannon divergence).

method

The method used ("mahalanobis" or "mutual_information").

additional

A list of additional computed quantities (e.g., weighted means and pooled covariance for Mahalanobis, or probability vectors for mutual_information).

Author(s)

Matthias Templ

See Also

Other comparison: ci_overlap(), compare_boxplots(), compare_chisq_gof(), compare_correlation_matrices(), compare_distributions_cont(), compare_embedding(), compare_feature_importance(), compare_histograms(), compare_ks_test(), compare_means_frequencies(), compare_missing_values(), compare_model_performance(), compare_multivariate_summary_statistics(), compare_outliers(), compare_pca(), compare_wasserstein(), densitydiff_1d_num(), densitydiff_kl_num(), densitydiff_pca()

Examples

# Continuous example using Mahalanobis distance (weighted)
set.seed(123)
X <- data.frame(
  income = rnorm(500, mean = 50000, sd = 10000),
  age = rnorm(500, mean = 40, sd = 10),
  weight = runif(500, 0.5, 1.5)
)
Y <- data.frame(
  income = rnorm(1000, mean = 48000, sd = 12000),
  age = rnorm(1000, mean = 42, sd = 11),
  weight = runif(1000, 0.5, 1.5)
)
result_mahal <- compare_multivariate_distribution(X, Y, vars = c("income", "age"),
                                                  method = "mahalanobis",
                                                  weight_X = "weight", weight_Y = "weight")
print(result_mahal)

# Nominal example using mutual information (Jensen-Shannon divergence)
set.seed(456)
X_nom <- data.frame(
  gender = factor(sample(c("Male", "Female"), 500, replace = TRUE)),
  region = factor(sample(c("North", "South", "East", "West"), 500, replace = TRUE))
)
Y_nom <- data.frame(
  gender = factor(sample(c("Male", "Female"), 1000, replace = TRUE)),
  region = factor(sample(c("North", "South", "East", "West"), 1000, replace = TRUE))
)
result_js <- compare_multivariate_distribution(X_nom, Y_nom, vars = c("gender", "region"),
                                               method = "mutual_information", n_bins = 5)
print(result_js)

Compare Multivariate Summary Statistics between Two Datasets

Description

This function computes higher-order summary statistics for multiple variables in two datasets (e.g., an original dataset and an anonymized/synthetic dataset) to assess whether the synthetic data replicate the joint properties of the original data. For continuous variables, the function calculates the multivariate mean vector, variance–covariance matrix, and correlation matrix. When sampling weights are provided, weighted estimates are computed. For categorical variables, joint frequency tables (and relative frequencies) are constructed. The function returns the statistics for each dataset along with the differences between them.

Usage

compare_multivariate_summary_statistics(X, ...)

## S3 method for class 'synth_pair'
compare_multivariate_summary_statistics(X, ...)

## Default S3 method:
compare_multivariate_summary_statistics(
  X,
  Y,
  cont_vars,
  cat_vars,
  weight_X = NULL,
  weight_Y = NULL,
  ...
)

Arguments

Details

For continuous variables, if sampling weights are provided the function computes weighted means using weighted.mean and weighted covariance matrices using a custom function. The correlation matrices are derived from the covariance matrices. For categorical variables, if weights are provided the frequencies are computed as the sum of weights; otherwise, raw counts are used. Joint frequency tables are normalized to obtain relative frequencies.

Value

A list with two elements:

continuous

A list containing:

mean_X

The multivariate mean vector for X.

mean_Y

The multivariate mean vector for Y.

mean_diff

The difference between the mean vectors (X minus Y).

cov_X

The variance–covariance matrix for X.

cov_Y

The variance–covariance matrix for Y.

cov_diff

The difference between the covariance matrices (X minus Y).

cor_X

The correlation matrix for X.

cor_Y

The correlation matrix for Y.

cor_diff

The absolute difference between the correlation matrices.

categorical

A list containing:

freq_X

The joint frequency table for the categorical variables in X.

freq_Y

The joint frequency table for the categorical variables in Y.

rel_freq_X

The relative frequency table for X.

rel_freq_Y

The relative frequency table for Y.

Author(s)

Matthias Templ

See Also

Other comparison: ci_overlap(), compare_boxplots(), compare_chisq_gof(), compare_correlation_matrices(), compare_distributions_cont(), compare_embedding(), compare_feature_importance(), compare_histograms(), compare_ks_test(), compare_means_frequencies(), compare_missing_values(), compare_model_performance(), compare_multivariate_distribution(), compare_outliers(), compare_pca(), compare_wasserstein(), densitydiff_1d_num(), densitydiff_kl_num(), densitydiff_pca()

Examples

set.seed(123)
X <- data.frame(
  income = rnorm(500, mean = 50000, sd = 10000),
  age = rnorm(500, mean = 40, sd = 10),
  gender = factor(sample(c("Male", "Female"), 500, replace = TRUE)),
  region = factor(sample(c("North", "South", "East", "West"), 500, replace = TRUE)),
  weight = runif(500, 0.5, 1.5)
)

Y <- data.frame(
  income = rnorm(1000, mean = 48000, sd = 12000),
  age = rnorm(1000, mean = 42, sd = 11),
  gender = factor(sample(c("Male", "Female"), 1000, replace = TRUE)),
  region = factor(sample(c("North", "South", "East", "West"), 1000, replace = TRUE)),
  weight = runif(1000, 0.5, 1.5)
)

result <- compare_multivariate_summary_statistics(X, Y,
              cont_vars = c("income", "age"),
              cat_vars = c("gender", "region"),
              weight_X = "weight", weight_Y = "weight")

print(result$continuous)
print(result$categorical)

Compare Outlier Detection Between Original and Synthetic Data

Description

This function compares the prevalence of outliers in an original dataset (X) and an anonymized/synthetic dataset (Y) using various outlier detection methods. Supported methods include:

• "zscore": Uses z-scores (default threshold = 3) to flag outliers.

• "iqr": Flags values outside 1.5 times the interquartile range.

• "dbscan": Uses DBSCAN clustering to identify noise points as outliers.

• "robust": Uses robust Mahalanobis distances based on the Minimum Covariance Determinant.

Usage

compare_outliers(X, ...)

## S3 method for class 'synth_pair'
compare_outliers(X, ...)

## Default S3 method:
compare_outliers(X, Y, method = "zscore", threshold = NULL, ...)

Arguments

Details

Input datasets must be numeric and of the same structure. For multivariate methods, the comparison is performed on the entire dataset.

Value

A list with:

• summary: A data frame summarizing outlier counts and proportions in X and Y.

• details: Method-specific details (e.g., per-variable counts or clustering output).

Author(s)

Matthias Templ

See Also

Other comparison: ci_overlap(), compare_boxplots(), compare_chisq_gof(), compare_correlation_matrices(), compare_distributions_cont(), compare_embedding(), compare_feature_importance(), compare_histograms(), compare_ks_test(), compare_means_frequencies(), compare_missing_values(), compare_model_performance(), compare_multivariate_distribution(), compare_multivariate_summary_statistics(), compare_pca(), compare_wasserstein(), densitydiff_1d_num(), densitydiff_kl_num(), densitydiff_pca()

Examples

set.seed(123)
X <- data.frame(a = rnorm(100), b = rnorm(100))
Y <- data.frame(a = rnorm(100, mean = 0.1), b = rnorm(100, mean = -0.1))

# Using z-score method (no extra packages required)
res_z <- compare_outliers(X, Y, method = "zscore", threshold = 3)

# Using IQR method
res_iqr <- compare_outliers(X, Y, method = "iqr")

# Using DBSCAN (requires the 'dbscan' package)
if (requireNamespace("dbscan", quietly = TRUE)) {
  res_db <- compare_outliers(X, Y, method = "dbscan", eps = 0.5, minPts = 5)
}

# Using robust Mahalanobis (requires the 'robustbase' package)
if (requireNamespace("robustbase", quietly = TRUE)) {
  res_robust <- compare_outliers(X, Y, method = "robust", threshold = 0.975)
}

Compare Principal Component Analysis (PCA) between Two Datasets with Separate Loadings

Description

This function performs Principal Component Analysis (PCA) on two datasets (X and Y) for a specified set of numeric variables. When side_by_side is FALSE, a combined PCA is performed on the union of the data for visualization of the point scores, while separate PCA analyses are computed for X and Y to obtain loadings. The function then overlays loadings arrows from X (blue) and Y (red) onto the combined PCA biplot. When side_by_side is TRUE, separate biplots for X and Y are produced and arranged side by side.

Usage

compare_pca(X, ...)

## S3 method for class 'synth_pair'
compare_pca(X, ...)

## Default S3 method:
compare_pca(
  X,
  Y,
  vars,
  center = TRUE,
  scale = TRUE,
  biplot = TRUE,
  side_by_side = FALSE,
  ...
)

Arguments

Details

When side_by_side is FALSE, the function first adds a dataset indicator to X and Y, combines them, and performs PCA on the combined dataset for the point scores. Then, PCA is computed separately for X and Y. Loadings from these separate analyses are scaled using the range of the combined PCA scores and overlaid on the combined biplot with blue arrows for X and red arrows for Y.

Value

A list with two elements:

pca

If side_by_side = FALSE, a list with three PCA objects: combined, X, and Y. If side_by_side = TRUE, a list with two PCA objects: X and Y.

plot

A ggplot2 object: either a combined biplot with separate loadings or separate plots arranged side by side.

Author(s)

Matthias Templ

See Also

Other comparison: ci_overlap(), compare_boxplots(), compare_chisq_gof(), compare_correlation_matrices(), compare_distributions_cont(), compare_embedding(), compare_feature_importance(), compare_histograms(), compare_ks_test(), compare_means_frequencies(), compare_missing_values(), compare_model_performance(), compare_multivariate_distribution(), compare_multivariate_summary_statistics(), compare_outliers(), compare_wasserstein(), densitydiff_1d_num(), densitydiff_kl_num(), densitydiff_pca()

Examples

set.seed(123)
X <- data.frame(
  income = rnorm(500, mean = 50000, sd = 10000),
  age = rnorm(500, mean = 40, sd = 10)
)
# Add a positively correlated variable
X$expenses <- X$income * 0.5 + rnorm(n = 500, mean = 1000, sd = 500)
Y <- data.frame(
  income = rnorm(1000, mean = 48000, sd = 12000),
  age = rnorm(1000, mean = 42, sd = 11)
)
Y$expenses <- Y$income * 0.5 + rnorm(n = 1000, mean = 1000, sd = 500)
# Combined PCA biplot with separate loadings for X (blue) and Y (red)
res_combined <- compare_pca(X, Y,
  vars = c("income", "age", "expenses"),
  biplot = TRUE, side_by_side = FALSE)
print(res_combined$pca)
print(res_combined$plot)

# Separate PCA biplots for X and Y arranged side by side
res_separate <- compare_pca(X, Y, vars = c("income", "age"), biplot = TRUE, side_by_side = TRUE)
print(res_separate$pca)
print(res_separate$plot)

Compare Distributions using the Wasserstein Distance

Description

This function computes the one-dimensional Wasserstein distance between a numeric variable in two datasets, X (original) and Y (anonymized or synthetic). For continuous variables, the Wasserstein distance is approximated by integrating the absolute differences between the quantile functions computed on a grid of probabilities. If sampling weights are provided, weighted quantiles are computed using Hmisc::wtd.quantile. For nominal (categorical) variables, the function computes the total variation distance (half the L1 distance between the probability distributions) assuming a unit cost for mismatches.

Usage

compare_wasserstein(X, ...)

## S3 method for class 'synth_pair'
compare_wasserstein(X, ...)

## Default S3 method:
compare_wasserstein(
  X,
  Y,
  num_var,
  cat_vars = NULL,
  weight_X = NULL,
  weight_Y = NULL,
  var_type = "auto",
  n_grid = 1000,
  ...
)

Arguments

Details

When grouping variables are provided (via cat_vars), the Wasserstein distance is computed within each group.

Value

A data.table with the computed Wasserstein distance for each group (or overall if no grouping is provided). For continuous variables, the Wasserstein distance is approximated by the mean absolute difference between the quantile functions. For nominal variables, the result is the total variation distance.

Author(s)

Matthias Templ

See Also

Other comparison: ci_overlap(), compare_boxplots(), compare_chisq_gof(), compare_correlation_matrices(), compare_distributions_cont(), compare_embedding(), compare_feature_importance(), compare_histograms(), compare_ks_test(), compare_means_frequencies(), compare_missing_values(), compare_model_performance(), compare_multivariate_distribution(), compare_multivariate_summary_statistics(), compare_outliers(), compare_pca(), densitydiff_1d_num(), densitydiff_kl_num(), densitydiff_pca()

Examples

set.seed(123)
X <- data.frame(
  income = rnorm(500, mean = 50000, sd = 10000),
  gender = sample(c("Male", "Female"), 500, replace = TRUE),
  region = sample(c("North", "South", "East", "West"), 500, replace = TRUE),
  weight = runif(500, 0.5, 1.5)
)

Y <- data.frame(
  income = rnorm(1000, mean = 48000, sd = 12000),
  gender = sample(c("Male", "Female"), 1000, replace = TRUE),
  region = sample(c("North", "South", "East", "West"), 1000, replace = TRUE),
  weight = runif(1000, 0.5, 1.5)
)

# Compare overall continuous distributions (weighted)
compare_wasserstein(X, Y, num_var = "income", var_type = "continuous",
                    weight_X = "weight", weight_Y = "weight")

# Compare distributions within groups defined by gender
compare_wasserstein(X, Y, num_var = "income", cat_vars = c("gender"),
                    var_type = "continuous", weight_X = "weight", weight_Y = "weight")
compare_wasserstein(X, Y, num_var = "income", cat_vars = c("gender","region"),
                    var_type = "continuous", weight_X = "weight", weight_Y = "weight")

# Example for nominal variables

# Create a synthetic dataset X with a nominal variable "gender"
set.seed(123)
X_nom <- data.frame(
  gender = sample(c("Male", "Female"), 500, replace = TRUE),
  weight = runif(500, 0.5, 1.5)
)

# Create a synthetic dataset Y with a nominal variable "gender"
Y_nom <- data.frame(
  gender = sample(c("Male", "Female"), 1000, replace = TRUE),
  weight = runif(1000, 0.5, 1.5)
)

# Compare the distributions of the nominal variable using the Wasserstein distance.
# Here, var_type is explicitly set to "nominal" so that the function uses frequency tables.
result_nominal <- compare_wasserstein(X_nom, Y_nom, num_var = "gender",
                                      var_type = "nominal",
                                      weight_X = "weight", weight_Y = "weight")

print(result_nominal)

Confidence Intervals for RAPID Risk Estimates

Description

Computes confidence intervals for the RAPID disclosure risk measure (proportion of records at risk). Three methods are available: Wald (normal approximation), Wilson (score interval, recommended), and bootstrap (percentile).

Usage

## S3 method for class 'rapid'
confint(
  object,
  parm = NULL,
  level = 0.95,
  method = c("wilson", "wald", "bootstrap"),
  n_bootstrap = 1000,
  seed = NULL,
  ...
)

Arguments

Details

The RAPID score is a sample proportion (number at risk / total). Because the model is trained on separate synthetic data and applied to the original records, the per-record risk indicators are conditionally i.i.d. Bernoulli, making binomial confidence intervals valid.

wald

Standard normal approximation \hat p \pm z_{\alpha/2}\sqrt{\hat p(1-\hat p)/n}. Can produce intervals outside [0,1].

wilson

Score interval (Wilson, 1927). Has better coverage near 0 and 1; recommended as the default.

bootstrap

Resamples the at_risk indicators from object$records and returns percentile confidence limits. No model refitting is needed.

Value

A 1 \times 2 matrix with columns for the lower and upper confidence limits, following the standard confint convention.

Author(s)

Oscar Thees, Matthias Templ

References

Wilson, E.B. (1927). Probable Inference, the Law of Succession, and Statistical Inference. Journal of the American Statistical Association, 22(158), 209–212.

See Also

rapid, rapid_test

Other rapid: rapid(), rapid_synthesizer_cv(), rapid_test(), rapid_threshold_select()

Examples

# Small runnable example
set.seed(42)
X <- data.frame(
  age = sample(20:60, 80, replace = TRUE),
  sex = sample(c("M", "F"), 80, replace = TRUE),
  income = rnorm(80, 50000, 10000)
)
Y <- X
Y$income <- Y$income + rnorm(80, 0, 5000)
r <- rapid(X, Y, key_vars = c("age", "sex"),
           target_var = "income", model_type = "lm")
confint(r)
confint(r, method = "wald")


# With random forest model and bootstrap CI
r2 <- rapid(X, Y, key_vars = c("age", "sex"),
            target_var = "income", model_type = "rf")
confint(r2)
confint(r2, method = "bootstrap", n_bootstrap = 500)

Contingency Table Fidelity for Categorical Dependence Comparison

Description

Compares bivariate contingency tables (joint distributions) for all pairs of categorical variables between original and synthetic data. For each pair, the total variation (TV) distance between the proportion tables is computed. The mean TV distance across all pairs summarizes how well the synthetic data preserves the bivariate categorical dependence structure.

Usage

contingency_fidelity(X, ...)

## S3 method for class 'synth_pair'
contingency_fidelity(X, ...)

## Default S3 method:
contingency_fidelity(X, Y, vars = NULL, na.rm = TRUE, ...)

Arguments

Details

Use this when the data contains multiple categorical variables and you want to check whether their bivariate dependence structure is preserved. This complements copula_fidelity, which handles numeric pairs.

The algorithm proceeds as follows:

1. Auto-detect categorical variables (factor or character). Numeric variables are skipped with an informational message.

2. For each pair of categorical variables (i, j):

   • Factor levels are aligned: the union of levels from both datasets is used so that cells present in one dataset but not the other receive a count of zero.

   • The bivariate contingency table (proportions) is computed for both original and synthetic data.

   • The total variation distance is computed as

     TV = \frac{1}{2} \sum_{c} |P_{\mathrm{orig}}(c) - P_{\mathrm{syn}}(c)|

     where the sum runs over all cells c in the cross-tabulation.

3. The overall fidelity is summarized as mean_tv, the mean of all pairwise TV distances. The utility score is 1 - mean_tv.

Total variation distance is bounded in [0, 1]: 0 means identical joint distributions and 1 means completely non-overlapping distributions. Pairs where a variable has only a single level in both datasets are skipped and receive NA.

Interpretation (heuristic thresholds):

• utility_score > 0.95: EXCELLENT – dependence structure very well preserved

• utility_score > 0.80: GOOD – reasonably preserved

• utility_score > 0.50: MODERATE – some differences

• utility_score <= 0.50: POOR – significant differences

Note that this measure only captures bivariate (pairwise) categorical associations. Higher-order interactions (3-way or more) are not assessed.

Value

An object of class "contingency_fidelity" containing:

• mean_tv: mean of all pairwise total variation distances

• utility_score: 1 - mean_tv, in [0, 1] (higher = better)

• pairwise: data.frame with columns var1, var2, tv_distance for each pair of variables

• n_vars: number of categorical variables used

• vars: names of categorical variables used

• n_X: number of rows in X (after NA removal)

• n_Y: number of rows in Y (after NA removal)

Author(s)

Matthias Templ

References

Snoke, J., Raab, G. M., Nowok, B., Dibben, C., and Slavkovic, A. (2018). General and Specific Utility Measures for Synthetic Data. Journal of the Royal Statistical Society: Series A, 181(3), 663-688.

See Also

copula_fidelity for numeric dependence comparison, compare_chisq_gof for univariate categorical comparison, hellinger for univariate Hellinger distance, regression_fidelity for analysis-specific fidelity, subgroup_utility for stratified utility assessment

Other utility: chisq_utility(), ci_proximity(), copula_fidelity(), energy_distance(), gower(), hellinger(), mmd(), mqs(), pMSE(), plot.rumap(), propscore(), regression_fidelity(), rumap(), specks(), subgroup_utility(), tail_fidelity(), tstr()

Examples

set.seed(42)
n <- 500
# Original data with dependence between cat variables
gender <- sample(c("M", "F"), n, replace = TRUE)
region <- sample(c("N", "S", "E", "W"), n, replace = TRUE)
edu <- ifelse(gender == "M",
              sample(c("low", "mid", "high"), n, replace = TRUE,
                     prob = c(0.2, 0.5, 0.3)),
              sample(c("low", "mid", "high"), n, replace = TRUE,
                     prob = c(0.4, 0.4, 0.2)))
X <- data.frame(gender = gender, region = region, education = edu,
                 stringsAsFactors = TRUE)

# Good synthetic data (preserves dependence)
gender2 <- sample(c("M", "F"), n, replace = TRUE)
edu2 <- ifelse(gender2 == "M",
               sample(c("low", "mid", "high"), n, replace = TRUE,
                      prob = c(0.2, 0.5, 0.3)),
               sample(c("low", "mid", "high"), n, replace = TRUE,
                      prob = c(0.4, 0.4, 0.2)))
Y_good <- data.frame(gender = gender2, region = sample(region),
                      education = edu2, stringsAsFactors = TRUE)

result <- contingency_fidelity(X, Y_good)
print(result)
summary(result)

# Using synth_pair
pair <- synth_pair(X, Y_good)
result2 <- contingency_fidelity(pair)

# Selecting specific variables
result3 <- contingency_fidelity(X, Y_good, vars = c("gender", "education"))


plot(result)

Copula Fidelity for Dependence Structure Comparison

Description

Compares the dependence structure between original and synthetic data using empirical copulas. For each pair of numeric variables, the bivariate empirical copula is estimated via rank-transformation and compared using the Cramer-von Mises (CvM) statistic on a grid. The mean pairwise CvM distance provides an overall measure of how well the synthetic data preserves the joint dependence structure.

Usage

copula_fidelity(X, ...)

## S3 method for class 'synth_pair'
copula_fidelity(X, ...)

## Default S3 method:
copula_fidelity(X, Y, vars = NULL, n_grid = 50L, na.rm = TRUE, ...)

Arguments

Details

The algorithm proceeds as follows:

1. Each numeric variable is rank-transformed to [0, 1]: rank(x) / (length(x) + 1).

2. For each pair of variables (i, j), the bivariate empirical CDF is computed for both original and synthetic data:

   F(u_1, u_2) = \frac{1}{n} \sum_{k=1}^{n} 1(U_{k,i} \le u_1, U_{k,j} \le u_2)

3. The Cramer-von Mises statistic measures the integrated squared difference between the two empirical copulas on a regular grid:

   CvM_{ij} = \text{mean}\left( (F_{\text{orig}}(u_1, u_2) - F_{\text{syn}}(u_1, u_2))^2 \right)

4. The overall fidelity is summarized as the mean of all pairwise CvM distances. A utility score is computed as 1 / (1 + mean_cvm * 100).

This measure is invariant to monotone transformations of individual variables (since only ranks matter) and focuses purely on the dependence structure. It complements marginal distribution comparisons (e.g., Wasserstein, Hellinger) by assessing whether the joint structure is preserved.

Value

An object of class "copula_fidelity" containing:

• mean_cvm: mean of all pairwise CvM distances

• utility_score: transformed score in [0,1] (higher = better), computed as 1 / (1 + mean_cvm * 100)

• pairwise: data.frame with columns var1, var2, cvm_distance for each pair of variables

• n_vars: number of numeric variables used

• vars: names of numeric variables used

• n_X: number of rows in X (after NA removal)

• n_Y: number of rows in Y (after NA removal)

• n_grid: grid resolution used

Author(s)

Matthias Templ

References

Genest, C. and Remillard, B. (2008). Validity of the parametric bootstrap for goodness-of-fit testing in semiparametric models. Annales de l'Institut Henri Poincare, Probabilites et Statistiques, 44(6), 1096-1127.

Deheuvels, P. (1979). La fonction de dependance empirique et ses proprietes. Un test non parametrique d'independance. Academie Royale de Belgique. Bulletin de la Classe des Sciences, 5e Serie, 65, 274-292.

See Also

compare_correlation_matrices for linear dependence, energy_distance for multivariate distribution comparison, mmd for kernel-based comparison

Other utility: chisq_utility(), ci_proximity(), contingency_fidelity(), energy_distance(), gower(), hellinger(), mmd(), mqs(), pMSE(), plot.rumap(), propscore(), regression_fidelity(), rumap(), specks(), subgroup_utility(), tail_fidelity(), tstr()

Examples

set.seed(123)
# Original data with dependence
n <- 300
x1 <- rnorm(n)
x2 <- 0.8 * x1 + rnorm(n, sd = 0.6)
x3 <- rnorm(n)
X <- data.frame(x1 = x1, x2 = x2, x3 = x3)

# Good synthetic data (preserves dependence)
s1 <- rnorm(n)
s2 <- 0.8 * s1 + rnorm(n, sd = 0.6)
s3 <- rnorm(n)
Y_good <- data.frame(x1 = s1, x2 = s2, x3 = s3)

# Poor synthetic data (independent variables)
Y_poor <- data.frame(x1 = rnorm(n), x2 = rnorm(n), x3 = rnorm(n))

result_good <- copula_fidelity(X, Y_good)
print(result_good)

result_poor <- copula_fidelity(X, Y_poor)
print(result_poor)


# Heatmap of pairwise copula distances
plot(result_poor)

Correct Attribution Probability (CAP/DCAP)

Description

Computes the Correct Attribution Probability for synthetic data disclosure risk. Measures the probability that an adversary can correctly infer a sensitive target variable using known quasi-identifier (key) attributes.

Usage

dcap(X, ...)

## S3 method for class 'synth_pair'
dcap(X, ...)

## Default S3 method:
dcap(
  X,
  Y,
  key_vars,
  target_var,
  method = c("exact", "gower"),
  gower_threshold = 0.1,
  cont_bins = 10,
  na.rm = TRUE,
  ...
)

Arguments

Details

The Correct Attribution Probability (CAP) measures attribute disclosure risk in synthetic data. For each original record, it finds matching synthetic records based on key attributes and computes the probability that the correct target value appears among the matches.

CAP_i = \frac{|\{j \in Y : keys_j = keys_i \land target_j = target_i\}|}{|\{j \in Y : keys_j = keys_i\}|}

Higher CAP values indicate higher disclosure risk. A CAP of 1 means the intruder can perfectly infer the target; a CAP equal to baseline means no information leakage beyond random guessing.

Two summaries are returned: the raw mean CAP (cap) and the differential CAP (dcap = \overline{CAP} - baseline), which subtracts the baseline so that values near 0 indicate no leakage beyond random guessing and larger values indicate more attribute disclosure (Taub et al. 2018).

Value

An object of class "dcap" containing:

• cap_scores: per-record CAP values

• cap: mean CAP (raw correct-attribution probability; synthpop's CAPd)

• cap_median: median CAP

• dcap: differential CAP, cap - baseline (attribution risk above random guessing; Taub et al. 2018)

• dcap_median: cap_median - baseline

• n_matched: records with at least one synthetic match

• n_unmatched: records with no matches

• baseline: expected CAP under random guessing

• key_vars, target_var, method: input parameters

Baseline computation

The baseline is computed as the maximum target category frequency in the synthetic data (Y), representing the best guess an intruder could make without any key information.

Note: This differs from synthpop's baseline computation, which uses the original data (X). The riskutility approach is more conservative: it measures what an intruder learns from synthetic data specifically, rather than from the original distribution. Both approaches are valid but answer slightly different questions.

Gower threshold selection

When method = "gower", the gower_threshold parameter determines how close records must be to be considered "matching." Guidelines:

• 0.05: Very strict matching, few matches, may miss disclosure

• 0.1 (default): Balanced threshold, suitable for most cases

• 0.2: More permissive, catches near-matches

• 0.3+: Very permissive, may inflate risk estimates

Consider the scale and nature of your quasi-identifiers when choosing. For datasets with many categorical variables, lower thresholds work well. For continuous-heavy data, slightly higher thresholds may be appropriate.

synth_pair method

If X is a synth_pair object, the function extracts original, synthetic, key_vars, and target_var from the object. The synth_pair must have key_vars and target_var set.

Author(s)

Matthias Templ

References

Taub, J., Elliot, M., Pampaka, M., & Smith, D. (2018). Differential Correct Attribution Probability for Synthetic Data: An Exploration. Privacy in Statistical Databases, 122-137.

See Also

synth_pair for creating a comparison pair, tcap for per-record CAP, disco for disclosive records

Other attribution-risk: disco(), tcap(), weap()

Examples

# Create example data
set.seed(123)
X <- data.frame(
  age = sample(20:60, 100, replace = TRUE),
  gender = sample(c("M", "F"), 100, replace = TRUE),
  region = sample(c("N", "S", "E", "W"), 100, replace = TRUE),
  income = sample(c("low", "medium", "high"), 100, replace = TRUE)
)
# Synthetic version with some noise
Y <- X
Y$income <- sample(Y$income)  # Shuffle target

# Compute DCAP (traditional API)
result <- dcap(X, Y,
               key_vars = c("age", "gender", "region"),
               target_var = "income")
print(result)
summary(result)

# Using synth_pair (container API)
pair <- synth_pair(X, Y,
                   key_vars = c("age", "gender", "region"),
                   target_var = "income")
result2 <- dcap(pair)

Distance to Closest Record (DCR)

Description

Computes the Distance to Closest Record privacy metric for synthetic data. DCR compares the distances from synthetic records to their nearest neighbors in the training data versus a holdout set to detect potential privacy leaks.

Usage

dcr(X, ...)

## S3 method for class 'synth_pair'
dcr(X, ...)

## Default S3 method:
dcr(
  X,
  Y,
  holdout = NULL,
  holdout_fraction = 0.5,
  vars = NULL,
  method = c("gower", "euclidean"),
  na.rm = TRUE,
  seed = NULL,
  progress = FALSE,
  null_test = TRUE,
  n_null = 100,
  ...
)

Arguments

Details

Distance to Closest Record (DCR) is a privacy metric that detects whether synthetic data has memorized or copied training records. The key insight is that synthetic records should be equally close to training and holdout data if no information leakage occurred.

The metric works by:

1. Splitting original data into training and holdout (or using provided holdout)

2. For each synthetic record, computing distance to nearest training neighbor

3. For each synthetic record, computing distance to nearest holdout neighbor

4. Comparing these distance distributions using statistical tests

Interpretation:

• DCR ratio ~ 1: Good privacy - synthetic equally distant from both

• DCR ratio < 1: Privacy concern - synthetic closer to training

• DCR share ~ 0.5: Ideal - 50% closer to training, 50% to holdout

• DCR share > 0.5: Privacy concern - too many closer to training

Value

An object of class "dcr" containing:

• dcr_train: distances from synthetic to closest training record

• dcr_holdout: distances from synthetic to closest holdout record

• dcr_ratio: ratio of mean distances (train/holdout), ideally ~1

• dcr_share: proportion of synthetic closer to training than holdout

• privacy_pass: logical, TRUE if dcr_share <= 0.55 and Wilcoxon p > 0.05

• wilcox_test: Wilcoxon test result comparing train vs holdout distances

• null_distribution: null distribution statistics (if null_test = TRUE)

• n_synthetic, n_train, n_holdout: dataset sizes

• method, vars: parameters used

Important limitations (The DCR Delusion)

Yao et al. (2025) demonstrate that DCR and related distance-based metrics can fail to detect privacy leakage. Key findings:

• False sense of security: Datasets deemed "private" by DCR can still be vulnerable to Membership Inference Attacks (MIAs).

• Null distribution matters: DCR values must be compared against a proper null distribution, not interpreted in absolute terms.

• Not sufficient alone: DCR should be used alongside other privacy metrics and, ideally, actual MIA evaluations.

• Metric gaming: Generators can produce low DCR while still leaking membership information through other channels.

This implementation includes statistical tests and null distribution comparison to provide more rigorous privacy assessment, but users should be aware that passing DCR tests does not guarantee privacy protection.

Holdout splitting (important)

When no external holdout is provided, DCR internally splits the original data X into training and holdout portions. This has important implications:

• Reduced training set: With default holdout_fraction = 0.5, only half of X is used for distance comparison. This may miss some disclosure risks if the synthetic data memorized records in the holdout.

• Variability: Results depend on the random split (use seed for reproducibility).

• Best practice: Provide a separate holdout set from the synthesis process if available. Many synthesis frameworks support train/test splits.

For small datasets (< 500 records), consider using a smaller holdout fraction (e.g., 0.2-0.3) to maintain sufficient training data for comparison.

Author(s)

Matthias Templ

References

Platzer, M. & Reutterer, T. (2021). Holdout-Based Empirical Assessment of Mixed-Type Synthetic Data. Frontiers in Big Data, 4, 679939. doi:10.3389/fdata.2021.679939

Park, N., et al. (2018). Data Synthesis based on Generative Adversarial Networks. Proceedings of the VLDB Endowment, 11(10), 1071–1083. doi:10.14778/3231751.3231757

Yao, Z., Krco, N., Ganev, G. & de Montjoye, Y.-A. (2025). The DCR Delusion: Measuring the Privacy Risk of Synthetic Data. https://arxiv.org/abs/2505.01524

Zhao, Z., et al. (2021). CTAB-GAN: Effective Table Data Synthesizing. Asian Conference on Machine Learning.

See Also

nndr for nearest neighbor distance ratio, ims for exact match detection, nnaa for nearest-neighbor adversarial accuracy

Other distance-risk: hitting_rate(), ims(), nnaa(), nndr(), repu(), rf_privacy()

Examples

# Create example data
set.seed(123)
X <- data.frame(
  age = rnorm(200, 40, 10),
  income = rnorm(200, 50000, 15000),
  gender = sample(c("M", "F"), 200, replace = TRUE),
  region = sample(c("N", "S", "E", "W"), 200, replace = TRUE)
)

# Good synthetic data (random, no memorization)
Y_good <- data.frame(
  age = rnorm(200, 40, 10),
  income = rnorm(200, 50000, 15000),
  gender = sample(c("M", "F"), 200, replace = TRUE),
  region = sample(c("N", "S", "E", "W"), 200, replace = TRUE)
)

# Compute DCR
result <- dcr(X, Y_good, seed = 42)
print(result)
summary(result)

delta-Presence Risk Assessment

Description

Computes the delta-presence risk measure for synthetic/released data relative to the original/population data. delta-Presence (Nergiz et al., 2007) bounds the membership probability for any record in the population, measuring how confidently an adversary can determine whether a specific individual contributed to the released dataset.

Usage

delta_presence(X, ...)

## S3 method for class 'synth_pair'
delta_presence(X, ...)

## Default S3 method:
delta_presence(X, Y, key_vars, delta_min = 0, delta_max = 1, na.rm = TRUE, ...)

Arguments

Details

delta-Presence is a formal privacy model that bounds the membership disclosure probability. Given a population dataset (the original data X) and a published subset (the synthetic/released data Y), delta-presence bounds the probability that any record from X appears in Y.

For each quasi-identifier combination k:

• f_k = frequency of combination k in the synthetic data Y

• F_k = frequency of combination k in the original data X

• Membership probability: Pr(t \in Y | t \in X) = f_k / F_k

The dataset satisfies (\delta_{min}, \delta_{max})-presence if for all records: \delta_{min} \le f_k / F_k \le \delta_{max}.

Membership probabilities are capped at 1.0 (when f_k > F_k, which can occur when the synthetic data overrepresents certain combinations).

Interpretation:

• Values close to 0: The QI combination is underrepresented in Y relative to X, meaning membership is unlikely

• Values close to 1: The QI combination has similar frequency in both datasets, meaning membership is highly probable

• Values = 0: The QI combination exists in X but not in Y (no membership risk for these records)

QI combinations in Y but not X: If QI combinations appear in the synthetic data Y that do not exist in the original data X, a warning is issued. These represent fabricated combinations that do not correspond to any real records.

Value

An object of class "delta_presence" containing:

• membership_prob: numeric vector of f_k/F_k per record in X

• per_combination: data.frame with QI combination, f_k, F_k, prob

• delta_min, delta_max: bounds used

• n_violations_lower: number of records below delta_min

• n_violations_upper: number of records above delta_max

• pct_violations: fraction of records violating bounds

• satisfies_delta: logical, all records within bounds

• privacy_pass: same as satisfies_delta

• n_original, n_synthetic: dataset sizes

• key_vars: quasi-identifier variables used

Author(s)

Matthias Templ

References

Nergiz, M. E., Atzori, M. & Clifton, C. (2007). Hiding the Presence of Individuals from Shared Databases. Proceedings of the 2007 ACM SIGMOD International Conference on Management of Data, 665–676. doi:10.1145/1247480.1247554

See Also

kanonymity for k-anonymity assessment, ldiversity for l-diversity assessment, tcloseness for t-closeness assessment, individual_risk for probabilistic risk assessment

Other privacy-models: attacker_risk(), disclosure_report(), domias(), drisk(), epsilon_identifiability(), individual_risk(), inspect_record(), kanonymity(), ldiversity(), linkability(), merge_per_record(), mia_classifier(), population_uniqueness(), recordLinkage(), risk_by_group(), singling_out(), suda(), tcloseness(), top_at_risk()

Examples

# Create example data
set.seed(123)
n <- 200
X <- data.frame(
  age = sample(c("young", "middle", "old"), n, replace = TRUE),
  gender = sample(c("M", "F"), n, replace = TRUE),
  region = sample(c("N", "S", "E", "W"), n, replace = TRUE)
)

# Synthetic data with similar distribution
Y <- data.frame(
  age = sample(c("young", "middle", "old"), n, replace = TRUE),
  gender = sample(c("M", "F"), n, replace = TRUE),
  region = sample(c("N", "S", "E", "W"), n, replace = TRUE)
)

result <- delta_presence(X, Y,
                         key_vars = c("age", "gender", "region"),
                         delta_min = 0.1, delta_max = 0.9)
print(result)
summary(result)
plot(result)


# Memorized data - all probabilities near 1
Y_copy <- X
result_bad <- delta_presence(X, Y_copy,
                             key_vars = c("age", "gender", "region"),
                             delta_max = 0.5)
print(result_bad)

Density ratio and entropy

Description

Computes the density ratio between two vectors and provides entropy measures and more.

Usage

densitydiff_1d_num(
  x,
  y,
  bayesspace = TRUE,
  stepsize = 1000,
  strata_x = NULL,
  strata_y = NULL
)

Arguments

Value

An object of class "denratio": a list with the evaluated densities, their ratio over a grid of points, and Kullback–Leibler (kl) and Jensen–Shannon (jsd) divergence measures. For stratified input, a list of such per-stratum objects. Has print and plot methods.

Author(s)

Matthias Templ

See Also

Other comparison: ci_overlap(), compare_boxplots(), compare_chisq_gof(), compare_correlation_matrices(), compare_distributions_cont(), compare_embedding(), compare_feature_importance(), compare_histograms(), compare_ks_test(), compare_means_frequencies(), compare_missing_values(), compare_model_performance(), compare_multivariate_distribution(), compare_multivariate_summary_statistics(), compare_outliers(), compare_pca(), compare_wasserstein(), densitydiff_kl_num(), densitydiff_pca()

Examples

# Simple example
set.seed(123)
x <- rnorm(200, mean = 40, sd = 10)
y <- rnorm(200, mean = 42, sd = 11)
d <- densitydiff_1d_num(x, y)
plot(d)

# With stratification
strata_x <- factor(sample(c("A", "B"), length(x), replace = TRUE))
strata_y <- factor(sample(c("A", "B"), length(y), replace = TRUE))
d2 <- densitydiff_1d_num(x, y, strata_x = strata_x, strata_y = strata_y)

Kullback-Leibler divergence

Description

Kullback-Leibler divergence between two variables or two matrices/data.frames.

Usage

densitydiff_kl_num(X, Y, stepsize = 1000)

Arguments

Details

First the probability distributions are estimated before the Kullback-Leiber divergence is calculated.

The Kullback-Leibler divergence is defined as:

KLD(X,Y) = \sum{X \times \log\left(\frac{X}{Y}\right)}

If the data are multivariate, a Kullback-Leibler inspired divergence is calculated through

KLD(X,Y) = \sum{X \times \log\left(\frac{X}{Y}\right)}

KLD(X,Y) = \sum_{i,j}{X_{i,j} \left( \log\left(\frac{X_{i,j}}{Y_{i,j}}\right) + \log(Y) - \log(X) \right)}

with

X = \sum{X_{i,j}}

and

Y = \sum{Y_{i,j}}

Value

The Kullback-Leibler divergence of X and Y.

Author(s)

Matthias Templ

See Also

Other comparison: ci_overlap(), compare_boxplots(), compare_chisq_gof(), compare_correlation_matrices(), compare_distributions_cont(), compare_embedding(), compare_feature_importance(), compare_histograms(), compare_ks_test(), compare_means_frequencies(), compare_missing_values(), compare_model_performance(), compare_multivariate_distribution(), compare_multivariate_summary_statistics(), compare_outliers(), compare_pca(), compare_wasserstein(), densitydiff_1d_num(), densitydiff_pca()

Examples

x <- rnorm(100)
y <- rnorm(100)

densitydiff_kl_num(x, y)

X <- MASS::mvrnorm(100, mu = c(0,0), Sigma = diag(2))
Y <- MASS::mvrnorm(100, mu = c(0,0), Sigma = diag(2))
densitydiff_kl_num(X, Y)

X <- MASS::mvrnorm(100, mu = c(0,0,0), Sigma = diag(3))
Y <- MASS::mvrnorm(100, mu = c(0,0,0), Sigma = diag(3))
densitydiff_kl_num(X, Y)

Density ratio and entropy of first PC's

Description

Computes the density ratio between the selected principal components and provides entropy measures and more.

Usage

densitydiff_pca(
  X,
  Y,
  bayesspace = TRUE,
  stepsize = 1000,
  strata_x = NULL,
  strata_y = NULL
)

Arguments

Value

An object of class "denpca": a list with per-principal-component Kullback–Leibler (kl) and Jensen–Shannon (jsd) divergences and the mean (mean_ratio) and standard deviation (sd_ratio) of the density ratio. For stratified input, a list of such per-stratum objects. Has print and plot methods.

Author(s)

Matthias Templ

See Also

Other comparison: ci_overlap(), compare_boxplots(), compare_chisq_gof(), compare_correlation_matrices(), compare_distributions_cont(), compare_embedding(), compare_feature_importance(), compare_histograms(), compare_ks_test(), compare_means_frequencies(), compare_missing_values(), compare_model_performance(), compare_multivariate_distribution(), compare_multivariate_summary_statistics(), compare_outliers(), compare_pca(), compare_wasserstein(), densitydiff_1d_num(), densitydiff_kl_num()

Examples

# Simple example with multivariate numeric data
set.seed(123)
X <- data.frame(
  var1 = rnorm(100, 50, 10),
  var2 = rnorm(100, 1000, 200),
  var3 = rnorm(100, 30, 5),
  var4 = rnorm(100, 500, 100)
)
Y <- data.frame(
  var1 = rnorm(100, 52, 11),
  var2 = rnorm(100, 980, 220),
  var3 = rnorm(100, 31, 6),
  var4 = rnorm(100, 510, 110)
)
d1 <- densitydiff_pca(X, Y, bayesspace = FALSE)
d1

Comprehensive Disclosure Risk Report

Description

Computes multiple disclosure risk metrics and produces a comprehensive report for data privacy assessment. Combines attribution-based measures (DCAP, TCAP, WEAP, DiSCO, RAPID), distance-based measures (DCR, NNDR, IMS, dRisk, hitting rate, RF Privacy), privacy models (k-anonymity, l-diversity, t-closeness), and membership inference attacks (singling out, linkability, NNAA, DOMIAS). Works for both synthetic and traditionally anonymized data.

Usage

disclosure_report(X, ...)

## S3 method for class 'synth_pair'
disclosure_report(X, ...)

## Default S3 method:
disclosure_report(
  X,
  Y,
  key_vars = NULL,
  target_var = NULL,
  holdout = NULL,
  holdout_fraction = 0.5,
  distance_vars = NULL,
  distance_method = c("gower", "euclidean"),
  compute = "all",
  rapid_model = "rf",
  na.rm = TRUE,
  seed = NULL,
  verbose = TRUE,
  ...
)

Arguments

Details

This function provides a one-stop assessment of disclosure risk by computing multiple complementary metrics across four families:

Attribution-Based Metrics (require key_vars and target_var):

• dcap: Overall correct attribution probability

• tcap: Per-record attribution risk with categories

• weap: Within equivalence class attribution (on synthetic only)

• disco: Count of disclosive synthetic records

• rapid: ML-based attribute inference risk (requires ranger package)

Distance-Based Metrics (use holdout comparison):

• dcr: Distance to closest record ratio

• nndr: Nearest neighbor distance ratio

• ims: Identical match share (exact copies)

• drisk: Disclosure risk for continuous variables

• hitting_rate: Fraction of records within threshold distance

• rf_privacy: Random forest proximity-based memorization test (requires ranger)

Privacy Models (single-dataset, require key_vars):

• kanonymity: Minimum equivalence class size

• ldiversity: Sensitive attribute diversity per EC (requires target_var)

• tcloseness: EMD between EC and overall distribution (requires target_var)

• individual_risk: Frequency-based per-record risk

Membership Inference (require holdout):

• singling_out: Predicate-based uniqueness attack (GDPR criterion)

• linkability: Record linkage attack (GDPR criterion)

• nnaa: Nearest-neighbor adversarial accuracy / privacy loss

• domias: Density-based membership inference (requires ranger package)

The overall risk is determined by the number of failed checks:

• Low: All metrics pass

• Medium: 1-2 metrics fail

• High: 3+ metrics fail

Value

An object of class "disclosure_report" containing:

• results: list of individual metric results

• summary: data frame with key metrics and pass/fail status

• overall_risk: character, overall risk assessment ("LOW", "MEDIUM", "HIGH")

• n_pass: number of metrics that passed

• n_warn: number of metrics that warned/failed

• parameters: list of input parameters used

Author(s)

Matthias Templ

See Also

dcap, tcap, weap, disco, rapid, dcr, nndr, ims, drisk, hitting_rate, rf_privacy, kanonymity, ldiversity, tcloseness, individual_risk, singling_out, linkability, nnaa, domias

Other privacy-models: attacker_risk(), delta_presence(), domias(), drisk(), epsilon_identifiability(), individual_risk(), inspect_record(), kanonymity(), ldiversity(), linkability(), merge_per_record(), mia_classifier(), population_uniqueness(), recordLinkage(), risk_by_group(), singling_out(), suda(), tcloseness(), top_at_risk()

Examples

# Create example data
set.seed(123)
n <- 200
original <- data.frame(
  age_group = sample(c("18-30", "31-45", "46-60", "60+"), n, replace = TRUE),
  gender = sample(c("M", "F"), n, replace = TRUE),
  region = sample(c("N", "S", "E", "W"), n, replace = TRUE),
  income = sample(c("low", "medium", "high"), n, replace = TRUE)
)

# Good synthetic data
synthetic <- data.frame(
  age_group = sample(c("18-30", "31-45", "46-60", "60+"), n, replace = TRUE),
  gender = sample(c("M", "F"), n, replace = TRUE),
  region = sample(c("N", "S", "E", "W"), n, replace = TRUE),
  income = sample(c("low", "medium", "high"), n, replace = TRUE)
)

# Generate comprehensive report
report <- disclosure_report(
  original, synthetic,
  key_vars = c("age_group", "gender", "region"),
  target_var = "income",
  seed = 42
)

print(report)
summary(report)
plot(report)

Disclosive in Synthetic Correct Original (DiSCO)

Description

Identifies synthetic records that could potentially disclose information about original records. A DiSCO record is one that matches an original record on key variables AND has the same target value, making it potentially disclosive.

Usage

disco(X, ...)

## S3 method for class 'synth_pair'
disco(X, ...)

## Default S3 method:
disco(
  X,
  Y,
  key_vars,
  target_var,
  disclosure_type = c("potential", "certain"),
  method = c("exact", "gower"),
  gower_threshold = 0.1,
  na.rm = TRUE,
  ...
)

Arguments

Details

DiSCO (Disclosive in Synthetic Correct Original) is a measure developed for the synthpop package to identify attribute disclosure risk. It identifies synthetic records that:

1. Match an original record on all key variables

2. Have the same target value as that original record

Two disclosure types are available:

Potential disclosure (disclosure_type = "potential"): Counts any synthetic record that matches an original on key+target. This is a broader measure that identifies all records that could potentially leak information. Use this when you want to identify all possible disclosure risks.

Certain disclosure (disclosure_type = "certain"): Only counts disclosures where the key combination uniquely determines the target in the original data. This means an intruder who matches a synthetic record to the original would learn the target with certainty. This is compatible with synthpop's DiSCO measure and is more conservative.

The function always computes both metrics for comparison, but the primary output (n_disco, pct_disco, disco_idx) reflects the chosen disclosure_type.

Interpretation:

• High DiSCO count: Many synthetic records could leak original information

• DiSCO/Baseline ratio > 1: Synthetic data leaks more than expected by chance

• DiSCO = 0: No exact key+target matches (lowest risk scenario)

• pct_original_disclosed: For "certain" method, shows what fraction of original records have their target certainly exposed

Value

An object of class "disco" containing:

• disco_idx: indices of DiSCO records in Y (synthetic data)

• disco_records: the actual DiSCO records from Y

• n_disco: number of DiSCO records (based on disclosure_type)

• pct_disco: percentage of synthetic records that are DiSCO

• matched_original_idx: for each DiSCO record, which original record(s) it matches

• n_disco_potential: count using "potential" method (always computed)

• n_disco_certain: count using "certain" method (always computed)

• pct_original_disclosed: percentage of original records disclosed (for "certain" method)

• baseline_disco: expected DiSCO count under random target assignment

• key_vars, target_var, method, disclosure_type: input parameters

Author(s)

Matthias Templ

References

Raab, G.M., Nowok, B., & Dibben, C. (2021). Assessing, visualizing and improving the utility of synthetic data. arXiv preprint arXiv:2109.12717.

See Also

tcap for targeted correct attribution probability, weap for within equivalence class attribution probability, dcap for differential correct attribution probability

Other attribution-risk: dcap(), tcap(), weap()

Examples

# Create example data
set.seed(42)
X <- data.frame(
  age = sample(20:40, 50, replace = TRUE),
  gender = sample(c("M", "F"), 50, replace = TRUE),
  disease = sample(c("none", "A", "B"), 50, replace = TRUE,
                   prob = c(0.7, 0.2, 0.1))
)
# Synthetic version - some records will match by chance
Y <- data.frame(
  age = sample(20:40, 100, replace = TRUE),
  gender = sample(c("M", "F"), 100, replace = TRUE),
  disease = sample(c("none", "A", "B"), 100, replace = TRUE,
                   prob = c(0.7, 0.2, 0.1))
)

# Find DiSCO records using "potential" disclosure (default)
result_potential <- disco(X, Y,
                          key_vars = c("age", "gender"),
                          target_var = "disease",
                          disclosure_type = "potential")
print(result_potential)

# Find DiSCO records using "certain" disclosure (synthpop-compatible)
result_certain <- disco(X, Y,
                        key_vars = c("age", "gender"),
                        target_var = "disease",
                        disclosure_type = "certain")
print(result_certain)

# Compare both methods
cat("Potential DiSCO:", result_potential$n_disco_potential, "\n")
cat("Certain DiSCO:", result_potential$n_disco_certain, "\n")

# View the disclosive records
if (result_potential$n_disco > 0) {
  head(result_potential$disco_records)
}

Density-based Membership Inference Attack (DOMIAS)

Description

Detects local overfitting in synthetic data by estimating density ratios between the synthetic data distribution and a reference distribution at each test point. Records where the synthetic density is much higher than the reference density are likely memorized. The AUC of classifying training versus holdout records by their density ratios serves as the attack metric.

Usage

domias(X, ...)

## S3 method for class 'synth_pair'
domias(X, ...)

## Default S3 method:
domias(
  X,
  Y,
  holdout = NULL,
  holdout_fraction = 0.5,
  radius = 0.1,
  vars = NULL,
  na.rm = TRUE,
  seed = NULL,
  ...
)

Arguments

Details

DOMIAS (Density-based Overfitting Membership Inference Attack on Synthetic data) detects whether a synthetic data generator has memorized specific training records. The key insight is that if a generator overfits to training data, the synthetic distribution will have higher density around training points compared to unseen holdout points.

The algorithm works by:

1. Splitting original data into training (used for synthesis) and holdout

2. For each test record, estimating synthetic density by counting neighbors within radius r in the synthetic data

3. Estimating reference density by counting neighbors within radius r in the full original data (training + holdout)

4. Computing the density ratio: (density_synth + 1) / (density_ref + 1), with Laplace smoothing to avoid division by zero

5. Comparing density ratios for training versus holdout records

Interpretation:

• AUC ~ 0.5: Good privacy - density ratios cannot distinguish training from holdout (no memorization detected)

• AUC > 0.5: Privacy concern - training records have higher density ratios, suggesting local overfitting

• AUC > 0.6: Likely memorization detected

• memorization_score ~ 1: No differential memorization

• memorization_score >> 1: Synthetic data concentrated around training records

This implementation uses Gower distance for neighbor counting, which handles mixed data types (numeric, categorical, ordinal). For purely numeric data, the radius parameter corresponds to the Gower distance (normalized between 0 and 1).

Value

An object of class "domias" containing:

• density_ratios_train: numeric vector of density ratios for training records

• density_ratios_holdout: numeric vector of density ratios for holdout records

• auc: AUC of train-vs-holdout classification by density ratio

• mean_ratio_train: mean density ratio for training records

• mean_ratio_holdout: mean density ratio for holdout records

• memorization_score: mean_ratio_train / mean_ratio_holdout (ideally ~1)

• privacy_pass: logical, TRUE if auc <= 0.6

• n_train, n_holdout, n_synthetic: dataset sizes

• radius: radius used for density estimation

• vars: variables used

Holdout splitting

When no external holdout is provided, the original data is split internally. The holdout serves as a control group: if the generator has not memorized training data, density ratios should be similar for training and holdout records. For best results, provide a separate holdout set from the synthesis process.

Choosing the radius

The radius parameter controls the bandwidth of the kernel density estimate. Guidelines:

• Smaller radius (0.01-0.05): More sensitive to local memorization but noisier estimates, needs more data

• Default radius (0.1): Good balance for most datasets

• Larger radius (0.2-0.5): Smoother estimates but may miss localized overfitting

Author(s)

Matthias Templ

References

Van Breugel, B., Sun, H., Qian, Z. & van der Schaar, M. (2023). Membership Inference Attacks against Synthetic Data through Overfitting Detection. Proceedings of the 26th International Conference on Artificial Intelligence and Statistics (AISTATS), PMLR 206, 3493–3514. https://proceedings.mlr.press/v206/breugel23a.html

See Also

dcr for distance to closest record, nndr for nearest neighbor distance ratio, nnaa for nearest-neighbor adversarial accuracy

Other privacy-models: attacker_risk(), delta_presence(), disclosure_report(), drisk(), epsilon_identifiability(), individual_risk(), inspect_record(), kanonymity(), ldiversity(), linkability(), merge_per_record(), mia_classifier(), population_uniqueness(), recordLinkage(), risk_by_group(), singling_out(), suda(), tcloseness(), top_at_risk()

Examples

# Create example data
set.seed(123)
n <- 200
X <- data.frame(
  age = rnorm(n, 40, 10),
  income = rnorm(n, 50000, 15000),
  gender = sample(c("M", "F"), n, replace = TRUE),
  region = sample(c("N", "S", "E", "W"), n, replace = TRUE)
)

# Good synthetic data (random, no memorization)
Y_good <- data.frame(
  age = rnorm(n, 40, 10),
  income = rnorm(n, 50000, 15000),
  gender = sample(c("M", "F"), n, replace = TRUE),
  region = sample(c("N", "S", "E", "W"), n, replace = TRUE)
)

result <- domias(X, Y_good, seed = 42)
print(result)
summary(result)


# Memorized data (Y is copy of X) - should show high AUC
Y_copy <- X[sample(nrow(X), n, replace = TRUE), ]
result_bad <- domias(X, Y_copy, seed = 42)
print(result_bad)

Disclosure Risk for Continuous Variables (dRisk / dRiskRMD)

Description

Computes disclosure risk measures for continuous key variables, based on interval overlap (dRisk) and Robust Mahalanobis Distance (dRiskRMD). These methods assess whether original records can be re-identified from synthetic data using numeric quasi-identifiers.

Usage

drisk(X, ...)

## S3 method for class 'synth_pair'
drisk(X, ...)

## Default S3 method:
drisk(
  X,
  Y,
  vars = NULL,
  method = c("both", "interval", "rmd"),
  outlier_par = 0.01,
  alpha = 0.05,
  na.rm = TRUE,
  ...
)

Arguments

Details

Both methods work ONLY on numeric variables. Non-numeric columns are automatically excluded (or cause an error if explicitly requested via vars).

Interval method (dRisk): For each variable v, an interval half-width is computed as:

d_v = \mathrm{outlier\_par} \times \mathrm{IQR}(X_v) / (2n)^{1/3}

where n = nrow(X). For each original record i, the record is flagged as at-risk if there exists at least one synthetic record j where ALL variables of record i fall within the intervals around record j:

|x_{iv} - y_{jv}| \le d_v \quad \forall v

The dRisk score is the fraction of original records that are at-risk.

RMD method (dRiskRMD): Uses the Minimum Covariance Determinant (MCD) estimator from cov.rob to obtain a robust covariance matrix from the original data. For each original record, the minimum Robust Mahalanobis Distance to any synthetic record is computed. Records with minimum RMD below the chi-squared threshold \chi^2_{1-\alpha, p} (where p = number of variables) are flagged as at-risk.

Value

An object of class "drisk" containing:

• drisk_interval: fraction of at-risk records (interval method), NA if not computed

• drisk_rmd: fraction of at-risk records (RMD method), NA if not computed

• at_risk_interval: logical vector per original record (interval method), NULL if not computed

• at_risk_rmd: logical vector per original record (RMD method), NULL if not computed

• min_rmd: numeric vector of minimum Robust Mahalanobis Distance per original record, NULL if method does not include "rmd"

• interval_widths: named numeric vector of interval half-widths d_v per variable, NULL if method does not include "interval"

• method: method(s) used

• privacy_pass: logical, max(drisk_interval, drisk_rmd) <= 0.1

• n_original: number of original records used

• n_synthetic: number of synthetic records used

• vars: variables used

• outlier_par: outlier parameter used

• alpha: significance level used

Interpretation

• dRisk close to 0: Low disclosure risk - few original records can be matched by synthetic data within the tolerance intervals.

• dRisk close to 1: High disclosure risk - most original records have close matches in the synthetic data.

• dRiskRMD close to 0: Low risk - original records are far from synthetic records in robust Mahalanobis distance.

• dRiskRMD close to 1: High risk - many original records have suspiciously close synthetic counterparts.

Author(s)

Matthias Templ

References

Templ, M. (2017). Statistical Disclosure Control for Microdata: Methods and Applications in R. Springer. doi:10.1007/978-3-319-50272-4

Templ, M., Kowarik, A. & Meindl, B. (2024). sdcMicro: Statistical Disclosure Control Methods for Anonymization of Data and Risk Estimation. R package. https://CRAN.R-project.org/package=sdcMicro

Rousseeuw, P. J. & Van Driessen, K. (1999). A fast algorithm for the Minimum Covariance Determinant estimator. Technometrics, 41(3), 212–223. doi:10.1080/00401706.1999.10485670

See Also

dcr for distance-based privacy with holdout comparison, nndr for nearest neighbor distance ratio, ims for identical match detection

Other privacy-models: attacker_risk(), delta_presence(), disclosure_report(), domias(), epsilon_identifiability(), individual_risk(), inspect_record(), kanonymity(), ldiversity(), linkability(), merge_per_record(), mia_classifier(), population_uniqueness(), recordLinkage(), risk_by_group(), singling_out(), suda(), tcloseness(), top_at_risk()

Examples

# Create example data
set.seed(123)
n <- 100
X <- data.frame(
  age = rnorm(n, 40, 10),
  income = rnorm(n, 50000, 15000),
  hours = rnorm(n, 40, 8)
)

# Good synthetic data (independent generation)
Y <- data.frame(
  age = rnorm(n, 40, 10),
  income = rnorm(n, 50000, 15000),
  hours = rnorm(n, 40, 8)
)

result <- drisk(X, Y)
print(result)
summary(result)


# Memorized data (Y is copy of X) - should show high risk
Y_copy <- X + rnorm(n * 3, sd = 0.01)
result_bad <- drisk(X, Y_copy)
print(result_bad)

Energy Distance for Multivariate Numeric Data

Description

Computes the energy distance between multivariate numeric distributions in two datasets. Energy distance is a statistical distance that characterizes equality of distributions and is zero if and only if the distributions are identical.

Usage

energy_distance(X, ...)

## S3 method for class 'synth_pair'
energy_distance(X, ...)

## Default S3 method:
energy_distance(
  X,
  Y,
  vars = NULL,
  standardize = TRUE,
  n_sample = 1000,
  na.rm = TRUE,
  seed = NULL,
  ...
)

Arguments

Details

The energy distance between distributions F and G is defined as:

D_E(F, G) = 2E||X - Y|| - E||X - X'|| - E||Y - Y'||

where X, X' are independent samples from F, and Y, Y' are independent samples from G. The energy distance:

• Is always non-negative

• Equals zero if and only if F = G

• Is sensitive to differences in both location and scale

• Does not require density estimation

For synthetic data evaluation, lower energy distance indicates better preservation of the multivariate numeric distribution.

The computation uses Euclidean distances. For large datasets, random sampling is applied to keep computation tractable (O(n^2) complexity).

Value

An object of class "energy_distance" containing:

• energy_distance: the computed energy distance

• energy_distance_normalized: energy distance divided by a reference value

• mean_dist_XY: mean distance between X and Y samples

• mean_dist_XX: mean distance within X samples

• mean_dist_YY: mean distance within Y samples

• n_X, n_Y: sample sizes used in computation

• n_vars: number of variables

• vars: variable names used

• standardized: whether standardization was applied

• utility_score: transformed score (higher = better utility)

Author(s)

Matthias Templ

References

Szekely, G. J. and Rizzo, M. L. (2013). Energy statistics: A class of statistics based on distances. Journal of Statistical Planning and Inference, 143(8), 1249-1272.

Rizzo, M. L. and Szekely, G. J. (2016). Energy Distance. WIREs Computational Statistics, 8(1), 27-38.

See Also

hellinger for categorical distributions, compare_wasserstein for univariate Wasserstein distance, gower for mixed-type data

Other utility: chisq_utility(), ci_proximity(), contingency_fidelity(), copula_fidelity(), gower(), hellinger(), mmd(), mqs(), pMSE(), plot.rumap(), propscore(), regression_fidelity(), rumap(), specks(), subgroup_utility(), tail_fidelity(), tstr()

Examples

set.seed(123)
# Original data
X <- data.frame(
  income = rnorm(500, mean = 50000, sd = 10000),
  age = rnorm(500, mean = 40, sd = 10),
  score = rnorm(500, mean = 100, sd = 15)
)

# Good synthetic data (similar distribution)
Y_good <- data.frame(
  income = rnorm(500, mean = 50000, sd = 10000),
  age = rnorm(500, mean = 40, sd = 10),
  score = rnorm(500, mean = 100, sd = 15)
)

# Poor synthetic data (shifted distribution)
Y_poor <- data.frame(
  income = rnorm(500, mean = 60000, sd = 15000),
  age = rnorm(500, mean = 45, sd = 15),
  score = rnorm(500, mean = 90, sd = 20)
)

result_good <- energy_distance(X, Y_good, seed = 42)
print(result_good)

result_poor <- energy_distance(X, Y_poor, seed = 42)
print(result_poor)

Epsilon Identifiability

Description

Computes the Epsilon Identifiability risk metric for synthetic data, a distance-entropy hybrid from the SynthEval framework. For each synthetic record, the metric finds its weighted Gower distance to the closest original record, where variable weights are the inverse of Shannon entropy (rare attributes are penalized more heavily). Records with minimum distance below the epsilon threshold are flagged as identifiable.

Usage

epsilon_identifiability(X, ...)

## S3 method for class 'synth_pair'
epsilon_identifiability(X, ...)

## Default S3 method:
epsilon_identifiability(
  X,
  Y,
  epsilon = 0.05,
  vars = NULL,
  na.rm = TRUE,
  seed = NULL,
  n_bins = 20L,
  ...
)

Arguments

Details

The Epsilon Identifiability metric combines distance-based and information-theoretic approaches:

1. Compute Shannon entropy H_v for each variable v in the original data. For numeric variables, values are discretized into n_bins equal-width bins. For categorical variables, entropy is computed directly from category frequencies.

2. Compute inverse-entropy weights w_v = 1 / H_v. Variables with zero entropy (constant) receive zero weight. Weights are normalized to sum to 1.

3. Compute weighted Gower distance from each synthetic record to all original records using VIM::gowerD with the entropy-based weights.

4. For each synthetic record, find the minimum weighted distance.

5. Flag records where distance < epsilon.

6. Compute identifiability rate = fraction of flagged records.

The intuition is that variables with low entropy (few distinct values or skewed distributions) carry more identifying power. A close match on a rare-valued variable is more concerning than a close match on a high-entropy variable.

Note: This is the SynthEval distance-entropy variant (Lautrup et al., 2025). It shares the name but not the definition of the probabilistic \epsilon-identifiability of Yoon et al. (2020, ADS-GAN); the two should not be conflated.

Value

An object of class "epsilon_identifiability" containing:

• distances: numeric vector of minimum weighted distances per synthetic record

• flagged: logical vector, TRUE where distance < epsilon

• identifiability_rate: fraction of synthetic records flagged

• entropy_weights: named numeric vector of per-variable weights (normalized, sum to 1)

• entropies: named numeric vector of per-variable Shannon entropies

• epsilon: threshold used

• n_flagged: count of flagged records

• privacy_pass: logical, TRUE if identifiability_rate <= 0.1

• n_original: number of original records

• n_synthetic: number of synthetic records

• vars: variables used

Choosing epsilon

The default epsilon = 0.05 is a conservative threshold. Smaller values are more strict (fewer flagged records). The appropriate threshold depends on the number of variables and the data domain. Use plot(result, which = 1) to visualize the distance distribution and assess whether the threshold is appropriate.

Author(s)

Matthias Templ

References

Lautrup, A., Hyrup, T., Zimek, A. & Blockeel, H. (2025). SynthEval: A Framework for Detailed Utility and Privacy Evaluation of Tabular Synthetic Data. Data Mining and Knowledge Discovery, 39(1). doi:10.1007/s10618-024-01081-4

See Also

dcr for distance to closest record, nndr for nearest neighbor distance ratio, ims for identical match share

Other privacy-models: attacker_risk(), delta_presence(), disclosure_report(), domias(), drisk(), individual_risk(), inspect_record(), kanonymity(), ldiversity(), linkability(), merge_per_record(), mia_classifier(), population_uniqueness(), recordLinkage(), risk_by_group(), singling_out(), suda(), tcloseness(), top_at_risk()

Examples

# Create example data
set.seed(123)
n <- 200
X <- data.frame(
  age = sample(20:70, n, replace = TRUE),
  income = rnorm(n, 50000, 15000),
  gender = sample(c("M", "F"), n, replace = TRUE),
  region = sample(c("N", "S", "E", "W"), n, replace = TRUE)
)

# Good synthetic data (random, no memorization)
Y_good <- data.frame(
  age = sample(20:70, n, replace = TRUE),
  income = rnorm(n, 50000, 15000),
  gender = sample(c("M", "F"), n, replace = TRUE),
  region = sample(c("N", "S", "E", "W"), n, replace = TRUE)
)

result <- epsilon_identifiability(X, Y_good)
print(result)
summary(result)


# Memorized data (Y is a copy of X) - should show high risk
Y_copy <- X[sample(nrow(X), n, replace = TRUE), ]
result_bad <- epsilon_identifiability(X, Y_copy)
print(result_bad)

Evaluation statistics

Description

Several kinds of evaluation statistics (MAPE, MAE, MSE, RMSE, AIT)

Usage

mape(x, y)

mae(x, y)

mse(x, y)

rmse(x, y)

ait(x, y)

Arguments

Value

The MAPE, MAE, MSE, RMSE, and (normalized) AIT (Aitchison distance) between two vectors.

Author(s)

Matthias Templ

See Also

rapid

Examples

x <- rnorm(10)
y <- rnorm(10)
mape(x, y)
mae(x, y)
mse(x, y)
rmse(x, y)
ait(x, y)

Extract comparison data from sdcMicro object

Description

Creates a synth_pair object from an sdcMicro sdcMicroObj.

Usage

from_sdcMicro(x, target_var = NULL, use_weights = TRUE, ...)

Arguments

Details

The sdcMicro package provides statistical disclosure control methods for microdata (local suppression, recoding, microaggregation, PRAM, noise addition, etc.). The sdcMicroObj S4 class stores both the original and anonymized data. This function extracts both into a synth_pair for evaluation with riskutility measures.

Note that sdcMicro produces anonymized data (perturbation of real records), not truly synthetic data. Nonetheless, many risk and utility measures apply equally. In particular, attribution-based measures (CAP family), distance-based measures, and all utility measures can be used to compare original vs. anonymized microdata.

Variable roles are automatically extracted from the sdcMicro object:

• key_vars: Categorical quasi-identifiers (keyVars slot)

• target_var: Sensitive variable (sensibleVar slot, first entry)

• weight_original: Sampling weight (weightVar slot)

• num_vars: Numeric key variables (numVars slot)

The anonymized data is reconstructed via sdcMicro::extractManipData(), which combines manipulated key, numeric, and PRAM variables with unmodified columns from the original data.

Value

An object of class "synth_pair"

See Also

synth_pair, from_synthpop, from_simPop

Examples

if (requireNamespace("sdcMicro", quietly = TRUE)) {
  # Create a simple sdcMicro object
  data("testdata2", package = "sdcMicro")
  sdc <- sdcMicro::createSdcObj(
    dat = testdata2,
    keyVars = c("urbrur", "roof", "walls", "water", "sex"),
    numVars = c("expend", "income", "savings"),
    weightVar = "sampling_weight"
  )
  # Apply some anonymization
  sdc <- sdcMicro::localSuppression(sdc)
  # Create synth_pair
  pair <- from_sdcMicro(sdc)
  print(pair)
}

Extract comparison data from simPop object

Description

Creates a synth_pair object from a simPop simPopObj.

Usage

from_simPop(
  x,
  key_vars = NULL,
  target_var = NULL,
  use_sample_weights = TRUE,
  ...
)

Arguments

Details

The simPop package generates synthetic populations from survey samples. Unlike synthpop which creates synthetic samples, simPop creates a full synthetic population. This has implications for interpretation:

• Original: The survey sample (typically has sampling weights)

• Synthetic: The generated population (typically no weights, or equal weights)

The simPop object stores both the original sample and the synthetic population, so unlike from_synthpop, you don't need to provide the original data.

Value

An object of class "synth_pair"

See Also

synth_pair, from_synthpop

Examples

if (requireNamespace("simPop", quietly = TRUE)) {
  # simPop example would go here
  # Note: simPop has complex dependencies, example kept minimal
}

Extract comparison data from synthpop object

Description

Creates a synth_pair object from a synthpop synds or synds.list object.

Usage

from_synthpop(x, original, key_vars = NULL, target_var = NULL, m = 1, ...)

Arguments

Details

The synthpop package creates synthetic data using sequential modeling. It returns objects of class synds (single synthesis) or synds.list (multiple syntheses). This function extracts the synthetic data and combines it with the original data into a synth_pair for analysis.

Note: synthpop does not store the original data in the output object, so you must provide it explicitly.

Value

An object of class "synth_pair"

See Also

synth_pair, from_simPop

Examples

if (requireNamespace("synthpop", quietly = TRUE)) {
  library(synthpop)

  # Create some example data
  original <- data.frame(
    age = sample(20:60, 100, replace = TRUE),
    sex = factor(sample(c("M", "F"), 100, replace = TRUE)),
    income = rnorm(100, 50000, 10000)
  )

  # Generate synthetic data
  syn_obj <- syn(original, seed = 123)

  # Create synth_pair
  pair <- from_synthpop(syn_obj, original,
                        key_vars = c("age", "sex"),
                        target_var = "income")
  print(pair)
}

Gower distance between two data frames

Description

Mean per-observation Gower distance between an original data frame and a row-corresponding anonymized version (e.g. a perturbative method such as noise addition, microaggregation, or rank swapping, where record i of the anonymized data corresponds to record i of the original). Returns an S3 object of class "gower" with print, summary, and plot methods.

Usage

gower(X, ...)

## S3 method for class 'synth_pair'
gower(X, ...)

## Default S3 method:
gower(X, Y, ...)

Arguments

Details

The two data frames must have the same number of rows; rows are compared pairwise, so the reported gower_distance is \frac{1}{n}\sum_{i=1}^{n} d_G(X_i, Y_i), the mean Gower distance between corresponding records, in [0, 1]. Identical data therefore yields distance 0 and utility_score 1. This measure assumes record correspondence (perturbative anonymization); for fully synthetic data without such correspondence use a distributional measure such as energy_distance, mmd, or compare_wasserstein.

Value

An object of class "gower" containing:

• gower_distance: the average Gower distance (lower = more similar)

• utility_score: 1 - gower_distance, in [0, 1] (higher = better utility)

• n_records: number of observations

• n_variables: number of variables

• n: alias for n_records (for backward compatibility)

• n_vars: alias for n_variables (for backward compatibility)

Author(s)

Matthias Templ. Based on the gowerD function from Alexander Kowarik in the VIM package.

References

Gower, J.C. (1971). A General Coefficient of Similarity and Some of Its Properties. Biometrics, 27(4), 857–871.

See Also

dcr, ims

Other utility: chisq_utility(), ci_proximity(), contingency_fidelity(), copula_fidelity(), energy_distance(), hellinger(), mmd(), mqs(), pMSE(), plot.rumap(), propscore(), regression_fidelity(), rumap(), specks(), subgroup_utility(), tail_fidelity(), tstr()

Examples

# Simple example with mixed data types
X <- data.frame(
  age = c(25, 30, 35, 40),
  income = c(30000, 45000, 50000, 60000),
  gender = factor(c("M", "F", "M", "F"))
)
Y <- data.frame(
  age = c(26, 31, 34, 42),
  income = c(32000, 44000, 52000, 58000),
  gender = factor(c("M", "F", "M", "F"))
)
result <- gower(X, Y)
result
summary(result)

plot(result)

Hellinger Distance for Categorical Distributions

Description

Computes the Hellinger distance between categorical variable distributions in two datasets. The Hellinger distance measures the similarity between probability distributions and is bounded between 0 (identical) and 1 (no overlap).

Usage

hellinger(X, ...)

## S3 method for class 'synth_pair'
hellinger(X, ...)

## Default S3 method:
hellinger(
  X,
  Y,
  vars = NULL,
  weight_X = NULL,
  weight_Y = NULL,
  na.rm = TRUE,
  ...
)

Arguments

Details

The Hellinger distance between two discrete probability distributions P and Q is:

H(P, Q) = \frac{1}{\sqrt{2}} \sqrt{\sum_{i} (\sqrt{p_i} - \sqrt{q_i})^2}

Properties:

• H = 0: Distributions are identical

• H = 1: Distributions have no overlap (disjoint support)

• Symmetric: H(P,Q) = H(Q,P)

• Bounded: 0 <= H <= 1

For utility assessment, lower Hellinger distance indicates better preservation of categorical distributions in synthetic data.

Value

An object of class "hellinger" containing:

• per_variable: data.frame with Hellinger distance for each variable

• hellinger_mean: mean Hellinger distance across all variables

• hellinger_max: maximum Hellinger distance across variables

• n_vars: number of variables compared

• vars: variable names used

• utility_score: 1 - hellinger_mean (higher = better utility)

Author(s)

Matthias Templ

References

Hellinger, E. (1909). Neue Begruendung der Theorie quadratischer Formen von unendlichvielen Veraenderlichen. Journal fuer die reine und angewandte Mathematik, 136, 210-271.

See Also

compare_chisq_gof for chi-squared goodness of fit, energy_distance for multivariate numeric comparison

Other utility: chisq_utility(), ci_proximity(), contingency_fidelity(), copula_fidelity(), energy_distance(), gower(), mmd(), mqs(), pMSE(), plot.rumap(), propscore(), regression_fidelity(), rumap(), specks(), subgroup_utility(), tail_fidelity(), tstr()

Examples

set.seed(123)
# Original data
X <- data.frame(
  gender = sample(c("Male", "Female"), 500, replace = TRUE),
  region = sample(c("North", "South", "East", "West"), 500, replace = TRUE),
  education = sample(c("High", "Medium", "Low"), 500, replace = TRUE)
)

# Good synthetic data (similar distributions)
Y_good <- data.frame(
  gender = sample(c("Male", "Female"), 500, replace = TRUE),
  region = sample(c("North", "South", "East", "West"), 500, replace = TRUE),
  education = sample(c("High", "Medium", "Low"), 500, replace = TRUE)
)

# Poor synthetic data (different distributions)
Y_poor <- data.frame(
  gender = sample(c("Male", "Female"), 500, replace = TRUE, prob = c(0.9, 0.1)),
  region = sample(c("North", "South", "East", "West"), 500, replace = TRUE,
                  prob = c(0.7, 0.1, 0.1, 0.1)),
  education = sample(c("High", "Medium", "Low"), 500, replace = TRUE,
                     prob = c(0.1, 0.1, 0.8))
)

result_good <- hellinger(X, Y_good)
print(result_good)

result_poor <- hellinger(X, Y_poor)
print(result_poor)

Get high-risk records

Description

Extract records with risk above a specified threshold.

Usage

high_risk_records(x, threshold = NULL)

Arguments

Value

data frame with high-risk record indices and their risk values

Hitting Rate

Description

Computes the Hitting Rate privacy metric for synthetic data. The Hitting Rate measures the fraction of synthetic records that fall within a configurable distance threshold of any original record. It bridges the Identical Match Share (IMS, threshold = 0) and the full Distance to Closest Record (DCR) distribution into a single, interpretable scalar.

Usage

hitting_rate(X, ...)

## S3 method for class 'synth_pair'
hitting_rate(X, ...)

## Default S3 method:
hitting_rate(
  X,
  Y,
  threshold = 0.05,
  vars = NULL,
  method = c("gower", "euclidean"),
  na.rm = TRUE,
  ...
)

Arguments

Details

The Hitting Rate provides a threshold-based view of synthetic data privacy. For each synthetic record, the minimum distance to any original record is computed. Records with a minimum distance at or below the threshold are counted as "hits" – near copies that may pose a disclosure risk.

The metric is defined as:

HR(\tau) = \frac{|\{y \in Y : \min_{x \in X} d(x, y) \le \tau\}|}{|Y|}

The hitting rate generalises two existing metrics:

• At threshold = 0, the hitting rate equals the Identical Match Share (ims).

• The full distribution of minimum distances is the same as the DCR training distribution in dcr, but without requiring a holdout set.

Interpretation:

• rate ~ 0: Good privacy – few synthetic records are close to any original record.

• rate > 0.1: Privacy concern – too many near copies.

• rate ~ 1: Severe memorization – nearly all synthetic records are close copies of original records.

The default threshold of 0.05 is appropriate for Gower distances (which are bounded between 0 and 1). For Euclidean distances on normalized data, adjust the threshold according to the dimensionality and scale of the data.

Value

An object of class "hitting_rate" containing:

• rate: fraction of synthetic records within threshold distance of any original record

• min_distances: numeric vector of minimum distances per synthetic record

• hits: logical vector indicating which synthetic records are hits (min_distance <= threshold)

• n_hits: count of hits

• threshold: the threshold used

• privacy_pass: logical, TRUE if rate <= 0.1

• n_original: number of original records

• n_synthetic: number of synthetic records

• method: distance method used

• vars: variables used

• rate_at_zero: fraction of synthetic records with exact matches (min_distance == 0), equivalent to IMS

Author(s)

Matthias Templ

References

Platzer, M. & Reutterer, T. (2021). Holdout-Based Empirical Assessment of Mixed-Type Synthetic Data. Frontiers in Big Data, 4, 679939. doi:10.3389/fdata.2021.679939

See Also

dcr for distance to closest record (with holdout comparison), ims for exact match detection (equivalent to hitting rate at threshold 0), nndr for nearest neighbor distance ratio

Other distance-risk: dcr(), ims(), nnaa(), nndr(), repu(), rf_privacy()

Examples

# Create example data
set.seed(123)
n <- 200
X <- data.frame(
  age = rnorm(n, 40, 10),
  income = rnorm(n, 50000, 15000),
  gender = sample(c("M", "F"), n, replace = TRUE),
  region = sample(c("N", "S", "E", "W"), n, replace = TRUE)
)

# Good synthetic data (random, no memorization)
Y_good <- data.frame(
  age = rnorm(n, 40, 10),
  income = rnorm(n, 50000, 15000),
  gender = sample(c("M", "F"), n, replace = TRUE),
  region = sample(c("N", "S", "E", "W"), n, replace = TRUE)
)

result <- hitting_rate(X, Y_good)
print(result)
summary(result)


# Memorized data (Y is copy of X) -- should show high rate
Y_bad <- X[sample(nrow(X), n, replace = TRUE), ]
result_bad <- hitting_rate(X, Y_bad)
print(result_bad)

# Sweep thresholds visually
plot(result, which = 2)

Identify Pareto-Optimal Solutions

Description

Identify Pareto-Optimal Solutions

Usage

identify_pareto(risk, utility)

Arguments

Value

Logical vector indicating which observations are Pareto-optimal

Identical Match Share (IMS)

Description

Computes the Identical Match Share privacy metric for synthetic data. IMS measures the proportion of synthetic records that are exact copies of training records, indicating potential memorization or data leakage.

Usage

ims(X, ...)

## S3 method for class 'synth_pair'
ims(X, ...)

## Default S3 method:
ims(X, Y, vars = NULL, na.rm = TRUE, ...)

Arguments

Details

For a variant that focuses only on unique (singleton) training records, use repu (Replicated Uniques).

Identical Match Share (IMS) is a straightforward privacy metric that detects exact copies of training records in synthetic data. Even well-designed synthetic data generators can occasionally produce records identical to training data by chance, especially for categorical-heavy datasets.

The metric computes:

IMS = \frac{|\{y \in Y : \exists x \in X, y = x\}|}{|Y|}

Interpretation:

• IMS = 0%: No exact copies - ideal for privacy

• IMS < 1%: Acceptable - likely coincidental matches

• IMS > 1%: Concerning - may indicate memorization

• IMS > 5%: Serious privacy issue - significant copying

For datasets with many categorical variables and few unique combinations, some identical matches may occur by chance. Compare IMS against the expected collision rate for your data characteristics.

Value

An object of class "ims" containing:

• ims: identical match share (proportion of synthetic with exact match)

• ims_pct: IMS as percentage

• n_identical: number of synthetic records with exact match

• identical_idx: indices of identical synthetic records in Y

• identical_records: the actual identical records from Y

• match_counts: number of training matches per identical synthetic record

• n_unique_matched: number of unique training records that were copied

• privacy_pass: logical, TRUE if IMS is acceptably low (< 1%)

• n_synthetic, n_train: dataset sizes

Author(s)

Matthias Templ

References

MOSTLY AI (2024). Synthetic data quality assurance. https://mostly.ai/

See Also

dcr for distance to closest record, nndr for nearest neighbor distance ratio, disco for disclosive records with matching target values

Other distance-risk: dcr(), hitting_rate(), nnaa(), nndr(), repu(), rf_privacy()

Examples

# Create example data
set.seed(123)
X <- data.frame(
  age = sample(20:60, 100, replace = TRUE),
  gender = sample(c("M", "F"), 100, replace = TRUE),
  region = sample(c("N", "S", "E", "W"), 100, replace = TRUE),
  income = sample(c("low", "medium", "high"), 100, replace = TRUE)
)

# Good synthetic data (random)
Y_good <- data.frame(
  age = sample(20:60, 100, replace = TRUE),
  gender = sample(c("M", "F"), 100, replace = TRUE),
  region = sample(c("N", "S", "E", "W"), 100, replace = TRUE),
  income = sample(c("low", "medium", "high"), 100, replace = TRUE)
)

result_good <- ims(X, Y_good)
print(result_good)

# Bad synthetic data (direct copies)
Y_bad <- X[sample(nrow(X), 100, replace = TRUE), ]
result_bad <- ims(X, Y_bad)
print(result_bad)

Individual Re-identification Risk

Description

Estimates individual re-identification risk based on population frequency models. Computes sample-based and model-based risk estimates using log-linear or Poisson regression models for population frequency estimation.

Usage

individual_risk(X, ...)

## S3 method for class 'synth_pair'
individual_risk(X, threshold = 0.1, data = c("synthetic", "original"), ...)

## Default S3 method:
individual_risk(
  X,
  key_vars,
  weight = NULL,
  method = c("both", "sample", "model"),
  threshold = 0.1,
  na.rm = TRUE,
  ...
)

Arguments

Details

Individual risk quantifies the probability that a specific record can be re-identified by an intruder who has access to the quasi-identifiers.

Sample-based Risk: For a record in an equivalence class of size f_k, the sample-based risk is:

r_k = \frac{1}{f_k}

This is a simple estimate assuming uniform probability within each class.

Model-based Risk: When sampling weights are available or for small samples, model-based approaches estimate the population frequency F_k using log-linear models:

r_k = \frac{1}{F_k}

The function fits a Poisson log-linear model to estimate expected frequencies in the population, accounting for the sampling design.

Risk Interpretation:

• Risk = 1: Unique record, definitely identifiable

• Risk > 0.5: High risk, appears in small equivalence class

• Risk > 0.1: Elevated risk, deserves attention

• Risk < 0.05: Generally acceptable for most applications

Value

An object of class "individual_risk" containing:

• risk: vector of individual risk values

• method: method used for risk estimation

• mean_risk: mean risk across all records

• max_risk: maximum individual risk

• n_high_risk: number of records exceeding threshold

• pct_high_risk: percentage of high-risk records

• threshold: risk threshold used

• risk_distribution: summary statistics of risk distribution

• ec_info: equivalence class information

Author(s)

Matthias Templ

References

Skinner, C. J., & Elliot, M. J. (2002). A measure of disclosure risk for microdata. Journal of the Royal Statistical Society: Series B, 64(4), 855-867.

Rinott, Y., & Shlomo, N. (2006). A generalized negative binomial smoothing model for sample disclosure risk estimation. In Privacy in Statistical Databases.

See Also

kanonymity for k-anonymity assessment, suda for Special Uniques Detection, ldiversity for l-diversity assessment

Other privacy-models: attacker_risk(), delta_presence(), disclosure_report(), domias(), drisk(), epsilon_identifiability(), inspect_record(), kanonymity(), ldiversity(), linkability(), merge_per_record(), mia_classifier(), population_uniqueness(), recordLinkage(), risk_by_group(), singling_out(), suda(), tcloseness(), top_at_risk()

Examples

# Create example data
set.seed(123)
data <- data.frame(
  age = sample(c("young", "middle", "old"), 500, replace = TRUE),
  gender = sample(c("M", "F"), 500, replace = TRUE),
  region = sample(c("N", "S", "E", "W"), 500, replace = TRUE),
  income = sample(c("low", "medium", "high"), 500, replace = TRUE)
)

# Compute individual risk
result <- individual_risk(data,
                          key_vars = c("age", "gender", "region"),
                          threshold = 0.2)
print(result)
summary(result)
plot(result)

# With sampling weights
data$weight <- runif(500, 0.5, 2)
result_weighted <- individual_risk(data,
                                   key_vars = c("age", "gender", "region"),
                                   weight = "weight")
print(result_weighted)

Information Surprisal

Description

Computes the self-information (surprisal) of an outcome x, given its probability p(x). Surprisal quantifies how unexpected or informative an event is, and can be used to evaluate privacy risks.

Usage

information_surprisal(p, logbase = 2)

Arguments

Details

In privacy evaluation, information surprisal measures how surprising an adversary would find a specific combination of attributes. The lower the frequency of this combination in the population, the higher the surprisal and the more identifiable the user becomes.

Mathematically, information surprisal is defined as:

I(x) = -\log_2(p(x))

A low probability p(x) results in a high surprisal value, indicating uniqueness or rarity, which may correspond to a higher risk of re-identification.

Value

A numeric vector of surprisal values.

Author(s)

Matthias Templ

References

Wagner, I., & Eckhoff, D. (2018). Technical Privacy Metrics: A Systematic Survey. ACM Computing Surveys, 51(3), Article 57. doi:10.1145/3168389

See Also

Other information-theory: Entropy, EntropyMeasures, max_info_leakage(), mutualInformation(), positive_information_disclosure(), privacy_score(), systemAnonymityLevel()

Examples

# Example 1: Basic surprisal values
probs <- c(0.5, 0.25, 0.125, 0.125)
information_surprisal(probs)

# Example 2: Using with a psychological dataset (e.g., personality profiles)
# Suppose these are relative frequencies of specific personality profiles
profile_freqs <- c("TypeA" = 0.4, "TypeB" = 0.35, "TypeC" = 0.2, "TypeD" = 0.05)
information_surprisal(profile_freqs)

# Example 3: Privacy evaluation - rare attribute combination
# Suppose an individual has a rare attribute combination with p(x) = 0.0001
information_surprisal(0.0001)
# Output: 13.29 bits - high risk due to uniqueness

Inspect a Single Record's Linkage Detail

Description

Returns detailed linkage diagnostics for a single record, including candidate IDs and (optionally) the data for the query record and its candidates. Requires recordLinkage(..., return_matches = TRUE).

Usage

inspect_record(x, ...)

## S3 method for class 'recordLinkageRisk'
inspect_record(x, i, data_orig = NULL, data_anon = NULL, ...)

## S3 method for class 'inspect_record'
print(x, ...)

Arguments

Value

An object of class "inspect_record" containing:

record_id

the record index

risk

re-identification risk

d_rank

rank of the true match among candidates

risk_band

categorical risk band

d_true, d_min

distances for the true match and closest candidate

n_candidates

number of candidates

true_in_set

whether the true match is in the candidate set

candidate_ids

vector of candidate indices

query_record

optional: data for the query record

candidate_records

optional: data for the candidate records

Author(s)

Matthias Templ, Oscar Thees

See Also

Other privacy-models: attacker_risk(), delta_presence(), disclosure_report(), domias(), drisk(), epsilon_identifiability(), individual_risk(), kanonymity(), ldiversity(), linkability(), merge_per_record(), mia_classifier(), population_uniqueness(), recordLinkage(), risk_by_group(), singling_out(), suda(), tcloseness(), top_at_risk()

Interpret pMSE results

Description

Interpret pMSE results

Usage

interpret_pMSE(pMSE_ratio, S_pMSE, SPECKS, PO50)

k-Anonymity Assessment

Description

Measures k-anonymity of a dataset based on specified quasi-identifier (key) variables. A dataset satisfies k-anonymity if every record is indistinguishable from at least k-1 other records with respect to the key variables.

Usage

kanonymity(X, ...)

## S3 method for class 'synth_pair'
kanonymity(X, k = 5, data = c("synthetic", "original"), ...)

## Default S3 method:
kanonymity(X, key_vars, k = 5, na.rm = TRUE, ...)

Arguments

Details

k-Anonymity (Sweeney, 2002) is a privacy model that requires each record to be indistinguishable from at least k-1 other records based on quasi-identifiers.

An equivalence class is the set of all records sharing the same values for all key variables. The k-anonymity level is the size of the smallest equivalence class.

Interpretation:

• k >= 5: Generally considered acceptable for public release

• k >= 3: Minimum for most applications

• k = 1: Unique records exist - high re-identification risk

• k = 2: Some records have only one match - elevated risk

Records in small equivalence classes (size < k) are at higher risk of re-identification and may need additional protection.

Value

An object of class "kanonymity" containing:

• k_level: the achieved k-anonymity level (minimum equivalence class size)

• satisfies_k: logical, whether the data satisfies k-anonymity for given k

• n_violating: number of records in equivalence classes smaller than k

• pct_violating: percentage of records violating k-anonymity

• n_ec: number of equivalence classes

• ec_sizes: table of equivalence class size distribution

• violating_records: indices of records violating k-anonymity

• equivalence_classes: data frame with equivalence class details

• risk_summary: summary of re-identification risk

Author(s)

Matthias Templ

References

Sweeney, L. (2002). k-Anonymity: A Model for Protecting Privacy. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 10(5), 557-570.

See Also

ldiversity for l-diversity assessment, tcloseness for t-closeness assessment, suda for special uniques detection, individual_risk for probabilistic risk assessment

Other privacy-models: attacker_risk(), delta_presence(), disclosure_report(), domias(), drisk(), epsilon_identifiability(), individual_risk(), inspect_record(), ldiversity(), linkability(), merge_per_record(), mia_classifier(), population_uniqueness(), recordLinkage(), risk_by_group(), singling_out(), suda(), tcloseness(), top_at_risk()

Examples

# Create example data
set.seed(123)
data <- data.frame(
  age = sample(c("young", "middle", "old"), 100, replace = TRUE),
  gender = sample(c("M", "F"), 100, replace = TRUE),
  region = sample(c("N", "S", "E", "W"), 100, replace = TRUE),
  income = sample(c("low", "medium", "high"), 100, replace = TRUE)
)

# Check k-anonymity
result <- kanonymity(data, key_vars = c("age", "gender", "region"), k = 5)
print(result)
summary(result)
plot(result)

# Check with more key variables (likely lower k)
result2 <- kanonymity(data, key_vars = c("age", "gender", "region", "income"), k = 3)
print(result2)

l-Diversity Assessment

Description

Measures l-diversity of a dataset, which extends k-anonymity by requiring diversity in sensitive attribute values within each equivalence class. Implements distinct l-diversity, entropy l-diversity, and recursive (c,l)-diversity.

Usage

ldiversity(X, ...)

## S3 method for class 'synth_pair'
ldiversity(X, l = 2, c = 2, data = c("synthetic", "original"), ...)

## Default S3 method:
ldiversity(X, key_vars, sensitive_var, l = 2, c = 2, na.rm = TRUE, ...)

Arguments

Details

l-Diversity (Machanavajjhala et al., 2007) addresses limitations of k-anonymity by requiring diversity in sensitive attributes within equivalence classes.

Distinct l-Diversity: Each equivalence class must have at least l distinct values of the sensitive attribute. The achieved level is the minimum number of distinct values across all equivalence classes.

Entropy l-Diversity: Each equivalence class must have entropy of the sensitive attribute >= log(l). Entropy is computed as: H = -\sum p_i \log(p_i)

Recursive (c,l)-Diversity: The most common sensitive value should not appear too frequently relative to other values. Specifically: r_1 < c(r_l + r_{l+1} + ... + r_m) where r_i is the frequency of the i-th most common value.

Interpretation:

• l >= 3: Good diversity protection

• l = 2: Minimum meaningful diversity

• l = 1: No diversity - sensitive value can be inferred

Value

An object of class "ldiversityRisk" containing:

• distinct_l: achieved distinct l-diversity level

• entropy_l: achieved entropy l-diversity level

• recursive_cl: whether data satisfies recursive (c,l)-diversity

• satisfies_l: logical, whether data satisfies l-diversity for given l

• n_violating_distinct: records in EC with fewer than l distinct values

• n_violating_entropy: records in EC with entropy < log(l)

• per_ec: data frame with per-equivalence-class diversity measures

• key_vars, sensitive_var: input parameters

Author(s)

Matthias Templ

References

Machanavajjhala, A., Kifer, D., Gehrke, J., & Venkitasubramaniam, M. (2007). l-Diversity: Privacy Beyond k-Anonymity. ACM Transactions on Knowledge Discovery from Data, 1(1), Article 3.

See Also

kanonymity for k-anonymity assessment, tcloseness for t-closeness assessment, tcap for targeted correct attribution probability

Other privacy-models: attacker_risk(), delta_presence(), disclosure_report(), domias(), drisk(), epsilon_identifiability(), individual_risk(), inspect_record(), kanonymity(), linkability(), merge_per_record(), mia_classifier(), population_uniqueness(), recordLinkage(), risk_by_group(), singling_out(), suda(), tcloseness(), top_at_risk()