Welcome to the DeepIRTools documentation!

DeepIRTools is a small Pytorch-based Python package that uses scalable deep learning methods to fit a number of different confirmatory and exploratory latent factors models, with a particular focus on item response theory (IRT) models. Graphics processing unit (GPU) support is available to speed up some computations.

Description

Latent factor models reduce the dimensionality of data by converting a large number of discrete or continuous observed variables (called items) into a smaller number of continuous unobserved variables (called latent factors), potentially making the data easier to understand. Latent factor models for discrete items are called item response theory (IRT) models.

Traditional maximum likelihood (ML) estimation methods for IRT models are computationally intensive when the sample size, the number of items, and the number of latent factors are all large. This issue can be avoided by approximating the ML estimator using an importance-weighted amortized variational estimator (I-WAVE) from the field of deep learning (for details, see Urban and Bauer, 2021). As an estimation byproduct, I-WAVE allows researchers to compute approximate factor scores and log-likelihoods for any observation – even new observations that were not used for model fitting.

DeepIRTools’ main functionality is the stand-alone IWAVE class contained in the iwave module. This class includes fit(), scores(), and log_likelihood() methods for fitting a latent factor model and for computing approximate factor scores and log-likelihoods for the fitted model.

The following (multidimensional) latent factor models are currently available…

• …for binary and ordinal items:

  • Graded response model

  • Generalized partial credit model

  • Nominal response model

• …for continuous items:

  • Normal (linear) factor model

  • Lognormal factor model

• …for count data:

  • Poisson factor model

  • Negative binomial factor model

DeepIRTools supports mixing item types, handling missing completely at random data, and predicting the mean of the latent factors with covariates (i.e., latent regression modeling); all models are estimable in both confirmatory and exploratory contexts. In the confirmatory context, constraints on the factor loadings, intercepts, and factor covariance matrix are implemented by providing appropriate arguments to fit(). In the exploratory context, the screeplot() function in the figures module may help identify the number of latent factors underlying the data.

Requirements

• Python 3.7 or higher

• torch

• pyro-ppl

• numpy

Installation

To install the latest version:

pip install deepirtools

Citation

To cite DeepIRTools in publications, use:

• Urban, C. J., & He, S. (2022). DeepIRTools: Deep learning-based estimation and inference for item response theory models. Python package. https://github.com/cjurban/deepirtools

To cite the method, use:

• Urban, C. J., & Bauer, D. J. (2021). A deep learning algorithm for high-dimensional exploratory item factor analysis. Psychometrika, 86 (1), 1–29. https://link.springer.com/article/10.1007/s11336-021-09748-3

and/or:

• Urban, C. J. (2021). Machine learning-based estimation and goodness-of-fit for large-scale confirmatory item factor analysis (Publication No. 28772217) [Master’s thesis, University of North Carolina at Chapel Hill]. ProQuest Dissertations Publishing. https://www.proquest.com/docview/2618877227/21C6C467D6194C1DPQ/

BibTeX entries for LaTeX users are:

@Manual{DeepIRTools,
     title = {{D}eep{IRT}ools: {D}eep learning-based estimation and inference for item response theory models},
     author = {Urban, Christopher J. and He, Shara},
     year = {2022},
     note = {Python package},
     url = {https://github.com/cjurban/deepirtools},
}

@article{UrbanBauer2021,
    author = {Urban, Christopher J. and Bauer, Daniel J.},
    year={2021},
    title={{A} deep learning algorithm for high-dimensional exploratory item factor analysis},
    journal = {Psychometrika},
    volume = {86},
    number = {1},
    pages = {1--29}
}

@phdthesis{Urban2021,
    author  = {Urban, Christopher J.},
    title   = {{M}achine learning-based estimation and goodness-of-fit for large-scale confirmatory item factor analysis},
    publisher = {ProQuest Dissertations Publishing},
    school  = {University of North Carolina at Chapel Hill},
    year    = {2021},
    type    = {Master's thesis},
}

deepirtools package

deepirtoools.iwave Module

class deepirtools.iwave.IWAVE(model_type: Union[str, List[str]], learning_rate: float = 0.001, device: str = 'cpu', gradient_estimator: str = 'dreg', log_interval: int = 100, verbose: bool = True, n_intervals: int = 100, **model_kwargs)

Importance-weighted amortized variational estimator (I-WAVE). 1 2

Parameters

• model_type (str or list of str)) –

  Measurement model type.

  Can either be a string if all items have same type or a list of strings specifying each item type.

  Let \(y_j\) be the response to item \(j\), \(\boldsymbol{x}\) be a \(\text{latent_size} \times 1\) vector of latent factors, and \(\boldsymbol{\beta}_j\) be a \(\text{latent_size} \times 1\) vector of factor loadings for item \(j\). Current measurement model options are:

  • "grm", graded response model:

    \[\begin{split}\begin{split} &\text{Pr}(y_j = k \mid \mathbf{x}) \\ &= \begin{cases} 1 - \sigma(\alpha_{j,k + 1} + \boldsymbol{\beta}_j^\top \mathbf{x}), & \text{if $k = 0$} \\ \sigma(\alpha_{j,k} + \boldsymbol{\beta}_j^\top \mathbf{x}) - \sigma(\alpha_{j,k+1} + \boldsymbol{\beta}_j^\top \mathbf{x}), & \text{if $k \in \{1, \ldots, K_j - 2\}$},\\ \sigma(\alpha_{j,k} + \boldsymbol{\beta}_j^\top \mathbf{x}), & \text{if $k = K_j - 1$}, \end{cases} \end{split}\end{split}\]

    where \(\alpha_{j, k}\) is the \(k^\text{th}\) category intercept for item \(j\), \(K_j\) is the number of response categories for item \(j\), and \(\sigma(z) = 1 / (1 + \exp[-z])\) is the inverse logistic link function.

  • "gpcm", generalized partial credit model:

    \[\text{Pr}(y_j = k - 1 \mid \boldsymbol{x}) = \frac{\exp\big[(k - 1)\boldsymbol{\beta}_j^\top \boldsymbol{x} + \sum_{\ell = 1}^k \alpha_{j, \ell} \big]}{\sum_{m = 1}^{K_j} \exp \big[ (m - 1) \boldsymbol{\beta}_j^\top\boldsymbol{x} + \sum_{\ell = 1}^m \alpha_{j, \ell} \big]},\]

    where \(k = 1, \ldots, K_j\) and \(\alpha_{j, k}\) is the \(k^\text{th}\) category intercept for item \(j\).

  • "nominal", nominal response model:

    \[\text{Pr}(y_j = k \mid \boldsymbol{x}) = \frac{\exp[\mathbb{1}(k \neq 0) (\alpha_{j, k} + \boldsymbol{\beta}_j^\top \boldsymbol{x}) ]}{1 + \sum_{\ell = 1}^{K_j} \exp [ \alpha_{j, \ell} + \boldsymbol{\beta}_j^\top \boldsymbol{x} ]},\]

    where \(k = 0, \ldots, K_j - 1\), \(\mathbb{1}(\cdot)\) is the indicator function, and \(\alpha_{j, k}\) is the \(k^\text{th}\) category intercept for item \(j\).

  • "poisson", poisson factor model:

    \[y_j \mid \boldsymbol{x} \sim \text{Pois}(\exp[\boldsymbol{\beta}_j^\top\boldsymbol{x} + \alpha_j]),\]

    where \(y_j \in \{ 0, 1, 2, \ldots \}\) and \(\alpha_j\) is the intercept for item \(j\).

  • "negative_binomial", negative binomial factor model:

    \[y_j \mid \boldsymbol{x} \sim \text{NB}(\exp[\boldsymbol{\beta}_j^\top\boldsymbol{x} + \alpha_j], p),\]

    where \(y_j \in \{ 0, 1, 2, \ldots \}\), \(\alpha_j\) is the intercept for item \(j\), and \(p\) is a success probability.

  • "normal", normal factor model:

    \[y_j \mid \boldsymbol{x} \sim \mathcal{N}(\boldsymbol{\beta}_j^\top\boldsymbol{x} + \alpha_j, \sigma_j^2),\]

    where \(y_j \in (-\infty, \infty)\), \(\alpha_j\) is the intercept for item \(j\), and \(\sigma_j^2\) is the residual variance for item \(j\).

  • "lognormal", lognormal factor model:

    \[\ln y_j \mid \boldsymbol{x} \sim \mathcal{N}(\boldsymbol{\beta}_j^\top\boldsymbol{x} + \alpha_j, \sigma_j^2),\]

    where \(y_j > 0\) and \(\alpha_j\) is the intercept for item \(j\).

• learning_rate (float, default = 0.001) –

  Step size for stochastic gradient optimizer.

  This is the main hyperparameter that may require tuning. Decreasing it typically improves optimization stability at the cost of increased fitting time.

• device (str, default = "cpu") –

  Computing device used for fitting.

  Current options are:

  • "cpu", central processing unit; and

  • "cuda", graphics processing unit.

• gradient_estimator (str, default = "dreg") –

  Gradient estimator for inference model parameters.

  Current options are:

  • "dreg", doubly reparameterized gradient estimator; and

  • "iwae", standard gradient estimator.

  "dreg" is the recommended option due to its bounded variance as the number of importance-weighted samples tends to infinity.

• log_interval (str, default = 100) – Number of mini-batches between printed updates during fitting.

• verbose (bool, default = True) – Whether to print updates during fitting.

• n_intervals (str, default = 100) – Number of 100-mini-batch intervals after which fitting is terminated if best average loss does not improve.

• latent_size (int) – Number of latent factors.

• n_cats (list of int and None, optional) –

  Number of response categories for each item.

  Only needed if some items are categorical. Any continuous items or counts are indicated with None.

  For example, setting n_cats = [3, 3, None, 2] indicates that items 1–2 are categorical with 3 categories, item 3 is continuous, and item 4 is categorical with 2 categories.

• n_items (int, optional) –

  Number of items.

  Only specified if all items are continuous. Not needed if n_cats is specified instead.

• inference_net_sizes (list of int, default = [100]) –

  Neural network inference model hidden layer dimensions.

  For example, setting inference_net_sizes = [100, 100] creates a neural network inference model with two hidden layers of size 100.

• fixed_variances (bool, default = True) –

  Whether to constrain variances of latent factors to one.

  Only applicable when use_spline_prior = False.

• fixed_means (bool, default = True) –

  Whether to constrain means of latent factors to zero.

  Only applicable when use_spline_prior = False.

• correlated_factors (list of int, default = []) –

  Which latent factors should be correlated.

  Only applicable when use_spline_prior = False.

  For example, setting correlated_factors = [0, 3, 4] in a model with 5 latent factors models the correlations between the first, fourth, and fifth factors while constraining the other correlations to zero.

• covariate_size (int, default = None) –

  Number of covariates for latent regression.

  Setting covariate_size > 0 when use_spline_prior = False models the distribution of the latent factors as:3 4

  \[\boldsymbol{x} \mid \boldsymbol{z} \sim \mathcal{N}(\boldsymbol{\Gamma}^\top \boldsymbol{z}, \boldsymbol{\Sigma}),\]

  where \(\boldsymbol{\Gamma}\) is a \(\text{latent_size} \times \text{covariate_size}\) matrix of regression weights, \(\boldsymbol{z}\) is a \(\text{covariate_size} \times 1\) vector of covariates, and \(\boldsymbol{\Sigma}\) is a \(\text{latent_size} \times \text{latent_size}\) factor covariance matrix.

• Q (Tensor, default = None) –

  Binary matrix indicating measurement structure.

  A \(\text{n_items} \times \text{latent_size}\) matrix. Elements of \(\mathbf{Q}\) are zero if the corresponding loading is set to zero and one otherwise:

  \[\beta_{j,d} = q_{j,d} \beta_{j,d}',\]

  where \(\beta_{j,d}\) is the loading for item \(j\) on factor \(d\), \(q_{j,d} \in \{0, 1\}\) is an element of \(\mathbf{Q}\), and \(\beta_{j,d}'\) is an unconstrained loading.

• A (Tensor, default = None) –

  Matrix imposing linear constraints on loadings.

  Linear constraints are imposed as follows:

  \[\boldsymbol{\beta} = \boldsymbol{b} + \boldsymbol{A} \boldsymbol{\beta}',\]

  where \(\boldsymbol{\beta} = (\beta_{1, 1}, \ldots, \beta_{\text{n_items}, 1}, \ldots, \beta_{1, \text{latent_size}}, \ldots, \beta_{\text{n_items}, \text{latent_size}})^\top\) is a \((\text{latent_size} \cdot \text{n_items}) \times 1\) vector of constrained loadings values, \(\boldsymbol{b}\) is a \((\text{latent_size} \cdot \text{n_items}) \times 1\) vector of constants, \(\boldsymbol{A}\) is a \((\text{latent_size} \cdot \text{n_items}) \times (\text{latent_size} \cdot \text{n_items})\) matrix of constants, and \(\boldsymbol{\beta}' = (\beta_{1, 1}', \ldots, \beta_{\text{n_items}, 1}', \ldots, \beta_{1, \text{latent_size}}', \ldots, \beta_{\text{n_items}, \text{latent_size}}')^\top\) is a \((\text{latent_size} \cdot \text{n_items}) \times 1\) vector of unconstrained loadings.

• b (Tensor, default = None) –

  Vector imposing linear constraints on loadings.

  See above for elaboration on linear constraints.

• ints_mask (Tensor, default = None) –

  Binary vector constraining specific intercepts to zero.

  A length \(\text{n_items}\) vector. For categorical items, only the smallest category intercept is constrained to zero.

• use_spline_prior (bool, default = False) – Whether to use spline/spline coupling prior distribution for the latent factors. 5 6

• spline_init_normal (bool, default = True) – Whether to initialize spline/spline coupling prior close to a standard normal distribution.

• flow_length (int, default = 2) –

  Number of spline coupling flow layers.

  Only applicable when latent_size > 1.

• count_bins (int, optional) – Number of segments for each spline transformation.

• bound (float, optional) – Quantity determining the bounding box of each spline transformation.

• spline_net_sizes (list of int, optional) –

  Hidden layer dimensions for neural network mapping from covariates to base density for spline/spline coupling prior.

  Corresponds to a more flexible version of the latent regression model.

loadings

Factor loadings matrix.

A \(\text{n_items} \times \text{latent_size}\) matrix.

Type

Tensor

intercepts

Intercepts.

When all items are continuous, a length \(\text{n_items}\) vector. When some items are graded (i.e., ordinal), a \(\text{n_items} \times M\) matrix where \(M\) is the maximum number of response categories across all items.

Type

Tensor

residual_std

Residual standard deviations.

A length \(\text{n_items}\) vector. Only applicable to normal and lognormal factor models.

Type

Tensor or None

probs

Success probabilities for Bernoulli trials.

A length \(\text{n_items}\) vector. Only applicable to negative binomial factor models.

Type

Tensor or None

cov

Factor covariance matrix.

A \(\text{latent_size} \times \text{latent_size}\) matrix. Only applicable when use_spline_prior = False.

Type

Tensor or None

mean

Factor mean vector.

A length \(\text{latent_size}\) vector. Only applicable when use_spline_prior = False.

Type

Tensor or None

latent_regression_weight

Latent regression weight matrix.

A \(\text{latent_size} \times \text{covariate_size}\). Only applicable to latent regression models.

Type

Tensor or None

grad_estimator

Gradient estimator for inference model parameters.

Type

str

device

Computing device used for fitting.

Type

str

verbose

Whether to print updates during fitting.

Type

bool

global_iter

Number of mini-batches processed during fitting.

Type

int

timerecords

Stores run times for various processes (e.g., fitting).

Type

dict

References

1(1,2)

Urban, C. J., & Bauer, D. J. (2021). A deep learning algorithm for high-dimensional exploratory item factor analysis. Psychometrika, 86 (1), 1–29. https://link.springer.com/article/10.1007/s11336-021-09748-3

2

Urban, C. J. (2021). Machine learning-based estimation and goodness-of-fit for large-scale confirmatory item factor analysis (Publication No. 28772217) [Master’s thesis, University of North Carolina at Chapel Hill]. ProQuest Dissertations Publishing. https://www.proquest.com/docview/2618877227/21C6C467D6194C1DPQ/

3

Camilli, G., & Fox, J.-P. (2015). An aggregate IRT procedure for exploratory factor analysis. Journal of Educational and Behavioral Statistics, 40, 377–401.

4

von Davier, M., & Sinharay, S. (2010). Stochastic approximation methods for latent regression item response models. Journal of Educational and Behavioral Statistics, 35 (2), 174–193.

5

Durkan, C., Bekasov, A., Murray, I., & Papamakarios, G. (2019). Neural spline flows. Advances in Neural Information Processing Systems, 32. https://papers.nips.cc/paper/2019/hash/7ac71d433f282034e088473244df8c02-Abstract.html

6

Dolatabadi, H. M., Erfani, S., & Leckie, C. (2020). Invertible generative modeling using linear rational splines. Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics, 108, 4236–4246. http://proceedings.mlr.press/v108/dolatabadi20a

7

Cremer, C., Morris, Q., & Duvenaud, D. (2017). Reinterpreting importance-weighted autoencoders. In 5th International Conference on Learning Representations. ICLR. https://arxiv.org/abs/1704.02916.

Examples

>>> import deepirtools
>>> from deepirtools import IWAVE
>>> import torch
>>> deepirtools.manual_seed(123)
>>> data = deepirtools.load_grm()["data"]
>>> n_items = data.shape[1]
>>> model = IWAVE(
...     model_type = "grm",
...     latent_size = 4,
...     n_cats = [3] * n_items,
...     Q = torch.block_diag(*[torch.ones([3, 1])] * 4),
...     correlated_factors = [0, 1, 2, 3],
... )
Initializing model parameters
Initialization ended in  0.0  seconds
>>> model.fit(data, iw_samples = 10)
Fitting started
Epoch =      100 Iter. =    25201 Cur. loss =   10.68   Intervals no change =  100
Fitting ended in  109.23  seconds
>>> model.loadings
tensor([[0.8295, 0.0000, 0.0000, 0.0000],
        [0.5793, 0.0000, 0.0000, 0.0000],
        [0.7116, 0.0000, 0.0000, 0.0000],
        [0.0000, 0.7005, 0.0000, 0.0000],
        [0.0000, 1.1687, 0.0000, 0.0000],
        [0.0000, 1.2890, 0.0000, 0.0000],
        [0.0000, 0.0000, 0.9268, 0.0000],
        [0.0000, 0.0000, 1.2653, 0.0000],
        [0.0000, 0.0000, 1.5622, 0.0000],
        [0.0000, 0.0000, 0.0000, 1.0346],
        [0.0000, 0.0000, 0.0000, 1.3641],
        [0.0000, 0.0000, 0.0000, 1.1348]])
>>> model.intercepts
tensor([[ 2.4245, -0.1637],
        [ 1.8219, -1.0013],
        [ 2.0811, -1.1320],
        [ 0.0948, -1.7253],
        [ 2.6597, -2.3412],
        [ 0.2610, -1.4938],
        [ 2.8196, -1.3281],
        [ 0.4833, -2.8053],
        [ 1.6395, -2.2220],
        [ 1.3482, -1.8870],
        [ 2.1606, -2.8600],
        [ 2.5318, -0.1333]])
>>> model.cov
tensor([[ 1.0000, -0.0737,  0.2130,  0.2993],
        [-0.0737,  1.0000, -0.1206, -0.3031],
        [ 0.2130, -0.1206,  1.0000,  0.1190],
        [ 0.2993, -0.3031,  0.1190,  1.0000]])
>>>  model.log_likelihood(data)
Computing approx. LL
Approx. LL computed in 33.27 seconds
Out[6]: -85961.69088745117
>>> model.scores(data)
tensor([[-0.6211,  0.1301, -0.7207, -0.7485],
        [ 0.2189, -0.2649,  0.0109, -0.2363],
        [ 0.0544,  0.9308,  0.7940, -0.8851],
        ...,
        [-0.2964, -0.9597, -0.8885, -0.0057],
        [-1.6015,  0.9812,  0.0486,  0.1773],
        [ 2.0448,  0.0583,  1.2005, -0.9317]])

fit(data: Tensor, batch_size: int = 32, missing_mask: Optional[Tensor] = None, covariates: Optional[Tensor] = None, max_epochs: int = 100000, **model_kwargs)

Fit model to a data set.

Parameters

• data (Tensor) –

  Data set.

  A \(\text{sample_size} \times \text{n_items}\) matrix.

• batch_size (int, default = 32) – Mini-batch size for stochastic gradient optimizer.

• missing_mask (Tensor, default = None) –

  Binary mask indicating missing item responses.

  A \(\text{sample_size} \times \text{n_items}\) matrix.

• covariates (Tensor, default = None) –

  Matrix of covariates.

  A \(\text{sample_size} \times \text{covariate_size}\) matrix .

• max_epochs (int, default = 100000) – Number of passes through the full data set after which fitting should be terminated if convergence not achieved.

• mc_samples (int, default = 1) –

  Number of Monte Carlo samples.

  Increasing this decreases the log-likelihood estimator’s variance.

• iw_samples (int, default = 5000) –

  Number of importance-weighted samples.

  Increasing this decreases the log-likelihood estimator’s bias.

load_model(model_name: str, load_path: str)

Load fitted model.

The initialized model should have the same hyperparameter settings as the fitted model that is being loaded (e.g., the same number of latent variables).

Parameters

• model_name (str) – Name of fitted model.

• load_path (str) – Where to load fitted model from.

log_likelihood(data: Tensor, missing_mask: Optional[Tensor] = None, covariates: Optional[Tensor] = None, mc_samples: int = 1, iw_samples: int = 5000)

Approximate log-likelihood of a data set.

Parameters

• data (Tensor) –

  Data set.

  A \(\text{sample_size} \times \text{n_items}\) matrix.

• missing_mask (Tensor, default = None) –

  Binary mask indicating missing item responses.

  A \(\text{sample_size} \times \text{n_items}\) matrix.

• covariates (Tensor, default = None) –

  Matrix of covariates.

  A \(\text{sample_size} \times \text{covariate_size}\) matrix.

• mc_samples (int, default = 1) –

  Number of Monte Carlo samples.

  Increasing this decreases the log-likelihood estimator’s variance.

• iw_samples (int, default = 5000) –

  Number of importance-weighted samples.

  Increasing this decreases the log-likelihood estimator’s bias.

Returns

ll – Approximate log-likelihood of the data set.

Return type

float

sample(sample_size: int, covariates: Optional[Tensor] = None, return_scores: bool = False)

Sample from model.

Parameters

• sample_size (int) – Number of observations to sample.

• covariates (Tensor, default = None) –

  Matrix of covariates.

  A \(\text{sample_size} \times \text{covariate_size}\) matrix.

• return_scores (bool, default = False) – Whether to reture sampled factor scores in addition to item responses.

Returns

samples – Sampled observations and factor scores.

Return type

dict

save_model(model_name: str, save_path: str)

Save fitted model.

Parameters

• model_name (str) – Name for fitted model.

• save_path (str) – Where to save fitted model.

scores(data: Tensor, missing_mask: Optional[Tensor] = None, covariates: Optional[Tensor] = None, mc_samples: int = 1, iw_samples: int = 5000, return_mu: bool = True)

Approximate expected a posteriori (EAP) factor scores given a data set.

Parameters

• data (Tensor) –

  Data set.

  A \(\text{sample_size} \times \text{n_items}\) matrix.

• missing_mask (Tensor, default = None) –

  Binary mask indicating missing item responses.

  A \(\text{sample_size} \times \text{n_items}\) matrix.

• covariates (Tensor, default = None) –

  Matrix of covariates.

  A \(\text{sample_size} \times \text{covariate_size}\) matrix.

• mc_samples (int, default = 1) –

  Number of Monte Carlo samples.

  Increasing this decreases the EAP estimator’s variance.

• iw_samples (int, default = 5000) –

  Number of importance-weighted samples.

  Increasing this decreases the EAP estimator’s bias. When iw_samples > 1, samples are drawn from the expected importance-weighted distribution using sampling- importance-resampling. 7

• return_mu (bool, default = True) –

  Whether to return exact approximate EAP scores (i.e., approximate EAP scores with no sampling error).

  Equivalent to using an infinite number of Monte Carlo samples. Only applicable when iw_samples = 1.

Returns

factor_scores – Approximate EAP factor scores given the data set.

A \(\text{sample_size} \times \text{latent_size}\) matrix.

Return type

Tensor

deepirtoools.data Module

deepirtools.data.load_grm()

Return data-generating parameters and sampled data for a graded response model.

The generating model has four correlated latent factors and twelve items with three categories each.

Returns

res – A dictionary containing sampled data and data-generating parameters. The returned dictionary includes the following key-value pairs:

• "data", the sampled data set of item responses;

• "loadings", the data-generating factor loadings;

• "intercepts", the data-generating category intercepts;

• "cov_mat", the data-generating factor covariance matrix; and

• "factor_scores", the sampled factor scores.

Return type

dict

deepirtoools.figures Module

deepirtools.figures.loadings_heatmap(loadings: Tensor, xlab: str = 'Factor', ylab: str = 'Item', title: str = 'Factor Loadings')

Make heatmap of factor loadings.

Result is saved as a PDF in the working directory.

Parameters

• loadings (Tensor) – Factor loadings matrix.

• xlab (str, default = "Factor") – Heatmap x-axis label.

• ylab (str, default = "Item") – Heatmap y-axis label.

• title (str, default = "Factor Loadings") – Heatmap title.

deepirtools.figures.screeplot(latent_sizes: List[int], data: Tensor, model_type: Union[str, List[str]], test_size: float, inference_net_sizes_list: Optional[List[List[int]]] = None, learning_rates: Optional[List[float]] = None, missing_mask: Optional[Tensor] = None, max_epochs: int = 100000, batch_size: int = 32, gradient_estimator: str = 'dreg', device: str = 'cpu', log_interval: int = 100, mc_samples_fit: int = 1, iw_samples_fit: int = 1, mc_samples_ll: int = 1, iw_samples_ll: int = 5000, random_seed: int = 1, xlab: str = 'Number of Factors', ylab: str = 'Predicted Approximate Negative Log-Likelihood', title: str = 'Approximate Log-Likelihood Scree Plot', **model_kwargs)

Make a log-likelihood screeplot. 1

Useful in the exploratory setting to detect the number of latent factors. Result is saved as a PDF in the working directory.

Parameters

• latent_sizes (list of int) – Latent dimensions to plot.

• data (Tensor) –

  Data set.

  A \(\text{sample_size} \times \text{n_items}\) matrix.

• model_type (str or list of str) –

  Measurement model type.

  Can either be a string if all items have same type or a list of strings specifying each item type.

  Current options are:

  • "grm", graded response model;

  • "gpcm", generalized partial credit model;

  • "nominal", nominal response model;

  • "poisson", poisson factor model;

  • "negative_binomial", negative binomial factor model;

  • "normal", normal factor model; and

  • "lognormal", lognormal factor model.

  See IWAVE class documentation for further details regarding each model type.

• test_size (float) – Proportion of data used for calculating LL. Range of values is \((0, 1)\).

• inference_net_sizes_list (list of list of int, default = None) –

  Neural net hidden layer sizes for each latent dimension in the screeplot.

  For example, when making a screeplot for three latent dimensions, setting inference_net_sizes_list = [[150, 75], [150, 75], [150, 75]] creates three neural nets each with two hidden layers of size 150 and 75, respectively.

  The default inference_net_sizes_list = None sets each neural net to have a single hidden layer of size 100.

• learning_rates (list of float, default = None) –

  Step sizes for stochastic gradient optimizers.

  The default learning_rates = None sets each optimizer’s step size to 0.001.

• missing_mask (Tensor, default = None) –

  Binary mask indicating missing item responses.

  A \(\text{sample_size} \times \text{n_items}\) matrix.

• max_epochs (int, default = None) – Number of passes through the full data set after which fitting should be terminated if convergence not achieved.

• batch_size (int, default = 32) – Mini-batch size for stochastic gradient optimizer.

• gradient_estimator (str, default = "dreg") –

  Gradient estimator for inference model parameters.

  Current options are:

  • "dreg", doubly reparameterized gradient estimator; and

  • "iwae", standard gradient estimator.

  "dreg" is the recommended option due to its bounded variance as the number of importance-weighted samples increases.

• device (int, default = "cpu") – Computing device used for fitting.

• log_interval (str, default = 100) – Frequency of updates printed during fitting.

• mc_samples_fit (int, default = 1) – Number of Monte Carlo samples for fitting.

• iw_samples_fit (int, default = 1) – Number of importance-weighted samples for fitting.

• mc_samples_ll (int, default = 1) – Number of Monte Carlo samples for calculating approximate log-likelihoods.

• iw_samples_ll (int, default = 5000) – Number of importance-weighted samples for calculating approximate log-likelihoods.

• random_seed (int, default = 1) – Seed for reproducibility.

• xlab (str, default = "Number of Factors") – Screeplot x-axis label.

• ylab (str, default = "Predicted Approximate Negative Log-Likelihood") – Screeplot y-axis label.

• title (str, default = "Approximate Log-Likelihood Scree Plot") – Screeplot title.

• **model_kwargs – Additional keyword arguments passed to IWAVE.__init__().

Returns

ll_list – List of approximate hold-out set log-likelihoods for each latent dimension.

Return type

list of float

Examples

>>> import deepirtools
>>> from deepirtools import screeplot
>>> deepirtools.manual_seed(123)
>>> data = deepirtools.load_grm()["data"]
>>> n_items = data.shape[1]
>>> screeplot(
...     latent_sizes = [2, 3, 4, 5, 6],
...     data = data,
...     model_type = "grm",
...     test_size = 0.1,
...     iw_samples_fit = 10,
...     n_cats = [3] * n_items,
... )
Latent size =  2

Initializing model parameters
Initialization ended in  0.0  seconds

Fitting started
Epoch =       79 Iter. =    17801 Cur. loss =   11.78   Intervals no change =  100
Fitting ended in  43.68  seconds

Computing approx. LL
Approx. LL computed in 2.02 seconds

Latent size =  3

Initializing model parameters
Initialization ended in  0.0  seconds

Fitting started
Epoch =      150 Iter. =    33901 Cur. loss =   10.66   Intervals no change =  100
Fitting ended in  73.85  seconds

Computing approx. LL
Approx. LL computed in 2.48 seconds

Latent size =  4

Initializing model parameters
Initialization ended in  0.0  seconds

Fitting started
Epoch =       97 Iter. =    22001 Cur. loss =   10.69   Intervals no change =  100
Fitting ended in  48.14  seconds

Computing approx. LL
Approx. LL computed in 2.54 seconds

Latent size =  5

Initializing model parameters
Initialization ended in  0.0  seconds

Fitting started
Epoch =       65 Iter. =    14801 Cur. loss =   10.23   Intervals no change =  100
Fitting ended in  32.25  seconds

Computing approx. LL
Approx. LL computed in 2.53 seconds

Latent size =  6

Initializing model parameters
Initialization ended in  0.0  seconds

Fitting started
Epoch =       87 Iter. =    19701 Cur. loss =   10.85   Intervals no change =  100
Fitting ended in  47.94  seconds

Computing approx. LL
Approx. LL computed in 2.37 seconds

[-8626.66552734375,
 -8601.913513183594,
 -8603.8798828125,
 -8604.689666748047,
 -8603.455841064453]

deepirtools.utils Module

deepirtools.utils.invert_cov(cov: Tensor, mat: Tensor)

Flip factor covariances according to loadings signs.

Parameters

• cov (Tensor) – Factor covariance matrix.

• mat (Tensor) – Loadings matrix.

Returns

inverted_cov – Inverted factor covariance matrix.

Return type

Tensor

deepirtools.utils.invert_factors(mat: Tensor)

For each factor, flip sign if sum of loadings is negative.

Parameters

mat (Tensor) – Loadings matrix.

Returns

inverted_mat – Inverted loadings matrix.

Return type

Tensor

deepirtools.utils.invert_latent_regression_weight(weight: Tensor, mat: Tensor)

Flip latent regression weights according to loadings signs.

Parameters

• weight (Tensor) – Latent regression weight matrix.

• mat (Tensor) – Loadings matrix.

Returns

inverted_weight – Inverted latent regression weight matrix.

Return type

Tensor

deepirtools.utils.invert_mean(mean: Tensor, mat: Tensor)

Flip factor means according to loadings signs.

Parameters

• mean (Tensor) – Factor mean vector.

• mat (Tensor) – Loadings matrix.

Returns

inverted_mean – Inverted factor mean vector.

Return type

Tensor

deepirtools.utils.manual_seed(seed: int)

Set random seeds to ensure reproducible results.

deepirtools.utils.normalize_ints(ints: Tensor, mat: Tensor, n_cats: List[int])

Convert intercepts to normal ogive metric (only for IRT models).

Parameters

• ints (Tensor) – Intercepts vector.

• mat (Tensor) – Loadings matrix.

• n_cats (list of int) – Number of categories for each item.

Returns

normalized_ints – Normalized intercepts vector.

Return type

Tensor

deepirtools.utils.normalize_loadings(mat: Tensor)

Convert loadings to normal ogive metric (only for IRT models).

Parameters

mat (Tensor) – Loadings matrix.

Returns

normalized_mat – Normalized loadings matrix.

Return type

Tensor

Indices and tables

• Index

• Search Page