Introduction
The primary aim of the dreval package is to evaluate and
compare one or more reduced dimension representations, in terms of how
well they retain the structure from a reference representation. Here, we
illustrate its application to a single-cell RNA-seq data set of
peripheral blood mononuclear cells (PBMCs), representing a subset of the
pbmc3k data set from the TENxPBMCData package.
The set of genes has been subset to only highly variable ones, and
several dimension reduction methods (PCA, tSNE and UMAP) have been
applied.
data(pbmc3ksub)
pbmc3ksub
#> class: SingleCellExperiment
#> dim: 1804 2700
#> metadata(1): log.exprs.offset
#> assays(2): counts logcounts
#> rownames(1804): LYZ S100A9 ... AGTRAP ATXN1L
#> rowData names(3): ENSEMBL_ID Symbol_TENx Symbol
#> colnames(2700): Cell1 Cell2 ... Cell2699 Cell2700
#> colData names(11): Sample Barcode ... Individual Date_published
#> reducedDimNames(8): PCA PCA_k2 ... UMAP_nn15 UMAP_nn30
#> mainExpName: NULL
#> altExpNames(0):Evaluation
The dreval() function is the main function in the
package, and calculates several metrics comparing each of the reduced
dimension representations in the SingleCellExperiment
object to the designed reference representation. Here, we use the
underlying normalized and log-transformed data contained in the
logcounts assay as the reference representation. For a list
of the calculated metrics, see the help page for
dreval().
Result visualization
The output of dreval() is a list with two elements. The
plots entry contains several diagnostic plot objects, while
the scores entry contains the calculated scores for each of
the evaluated reduced dimension representations. These can be summarized
visually with the plotRankSummary() function.
dre$scores
#> Method dimensionality SpearmanCorrDist PearsonCorrDist KSStatDist
#> D...1 PCA 25 0.8154994 0.8573328 0.3444569
#> D...2 PCA_k2 2 0.6847185 0.6703615 0.5003768
#> D...3 TSNE_perp30 2 0.6529265 0.5634928 0.3798717
#> D...4 TSNE_perp5 2 0.4319733 0.3580134 0.3879359
#> D...5 TSNE_perp15 2 0.6082011 0.5531568 0.3824449
#> D...6 TSNE_perp100 2 0.6530342 0.5303586 0.4009860
#> D...7 UMAP_nn15 2 0.6295305 0.5729313 0.4337395
#> D...8 UMAP_nn30 2 0.6392410 0.5452490 0.4644088
#> EuclDistBetweenDists SammonStress Trustworthiness_k10 Continuity_k10
#> D...1 0.2689303 0.07391334 0.9078543 0.8846762
#> D...2 0.5018734 0.26715742 0.8079315 0.8337003
#> D...3 0.3986876 0.16723525 0.8485094 0.8323802
#> D...4 0.4113424 0.17301868 0.8266943 0.7934712
#> D...5 0.3934171 0.16283086 0.8430795 0.8150423
#> D...6 0.4569962 0.22048310 0.8560838 0.8381717
#> D...7 0.5211990 0.29443922 0.8326147 0.8223748
#> D...8 0.5683829 0.34864929 0.8358089 0.8377692
#> Trustworthiness_k75 Continuity_k75 MeanJaccard_k10 MeanJaccard_k75
#> D...1 0.9188758 0.9266787 0.1705789 0.4113410
#> D...2 0.8509984 0.8861510 0.1101979 0.3206625
#> D...3 0.8523844 0.8797668 0.1385658 0.3234358
#> D...4 0.7956347 0.8046975 0.1276278 0.2714622
#> D...5 0.8486944 0.8622888 0.1353894 0.3186285
#> D...6 0.8618369 0.8924744 0.1371832 0.3381405
#> D...7 0.8537153 0.8733586 0.1266815 0.3208645
#> D...8 0.8616588 0.8900690 0.1278621 0.3275634
#> coRankingQlocal coRankingQglobal KmaxLCMC
#> D...1 0.3592266 0.2536750 115
#> D...2 0.2630986 0.1954727 122
#> D...3 0.2767429 0.1874120 120
#> D...4 0.2171358 0.1246179 106
#> D...5 0.2637424 0.1737807 114
#> D...6 0.2854750 0.1997314 116
#> D...7 0.2660368 0.1817096 116
#> D...8 0.2793017 0.1948339 120
suppressPackageStartupMessages({
library(ggplot2)
})
cowplot::plot_grid(plotlist = lapply(dre$plots$disthex, function(w) w +
theme(axis.title = element_text(size = 7))), ncol = 2)
#> Warning: `stat(density)` was deprecated in ggplot2 3.4.0.
#> ℹ Please use `after_stat(density)` instead.
#> ℹ The deprecated feature was likely used in the dreval package.
#> Please report the issue at <https://github.com/csoneson/dreval/issues>.
#> This warning is displayed once every 8 hours.
#> Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
#> generated.
plotRankSummary(dre$scores) +
theme(axis.text.x = element_text(angle = 45, hjust = 1, vjust = 1))
Using a reduced dimension representation as the reference space
In the example above, we used the original feature space (the
logcounts assay) as the reference space, to which all
reduced dimension representations were compared. It is also possible to
use one of the reduced dimension representations as the reference space,
as we will illustrate here.
dre <- dreval(
sce = pbmc3ksub, refType = "dimred", refDimRed = "PCA",
dimReds = NULL, nSamples = 500, kTM = c(10, 75),
verbose = FALSE, distNorm = "l2"
)
dre$scores
#> Method dimensionality SpearmanCorrDist PearsonCorrDist KSStatDist
#> D...1 PCA_k2 2 0.9233915 0.8942498 0.2644088
#> D...2 TSNE_perp30 2 0.7675047 0.7094667 0.1252665
#> D...3 TSNE_perp5 2 0.5943282 0.5488952 0.1078958
#> D...4 TSNE_perp15 2 0.7263592 0.6940593 0.1089699
#> D...5 TSNE_perp100 2 0.7851810 0.7114814 0.2111102
#> D...6 UMAP_nn15 2 0.7984050 0.7682800 0.3160000
#> D...7 UMAP_nn30 2 0.8475167 0.7620543 0.3751743
#> EuclDistBetweenDists SammonStress Trustworthiness_k10 Continuity_k10
#> D...1 0.2897892 0.1138349 0.9010931 0.9417259
#> D...2 0.3186966 0.1071415 0.9591690 0.9607034
#> D...3 0.3812950 0.1500976 0.9401556 0.9386072
#> D...4 0.3224079 0.1130083 0.9538675 0.9540698
#> D...5 0.3555046 0.1353942 0.9630918 0.9671030
#> D...6 0.3808133 0.1843372 0.9508854 0.9564619
#> D...7 0.4192053 0.2220888 0.9535897 0.9631501
#> Trustworthiness_k75 Continuity_k75 MeanJaccard_k10 MeanJaccard_k75
#> D...1 0.9403377 0.9574120 0.1821428 0.4777051
#> D...2 0.9379067 0.9463695 0.3427420 0.5432202
#> D...3 0.8737818 0.9015667 0.2960875 0.4513358
#> D...4 0.9259113 0.9379746 0.3246281 0.5226296
#> D...5 0.9495374 0.9564119 0.3597027 0.5686795
#> D...6 0.9429501 0.9511481 0.3181976 0.5445045
#> D...7 0.9443877 0.9551933 0.3204885 0.5482587
#> coRankingQlocal coRankingQglobal KmaxLCMC
#> D...1 0.4315576 0.2709721 137
#> D...2 0.4883456 0.2697993 52
#> D...3 0.4176202 0.2046238 50
#> D...4 0.4683484 0.2539875 52
#> D...5 0.5039833 0.2913327 50
#> D...6 0.4683770 0.2737537 53
#> D...7 0.4691012 0.2962409 50
cowplot::plot_grid(plotlist = lapply(dre$plots$disthex, function(w) w +
theme(axis.title = element_text(size = 7))), ncol = 2)
plotRankSummary(dre$scores) +
theme(axis.text.x = element_text(angle = 45, hjust = 1, vjust = 1))
Computational efficiency
dreval relies heavily on the calculation of pairwise
distances among samples, and is consequently computationally demanding
for large data sets. The dreval() function allows for
subsampling a smaller number of samples, as well as selecting a subset
of the variables, to speed up calculations and reduce the memory
footprint. Here we illustrate that for the example data set, the
calculated metrics are stable under subsampling of cells.
suppressPackageStartupMessages({
library(dplyr)
library(tidyr)
library(ggplot2)
})
set.seed(1)
nS <- c(100, 400, 1000, 1500, 2700)
v <- lapply(nS, function(n) {
if (n == 2700) nR <- 1
else nR <- 2
lapply(seq_len(nR), function(i) {
message(n, " samples, replicate ", i)
dreval(sce = pbmc3ksub, refAssay = "logcounts", dimReds = NULL,
nSamples = n, kTM = 50, verbose = FALSE)$scores %>%
dplyr::mutate(nSamples = n, replicate = i)
})
})
vv <- do.call(dplyr::bind_rows, v)
vv <- vv %>%
tidyr::gather(
key = "measure", value = "value",
-Method, -dimensionality, -nSamples, -replicate
)
suppressWarnings(
ggplot(vv, aes(x = nSamples, y = value, color = Method)) +
geom_point(size = 3) + facet_wrap(~ measure, scales = "free_y") +
geom_smooth(se = FALSE) + theme_bw()
)Session info
sessionInfo()
#> R version 4.5.1 (2025-06-13)
#> Platform: aarch64-apple-darwin20
#> Running under: macOS Sonoma 14.7.6
#>
#> Matrix products: default
#> BLAS: /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRblas.0.dylib
#> LAPACK: /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRlapack.dylib; LAPACK version 3.12.1
#>
#> locale:
#> [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
#>
#> time zone: UTC
#> tzcode source: internal
#>
#> attached base packages:
#> [1] stats graphics grDevices utils datasets methods base
#>
#> other attached packages:
#> [1] ggplot2_3.5.2 dreval_0.1.6
#>
#> loaded via a namespace (and not attached):
#> [1] SummarizedExperiment_1.38.1 gtable_0.3.6
#> [3] xfun_0.52 bslib_0.9.0
#> [5] sparsesvd_0.2-2 Biobase_2.68.0
#> [7] lattice_0.22-7 vctrs_0.6.5
#> [9] tools_4.5.1 generics_0.1.4
#> [11] stats4_4.5.1 tibble_3.3.0
#> [13] cluster_2.1.8.1 pkgconfig_2.0.3
#> [15] Matrix_1.7-3 RColorBrewer_1.1-3
#> [17] desc_1.4.3 S4Vectors_0.46.0
#> [19] lifecycle_1.0.4 GenomeInfoDbData_1.2.14
#> [21] compiler_4.5.1 farver_2.1.2
#> [23] iotools_0.3-5 textshaping_1.0.1
#> [25] coRanking_0.2.5 GenomeInfoDb_1.44.0
#> [27] htmltools_0.5.8.1 sass_0.4.10
#> [29] yaml_2.3.10 hexbin_1.28.5
#> [31] pillar_1.11.0 pkgdown_2.1.3.9000
#> [33] crayon_1.5.3 jquerylib_0.1.4
#> [35] tidyr_1.3.1 MASS_7.3-65
#> [37] SingleCellExperiment_1.30.1 DelayedArray_0.34.1
#> [39] cachem_1.1.0 abind_1.4-8
#> [41] tidyselect_1.2.1 digest_0.6.37
#> [43] dplyr_1.1.4 purrr_1.1.0
#> [45] labeling_0.4.3 cowplot_1.2.0
#> [47] fastmap_1.2.0 grid_4.5.1
#> [49] cli_3.6.5 SparseArray_1.8.0
#> [51] magrittr_2.0.3 S4Arrays_1.8.1
#> [53] withr_3.0.2 UCSC.utils_1.4.0
#> [55] scales_1.4.0 rmarkdown_2.29
#> [57] XVector_0.48.0 httr_1.4.7
#> [59] matrixStats_1.5.0 ragg_1.4.0
#> [61] evaluate_1.0.4 knitr_1.50
#> [63] GenomicRanges_1.60.0 IRanges_2.42.0
#> [65] rlang_1.1.6 Rcpp_1.1.0
#> [67] wordspace_0.2-8 glue_1.8.0
#> [69] BiocGenerics_0.54.0 jsonlite_2.0.0
#> [71] R6_2.6.1 MatrixGenerics_1.20.0
#> [73] systemfonts_1.2.3 fs_1.6.6