Interface

SingleCellProjections.annotationMethod
SCP.annotation(table, colname) -> Job

Extract the ID column and the column named colname from table, returning a two-column table. Useful for passing annotations to filtering or covariate specification.

source
SingleCellProjections.filter_obsMethod
SCP.filter_obs(fobs, data; kwargs...) -> Job

Filter observations by the predicate fobs. fobs can be:

  • An integer range or vector of indices.
  • A Pair of column name and predicate (e.g. "celltype" => isequal("Monocyte")).
  • An annotation table Job with a predicate.

(TODO: Add an example with predicate. Cannot use the relative_std one.)

See also filter_var, filter_matrix, subset_obs.

source
SingleCellProjections.filter_varMethod
SCP.filter_var(fvar, data; kwargs...) -> Job

Filter variables by the predicate fvar. fvar can be:

  • An integer range or vector of indices (e.g. 1:100).
  • A Pair of column name and predicate (e.g. "name" => >("D")).
  • An annotation table Job with a predicate (e.g. SCP.relative_std(data) => >=(0.1)).

See also filter_obs, filter_matrix, subset_var.

source
SingleCellProjections.find_optimal_coord_transformMethod
SCP.find_optimal_coord_transform(data, group_filters...; kwargs...) -> Job

Find an optimal rotation matrix that aligns data coordinates so that specified cell groups are separated along the principal axes. The first group filter defines the direction of the first axis (up), the second group the second axis, and so on — each is made orthogonal to the preceding axes.

Each group_filter is a Pair of column name and predicate (e.g. "celltype" => isequal("HSC")).

Examples

Rotation of 3D plot:

julia> transform = SCP.find_optimal_coord_transform(fl,
           "celltype"=>isequal("HSC"),
           "celltype"=>isequal("T-cells"),
           "celltype"=>isequal("B-cells"))
julia> fl_rotated = SCP.transform_coords(fl, transform; keep_var=true)

Rotation of 2D plot:

julia> transform = SCP.find_optimal_coord_transform(fl_2d,
           "celltype"=>isequal("HSC"),
           "celltype"=>isequal("T-cells"))
julia> fl_rotated = SCP.transform_coords(fl_2d, transform; keep_var=true)

See also transform_coords, force_layout.

source
SingleCellProjections.force_layoutMethod
SCP.force_layout(data; ndim=3, kwargs...) -> Job

Compute a force-directed layout embedding of data. Returns a DataMatrix with ndim layout dimensions as variables.

Keyword arguments:

  • k — number of nearest neighbors for the graph.
  • k_fraction — alternative to k, specify neighbors as a fraction of observations.
  • niter — number of force simulation iterations (default 100).
  • link_distance, link_strength — link force parameters (defaults 40, 0.05).
  • charge, charge_min_distance, theta — repulsion parameters (defaults 40, 1, 0.9).
  • center_strength — centering force (default 0.05).
  • velocity_decay — velocity damping (default 0.9).
  • initialAlpha, finalAlpha — simulation temperature schedule (defaults 1.0, 1e-3).
  • initialScale — initial coordinate scale (default 10).
  • seed — random seed (default 1234).
  • k_projection — neighbors used when projecting onto this layout (default 10).

Examples

julia> SCP.force_layout(reduced; ndim=3, seed=4567, k=100, k_projection=25)

See also transform_coords, find_optimal_coord_transform, umap, tsne.

source
SingleCellProjections.ftestMethod
SCP.ftest(data, h1; h0=(), center=true, kwargs...) -> Job

Perform an F-test for each variable comparing the full model h1 against the null model h0. Returns a table with test statistics and p-values.

h1 and h0 are covariates specified as column name strings or Pairs of column name and covariate description. The covariate type (categorical/numerical) is normally autodetected. With a single categorical covariate, this is equivalent to a one-way ANOVA.

Keyword arguments:

  • statistic_col="F" / pvalue_col="pValue" - output column names (set to nothing to omit).
  • do_sort=true - sort variables by the F statistic (most significant first).

(TODO: Examples.)

See also ttest, normalize_matrix.

source
SingleCellProjections.load_countsMethod
SCP.load_counts(filenames; sample_names, feature_filenames=nothing, barcode_filenames=nothing, prefilter="feature_type"=>isequal("Gene Expression"), extra_id_cols="feature_type", kwargs...) -> Job

Load raw count matrices from one or more 10x files. Returns a Job whose result is a DataMatrix with genes as variables and cells as observations.

Each file can be a 10x HDF5 (.h5) file, or a CellRanger Matrix Market matrix (.mtx[.gz]). For a .mtx file, the matching feature and barcode files are found in the same folder (following the CellRanger naming convention), or can be given explicitly.

  • sample_names is required and assigns a name to each sample.
  • feature_filenames / barcode_filenames — explicit feature/barcode files (a single filename or a vector matching filenames). When nothing (default), they are guessed from each .mtx filename; for a .h5 file the file itself is used.
  • prefilter selects which features to keep (defaults to Gene Expression only).
  • extra_id_cols specifies additional columns used (together with the first column) to uniquely identify variables when merging samples. Variables with matching ID columns are combined.

Examples

Load a single sample:

julia> SCP.load_counts("SampleA.h5"; sample_names="SampleA")

Load multiple samples:

julia> SCP.load_counts(["SampleA.h5", "SampleB.h5"]; sample_names=["SampleA","SampleB"])

Load from a Matrix Market file (features/barcodes found in the same folder):

julia> SCP.load_counts("matrix.mtx.gz"; sample_names="SampleA")

See also load_csv.

source
SingleCellProjections.load_csvFunction
SCP.load_csv(filepath; kwargs...) -> Job

Load a CSV or TSV file as a table Job. The file path is automatically checksummed for cache invalidation. Requires the CSV package to be loaded.

See also load_counts.

source
SingleCellProjections.load_h5adFunction
SCP.load_h5ad([T], filepath; layer=nothing, obsm=nothing, obsp=nothing, varm=nothing, varp=nothing, kwargs...) -> Job

Load a .h5ad (AnnData) file as a DataMatrix Job. Requires the Muon package to be loaded.

The optional type parameter T determines the eltype of the matrix. If specified, the matrix will be converted (e.g. Int for count matrices stored as floats).

By default, the main matrix X is loaded. Use one of the following mutually exclusive kwargs to load from a different source:

  • layer — a named layer from layers (e.g. "raw_counts")
  • obsm — observation embeddings (e.g. "X_umap"), var is set to synthetic dimension IDs
  • obsp — observation pairwise matrix, both var and obs are set to obs annotations
  • varm — variable embeddings, obs is set to synthetic dimension IDs
  • varp — variable pairwise matrix, both var and obs are set to var annotations

Examples

Load the main matrix X.

julia> SCP.load_h5ad("data.h5ad")

Load raw counts. Note that we want to specify the eltype Int, because h5ad typically stores counts as Float32.

julia> SCP.load_h5ad(Int, "data.h5ad"; layer="raw_counts")

Load a UMAP embedding.

julia> SCP.load_h5ad("data.h5ad"; obsm="X_umap")

See also load_counts, load_csv.

source
SingleCellProjections.loadingsMethod
SCP.loadings(data; nsv, seed=1234, kwargs...) -> Job

Extract PCA loadings from data. Returns a DataMatrix where each column is a loading vector. The loadings are not affected by projection. Uses the same randomized SVD algorithm as SCP.pca and accepts the same keyword arguments (nsv, seed, subspacedims, niter).

Examples

Compute the loadings of normalized for the 100 first principal components. Useful in combination with a call to SCP.pca (with the same parameters).

julia> SCP.loadings(normalized; nsv=100)

See also pca, svd.

source
SingleCellProjections.local_outlier_factorMethod
SCP.local_outlier_factor(data, full; k=10, col="LOF") -> Job

Compute the Local Outlier Factor for each observation in data relative to the full dataset full, using k nearest neighbors. Returns a table with IDs and LOF scores in a column named col.

When projecting, only neighbors in the base dataset are considered.

source
SingleCellProjections.logtransformMethod
SCP.logtransform([T=Float64,] counts; scale_factor=10_000, kwargs...) -> Job

Apply log transformation: log(1 + x * scale_factor / total_counts). Returns a DataMatrix with the transformed matrix. The element type of the resulting matrix is T.

(TODO: Add example.)

See also sctransform, normalize_matrix.

source
SingleCellProjections.mannwhitneyMethod
SCP.mannwhitney(data, column, [group_a, group_b]; h1_missing=:skip, kwargs...) -> Job

Perform a Mann-Whitney U-test (a.k.a. Wilcoxon rank-sum test) between two groups of observations, for each variable. The U statistic is corrected for ties, and p-values are computed using a normal approximation. Returns a table with variable IDs, U statistics and p-values, sorted by significance (see below).

data must contain a sparse matrix. It is recommended to first logtransform (or tf_idf_transform) the raw counts.

column selects a column in data.obs that determines group membership:

  • If neither group_a nor group_b is given, column must have exactly two unique values (ignoring missing).
  • If only group_a is given, observations equal to group_a are compared against all others (ignoring missing).
  • If both are given, observations equal to group_a are compared against those equal to group_b.

Keyword arguments:

  • h1_missing=:skip - :skip excludes missing values in column; :error throws if any are present.
  • statistic_col="U" / pvalue_col="pValue" / z_col=nothing - output column names (set to nothing to omit; z is omitted by default).
  • do_sort=true - sort variables by |z| (most significant first).

Results are sorted by the absolute standardized statistic |z|, where z = (U - n1*n2/2)/σ. This orders variables by significance without the underflow that sorting by pValue suffers (p-values collapse to 0 for strongly-separated variables). The signed z (available via z_col) is monotone with the p-value and also indicates the direction of the effect.

The test is projectable: when projecting onto other data, the group labels resolved here are reused and the test is recomputed on the projected observations.

See also ftest, ttest, logtransform.

source
SingleCellProjections.normalize_matrixMethod
SCP.normalize_matrix(data, covariates...; center=true, kwargs...) -> Job

Normalize data by centering and regressing out covariates. Covariates can be column names (strings) or Pairs of column name and covariate description.

Optional keyword arguments for annotating per-variable statistics:

  • annotate_variance: Set to true to add a column with per-variable variance.
  • annotate_std: Set to true to add a column with per-variable standard deviation.
  • annotate_relative_std: Set to true to add a column with per-variable relative standard deviation.
  • variance_col: Custom name for the variance column.
  • std_col: Custom name for the std column.
  • relative_std_col: Custom name for the relative standard deviation.

Examples

Center transformed data:

julia> SCP.normalize_matrix(transformed)

Center transformed data and regress out the fraction_mt covariate.

julia> SCP.normalize_matrix(transformed, "fraction_mt")

Annotate by relative std:

julia> SCP.normalize_matrix(transformed; annotate_relative_std=true)

Annotate by variance, using a custom name:

julia> SCP.normalize_matrix(transformed; variance_col="my_variance_column")

See also sctransform, logtransform, designmatrix.

source
SingleCellProjections.obs_counts_fractionFunction
SCP.obs_counts_fraction(counts, col, sub_filter, tot_filter=Returns(true); project_ids=:no) -> Job

Compute the fraction of counts from a subset of observations (cells) for each variable, and add it as a new variable annotation column named col.

sub_filter and tot_filter are predicates applied to the variable annotations to select the subset and total gene sets respectively.

See also obs_counts_sum, var_counts_fraction.

source
SingleCellProjections.obs_counts_sumFunction
SCP.obs_counts_sum([f,] counts, col, filter=Returns(true); project_ids=:no) -> Job

Compute the sum of counts (optionally transformed by f) from a filtered subset of observations for each variable, and add it as a new variable annotation column named col.

Examples

For each variable, count the number of cells with a non-zero value.

julia> SCP.obs_counts_sum(!iszero, counts, "nonzero_cell_count")

See also obs_counts_fraction, var_counts_sum.

source
SingleCellProjections.pcaMethod
SCP.pca(data; nsv, seed=1234, kwargs...) -> Job

Compute PCA of data, keeping nsv principal components. Returns a DataMatrix where the variables are the principal components and the observations are unchanged. Uses a randomized SVD algorithm based on Halko, Martinsson, and Tropp (2011).

The returned principal components are scaled by the singular values, to make this an accurate nsv-dimensional approximation of the original data.

Keyword arguments controlling the iterative procedure:

  • seed — random seed for reproducibility.
  • subspacedims — dimension of the random subspace (default 4nsv).
  • niter — number of power iterations (default 3).

Examples

Compute a 100-dimensional PCA of normalized.

julia> SCP.pca(normalized; nsv=100)

See also svd, loadings, normalize_matrix.

source
SingleCellProjections.population_matrixMethod
SCP.population_matrix(obs, obs_covariate1, obs_covariates...; new_var_covariates, kwargs...) -> Job

Create a matrix where each entry is the fraction of cells belonging to each combination of new_var_covariates within each group defined by the observation covariates. The observation covariates define the columns (samples/groups) and new_var_covariates define the rows (e.g. cell type proportions per sample).

(TODO: Add an example.)

See also pseudobulk.

source
SingleCellProjections.projectMethod
SCP.project(onto, old => new, ...; kwargs...) -> Job

Projects a dataset onto another, while replacing old=>new. Multiple replacement pairs can be specified. (TODO: Describe projection properly.)

Examples

Given a force layout Job fl, we here project proj_raw_counts onto that force layout, by replacing raw_counts with proj_raw_counts.

julia> SCP.project(fl, raw_counts=>proj_raw_counts)
source
SingleCellProjections.pseudobulkMethod
SCP.pseudobulk(data, obs_covariate1, obs_covariates...; kwargs...) -> Job

Aggregate single-cell data into pseudobulk by grouping observations according to the specified covariates. Returns a DataMatrix where each column is a pseudobulk sample.

(TODO: Add example.)

See also population_matrix.

source
SingleCellProjections.relative_stdMethod
SCP.relative_std(data; assume_centered, col="relative_std", project=:no) -> Job

Compute the standard deviation of each variable in data relative to the maximum standard deviation, returning a table with IDs and values in [0,1].

Useful for filtering variables: SCP.filter_var(SCP.relative_std(data) => >=(f), data) keeps only variables whose std is at least a fraction f of the highest-std variable.

  • assume_centered (required) must be set to true to confirm that data is mean-centered; the std is computed assuming a mean of zero.
  • col is the name of the annotation column, defaults to "relative_std".
  • project can be :no (default) or :yes. If :no, it will compute the std of the base data set, and if :yes, it will compute the std of the projected data set.

See also variance, std, normalize_matrix.

source
SingleCellProjections.sctransformMethod
SCP.sctransform([T=Float64,] counts; kwargs...) -> Job

Apply SCTransform (variance-stabilizing transformation) to raw count data. Returns a DataMatrix with the transformed matrix. The element type of the resulting matrix is T.

Keyword arguments:

  • var_filter — filter variables used for parameter estimation (default :).
  • min_cells — minimum number of cells with nonzero counts for a variable to be included (default 5).
  • annotate — if true, add SCTransform parameters to var annotations.

Examples

SCTransform a counts data matrix.

julia> SCP.sctransform(counts)

See also logtransform, normalize_matrix.

source
SingleCellProjections.signatureMethod
SCP.signature(data, var_filter, out_col_name; loadings=false, kwargs...) -> Job

Compute a gene signature score for each observation by filtering to genes matching var_filter, normalizing, and extracting the first principal component. Returns a table with IDs and the signature scores in a column named out_col_name.

(TODO: Example.)

See also pca, loadings.

source
SingleCellProjections.stdMethod
SCP.std(data; assume_centered, col="std", project=:no) -> Job

Compute the standard deviation of each variable in data, returning a table with IDs and values.

  • assume_centered (required) must be set to true to confirm that data is mean-centered; the std is computed assuming a mean of zero.
  • col is the name of the annotation column, defaults to "std".
  • project can be :no (default) or :yes. If :no, it will compute the std of the base data set, and if :yes, it will compute the std of the projected data set.

See also variance, relative_std, normalize_matrix.

source
SingleCellProjections.svdMethod
SCP.svd(data; nsv, seed=1234, kwargs...) -> Job

Compute a truncated SVD of data, keeping nsv singular values. Returns a DataMatrix containing the SVD result. Uses a randomized algorithm based on Halko, Martinsson, and Tropp (2011).

Keyword arguments controlling the iterative procedure:

  • seed — random seed for reproducibility.
  • subspacedims — dimension of the random subspace (default 4nsv).
  • niter — number of power iterations (default 3).

See also pca, loadings.

source
SingleCellProjections.tf_idf_transformMethod
SCP.tf_idf_transform([T=Float64,] counts; scale_factor=10_000, annotate=false, kwargs...) -> Job

Apply the TF-IDF (term frequency-inverse document frequency) transform to raw count data. Returns a DataMatrix with the transformed matrix. The element type of the resulting matrix is T.

The transform is log(1 + scale_factor * tf * idf), where the term frequency is tf = counts ./ max.(1, sum(counts; dims=1)) and the inverse document frequency is idf = nobs ./ max.(1, sum(counts; dims=2)).

idf is estimated from counts and stored in the model, so that projecting onto another dataset reuses it (remapping to the projected variables by ID) rather than recomputing.

Keyword arguments:

  • scale_factor — term-frequency scale factor (default 10_000).
  • annotate — if true, add the idf vector as a var annotation.

See also logtransform, sctransform, normalize_matrix.

source
SingleCellProjections.transfer_annotationMethod
SCP.transfer_annotation(base, new, covariate; k, kwargs...) -> Job

Transfer cell annotations from base to new using kNN-based label transfer. The covariate specifies which annotation column to transfer. k is the number of nearest neighbors used for voting.

Returns a table with the transferred labels and confidence scores.

(TODO: Add example - maybe I need to construct one? It should be about celltype transfer.)

source
SingleCellProjections.transform_annotationMethod
SCP.transform_annotation(f, table; kwargs...) -> Job

Apply function f element-wise to the value column of table, returning a new table with transformed values. The table must have exactly two columns (ID and value). Use new_name to rename the value column.

(TODO: Example.)

source
SingleCellProjections.tsneFunction
SCP.tsne(data; ndim=3, kwargs...) -> Job

Compute a t-SNE embedding of data with ndim dimensions. Returns a DataMatrix with t-SNE dimensions as variables. Requires the TSne package to be loaded.

Additional keyword arguments (max_iter, perplexity, etc.) are forwarded to TSne.tsne.

See also force_layout, umap.

source
SingleCellProjections.ttestMethod
SCP.ttest(data, h1; h0=(), center=true, kwargs...) -> Job

Perform a t-test for each variable testing the effect of h1 while controlling for h0. Returns a table with test statistics and p-values. h1 must be a numerical covariate or a two-group covariate.

Keyword arguments:

  • statistic_col="t" / pvalue_col="pValue" / difference_col="difference" - output column names (set to nothing to omit).
  • do_sort=true - sort variables by |t| (most significant first).

(TODO: Examples.)

See also ftest, normalize_matrix, twogroup_covariate.

source
SingleCellProjections.umapFunction
SCP.umap(data; ndim, seed=1234, kwargs...) -> Job

Compute a UMAP embedding of data with ndim dimensions. Returns a DataMatrix with UMAP dimensions as variables. Requires the UMAP package to be loaded.

seed is used to reset the global RNG for reproducibility, but results may still vary across runs due to threading differences in the UMAP nearest neighbor search.

Additional keyword arguments are forwarded to UMAP.fit.

See also force_layout, tsne.

source
SingleCellProjections.var_counts_fractionFunction
SCP.var_counts_fraction(counts, col, sub_filter, tot_filter=Returns(true); project_ids=:intersect) -> Job

Compute the fraction of counts from a subset of variables (genes) for each observation, and add it as a new observation annotation column named col.

sub_filter and tot_filter are predicates applied to the variable annotations to select the subset and total gene sets respectively.

Examples

Count the fraction of reads that come from Mitochondrial genes.

julia> SCP.var_counts_fraction(counts, "fraction_mt", "name"=>startswith("MT-"))

See also var_counts_sum, obs_counts_fraction.

source
SingleCellProjections.var_counts_sumFunction
SCP.var_counts_sum([f,] counts, col, filter=Returns(true); project_ids=:intersect) -> Job

Compute the sum of counts (optionally transformed by f) from a filtered subset of variables for each observation, and add it as a new observation annotation column named col.

Examples

Let counts be the raw counts.

To count the total number of reads in each cell:

julia> SCP.var_counts_sum(counts, "total_RNA_count")

To count the number of genes that have a non-zero value:

julia> SCP.var_counts_sum(!iszero, counts, "nonzero_RNA_count")

See also var_counts_fraction, obs_counts_sum, load_counts.

source
SingleCellProjections.varianceMethod
SCP.variance(data; assume_centered, col="variance", project=:no) -> Job

Compute the variance of each variable in data, returning a table with IDs and variances.

  • assume_centered (required) must be set to true to confirm that data is mean-centered; the variance is computed assuming a mean of zero.
  • col is the name of the annotation column, defaults to "variance".
  • project can be :no (default) or :yes. If :no, it will compute the variance of the base data set, and if :yes, it will compute the variance of the projected data set.

See also std, relative_std, normalize_matrix.

source