Memory issue

[ianchlee]

Hi,

I have been trying to use scTenifoldNet to build a comparison on a model, however I kept receiving an error saying Error: cannot allocate vector of size 24.6 Gb. I have tried to down sample it so many times and increasing the future.globals.maxSize parameters to no avail. Is this an optimisation issue or server issue? Below is my pseudocode I tried to run which produces the error, with X = 18172 * 13, Y = 18172 * 12 (which is in the same dimension to my real data).

nCores = parallelly::availableCores()

print(paste0("Available cores = ", nCores))
# Available cores = 16

options(future.globals.maxSize = 900 * 1024^4)

nCells = 13
nGenes = 18172

set.seed(1)
X <- rnbinom(n = nGenes * nCells, size = 20, prob = 0.98)
X <- round(X)
X <- matrix(X, ncol = nCells)
rownames(X) <- c(paste0('ng', 1:(18172-10)), paste0('mt-', 1:10))

# Generating a perturbed network modifying the expression of genes 10, 2 and 3
Y <- X[,-1]
Y[10,] <- Y[50,]
Y[2,] <- Y[11,]
Y[3,] <- Y[5,]

output <- scTenifoldNet(X = X, Y = Y, qc = FALSE,
                        nc_nNet = 10, nc_nCells = 10,
                        td_K = 3, 
                        nCores = nCores)

I've tried to run it as a batch job with 1TB memory allocated to it already.