Package: Rfmalloc 0.1.0

Rfmalloc: Out-of-Core Arrays with Pluggable Codecs and Compute Backends

Computes on arrays that do not fit in memory. Vectors, matrices and tensors are allocated inside a memory-mapped file through R's ALTREP interface, backed by a patched copy of the 'fmalloc' allocator, so available RAM becomes a speed gradient rather than a hard limit. Two plugin registries sit on that substrate: a codec registry deciding how bytes are stored, from lossless floating-point and sparse encodings to compressed and quantized blocks, and a matrix-multiply backend registry deciding how products are computed, defaulting to whatever 'BLAS' the R build links against. Matrix products stream in bounded panels and release the pages they have consumed, so operands larger than memory still multiply. A registered backend may decline any product, in which case the default path answers it, so results never depend on which backend is selected. Also provides persistent and scratch runtimes, durable reference-based serialization, explicit vector lifecycle management, multiple runtime handles per session, and out-of-core reductions and principal components for genomics-scale matrices.

Authors:Sounkou Mahamane Toure [aut, cre], Kenichi Yasukata [cph], Wolfram Gloger [cph], Free Software Foundation, Inc. [cph]

Rfmalloc_0.1.0.tar.gz
Rfmalloc_0.1.0.zip(r-4.7)Rfmalloc_0.1.0.zip(r-4.6)Rfmalloc_0.1.0.zip(r-4.5)
Rfmalloc_0.1.0.tgz(r-4.6-x86_64)Rfmalloc_0.1.0.tgz(r-4.6-arm64)Rfmalloc_0.1.0.tgz(r-4.5-x86_64)Rfmalloc_0.1.0.tgz(r-4.5-arm64)
Rfmalloc_0.1.0.tar.gz(r-4.7-arm64)Rfmalloc_0.1.0.tar.gz(r-4.7-x86_64)Rfmalloc_0.1.0.tar.gz(r-4.6-arm64)Rfmalloc_0.1.0.tar.gz(r-4.6-x86_64)
Rfmalloc_0.1.0.tgz(r-4.6-emscripten)
manual.pdf |manual.html
DESCRIPTION |NEWS
card.svg |card.png
Rfmalloc/json (API)

# Install 'Rfmalloc' in R:
install.packages('Rfmalloc', repos = c('https://sounkou-bioinfo.r-universe.dev', 'https://cloud.r-project.org'))

Bug tracker:https://github.com/sounkou-bioinfo/rfmalloc/issues

Pkgdown/docs site:https://sounkou-bioinfo.github.io

Uses libs:
  • c++– GNU Standard C++ Library v3

On CRAN:

Conda:

altrepblasgenomicsggmlggufllmllm-inferencemallocmemory-mappedmmapout-of-corequantizationsimdcpp

4.69 score 4 stars 3 packages 45 scripts 58 exports 0 dependencies

Last updated from:6e05ad0361. Checks:13 OK. Indexed: yes.

TargetResultTimeFilesSyslog
linux-devel-arm64OK125
linux-devel-x86_64OK125
source / vignettesOK185
linux-release-arm64OK123
linux-release-x86_64OK176
macos-release-arm64OK74
macos-release-x86_64OK207
macos-oldrel-arm64OK78
macos-oldrel-x86_64OK215
windows-develOK133
windows-releaseOK117
windows-oldrelOK104
wasm-releaseOK103

Exports:as_fmalloc_arrayas_fmalloc_data_frameas_fmalloc_matrixas_fmalloc_tensorcleanup_fmalloccolMeanscolSumscreate_fmalloc_arraycreate_fmalloc_data_framecreate_fmalloc_haplotypescreate_fmalloc_matrixcreate_fmalloc_tensorcreate_fmalloc_vectordestroy_fmalloc_vectordiagnose_fmalloc_runtimefmalloc_addfmalloc_bedfmalloc_bed_standardizefmalloc_colVarsfmalloc_crossprod_oocfmalloc_default_runtimefmalloc_divfmalloc_dosagefmalloc_dosage_standardizefmalloc_fillfmalloc_hap_materializefmalloc_haplotypesfmalloc_ldfmalloc_matmul_backendfmalloc_matmul_backendsfmalloc_matmul_oocfmalloc_mulfmalloc_pcafmalloc_rowVarsfmalloc_runtimefmalloc_runtime_infofmalloc_setfmalloc_storage_advisefmalloc_subfmalloc_syncfmalloc_tcrossprod_oocfmalloc_tensor_codecsfmalloc_tensor_dtypefmalloc_tensor_materializefmalloc_vector_infofmalloc_vector_lengthfmalloc_vector_payload_ptrfmalloc_vector_typeinit_fmallocis_fmalloc_runtimeis_fmalloc_vectorld_colld_ncolld_pairlist_fmalloc_allocationsopen_fmallocrowMeansrowSums

Dependencies:

Readme and manuals

Help Manual

Help pageTopics
Rfmalloc: Memory-Mapped File Allocation for RRfmalloc-package Rfmalloc
Convert a vector to fmalloc array metadataas_fmalloc_array
Convert to data.frame for fmalloc vectorsas_fmalloc_data_frame
Convert a vector to fmalloc matrix metadataas_fmalloc_matrix
Clean Up fmalloccleanup_fmalloc
Create Array Using fmalloccreate_fmalloc_array
Construct data.frame from fmalloc columnscreate_fmalloc_data_frame
Create Matrix Using fmalloccreate_fmalloc_matrix
Create Vector Using fmalloccreate_fmalloc_vector
Explicitly destroy a fmalloc vectordestroy_fmalloc_vector
Diagnose fmalloc runtime statediagnose_fmalloc_runtime
Public Rfmalloc API helpersfmalloc_api fmalloc_default_runtime fmalloc_runtime fmalloc_runtime_info fmalloc_vector_info fmalloc_vector_length fmalloc_vector_payload_ptr fmalloc_vector_type is_fmalloc_runtime is_fmalloc_vector
Pluggable matrix-multiply backendfmalloc_backend fmalloc_matmul_backend fmalloc_matmul_backends
PLINK 1 genotypes as a 2-bit fmalloc tensorfmalloc_bed
Bake per-variant standardization into a bed tensorfmalloc_bed_standardize
Column / row variances of an fmalloc matrixfmalloc_colVars fmalloc_rowVars
Fractional genotype dosages as a 1-byte fmalloc tensorfmalloc_dosage
Bake per-variant standardization into a dosage tensorfmalloc_dosage_standardize
Adapt a bit-packed haplotype store to a 0/1 matrixfmalloc_hap_materialize
Phased haplotypes as a 1-bit fmalloc storecreate_fmalloc_haplotypes dim.fmalloc_haplotypes fmalloc_haplotypes print.fmalloc_haplotypes
In-place (by-reference) mutation of fmalloc vectorsfmalloc_add fmalloc_div fmalloc_fill fmalloc_insitu fmalloc_mul fmalloc_set fmalloc_sub
Banded LD (correlation) matrix as a compressed fmalloc storedim.fmalloc_ld fmalloc_ld print.fmalloc_ld
Matrix algebra for fmalloc-backed vectors and matrices%*%.fmalloc crossprod.fmalloc fmalloc_linalg matrixOps.fmalloc tcrossprod.fmalloc
Out-of-core matrix product for fmalloc matrices larger than RAMfmalloc_crossprod_ooc fmalloc_matmul_ooc fmalloc_tcrossprod_ooc
Out-of-core PCA / truncated SVD for fmalloc matricesfmalloc_pca
Matrix reduction helpers for fmalloc-backed matricescolMeans colSums fmalloc_reduction_methods rowMeans rowSums
Give the pager an access hint for typed storagefmalloc_storage_advise
Flush an fmalloc runtime's backing store to diskfmalloc_sync
Typed fmalloc tensors%*%.fmalloc_tensor as_fmalloc_tensor create_fmalloc_tensor crossprod.fmalloc_tensor dim.fmalloc_tensor fmalloc_tensor fmalloc_tensor_codecs fmalloc_tensor_dtype fmalloc_tensor_materialize matrixOps.fmalloc_tensor print.fmalloc_tensor tcrossprod.fmalloc_tensor
Initialize fmallocinit_fmalloc
Neighbour run of one column of a banded LD storeld_col
Number of variants in a banded LD storeld_ncol
Correlation between two variants of a banded LD storeld_pair
List Persistent fmalloc Allocationslist_fmalloc_allocations
Open an fmalloc Runtimeopen_fmalloc