Package: Rllm 0.1.0
Rllm: Native Quantized Matrix Products and LLM Inference over 'Rfmalloc' Tensors
The composition layer of the 'Rfmalloc' ecosystem: it registers 'Rggml' (a vendored 'GGML' build with runtime-SIMD-dispatched quantized kernels) as a codec-aware matrix-multiply backend for 'Rfmalloc', so that products of file-backed, quantized tensors run natively in quantized space (weights stay 'Q4_K'/'Q6_K'/... encoded, activations are quantized on the fly) instead of being decoded to double first. Combined with 'Rgguf', which exposes 'GGUF' model weights as 'Rfmalloc'-backed tensors, this lets a larger-than-memory model's linear layers be multiplied through 'GGML's SIMD-accelerated dot kernels zero-copy from the memory-mapped payload. A transformer forward-pass graph builder over these primitives is planned.
Authors:
Rllm_0.1.0.tar.gz
Rllm_0.1.0.zip(r-4.7)Rllm_0.1.0.zip(r-4.6)Rllm_0.1.0.zip(r-4.5)
Rllm_0.1.0.tgz(r-4.6-x86_64)Rllm_0.1.0.tgz(r-4.6-arm64)Rllm_0.1.0.tgz(r-4.5-x86_64)Rllm_0.1.0.tgz(r-4.5-arm64)
Rllm_0.1.0.tar.gz(r-4.7-arm64)Rllm_0.1.0.tar.gz(r-4.7-x86_64)Rllm_0.1.0.tar.gz(r-4.6-arm64)Rllm_0.1.0.tar.gz(r-4.6-x86_64)
Rllm_0.1.0.tgz(r-4.6-emscripten)
manual.pdf |manual.html✨
DESCRIPTION |NEWS
card.svg |card.png
Rllm/json (API)
| # Install 'Rllm' in R: |
| install.packages('Rllm', repos = c('https://sounkou-bioinfo.r-universe.dev', 'https://cloud.r-project.org')) |
Bug tracker:https://github.com/sounkou-bioinfo/rfmalloc/issues
Pkgdown/docs site:https://sounkou-bioinfo.github.io
altrepblasgenomicsggmlggufllmllm-inferencemallocmemory-mappedmmapout-of-corequantizationsimd
Last updated from:921d01f678. Checks:10 OK, 3 NOTE. Indexed: yes.
| Target | Result | Time | Files | Syslog |
|---|---|---|---|---|
| linux-devel-arm64 | OK | 120 | ||
| linux-devel-x86_64 | OK | 107 | ||
| source / vignettes | OK | 166 | ||
| linux-release-arm64 | OK | 124 | ||
| linux-release-x86_64 | OK | 99 | ||
| macos-release-arm64 | OK | 106 | ||
| macos-release-x86_64 | OK | 170 | ||
| macos-oldrel-arm64 | NOTE | 103 | ||
| macos-oldrel-x86_64 | NOTE | 165 | ||
| windows-devel | OK | 75 | ||
| windows-release | OK | 90 | ||
| windows-oldrel | NOTE | 74 | ||
| wasm-release | OK | 92 |
Exports:rllm_backend_enabledrllm_decoderllm_encoderllm_forwardrllm_generaterllm_gguf_modelrllm_kv_cacherllm_quantize_tensorrllm_use_ggml
Readme and manuals
Help Manual
| Help page | Topics |
|---|---|
| Convert between token ids and raw bytes | rllm_decode rllm_encode |
| Run a transformer forward pass and return the logits | rllm_forward |
| Generate tokens with a KV cache (greedy or sampled) | rllm_generate |
| Load a llama-architecture GGUF model for forward passes | rllm_gguf_model |
| Create a KV cache for incremental decoding | rllm_kv_cache |
| Quantize a matrix into an Rfmalloc-backed quantized tensor | rllm_quantize_tensor |
| Enable or disable the ggml quantized matmul backend | rllm_backend_enabled rllm_use_ggml |
