LoRA/QLoRA Effective Scaling Calculator
Enter one or more adapters' rank, alpha, and whether they use
rank-stabilized LoRA (rsLoRA), and get the real effective
scaling factor each one applies — alpha / r for
standard LoRA, alpha / sqrt(r) for rsLoRA. With
more than one adapter, flags the specific mistake of mixing
rsLoRA and non-rsLoRA settings in a weighted merge, which has
caused real bugs even in reference implementations.
What is the actual formula for LoRA's effective scaling factor?
Standard LoRA (Hu et al., 2021) scales the adapter's low-rank
update by alpha divided by rank (alpha / r).
Rank-Stabilized LoRA, or rsLoRA (Kalajdzievski, 2023,
arXiv:2312.03732),
instead scales by alpha divided by the square root of rank
(alpha / sqrt(r)), specifically to keep training
stable at higher ranks where alpha/r would shrink
the update too aggressively. Which formula applies depends
entirely on whether the adapter was trained with
use_rslora enabled — using the wrong one silently
changes how strongly the adapter's learned behavior is
applied.
Why does merging multiple LoRA adapters sometimes produce wrong results?
When combining multiple adapters with different ranks,
alphas, or rsLoRA settings, each adapter's contribution needs
its own correctly-computed scaling factor before being
weighted and summed. Real bugs from getting this wrong have
been reported even in reference implementations: HuggingFace's
own PEFT library's add_weighted_adapter has been
reported to silently produce a wrong combined adapter for
rsLoRA and other variants, and vLLM and TensorRT-LLM have both
had issues applying the wrong scaling magnitude for rsLoRA
adapters in serving. Mixing rsLoRA and non-rsLoRA adapters in
one merge is exactly where this goes wrong, which is why this
tool flags it explicitly.
Does this tool send my adapter config anywhere?
No. All parsing and calculation happens entirely in your browser with plain JavaScript. Nothing you paste or enter is uploaded or transmitted.