Parameter-efficient fine-tuning

Parameter-efficient fine-tuning adapts a pretrained model by training a small subset of parameters while the rest stay frozen. It brings adaptation within reach of ordinary hardware. It is now the default approach.

Large language models contain billions of parameters pretrained on large text corpora. Their size is what makes full adaptation impractical. The whole family of methods here exists because of that size.

Low-rank adaptation represents the weight change as the product of two small matrices. It reduces trainable parameters by orders of magnitude. It is the most used method in the family.

If training a tiny fraction of the weights cost quality, the accuracy rows would drop with the parameter count. They stay level.

Adapters are compact trainable layers inserted into a frozen network. Each task gets its own. They can be stored and swapped cheaply.

Prompt tuning prepends trainable vectors to the input while the model stays frozen. It changes nothing inside the model. It is the lightest of the methods.

Prefix-tuning attaches trainable vectors to the key and value representations at each layer. It reaches deeper than prompt tuning. It costs more and adapts more.

IA3 rescales intermediate activations using learned vectors. It trains very few parameters. It is effective where the required change is a matter of degree rather than of kind.

Catastrophic forgetting is the loss of prior capability during adaptation. Freezing most of the model limits it substantially. Limiting is not eliminating.

Full fine-tuning updates every parameter. It is the baseline these methods are compared against. It remains better for large shifts in domain and worse for everything else.

vLLM is a memory-efficient inference server for serving models with adapters. It is infrastructure rather than method. It is what makes several adapters servable at once.

Gradient checkpointing recomputes intermediate activations rather than storing them, trading compute for memory. It is what makes training fit. It slows training by a manageable margin.

QLoRA combines a quantized base model with low-rank adaptation, allowing large models to be tuned on a single accelerator. It is the technique that democratised fine-tuning. Its precision cost is small and real.

BitFit updates only the bias parameters. It is the most extreme reduction in trainable parameters that still works. It is a useful baseline for whether a task needs more.

DoRA decomposes the weight update into magnitude and direction before applying low-rank adaptation. It closes part of the gap to full fine-tuning. It is a refinement of the dominant method.

GPU memory is the binding hardware constraint in fine-tuning. Every method in this family exists to fit within it. It is the reason the field looks as it does.

It covers methods and memory savings. It leaves out the optimisation details that decide whether training succeeds, the theory that explains why low rank suffices, the operational practice around adaptation, and every legal, financial and organisational question that decides whether it should be done.

Sparsified fine-tuning zeroes out individual weights rather than constraining updates to a low rank. It is a different route to the same saving.

Weight decay penalises large parameter values during training. It is standard regularisation.

Singular value decomposition is the factorisation that low-rank adaptation approximates. The mathematics explains both why it works and where it fails.

Quantization-aware training simulates reduced precision during training so the model adapts to it. Doing it during training yields better quality at equal size.

Activation checkpointing is the same technique as gradient checkpointing under its other name. The duplicate vocabulary obscures that they are one thing.

Adapter fusion combines several task adapters so one model serves several purposes. Combination is where the operational value lies.

Intrinsic dimensionality is the number of dimensions a task actually needs. It is why a low rank suffices.

Gradient accumulation simulates a large batch by summing gradients over several small ones. It is how effective batch size is reached on small hardware.

The Fisher information matrix indicates which parameters matter most for a task. It offers a principled way to choose what to train.

A learning rate schedule varies the rate across training, typically with a warmup and a decay. It often matters more than the initial value.

Mixed precision training uses reduced precision for most operations. It halves memory and speeds training.

Orthogonal fine-tuning constrains updates to rotations, preserving more of the original behaviour. It is an alternative to low-rank adaptation.

Kullback-Leibler divergence measures how far the adapted model's output distribution has drifted from the original. Accuracy can rise while behaviour elsewhere degrades unmeasured.

An out-of-memory error is the practical failure that ends most first attempts. The numbers that matter are the ones that cause the crash.

The zero redundancy optimizer distributes optimiser state across devices to remove duplication. It is the multi-device counterpart to these single-device methods.

Supervised fine-tuning is adaptation on labelled input-output pairs. It names what the data has to look like.

Direct preference optimization aligns a model on preferred and rejected outputs. It combines with efficient adaptation.

Weight interpolation blends fine-tuned models by averaging parameters. It produces combined behaviour without retraining.

The Hessian describes the curvature of the loss surface, which second-order methods use. Curvature explains why some learning rates destroy a model and others do nothing.

Layer-wise relevance propagation attributes an output back through the layers that produced it. It is an interpretability method.

Continuous integration and deployment treats adaptation as a repeatable pipeline rather than a script somebody ran. Reproducibility is an operational property, not a technical one.

Data contamination is overlap between training data and evaluation sets. A contaminated evaluation makes a bad adaptation look successful.

Model poisoning is the insertion of malicious behaviour through compromised weights. Downloading an adapter is running somebody else's parameters.

Memristive hardware is a neuromorphic alternative to conventional accelerators. Standard graphics hardware is the reasonable assumption today and not a permanent one.

Static random-access memory is the fast on-chip memory that constrains what fits close to the compute. The on-chip limit is often the real one.

A non-disclosure agreement governs what may be done with proprietary training material. Whether you may fine-tune on a dataset is a legal question before it is a technical one.

An MLOps specialist is the person responsible for running, versioning and monitoring models in production. The role decides whether an adaptation survives contact with production.

A data annotator produces the labelled examples adaptation requires. The annotation workflow is usually the largest cost.

A model registry versions models and adapters as tracked artefacts. Without a registry, nobody can say which version is in production.

A tensor processing unit is an alternative accelerator architecture. Hardware choice changes which methods are practical.

A service level agreement sets binding performance commitments. What can be promised to a customer is a different number from what a benchmark reports.

Thermal design power is the heat and electricity an accelerator demands. Memory savings and energy savings are not the same quantity.

A carbon footprint assessment measures the environmental cost of training and serving. Efficiency per run says nothing about the total.

Total cost of ownership covers hardware, energy, personnel and maintenance over the asset's life. The efficiency of the method is a small term in that sum.

ISO/IEC 42001 is the international management standard for artificial intelligence systems. Where an organisation is certified, adaptation falls under it.

A supply chain audit inspects the provenance of software and model components. A downloaded base model and a downloaded adapter are both supply chain items.

A depreciation period is the accounting life of an asset. A tuned model is an asset that ages and has to be written down.

A data protection officer oversees compliance where personal data is processed. Fine-tuning on customer data is a processing activity with an owner.

The European Union AI Act sets statutory obligations for artificial intelligence systems by risk category. Adaptation can change which category a system falls into.

Hugging Face is the dominant hub for models, adapters and datasets. In practice it is where almost all of this happens.

The technical question is whether it fits on the hardware, and it usually does. The real questions are whether the data may be used, who maintains the result, what it costs over its life, and what the model can no longer do afterwards. None of those appear in the technical account.