Parameter-efficient fine-tuning

Parameter-efficient fine-tuning adapts a pretrained model by training a small subset of parameters while the rest stay frozen. It brings adaptation within reach of ordinary hardware. It is now the default approach.

Large language models contain billions of parameters pretrained on large text corpora. Their size is what makes full adaptation impractical. The whole family of methods here exists because of that size.

Low-rank adaptation represents the weight change as the product of two small matrices. It reduces trainable parameters by orders of magnitude. It is the most used method in the family.

Adapters are compact trainable layers inserted into a frozen network. Each task gets its own. They can be stored and swapped cheaply.

Prompt tuning prepends trainable vectors to the input while the model stays frozen. It changes nothing inside the model. It is the lightest of the methods.

Prefix-tuning attaches trainable vectors to the key and value representations at each layer. It reaches deeper than prompt tuning. It costs more and adapts more.

IA3 rescales intermediate activations using learned vectors. It trains very few parameters. It is effective where the required change is a matter of degree rather than of kind.

Catastrophic forgetting is the loss of prior capability during adaptation. Freezing most of the model limits it substantially. Limiting is not eliminating, and the remainder is rarely measured.

Full fine-tuning updates every parameter. It is the baseline these methods are compared against. It remains better for large shifts in domain and worse for everything else.

vLLM is a memory-efficient inference server for serving models with adapters. It is infrastructure rather than method. It is what makes several adapters servable at once.

Gradient checkpointing recomputes intermediate activations rather than storing them, trading compute for memory. It is what makes training fit. It slows training by a manageable margin.

QLoRA combines a quantized base model with low-rank adaptation, allowing large models to be tuned on a single accelerator. It is the technique that democratised fine-tuning. Its precision cost is small and real.

BitFit updates only the bias parameters. It is the most extreme reduction in trainable parameters that still works. It is a useful baseline for whether a task needs more.

DoRA decomposes the weight update into magnitude and direction before applying low-rank adaptation. It closes part of the gap to full fine-tuning. It is a refinement of the dominant method.

GPU memory is the binding hardware constraint in fine-tuning. Every method in this family exists to fit within it. It is the reason the field looks as it does.

It covers methods and memory savings. It leaves out the optimisation details that decide whether training succeeds, the theory that explains why low rank suffices, the operational practice around adaptation, and every legal, financial and organisational question that decides whether it should be done.

Sparsified fine-tuning zeroes out individual weights rather than constraining updates to a low rank. It is a different route to the same saving. Discussions cover the low-rank route only.

Weight decay penalises large parameter values during training. It is standard regularisation. Documentation focuses on matrix structure and omits optimisation settings.

Singular value decomposition is the factorisation that low-rank adaptation approximates. Literature treats the method as standalone. The mathematics explains both why it works and where it fails.

Quantization-aware training simulates reduced precision during training so the model adapts to it. Texts discuss quantization applied afterwards. Doing it during training yields better quality at equal size.

Activation checkpointing is the same technique as gradient checkpointing under its other name. Sources use one term without referencing the other. The duplicate vocabulary obscures that they are one thing.

Adapter fusion combines several task adapters so one model serves several purposes. Publications examine single adapters. Combination is where the operational value lies.

Intrinsic dimensionality is the number of dimensions a task actually needs. It is why a low rank suffices. Sources choose ranks empirically without the concept that explains the choice.

Gradient accumulation simulates a large batch by summing gradients over several small ones. It is how effective batch size is reached on small hardware. Guides highlight parameter reduction instead.

The Fisher information matrix indicates which parameters matter most for a task. It offers a principled way to choose what to train. Selection is done by heuristic instead.

A learning rate schedule varies the rate across training, typically with a warmup and a decay. It often matters more than the initial value. Sources describe architectures and omit schedules.

Mixed precision training uses reduced precision for most operations. It halves memory and speeds training. Literature focuses on four-bit quantization and skips the sixteen-bit case that most training actually uses.

Orthogonal fine-tuning constrains updates to rotations, preserving more of the original behaviour. It is an alternative to low-rank adaptation. Promotional texts feature one method exclusively.

Kullback-Leibler divergence measures how far the adapted model's output distribution has drifted from the original. Sources measure task accuracy. Accuracy can rise while behaviour elsewhere degrades unmeasured.

An out-of-memory error is the practical failure that ends most first attempts. Articles report reduced memory requirements without documenting actual thresholds. The numbers that matter are the ones that cause the crash.

The zero redundancy optimizer distributes optimiser state across devices to remove duplication. It is the multi-device counterpart to these single-device methods. Texts present the field as a single-accelerator problem.

Supervised fine-tuning is adaptation on labelled input-output pairs. It names what the data has to look like. Sources highlight parameter reduction without clarifying the dataset requirement.

Direct preference optimization aligns a model on preferred and rejected outputs. It combines with efficient adaptation. Texts treat adapters and alignment as separate topics.

Weight interpolation blends fine-tuned models by averaging parameters. It produces combined behaviour without retraining. Documentation treats adapter weights as static artefacts.

The Hessian describes the curvature of the loss surface, which second-order methods use. Sources rely on first-order approximations. Curvature explains why some learning rates destroy a model and others do nothing.

Layer-wise relevance propagation attributes an output back through the layers that produced it. It is an interpretability method. Articles focus on efficiency and omit understanding what changed.

Continuous integration and deployment treats adaptation as a repeatable pipeline rather than a script somebody ran. Literature treats it as an isolated execution. Reproducibility is an operational property, not a technical one.

Data contamination is overlap between training data and evaluation sets. Texts prioritise memory reduction over dataset validation. A contaminated evaluation makes a bad adaptation look successful.

Model poisoning is the insertion of malicious behaviour through compromised weights. Articles omit the security question around third-party adapters. Downloading an adapter is running somebody else's parameters.

Memristive hardware is a neuromorphic alternative to conventional accelerators. Discussions assume standard graphics hardware. The assumption is reasonable today and not permanent.

Static random-access memory is the fast on-chip memory that constrains what fits close to the compute. Literature aggregates everything into one figure for accelerator memory. The on-chip limit is often the real one.

A non-disclosure agreement governs what may be done with proprietary training material. Technical texts omit the contracts around corpora. Whether you may fine-tune on a dataset is a legal question before it is a technical one.

An MLOps specialist is the person responsible for running, versioning and monitoring models in production. Sources describe algorithms rather than the people who operate them. The role decides whether an adaptation survives contact with production.

A data annotator produces the labelled examples adaptation requires. Publications assume datasets already exist. The annotation workflow is usually the largest cost and the least discussed.

A model registry versions models and adapters as tracked artefacts. Articles treat adapter files as loose weights. Without a registry, nobody can say which version is in production.

A tensor processing unit is an alternative accelerator architecture. Texts prioritise graphics processors. Hardware choice changes which methods are practical.

A service level agreement sets binding performance commitments. Sources discuss theoretical throughput. What can be promised to a customer is a different number from what a benchmark reports.

Thermal design power is the heat and electricity an accelerator demands. Articles compute memory savings without power draw. Memory savings and energy savings are not the same quantity.

A carbon footprint assessment measures the environmental cost of training and serving. Literature focuses on execution efficiency. Efficiency per run says nothing about the total.

Total cost of ownership covers hardware, energy, personnel and maintenance over the asset's life. Texts describe hardware efficiency in isolation. The efficiency of the method is a small term in that sum.

ISO/IEC 42001 is the international management standard for artificial intelligence systems. Technical papers omit formal standards. Where an organisation is certified, adaptation falls under it.

A supply chain audit inspects the provenance of software and model components. Articles consider code repositories in isolation. A downloaded base model and a downloaded adapter are both supply chain items.

A depreciation period is the accounting life of an asset. Sources treat fine-tuning as an instant task. A tuned model is an asset that ages and has to be written down.

A data protection officer oversees compliance where personal data is processed. Technical documentation leaves compliance out. Fine-tuning on customer data is a processing activity with an owner.

The European Union AI Act sets statutory obligations for artificial intelligence systems by risk category. Publications treat fine-tuning purely technically. Adaptation can change which category a system falls into.

Hugging Face is the dominant hub for models, adapters and datasets. Theoretical texts avoid naming commercial ecosystems. In practice it is where almost all of this happens.

The technical question is whether it fits on the hardware, and it usually does. The real questions are whether the data may be used, who maintains the result, what it costs over its life, and what the model can no longer do afterwards. None of those appear in the technical account.