Wednesday, December 31, 2025

Finetuning Large Language Models -- From Foundations to Adaptation (Chapter 7)

Download Book

<<< Previous Chapter Next Chapter >>>

Outline

Section 1: Introduction + The Evolution of Finetuning
Section 2: Memory Bottlenecks Explained
Section 3: Parameter-Efficient Finetuning (PEFT) & Adapter-Based Methods
Section 4: LoRA Deep Dive + Variants
Section 5: Model Merging, Frameworks, Hyperparameters, and Conclusion

Great! Here’s Section 1 of the blog post (~1200 words).

Finetuning Large Language Models: From Foundations to Adaptation

Section 1 – Introduction & The Evolution of Finetuning

Introduction: Why Finetuning Matters in the Age of Foundation Models

In the past few years, the way we build artificial intelligence (AI) applications has changed dramatically. At the center of this transformation are foundation models—large language models (LLMs) like GPT-4, Claude, Llama 3, and Mistral. These models come pre-trained on massive amounts of text data, enabling them to perform a wide variety of tasks right out of the box: answering questions, generating code, summarizing documents, and even creating poetry.

But while these models are incredibly powerful, they are not perfect for every situation. Imagine using a general-purpose LLM to handle specialized legal queries, medical diagnostics, or customer service conversations in a very specific tone of voice. The model may be good enough, but it may not fully align with your domain, data, or desired output format.

This is where finetuning comes into play.

Finetuning is the process of adapting a pre-trained model by training it further—either on your specific data, or with modifications that make it better suited for a specialized task. Unlike prompt engineering, which adjusts the input you feed into a model, finetuning adjusts the weights inside the model itself. The difference is subtle but significant: finetuning rewires how the model thinks, while prompting merely nudges it in the right direction.

To borrow an analogy:

Prompting is like instructing a professional chef to cook a new dish by giving detailed step-by-step instructions.
Finetuning is like training the chef to permanently learn that dish so they can cook it without explicit reminders in the future.

Finetuning is not just about better outputs—it’s about efficiency, alignment, and control. It allows developers and organizations to:

Improve instruction following (e.g., ensuring outputs always come in JSON).
Reduce biases by retraining on curated datasets.
Enhance domain-specific performance, such as financial modeling or SQL generation.
Create smaller, cheaper models that outperform larger ones on targeted tasks.

Of course, finetuning comes at a cost. It requires high-quality data, computational resources, and expertise in machine learning (ML). It also raises questions: When should you finetune? When is Retrieval-Augmented Generation (RAG) enough? What are the trade-offs?

This blog series explores these questions in depth. In this first section, we’ll walk through the evolution of finetuning—from its early days to today’s parameter-efficient techniques.

The Roots of Finetuning: Transfer Learning

The story of finetuning begins with transfer learning, a concept first introduced by Bozinovski and Fulgosi in 1976. Transfer learning is the idea that knowledge gained in one task can be transferred to accelerate learning in another. Humans do this all the time—if you know how to play the piano, learning the guitar is easier because you already understand rhythm, scales, and coordination.

In machine learning, transfer learning became popular in computer vision. Models trained on ImageNet, a massive image dataset, learned general features like edges, textures, and shapes. These features could then be reused for new tasks like detecting tumors in X-rays or recognizing traffic signs. Instead of training a new model from scratch, developers fine-tuned the ImageNet model for their specific problem.

The same principle applies to LLMs. A language model trained on billions of words already “knows” grammar, facts, and reasoning patterns. Instead of starting from scratch, developers finetune it for specialized tasks like legal document analysis, financial forecasting, or code generation.

A famous early success was Google’s multilingual translation system (Johnson et al., 2016). The system could translate Portuguese ↔ English and English ↔ Spanish. Surprisingly, without explicit examples, it learned to translate Portuguese ↔ Spanish—a task it was never directly trained on. That was transfer learning in action.

Finetuning in the Era of Large Language Models

When it comes to LLMs, finetuning builds on the pre-training phase. Pre-training is typically done in a self-supervised manner, meaning the model learns by predicting the next word in massive text datasets. This equips the model with broad knowledge but not necessarily the ability to follow instructions well or produce outputs in a specific style.

Here’s where post-training and finetuning enter:

Supervised Finetuning (SFT) – The model is trained on pairs of (instruction, response). For example:
- Instruction: “Summarize this contract in simple terms.”
- Response: “This contract states that the employee will…”
  SFT improves alignment with human expectations.
Preference Finetuning (RLHF) – The model learns to prefer outputs that humans like, usually through reinforcement learning with human feedback. This makes responses more helpful, harmless, and honest.
Continued Pre-Training – The model is further trained on large amounts of raw domain-specific text (e.g., legal documents or medical literature) before fine-grained supervised tuning.

Each of these forms of finetuning is an extension of pre-training. The difference is in how much data you use, what kind of data, and which parameters you update.

Why Finetuning?

Let’s consider some real-world scenarios:

Structured Outputs: Suppose you need a model to always respond in strict JSON format for downstream automation. Prompting can work, but it’s fragile. Finetuning ensures the model internalizes this requirement.
Domain-Specific Jargon: A healthcare chatbot may need to understand abbreviations like “BP” (blood pressure) or “HbA1c” (a diabetes measure). General models may stumble; finetuning on medical data fixes this.
Bias Mitigation: If a model frequently associates “CEO” with male names, finetuning on curated datasets that include female CEOs can rebalance the bias.
Smaller, Faster Models: Grammarly reported that their finetuned Flan-T5 models (much smaller than GPT-3) outperformed GPT-3 on text editing tasks. This made their models cheaper and faster in production.

In short, finetuning helps bridge the gap between a model’s general intelligence and the specific intelligence you need.

The Shift from Full to Efficient Finetuning

In the early days, finetuning meant updating all the parameters of a model. This is called full finetuning. For small models, this was feasible. But as models grew to billions of parameters, full finetuning became impractical:

A 7B parameter model in 16-bit precision requires ~14 GB just to store the weights.
To finetune it with optimizers like Adam, you may need ~56 GB or more.
This exceeds the memory of most consumer GPUs.

To solve this, researchers moved toward partial finetuning: instead of updating the entire model, they updated only parts—often the final layers. While this reduced memory requirements, it was inefficient: performance dropped compared to full finetuning.

The breakthrough came with Parameter-Efficient Finetuning (PEFT). Instead of touching billions of parameters, PEFT techniques add or modify only a small fraction of the model while freezing the rest. The result: near-full performance at a fraction of the cost.

PEFT methods fall broadly into two categories:

Adapter-based methods: Insert small modules (adapters) into the model and train only them.
Soft prompt–based methods: Add trainable embeddings (soft prompts) to guide the model.

One adapter-based technique, LoRA (Low-Rank Adaptation), has become especially dominant. LoRA allows developers to finetune massive models using only a few percent of the parameters, without hurting inference speed.

Looking Ahead

We’ve set the stage:

Finetuning began as transfer learning.
It evolved into full and partial finetuning.
The need for scalability gave rise to PEFT methods like LoRA.

In the next section, we’ll explore why finetuning is so memory-intensive, breaking down the concepts of trainable parameters, backpropagation, and numerical representations. Understanding these bottlenecks is key to appreciating why PEFT matters so much.

Finetuning Large Language Models: From Foundations to Adaptation

Section 2 – Memory Bottlenecks Explained

Why Memory Is the Achilles’ Heel of Finetuning

When developers first attempt to finetune a large language model (LLM), one of the most common errors they see is:

csharp
RuntimeError: CUDA out of memory

This isn’t just a beginner’s mistake—it’s a fundamental reality of working with models that contain billions of parameters. Memory is the biggest bottleneck in finetuning, and unless you understand where that memory goes, you’ll quickly hit a wall.

At inference time (when the model is simply generating outputs), memory requirements are high but manageable. Finetuning, however, is far more demanding. That’s because finetuning doesn’t just run the model—it also needs to update its parameters, store gradients, and keep optimizer states. All of this adds up.

In this section, we’ll unpack the sources of memory consumption in finetuning, run through some back-of-the-envelope calculations, and explore techniques like quantization and gradient checkpointing that help mitigate the problem.

Parameters, Trainable Parameters, and Frozen Layers

Every model is made up of parameters—the numerical weights that determine how it processes input. For example:

GPT-3 has 175 billion parameters.
Llama 2 has versions ranging from 7 billion to 70 billion parameters.
Even a “small” model like Mistral 7B requires tens of gigabytes to store.

When you perform inference, the model loads all its parameters into memory but does not modify them. That’s relatively simple.

When you perform finetuning, however, you allow some or all of those parameters to be updated. The parameters that can change are called trainable parameters.

Full finetuning: All parameters are trainable.
Partial finetuning: Some parameters are trainable, the rest are frozen.
PEFT methods (e.g., LoRA): Only a tiny fraction of parameters are added or modified.

The more trainable parameters you have, the higher the memory footprint.

Backpropagation: The Memory Multiplier

The reason finetuning consumes so much more memory than inference comes down to the training process, specifically backpropagation.

Training happens in two stages:

Forward pass – The model computes an output given an input. (This is the same as inference.)
Backward pass – The model computes how wrong its output was compared to the expected result (the loss) and adjusts its weights accordingly.

During the backward pass, each trainable parameter requires extra memory for:

Its gradient (how much that parameter contributed to the error).
Its optimizer states (values used by optimizers like Adam to update weights efficiently).

For example, with the Adam optimizer:

Each trainable parameter requires 3 extra values: 1 gradient + 2 optimizer states.
If you store each value in 2 bytes (16-bit precision), that’s an extra 6 bytes per parameter.

So for a 13B-parameter model, updating all parameters means:

python
13B parameters × 6 bytes = 78 GB

And that’s just for gradients and optimizer states! Add in the original weights (~26 GB in 16-bit precision) plus activations, and you’re well beyond the capacity of a single GPU.

Activations: The Hidden Memory Hog

Parameters aren’t the only concern. Neural networks also generate activations—the intermediate values produced as data flows through the layers.

During inference, activations are ephemeral: once the output is generated, they can be discarded.

During training, however, activations must be stored so they can be reused in the backward pass to calculate gradients. This means activations can dwarf the size of the model’s weights.

Research from Microsoft (Korthikanti et al., 2022) showed that for large transformer models, activation memory often exceeds weight memory.

To cope with this, developers use gradient checkpointing (activation recomputation). Instead of storing all activations, the model recomputes them when needed. This reduces memory at the cost of slower training.

Memory Math: A Simple Formula

To get a rough estimate of how much memory finetuning requires, you can use this formula:

java
Training memory = Weights + Activations + Gradients + Optimizer states

Let’s work through a practical example.

Example: 13B-parameter model (LLaMA 2-13B)

Weights (16-bit) = 13B × 2 bytes = 26 GB
Gradients + Optimizer states (Adam, 16-bit) = 13B × 3 × 2 bytes = 78 GB
Activations (approx. 20% of weights) = ~5 GB (in practice often much higher)

Total = 26 + 78 + 5 ≈ 109 GB

That’s well beyond even high-end GPUs. The Nvidia A100 tops out at 80 GB of memory, and most developers use far less powerful cards (24–48 GB).

This is why full finetuning of large models is impractical for most organizations.

Numerical Representations: FP32, FP16, BF16, and INT8

Another key factor in memory usage is the numerical format used to store values.

Traditionally, models were trained in FP32 (32-bit floating point). But this is overkill:

FP32 = 4 bytes per value.
FP16 (half precision) = 2 bytes per value.
BF16 (Google’s “Brain Floating Point”) also = 2 bytes, but with different trade-offs (wider range, less precision).
INT8 and INT4 = 1 byte and 0.5 bytes, respectively.

For example, a 13B-parameter model in FP32 requires 52 GB just for weights. In FP16 or BF16, that drops to 26 GB. In INT8, it’s 13 GB.

This reduction is called quantization—representing values with fewer bits.

Post-training quantization (PTQ): Quantize after the model is trained. Common for inference.
Quantization-aware training (QAT): Simulate low precision during training so the model learns to handle it.

Quantization can reduce both memory and compute time, but it risks accuracy loss. Striking the right balance is an active area of research.

Inference vs. Training Memory Profiles

It’s important to distinguish between inference and training memory requirements:

Inference memory ≈ Weights + Activations (scaled by sequence length & batch size)
Training memory ≈ Weights + Activations + Gradients + Optimizer states

That extra overhead for gradients and optimizer states is what makes finetuning orders of magnitude more demanding than inference.

This difference is why we see specialized chips for inference (optimized for low precision, speed, and efficiency) versus training (optimized for high throughput and precision).

Tricks to Reduce Memory Bottlenecks

Several strategies exist to make finetuning possible without 100+ GB GPUs:

Gradient Checkpointing
- Don’t store all activations; recompute them when needed.
- Saves memory, costs extra compute time.
Mixed Precision Training
- Store some values (like gradients) in FP16 or BF16 while keeping others (like embeddings) in FP32.
- Most ML frameworks (PyTorch AMP, TensorFlow) support this out of the box.
Quantization
- Use INT8, INT4, or even experimental 1.58-bit formats (BitNet, 2024).
- Reduces memory, but may hurt accuracy if not done carefully.
PEFT (Parameter-Efficient Finetuning)
- Instead of updating all parameters, update only a small fraction (we’ll dive deep into this in the next section).
- Can reduce trainable parameters from billions to millions.
Offloading
- Move some weights to CPU or disk (e.g., DeepSpeed ZeRO).
- Slower, but allows fitting larger models.
Smaller Models
- Don’t underestimate the power of smaller models! A well-finetuned 7B model may outperform an unfined 70B model on your task.

Why This Matters

Understanding memory bottlenecks is not just about avoiding runtime errors. It’s about strategic decision-making:

Do you attempt full finetuning, or is PEFT sufficient?
Should you invest in expensive GPUs, or use clever memory-saving techniques?
Do you quantize aggressively for efficiency, or preserve precision for accuracy?

Every organization will make different trade-offs. But the key takeaway is this: memory, not compute, is the limiting factor in finetuning today’s LLMs.

Looking Ahead

We’ve now seen why finetuning is so challenging from a memory perspective. The natural next step is to explore how researchers solved this problem—enter Parameter-Efficient Finetuning (PEFT).

In the next section, we’ll dive into adapter-based methods and soft prompts, the two main families of PEFT techniques that have made finetuning practical for developers around the world.

Finetuning Large Language Models: From Foundations to Adaptation

Section 3 – Parameter-Efficient Finetuning (PEFT) & Adapter-Based Methods

The Rise of Parameter-Efficient Finetuning (PEFT)

In the previous section, we saw how memory bottlenecks make full finetuning of large language models (LLMs) impractical for most developers. Training all parameters in a multi-billion-parameter model requires hundreds of gigabytes of memory—far more than what a typical GPU setup can handle.

So researchers asked: What if we didn’t need to update all the parameters?

This gave rise to Parameter-Efficient Finetuning (PEFT). Instead of retraining billions of parameters, PEFT techniques modify or add a tiny fraction of parameters—sometimes as little as 0.1% of the model—while freezing the rest. The results are astonishing: models finetuned with PEFT often perform within a few percentage points of full finetuning, at a fraction of the cost.

Why does this work? Because foundation models already know a lot. Finetuning isn’t about teaching them entirely new concepts; it’s about nudging them into the right shape for a specific task or domain. You don’t need to rewire the whole brain, just adjust a few critical pathways.

Full vs. Partial vs. Parameter-Efficient Finetuning

Let’s quickly compare the three approaches:

Full Finetuning
- Updates all model parameters.
- Very expensive (memory + compute).
- Highest performance, but diminishing returns compared to efficient methods.
Partial Finetuning
- Updates only a subset of layers (e.g., the final transformer block).
- Reduces cost but often requires 20–30% of parameters to match full performance.
- Parameter-inefficient: too many parameters for too little gain.
PEFT
- Updates or adds only a very small number of parameters.
- Can achieve near full-finetuning performance with 1–5% of parameters (sometimes less).
- The dominant approach today for adapting large LLMs.

The breakthrough paper by Houlsby et al. (2019) introduced the first widely adopted PEFT method: adapters. Let’s dive into how adapter-based methods work.

Adapter-Based Methods

The Core Idea

Adapters are small, trainable modules inserted into each transformer block of an LLM. Instead of updating the billions of frozen parameters in the base model, finetuning updates only the adapter weights.

Think of adapters as plug-ins for the model:

The frozen base model is the operating system.
The adapters are small applications that customize behavior.

During inference, the model processes inputs as usual, but the adapters intercept the flow, add their learned adjustments, and pass the signal back. This way, adapters can teach the model new tricks without rewriting its entire knowledge base.

Houlsby Adapters (2019)

The original adapter method added two small feed-forward layers into each transformer block. During training:

The base model’s parameters stayed frozen.
Only the adapter layers were updated.

The results were striking. On the GLUE benchmark for natural language understanding, Houlsby adapters achieved performance within 0.4% of full finetuning, while training only 3% of parameters.

This was a game-changer. Suddenly, adapting large models became feasible on modest hardware.

BitFit (2021)

BitFit took the idea even further: instead of inserting new layers, it updated only the bias terms of the model (hence the name). Since bias parameters make up a tiny fraction of the total weights, this meant updating less than 0.1% of parameters.

Surprisingly, BitFit still delivered competitive results in many tasks. It wasn’t always as strong as adapters, but it showed just how much efficiency was possible.

IA³ (2022)

IA³ (Infused Adapter by Inhibiting and Amplifying Inner Activations) proposed a more elegant method: instead of adding full adapter layers, it learned small scaling vectors that multiplied existing activations inside the model.

This meant:

Extremely lightweight overhead.
Easy batching for multi-task finetuning.
In some cases, even better performance than full finetuning.

LongLoRA (2023)

One limitation of many LLMs is context length—how much text they can handle at once. Most models are trained with a context window of a few thousand tokens.

LongLoRA adapted LoRA (which we’ll cover in depth in Section 4) to extend context length. By tweaking attention mechanisms and training with long-sequence data, LongLoRA enabled models to handle much larger contexts (e.g., from 4k tokens to 16k tokens or more).

This made it especially useful for tasks like:

Handling long code files.
Processing legal or scientific documents.
Multi-turn dialogue with extended histories.

Soft Prompt–Based Methods

While adapter-based methods add trainable parameters inside the model, soft prompt methods modify the inputs the model sees.

Here’s the difference:

Hard prompts = Human-readable text prompts (e.g., “Translate to French:”).
Soft prompts = Trainable vectors (continuous embeddings) prepended to the input.

Soft prompts aren’t human-readable, but they act like invisible “hints” that steer the model’s behavior.

Prompt Tuning (2021)

In prompt tuning, a small set of trainable embeddings is added to the input at the embedding layer. During training:

The base model stays frozen.
Only the soft prompts are updated.

This allows a single base model to be quickly adapted to many tasks by swapping out the soft prompts.

Prefix Tuning (2021)

Prefix tuning goes a step further: instead of adding tokens only at the input, it prepends soft prompts at every transformer layer. This provides more control over the model’s internal activations.

Prefix tuning is more powerful but also slightly heavier than prompt tuning.

P-Tuning (2021)

P-Tuning is another variant that optimizes where and how soft prompts are injected. These subtle differences in architecture make each method slightly better for certain tasks, but the underlying idea is the same: train a small set of vectors instead of the whole model.

Comparing Adapter vs. Soft Prompt Methods

Feature	Adapter-Based	Soft Prompt–Based
Where it acts	Inside transformer layers	Input embeddings (and sometimes layers)
Parameter efficiency	1–5% trainable params	Often <1% trainable params
Performance	Strong, near full finetuning	Competitive, but sometimes weaker
Modularity	Can stack multiple adapters	Can swap prompts easily
Inference speed	Slightly slower (extra layers)	Same as base model
Use cases	Domain adaptation, syntax enforcement	Multi-task, lightweight customization

Both families of methods are actively used today, depending on the application and constraints.

Real-World Adoption

To see what’s popular in practice, researchers analyzed thousands of GitHub issues from the Hugging Face PEFT library (2024). The results showed:

LoRA dominates adapter-based methods.
Soft prompts are less common but gaining traction among teams that want more flexibility than prompting but less complexity than finetuning.

This aligns with industry trends: LoRA is the go-to choice for finetuning today, thanks to its balance of efficiency and performance.

Why PEFT Matters for Developers

For most application developers, full finetuning is out of reach. Even partial finetuning may be impractical. PEFT bridges the gap by making it possible to:

Run finetuning on a single GPU or small cluster, instead of massive infrastructure.
Maintain multiple task-specific models by swapping adapters or prompts, rather than training from scratch each time.
Deploy cheaper, faster models that still meet production needs.

In other words, PEFT democratizes LLM customization.

Looking Ahead

We’ve covered the two main families of PEFT: adapters and soft prompts. But one method in particular—LoRA (Low-Rank Adaptation)—has become the de facto standard, powering everything from open-source community models to enterprise deployments.

In the next section, we’ll dive deep into LoRA, exploring how it works, why it’s so effective, and the many variants that have sprung from it.

Finetuning Large Language Models: From Foundations to Adaptation

Section 4 – LoRA Deep Dive & Variants

The Emergence of LoRA

Among all the parameter-efficient finetuning (PEFT) methods, LoRA (Low-Rank Adaptation) has emerged as the most widely used. Originally introduced by Hu et al. (2021), LoRA was designed with a simple but powerful idea: instead of updating huge weight matrices directly, approximate their updates using much smaller low-rank matrices.

This approach slashes the number of trainable parameters by orders of magnitude while preserving performance. Today, if you see a community finetuned model on Hugging Face, chances are it was trained using LoRA.

But why did LoRA succeed where others were just “good enough”? To answer that, let’s break down how it works.

The Core Idea of LoRA

Transformer models, like GPT and LLaMA, are filled with linear layers—essentially giant matrices that multiply with vectors to process data. For example:

ini
y = W × x

where:

W is a weight matrix (often billions of parameters).
x is the input vector.
y is the output.

In finetuning, we want to adjust W. But directly storing and updating such a massive matrix is impractical.

LoRA introduces a clever trick:

Instead of updating W directly, represent the update ΔW as the product of two smaller matrices, A and B.


W’ = W + ΔW  
ΔW = A × B

Here:

A is an m × r matrix.
B is an r × n matrix.
r is the rank (a small number, often 4, 8, or 16).

Because r is small, A × B has far fewer parameters than W. This is the “low-rank” in Low-Rank Adaptation.

Why LoRA Works: The Intrinsic Dimension Hypothesis

At first glance, you might wonder: How can tiny matrices possibly capture the complexity of massive weight updates?

The answer lies in the intrinsic dimension hypothesis. Research suggests that many high-dimensional tasks can actually be solved in a much lower-dimensional subspace. In other words, even though models have billions of parameters, the meaningful changes for a given task lie in a much smaller space.

LoRA exploits this by only learning updates within that smaller subspace. The base model provides the “knowledge,” and LoRA adapters learn the “specialization.”

Practical Benefits of LoRA

LoRA offers several advantages that explain its widespread adoption:

Efficiency
- Only a tiny fraction of parameters are trained.
- Finetuning becomes feasible on a single GPU.
Modularity
- You can train different LoRA adapters for different tasks.
- Swap them in and out without retraining the whole model.
Composability
- Multiple LoRAs can be combined (e.g., one for medical text, another for legal text).
- This makes models highly customizable.
No Inference Latency Penalty
- LoRA updates are folded into the weight multiplication, so inference remains just as fast.

These factors made LoRA the go-to choice for researchers and developers alike.

Variants of LoRA

Like all good ideas in AI, LoRA quickly inspired numerous extensions and adaptations. Let’s look at the major ones.

Quantized LoRA (QLoRA)

Introduced by Dettmers et al. (2023), QLoRA combined two powerful ideas:

Quantization – Store base model weights in 4-bit precision to reduce memory.
LoRA – Train low-rank adapters on top of the quantized model.

This made it possible to finetune 65B-parameter models on a single 48 GB GPU. Before QLoRA, such large-scale finetuning was reserved for top-tier labs.

QLoRA democratized finetuning by drastically lowering hardware requirements. Today, it’s one of the most practical setups for community-driven projects.

LongLoRA

We briefly mentioned LongLoRA earlier. It modifies the attention mechanism to allow models to handle longer sequences during finetuning. Combined with LoRA, it makes training for long-context tasks affordable.

Use cases:

Reading entire legal contracts.
Summarizing research papers.
Multi-turn chatbots with extended histories.

ReLoRA (Restarted LoRA)

ReLoRA addresses a subtle issue: as training progresses, low-rank updates may lose effectiveness. ReLoRA periodically resets the low-rank matrices and reinitializes them, allowing the model to keep learning efficiently.

This is especially useful for longer training runs where LoRA might otherwise stagnate.

GaLore (Gradient Low-Rank Projection)

GaLore, a 2024 innovation, applies the low-rank trick not just to weight updates, but directly to gradients. Instead of storing full gradients (which consume huge memory), it projects them into a low-rank space.

This slashes memory requirements for backpropagation and makes finetuning even more efficient.

Multi-LoRA Serving

One of LoRA’s killer features is that you can serve multiple adapters on the same base model. For example:

A customer service company could maintain separate LoRAs for retail, banking, and healthcare clients.
A developer could train LoRAs for different tasks (summarization, sentiment analysis, translation) and load whichever is needed on demand.

This modularity is similar to having multiple browser extensions installed on the same web browser. The core system remains unchanged; each extension adds a specialization.

Serving frameworks like Hugging Face’s peft library and 🤗 Transformers already support this capability, making LoRA-based deployments production-ready.

Real-World Applications of LoRA

Community Models
- Hugging Face is filled with LoRA-finetuned models.
- From roleplay chatbots to code assistants, LoRA makes experimentation accessible.
Enterprise Customization
- Companies use LoRA to adapt foundation models to domain-specific needs.
- Example: A financial firm finetuning a Llama model with LoRA for SEC filings and earnings calls.
Education and Research
- Universities and independent researchers can now finetune models without massive clusters.
- LoRA has become the default teaching method for “hands-on LLM finetuning.”

Why LoRA Dominates

While other PEFT methods (like adapters and soft prompts) are useful, LoRA has several unique advantages that explain its dominance:

Performance – Often matches or beats other methods.
Ease of Integration – Works seamlessly with existing architectures.
Scalability – Handles small and very large models.
Community Momentum – Supported by libraries like Hugging Face PEFT, PyTorch, and ColossalAI.

In fact, many researchers now assume LoRA as the default PEFT baseline in experiments.

Beyond LoRA: The Future of Low-Rank Methods

LoRA is not the end of the story. Several exciting directions are emerging:

Dynamic rank selection – Adapting rank r per layer for optimal efficiency.
Low-rank pre-training – Training models from scratch with low-rank constraints.
Hybrid approaches – Combining LoRA with soft prompts or adapters for maximum flexibility.

The field is evolving fast, but one thing is clear: the low-rank principle has unlocked a new era of scalable, affordable finetuning.

Looking Ahead

We’ve now explored LoRA and its variants in depth. But LoRA isn’t the only alternative to full finetuning. Another powerful approach is model merging, where instead of training adapters, developers directly combine pre-trained models.

In the next section, we’ll cover:

Model merging techniques (summing, stacking, concatenation).
Finetuning frameworks and hyperparameters.
Practical decision-making for when and how to finetune.

Finetuning Large Language Models: From Foundations to Adaptation

Section 5 – Model Merging, Frameworks, Hyperparameters, and Conclusion

Beyond Finetuning: The Idea of Model Merging

So far, we’ve focused on finetuning—whether full, partial, or parameter-efficient. But there’s another emerging paradigm: model merging.

Instead of training adapters or updating weights, model merging combines already finetuned models into a single new model.

Why does this work? Because finetuned models often contain complementary knowledge. By merging, we can capture the strengths of multiple models without retraining from scratch.

Techniques for Model Merging

Weight Averaging (Model Soup)
- Proposed by Wortsman et al. (2022).
- Merge multiple models by taking a weighted average of their parameters.
- Example: If you have three LoRA-finetuned models for different NLP tasks, averaging them can yield a single, more general model.
Layer Stacking
- Stack layers from different models.
- Example: Lower layers from a language model trained on code + higher layers from a model trained on dialogue.
Concatenation
- Combine embeddings or intermediate representations from multiple models.
- More experimental, but promising for multimodal tasks (e.g., text + image).

Real-World Examples

SOLAR 10.7B (2023): A model built by merging multiple open-weight models (up to 16). Despite being only 10.7B parameters, it achieved performance close to LLaMA-2-70B on benchmarks.
On-device models: Small models on smartphones can be “merged” from different finetuned variants, allowing them to perform diverse tasks without huge storage overhead.

Model merging isn’t yet as mature as LoRA, but it shows the future may involve mixing and matching specialized models rather than finetuning one base model repeatedly.

Choosing the Right Base Model

When planning finetuning or merging, the choice of base model is critical.

Open-source models: LLaMA, Mistral, Falcon, Mixtral, Gemma.
- Pros: Customizable, transparent, cheap to run.
- Cons: May lack the polish of commercial APIs.
Proprietary models: OpenAI GPT, Anthropic Claude, Google Gemini.
- Pros: State-of-the-art performance, strong support.
- Cons: Limited finetuning access, high costs.

A common strategy:

Use open-source models for heavily customized or private data tasks.
Use APIs for general-purpose or low-maintenance tasks.

Finetuning Frameworks

Several frameworks make PEFT and LoRA finetuning more accessible:

Hugging Face PEFT
- Most widely used library for LoRA, QLoRA, prefix tuning, etc.
- Integrates with Transformers and Accelerate.
Axolotl
- Community-driven framework for multi-GPU LoRA/QLoRA training.
- Popular for fine-tuning LLaMA, Mistral, and other open-source models.
LitGPT
- Lightweight, research-friendly framework.
- Focused on reproducibility and simplicity.
Unsloth
- Emerging framework specializing in extremely efficient LoRA training.
- Often used for rapid experimentation.
Distributed Training Libraries
- DeepSpeed (Microsoft): Sharding and offloading for large models.
- ColossalAI: Scales LoRA finetuning efficiently.
- PyTorch Distributed: General-purpose parallel training.

For most developers, PEFT + Transformers is enough. For larger teams, frameworks like DeepSpeed and ColossalAI unlock scalability.

Hyperparameters That Matter

Finetuning involves many knobs, but a few matter more than others:

Learning Rate
- Too high → catastrophic forgetting (model loses general knowledge).
- Too low → underfitting.
- Rule of thumb: LoRA adapters often use learning rates around 1e-4 to 1e-5.
Batch Size
- Larger batches → more stable gradients, but higher memory.
- Smaller batches → noisier but sometimes better generalization.
Sequence Length
- Longer sequences capture more context but increase memory.
- Use gradient checkpointing or LongLoRA for efficiency.
Prompt Loss Weighting
- Important when balancing instruction-following with task-specific objectives.
Epochs & Early Stopping
- Overtraining can lead to overfitting or forgetting.
- Early stopping helps maintain balance.

In practice, hyperparameter tuning is as much art as science. Many teams start with community defaults and adjust based on validation performance.

Practical Development Paths

OpenAI’s Path

OpenAI suggests a progression for developers:

Prompt engineering – Cheapest, simplest.
Few-shot prompting – Better results with examples.
RAG (Retrieval-Augmented Generation) – Bring external knowledge in.
Finetuning (SFT/LoRA) – When you need strict formats or domain alignment.

This ensures you don’t overinvest in finetuning when prompting or RAG would suffice.

Distillation Path

Another popular path is distillation:

Train a small student model to mimic a large teacher model.
Use finetuned LLMs as teachers.
Deploy smaller, faster students for production.

This allows enterprises to get 80–90% of the performance at a fraction of the cost.

Challenges and Trade-Offs

Data Availability
- Good finetuning data is rare and expensive.
- Synthetic data generation (using GPT-4 or Claude) helps but can introduce noise.
Latency and Deployment
- Adapters can increase latency if not merged back into the model.
- Some frameworks support merging LoRA weights into base weights for deployment.
When Not to Finetune
- If the goal is to add new facts → Use RAG.
- If the goal is structured output or style → Use finetuning.
- Key mantra: “Finetuning is for form, RAG is for facts.”

The Future of Finetuning

Where is finetuning headed? Several trends are emerging:

Low-rank pre-training: Training models from scratch with LoRA-like constraints.
Composable adapters: Building “adapter stores” where LoRAs can be mixed like Lego blocks.
Model merging at scale: Building models like SOLAR that rival much larger ones.
Ultra-low-bit training: BitNet (2024) showed training in 1.58-bit precision can be stable, hinting at massive efficiency gains.

The direction is clear: finetuning will become cheaper, more modular, and more accessible.

Conclusion: A New Era of Customization

We’ve journeyed through the evolution of finetuning:

From full finetuning and its memory bottlenecks…
To PEFT methods like adapters and soft prompts…
To the dominance of LoRA and its variants…
To emerging approaches like model merging.

The takeaway is simple: you no longer need a billion-dollar budget to adapt powerful LLMs. With techniques like LoRA and QLoRA, even modest setups can achieve near state-of-the-art results.

For developers and enterprises, the message is empowering:

Use prompting and RAG when possible.
Use finetuning when you need control, style, or structure.
Explore merging and distillation to push efficiency further.

The field is moving fast, but one principle remains constant: foundation models are the clay, and finetuning is how we sculpt them into tools that serve our needs.

Tags: Artificial Intelligence,Generative AI,Agentic AI,Technology,Book Summary,

Saturday, December 27, 2025

Installing Draw.IO on Linux Ubuntu Laptop

All Linux Lessons

How do I install this software on my linux machine:

It is available in following architectures:
Linux - deb, AppImage or rpm

You can install draw.io (diagrams.net desktop) on Linux in three common ways. Pick the one that best fits your distro and preference.

✅ Option 1: DEB package (Best for Ubuntu / Linux Mint / Debian)

Recommended if you want system integration and automatic updates.

Download the .deb file from the release page.
Open a terminal in the download folder and run:
```
bash
sudo apt update
sudo apt install ./drawio-amd64-29.0.3.deb
```
Replace the filename if the version differs.
Launch from the app menu or run:
```
bash
drawio
```

✅ Option 2: AppImage (Works on almost all distros)

No installation needed; great for portability.

Download the .AppImage file.

Make it executable:

bash
chmod +x drawio-29.0.3-x86_64.AppImage

Run it:

bash
./drawio-29.0.3-x86_64.AppImage

Optional: Move it to /opt and create a desktop shortcut if you want.

✅ Option 3: RPM package (Fedora / RHEL / openSUSE)

Use this if you’re on an RPM-based distro.

bash
sudo dnf install drawio-29.0.3.x86_64.rpm

or on openSUSE:

bash
sudo zypper install drawio-29.0.3.x86_64.rpm

🔍 Which should you choose?

Ubuntu / Linux Mint / Debian → DEB
Any distro / portable use → AppImage
Fedora / RHEL / openSUSE → RPM

If you tell me your exact distro (e.g., Ubuntu 22.04, Mint 21, Lubuntu, etc.), I’ll give you the exact filename and best command.

...

(base) jain@jain-ThinkPad-T440s:~/Downloads$ chmod +x drawio-x86_64-29.0.3.AppImage 
(base) jain@jain-ThinkPad-T440s:~/Downloads$ ./drawio-x86_64-29.0.3.AppImage 
dlopen(): error loading libfuse.so.2

AppImages require FUSE to run. 
You might still be able to extract the contents of this AppImage 
if you run it with the --appimage-extract option. 
See https://github.com/AppImage/AppImageKit/wiki/FUSE 
for more information

This is a very common AppImage issue 👍
Your system is missing FUSE v2 (libfuse.so.2), which AppImages require.

Let’s fix it cleanly.

✅ Fix 1 (Recommended): Install FUSE (libfuse2)

On Ubuntu / Linux Mint / Lubuntu:

bash
sudo apt update
sudo apt install libfuse2

Then run again:

bash
./drawio-x86_64-29.0.3.AppImage

This should start draw.io immediately.

🔎 Why this happens

AppImages use FUSE (Filesystem in Userspace) to mount themselves.
Newer distros ship fuse3, but AppImages still expect fuse2.
Hence the missing libfuse.so.2.

🛠 Fix 2 (If you cannot install FUSE): Extract AppImage

If you’re on a restricted system:

bash
./drawio-x86_64-29.0.3.AppImage --appimage-extract

Then run it manually:

bash
cd squashfs-root
./drawio

⚠️ This works but no desktop integration.

🔁 Alternative (Most Stable): Install DEB instead

If you’re on Ubuntu/Mint/Lubuntu, the DEB version is simpler:

bash
sudo apt install ./drawio-amd64-29.0.3.deb

✅ Quick Recommendation

Situation	Best Choice
Ubuntu / Mint / Lubuntu	DEB package
Portable / no install	AppImage + libfuse2
Locked-down system	AppImage extract

If you want, I can:

Create a desktop launcher for the AppImage
Help you switch to the DEB version
Check if your system already has fuse3 vs fuse2

Just tell me 👍

... ... ...

(base) jain@jain-ThinkPad-T440s:~/Downloads$ ./drawio-x86_64-29.0.3.AppImage 
[6616:1227/063742.336623:FATAL:sandbox/linux/suid/client/setuid_sandbox_host.cc:166] The SUID sandbox helper binary was found, but is not configured correctly. Rather than run without sandboxing I'm aborting now. You need to make sure that /tmp/.mount_drawiodOPId7/chrome-sandbox is owned by root and has mode 4755.
Trace/breakpoint trap (core dumped)

This is another very common Electron + AppImage issue 👍
Nothing is “broken” — it’s a sandbox permission problem.

The error means:

Electron found the chrome-sandbox helper inside the AppImage, but it cannot be setuid root, which is required for sandboxing.

Because AppImages run from /tmp/.mount_*, Linux does not allow setuid binaries there.

✅ Fastest & Safest Fix (Recommended)

Run draw.io without the sandbox.

👉 Just run:

bash
./drawio-x86_64-29.0.3.AppImage --no-sandbox

This is safe for desktop apps like draw.io and is the official workaround.

🔁 Make it permanent (so you don’t type it every time)

Create a small wrapper:

bash
nano ~/bin/drawio

Paste:

bash
#!/bin/bash
~/Downloads/drawio-x86_64-29.0.3.AppImage --no-sandbox

Save (Ctrl+O, Enter, Ctrl+X) and make executable:

bash
chmod +x ~/bin/drawio

Now you can just run:

bash
drawio

🧠 Why this happens (short explanation)

draw.io is an Electron app (Chromium-based)
Chromium wants a setuid sandbox
AppImages mount under /tmp/.mount_*
Linux disallows setuid binaries there
Result → crash unless sandbox is disabled

🟢 Best Long-Term Solution (If you want zero issues)

Since you’re on Ubuntu/Mint/Lubuntu, the DEB package is better than AppImage:

bash
sudo apt install ./drawio-amd64-29.0.3.deb

✔ Proper sandbox
✔ Menu integration
✔ No flags needed
✔ Auto updates possible

🏁 Recommendation

Method	Stability
AppImage + `--no-sandbox`	✅ Works
AppImage without flag	❌ Crashes
DEB package	⭐⭐⭐⭐⭐ Best

If you want, I can:

Create a desktop icon for the AppImage
Help you switch cleanly to DEB
Explain security implications (briefly)

Just tell me 🙂

Tags: Technology,Linux,FOSS,

Friday, December 26, 2025

Accenture Skill Proficiency Test - Large Language Models - Dec 2025

See All: Miscellaneous Interviews @ Accenture

📘 Accenture Proficiency Test on LLMs

Q1. Few-shot Learning with GPT-3

Question (Cleaned)

You are developing a large language model using GPT-3 and want to apply few-shot learning techniques. You have a limited dataset for a specific task and want the model to generalize well. Which approach would be most effective?

Options:
a) Train the model on the entire dataset, then fine-tune on a small subset
b) Provide examples of the task in the input and let the model generate
c) LLMs are unable to handle single tasks

✅ Correct Answer

✔ b) Provide examples of the task in the input and let the model generate

💡 Hint

Few-shot learning = prompt engineering
No retraining required
You show the task via examples in the prompt

Q2. Edge AI with LLMs

Question

You are using an LLM for an edge AI application requiring real-time object detection. Which approach is most efficient?

Options:
a) Cloud-based LLM processing
b) Use a complex LLM regardless of compute
c) Use an LLM optimized for edge devices balancing accuracy and efficiency
d) Store data and process later
e) Manual input-based LLM

✅ Correct Answer

✔ c) Use an LLM optimized for edge devices

💡 Hint

Edge AI prioritizes low latency + low compute
Cloud = latency bottleneck
“Optimized” is the keyword

Q3. Improving Fine-tuned LLM Performance (Select Two)

Question

You fine-tuned a pre-trained LLM but performance is poor. What steps improve it?

Options:
a) Gather more annotated Q&A data with better supervision
b) Change architecture to Mixture of Experts / Mixture of Tokens
c) Simplify the task definition
d) Smaller chunks reduce retrieval complexity
e) Smaller chunks improve generation accuracy

✅ Correct Answers

✔ a) Gather more annotated data
✔ c) Simplify the task definition

💡 Hint

First fix data quality & task framing
Architecture changes come later
Accenture favors practical ML hygiene

Q4. Chunking in RAG Systems

Question

Why do smaller chunks improve a RAG pipeline?

Correct Statements:
✔ Smaller chunks reduce retrieval complexity
✔ Smaller chunks improve generation accuracy

💡 Hint

Retrieval works better with semantic focus
Too large chunks dilute meaning

Q5. Challenges of Local LLMs in Chatbots

Question

What is a potential challenge local LLMs face in long-term task planning?

Options:
a) Unable to adjust plans when errors occur
b) Unable to handle complex tasks
c) Unable to handle multiple queries
d) Unable to use task decomposition

✅ Correct Answer

✔ a) Unable to adjust plans when errors occur

💡 Hint

Local models lack persistent memory & feedback loops
Planning correction is the real limitation

Q6. RAG Pipeline – Poor Semantic Representation

Question

Why might embeddings not represent semantic meaning correctly?

Options:
a) Encoder not trained on similar data
b) Text chunks too large
c) Encoder incompatible with RAG
d) Incorrect chunk splitting
e) Encoder not initialized

✅ Correct Answers

✔ a) Domain mismatch in training data
✔ b) Chunk size too large

💡 Hint

Embeddings fail due to domain shift or context overflow
Initialization issues are rare in practice

Q7. Designing Advanced RAG – Chunking Decision

Question

Which is NOT a valid reason for splitting documents into smaller chunks?

Options:
a) Large chunks are harder to search
b) Small chunks won’t fit in context window
c) Smaller chunks improve indexing efficiency

✅ Correct Answer

✔ b) Small chunks won’t fit in context window

💡 Hint

Smaller chunks fit better, not worse
This is a classic reverse-logic trap

Q8. Intent in Chatbots

Question

What is the purpose of an intent in a chatbot?

✅ Correct Answer

✔ To determine the user’s goal

💡 Hint

Intent ≠ entity
Intent answers “what does the user want?”

Q9. Healthcare LLM Security

Question

Which strategy best ensures privacy and compliance for patient data?

Options:
a) Public API & public cloud
b) Layered security: encryption, access control, audits, private network
c) No security changes
d) Plaintext storage
e) Unverified 3rd party services

✅ Correct Answer

✔ b) Layered security approach

💡 Hint

Healthcare = defense in depth
Accenture loves encryption + audits + private infra

Q10. Edge AI Programming Language

Question

Which language is commonly used for developing ML models in Edge AI?

Options:

Java
Python
C++
JavaScript
Ruby

✅ Correct Answer

✔ Python

💡 Hint

Python dominates ML tooling
C++ is used for deployment, not modeling

Q11. Customizing METEOR Scoring

Question

How do you customize METEOR’s scoring function?

✅ Correct Answer

✔ Modify tool configuration or run with command-line flags

💡 Hint

METEOR supports custom weighting
Not hardcoded, no paid version needed

Q12. Bias Mitigation in LLMs

Question

First step when an LLM is found biased?

✅ Correct Answer

✔ Identify the source of bias

💡 Hint

Diagnosis before correction
Retraining comes later

Q13. DeepEval Tool – Advanced Features

Question

Which statement is correct about DeepEval advanced usage?

✅ Correct Answer

✔ Configure advanced features in Python scripts

💡 Hint

Tool-level configuration
No paid version needed

Tags: Interview Preparation,Generative AI,Large Language Models,

Accenture Skill Proficiency Test - Natural Language Processing (NLP) - Dec 2025

See All: Miscellaneous Interviews @ Accenture

📘 Accenture Skill Proficiency Test - NLP - Question Report

🧠 SECTION 1: NLP & WORD EMBEDDINGS

Q1. What is GloVe?

Extracted options (interpreted):

Matrix factorization of (raw) PMI values with respect to squared loss
Matrix factorization of log-counts with respect to weighted squared loss
Neural network that predicts words in context and learns from that
Neural network that predicts words based on similarity and embedding

✅ Correct Answer

✔ Matrix factorization of log-counts with respect to weighted squared loss

💡 Hint

GloVe = Global Vectors
Combines count-based methods (co-occurrence matrix) with predictive embedding ideas
Objective minimizes weighted squared error between word vectors and log(co-occurrence counts)

🧠 SECTION 2: TRANSFORMERS & CONTEXTUAL MODELS

Q2. In which architecture are relationships between all words in a sentence modeled irrespective of their position?

Extracted options:

OpenAI GPT
BERT
ULMFiT
ELMo

✅ Correct Answer

✔ BERT

💡 Hint

BERT uses bidirectional self-attention
Every token attends to all other tokens simultaneously
GPT = causal (left-to-right), not fully bidirectional

📊 SECTION 3: EVALUATION METRICS

Q3. Log loss evaluation metric can have negative values

Options:

True
False

✅ Correct Answer

✔ False

💡 Hint

Log loss = negative log likelihood
Probabilities ∈ (0,1) → log(probability) ≤ 0 → negative sign makes loss ≥ 0
Log loss is always ≥ 0

Q4. Which metric is used to evaluate STT (Speech-to-Text) transcription?

Extracted options:

ROUGE
BLEU
WER
METEOR

✅ Correct Answer

✔ WER (Word Error Rate)

💡 Hint

WER = (Insertions + Deletions + Substitutions) / Total words
Standard metric for speech recognition

🧩 SECTION 4: NLP ALGORITHMS & TAGGING

Q5. Best Cut algorithm works for:

Extracted options:

Text classification
Coreference resolution
POS tagging

✅ Correct Answer

✔ Text classification

💡 Hint

Best Cut / Min Cut → graph-based partitioning
Often used for document clustering / classification

Q6. BIO tagging is applicable for:

Extracted options:

Text classification
Coreference resolution
NER
N-grams

✅ Correct Answer

✔ NER (Named Entity Recognition)

💡 Hint

BIO = Begin – Inside – Outside
Used in sequence labeling tasks
Especially common in NER and chunking

🎯 SECTION 5: ATTENTION MECHANISMS

Q7. Which among the following are attention mechanisms generally used in neural network models?

Extracted options:

Bahdanau attention
Karpathy attention
Luong attention
ReLU attention
Sigmoid attention

✅ Correct Answers

✔ Bahdanau attention
✔ Luong attention

❌ Incorrect

Karpathy (not an attention mechanism)
ReLU / Sigmoid (activation functions, not attention)

💡 Hint

Bahdanau = additive attention
Luong = multiplicative (dot-product) attention
Activations ≠ attention

📚 SECTION 6: TEXT PROCESSING & TOPIC MODELING

Q8. Porter Stemmer is used for:

✅ Correct Answer

✔ Stemming words to their root form

💡 Hint

Example: running → run
Reduces vocabulary size
Rule-based, not dictionary-based

Q9. LDA stands for:

✅ Correct Answer

✔ Latent Dirichlet Allocation

💡 Hint

Probabilistic topic modeling
Documents = mixture of topics
Topics = distribution over words

Q10. Which of the following are true to choose optimal number of topics in LDA for topic modeling?

Extracted options:

Min coherence value
Max model perplexity
Max coherence value
Min model perplexity

✅ Correct Answers

✔ Max coherence value
✔ Min model perplexity

💡 Hint

Coherence → interpretability (higher is better)
Perplexity → generalization (lower is better)
Best practice: balance both

Tags: Interview Preparation,Natural Language Processing,

Pages

Wednesday, December 31, 2025