Skip to content

[WIP] [GenAI] Lora Finetune #7288

Closed
Closed

Description

@LittleLittleCloud

Lora fine-tuning is an adapter-based technique to fine-tune an LLM. It changes LLM model architecture by adding learnable lora layers to transformers. During fine-tuning, only lora weights are adjustable and the LLM weights are frozen, so it requires much less GPU memory comparing to a full-layer fine-tuning. Based on this table, it requires 16GB memory to fine-tuning a 7B size model in 16bits, which can be fit in rtx 3090, 4080 and 4090. A wider range of GPUs can be fit on 3.8B LLMs like phi-3.5-mini

API design (wip)

Package: Microsoft.ML.GenAI.Lora

interface ICausalLMLoraPipeline {} // pipeline for loading causal LM + lora layers

class LoraConfiguration // lora configuration

Activity

  1. michaelgsharp commented on Mar 10, 2025

    @michaelgsharp
    Contributor

    @LittleLittleCloud could you add this to a Milestone for when you are planning on getting this done by?

  2. added
    area-GenAIGenerative AI models and APIs
    and removed on Sep 8, 2026
  3. rosebyte commented on Oct 14, 2026

    @rosebyte
    Member

    Triage: we have discontinued the GenAI packages.

  4. locked and limited conversation to collaborators on Nov 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

area-GenAIGenerative AI models and APIsenhancementNew feature or request

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions