Lunar Boom Learning
Chapter 4
Chapter 4
Training and Fine-Tuning a Music Model
From optimisation and model adaptation to reproducible experiments and systematic debugging
Chapter 3 explained how a trained model generates music. This chapter examines how that behaviour is learned, adapted, tested, resourced, monitored, and repaired. It follows the complete practical path from one optimisation step to a reproducible and debuggable music-model experiment.
What you’ll learn
- Trace a complete model-training step from batch loading and forward prediction to backpropagation and parameter updates.
- Distinguish training from scratch, continued pretraining, full and partial fine-tuning, adapters, LoRA, conditioning modules, and inference-only use.
- Choose an adaptation strategy based on model compatibility, data, compute, rights, deployment, and retention requirements.
- Design a small symbolic-music experiment with a narrow hypothesis, grouped data splits, reproducible tokenisation, baselines, and predefined decision criteria.
- Distinguish underfitting, generalisation, overfitting, leakage, duplication, memorisation, and data replication.
- Explain how model size, sequence length, batch size, precision, trainable parameter count, and optimiser state affect training memory and compute.
- Design recoverable checkpoint, storage, experiment-tracking, and local, cloud, or distributed training workflows.
- Debug data, optimisation, conditioning, generation, and infrastructure failures using small reproducible tests.
Questions to carry with you
How does prediction error become a parameter update?
When should a project fine-tune an existing model instead of training a new one?
What can adapters, LoRA, and new conditioning modules change?
What makes a music-generation experiment small, controlled, and reproducible?
How can a model improve on training data while becoming worse on unseen music?
How can generated music be audited for memorisation or reproduction of training examples?
What occupies GPU memory during training, and which techniques reduce each component?
How can a broken data, conditioning, or decoding pipeline imitate model failure?
What evidence is needed to identify and verify the root cause of a failed training run?
Sections
Section 4.1 · 15 min
What Happens During Training?
Follow a complete training step from loading a batch and making predictions to calculating loss, backpropagating gradients, updating parameters, validating progress, and beginning the next step.
Section 4.2 · 15 min
Training From Scratch Versus Fine-Tuning
Compare training a music model from random initialisation with continued pretraining, full and partial fine-tuning, adapters, LoRA, new conditioning modules, and inference-only use.
Section 4.3 · 16 min
Designing a Small Experiment
Design a reproducible symbolic melody experiment with a narrow objective, traceable MIDI data, leakage-resistant splits, simple baselines, controlled model comparisons, and clear success and stopping criteria.
Section 4.4 · 16 min
Overfitting and Memorisation
Learn how to distinguish useful generalisation from overfitting, split leakage, duplicate exposure, memorisation, and exact or approximate reproduction in generated music.
Section 4.5 · 16 min
Compute and Infrastructure
Learn what occupies GPU memory, how music representation and sequence length affect training cost, when memory-saving and distributed techniques help, and how to build a recoverable local, cloud, or hybrid workflow.
Section 4.6 · 16 min
Debugging Model Training
Learn how to reduce a failed training run to a reproducible test, verify each stage of the pipeline, interpret loss and gradient symptoms, diagnose conditioning failures, and separate model problems from generation or infrastructure problems.
Chapter recap
Chapter takeaway
Chapter 4 follows the complete model-development workflow. It begins with the optimisation loop, compares ways to reuse or adapt pretrained models, turns an idea into a controlled symbolic-music experiment, examines overfitting and memorisation, plans the compute and storage system, and ends with a structured method for debugging failures.
Training repeatedly converts prediction errors into gradients and optimiser updates, but the loss function determines what behaviour the model is actually encouraged to learn.
Model adaptation exists on a spectrum from inference-only use to training a complete model from scratch.
The smallest adaptation capable of expressing the required change is often the most practical starting point.
A useful experiment requires traceable data, fixed representations, grouped splits, simple baselines, controlled comparisons, and predefined success and stopping criteria.
Overfitting is a generalisation failure, while memorisation concerns retention of example-specific information. They must be tested separately.
Duplicate exposure, split leakage, and weak validation design can make training results appear stronger than they are.
Training feasibility depends on parameters, gradients, optimiser states, activations, sequence length, batch settings, precision, storage, and data throughput.
Mixed precision, gradient accumulation, activation checkpointing, parameter-efficient tuning, and distributed sharding address different resource bottlenecks.
Debugging should identify the earliest incorrect pipeline stage using a reproducible failure, a fixed batch, targeted assertions, controlled ablations, and verification tests.
A model that trains successfully is not automatically a model that is musically strong, safe, original, governable, or ready for release.
Chapter 4 established how to train, adapt, test, resource, and debug a music model. Chapter 5, Evaluating, Releasing, and Governing the Model, examines how to judge musical quality and control adherence, assess safety and replication risks, document provenance, communicate limitations, monitor deployment, and make responsible release decisions.
Check your understanding
Chapter 4 Assessment
Review the optimisation loop, model-adaptation choices, experiment design, generalisation and memorisation, compute infrastructure, and systematic debugging.