AI Computing Systems (EE219)

School of Science and Technology (SIST), ShanghaiTech University

Shanghai Institute of Semiconductor Innovation (SISI)

Lab 2

Introduction

In this lab, you will study model compression techniques on the CIFAR-10 CNN from Lab 1. This lab covers pruning and INT8 post-training quantization (PTQ). Complete the TODOs and answer questions in this notebook.

Lab Overview

Lab Tasks

Study model compression on the Lab 1 CIFAR-10 CNN across four required sections:

  • Measure the original model (FP32 baseline): accuracy, parameter count, serialized state_dict bytes, and CPU latency.
  • Apply global unstructured pruning to produce a pruned model and fine-tune it, then analyze the sparsity recovery and how sparsity relates to dense model size and measured latency.
  • Explore structured pruning as a design-space study, including the six dims = [0, 1] × ratios = [0.1, 0.3, 0.5] configurations and the required four-dimension comparison.
  • Apply FX graph-mode INT8 post-training quantization (PTQ) to produce the quantized model, with observer preparation, calibration, and conversion.

Models are named by their state: the original model (the unmodified Lab 1 FP32 checkpoint), a pruned model (zero-masked by pruning), the quantized model (the INT8 model produced by FX PTQ), and the pruned and quantized model (a pruned model that is quantized afterwards).

Grading (100 pts): FP32 baseline: 10 pts. Global unstructured pruning: 30 pts. Structured pruning exploration: 25 pts. FX INT8 PTQ: 35 pts. Knowledge distillation is an optional bonus and is not part of the required total.

Note: The description on this page provides only a summary of the lab. Please refer to the provided IPython notebook for the detailed requirements and instructions.

Lab Files

Download the Lab 2 notebook: EE219-Lab2.ipynb. Keep it in Lab2/.

This lab reuses the Lab 1 model and data: use the Lab 1 checkpoint ../Lab1/model_2026fall.pth and the CIFAR-10 data in ../Lab1/data/CIFAR10/ (input shape 3x32x32). See Lab 1 if you need the Lab 1 files.

All supplied helper functions are embedded in the notebook; do not edit them. Complete the student TODOs, run the experiments, and answer the questions.

This lab measures CPU latency, and the FX PTQ part is a CPU experiment. Report latency as a hardware-dependent measurement.

Requirements

Complete the provided EE219-Lab2.ipynb.

  1. Complete the code independently.
  2. Make sure your results are reproducible.
  3. Do not fine-tune the structured pruning models.
  4. Use prune.ln_structured with L2 norm (n=2) on Conv2d weights only.
  5. Keep the supplied helper functions unchanged.

Learning Objectives

  • Understand the difference between parameter sparsity, model size, computation cost, and measured latency.
  • Apply unstructured pruning and analyze sparsity recovery through fine-tuning.
  • Explore how structured pruning choices affect neural network structure and deployment cost.
  • Apply FX graph-mode INT8 post-training quantization with observer preparation, calibration, and conversion.

Structured Pruning Experiment

The pruning ratio is the fraction to prune (zero-mask), not the fraction to retain, and ranges from 0 to 1. For unstructured pruning it applies to individual weights; for structured pruning it applies to groups along the selected dimension in each layer. For example, ratio=0.3 requests pruning 30% and retaining 70%. Structured pruning rounds to a whole number of groups, so the effective ratio may differ. PyTorch pruning APIs name this argument amount; pass amount=ratio.

Evaluate structured pruning as a design-space exploration. Compare dims = [0, 1] with ratios = [0.1, 0.3, 0.5] (six configurations), then run the required four-dimension comparison with dims = [0, 1, 2, 3] at ratio = 0.25. Do not fine-tune these models. Use prune.ln_structured with L2 norm (n=2) on Conv2d weights only:

Dimension Meaning Expected observation
dim=0 Output channel/filter pruning Creates zero output filters; graph rewriting can reduce convolution computation
dim=1 Input channel pruning Creates zero input-channel slices; connected layers must be consistent in a physically smaller model
dim=2 Kernel-height slice pruning Creates zero kernel rows; weight shapes are unchanged, so dense size and latency do not automatically improve
dim=3 Kernel-width slice pruning Creates zero kernel columns; same caveat as dim=2

For every combination, evaluate accuracy and collect compression_report values. Display dim, ratio, accuracy, parameters, sparsity, and CPU latency in the summary table; retain nonzero parameter count and serialized bytes in structured_results.

Inspect a masked model with print(model) and compare its weight shapes and zeroed groups. A structured mask does not automatically create a smaller dense model; explain the additional model rewrite needed to exploit removed channels.

The four-dimension comparison at ratio=0.25 is required, and its results are graded as part of Question 3.

Grading

Required section Code Analysis Total
FP32 baseline 0 10 10
Global unstructured pruning 15 15 30
Structured pruning exploration 10 15 25
FX INT8 PTQ 25 10 35
Required total 50 50 100

Bonus

Bonus (optional): knowledge distillation. See the bonus section of the notebook for the questions; the bonus does not replace any required task.

Submission

Before submitting:

  • Submit only the completed EE219-Lab2.ipynb, including answers, experimental results, and a comparison table.
  • Record accuracy, total/nonzero parameters, sparsity, serialized bytes, and CPU latency for every result.
  • Record the PyTorch version, CPU quantization backend, calibration samples, seed, and measurement hardware.

Important

Report CPU latency as a hardware-dependent measurement; sparsity alone does not imply speedup. FX PTQ is a CPU experiment, not a direct RTL export.