Header
Comparison of Data Augmentation Technics

Data Scientist

Hello, I'm Takuoko, Kaggle Grandmaster.

In this post, I would like to compare data augmentation technics, for both image classification and testing. Data augmentation is a powerful technique in CV competition.
I have compared the techniques used for image classification, which is the most standard in CV competition.

As a member of the Z by HP & NVIDIA Global Data Science Ambassador, this article is an experiment supported by Z by HP and NVIDIA.
# development environment

Z8G4 powered with 2x RTX 6000    ZBook Studio powered with RTX 5000

I am working on a CV competition with the above powerful systems. Compared to the time when I used V100, I now have systems that can run freely at any time and have a higher speed, so the number of experiments that I can run has increased greatly compared to before, and I can also run comparative tests of various papers.

Environments such as pytorch PyTorch and cuda CUDA are pre-installed in the systems, so there was no need for me to build the environment.

# Comparison images of data augmentation techniques

I would like to compare two images of CIFAR-100 by mixing them. Some of the figures are cited from papers.

 ## Mixup

[paper with code](https://paperswithcode.com/paper/mixup-beyond-empirical-risk-minimization)

A technique of mixing two images with lam * img1 + (1 - lam) * img2.

## Manifold Mixup

[paper with code](https://paperswithcode.com/paper/manifold-mixup-better-representations-by)

Example of mixup in the middle layer.

## CutMix

[paper with code](https://paperswithcode.com/paper/cutmix-regularization-strategy-to-train)

A technique in which one image is cut out with a bbox of a certain size and pasted into the other image.

There are two methods: corresponding paste which pastes at the same position as the cut-out position, and random paste which pastes at random positions.

## PatchUp

[paper with code](https://paperswithcode.com/paper/patchup-a-regularization-technique-for)

A method of running Cutmix in the middle layer.

## ResizeMix

[paper with code](https://paperswithcode.com/paper/resizemix-mixing-data-with-preserved-object)

A technique in which one image is resized and pasted into the other.

 ## fmix

[paper with code](https://paperswithcode.com/paper/understanding-and-enhancing-mixed-sample-data)

 Compared to CutMix, a mask can be generated and mixed regardless of shapes (does not have to be square shaped).

## SnapMix

[paper with code](https://paperswithcode.com/paper/snapmix-semantically-proportional-mixing-for)

Reduce background image noise by using CAM to weight the label after mixing. Figure is cited from the paper.

## PuzzleMix

[paper with code](https://paperswithcode.com/paper/puzzle-mix-exploiting-saliency-and-local-1)

Reduces background noise by overlapping important parts. Figure is cited from the paper.

# Comparative testing of data augmentation technics

Test Settings
Dataset:Kaggle’s [Cassava Leaf Disease Classification](https://www.kaggle.com/c/cassava-leaf-disease-classification)

Model:resnet34d
Loss:CrossEntropyLoss
Optimizer:Adam+Lookahead
Image Size:256
Batch Size:64
Epochs:20

Results

Technic

5fold CV

CutMix random paste

0.8708

CutMix corresponding paste

0.8694

PatchUp

0.8694

ManifoldMixup

0.8692

ResizeMix

0.8686

Cutmix corresponding paste + Mixup

0.868

Fmix

0.8675

PuzzleMix

0.8662

Mixup

0.8661

Summary
As noted in ResizeMix's paper, random paste was more accurate for Cutmix. ResizeMix, on the other hand, was less accurate than Cutmix, which is a different result from the paper.

Fmix's paper also mentioned that Cutmix + Mixup and Fmix + Mixup were more accurate than either method alone, but this could not replicated.

As for the methods, I felt that random paste and ResizeMix were methods which needed careful consideration of where to apply, since accuracy would be low in image sets in which the position is fixed in the whole image sets, such as medical images. Technics such as PuzzleMix and SnapMix are likely to be more effective for tasks in which the subtle points hold importance.

Further hopes
I will continue to use the systems supported for Z by HP & NVIDIA Global Data Science Ambassadors to compare and test methods of various papers in CV competitions.