Why it matters

Data quality is scaling frontier. Understanding shapes strategy.

Advertisement

The architecture

Quality filters: perplexity, classifiers, human review.

Synthetic data augmentation.

Domain curation.

Data quality scalingFilter noisyperplexity + classifiersCurate domainsweb + code + papersSynthetic augmentquality dataPhi-3 small model + quality data matches larger noisy models; growing pattern
Data quality.
Advertisement

How it works end to end

Phi paper: small + quality = large + noisy.

FineWeb-Edu, DCLM curation.

Synthetic data (Alpaca, WizardLM).

Dedup essential.