The Medium post goes over various flavors of distillation, including response-based distillation, feature-based distillation and relation-based distillation. It also covers two fundamentally different modes of distillation – off-line and online distillation.

Since Chinese artificial intelligence (AI) start-up DeepSeek rattled Silicon Valley and Wall Street with its cost-effective models, the company has been accused of data theft through a practice that is common across the industry.

David Sacks says OpenAI has evidence that Chinese company DeepSeek used a technique called "distillation" to build a rival model.