Data Science & Developer Roadmaps with Chat & Free Learning Resources
Turning Up the Heat: The Mechanics of Model Distillation
When I first read this paper, I was struck by twin impulses. The first was that I should absolutely write a post explaining it, because of how many of its ideas are elegant and compelling — from its…
Read more at Towards Data Science | Find similar documentsWhat is Knowledge Distillation?
Knowledge distillation is a fascinating concept, we’ll cover briefly why we need it, how it works.
Read more at Towards Data Science | Find similar documentsOn DINO, Self-Distillation with no labels
It has been clear for some time that the Transformers had arrived in the field of computer vision to amaze, but hardly anyone could have imagined such astonishing results from a Vision Transformer in…...
Read more at Towards Data Science | Find similar documentsUsing Distillation to Protect Your Neural Networks
Distillation is a hot research area. For distillation, you first train a deep learning model, the teacher network, to solve your task. Then, you train a student network, which can be any model. While…...
Read more at Towards Data Science | Find similar documentsDistill Hiatus
Over the past five years, Distill has supported authors in publishing artifacts that push beyond the traditional expectations of scientific papers. From Gabriel Goh’s interactive exposition of momentu...
Read more at Distill | Find similar documentsKnowledge Distillation : Simplified
Neural models in recent years have been successful in almost every field including extremely complex problem statements. However, these models are huge in size, with millions (and billions) of…
Read more at Towards Data Science | Find similar documentsDistilling Step-by-Step : Paper Review
Exploring one of the most recent and innovative methods in LLM compression Continue reading on Towards AI
Read more at Towards AI | Find similar documentsPatient Knowledge Distillation
With the advent of deep learning, newer and more complex models are constantly improving performance on a variety of tasks. However, this improvement comes at the cost of computational and storage…
Read more at Towards Data Science | Find similar documentsDistill Update 2018
Things that Worked Well Interfaces for Ideas Engagement as a Spectrum Software Engineering Best Practices for Scientific Publishing Challenges & Improvements The Distill Prize A Small Community Revie...
Read more at Distill | Find similar documentsTernaryBERT: Quantization Meets Distillation
The ongoing trend of building ever larger models like BERT and GPT-3 has been accompanied by a complementary effort to reduce their size at little or no cost in accuracy. Effective models are built…
Read more at Towards Data Science | Find similar documentsKnowledge Distillation — A Survey Through Time
In 2012, AlexNet outperformed all the existing models on the ImageNet data. Neural networks were about to see major adoption. By 2015, many state of the arts were broken. The trend was to use neural…
Read more at Towards Data Science | Find similar documentsExtract Cooling for Espresso
Coffee Data Science Applying modern techniques to hack flavor Espresso is generally brewed hot, but one of the downsides is that aroma continues to be lost to evaporation. Generally, there is a notio...
Read more at Towards Data Science | Find similar documentsEspresso Preparation: Grinding, Distribution, and Tamping
Previously, we looked at pre-infusion, pressure, and temperature with a few data sources to understand what provided the best extraction. Now, let’s look at grinding, distribution, and tamping. The…
Read more at Towards Data Science | Find similar documentsComparing Methods for Measuring Extraction Yield in Espresso
When brewing coffee, the main quantitative quality metric is Extraction Yield (EY). EY is determined in two ways: drying spent coffee and measuring Total Dissolved Solids (TDS) to compute EY. I…
Read more at Towards Data Science | Find similar documentsDistillation of Knowledge in Neural Networks
Currently, especially in NLP, very large scale models are being trained. A large portion of those can’t even fit on an average person’s hardware. Plus, due to the Law of diminishing returns, a great…
Read more at Towards Data Science | Find similar documentsThe Duality of Coffee: An Extract and a Filter
Previously, I experimented with trying to turbo charge my espresso shot by feeding espresso back into the puck to extract more coffee. Instead, I found that used or spent coffee grounds filters…
Read more at Towards Data Science | Find similar documentsPeeling Back the Mystery of Espresso Extraction in Staccato
I have examined extraction by layer for staccato and staccato tamped espresso before, but those shots were usually 3 to 1 (output to input). Most of my shots are split between the 1:1 and the rest of…...
Read more at Towards Data Science | Find similar documentsEspresso Extraction by Layer
While developing the staccato shot, I noticed the spent coffee layers stayed separated after the shot. I decided to dry out some of these pucks to take a look at coffee extraction per layer. I was…
Read more at Towards Data Science | Find similar documentsThe Power of Knowledge Distillation in Modern AI: Bridging the Gap between Powerful and Compact…
What is Knowledge Distillation? At its core, knowledge distillation is about transferring knowledge from a large, complex model (often called the teacher ) to a smaller, simpler model (the student ). ...
Read more at Towards AI | Find similar documentsDissecting Coffee Extraction: Part 2
Coffee Data Science Splitting extraction by layer Previously, I looked at some extraction for dried coffee pucks, and some of the shots had a paper filter in between layers. I used this paper filter ...
Read more at Towards Data Science | Find similar documentsStaccato Tamping: Improving Espresso without a Sifter
Most of my coffee experiments are done casually throughout the day. Because of the shelter in place orders, I moved my coffee bar to my house, and I’ve had a fair amount of lattes and non-staccato…
Read more at Towards Data Science | Find similar documentsDecrease Neural Network Size and Maintain Accuracy: Knowledge Distillation
Practical machine learning is all about tradeoffs. We can get better accuracy from neural networks by making them bigger, but in real life, large neural nets are hard to use. Specifically, the…
Read more at Towards Data Science | Find similar documentsThe Diminishing Returns of Tamping for Espresso
This experiment started from a Facebook discussion in an espresso group about tamping vs tapping. The thinking was tamping compresses grounds at the top and the bottom more than the middle, and this…
Read more at Towards Data Science | Find similar documentsExtraction over the Life of the Coffee Bean
I’ve been collecting data on my shots for almost 2 years. I wanted to look through that information to see if I could see any trends, particularly the age of a roast. The main caveats are: So I…
Read more at Towards Data Science | Find similar documents- «
- ‹
- …