Conclusion

I worry I've been a bit grumpy. But I'm sure the Internet is the right place for that. Time for some nuance, so here's what I think:

• Metaphors are powerful ways to think about problems, very useful for getting a grip.
• They're limited, and we need to be aware of those limits.
• It's a good idea to be aware of when we're doing metaphor thinking, and critique appropriately.

Maybe we need some more metaphors; that'd be fun! My ex-colleague said training deep learning models is like cooking. I like that. But sometimes we need to be careful:

Like a pair of shoes that eventually wears out, or start walking you in the wrong direction; wait, no, it turns out they were the wrong shoes all along (dammit, shoe metaphor)... we need to know when our metaphors might break down.

• Introduction
• A Neural Network
• The Loss Landscape
• Attention
• Mixture of Experts
• Finally, a conclusion, thanks for reading!

Graveyard

Here are some others I considered, but didn't write about. Perhaps they're not even metaphors. But in the abstract world of computer science and machine learning, most of our words started out as metaphors. So maybe it's just a matter of perspective.

• Model
• Layer
• Checkpoint
• Agent
• Policy
• Adversary
• Inference/Training
• Framework/Library