Metaphor thinking in deep learning

As a deep learning researcher, I find my self doing metaphor thinking quite a lot. Metaphor thinking is where you leap into a metaphor, a figure of speech, and use it to explore a subject as if it exists wholly within the metaphor. I suppose "to leap in & explore" in that sentence was itself metaphor thinking (or metaphor speaking).

Metaphor thinking is a very powerful tool, but also quite dangerous, as our metaphors often don't quite match up to reality. This means that conclusions reached in metaphor thinking aren't always valid in the real world.

I thought it'd be fun to consider a few common deep learning metaphors and critique them. I've written a short explanation and discussion of four that I find particularly interesting in this series.

Health warning: to keep things simple I haven't said "in my opinion" every sentence, as it gets tiresome. But it is all just the opinion of an ignorant & poorly-read researcher/engineer, please take it as such! This series is therefore also very light on facts and even properly constructed arguments. Forgive me.