The final video in the `makemore` series involves making the previous model deeper with a tree-like structure. The end result is the convolutional neural network architecture inspired by WaveNet. What’s particularly interesting is the convolutional aspect, where a technique more commonly used for images is used to take advantage of the ‘for’ loop taking place inside the CUDA kernel. Emphasis is also placed on building out an experimental harness to track how training and validation loss changes over time.
Today we also had the final functional programming study group for the half-batch with the topic “What is functional programming?”. What I’ll continue to ponder beyond the session is the notion of ‘what is ergonomic’ vs ‘what is possible’.
For example, in Idris you can set a maximum size to a list and this check is enforced without any additional ‘ceremony', whereas in Python you’ll have to create either a custom class or a custom function to enforce this. Do all Turing-complete language allow you to do the 'same thing' but at varying levels of ease, or are there things that are possible to do in one language but impossible to do in another? Perhaps what's worth reflecting on is, is this even the right question?