The first draft of this post was handwritten on paper. This was by design.
I decided to switch gears to learning how ChatGPT works under the hood, post-conversation with a friend last week. I also took the opportunity to update my LinkedIn profile. This was something I was initially hesitant about doing; partly to concentrate on functional programming, partly to spend a bit of time more deliberately thinking what I'd like to do next.
At the start of RC, I wasn't sure whether to learn functional programming or spend time on Andrej Karpathy's videos. Why Andrej Karpathy? I really enjoyed the material he put together for the Stanford CS231n course on computer vision, which then inspired me to come up with a 1-hour intro on neural networks for SquareU.
I spent a bit of time going through the video on a minimal backprop; it's deceptively simple. On a related note, the intro above had a graphic on how the weights change when it goes through stochastic gradient descent but only for a 1-layer network.
The rest of the day I used ChatGPT as a consumer. The first use case was creating Anki flashcards; I did a lot of courses at Bradfield but I've also forgotten a lot of what I learned. GPT-4 is very good at creating flashcards. I would use the text from OSTEP into the prompts for the final version, but the question-and-answers sets it came out with when I tried prompting without any OSTEP text (hence completely made up) were impressive!
What I wasn't able to get GPT-4 to do was implement Raft in Python. I copied the the tests from my implementation into the prompt, and tried to get GPT-4 to write code that would pass the tests. If errors were raised, I would paste the error in full as a prompt to get the next iteration. This failed on the same test 10 times in a row. Next time I'll try helping it along after repeated failures, to see how far we end up.
The amusing part was how, after a few failed iterations, ChatGPT tried to change the function's parameters from 4 to 5 and asked me to change the tests accordingly! I shared this with a friend, who then responded as follows.
A candidate tried that with me during an interview. Human level AI.