In my previous post, I shared thoughts on building patience. A discussion followed on RC chat which highlighted two aspects of patience.
OK we’re back on to 'sharing sketches of my blog post ideas along the way’, here we go.
One powerful design strategy, which is particularly appropriate to the construction of programs for modeling physical systems, is to base the structure of our programs on the structure of the system being modeled.
I had been wondering what to write about to recap Week 1 of RC. I came across a post by Cindy Wu sharing her RC application, which inspired me to review my own. My plan at RC has narrowed a fair bit. To the question "What would you like to work on at RC?”, I described two ideas.
The first is to work through the Haskell, Prolog and Idris sections of Seven Languages in Seven Weeks. I love seeing the trade-offs different languages make, and keen to extend functional programming best practices to imperative code. This would also be an extension of the SICP course I completed recently. A lingering wonder I have is how concepts like composition help frame abstractions for domain-specific software - what's the reason Cognitech joined Nubank and Jane St love OCaml?
The second is to build a toy ChatGPT clone. I've build [sic] machine learning and neural network models from scratch before (and presented on them at RC and at my workplace), but generative models are more recent and a lot more resource-intensive. My plan is to spend about two weeks to get a basic understanding how they work, and either build one from scratch or use an existing model and 'retrain' one to write in my writing style say.
I would be more deliberate in listening to what my batch mates are interested in, and use that to choose on things to work on based on what I can collaborate with and pair on.
There's something lethargic at best and dangerous at worst about idleness.
In my previous post, I shared snippets from my RC application and how my plans have changed over the first week. This was inspired by Cindy Wu sharing her own RC application, amusingly when she had trouble writing about RC.
What can I say? Week 1 of RC has been a blast! It’s affirming seeing the diversity of backgrounds and knowledge. As a self-taught programmer, this week helped illuminate how the path towards mastery is not linear. It’s multi-dimensional.
Maybe the first run, they took an idea they’ve had for a decade or three decades and they expounded on it and it sounded great. And then in the sequel, whether it’s a book or a movie, they have to come up with something equally good, but they only have two years to do it. And it’s just really difficult to do.
In a previous post, I shared how my plan at RC was inspired by the courses I did with David Beazley. I wasn’t sure then if I was paraphrasing him accurately, but yesterday he wrote in his newsletter:
Meanwhile, I've managed to dig myself into a pretty deep rabbit hole as a result of the month-long courses I offered back in January. The extra time allowed me to make an attempt at writing the compiler project in Haskell, which to be honest, kind of blew my mind and exposed a totally different way of doing things in Python. I'll have more to say about that once I finish putting the pieces of my brain back together.
Lazy evaluation makes it hard to reason about when things will be evaluated; hence including side effects in a lazy language would be extremely unintuitive. Historically, this is the reason Haskell is pure: initially, the designers of Haskell wanted to make a lazy functional language, and quickly realized it would be impossible unless it also disallowed side effects.
foldl (+) 0 [1, 2, 3]
= foldl (+) (0 + 1) [2,3]
= foldl (+) ((0 + 1) + 2) [3]
= foldl (+) (((0 + 1) + 2) + 3) []
= (((0 + 1) + 2) + 3)
= ((1 + 2) + 3)
= (3 + 3)
= 6
foldr (+) 0 [1, 2, 3]
= foldr (+) (3 + 0) [1, 2]
= foldr (+) (2 + (3 + 0)) [1]
= foldr (+) (1 + (2 + (3 + 0))) []
= (1 + (2 + (3 + 0)))
= (1 + (2 + 3))
= (1 + 5)
= 6
For non-technical talks, I presented on payments and chargebacks. In the spirit of the talk being non-technical, I left out the parts on the software stack and ML models (perhaps for a different post). The write-up below is roughly what I talked about.
Today I prepared slides for Friday’s presentation. I always find it helpful to a dry run the day before, I’m told magic happens when you sleep on it.
I was happy to draw a line connecting SICP to Haskell, and it’s a good periodic reminder when delving into functional programming. What's new is I feel pulled into a lot of theory (lambda calculus, type theory, category theory, proofs), discover this is normal, and spending time on all this doesn’t even cover what the compiler does in practice.
Re: compiler, I was previously under the impression that lazy evaluation can be inefficient because you’re doing the same calculation more than once. For example, in calculating `square (1 + 2)`, you have to do `(1 + 2)` twice in `((1 + 2) * (1 + 2))` instead of just once when do the sum first before you square. This is from CIS 194 lecture notes.
In particular, GHC uses a technique called graph reduction, where the expression being evaluated is actually represented as a graph, so that different parts of the expression can share pointers to the same subexpression.
stack exec cis194-exe 2.27s user 0.85s system 116% cpu 2.672 total
stack exec cis194-exe 13.14s user 2.97s system 101% cpu 15.849 total
There are times I’m stuck not knowing what to write about, and resign myself to the possibility that the post may end up being just a single paragraph that I forced myself to write. Then I start writing that paragraph and end up with a few paragraphs, and wonder how I resigned myself to writing just one.
At times I feel going into Haskell is escapism. Let's stick to it for a week and see where this goes.
One last thing: don't let anyone tell you that the tech/engineering is the easy part. It's not. It's hard. Soft skills are also hard. It's ALL hard, and both are required to succeed.
To those who maintain a blog and wonder if you’re putting in too much effort relative to the value you get out of it: (1) Let’s chat (2) The person who finds this blog most helpful is myself.
Different people have different learning styles. The style that I find works well is to go through the material end-to-end in multiple iterations. The first pass usually as fast as I can for the high-level overview, then hone in on specific sections. Perhaps this is why learning SICP with David Beazley was pure joy.