Hello, World!
I recently began working on The Test Cabinet, which is a benchmark I created to evaluate large language models and harnesses on large-scale software engineering challenges. And by large scale, I don’t mean “write 1000-2000 lines of code”. I mean more like “Write a game engine from scratch and implement a small game on top of it”. That has shown not only the very obvious limits of current LLMs, but also how capable they’ve gotten.
While LLMs fail pretty hard at the more challenging test cases, they’re now good enough that it’s trivially simple to use them for one-off implementations or simple projects, which opens up a new opportunity that wasn’t previously available to me.
I taught myself to code by looking for tutorials and other learning resources whenever I wanted to implement something that I didn’t know how to write myself. Searching for material to teach myself new techniques led me to some remarkably useful sites, written by incredibly talented people. Gaffer on Games taught me a ton about writing networking code for games. Red Blob Games has a ton of miscellaneous content covering all sorts of topics useful for game developers. And Cat Like Coding’s hex map tutorial is one of the best I’ve ever seen for how to generate geometry for hex terrain.
What all of these blogs have in common is a heavy emphasis on visuals and/or interactive examples. For example, the Gaffer on Games post about Deterministic Lockstep includes side by side comparisons showing the effects of non-determinism. Red Blob Games’ A* article includes animations that you can play step by step and examples where you can modify the map and watch the algorithm adjust in real time. And having written plenty of code for generating meshes from data, Cat Like Coding’s heavy emphasis on visuals is invaluable for working out how complex graphical techniques can be generated entirely in code.
My goal for this blog is to explore various techniques and write them up as articles or tutorial series that go through how the techniques work, when they’re applicable, and how to build them yourself. This is significantly more viable now that LLMs exist, since I can focus on what the techniques I want to show are and let the LLM generate the code demonstrating the technique. I expect I’ll end up handling this largely using Rust compiled to WASM, which is how this example is handled:
Loading simulation…
I hope this blog will end up being as valuable to others as the blogs that helped teach me.