Thanks to Go's profiling tools, I discovered that the memoization, while it was cutting down runtime significantly, was itself slow because it was using arrays. Swapping those arrays out for maps made a _massive_ difference (4s/14s part1/part2 to 1ms/2ms with no other changes). Lesson learned. Again.
The IntHeap rename was long overdue since I took the code originally from Go's sample docs for priority queues.