Sorry, man, we never answered your basic question: ya, you would want to clean up the deck as best as possible with light rotating disc, green scratch pads, whatever. But, it is best to keep the crudskis from falling in your holes.
A clean hole is a happy hole!
Clean up the head, too and be careful with the soft aluminum surface. I suppose you could check out how many dents you have in the pistons and what the bores look like. Sometimes, stuff like this can be the proverbial "Pandora's box."
Sometimes, ignorance is bliss.
Added: Lucas would want me to remind you to make sure and NOT chamfer the holes (oil and water) when cleaning with abrasives so that you don't encourage inefficient gasket sealing.