Welcome Puddleglum, Even Lurkers finally come out of the dark to get additional help.
So ya gots a 1.5L eh? Not so bad, some say the ideal Classic Rabbit engine. After all, it was #1.
So let's go down the list of work you have completed and see if there is anything additional to do.
WORK DONE:
Replaced head gasket when it failed recently Did you check that head for proper tolerance of like .004 thou of an inch?
Replaced timing belt Always a good thing on an unknown mileage or serviced engine.
Installed new injector nozzles and sent them out for pop-testing and balancing. OK, so system delivery is solid from that point on but IP? I will come back to this if you haven't listed it below.
New rubber fuel lines Good to have
New fuel filter A must to have.
New engine mounts (except mount under radiator which looked fine) Ahh, perhaps it is not as good as it needs to be, or is hard as a rock so it doesn't absorb any vibration?
Engine “aligned” per Bentley manual OK, that is generally a miss for most folks.
I don’t have dial indicator adapter so timed by “ear”. Ahh, this is not going to work. Get a dial gauge so you know your numbers as old engines may have less pressure in their IP pumps and even running the right # of mm of advance they can still be out of time with the performance. Knowing where you are at for a starting point is a gotta have. At least in MHO.
OTHER NOTES:
-Before head gasket blew, I was getting compression overall just below spec with two cylinders lower (I forget how much, something like 10-15 lower). I am planning to recheck when I can borrow the tool. The addition of a new gasket may have helped get you back to spec but maybe not. 1980s motor, may need a new set of rings to be close to spec and if you live in a cold climate you will need that compression to get it to fire up when below 20 F.
-When I changed the gasket, I found the old was a four notch and I measured for and installed a three notch gasket.
If you measured that is kudos to you for doing that. Protrusion is so important with that diesel engine.
-Cylinder walls and valves all looked visually good to my untrained eye while it was apart. No scoring or burning. I do not have a perfect straight edge, but by comparing various checks, I do think the head is a bit warped. Maybe 4 thousandths? I can’t be sure with the straightedge I have. Ahh, so you did check and are not sure about how good it was, that answers my previous comment on that item. A compression check should verify any leaks so get the tools back.
-Since head gasket, now engine starts more easily. Always caught instantly if warm, but now with glow plugs it starts quickly cold. It hated cold starts before. So you gained some compression, but is it balanced or just making starting easier. Tools and check, please.
-The technician at the diesel shop says to get the pump rebuilt, but that’s $700 to him so he would be biased. I also have heard the pump rarely goes bad. Ok here is the deal I know on IP's. Designed to use thicker fuel in the 80's. That is not available anymore so to get it back there are additives you can put in the tank to make it have the lube and the viscosity it should have had in the past.
If you really want to try something quick and have a spare 1-gallon fuel can around try this. Go to the store and find some of that canning wax. Do you know the bricks of paraffin? Put about 3/4 of a gallon of diesel in the can and set it in the sun to warm it up some. While that is happening take half of one of those pieces of wax and shave it into thin slices with a knife. Don't cut yourself. Now put the shavings into the fuel and let them dissolve into it. It takes time but it does happen. You have just created a super additive mix that you will now pour into your tank.
This fuel mix contains lube, a thickener, and what clinches the deal is CETANE! That is what a diesel likes to run on.
Old IPs also wear out, and that affects the timing and the advance. The internal pump pressure may be down too and so the whole system can be out of spec. There are simple tests and devices that you can add to check the health of the IP you have. Rebuilding mostly replaces the gaskets and cleans the internals. Rebuilding pumps to spec is a very complex job and most of the wear can be dealt with by adjusting some parts to compensate for the wear.
-When timing, I can adjust the pump through its full range of motion without affecting sound significantly. With less advance, I get white smoke and sluggish response. More advance takes the smoke away and improves the responsiveness. I never hear the diesel sound get much more harsh and the cold start doesn’t seem to change anything. This all sounds normal to me. But without a dial gauge, you are chasing butterflies, my man. If you could measure the full range you have then you would know how retarded and advanced you are adjusting it. The harsh sound of a diesel may be related to the poor adjustment of valve lash and not necessarily advanced timing.
-when returning to idle, the engine hangs at 1500-2000 rpm for an extra second or two on its way down I think this is a condition of a poorly adjusted or worn throttle bushing. My pea brain is failing me at the moment.
-twice on the last test drive, it threw a single puff of black smoke seemingly at random during medium acceleration
This sounds wrong if you have new injectors in the block. Maybe some diesel injector cleaner in the current tank of fuel. Or that wax mix will do some cleaning of the fuel system too. Delivery valves can stick too, you know the ones at the end of the head of the IP.
This should be plenty for you to chew on and digest for a bit. But still keep on it, that little 1.5 may run high revs in a 4 spd but still got plenty of MPG doing it. The 5 spd was a good addition later on dropping that whine down to a reasonable level at 60 MPH.
Welcome and enjoy.