Most, if not all diesels need a large injection to get them started. There is a part of the linkage within the governor/speed control assembly that is supposed to give you full throttle to start it. I'd suspect that you either have a sticky spot in the sliding portion of the governor shaft, or you have a problem with the speed control assembly.
Either way, I think that you are probably looking at having the pump gone through, though if everything else works fine, I'd just run it for now. Unless your idle is low, I'd probably not try to compensate by adjusting the residual fuel screw.