Better handling of jobs in corner cases. Fixes BUGZ-3976

Some changes to handle server restart and race conditions is n job processing better. New log messages for detecting unexpected conditions.