I hope you don't have to go through another cold night.
Check for a High limit switch, the unit may be getting too hot for the worn out switch. It will look like a big, big watch battery with a little frame around it with 2 screws to hold it on.
If you have a multitester you could fire the furnace up, check for voltage across the switch; it should be a 'normally closed' switch. Keep the tester on the switch and just before the unit kicks off the switch will open, That's what kills the fuel supply.
Another thing about kerosene heaters is a photo-cell that checks for a flame and, if it doesn't detect the flame after 90 seconds it shuts off the fuel. Take the photo-cell out and clean the surface of it; it may be carbon coated to the point it can't see the flame.