Change maketempfile() to return a FILE* so as to eliminate the fopen()

Change maketempfile() to return a FILE* so as to eliminate the fopen()
that immediately follows the only call to it. maketempfile() uses
mkstemp(), so the temporary file has already been opened and using
fopen() again just opens the file twice. This also fixes the invalid
mode used on the fopen().
While here, assign NULL to fxref after fclose() because we test for
fxref being !NULL to determine if we have the (temporary) hints file
open.