os: explicitly check for invalid FD in NewFile

CL 497075 refactored NewFile to unconditionally dereference the file
returned by newFile. However, newFile can return nil if passed a
negative FD, which now causes a crash.

Resolve this by moving the invalid check earlier in NewFile, which also
lets us avoid a useless fcntl syscall on a negative FD.

Since we convert to int to check sign, adjust newFile to take an int
rather than uintptr, which cleans up a lot of conversions.

Fixes #60406

Change-Id: I382a74e22f1cc01f7a2dcf1ff4efca6a79c4dd57
Reviewed-on: https://go-review.googlesource.com/c/go/+/497877


Run-TryBot: Michael Pratt <mpratt@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>