When arguments of types narrower than a register are passed to a C
function in a register, the ABI typically requires that they be
sign-extended or zero-extended to the full width of the register. The
compiler, generating code for the called function, may presume that the
registers have been properly extended (and GCC is getting increasingly
good at avoiding redundant sign and zero extensions). In turn, it may
generate instructions for which the processor presumes the values for
properly extended - for example, on MIPS64, 32-bit arithmetic instructions
are documented as yielding unpredicatable results if the operands are not
sign-extended to 64 bits.
Consider, for example,
<http://sourceware.org/bugzilla/show_bug.cgi?id=4459>. Here glibc has
passed an improperly extended value to a syscall, so the syscall
implementation (written in C) receives a register value not conforming to
the ABI, and undefined behavior in the kernel duly ensues. Depending on
the particular form the undefined behavior takes for a given function,
compiler and CPU implementation, security issues might arise if sanity
checks of the arguments to a syscall fail to allow for values that can be
passed in registers but are beyond those permitted by the ABI.
What should the kernel syscall ABI be in such cases (any case where the
syscall implementations expect arguments narrower than registers, so
mainly 32-bit arguments on 64-bit platforms)? There are two obvious
(a) The upper bits of 32-bit syscall arguments are undefined, the kernel
should deal with this.
(b) The upper bits of 32-bit syscall arguments must be extended according
to the ABI, the kernel should detect invalid register values and treat
them as erroneous syscall arguments (probably returning EINVAL).
In either case, the kernel needs a way to handle the improperly extended
syscall arguments. Possibilities include:
* For (a), a new compiler option (or function attribute) to change the ABI
so that improperly extended arguments are valid; the compiler would then
generate the necessary code to convert them to properly extended values in
* Code at the assembly level, before syscalls get passed to their C
implementations, that uses a table of which arguments to which syscalls
are narrower than registers and either extends (for (a)) or returns EINVAL
for improperly extended values (for (b)).
* Making the C syscall implementations take register-sized arguments, with
some macros to check they are properly extended (for (b)) or reduce them
in width to variables of the narrower type (for (a)).
If (a), existing glibc is fine for 32-bit arguments on all targets - but
cases which glibc passes a 64-bit argument and the kernel expects a 32-bit
one (e.g. passing size_t where the kernel expects int) would have such
arguments silently truncated to 32 bits.
If (b) (which I prefer), MIPS64 (only) would need a new glibc that
properly sign-extends 32-bit syscall arguments to 64-bit values, in order
to work with a kernel that detects improper extension. This mainly
affects -1 as a uid/gid argument - a case we see is currently broken
anyway. Such a glibc should work on older kernels as well, and this need
for a glibc change should not affect platforms where unsigned 32-bit
values are zero-extended rather than sign-extended to 64 bits.
Joseph S. Myers