akpm@osdl.org 838cd153a5 [PATCH] N32 sigset and __COMPAT_ENDIAN_SWAP__
I'm testing glibc on MIPS64, little-endian, N32, O32 and N64 multilibs.

Among the NPTL test failures seen are some arising from sigsuspend problems
for N32: it blocks the wrong signals, so SIGCANCEL (SIGRTMIN) is blocked
despite glibc's carefully excluding it from sets of signals to block.
Specifically, testing suggests it blocks signal N^32 instead of signal N,
so (in the example tested) blocking SIGUSR1 (17) blocks signal 49 instead.

glibc's sigset_t uses an array of unsigned long, as does the kernel.
In both cases, signal N+1 is represented as
(1UL << (N % (8 * sizeof (unsigned long)))) in word number
(N / (8 * sizeof (unsigned long))).

Thus the N32 glibc uses an array of 32-bit words and the N64 kernel uses an
array of 64-bit words.  For little-endian, the layout is the same, with
signals 1-32 in the first 4 bytes, signals 33-64 in the second, etc.; for
big-endian, userspace has that layout while in the kernel each 8 bytes have
the two halves swapped from the userspace layout.

The N32 sigsuspend syscall uses sigset_from_compat to convert the userspace
sigset to kernel format.  If __COMPAT_ENDIAN_SWAP__ is *not* set, this uses
logic of the form

  set->sig[0] = compat->sig[0] | (((long)compat->sig[1]) << 32 )

to convert the userspace sigset to a kernel one.  This looks correct to me
for both big and little endian, given that in userspace compat->sig[1] will
represent signals 33-64, and so will the high 32 bits of set->sig[0] in the
kernel.  If however __COMPAT_ENDIAN_SWAP__ *is* set, as it is for
__MIPSEL__, it uses

  set->sig[0] = compat->sig[1] | (((long)compat->sig[0]) << 32 );

which seems incorrect for both big and little endian, and would
explain the observed symptoms.

This code is the only use of __COMPAT_ENDIAN_SWAP__, so if incorrect
then that macro serves no purpose, in which case something like the
following patch would seem appropriate to remove it.

Signed-off-by: Joseph Myers <joseph@codesourcery.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-06-25 10:01:15 -07:00
..
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-02-08 17:52:25 +00:00
2006-06-19 17:39:22 +01:00
2005-09-07 16:57:21 -07:00
2005-04-16 15:20:36 -07:00
2005-10-29 19:30:34 +01:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-06-01 00:28:35 +01:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-11-07 18:05:40 +00:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-02-15 15:32:22 -08:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-06-19 17:39:19 +01:00
2005-04-16 15:20:36 -07:00
2006-04-19 04:14:22 +02:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-06-06 00:15:20 +01:00
2005-10-29 19:32:40 +01:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-03-22 00:13:35 +01:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00