brk: COMPAT_BRK: fix detection of randomized brk
authorJiri Kosina <jkosina@suse.cz>
Thu, 14 Apr 2011 22:22:09 +0000 (15:22 -0700)
committerBrad Figg <brad.figg@canonical.com>
Wed, 27 Apr 2011 18:42:18 +0000 (11:42 -0700)
commit98439bf9a7f547ecad1604cd0a683b5ec4926cfb
tree9e7a4d51260cddfbc03b582eb9e1f98ab0d4b885
parenta21687f60f83066e313305567cdaae059ae03a6b
brk: COMPAT_BRK: fix detection of randomized brk

BugLink: http://bugs.launchpad.net/bugs/769042

commit 4471a675dfc7ca676c165079e91c712b09dc9ce4 upstream.

5520e89 ("brk: fix min_brk lower bound computation for COMPAT_BRK")
tried to get the whole logic of brk randomization for legacy
(libc5-based) applications finally right.

It turns out that the way to detect whether brk has actually been
randomized in the end or not introduced by that patch still doesn't work
for those binaries, as reported by Geert:

: /sbin/init from my old m68k ramdisk exists prematurely.
:
: Before the patch:
:
: | brk(0x80005c8e)                         = 0x80006000
:
: After the patch:
:
: | brk(0x80005c8e)                         = 0x80005c8e
:
: Old libc5 considers brk() to have failed if the return value is not
: identical to the requested value.

I don't like it, but currently see no better option than a bit flag in
task_struct to catch the CONFIG_COMPAT_BRK && randomize_va_space == 2
case.

Signed-off-by: Jiri Kosina <jkosina@suse.cz>
Tested-by: Geert Uytterhoeven <geert@linux-m68k.org>
Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Signed-off-by: Leann Ogasawara <leann.ogasawara@canonical.com>
fs/binfmt_elf.c
include/linux/sched.h
mm/mmap.c