linux-mips-fnet
[Top] [All Lists]

Re: issues with non-PIC glibc on Linux/MIPS

To: "Ralf Baechle" <ralf@uni-koblenz.de>
Subject: Re: issues with non-PIC glibc on Linux/MIPS
From: "Jay Carlson" <nop@nop.com>
Date: Sun, 21 Nov 1999 23:27:28 -0500
Cc: "Dominic Sweetman" <dom@algor.co.uk>, <linux-mips@fnet.fr>, <glibc-linux@ricardo.ecn.wfu.edu>, <linuxce-devel@linuxce.org>
References: <1211d01bf2149$2199ccc0$0a00000a@nop.com> <199910290847.JAA00269@gladsmuir.algor.co.uk> <19991030003913.B15510@uni-koblenz.de> <1221301bf23b5$14e4b990$0a00000a@nop.com> <19991105084610.B2970@uni-koblenz.de>
A few weeks ago Ralf Baechle wrote:

> On Sun, Oct 31, 1999 at 10:32:01AM -0500, Jay Carlson wrote:
>
> > What I'm wondering is how bad we will lose if we avoid the fixup during
> > loading and just nail libc.so down at a fixed address.  libc procedures
> > would not need to be patched to call each other, and all data used by
libc
> > would be placed in memory at fixed addresses.  We would lose dynamic
> > linking, but not dynamic loading for shared libraries.
>
> The way SVr4 PIC code works it it not necessary to patch libc procedures.
> This still requires performing all the R_MIPS_REL32 relocations by the
> dynamic linker.  Most of this can be avoided by a concept like
``Quickstart''
> which is being used by RISC/os and IRIX.  The idea is to assign load
> addresses to binaries in advance and perform all relocations to this
> addresses in advance on the library on disk.  If on startup the library
> then can actually be mapped to the pre-relocated address most of the
> relocation business can be skipped.

But the code has still paid the price for the *potential* of having its
symbols relocated.  All references to symbols get an indirection, and
because gcc itself doesn't emit the PIC instructions, it depends on gas to
rewrite
the references.  And gas doesn't do much optimization, so every
jump-and-link from gcc ends up as four instructions, two of which are nops.

I've made some progress on at least intercall between PIC and non-PIC code
by hacking on binutils 2.8.1+mips.  One patch forces the .cpload pseudo-op
to figure out its own address rather than depending on the caller to load it
into register $25.  The other changes call-the-shared-library stubs to load
the address of the main program's GOT.  They're at
ftp://ftp.place.org/pub/nop/linuxce/linuxce/binutils-nonpic-intercall.patch
but you won't want to use them unless you're playing with this.

As far as I know, this solves the problem of a non-PIC main program calling
a PIC shared library and vice versa.  What it doesn't solve is references
from the non-PIC main program to data symbols in the shared library.  The
first big offender is errno, and then there's stdio stuff and toupper and
friends.  I'm still figuring out what approach to take for this besides
ad-hoc fixes.

> > One justification for including glibc-linux in the To list is that I'm
> > hoping to get a feel for how much stuff will break without dynamic
linking.
>
> Ulrich Drepper has worked on a Quickstart-like scheme which is supposed
> to be better.

Quickstart is a good thing, but it's not directly relevant.

> > There are really bizarre things that can be done to fix up the memory
usage
> > problem.  Consider a dynamic linking setup that patched up references to
> > shared symbols in the code as they were encountered---perhaps illegal
> > instructions for an exception handler to resolve.  Because, in some
sense,
> > the code pages' semantics are not changed by this process, they can be
> > freely discarded by the kernel and they'll just be fixed up the next
time
> > their original data gets faulted in and executed.  If the fixup handler
is
> > trusted, even the fixed-up pages could be shared across processes with
the
> > same execution environment.  An interesting thought experiment, at
least.

> This mixes the userspace problem of dynamic linking with the kernel side
> of demand loading.  Without going into details - you'll not be able to
> solve this efficiently using the current kernel API.

Yes.  I called it bizarre because it violates several tenets of the current
execution model.  At the minimum there would be new system calls and
non-trivial modifications to the kernel memory manager.  But I brought it up
*because* it violates the rules.  I don't know where the cost/benefit line
is in discarding these kinds of conventions and abstractions, but I do think
it's closer to the "benefit" side on semi-embedded devices.  The thought
experiment is worthwhile.

(...I'm not planning on implementing that particular mess any time soon
though... :-)

> I'd actually consider to rewrite a number of programs into smaller, more
> space-efficient versions.

Yeah, we've been picking those up from various sources; people working on
x86 boot disks have been a great source of tiny apps.  uClinux has more.
But the potential payoff of global code size reductions from toolchain
modifications was tempting enough to distract me from that.

> Multithreaded programs would have the per-process
> relocation overhead only once instead of once per process.

?

> Or have you
> ever checked how much rarely needed baggage libc carries around?  A
> chainsawed version of libc could help alot.

Yeah, libc carries a lot of baggage.  In some cases we've been using a
ported Debian libc reducer
 ftp://ftp.place.org/pub/nop/linuxce/mipsel-libc-reducer.tar.gz ) which
spits out the minimum libc required to support a given set of executables.
900k->500k is typical.  Unfortunately, the reducer is too simple-minded to
get libnss right, so for the ramdisks that need networking we've just been
shipping the whole libc.  We're not doing anything special at the source
code level right now.

glibc 2.1 by all reports is much worse.  For instance, touching just about
anything tows in the whole internationalization package.  H.J. Lu did some
work on that, but I don't see much interest from the glibc core team on
fixing this kind of issue; I suppose that's a reasonable allocation of
resources, but along with the well-known code size increase, this has made
2.1 a non-starter for LinuxCE projects.

> > Yeah, we should take a look at that.  It's a shame to lose the
possibility
> > of execute-in-place though.  I wonder if any of the ext2 compression
patches
> > work on linux/mips...
>
> I don't have reports about this but I imagine that if changes are
necessary
> they won't be too hard.

Turns out 2.3.7 significantly changed a bunch of kernel interfaces and
e2compr has not yet been updated.  See
http://opensource.captech.com/e2compr/software-04x.html .

Some people have been working on compression for romfs; that might make
sense.  But you lose any hope of eXecute In Place in either case.

Jay


<Prev in Thread] Current Thread [Next in Thread>