Brian Foster wrote:
Well, he's *almost* right about that. The delay slot emulation function
executes a single instruction off the user stack/vdso slot, which is
followed in memory by an instruction that provokes an address
exception. The address exception handler detects the special case (and
it should be noted that detecting the special case could be made
simpler and more reliable if a vdso-type region were used), cleans up,
and restores normal stack behavior. That "clean up" could, of course,
include any necessary vdso slot management. But what about cases that
won't get to the magic alignment trap?
On Friday 24 April 2009 17:30:35 David Daney wrote:
Kevin D. Kissell wrote:
Brian Foster wrote:
On Wednesday 22 April 2009 20:01:44 David Daney wrote:
Kevin D. Kissell wrote:
David Daney wrote:
This is a preliminary patch to add a vdso to all user processes.
[ ... ]
Note that for FPU-less CPUs, the kernel FP emulator also uses a user
stack trampoline to execute instructions in the delay slots of emulated
FP branches. [ ... ]
As David says, this is a Very Ugly Problem. Each FP trampoline
is effectively per-(runtime-)instance per-thread [ ... ]
I haven't reviewed David's code in detail, but from his description, I
thought that there was a vdso page per task/thread. If there's only one
per processor, then, yes, that poses a challenge to porting the FPU
emulation code to use it, since, as you observe, the instruction
sequence to be executed may differ for each delay slot emulation. It
should still be possible, though. [ ... ]
Kevin is right, this is ugly.
My current plan is to map an anonymous page with execute permission for
each vma (process) and place all FP trampolines there. Each thread that
needs a trampoline will allocate a piece of this page and write the
trampoline. We can arrange it so that the only way a thread can exit
the trampoline is by taking some sort of fault (currently this is true
for the normal case), or exiting.
The above is the bit which has always stumped me.
Having a per-process(or similar) page for the FP
trampoline(s) is the “obvious” approach, but what
has had me going around in circles is how to know
when an allocated slot/trampoline can be freed.
As you imply, in the normal case, it seems trivial.
It's the not-normal cases which aren't clear (or at
least aren't clear to me!).
You say (EMPHASIS added) “We can arrange it so
that the ONLY way a thread can exit the trampoline
is by taking some sort of fault ... or exiting”,
which if true, could solve the issue. Could you
elucidate on this point, please?
As the instruction being executed is extracted from a branch delay
slot, we know it's not legal for it to be any sort of branch or jump
instruction. But it *could* be a trap or system call instruction, or a
load/store that would provoke a TLB exception. In the usual cases,
however, as I believe David was alluding, either the exception will
ultimately unwind to return to execute the magic alignment trap, or the
thread will exit, and could free the emulation slot as part of general
But there's a case that isn't handled in this model, and that's the
case of an exception (or interrupt that falls in the 2-instruction
window) resulting in a signal that is caught and dispatched, and where
either the signal handler does a longjmp and restarts FP computation,
or where the signal handler itself contains a FP branch with yet
another delay slot to be emulated. One *could* get alarm signal before
the original delay slot instruction is executed, so recycling the same
vdso cache line would be premature. It's hard to get away from
something distinctly stack-like if one wants to cover these cases.
My short-term suggestion would be to leave FP emulator delay slot
handling on the (executable) user stack, even if signal trampolines use
the vdso. Longer term, we might consider what sorts of crockery would
be necessary to deal with delay slot abandonment and recursion. That
might mean adding cruft to the signal dispatch logic to detect that
we're in mid-delay-slot-emulation and defer the signal until after the
alignment trap cleanup is done (adds annoying run-time overhead, but is
probably the smallest increase in footprint and complexity), or it
might mean changing the delay slot emulation paradigm completely and
bolting a full instruction set emulator into the FP emulator, so that
the delay slot instruction is simulated in kernel mode, rather than
requiring execution in user mode. I rejected that idea out-of-hand
when I first did the FP emulator integration with the kernel, years
ago, but maybe the constraints have changed...