> I am working on an app where I want to give one or more user
> processes access to a largish range of physical address space
> (specifically, this is a Broadcom 1125 running a 32 bit kernel, and
> for now the region is accessible via KSEG0/1 (physical address < 512
> MB)). mmap() on /dev/mem does this just fine, and setting (or not
> setting) O_SYNC on open seems to control caching. But I just
> realized a disadvantage to doing this in user space: the user
> process accesses have to be mapped (since a user process can't, I
> believe, use KSEG0 or KSEG1 addresses), so you have to go through
> the (64 entry) TLB, and if you had signficant non-locality of
> reference, you'd possibly risk thrashing the TLB (which doesn't
> happen in kernel space, since the region can be directly accessed).
As usual, I guess the first thing is to try doing it the standard way
and then try to measure how much time is being spent in extra TLB misses
generated by your application. Some MIPS CPUs have "performance
counters" which might be able to count TLB misses, but you'll more
likely have to instrument the TLB miss code.
If it does turn out that TLB replacement is a big drain:
Most MIPS CPU hardware allows you to map large chunks of memory with a
single TLB entry: often up to 16Mbytes at a time. But I don't know
how you'd persuade Linux how to do that.