> I am working with some hardware that has a "feature" that I'd like some
> advice on how to handle. The PCI bridge has a read-ahead buffer between
> the PCI bus and system memory - used by PCI bus masters. The buffer can
> only be invalidated from software.
So it also acts as a cache. Interesting.
> An example of the problem it causes is an ethernet device is kicked off
> to go through its ring buffers. The first one has a flag saying there is
> no data, so it stops. The kernel then puts data in the buffer, toggles
> the flag, and kicks off the ethernet device again. The old value of the
> flag is still in the read-ahead buffer so the device stops again. The
> fix is obviously to invalidate the read-ahead buffer before kicking off
> the device. The question is, how to do this in a generic way?
This is analogous to the problem you get when accessing device buffers
through the MIPS cache; so we know there's no easy fix. You'll have
to locate every write made to a shared memory structure, and then
invalidate the cache in the bridge. If you're mapping the shared
memory cached, that would be just after doing a forced write-back of
the CPU cache... but the shared memory is more likely accessed
But as a result of the MIPS cache experience there are a collection of
cache-safety buffer functions (rely on a proper Linux expert to tell
you about them, sorry). So you can update your driver to call them,
then change your local implementation of the functions to invalidate
the bridge's cache too. You still have to change all the
non-compliant drivers, of course, but at least you have the warm
feeling that you're *improving* them.
[Just a note: I suppose you've checked you're not accidentally working
through a "writeback" CPU cache, which would cause the identical
symptom? This feature would cause such routine trouble
in a bridge that I'm surprised it isn't at least easy to disable...]
> I don't want to modify the driver for every PCI device that might be
> used. The only other way seems to be to add the buffer invalidation code
> to outb() etc. (and hope that no driver wants to use memory mapped
But lots of drivers use memory mapped registers. x86 I/O space
(inb/outb etc) maps to PCI I/O space, the use of which is deprecated
except for legacy software.
The Fruit Farm, Ely Road, Chittering, CAMBS CB5 9PH, ENGLAND
phone: +44 1223 706200 / fax: +44 1223 706250 / home: +44 20 7226 0032