Dominic Sweetman wrote:
>
> Phil,
>
> > I am working with some hardware that has a "feature" that I'd like some
> > advice on how to handle. The PCI bridge has a read-ahead buffer between
> > the PCI bus and system memory - used by PCI bus masters. The buffer can
> > only be invalidated from software.
>
> So it also acts as a cache. Interesting.
>
> > An example of the problem it causes is an ethernet device is kicked off
> > to go through its ring buffers. The first one has a flag saying there is
> > no data, so it stops. The kernel then puts data in the buffer, toggles
> > the flag, and kicks off the ethernet device again. The old value of the
> > flag is still in the read-ahead buffer so the device stops again. The
> > fix is obviously to invalidate the read-ahead buffer before kicking off
> > the device. The question is, how to do this in a generic way?
>
> This is analogous to the problem you get when accessing device buffers
> through the MIPS cache; so we know there's no easy fix. You'll have
> to locate every write made to a shared memory structure, and then
> invalidate the cache in the bridge. If you're mapping the shared
> memory cached, that would be just after doing a forced write-back of
> the CPU cache... but the shared memory is more likely accessed
> uncached.
>
> But as a result of the MIPS cache experience there are a collection of
> cache-safety buffer functions (rely on a proper Linux expert to tell
> you about them, sorry). So you can update your driver to call them,
> then change your local implementation of the functions to invalidate
> the bridge's cache too. You still have to change all the
> non-compliant drivers, of course, but at least you have the warm
> feeling that you're *improving* them.
>
> [Just a note: I suppose you've checked you're not accidentally working
> through a "writeback" CPU cache, which would cause the identical
> symptom? This feature would cause such routine trouble
> in a bridge that I'm surprised it isn't at least easy to disable...]
It's definately in the bridge.
> > I don't want to modify the driver for every PCI device that might be
> > used. The only other way seems to be to add the buffer invalidation code
> > to outb() etc. (and hope that no driver wants to use memory mapped
> > registers).
>
> But lots of drivers use memory mapped registers. x86 I/O space
> (inb/outb etc) maps to PCI I/O space, the use of which is deprecated
> except for legacy software.
Oh dear!
Phil
|