the general problem that we have under Linux is that the way the Linux
device drivers for shared memory and DMA hardware on ISA/EISA/PCI bus
are implemented is not really suitable for use on a MIPS system.
The reason for this is easy; the Linux device drivers were originally
written on for Intel where virtual and physical addresses are the same.
Later on this model was extended for the Alpha; the device driver model
now contains bus addresses which are the addresses under which the
memory is accessible from a bus device. Still alot of device drivers
get that wrong.
Nothing however has so far been done to support DMA cache coherence in
software because Intel and Alpha solve this in hardware invisible to
Fixing is easy in general and involves adding some calls to the cache
flushing routines as well as some converting some addresses into KSEG1
addresses. I've already done this for several drivers; it's however
not a very nice way to solve the problem because it's either unportable
or the extra stuff needs to be wrapped by #ifdefs.
Finally we have to deal with the more sophisticated DMA hardware available
in some MIPS systems, in particular the Jazz machines. On these machines
DMA bus addresses are being translated into virtual addresses by an
onboard MMU. Note that this nukes the simple model of physical and bus
address most drivers have. We currently cheat on these machines by
setting up an 1:1 address translation table; more sophisticated drivers
which have knowledge of the hardware like the floppy driver and the
Sonic Ethernet driver install special mappings.
(The 1:1 mapping is a bad thing; we don't use the capability to do scatter/
gather for all DMA devices. This is in particular interesting for modules
like the floppy driver which is plagued by the fact that it cannot allocate
DMA memory due fragmentation of the free page pool)
Furthermore we have machines (Wreckstation Tyne) where DMA-able memory
is allocated from a special pool.
I'd like to solve all that by adding an hook to kmalloc(),
__get_free_pages() and __free_pages() which will allow to plug architecture
specific routines in. For some rare cases it might also make sense to
have a portable way to flush caches. For that I suggest an hook similar
to the other cache flushing functions.
> > As far as the SCSI driver it seems that you need both the uncached access
> > space and the cacheflush commands in a few key spots.
> Uncached accesses and correct cache management (writeback and/or
> invalidation) are equivalent. If they act differently there are bugs
> in the driver code.
> I can guess where one of these might be, so check out the following.
> The 53C810 SCSI controller used on your P-4032 board is a very
> enthusiastic user of shared memory. As well as its buffers and buffer
> control structures, it uses shared memory to fetch "script" microcode
> and to maintain status words.
> It's the status word which is most commonly the problem. Some of the
> 53Cxx0 drivers in common circulation use a field in a per-controller
> software structure to maintain device status. But the structure also
> contains other software-only fields. If one cache line contains both
> the device status word (being updated by the SCSI controller) and
> commonly-used software variables then the system will be unreliable,
> because the CPU will often see a 'stale' copy of the status word which
> got brought into cache as a side-effect of a reference to another
> You'll need to change the driver to make sure that any location
> read/written by the SCSI controller is safely fenced away from
> software-only fields.
> Dominic Sweetman
> Algorithmics Ltd
Fido: Ralf Baechle 2:245/5618.2