[Top] [All Lists]

Re: [patch] linux 2.4.17: An mb() rework

To: Jason Gunthorpe <>
Subject: Re: [patch] linux 2.4.17: An mb() rework
From: "Maciej W. Rozycki" <>
Date: Thu, 31 Jan 2002 13:17:39 +0100 (MET)
In-reply-to: <>
Organization: Technical University of Gdansk
On Wed, 30 Jan 2002, Jason Gunthorpe wrote:

> I was under the impression that the mb() functions existed to maintain
> order of memory access and were not defined as flushes per say - so is
> the time delay a concern (perhaps sticking a sync before ERET is
> appropriate?)

 Well, other architectures seem to define the macros such that wmb() is a
pure ordering barrier (to assure strong ordering between writes) and rmb()
is a flush (since reads are synchronous by their nature this implies all
other transactions have to be finished before).

 Putting a "sync" before "eret" certainly doesn't work.  The case I've
identified is mask_and_ack() in an interrupt handler.  It masks an active
IRQ line in an external controller so that handle_IRQ_event() can unmask
interrupts in the CPU (this implies mask_and_ack() is synchronous).  But
due to the current lack of proper synchronization, the write isn't
executed until after the __sti() there.  As a result for almost every IRQ
routed through the external controler there is a spurious one signalled
shortly afterwards.  There is still a long path from here to an "eret". 

> I spent some time talking with the Sandcraft people about memory barrier
> issues, and it turns out that at least on the SR71000 (and in most cases
> the RM7K) the order of SysAD transactions will always match the order of
> the instruction stream, but all writes are posted and all reads are split
> - that is the CPU can execute two back to back uncached loads and several
> back to back uncached stores without stalling the pipeline, or getting the
> IO's out of order. Adding sync's and uncached loads only slows things down
> for these chips. 

 Ok, what about assuring a value written to an I/O memory resource has
actually been commited?  That's what rmb() is for.  Since the CPU is
strongly-orderes wmb() (a "sync") should be indistinguishable from a

> I understand this is because the CPUs have a single load/store unit and do
> not do out of order execution. Many older/embedded MIPS designs probably
> have a similar configuration, they could likely also run with out the
> syncs. 

 Putting a wmb() or not should be the matter of requirements of specific
drivers.  Wasting a "nop" (effectively) is surely a justifiable price for
system's consistency.

> So - could you add something like CONFIG_IN_ORDER_IO which would nullify
> the syncs for these processors?

 Hmm, the option seems to exist already, namely CONFIG_NONCOHERENT_IO, but
is it really worth making the third variation to save a single "nop",
especially as barriers are relatively rare? I'll do it, if it is.

> BTW, does anyone know what CONFIG_WB is ment to mean? The CPU has a write
> buffer that does not preserve order? 

 Certain DECstation models have a write-back buffer that needs to be
handled explicitly.  For example rmb() is "1: bc0f 1b" for the R3220 WB
chip.  Wmb() is null, certainly, as the buffer is strongly-ordered.  See
arch/mips/dec/wbflush.c for details.


+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+        e-mail:, PGP key available        +

<Prev in Thread] Current Thread [Next in Thread>