From alan@lxorguk.ukuu.org.uk Fri Feb 1 00:59:04 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id AAA02203; Fri, 1 Feb 2002 00:59:04 +0100 (MET) Received-Date: Fri, 1 Feb 2002 00:59:04 +0100 (MET) Received: from lightning.swansea.linux.org.uk(194.168.151.1), claiming to be "the-village.bc.nu" via SMTP by guadalquivir.fnet.fr, id smtpd002201; Fri Feb 1 00:59:00 2002 Received: from alan by the-village.bc.nu with local (Exim 3.33 #5) id 16WQn9-0003XW-00; Thu, 31 Jan 2002 23:38:55 +0000 Subject: Re: [patch] linux 2.4.17: An mb() rework To: jgg@debian.org Date: Thu, 31 Jan 2002 23:38:54 +0000 (GMT) Cc: macro@ds2.pg.gda.pl (Maciej W. Rozycki), linux-mips@fnet.fr, linux-mips@oss.sgi.com In-Reply-To: from "Jason Gunthorpe" at Jan 31, 2002 01:35:42 PM X-Mailer: ELM [version 2.5 PL6] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-Id: From: Alan Cox Content-Length: 1051 Lines: 30 > A little more qualification would be that: > write(...,device); // Disable int > wmb() > enable_ints(); > Is expected to have a potential spurious interrupt. But, perhaps this is > OK: > outl(...,device); > wmb(); > enable_ints(); > This is consistant with how the PCI spec discusses ordering/etc and > barriers are frequently used in the PCI drivers. Looking at the i386 code > this is what I would expect to see. > > Anyone know for sure? The x86 behaviour forced as I understand it is barrier() - compiler level store barrier rmb() - read barrier to bus/DMA level [no operation] wmb() - write barrier to bus/DMA level [synchronizing instruction sequence of locked add of 0 to stack top] (mb and wmb as names come from Alpha so I guess its definitive 8)) It does not enforce PCI posting. Also your spurious interrupt case is wrong for other horrible reasons. Interrupt delivery must never be assumed to be synchronous in a portable driver. (In fact you'll see async irq delivery on an X86) From dom@mudchute.algor.co.uk Fri Feb 1 09:55:32 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id JAA08852; Fri, 1 Feb 2002 09:55:32 +0100 (MET) Received-Date: Fri, 1 Feb 2002 09:55:32 +0100 (MET) Received: from oval.algor.co.uk(62.254.210.250) via SMTP by guadalquivir.fnet.fr, id smtpd008848; Fri Feb 1 09:55:30 2002 Received: from mudchute.algor.co.uk (dom@mudchute.algor.co.uk [62.254.210.251]) by oval.algor.co.uk (8.11.6/8.10.1) with ESMTP id g118tEa10310; Fri, 1 Feb 2002 08:55:14 GMT Received: (from dom@localhost) by mudchute.algor.co.uk (8.8.5/8.8.5) id IAA00394; Fri, 1 Feb 2002 08:55:10 GMT Date: Fri, 1 Feb 2002 08:55:10 GMT Message-Id: <200202010855.IAA00394@mudchute.algor.co.uk> From: Dominic Sweetman MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: Jason Gunthorpe Cc: "Maciej W. Rozycki" , linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: Re: [patch] linux 2.4.17: An mb() rework In-Reply-To: References: X-Mailer: VM 6.34 under 19.16 "Lille" XEmacs Lucid Content-Length: 2788 Lines: 62 > > > No, a sync will still empty the write buffer. It may halt the > > >pipeline for many (~80 perhaps) cycles while posted write data is > > >dumped to the system controller. > > > > Then it's an implementation bug. For a CPU in the UP mode "sync" > > is only defined to prevent write and read reordering -- there is > > nothing about flushing. > > Did some more research + phoning.. RM7K is definately documented to dump > the write buffer on 'sync'. The RM7000 sync does more than is required by the ISA. However, since the write-buffer is not part of the architecture, this is not a bug. Though it might well be held to be undesirable. And there has to be *some* way to force the write buffer to empty, and this is cheaper than the standard alternative of an uncached read. I believe the RM7000 write buffer can hold one pending cache-line writeback or up to four uncached writes. Emptying the write buffer can only occupy more than a handful of cycles when the system controller is already backed-up with pending writes; under those circumstances the system was probably about to stall anyway, so I wouldn't be too worried about the performance implications of a 'sync'. > The RM7K guide even has an example (7.8.5) > where it implies that sync also forces a write back of any dirty cache > lines - gah! (Hard to belive though..) There's not a word of truth in that (there just aren't the right wires in the hardware to make that possible). But I think what it means is that it stops the CPU until a pending cache line write back is complete. It's hard to see why that would be useful, but since it uses the same write buffer hardware it's easy to see why it would happen. Of course (as everyone has been piling in and pointing out) the real problems are: 1. The MIPS people who invented 'sync' had a much more sophisticated understanding of read/write behaviour than is common in those who support, document and program uniprocessor "embedded" CPUs; so there's lots of scope for misunderstanding 'sync'. 2. When write-posting bites you, it's usually not in the CPU but somewhere further down the system. You should never believe a write has actually arrived at the device you sent it to, until you see the device itself change state... And a plea: the word "flush" in Linux is already abused for caches (where it means invalidate, write back, or possibly both). That's enough trouble: if you also "flush" write buffers, your confusion will be complete. Maybe we should leave "flush" to plumbers and use more precise terminology for computers... -- Dominic Sweetman, Algorithmics Ltd The Fruit Farm, Ely Road, Chittering, CAMBS CB5 9PH, ENGLAND phone: +44 1223 706200 / fax: +44 1223 706250 / direct: +44 1223 706205 http://www.algor.co.uk From macro@ds2.pg.gda.pl Fri Feb 1 13:02:31 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id NAA12711; Fri, 1 Feb 2002 13:02:31 +0100 (MET) Received-Date: Fri, 1 Feb 2002 13:02:31 +0100 (MET) Received: from delta.ds2.pg.gda.pl(213.192.72.1) via SMTP by guadalquivir.fnet.fr, id smtpd012709; Fri Feb 1 13:02:28 2002 Received: from localhost by delta.ds2.pg.gda.pl (8.9.3/8.9.3) with SMTP id NAA27210; Fri, 1 Feb 2002 13:01:57 +0100 (MET) Date: Fri, 1 Feb 2002 13:01:57 +0100 (MET) From: "Maciej W. Rozycki" To: Jason Gunthorpe cc: linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: Re: [patch] linux 2.4.17: An mb() rework In-Reply-To: Message-ID: Organization: Technical University of Gdansk MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Length: 994 Lines: 22 On Thu, 31 Jan 2002, Jason Gunthorpe wrote: > I'm afraid I don't have any manuals for any of the MIPS chips just 3rd > party ones SB1, RM7K and SR71000 - which is why I have some many > odd questions. :> Please feel free to grab a few from 'http://www.mips.com/Documentation/' and 'http://www.mips.com/declassified/'. ;-) I have a few excellent IDT manuals on their original CD-ROM as well. If they can't be downloaded anymore, I may put them online at my site. > Did some more research + phoning.. RM7K is definately documented to dump > the write buffer on 'sync'. The RM7K guide even has an example (7.8.5) > where it implies that sync also forces a write back of any dirty cache > lines - gah! (Hard to belive though..) If RM7K is always strongly ordered, I may code a workaround. -- + Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: macro@ds2.pg.gda.pl, PGP key available + From macro@ds2.pg.gda.pl Fri Feb 1 13:05:02 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id NAA13285; Fri, 1 Feb 2002 13:05:02 +0100 (MET) Received-Date: Fri, 1 Feb 2002 13:05:02 +0100 (MET) Received: from delta.ds2.pg.gda.pl(213.192.72.1) via SMTP by guadalquivir.fnet.fr, id smtpd013207; Fri Feb 1 13:04:54 2002 Received: from localhost by delta.ds2.pg.gda.pl (8.9.3/8.9.3) with SMTP id NAA27319; Fri, 1 Feb 2002 13:04:24 +0100 (MET) Date: Fri, 1 Feb 2002 13:04:23 +0100 (MET) From: "Maciej W. Rozycki" To: Dominic Sweetman cc: Jason Gunthorpe , linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: Re: [patch] linux 2.4.17: An mb() rework In-Reply-To: <200202010855.IAA00394@mudchute.algor.co.uk> Message-ID: Organization: Technical University of Gdansk MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Length: 673 Lines: 15 On Fri, 1 Feb 2002, Dominic Sweetman wrote: > And a plea: the word "flush" in Linux is already abused for caches > (where it means invalidate, write back, or possibly both). That's > enough trouble: if you also "flush" write buffers, your confusion will > be complete. Maybe we should leave "flush" to plumbers and use more > precise terminology for computers... Too bad the original name for the function is wbflush() already... At least IDT uses it in their docs. -- + Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: macro@ds2.pg.gda.pl, PGP key available + From macro@ds2.pg.gda.pl Fri Feb 1 13:28:31 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id NAA18262; Fri, 1 Feb 2002 13:28:31 +0100 (MET) Received-Date: Fri, 1 Feb 2002 13:28:31 +0100 (MET) Received: from delta.ds2.pg.gda.pl(213.192.72.1) via SMTP by guadalquivir.fnet.fr, id smtpd018260; Fri Feb 1 13:28:29 2002 Received: from localhost by delta.ds2.pg.gda.pl (8.9.3/8.9.3) with SMTP id NAA27942; Fri, 1 Feb 2002 13:27:42 +0100 (MET) Date: Fri, 1 Feb 2002 13:27:42 +0100 (MET) From: "Maciej W. Rozycki" To: Alan Cox cc: jgg@debian.org, linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: Re: [patch] linux 2.4.17: An mb() rework In-Reply-To: Message-ID: Organization: Technical University of Gdansk MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Length: 2049 Lines: 47 On Thu, 31 Jan 2002, Alan Cox wrote: > The x86 behaviour forced as I understand it is > > barrier() - compiler level store barrier > rmb() - read barrier to bus/DMA level > [no operation] > wmb() - write barrier to bus/DMA level > [synchronizing instruction sequence > of locked add of 0 to stack top] > > (mb and wmb as names come from Alpha so I guess its definitive 8)) Well, after looking at the Alpha Architecture Handbook I see "mb" and "wmb" are pure ordering barriers -- any transactions at the CPU bus (pins) may still be deferred or prefetched (architecturally -- can't comment on specific chips). So after all, maybe all the macros should be purely "sync" for MIPS ("" for MIPS I, and mb() equal to wbflush() for R3220 and similar setups) and anything that wants to see all writes actually committed should use wbflush(), which would be defined as "mb(); uncached_read();" (or in a system-specific way, for R3220, etc.)? The i386 implementation seems stronger than it should be, but that's probably because of the limited choice available. Any thoughts? > It does not enforce PCI posting. Also your spurious interrupt case is > wrong for other horrible reasons. Interrupt delivery must never be > assumed to be synchronous in a portable driver. (In fact you'll see async > irq delivery on an X86) For interrupts arriving to an interrupt controller -- agreed. But we don't generally expect a spurious interrupt from a line that was already masked at the controller level. In other words mask_and_ack() must undertake any means possible, to assure the addressed controller received the new mask. If an interrupt passes by ocassionally anyway, then it's not fatal, i.e. we can handle it, but it shouldn't be a rule (i.e. receiving as many spurious interrupts as real ones). Am I right? Maciej -- + Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: macro@ds2.pg.gda.pl, PGP key available + From macro@ds2.pg.gda.pl Sat Feb 2 13:10:36 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id NAA01161; Sat, 2 Feb 2002 13:10:36 +0100 (MET) Received-Date: Sat, 2 Feb 2002 13:10:36 +0100 (MET) Received: from delta.ds2.pg.gda.pl(213.192.72.1) via SMTP by guadalquivir.fnet.fr, id smtpd001159; Sat Feb 2 13:10:33 2002 Received: from localhost by delta.ds2.pg.gda.pl (8.9.3/8.9.3) with SMTP id NAA24037; Sat, 2 Feb 2002 13:09:48 +0100 (MET) Date: Sat, 2 Feb 2002 13:09:47 +0100 (MET) From: "Maciej W. Rozycki" To: linux-mips@fnet.fr, linux-mips@oss.sgi.com cc: jgg@debian.org Subject: Re: [patch] linux 2.4.17: An mb() rework In-Reply-To: Message-ID: Organization: Technical University of Gdansk MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Length: 5206 Lines: 172 On Fri, 1 Feb 2002, Maciej W. Rozycki wrote: > Well, after looking at the Alpha Architecture Handbook I see "mb" and > "wmb" are pure ordering barriers -- any transactions at the CPU bus (pins) > may still be deferred or prefetched (architecturally -- can't comment on > specific chips). So after all, maybe all the macros should be purely > "sync" for MIPS ("" for MIPS I, and mb() equal to wbflush() for R3220 and > similar setups) and anything that wants to see all writes actually > committed should use wbflush(), which would be defined as "mb(); > uncached_read();" (or in a system-specific way, for R3220, etc.)? > > The i386 implementation seems stronger than it should be, but that's > probably because of the limited choice available. > > Any thoughts? Well, I've seen no thoughts so far. :-( Anyway here is a new patch, which takes my considerations about a synchronization vs a write commitment into account. This implementation only assures the absolute minimum specified in the operations. I am also tempted to add "#define iob() wbflush()" to system.h and attempt to define the macro for other architectures as well. The intended semantics is to assure all preceding transactions arrived at a system bus and none of following ones were started yet. The "system bus" here is defined: "the bus that connects the bus controller of a CPU (whether internal to the chip or not) to the rest of a system." -- + Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: macro@ds2.pg.gda.pl, PGP key available + patch-mips-2.4.17-20020111-mb-wb-1 diff -up --recursive --new-file linux-mips-2.4.17-20020111.macro/arch/mips/config.in linux-mips-2.4.17-20020111/arch/mips/config.in --- linux-mips-2.4.17-20020111.macro/arch/mips/config.in Mon Jan 7 05:27:13 2002 +++ linux-mips-2.4.17-20020111/arch/mips/config.in Sat Jan 26 02:35:35 2002 @@ -363,6 +363,12 @@ else fi fi fi +if [ "$CONFIG_CPU_R3000" = "y" -o \ + "$CONFIG_CPU_TX39XX" = "y" ]; then + define_bool CONFIG_CPU_HAS_SYNC n +else + define_bool CONFIG_CPU_HAS_SYNC y +fi endmenu mainmenu_option next_comment diff -up --recursive --new-file linux-mips-2.4.17-20020111.macro/include/asm-mips/system.h linux-mips-2.4.17-20020111/include/asm-mips/system.h --- linux-mips-2.4.17-20020111.macro/include/asm-mips/system.h Thu Dec 13 05:28:09 2001 +++ linux-mips-2.4.17-20020111/include/asm-mips/system.h Sat Feb 2 00:05:19 2002 @@ -167,32 +167,31 @@ extern void __global_restore_flags(unsig #define local_irq_disable() __cli(); #define local_irq_enable() __sti(); -/* - * These are probably defined overly paranoid ... - */ +#ifdef CONFIG_CPU_HAS_SYNC +#define __sync() __asm__ __volatile__("sync" : : : "memory") +#else +#define __sync() do { } while(0) +#endif + +#define fast_wmb() __sync() +#define fast_rmb() __sync() +#define fast_mb() __sync() + #ifdef CONFIG_CPU_HAS_WB #include -#define rmb() do { } while(0) -#define wmb() wbflush() -#define mb() wbflush() - -#else /* CONFIG_CPU_HAS_WB */ - -#define mb() \ -__asm__ __volatile__( \ - "# prevent instructions being moved around\n\t" \ - ".set\tnoreorder\n\t" \ - "# 8 nops to fool the R4400 pipeline\n\t" \ - "nop;nop;nop;nop;nop;nop;nop;nop\n\t" \ - ".set\treorder" \ - : /* no output */ \ - : /* no input */ \ - : "memory") -#define rmb() mb() -#define wmb() mb() -#endif /* CONFIG_CPU_HAS_WB */ +#define wmb() fast_wmb() +#define rmb() fast_rmb() +#define mb() wbflush(); + +#else /* !CONFIG_CPU_HAS_WB */ + +#define wmb() fast_wmb() +#define rmb() fast_rmb() +#define mb() fast_mb() + +#endif /* !CONFIG_CPU_HAS_WB */ #ifdef CONFIG_SMP #define smp_mb() mb() diff -up --recursive --new-file linux-mips-2.4.17-20020111.macro/include/asm-mips/wbflush.h linux-mips-2.4.17-20020111/include/asm-mips/wbflush.h --- linux-mips-2.4.17-20020111.macro/include/asm-mips/wbflush.h Fri Sep 7 04:26:33 2001 +++ linux-mips-2.4.17-20020111/include/asm-mips/wbflush.h Sat Feb 2 00:05:21 2002 @@ -6,28 +6,49 @@ * for more details. * * Copyright (c) 1998 Harald Koerfgen + * Copyright (C) 2002 Maciej W. Rozycki */ #ifndef __ASM_MIPS_WBFLUSH_H #define __ASM_MIPS_WBFLUSH_H #include -#if defined(CONFIG_CPU_HAS_WB) -/* - * R2000 or R3000 - */ -extern void (*__wbflush) (void); +#include +#include -#define wbflush() __wbflush() +#define __fast_wbflush() \ + __asm__ __volatile__( \ + ".set push\n\t" \ + ".set noreorder\n\t" \ + "lw $0,%0\n\t" \ + "nop\n\t" \ + ".set pop" \ + : /* no output */ \ + : "m" (*(int *)KSEG1) \ + : "memory") + +#define fast_wbflush() \ + do { \ + fast_mb(); \ + __fast_wbflush(); \ + } while (0) + + +#ifdef CONFIG_CPU_HAS_WB + +extern void (*__wbflush)(void); + +#define wbflush() \ + do { \ + fast_mb(); \ + __wbflush(); \ + } while (0) -#else -/* - * we don't need no stinkin' wbflush - */ +#else /* !CONFIG_CPU_HAS_WB */ -#define wbflush() do { } while(0) +#define wbflush() fast_wbflush() -#endif +#endif /* !CONFIG_CPU_HAS_WB */ extern void wbflush_setup(void); From macro@ds2.pg.gda.pl Mon Feb 4 13:10:40 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id NAA21238; Mon, 4 Feb 2002 13:10:40 +0100 (MET) Received-Date: Mon, 4 Feb 2002 13:10:40 +0100 (MET) Received: from delta.ds2.pg.gda.pl(213.192.72.1) via SMTP by guadalquivir.fnet.fr, id smtpd021236; Mon Feb 4 13:10:37 2002 Received: from localhost by delta.ds2.pg.gda.pl (8.9.3/8.9.3) with SMTP id NAA11331; Mon, 4 Feb 2002 13:10:25 +0100 (MET) Date: Mon, 4 Feb 2002 13:10:25 +0100 (MET) From: "Maciej W. Rozycki" To: linux-mips@fnet.fr, linux-mips@oss.sgi.com cc: Ralf Baechle Subject: [patch] linux 2.4.17: The second mb() rework Message-ID: Organization: Technical University of Gdansk MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Length: 6165 Lines: 242 Hello, Here is the second version of the mb() approach again. It adds iob() as the strongest barrier, implying committing all writes to the CPU bus. I consider this version final. Any objections (barring "sync" workarounds we may consider later)? Maciej -- + Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: macro@ds2.pg.gda.pl, PGP key available + patch-mips-2.4.17-20020129-mb-wb-5 diff -up --recursive --new-file linux-mips-2.4.17-20020129.macro/arch/mips/config.in linux-mips-2.4.17-20020129/arch/mips/config.in --- linux-mips-2.4.17-20020129.macro/arch/mips/config.in Fri Jan 25 05:26:34 2002 +++ linux-mips-2.4.17-20020129/arch/mips/config.in Mon Feb 4 00:07:13 2002 @@ -384,6 +384,12 @@ else fi fi fi +if [ "$CONFIG_CPU_R3000" = "y" -o \ + "$CONFIG_CPU_TX39XX" = "y" ]; then + define_bool CONFIG_CPU_HAS_SYNC n +else + define_bool CONFIG_CPU_HAS_SYNC y +fi endmenu mainmenu_option next_comment diff -up --recursive --new-file linux-mips-2.4.17-20020129.macro/include/asm-mips/system.h linux-mips-2.4.17-20020129/include/asm-mips/system.h --- linux-mips-2.4.17-20020129.macro/include/asm-mips/system.h Sun Jan 27 05:27:59 2002 +++ linux-mips-2.4.17-20020129/include/asm-mips/system.h Mon Feb 4 02:10:33 2002 @@ -18,9 +18,12 @@ #include #include -#include + #include +#include +#include + __asm__ ( ".macro\t__sti\n\t" ".set\tpush\n\t" @@ -166,32 +169,57 @@ extern void __global_restore_flags(unsig #define local_irq_disable() __cli(); #define local_irq_enable() __sti(); -/* - * These are probably defined overly paranoid ... - */ +#ifdef CONFIG_CPU_HAS_SYNC +#define __sync() \ + __asm__ __volatile__( \ + ".set push\n\t" \ + ".set noreorder\n\t" \ + "sync\n\t" \ + ".set pop" \ + : /* no output */ \ + : /* no input */ \ + : "memory") +#else +#define __sync() do { } while(0) +#endif + +#define __fast_iob() \ + __asm__ __volatile__( \ + ".set push\n\t" \ + ".set noreorder\n\t" \ + "lw $0,%0\n\t" \ + "nop\n\t" \ + ".set pop" \ + : /* no output */ \ + : "m" (*(int *)KSEG1) \ + : "memory") + +#define fast_wmb() __sync() +#define fast_rmb() __sync() +#define fast_mb() __sync() +#define fast_iob() \ + do { \ + __sync(); \ + __fast_iob(); \ + } while (0) + #ifdef CONFIG_CPU_HAS_WB #include -#define rmb() do { } while(0) -#define wmb() wbflush() -#define mb() wbflush() - -#else /* CONFIG_CPU_HAS_WB */ - -#define mb() \ -__asm__ __volatile__( \ - "# prevent instructions being moved around\n\t" \ - ".set\tnoreorder\n\t" \ - "# 8 nops to fool the R4400 pipeline\n\t" \ - "nop;nop;nop;nop;nop;nop;nop;nop\n\t" \ - ".set\treorder" \ - : /* no output */ \ - : /* no input */ \ - : "memory") -#define rmb() mb() -#define wmb() mb() -#endif /* CONFIG_CPU_HAS_WB */ +#define wmb() fast_wmb() +#define rmb() fast_rmb() +#define mb() wbflush(); +#define iob() wbflush(); + +#else /* !CONFIG_CPU_HAS_WB */ + +#define wmb() fast_wmb() +#define rmb() fast_rmb() +#define mb() fast_mb() +#define iob() fast_iob() + +#endif /* !CONFIG_CPU_HAS_WB */ #ifdef CONFIG_SMP #define smp_mb() mb() diff -up --recursive --new-file linux-mips-2.4.17-20020129.macro/include/asm-mips/wbflush.h linux-mips-2.4.17-20020129/include/asm-mips/wbflush.h --- linux-mips-2.4.17-20020129.macro/include/asm-mips/wbflush.h Fri Sep 7 04:26:33 2001 +++ linux-mips-2.4.17-20020129/include/asm-mips/wbflush.h Mon Feb 4 02:52:11 2002 @@ -6,29 +6,30 @@ * for more details. * * Copyright (c) 1998 Harald Koerfgen + * Copyright (C) 2002 Maciej W. Rozycki */ #ifndef __ASM_MIPS_WBFLUSH_H #define __ASM_MIPS_WBFLUSH_H #include -#if defined(CONFIG_CPU_HAS_WB) -/* - * R2000 or R3000 - */ -extern void (*__wbflush) (void); +#ifdef CONFIG_CPU_HAS_WB -#define wbflush() __wbflush() +extern void (*__wbflush)(void); +extern void wbflush_setup(void); -#else -/* - * we don't need no stinkin' wbflush - */ +#define wbflush() \ + do { \ + __sync(); \ + __wbflush(); \ + } while (0) -#define wbflush() do { } while(0) +#else /* !CONFIG_CPU_HAS_WB */ -#endif +#define wbflush_setup() do { } while (0) -extern void wbflush_setup(void); +#define wbflush() fast_iob() + +#endif /* !CONFIG_CPU_HAS_WB */ #endif /* __ASM_MIPS_WBFLUSH_H */ diff -up --recursive --new-file linux-mips-2.4.17-20020129.macro/include/asm-mips64/system.h linux-mips-2.4.17-20020129/include/asm-mips64/system.h --- linux-mips-2.4.17-20020129.macro/include/asm-mips64/system.h Sun Jan 27 05:27:59 2002 +++ linux-mips-2.4.17-20020129/include/asm-mips64/system.h Mon Feb 4 02:12:27 2002 @@ -11,12 +11,13 @@ #define _ASM_SYSTEM_H #include - #include -#include #include +#include +#include + __asm__ ( ".macro\t__sti\n\t" ".set\tpush\n\t" @@ -163,20 +164,32 @@ extern void __global_restore_flags(unsig #define local_irq_disable() __cli(); #define local_irq_enable() __sti(); -/* - * These are probably defined overly paranoid ... - */ -#define mb() \ -__asm__ __volatile__( \ - "# prevent instructions being moved around\n\t" \ - ".set\tnoreorder\n\t" \ - "sync\n\t" \ - ".set\treorder" \ - : /* no output */ \ - : /* no input */ \ - : "memory") -#define rmb() mb() -#define wmb() mb() +#define __sync() \ + __asm__ __volatile__( \ + ".set push\n\t" \ + ".set noreorder\n\t" \ + "sync\n\t" \ + ".set pop" \ + : /* no output */ \ + : /* no input */ \ + : "memory") + +#define wmb() __sync() +#define rmb() __sync() +#define mb() __sync() +#define iob() \ + do { \ + __sync(); \ + __asm__ __volatile__( \ + ".set push\n\t" \ + ".set noreorder\n\t" \ + "lw $0,%0\n\t" \ + "nop\n\t" \ + ".set pop" \ + : /* no output */ \ + : "m" (*(int *)KSEG1) \ + : "memory"); \ + } while (0) #ifdef CONFIG_SMP #define smp_mb() mb() From ralf@linux-mips.net Mon Feb 4 19:19:08 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id TAA26882; Mon, 4 Feb 2002 19:19:08 +0100 (MET) Received-Date: Mon, 4 Feb 2002 19:19:08 +0100 (MET) Received: from a1as18-p231.stg.tli.de(195.252.193.231), claiming to be "dea.linux-mips.net" via SMTP by guadalquivir.fnet.fr, id smtpd026880; Mon Feb 4 19:19:02 2002 Received: (from ralf@localhost) by dea.linux-mips.net (8.11.6/8.11.1) id g14H2w404022; Mon, 4 Feb 2002 18:02:58 +0100 Date: Mon, 4 Feb 2002 18:02:58 +0100 From: Ralf Baechle To: "Maciej W. Rozycki" Cc: Jason Gunthorpe , linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: Re: [patch] linux 2.4.17: An mb() rework Message-ID: <20020204180258.A6124@dea.linux-mips.net> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from macro@ds2.pg.gda.pl on Thu, Jan 31, 2002 at 01:17:39PM +0100 X-Accept-Language: de,en,fr Content-Length: 868 Lines: 18 On Thu, Jan 31, 2002 at 01:17:39PM +0100, Maciej W. Rozycki wrote: > Certain DECstation models have a write-back buffer that needs to be > handled explicitly. For example rmb() is "1: bc0f 1b" for the R3220 WB > chip. Wmb() is null, certainly, as the buffer is strongly-ordered. See > arch/mips/dec/wbflush.c for details. Just as an aside that isn't directly relevant to DECstations. To date all MIPS _processors_ are strongly ordered. I now know of at least one processor that implements a weakly ordered memory model, so the assumption of a strongly ordered memory model has become void for large parts of the kernel. Even before that some systems had strongly ordered processors in system environments that may reorder requests. Bugs due to surprise memory reordering are entirely unfun to debug, so be paranoid. They're out there to get you ... Ralf From macro@ds2.pg.gda.pl Tue Feb 5 11:34:24 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id LAA06434; Tue, 5 Feb 2002 11:34:24 +0100 (MET) Received-Date: Tue, 5 Feb 2002 11:34:24 +0100 (MET) Received: from delta.ds2.pg.gda.pl(213.192.72.1) via SMTP by guadalquivir.fnet.fr, id smtpd006432; Tue Feb 5 11:34:17 2002 Received: from localhost by delta.ds2.pg.gda.pl (8.9.3/8.9.3) with SMTP id LAA10108; Tue, 5 Feb 2002 11:33:24 +0100 (MET) Date: Tue, 5 Feb 2002 11:33:23 +0100 (MET) From: "Maciej W. Rozycki" To: Ralf Baechle cc: Jason Gunthorpe , linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: Re: [patch] linux 2.4.17: An mb() rework In-Reply-To: <20020204180258.A6124@dea.linux-mips.net> Message-ID: Organization: Technical University of Gdansk MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Length: 561 Lines: 13 On Mon, 4 Feb 2002, Ralf Baechle wrote: > Bugs due to surprise memory reordering are entirely unfun to debug, so be > paranoid. They're out there to get you ... The rule of thumb is to use barriers whenever a *device* requires them and depend on the platform to define them appropriately. They will expand to nothing if unneeded, so there is no trade-off. -- + Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: macro@ds2.pg.gda.pl, PGP key available + From macro@ds2.pg.gda.pl Mon Feb 11 16:46:08 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id QAA02885; Mon, 11 Feb 2002 16:46:08 +0100 (MET) Received-Date: Mon, 11 Feb 2002 16:46:08 +0100 (MET) Received: from delta.ds2.pg.gda.pl(213.192.72.1) via SMTP by guadalquivir.fnet.fr, id smtpd002883; Mon Feb 11 16:46:01 2002 Received: from localhost by delta.ds2.pg.gda.pl (8.9.3/8.9.3) with SMTP id QAA24645; Mon, 11 Feb 2002 16:45:54 +0100 (MET) Date: Mon, 11 Feb 2002 16:45:54 +0100 (MET) From: "Maciej W. Rozycki" To: Ralf Baechle cc: linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: [patch] linux 2.4.17: The second mb() rework (final) Message-ID: Organization: Technical University of Gdansk MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Length: 5978 Lines: 238 Ralf, I haven't seen any objections. Could you please apply it? Maciej -- + Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: macro@ds2.pg.gda.pl, PGP key available + patch-mips-2.4.17-20020129-mb-wb-5 diff -up --recursive --new-file linux-mips-2.4.17-20020129.macro/arch/mips/config.in linux-mips-2.4.17-20020129/arch/mips/config.in --- linux-mips-2.4.17-20020129.macro/arch/mips/config.in Fri Jan 25 05:26:34 2002 +++ linux-mips-2.4.17-20020129/arch/mips/config.in Mon Feb 4 00:07:13 2002 @@ -384,6 +384,12 @@ else fi fi fi +if [ "$CONFIG_CPU_R3000" = "y" -o \ + "$CONFIG_CPU_TX39XX" = "y" ]; then + define_bool CONFIG_CPU_HAS_SYNC n +else + define_bool CONFIG_CPU_HAS_SYNC y +fi endmenu mainmenu_option next_comment diff -up --recursive --new-file linux-mips-2.4.17-20020129.macro/include/asm-mips/system.h linux-mips-2.4.17-20020129/include/asm-mips/system.h --- linux-mips-2.4.17-20020129.macro/include/asm-mips/system.h Sun Jan 27 05:27:59 2002 +++ linux-mips-2.4.17-20020129/include/asm-mips/system.h Mon Feb 4 02:10:33 2002 @@ -18,9 +18,12 @@ #include #include -#include + #include +#include +#include + __asm__ ( ".macro\t__sti\n\t" ".set\tpush\n\t" @@ -166,32 +169,57 @@ extern void __global_restore_flags(unsig #define local_irq_disable() __cli(); #define local_irq_enable() __sti(); -/* - * These are probably defined overly paranoid ... - */ +#ifdef CONFIG_CPU_HAS_SYNC +#define __sync() \ + __asm__ __volatile__( \ + ".set push\n\t" \ + ".set noreorder\n\t" \ + "sync\n\t" \ + ".set pop" \ + : /* no output */ \ + : /* no input */ \ + : "memory") +#else +#define __sync() do { } while(0) +#endif + +#define __fast_iob() \ + __asm__ __volatile__( \ + ".set push\n\t" \ + ".set noreorder\n\t" \ + "lw $0,%0\n\t" \ + "nop\n\t" \ + ".set pop" \ + : /* no output */ \ + : "m" (*(int *)KSEG1) \ + : "memory") + +#define fast_wmb() __sync() +#define fast_rmb() __sync() +#define fast_mb() __sync() +#define fast_iob() \ + do { \ + __sync(); \ + __fast_iob(); \ + } while (0) + #ifdef CONFIG_CPU_HAS_WB #include -#define rmb() do { } while(0) -#define wmb() wbflush() -#define mb() wbflush() - -#else /* CONFIG_CPU_HAS_WB */ - -#define mb() \ -__asm__ __volatile__( \ - "# prevent instructions being moved around\n\t" \ - ".set\tnoreorder\n\t" \ - "# 8 nops to fool the R4400 pipeline\n\t" \ - "nop;nop;nop;nop;nop;nop;nop;nop\n\t" \ - ".set\treorder" \ - : /* no output */ \ - : /* no input */ \ - : "memory") -#define rmb() mb() -#define wmb() mb() -#endif /* CONFIG_CPU_HAS_WB */ +#define wmb() fast_wmb() +#define rmb() fast_rmb() +#define mb() wbflush(); +#define iob() wbflush(); + +#else /* !CONFIG_CPU_HAS_WB */ + +#define wmb() fast_wmb() +#define rmb() fast_rmb() +#define mb() fast_mb() +#define iob() fast_iob() + +#endif /* !CONFIG_CPU_HAS_WB */ #ifdef CONFIG_SMP #define smp_mb() mb() diff -up --recursive --new-file linux-mips-2.4.17-20020129.macro/include/asm-mips/wbflush.h linux-mips-2.4.17-20020129/include/asm-mips/wbflush.h --- linux-mips-2.4.17-20020129.macro/include/asm-mips/wbflush.h Fri Sep 7 04:26:33 2001 +++ linux-mips-2.4.17-20020129/include/asm-mips/wbflush.h Mon Feb 4 02:52:11 2002 @@ -6,29 +6,30 @@ * for more details. * * Copyright (c) 1998 Harald Koerfgen + * Copyright (C) 2002 Maciej W. Rozycki */ #ifndef __ASM_MIPS_WBFLUSH_H #define __ASM_MIPS_WBFLUSH_H #include -#if defined(CONFIG_CPU_HAS_WB) -/* - * R2000 or R3000 - */ -extern void (*__wbflush) (void); +#ifdef CONFIG_CPU_HAS_WB -#define wbflush() __wbflush() +extern void (*__wbflush)(void); +extern void wbflush_setup(void); -#else -/* - * we don't need no stinkin' wbflush - */ +#define wbflush() \ + do { \ + __sync(); \ + __wbflush(); \ + } while (0) -#define wbflush() do { } while(0) +#else /* !CONFIG_CPU_HAS_WB */ -#endif +#define wbflush_setup() do { } while (0) -extern void wbflush_setup(void); +#define wbflush() fast_iob() + +#endif /* !CONFIG_CPU_HAS_WB */ #endif /* __ASM_MIPS_WBFLUSH_H */ diff -up --recursive --new-file linux-mips-2.4.17-20020129.macro/include/asm-mips64/system.h linux-mips-2.4.17-20020129/include/asm-mips64/system.h --- linux-mips-2.4.17-20020129.macro/include/asm-mips64/system.h Sun Jan 27 05:27:59 2002 +++ linux-mips-2.4.17-20020129/include/asm-mips64/system.h Mon Feb 4 02:12:27 2002 @@ -11,12 +11,13 @@ #define _ASM_SYSTEM_H #include - #include -#include #include +#include +#include + __asm__ ( ".macro\t__sti\n\t" ".set\tpush\n\t" @@ -163,20 +164,32 @@ extern void __global_restore_flags(unsig #define local_irq_disable() __cli(); #define local_irq_enable() __sti(); -/* - * These are probably defined overly paranoid ... - */ -#define mb() \ -__asm__ __volatile__( \ - "# prevent instructions being moved around\n\t" \ - ".set\tnoreorder\n\t" \ - "sync\n\t" \ - ".set\treorder" \ - : /* no output */ \ - : /* no input */ \ - : "memory") -#define rmb() mb() -#define wmb() mb() +#define __sync() \ + __asm__ __volatile__( \ + ".set push\n\t" \ + ".set noreorder\n\t" \ + "sync\n\t" \ + ".set pop" \ + : /* no output */ \ + : /* no input */ \ + : "memory") + +#define wmb() __sync() +#define rmb() __sync() +#define mb() __sync() +#define iob() \ + do { \ + __sync(); \ + __asm__ __volatile__( \ + ".set push\n\t" \ + ".set noreorder\n\t" \ + "lw $0,%0\n\t" \ + "nop\n\t" \ + ".set pop" \ + : /* no output */ \ + : "m" (*(int *)KSEG1) \ + : "memory"); \ + } while (0) #ifdef CONFIG_SMP #define smp_mb() mb() From macro@ds2.pg.gda.pl Tue Feb 12 12:41:22 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id MAA12016; Tue, 12 Feb 2002 12:41:22 +0100 (MET) Received-Date: Tue, 12 Feb 2002 12:41:22 +0100 (MET) Received: from delta.ds2.pg.gda.pl(213.192.72.1) via SMTP by guadalquivir.fnet.fr, id smtpd012014; Tue Feb 12 12:41:15 2002 Received: from localhost by delta.ds2.pg.gda.pl (8.9.3/8.9.3) with SMTP id MAA17975; Tue, 12 Feb 2002 12:41:02 +0100 (MET) Date: Tue, 12 Feb 2002 12:41:01 +0100 (MET) From: "Maciej W. Rozycki" To: Matthew Dharm cc: Ralf Baechle , linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: RE: [patch] linux 2.4.17: The second mb() rework (final) In-Reply-To: Message-ID: Organization: Technical University of Gdansk MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Length: 477 Lines: 12 On Mon, 11 Feb 2002, Matthew Dharm wrote: > Aren't there more processors that have sync? I thought the RM7000 > (PMC-Sierra) did too... Obviously there are as "sync" is standard for MIPS II and above. That's why only CONFIG_CPU_R3000 and CONFIG_CPU_TX39XX disable "sync". -- + Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: macro@ds2.pg.gda.pl, PGP key available + From kevink@mips.com Tue Feb 12 13:01:32 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id NAA14383; Tue, 12 Feb 2002 13:01:32 +0100 (MET) Received-Date: Tue, 12 Feb 2002 13:01:32 +0100 (MET) Received: from mx.mips.com(206.31.31.226) via SMTP by guadalquivir.fnet.fr, id smtpd014380; Tue Feb 12 13:01:23 2002 Received: from newman.mips.com (ns-dmz [206.31.31.225]) by mx.mips.com (8.9.3/8.9.0) with ESMTP id EAA18097; Tue, 12 Feb 2002 04:01:13 -0800 (PST) Received: from Ulysses (ulysses [192.168.236.13]) by newman.mips.com (8.9.3/8.9.0) with SMTP id EAA28977; Tue, 12 Feb 2002 04:01:09 -0800 (PST) Message-ID: <010601c1b3bd$1da618e0$0deca8c0@Ulysses> From: "Kevin D. Kissell" To: "Maciej W. Rozycki" , "Matthew Dharm" Cc: "Ralf Baechle" , , References: Subject: Re: [patch] linux 2.4.17: The second mb() rework (final) Date: Tue, 12 Feb 2002 13:02:13 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 Content-Length: 531 Lines: 14 From: "Maciej W. Rozycki" : > On Mon, 11 Feb 2002, Matthew Dharm wrote: > > > Aren't there more processors that have sync? I thought the RM7000 > > (PMC-Sierra) did too... > > Obviously there are as "sync" is standard for MIPS II and above. That's > why only CONFIG_CPU_R3000 and CONFIG_CPU_TX39XX disable "sync". According to the (rather old) Tx39xx specs at my disposal, those parts should also support the SYNC instruction. Does anyone know for a fact that some of them do not? Kevin K. From macro@ds2.pg.gda.pl Tue Feb 12 13:48:41 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id NAA16785; Tue, 12 Feb 2002 13:48:41 +0100 (MET) Received-Date: Tue, 12 Feb 2002 13:48:41 +0100 (MET) Received: from delta.ds2.pg.gda.pl(213.192.72.1) via SMTP by guadalquivir.fnet.fr, id smtpd016783; Tue Feb 12 13:48:33 2002 Received: from localhost by delta.ds2.pg.gda.pl (8.9.3/8.9.3) with SMTP id NAA19778; Tue, 12 Feb 2002 13:45:22 +0100 (MET) Date: Tue, 12 Feb 2002 13:45:21 +0100 (MET) From: "Maciej W. Rozycki" To: "Kevin D. Kissell" cc: Matthew Dharm , Ralf Baechle , linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: Re: [patch] linux 2.4.17: The second mb() rework (final) In-Reply-To: <010601c1b3bd$1da618e0$0deca8c0@Ulysses> Message-ID: Organization: Technical University of Gdansk MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Length: 973 Lines: 20 On Tue, 12 Feb 2002, Kevin D. Kissell wrote: > > Obviously there are as "sync" is standard for MIPS II and above. That's > > why only CONFIG_CPU_R3000 and CONFIG_CPU_TX39XX disable "sync". > > According to the (rather old) Tx39xx specs at my disposal, > those parts should also support the SYNC instruction. Does > anyone know for a fact that some of them do not? I set these values according to what I found in arch/mips/setup.c -- Tx39xx are marked as MIPS_CPU_ISA_I, but obviously if they still implement "sync" as an extension they should be marked to enable the code that uses it. The patch should be applied as is anyway -- until it's clear if all members of the Tx39xx family implement "sync", it's better to leave their configuration in the current state. -- + Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: macro@ds2.pg.gda.pl, PGP key available + From nemoto@toshiba-tops.co.jp Wed Feb 13 02:23:47 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id CAA24045; Wed, 13 Feb 2002 02:23:47 +0100 (MET) Received-Date: Wed, 13 Feb 2002 02:23:47 +0100 (MET) Received: from topsns.toshiba-tops.co.jp(202.230.225.5) via SMTP by guadalquivir.fnet.fr, id smtpd024043; Wed Feb 13 02:23:45 2002 Received: from inside-ms1.toshiba-tops.co.jp by topsns.toshiba-tops.co.jp via smtpd (for guadalquivir.fnet.fr [193.104.112.133]) with SMTP; 13 Feb 2002 01:23:43 UT Received: from srd2sd.toshiba-tops.co.jp (gw-chiba7.toshiba-tops.co.jp [172.17.244.27]) by topsms.toshiba-tops.co.jp (Postfix) with ESMTP id EF68EB46B; Wed, 13 Feb 2002 10:23:33 +0900 (JST) Received: by srd2sd.toshiba-tops.co.jp (8.9.3/3.5Wbeta-srd2sd) with ESMTP id KAA44828; Wed, 13 Feb 2002 10:23:33 +0900 (JST) Date: Wed, 13 Feb 2002 10:28:05 +0900 (JST) Message-Id: <20020213.102805.74755945.nemoto@toshiba-tops.co.jp> To: kevink@mips.com Cc: macro@ds2.pg.gda.pl, mdharm@momenco.com, ralf@uni-koblenz.de, linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: Re: [patch] linux 2.4.17: The second mb() rework (final) From: Atsushi Nemoto In-Reply-To: <010601c1b3bd$1da618e0$0deca8c0@Ulysses> References: <010601c1b3bd$1da618e0$0deca8c0@Ulysses> X-Fingerprint: EC 9D B9 17 2E 89 D2 25 CE F5 5D 3D 12 29 2A AD X-Pgp-Public-Key: http://pgp.nic.ad.jp/cgi-bin/pgpsearchkey.pl?op=get&search=0xB6D728B1 Organization: TOSHIBA Personal Computer System Corporation X-Mailer: Mew version 2.1 on Emacs 20.7 / Mule 4.1 (AOI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Length: 510 Lines: 14 >>>>> On Tue, 12 Feb 2002 13:02:13 +0100, "Kevin D. Kissell" said: kevink> According to the (rather old) Tx39xx specs at my disposal, kevink> those parts should also support the SYNC instruction. Does kevink> anyone know for a fact that some of them do not? There are two types of TX39: TX39/H and TX39/H2. To my knowledge, both TX39/H (3904, 3912, etc.) and TX39/H2 (3922, 3927, etc.) have then SYNC instruction. TX39/H2 also have a write buffer but TX39/H does not. --- Atsushi Nemoto From nemoto@toshiba-tops.co.jp Fri Feb 15 04:27:13 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id EAA10486; Fri, 15 Feb 2002 04:27:13 +0100 (MET) Received-Date: Fri, 15 Feb 2002 04:27:13 +0100 (MET) Received: from topsns.toshiba-tops.co.jp(202.230.225.5) via SMTP by guadalquivir.fnet.fr, id smtpd010484; Fri Feb 15 04:27:10 2002 Received: from inside-ms1.toshiba-tops.co.jp by topsns.toshiba-tops.co.jp via smtpd (for guadalquivir.fnet.fr [193.104.112.133]) with SMTP; 15 Feb 2002 03:27:08 UT Received: from srd2sd.toshiba-tops.co.jp (gw-chiba7.toshiba-tops.co.jp [172.17.244.27]) by topsms.toshiba-tops.co.jp (Postfix) with ESMTP id 2653AB46B; Fri, 15 Feb 2002 12:26:59 +0900 (JST) Received: by srd2sd.toshiba-tops.co.jp (8.9.3/3.5Wbeta-srd2sd) with ESMTP id MAA50808; Fri, 15 Feb 2002 12:26:58 +0900 (JST) Date: Fri, 15 Feb 2002 12:31:24 +0900 (JST) Message-Id: <20020215.123124.70226832.nemoto@toshiba-tops.co.jp> To: kevink@mips.com Cc: macro@ds2.pg.gda.pl, mdharm@momenco.com, ralf@uni-koblenz.de, linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: Re: [patch] linux 2.4.17: The second mb() rework (final) From: Atsushi Nemoto In-Reply-To: <20020213.102805.74755945.nemoto@toshiba-tops.co.jp> References: <010601c1b3bd$1da618e0$0deca8c0@Ulysses> <20020213.102805.74755945.nemoto@toshiba-tops.co.jp> X-Fingerprint: EC 9D B9 17 2E 89 D2 25 CE F5 5D 3D 12 29 2A AD X-Pgp-Public-Key: http://pgp.nic.ad.jp/cgi-bin/pgpsearchkey.pl?op=get&search=0xB6D728B1 Organization: TOSHIBA Personal Computer System Corporation X-Mailer: Mew version 2.1 on Emacs 20.7 / Mule 4.1 (AOI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Length: 383 Lines: 11 >>>>> On Wed, 13 Feb 2002 10:28:05 +0900 (JST), Atsushi Nemoto said: nemoto> TX39/H2 also have a write buffer but TX39/H does not. Sorry, this is wrong. TX39/H also have a write buffer. Note that SYNC on TX39/H and TX39/H2 does not flush a write buffer. Some operation (for example, bc0f loop) are required to flush a write buffer. --- Atsushi Nemoto From kevink@mips.com Fri Feb 15 10:49:21 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id KAA13477; Fri, 15 Feb 2002 10:49:21 +0100 (MET) Received-Date: Fri, 15 Feb 2002 10:49:21 +0100 (MET) Received: from mx.mips.com(206.31.31.226) via SMTP by guadalquivir.fnet.fr, id smtpd013475; Fri Feb 15 10:49:14 2002 Received: from newman.mips.com (ns-dmz [206.31.31.225]) by mx.mips.com (8.9.3/8.9.0) with ESMTP id BAA09499; Fri, 15 Feb 2002 01:49:05 -0800 (PST) Received: from Ulysses (ulysses [192.168.236.13]) by newman.mips.com (8.9.3/8.9.0) with SMTP id BAA08013; Fri, 15 Feb 2002 01:48:52 -0800 (PST) Message-ID: <006e01c1b606$27b1b060$0deca8c0@Ulysses> From: "Kevin D. Kissell" To: "Atsushi Nemoto" Cc: , , , , References: <010601c1b3bd$1da618e0$0deca8c0@Ulysses><20020213.102805.74755945.nemoto@toshiba-tops.co.jp> <20020215.123124.70226832.nemoto@toshiba-tops.co.jp> Subject: Re: [patch] linux 2.4.17: The second mb() rework (final) Date: Fri, 15 Feb 2002 09:30:45 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 Content-Length: 842 Lines: 18 From: "Atsushi Nemoto" : > Note that SYNC on TX39/H and TX39/H2 does not flush a write buffer. > Some operation (for example, bc0f loop) are required to flush a write > buffer. That is, I would say, a bug in the TX39 implementation of SYNC. The specification is states that all stores prior to the SYNC must complete before any memory ops after the sync, and that the definition of a store completing is that all stored values be "visible to every other processor in the system", which pretty clearly implies that the write buffers must be flushed. So I think that the Linux code was perfectly correct in considering the TX39 to be without SYNC, just as a Vr4101 must be consdered to be without LL/SC. They decode the instructions, but they don't actually implement them as specified. Kevin K. From ralf@linux-mips.net Fri Feb 15 12:50:30 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id MAA15759; Fri, 15 Feb 2002 12:50:30 +0100 (MET) Received-Date: Fri, 15 Feb 2002 12:50:30 +0100 (MET) Received: from a1as06-p212.stg.tli.de(195.252.187.212), claiming to be "dea.linux-mips.net" via SMTP by guadalquivir.fnet.fr, id smtpd015757; Fri Feb 15 12:50:22 2002 Received: (from ralf@localhost) by dea.linux-mips.net (8.11.6/8.11.1) id g1FAo6S26781; Fri, 15 Feb 2002 11:50:06 +0100 Date: Fri, 15 Feb 2002 11:50:06 +0100 From: Ralf Baechle To: "Kevin D. Kissell" Cc: Atsushi Nemoto , macro@ds2.pg.gda.pl, mdharm@momenco.com, linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: Re: [patch] linux 2.4.17: The second mb() rework (final) Message-ID: <20020215115006.B26756@dea.linux-mips.net> References: <010601c1b3bd$1da618e0$0deca8c0@Ulysses><20020213.102805.74755945.nemoto@toshiba-tops.co.jp> <20020215.123124.70226832.nemoto@toshiba-tops.co.jp> <006e01c1b606$27b1b060$0deca8c0@Ul Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <006e01c1b606$27b1b060$0deca8c0@Ulysses>; from kevink@mips.com on Fri, Feb 15, 2002 at 09:30:45AM +0100 X-Accept-Language: de,en,fr Content-Length: 831 Lines: 17 On Fri, Feb 15, 2002 at 09:30:45AM +0100, Kevin D. Kissell wrote: > That is, I would say, a bug in the TX39 implementation of SYNC. > The specification is states that all stores prior to the SYNC must > complete before any memory ops after the sync, and that the > definition of a store completing is that all stored values be > "visible to every other processor in the system", which pretty > clearly implies that the write buffers must be flushed. In practice sync just isn't good enough. Most systems these days use an I/O bus like PCI which uses posted writes or have some other non-strongly ordered memory model. Which is why something wmb() or mb() aren't good enough in driver. I'm just having a nice discussion about this topic with SGI's IA64 people; we have to come up with a portable and efficient ABI. Ralf From dom@mudchute.algor.co.uk Fri Feb 15 12:58:00 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id MAA16755; Fri, 15 Feb 2002 12:58:00 +0100 (MET) Received-Date: Fri, 15 Feb 2002 12:58:00 +0100 (MET) Received: from oval.algor.co.uk(62.254.210.250) via SMTP by guadalquivir.fnet.fr, id smtpd016753; Fri Feb 15 12:57:54 2002 Received: from mudchute.algor.co.uk (dom@mudchute.algor.co.uk [62.254.210.251]) by oval.algor.co.uk (8.11.6/8.10.1) with ESMTP id g1FBv5402720; Fri, 15 Feb 2002 11:57:05 GMT Received: (from dom@localhost) by mudchute.algor.co.uk (8.8.5/8.8.5) id LAA17485; Fri, 15 Feb 2002 11:56:59 GMT Date: Fri, 15 Feb 2002 11:56:59 GMT Message-Id: <200202151156.LAA17485@mudchute.algor.co.uk> From: Dominic Sweetman MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: "Kevin D. Kissell" Cc: "Atsushi Nemoto" , , , , , Subject: Re: [patch] linux 2.4.17: The second mb() rework (final) In-Reply-To: <006e01c1b606$27b1b060$0deca8c0@Ulysses> References: <010601c1b3bd$1da618e0$0deca8c0@Ulysses> <20020213.102805.74755945.nemoto@toshiba-tops.co.jp> <20020215.123124.70226832.nemoto@toshiba-tops.co.jp> <006e01c1b606$27b1b060$0deca8c0@Ulysses> X-Mailer: VM 6.34 under 19.16 "Lille" XEmacs Lucid Content-Length: 653 Lines: 19 Kevin D. Kissell (kevink@mips.com) writes: > > Note that SYNC on TX39/H and TX39/H2 does not flush a write buffer. > > Some operation (for example, bc0f loop) are required to flush a write > > buffer. > > That is, I would say, a bug in the TX39 implementation of SYNC. That's only a problem if the CPU permitted reads to overtake buffered writes. [Early R3000 write buffers did that (with an address check to avoid the disaster of allowing a read to overtake a write to the same location).] But my recollection is that the TX39 does all memory operations in order: so SYNC has very little to do, but it isn't a bug. Dominic Sweetman Algorithmics From macro@ds2.pg.gda.pl Fri Feb 15 13:09:30 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id NAA19552; Fri, 15 Feb 2002 13:09:30 +0100 (MET) Received-Date: Fri, 15 Feb 2002 13:09:30 +0100 (MET) Received: from delta.ds2.pg.gda.pl(213.192.72.1) via SMTP by guadalquivir.fnet.fr, id smtpd019550; Fri Feb 15 13:09:27 2002 Received: from localhost by delta.ds2.pg.gda.pl (8.9.3/8.9.3) with SMTP id NAA00974; Fri, 15 Feb 2002 13:08:21 +0100 (MET) Date: Fri, 15 Feb 2002 13:08:20 +0100 (MET) From: "Maciej W. Rozycki" To: Atsushi Nemoto cc: kevink@mips.com, mdharm@momenco.com, ralf@uni-koblenz.de, linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: Re: [patch] linux 2.4.17: The second mb() rework (final) In-Reply-To: <20020215.123124.70226832.nemoto@toshiba-tops.co.jp> Message-ID: Organization: Technical University of Gdansk MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Length: 1012 Lines: 25 On Fri, 15 Feb 2002, Atsushi Nemoto wrote: > Note that SYNC on TX39/H and TX39/H2 does not flush a write buffer. That's perfectly fine -- it does not need to. It's only a synchronization point. > Some operation (for example, bc0f loop) are required to flush a write > buffer. This is bad, though. No extra operations must be needed. After a "sync" the buffer should be written back as soon as not doing so would be visible externally. IOW, two consecutive writes separated by a "sync" need not imply a write-back, but as soon as a read happens a preceding write-back is a must. It sounds weird: it looks like "sync" is useless and basically a "nop". If this is the case the processors need their own wbflush() implementation. Can you point to specific chapters of specifications that document it? -- + Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: macro@ds2.pg.gda.pl, PGP key available + From macro@ds2.pg.gda.pl Fri Feb 15 13:12:30 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id NAA20514; Fri, 15 Feb 2002 13:12:30 +0100 (MET) Received-Date: Fri, 15 Feb 2002 13:12:30 +0100 (MET) Received: from delta.ds2.pg.gda.pl(213.192.72.1) via SMTP by guadalquivir.fnet.fr, id smtpd020509; Fri Feb 15 13:12:13 2002 Received: from localhost by delta.ds2.pg.gda.pl (8.9.3/8.9.3) with SMTP id NAA01131; Fri, 15 Feb 2002 13:11:16 +0100 (MET) Date: Fri, 15 Feb 2002 13:11:15 +0100 (MET) From: "Maciej W. Rozycki" To: "Kevin D. Kissell" cc: Atsushi Nemoto , mdharm@momenco.com, ralf@uni-koblenz.de, linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: Re: [patch] linux 2.4.17: The second mb() rework (final) In-Reply-To: <006e01c1b606$27b1b060$0deca8c0@Ulysses> Message-ID: Organization: Technical University of Gdansk MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Length: 700 Lines: 15 On Fri, 15 Feb 2002, Kevin D. Kissell wrote: > So I think that the Linux code was perfectly correct in considering > the TX39 to be without SYNC, just as a Vr4101 must be > consdered to be without LL/SC. They decode the instructions, > but they don't actually implement them as specified. The code is not correct if "bc0f" is needed to be sure a write-back happened. If that is the case, the processors need their own wbflush() implementation like R2k/R3k configurations in older DECstations. -- + Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: macro@ds2.pg.gda.pl, PGP key available + From macro@ds2.pg.gda.pl Fri Feb 15 13:33:10 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id NAA23352; Fri, 15 Feb 2002 13:33:10 +0100 (MET) Received-Date: Fri, 15 Feb 2002 13:33:10 +0100 (MET) Received: from delta.ds2.pg.gda.pl(213.192.72.1) via SMTP by guadalquivir.fnet.fr, id smtpd023350; Fri Feb 15 13:33:07 2002 Received: from localhost by delta.ds2.pg.gda.pl (8.9.3/8.9.3) with SMTP id NAA01735; Fri, 15 Feb 2002 13:31:44 +0100 (MET) Date: Fri, 15 Feb 2002 13:31:43 +0100 (MET) From: "Maciej W. Rozycki" To: Ralf Baechle cc: "Kevin D. Kissell" , Atsushi Nemoto , mdharm@momenco.com, linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: Re: [patch] linux 2.4.17: The second mb() rework (final) In-Reply-To: <20020215115006.B26756@dea.linux-mips.net> Message-ID: Organization: Technical University of Gdansk MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Length: 1944 Lines: 39 On Fri, 15 Feb 2002, Ralf Baechle wrote: > In practice sync just isn't good enough. Most systems these days use > an I/O bus like PCI which uses posted writes or have some other > non-strongly ordered memory model. Which is why something wmb() or > mb() aren't good enough in driver. I'm just having a nice discussion It depends on what a driver needs. They are perfectly sufficient and actually needed for host bus devices if you need to be sure writes and/or reads happen in order. It's only when a side-effect of a write needs to happen immediately a write-back flush is needed. It doesn't happen that often. AFAIK, PCI only allows posted writes -- it doesn't allow write or read reordering, so mb() and friends are sufficient. Are there any busses we support or are going to in foreseeable future that have a weak ordering model? > about this topic with SGI's IA64 people; we have to come up with a > portable and efficient ABI. ABI? API, I suppose. I can't see anything special here -- drivers already know the bus they are on and they know if it needs any specific handling. I don't think it's possibly to define general functions for I/O buses as actions depend on the device that's being addressed. E.g. for a PCI device a driver needs to perform a read from one of the device's assigned regions and which location exactly, depends on what memory or registers the device implements (surely it can't read one that implies side effects). It can't just do an arbitrary read from a location decoded to the bus. Anyway, will my patch get applied or do we have to wait until the discussion concludes? The patch doesn't affect any I/O bus-specific operations, so it's orthogonal. Also a new API is surely a 2.5 item. -- + Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: macro@ds2.pg.gda.pl, PGP key available + From kevink@mips.com Fri Feb 15 13:44:30 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id NAA25259; Fri, 15 Feb 2002 13:44:30 +0100 (MET) Received-Date: Fri, 15 Feb 2002 13:44:30 +0100 (MET) Received: from mx.mips.com(206.31.31.226) via SMTP by guadalquivir.fnet.fr, id smtpd025257; Fri Feb 15 13:44:22 2002 Received: from newman.mips.com (ns-dmz [206.31.31.225]) by mx.mips.com (8.9.3/8.9.0) with ESMTP id EAA11123; Fri, 15 Feb 2002 04:44:14 -0800 (PST) Received: from grendel (grendel [192.168.236.16]) by newman.mips.com (8.9.3/8.9.0) with SMTP id EAA11277; Fri, 15 Feb 2002 04:44:11 -0800 (PST) Message-ID: <008601c1b61e$ff4d88b0$10eca8c0@grendel> From: "Kevin D. Kissell" To: "Maciej W. Rozycki" Cc: "Atsushi Nemoto" , , , , References: Subject: Re: [patch] linux 2.4.17: The second mb() rework (final) Date: Fri, 15 Feb 2002 13:47:12 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 Content-Length: 754 Lines: 17 > On Fri, 15 Feb 2002, Kevin D. Kissell wrote: > > > So I think that the Linux code was perfectly correct in considering > > the TX39 to be without SYNC, just as a Vr4101 must be > > consdered to be without LL/SC. They decode the instructions, > > but they don't actually implement them as specified. > > The code is not correct if "bc0f" is needed to be sure a write-back > happened. If that is the case, the processors need their own wbflush() > implementation like R2k/R3k configurations in older DECstations. Note that I did not say that "the code is correct", only that it is correct *in considering the TX39 to be effectively SYNC-less*. I'm sure that the code is anyway broken six ways from Wednesday, as usual. ;-) Kevin K. From macro@ds2.pg.gda.pl Fri Feb 15 13:55:40 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id NAA27224; Fri, 15 Feb 2002 13:55:40 +0100 (MET) Received-Date: Fri, 15 Feb 2002 13:55:40 +0100 (MET) Received: from delta.ds2.pg.gda.pl(213.192.72.1) via SMTP by guadalquivir.fnet.fr, id smtpd027209; Fri Feb 15 13:55:25 2002 Received: from localhost by delta.ds2.pg.gda.pl (8.9.3/8.9.3) with SMTP id NAA02328; Fri, 15 Feb 2002 13:53:51 +0100 (MET) Date: Fri, 15 Feb 2002 13:53:50 +0100 (MET) From: "Maciej W. Rozycki" To: Dominic Sweetman cc: "Kevin D. Kissell" , Atsushi Nemoto , mdharm@momenco.com, ralf@uni-koblenz.de, linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: Re: [patch] linux 2.4.17: The second mb() rework (final) In-Reply-To: <200202151156.LAA17485@mudchute.algor.co.uk> Message-ID: Organization: Technical University of Gdansk MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Length: 1045 Lines: 23 On Fri, 15 Feb 2002, Dominic Sweetman wrote: > That's only a problem if the CPU permitted reads to overtake buffered > writes. [Early R3000 write buffers did that (with an address check to > avoid the disaster of allowing a read to overtake a write to the same > location).] The R2020 and the R3220 write buffers that are used in older DECstations seem to provide buffered values themselves if hit by a read. This way they are completely safe for ordinary memory references and there is no need to stall for a write-back completion for memory operations. At least this is what DECstation specifications imply -- it seems hard to get to original docs for the chips these days. For I/O resources implying side effects a stall is needed, of course. The way the chips work is not that uncommon -- e.g. Intel's i486 does exactly the same. -- + Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: macro@ds2.pg.gda.pl, PGP key available + From macro@ds2.pg.gda.pl Fri Feb 15 14:13:20 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id OAA28706; Fri, 15 Feb 2002 14:13:20 +0100 (MET) Received-Date: Fri, 15 Feb 2002 14:13:20 +0100 (MET) Received: from delta.ds2.pg.gda.pl(213.192.72.1) via SMTP by guadalquivir.fnet.fr, id smtpd028704; Fri Feb 15 14:13:16 2002 Received: from localhost by delta.ds2.pg.gda.pl (8.9.3/8.9.3) with SMTP id OAA02690; Fri, 15 Feb 2002 14:03:45 +0100 (MET) Date: Fri, 15 Feb 2002 14:03:45 +0100 (MET) From: "Maciej W. Rozycki" To: Hartvig Ekner cc: Dominic Sweetman , "Kevin D. Kissell" , Atsushi Nemoto , mdharm@momenco.com, ralf@uni-koblenz.de, linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: Re: [patch] linux 2.4.17: The second mb() rework (final) In-Reply-To: <200202151205.NAA25636@copsun18.mips.com> Message-ID: Organization: Technical University of Gdansk MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Length: 1232 Lines: 27 On Fri, 15 Feb 2002, Hartvig Ekner wrote: > But in-order for the CPU is not enough - it also needs to make sure all > written data is visible for DMA accesses to memory from the outside, which > is why the writebuffer needs to be flushed by a SYNC as well. From the > definition of "SYNC": > > "A store is completed when the stored value is visible to every other > processor in the system". > > Which presumably also includes DMA I/O devices... The description found in the R4k manual seems to contradict. A "sync" on a single CPU only assures transactions on it's external bus will happen in order wrt the "sync". In a multiple-CPU environment all CPUs interested in a barrier need to execute a "sync". That's seems natural -- how would you define a synchronization point for a CPU that does not execute a "sync". Since I/O devices have no means to signal a CPU a "sync" operation, how can you expect a "sync" on the CPU to complete all preceding writes to the device? It's only a following read that may imply it. -- + Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: macro@ds2.pg.gda.pl, PGP key available + From macro@ds2.pg.gda.pl Fri Feb 15 14:20:10 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id OAA29706; Fri, 15 Feb 2002 14:20:10 +0100 (MET) Received-Date: Fri, 15 Feb 2002 14:20:10 +0100 (MET) Received: from delta.ds2.pg.gda.pl(213.192.72.1) via SMTP by guadalquivir.fnet.fr, id smtpd029687; Fri Feb 15 14:20:04 2002 Received: from localhost by delta.ds2.pg.gda.pl (8.9.3/8.9.3) with SMTP id OAA03199; Fri, 15 Feb 2002 14:17:30 +0100 (MET) Date: Fri, 15 Feb 2002 14:17:30 +0100 (MET) From: "Maciej W. Rozycki" To: "Kevin D. Kissell" cc: Atsushi Nemoto , mdharm@momenco.com, ralf@uni-koblenz.de, linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: Re: [patch] linux 2.4.17: The second mb() rework (final) In-Reply-To: <008601c1b61e$ff4d88b0$10eca8c0@grendel> Message-ID: Organization: Technical University of Gdansk MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Length: 1351 Lines: 28 On Fri, 15 Feb 2002, Kevin D. Kissell wrote: > > The code is not correct if "bc0f" is needed to be sure a write-back > > happened. If that is the case, the processors need their own wbflush() > > implementation like R2k/R3k configurations in older DECstations. > > Note that I did not say that "the code is correct", only that it > is correct *in considering the TX39 to be effectively SYNC-less*. Then you are probably misinterpreting CONFIG_CPU_HAS_SYNC (shame on me for not documenting it at all). It means a CPU has the "sync" instruction but it does not imply a "sync" is sufficient for mb() on the CPU. See the code in include/asm-mips/system.h in my patch. If CONFIG_CPU_HAS_WB is set wbflush() is executed for mb() regardless the setting of CONFIG_CPU_HAS_SYNC. Fast_mb() is set to a "sync" in case a wbflush() implementation needs it for something -- the DECstation's wbflush() actually uses it in a patch I have queued waiting for my pending submissions to be applied by Ralf. > I'm sure that the code is anyway broken six ways from Wednesday, > as usual. ;-) I hope with my fixes the number of ways decreases, though. ;-) -- + Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: macro@ds2.pg.gda.pl, PGP key available + From macro@ds2.pg.gda.pl Fri Feb 15 20:39:18 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id UAA04853; Fri, 15 Feb 2002 20:39:18 +0100 (MET) Received-Date: Fri, 15 Feb 2002 20:39:18 +0100 (MET) Received: from delta.ds2.pg.gda.pl(213.192.72.1) via SMTP by guadalquivir.fnet.fr, id smtpd004851; Fri Feb 15 20:39:12 2002 Received: from localhost by delta.ds2.pg.gda.pl (8.9.3/8.9.3) with SMTP id UAA14500; Fri, 15 Feb 2002 20:39:11 +0100 (MET) Date: Fri, 15 Feb 2002 20:39:09 +0100 (MET) From: "Maciej W. Rozycki" To: Jason Gunthorpe cc: "Kevin D. Kissell" , linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: Re: [patch] linux 2.4.17: The second mb() rework (final) In-Reply-To: Message-ID: Organization: Technical University of Gdansk MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Length: 1304 Lines: 30 On Fri, 15 Feb 2002, Jason Gunthorpe wrote: > Sorry, why? If the TX39 is the only processor in the system then write > buffers can be left alone. You can't consider PCI IO devices to be That depends on the implementation of the buffers. > processors because the bus protocols would never allow you to satisfy the > requirements for 'sync'. Fully agreed -- I've already expressed that in another mail. > IMHO the only time *mb should care about a write buffer is if the buffer > breaks PCI ordering semantics, by, say returning reads from posted write > data, re-ordering, etc. Well, the "classic" MIPS R2020 and R3220 ones would break PCI (or actually any I/O) ordering semantics as they return data from a posted write upon a hit. The affected read never appears at the I/O bus in that case. They never reorder writes though, as they work as FIFOs (the former is four stage deep and the latter is six stage deep), so wmb() may be null for them. I've read a suggestion a "bc0f" might be needed for the TX39's write buffer as a barrier. That means the buffer behaves as the "classic" ones. -- + Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: macro@ds2.pg.gda.pl, PGP key available + From macro@ds2.pg.gda.pl Mon Feb 18 15:30:47 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id PAA23599; Mon, 18 Feb 2002 15:30:47 +0100 (MET) Received-Date: Mon, 18 Feb 2002 15:30:47 +0100 (MET) Received: from delta.ds2.pg.gda.pl(213.192.72.1) via SMTP by guadalquivir.fnet.fr, id smtpd023590; Mon Feb 18 15:30:38 2002 Received: from localhost by delta.ds2.pg.gda.pl (8.9.3/8.9.3) with SMTP id PAA18433; Mon, 18 Feb 2002 15:24:02 +0100 (MET) Date: Mon, 18 Feb 2002 15:24:02 +0100 (MET) From: "Maciej W. Rozycki" To: Hartvig Ekner cc: Dominic Sweetman , "Kevin D. Kissell" , Atsushi Nemoto , mdharm@momenco.com, ralf@uni-koblenz.de, linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: Re: [patch] linux 2.4.17: The second mb() rework (final) In-Reply-To: <200202181252.NAA03169@copsun18.mips.com> Message-ID: Organization: Technical University of Gdansk MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Length: 1266 Lines: 26 On Mon, 18 Feb 2002, Hartvig Ekner wrote: > You are no doubt correct regarding the R4K manual - so my comment probably > only applies for CPU's that claim to be MIPS32/MIPS64 compliant. All MTI's > CPU offerings (cores only) do in fact flush the WB on a SYNC to comply with > the current spec. Note that the MIPS II spec doesn't forbid a "sync" implementation to be stronger than required. > This I do have a problem understanding. If the SYNC does not flush the WB > on some processor/writebuffer combinations, and a read can be satisfied > out of the WB, how would you ever be able to guarantee that DMA data written > by the CPU has reached memory before triggering the IO device? If after a "sync" an uncached read could be satisfied from an uncommittedd write pending since before the "sync" in a CPU's oncore WB, then the CPU would violate the MIPS II spec of "sync", as the order of transactions at this CPU's external bus would not be preserved. You may exploit this property to do a write-back flush to the host bus -- that's what I added iob() for. -- + Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: macro@ds2.pg.gda.pl, PGP key available + From nemoto@toshiba-tops.co.jp Thu Feb 21 12:37:16 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id MAA16672; Thu, 21 Feb 2002 12:37:16 +0100 (MET) Received-Date: Thu, 21 Feb 2002 12:37:16 +0100 (MET) Received: from topsns.toshiba-tops.co.jp(202.230.225.5) via SMTP by guadalquivir.fnet.fr, id smtpd016670; Thu Feb 21 12:37:07 2002 Received: from inside-ms1.toshiba-tops.co.jp by topsns.toshiba-tops.co.jp via smtpd (for guadalquivir.fnet.fr [193.104.112.133]) with SMTP; 21 Feb 2002 11:37:05 UT Received: from srd2sd.toshiba-tops.co.jp (gw-chiba7.toshiba-tops.co.jp [172.17.244.27]) by topsms.toshiba-tops.co.jp (Postfix) with ESMTP id 6087BB46B; Thu, 21 Feb 2002 20:36:56 +0900 (JST) Received: by srd2sd.toshiba-tops.co.jp (8.9.3/3.5Wbeta-srd2sd) with ESMTP id UAA70710; Thu, 21 Feb 2002 20:36:56 +0900 (JST) Date: Thu, 21 Feb 2002 20:41:20 +0900 (JST) Message-Id: <20020221.204120.102764790.nemoto@toshiba-tops.co.jp> To: macro@ds2.pg.gda.pl Cc: kevink@mips.com, mdharm@momenco.com, ralf@uni-koblenz.de, linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: Re: [patch] linux 2.4.17: The second mb() rework (final) From: Atsushi Nemoto In-Reply-To: References: <20020215.123124.70226832.nemoto@toshiba-tops.co.jp> X-Fingerprint: EC 9D B9 17 2E 89 D2 25 CE F5 5D 3D 12 29 2A AD X-Pgp-Public-Key: http://pgp.nic.ad.jp/cgi-bin/pgpsearchkey.pl?op=get&search=0xB6D728B1 Organization: TOSHIBA Personal Computer System Corporation X-Mailer: Mew version 2.1 on Emacs 20.7 / Mule 4.1 (AOI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Length: 1062 Lines: 26 >>>>> On Fri, 15 Feb 2002 13:08:20 +0100 (MET), "Maciej W. Rozycki" said: macro> IOW, two consecutive writes separated by a "sync" need not macro> imply a write-back, but as soon as a read happens a preceding macro> write-back is a must. TX39 satisfy this on uncached load/store. "sync" does not flush a write buffer, but any uncached load are executed AFTER completion of pending uncached store. On combination of cached and uncached operation, TX39 does not satisfy this order. macro> It sounds weird: it looks like "sync" is useless and basically macro> a "nop". TX39 has a function called "non-blocking load". This function is described on chapter 4.4 of TX39/H2 manual. "sync" operation is meaningful with this function. macro> If this is the case the processors need their own wbflush() macro> implementation. Can you point to specific chapters of macro> specifications that document it? Chapter 4.9.4 in TX39/H2 Japanese manual describes write buffer. But I could not find it in the English manual... --- Atsushi Nemoto From nemoto@toshiba-tops.co.jp Thu Feb 21 12:56:36 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id MAA17741; Thu, 21 Feb 2002 12:56:36 +0100 (MET) Received-Date: Thu, 21 Feb 2002 12:56:36 +0100 (MET) Received: from topsns.toshiba-tops.co.jp(202.230.225.5) via SMTP by guadalquivir.fnet.fr, id smtpd017739; Thu Feb 21 12:56:32 2002 Received: from inside-ms1.toshiba-tops.co.jp by topsns.toshiba-tops.co.jp via smtpd (for guadalquivir.fnet.fr [193.104.112.133]) with SMTP; 21 Feb 2002 11:56:30 UT Received: from srd2sd.toshiba-tops.co.jp (gw-chiba7.toshiba-tops.co.jp [172.17.244.27]) by topsms.toshiba-tops.co.jp (Postfix) with ESMTP id AE931B46B; Thu, 21 Feb 2002 20:56:28 +0900 (JST) Received: by srd2sd.toshiba-tops.co.jp (8.9.3/3.5Wbeta-srd2sd) with ESMTP id UAA70757; Thu, 21 Feb 2002 20:56:28 +0900 (JST) Date: Thu, 21 Feb 2002 21:00:52 +0900 (JST) Message-Id: <20020221.210052.38718643.nemoto@toshiba-tops.co.jp> To: macro@ds2.pg.gda.pl Cc: jgg@debian.org, kevink@mips.com, linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: Re: [patch] linux 2.4.17: The second mb() rework (final) From: Atsushi Nemoto In-Reply-To: References: X-Fingerprint: EC 9D B9 17 2E 89 D2 25 CE F5 5D 3D 12 29 2A AD X-Pgp-Public-Key: http://pgp.nic.ad.jp/cgi-bin/pgpsearchkey.pl?op=get&search=0xB6D728B1 Organization: TOSHIBA Personal Computer System Corporation X-Mailer: Mew version 2.1 on Emacs 20.7 / Mule 4.1 (AOI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Length: 1031 Lines: 23 >>>>> On Fri, 15 Feb 2002 20:39:09 +0100 (MET), "Maciej W. Rozycki" said: macro> Well, the "classic" MIPS R2020 and R3220 ones would break PCI macro> (or actually any I/O) ordering semantics as they return data macro> from a posted write upon a hit. The affected read never macro> appears at the I/O bus in that case. They never reorder writes macro> though, as they work as FIFOs (the former is four stage deep macro> and the latter is six stage deep), so wmb() may be null for macro> them. macro> I've read a suggestion a "bc0f" might be needed for the TX39's macro> write buffer as a barrier. That means the buffer behaves as macro> the "classic" ones. As I wrote in another mail, TX39's uncached load does NOT return data from a write buffer. Uncached load/store always appears on I/O bus in same order. The problem of TX39's write buffer is that cached load/store operation can overtake preceding uncached store operation (even if "SYNC" was exist between these operations). --- Atsushi Nemoto From macro@ds2.pg.gda.pl Thu Feb 21 15:15:15 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id PAA20106; Thu, 21 Feb 2002 15:15:15 +0100 (MET) Received-Date: Thu, 21 Feb 2002 15:15:15 +0100 (MET) Received: from delta.ds2.pg.gda.pl(213.192.72.1) via SMTP by guadalquivir.fnet.fr, id smtpd020104; Thu Feb 21 15:15:05 2002 Received: from localhost by delta.ds2.pg.gda.pl (8.9.3/8.9.3) with SMTP id PAA08434; Thu, 21 Feb 2002 15:12:44 +0100 (MET) Date: Thu, 21 Feb 2002 15:12:43 +0100 (MET) From: "Maciej W. Rozycki" Reply-To: "Maciej W. Rozycki" To: Atsushi Nemoto cc: kevink@mips.com, mdharm@momenco.com, ralf@uni-koblenz.de, linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: Re: [patch] linux 2.4.17: The second mb() rework (final) In-Reply-To: <20020221.204120.102764790.nemoto@toshiba-tops.co.jp> Message-ID: Organization: Technical University of Gdansk MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Length: 2398 Lines: 50 On Thu, 21 Feb 2002, Atsushi Nemoto wrote: > TX39 satisfy this on uncached load/store. "sync" does not flush a > write buffer, but any uncached load are executed AFTER completion of > pending uncached store. On combination of cached and uncached > operation, TX39 does not satisfy this order. With respect to cache refills (what is already cached is irrelevant, obviously, as read accesse to it don't appear externally), "32-bit RISC Microprocessor TX39 Family Core Architecture User's Manual" seems to contradict. In the description of the "sync" instruction it states: "Interlocks the pipeline until the load, store or data cache refill operation of the previous instruction is completed. The R3900 Processor Core can continue processing instructions following a load instruction even if a cache refill is caused by the load instruction or a load is made from a noncacheable area. Executing a SYNC instruction interlocks subsequent instructions until the SYNC instruction execution is completed. This ensures that the instructions following a load instruction are executed in the proper sequence." It's clear "sync" is strong on the TX39, stronger then required by MIPS II. > TX39 has a function called "non-blocking load". This function is > described on chapter 4.4 of TX39/H2 manual. "sync" operation is > meaningful with this function. Chapter 4.3 ("") of the quoted manual. The statement I quoted assures it, indeed (modulo errata, which hopefully don't exist here). > Chapter 4.9.4 in TX39/H2 Japanese manual describes write buffer. But > I could not find it in the English manual... The write buffer is described in the part about "TMPR3901F" (it appears two manuals are concatenated in a single file), chapter 2.3 ("Bus Interface Unit (Bus Controller / Write Buffer)"). It looks like a "bc0f" loop is needed for mb(). The only difference comparing to R2020/R3220 is only a single "nop" is needed after a write for the coprocessor status to become valid, it would seem. It's not documented explicitly but the supplied example code suggests so. The document I'm referring to is available at: "http:/pdf.toshiba.com/taec/components/Generic/TX39_Core_um.pdf". -- + Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: macro@ds2.pg.gda.pl, PGP key available + From macro@ds2.pg.gda.pl Thu Feb 21 15:24:25 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id PAA20659; Thu, 21 Feb 2002 15:24:25 +0100 (MET) Received-Date: Thu, 21 Feb 2002 15:24:25 +0100 (MET) Received: from delta.ds2.pg.gda.pl(213.192.72.1) via SMTP by guadalquivir.fnet.fr, id smtpd020636; Thu Feb 21 15:22:07 2002 Received: from localhost by delta.ds2.pg.gda.pl (8.9.3/8.9.3) with SMTP id PAA08638; Thu, 21 Feb 2002 15:19:26 +0100 (MET) Date: Thu, 21 Feb 2002 15:19:25 +0100 (MET) From: "Maciej W. Rozycki" To: Atsushi Nemoto cc: jgg@debian.org, kevink@mips.com, linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: Re: [patch] linux 2.4.17: The second mb() rework (final) In-Reply-To: <20020221.210052.38718643.nemoto@toshiba-tops.co.jp> Message-ID: Organization: Technical University of Gdansk MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Length: 857 Lines: 21 On Thu, 21 Feb 2002, Atsushi Nemoto wrote: > As I wrote in another mail, TX39's uncached load does NOT return data > from a write buffer. Uncached load/store always appears on I/O bus in > same order. Well, the specification suggests that a load doesn't stall until all data from the buffer are written back. Therefore a load will appear on the host bus before pending writes. It implies mb() has to stall on the "buffer not empty" condition. > The problem of TX39's write buffer is that cached load/store operation > can overtake preceding uncached store operation (even if "SYNC" was > exist between these operations). It's implied by the above. -- + Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: macro@ds2.pg.gda.pl, PGP key available + From nemoto@toshiba-tops.co.jp Fri Feb 22 03:32:21 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id DAA29332; Fri, 22 Feb 2002 03:32:21 +0100 (MET) Received-Date: Fri, 22 Feb 2002 03:32:21 +0100 (MET) Received: from topsns.toshiba-tops.co.jp(202.230.225.5) via SMTP by guadalquivir.fnet.fr, id smtpd029330; Fri Feb 22 03:32:14 2002 Received: from inside-ms1.toshiba-tops.co.jp by topsns.toshiba-tops.co.jp via smtpd (for guadalquivir.fnet.fr [193.104.112.133]) with SMTP; 22 Feb 2002 02:32:12 UT Received: from srd2sd.toshiba-tops.co.jp (gw-chiba7.toshiba-tops.co.jp [172.17.244.27]) by topsms.toshiba-tops.co.jp (Postfix) with ESMTP id 5D24EB474; Fri, 22 Feb 2002 11:32:10 +0900 (JST) Received: by srd2sd.toshiba-tops.co.jp (8.9.3/3.5Wbeta-srd2sd) with ESMTP id LAA72359; Fri, 22 Feb 2002 11:32:10 +0900 (JST) Date: Fri, 22 Feb 2002 11:36:34 +0900 (JST) Message-Id: <20020222.113634.45519920.nemoto@toshiba-tops.co.jp> To: macro@ds2.pg.gda.pl Cc: kevink@mips.com, mdharm@momenco.com, ralf@uni-koblenz.de, linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: Re: [patch] linux 2.4.17: The second mb() rework (final) From: Atsushi Nemoto In-Reply-To: References: <20020221.204120.102764790.nemoto@toshiba-tops.co.jp> X-Fingerprint: EC 9D B9 17 2E 89 D2 25 CE F5 5D 3D 12 29 2A AD X-Pgp-Public-Key: http://pgp.nic.ad.jp/cgi-bin/pgpsearchkey.pl?op=get&search=0xB6D728B1 Organization: TOSHIBA Personal Computer System Corporation X-Mailer: Mew version 2.1 on Emacs 20.7 / Mule 4.1 (AOI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Length: 1509 Lines: 31 >>>>> On Thu, 21 Feb 2002 15:12:43 +0100 (MET), "Maciej W. Rozycki" said: macro> With respect to cache refills (what is already cached is macro> irrelevant, obviously, as read accesse to it don't appear macro> externally), "32-bit RISC Microprocessor TX39 Family Core macro> Architecture User's Manual" seems to contradict. In the macro> description of the "sync" instruction it states: macro> "Interlocks the pipeline until the load, store or data cache macro> refill operation of the previous instruction is completed. The macro> R3900 Processor Core can continue processing instructions macro> following a load instruction even if a cache refill is caused macro> by the load instruction or a load is made from a noncacheable macro> area. Executing a SYNC instruction interlocks subsequent macro> instructions until the SYNC instruction execution is completed. macro> This ensures that the instructions following a load instruction macro> are executed in the proper sequence." The contradiction is came from some confusion about usage of a word "Core" in TX39 manual. Maybe a writer of the quoted statements assumes a write buffer is outside of a "R3900 Processor Core". So if he said "operation is completed" it means "data are sent to a write buffer". Of course this point of view is not acceptable for software programmers... macro> It's clear "sync" is strong on the TX39, stronger then required macro> by MIPS II. So unfortunately this is not true. --- Atsushi Nemoto From macro@ds2.pg.gda.pl Fri Feb 22 15:14:27 2002 Received: (uucp@localhost) by guadalquivir.fnet.fr (8.8.8/97.02.12/Guadalquivir); id PAA04285; Fri, 22 Feb 2002 15:14:27 +0100 (MET) Received-Date: Fri, 22 Feb 2002 15:14:27 +0100 (MET) Received: from delta.ds2.pg.gda.pl(213.192.72.1) via SMTP by guadalquivir.fnet.fr, id smtpd004278; Fri Feb 22 15:14:01 2002 Received: from localhost by delta.ds2.pg.gda.pl (8.9.3/8.9.3) with SMTP id PAA08019; Fri, 22 Feb 2002 15:10:55 +0100 (MET) Date: Fri, 22 Feb 2002 15:10:55 +0100 (MET) From: "Maciej W. Rozycki" To: Atsushi Nemoto cc: kevink@mips.com, mdharm@momenco.com, ralf@uni-koblenz.de, linux-mips@fnet.fr, linux-mips@oss.sgi.com Subject: Re: [patch] linux 2.4.17: The second mb() rework (final) In-Reply-To: <20020222.113634.45519920.nemoto@toshiba-tops.co.jp> Message-ID: Organization: Technical University of Gdansk MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Length: 1041 Lines: 27 On Fri, 22 Feb 2002, Atsushi Nemoto wrote: > The contradiction is came from some confusion about usage of a word > "Core" in TX39 manual. Maybe a writer of the quoted statements > assumes a write buffer is outside of a "R3900 Processor Core". So if > he said "operation is completed" it means "data are sent to a write That's how I understand it. > buffer". Of course this point of view is not acceptable for software > programmers... If we handle it for the DECstation, we can do so for the TX39 as well. > macro> It's clear "sync" is strong on the TX39, stronger then required > macro> by MIPS II. > > So unfortunately this is not true. It is, considering the write buffer is actually external to the CPU. It's even required to be executed before checking the write buffer, as otherwise you may get a false positive result. -- + Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: macro@ds2.pg.gda.pl, PGP key available +