[Top] [All Lists]

Re: [PATCH 1/2] staging: octeon-ethernet: don't assume that CPU 0 is spe

To: Aaro Koskinen <>
Subject: Re: [PATCH 1/2] staging: octeon-ethernet: don't assume that CPU 0 is special
From: David Daney <>
Date: Mon, 30 Sep 2013 12:41:59 -0700
Cc: <>, <>, "Greg Kroah-Hartman" <>, David Daney <>, <>
In-reply-to: <>
List-archive: <>
List-help: <>
List-id: linux-mips <>
List-owner: <>
List-post: <>
List-software: Ecartis version 1.0.0
List-subscribe: <>
List-unsubscribe: <>
Original-recipient: rfc822;
References: <> <> <>
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130625 Thunderbird/17.0.7
On 09/30/2013 12:35 PM, Aaro Koskinen wrote:

On Mon, Sep 30, 2013 at 10:23:10AM -0700, David Daney wrote:
On 09/28/2013 12:50 PM, Aaro Koskinen wrote:
Currently the driver assumes that CPU 0 is handling all the hard IRQs.
This is wrong in Linux SMP systems where user is allowed to assign to
hardware IRQs to any CPU. The driver will stop working if user sets
smp_affinity so that interrupts end up being handled by other than CPU
0. The patch fixes that.

Signed-off-by: Aaro Koskinen <>
  drivers/staging/octeon/ethernet-rx.c | 12 ++++++------
  1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/staging/octeon/ethernet-rx.c 
index e14a1bb..de831c1 100644
--- a/drivers/staging/octeon/ethernet-rx.c
+++ b/drivers/staging/octeon/ethernet-rx.c
@@ -80,6 +80,8 @@ struct cvm_oct_core_state {

  static struct cvm_oct_core_state core_state __cacheline_aligned_in_smp;

+static int cvm_irq_cpu = -1;
  static void cvm_oct_enable_napi(void *_)
        int cpu = smp_processor_id();
@@ -112,11 +114,7 @@ static void cvm_oct_no_more_work(void)
        int cpu = smp_processor_id();

-       /*
-        * CPU zero is special.  It always has the irq enabled when
-        * waiting for incoming packets.
-        */
-       if (cpu == 0) {
+       if (cpu == cvm_irq_cpu) {
                enable_irq(OCTEON_IRQ_WORKQ0 + pow_receive_group);
@@ -135,6 +133,7 @@ static irqreturn_t cvm_oct_do_interrupt(int cpl, void 
        /* Disable the IRQ and start napi_poll. */
        disable_irq_nosync(OCTEON_IRQ_WORKQ0 + pow_receive_group);
+       cvm_irq_cpu = smp_processor_id();

        return IRQ_HANDLED;
@@ -547,8 +546,9 @@ void cvm_oct_rx_initialize(void)
        cvmx_write_csr(CVMX_POW_WQ_INT_PC, int_pc.u64);

-       /* Scheduld NAPI now.  This will indirectly enable interrupts. */
+       /* Schedule NAPI now. */
+       enable_irq(OCTEON_IRQ_WORKQ0 + pow_receive_group);

The fact that you have to manually enable irqs here indicates that
the patch is not good.

Either the enable_irq() is unnecessary, or you broke the logic for
enabling NAPI on more than one CPU.

I am not sure which is the case, but I think it would be best if you
supplied a fixed patch set that corrects whichever happens to be the

No, the original logic was already broken. The code assumed that the
NAPI scheduled by the driver init gets executed always on CPU 0. The
IRQ got enabled just because we are lucky.

No. The default affinity for all irqs is CPU0 for just this reason. So there was no luck involved.

The patch removes such assumption. During the driver init, the IRQ is
disabled and cvm_irq_cpu is -1. So when the NAPI scheduled (on whatever
CPU) by the init is done, the check in cvm_oct_no_more_work() will be
false and the IRQ remains disabled. So the init routine has to enable IRQ
"manually". This is a special case only for the driver initialization.

No. We schedule NAPI at this point. If there are no packets available, the normal driver logic enables the irq. There is no need to explicitly enable it in cvm_oct_rx_initialize().

If this doesn't work, and you need to have an additional place that you enable the irq, then you broke something.

During the normal operation, the IRQ handler record the CPU (on which
it's going the schedule for the first NAPI poll) in cvm_irq_cpu so
cvm_oct_no_more_work() will always re-enable the IRQ correctly.


<Prev in Thread] Current Thread [Next in Thread>