On Tue, 2011-01-04 at 09:40 -0800, Kevin D. Kissell wrote:
> On 01/04/11 05:02, Anoop P A wrote:
> > On Tue, 2011-01-04 at 00:17 -0800, Kevin D. Kissell wrote:
> >> Those interrupt counters show that IPIs are being taken everywhere,
> >> though very few by CPUs 5 and 6. If I understand the configuration
> >> correctly, CPU 4 is a TC in VPE 1, and it's getting a reasonable IPI
> > Yes CPU4 is in second VPE
> >
> >> rate, *if* we're looking at a tickless kernel under low load. But there
> > No it was not the tickless kernel.I had selected 250 MHz timer. can't we
> > expect IPI / timer interrupt for all the threads in this case ?.
>
> In that case, you should expect a distribution of timer interrupts that
> favors the low-numbered TCs within the VPE, as you do in VPE0, and a
> distribution of IPIs that is sort-of the inverse, as you do in VPE0.
> But the low counts on VPE1 are indeed suspicious, as you note.
>
> >> may be a clue there to part of your problem. I have no idea why the
> >> behavior would have changed from 2.6.36 to 2.6.37, but it looks as if
> >> you're getting your clock interrupts through the MSP CIC interrupt
> >> controller on VPE 0. There's nothing symmetric for VPE1. The Malta
> >> example code is perhaps deceptively simple, in that both VPEs have their
> >> count/compare indication wired directly to the 2 clock interrupt inputs,
> >> so that having both of them running with only a single set of irq state
> >> just works. I don't know whether the MSP CIC timer interrupt is a
> > In my case it is separate irq. MSP_INT_VPE1_TIMER (34) and
> > MSP_INT_VPE0_TIMER (25) are wired to CIC . CIC interrupt has been
> > connected to cpu irq 6.
> >
> > I can reproduce cpu stall in VSMP mode If I don't setup VPE1 timer
> > interrupt . Don't we have support for separate irq in SMTC
> > implementation ?..
>
> There are hooks for platform-specific SMTC support, which is implemented
> for the Malta in arch/mips/mti-malta/malta-smtc.c. See
> msmtc_init_secondary(), for example, where the clock/compare, profile,
> and IPI interrupts are armed for VPE 1, while I/O peripheral interrupts
> are inhibited.
>
> >> gating of the VPE0 count/compare output, or whether it's it's own
> >> interval timer, but I suspect that you may need to do some further
> >> low-level initialization in the platform-specific code to set up an
> >> interrupt on the VPE1 side. I don't think the snippet you've got below
> >> would work as written.
> > The routine which I copied works fine for VSMP mode .
> >
> > / # cat /proc/interrupts
> > CPU0 CPU1
> > 0: 187 254 MIPS IPI_resched
> > 1: 77 174 MIPS IPI_call
> > 6: 0 0 MIPS MSP CIC cascade
> > 8: 0 0 MSP_CIC Softreset button
> > 9: 0 0 MSP_CIC Standby switch
> > 21: 0 0 MSP_CIC MSP PER cascade
> > 25: 37077 0 MSP_CIC timer
> > 27: 188 0 MSP_CIC serial
> > 34: 0 36986 MSP_CIC timer
> >
> > Do I want to change anything specific for SMTC ? .
>
> If it works (which I doubt), then we can critique stylistic points like
> using
>
> if ((1==get_current_vpe())
>
> Instead of the more readable and general
>
> if (get_current_vpe()> 0)
>
>
> But I think you're generally looking in the wrong place. Look at the
> Malta code and see what's done where. The initial SMTC code had a lot
> of Malta assumptions in the main line that I pushed out to platform code
> in later patches. I can see how things could be made even more modular,
> but for the moment I think it's just that there's some stuff that ought
> to be done in a "msp_smtc.c" file that doesn't exist in 2.6.37.
Yes , I am doing similar stuff in msp_smtc.c . Attaching code for your
reference. I am not seeing a VPE1 timer interrupt.
>
> Regards,
>
> Kevin K.
> >
> >
> >> If it's purely an issue with clock distribution on VPE1, then a boot
> >> with maxvpes=1 maxtcs=4 should be stable.
> > Yes the kernel seems to be stable if I boot with maxvpes=1 maxtcs=4 .
> >
> >> /K.
> >>
> >> On 1/3/2011 11:20 AM, Anoop P A wrote:
> >>> Hi Kevin,
> >>>
> >>> On Mon, 2011-01-03 at 08:14 -0800, Kevin D. Kissell wrote:
> >>>> The very first SMTC implementations didn't support full kernel-mode
> >>>> preemption, which anyway wasn't a priority, given the hardware event
> >>>> response support in MIPS MT. I believe it was later made compatible,
> >>>> but it was never extensively exercised. Since SMTC has fingers in some
> >>>> pretty low-level atomicity mechanisms, if a new, parallel set was
> >>>> implemented for RCU, I can easily imagine that nobody has yet
> >>>> implemented SMTC-ified variants of that set.
> >>>>
> >>>> Your last statement isn't very clear, though. Are you saying that if
> >>>> you configure for no forced preemption and with TREE_CPU, the 2.6.37
> >>>> kernel boots all the way up, or that it simply hangs later? What's the
> >>>> last rev kernel that actually boots all the way up?
> >>> I have debugged this a bit more. It seems that kernel getting stalled
> >>> while executing on TC's of second VPE .
> >>> INFO: rcu_sched_state detected stalls on CPUs/tasks: { 4 5 6} (detected
> >>> by 1, t=2504 jiffies)
> >>> INFO: rcu_sched_state detected stalls on CPUs/tasks: { 4 5 6} (detected
> >>> by 1, t=10036 jiffies)
> >>> INFO: rcu_sched_state detected stalls on CPUs/tasks: { 4 5 6} (detected
> >>> by 1, t=17568 jiffies)
> >>> INFO: rcu_sched_state detected stalls on CPUs/tasks: { 4 5 6} (detected
> >>> by 1, t=25100 jiffies)
> >>> INFO: rcu_sched_state detected stalls on CPUs/tasks: { 4 5 6} (detected
> >>> by 1, t=32632 jiffies)
> >>> INFO: rcu_sched_state detected stalls on CPUs/tasks: { 4 5 6} (detected
> >>> by 1, t=40164 jiffies)
> >>>
> >>> With CONFIG_TREE_CPU we were not hitting this scenario very often.
> >>> However with CONFIG_PREEMPT_TREE_CPU stall happens most of the time.
> >>>
> >>> I presume some issue in my timer setup . I am not seeing timer interrupt
> >>> (or IPI interrupt) getting incremented for VPE1 tcs on a completely
> >>> booted 2.6.32-stable kernel.
> >>>
> >>> / # cat /proc/interrupts
> >>> CPU0 CPU1 CPU2 CPU3 CPU4 CPU5
> >>> CPU6
> >>> 1: 148 15023 15140 15093 3779 8
> >>> 2 MIPS SMTC_IPI
> >>> 6: 0 0 0 0 0 0
> >>> 0 MIPS MSP CIC cascade
> >>> 8: 0 0 0 0 0 0
> >>> 0 MSP_CIC Softreset button
> >>> 9: 0 0 0 0 0 0
> >>> 0 MSP_CIC Standby switch
> >>> 21: 0 0 0 0 0 0
> >>> 0 MSP_CIC MSP PER cascade
> >>> 25: 15113 341 4 7 0 0
> >>> 0 MSP_CIC timer
> >>> 27: 260 9 0 1 0 0
> >>> 0 MSP_CIC serial
> >>> 34: 0 0 0 0 0 0
> >>> 0 MSP_CIC timer
> >>>
> >>> Can't we use separate timer interrupts for VPE1 and VPE0 in SMTC ?.
> >>>
> >>> I have tried setting up VPE1 timer from get_co_compare_int as follows
> >>>
> >>> unsigned int __cpuinit get_c0_compare_int(void)
> >>> {
> >>> if ((1==get_current_vpe())&& !vpe1_timr_installed){
> >>>
> >>> memcpy(&timer_vpe1,&c0_compare_irqaction,sizeof(timer_vpe1));
> >>>
> >>> setup_irq(MSP_INT_VPE1_TIMER,&timer_vpe1);
> >>> vpe1_timr_installed++;
> >>> }
> >>> return (get_current_vpe() ? MSP_INT_VPE1_TIMER :
> >>> MSP_INT_VPE0_TIMER);
> >>> }
> >>>
> >>> Thanks
> >>> Anoop
>
msp_smtc.c
Description: Text Data
|