Re: [RFC][PATCH 03/16] sched: Wrap rq::lock access

[Subhra Mazumdar]

On 3/29/19 3:23 PM, Subhra Mazumdar wrote:
> On 3/29/19 6:35 AM, Julien Desfossez wrote:
> > On Fri, Mar 22, 2019 at 8:09 PM Subhra Mazumdar 
> > <subhra.mazumdar@xxxxxxxxxx>
> > wrote:
> > > Is the core wide lock primarily responsible for the regression? I ran
> > > upto patch
> > > 12 which also has the core wide lock for tagged cgroups and also calls
> > > newidle_balance() from pick_next_task(). I don't see any 
> > > regression. Of
> > > course
> > > the core sched version of pick_next_task() may be doing more but
> > > comparing with
> > > the __pick_next_task() it doesn't look too horrible.
> > On further testing and investigation, we also agree that spinlock 
> > contention
> > is not the major cause for the regression, but we feel that it should 
> > be one
> > of the major contributing factors to this performance loss.
> I finally did some code bisection and found the following lines are
> basically responsible for the regression. Commenting them out I don't see
> the regressions. Can you confirm? I am yet to figure if this is needed 
> for
> the correctness of core scheduling and if so can we do this better?
>
> -------->8-------------
>
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index fe3918c..3b3388a 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -3741,8 +3741,8 @@ pick_next_task(struct rq *rq, struct task_struct 
> *prev, struct rq_flags *rf)
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ * If there weren't no cookies; we 
> don't need
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ * to bother with the other siblings.
> */
> -ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ if (i == cpu && !rq->core->core_cookie)
> -ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ goto next_class;
> +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ //if (i == cpu && !rq->core->core_cookie)
> +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ //goto next_class;
>
> continue;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ }
AFAICT this condition is not needed for correctness as cookie matching will
sill be enforced. Peter any thoughts? I get the following numbers with 1 DB
and 2 DB instance.

1 DB instance
users baseline %idle core_sched %idle
16ÂÂÂÂ 1ÂÂÂÂÂÂÂÂÂ 84ÂÂÂÂ -5.5% 84
24ÂÂÂÂ 1ÂÂÂÂÂÂÂÂÂ 76ÂÂÂÂ -5% 76
32ÂÂÂÂ 1ÂÂÂÂÂÂÂÂÂ 69ÂÂÂÂ -0.45% 69

2 DB instance
users baseline %idle core_sched %idle
16ÂÂÂÂ 1ÂÂÂÂÂÂÂÂÂ 66ÂÂÂÂ -23.8% 69
24ÂÂÂÂ 1ÂÂÂÂÂÂÂÂÂ 54ÂÂÂÂ -3.1% 57
32ÂÂÂÂ 1ÂÂÂÂÂÂÂÂÂ 42ÂÂÂÂ -21.1%ÂÂÂÂÂ 48