Re: [PATCH 0/4] Allow CPU0 to be nohz full

[Thomas Gleixner]

On Fri, 5 Apr 2019, Nicholas Piggin wrote:
> Thomas Gleixner's on April 5, 2019 12:36 am:
> > On Thu, 4 Apr 2019, Nicholas Piggin wrote:
> > 
> >> I've been looking at ways to fix suspend breakage with CPU0 as a
> >> nohz CPU. I started looking at various things like allowing CPU0
> >> to take over do_timer again temporarily or allowing nohz full
> >> to be stopped at runtime (that is quite a significant change for
> >> little real benefit). The problem then was having the housekeeping
> >> CPU go offline.
> >> 
> >> So I decided to try just allowing the freeze to occur on non-zero
> >> CPU. This seems to be a lot simpler to get working, but I guess
> >> some archs won't be able to deal with this? Would it be okay to
> >> make it opt-in per arch?
> > 
> > It needs to be opt in. x86 will fall on its nose with that.
> 
> Okay I can add that.
> 
> > Now the real interesting question is WHY do we need that at all?
> 
> Why full nohz for CPU0? Basically this is how their job system was
> written and used, testing nohz full was a change that came much later 
> as an optimisation.
> 
> I don't think there is a fundamental reason an equivalent system
> could not be made that uses a different CPU for housekeeping, but I
> was assured the change would be quite difficult for them.
> 
> If we can support it, it seems nice if you can take a particular
> configuration and just apply nohz_full to your application processors
> without any other changes.

This wants an explanation in the patches. And patch 4 has in the changelog:

   nohz_full has been successful at significantly reducing jitter for a
   large supercomputer customer, but their job control system requires CPU0
   to be for housekeeping.

which just makes me dazed and confused :)

Other than some coherent explanation and making it opt in, I don't think
there is a fundamental issue with that.

Thanks,

	tglx