GNU bug report logs - #42044
nproc says "1" on mobile processors

Previous Next

To reply to this bug, email your comments to 42044 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Adam Borowski <kilobyte <at> angband.pl>
To: bug-coreutils <at> gnu.org
Subject: nproc says "1" on mobile processors
Date: Thu, 25 Jun 2020 19:37:39 +0200

Hi!
I'm afraid that `nproc` shows only the number of _currently_ online CPUs,
which on mobile processors tends to be 1 when starting a job.  As there's
a need to conserve power, holding cores online when they have nothing to
do would be a waste, thus they constantly get onlined and offlined.

For example, on a 10-core box where the processor consists of three
clusters, 4+4+2 cores, topline graph shows:

(oooo▁▁▁ oo)
(oooo▁ ▁ oo)
(▃ooooooooo)
(oooo▁▁▁▁oo)
(▂ooooooooo)
(oooo▁▁▁ oo)
(oooo▁ ▁ oo)
(oooo▁▁  oo)
(oooo▁  ▁oo)
(oooo▁▁  oo)
(oooo▁   oo)

(1 line = 1 second, "o" means offline core, otherwise it's utilization
level.)

Because of noisy GUI (that eg. draws this very graph), the machine keeps
flipping between onlining just CPU 0, or a cluster of CPU4..7.  Thus,
`nproc` will randomly says either "1" or "4", while the user expects to
run her compile with all 10 cores.

I found out that numa_num_task_cpus() from libnuma gets the right answer,
handling both affinity mask and CPUs present-but-currently-offline.  But
alas, it doesn't provide a command-line tool, and thousands of scripts
already use nproc, thus switching a tool would be a waste of effort.

Ie, could you please make nproc include all available CPUs rather than
only online ones?


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ in the beginning was the boot and root floppies and they were good.
⢿⡄⠘⠷⠚⠋⠀                                       -- <willmore> on #linux-sunxi
⠈⠳⣄⠀⠀⠀⠀

Message #8 received at 42044 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Adam Borowski <kilobyte <at> angband.pl>, 42044 <at> debbugs.gnu.org
Subject: Re: bug#42044: nproc says "1" on mobile processors
Date: Fri, 26 Jun 2020 19:26:16 +0100

On 25/06/2020 18:37, Adam Borowski wrote:
> Hi!
> I'm afraid that `nproc` shows only the number of _currently_ online CPUs,
> which on mobile processors tends to be 1 when starting a job.  As there's
> a need to conserve power, holding cores online when they have nothing to
> do would be a waste, thus they constantly get onlined and offlined.
> 
> For example, on a 10-core box where the processor consists of three
> clusters, 4+4+2 cores, topline graph shows:
> 
> (oooo▁▁▁ oo)
> (oooo▁ ▁ oo)
> (▃ooooooooo)
> (oooo▁▁▁▁oo)
> (▂ooooooooo)
> (oooo▁▁▁ oo)
> (oooo▁ ▁ oo)
> (oooo▁▁  oo)
> (oooo▁  ▁oo)
> (oooo▁▁  oo)
> (oooo▁   oo)
> 
> (1 line = 1 second, "o" means offline core, otherwise it's utilization
> level.)
> 
> Because of noisy GUI (that eg. draws this very graph), the machine keeps
> flipping between onlining just CPU 0, or a cluster of CPU4..7.  Thus,
> `nproc` will randomly says either "1" or "4", while the user expects to
> run her compile with all 10 cores.
> 
> I found out that numa_num_task_cpus() from libnuma gets the right answer,
> handling both affinity mask and CPUs present-but-currently-offline.  But
> alas, it doesn't provide a command-line tool, and thousands of scripts
> already use nproc, thus switching a tool would be a waste of effort.
> 
> Ie, could you please make nproc include all available CPUs rather than
> only online ones?

Does `nproc --all` suffice for your use case?

Message #11 received at 42044 <at> debbugs.gnu.org (full text, mbox):

From: Adam Borowski <kilobyte <at> angband.pl>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 42044 <at> debbugs.gnu.org
Subject: Re: bug#42044: nproc says "1" on mobile processors
Date: Fri, 26 Jun 2020 21:18:57 +0200

On Fri, Jun 26, 2020 at 07:26:16PM +0100, Pádraig Brady wrote:
> On 25/06/2020 18:37, Adam Borowski wrote:
> > Hi!
> > I'm afraid that `nproc` shows only the number of _currently_ online CPUs,
> > which on mobile processors tends to be 1 when starting a job.  As there's
> > a need to conserve power, holding cores online when they have nothing to
> > do would be a waste, thus they constantly get onlined and offlined.

> > Ie, could you please make nproc include all available CPUs rather than
> > only online ones?
> 
> Does `nproc --all` suffice for your use case?

It works on the mobile box, but doesn't obey affinity mask anymore:

[~]$ numactl -N 0 nproc --all
64
[~]$ numactl -N 0 nproc 
16

I'd wish for fitting all uses, big and small.  Ie, the answer "on this
machine, with currently imposed limits, what's the fastest level of
parallelism?".


Meow.
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ in the beginning was the boot and root floppies and they were good.
⢿⡄⠘⠷⠚⠋⠀                                       -- <willmore> on #linux-sunxi
⠈⠳⣄⠀⠀⠀⠀

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.