Comments on: Gentoo: Portage’s new –jobs feature

By: Aniruddha

Aniruddha — Thu, 31 Jul 2008 19:48:25 +0000

@jolexa
Lol, I didn’t even realize this was a bug. Thanks for filing the bugreport!

By: jolexa

jolexa — Tue, 29 Jul 2008 17:01:39 +0000

@Aniruddha, nice find. I spoke with zmedico and filed a bug at his request. https://bugs.gentoo.org/show_bug.cgi?id=233296

By: Aniruddha

Aniruddha — Mon, 28 Jul 2008 07:43:45 +0000

Thanks for the answer. i noticed that running multiple jobs is problematic when you need to accept a license. My emerge world couldn’t continue because I needed to accept the ut2004 license and I couldn’t see the “accept license screen”

By: jolexa

jolexa — Sat, 26 Jul 2008 11:41:35 +0000

@Aniruddha: For a dual core proc, I would probably recommend –jobs=2 MAKEOPTS=-j3. Depends on alot of things though, RAM, IO wait, etc. Please try out a few things but –jobs=8 is high for only 2 cores. (My host has 8 cores)

Thrashing the CPU, is really using your CPU for more than you have available. ie. Trying to do way more than it can handle and making it switch processes alot. Here is a definition of disk thrashing, apply it to CPUs: http://www.cryer.co.uk/glossary/d/disk_thrashing.htm

By: What’s Going On? » Gentoo - Gentoo Lives!

What’s Going On? » Gentoo - Gentoo Lives! — Sat, 26 Jul 2008 07:20:18 +0000

[...] blog post with more [...]

By: Aniruddha

Aniruddha — Sat, 26 Jul 2008 05:23:48 +0000

So –jobs=8, MAKEOPTS=3, load-average=10 is the recommended setting for a dualcore processor? What do you mean with “trashing your cpu”?

By: jolexa

jolexa — Fri, 25 Jul 2008 15:01:18 +0000

It was suggested that I was thrashing my CPU with MAKEOPTS set so high. I agree with that on the last trial where MAKEOPTS was set to 40, but I didn't agree with it on any of the other trials based solely on load average. It was also suggested that I may be trashing my IO. The suggested test was to do --jobs=8 and MAKEOPTS="-j5" to keep both IO & CPU busy but not thrashing. I obliged:

With --jobs=8, MAKEOPTS=5, load-average=10: real 58m41.048s user 41m43.281s sys 34m6.764s

By: jolexa

jolexa — Fri, 25 Jul 2008 13:03:39 +0000

@cynyr: Assuming all deps are serial, yes –jobs will do nothing. Also, assuming that all of the jobs compile at the exact same time (unlikely, IME) then you will get a bottleneck at the CPU again.

@Duncan: 4 gigs of ram but no tmpfs. I do understand that there would be an improvement here but I do not have the time to set that up. Plus, this is actually on RHEL4 which is annoying enough to deal with . Looks like it is time to try tmpfs on my home box again.

By: Duncan

Duncan — Fri, 25 Jul 2008 11:02:03 +0000

You don’t mention… how much memory do you have and where do you have PORTAGE_TMPDIR pointing? Is it per chance at a tmpfs or similar?

I’ve found that makes a big difference here (not nearly your “toy”, but dual Opteron 290, so dual dual-core 2.8 GHz, 8 gig RAM, and a 4-way RAID (kernel/mdp RAID-6 for system and user data, mdp RAID-0 for PORTDIR and ccache, and 4-way striped swap).

I’ve been doing manual parallel compiles for some time now, and really, with per-user kernel scheduling and PORTAGE_NICENESS=19 (to get the batch priority timeslice length bonus in addition to the better interactivity), I’ve found load average is pretty much irrelevant to system responsiveness. I can have a load average of hundreds (kernel and IIRC glibc builds are nice for their many parallelizable jobs) and still have a nicely usable system, as long as I’m running my interactive stuff mostly as a different user, and don’t bog down the kernel in I/O-wait.

Pointing PORTAGE_TMPDIR at tmpfs definitely helps with I/O, but then of course one has to track memory usage. With the 4-way-striped swap, I’ve found I can go a gig into swap before things start noticeably slowing down there, but I use job limits and load-average control way more as an indirect method of limiting memory and tmpfs usage than to really limit load average, which as I said, is pretty much irrelevant in and of itself, provided I’m interacting as a different user with its own kernel enforced time-share if user has need of it.

I’m running the 2.2 rcs but haven’t yet updated this week, so am still on rc1. Today’s update day, tho, so I’ll soon be playing with the new options too! =8^)

By: cynyr

cynyr — Thu, 24 Jul 2008 23:42:05 +0000

I was wondering how well the –jobs option handles build time dependencies, run time ones, and trying to keep the number of jobs up. It would seem that if a package’s tree was completely serial (every package is needed for the next) that –jobs does nothing to speed this up, and you would want makeopts=-j$(cores), but of something has 12 deps that can all be made at once you wouldn’t want each to have the same makeopts=-j$(cores). So can it balance the two cases?