bash - Can GNU Parallel execute more parallel processes? -


can example execute:

parallel -j 200 < list0

where "list" has:

nice -n -20 parallel -j 100 < list2 nice -n -20 parallel -j 100 < list1

would feasible/possible?

the number of processes limited. in practice, don't run more few dozens of processes in parallel (at least on personal computer; on million dollars machines limit bigger).

read fork(2), execve(2), setrlimit(2) man pages, understand when , how can fail.

and before reaching limit fork fail, you'll reach threshold doing more parallel processes slow down entire computation, since scheduler inside kernel overstressed.

if have access network of computers (perhaps 2 of them) consider mpi. if can recode of applications read pthreads (e.g. this tutorial).


Comments

Popular posts from this blog

python 3.x - Mapping specific letters onto a list of words -

javascript - jquery or ashx not working -

inno setup - TLabel or TNewStaticText - change .Font.Style on Focus like Cursor changes with .Cursor -