bash - Can GNU Parallel execute more parallel processes? -


can example execute:

parallel -j 200 < list0

where "list" has:

nice -n -20 parallel -j 100 < list2 nice -n -20 parallel -j 100 < list1

would feasible/possible?

the number of processes limited. in practice, don't run more few dozens of processes in parallel (at least on personal computer; on million dollars machines limit bigger).

read fork(2), execve(2), setrlimit(2) man pages, understand when , how can fail.

and before reaching limit fork fail, you'll reach threshold doing more parallel processes slow down entire computation, since scheduler inside kernel overstressed.

if have access network of computers (perhaps 2 of them) consider mpi. if can recode of applications read pthreads (e.g. this tutorial).


Comments

Popular posts from this blog

apache - Remove .php and add trailing slash in url using htaccess not loading css -

inno setup - TLabel or TNewStaticText - change .Font.Style on Focus like Cursor changes with .Cursor -