OSDev.org

The Place to Start for Operating System Developers
It is currently Fri Apr 19, 2024 1:25 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 5 posts ] 
Author Message
 Post subject: Tried to implement simple barrier in linux shell but fail
PostPosted: Mon Sep 21, 2020 7:38 pm 
Offline
Member
Member

Joined: Wed Nov 18, 2015 3:04 pm
Posts: 396
Location: San Jose San Francisco Bay Area
I think I am doing something wrong when I tried to implement simple barrier in linux shell script but seem to fail spectacularly.
Code:
# some code block before launching scripts in parallel.
<snippet1>
...
for loop
    # should launch instances of each script1 in background since ampersand implied.
    launch another shell script in <script1> in background &
done

#barrier implementation, wait loop until all instances of script1 finished running.
for loop
   count=`ps -ef | grep <script1>  | wc -l`
   if count -eq 0
      break
      fi
done
...
# some code block after launching scripts in parallel.
<snippet2>


The problem is I can see <snippet1> and all instances of <script1> is launched
But output of code block in <snippet2> appearing before the output of instances of <script1>.
Because there is a dependency where <script1> can not run after <snippet2> is executed it runs into all sort of errors.

_________________
key takeaway after spending yrs on sw industry: big issue small because everyone jumps on it and fixes it. small issue is big since everyone ignores and it causes catastrophy later. #devilisinthedetails


Top
 Profile  
 
 Post subject: Re: Tried to implement simple barrier in linux shell but fai
PostPosted: Mon Sep 21, 2020 8:26 pm 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 5137
Try replacing your barrier with this:
Code:
wait


Top
 Profile  
 
 Post subject: Re: Tried to implement simple barrier in linux shell but fai
PostPosted: Tue Sep 22, 2020 5:50 am 
Offline
Member
Member
User avatar

Joined: Thu Oct 13, 2016 4:55 pm
Posts: 1584
ggodw000 wrote:
Code:
count=`ps -ef | grep <script1>  | wc -l`
This always going to be larger than 0, because grep itself is included in ps output. (btw the flag "f" makes no sense here.)
Code:
$ ps -ef | grep something
bzt       780842  147715  0 13:44 pts/0    00:00:00 grep something

This trick I've learned from an old UNIX guru:
Code:
$ ps -ef | grep [s]omething
This will list all processes containing "something" except the grep :-) The reason is, the command will be "grep [s]omething", however [] is replaced by the shell, and "grep something" will actually be executed, that doesn't match "[s]omething".

Btw, "wait" is a much better solution, I agree.

Cheers,
bzt


Top
 Profile  
 
 Post subject: Re: Tried to implement simple barrier in linux shell but fai
PostPosted: Tue Sep 22, 2020 7:58 am 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 1604
bzt wrote:
This always going to be larger than 0, because grep itself is included in ps output. (btw the flag "f" makes no sense here.)
Not necessarily (and that can lead to some fun bugs): The shell will spawn all parts of the pipeline in parallel. So whether "grep" has been exec'd by the time ps gets to that process is not defined. There is no sequence that makes that a necessity, and indeed, in some situations ps fails to output the line with "grep" in it (if grep happens to get a very small PID and ps has already iterated over it by the time grep is exec'd). And so you end up with the grep line sometimes being in the output and sometimes not. Happy debugging!

The trick you showed is a good one, however. I already knew it, but it is a good one for newbies (regular expressions don't have to match themselves). Alternatively, you can filter the "grep" line afterwards with "grep -v grep". But for this particular problem, Octocontrabass was entirely correct: There is no need to iterate over all processes when you only need to wait for your child processes.

_________________
Carpe diem!


Top
 Profile  
 
 Post subject: Re: Tried to implement simple barrier in linux shell but fai
PostPosted: Tue Sep 22, 2020 8:55 am 
Offline
Member
Member

Joined: Wed Nov 18, 2015 3:04 pm
Posts: 396
Location: San Jose San Francisco Bay Area
thanks I used wait and apparently working much better.

_________________
key takeaway after spending yrs on sw industry: big issue small because everyone jumps on it and fixes it. small issue is big since everyone ignores and it causes catastrophy later. #devilisinthedetails


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 5 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: Bing [Bot] and 46 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group