Have an interesting case I want to share and save for future.
One small problem in this system: https://github.com/trianglesis/BMC_TPL_IDE is showing when some process is running and was not hanged up.
Sometimes we need to run a huge amount of checks and tests and would be great to know if something hangs before killing it mannually.
I used different scenarios and mostly a module pregressbar2, and this is what I finally can compose:
- Make worker which will execute job sorf of randomly:
import time import random print("START SOMETHING") for i in range(5): sec = random.randint(1, 3) print("Sleeping for: %d" % sec) time.sleep(sec) print("Making job, line: %d" % i) print("END SOMETHING")
- Make process executor:
import sys import subprocess import progressbar # Draw spinner: bar = progressbar.ProgressBar(max_value=progressbar.UnknownLength) # Execute some job with multiple lines on stdout: p = subprocess.Popen("C:\Python34\python.exe worker.py", shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) # Lines will be collected in list: result =  # Until I get last line and the end of string: while p.stdout is not None: # Update spinner on one step: # It will update only when any line was printed to stdout! bar.update() # Read each line: line = p.stdout.readline() # Add line in list and remove carriage return result.append(line.decode('UTF-8').rstrip('\r')) # When no lines appears: if not line: print("\n") p.stdout.flush() break # Show finish message, it also useful because bar cannot start new line on console, why? print("Finish:") # Results as string: print(''.join(result))
Voilla! But there is still a small problem – this bar will update it’s state only when process post something in stdout. Between posts it will stay calm for random amount of time (as designed in worker). If I’ll update while or somehow try to make this loop faster – it becomes a deadlock.
Next example is from: https://github.com/trianglesis/BMC_TPL_IDE
In this example – bar works better, because It have a list of tests, so I can enumerate its len and config bar to show how much progress were already done.
But yes, it will update only each one itetarion, and till test run, for 600+ sec – it wouldn’t move.
def tests_executor(self, tests_list): tests_len = len(tests_list) if progressbar: progressbar.streams.flush() progressbar.streams.wrap_stderr() bar = progressbar.ProgressBar(max_value=tests_len) else: log.debug("Module progressbar2 is not installed, will show progress in usual manner.") pass log.info("-==== START RELATED TESTS EXECUTION ====-") log.debug("Tests related to: "+str(tests_list['pattern'])) log.debug("All tests len: "+str(tests_len)) for i, test in enumerate(tests_list): log.info("Start test:" + str(test['rem_test_path'])) if progressbar: bar(range(tests_len)) bar.update(i) else: pass log.info("%d test of "+str(tests_len), i+1) # Just print pre_cmd = ". ~/.bash_profile;" wd_cmd = "cd "+test['rem_test_wd']+";" cmd_test = "/usr/tideway/bin/python -u "+test['rem_test_path']+" --verbose" cmd = pre_cmd + wd_cmd + cmd_test log.debug("Run: "+str(cmd)) try: _, stdout, stderr = self.ssh_cons.exec_command(cmd) if stdout: output = stdout.readlines() raw_out = "".join(output) log.info("-==== DETAILED LOG ====-") log.info("\n"+raw_out) if stderr: output = stderr.readlines() raw_out = "".join(output) log.info("-==== UNITTEST LOG ====-") log.info("\n\n"+raw_out) except: log.error("Test execution command cannot run: "+str(cmd)) if progressbar: bar.finish() else: pass log.info("-==== END OF RELATED TESTS EXECUTION ====-")
So the problem solved, partially.
Still wondering – how to make progressbar or spinner update constantly till process not die?