Repository navigation
Logs are not complete and scrambled #142
Description
The logs of my jobs are missing some lines (I print one line every batch, approx 6 out of 7 are missing!) and sometimes the lines are scrambled (the outputs of different epochs are interleaved). Note that these are two separate issues, i.e., I checked that the missing lines are not just somewhere else in the log, they are actually missing.
It might have to do with progressbar2, but I have no evidence of this. The only reason for this claim is that there might be an incompatibility with progressbar rewriting multiple times the same line.
Activity
mgermain commentedon Oct 27, 2016
I've never heard of this one. Might be related to progressbar2 which I've never tried with smartdispatch. I don't know if @MarcCote knows more about this issue. For now, can you tell us on which cluster you've seen this issue and do you have code to reproduce?
fvisin commentedon Oct 27, 2016
I was using Helios. You can almost certaintly reproduce it with this code (I'm using the same main loop, which is where the progressbar is called):
https://github.com/fvisin/reseg
Thank you for investigating the issue!