Concurrent Futures is a concurrency library present in Python since 2.7. It allows easy management of parallel/background jobs. I had a project
recently where my background threads would appear to die silently. I was expecting to see an Exception somewhere in my logs but found nothing.
It turns out somewhere in concurrent.futures.ThreadpoolExecutor, the exception gets gobbled up. The documentation page shows the Futures object
actually has an exception() method which will return the Exception raised in the background thread. However there is a catch, all we get is the exception
message which might not be all that useful in actually tracking down the bug. What we really want is the traceback, showing which file and line caused
We can however fix this issue by catching the exception, and re-raising it with more information in the message. Our worker function looks something
The above snippet is fairly simple, we get the traceback from sys.exc_info(). We can then determine the filename, line number and actual traceback. Its
just a matter of constructing a new message with that information contained within and then raising a RuntimeError().