At present we sometimes see problems in gitlab where the environment has
0x80 characters or sequences which are not valid UTF-8.

Avoid this by using bytes for the environment, both internal to buildman
and when writing out the 'env' file. Add a test to make sure this works
as expected.

Reported-by: Marek Vasut <marex@denx.de>
Fixes: e5fc79ea718 ("buildman: Write the environment out to an 'env' file")
Signed-off-by: Simon Glass <sjg@chromium.org>

There have been at least a few cases where an exception has occurred in a
thread and resulted in buildman hanging: running out of disk space and
getting a unicode error.

Handle these by collecting a list of exceptions, printing them out and
reporting failure if any are found. Add a test for this.

Signed-off-by: Simon Glass <sjg@chromium.org>

At present the code to report a build result is duplicated. Put it in a
common function to avoid this.

Signed-off-by: Simon Glass <sjg@chromium.org>

Add some function comments which are missing, or missing arguments.

Signed-off-by: Simon Glass <sjg@chromium.org>

The environment may contain some unicode characters. At least that is what
seemed to happen on one commit:

Building current source for 1 boards (0 threads, 64 jobs per thread)
    0    0    0 /1       -1      (starting)
Traceback (most recent call last):
  File ".../tools/buildman/buildman", line 64, in <module>
    ret_code = control.DoBuildman(options, args)
  File "tools/buildman/control.py", line 372, in DoBuildman
    options.keep_outputs, options.verbose)
  File ".../tools/buildman/builder.py", line 1704, in BuildBoards
    results = self._single_builder.RunJob(job)
  File ".../tools/buildman/builderthread.py", line 526, in RunJob
    self._WriteResult(result, job.keep_outputs, job.work_in_output)
  File ".../tools//buildman/builderthread.py", line 349, in _WriteResult
    print('%s="%s"' % (var, env[var]), file=fd)
UnicodeEncodeError: 'ascii' codec can't encode characters in position
	311-312: ordinal not in range(128)

The problem defies repetition with any change at all to buildman. But
let's set an encoding in any case.

Signed-off-by: Simon Glass <sjg@chromium.org>

At present even if only a single thread is in use, buildman still uses
threading.

For some debugging it is helpful to do everything in the main process.
Allow -T0 to support this.

Signed-off-by: Simon Glass <sjg@chromium.org>

The isAlive() method was deprecated in Python 3.8 and has been removed in
Python 3.9. See https://bugs.python.org/issue37804. Use is_alive() instead.

Since Python 2.6 is_alive() has been a synonym for isAlive(). So there
should be no problems for users using elder Python 3 versions.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Reviewed-by: Simon Glass <sjg@chromium.org>

Buildman reuses build directories from previous builds to avoid the cost
of 'make mrproper' for every build. If the previous build produced an SPL
image but the current one does not, the SPL image will remain and buildman
will think it is a result of building the current board.

Remove these files before building, to avoid this problem.

Signed-off-by: Simon Glass <sjg@chromium.org>

Rename these functions to lower case as per PEP8.

Signed-off-by: Simon Glass <sjg@chromium.org>

This patch makes buildman create linked working trees instead of clones
of the source repository, but keeps updating the older clones of the
repository that might already exist. These worktrees share "everything
except working directory specific files such as HEAD, index, etc." with
the source repository. See the git-worktree(1) manual page for more
information.

If git-worktree isn't available, silently falls back to cloning the
repository.

Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested-by: Simon Glass <sjg@chromium.org>