Improvements to the Emscripten buildbot

[hoodmane]

• Use the build script from its new Platform/emscripten/__main__.py location
• Use python3 Platform/emscripten run to avoid hard coding build output path
• Use new EMSDK_CACHE environment variable to allow build script to manage
  Emscripten installations
• Build libmpdec
• Cache libffi and libmpdec
• Allow build script to manage node installations

python/cpython#146156 has to land first

[vstinner]

python/cpython#146156 has to land first

It landed, but its 3.14 backport didn't land yet: python/cpython#146159

[hoodmane]

I guess we'll also want to merge and backport python/cpython#146158 and python/cpython#146160 first so we can move more configuration from here into the cpython repo.

[freakboy3742]

One typo; but otherwise this looks good to me. All the upstream PRs have landed, including 3.14 backports; and I've manually tested the commands on the buildbot.

[vstinner]

All the upstream PRs have landed, including 3.14 backports

Ok. I merged the PR (the new config will be deployed in a few minutes). Let's see how it goes :-)

• WASM Emscripten 3.x
• WASM Emscripten 3.14

[hoodmane]

Thanks @vstinner!

[hoodmane]

Well it is not quite right:
#688

[hoodmane]

Looks working now:
https://buildbot.python.org/all/#/builders/1809/builds/108

[mhsmith]

Since this PR was merged, the Emscripten buildbot has posted some comments on GitHub in which the "traceback logs" appears to be the entire stderr of the build and test process:

• gh-146228: Better fork support in cached FastPath cpython#146231 (comment)
• gh-144319: obtain SeLockMemoryPrivilege on Windows cpython#144928 (comment)

These were brought to my attention because I have a filter set up for the word "Android", which shows up in the log because the Android test has been (correctly) skipped. I haven't had this happen with other buildbots.

Separately, the logs are a bit mangled both in the GitHub comment and the Buildbot web interface because they include ANSI color codes. The other buildbots don't do this either.

[hoodmane]

Seems likely to be because the interpreter crashed? Presumably the bot looked for the pattern that it expects failing unittests to generate and didn't find it so instead it included all of stdout.

[mhsmith]

Yes, it looks like you're right, except it's stderr only. Maybe skip messages shouldn't be printed on stderr, but that's a separate discussion.

[hoodmane]

On the other hand the color handling does seem to be unusual. It's unfortunate because I think the console view is more readable with color, but we probably should make the Emscripten buildbot do the typical thing.

An Ubuntu build bot

The Emscripten build bot

[mhsmith]

Even in the buildbot web interface, in this run the color codes stopped working half way through for some reason (the red is the web interface's default rendering of stderr):

[hoodmane]

That is odd. Do you have any idea how the colors are supposed to get disabled? None of the other build bots seem to explicitly disable them. I guess I should look at the unittest code that decides whether color is on or off.

[hoodmane]

can_colorize

    if not sys.flags.ignore_environment:
        if _safe_getenv("PYTHON_COLORS") == "0":
            return False
        if _safe_getenv("PYTHON_COLORS") == "1":
            return True
    if _safe_getenv("NO_COLOR"):
        return False
    if _safe_getenv("FORCE_COLOR"):
        return True
    if _safe_getenv("TERM") == "dumb":
        return False

    if not hasattr(file, "fileno"):
        return False

    try:
        return os.isatty(file.fileno())
    except OSError:
        return hasattr(file, "isatty") and file.isatty()
``


</p>
</details> 

[hoodmane]

Well you can see that none of the environment variables are set from the listing, and it doesn't seem that an argument is passed to python -m test to turn off the colors. Maybe os.isatty(stdout) returns True on Emscripten and False in other buildbots?

[hoodmane]

Indeed in an ubuntu buildbot os.isatty(file.fileno()) is False but in the Emscripten buildbot os.isatty(file.fileno()) returns True.

$ python Platforms/emscripten/ run -- -c \
       'import sys; import os; print(os.isatty(sys.stdout.fileno()))' | cat
True

[hugovk]

Setting a NO_COLOR=1 env var on this buildbot should disable it globally.

[hoodmane]

Yeah though I'd rather fix the problem with streams. Pyodide handles this correctly, so it's just a matter of pulling in the right part of https://github.com/pyodide/pyodide/blob/main/src/js/streams.ts

Some day this stuff should go into Emscripten, but I've never gotten around to it.