Seperate Build systems might be overriding each others release tag #909

Rolzad73 · 2017-01-28T06:17:56Z

AppVeyor (Win, IOS) build upload to latest-master might be getting overridden by Travis CI (everything else) build latest-master branch uploads because a new tag is created when each build system finishes and uploads.

Rolzad73 · 2017-01-28T09:15:18Z

As well, I think that non master branch builds from Travis are also getting uploaded to latest-master tag, which would be wrong.

Rolzad73 · 2017-02-01T01:34:40Z

Last few builds I've seen and things seem to be working normally. Closing.

Rolzad73 · 2017-02-02T19:38:56Z

Seems to be happening again. Now I am seeing that the Core.zip is not being uploaded.

Rolzad73 · 2017-02-04T03:43:42Z

Here is a recent log portion from latest-master Travis CI build during the binary upload section (in this case, core.zip):

$ python tools/upload-binaries.py -t $GH_TOKEN -s $TRAVIS_COMMIT -l core.zip -r Wagic-core.zip -b $TRAVIS_BRANCH
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:334: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:132: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:132: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/connectionpool.py:852: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
File core.zip has been uploaded as Wagic-core.zip.

It might indicate an older version of python contributing to the upload issue. But then again, the Ubuntu version is Ubuntu 12.04.5 LTS, so I am unsure if an upgrade is even possible. It might not solve the issue anyways, but thought it was good to add the possibility to the issue comments.

xawotihs · 2017-02-05T09:48:20Z

Unless something new happens, I think the bug you're tracking is well known for years and easily worked around: as soon as an artifact fails building, it will not be uploaded in the latest-master release. If it is one of the first artifacts (one for appveyor, one for travis, one for the mac_travis) of the next build run is trying the upload, it will cause releases to be partial for ever because each build will first upload in the next release to complete the artifact set then will detects that artifact are being overwritten and create a new release ...

One easy way to work around that is to manually re-run the build of the missing artifact like I just did with Appveyor.

Rolzad73 · 2017-02-05T16:07:04Z

@xawotihs Thanks for the info. Yes for Travis, I have been able to restart builds that succeeded on the second pass with no code changes to work around this.

For Appveyor however, I do not have that kind of access. I guess we would have to rely on you to catch the fail and spark a new build. Seems like a hassle to ask you to watch every build though.

Really only affects the Windows build as it is from Appveyor. Speaking of which, whatever happened to the IOS auto build? I think it's disabled in builds now.

xawotihs · 2017-02-05T16:43:42Z

To fix properly the build, I think there is a need to update the python script to only upload an artifact to the release matching the github commit. The issue is that currently the mac/iOS build is actually not the same github commit than the Windows and android/linux builds because of the automatic additional commit creating the mac branch. I actually fixed most of that mess in the cmake branch. I modified the Travis build to actually rely on matrix, this matrix support mac and linux builds on the same commit.
The Theos project (the makefile system used by the iOS build) was modified, I tried updating the build to use the new Theos, it's better but it still does not work. At least it goes a bit further.

kevlahnota · 2017-03-11T03:25:55Z

What happened to the releases? It seems wagic qt is the only one uploaded?

Rolzad73 · 2017-03-11T16:21:47Z

Haven't been involved for a while (new job), but have seen this is still an issue. Although I have looked in every once in a while and saw at least some more build artifacts published than just QT linux, but as always, is a roll of the dice which ones show up.

Just looked at the last build log on travis, first of all, even looking at "raw" log shows colour escape codes in it. Don't think that is related in any way, but may indicate how broken that system is.

Otherwise, builds compile fine, the upload messages at the end look fine except for the SSL warnings.

So. I manually restarted the master branch build just on Travis, let's see what happens.

Rolzad73 · 2017-03-11T16:58:28Z

UPDATE:
master build was fine again, this time the log had no color escapes codes in it (yay?).
But no overwrite on latest-master release.

Rolzad73 · 2017-03-11T17:52:44Z

UPDATE UPDATE:
forced another build, this time via commit, master build was fine, Again, only linux QT build uploaded.

kevlahnota · 2017-03-13T01:34:54Z

I have just restarted manually my commit and the results the same. Only Linux Qt is available. Does the phyton script erases previous one uploaded, becuase the last list is Linux Qt, so perhaps it erases/overwrite others?

`travis_time:end:1c1829c0:start=1489368220355950251,finish=1489368261233355472,duration=40877405221
�[0Ktravis_fold:end:after_success.1
�[0Ktravis_fold:start:after_success.2
�[0Ktravis_time:start:07024e72
�[0K$ python tools/upload-binaries.py -t $GH_TOKEN -s $TRAVIS_COMMIT -l core.zip -r Wagic-core.zip -b $TRAVIS_BRANCH
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:334: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:132: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:132: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/connectionpool.py:852: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
File core.zip has been uploaded as Wagic-core.zip.

travis_time:end:07024e72:start=1489368261238777853,finish=1489368263823758258,duration=2584980405
�[0Ktravis_fold:end:after_success.2
�[0Ktravis_fold:start:after_success.3
�[0Ktravis_time:start:1de332ee
�[0K$ python tools/upload-binaries.py -t $GH_TOKEN -s $TRAVIS_COMMIT -l projects/mtg/Android/bin/Wagic-debug.apk -r Wagic-android.apk -b $TRAVIS_BRANCH
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:334: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:132: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:132: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/connectionpool.py:852: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
File projects/mtg/Android/bin/Wagic-debug.apk has been uploaded as Wagic-android.apk.

travis_time:end:1de332ee:start=1489368263828758614,finish=1489368265495832564,duration=1667073950
�[0Ktravis_fold:end:after_success.3
�[0Ktravis_fold:start:after_success.4
�[0Ktravis_time:start:064e0e4c
�[0K$ python tools/upload-binaries.py -t $GH_TOKEN -s $TRAVIS_COMMIT -l projects/mtg/psprelease.zip -r Wagic-psp.zip -b $TRAVIS_BRANCH
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:334: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:132: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:132: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/connectionpool.py:852: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
File projects/mtg/psprelease.zip has been uploaded as Wagic-psp.zip.

travis_time:end:064e0e4c:start=1489368265504920045,finish=1489368267010882757,duration=1505962712
�[0Ktravis_fold:end:after_success.4
�[0Ktravis_fold:start:after_success.5
�[0Ktravis_time:start:1185daed
�[0K$ python tools/upload-binaries.py -t $GH_TOKEN -s $TRAVIS_COMMIT -l qt-gui-build/linuxqtrelease.zip -r Wagic-linux-QT.zip -b $TRAVIS_BRANCH
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:334: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:132: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
!deleting old release! -> latest-master
!deleting old tag! -> latest-master
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:132: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/connectionpool.py:852: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
File qt-gui-build/linuxqtrelease.zip has been uploaded as Wagic-linux-QT.zip.`

Rolzad73 · 2017-03-13T11:25:17Z

@kevlahnota This is just a guess, but possibly because the upload script is getting called a bunch of times at once, maybe there is a threading oddness, where something in python abandons previous calls and only the last one finishes. Or something along those lines.

That is just a guess. Really have no idea, but want to fix this for sure. I think many people are missing out on Wagic because of this.

Rolzad73 · 2017-03-13T11:27:27Z

OH wait!
I see this is happening right before the QT upload:

!deleting old release! -> latest-master
!deleting old tag! -> latest-master

This should be happening far earlier in the sequence. It may not be a script error, but again some kind of threading oddness.

Rolzad73 · 2017-03-13T11:55:00Z

Actually, I was wrong.
in def checkRelease(repository, remote): of upload-binaries.py

I see how the upload script goes through a loop of checking assets and if it finds a match, it deletes the tag? and because recently, only the QT artifact was there, that is when it did the tag delete.

does the script block on this line before continuing?
r = checkRelease(repository, options.remote)

I admit, I am a python newb. Perhaps I will experiment on my fork of the project.

Rolzad73 · 2017-03-15T12:30:47Z

@kevlahnota Well SOMETHING happened, on that last commit/Travis build, now all the release artifacts are there.

Should we stamp a build for permenant release? v0.20.1 was July 2016.

I am not sure of what the versioning scheme was decided upon.

kevlahnota · 2017-03-16T00:23:43Z

Yes but how? I have some commits left (still testing some cards) but I think to be safe, I agree on stamping the new build for permanent release.

Rolzad73 · 2017-03-18T16:33:15Z

I saw today that the build artifacts got messed up again. I went to try another build in order to try to stamp something and this time, the core failed to upload:

Traceback (most recent call last):
  File "tools/upload-binaries.py", line 73, in <module>
    main()
  File "tools/upload-binaries.py", line 68, in main
    asset = r.upload_asset('application/zip', filename , fd)
  File "/usr/local/lib/python2.7/dist-packages/github3/decorators.py", line 38, in auth_wrapper
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/github3/repos/release.py", line 134, in upload_asset
    verify=False)
  File "/usr/local/lib/python2.7/dist-packages/github3/models.py", line 145, in _post
    return self._session.post(url, data, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 535, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/github3/session.py", line 81, in request
    response = super(GitHubSession, self).request(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 473, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', error(32, 'Broken pipe'))

Rolzad73 closed this as completed Feb 1, 2017

Rolzad73 reopened this Feb 2, 2017

Rolzad73 added the Priority-Low label Feb 5, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Seperate Build systems might be overriding each others release tag #909

Seperate Build systems might be overriding each others release tag #909

Rolzad73 commented Jan 28, 2017 •

edited

Loading

Rolzad73 commented Jan 28, 2017 •

edited

Loading

Rolzad73 commented Feb 1, 2017

Rolzad73 commented Feb 2, 2017

Rolzad73 commented Feb 4, 2017

xawotihs commented Feb 5, 2017

Rolzad73 commented Feb 5, 2017

xawotihs commented Feb 5, 2017

kevlahnota commented Mar 11, 2017

Rolzad73 commented Mar 11, 2017

Rolzad73 commented Mar 11, 2017

Rolzad73 commented Mar 11, 2017

kevlahnota commented Mar 13, 2017

Rolzad73 commented Mar 13, 2017

Rolzad73 commented Mar 13, 2017

Rolzad73 commented Mar 13, 2017

Rolzad73 commented Mar 15, 2017

kevlahnota commented Mar 16, 2017

Rolzad73 commented Mar 18, 2017

Seperate Build systems might be overriding each others release tag #909

Seperate Build systems might be overriding each others release tag #909

Comments

Rolzad73 commented Jan 28, 2017 • edited Loading

Rolzad73 commented Jan 28, 2017 • edited Loading

Rolzad73 commented Feb 1, 2017

Rolzad73 commented Feb 2, 2017

Rolzad73 commented Feb 4, 2017

xawotihs commented Feb 5, 2017

Rolzad73 commented Feb 5, 2017

xawotihs commented Feb 5, 2017

kevlahnota commented Mar 11, 2017

Rolzad73 commented Mar 11, 2017

Rolzad73 commented Mar 11, 2017

Rolzad73 commented Mar 11, 2017

kevlahnota commented Mar 13, 2017

Rolzad73 commented Mar 13, 2017

Rolzad73 commented Mar 13, 2017

Rolzad73 commented Mar 13, 2017

Rolzad73 commented Mar 15, 2017

kevlahnota commented Mar 16, 2017

Rolzad73 commented Mar 18, 2017

Rolzad73 commented Jan 28, 2017 •

edited

Loading

Rolzad73 commented Jan 28, 2017 •

edited

Loading