Cannot access MDL server

[ISkWW-Z3]

I tried to access MDL using the sample code (pre-trained_model_library.ipynb), but I got the following error.

ConnectionError: HTTPConnectionPool(host='xenon.ism.ac.jp', port=80): Max retries exceeded with url: /api (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0000022531271788>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed'))

Does anyone know what my mistake is?

[ISkWW-Z3]

(I accidentally deleted my previous comment. )
I tried to access MDL using the sample code (pre-trained_model_library.ipynb), but I got the following error.

ConnectionError: HTTPConnectionPool(host='xenon.ism.ac.jp', port=80): Max retries exceeded with url: /api (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0000022531271788>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed'))

Does anyone know what my mistake is?

[TsumiNa]

@ISkWW-Z3 Hi, thank you for your feedback. Our server found some technical issues from last week. We are trying to fix these, but the schedule still open. We hope everything back to normal next week.

[ISkWW-Z3]

Thank you for your reply.
I'm looking forward to retrying.

[ISkWW-Z3]

@TsumiNa Hi. Has your server returned to normal ?
I still have the same problem. If the server is normal, I think my problem is for another reason.

[TsumiNa]

@ISkWW-Z3 We are working on it. Technically everything is ready, but our server is running behind our institute's network. We have to reapply for public access permission from the system manager. This will be the main time delay for the recovery. We plan to resume everything before Christmas this year. Happy new year!!

[ISkWW-Z3]

@TsumiNa Thank you for the detailed information.

[rkawasumi]

I also tried mdl.version using sample code, but it returned the following error.

HTTPError: status_code: 502, Server did not responce.

Is this the same problem as above?

[stewu5]

@ISkWW-Z3 @rkawasumi Thank you for raising the issue. We are expecting a recovery of our model server by the end of Jan. We will provide update if there are any new changes to the plan. Sorry for all the inconvenience causes.

[YIshiyama8]

@stewu5 When I ran mdl.version in the transfer learning tutorial, I got the following error:

HTTPError: status_code: 503, Server did not responce.

Is the model server not yet restored? Or is it my problem?

[TsumiNa]

@YIshiyama8 Sorry for the inconvenience, our server will keep off-line for about two weeks. We are trying to fix the technical problem and bring some critical improvement.

[yoshida-chem]

I also tried to access MDL using the sample code but, I got the following error.
Is this problem still unresolved?

RemoteDisconnected Traceback (most recent call last)
~/opt/anaconda3/envs/xenonpy/lib/python3.7/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
705 headers=headers,
--> 706 chunked=chunked,
707 )

~/opt/anaconda3/envs/xenonpy/lib/python3.7/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
444 # Otherwise it looks like a bug in the code.
--> 445 six.raise_from(e, None)
446 except (SocketTimeout, BaseSSLError, SocketError) as e:

~/opt/anaconda3/envs/xenonpy/lib/python3.7/site-packages/urllib3/packages/six.py in raise_from(value, from_value)

~/opt/anaconda3/envs/xenonpy/lib/python3.7/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
439 try:
--> 440 httplib_response = conn.getresponse()
441 except BaseException as e:

~/opt/anaconda3/envs/xenonpy/lib/python3.7/http/client.py in getresponse(self)
1368 try:
-> 1369 response.begin()
1370 except ConnectionError:

~/opt/anaconda3/envs/xenonpy/lib/python3.7/http/client.py in begin(self)
309 while True:
--> 310 version, status, reason = self._read_status()
311 if status != CONTINUE:

~/opt/anaconda3/envs/xenonpy/lib/python3.7/http/client.py in _read_status(self)
278 # sending a valid response.
--> 279 raise RemoteDisconnected("Remote end closed connection without"
280 " response")

RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

ProtocolError Traceback (most recent call last)
~/opt/anaconda3/envs/xenonpy/lib/python3.7/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
448 retries=self.max_retries,
--> 449 timeout=timeout
450 )

~/opt/anaconda3/envs/xenonpy/lib/python3.7/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
755 retries = retries.increment(
--> 756 method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
757 )

~/opt/anaconda3/envs/xenonpy/lib/python3.7/site-packages/urllib3/util/retry.py in increment(self, method, url, response, error, _pool, _stacktrace)
530 if read is False or not self._is_method_retryable(method):
--> 531 raise six.reraise(type(error), error, _stacktrace)
532 elif read is not None:

~/opt/anaconda3/envs/xenonpy/lib/python3.7/site-packages/urllib3/packages/six.py in reraise(tp, value, tb)
733 if value.traceback is not tb:
--> 734 raise value.with_traceback(tb)
735 raise value

~/opt/anaconda3/envs/xenonpy/lib/python3.7/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
705 headers=headers,
--> 706 chunked=chunked,
707 )

~/opt/anaconda3/envs/xenonpy/lib/python3.7/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
444 # Otherwise it looks like a bug in the code.
--> 445 six.raise_from(e, None)
446 except (SocketTimeout, BaseSSLError, SocketError) as e:

~/opt/anaconda3/envs/xenonpy/lib/python3.7/site-packages/urllib3/packages/six.py in raise_from(value, from_value)

~/opt/anaconda3/envs/xenonpy/lib/python3.7/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
439 try:
--> 440 httplib_response = conn.getresponse()
441 except BaseException as e:

~/opt/anaconda3/envs/xenonpy/lib/python3.7/http/client.py in getresponse(self)
1368 try:
-> 1369 response.begin()
1370 except ConnectionError:

~/opt/anaconda3/envs/xenonpy/lib/python3.7/http/client.py in begin(self)
309 while True:
--> 310 version, status, reason = self._read_status()
311 if status != CONTINUE:

~/opt/anaconda3/envs/xenonpy/lib/python3.7/http/client.py in _read_status(self)
278 # sending a valid response.
--> 279 raise RemoteDisconnected("Remote end closed connection without"
280 " response")

ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

ConnectionError Traceback (most recent call last)
in
2 mdl
3
----> 4 mdl.version

~/opt/anaconda3/envs/xenonpy/lib/python3.7/site-packages/xenonpy/mdl/mdl.py in version(self)
95 @Property
96 def version(self):
---> 97 return GetVersion()()
98
99 def call(self, *query: str,

~/opt/anaconda3/envs/xenonpy/lib/python3.7/site-packages/xenonpy/mdl/base.py in call(self, file, return_json, *querying_vars)
77 data=payload,
78 headers={"content-type": "application/json",
---> 79 'api_key': self._api_key})
80 else:
81 file = Path(file).resolve()

~/opt/anaconda3/envs/xenonpy/lib/python3.7/site-packages/requests/api.py in post(url, data, json, **kwargs)
117 """
118
--> 119 return request('post', url, data=data, json=json, **kwargs)
120
121

~/opt/anaconda3/envs/xenonpy/lib/python3.7/site-packages/requests/api.py in request(method, url, **kwargs)
59 # cases, and look like a memory leak in others.
60 with sessions.Session() as session:
---> 61 return session.request(method=method, url=url, **kwargs)
62
63

~/opt/anaconda3/envs/xenonpy/lib/python3.7/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
540 }
541 send_kwargs.update(settings)
--> 542 resp = self.send(prep, **send_kwargs)
543
544 return resp

~/opt/anaconda3/envs/xenonpy/lib/python3.7/site-packages/requests/sessions.py in send(self, request, **kwargs)
653
654 # Send the request
--> 655 r = adapter.send(request, **kwargs)
656
657 # Total elapsed time of the request (approximately)

~/opt/anaconda3/envs/xenonpy/lib/python3.7/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
496
497 except (ProtocolError, socket.error) as err:
--> 498 raise ConnectionError(err, request=request)
499
500 except MaxRetryError as e:

ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

[stewu5]

@yyamaguchi Sorry for the inconvenience. We have decided to make a completely new model library server and it is expected to be ready in March. We will make the announcement as soon as it is ready again.

[yoshida-chem]

@stewu5
Thank you for letting me know.
I'm looking forward to the new model library.

[okikosuke]

@TsumiNa @stewu5
Is it ready to use the new library now?
I want to use MDL.

[TsumiNa]

Hi @okikosuke, we are moving on to provide the new MDL system for all users. Here, we prepared some scripts to access our under-developing MDL service, but please keep in mind that the API will change without preannouncement.

Also, we are planning to retrain all organic/polymer models because of the changes from rdkit. That will be a time-consuming task.

Scripts: https://www.dropbox.com/sh/s2i90n8dx1at90r/AADVC9AVoT2dENE8xci4Ss-Ra?dl=0

[okikosuke]

@TsumiNa
Thank you for your reply. I understand the current status of MDL. I'm looking forward to the new MDL system being ready.

[youngseonshim]

I have been trying to access to MDL, but i have found the connection errors. I'm not sure whether the error is coming out from your MDL system or not. Please, let me know when the new library is set up. Thanks a lot!

[TsumiNa]

@yyamaguchi Hi! MDL services are renewing to match more complicated searching cases. You can try to use the temporary scripts described here to access our developing server for a review.

If you still find it hard to access the server, please don't be hesitate to let me know.

[youngseonshim]

@yyamaguchi Hi! MDL services are renewing to match more complicated searching cases. You can try to use the temporary scripts described here to access our developing server for a review.

If you still find it hard to access the server, please don't be hesitate to let me know.

Thanks for your temporary scripts. I have been trying to access your developing server, but i couldn't.
I got error messages. Could you figure out the problem? Thanks.

---

gaierror Traceback (most recent call last)
~/venv-test/lib/python3.7/site-packages/urllib3/connection.py in _new_conn(self)
169 conn = connection.create_connection(
--> 170 (self._dns_host, self.port), self.timeout, **extra_kw
171 )

~/venv-test/lib/python3.7/site-packages/urllib3/util/connection.py in create_connection(address, timeout, source_address, socket_options)
72
---> 73 for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
74 af, socktype, proto, canonname, sa = res

/apps/GPGPU/2021_supercom/Python/python3.7/Python-3.7.10_210715/lib/python3.7/socket.py in getaddrinfo(host, port, family, type, proto, flags)
751 addrlist = []
--> 752 for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
753 af, socktype, proto, canonname, sa = res

gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

NewConnectionError Traceback (most recent call last)
~/venv-test/lib/python3.7/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
705 headers=headers,
--> 706 chunked=chunked,
707 )

~/venv-test/lib/python3.7/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
381 try:
--> 382 self._validate_conn(conn)
383 except (SocketTimeout, BaseSSLError) as e:

~/venv-test/lib/python3.7/site-packages/urllib3/connectionpool.py in _validate_conn(self, conn)
1009 if not getattr(conn, "sock", None): # AppEngine might not have .sock
-> 1010 conn.connect()
1011

~/venv-test/lib/python3.7/site-packages/urllib3/connection.py in connect(self)
352 # Add certificate verification
--> 353 conn = self._new_conn()
354 hostname = self.host

~/venv-test/lib/python3.7/site-packages/urllib3/connection.py in _new_conn(self)
181 raise NewConnectionError(
--> 182 self, "Failed to establish a new connection: %s" % e
183 )

NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f10e2983e10>: Failed to establish a new connection: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

MaxRetryError Traceback (most recent call last)
~/venv-test/lib/python3.7/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
448 retries=self.max_retries,
--> 449 timeout=timeout
450 )

~/venv-test/lib/python3.7/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
755 retries = retries.increment(
--> 756 method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
757 )

~/venv-test/lib/python3.7/site-packages/urllib3/util/retry.py in increment(self, method, url, response, error, _pool, _stacktrace)
573 if new_retry.is_exhausted():
--> 574 raise MaxRetryError(_pool, url, error or ResponseError(cause))
575

MaxRetryError: HTTPSConnectionPool(host='xenon.ism.ac.jp', port=443): Max retries exceeded with url: /api (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f10e2983e10>: Failed to establish a new connection: [Errno -2] Name or service not known'))

[chotoQ]

I tried the following program, but I got the error.

--------------program(name: ex.py)-----------
from xenonpy.mdl import MDL
mdl = MDL()
mdl.version

-------------------error-------------------------
Traceback (most recent call last):
File "ex.py", line 3, in
mdl.version
File "/home/.local/lib/python3.8/site-packages/xenonpy/mdl/mdl.py", line 97, in version
return GetVersion()()
File "/home/.local/lib/python3.8/site-packages/xenonpy/mdl/base.py", line 99, in call
message = ret.json()
File "/usr/lib/python3/dist-packages/requests/models.py", line 897, in json
return complexjson.loads(self.text, **kwargs)
File "/usr/lib/python3/dist-packages/simplejson/init.py", line 518, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 370, in decode
obj, end = self.raw_decode(s)
File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 400, in raw_decode
return self.scan_once(s, idx=_w(s, idx).end())
simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Could you figure out the problem?

[deepakorani]

I have the same issue @chotoQ. @TsumiNa any guidance on this matter would be greatly appreciated. Thankyou!

[TsumiNa]

@deepakorani Could you please give us your error messages? We did some tests on our machines (macOS and Ubuntu) and found it works well.

BTW, you have to use the temporary code to access the server.

Hi @okikosuke, we are moving on to provide the new MDL system for all users. Here, we prepared some scripts to access our under-developing MDL service, but please keep in mind that the API will change without preannouncement.

Also, we are planning to retrain all organic/polymer models because of the changes from rdkit. That will be a time-consuming task.

Scripts: https://www.dropbox.com/sh/s2i90n8dx1at90r/AADVC9AVoT2dENE8xci4Ss-Ra?dl=0

If you can not or have no permission to access Dropbox. We can mail you the temporary codes.

[deepakorani]

@TsumiNa Thanks for your prompt reply; I am not sure but I am trying to run this on Google Cloud; and I am using Windows; This is the error I am getting when I access the temporary codes.

[TsumiNa]

@TsumiNa Thanks! This error seems from getting an empty return from the server. I will try to reproduce the error on my machine and check what happened.

[deepakorani]

Sure @TsumiNa. I think the server that the API is calling from is down for me. I am not sure if that's the issue. 👍

[chotoQ]

@TsumiNa @deepakorani After I did this 1.~4. (https://xenonpy.readthedocs.io/en/latest/installation.html), the JSONDecodeError was disappeared. ("cuda101.yml" was not exist, so I got "cuda111.yml".)
But I got the following error.

HTTPError: status_code: 502, Server did not responce.

Is this my problem?

[TsumiNa]

@chotoQ After you followed the installation guides, it should give you a clean environment to run xenonpy.

It seems someone else also got this error code. If you are using Windows, a considerable reason is incompatible with the requests package. Could you give me the error messages when the HTTPError was showing?

[chotoQ]

@TsumiNa I'm using WSL and I got the error messages:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/miniconda3/envs/xenonpy/lib/python3.9/site-packages/xenonpy/mdl/mdl.py", line 97, in version
return GetVersion()()
File "/home/miniconda3/envs/xenonpy/lib/python3.9/site-packages/xenonpy/mdl/base.py", line 103, in __call__
raise HTTPError('status_code: %s, %s' % (ret.status_code, message))
requests.exceptions.HTTPError: status_code: 502, Server did not responce.

[arifin-chemist89]

@TsumiNa do you aware that the end point https://xenon.ism.ac.jp/api caught 502 error, which also shown in your ipynb script.

It seems recommended to look at the /var/log/nginx/error.log on the server side to find out more about the reason of the error. I am not pretty sure if this webpage could give some idea how to solve it. FYI, https://www.datadoghq.com/blog/nginx-502-bad-gateway-errors-php-fpm/

Edit1: Somehow after accessing https://xenon.ism.ac.jp/api, https://xenon.ism.ac.jp/api/auth/signin also has same error..
Edit2: This page also seems could give some idea. https://www.bit-hive.com/articles/20190319

[TsumiNa]

@arifin-chemist89 Thanks! It seems the proxy server blocked the connection somehow. I will check the details next week, help we can fix it immediately.

[deepakorani]

Hi @TsumiNa is their any update on this error? I am still getting the same error till now!

[stewu5]

@deepakorani Sorry for the inconvenience. Please refer to our first page for more information (The top of our Readme page).

[mVenetos97]

@stewu5 @TsumiNa Are there any updates on connecting to the server?

[stewu5]

@mVenetos97 Sorry for the delayed response. Due to many incompatibility issues between models, packages, and the upgraded server, we fail to resolve the problem and now exploring to completely re-structure the model library. We can try to manually provide the models you need through email request. Sorry for the inconvenience.