Skip to content

WorkProduct manifests generation fails while validating Manifest.1.0.0.json

WorkProduct manifests generation fails with the following error each time Manifest.1.0.0.json is validated:

2024-01-10 09:46:53 ERROR Unable to process document file: 69_D_CH_11
Traceback (most recent call last):
  File "/home/zhubin/.local/lib/python3.10/site-packages/jsonschema/validators.py", line 1098, in resolve_from_url
    document = self.store[url]
  File "/home/zhubin/.local/lib/python3.10/site-packages/jsonschema/_utils.py", line 20, in __getitem__
    return self.store[self.normalize(uri)]
KeyError: 'https://schema.osdu.opengroup.org/json/abstract/GenericWorkProductComponent.1.0.0.json'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/zhubin/.local/lib/python3.10/site-packages/urllib3/connection.py", line 159, in _new_conn
    conn = connection.create_connection(
  File "/home/zhubin/.local/lib/python3.10/site-packages/urllib3/util/connection.py", line 61, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/lib/python3.10/socket.py", line 955, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/zhubin/.local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "/home/zhubin/.local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "/home/zhubin/.local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 978, in _validate_conn
    conn.connect()
  File "/home/zhubin/.local/lib/python3.10/site-packages/urllib3/connection.py", line 309, in connect
    conn = self._new_conn()
  File "/home/zhubin/.local/lib/python3.10/site-packages/urllib3/connection.py", line 171, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f67bb1e4e80>: Failed to establish a new connection: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/zhubin/.local/lib/python3.10/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/home/zhubin/.local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 726, in urlopen
    retries = retries.increment(
  File "/home/zhubin/.local/lib/python3.10/site-packages/urllib3/util/retry.py", line 446, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='schema.osdu.opengroup.org', port=443): Max retries exceeded with url: /json/abstract/GenericWorkProductComponent.1.0.0.json (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f67bb1e4e80>: Failed to establish a new connection: [Errno -2] Name or service not known'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/zhubin/.local/lib/python3.10/site-packages/jsonschema/validators.py", line 1101, in resolve_from_url
    document = self.resolve_remote(url)
  File "/home/zhubin/.local/lib/python3.10/site-packages/jsonschema/validators.py", line 1202, in resolve_remote
    result = requests.get(uri).json()
  File "/home/zhubin/.local/lib/python3.10/site-packages/requests/api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "/home/zhubin/.local/lib/python3.10/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/zhubin/.local/lib/python3.10/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/zhubin/.local/lib/python3.10/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/home/zhubin/.local/lib/python3.10/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='schema.osdu.opengroup.org', port=443): Max retries exceeded with url: /json/abstract/GenericWorkProductComponent.1.0.0.json (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f67bb1e4e80>: Failed to establish a new connection: [Errno -2] Name or service not known'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/zhubin/open-test-data/rc--3.0.0/2-scripts/load_manifest_scripts/src/loading_manifest/osdu_document_manifest.py", line 155, in create_document_manifest_from_path
    obj=create_document_manifest(
  File "/home/zhubin/open-test-data/rc--3.0.0/2-scripts/load_manifest_scripts/src/loading_manifest/osdu_document_manifest.py", line 75, in create_document_manifest
    document = cm.create_loading_manifest(
  File "/home/zhubin/open-test-data/rc--3.0.0/2-scripts/load_manifest_scripts/src/loading_manifest/common_manifest.py", line 272, in create_loading_manifest
    validate_schema(lm_group, schema_id_lm, dict_schemas)
  File "/home/zhubin/open-test-data/rc--3.0.0/2-scripts/load_manifest_scripts/src/loading_manifest/common_manifest.py", line 661, in validate_schema
    jsonschema.validate(lm, schema, resolver=resolver)
  File "/home/zhubin/.local/lib/python3.10/site-packages/jsonschema/validators.py", line 1306, in validate
    error = exceptions.best_match(validator.iter_errors(instance))
  File "/home/zhubin/.local/lib/python3.10/site-packages/jsonschema/exceptions.py", line 456, in best_match
    best = next(errors, None)
  File "/home/zhubin/.local/lib/python3.10/site-packages/jsonschema/validators.py", line 368, in iter_errors
    for error in errors:
  File "/home/zhubin/.local/lib/python3.10/site-packages/jsonschema/_keywords.py", line 295, in properties
    yield from validator.descend(
  File "/home/zhubin/.local/lib/python3.10/site-packages/jsonschema/validators.py", line 416, in descend
    for error in errors:
  File "/home/zhubin/.local/lib/python3.10/site-packages/jsonschema/_keywords.py", line 295, in properties
    yield from validator.descend(
  File "/home/zhubin/.local/lib/python3.10/site-packages/jsonschema/validators.py", line 416, in descend
    for error in errors:
  File "/home/zhubin/.local/lib/python3.10/site-packages/jsonschema/_legacy_keywords.py", line 135, in items_draft6_draft7_draft201909
    yield from validator.descend(item, items, path=index)
  File "/home/zhubin/.local/lib/python3.10/site-packages/jsonschema/validators.py", line 416, in descend
    for error in errors:
  File "/home/zhubin/.local/lib/python3.10/site-packages/jsonschema/_keywords.py", line 274, in ref
    yield from validator._validate_reference(ref=ref, instance=instance)
  File "/home/zhubin/.local/lib/python3.10/site-packages/jsonschema/validators.py", line 461, in _validate_reference
    scope, resolved = resolve(ref)
  File "/home/zhubin/.local/lib/python3.10/site-packages/jsonschema/validators.py", line 1087, in resolve
    return url, self._remote_cache(url)
  File "/home/zhubin/.local/lib/python3.10/site-packages/jsonschema/validators.py", line 1103, in resolve_from_url
    raise exceptions._RefResolutionError(exc)
jsonschema.exceptions._RefResolutionError: HTTPSConnectionPool(host='schema.osdu.opengroup.org', port=443): Max retries exceeded with url: /json/abstract/GenericWorkProductComponent.1.0.0.json (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f67bb1e4e80>: Failed to establish a new connection: [Errno -2] Name or service not known'))

After discussion with @deepapathak she found out that changing the version of jsonschema to 3.2.0 in open-test-data/rc--3.0.0/2-scripts/load_manifest_scripts/requirements.txt fixes the issue: