Resolve "Sync with DD, latest schemas and reference-values" M9

Closes #21 (closed)

Summary of Changes Schemas

(since last update July 29, 2021 SHA 9118f8a7a7e1a4361335fbdf96c8c1744ed951a5) Links lead to Member GitLab.

1. Anonymous object fragments
   1. The up to now anonymous object in the data allOf array is programmatically named IndividualProperties to assist the schema version validation. Objects will be identified via their title to perform matching and comparison for schema validation.
   2. Also added a title for Well and Wellbore's VerticalMeasurementID element.
      
   3. Also improved: the large table describing the data properties in the E-R markdown are split into sections by allOf array members.
2. WellboreMarkerSet
   Similar modernization as for WellLog;1.1.0 and WellboreTrajectory:1.1.0. The content for the 'bulk' data (e.g., in the CSV file) ws worked out earlier by Chris Hough and Daniel Perna, see here:
   1. Added osdu:wks:reference-data--MarkerPropertyType:1.0.0 to hold the well-known marker property types - or in context of CSV files column definitions. This type is based on osdu:wks:AbstractReferencePropertyType:1.0.0.
   2. Created osdu:wks:work-product-component--WellboreMarkerSet:1.1.0 to add
      1. data.AvailableMarkerProperties[] The array of MarkerProperty definitions describing the available properties for this instance of WellboreMarkerSet
      2. data.Markers[].MarkerSubSeaVerticalDepth The Marker's TVD converted to a Sub-Sea Vertical depth, i.e., below Mean Sea Level. Note that TVD values above MSL are negative. This is the same as true vertical depth referenced to the vertical CRS “MSL depth”.
3. PersistedCollection
   Moved master-data--Collection (created earlier above) to the work-product-component group-type as osdu:wks:work-product-component--PersistedCollection:1.0.0.
   1. Added support for a generic collection entity, which can refer to any master-data, reference-data, work-product and/or work-product-component group-type member. PersistedCollection is not to be confused with the transient DataCollection. The new entity is documented here: osdu:wks:work-product-component--PersistedCollection:1.0.0.
   2. The collection purpose or intent is classified by a new reference value type osdu:wks:reference-data--CollectionPurpose:1.0.0.
4. Generic Geometry Datasets
   In preparation of Generic Geometry Representations, we need content descriptions, which are specific enough to let consumers understand whether they can parse the data content or not.
   1. osdu:wks:dataset--File.GeoJSON:1.0.0 for WGS 84 vector geometries.
   2. osdu:wks:dataset--FileCollection.Esri.Shape:1.0.0 for shapefiles. Shapefiles can refer to any coordinate reference system (CRS) but not necessarily to a complete spatial reference allowing the geometry to be placed on the world map. The full CRS specification is provided in the dataset via the schema fragment osdu:wks:AbstractSpatialReference:1.0.0
   3. Reference values for EncodingFormatType and SchemaFormatType are delivered in parallel.
   4. For further details and examples, please refer to the WorkedExamples
5. Governance Change for EncodingFormatType, SchemaFormatType
   New reference values have been added to support generic geometries and images.
   1. osdu:wks:reference-data--EncodingFormatType:1.0.0 changed from LOCAL to OPEN.
   2. osdu:wks:reference-data--SchemaFormatType:1.0.0 changed from LOCAL to OPEN.
6. AbstractFacility and FacilityType
   1. Add a description to AbstractFacility clarifying the term facility in OSDU.
   2. Update the description for FacilityType accordingly.
7. WellLog:1.1.0 Corrections
   Based on issue #255 "Well Log Curve depths and units in WellLog_1.1.0", some deprecated properties (data.Curves[].TopDepth, data.Curves[].BaseDepth, data.Curves[].DepthUnit) have been revived. The names stay unchanged to avoid disruptions for consumers. The description have been updated to note that the values may refer to other references than only depth.
8. ActivityTemplate, ActivityTemplateArc
   1. Added data.Name and data.Description to osdu:wks:master-data--ActivityTemplate:1.0.0, which was an oversight during the definition phase. It is expected that this type is not in widespread use; therefore the minor version is not incremented.
   2. Added data.Description to osdu:wks:master-data--ActivityTemplateArc:1.0.0. It is expected that this type is not in widespread use; therefore the minor version is not incremented.

Summary of Changes Reference Values

Corresponding to reference-values repo SHA cb1e513c44dfc924b821a6ee2a3dbcc5f83de442 M9

Referential Integrity

1. Ingestion tests have shown that some relationships were incorrect or referring to non-existing records. The manifest generation process has been enhanced to check the existence of every related object. All broken relationships have been corrected
2. Some reference value types have references/relationships to other reference types, some have even cyclic references. During the schema generation these dependencies are detected and captured in the ReferenceValueTypeDependencies.json. Types, which have interdependencies are merged into one manifest file (MergedManifests in ReferenceValueTypeDependencies.json).
3. All manifests with internal parent-child dependencies are re-ordered such that parents get created before their children. This applies to the merged manifests in particular.

UTC DateTime

• All date-times have been adjusted to conform to either UTC Zulu time or UTC+offset. This change enables successful indexing of date-times.

Manifest Loading Sequence: IngestionSequence.json

Following the ReferenceValueTypeDependencies.json IngestionSequence it is possible to construct a IngestionSequence.json. It is a simple list of JSON objects linking the kind or the merged manifest name to a relative file location. The first two records are shown below:

[
  {
    "kind": "ANT_ANTC_OT_SO",
    "Key": "ANT_ANTC_OT_SO",
    "FileName": "Manifests/reference-data/OPEN/ANT_ANTC_OT_SO.json"
  },
  {
    "kind": "reference-data--ActivityType:1.0.0",
    "Key": "ActivityType.1.0.0",
    "FileName": "Manifests/reference-data/LOCAL/ActivityType.1.0.0.json"
  }
]

1. The first element refers to a merged manifest, in this case "AliasNameType.1.0.0", "AliasNameTypeClass.1.0.0", " OrganisationType.1.0.0", "StandardsOrganisation.1.0.0".
2. The second element refers to an ordinary manifest.
3. The inline example is truncated after the first two elements. See IngestionSequence.json for the full details.

If the manifests are loaded in the sequence as listed in IngestionSequence.json all reference-value relationships are resolvable.

Cleanups

1. A duplicate CollectionPurpose manifest was removed from the LOCAL folder.
2. Governance Change for AliasNameTypeClass from LOCAL to OPEN.
3. Additional values for:
   1. AzimuthReferenceType
   2. FacilityType
   3. AgreementType (Aliases only)