ADR - Project & Workflow Services - ADR Summary
Decision Title
Overview about Project & Workflow Services
Status
-
Proposed -
Trialing -
Under review -
Approved -
Retired
Context & Scope
From its inception, the OSDU® Forum community have aimed to deliver functionality which supports more efficient and better decision making on large scale capital investments. The aim of the Project & Workflow Service (P&WFS) capability is to deliver a service which achieves that vision. This capability group are aiming for a step change in collaboration, workflow processes and information recorded throughout projects and decision gates. Enabling enhanced assurance, rich contextual information capture, and the pursuit of excellence in project decision-making.
The OSDU® Data Platform has a growing base of supported domain data types and domain data management services which support liberating information from separated scientific & engineering domains and their associated applications. OSDU's potential to support systems of engagements however has not yet matured, and it’s this area where this capability group will focus.
The OSDU® Data Platform has some rudimentary services or functions like the flexible WKS' for capturing meta data, the activity model and notification service, which do combine to facilitate application integration. The current services and data definitions however fall short of actually eliminating workflow bottlenecks and do not provide the depth and granularity of information required to capture contextual metadata from hierarchical, nonlinear, highly iterative and multi-disciplinary energy workflows. Examples of current gaps are:
-
Lack of a defined approach for how existing services & mechanisms should be consistently employed across workflows and applications.
-
Facilitation of a collaboration space used by teams as they work, using either legacy or cloud native applications. Team members and their applications require to work from a consistent defined collection of data for a project with all the E&O and security provided by the OSDU® Data Platform, and create new "work-in-progress" datasets within that Space based on existing data without polluting the System of Record with interim or experimental data which can be low quality and intentionally temporary.
-
An efficient service for datasets that can be refreshed as they are updated by other applications in a controlled fashion.
-
An appropriate level of contextual information associated with project derived data, which delivers sufficiently granular information between project tasks, capturing decisions, lineage and assurance criteria.
To achieve the required functionality P&WFS will deliver a Collaboration Project space and suite of associated services for multi-disciplinary teams (and multi organisational teams? JV's) to work on defined data collections and share interim results as they iterate towards their interpretation deliverables. P&WFS will also provide the ability to record a rich set of information about the workflows. Tasks, interpreters, applications, progress, peer reviews and more will be captured so that when data are produced/published to OSDU there is traceability of both the technical processes and the business decisions they inform.
Background why OSDU System of Record data does not yet address existing data challenges around project data:
The OSDU data platform has been positioned as an innovative data platform that can function as a permanent and authoritative storage system for company data. Also called the System of Record. It does so through concepts for structuring meta data that offers the use of innovative contextual information for end users:
-
Rich meta data that helps users to find data easier,
-
Meta data that explains the relationships between data (lineage model)
-
Meta data to specify for what purpose data can be used (assurance model)
-
Capture steps and key parameters of a workflow to describe how new data is generated (activity model)
To use this data for analysis, decision making and ultimately value creation, OSDU also needs to define ways to interact with this data and enable applications to work with it. The traditional way would be to extract the relevant data from the SOR and store it in a temporary location or application store. Upon completion of a workflow element generated data is either passed on to another application or remains in the temporary store. If the user is diligent the final result is published back into the SOR and the temporary store cleaned. Unfortunately this is typically the weak part and users do not publish back to a SOR or clean up. In addition, the contextual information is mostly not generated automatically and not passed on to other applications. This leads to "project" or "application" data stores that become an amalgamation of intermediate data and final data. As this accumulates over time, these stores keep growing and contain ever more ill-defined versions of data undermining efficiency and effectiveness of other users who then struggle to find the right data.
Business driven functional requirements of P&WFS:
-
To initialize a new P&WFS Collaboration Project.;
Which will define the working Collaboration Project, assign team members (form multiple OSDU organizations?), security level, agreed rules and business outcome to be determined, simulated or defined in this Project Collaboration Space (Our working assumption is that this service will be Outcome driven and time bound.). P&WFS will also need to create a "session manager" to track and record the elements which will become part of the activity or assurance model for any data that will be output from this environment.
-
To select and manage Collaboration Project Data Collection.;
These datasets reside inside OSDU® Data Platform/s and could be represented in a P&WFS instance as meta-data shared with any consumption zone or application able to use the service. (It may also include external data-content as required?) This includes adding new datasets as well as handling updates from the core OSDU platform/s that might originate from other sources.
-
To create new Work-in-Progress Data, associated with Collaboration Project.;
The created data must be stored in OSDU, but NOT automatically made visible outside the Project Collaboration Space.
-
To publish selected Work-in-Progress Data to the OSDU System of Record.;
WIP Project Data Collections comprising OSDU defined data types and/or custom data needs to be made available in the OSDU** System of Record with a known quality and rich context, with its associated lineage / assurance, activity & decision meta information. This should be possible at any time during Collaboration Project's life, according to internal workflow rules as well as cross functional integration needs. A Collaboration Project can also be terminated without any data being published to the System or Record.
-
An effective notification service:
Delivering the P&WFS assigned team information from the System of Record related to relevant changes to the Collaboration Project Data Collection, but also within P&WFS for efficient collaboration using interim results not published to the System of Record.
-
The P&WFS will work within the Core Architecture principles;
As far as possible, this capability group will employ core service features to deliver the outcomes required. We'll only deviate from that principle if an when a feature does not > exist or breaks the capability.
Use Case Examples: (From AhA)
-
Seismic interpretation: A seismic interpreter creates a new horizon interpretation during several work sessions. How to store the temporary work products? How to make data entities that are work-in-progress discoverable and sharable?
-
Selection and sharing of project data: The members of project team select data from OSDU using a data discovery tool for a reservoir review. Petrophysical logs, seismic data, reports, etc... How to organize project data?
-
Well Trajectory Planning: A Well Engineer and a Geoscientist plan a well trajectory. Between them there are many iterations needed to plan a trajectory that satisfies all requirements. Different OSDU based software applications are used to perform the work iterations. How to facilitate data exchange?
-
Assurance workflow: Data supporting a well proposal are stored on OSDU and submitted for technical assurance review. The review is pending. How to avoid that the data is changed/new versions are created while the review is pending?
Areas which require further technical definition:
- DDMS interaction with P&WFS
- Application integration with P&WFS (Through current or new OSDU API's and/or a DDMS style delivery.
- Defining the two publishing scenarios, Within P&WFS & back to SOR.
- Deep dive on E&O
- "As this document is also defining the scope, can we also indicate here different types of applications/users which can use P&WS and different datatypes which can be supported by P&WS"
OSDU Forum Activates & Artefacts Influencing The P&WFS Capability Group
- Aha Submission:
- Collaboration Space:
- Namespace:
- Activity Model/ Activity Template:
- Business Decisions:
- Persisted Collections:
- Technical Assurance:
- External Data Services EDS:
- User Profile:
- Interoperability Working Group:
Frequently Asked Questions:
-
Is this intended to be a joint working application store? NO!
The Collaboration Project should define a Collaboration Project Data Collection so that version control and consistency are maintained for actors working on the data asynchronously. Interim Work-in-progress results can be published back to this space for utilization by other team members but the space is not intended to be a universal application store.
Add more . . .
Working Questions:
-
Will P&WS support multi-region OSDU / global data facilitation across different OSDU instances?
a. Can P&WFS Support 3 separate OSDU Instances? Because OSDU at a core level does not support this.
-
BR 4: do we need to publish the entire associated lineage from PWS back into SoR? (From Suhas for further elaboration.)
Drawings
Proposal for P&WFS MVP 1
Collaboration Space Concept
