Closes #8
Sunil Garg (827da03b) at 04 Oct 22:12
Read the Stream Setup and initialize resources (containers need to be initialized) to make sure that the source stream can be received by the parser and parser can further feed the messages to the Kafka source topic.
This should be a Python app packaged to a Docker container that takes input parameters as env variables and uses them to do the following:
Input parameters are passed as env variables:
bootstrap.servers - list of brokers to bootstrap kafka connection
OSDU_STREAMS_SUBSCRIBEIDS - the list of message keys to monitor in the source topic and route to sink topic
OSDU_STREAMS_SOURCEBINDINGS - the list of source topics to read messages from
OSDU_STREAMS_SINKBINDINGS - the list of sink topics to write the messages to
Schema of the message is to be discussed!
Create a Logical Stream for the data Source - Register all the information required for initialization of the resources, setup the Kafka topics to make sure that the source stream can be received by the parser and parser can further feed the messages to the Kafka source topic