Sink / Web Messages Consumer
Write a simple web app (Python|Node|Java) packaged as a docker container that connects to Kafka, consumes messages from a topic and visualizes this information on a web page.
Input parameters are passed as env variables:
bootstrap.servers - list of brokers to bootstrap kafka connection
OSDU_STREAMS_SUBSCRIBEIDS - the list of message keys to monitor in the source topic and route to sink topic
OSDU_STREAMS_SOURCEBINDINGS - the list of source topics to read messages from
OSDU_STREAMS_SINKBINDINGS - the list of sink topics to write the messages to
-
On start-up extract the parameters from env variables: -
bootstrap.servers = localhost:9092 (list of brokers to bootstrap Kafka connection) -
OSDU_STREAMS_SUBSCRIBEIDS = "opendes:work-product-component--WellLog:be54a691c0384182944d71c6b2b6f699" -
OSDU_STREAMS_SOURCEBINDINGS = "opendes_myApp--WellLog_1.0.0" -
OSDU_STREAMS_SINKBINDINGS = "http://localhost:8080" (not used in this application)
-
-
establish connection with Kafka using bootstrap servers and topics information -
run infinite loop of reading messages from the source topic, extract the measurements from the avro message and visualize it on the page -
create tests for the app