<!--
SPDX-FileCopyrightText: 2019-2024 Helmholtz Centre Potsdam GFZ German Research Centre for Geosciences
SPDX-FileCopyrightText: 2021-2024 Helmholtz-Zentrum hereon GmbH

SPDX-License-Identifier: CC-BY-4.0
-->

# Messaging and Message Brokers

## Messaging

In standard web-based frameworks, communication works via HTTP requests. The
client (e.g. the frontend of a website) makes a GET or POST to server (backend)
and receives a response. This standard framework, however, exposes your backend
to a lot of security issues and it takes a lot of effort making it secure.

DASF uses a slightly different approach. Here, client and backend cannot
communicate with each other directly. Instead, we use a so-called message
broker. The client stub (see [above](concept-rpc)) sends the request to a
so-called message broker which forwards the request to the server stub.

```{digraph} request
---
caption: Request to the backend module
alt: Request to the backend module
---
    rankdir="LR";
    "client" -> "message broker" -> "server";
```

Then the server processes the request and sends back a response to the client, again via the message broker.

```{digraph} response
---
caption: Response from the backend module
alt: Response from the backend module
---
    rankdir="LR";
    "server" -> "message broker" -> "client";
```

It is somehow comparable to a chat system. Assume Alice wants to
chat with Bob using their mobile phones. The mobile phones do not know each
other but instead they both register at a server (the message broker). When
Alice wants to send a message to Bob, she actually sends it to the server and
the server forwards it to Bobs mobile phone.

The advantage of such a framework is that neither the client nor the server
need to be accessible from the web. This is especially important if you want to
use protected computing resources such as internal servers or HPC clusters. The
only thing you have to do is to protect and secure the message broker, this
already takes most of the burden to secure your _backend module_.

## Message Brokers

### Concept

Message brokers distinguish between _consumers_ and _producers_. As the names
suggest, _producers_ produce a message and _consumers_ consume the message.
When a client stub makes a request, the client is the _producer_, and the
server is the _consumer_.

```{digraph} requestbroker
---
caption: Request to the backend module. The client **produces** a message, the server **consumes**.
alt: Request to the backend module
---
    rankdir="LR";
    "client (producer)" -> "message broker" -> "server (consumer)";
```

And for the response, the client is the _consumer_
and the server is the _producer_.

```{digraph} responsebroker
---
caption: Response from the backend module. The server **produces** a message, the client **consumes**.
alt: Response from the backend module
---
    rankdir="LR";
    "server (producer)" -> "message broker" -> "client (consumer)";
```

To connect producer and consumer, we use so-called _topics_. You can think of
a topic to be the address of the _consumer_ at the message broker. A client
that wants to send a request to a backend module, needs to know the topic that
the backend module is listening to (i.e. that the backend module is
_consuming_). Likewise, the backend module needs to know where it
should send the response to (i.e. it needs to know the topic that the client is
_consuming_).

Within the DASF Framework, you specify the topic when you start listening for
incoming messages. This is specified with the `-t` option

```bash
python ExampleMessageConsumer.py -t my-personal-topic listen
```

or you specify it in the call of the {func}`~demessaging.backend.main` function,

```python
if __name__ == "__main__":
    main(messaging_config=dict(topic="my-personal-topic"))
```

The response topic, i.e. the topic that the client is consuming, is specified
within the properties of the request (see
{class}`demessaging.messaging.constants.PropertyKeys`).
From this, the backend module automatically knows where to send the response
to.

(messagebroker)=
### Choosing the message broker

There are a variety of message brokers around, such as
[Apache Pulsar](https://pulsar.apache.org/) or [MQTT](https://mqtt.org/). And
we are working on a [Django-based message broker][dasf-broker-django] in the
DASF context.

We aim to be agnostic to what message broker is used. Currently, however, we
only tested our workflows with the Apache Pulsar and our Django-based message
broker. An important point is that, by design, our connection to the message
broker does not use provided client libraries, such as the `pulsar-client`,
etc.. Instead, we use the websocket protocol to connect to the message broker.
This is necessary to guarantee flexibility of the framework. So the choice of
your message broker depends on the application that you want to build.

[dasf-broker-django]: https://dasf-broker-django.readthedocs.io/en/latest/