personalfoki.blogg.se

Message queue system
Message queue system








message queue system

However, a server would have to ack a message if the client isn't connected to that server, which means if the client isn't connected at all, all servers will ack and the message will never get sent when the client connects. I could have a queue created whenever a new server spins up, and the publisher sends each message to all queues (fanout). However, this requires that queues be dynamically, and ideally quickly, created either by the sender or by the subscriber (as it's not tenable to pre-create a queue for every client id), depending on whether a message or connection happens first, and that isn't a feature I can find. It also has the benefit that the log of messages in the queue is exactly the log of messages sent to that client, so it's easier to trace and debug client sessions.

message queue system

a server subscribes to a client's queue when the client connects. This probably has the cleanest semantics. One major design choice is whether there's a single, global queue (or pubsub topic, etc) and the target client id is an attribute or metadata on the message, or a queue per server, or a queue per client. However this use case doesn't seem to cleanly line up with how they're designed. There are a bunch of options here: AWS SQS, Google PubSub, RabbitMQ, etc. It feels like a design smell to force direct connections to ephemeral servers Servers in a load balancing cluster are not supposed to be individually addressable.Message senders are responsible for queueing/retrying/handling acks/most of the other benefits a message queue framework provides.Message senders are responsible for holding the message if the client isn't in the registry.It forces push semantics and an open port on the application servers to listen for messages.However, this has a bunch of undesirable characteristics I want to avoid: Servers would be responsible for maintaining it as clients connect and disconnect, and message senders would be responsible for doing the lookup and initiating connections to the application servers. I could have a global lookup or registry that stores the mapping between server instance and the list of client ids connected to it. I'm trying to figure out the best routing/queue design to get that message to the correct server, but it doesn't seem to match up with the assumptions underlying existing messaging/pubsub/notification/etc frameworks. A message might be sent before the client connects (in which case it should be held until the client connects).

message queue system

Each server knows the id of each client connected to it.Īt some point another system component will want to send a message to a specific one of the connected clients. I have a system where a potentially large number of clients make persistent streaming connections to an auto-scaling cluster of application servers in the cloud.










Message queue system