# Priority between digest message and packet\_In

**URL:** https://forum.p4.org/t/priority-between-digest-message-and-packet-in/271
**Category:** Getting Started with P4
**Created:** [March 13, 2022, 5:58pm UTC](https://forum.p4.org/t/priority-between-digest-message-and-packet-in/271 "2022-03-13T17:58:53Z")
**Posts on this page:** 13
**Page:** 1

<div class="post-metadata">

### Author: ![ppavlidis](https://avatars.discourse-cdn.com/v4/letter/p/ec9cab/32.png) [@ppavlidis](https://forum.p4.org/u/ppavlidis)
#### Post date: [March 13, 2022, 5:58pm UTC](https://forum.p4.org/t/priority-between-digest-message-and-packet-in/271/1 "2022-03-13T17:58:53Z")

</div>

Hello,  
Is there a priority between a digest\_message and packet\_in message about how these different messages send from the data plane to the controller?  
In my case, i firstly send digest message and immediate after that a packet\_in message. in the log file i see that first is sent the digest message and then the packet\_in, but in the controller i receive the packet\_in message from the stream\_message\_response queue.

Does anybody know why is this happen?

---

<div class="post-metadata">

### Author: ![andyfingerhut](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.p4.org/andyfingerhut/32/17_2.png) [@andyfingerhut](https://forum.p4.org/u/andyfingerhut)
#### Post date: [March 13, 2022, 9:18pm UTC](https://forum.p4.org/t/priority-between-digest-message-and-packet-in/271/2 "2022-03-13T21:18:58Z")

</div>

There is nothing in the definition of the P4 architectures like v1model and PSA that mandates what order digest messages and PacketIn messages arrive at a controller. In general, different implementations might send each of these through two or more different queues in driver software, and it is up to the driver software and perhaps many other factors exactly what order they arrive at the controller.

An implementation should give you some assurances of what relative order different PacketIn messages from the switch to the controller will arrive, e.g. first-in/first-out per class of service, perhaps, but even that is target-dependent and not specified by the architecture definitions today.

Similar for the relative order of different Digest messages generated by the data plane.

---

<div class="post-metadata">

### Author: ![ppavlidis](https://avatars.discourse-cdn.com/v4/letter/p/ec9cab/32.png) [@ppavlidis](https://forum.p4.org/u/ppavlidis)
#### Post date: [March 14, 2022, 6:50pm UTC](https://forum.p4.org/t/priority-between-digest-message-and-packet-in/271/3 "2022-03-14T18:50:05Z")

</div>

Because i want to receive first the digest message in the contorller, can i somehow in the p4 program put the packet\_in message to a queue and send first the digest message?

---

<div class="post-metadata">

### Author: ![andyfingerhut](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.p4.org/andyfingerhut/32/17_2.png) [@andyfingerhut](https://forum.p4.org/u/andyfingerhut)
#### Post date: [March 14, 2022, 7:01pm UTC](https://forum.p4.org/t/priority-between-digest-message-and-packet-in/271/4 "2022-03-14T19:01:15Z")

</div>

Why not add a custom header to the packet\_in messages that contains what you are currently putting into a digest message? Then you can be assured that they will arrive to the controller simultaneously.

---

<div class="post-metadata">

### Author: ![ppavlidis](https://avatars.discourse-cdn.com/v4/letter/p/ec9cab/32.png) [@ppavlidis](https://forum.p4.org/u/ppavlidis)
#### Post date: [March 14, 2022, 7:16pm UTC](https://forum.p4.org/t/priority-between-digest-message-and-packet-in/271/5 "2022-03-14T19:16:12Z")

</div>

This case does not always appear and my program works fine when this case does not appear. If i do such a change i should make a lot of changes in the code, so i try to find an alternative,if it exists.

---

<div class="post-metadata">

### Author: ![andyfingerhut](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.p4.org/andyfingerhut/32/17_2.png) [@andyfingerhut](https://forum.p4.org/u/andyfingerhut)
#### Post date: [March 14, 2022, 7:26pm UTC](https://forum.p4.org/t/priority-between-digest-message-and-packet-in/271/6 "2022-03-14T19:26:02Z")

</div>

I am not aware of any way in existing P4-programmable devices to force a digest generated by the data plane to reach the controller before a corresponding packet\_in packet generated by the data plane, nor vice versa.

If you can think of a way to make your controller software react correctly regardless of the order that these two things arrive relative to each other, I would recommend thinking about how much work that would be for you.

---

<div class="post-metadata">

### Author: ![antonin](https://avatars.discourse-cdn.com/v4/letter/a/d07c76/32.png) [@antonin](https://forum.p4.org/u/antonin)
#### Post date: [March 15, 2022, 7:04pm UTC](https://forum.p4.org/t/priority-between-digest-message-and-packet-in/271/7 "2022-03-15T19:04:40Z")

</div>

I would +1 Andy’s suggestion. There is nothing in the P4Runtime spec that guarantees the order in which the stream messages ought to be delivered to the controller. Digest messages are typically generated with information from several dataplane packets (for example a single Digest message will be used to learn multiple MAC addresses), whereas a PacketIn message corresponds to a single dataplane packet.

In the bmv2 + PI implementation, the different message types are processed separately and asynchronously, and they go through completely different queues. Knowing the implementation, it is more likely than the digest message will be sent out first, but there is 0 guarantee,

> [@andyfingerhut](#):
>
> An implementation should give you some assurances of what relative order different PacketIn messages from the switch to the controller will arrive, e.g. first-in/first-out per class of service, perhaps, but even that is target-dependent and not specified by the architecture definitions today.
> 
> Similar for the relative order of different Digest messages generated by the data plane.

I think the ability to prioritize messages is fine, but providing ordering guarantees is another thing. I would be wary of a controller implementation that requires reliable delivery of stream messages from the P4Runtime server, let alone one that requires in-order delivery.

---

<div class="post-metadata">

### Author: ![ppavlidis](https://avatars.discourse-cdn.com/v4/letter/p/ec9cab/32.png) [@ppavlidis](https://forum.p4.org/u/ppavlidis)
#### Post date: [March 16, 2022, 9:41am UTC](https://forum.p4.org/t/priority-between-digest-message-and-packet-in/271/8 "2022-03-16T09:41:51Z")

</div>

So we conclude that the most suitable solution to my problem is what Andy said:

> Why not add a custom header to the packet\_in messages that contains what you are currently putting into a digest message? Then you can be assured that they will arrive to the controller simultaneously.

For the priority part, it still exists the case that digest message comes second after the packet\_in message, right?  
But how can i achieve to prioritize messages?

---

<div class="post-metadata">

### Author: ![andyfingerhut](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.p4.org/andyfingerhut/32/17_2.png) [@andyfingerhut](https://forum.p4.org/u/andyfingerhut)
#### Post date: [March 16, 2022, 1:06pm UTC](https://forum.p4.org/t/priority-between-digest-message-and-packet-in/271/9 "2022-03-16T13:06:35Z")

</div>

“For the priority part, it still exists the case that digest message comes second after the packet\_in message, right?”

As we both wrote above, there is nothing in any implementation that we know about that guarantees that the digest message will become before the PacketIn, nor to guarantee that the PacketIn will come before the digest. If, in your testing, you have consistently seen one arrive at the controller before the other, we believe that is just what you have happened to see occur so far, and if conditions were different somehow, perhaps they might arrive in the other order at the controller. Or maybe it really is guaranteed in the version of BMv2 they will always arrive in that order, but if so, maybe the next version of BMv2 might change that for implementation-specific reasons.

We are both recommending that you DO NOT write controller code that ASSUMES there is a particular relative arrival order of such kinds of messages at the controller. If you do, the system will likely disappoint you at some point in the future, because you made an assumption that was not true.

---

<div class="post-metadata">

### Author: ![ppavlidis](https://avatars.discourse-cdn.com/v4/letter/p/ec9cab/32.png) [@ppavlidis](https://forum.p4.org/u/ppavlidis)
#### Post date: [March 16, 2022, 3:24pm UTC](https://forum.p4.org/t/priority-between-digest-message-and-packet-in/271/10 "2022-03-16T15:24:22Z")

</div>

I want to mention that in different executions i get the correct results so the BMv2 version is not guarantee that.  
Thank you guys for your help, i will find a way to fix it.  
For my last question, is there a way to prioritize messages?

---

<div class="post-metadata">

### Author: ![andyfingerhut](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.p4.org/andyfingerhut/32/17_2.png) [@andyfingerhut](https://forum.p4.org/u/andyfingerhut)
#### Post date: [March 16, 2022, 3:55pm UTC](https://forum.p4.org/t/priority-between-digest-message-and-packet-in/271/11 "2022-03-16T15:55:37Z")

</div>

To prioritize which messages? Prioritize them between which two points of the system?

---

<div class="post-metadata">

### Author: ![ppavlidis](https://avatars.discourse-cdn.com/v4/letter/p/ec9cab/32.png) [@ppavlidis](https://forum.p4.org/u/ppavlidis)
#### Post date: [March 17, 2022, 4:59pm UTC](https://forum.p4.org/t/priority-between-digest-message-and-packet-in/271/12 "2022-03-17T16:59:08Z")

</div>

As priority i mean something like QoS. Is there any reference related to it?

---

<div class="post-metadata">

### Author: ![andyfingerhut](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.p4.org/andyfingerhut/32/17_2.png) [@andyfingerhut](https://forum.p4.org/u/andyfingerhut)
#### Post date: [March 17, 2022, 10:02pm UTC](https://forum.p4.org/t/priority-between-digest-message-and-packet-in/271/13 "2022-03-17T22:02:54Z")

</div>

Do you mean you want to have multiple output queues for a single physical output port in the traffic manager, with a packet scheduling algorithm run on those queues to decide which packet is transmitted next, such as a strict priority queue, or weighted fair queueing, etc.?

If so, currently the answer is “it depends upon the target device”. Which target device do you have in mind?

If the BMv2 software switch, there is an option to enable multiple queues per output port, but only if you make a one-line change to the source code and recompile it from source code. Here is an earlier Github issue where this question was asked, and the answer was given for how to change the source code to enable this: [Does simple\_switch\_grpc support priority queue ? · Issue #877 · p4lang/behavioral-model · GitHub](https://github.com/p4lang/behavioral-model/issues/877)

It would be great if someone was willing to enhance BMv2 source code so that multiple queues per port could be enabled with just a command line option, without having to recompile. See here if you are interested in that: [Consider making it a command line option to enable priority queues · Issue #1065 · p4lang/behavioral-model · GitHub](https://github.com/p4lang/behavioral-model/issues/1065)

If I have completely missed the mark on your question, then I ask you again “prioritize which messages? And prioritize them between which two points of the system?”
