I am facing an issue while using the Clone function type E2E (Egress to Egress).
My idea is to write INT information to the INT header stacks on the packets that come into the Egress pipeline and send a clone of the full packet on port 1 to a monitoring system.
So far so good, everything is working well. However, as I am using VXLAN encapsulation, I have to decapsulate the original packet from outer headers and VXLAN headers, and as a consequence also remove the INT headers, to send a regular packet to the destination host.
In my case, the program is removing all the corresponding headers and is then cloning the decapped packet, resulting in two small packets while neither of them carries outer headers, VXLAN and INT headers.
Does someone have an idea how to send a copy of the full packet out of port 1 before removing the headers?
for further understanding, I am writing the INT information in the Egress pipeline because I for instance need to hop latency on the switch (deq_timedelta), and I also require the packets and INT information to be exactly the same on both instances. So cloning from Ingress to Egress should not be an option if possible…
This is a guess, but I think that making the headers invalid before cloning could be the actual issue. You mention “removing all corresponding headers”, so I guess you mean VXLAN and INT headers + metadata. The clone function, I assume, will only clone the valid headers. This makes sense to me, but I have not tested it, so not sure if my statement is completely true. I guess you already achieved the same conclusion.
Your program should be able to determine when a switch is first hop (source), last hop (sink) or transit. It should also be able to either clone the packet, determine if the packet is cloned and encapsulate it into a report or remove telemetry headers and send it to the end host. You can track if the packet is first, last hop or anything else using local metadata, which is stateful per se across Ingress and Egress in bmv2. In any case, at the Egress control block, you should be able to play with if-else blocks to determine what to do:
Shall this packet be cloned (it means this is a last hop)?
Shall this packet be forwarded as originally perceived? (it means you just cloned the packet and you forward it as)
Has this packet just been cloned? (it means you just checked the instance type and you know this is a cloned packet?
I am going to make the first version of the code public in approx 5 hours. I want to make a public version of the VM already working, but I do not have time for that. At least, I will make the code available to anyone.
If you read the details about clone operations in v1model / simple_switch here behavioral-model/simple_switch.md at main · p4lang/behavioral-model · GitHub you can see that for clone operations performed during egress, the resulting clone packet will be as the packet was at the end of egress processing, so if your egress processing is invalidating one or more headers, the cloned packets will not have those headers, either.
That documentation also mentions that for clone operations performed during ingress processing, the cloned packets will be as the packet was when it began ingress processing. Thus if you can arrange to make the clone decision during ingress instead of egress, that might result in behavior that you want.