State in a P4 program

I am interested in creating a P4 program that is stateful. Specifically, I am adding to the basic P4 network in the tutorial exercises to process a custom header that has the ability to change the state of the switch. If one of the fields in the custom header is a 1, I want the target switch to drop all subsequent icmp packets. If that same field is a 0, I want the target switch to forward all subsequent icmp packets. I have already successfully changed the P4 program to process the custom header and perform different actions based on if the field is a 1 or a 0, but I am unsure how to have it change the subsequent behavior of the switch. (My first thought was to have some sort of variable that stores the value of the field, but the only variables that can be declared that seem to survive between packets are constants) Is this possible with P4 and if so, what is the best way to do so?
Byron Denham

There are two main ways, which differ in aspects like the rate of state changes that can be supported, how much code you need to write in your controller program, and also somewhat in how many resources are required in the data plane.

(a) Use a P4 register array. These can be read and written from the P4 program, so you could write a P4 program that on receiving a packet that should change the future processing behavior of ICMP packets, your P4 program would recognize those packets and write to a P4 register array element. When processing ICMP packets, your P4 code would read that same element of the P4 register array and decide what to do with the ICMP packet.

This approach requires little or no controller software, and can support rates of change to the P4 register array contents as fast as the data plane can process packets. Typically the packets writing to the P4 register array will have the write effects visible to the very next packet received by the device, so the latency between receiving a packet that affects how future ICMP packets should be processed, can take effect “immediately”, i.e. on the very next packet.

(b) “Punt” selected packets to the controller, which is programmed to recognize those packets and modify P4 table entries, which affect how future packets are processed. When your P4 code processes ICMP packets, it would look up an entry in some P4 table to decide what to do with that ICMP packet. When your P4 code processes packets that should change how future ICMP packets are processed, it sends the packet to a CPU port, which is received by your controller software, which modifies however many table entries are needed to affect future ICMP packet processing.

This approach requires a little bit of control plane software to recognize the packets and update the required table entries. Packets sent to the CPU port and processed by the controller can be lost if they can arrive in very long high-rate bursts such that the controller cannot keep up with their arrival rate. It is up to your use case whether you consider that likely or unlikely to happen. There is a noticeable latency between the time that the data plane receives packets that should change the state, vs. when the controller gets to the point of updating the table entry, so it might be hundreds, thousands, or millions of packet times before the behavior of future ICMP packets are processed, after receiving the control packet. Again, it is up to your use case whether that is still good enough for you, or not.

Ok great! Are there any examples or documentation that you would suggest for learning how to interact with the register array that you mention in option A from a p4 program in the lab environment? (It seems like it might be specific to the environment, specifically the BMv2 switch)

I am sure there are many such example programs, but a fairly simple one can be found here: p4-guide/ptf-tests/registeraccess at master · jafingerhut/p4-guide · GitHub

That program would be quite a bit shorter, except that it also demonstrates a way to enable the controller to read & write elements of the register array, by injecting packets into the data plane from the controller. Unfortunately, the current status is that in its open source implementation, it is not possible to read or write register array elements from the controller using P4Runtime API.

not too much, there is no inter-packet-state and between the stages you need to use bridged metadata much like in the real asics…