There are many key-value pairs in the registers that I want to insert into the packet to forward. If I fill them in the packet as a custom header, will the protocol layer be too complex, and it would be unwise to manually define many key-value pairs in header struct.
It turns out that there is an open source P4 program for Tofino1 that does this, which you can examine here: GitHub - p4lang/p4app-switchML: Switch ML Application That particular program can add / remove either 128 or 256 bytes of the packet per pass (with recirculation operations done between passes), either writing data from received packets into register, or reading data from registers and prepending it into new packet headers. I do not recall off-hand whether it is 128 or 256 bytes per pass, but I am pretty sure it is one of those two values. The precise upper limit that is possible might be larger than that, but probably not twice as large as that.
Thank you very much. There are two 128-byte data headers to support harvesting 256 bytes with recirculation in Switch ML Application. Your reply is very helpful to me.