Device ID assignment for server with multiple devices

P4 Dev
1

I see that the device ID is being assigned to the server by the controller using the SetForwardingPipelineConfig message. Is the expectation is that each device should be managed by a server?

2

Yes, in the P4Runtime API at least, that is the expectation.

Note that here the word “server” is used in the sense of “client/server”, so the server here could be a process running on a tiny cheap embedded CPU core inside the device, or a separate CPU on a logic board next to the network device. It doesn’t necessarily have to be a “server class system” that costs thousands of dollars, it can be quite cheap.

3

Depends on what you mean by this. If you mean, “each device should be managed by its own server”, then no this is not the expectation. There can be a single P4Runtime server managing multiple / many devices, e.g. on a multi-ASIC chassis, and the device_id field in P4Runtime messages is used to identify which device is targeted.

That’s not really how it is meant to work. The device ID is not “assigned” to the server by the controller. The device ID is how the controller and server know they are talking about the same device. For a P4Runtime switch, the device ID is typically assigned to the switch through some out-of-band mechanism (can be as simple as a command-line argument when starting the drivers / P4Runtime server process on the switch).

Note that if you use bmv2 simple_switch_grpc, which includes a P4Runtime gRPC server, each process only supports one P4 device. So this is a special case (perhaps common for P4 switches) where each device has its own P4Runtime server.

4

Thank you Andy,
So the server assigned IDs to the devices it manage. The question is how does the controller knows the devices (and their IDs) on the server side? Is that should be done via the capabilities message?
Another questions regarding client/controller server binding:

  1. Is there is a way to operate at a pipe level resolution (as opposed to device level resolution assuming that one device has multiple pipes)? I.e. is it possible to load one program for pipe A and another for pipe B on the same device?
  2. Warm boot for server device, is that done through SetForwardingPipelineConfig with action = VERIFY?
  3. Is there a method (message) to specify the expected runtime configuration (i.e. table values, action profiles, etc…) or this can simply fit into a regular write message?
5
  1. There have been some issues on Github that I believe have been discussed in P4 API working group meetings, but I was not present at any of them. Here are some sample public issues you can look at. I do not know the status of their implementation:

  1. I do not know this aspect of P4Runtime off the top of my head well enough to answer today. Perhaps someone else might.

  2. P4Runtime controllers typically use write messages to change the configuration of a device.

Regarding the beginning of your latest message, I believe I may have misled you in my earlier answer, and Antonin Bas’s answer is correct.

6

Thank you Antonin,
so if the server manages the devices under it, how does the controller knows which IDs assigned to each device? How does the controller knows which type of device it is? Is there some sort of discovery mechanism like using the capabilities message?

7

That depends and there is no mechanism mandated by P4Runtime.

But a good example may be the Stratum NOS, which I think typically supports 2 modes:

  • the Stratum NOS is started on the switch and provided a “ChassisConfig” file, which includes device id and port configuration (e.g. stratum-onos-demo/leaf4.pb.txt at master · opennetworkinglab/stratum-onos-demo · GitHub). The controller can then retrieve this configuration over gNMI and use the device id obtained from it for P4Runtime.
  • the Stratum NOS is started on the switch without a “ChassisConfig”. The controller then needs to provide it over gNMI before any P4Runtime message can be sent.

So as you can see, in the first case the device id comes from the switch (which is the case I was describing in my original message), while in the second case it comes from the controller. It is possible that the “controller” assigns a device id to the switch, but this is not done over P4Runtime (in Stratum they use gNMI with a Protobuf representation of Openconfig models).