# Device ID assignment for server with multiple devices **URL:** \ **Category:** P4 Dev\ **Created:** [February 12, 2022, 2:37am UTC](https://forum.p4.org/t/device-id-assignment-for-server-with-multiple-devices/240 "2022-02-12T02:37:51Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![vainron](https://avatars.discourse-cdn.com/v4/letter/v/f4b2a3/32.png) [@vainron](https://forum.p4.org/u/vainron)\ **Post date:** [February 12, 2022, 2:37am UTC](https://forum.p4.org/t/device-id-assignment-for-server-with-multiple-devices/240/1 "2022-02-12T02:37:51Z")
I see that the device ID is being assigned to the server by the controller using the SetForwardingPipelineConfig message. Is the expectation is that each device should be managed by a server? ---
**Author:** ![andyfingerhut](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.p4.org/andyfingerhut/32/17_2.png) [@andyfingerhut](https://forum.p4.org/u/andyfingerhut)\ **Post date:** [February 12, 2022, 3:15pm UTC](https://forum.p4.org/t/device-id-assignment-for-server-with-multiple-devices/240/2 "2022-02-12T15:15:07Z")
Yes, in the P4Runtime API at least, that is the expectation. Note that here the word “server” is used in the sense of “client/server”, so the server here could be a process running on a tiny cheap embedded CPU core inside the device, or a separate CPU on a logic board next to the network device. It doesn’t necessarily have to be a “server class system” that costs thousands of dollars, it can be quite cheap. ---
**Author:** ![antonin](https://avatars.discourse-cdn.com/v4/letter/a/d07c76/32.png) [@antonin](https://forum.p4.org/u/antonin)\ **Post date:** [February 14, 2022, 5:40pm UTC](https://forum.p4.org/t/device-id-assignment-for-server-with-multiple-devices/240/3 "2022-02-14T17:40:25Z")
> [@vainron](#): > > Is the expectation is that each device should be managed by a server? Depends on what you mean by this. If you mean, “each device should be managed by its own server”, then no this is not the expectation. There can be a single P4Runtime server managing multiple / many devices, e.g. on a multi-ASIC chassis, and the device\_id field in P4Runtime messages is used to identify which device is targeted. > [@vainron](#): > > I see that the device ID is being assigned to the server by the controller using the SetForwardingPipelineConfig message. That’s not really how it is meant to work. The device ID is not “assigned” to the server by the controller. The device ID is how the controller and server know they are talking about the same device. For a P4Runtime switch, the device ID is typically assigned to the switch through some out-of-band mechanism (can be as simple as a command-line argument when starting the drivers / P4Runtime server process on the switch). Note that if you use bmv2 simple\_switch\_grpc, which includes a P4Runtime gRPC server, each process only supports one P4 device. So this is a special case (perhaps common for P4 switches) where each device has its own P4Runtime server. ---
**Author:** ![vainron](https://avatars.discourse-cdn.com/v4/letter/v/f4b2a3/32.png) [@vainron](https://forum.p4.org/u/vainron)\ **Post date:** [February 14, 2022, 8:02pm UTC](https://forum.p4.org/t/device-id-assignment-for-server-with-multiple-devices/240/4 "2022-02-14T20:02:59Z")
Thank you Andy, So the server assigned IDs to the devices it manage. The question is how does the controller knows the devices (and their IDs) on the server side? Is that should be done via the capabilities message? Another questions regarding client/controller server binding: 1. Is there is a way to operate at a pipe level resolution (as opposed to device level resolution assuming that one device has multiple pipes)? I.e. is it possible to load one program for pipe A and another for pipe B on the same device? 2. Warm boot for server device, is that done through SetForwardingPipelineConfig with action = VERIFY? 3. Is there a method (message) to specify the expected runtime configuration (i.e. table values, action profiles, etc…) or this can simply fit into a regular _write_ message? ---
**Author:** ![andyfingerhut](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.p4.org/andyfingerhut/32/17_2.png) [@andyfingerhut](https://forum.p4.org/u/andyfingerhut)\ **Post date:** [February 14, 2022, 11:43pm UTC](https://forum.p4.org/t/device-id-assignment-for-server-with-multiple-devices/240/5 "2022-02-14T23:43:20Z")
1. There have been some issues on Github that I believe have been discussed in P4 API working group meetings, but I was not present at any of them. Here are some sample public issues you can look at. I do not know the status of their implementation: > > > \### Problem > > Some architectures like the Tofino Native Architecture (TNA) for …the Barefoot Tofino chip consist of multiple parallel hardware pipelines, instead of the usual one. They are attached to distinct sets of front panel ports, can be individually programmed and have independent state. Currently there is no mechanism in P4Runtime to target such a subset of the pipelines. > > Examples > \- Asymmetric table entries > In some cases table entries only apply to a subset of ports (e.g. setting VLAN id based on egress port), but all rules still are inserted into all pipelines. By selectively inserting there entries in only the relevant subset, resources can be used more efficiently. > \- Registers & counters > Since the pipelines share no state, the registers and counters in them are not in sync. While this can be mitigated for counters (totaling them up), this does not make sense for general purpose registers (e.g. calculating flow hashes). > > \### Proposed changes > > Adding an optional scope message to the \`Entity\` allows targeting on both the read and write path. The absence of a Scope message specifies the current behavior. If set on entities that do not allow or support scopes (e.g. PRE on Tofino), the server return an INVALID\_ARGUMENT error. > > \`\`\`proto > message Entity { > message Scope { > repeated uint32 scope = 1; > } > oneof entity { > ExternEntry extern\_entry = 1; > TableEntry table\_entry = 2; > ActionProfileMember action\_profile\_member = 3; > ActionProfileGroup action\_profile\_group = 4; > MeterEntry meter\_entry = 5; > DirectMeterEntry direct\_meter\_entry = 6; > CounterEntry counter\_entry = 7; > DirectCounterEntry direct\_counter\_entry = 8; > PacketReplicationEngineEntry packet\_replication\_engine\_entry = 9; > ValueSetEntry value\_set\_entry = 10; > RegisterEntry register\_entry = 11; > DigestEntry digest\_entry = 12; > } > Scope scope = 13; > } > \`\`\` > > \### Open Questions > > Should the scope also apply to the \`ForwardingPipelineConfig\` message to allow programming individual pipelines? > > How should the scope be specified? Is an array of integers enough for all types of entities (e.g. parser value sets)? > > P4Info has a \[Scope enum\](https://github.com/p4lang/p4runtime/blob/master/proto/p4/config/v1/p4info.proto#L226), should we chose a different name? > > \### Alternatives > > Externs provide a way to extend the P4RT protocol unilaterally for a platform/architecture/vendor. Since the details are then defined by the specific implementation, interoperability between SwitchOSs and control planes might be lower than with a standardized solution. > > Treating each pipe as a separate device with its own device id provides an even higher level of isolation. > Assuming the user only wants to program a small part of the tables / entities asymmetrically, this could result in extra overhead for the controller (maintaining the StreamChannel, ensuring all common entries are inserted). > > > TODOs: > > \- \[\] Update spec > > > Fixes #316 1. I do not know this aspect of P4Runtime off the top of my head well enough to answer today. Perhaps someone else might. 2. P4Runtime controllers typically use write messages to change the configuration of a device. Regarding the beginning of your latest message, I believe I may have misled you in my earlier answer, and Antonin Bas’s answer is correct. ---
**Author:** ![vainron](https://avatars.discourse-cdn.com/v4/letter/v/f4b2a3/32.png) [@vainron](https://forum.p4.org/u/vainron)\ **Post date:** [February 15, 2022, 6:10pm UTC](https://forum.p4.org/t/device-id-assignment-for-server-with-multiple-devices/240/6 "2022-02-15T18:10:04Z")
Thank you Antonin, so if the server manages the devices under it, how does the controller knows which IDs assigned to each device? How does the controller knows which type of device it is? Is there some sort of discovery mechanism like using the capabilities message? ---
**Author:** ![antonin](https://avatars.discourse-cdn.com/v4/letter/a/d07c76/32.png) [@antonin](https://forum.p4.org/u/antonin)\ **Post date:** [February 16, 2022, 7:07pm UTC](https://forum.p4.org/t/device-id-assignment-for-server-with-multiple-devices/240/7 "2022-02-16T19:07:32Z")
That depends and there is no mechanism mandated by P4Runtime. But a good example may be the Stratum NOS, which I think typically supports 2 modes: - the Stratum NOS is started on the switch and provided a “ChassisConfig” file, which includes device id and port configuration (e.g. [stratum-onos-demo/leaf4.pb.txt at master · opennetworkinglab/stratum-onos-demo · GitHub](https://github.com/opennetworkinglab/stratum-onos-demo/blob/master/demo/chassis-config/leaf4.pb.txt)). The controller can then retrieve this configuration over gNMI and use the device id obtained from it for P4Runtime. - the Stratum NOS is started on the switch without a “ChassisConfig”. The controller then needs to provide it over gNMI before any P4Runtime message can be sent. So as you can see, in the first case the device id comes from the switch (which is the case I was describing in my original message), while in the second case it comes from the controller. It _is_ possible that the “controller” assigns a device id to the switch, but _this is not done over P4Runtime_ (in Stratum they use gNMI with a Protobuf representation of Openconfig models).