Need help with digest p4runtime

Getting Started with P4
1

Hello,
I want to send digest message from bmv2 to controller through p4runtime, but I can’t make it work.
I tried different environments, but none of them will work. Maybe I missed something.

To my understanding, I have to send WriteRequest to insert a DigestEntry, so the switch will start to send digest messages to controller through StreamChannel.
With stratum_bmv2, I can insert the entry, but I didn’t receive any digest message. I can still get PacketIn messages meanwhile.
With simple_switch_grpc, I can’t insert the entry. I received a helpless message as below

Traceback (most recent call last):                                                                                                                              
  File "./p4runtime_test.py", line 152, in <module>                                                                                                             
    main2(stub)                                                                                                                                                 
  File "./p4runtime_test.py", line 141, in main2                                                                                                                
    insert_digest(stub, 401776493)                                                                                                                              
  File "./p4runtime_test.py", line 117, in insert_digest                                                                                                        
    response = stub.Write(req)                                                                                                                                  
  File "/home/doraeric/venv/lib/python3.8/site-packages/grpc/_channel.py", line 946, in __call__                                                                
    return _end_unary_response_blocking(state, call, False, None)                                                                                               
  File "/home/doraeric/venv/lib/python3.8/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking                                            
    raise _InactiveRpcError(state)                                                                                                                              
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:                                                                                
        status = StatusCode.UNKNOWN                                                                                                                             
        details = ""                                                                                                                                            
        debug_error_string = "{"created":"@1654597469.924974618","description":"Error received from peer ipv6:[::1]:50051","file":"src/core/lib/surface/call.cc"
,"file_line":952,"grpc_message":"","grpc_status":2}"

The server didn’t response any details about the error. I can receive PacketIn in the setup, too.

I tried both using p4runtime-shell and writing grpc client by myself, neither of them have received digest messages.

The P4 program with digest call can be compiled and upload to switch through SetForwardingPipelineConfig without error, but it didn’t send any digest message.

How can I send digest messages?
Did I miss some steps?
Is there any working example or tool I can try?

This is the python I used to insert digest entry:

def insert_digest(stub, digest_id):
    req = p4runtime_pb2.WriteRequest()
    req.device_id = device_id
    req.election_id.high = 0
    req.election_id.low = 1
    req.role_id = 0
    update = req.updates.add()
    update.type = p4runtime_pb2.Update.INSERT
    digest_entry = update.entity.digest_entry
    digest_entry.digest_id = digest_id
    digest_entry.config.max_timeout_ns = 0
    response = stub.Write(req)
    print('digest added')

Here are some environment information. I truncated some less important output.

# stratum
$ stratum_bmv2 --version
stratum_bmv2 version e00eac942a877e22e3927ef8ec0818685622039d
$ ps aux | grep '[s]tratum_bmv2'
root          73  1.6  0.2 1089148 53272 ?       Ssl  09:37   2:41 stratum_bmv2 -device_id=1 -chassis_config_file=/tmp/s1/chassis-config.txt -forwarding_pipeline_configs_file=/tmp/s1/pipe.txt -persistent_config_dir=/tmp/s1 -initial_pipeline=/root/dummy.json -cpu_port=255 -external_stratum_urls=0.0.0.0:50001 -local_stratum_url=localhost:55361 -max_num_controllers_per_node=10 -write_req_log_file=/tmp/s1/write-reqs.txt -bmv2_log_level=warn

# simple_switch_grpc
$ simple_switch_grpc --version
1.14.0-2de095c7
$ ps aux | grep '[s]imple_switch'
root       57014  0.7  1.1 1267316 45252 pts/7   Sl+  16:48   1:34 simple_switch_grpc -i 1@s1-eth1 -i 2@s1-eth2 -i 3@s1-eth3 -i 4@s1-eth4 --pcap /home/doraeric/tutorials/exercises/basic/pcaps --nanolog ipc:///tmp/bm-0-log.ipc --device-id 0 build/basic.json --log-console --thrift-port 9090 -- --grpc-server-addr 0.0.0.0:50051 --cpu-port 255
2

I found a working example finally.

I think the problem is at the P4 source code.
The following code does NOT work on simple_switch_grpc.

digest(1, meta.test_digest);
digest(1, hdr.ethernet.srcAddr);

The code is found from here and p4c testdata.

Here is a working example from the first link:

struct mac_learn_digest_t {
    EthernetAddress srcAddr;
    PortId_t        ingress_port;
}

digest<mac_learn_digest_t>(1, {hdr.ether.srcAddr, st_md.ingress_port});

I also set the digest list size to 1 as he does.

digest_entry.config.max_list_size = 1

With the original not working p4 code, there is no warning or error when compiling and calling SetForwardingPipelineConfig.
The p4runtime server by simple_switch_grpc will only give you an unknown status code when you try to enable digest by inserting a digest entry. After updating p4 code, it works now.
stratum_bmv2 doesn’t seem to support digest function. There is a closed issue on github without clarifying its functionality. And I can’t make it work even with fixed p4 code.

3

If you are able to publish your working example, and also the smallest modification you can find to it related to the digest call that makes it no longer work (e.g. make it “one step closer” to the non-working examples you found), and also publish that, someone else may be able to reproduce your situation more easily, and perhaps determine if there is a bug in the compiler related to this. Would you be willing to do so?

1 Like
4

Here is my code. I tried to make it as minimal as I can.

P4 code. Check line 46 and 49 and choose one to use.
The argument for second digest is not a struct. Maybe that’s the reason it failed.

// main.p4
#include <core.p4>
#include <v1model.p4>

header ethernet_t {
    bit<48> dst_addr;
    bit<48> src_addr;
    bit<16> ether_type;
}
struct headers_t {
    ethernet_t ethernet;
}

struct mac_learn_digest_t {
    bit<48> src_addr;
    bit<9>  ingress_port;
}
struct local_metadata_t { }

parser parser_impl(
        packet_in pkt,
        out headers_t hdr,
        inout local_metadata_t user_md,
        inout standard_metadata_t st_md) {
    state start { transition parse_ethernet; }
    state parse_ethernet {
        pkt.extract(hdr.ethernet);
        transition accept;
    }
}

control deparser(
        packet_out pkt,
        in headers_t hdr) {
    apply {
        pkt.emit(hdr.ethernet);
    }
}

control ingress(
        inout headers_t hdr,
        inout local_metadata_t user_md,
        inout standard_metadata_t st_md) {
    apply {
        // passed
        digest<mac_learn_digest_t>(1, {hdr.ethernet.src_addr, st_md.ingress_port});

        // failed
        // digest(1, hdr.ethernet.src_addr);
    }
}
control egress(
        inout headers_t hdr,
        inout local_metadata_t user_md,
        inout standard_metadata_t st_md) {
    apply { }
}
control no_verify_checksum(
        inout headers_t hdr,
        inout local_metadata_t user_md) {
    apply { }
}
control no_compute_checksum(
        inout headers_t hdr,
        inout local_metadata_t user_md) {
    apply { }
}
V1Switch(parser_impl(),
         no_verify_checksum(),
         ingress(),
         egress(),
         no_compute_checksum(),
         deparser()
) main;

python controller. Check which digest_id to use before running.

#!/usr/bin/env python3
"""
mininet >
h1 route add default gw 10.0.0.254 dev h1-eth0
h1 arp -i h1-eth0 -s 10.0.0.254 00:00:0a:00:00:fe
"""
import logging
import os
import queue
import sys
import threading
import traceback

import google.protobuf.text_format
from google.rpc import status_pb2, code_pb2
import grpc
from p4.v1 import p4runtime_pb2
from p4.v1 import p4runtime_pb2_grpc

# parameters
device_id = 1
P4INFO = os.getenv('P4INFO', 'build/p4info.txt')
P4BIN = os.getenv('P4BIN', 'build/main.json')
passed_digest_id = 402184575
failed_digest_id = 401776493
digest_id = passed_digest_id

logging.basicConfig(
        format='%(asctime)s.%(msecs)03d: %(process)d: %(levelname).1s/%(name)s: %(filename)s:%(lineno)d: %(message)s',
        datefmt='%Y-%m-%d %H:%M:%S',
        level=logging.INFO)
send_queue = queue.Queue()
recv_queue = queue.Queue()

def gen_handshake(election_id):
    req = p4runtime_pb2.StreamMessageRequest()
    arbitration = req.arbitration
    arbitration.device_id = device_id
    eid = arbitration.election_id
    eid.high = election_id[0]
    eid.low = election_id[1]
    return req

def check_handshake():
    rep = recv_queue.get(timeout=2)
    if rep is None:
        logging.critical("Failed to establish session with server")
        sys.exit(1)
    is_primary = (rep.arbitration.status.code == code_pb2.OK)
    logging.debug("Session established, client is '%s'", 'primary' if is_primary else 'backup')
    if not is_primary:
        logging.info("You are not the primary client, you only have read access to the server")
    else:
        logging.info('You are primary')

def stream(stub):
    def recv_handler(responses):
        for response in responses:
            logging.info('Receive response')
            logging.info(response)
            recv_queue.put(response)
    responses = stub.StreamChannel(iter(send_queue.get, None))
    logging.info('created channel')
    recv_thread = threading.Thread(target=recv_handler, args=(responses,))
    recv_thread.start()
    send_queue.put(gen_handshake(election_id=(0, 1)))
    check_handshake()
    logging.info('handshaked')
    return recv_thread

def insert_digest(stub, digest_id):
    req = p4runtime_pb2.WriteRequest()
    req.device_id = device_id
    req.election_id.high = 0
    req.election_id.low = 1
    req.role_id = 0
    update = req.updates.add()
    update.type = p4runtime_pb2.Update.INSERT
    digest_entry = update.entity.digest_entry
    digest_entry.digest_id = digest_id
    digest_entry.config.max_timeout_ns = 0
    digest_entry.config.max_list_size = 1
    digest_entry.config.ack_timeout_ns = 0
    response = stub.Write(req)

def set_fwd_pipe_config(stub, p4info_path, bin_path):
    req = p4runtime_pb2.SetForwardingPipelineConfigRequest()
    req.device_id = device_id
    election_id = req.election_id
    election_id.high = 0
    election_id.low = 1
    req.action = p4runtime_pb2.SetForwardingPipelineConfigRequest.VERIFY_AND_COMMIT
    with open(p4info_path, 'r') as f1:
        with open(bin_path, 'rb') as f2:
            try:
                google.protobuf.text_format.Merge(f1.read(), req.config.p4info)
            except google.protobuf.text_format.ParseError:
                logging.error("Error when parsing P4Info")
                raise
            req.config.p4_device_config = f2.read()
    return stub.SetForwardingPipelineConfig(req)

def client_main(stub):
    logging.info('SetForwardingPipelineConfig...')
    set_fwd_pipe_config(stub, P4INFO, P4BIN)
    logging.info('SetForwardingPipelineConfig passed')
    logging.info('insert_digest...')
    insert_digest(stub, digest_id)
    logging.info('insert_digest passed')

with grpc.insecure_channel('localhost:50001') as channel:
    stub = p4runtime_pb2_grpc.P4RuntimeStub(channel)
    recv_t = stream(stub)
    try:
        client_main(stub)
        while True:
            cmd = input('> ')
            if cmd.lower() == 'exit': break
            if cmd.lower() == 'quit': break
    except (KeyboardInterrupt, EOFError):
        pass
    except:
        traceback.print_exc()
    send_queue.put(None)
    recv_t.join()

Here are the steps to reproduce:

  1. Start bmv2 switch from docker, so the environment is consistent for everyone.
    There are two types of switches as I mentioned earlier: stratum_bmv2 and simple_switch_grpc. Choose the one you want to use.
  • stratum_bmv2: docker run --privileged --rm -it --net=host opennetworking/mn-stratum --topo single,1
  • simple_switch_grpc: docker run --privileged --rm -it --net=host opennetworking/p4mn --topo single,1
  1. Make mininet host usable. Type in the following commands in mininet shell.
h1 route add default gw 10.0.0.254 dev h1-eth0
h1 arp -i h1-eth0 -s 10.0.0.254 00:00:0a:00:00:fe
  1. Compile p4. Check ingress to use the digest you want.
p4c --std p4-16 -b bmv2 --p4runtime-files build/p4info.txt -o build src/main.p4
  1. Start python controller. Save the python code to a file. Check line 26 to change digest_id before running.

Results

With passed_digest and simple_switch_grpc, everything works.

You can verify by entering the command h1 ping 1.1.1.1 in mininet shell and seeing the digest messages in python controller.

With passed_digest and stratum_bmv2, there is no error message. No digest message is received on controller while there should be messages.

With failed_digest and any bmv2, there is only a warning message complaining about name:

src/main.p4(49): [--Wwarn=mismatch] warning: Cannot find a good name for digest method call, using auto-generated name 'digest_0'
        digest(1, hdr.ethernet.src_addr);
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

There is no other error message during compiling or setting forwarding pipeline. It only gives an error when inserting digest entry:

grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.UNKNOWN
        details = ""
        debug_error_string = "{"created":"@1655044976.054394098","description":"Error received from peer ipv6:[::1]:50001","file":"src/core/lib/surface/call.cc","file_line":1074,"grpc_message":"","grpc_status":2}"
>

If the second digest is not a valid p4 statement, I think it should give me error messages during compiling or during setting forwarding pipeline.

1 Like