The unit of ingress/egress_global_timestamp

Hello everyone,

I’m new to P4 and I would greatly appreciate any assistance. I’m trying to calculate the time a packet spends in both ingress processing and egress processing in BMv2 (v1model). To achieve this, I subtract the value of ingress_global_timestamp from egress_global_timestamp in the egress processing stage. Here’s the relevant code:
hdr.myPayload.time = (bit<64>)standard_metadata.egress_global_timestamp - (bit<64>)standard_metadata.ingress_global_timestamp

I have a couple of questions regarding this:

  1. What is the unit of measurement for the two timestamps mentioned above?
    According to the documentation (/docs/simple+switch.md), the unit for both timestamps should be microseconds. I also checked the simple_switch.h and simple_switch.cpp files, where ts_res is set to std::chrono::microseconds. However, in my experiments, the value I obtain from hdr.myPayload.time is around 60000, while my round-trip time (RTT) is approximately 0.9 milliseconds. Additionally, in p4src/includes/intrinsic.p4, the unit for ingress_global_timestamp is specified as nanoseconds. This inconsistency has left me confused.
  2. Will the parallel nature of packet processing in P4 switches affect the accuracy of this recorded time?

Sincerely,
Logan

You mention a file p4src/includes/intrinsic.p4 that says the units is in nanoseconds. Is this file in some public repository somewhere? Which one? If it is in some switch-ASIC-specific file somewhere, that would not surprise me, e.g. the Tofino ASIC implements units of nanoseconds for similar fields.

Regarding your question “Will the parallel nature of packet processing in P4 switches affect the accuracy of this recorded time?” Unless there is a pretty bad bug in the implementation you are using, I would be very surprised if this is the case. Providing a readable time value in the data plane is a fairly straightforward thing to do.

Warning: In the simple_switch_grpc implementation, the type of ingress/egress_global_timestamp is bit<48>. You are casting these values to bit<64> and then subtracting. It would take about 8.9 years for these bit<48> values to wrap around, but when/if they do, your subtraction will give the wrong result. I would strongly recommend removing the bit<64> casts and keeping the results in type bit<48>. They might not wrap around in a short enough time on simple_switch_grpc for you to ever observe, but in packet processing devices that uses nanoseconds or similar smaller time units for timestamps, they typically wrap around much more often, e.g. every few minutes, and you would see latency calculation bugs quite often.

I would enable logging in simple_switch_grpc using one of these sets of command line options when you start it:

  • --log-console --dump-packet-data 10000 if you want the logging messages to go to the console
  • --log-file ss-log --log-flush --dump-packet-data 10000 if you want logging messages to be written to a file ss-log.txt
    You can use the log_msg intrinsic function if you want to cause extra messages to be added to the packet processing log, if the information you would like to see is not already there: p4c/v1model.p4 at main · p4lang/p4c · GitHub

Print out the value of ingress_global_timestamp and egress_global_timestamp before performing the subtraction, and print the result of the subtraction.

In simple_switch_grpc, both of those values should be in units of microseconds. Those values should be 0 at the time you start simple_switch_grpc, and increment once per microsecond after that.

Possible problems I would guess you might be encountering:

The value of ingress_global_timestamp might be 0 if you attempt to read its value in the egress control (I have not verified this, but it seems possible that the implementation only supports a correct value of ingress_global_timestamp in ingress processing). If that is the case, I would recommend declaring a user metadata field with the same type as ingress_global_timestamp, and copying the value of ingress_global_timestamp to that field during ingress processing. Then use that user metadata field, not ingress_global_timestamp, when performing the subtraction during egress processing.

Hi Andy,

I greatly appreciate your prompt response. You are absolutely right that the p4src/includes/intrinsic.p4 file can be found in the p4lang/switch repository.

  1. I am currently using the simple_switch and have chosen to disable logging to improve performance. I would like to know if it is feasible for me to store the calculated value in the packet and retrieve it when the packet is received.

  2. Fortunately, I haven’t encountered the issue of ingress_global_timestamp being 0 when attempting to read its value in the egress control. However, to eliminate the possibility of this bug, I plan to modify my code. I’m curious, instead of declaring a user metadata field, is it possible to store the value of ingress_global_timestamp somewhere in my packet header?

Thank you for your valuable input.

Regarding question #2: You certainly could choose to preserve the value of ingress_global_timestamp with the packet by copying it into the packet contents somewhere. Of course you should be careful not to overwrite some value that you want to keep.

Regarding question #1: What you are doing definitely seems feasible. You can enable logging temporarily just to debug your program more easily, then disable it again when you do not need it, if you are concerned about performance. The value of 60,000 microseconds = 60 millisec definitely seems inconsistent with a measured RTT o 0.9 milliseconds, and I do not know why you are observing that, but extra debugging information can often help narrow down the cause of such problems.

I will enable the logging to see if I could find the reason and keep you updated if anything valuable found. Thank you so much for you time!