How to embed Ghost Thread feature in P4 code

Hi!

Recently, I’ve been struggling to implement Bolt (NSDI ‘23). It’s quite frustrating that I cannot find any reference code except sfc_ghost.p4, sfc_trigger.p4 (under open-p4studio/pkgsrc/switch-p4-16/p4src/shared at main · p4lang/open-p4studio · GitHub ), switch_sfc_signal.py, and switch_sfc_trigger.py (under open-p4studio/pkgsrc/switch-p4-16/ptf/api at main · p4lang/open-p4studio · GitHub). I’m confused about the differences between these codes and how to embed the P4 code into my code. Could someone give me a hint?

Thanks in advance.

Dear @herano1999 ,

The difference between the files you are referencing is quite simple: .p4 files are the data plane code, and .py files are PTF tests for the corresponding feature.

In terms of integrating that code into your program, you should probably look at it as an example, no more and then implement whatever you need. The code you are referring to was written specifically for the big data plane program, called switch.p4 and makes a lot of assumptions about it, not to mention that implements certain specific functionality as was envisioned by the program designers (or requested by some Tofino customers).

I would also recommend reaching out to the paper authors.

Happy hacking,
Vladimir

Hi @p4prof ,

Thanks for your reply. After checking the tofino2_arch.p4, I think at the dataplane, I should leverage the gh_intr_md in IngressT to get the queue length at the egress. But I’m not sure about it, because there is another control block GhostT. However, this control block has no out data, so I’m wondering why this control block was designed this way.

For the previous question, I’m sorry I didn’t explain the question clearly. I want to know the difference between the two P4 codes and the two Python codes. After diving into these codes, I think the two Python codes are testing different features, and the P4 codes both implement the SFC. Though I still cannot find where the relative control blocks (such as IngressSfcPrepare, IngressSfcTrigger and so on) are called, I think it isn’t a matter if I can directly leverage the gh_intr_md.

Thanks!

Vladimir knows this much better than I do, but I might be able to convey the basic idea correctly.

The “ghost” thread runs in the ingress pipeline, and runs along side of ingress packet processing, and it has access to some ingress resources. It is not intended to have any output packet, but to cause changes to the state of ingress resources, typically by writing to one or more P4 register arrays, that the normal packet ingress processing code can then read.

So for example, the ghost thread might write queue depth information to a P4 register array, that ingress packet processing might then read and use to affect how ingress packets are processed.

1 Like

Hi, and thank you @andyfingerhut! It sounds reasonable to me. So why is there another control block GhostT. This control block only contains one input gh_intr_md, and seems to overlap with the input from Ingress. I can envision two possible implementation approaches:

  • Leverage the gh_intr_md at ingress directly.
  • Or send CPU packets by the GhostT block to modify the control plane. Read the relevant table at ingress.

Which approach is correct (or both OK/not OK)? The second approach may be somewhat complicated to implement, and I suspect it will introduce overhead.

@herano1999 ,

@andyfingerhut is correct – the ghost thread is a separate control that can be included in a T2NA switch package. It runs independently in the ingress pipeline and GhostT is the control that describes the processing that can occur there. As you can see in the definition, there is not much there, just a single input argument, meaning that you cannot send a packet or anything.

What you can do is you can instantiate a Register() extern at the top level (i.e., outside any control) and then use the Ghost control to store values there and let the Ingress control to read these values and do whatever it needs.

That’s how it works. The ghost thread can do a little bit of additional processing, but not much, really. The file you pointed to (sfc_ghost.p4) is a pretty good example, actually.

Happy hacking,
Vladimir

I think I’m starting to get it. I will try to implement. Thank you for your help @p4prof @andyfingerhut !

Cheers!