Hi, I am using the python code in exercises → p4runtime → mycontroller.py to add a table entry to the existing tables installed on switches. But, after I run the mycontroller.py, although the new rules (table entries) will be installed on the switch, the previous exiting rule will be deleted. Please let me know how I should change the mycontroller.py, so that, the previous table entries will not delete.
(I searched for a way to “add” to table entries in runtime, but I could not find any brief and concise answer.)
Hi @sorosuh,
How are you actually running the script? Do you run it once, then modify the Python script (with new table entries) and run it again?
Could this happen because each time that mycontroller.py ends, it calls the ShutdownAllSwitchConnections() (see here)? I wonder if that call has some method that removes table entries of the controller that was connected in that stream. Of course, I am assuming you are not running and closing mininet every time you test your rules.
Cheers,
Hi @ederollora,
I just run everything once. I first run the mininet. I have a fattree topology, and the routing tables of all switches are complete, except for one. Actually I intentionally deleted one table entry before running the mininet. Then I try to add that missing table entry during runtime using mycontroller.py. But, running the mycontroller().py removes all other table entries on that specific switch.
I believe that the problem happens after the SetForwardingPipelineConfig command. I commented the line you mentioned (ShutdownAllSwitchConnections), but the problem was not resolved.
1 Like
This is the expected behavior. By default, running SetForwardingPipelineConfig will “reset” the P4 pipeline and all forwarding entries are removed. You can refer to the P4Runtime Specification. This is the only sensical behavior as SetForwardingPipelineConfig lets you push an arbitrary P4 pipeline to the switch: it can be completely unrelated to the previous one and the existing forwarding entries may not apply any more (because match-action tables have changed). In your case, if you know you are pushing the same P4 pipeline, you can change the SetForwardingPipelineConfig configuration action and use RECONCILE_AND_COMMIT instead of VERIFY_AND_COMMIT. Alternatively, maybe you can skip calling SetForwardingPipelineConfig altogether when you run mycontroller.py multiple times (you only need to call it once initially to configure the switch with your P4 pipeline).
That being said, I do not know the tutorials code very well, but I would say that mycontroller.py is meant to install all the entries in one go, and is meant to be run once only.
2 Likes
It looks like what @sorosuh and @antonin said makes sense.
This was my thought yesterday when reading @sorosuh last message. If I were you @sorosuh , I would separate MasterArbitrationUpdate and SetForwardingPipelineConfig-like statements to one method or function and writeTunnelRules-like statements to another one. In this way you can run them once or multiple times without affecting the pipeline. Finally when you terminate your controller execution (pressing Control+C which can be catched with signal.signal(signal.SIGINT, handler) or a similar action) you can call ShutdownAllSwitchConnections.
As @antonin said, the tutorial program connects the controller to the switch (sets the pipeline), installs rules and disconnects all at once. So every time that you need to “install rules”, you do everything at once. That is why it makes sense to separate these actions to run once and/or multiple times depending on your interested.
Cheers,
1 Like
Thank you @antonin and @ederollora.
@antonin, I think your suggestion to use RECONCILE_AND_COMMIT instead of VERIFY_AND_COMMIT is the best option for me. Because the scenario I am developing is a datacenter, which is managed by a cloud provider, and the cloud provider needs to add/delete table entries periodically to manage a monitoring system. However, unfortunately, I get a ‘socket closed’ error, when I put RECONCILE_AND_COMMIT instead of VERIFY_AND_COMMIT.
Now, I understand that when I call the MakeFile at first, the SetForwardingPipelineConfig is called to install table entries that I specified in s#-runtime.json files. Then, in runtime, when I call the mycontroller.py file to add table entries, either I have to add the whole table entries again, or I should use the RECONCILE_AND_COMMIT which did not work for me.
@ederollora, I think I did not understand your suggestion correctly. But, if you mean I should call the SetForwardingPipelineConfig command only once; as I described above, it is called once at the beginning, and when I want to run my controller (it could be mycontroller.py or any other controller I can use to execute my monitoring policies!), I have to call it again. Of course, there is a solution for me: running my controller, adding all table entries, and not terminating the controller, and adding/deleting table entries whenever I want, just using a loop to call writeTunnelRules.
But at this point, I am curious why @antonin’s suggestion (using RECONCILE_AND_COMMIT) did not work for me.
To be honest, a solution that uses RECONCILE_AND_COMMIT seems beneficial for a use case is running a P4-programmable device and needs to install a new P4 program (this is only an opinion from reading the definition). If the device cannot be replaced in the pipeline upgrade or the traffic diverted to another device, the aforementioned solution would work because you can probably keep the forwarding state and include new tables and logic. See from P4runtime spec:
RECONCILE_AND_COMMIT: verifies, saves and realizes the given config, while preserving the forwarding state in the target. This is an advanced use case to enable changes to the P4 forwarding pipeline configuration with minimal traffic loss. (…)
An SDN controller or a general control plane is, in principle (and naively summarized), a never-ending process that reacts to PACKET_IN events, northbound requests, and other types of events. If your controller starts and ends right away, then you have a problem (if you want to install other rules). The mycontroller.py code is easy to exemplify a controller, but it is probably not the best way to make it work as a reactive controller. For instance, a switch sends a packet to the controller (because it does not know what to do with it or because it is interesting to be processed by the control plane), the controller processes it and installs rules and/or sends a PACKET_OUT. In your case, as you mentioned, there is a solution for you and that is the one I meant. You should only run once the pipeline setting (SetForwardingPipelineConfig) and wait for events like a PACKET_IN (an example here for the P4 code and controller code), or a timed event, or keyboard input. When you are done, terminate the command and make it ShutdownAllSwitchConnections.
Cheers,
2 Likes
Just wanted to explicitly +1 what @ederollora explained above.
An SDN controller or a general control plane is, in principle (and naively summarized), a never-ending process
The tutorials repo is just that, some place where you can get started and get familiar with the tools. But as soon as you get into more advanced use cases, sticking to the tutorials code and the Makefile-driven steps there is a mistake. You should make sure you know where to look for the bmv2 / P4Runtime logs if you want to be able to troubleshoot anything.
Your controller should ideally be a long-running process and maintain a “connection” (in P4Runtime terms, it is an arbitration stream) to the P4Runtime servers / switches it is controlling. It can in turn expose an application-aware interface to you which you can use to send directives to the controller, which in turn will send the appropriate P4Runtime messages to the switch.
If you do not want to write a full-fledged controller because you feel you didn’t need it, you can look at GitHub - p4lang/p4runtime-shell: An interactive Python shell for P4Runtime and see if it works for you. the first time you call it, you would need to use the --config flag and provide a P4 pipeline config: the shell will the call SetForwardingPipelineConfig for you. After that, you would invoke the shell without the --config flag. The shell can also be used programmatically. Note that in the end, the shell is just a wrapper around a P4Runtime client.
1 Like