Hi,
Has anyone been able to run BMV2 simple_switch on standalone linux based switchboxes or servers? I am able to run simple_switch in a VM and am able to ensure connectivity between two VM endhosts in Virtual box.
I am using simple_switch -i 1@if1 -i 2@if2 basic.json command to map physical interfaces to the bmv2 switch. But so far I have not been able to connect two physical end hosts with the bmv2 switch in the baremetal switchbox. Is there something I am doing wrong?
Basically, my P4 program is similar to the basic forwarding program from the tutorial. The same program works with BMV2 in a VM with two VM end-hosts. However, when replicated in a real-world scenario with linux based machines, this does not work.
For example, the end-hosts don’t respond to ICMP request even when packets correctly reach the interface. I have confirmed this with wireshark and I’m not really sure what I might be doing wrong.
Hi,
Can you please let me know more about the topology you are trying to test ?
As an additional test could you try running the simple_switch command with -i 1@eth1 -i 2@eth2 ? I think this may be part of your issue the simple_switch process is attempting to mount the interfaces named if1 and if2 .
Hi,
I have met with a similar problem in the past and was able to use BMv2 as a software switch between physical hosts. In addition, I have also used BMv2 as the software switch for multiple KVMs and Docker containers to build a simple testbed back then.
You can check out this email thread on the p4-dev mailing list.
https://lists.p4.org/empathy/thread/YD2XZ7VZR2UMLN6TO7TESFNJN4XHEW7P?hash=6AKBAGB4WBVVRZILY46UPM4ARQMHI7Z4#6AKBAGB4WBVVRZILY46UPM4ARQMHI7Z4
To bind interfaces, you probably need to do it with superuser privileges.
Then, after binding the interfaces, make sure to disable certain NIC offloads such as RX, TX using ethtool for the interfaces bound with BMv2.
my topology is very simple:
[host1] <—> [p4-switch] <—> [host2]
if1 and if2 are generic names I have given for the interfaces to explain my issue in this post. I use the original interface names when I execute the command.
@khooixz , thank you very much. I am pretty sure I am using superuser privileges when the command is executed. I will try disabling NIC offloads.
One more thing to add on, I think I gave a wrong suggestion.
Instead of disabling the offloads on the interfaces bound to BMv2, you should disable the ones on the end hosts.
My bad, I hope this info helps.
@khooixz, thank you very much! Disabling the offloads on the physical end host interfaces did the trick. I have tried every trick in my disposal such as using linux bridges, separate namespaces and even virtual interfaces which did not work. But this helps a lot for me to work with. I was hesitant to use P4 to DPDK (T4P4S) based compilers for running my P4 application, since I am a noob with DPDK and am not well versed with its usage. This saves a lot of my time! 
In case the link does not work in the future, I am quoting the solution here:
Hi,
Sorry if this is something you already did.
But did you try to spawn 2 simple_switch on the same VMs ?If really network isolation is needed you can create Linux namespace in order to separate them.
As a side note you might be hit by the bug Andy put in evidence ?
https://github.com/jafingerhut/p4-guide/tree/master/linux-veth-bug
If not its s also a good idea to switch off linux kernel TCP offload (in your end host [VM or Physical device]) :
As superuser you can run: ($1 your interface name)
ifconfig $1 multicast allmulti promisc mtu 1500 up
ethtool -K $1 rx off
ethtool -K $1 tx off
ethtool -K $1 sg off
ethtool -K $1 tso off
ethtool -K $1 ufo off
ethtool -K $1 gso off
ethtool -K $1 gro off
ethtool -K $1 lro off
ethtool -K $1 rxvlan off
ethtool -K $1 txvlan off
ethtool -K $1 ntuple off
ethtool -K $1 rxhash off
ethtool --set-eee $1 eee off