I am trying to understand the SRAM block usage and size allocation behavior of Match-Action Tables (MATs) on Intel Tofino2.In my experiments, I observed the following:
For a MAT with only 1 entry:
When Key Width = 16 bits or 32 bits, the compiler reports 4 SRAM blocks used, and Size = 4096.
When Key Width = 1 bit, 4 bits, or 8 bits, the compiler reports 1 SRAM block used, and Size = 1024.
Questions：
1.**How can this phenomenon be explained?**Why does the table consume 4 SRAM blocks and a larger “Size” (4096) when the key width increases, even though the table only has 1 entry?
2.What does the reported “Size” mean exactly?
3.What is the maximum number of Match-Action Tables that can be placed in a single stage on Tofino2?
What you are seeing is a result of (a) specifics of the Tofino2 hardware and the heuristics applied by the compiler during the table allocation.
Here are some basic facts that explain what you see.
The crossbars pull the data that form the lookup keys in whole bytes and not individual bits. As a result, there is a good chance that if you use a 1-bit, 4-bit or 8-bit-wide key for the table, it will occupy the same amount of space, especially in a simple test case like yours.
Note, however, that this might not always be the case. For example, if the compiler splits a field across multiple individual bytes the amount of memory required to store a key might grow significantly.
Same thing often happens if you use multiple unrelated fields as a key. Even though the total key width might still be equal to 8 bits, the actual key sie might be as high as 8x8=64 bits.
The compiler does try hard to ensure that the effective key size is as small as possible
Each SRAM block on Tofino2 contains 1024 128-bit-wide words. As a result the minimum number of entries an exact-match table can have on Tofino is 1024. If you request less than that, the compiler still allocates 1024 and then the driver artificially limits the number of entries one can add.
(2) and (3) should explain your first observation. Note, that even though you requested only one entry, we still must allocate at least 1024 entries, because nobody knows into which of the actual physical words the requested entry will need to be placed. It all depends on the calculated hash value and assuming 10-bit-wide hash, it could be anywhere between words 0 and 1023. Because you specified 16 or 32-bit key, the compiler decided to allocate 4-way hash table. Granted, it is not necessary of 1 entry (since there can be no hash collisions), but that’s the default. It can be changed using @ways() annotation. But that’s why you see 4K entries (4 ways, 1K entries each)
(1), (2) and (4) should answer your second observation. Because the number of entries (1024) is greater than 2^W, one way is enough and so you get 1024 entries in 1 SRAM block.
As for your further questions:
Because in the hash tables the key needs to be stored alongside the data, increasing the width of the key might result in more memory required to store the table (in addition to the facts (3) and (4) I discussed above)
Reported size shows the actual number of entries in the allocated table. As you already noticed, it might be higher than what you requested.
The maximum number of match-action tables that can be placed in one stage of Tofino2 is 16 (assuming you have the resources and do not exceed other limits).
If you want to further explore the subject, I recommend you look at the classes ICA-XFG-101, ICA-1141 and ICA-1142 available from P4ica.
Thank you so much, Vladimir!
Your explanation was extremely clear and answered all my questions in detail.
I really appreciate your time and the insights you shared!
