Device Tree
Using the Device Tree (DT), we describe the contents of the firmware for the utility software: base addresses, versions and features of the individual components. To a certain extent, the Device Tree communicates to the user basic information about the HW platform, firmware version, etc. The Device Tree structure will be closely linked to a specific firmware - integrated inside.
DT integration in build system
There are scripts in the translation system that go through components using Modules.tcl. The top-level DevTree.tcl script (in ofm/build) first inserts general information in the form of DT-properties into the DTS framework (build time, current repository revision, author of the firmware build, etc.). Then, if it exists, it calls a function called dts_build_project, and it calls the dts_build_netcope function, which should be included in each base of the card project (typically a top-level directory with fpga_common.vhd). Here, the card project already ensures the insertion of specific information (eg card type) itself, including the instancing of its subcomponents (typically dts_boot_controller, dts_dma_module, dts_application…) and passing generics to these functions.
After running make (for compiling the firmware of the selected card) a file DevTree.dts (generated by passing TCL scripts DevTree.tcl) and VHDL package DevTree.vhd are created, which contains std_logic_vector DTB_DATA (which is a binary representation of compiled (dtc) and compressed (xz) input DevTree.dts file). This package is used by the PCI_EXT_CAP component, see the next chapter.
Because constants in packages cannot be accessed directly in TCL, the user_const.vhd package files are generated by the TCL script user_const.tcl. This is the only way to ensure consistency in top-level generations and Device Tree descriptions. Translation scripts need the dtc compiler for their operation.
Location of DTB in the firmware
The DT blob is located in the PCI configuration space. A custom extension (PCIe VSEC - Vendor-Specific Extended Capability) has been created. Because the configuration space is relatively small, there are only a few control registers that make the entire DTB stored in BRAM accessible. In this way, the DTB does not load the MI bus and cannot be easily or accidentally removed from the design. You can use the dtc tool to read the complete Device Tree from the FPGA card:
# dtc -I dtb /dev/nfb0
Example of DTS of one component
/*
* ref_name: instance name, typically populated by the parent module when needed
* get reference to the installed component (can be empty)
* my_comp: component name, populates the dts_my_comp function
* reg: component address space
* - the first value indicates the base address on the MI,
* - the second value indicates the internal space size used
* compatible: a driver submodule or software tool is bound to this string
* There must be a node specification for a specific compatible string:
* - properties and subnode names
* - their types and connection to HW functionality
* - mandatory or optional items
* no SW tool works directly with node without compatible property
* version: version of the component in terms of SW interface (register functions)
* type: additional / optional property
* - can specify large changes in the component
* - possibly extends the property compatible
* others: other properties / subnodes that the software can use
*/
ref_name: my_comp {
reg = <$BASE_ADDRESS 0x40>;
compatible = "cesnet,my_comp";
version = <0x00010004>;
type = "reduced";
};
Example of generated DTS for FPGA card
/dts-v1/;
/ {
firmware {
build-tool = "Quartus Version 22.4.0 Build 94 12/07/2022 SC Pro Edition";
build-author = "no-reply@liberouter.org";
build-revision = "95415f0";
build-time = <0x65c33529>;
card-name = "N6010";
project-name = "NDK_MINIMAL";
project-variant = "100G2";
project-version = "0.5.8";
mi0: mi_bus0 {
#address-cells = <0x01>;
#size-cells = <0x01>;
compatible = "netcope,bus,mi";
resource = "PCI0,BAR0";
width = <0x20>;
boot: ofs_pmci {
compatible = "cesnet,pmci";
version = <0x01>;
reg = <0x2000 0x1000>;
};
mi_test_space {
compatible = "cesnet,ofm,mi_test_space";
reg = <0x00 0x100>;
};
tsu: tsu {
compatible = "netcope,tsu";
reg = <0x4000 0x1000>;
type = <0x01>;
version = <0x01>;
};
dma_module@0x01000000 {
#address-cells = <0x01>;
#size-cells = <0x01>;
dma_params_rx0: dma_params_rx0 {
frame_size_max = <0x3fff>;
frame_size_min = <0x3c>;
phandle = <0x01>;
};
dma_params_tx0: dma_params_tx0 {
frame_size_max = <0x3fff>;
frame_size_min = <0x3c>;
phandle = <0x02>;
};
dma_ctrl_ndp_rx0 {
compatible = "netcope,dma_ctrl_ndp_rx";
reg = <0x1000000 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x01>;
};
dma_ctrl_ndp_rx1 {
compatible = "netcope,dma_ctrl_ndp_rx";
reg = <0x1000080 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x01>;
};
dma_ctrl_ndp_rx2 {
compatible = "netcope,dma_ctrl_ndp_rx";
reg = <0x1000100 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x01>;
};
dma_ctrl_ndp_rx3 {
compatible = "netcope,dma_ctrl_ndp_rx";
reg = <0x1000180 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x01>;
};
dma_ctrl_ndp_rx4 {
compatible = "netcope,dma_ctrl_ndp_rx";
reg = <0x1000200 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x01>;
};
dma_ctrl_ndp_rx5 {
compatible = "netcope,dma_ctrl_ndp_rx";
reg = <0x1000280 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x01>;
};
dma_ctrl_ndp_rx6 {
compatible = "netcope,dma_ctrl_ndp_rx";
reg = <0x1000300 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x01>;
};
dma_ctrl_ndp_rx7 {
compatible = "netcope,dma_ctrl_ndp_rx";
reg = <0x1000380 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x01>;
};
dma_ctrl_ndp_rx8 {
compatible = "netcope,dma_ctrl_ndp_rx";
reg = <0x1000400 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x01>;
};
dma_ctrl_ndp_rx9 {
compatible = "netcope,dma_ctrl_ndp_rx";
reg = <0x1000480 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x01>;
};
dma_ctrl_ndp_rx10 {
compatible = "netcope,dma_ctrl_ndp_rx";
reg = <0x1000500 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x01>;
};
dma_ctrl_ndp_rx11 {
compatible = "netcope,dma_ctrl_ndp_rx";
reg = <0x1000580 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x01>;
};
dma_ctrl_ndp_rx12 {
compatible = "netcope,dma_ctrl_ndp_rx";
reg = <0x1000600 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x01>;
};
dma_ctrl_ndp_rx13 {
compatible = "netcope,dma_ctrl_ndp_rx";
reg = <0x1000680 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x01>;
};
dma_ctrl_ndp_rx14 {
compatible = "netcope,dma_ctrl_ndp_rx";
reg = <0x1000700 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x01>;
};
dma_ctrl_ndp_rx15 {
compatible = "netcope,dma_ctrl_ndp_rx";
reg = <0x1000780 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x01>;
};
dma_ctrl_ndp_tx0 {
compatible = "netcope,dma_ctrl_ndp_tx";
reg = <0x1200000 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x02>;
};
dma_ctrl_ndp_tx1 {
compatible = "netcope,dma_ctrl_ndp_tx";
reg = <0x1200080 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x02>;
};
dma_ctrl_ndp_tx2 {
compatible = "netcope,dma_ctrl_ndp_tx";
reg = <0x1200100 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x02>;
};
dma_ctrl_ndp_tx3 {
compatible = "netcope,dma_ctrl_ndp_tx";
reg = <0x1200180 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x02>;
};
dma_ctrl_ndp_tx4 {
compatible = "netcope,dma_ctrl_ndp_tx";
reg = <0x1200200 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x02>;
};
dma_ctrl_ndp_tx5 {
compatible = "netcope,dma_ctrl_ndp_tx";
reg = <0x1200280 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x02>;
};
dma_ctrl_ndp_tx6 {
compatible = "netcope,dma_ctrl_ndp_tx";
reg = <0x1200300 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x02>;
};
dma_ctrl_ndp_tx7 {
compatible = "netcope,dma_ctrl_ndp_tx";
reg = <0x1200380 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x02>;
};
dma_ctrl_ndp_tx8 {
compatible = "netcope,dma_ctrl_ndp_tx";
reg = <0x1200400 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x02>;
};
dma_ctrl_ndp_tx9 {
compatible = "netcope,dma_ctrl_ndp_tx";
reg = <0x1200480 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x02>;
};
dma_ctrl_ndp_tx10 {
compatible = "netcope,dma_ctrl_ndp_tx";
reg = <0x1200500 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x02>;
};
dma_ctrl_ndp_tx11 {
compatible = "netcope,dma_ctrl_ndp_tx";
reg = <0x1200580 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x02>;
};
dma_ctrl_ndp_tx12 {
compatible = "netcope,dma_ctrl_ndp_tx";
reg = <0x1200600 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x02>;
};
dma_ctrl_ndp_tx13 {
compatible = "netcope,dma_ctrl_ndp_tx";
reg = <0x1200680 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x02>;
};
dma_ctrl_ndp_tx14 {
compatible = "netcope,dma_ctrl_ndp_tx";
reg = <0x1200700 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x02>;
};
dma_ctrl_ndp_tx15 {
compatible = "netcope,dma_ctrl_ndp_tx";
reg = <0x1200780 0x80>;
version = <0x20000>;
pcie = <0x00>;
params = <0x02>;
};
};
i2c0: i2c0 {
compatible = "netcope,i2c";
reg = <0x3010 0x08>;
phandle = <0x04>;
};
pmdctrl0: pmdctrl0 {
reg = <0x301c 0x04>;
version = <0x10000>;
phandle = <0x03>;
};
pmd0: pmd0 {
compatible = "netcope,transceiver";
type = "QSFP";
status-reg = <0x03>;
control = <0x04>;
phandle = <0x08>;
control-param {
i2c-addr = <0xa0>;
};
};
i2c1: i2c1 {
compatible = "netcope,i2c";
reg = <0x3110 0x08>;
phandle = <0x06>;
};
pmdctrl1: pmdctrl1 {
reg = <0x311c 0x04>;
version = <0x10000>;
phandle = <0x05>;
};
pmd1: pmd1 {
compatible = "netcope,transceiver";
type = "QSFP";
status-reg = <0x05>;
control = <0x06>;
phandle = <0x0d>;
control-param {
i2c-addr = <0xa0>;
};
};
regarr0: regarr0 {
compatible = "netcope,pcsregs";
reg = <0x800000 0x40000>;
phandle = <0x07>;
};
pcspma0: pcspma0 {
type = "100G";
control = <0x07>;
phandle = <0x09>;
control-param {
ip-name = "E_TILE";
};
};
txmac0: txmac0 {
compatible = "netcope,txmac";
type = "tx_mac_lite";
speed = "100";
version = <0x02>;
reg = <0x8000 0x200>;
mtu = <0x3fff>;
phandle = <0x0b>;
};
rxmac0: rxmac0 {
compatible = "netcope,rxmac";
type = "rx_mac_lite";
speed = "100";
version = <0x02>;
reg = <0x8200 0x200>;
mtu = <0x3fff>;
phandle = <0x0a>;
};
eth0 {
compatible = "netcope,eth";
pmd = <0x08>;
pcspma = <0x09>;
rxmac = <0x0a>;
txmac = <0x0b>;
pmd-params {
lines = <0x00 0x01 0x02 0x03>;
};
};
regarr1: regarr1 {
compatible = "netcope,pcsregs";
reg = <0xa00000 0x40000>;
phandle = <0x0c>;
};
pcspma1: pcspma1 {
type = "100G";
control = <0x0c>;
phandle = <0x0e>;
control-param {
ip-name = "E_TILE";
};
};
txmac1: txmac1 {
compatible = "netcope,txmac";
type = "tx_mac_lite";
speed = "100";
version = <0x02>;
reg = <0xa000 0x200>;
mtu = <0x3fff>;
phandle = <0x10>;
};
rxmac1: rxmac1 {
compatible = "netcope,rxmac";
type = "rx_mac_lite";
speed = "100";
version = <0x02>;
reg = <0xa200 0x200>;
mtu = <0x3fff>;
phandle = <0x0f>;
};
eth1 {
compatible = "netcope,eth";
pmd = <0x0d>;
pcspma = <0x0e>;
rxmac = <0x0f>;
txmac = <0x10>;
pmd-params {
lines = <0x00 0x01 0x02 0x03>;
};
};
intel_sdm_controller {
compatible = "netcope,intel_sdm_controller";
reg = <0x1000 0x2c>;
type = <0x00>;
version = <0x01>;
boot_en = <0x00>;
};
intel_jtag_op_controller {
compatible = "cesnet,ofm,intel_jtag_op_ctrl";
reg = <0x10000 0xc000>;
type = <0x00>;
version = <0x01>;
};
app: application {
app_core_minimal_0 {
reg = <0x2000000 0x800000>;
compatible = "cesnet,minimal,app_core";
rx_chan_router {
compatible = "cesnet,ofm,mvb_channel_router";
reg = <0x2000000 0x04>;
};
};
app_core_minimal_1 {
reg = <0x2800000 0x800000>;
compatible = "cesnet,minimal,app_core";
rx_chan_router {
compatible = "cesnet,ofm,mvb_channel_router";
reg = <0x2800000 0x04>;
};
};
ddr_tester_0: mem_tester_0 {
reg = <0x3000000 0x100>;
compatible = "netcope,mem_tester";
};
ddr_tester_1: mem_tester_1 {
reg = <0x3020000 0x100>;
compatible = "netcope,mem_tester";
};
ddr_tester_2: mem_tester_2 {
reg = <0x3040000 0x100>;
compatible = "netcope,mem_tester";
};
ddr_tester_3: mem_tester_3 {
reg = <0x3060000 0x100>;
compatible = "netcope,mem_tester";
};
ddr_logger_0: mem_logger_0 {
reg = <0x3080000 0x30>;
compatible = "netcope,mem_logger";
};
ddr_logger_1: mem_logger_1 {
reg = <0x30a0000 0x30>;
compatible = "netcope,mem_logger";
};
ddr_logger_2: mem_logger_2 {
reg = <0x30c0000 0x30>;
compatible = "netcope,mem_logger";
};
ddr_logger_3: mem_logger_3 {
reg = <0x30e0000 0x30>;
compatible = "netcope,mem_logger";
};
};
dbg_gls0 {
compatible = "cesnet,ofm,gen_loop_switch";
reg = <0x5000 0x200>;
version = <0x01>;
mfb_gen2dma {
compatible = "cesnet,ofm,mfb_generator";
reg = <0x5080 0x40>;
version = <0x01>;
};
mfb_gen2eth {
compatible = "cesnet,ofm,mfb_generator";
reg = <0x50c0 0x40>;
version = <0x01>;
};
};
dbg_gls1 {
compatible = "cesnet,ofm,gen_loop_switch";
reg = <0x5200 0x200>;
version = <0x01>;
mfb_gen2dma {
compatible = "cesnet,ofm,mfb_generator";
reg = <0x5280 0x40>;
version = <0x01>;
};
mfb_gen2eth {
compatible = "cesnet,ofm,mfb_generator";
reg = <0x52c0 0x40>;
version = <0x01>;
};
};
};
};
};
Note
The phandle is a unique identifier of the (parent) node within one DTS. With this mechanism, nodes can be referenced in any properties.
Requirements for developers
- Requirements for SW developers:
The developer must know the basic structure of the Device Tree.
Must be familiar with the functions of the libfdt library.
Must know the DT node specification of the component for which the utility software writes.
- Requirements for HW developers:
The developer must know the basic syntax of DTS to implement the unique
dts_my_compfunction for his component.Specifies a compatible string, ie creates a description of each “property” (or “subnode”) of the Device Tree for the SW developer so that the control software can write according to them.
It must take into account any generic parameters that change the MI address space of a component or the function of its registers. Such parameters and features must be included in the DTS so that the utility software can work with it properly.
Generics that do not change the address space should not be included in the component’s DTS node. Component modifications and bug fixes that do not change the address space should also not be included in the DT description, because they can be found from the revision (which is one of the basic properties of the DT project). However, it is possible to consider, for example, to increase the minor version number, if the property version is present.
Implements
dts_my_compideally as a function inDevTree.tcl, which is implemented usingsource $ENTITY_BASE/DevTree.tclinModules.tcl. It is necessary to comment well on the input parameters and inform about the default values.When changing a generic that changes the address space, the developer must also update the DT description of the component or the DT description specification. Changes to the VHDL generic and its default values must match the values in the DT description. Otherwise, it is a bug in the firmware.
If it instantiates a subcomponent in its component that contains another
dts_my_compfunction, it should include (call) it in its description as well. In this case, it must correctly pass the base addresses and all other generics required by thedts_my_compfunction. This is especially important for a top-level project where the main components are installed.