Hi everyone, This email is to start a discussion about the format into which NVIDIA firmware is going to be provided. If you had a look at the linux-firmware branch we pushed earlier [1] you may already have an idea of the general organization, but this email is to discuss more specific details. Official firmware is organized per-chip, with an additional level of hierarchy for the different managed subsystems. For example, gm200 currently has two sub-directories, acr and gr, which contain the firmware files for secure boot (ACR) and PGRAPH (GR). ACR is a particular case and comes in the form of self-contained units (code, data, signature) that can be run on a high-secure falcon (currently PMU). It consumes a blob that is built by the kernel and contains the signed firmwares of the low-secure falcons to load and manage. The ACR blob is made of a header describing the managed falcons and the offses of their bootloader, code and data within the blob, as well as bootloader/code/data sections for each falcon. A signed, low-secure falcon firmware in the ACR blob is thus the aggregation of three different components: - An image containing the bl, code and data sections - A descriptor with the offsets of these sections within the image - A signature that the ACR will verify against These three components can come as files to be directly loaded. However for the current GR firmware we took the approach of splitting the bl, code and data sections into their own files, and building the image and descriptor on-the-fly, as you can see from gm200/gr: gm200/gr/fecs_bl.bin gm200/gr/fecs_data.bin gm200/gr/fecs_inst.bin gm200/gr/fecs_sig.bin The bl, data, and inst files are loaded and combined into an image while the corresponding descriptor is built. This is done in the ls_ucode_img_build() function. The main reason for doing this is there is that for a given GPU generation, the _bl and _inst files are very likely going to be exactly the same, with only the data and signature varying. Splitting the sections allow us to symlink identical files. For instance, gr/gm200 weights 61KB, while gm204/gr, which mostly symlinks to the former, only takes 8.5KB. Another advantage is that this also allows the code and data to be directly loaded via the traditional method into a fused non-secure board, although this advantage is not too relevant for the community. That's the design we took for now - it is possible to switch to a more smaller number of files per chip, and remove a bit of kernel code, at the cost of firmware footprint. I just wanted to make sure this design was ok and take any objection into account before the planned merge of the kernel support for signed firmware, hopefully next week. Thanks, Alex. [1] https://github.com/Gnurou/linux-firmware/tree/secboot/nvidia
On Wed, Feb 17, 2016 at 10:39 PM, Alexandre Courbot <acourbot at nvidia.com> wrote:> Hi everyone, > > This email is to start a discussion about the format into which NVIDIA > firmware is going to be provided. If you had a look at the linux-firmware > branch we pushed earlier [1] you may already have an idea of the general > organization, but this email is to discuss more specific details. > > Official firmware is organized per-chip, with an additional level of > hierarchy for the different managed subsystems. > > For example, gm200 currently has two sub-directories, acr and gr, which > contain the firmware files for secure boot (ACR) and PGRAPH (GR). > > ACR is a particular case and comes in the form of self-contained units > (code, data, signature) that can be run on a high-secure falcon (currently > PMU). It consumes a blob that is built by the kernel and contains the signed > firmwares of the low-secure falcons to load and manage. > > The ACR blob is made of a header describing the managed falcons and the > offses of their bootloader, code and data within the blob, as well as > bootloader/code/data sections for each falcon. > > A signed, low-secure falcon firmware in the ACR blob is thus the aggregation > of three different components: > > - An image containing the bl, code and data sections > - A descriptor with the offsets of these sections within the image > - A signature that the ACR will verify against > > These three components can come as files to be directly loaded. However for > the current GR firmware we took the approach of splitting the bl, code and > data sections into their own files, and building the image and descriptor > on-the-fly, as you can see from gm200/gr: > > gm200/gr/fecs_bl.bin > gm200/gr/fecs_data.bin > gm200/gr/fecs_inst.bin > gm200/gr/fecs_sig.bin > > The bl, data, and inst files are loaded and combined into an image while the > corresponding descriptor is built. This is done in the ls_ucode_img_build() > function. > > The main reason for doing this is there is that for a given GPU generation, > the _bl and _inst files are very likely going to be exactly the same, with > only the data and signature varying. Splitting the sections allow us to > symlink identical files. For instance, gr/gm200 weights 61KB, while > gm204/gr, which mostly symlinks to the former, only takes 8.5KB. > > Another advantage is that this also allows the code and data to be directly > loaded via the traditional method into a fused non-secure board, although > this advantage is not too relevant for the community. > > That's the design we took for now - it is possible to switch to a more > smaller number of files per chip, and remove a bit of kernel code, at the > cost of firmware footprint. > > I just wanted to make sure this design was ok and take any objection into > account before the planned merge of the kernel support for signed firmware, > hopefully next week.Since the firmware is completely separate from the kernel, you need to think about versioning. The firmware presents an ABI to the kernel, and unless you promise to never ever ever ever ever change the ABI with later updates, versioning the firmware files is something you're going to have to think about. Sometimes it's done via filenames, e.g. -1, -2, etc. Sometimes it's done by packing multiple data files into a single one, allowing the code to pick whichever one it wants. -ilia
On 02/18/2016 12:47 PM, Ilia Mirkin wrote:> On Wed, Feb 17, 2016 at 10:39 PM, Alexandre Courbot <acourbot at nvidia.com> wrote: >> Hi everyone, >> >> This email is to start a discussion about the format into which NVIDIA >> firmware is going to be provided. If you had a look at the linux-firmware >> branch we pushed earlier [1] you may already have an idea of the general >> organization, but this email is to discuss more specific details. >> >> Official firmware is organized per-chip, with an additional level of >> hierarchy for the different managed subsystems. >> >> For example, gm200 currently has two sub-directories, acr and gr, which >> contain the firmware files for secure boot (ACR) and PGRAPH (GR). >> >> ACR is a particular case and comes in the form of self-contained units >> (code, data, signature) that can be run on a high-secure falcon (currently >> PMU). It consumes a blob that is built by the kernel and contains the signed >> firmwares of the low-secure falcons to load and manage. >> >> The ACR blob is made of a header describing the managed falcons and the >> offses of their bootloader, code and data within the blob, as well as >> bootloader/code/data sections for each falcon. >> >> A signed, low-secure falcon firmware in the ACR blob is thus the aggregation >> of three different components: >> >> - An image containing the bl, code and data sections >> - A descriptor with the offsets of these sections within the image >> - A signature that the ACR will verify against >> >> These three components can come as files to be directly loaded. However for >> the current GR firmware we took the approach of splitting the bl, code and >> data sections into their own files, and building the image and descriptor >> on-the-fly, as you can see from gm200/gr: >> >> gm200/gr/fecs_bl.bin >> gm200/gr/fecs_data.bin >> gm200/gr/fecs_inst.bin >> gm200/gr/fecs_sig.bin >> >> The bl, data, and inst files are loaded and combined into an image while the >> corresponding descriptor is built. This is done in the ls_ucode_img_build() >> function. >> >> The main reason for doing this is there is that for a given GPU generation, >> the _bl and _inst files are very likely going to be exactly the same, with >> only the data and signature varying. Splitting the sections allow us to >> symlink identical files. For instance, gr/gm200 weights 61KB, while >> gm204/gr, which mostly symlinks to the former, only takes 8.5KB. >> >> Another advantage is that this also allows the code and data to be directly >> loaded via the traditional method into a fused non-secure board, although >> this advantage is not too relevant for the community. >> >> That's the design we took for now - it is possible to switch to a more >> smaller number of files per chip, and remove a bit of kernel code, at the >> cost of firmware footprint. >> >> I just wanted to make sure this design was ok and take any objection into >> account before the planned merge of the kernel support for signed firmware, >> hopefully next week. > > Since the firmware is completely separate from the kernel, you need to > think about versioning. The firmware presents an ABI to the kernel, > and unless you promise to never ever ever ever ever change the ABI > with later updates, versioning the firmware files is something you're > going to have to think about. Sometimes it's done via filenames, e.g. > -1, -2, etc. Sometimes it's done by packing multiple data files into a > single one, allowing the code to pick whichever one it wants.For versioning purposes, I thought about using different filenames. It is simple and effective, and since I cannot predict the scope of changes these files may undergo, it also seems to be the most flexible solution. Note that the format of files named similarly for different GPUs might also be different. What is guaranteed is that a given file will forever remain backward-compatible. There already are differences between the GM20B (Tegra) firmware files and the other GM20X due to GM20B coming from a different tree, so although it may be a little bit confusing this is a necessary evil. And it's not like we are not used to dealing with chip-specific ops in Nouveau anyway. :)