Awaiting product image

PyNVMe3 User Guide

PyNVMe3 User Guide

Last Modified: August 25, 2023

Copyright © 2021-2023 GENG YUN Technology Pte. Ltd.
All Rights Reserved.

Change Log

version author date changes
0.1 cranechu 2021-01-11 initial version
0.9 cranechu 2021-02-06 updated for PyNVMe3 R21.2
1.0 cranechu 2021-02-25 added an email for tech support
1.1 cranechu 2021-05-25 changed recommended platform
keep reports in folder results
1GB hugepage setup
updated for PyNVMe3 R21.5
1.2 cranechu 2021-08-03 add benchmark test suite
Quarch PAM setup
1.3 cranechu 2021-11-25 PAM is required by TCG and benchmark tests
1.4 cranechu 2022-05-26 support NVMe v2.0
updated for PyNVMe3 R22.5
1.5 cranechu 2022-11-28 1GB-hugepage is a required configuration
updated for PyNVMe3 R22.11
2.0 cranechu 2022-12-28 re-distribute as an online document
2.1 cranechu 2023-08-25 PyNVMe3 R23.8 supports Ubuntu 20.04 and 22.04

Introduction

PyNVMe3 is a test collection for NVMe SSD implemented in Python3 scripts, powered by an in-house NVMe test-dedicated driver. The driver takes full control of the Device Under Test (DUT), from registers to memory, from data buffer to commands. It provides comprehensive and powerful functions in easy-to-use Python API. PyNVMe3 builds a professional and well maintained collection of NVMe test scripts based on this driver.

pic
Image 1 PyNVMe3 Block Diagram

PyNVMe3 refers to the NVMe Specification v1.4 and v2.0, but it is not limited by the NVMe Specification.

Requirement

The platform running PyNVMe3 has to meet these minimum requirements:

  1. CPU: Intel® Core™ i5
  2. Memory: 8GB
  3. OS: Ubuntu 20.04.x and 22.04.x
  4. Secure boot: disabled
  5. Storage: enable AHCI/NVMe mode
pic
Image 2 BIOS Setup

We recommend Dell Precision 3560 Mobile Workstation as the test system of PyNVMe3. It has dual M.2 slots. SSD-1 slot is used for a SATA SSD where OS installed, and SSD-2 (PCIe Gen4) slot is for NVMe DUT.

pic
Image 3 Recommended System

Install

In Ubuntu, open a Terminal to install PyNVMe3 in command lines. Internet access is required when installing PyNVMe3.

  1. install pip3

    sudo apt install -y python3-pip

  2. uninstall PyNVMe3 if it has been installed before

    sudo pip3 uninstall PyNVMe3
    sudo rm -rf /usr/local/PyNVMe3

  3. install PyNVMe3 package with pip3

    sudo pip3 install PyNVMe3-23.8.tar.gz

Now, PyNVMe3 is installed under folder /usr/local/PyNVMe3.

Configuration

1GB-hugepage is required when scripts allocate huge buffers (e.g. HMB). Furthermore, we need to disable IOMMU on AMD platforms. We can make the configurations in following steps:

  1. modify the line of GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub 
    GRUB_CMDLINE_LINUX_DEFAULT="quiet splash default_hugepagesz=2M hugepagesz=1G hugepages=1 amd_iommu=off"
  2. sudo update-grub
  3. add one line in file /etc/fstab 
    none /mnt/huge hugetlbfs pagesize=1G,size=1G 0 0
  4. sudo reboot

PAM Setup

Quarch Power Analysis Module (PAM) is a 3-rd party power module used to control PCIe power (on/off) and monitor device power consumption. It is required by TCG tests, as well as power-related benchmark tests in PyNVMe3. PAM consists of 2 units: a control unit in a white box, and the m.2 fixture in 2280 form factor.

pic
Image 4 Front-view of the control unit
pic
Image 5 Back-view of the control unit
pic
Image 6 M.2 Fixture

There 2 units are connected to the System Under Test (SUT) in the following steps:

  1. insert the m.2 fixture to the SUT, for example, SSD-2 (PCIe Gen4) slot of the recommended system;
  2. plug the DUT into the m.2 fixture;
  3. connect the control unit and the m.2 fixture with the (Quarch provided) type-C to type-C USB cable;
    pic
    Image 7 Connection among SUT, DUT, fixture, and control unit
  4. connect the control unit to the SUT with the (Quarch provided) type-B to type-A USB cable;
  5. connect (Quarch provided) power adaptor to the control unit, and power on;
  6. power on the SUT.
    pic
    Image 8 Connection between SUT and controller unit

In step 4, as an alternative, when the scripts neither require PAM to control the power on/off, nor collect power data by the scripts themselves, we can connect the control unit to another computer where Quarch Power Studio (QPS) collects and presents the power data.

Test

In Ubuntu, open a Terminal to run PyNVMe3 in command lines. Internet access is not required to run PyNVMe3 tests.

  1. change directory to PyNVMe3 folder.

    cd /usr/local/PyNVMe3

  2. switch to root user.

    sudo su

  3. run the conformance tests with NVMe DUT.

    make test

  4. or, run the specified tests with NVMe DUT in following ways.

    make test TESTS=scripts/test_folder
    make test TESTS=scripts/test_folder/test_file.py
    make test TESTS=scripts/test_folder/test_file.py::test_function

    For more information, please refer to pytest documents.

  5. or when multiple NVMe DUTs are attached in the SUT, run the tests with a specified NVMe DUT by its BDF address.

    make test pciaddr=BB:DD.F

NOTICE: The test may remove or overwrite the data in your NVMe DUT. Before running any PyNVMe3 test, please backup your data and delete all partitions in your DUT.

When PyNVMe3 test is running, the test-dedicated driver takes over control of NVMe DUT from Linux Kernel’s inbox driver. So, the Kernel, as well as the Kernel-based utilities (e.g. fio, nvme-cli), cannot detect and manipulate the NVMe DUT during PyNVMe3 tests.

Conformance Tests

PyNVMe3 provides conformance test suite to test DUT against specifications. Quarch PAM is required by some test scripts (e.g. TCG tests). If PAM is not available, these tests are skipped.

Benchmark Tests

PyNVMe3 provides benchmark test suite to test DUT in real workloads. We can compare test results of different DUTs. Quarch PAM is required by some of the benchmark tests (e.g. power_cycle.py). Users can run benchmark test as below:

make test TESTS=./scripts/benchmark/power_cycle.py

Results

After the test is completed, we can find the test report (.xlsx) and log (.log) files in folder PyNVMe3/results. Benchmark tests could generate more files and pictures in this folder.

pic
Image 9 Test result files

There are some possible test results of each test item:

  1. SKIPPED: test skipped.
  2. FAILED: test failed.
  3. PASSED: test passed, but may have some warnings.

For more information on the test log and report, please refer to pytest.

Troubleshooting

When the test is terminated abnormally, users can find more debug information from the log files in the PyNVMe3/results folder. Users can also add a prefix ‘_’ to the function name of the FAILED tests to bypass them temporarily. Please provide us the test log file when we need to investigate issues together.

Contact