Added initial support for test plans

.gitignore

@@ -52,6 +52,12 @@ Module.symvers
 Mkfile.old
 dkms.conf
 
+# Built documentation
+doc/_build/pdf/*.pdf
+**/doc/pdf/*.pdf
+**doc/_build/**
+
+
 # Tmp files
 *.*~
 *~

CHANGELOG.rst

@@ -7,6 +7,15 @@ UNRELEASED
   * ADDED:     AssertiveComparisonChecker's suppress_multidrive_messages support/param to
     ComparisonChecker
   * ADDED:     Methods in Xsi class for getting the xsim tick frequency
+  * ADDED:     Reusable testplan pytest plugin documentation under doc/rst, including
+    central parametrization, automatic pytest marker discovery, generated RST
+    results, and optional executable sequence rendering
+  * ADDED:     testplan generated RST can include optional sequence tables from pure
+    sequence builder functions
+  * ADDED:     testplan generated RST includes requirement coverage and result
+    summaries based on authored verifies links
+  * ADDED:     testplan requirement coverage supports unsupported requirements as
+    deliberate exclusions from coverage gaps
   * CHANGED:   Pyxsim CMake build uses XCommon CMake
   * CHANGED:   The way time is incremented by time_step for better floating point precision
   * CHANGED:   ComparisonChecker only prints expected output when verbosity is 2 or higher (i.e.

README.rst

@@ -1,78 +1,66 @@
+:orphan:
 
-Test Support
-============
+#########################################
+test_support: XMOS test support helpers
+#########################################
 
-This repo contains helpers for testing XMOS xCORE applictions and includes the following:
+:vendor: XMOS
+:version: 2.0.0
+:scope: General Use
+:description: Reusable pytest, simulator, coverage, and test-plan helpers for XMOS software repositories
+:category: Test
+:keywords: pytest, xsim, pyxsim, xcoverage, testplan
+:devices: xcore-200, xcore.ai
 
-- Python wrapper for the xCORE simulator (xsim)
-- Python access functions for XE files
-- Python code coverage measurement (xcov) for pytest 
+*******
+Summary
+*******
 
-Basic usage: xcoverage
-----------------------
+``test_support`` provides reusable Python helpers for testing XMOS xCORE
+applications. It includes wrappers for running tests on the xCORE simulator,
+code coverage processing utilities, and pytest helpers for generating structured
+test-plan result documentation.
 
-This only suit for xsim.
+********
+Features
+********
 
-It requires disassembly and elf file which dumped from binary file (.xe file) by:
+* Python wrapper for the xCORE simulator, ``xsim``.
+* Python access functions for XE files.
+* Python code coverage measurement helpers for pytest.
+* Reusable pytest test-plan generation helpers for central parametrization,
+  generated result summaries, and optional executable sequence tables.
 
- * xobjdump --split [.xe].
- * xobjdump -S [.xe] -o [output_file_name.dump].
- * run the above 2 step by youself or run method from xcov: generate_elf_disasm("/path_to/(name-of-xe).xe", "/path_where_store_elf_and_disasm", "/path_to/(name-of-disasm).dump")
+************
+Known issues
+************
 
-.xe must make with -g flag to enable the gdb bugger otherwise xcoverage won't work!.
-
-It also needs a tracing file from xsim by running:
-
- * xsim --trace-to [output_file_name.txt] [.xe].
-
-``xcov_process``
-.......................
-
-This is the main function to be called in your test.
-It returns the average coverage and save the data in .xcov file in xcov dir.
-.xcov file is necessary for the below "xcov_combine" function.
-
-xcov_process(disasm, trace, xcov_dir).
-
- * @param disam: path to disasm file.
- * @param trace: path to trace file.
- * @param xcov_dir : path where xcov directory locates.
- * @return average coverage of all src file.
- * @output generate xcov file for xcov_combine and save in xcov dir.
-
-``xcov_combine``
-.......................
-
-see example in examples/code_coverage
-
-``combine_process``
-.......................
-
-see example in examples/code_coverage
-
-``Mark the source code as not expected to be hit``
-........................................................
-
-Add a comment "//NOCOVER" or "//NOCOVERSTART" and "//NOCOVEREND" beside you source code. It wouldn't be counted in coverage.
-
-see example in test/test_xcoverage
-
-``Excluded File``
-........................................................
-Passing an excluded_file arg in xcov_process(), eg:
-
-xcov_process(disasm, trace, xcov_dir, excluded_file=["/tests/shared/test_main.xc","/tests/shared/shared.h" ])
-
-Software version and dependencies
-.................................
-
-The CHANGELOG contains information about the current and previous versions.
-For a list of direct dependencies, look for DEPENDENT_MODULES in test_support/module_build_info.
+* None
 
+****************
+Development repo
+****************
 
+* ``test_support`` is maintained as part of XMOS software infrastructure.
 
+**************
+Required tools
+**************
 
+* XMOS XTC Tools
+* Python 3
+* pytest
 
+*********************************
+Required libraries (dependencies)
+*********************************
 
+* ``colorama``
+* ``PyYAML``
 
+*******
+Support
+*******
 
+This package is supported by XMOS Ltd. Issues can be raised against the software at
+`www.xmos.com/support <https://www.xmos.com/support>`_.

doc/exclude_patterns.inc

@@ -0,0 +1,3 @@
+CHANGELOG.rst
+LICENSE.rst
+examples/**

doc/rst/code_coverage.rst

@@ -0,0 +1,76 @@
+#############
+Code Coverage
+#############
+
+``test_support`` includes Python code coverage measurement helpers for pytest
+tests that run on ``xsim``.
+
+************
+Requirements
+************
+
+Coverage processing requires a disassembly file and ELF file extracted from the
+``.xe`` binary:
+
+.. code-block:: bash
+
+   xobjdump --split app.xe
+   xobjdump -S app.xe -o app.dump
+
+The same files can be generated from Python by calling
+``generate_elf_disasm(xe_path, output_dir, dump_path)``.
+
+The ``.xe`` file must be built with debug information enabled using ``-g`` so
+that coverage can be mapped back to source lines.
+
+Coverage also requires a trace file from ``xsim``:
+
+.. code-block:: bash
+
+   xsim --trace-to trace.txt app.xe
+
+************
+xcov_process
+************
+
+``xcov_process(disasm, trace, xcov_dir)`` processes a disassembly file and xsim
+trace, returns the average source coverage, and writes an ``.xcov`` file under
+``xcov_dir``. The generated ``.xcov`` file is used by the combine helpers.
+
+Arguments:
+
+* ``disasm``: path to the disassembly file.
+* ``trace``: path to the xsim trace file.
+* ``xcov_dir``: directory where ``.xcov`` files are stored.
+
+************
+xcov_combine
+************
+
+``xcov_combine`` combines coverage data from multiple ``.xcov`` files. See
+``examples/code_coverage`` for example usage.
+
+***************
+combine_process
+***************
+
+``combine_process`` provides a higher-level combine flow. See
+``examples/code_coverage`` for example usage.
+
+*******************
+Coverage Exclusions
+*******************
+
+Add ``//NOCOVER`` beside a source line to exclude it from coverage. Use
+``//NOCOVERSTART`` and ``//NOCOVEREND`` to exclude a block.
+
+An excluded file list can also be passed to ``xcov_process``:
+
+.. code-block:: python
+
+   xcov_process(
+       disasm,
+       trace,
+       xcov_dir,
+       excluded_file=["/tests/shared/test_main.xc", "/tests/shared/shared.h"],
+   )

doc/rst/test_support.rst

@@ -0,0 +1,54 @@
+#########################################
+test_support: XMOS test support helpers
+#########################################
+
+************
+Introduction
+************
+
+``test_support`` contains reusable helpers for testing XMOS xCORE software from
+pytest. The package is intended to be installed into consuming repositories and
+used by their local test suites.
+
+The package includes simulator wrappers, coverage processing helpers, and a
+generic pytest test-plan plugin that can generate documentation-ready result
+includes for projects using ``sphinx-needs``.
+
+************
+Installation
+************
+
+Consuming repositories commonly install ``test_support`` as an editable Python
+dependency from their development environment:
+
+.. code-block:: bash
+
+   pip install -e ../test_support
+
+The exact path depends on the repository layout used by the project under test.
+
+********
+Contents
+********
+
+.. toctree::
+   :maxdepth: 2
+
+   code_coverage
+   testplan
+
+******
+Pyxsim
+******
+
+``Pyxsim`` provides Python helpers for running xCORE simulator based tests from
+pytest. Tests can execute XE files under ``xsim`` and compare simulator output
+using the provided tester classes.
+
+For examples, see ``examples/pyxsim`` in this package.
+
+*********
+Changelog
+*********
+
+Release notes are recorded in ``CHANGELOG.rst`` at the package root.