Version 0.5: HLSL++ Math, Graphics API Optimizations, Automated Static Code Analysis, Tests Code Coverage, Code Improvements and Refactoring

[egorodet]

• Samples and tutorials:
  • Migrated tutorials and samples from CML to HLSL++ vector math library with SIMD intrinsics acceleration for CPU. Removed explicit alignment of uniform structure fields, which is not required anymore for HLSL++ types, since they already use 16-bytes aligned data storage.
  • Asteroids sample was optimized with up to 50% FPS increase:
    • Uniforms buffer size was reduced by reusing scene view-projection matrix in shaders (+7% FPS)
    • Uniforms buffer data transfer is made asynchronously with draw commands encoding (+20% FPS)
    • Disabled repeated retaining of the bound resources in command list draw calls (+30% FPS)
    • Disabled automatic resource barriers setup during parallel asteroids rendering to take advantage of automatic state propagation from Common state.
    • Got rid of excessive program binding calls in DirectX as a result of using Frame-Constant argument accessors in program bindings.
• Common libraries:
  • Primitives library extended with error handling (closed Define custom exception types and throw from META_CHECK_* macroses project wide #74):
    • Methane/Exceptions.hpp was added with common exception type defintions
    • Methane/Checks.hpp was added with error checking macro definitions aka META_CHECK*
    • META_LOG macros has been updated to make string formatting using FMT library inside of the macro using variadic arguments
• Data libraries
  • RawVector template class was added as a replacement of cml::vector for dense-packed components storage used in vertex attributes.
  • Point template class now use underlying HLSL++ vector and unifies 2D and 3D implementations.
  • Rect and RectSize, Volume and VolumeSize classes were refactored. Made class fields private, available through accessors.
  • Constants template class was added with math constants.
• Platform libraries
  • Platform App library was updated with unified error handling logic in AppBase class, moved from platform specific classes
• Graphics libraries
  • Graphics Core changes:
    • Program binding improvements:
      • Program::Argument and Program::ArgumentAccessor (previously Program::ArgumentDesc) were refactored from struct to class style with field accesor functions. Program::Argument::Modifiers bit-mask was replaced with Program::ArgumentAccessor::Type enum.
      • New program argument accessor type was added for frame-constant arguments Program::ArgumentAccessor::Type::FrameConstant to reduce number of excessive argument binding setup calls. Frame-constant bindings are properly supported by DX descriptor table layout.
      • Frame-buffer index was added to ProgramBindings::Create function as an optional argument to support frame-constant bindings.
      • Applying of DirectX resource states and root parameter bindings was reworked in ProgramBindingsDX.
    • Resource upload and state synchronization improvements:
      • COPY comamnd queue is now used by default for BlitCommandList execution in DirectX (instead of DIRECT command queue used before). So the resources upload is also done with a COPY queue in DX now. It can be switched back to using DIRECT queue with RenderContext::Settings::options_mask bit-mask flag Context::Options::BlitWithDirectQueueOnWindows.
      • Resource::SetData method is extended with an optional argument of command queue for resource state synchronization. Command queue where resource is actually used should be passed whenever possible.
      • Resource barriers setup was moved from the Resource::SetData method implementations to the SetProgramBindings, SetVertexBuffer and SetIndexBuffer calls, while setup resource barriers are still cached inside Resource object.
      • Resource::State and Resource::Barriers types were moved to the public header files and split from Resource interface definition.
    • CommandKit interface was added to Graphics Core to provide unified access from Context::GetDefaultCommandKit to the command queue, command lists, command list set and synchronization fences made for working together with the same CommandList::Type. This interface allowed to replace methods Context::GetUploadCommandQueue, Context::GetUploadCommandList, Context::GetUploadCommandListSet and RenderContext::GetRenderCommandQueue with Context::GetDefaultCommandKit.
    • CommandList changes:
      • CommandList::SetResourceBarriers(...) call was added to the public interface (was private before).
    • RenderCommandList changes:
      • RenderCommandList::ResetWithStateOnce method was added additionally to ResetWithState to skip command list reset when it is already in Encoding state with the same RenderState already applied.
      • RenderCommandList::SetIndexBuffers method was adedd, index buffer is now set separately from DrawIndexed call.
      • RenderCommandList::Set[Index|Vertex]Buffers methods were exnteded with an optional argument which allows to disable setting resource barriers automatically.
    • ParallelRenderCommandList changes:
      • Child render command lists are now committed in parallel.
    • ImageLoader now initializes texture names inside Load functions.
    • Changed ResourceBase derived classes hierarchy based on template implementation of Resource[DX|MT|VK]<ResourceBaseType> which is replacing ResourceNT alias class
    • Removed almost all Native platform alias headers
    • Fixed some memory leaks in Metal implementation on MacOS
    • Fixed const correctness for Device and Context references passed to core interface factory functions
  • Graphics::App template class implementation was split to Graphics::AppBase to reduce template code and speedup compilation
  • Color template class was rewritten from scratch and now can use both integer and floating point components via underlying HLSL++ vector storage
  • ArcBallCamera unit-tests were improved and uncovered some issues in tested class implementation. Issues were fixed
  • SphereMesh generation was simplified with matrix-free implementation
  • Fixed missing debug names of graphics resources (regression of previous refactoring)
  • Fixed mip-levels count in depth texture descriptor for DirectX12
  • Fixed limited rendering FPS with VSync turned OFF on Nvidia GPUs with latest drivers (Jan 2021)
  • Enabled naming of Tracy GPU contexts
• User Interface libraries
  • Unified UnitType derived classes implementation within one template class
  • Fixed bug in Text mesh generation with multiple zero characters at the end of the string
  • Text rendering is now calling ResetWithStateOnce instead of ResetWithState to minimize rendering state resets in sequence of several text blocks
• Unit-tests coverage was expanded from 5.7% to 13%:
  • RawVector, Point new template classes were covered with unit-tests.
  • Rect and RectSize, Volume and VolumeSize existing template classes were covered with unit-tests.
  • Color template class was covered with unit-tests.
  • UnitType template class was covered with unit tests.
  • UserInterface::Context class unit convertors were covered with unit tests.
  • FakeRenderContext class was added to UI unit-testing infrastructure.
  • Replaced deprecated ParseAndAddCatchTests CMake function with catch_discover_tests to fix Catch warnings.
• Global code changes:
  • Replaced CML math library with HLSL++ to improve math performance with SIMD instructions (close Replace math library with vector instructions implementation #78).
  • Debug logging was extended for resource barriers synchronization, render and view state parameters and cleaned up: properly used magic_enum, removed manual enum to string conversion functions, fixed logs formatting.
  • Properly implemented or enabled default generation of assignment operators, copy and move constructors for all applicable types
  • Used MagicEnum library extensively to remove enum code bloat and improve type safety with enum class types:
    • Uses enum class types for bit-masks instead of raw enum types to improve type safety.
    • Generating string names of enum values with a single line of code
    • Iterating over enum values and getting values count
  • All modules updated with fixes of static code analysis issues from SonarCloud (closed Fix most significant static analysis issues from Sonar Cloud #73):
    • Replaced all explicit exception throws and asserts with META_CHECK* macroses throwing specific exception types, which simplified error checking code and fixed many issues related to non-specific exception throws across Methane code base
    • Throw undefined enum value exception in default branches of all switch blocks to fix repeated analysis issue
    • Replaced std::function usage with template arguments wherever possible to improve performance
    • Used fmt::format(...) for string formatting wherever possible project wide
    • Used magic_enum library to replace raw enum types of bit-masks to enum class types, use enum names and counts generated in compile time.
    • Uppercase literal suffixes
    • Fixed new C++17 static analysis issues:
      • Use std::map::try_emplace instead of std::map::emplace
      • Use std::byte instead of uint8_t and char for byte-access to memory buffers
      • Use structured bindings in for loops over elements of std::map
      • Use [[nodiscard]] attribute for constant methods like getters
      • Use std::string_view in some cases for input string arguments
      • Use inline initialiser statements in if and switch statements
      • Use scoped_lock instead of lock_guard
• External libraries:
  • HLSL++ library v3.0 was added
  • CML library was removed
  • FMT v7.1.3 library was added
  • MagicEnum library v0.7.2 was added
  • OpenCppCoverage Windows tool binaries were added
  • CMRC library was updated to latest version.
  • CLI11 library was updated to v1.9.1
  • Catch2 library was updated to v2.13.5
  • IttApi library was updated to v3.18.8+
  • STB library was updated to latest revision from 13 Jul 2020
  • Taskflow library was updated to v3.1.0
  • Tracy library was updated to v0.7.7
  • Removed Sonar Scanner binaries from repository: binaries are loaded from official web site now
• Build Infrastructure:
  • Speedup Methane Kit builds up to 30% with CMake precompiled headers generated once and shared across all modules (closed Accelerate build with CMake precompiled headers #67)
  • Speedup Graphics Core build with CMake Unity build which compiles multiple C++ source files in batches as one source.
  • Automated Sonar Cloud static code analysis is integrated into Azure Pipelines continuous build system. Scan results are now available with every master and develop build separately for every supported platform (closed Integrate Sonar Cloud static code analysis in Azure Pipelines CI #68)
  • Tests code coverage results collection was setup in CI with Azure Pipelines and Sonar Cloud on Windows with OpenCppCoverage tool, on MacOS with llvm-profdata and on Linux with Gcov (closed Setup code coverage results collection with Azure Pipelines and Sonar Cloud #72)
  • Do not fail build when some unit tests fail or Sonar Scanner jobs fail with error
  • Publish Sonar Scanner error reproducer to build artifacts in case of scan failure
  • Fix Sonar Scanner cache invalidation by adding plugin version to cache key
  • CodeQL static analysis was enabled with GitHub Actions workflow and results available in Security tab on repository page
  • Use MacOS 10.15 and Ubuntu 20.04 agents with GCC 9+ in Azure Pipelines
  • GitPod was switched to VSCode UI and extensions were updated
  • CMake profiles were added for CLion 2021

---

This discussion was created from the release Version 0.5: HLSL++ Math, Graphics API Optimizations, Automated Static Code Analysis, Tests Code Coverage, Code Improvements and Refactoring.