feat: automatically detect end of include guard instead of relying on regex match

[Viatorus]

Fixes #60

[Copilot]

Pull request overview

This PR enhances include guard detection by automatically tracking the nesting level of preprocessor conditionals instead of relying on regex matching of #endif comments. The implementation introduces a tracker class that counts the depth of conditional blocks to determine when an include guard ends.

• Adds IncludeGuardsTracker class to track nested conditional blocks
• Introduces tokenization support for #if and #ifdef directives
• Updates include guard detection logic to use depth tracking instead of comment regex matching

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File	Description
tests/test_quom/test_include_guards.py	Adds comprehensive test cases for simple and complex include guard scenarios with nested conditionals
src/quom/tokenizer/preprocessor_tokenizer.py	Adds token classes and parsing logic for #if and #ifdef directives
src/quom/tokenizer/init.py	Exports newly added preprocessor token types
src/quom/quom.py	Implements IncludeGuardsTracker class and refactors include guard detection to use depth tracking

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The tracker's if_reached() is called for all conditional preprocessor directives, but this happens before checking if the token is actually an include guard. This means the tracker will incorrectly increment depth for all #ifndef directives, not just those that are include guards. The if_reached() call should only occur after confirming the token matches an include guard pattern.

[Copilot]

The IncludeGuardsTracker class lacks a docstring explaining its purpose and how it tracks nested include guards. Consider adding documentation that describes the tracking mechanism and the meaning of the integer values stored in the __guards list.

Suggested change

	class IncludeGuardsTracker:
	class IncludeGuardsTracker:
	"""
	Tracks the lifetime of one or more include guards across nested
	preprocessor conditionals.
	Each time an include guard is detected, :meth:`include_guard_found`
	is called and an entry with value ``1`` is appended to ``__guards``.
	The integers stored in ``__guards`` represent the remaining nesting
	depth until the corresponding include guard is closed:
	* :meth:`if_reached` is called when a new ``#if``-like block is
	entered and increments all stored depths.
	* :meth:`endif_reached` is called when an ``#endif`` is reached and
	decrements all stored depths, removing any entry that reaches
	zero and returning ``True`` if at least one include guard has just
	ended.
	"""

[Copilot]

This loop can be replaced with a list comprehension for better readability and performance: self.__guards = [x + 1 for x in self.__guards].

Suggested change

	for i in range(len(self.__guards)):
	self.__guards[i] += 1
	self.__guards = [guard + 1 for guard in self.__guards]

[Copilot]

The logic for decrementing guards and filtering can be simplified. The current approach mutates the list in place and then filters it, which is inefficient. Consider: first decrement all values, then check if any reached zero, and finally filter in a single pass.

Suggested change

	# Decrease all guards by one level.
	for i in range(len(self.__guards)):
	self.__guards[i] -= 1
	if self.__guards[i] == 0:
	include_guard_ended = True
	if include_guard_ended:
	self.__guards = [x for x in self.__guards if x > 0]
	# Decrease all guards by one level, tracking which ones end, in a single pass.
	new_guards = []
	for level in self.__guards:
	new_level = level - 1
	if new_level == 0:
	include_guard_ended = True
	elif new_level > 0:
	new_guards.append(new_level)
	self.__guards = new_guards