Improve dict generation for key-dependent branches

[Shikaikai02]

Hi, I’m exploring the input space of Python APIs for a research project. Many of the functions I target accept **kwargs or other dictionary parameters.
Currently Pynguin builds dicts at random, so code like

def foo(d: dict[str, int]):
    if d['k'] == 10: ...

never enters the if.
Do you have any plans to treat constant keys ('k') as guidance hints, as scalars already are? This would greatly help coverage for dict-heavy APIs.
Thanks!

[BergLucas]

Hi @Shikaikai02,

As far as I know, Pynguin already captures constants of type str to guide its algorithms. Doesn't the problem stem from the fact that the probability of generating a dictionary with exactly the right keys/values that your functions need is very low?

Have a nice day!

[LuKrO2011]

Hi @Shikaikai02,

thank you for reporting this.

This code snippet also does not work very well:

def foo(d: dict[str, int]):
    if d['abcdef']: ...

whereas this works as expected due to constant_seeding:

def foo(d: str):
    if d == 'abcdef': ...

So I think there is indeed a problem with using the collected seeds as dict keys, but maybe one can improve the fitness_function for comparisons of dict items on top of that.

Feel free to contribute to the project by suggesting one or multiple PRs.

[LuKrO2011]

One problem was that the constant seeding only worked for modules within packages (having a __init__.py). I explicitly also added the SUT module itself to scan for seeds in case it is not part of a package in 55fb6c5.

For dict keys, values of StringPrimitiveStatement (among others) are used and the values of these are seeded, so there is no problem with using the collected seeds as dict keys.

Regarding the fitness_function the main issue is, that if the dict-access raises an exception (e.g., missing key) and so the compare predicate (e.g. == 10 or existence) is not reached, and no fitness is calculated. So finding the right dict-key is currently, beyond getting lucky with seeding, purely guessing.

[Shikaikai02]

Thank you very much for the detailed explanation. Your clarification about dict key seeding and the current limitation is really helpful.
Pynguin is a great project, and I hope it continues to evolve and become even more powerful.
Thanks again for your time and for maintaining the project!