Provide pytest fixture for user test startup

[Rollmops]

Motivation

In some circumstances, it is not feasible to use a file (e.g. main.py) in your project to startup the ui for testing (via the main_file pytest ini config).

E.g. Imagine your production code uses a database that is created before calling ui.run and passed to the pages via some injection mechanism. In your test you want to:

1. provide a mock database (e.g. in-memory) and pass this through your production code (this would already be possible by using the existing main_fileapproach)
2. you also want to access the mocked database in you test to arrange the test data and to assert that the data was manipulated correctly be the pages. This is not possible (at least not without wild hacks).

By providing a pytest.fixture that is starting up the ui, the test database could be injected there and in your actual test by another ´pytest.fixture´.

This is of course not limited to a database.

Implementation

Using pytest fixtures to provide custom startup functions is a natural and backward-compatible way of setting up your test startups.
By default, the fixture user_startup returns the function that implements the 'old' behaviour. By overriding this fixture, you can provide your own startup function.

Progress

• I chose a meaningful title that completes the sentence: "If applied, this PR will..."
• The implementation is complete.
• Pytests have been added (or are not necessary).
• Documentation has been added (or is not necessary).

[Rollmops]

@rodja thanks for the review and fixing things 👍

I was wondering if we should call the fixture user_startup_func or user_startup_function since its returning a function rather than executing it. What do you think?

[rodja]

Very clever @Rollmops. Thanks for providing the code. I like the implementation but hesitated with the demo you provided.

First: it should be in the user fixture documentation and not in the main testing section.

Second: The current demo is not compelling. The user_startup fixture in pytest has only a super narrow, specific purpose, which makes it difficult to find obvious examples for when its useful to replace it. Here's why:

1. Overlap with user fixture capabilities: Most customizations (environment variables, mocks, test instrumentation) can be accomplished by overriding the user fixture directly using pytest's fixture inheritance pattern:

   @pytest.fixture
   async def user(user):  # this overrides and uses NiceGUI's provided fixture
       os.environ['TEST_MODE'] = 'true'
       yield user

2. Existing main_file mechanism: The main_file configuration (via pytest.ini or @pytest.mark.nicegui_main_file) already handles most "different app startup" scenarios.
   Especially if you consider that the (quite undocumented) main_file configuration does not run a file if you set it to an empty string. That should also work for your use case.

3. Limited relevance of ui.run() parameters: Most parameters passed to ui.run() (like reload, show_welcome_message, etc.) are irrelevant in simulated testing since there's no real server or browser involved.

Still, the most compelling reason to override user_startup I could come up with is to control the ui.run() parameters, particularly when you want to test your application without a storage_secret:

@pytest.fixture
def user_startup():
    def startup():
        ui.run()  # No storage_secret - test graceful degradation
    return startup

Because the default user_startup hardcodes storage_secret='simulated secret' you cannot prevent this by overriding user. I find this a legitimate (still a bit artificial) test scenario.

I'm therefore not sure if we should have the user_startup fixture at all.
Maybe you could try the approach with an empty main_file configuration for your use case before we proceed?

[Rollmops]

@rodja Let me fiddle around and try to find a way how our use case can be accomplished with a main.py file. Maybe I gave up too early with that approach, because I find it feels a little unnatural to point to a file main.py which is not really a main for the project (I know, you can change the name, but than all users have to provide a pytest.ini) and that is somewhere in your project and not in the test directory.

Thanks for your effort and I will report back :-)

[rodja]

try to find a way how our use case can be accomplished with a main.py file

You can do it without a main file. See this mini-example:

def custom_setup():
    @ui.page('/')
    def page():
        ui.label('Hello from my custom startup :-)')


@pytest.mark.nicegui_main_file('')
async def test_custom_startup(user: User) -> None:
    custom_setup()

    await user.open('/')
    await user.should_see('Hello from my custom startup :-)')

[Rollmops]

try to find a way how our use case can be accomplished with a main.py file

You can do it without a main file. See this mini-example:

def custom_setup():
    @ui.page('/')
    def page():
        ui.label('Hello from my custom startup :-)')


@pytest.mark.nicegui_main_file('')
async def test_custom_startup(user: User) -> None:
    custom_setup()

    await user.open('/')
    await user.should_see('Hello from my custom startup :-)')

That is not working in my case, since i pass a root to ui.run() which is just executed like ui.run(storage_secret='simulated secret') if the main file is not set.

I keep trying :-)

[Rollmops]

Ok, I played with the main_file and thought I have it. But not.
Here is where the gap is and why I had a bad feeling about a test file.

Hopefully, a simplified example should demonstrates the problem I am running into:

Our app server provides a database connection that can be used by the pages to do stuff.
In my test, I want

1. prepare test data
2. access this database object to assert data that was manipulated by the app

pytest.ini:

[pytest]
asyncio_mode = auto
main_file = test/main.py

test/main.py:

import ...

start_test_server(my_package_under_test)

start_test_server() creates the test context for the app (e.g. creating a sqlite database that then gets passed through injection to the apps) and run ui.run() (which receives the root page).

I could prepare the test data here, that would work (although, we have different apps with different test data in this project, so it would be necessary to provide multiple main.py files, but ok.

The Problem is, I have no way to access the test database object in my test (I even tried working with global variables ...).

test/my_test.py:

import pytest
from nicegui.testing import User


async def test_my_app_does_something_with_the_db(user: User):
    await user.open("/my-app")
    
    # here i need the database to assert data 

The database is just an example.

In nicegui<3 that was possible all with pytest.fixture (I provided fixtures that returned the test database).

Maybe I am overseeing something, but currently we are not able to migrate to nicegui-3 :-(

[rodja]

Ah I see. That looks really complicated. In my analysis I stated that ui.run() parameters have limited relevance to the user fixture. But we now have introduced root which is super crucial. As said before, your contribution with user_startup is great. Only when seeking a short example for the documentation I started to struggle. Maybe you can simply add some hint, without any code demo to the user fixture docs. Then I would be fine with merging this to support unorthodox but super interesting scenarios like your multiple root functions etc.

[Rollmops]

Just to clarify our problem with the main.py approach. It is not to provide a root function - that would be possible with the main.py approach.
The problem is that we want to access objects in the test functions (to arrange and assert test data), that we also need to inject in the startup code (i.e. in the main.py file) to be accessible by the productive code that work with those objects (e.g. database).

The motivation in me initial post did not make that clear, I know :-)

I will improve the docs asap.

Edit: I updated the motivation in the PR description.

[Rollmops]

@rodja I just moved the documentation to the user fixture section and added a more meaningful example. If you think this might confuse users too much, I can remove it.

Furthermore, I renamed the fixture to user_startup_func due to the fact that this fixture returns a function.

[rodja]

@Rollmops why are you keeping to force-push to this branch. It makes review quite hard.

[Rollmops]

Sorry, I just wanted to keep the branch in sync with the main branch. I will stop it.

[evnchn]

keep the branch in sync with the main branch

We (I think GitHub in large) prefer merge. You may be coming from GitLab where their UI promotes rebase and force-push as a default?

Also, generally speaking, no need to keep the branch in sync unless something new off main is needed in your code / there is merge conflict.

(probably should document this in CONTRIBUTING.md?)

[rodja]

I just wanted to finish this review and then noticed that #5380 might be a more universal solution to your underlying problem. What do you think @Rollmops?

[Rollmops]

@rodja #5380 looks promising. As I understood, the motivation is similar. Only the approach is a little different.
Maybe you can move #5380 to Milestone 3.3 and leave this PR open?
I will try to take deeper look at #5380 asap.

[himbeles]

#5380 has now been merged for 3.4.