Recommended `vite` usage significantly hurts build performance

[davidtaylorhq]

If we follow the import.meta.glob(...) advice in the README, then we're creating a dynamic entrypoint for every single test file in the app. In large apps, that can be hundreds/thousands of files. Introducing that many entrypoints can cause a massive hit to rollup/rolldown build performance.

Multiple entrypoints are necessary because Ember Exam needs to control the execution of the QUnit.* methods for filtering & load balancing. Our workaround has been to wrap all our test modules in a default-exported function, add eager: true to the import.meta.glob, and then 'trick' Ember Exam into running those functions when it asks for the module.

const rawModules = import.meta.glob("./**/*-test.{gjs,js}", {
  eager: true, // << this is the important bit for improved build performance
});

examModules = {};
for (const [key, value] of Object.entries(rawModules)) {
  examModules[key] = value.default; // `.default` returns our wrapper functions
}

startEmberExam({
  availableModules: examModules,
  // ...
});

This works because Ember Exam expects availableModules to be a map of moduleName => function. It doesn't care whether the function is async-or-not, and doesn't care about the return value.

For reference, this is the rolldown plugin we use to wrap the modules automatically. The same effect could be achieved by making this change in the actual source files:

Rolldown test module wrapping plugin

const TEST_FILE_RE = /tests\/(?!helpers\/).*-test\.(?:gjs|js)$/;

// Technically not 100% safe. Would be tripped up by top-level-await, imports in weird places, etc.
export default function wrapTestModulesPlugin() {
  return {
    name: "wrap-test-modules",
    transform: {
      filter: { id: TEST_FILE_RE },
      handler(code, id, { magicString }) {
        const ast = this.parse(code);

        let lastImportEnd = 0;
        for (const node of ast.body) {
          if (node.type === "ImportDeclaration") {
            lastImportEnd = node.end;
          }
        }

        if (lastImportEnd >= code.length) {
          return null;
        }

        magicString.appendLeft(
          lastImportEnd,
          "\n\nexport default function () {\n"
        );
        magicString.append("\n}\n");

        return {
          code: magicString,
        };
      },
    },
  };
}

In Discourse, this workaround cuts around 680 entrypoints from our build, and reduces overall Rolldown build time by almost 30% (from ~7s to ~5s in my A/B test just now).

Clearly... the workaround is not ideal. But I wanted to share our findings in case it helps anyone else, and so that it can be a starting point for a more 'official' solution to the problem.

[NullVoxPopuli]

Ember exam should probably be able to handle both eager and non eager.

At work, we also have thousands of tests, but opted for having all the entrypoints, because the tests can pull an massive dep subgraphs which then slow down dev time things. Since devs never rut the full test suite, 🙈, this is the tradeoff. Def can be improved generally tho

[davidtaylorhq]

Ember exam should probably be able to handle both eager and non eager.

That would be great, but it's tricky because Ember Exam is just wrapping around QUnit. QUnit itself has no visibility of module names/paths for filtering. Once the test modules are evaluated, they're in QUnit's queue.

Actually authoring tests inside an export default function() { could be a solution... but it's quite a big change.

the tests can pull an massive dep subgraphs which then slow down dev time things

We do still have a separate entrypoint for "tests", so it doesn't impact general development. But it's just one entrypoint for all qunit tests, instead of one-entrypoint-per-test-file.