reference_accesses can be very slow

[sergisiso]

This is found testing just the fronted part of time psyclone ukca_aero_bug/ukca_aero_ctl.F90

On master it takes: 1m50.579s !

Most of the time is doing reference_accesses. And this is not calling it multiple times, a single call to the routine.refernce_accesses takes multiple seconds. Which is done multiple times in the fronted through the symbol_table.py::_validate_remove_routinesymbol

If I go back before all recent changes to call reference_access to look for the routine declarations, it is still slow, but an improvement.
0m52.943s

If I prune the call.reference_access altogher it's significantly faster.
0m13.700s

One think to notice is that symbol_table.py::_validate_remove_routinesymbol does not need the access types, so we could implement a faster "get_me_all_references" method wihtout the map updates. However, the problem happens again when applying transformations, there we may need them.

[sergisiso]

After #3194, almost all of the time inside reference_access comes from get_callee:

The problems seems to be (in order of importance):

• A lot of time is spent building error messages even when no error is produced! I can try to solve this by using LazyString
• The get_argument_map uses symbolic_math to do the shape comparisons (I am not sure what is the solution to this)
• The get_callees, can currently decide to go and explore other files, this is the subject of Improve handling of loading external files #3162

[sergisiso]

Another detail from the profile above. The debug_string, which has a _DISABLE_LOWERING = True, calls lower_to_language_level?

[LonelyCat124]

Another detail from the profile above. The debug_string, which has a _DISABLE_LOWERING = True, calls lower_to_language_level?

This is really weird, I can't replicate this with (e.g.) a Literal from a python terminal

[sergisiso]

Oh, you are right, adding a breakpoint for such case in never happens. I may be the profiler not using the full stack and adding frames that are inside __call__ but not inside debug_string->__call__