seg fault / running GC on python 3.11

[danielebriggi]

On SQLite rag, use python 3.11 and run tests: pytest --cov --cov-branch --cov-report=xml -v -m "not slow" ./tests.
At the end of the test, often but not always, I get:

tests/test_sqlite_rag.py::TestSQLiteRagSearch::test_search_uses_retrieval_query_template PASSED [ 97%]
tests/test_sqlite_rag.py::TestSQLiteRagSearch::test_search_with_prompt_template[True] PASSED [ 98%]
Fatal Python error: Segmentation fault

Current thread 0x00007f6db2bc6b80 (most recent call first):
  Garbage-collecting
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/xml/dom/minidom.py", line 372 in _get_localName
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/xml/dom/minidom.py", line 793 in setAttributeNode
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/xml/dom/minidom.py", line 753 in setAttribute
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/coverage/xmlreport.py", line 221 in xml_file
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/coverage/xmlreport.py", line 110 in report
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/coverage/report_core.py", line 59 in render_report
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/coverage/control.py", line 1281 in xml_report
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/pytest_cov/engine.py", line 248 in summary
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/pytest_cov/engine.py", line 57 in ensure_topdir_wrapper
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/pytest_cov/plugin.py", line 368 in pytest_runtestloop
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/pluggy/_callers.py", line 152 in _multicall
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/pluggy/_hooks.py", line 512 in __call__
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/_pytest/main.py", line 343 in _main
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/_pytest/main.py", line 289 in wrap_session
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/_pytest/main.py", line 336 in pytest_cmdline_main
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/pluggy/_callers.py", line 121 in _multicall
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/pluggy/_hooks.py", line 512 in __call__
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/_pytest/config/__init__.py", line 175 in main
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/_pytest/config/__init__.py", line 201 in console_main
  File "/opt/hostedtoolcache/Python/3.11.13/x64/bin/pytest", line 7 in <module>

Extension modules: charset_normalizer.md, requests.packages.charset_normalizer.md, requests.packages.chardet.md, numpy._core._multiarray_umath, numpy.linalg._umath_linalg, lxml._elementpath, lxml.etree, _cffi_backend, pyarrow.lib, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._pcg64, numpy.random._mt19937, numpy.random._generator, numpy.random._philox, numpy.random._sfc64, numpy.random.mtrand, pandas._libs.tslibs.ccalendar, pandas._libs.tslibs.np_datetime, pandas._libs.tslibs.dtypes, pandas._libs.tslibs.base, pandas._libs.tslibs.nattype, pandas._libs.tslibs.timezones, pandas._libs.tslibs.fields, pandas._libs.tslibs.timedeltas, pandas._libs.tslibs.tzconversion, pandas._libs.tslibs.timestamps, pandas._libs.properties, pandas._libs.tslibs.offsets, pandas._libs.tslibs.strptime, pandas._libs.tslibs.parsing, pandas._libs.tslibs.conversion, pandas._libs.tslibs.period, pandas._libs.tslibs.vectorized, pandas._libs.ops_dispatch, pandas._libs.missing, pandas._libs.hashtable, pandas._libs.algos, pandas._libs.interval, pandas._libs.lib, pyarrow._compute, pandas._libs.ops, pandas._libs.hashing, pandas._libs.arrays, pandas._libs.tslib, pandas._libs.sparse, pandas._libs.internals, pandas._libs.indexing, pandas._libs.index, pandas._libs.writers, pandas._libs.join, pandas._libs.window.aggregations, pandas._libs.window.indexers, pandas._libs.reshape, pandas._libs.groupby, pandas._libs.json, pandas._libs.parsers, pandas._libs.testing, PIL._imaging, PIL._imagingft (total: 61)
/home/runner/work/_temp/165c415c-283c-4e3d-89e5-d308fa0def8c.sh: line 1:  2071 Segmentation fault      (core dumped) pytest --cov --cov-branch --cov-report=xml -v -m "not slow" ./tests
tests/test_sqlite_rag.py::TestSQLiteRagSearch::test_search_with_prompt_template[False] PASSED [100%]

Sometimes I also see this message: Error executing INSERT INTO ai_log (type, message) VALUES (?, ?); in db_write (bad parameter or other API misuse).Error executing INSERT INTO ai_log (type, message) VALUES (?, ?); in db_write (bad parameter or other API misuse).

I solved the SEG Fault by helping Python to llm_model_free and close db connection during GC.

[andinux]

this segmentation fault should have been fixed with the release https://github.com/sqliteai/sqlite-ai/releases/tag/0.7.58, in particulare with commit c3fd345

• fix(chat): remove the duplicate_content arg in llm_messages_append to avoid a crash on llm_chat_free()
  Refactored llm_messages_append to always duplicate message content using sqlite_strdup, removing the duplicate_content parameter. This option was causing a double-free crash in llm_chat_free with non-duplicated content. Also, the ai->chat.response buffer is reset on new prompts in the same chat so we cannot rely on that buffer to store previous response messages.

I ran pytest --cov --cov-branch --cov-report=xml -v -m "not slow" ./tests several times in the Dev Container with python 3.11 and I never got the seg fault.
@danielebriggi please confirm that the seg fault doesn't happen anymore to close the this issue.

[danielebriggi]

After running tests, I close the connection to the database. If I do so, without executing SELECT llm_model_free(); I still run into SEG FAULT in py3.11 even with v0.7.58.
If I do not execute the close() connection and I do not execute SELECT llm_model_free(); I do not fall into SEG FAULT.
The best practice is already place in the test suite (close and free model) but the SEG FAULT should not happen anyway