KeyError: 'COMP_WORDS'

[onpaws]

I'm trying to enable shell completion to a Python CLI app that does not use setuptools.

I'm on Ubuntu, using Poetry to handle deps, and PyInstaller to build a binary, AFAICT everything should be pretty vanilla.

If I understand correctly, it seems COMP_WORDS is supposed to be an env var populated by shell completion, but who makes it and how/when its consumed isn't yet clear to me.

It seems either I found found a bug in Click, or I'm misunderstanding the problem. If this is not actually a Click bug, apologies, I will close as appropriate.

minimal repro

example.py

import click

@click.command()
def hello():
    pass

hello()

$ git clone [email protected]:onpaws/click-repro.git; cd click-repro
$ poetry install
Creating virtualenv example-xPTbDj88-py3.10 in /Users/paws/Library/Caches/pypoetry/virtualenvs
Installing dependencies from lock file

Package operations: 5 installs, 0 updates, 0 removals

  • Installing altgraph (0.17.2)
  • Installing macholib (1.16)
  • Installing pyinstaller-hooks-contrib (2022.8)
  • Installing click (8.1.3)
  • Installing pyinstaller (4.5.1)

Installing the current project: example (0.1.0)

$ poetry run pyinstaller example.py 
35 INFO: PyInstaller: 4.5.1
35 INFO: Python: 3.9.4
38 INFO: Platform: Linux-5.15.0-1013-gcp-x86_64-with-glibc2.31
39 INFO: wrote /home/ubuntu/example/example.spec
41 INFO: UPX is not available.
42 INFO: Extending PYTHONPATH with paths
['/home/ubuntu/example', '/home/ubuntu/example']
161 INFO: checking Analysis
168 INFO: checking PYZ
171 INFO: checking PKG
171 INFO: Bootloader /home/ubuntu/.cache/pypoetry/virtualenvs/example-qS9psWQd-py3.9/lib/python3.9/site-packages/PyInstaller/bootloader/Linux-64bit-intel/run
171 INFO: checking EXE
172 INFO: checking COLLECT
173 INFO: Building COLLECT COLLECT-00.toc
229 INFO: Building COLLECT COLLECT-00.toc completed successfully.

Binary built in the expected way, so far so good 🎉
Run it manually like this:

$ ./dist/example/example  
# works

Now we attempt to enable the completion, per the docs:

$ _EXAMPLE_COMPLETE=bash_complete ./dist/example/example 
Traceback (most recent call last):
  File "example.py", line 7, in <module>
    hello()
  File "click/core.py", line 1130, in __call__
  File "click/core.py", line 1050, in main
  File "click/core.py", line 1125, in _main_shell_completion
  File "click/shell_completion.py", line 49, in shell_complete
  File "click/shell_completion.py", line 290, in complete
  File "click/shell_completion.py", line 330, in get_completion_args
  File "os.py", line 679, in __getitem__
KeyError: 'COMP_WORDS'
[1072535] Failed to execute script 'example' due to unhandled exception!

It's my first time using this feature so I'm not sure what to expect exactly (i think some bash?) but definitely not an error. Did I fail to provide something?

Thanks for any pointers :)

Environment

$ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.4 LTS"

$ bash --version
GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu)

$ python --version
Python 3.9.4

$ poetry show click
name         : click
version      : 8.1.3
description  : Composable command line interface toolkit

dependencies
 - colorama *

[ssbarnea]

@davidism I am curious why the original bug report was converted into a discussion without any explanation, especially as it did not look a poorly described. I found myself being hit by the save bug many months after.

Doing a simple search on source code, is not hard to see how an unhandled exception might occur when COMP_WORDS is not defined:

src/click/shell_completion.py:330:47:        cwords = split_arg_string(os.environ["COMP_WORDS"])
src/click/shell_completion.py:352:47:        cwords = split_arg_string(os.environ["COMP_WORDS"])
src/click/shell_completion.py:374:47:        cwords = split_arg_string(os.environ["COMP_WORDS"])

[onpaws]

Thanks for sharing your experience @ssbarnea .

I'm sorry to say last summer I ended up abandoning my efforts with click because tracking this down took more time than I had.
"Smart" tab completion remains one of my favorite aspects of using the CLI, so I remain hopeful someone cleverer than me can share knowledge on whatever this issue is🤞

[voneiden]

@onpaws @ssbarnea

The issue is ultimately brain malfunction while consuming the docs. Don't ask me how I know 😂

The docs tell (also in 2023) to use

_FOO_BAR_COMPLETE=bash_source

however as we can see, in this issue report bash_complete was used instead of bash_source. source is for generating the completion source code (which is what you want), complete is what the shell is supposed to call behind the scenes during active completion. The COMP_WORDS and COMP_CWORD environment variables exist only in the context of an active completion operation by the shell.