Git dependency install with insteadOf URL fails with poetry>2 #10167
Description
Description
When installing a private Git dependencies of a project, with a global Git configuration that uses url.insteadOf the install fails with a dulwich error. This is a setup in GitLab CI pipelines to install dependencies from internal Git repositories without setting up / deploying to additional PyPI repositories.
HangupException
The remote server unexpectedly closed the connection.
at ~/.local/pipx/venvs/poetry/lib/python3.10/site-packages/dulwich/protocol.py:219 in read_pkt_line
215│
216│ try:
217│ sizestr = read(4)
218│ if not sizestr:
→ 219│ raise HangupException()
220│ size = int(sizestr, 16)
221│ if size == 0:
222│ if self.report_activity:
223│ self.report_activity(4, "read")See Poetry runtime logs for full traceback.
In the CI pipeline the Git URL replacement is configured in the before_script section using:
git config --global url."https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.acme.com/".insteadOf "[email protected]:"When reproducing the issue manually, poetry prompts for a host key verification which indicates that SSH is used for cloning the package instead of HTTPS as configured in the gitconfig.
The authenticity of host 'gitlab.acme.com (172.23.1.12)' can't be established.
ED25519 key fingerprint is SHA256:cjWQPU5PdrmIwNFk61/cRzi/lXg2n18evVMojxsHfww.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? noWhen selecting no, the full traceback from the Poetry runtime logs is visible.
Workarounds
Downgrade to poetry < 2
Poetry Installation Method
pipx
Operating System
Ubuntu 22.04
Poetry Version
Poetry (version 2.0.1)
Poetry Configuration
cache-dir = "/root/.cache/pypoetry"
installer.max-workers = null
installer.no-binary = null
installer.only-binary = null
installer.parallel = true
installer.re-resolve = true
keyring.enabled = true
requests.max-retries = 0
solver.lazy-wheel = true
system-git-client = false
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}/virtualenvs" # /root/.cache/pypoetry/virtualenvs
virtualenvs.prompt = "{project_name}-py{python_version}"
virtualenvs.use-poetry-python = falsePython Sysconfig
sysconfig.log
Paste the output of 'python -m sysconfig', over this line.
Example pyproject.toml
Poetry Runtime Logs
poetry-runtime.log
root@fe2b0c78534d:/workspaces/myrepo# poetry install -vvv
Loading configuration file /root/.config/pypoetry/config.toml
Trying to detect current active python executable as specified in the config.
Found: /usr/local/bin/python
Trying to detect current active python executable as specified in the config.
Found: /usr/local/bin/python
Creating virtualenv myrepo-dB_rVOTu-py3.11 in /root/.cache/pypoetry/virtualenvs
[virtualenv:virtualenv.discovery.builtin] find interpreter for spec PythonSpec(path=/root/.local/pipx/venvs/poetry/bin/python)
[virtualenv:virtualenv.info] filesystem is case-sensitive
[filelock:filelock] Attempting to acquire lock 139951967103376 on /root/.local/share/virtualenv/py_info/2/1a1af0ee75eeea9e2e1ee996c87e7a2b11a0bebd85af04bb136d915cefc0abce.lock
[filelock:filelock] Lock 139951967103376 acquired on /root/.local/share/virtualenv/py_info/2/1a1af0ee75eeea9e2e1ee996c87e7a2b11a0bebd85af04bb136d915cefc0abce.lock
[virtualenv:virtualenv.app_data.via_disk_folder] got python info of %s from (PosixPath('/usr/local/bin/python3.11'), PosixPath('/root/.local/share/virtualenv/py_info/2/1a1af0ee75eeea9e2e1ee996c87e7a2b11a0bebd85af04bb136d915cefc0abce
.json'))
[filelock:filelock] Attempting to release lock 139951967103376 on /root/.local/share/virtualenv/py_info/2/1a1af0ee75eeea9e2e1ee996c87e7a2b11a0bebd85af04bb136d915cefc0abce.lock
[filelock:filelock] Lock 139951967103376 released on /root/.local/share/virtualenv/py_info/2/1a1af0ee75eeea9e2e1ee996c87e7a2b11a0bebd85af04bb136d915cefc0abce.lock
[virtualenv:virtualenv.discovery.builtin] proposed PythonInfo(spec=CPython3.11.11.final.0-64, exe=/usr/local/bin/python3.11, platform=linux, version='3.11.11 (main, Feb 4 2025, 04:55:09) [GCC 12.2.0]', encoding_fs_io=utf-8-utf-8)
[virtualenv:virtualenv.discovery.builtin] accepted PythonInfo(spec=CPython3.11.11.final.0-64, exe=/usr/local/bin/python3.11, platform=linux, version='3.11.11 (main, Feb 4 2025, 04:55:09) [GCC 12.2.0]', encoding_fs_io=utf-8-utf-8)
[virtualenv:virtualenv.run.session] create virtual environment via CPython3Posix(dest=/root/.cache/pypoetry/virtualenvs/myrepo-dB_rVOTu-py3.11, clear=False, no_vcs_ignore=False, global=False)
[virtualenv:virtualenv.util.path._sync] create folder /root/.cache/pypoetry/virtualenvs/myrepo-dB_rVOTu-py3.11/bin
[virtualenv:virtualenv.util.path._sync] create folder /root/.cache/pypoetry/virtualenvs/myrepo-dB_rVOTu-py3.11/lib/python3.11/site-packages
[virtualenv:virtualenv.create.pyenv_cfg] write /root/.cache/pypoetry/virtualenvs/myrepo-dB_rVOTu-py3.11/pyvenv.cfg
[virtualenv:virtualenv.create.pyenv_cfg] home = /usr/local/bin
[virtualenv:virtualenv.create.pyenv_cfg] implementation = CPython
[virtualenv:virtualenv.create.pyenv_cfg] version_info = 3.11.11.final.0
[virtualenv:virtualenv.create.pyenv_cfg] virtualenv = 20.29.1
[virtualenv:virtualenv.create.pyenv_cfg] include-system-site-packages = false
[virtualenv:virtualenv.create.pyenv_cfg] base-prefix = /usr/local
[virtualenv:virtualenv.create.pyenv_cfg] base-exec-prefix = /usr/local
[virtualenv:virtualenv.create.pyenv_cfg] base-executable = /usr/local/bin/python3.11
[virtualenv:virtualenv.util.path._sync] symlink /usr/local/bin/python3.11 to /root/.cache/pypoetry/virtualenvs/myrepo-dB_rVOTu-py3.11/bin/python
[virtualenv:virtualenv.create.via_global_ref.api] create virtualenv import hook file /root/.cache/pypoetry/virtualenvs/myrepo-dB_rVOTu-py3.11/lib/python3.11/site-packages/_virtualenv.pth
[virtualenv:virtualenv.create.via_global_ref.api] create /root/.cache/pypoetry/virtualenvs/myrepo-dB_rVOTu-py3.11/lib/python3.11/site-packages/_virtualenv.py
[virtualenv:virtualenv.run.session] ============================== target debug ==============================
[virtualenv:virtualenv.create.creator] debug via /root/.cache/pypoetry/virtualenvs/myrepo-dB_rVOTu-py3.11/bin/python /root/.local/pipx/venvs/poetry/lib/python3.11/site-packages/virtualenv/create/debug.py
[virtualenv:virtualenv.run.session] {
[virtualenv:virtualenv.run.session] "sys": {
[virtualenv:virtualenv.run.session] "executable": "/root/.cache/pypoetry/virtualenvs/myrepo-dB_rVOTu-py3.11/bin/python",
[virtualenv:virtualenv.run.session] "_base_executable": "/usr/local/bin/python3.11",
[virtualenv:virtualenv.run.session] "prefix": "/root/.cache/pypoetry/virtualenvs/myrepo-dB_rVOTu-py3.11",
[virtualenv:virtualenv.run.session] "base_prefix": "/usr/local",
[virtualenv:virtualenv.run.session] "real_prefix": null,
[virtualenv:virtualenv.run.session] "exec_prefix": "/root/.cache/pypoetry/virtualenvs/myrepo-dB_rVOTu-py3.11",
[virtualenv:virtualenv.run.session] "base_exec_prefix": "/usr/local",
[virtualenv:virtualenv.run.session] "path": [
[virtualenv:virtualenv.run.session] "/usr/local/lib/python311.zip",
[virtualenv:virtualenv.run.session] "/usr/local/lib/python3.11",
[virtualenv:virtualenv.run.session] "/usr/local/lib/python3.11/lib-dynload",
[virtualenv:virtualenv.run.session] "/root/.cache/pypoetry/virtualenvs/myrepo-dB_rVOTu-py3.11/lib/python3.11/site-packages"
[virtualenv:virtualenv.run.session] ],
[virtualenv:virtualenv.run.session] "meta_path": [
[virtualenv:virtualenv.run.session] "<class '_virtualenv._Finder'>",
[virtualenv:virtualenv.run.session] "<class '_frozen_importlib.BuiltinImporter'>",
[virtualenv:virtualenv.run.session] "<class '_frozen_importlib.FrozenImporter'>",
[virtualenv:virtualenv.run.session] "<class '_frozen_importlib_external.PathFinder'>"
[virtualenv:virtualenv.run.session] ],
[virtualenv:virtualenv.run.session] "fs_encoding": "utf-8",
[virtualenv:virtualenv.run.session] "io_encoding": "utf-8"
[virtualenv:virtualenv.run.session] },
[virtualenv:virtualenv.run.session] "version": "3.11.11 (main, Feb 4 2025, 04:55:09) [GCC 12.2.0]",
[virtualenv:virtualenv.run.session] "makefile_filename": "/usr/local/lib/python3.11/config-3.11-x86_64-linux-gnu/Makefile",
[virtualenv:virtualenv.run.session] "os": "<module 'os' (frozen)>",
[virtualenv:virtualenv.run.session] "site": "<module 'site' (frozen)>",
[virtualenv:virtualenv.run.session] "datetime": "<module 'datetime' from '/usr/local/lib/python3.11/datetime.py'>",
[virtualenv:virtualenv.run.session] "math": "<module 'math' from '/usr/local/lib/python3.11/lib-dynload/math.cpython-311-x86_64-linux-gnu.so'>",
[virtualenv:virtualenv.run.session] "json": "<module 'json' from '/usr/local/lib/python3.11/json/__init__.py'>"
[virtualenv:virtualenv.run.session] }
[virtualenv:virtualenv.run.session] add seed packages via FromAppData(download=False, pip=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv)
[virtualenv:virtualenv.seed.embed.via_app_data.via_app_data] install pip from wheel /root/.local/pipx/venvs/poetry/lib/python3.11/site-packages/virtualenv/seed/wheels/embed/pip-24.3.1-py3-none-any.whl via CopyPipInstall
[filelock:filelock] Attempting to acquire lock 139951964774032 on /root/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/pip-24.3.1-py3-none-any.lock
[filelock:filelock] Lock 139951964774032 acquired on /root/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/pip-24.3.1-py3-none-any.lock
[filelock:filelock] Attempting to release lock 139951964774032 on /root/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/pip-24.3.1-py3-none-any.lock
[filelock:filelock] Lock 139951964774032 released on /root/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/pip-24.3.1-py3-none-any.lock
[virtualenv:virtualenv.util.path._sync] copy directory /root/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/pip-24.3.1-py3-none-any/pip to /root/.cache/pypoetry/virtualenvs/myrepo-dB_rVOTu-py3.11/lib/python3.11/site-packa
ges/pip
[virtualenv:virtualenv.util.path._sync] copy directory /root/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/pip-24.3.1-py3-none-any/pip-24.3.1.dist-info to /root/.cache/pypoetry/virtualenvs/myrepo-dB_rVOTu-py3.11/lib/pyth
on3.11/site-packages/pip-24.3.1.dist-info
[virtualenv:virtualenv.util.path._sync] copy /root/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/pip-24.3.1-py3-none-any/pip-24.3.1.virtualenv to /root/.cache/pypoetry/virtualenvs/myrepo-dB_rVOTu-py3.11/lib/python3.11/si
te-packages/pip-24.3.1.virtualenv
[distlib:distlib.util] changing mode of /root/.cache/pypoetry/virtualenvs/myrepo-dB_rVOTu-py3.11/bin/pip-3.11 to 755
[distlib:distlib.util] changing mode of /root/.cache/pypoetry/virtualenvs/myrepo-dB_rVOTu-py3.11/bin/pip3 to 755
[distlib:distlib.util] changing mode of /root/.cache/pypoetry/virtualenvs/myrepo-dB_rVOTu-py3.11/bin/pip3.11 to 755
[distlib:distlib.util] changing mode of /root/.cache/pypoetry/virtualenvs/myrepo-dB_rVOTu-py3.11/bin/pip to 755
[virtualenv:virtualenv.seed.embed.via_app_data.pip_install.base] generated console scripts pip-3.11 pip3.11 pip pip3
[virtualenv:virtualenv.run.session] add activators for Bash, CShell, Fish, Nushell, PowerShell, Python
[virtualenv:virtualenv.create.pyenv_cfg] write /root/.cache/pypoetry/virtualenvs/myrepo-dB_rVOTu-py3.11/pyvenv.cfg
[virtualenv:virtualenv.create.pyenv_cfg] home = /usr/local/bin
[virtualenv:virtualenv.create.pyenv_cfg] implementation = CPython
[virtualenv:virtualenv.create.pyenv_cfg] version_info = 3.11.11.final.0
[virtualenv:virtualenv.create.pyenv_cfg] virtualenv = 20.29.1
[virtualenv:virtualenv.create.pyenv_cfg] include-system-site-packages = false
[virtualenv:virtualenv.create.pyenv_cfg] base-prefix = /usr/local
[virtualenv:virtualenv.create.pyenv_cfg] base-exec-prefix = /usr/local
[virtualenv:virtualenv.create.pyenv_cfg] base-executable = /usr/local/bin/python3.11
[virtualenv:virtualenv.create.pyenv_cfg] prompt = myrepo-py3.11
Using virtualenv: /root/.cache/pypoetry/virtualenvs/myrepo-dB_rVOTu-py3.11
Updating dependencies
Resolving dependencies...
1: fact: myrepo is 0.1.0
1: derived: myrepo
The authenticity of host 'gitlab.acme.com (172.23.1.12)' can't be established.
ED25519 key fingerprint is SHA256:cjWQPU5PdrmIwNFk61/cRzi/lXg2n18evVMojxsHfww.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? no
1: Version solving took 70.241 seconds.
1: Tried 1 solutions.
Stack trace:
26 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/cleo/application.py:327 in run
325│
326│ try:
→ 327│ exit_code = self._run(io)
328│ except BrokenPipeError:
329│ # If we are piped to another process, it may close early and send a
25 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/console/application.py:236 in _run
234│
235│ with directory(self._working_directory):
→ 236│ exit_code: int = super()._run(io)
237│
238│ return exit_code
24 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/cleo/application.py:431 in _run
429│ io.input.interactive(interactive)
430│
→ 431│ exit_code = self._run_command(command, io)
432│ self._running_command = None
433│
23 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/cleo/application.py:473 in _run_command
471│
472│ if error is not None:
→ 473│ raise error
474│
475│ return terminate_event.exit_code
22 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/cleo/application.py:457 in _run_command
455│
456│ if command_event.command_should_run():
→ 457│ exit_code = command.run(io)
458│ else:
459│ exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED
21 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/cleo/commands/base_command.py:117 in run
115│ io.input.validate()
116│
→ 117│ return self.execute(io) or 0
118│
119│ def merge_application_definition(self, merge_args: bool = True) -> None:
20 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/cleo/commands/command.py:61 in execute
59│
60│ try:
→ 61│ return self.handle()
62│ except KeyboardInterrupt:
63│ return 1
19 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/console/commands/install.py:173 in handle
171│ self.installer.verbose(self.io.is_verbose())
172│
→ 173│ return_code = self.installer.run()
174│
175│ if return_code != 0:
18 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/installation/installer.py:103 in run
101│ self.verbose(True)
102│
→ 103│ return self._do_install()
104│
105│ def dry_run(self, dry_run: bool = True) -> Installer:
17 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/installation/installer.py:241 in _do_install
239│ source_root=self._env.path.joinpath("src")
240│ ):
→ 241│ solved_packages = solver.solve(
242│ use_latest=self._whitelist
243│ ).get_solved_packages()
16 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/puzzle/solver.py:85 in solve
83│ with self._progress(), self._provider.use_latest_for(use_latest or []):
84│ start = time.time()
→ 85│ packages = self._solve()
86│ # simplify markers by removing redundant information
87│ for transitive_info in packages.values():
15 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/puzzle/solver.py:166 in _solve
164│
165│ try:
→ 166│ result = resolve_version(self._package, self._provider)
167│
168│ packages = result.packages
14 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/mixology/__init__.py:18 in resolve_version
16│ solver = VersionSolver(root, provider)
17│
→ 18│ return solver.solve()
19│
13 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/mixology/version_solver.py:174 in solve
172│ while next is not None:
173│ self._propagate(next)
→ 174│ next = self._choose_package_version()
175│
176│ return self._result()
12 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/mixology/version_solver.py:514 in _choose_package_version
512│ package = locked
513│
→ 514│ package = self._provider.complete_package(package)
515│
516│ conflict = False
11 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/puzzle/provider.py:545 in complete_package
543│ if locked is not None and locked.package.is_same_package_as(dep):
544│ continue
→ 545│ self.search_for_direct_origin_dependency(dep)
546│
547│ dependencies = self._get_dependencies_with_overrides(_dependencies, package)
10 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/puzzle/provider.py:218 in search_for_direct_origin_dependency
216│ elif dependency.is_vcs():
217│ dependency = cast("VCSDependency", dependency)
→ 218│ package = self._search_for_vcs(dependency)
219│
220│ elif dependency.is_file():
9 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/puzzle/provider.py:293 in _search_for_vcs
291│ and get the information we need by checking out the specified reference.
292│ """
→ 293│ package = self._direct_origin.get_package_from_vcs(
294│ dependency.vcs,
295│ dependency.source,
8 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/packages/direct_origin.py:116 in get_package_from_vcs
114│ raise ValueError(f"Unsupported VCS dependency {vcs}")
115│
→ 116│ return _get_package_from_git(
117│ url=url,
118│ branch=branch,
7 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/packages/direct_origin.py:34 in _get_package_from_git
32│ source_root: Path | None = None,
33│ ) -> Package:
→ 34│ source = Git.clone(
35│ url=url,
36│ source_root=source_root,
6 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/vcs/git/backend.py:453 in clone
451│ try:
452│ if not cls.is_using_legacy_client():
→ 453│ local = cls._clone(url=url, refspec=refspec, target=target)
454│ cls._clone_submodules(repo=local)
455│ return local
5 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/vcs/git/backend.py:270 in _clone
268│ local = Repo(str(target))
269│
→ 270│ remote_refs = cls._fetch_remote_refs(url=url, local=local)
271│
272│ logger.debug(
4 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/vcs/git/backend.py:215 in _fetch_remote_refs
213│
214│ with local:
→ 215│ result: FetchPackResult = client.fetch(
216│ path,
217│ local,
3 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/dulwich/client.py:981 in fetch
979│ f, commit, abort = target.object_store.add_pack()
980│ try:
→ 981│ result = self.fetch_pack(
982│ path,
983│ determine_wants,
2 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/dulwich/client.py:1352 in fetch_pack
1350│ raise ValueError(f"unknown Git protocol version {protocol_version}")
1351│ proto, can_read, stderr = self._connect(b"upload-pack", path, protocol_version)
→ 1352│ server_protocol_version = negotiate_protocol_version(proto)
1353│ if server_protocol_version not in GIT_PROTOCOL_VERSIONS:
1354│ raise ValueError(
1 ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/dulwich/client.py:259 in negotiate_protocol_version
257│
258│ def negotiate_protocol_version(proto) -> int:
→ 259│ pkt = proto.read_pkt_line()
260│ if pkt is not None and pkt.strip() == b"version 2":
261│ return 2
HangupException
The remote server unexpectedly closed the connection.
at ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/dulwich/protocol.py:255 in read_pkt_line
251│
252│ try:
253│ sizestr = read(4)
254│ if not sizestr:
→ 255│ raise HangupException
256│ size = int(sizestr, 16)
257│ if size == 0 or size == 1: # flush-pkt or delim-pkt
258│ if self.report_activity:
259│ self.report_activity(4, "read")