[refactor] fix third_party issue

[SkqLiao]

What does this PR do?

Fixes # (issue)

Before submitting

• Did you read the contributor guideline?
• Did you write any new necessary tests?

[gemini-code-assist]

Summary of Changes

Hello @SkqLiao, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request refactors the project's external dependency management by relocating existing third_party submodules to a more central location and integrating a new custom_flashinfer submodule. It also simplifies the install.sh script by removing automated dependency installation and updating build command examples, along with minor code formatting improvements in AMX-related test files.

Highlights

• Restructured third_party dependencies: The llama.cpp and pybind11 submodules have been moved from kt-kernel/third_party to a top-level third_party directory. This involved updating their paths in .gitmodules and CMakeLists.txt.
• Introduced custom_flashinfer submodule: A new submodule, custom_flashinfer, has been added to the third_party directory, tracking a specific branch from kvcache-ai/custom_flashinfer.git.
• Simplified install.sh script: The install_dependencies function and the --skip-deps option have been removed from install.sh, streamlining the build process. Usage examples and documentation for CPU instruction sets (including a new FANCY option) have also been updated.
• Minor code style adjustment: Braces were added around ggml_amx_init() calls in mmq-test.cpp and mmq.cpp for consistent code formatting.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request primarily refactors the project structure by moving the third_party directory to the repository root, which is a good architectural improvement. It also introduces a new submodule for custom_flashinfer and improves the install.sh script by making dependency installation part of explicit subcommands rather than an automatic action. The changes are generally positive, but I have one recommendation regarding the new submodule to ensure reproducible builds. The pull request description is quite sparse; for future changes of this magnitude, a more detailed explanation would be beneficial for reviewers.

[gemini-code-assist]

The new submodule custom_flashinfer is configured to track the fix-precision-mla-merge-main branch. For better build reproducibility and stability, it is strongly recommended to pin submodules to a specific commit hash rather than a branch. Branches can be updated, which could introduce breaking changes unexpectedly.

To fix this, you can go into the third_party/custom_flashinfer directory, check out the exact commit you want to use, then return to the parent repository's root and run git add third_party/custom_flashinfer. This will stage the submodule at that specific commit.