Add files via upload by wpadim · Pull Request #258 · FlagAI-Open/OpenSeek

wpadim · 2026-05-29T03:11:08Z

No description provided.

gemini-code-assist

Code Review

This pull request configures the annotation pipeline for Huawei Ascend by switching the active annotator, updating model and dataset paths to absolute paths under /root, reducing the number of ICL examples to 50, and adding special handling for Task 8 (Triton code generation) to return raw model output. It also introduces a requirements.txt file. Feedback highlights critical issues: switching back to NVIDIA annotation will cause a runtime TypeError and fail on Task 8 because annotate_nvidia was not updated to accept task_id or handle Task 8. Additionally, the requirements.txt specifies non-existent versions of torch and torch_npu (>=2.8.0), which will break installation. Finally, several hardcoded absolute paths should be replaced with dynamic path resolution or environment variables to improve portability.

gemini-code-assist · 2026-05-29T03:12:33Z

+        #     prediction = annotate(input_prompt, task_id)
        #     test_record['prediction'] = prediction
-        prediction = annotate(input_prompt)
+        prediction = annotate(input_prompt, task_id)


Toggling the active annotation function to annotate_nvidia (by uncommenting line 9 and commenting line 10) will cause a runtime TypeError because annotate_nvidia does not accept the task_id parameter.

To ensure compatibility, please update the signature of annotate_nvidia in method.py to accept task_id: int = None as well, even if it is not used inside the function.

gemini-code-assist · 2026-05-29T03:12:33Z

    return prediction

-def annotate_ascend(input_prompt:str)->list[str]:
+def annotate_ascend(input_prompt:str, task_id:int=None)->list[str]:


While adding task_id support to annotate_ascend to handle Task 8 (returning raw Triton code), annotate_nvidia was not updated. If a user switches to annotate_nvidia for Task 8, it will fail to return the raw code because it lacks the task_id check and will instead run count_answer (which returns None since Task 8 output has no <label> tags).

Please update annotate_nvidia to also support task_id and handle Task 8 similarly.

gemini-code-assist · 2026-05-29T03:12:33Z

+torch>=2.8.0
+torch_npu>=2.8.0


PyTorch version 2.8.0 does not exist on PyPI (as of early 2025, the latest stable versions are in the 2.5.x/2.6.x range). Specifying torch>=2.8.0 and torch_npu>=2.8.0 will cause pip install -r requirements.txt to fail with a resolution error. Please update these to the correct, existing versions compatible with your Huawei Ascend environment (e.g., 2.4.0 or 2.1.0).

gemini-code-assist · 2026-05-29T03:12:33Z

+    1: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-1_closest_integers.json',
+    2: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-2_count_nouns_verbs.json',
+    3: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-3_collatz_conjecture.json',
+    4: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-4_conala_concat_strings.json',
+    5: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-5_semeval_2018_task1_tweet_sadness_detection.json',
+    6: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-6_mnli_same_genre_classification.json',
+    7: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-7_jeopardy_answer_generation_all.json',
+    8: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-8_kernel_generation.json',


Hardcoding absolute paths like /root/OpenSeek/... makes the codebase non-portable and will break on any other environment. It is highly recommended to resolve these paths dynamically relative to the current file's directory.

Suggested change

1: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-1_closest_integers.json',

2: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-2_count_nouns_verbs.json',

3: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-3_collatz_conjecture.json',

4: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-4_conala_concat_strings.json',

5: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-5_semeval_2018_task1_tweet_sadness_detection.json',

6: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-6_mnli_same_genre_classification.json',

7: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-7_jeopardy_answer_generation_all.json',

8: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-8_kernel_generation.json',

1: os.path.join(os.path.dirname(__file__), '../data/openseek-1_closest_integers.json'),

2: os.path.join(os.path.dirname(__file__), '../data/openseek-2_count_nouns_verbs.json'),

3: os.path.join(os.path.dirname(__file__), '../data/openseek-3_collatz_conjecture.json'),

4: os.path.join(os.path.dirname(__file__), '../data/openseek-4_conala_concat_strings.json'),

5: os.path.join(os.path.dirname(__file__), '../data/openseek-5_semeval_2018_task1_tweet_sadness_detection.json'),

6: os.path.join(os.path.dirname(__file__), '../data/openseek-6_mnli_same_genre_classification.json'),

7: os.path.join(os.path.dirname(__file__), '../data/openseek-7_jeopardy_answer_generation_all.json'),

8: os.path.join(os.path.dirname(__file__), '../data/openseek-8_kernel_generation.json'),

gemini-code-assist · 2026-05-29T03:12:33Z

                        help='Prefix path to save the evaluation logs.')
    parser.add_argument('--tokenizer_path', type=str,
-                        default='/share/project/wuhaiming/spaces/data_agent/OpenSeek-main/openseek/competition/LongContext-ICL-Annotation/src/Qwen3-4B')
+                        default='/root/Qwen3-4B')


Hardcoding the absolute path /root/Qwen3-4B as the default tokenizer path reduces portability. Consider using an environment variable with a sensible relative default instead.

Suggested change

default='/root/Qwen3-4B')

default=os.environ.get('TOKENIZER_PATH', './Qwen3-4B'))

gemini-code-assist · 2026-05-29T03:12:34Z

    # 初始化Qwen3-4B的tokenizer（自动下载/加载千问3-4B的分词器）
    # 若本地已下载模型，可替换为本地路径，如 "./qwen3-4b"
-    tokenizer = AutoTokenizer.from_pretrained("/share/project/wuhaiming/spaces/data_agent/OpenSeek-main/openseek/competition/LongContext-ICL-Annotation/src/Qwen3-4B", trust_remote_code=True)
+    tokenizer = AutoTokenizer.from_pretrained("/root/Qwen3-4B", trust_remote_code=True)


Hardcoding the absolute path /root/Qwen3-4B reduces portability. Consider using an environment variable with a relative path fallback.

Suggested change

tokenizer = AutoTokenizer.from_pretrained("/root/Qwen3-4B", trust_remote_code=True)

import os

tokenizer = AutoTokenizer.from_pretrained(os.environ.get("TOKENIZER_PATH", "./Qwen3-4B"), trust_remote_code=True)

gemini-code-assist · 2026-05-29T03:12:34Z

    openai.api_key = "EMPTY"
    openai.base_url = "http://localhost:9010/v1/"
-    model = "Qwen3-4B-ascend-flagos"
+    model = "/root/Qwen3-4B"


Hardcoding the absolute path /root/Qwen3-4B as the model name reduces portability. Consider using an environment variable with a fallback.

Suggested change

model = "/root/Qwen3-4B"

import os

model = os.environ.get("MODEL_PATH", "/root/Qwen3-4B")

Add files via upload

7986f56

gemini-code-assist Bot reviewed May 29, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add files via upload#258

Add files via upload#258
wpadim wants to merge 1 commit into
FlagAI-Open:mainfrom
wpadim:main

wpadim commented May 29, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

gemini-code-assist Bot May 29, 2026

Uh oh!

gemini-code-assist Bot May 29, 2026

Uh oh!

gemini-code-assist Bot May 29, 2026

Uh oh!

gemini-code-assist Bot May 29, 2026

Uh oh!

gemini-code-assist Bot May 29, 2026

Uh oh!

gemini-code-assist Bot May 29, 2026

Uh oh!

gemini-code-assist Bot May 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

	default='/root/Qwen3-4B')
	default=os.environ.get('TOKENIZER_PATH', './Qwen3-4B'))

	tokenizer = AutoTokenizer.from_pretrained("/root/Qwen3-4B", trust_remote_code=True)
	import os
	tokenizer = AutoTokenizer.from_pretrained(os.environ.get("TOKENIZER_PATH", "./Qwen3-4B"), trust_remote_code=True)

	model = "/root/Qwen3-4B"
	import os
	model = os.environ.get("MODEL_PATH", "/root/Qwen3-4B")

		torch>=2.8.0
		torch_npu>=2.8.0

Uh oh!

Conversation

wpadim commented May 29, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot May 29, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 29, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 29, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 29, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 29, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 29, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 29, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant