Add files via upload by ttstopll85 · Pull Request #257 · FlagAI-Open/OpenSeek

ttstopll85 · 2026-05-29T03:08:56Z

No description provided.

gemini-code-assist

Code Review

This pull request introduces a differentiated prompting and parameter strategy for various tasks, implementing Chain-of-Thought (CoT) prompts for mathematical and string tasks, updating parsing logic with fallback strategies, and switching the default annotation backend to Huawei Ascend. Key feedback points out a mismatch in task handling between main.py and method.py for CoT prompts, the use of hardcoded absolute paths that hinder portability, and a potential signature mismatch if switching back to Nvidia annotation. Additionally, the reviewer noted that hardcoded tokenizer paths bypass command-line arguments, type hints for several functions are incorrect, and the new API call lacks error handling.

gemini-code-assist · 2026-05-29T03:10:39Z

    )
    return prompt

+def build_prompt_cot(task_description: str, text2annotate: str, task_id: int) -> str:


There is a mismatch between main.py and method.py regarding Task 4 and Task 8.

main.py calls build_prompt_cot for task_id in [3, 4], but build_prompt_cot has no specific implementation for task_id == 4 and falls back to the standard prompt.

build_prompt_cot contains a specific CoT prompt implementation for task_id == 8, but main.py explicitly avoids calling it for Task 8 because 'CoT was harmful'.

Please align the implementations so that Task 4 has its CoT prompt implemented if intended, and remove or clarify the unused Task 8 CoT prompt.

gemini-code-assist · 2026-05-29T03:10:39Z

+    1: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-1_closest_integers.json',
+    2: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-2_count_nouns_verbs.json',
+    3: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-3_collatz_conjecture.json',
+    4: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-4_conala_concat_strings.json',
+    5: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-5_semeval_2018_task1_tweet_sadness_detection.json',
+    6: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-6_mnli_same_genre_classification.json',
+    7: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-7_jeopardy_answer_generation_all.json',
+    8: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-8_kernel_generation.json',


Using hardcoded absolute paths starting with /root/ makes the script non-portable and environment-dependent. It is highly recommended to resolve these paths dynamically relative to the script's location using os.path.

Suggested change

1: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-1_closest_integers.json',

2: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-2_count_nouns_verbs.json',

3: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-3_collatz_conjecture.json',

4: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-4_conala_concat_strings.json',

5: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-5_semeval_2018_task1_tweet_sadness_detection.json',

6: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-6_mnli_same_genre_classification.json',

7: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-7_jeopardy_answer_generation_all.json',

8: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-8_kernel_generation.json',

1: os.path.abspath(os.path.join(os.path.dirname(__file__), '../data/openseek-1_closest_integers.json')),

2: os.path.abspath(os.path.join(os.path.dirname(__file__), '../data/openseek-2_count_nouns_verbs.json')),

3: os.path.abspath(os.path.join(os.path.dirname(__file__), '../data/openseek-3_collatz_conjecture.json')),

4: os.path.abspath(os.path.join(os.path.dirname(__file__), '../data/openseek-4_conala_concat_strings.json')),

5: os.path.abspath(os.path.join(os.path.dirname(__file__), '../data/openseek-5_semeval_2018_task1_tweet_sadness_detection.json')),

6: os.path.abspath(os.path.join(os.path.dirname(__file__), '../data/openseek-6_mnli_same_genre_classification.json')),

7: os.path.abspath(os.path.join(os.path.dirname(__file__), '../data/openseek-7_jeopardy_answer_generation_all.json')),

8: os.path.abspath(os.path.join(os.path.dirname(__file__), '../data/openseek-8_kernel_generation.json')),

gemini-code-assist · 2026-05-29T03:10:39Z

+        #     prediction = annotate(input_prompt, task_id)
        #     test_record['prediction'] = prediction
-        prediction = annotate(input_prompt)
+        prediction = annotate(input_prompt, task_id)


If the user switches the active annotation function to annotate_nvidia (by uncommenting line 9 and commenting out line 10), this call will raise a TypeError because annotate_nvidia does not accept a task_id argument. Consider updating annotate_nvidia's signature in method.py to accept task_id: int = None to maintain interface compatibility, or handle it dynamically here.

Suggested change

prediction = annotate(input_prompt, task_id)

prediction = annotate(input_prompt, task_id) if annotate.__name__ == 'annotate_ascend' else annotate(input_prompt)

gemini-code-assist · 2026-05-29T03:10:39Z

    # 初始化Qwen3-4B的tokenizer（自动下载/加载千问3-4B的分词器）
    # 若本地已下载模型，可替换为本地路径，如 "./qwen3-4b"
-    tokenizer = AutoTokenizer.from_pretrained("/share/project/wuhaiming/spaces/data_agent/OpenSeek-main/openseek/competition/LongContext-ICL-Annotation/src/Qwen3-4B", trust_remote_code=True)
+    tokenizer = AutoTokenizer.from_pretrained("/root/Qwen3-4B", trust_remote_code=True)


The tokenizer path /root/Qwen3-4B is hardcoded here, which ignores the --tokenizer_path argument passed to main.py. Consider using an environment variable to allow overriding this path dynamically without changing the function signature.

Suggested change

tokenizer = AutoTokenizer.from_pretrained("/root/Qwen3-4B", trust_remote_code=True)

import os

tokenizer_path = os.getenv("TOKENIZER_PATH", "/root/Qwen3-4B")

tokenizer = AutoTokenizer.from_pretrained(tokenizer_path, trust_remote_code=True)

gemini-code-assist · 2026-05-29T03:10:39Z

 def count_answer(text: str) -> tuple[list, dict]:
    """
    提取字符串中<label>标签内的所有内容（字符串形式），统计出现次数最多的内容
+    M01 优化版本：增加兜底匹配逻辑


The function's type hint indicates it returns tuple[list, dict], but the actual implementation returns a single str (or None). Please update the type hint to str | None (or Optional[str]) to match the actual return type and avoid confusing callers or static analysis tools.

gemini-code-assist · 2026-05-29T03:10:39Z

    return prediction

-def annotate_ascend(input_prompt:str)->list[str]:
+def annotate_ascend(input_prompt:str, task_id:int=None)->list[str]:


The function's type hint indicates it returns list[str], but the actual implementation returns a single str (or None). Please update the type hint to str | None to match the actual return type.

Suggested change

def annotate_ascend(input_prompt:str, task_id:int=None)->list[str]:

def annotate_ascend(input_prompt:str, task_id:int=None)->str | None:

gemini-code-assist · 2026-05-29T03:10:39Z

    response = openai.chat.completions.create(
        model=model,
        messages=messages,
-        temperature=0.7,
+        temperature=temperature,
        top_p=0.95,
-        max_tokens=10_000,
+        max_tokens=max_tokens,
        stream=False,
    )
    whole_result = response.choices[0].message.content


The OpenAI API call lacks error handling. If a network issue, rate limit, or API error occurs, the entire evaluation loop will crash, losing progress. Consider wrapping the API call in a try-except block to handle exceptions gracefully.

Suggested change

response = openai.chat.completions.create(

model=model,

messages=messages,

temperature=0.7,

temperature=temperature,

top_p=0.95,

max_tokens=10_000,

max_tokens=max_tokens,

stream=False,

)

whole_result = response.choices[0].message.content

try:

response = openai.chat.completions.create(

model=model,

messages=messages,

temperature=temperature,

top_p=0.95,

max_tokens=max_tokens,

stream=False,

)

whole_result = response.choices[0].message.content

except Exception as e:

print(f"Error calling OpenAI API: {e}")

whole_result = ""

Add files via upload

817290c

gemini-code-assist Bot reviewed May 29, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add files via upload#257

Add files via upload#257
ttstopll85 wants to merge 1 commit into
FlagAI-Open:mainfrom
ttstopll85:main

ttstopll85 commented May 29, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

gemini-code-assist Bot May 29, 2026

Uh oh!

gemini-code-assist Bot May 29, 2026

Uh oh!

gemini-code-assist Bot May 29, 2026

Uh oh!

gemini-code-assist Bot May 29, 2026

Uh oh!

gemini-code-assist Bot May 29, 2026

Uh oh!

gemini-code-assist Bot May 29, 2026

Uh oh!

gemini-code-assist Bot May 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

	prediction = annotate(input_prompt, task_id)
	prediction = annotate(input_prompt, task_id) if annotate.__name__ == 'annotate_ascend' else annotate(input_prompt)

-    tokenizer = AutoTokenizer.from_pretrained("/root/Qwen3-4B", trust_remote_code=True)
+    import os
+    tokenizer_path = os.getenv("TOKENIZER_PATH", "/root/Qwen3-4B")
+    tokenizer = AutoTokenizer.from_pretrained(tokenizer_path, trust_remote_code=True)

	def annotate_ascend(input_prompt:str, task_id:int=None)->list[str]:
	def annotate_ascend(input_prompt:str, task_id:int=None)->str \| None:

Uh oh!

Conversation

ttstopll85 commented May 29, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot May 29, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 29, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 29, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 29, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 29, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 29, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 29, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant