Add files via upload

[rubhudegaar]

No description provided.

[gemini-code-assist]

Code Review

This pull request introduces several optimizations for the long-context in-context learning (ICL) annotation system, including task-specific prompt routing, Chain-of-Thought (CoT) prompt generation, dynamic example selection strategies (M05, M19, M20), and post-processing cleanups for Huawei Ascend model outputs. The reviewer feedback highlights critical issues that need to be addressed: a typo in the regular expression for thought-chain cleaning, a logic error where dynamically retrieved examples are incorrectly cached and reused across all samples, hardcoded absolute paths, a type annotation mismatch, and performance bottlenecks caused by repeatedly loading the tokenizer instead of reusing the global singleton.

[gemini-code-assist]

There is a typo in the regular expression used to clean the thought process from the LLM output. The pattern r'<think&rt;.*?</think&rt;' uses &rt; instead of > (or >). Furthermore, since the OpenAI API returns plain text, the tags in whole_result are literal <think> and </think> tags, not HTML entities. As a result, this regex will fail to match and clean the thought process, which can lead to incorrect final answers or code block extraction.

Suggested change

	# Step 1: 过滤<think&rt;标签内容（去思维链）
	cleaned_result = re.sub(r'<think&rt;.*?</think&rt;', '', whole_result, flags=re.DOTALL)
	# Step 1: 过滤<think>标签内容（去思维链）
	cleaned_result = re.sub(r'<think>.*?</think>', '', whole_result, flags=re.DOTALL)

[gemini-code-assist]

The newly introduced select_examples_M19 is a dynamic retrieval strategy that selects few-shot examples based on similarity to the current sample's input (text2annotate). However, because examples_str is defined outside the loop and guarded by if examples_str is None:, it is only computed once for the first sample and then reused for all subsequent samples. This completely defeats the purpose of dynamic retrieval.

To fix this, you should remove the if examples_str is None: check so that the examples are dynamically retrieved for each sample.

        # M19优化：使用检索+重试+后处理组合策略
        examples_str = select_examples_M19(icl_examples, task_description, text2annotate, task_id, sample_idx)

[gemini-code-assist]

Hardcoding absolute paths like /root/OpenSeek/... makes the code non-portable and prone to failure when run in different environments or by other users. It is highly recommended to construct these paths dynamically relative to the script's location using os.path.

SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
DATA_DIR = os.path.join(os.path.dirname(SCRIPT_DIR), 'data')

TASK_FILES = {
    1: os.path.join(DATA_DIR, 'openseek-1_closest_integers.json'),
    2: os.path.join(DATA_DIR, 'openseek-2_count_nouns_verbs.json'),
    3: os.path.join(DATA_DIR, 'openseek-3_collatz_conjecture.json'),
    4: os.path.join(DATA_DIR, 'openseek-4_conala_concat_strings.json'),
    5: os.path.join(DATA_DIR, 'openseek-5_semeval_2018_task1_tweet_sadness_detection.json'),
    6: os.path.join(DATA_DIR, 'openseek-6_mnli_same_genre_classification.json'),
    7: os.path.join(DATA_DIR, 'openseek-7_jeopardy_answer_generation_all.json'),
    8: os.path.join(DATA_DIR, 'openseek-8_kernel_generation.json'),
}

[gemini-code-assist]

The return type annotation for annotate_ascend is specified as list[str], but the function actually returns a single string (str) representing the cleaned prediction or code block. This type mismatch can mislead static analysis tools and other developers.

Suggested change

	def annotate_ascend(input_prompt:str, task_id:int=None)->list[str]:
	def annotate_ascend(input_prompt:str, task_id:int=None)->str:

[gemini-code-assist]

Loading the tokenizer from disk/network using AutoTokenizer.from_pretrained on every call to select_examples_M05 is extremely slow and will cause a massive performance bottleneck in the evaluation loop. Since you already implemented a global tokenizer singleton get_tokenizer_m20(), you should reuse it here to avoid redundant loading.

Suggested change

	tokenizer = AutoTokenizer.from_pretrained("/root/Qwen3-4B", trust_remote_code=True)
	tokenizer = get_tokenizer_m20()

[gemini-code-assist]

Loading the tokenizer from disk/network using AutoTokenizer.from_pretrained on every call to select_examples_M19 is extremely slow and will cause a massive performance bottleneck in the evaluation loop. Since you already implemented a global tokenizer singleton get_tokenizer_m20(), you should reuse it here to avoid redundant loading.

Suggested change

	tokenizer = AutoTokenizer.from_pretrained("/root/Qwen3-4B", trust_remote_code=True)
	tokenizer = get_tokenizer_m20()