Add files via upload

[jtthie]

No description provided.

[gemini-code-assist]

Code Review

This pull request introduces a dynamic example selection scheme based on Jaccard similarity, adds Chain-of-Thought (CoT) prompting for specific tasks, and implements a differentiated annotation strategy for Huawei Ascend with task-specific temperatures and token limits. Feedback on these changes highlights a critical bug where the dynamic example selection is defeated because the selected examples are cached outside the loop and reused across all samples. To resolve this and prevent severe performance degradation, the tokenizer should be pre-loaded and passed directly to the selection function. Additionally, reviewers recommend replacing hardcoded absolute paths with dynamic relative paths, correcting the return type annotation for 'annotate_ascend', and adding defensive checks on the API response to avoid potential runtime errors.

[gemini-code-assist]

The variable examples_str is initialized to None outside the loop and cached after the first iteration. This means select_examples is only called once for the very first test sample, and the same examples are reused for all subsequent test samples.

This completely defeats the purpose of the new "M02 dynamic selection scheme" (select_examples with Jaccard similarity), which is designed to dynamically select the most relevant examples for each test sample based on text2annotate.

To fix this, we should call select_examples for every test sample. Additionally, to avoid loading the tokenizer from disk on every call, we should pass the pre-loaded qwen_tokenizer to select_examples.

        examples_str = select_examples(icl_examples, task_description, text2annotate, qwen_tokenizer)

[gemini-code-assist]

If select_examples is called for every test sample to enable dynamic selection, loading the tokenizer from disk via AutoTokenizer.from_pretrained("/root/Qwen3-4B") on every single call will cause severe performance degradation.

We should update the function signature to accept an optional tokenizer argument and reuse the pre-loaded tokenizer from main.py.

Suggested change

	def select_examples(all_examples: list[dict], task_description: str, text2annotate: str) -> str:
	"""
	M02优化版本：按样本动态选例方案
	为每个样本动态选择最相关的示例，而非按固定顺序选择
	Parameters:
	all_examples: 所有示例列表，每个示例包含'input'和'output'键
	task_description: 任务描述
	text2annotate: 待标注文本（用于相似度计算）
	"""
	# 初始化Qwen3-4B的tokenizer
	tokenizer = AutoTokenizer.from_pretrained("/root/Qwen3-4B", trust_remote_code=True)
	def select_examples(all_examples: list[dict], task_description: str, text2annotate: str, tokenizer: AutoTokenizer = None) -> str:
	"""
	M02优化版本：按样本动态选例方案
	为每个样本动态选择最相关的示例，而非按固定顺序选择
	Parameters:
	all_examples: 所有示例列表，每个示例包含'input'和'output'键
	task_description: 任务描述
	text2annotate: 待标注文本（用于相似度计算）
	tokenizer: Qwen3-4B的tokenizer实例
	"""
	# 初始化Qwen3-4B的tokenizer
	if tokenizer is None:
	tokenizer = AutoTokenizer.from_pretrained("/root/Qwen3-4B", trust_remote_code=True)

[gemini-code-assist]

Using absolute paths hardcoded to /root/... makes the script non-portable and prone to FileNotFoundError when run in different environments or by other users.

It is highly recommended to construct these paths dynamically relative to the script's directory using os.path.abspath and os.path.join.

CURRENT_DIR = os.path.dirname(os.path.abspath(__file__))
DATA_DIR = os.path.abspath(os.path.join(CURRENT_DIR, "..", "data"))

TASK_FILES = {
    1: os.path.join(DATA_DIR, 'openseek-1_closest_integers.json'),
    2: os.path.join(DATA_DIR, 'openseek-2_count_nouns_verbs.json'),
    3: os.path.join(DATA_DIR, 'openseek-3_collatz_conjecture.json'),
    4: os.path.join(DATA_DIR, 'openseek-4_conala_concat_strings.json'),
    5: os.path.join(DATA_DIR, 'openseek-5_semeval_2018_task1_tweet_sadness_detection.json'),
    6: os.path.join(DATA_DIR, 'openseek-6_mnli_same_genre_classification.json'),
    7: os.path.join(DATA_DIR, 'openseek-7_jeopardy_answer_generation_all.json'),
    8: os.path.join(DATA_DIR, 'openseek-8_kernel_generation.json'),
}

[gemini-code-assist]

The return type annotation for annotate_ascend is specified as list[str], but the function actually returns a single string (either whole_result.strip() or the result of count_answer which is a string or None). The type annotation should be updated to reflect this.

Suggested change

	def annotate_ascend(input_prompt:str, task_id:int=None)->list[str]:
	def annotate_ascend(input_prompt:str, task_id:int=None)->str | None:

[gemini-code-assist]

If the API call fails, returns an empty response, or if response.choices is empty, accessing response.choices[0].message.content directly will raise an IndexError or AttributeError.

We should defensively check if response.choices is non-empty and if the content is not None before using it.

Suggested change

	whole_result = response.choices[0].message.content
	whole_result = response.choices[0].message.content if (response.choices and response.choices[0].message.content) else ""