feat: add new long-context ICL annotation method#261
Conversation
There was a problem hiding this comment.
Code Review
This pull request implements a comprehensive automatic data annotation solution for the OpenSeek competition (Tasks 1-8), featuring hybrid ICL retrieval, spaCy-based POS tagging, and ReAct-style verification loops. The code review feedback highlights critical security concerns regarding hardcoded DeepSeek API keys in build_cot_training_set.py and deepseek_client.py. Additionally, there are opportunities to improve portability by removing hardcoded CUDA_VISIBLE_DEVICES settings, clean up leftover debugging print statements in the Task 5 inference scripts, and enhance the robustness of the output comparison script by supporting alternative field names.
| if not args.dry_run and not api_key: | ||
| api_key = 'sk-340c73f8209e4ba3ae44c7e6cf570506' |
There was a problem hiding this comment.
Hardcoding API keys is a severe security risk. It is highly recommended to remove the hardcoded key and raise an error if the DEEPSEEK_API_KEY environment variable is not set.
| if not args.dry_run and not api_key: | |
| api_key = 'sk-340c73f8209e4ba3ae44c7e6cf570506' | |
| if not args.dry_run and not api_key: | |
| raise ValueError("DEEPSEEK_API_KEY environment variable is not set.") |
|
|
||
| from openai import OpenAI | ||
|
|
||
| DEEPSEEK_API_KEY = "sk-340c73f8209e4ba3ae44c7e6cf570506" |
There was a problem hiding this comment.
| continue | ||
| try: | ||
| row = json.loads(line) | ||
| except json.JSONDecodeError: |
| converted.append(EMOJI_TEXT_MAP_ZH.get(ch, _emoji_to_alias(ch))) | ||
| else: | ||
| converted.append(_emoji_to_alias(ch)) | ||
| print('converted:', converted) |
| sample_id = obj.get("test_sample_id") | ||
| prediction = obj.get("prediction") | ||
| if sample_id is None or prediction is None: | ||
| raise ValueError( | ||
| f"{path} 第 {line_no} 行缺少字段,必须包含 test_sample_id 和 prediction" | ||
| ) |
There was a problem hiding this comment.
The current implementation only supports test_sample_id and prediction fields. Supporting both test_sample_id/example_id and prediction/model_output makes this comparison script much more robust and compatible with both examples and test samples.
| sample_id = obj.get("test_sample_id") | |
| prediction = obj.get("prediction") | |
| if sample_id is None or prediction is None: | |
| raise ValueError( | |
| f"{path} 第 {line_no} 行缺少字段,必须包含 test_sample_id 和 prediction" | |
| ) | |
| sample_id = obj.get("test_sample_id") or obj.get("example_id") | |
| prediction = obj.get("prediction") or obj.get("model_output") | |
| if sample_id is None or prediction is None: | |
| raise ValueError( | |
| f"{path} line {line_no} is missing required fields (must contain test_sample_id/example_id and prediction/model_output)" | |
| ) |
| import concurrent.futures | ||
| import json | ||
| import os | ||
| os.environ.setdefault("CUDA_VISIBLE_DEVICES", "2") |
Signed-off-by: xiamaozi11 <879989535@qq.com>
498c1b0 to
6802a68
Compare
No description provided.