feat: add frequency penalty to inference sampling pipeline

- Add FrequencyPenaltyStrategy (logit -= penalty * count)
- Per-task rep_window for penalty history lookup
- Wire through engine, task, executor, API layer
- Add --frequency_penalty and --rep_window to stream_chat.py
- 9 unit tests for frequency penalty strategy
This commit is contained in:
2026-07-17 21:28:31 +08:00
parent a1ea26d367
commit d08a92c7bd
9 changed files with 370 additions and 20 deletions
+15
View File
@@ -42,6 +42,19 @@ def parse_args():
default=2048,
help="Maximum tokens to generate",
)
parser.add_argument(
"--frequency_penalty",
type=float,
default=0.5,
help="Penalty per occurrence for repeated tokens (0.0 disables, "
"range -2.0~2.0, typical 0.3-1.0)",
)
parser.add_argument(
"--rep_window",
type=int,
default=64,
help="Number of recent prompt tokens to include in penalty history",
)
parser.add_argument(
"--system_prompt",
type=str,
@@ -79,6 +92,8 @@ def chat():
temperature=args.temperature,
top_p=args.top_p,
top_k=args.top_k,
frequency_penalty=args.frequency_penalty,
rep_window=args.rep_window,
):
print(token, end="", flush=True)
full_response += token