Fix/allowed token ids validation

[DivyaNarahari97]

fix(sampling): correct stop-sequence string pairing and allowed_token_ids validation

Summary

Two independent bugs in lightllm/server/core/objs/sampling_params.py, plus a repair to the test module that was supposed to be covering them.

---

1. Stop sequence token ids could be paired with the wrong string

stop_sentences_to_token_ids() drops entries that encode to no tokens ("", [], or any string the tokenizer maps to nothing). StopSequenceGroups.initialize() then indexed the original stop_sequences list using the filtered list's index:

groups = self.stop_sentences_to_token_ids(stop_sequences, tokenizer)  # filtered, may be shorter
self.size = len(groups)

for group_idx in range(self.size):
    if isinstance(stop_sequences[group_idx], str):                    # original, unfiltered
        self.groups[group_idx].initialize(groups[group_idx], sequence_str=stop_sequences[group_idx])

Every entry after a dropped one shifts by a position, so a group's sequence_str comes from the wrong stop entry.

Impact — two user-visible failures:

stop=	to_strings() before	Effect
["", "END"]	[]	"END" silently loses string matching entirely
["unknown", "stop2"]	["unknown"]	stops generation on a string the caller never requested

The second is the damaging one. These strings drive stop matching in DecodeReq.stop_sequences_str_match() and trailing-stop trimming in the OpenAI completion path, so a request can terminate early on unrelated text.

Fix — carry (token_ids, original_entry) pairs through the filter so the two can't drift apart. stop_sentences_to_token_ids() keeps its original signature as a thin wrapper over the new helper, so no callers change.

---

2. AllowedTokenIds.initialize() validated the wrong variable

def initialize(self, ids: List[int]):
    self.size = len(ids)
    assert self.size <= ALLOWED_TOKEN_IDS_MAX_LENGTH, "Too many allowed token IDs."
    assert all(isinstance(e, int) for e in self.ids), "all must be int"   # <-- self.ids, not ids
    self.ids[: self.size] = ids[:]

self.ids is the c_int array being written into, still zero-filled at that point. Iterating a ctypes c_int array always yields Python ints, so the assertion is vacuously true for every input and rejects nothing.

Impact — low. Bad input was still rejected, just one line later and with a confusing error:

before:  TypeError: 'str' object cannot be interpreted as an integer
after:   AssertionError: all must be int

Fix — assert over ids (the argument). This matches StopSequence.initialize() a few classes up, which validates its argument correctly.

---

3. The test module for all of the above never ran

unit_tests/server/core/objs/test_sampling_params.py imported DecodeNode, a class that no longer exists in sampling_params. The module failed at collection, so none of its tests had been running. Replaced with an equivalent NodeUUId round-trip test.

Testing

• Added test_stop_sequence_groups_keeps_ids_and_strings_aligned, covering the aligned case, both drop-then-shift cases, and pure-id entries.
• Added test_allowed_token_ids_rejects_non_int.
• Both confirmed failing before the fix and passing after.
• The pre-existing tests in the module now execute for the first time and pass.