Devon's suggestion: extend the workflow from a single LLM call to an agentic
loop, where the model inspects the output of a failed conversion and queries
itself to fix it.
Scoped as future work, not Fall semester scope. Devon raised it as an avenue to
try if the single-shot approach underperforms.
Why it is worth exploring:
- The signals already exist. Type checking, per-column confidence, and the
actionable feedback on unmapped columns are what a retry loop would consume.
Today they go to the user. Agentically they would go back to the model.
- It is the same problem as adaptive coding for irregular formats, for example
DIA-NN packing several values into one cell. A try, inspect, fix loop is one
mechanism for that.
The trade-off:
- Reproducibility. The current design keeps conversions reproducible because the
model emits a mapping once and deterministic code executes it. Several
stochastic, path-dependent calls weaken that, and reproducibility is part of
the argument for the local-model path.
- Cost and latency on the hosted-model path.
- Harder to characterise failure modes for the paper.
Middle ground worth considering first:
A bounded retry rather than a general agent. On a type-check failure, feed the
error back once, cap the number of attempts, and log the full trace so a run
stays auditable. Most of the robustness, far less of the reproducibility cost,
and a much smaller build.
Related work:
- Cover single-shot versus agentic as a decision point in the design document.
- Include in the future work section of the technical note. Reviewers are likely
to ask why the approach does not iterate, so the reasoning is worth stating.
Devon's suggestion: extend the workflow from a single LLM call to an agentic
loop, where the model inspects the output of a failed conversion and queries
itself to fix it.
Scoped as future work, not Fall semester scope. Devon raised it as an avenue to
try if the single-shot approach underperforms.
Why it is worth exploring:
actionable feedback on unmapped columns are what a retry loop would consume.
Today they go to the user. Agentically they would go back to the model.
DIA-NN packing several values into one cell. A try, inspect, fix loop is one
mechanism for that.
The trade-off:
model emits a mapping once and deterministic code executes it. Several
stochastic, path-dependent calls weaken that, and reproducibility is part of
the argument for the local-model path.
Middle ground worth considering first:
A bounded retry rather than a general agent. On a type-check failure, feed the
error back once, cap the number of attempts, and log the full trace so a run
stays auditable. Most of the robustness, far less of the reproducibility cost,
and a much smaller build.
Related work:
to ask why the approach does not iterate, so the reasoning is worth stating.