Output hidden states like hugging face

[Sami6720]

Hi,

I am trying to get all the hidden states from the forward pass of a llama model using vllm. Sort of like what is possible with hugging face's out_hidden_states = True but with all the optimizations that film does to speed up inference that one doesn't get using vanilla hf.

Is this even possible?

I have tried setting task = embedding in the LLM class, but that only gives the last layer's last embedding. I would want to have access to the outputs from each layer.

[DarkLight1337]

See #15434

[d8543502-spec]

Interesting feature request! Having access to hidden states, similar to Hugging Face, would give developers more flexibility for analysis and model debugging. It could also improve AI-powered food applications, such as recipe recommendations, menu classification, and nutrition assistants by enabling deeper insight into model behavior.

[d8543502-spec]

Great suggestion! Exposing hidden states would make it easier to understand and fine-tune model outputs for advanced use cases. In food-tech projects, this could support better recipe generation, ingredient recognition, and personalized meal recommendations while giving developers more control over AI performance.