Basic LLM Chain's Output Format prompt does not include the defined schema in its message.

[YuzuruT]

Describe the problem/error/question

The output from the Basic LLM Chain does not comply with the assigned Json Schema for the output format. The output include hallucinated fields.

I do suspect the cause is because the prompt appended to indicate the LLM to return a JSON format response does not include the provided schema in its definition. It simply tells that it must return in a Json format.

What is the error message (if any)?

No error message. Just the output is not what is expected.

Please share your workflow/screenshots/recording

The LLM I am using is Gemini-1.0-pro.
Below is the node for the Basic LLM Chain and the Structured Output Parser.

Basic LLM Chain
{
"nodes": [
{
"parameters": {
"promptType": "define",
"text": "= {{ $json.output }}",
"hasOutputParser": true,
"messages": {
"messageValues": [
{
"message": "Extract the client_id, and action needed to be taken by the Healsonic technician from the provided message. If none are found, leave the fields as null. Keep the original message as output_text field."
}
]
}
},
"type": "@n8n/n8n-nodes-langchain.chainLlm",
"typeVersion": 1.5,
"position": [
1380,
1120
],
"id": "f8003ce5-eb81-409b-9e9c-0b27419f6e13",
"name": "Basic LLM Chain"
}
],
"connections": {},
"pinData": {},
"meta": {
"templateId": "2753",
"templateCredsSetupCompleted": true,
"instanceId": "1db3826a53944b70bd3663b3d3df73cf686a59b54e518109bc3737fab1dd588f"
}
}

Structured Output Parser.
{
"nodes": [
{
"parameters": {
"schemaType": "manual",
"inputSchema": "{\n "output_text": "string",\n "client_id": "string",\n "action": "string"\n}"
},
"type": "@n8n/n8n-nodes-langchain.outputParserStructured",
"typeVersion": 1.2,
"position": [
1500,
1300
],
"id": "14bc7f37-610e-451c-9b7b-56a6d0b256b8",
"name": "Structured Output Parser"
}
],
"connections": {
"Structured Output Parser": {
"ai_outputParser": [
[]
]
}
},
"pinData": {},
"meta": {
"templateId": "2753",
"templateCredsSetupCompleted": true,
"instanceId": "1db3826a53944b70bd3663b3d3df73cf686a59b54e518109bc3737fab1dd588f"
}
}

(Select the nodes on your canvas and use the keyboard shortcuts CMD+C/CTRL+C and CMD+V/CTRL+V to copy and paste the workflow.)

Share the output returned by the last node

{
"output": {
"client_id": null,
"action_needed": "fix_qr script needs to be run by the Healsonic technician"
}
}

Debug info

The prompt sent to the Gemini included the following text at the end which did NOT include the schema defined in the Structured Output Parser.

'''
You must format your output as a JSON value that adheres to a given "JSON Schema" instance.

"JSON Schema" is a declarative language that allows you to annotate and validate JSON documents.

For example, the example "JSON Schema" instance {{"properties": {{"foo": {{"description": "a list of test words", "type": "array", "items": {{"type": "string"}}}}}}, "required": ["foo"]}}}}
would match an object with one required property, "foo". The "type" property specifies "foo" must be an "array", and the "description" property semantically describes it as "a list of test words". The items within "foo" must be strings.
Thus, the object {{"foo": ["bar", "baz"]}} is a well-formatted instance of this example "JSON Schema". The object {{"properties": {{"foo": ["bar", "baz"]}}}} is not well-formatted.

Your output will be parsed and type-checked according to the provided schema instance, so make sure all fields in your output match the schema exactly and there are no trailing commas!

Here is the JSON Schema instance your output must adhere to. Include the enclosing markdown codeblock:

{"type":"object","properties":{"output":{}},"additionalProperties":false,"$schema":"http://json-schema.org/draft-07/schema#"}
'''

### core

- n8nVersion: 1.75.2
- platform: npm
- nodeJsVersion: 20.18.1
- database: sqlite
- executionMode: regular
- concurrency: 5
- license: community

### storage

- success: all
- error: all
- progress: false
- manual: true
- binaryMode: filesystem

### pruning

- enabled: true
- maxAge: 168 hours
- maxCount: 2500 executions

### client

- userAgent: mozilla/5.0 (macintosh; intel mac os x 10_15_7) applewebkit/537.36 (khtml, like gecko) chrome/131.0.0.0 safari/537.36
- isTouchDevice: false

Generated at: 2025-01-29T00:25:40.417Z}

[Joffcom]

Hey @YuzuruT,

We have created an internal ticket to look into this which we will be tracking as "GHC-650"