fix: resolve select field label-to-ID mapping for custom fields

[SEWADE]

Problem

When using custom fields of type select, paperless-gpt fails to update documents with a 400 error from the Paperless-ngx API:

error updating document: 400, {"custom_fields":[{"non_field_errors":["Value must be an id of an element in [...]"]}]}

Root cause: Two issues combined:

1. The CustomField struct did not include ExtraData, so the select_options from the API response were silently discarded during JSON parsing.

2. The LLM prompt for custom field suggestions only contained type="select" without the available options. The LLM therefore returned human-readable labels (e.g. "Sonstiges") instead of the required option IDs (e.g. "Ien0jKSg1o3E4E0t"), causing the API to reject the update.

Fix

• Added SelectOption and CustomFieldExtraData structs and an ExtraData field to CustomField so select options are properly deserialized.
• Extended the LLM prompt XML to include <option id="...">label</option> entries for select-type fields, so the LLM can return the correct option ID directly.

Tested

Verified on a Paperless-ngx instance with two select-type custom fields (8 and 12 options respectively). Documents are now processed and saved successfully.

Summary by CodeRabbit

• New Features
  • Improved custom field support: added structured handling for select option data so multi-option fields render correctly.
• Bug Fixes
  • Safer XML output: attribute and text values are now escaped to prevent malformed XML when custom field names, types, or option labels contain special characters.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 9254e4b3-5d0d-4ecf-aca4-eb6e99e376c3

📥 Commits

Reviewing files that changed from the base of the PR and between f24c3d1 and c8b434f.

📒 Files selected for processing (1)

• app_llm.go

---

📝 Walkthrough

Walkthrough

Adds Paperless-ngx select-option modeling and XML escaping helpers, and updates prompt XML generation to emit escaped attribute/text and nested <option> child elements for select custom fields when options are present.

Changes

Cohort / File(s)	Summary
Data model paperless.go	Added SelectOption and CustomFieldExtraData types; extended exported CustomField with optional ExtraData *CustomFieldExtraData \json:"extra_data"`` to hold select options.
Prompt/XML generation app_llm.go	Added escapeXMLAttr/escapeXMLText helpers and integrated them into getSuggestedCustomFields; select fields with ExtraData.SelectOptions now render a multiline <field ...> with escaped child <option id="...">label</option> entries, otherwise preserve single-tag <field> with escaped attributes.

Sequence Diagram(s)

(omitted)

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related issues

• Custom fields with type "select" fail: sends label instead of ID to Paperless-ngx API #902: Implements select-option types and XML serialization with option IDs/labels, addressing the issue's objective to map option labels to IDs.

Poem

🐰 I nibble code and tidy tags,
Options nested, neat as bags,
Escaped attributes, text so clean,
Paperless fields in XML sheen,
Hops of joy — a rabbit's engineering! 🥕

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main change: adding select field label-to-ID mapping support for custom fields by including options in the LLM prompt and properly deserializing extra_data.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@app_llm.go`:
- Around line 404-412: The XML being built in the block that writes field and
option nodes must escape attribute values and text to avoid breaking XML when
names/labels contain special characters; add helper replacers/escape functions
(e.g., xmlAttrEscaper/xmlTextEscaper with escapeXMLAttr and escapeXMLText) and
use them when writing attributes and text in xmlBuilder.WriteString (escape
field.Name and field.DataType for attributes, option IDs with escapeXMLAttr, and
option labels with escapeXMLText) so all interpolated values are properly
encoded before writing the <field> and <option> nodes.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 4b620b93-2f62-4681-b89f-356be646f40f

📥 Commits

Reviewing files that changed from the base of the PR and between 94c8428 and f24c3d1.

📒 Files selected for processing (2)

• app_llm.go
• paperless.go