diff --git a/CHANGELOG.md b/CHANGELOG.md index 02d884321..f65875e41 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,18 @@ follow semantic versioning; release dates are ISO 8601. ### Public API +- **A table of contents keeps its page numbers in Word, and they stay right.** The DOCX export + dropped every page reference, so `addTableOfContents(...)` reached Word with its entries and + no numbers, and `addPageReference(...)` wrote nothing. Each is now Word's own `PAGEREF` + field to the anchor's bookmark, as a hyperlink, storing the page the layout resolved — so + the file opens reading the PDF's numbers, and the editor recomputes them as the document + changes. Measured in LibreOffice: with the stored numbers replaced by 99, the table of + contents still showed pages 2, 3 and 5. A reference to an anchor the document has no + bookmark for is written as its placeholder text rather than a field Word would turn into + "Error! Bookmark not defined.". The export does not set `w:updateFields`, which would make + Word ask on every open to recompute numbers that already read correctly; the recipe states + which field each page number is and what updates it. + - **A link to a section, table or image reaches it in Word.** The DOCX export wrote a bookmark only for a paragraph's `anchor(...)`; an anchor on a section, container, table or image was dropped, so an internal link to it went nowhere in Word. Such a block's anchor is diff --git a/docs/architecture/backend-capability-matrix.md b/docs/architecture/backend-capability-matrix.md index fa83e6ff5..45f6c6c62 100644 --- a/docs/architecture/backend-capability-matrix.md +++ b/docs/architecture/backend-capability-matrix.md @@ -93,6 +93,7 @@ Payload records live in `core` under |---|---|---|---| | External hyperlinks (fragment- and run-level) | ✅ `PdfLinkAnnotationWriter` + link rects in `PdfFixedLayoutBackend` | ✅ `PptxNavigationWriter` (transparent hotspots for measured span, line, and fragment rectangles, emitted above all content after the fragment pass) | ❌ | | Internal links (anchor jump, forward references) | ✅ `PdfInternalLinkWriter` (two-pass) | ✅ `PptxNavigationWriter` (deferred slide-jump hyperlinks, resolved after all fragments — including across sections) | ✅ `DocxSemanticBackend` — an internal `linkTarget` is a `w:hyperlink` with `w:anchor`, and every anchor the export writes is a bookmark: a paragraph's around its text, a section's, container's, table's or image's around everything the block wrote (`bookmarkAround`). An anchor on a node the export drops (a shape, a barcode) has nothing to mark | +| Page references — a table of contents' numbers, `addPageReference(...)` (`PageReferenceNode`) | ✅ `PageReferenceDefinition` lays out the resolved page as text, drawn by `PdfParagraphFragmentRenderHandler` | ✅ the same laid-out text through `PptxParagraphFragmentRenderHandler` | ✅ `DocxSemanticBackend.writePageReference` — a `PAGEREF` field to the anchor's bookmark, as a hyperlink (`\h`), storing the page the layout resolved; the editor recomputes it (LibreOffice on layout, Word on a field update). A reference to an anchor the export writes no bookmark for is its placeholder text, since Word turns a `PAGEREF` to a missing bookmark into an error. `w:updateFields` is not set | | Document outline / bookmarks tree | ✅ `PdfBookmarkOutlineWriter` | ⚠️ `PptxNavigationWriter` (no PPTX outline concept — slide names where 1:1, extra bookmarks dropped with a note) | ❌ | ## Document chrome and output options diff --git a/docs/recipes/docx-export.md b/docs/recipes/docx-export.md index a2e35b82e..9d818a350 100644 --- a/docs/recipes/docx-export.md +++ b/docs/recipes/docx-export.md @@ -82,6 +82,28 @@ Page geometry (size, margins and orientation — a page wider than it is tall is landscape) and session metadata (title, author, subject, keywords) carry into the Word document as well. +## Fields, and when they update + +Everything that states a page number is a Word field rather than text, so it stays right +when the reader edits the document: + +| Where | Field | Updated by | +|---|---|---| +| a page zone's `pageNumber()` | `PAGE` | the editor, every time it lays the pages out | +| a page zone's `pageTotal()` | `NUMPAGES`, or `SECTIONPAGES` in a multi-section document | the editor, every time it lays the pages out — except LibreOffice, which does not update `SECTIONPAGES` | +| a table of contents' page numbers, `addPageReference(...)` | `PAGEREF` to the anchor's bookmark, as a hyperlink | LibreOffice on every layout (measured: a field whose stored number was replaced by 99 showed the real page); Word when fields are updated — F9, or printing with field updates on | + +Each field also stores a result, which is what a reader sees before an editor updates it +and what a text extractor finds: the page the layout resolved, and for a page total the +number of pages it laid out. A file therefore opens reading the same numbers as the PDF. + +The export does not set `w:updateFields`. It would make Word ask, on every open, whether to +update fields — to recompute numbers that already read correctly. + +A page reference to an anchor the document does not bookmark is written as its text, the +placeholder the page prints: Word turns a `PAGEREF` to a missing bookmark into "Error! +Bookmark not defined." the first time it updates. + ## Several sections in one document A `MultiSectionDocument` — a cover in one page size, a body in another — exports to Word diff --git a/render-docx/src/main/java/com/demcha/compose/document/backend/semantic/docx/DocxLayoutMetrics.java b/render-docx/src/main/java/com/demcha/compose/document/backend/semantic/docx/DocxLayoutMetrics.java index 57d079674..db701b039 100644 --- a/render-docx/src/main/java/com/demcha/compose/document/backend/semantic/docx/DocxLayoutMetrics.java +++ b/render-docx/src/main/java/com/demcha/compose/document/backend/semantic/docx/DocxLayoutMetrics.java @@ -312,6 +312,28 @@ OptionalDouble lineHeight(DocumentNode node) { return OptionalDouble.empty(); } + /** + * The text a node laid out as paragraph lines, as the layout wrote it. + * + *

A page reference's number is known only once the document is paginated, so the + * layout resolves it and lays out the number as text; this reads that text back rather + * than resolving the page a second time.

+ * + * @param node any node that lays out as paragraph lines + * @return the laid-out text, or empty when the node laid out nothing + */ + java.util.Optional laidOutText(DocumentNode node) { + for (PlacedFragment fragment : fragmentsOf(node)) { + if (fragment.payload() instanceof ParagraphFragmentPayload paragraph + && !paragraph.lines().isEmpty()) { + StringBuilder text = new StringBuilder(); + paragraph.lines().forEach(line -> text.append(line.text())); + return java.util.Optional.of(text.toString()); + } + } + return java.util.Optional.empty(); + } + /** * The resolved width of every column of a table. * diff --git a/render-docx/src/main/java/com/demcha/compose/document/backend/semantic/docx/DocxSemanticBackend.java b/render-docx/src/main/java/com/demcha/compose/document/backend/semantic/docx/DocxSemanticBackend.java index 24fdc812f..a6a24ce40 100644 --- a/render-docx/src/main/java/com/demcha/compose/document/backend/semantic/docx/DocxSemanticBackend.java +++ b/render-docx/src/main/java/com/demcha/compose/document/backend/semantic/docx/DocxSemanticBackend.java @@ -234,6 +234,8 @@ public final class DocxSemanticBackend implements SemanticBackend { // The outline levels this document asks for, so the styles part defines those and no // others. Filled before the styles part is written, which comes before the body. private java.util.Set headingLevels = java.util.Set.of(); + // Anchors this export writes a bookmark for, so a page reference knows it has a target. + private java.util.Set bookmarkedAnchors = java.util.Set.of(); // Where the finished report goes, when the caller configured somewhere for it to go. private final java.util.function.Consumer reportSink; // The instant every clock in the package is pinned to, or null for live timestamps. @@ -450,6 +452,7 @@ private byte[] write(List sections, Path outputFile) throws Exc report = new DocxExportReport.Builder(); bookmarkNames = new DocxBookmarkNames(); headingLevels = headingLevelsIn(whole); + bookmarkedAnchors = bookmarkedAnchorsIn(whole); wordFamilies = DocxFontTable.familiesByName(fonts); documentDefaultStyle = dominantTextStyle(whole); currentCell = null; @@ -1119,6 +1122,8 @@ private static void keepOnOnePage(List written, boolean withNext) private void writeNodeContent(XWPFDocument document, DocumentNode node) throws Exception { if (node instanceof ParagraphNode paragraph) { writeParagraph(document, paragraph); + } else if (node instanceof com.demcha.compose.document.node.PageReferenceNode reference) { + writePageReference(document, reference); } else if (node instanceof ImageNode image) { writeImage(document, image); } else if (node instanceof TableNode table) { @@ -1807,6 +1812,25 @@ private void applyHeadingRole(XWPFParagraph para, ParagraphNode node) { * A large paragraph is a large paragraph; a document that never asked for an outline * does not get one invented from its typography.

*/ + /** + * Every anchor this export writes a bookmark for: a paragraph's, and a block's the export + * writes (see {@link #blockAnchorOf}). A page reference is a live field only when its + * anchor is one of these. + */ + private static java.util.Set bookmarkedAnchorsIn(DocumentGraph graph) { + java.util.Set anchors = new java.util.HashSet<>(); + java.util.ArrayDeque pending = new java.util.ArrayDeque<>(graph.roots()); + while (!pending.isEmpty()) { + DocumentNode node = pending.pop(); + String anchor = node instanceof ParagraphNode paragraph ? paragraph.anchor() : blockAnchorOf(node); + if (anchor != null && !anchor.isBlank()) { + anchors.add(anchor.trim()); + } + pending.addAll(node.children()); + } + return anchors; + } + private static java.util.Set headingLevelsIn(DocumentGraph graph) { java.util.Set levels = new java.util.TreeSet<>(); for (DocumentNode root : graph.roots()) { @@ -2128,6 +2152,46 @@ private void writeShapeContainer(XWPFDocument document, ShapeContainerNode node) } } + /** + * Writes a page reference — a table of contents' page number, a "see page N" — as Word's + * own {@code PAGEREF} field on the anchor's bookmark. + * + *

The export dropped the node, so a table of contents reached Word with its entries + * and without a single page number. A number written as text would be right until the + * reader edits the document; the field is the page Word counts, a hyperlink to it + * ({@code \h}) as the entry's label already is. What it reads before an editor updates it + * is the page the layout resolved, so the file opens showing the numbers the PDF does.

+ * + *

A reference whose anchor this export writes no bookmark for is written as its text + * alone: Word turns a {@code PAGEREF} to a missing bookmark into "Error! Bookmark not + * defined." the first time the field updates, which is worse than a number that does not + * move.

+ */ + private void writePageReference(XWPFDocument document, + com.demcha.compose.document.node.PageReferenceNode node) { + String shown = layout.laidOutText(node).orElse(node.placeholderText()); + // The layout lays a page reference out as this paragraph, so its properties are + // written exactly as that paragraph's would be. + ParagraphNode asLaidOut = new ParagraphNode(node.name(), shown, node.textStyle(), node.align(), + 0.0, node.padding(), node.margin()); + XWPFParagraph para = newBodyParagraph(document); + applyParagraphProperties(para, asLaidOut); + applyLineHeight(para, layout.lineHeight(node)); + String bookmark = bookmarkedAnchors.contains(node.anchor()) + ? bookmarkNames.nameFor(node.anchor()) + : null; + XWPFRun run; + if (bookmark == null) { + run = para.createRun(); + } else { + CTSimpleField field = para.getCTP().addNewFldSimple(); + field.setInstr(" PAGEREF " + bookmark + " \\h "); + run = new XWPFRun(field.addNewR(), para); + } + applyStyle(run, node.textStyle()); + run.setText(shown); + } + private void writeParagraph(XWPFDocument document, ParagraphNode node) { XWPFParagraph para = newBodyParagraph(document); boolean rightToLeft = applyParagraphProperties(para, node); diff --git a/render-docx/src/test/java/com/demcha/compose/document/backend/semantic/docx/DocxPageReferenceTest.java b/render-docx/src/test/java/com/demcha/compose/document/backend/semantic/docx/DocxPageReferenceTest.java new file mode 100644 index 000000000..5764890f5 --- /dev/null +++ b/render-docx/src/test/java/com/demcha/compose/document/backend/semantic/docx/DocxPageReferenceTest.java @@ -0,0 +1,158 @@ +package com.demcha.compose.document.backend.semantic.docx; + +import com.demcha.compose.GraphCompose; +import com.demcha.compose.document.api.DocumentSession; +import com.demcha.compose.document.style.DocumentInsets; +import org.apache.poi.openxml4j.opc.PackagePart; +import org.apache.poi.openxml4j.opc.PackagingURIHelper; +import org.apache.poi.xwpf.usermodel.XWPFDocument; +import org.apache.poi.xwpf.usermodel.XWPFParagraph; +import org.apache.poi.xwpf.usermodel.XWPFTable; +import org.apache.poi.xwpf.usermodel.XWPFTableCell; +import org.apache.poi.xwpf.usermodel.XWPFTableRow; +import org.junit.jupiter.api.Test; +import org.openxmlformats.schemas.wordprocessingml.x2006.main.CTBookmark; +import org.openxmlformats.schemas.wordprocessingml.x2006.main.CTSimpleField; + +import java.io.ByteArrayInputStream; +import java.io.InputStream; +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * A page reference is Word's own {@code PAGEREF} field, reading the page the layout resolved. + * + *

The export dropped page references, so a table of contents reached Word with its entries + * and no page numbers. As a field the number is the page the editor counts — measured in + * LibreOffice, a field whose cached number was replaced by 99 still showed the real pages — + * and until an editor updates it, it reads the page the PDF shows.

+ * + * @author Artem Demchyshyn + */ +class DocxPageReferenceTest { + + @Test + void aTableOfContentsCarriesAFieldPerEntryReadingTheLaidOutPage() throws Exception { + try (XWPFDocument document = export(threeChapters())) { + Map fields = pageReferences(document); + + assertThat(fields).containsExactly( + Map.entry("PAGEREF intro \\h", "2"), + Map.entry("PAGEREF terms \\h", "3"), + Map.entry("PAGEREF prices \\h", "4")); + assertThat(bookmarkNames(document)) + .as("every field names a bookmark the document has, a section's anchor included") + .contains("intro", "terms", "prices"); + } + } + + @Test + void aStandalonePageReferenceIsAFieldToo() throws Exception { + try (XWPFDocument document = export(page -> page + .addPageReference("terms") + .addPageBreak(b -> { }) + .addSection(s -> s.anchor("terms").addParagraph(p -> p.text("Terms"))))) { + assertThat(pageReferences(document)).containsExactly(Map.entry("PAGEREF terms \\h", "2")); + } + } + + @Test + void aReferenceToAnAnchorTheDocumentDoesNotHaveIsPlainText() throws Exception { + try (XWPFDocument document = export(page -> page + .addPageReference("nowhere") + .addParagraph(p -> p.text("Body")))) { + assertThat(pageReferences(document)) + .as("Word would turn it into 'Error! Bookmark not defined.' on the first update") + .isEmpty(); + assertThat(document.getParagraphs().get(0).getText()) + .as("the placeholder the page prints for an unresolved reference — empty unless set") + .isEmpty(); + } + } + + @Test + void theDocumentDoesNotAskToUpdateItsFieldsOnOpen() throws Exception { + try (XWPFDocument document = export(threeChapters())) { + PackagePart settings = document.getPackage() + .getPart(PackagingURIHelper.createPartName("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/word/settings.xml")); + String xml; + try (InputStream input = settings.getInputStream()) { + xml = new String(input.readAllBytes(), StandardCharsets.UTF_8); + } + + // The fields open reading the layout's numbers; w:updateFields would only add a + // prompt asking the reader to recompute what is already right. + assertThat(xml).doesNotContain("updateFields"); + } + } + + private static java.util.function.Consumer threeChapters() { + return page -> page + .addTableOfContents(toc -> toc.title("Contents") + .entry("Intro", "intro") + .entry("Terms", "terms") + .entry("Prices", "prices")) + .addPageBreak(b -> { }) + .addParagraph(p -> p.text("Intro").anchor("intro")) + .addPageBreak(b -> { }) + .addSection(s -> s.anchor("terms").addParagraph(p -> p.text("Terms"))) + .addPageBreak(b -> { }) + .addSection(s -> s.anchor("prices").addParagraph(p -> p.text("Prices"))); + } + + private static XWPFDocument export( + java.util.function.Consumer content) throws Exception { + byte[] docx; + try (DocumentSession session = GraphCompose.document() + .pageSize(300, 300) + .margin(DocumentInsets.of(30)) + .create()) { + session.pageFlow(content::accept); + docx = session.toDocxBytes(); + } + return new XWPFDocument(new ByteArrayInputStream(docx)); + } + + /** Every PAGEREF field in the body and its tables, instruction to the text it reads. */ + private static Map pageReferences(XWPFDocument document) { + Map fields = new LinkedHashMap<>(); + for (XWPFParagraph paragraph : allParagraphs(document)) { + for (CTSimpleField field : paragraph.getCTP().getFldSimpleList()) { + String instruction = field.getInstr().trim(); + if (instruction.startsWith("PAGEREF")) { + StringBuilder text = new StringBuilder(); + field.getRList().forEach(run -> run.getTList().forEach(t -> text.append(t.getStringValue()))); + fields.put(instruction, text.toString()); + } + } + } + return fields; + } + + private static List bookmarkNames(XWPFDocument document) { + List names = new ArrayList<>(); + for (XWPFParagraph paragraph : allParagraphs(document)) { + for (CTBookmark bookmark : paragraph.getCTP().getBookmarkStartList()) { + names.add(bookmark.getName()); + } + } + return names; + } + + private static List allParagraphs(XWPFDocument document) { + List paragraphs = new ArrayList<>(document.getParagraphs()); + for (XWPFTable table : document.getTables()) { + for (XWPFTableRow row : table.getRows()) { + for (XWPFTableCell cell : row.getTableCells()) { + paragraphs.addAll(cell.getParagraphs()); + } + } + } + return paragraphs; + } +}