Merge 2.7.2 into main (#2432)

CHANGELOG.md

@@ -1,5 +1,45 @@
 # novelWriter Changelog
 
+## Version 2.7.2 [2025-06-24]
+
+### Release Notes
+
+This is a patch release that fixes several issues related to DocX and PDF manuscript documents,
+improves how line breaks, alignment and indentation is processed, and fixes some issues with
+displaying tags for novel documents on the user interface.
+
+### Detailed Changelog
+
+**Bugfixes**
+
+* Fixed an issue where escaped markup characters were not replaced properly in DocX and PDF
+  documents, and in previews. Issue #2410. PR #2411.
+* Fixed an issue where titles with line breaks in them would have page breaks applied to both lines
+  for preview and PDF documents. Issue #2415. PR #2416.
+* Fixed an issue where the justified text setting would not be properly applied to a paragraph in a
+  manuscript document if there was a line break in the paragraph, but single line breaks were set
+  to be ignored. Issue #2426. PR #2427.
+* Fixed an issue where the "Tag" field of the Outline View details panel remained blank even if a
+  tag was set for the novel document. Issue #2428. PR #2429.
+
+**Improvements**
+
+* When comments are enabled in the viewer, story comments should also be visible. A separate filter
+  button will be added for this in 2.8. PR #2420.
+* When a paragraph has line breaks in it, the alignment tag will now override first line
+  indentation even if the alignment tag is not on the first line. This is more consistent with the
+  alignment behaviour for multi-line paragraphs in general. Issue #2425. PR #2427.
+* Tags will now be shown in the Novel View tooltip pop-out under the triangle button, together with
+  all the other meta data collected about a document or heading. PR #2429.
+
+**Documentation**
+
+* The documentation on how alignment and first line indentation works in conjunction with
+  in-paragraph line breaks, the setting to keep or ignore such line breaks, and the forced line
+  break shortcode, has been improved. Issue #2425. PR #2427.
+
+----
+
 ## Version 2.7.1 [2025-06-09]
 
 ### Release Notes

docs/source/usage/advanced_formatting.rst

@@ -56,6 +56,20 @@ activated by clicking the left-most icon button in the editor header.
 .. versionadded:: 2.2
 
 
+.. _docs_usage_formatting_shortcodes_break:
+
+Forced Line Break
+-----------------
+
+Inserting ``[br]`` in the text will ensure a line break is always inserted in that place, even if
+you turn off **Preserve Hard Line Breaks** in your manuscript build settings.
+
+You can add a manual line break after it too, for a better visual representation in the editor, but
+keep in mind that this line break is removed before the text is processed, so the text on either
+side of the ``[br]`` shortcode will be considered as belonging to the same line. This can affect
+how alignment is treated. See :ref:`docs_usage_align_indent_forced` for more details.
+
+
 .. _docs_usage_formatting_breaks:
 
 Vertical Space and Page Breaks

docs/source/usage/alignment_and_indent.rst

@@ -53,18 +53,33 @@ the entire paragraph. For the following text, all lines will be centred:
 
 .. code-block:: md
 
-   >> I am the very model of a modern Major-General
+   >> I am the very model of a modern Major-General <<
    I've information vegetable, animal, and mineral
    I know the kings of England, and I quote the fights historical
-   From Marathon to Waterloo, in order categorical <<
+   From Marathon to Waterloo, in order categorical
+
+If you have multiple conflicting alignments on a paragraph, only one is applied. The order of
+precedence is:
+
+#. Left alignment
+#. Right alignment
+#. Centred text
+#. Justified text
+
+.. note::
+
+   It is strongly recommended that you keep the **Preserve Hard Line Breaks** setting enabled in
+   your manuscript build settings. This setting assumes all single line breaks in your text are
+   intended. Turning this off makes adding line breaks more complicated, but it is still possible.
+   See :ref:`docs_usage_align_indent_forced`.
 
 
 Alignment with First Line Indent
 ================================
 
 If you have first line indent enabled in your manuscript build settings, you probably want to
-disable it for text in verses. Adding any alignment tags will cause the first line indent to be
+disable it for text in verses. Adding any alignment tags on a paragraph will cause the first
-switched off for that paragraph.
+line indent to be switched off for that paragraph.
 
 :bdg-info:`Example`
 
@@ -76,3 +91,35 @@ The following text will always be aligned against the left margin:
    I've information vegetable, animal, and mineral
    I know the kings of England, and I quote the fights historical
    From Marathon to Waterloo, in order categorical
+
+
+.. _docs_usage_align_indent_forced:
+
+Alignment with Forced Line Breaks
+=================================
+
+If you turn off **Preserve Hard Line Breaks** in your manuscript build settings, you can still
+force line breaks in paragraphs using the ``[br]`` shortcode. For clarity in the text, you can add
+a line break after it as well. It doesn't result in two line breaks.
+
+Keep in mind that when the text is processed, the lines on either side of a ``[br]`` shortcode are
+combined, and any trailing hard line break is *ignored*. This means that when such a paragraph is
+processed, these line breaks count as the same line. This affects how alignment tags are handled.
+For instance, this text becomes centred instead of left aligned.
+
+.. code-block:: md
+
+   >> I am the very model of a modern Major-General[br]
+   I've information vegetable, animal, and mineral[br]
+   I know the kings of England, and I quote the fights historical[br]
+   From Marathon to Waterloo, in order categorical <<
+
+Since this is understood as one line, this is the only way you can actually centre this paragraph.
+
+.. caution::
+
+   Due to this difference in how text with ``[br]`` tags are processed, it is generally better to
+   stick with the **Preserve Hard Line Breaks** setting enabled. It ensures a better correspondence
+   between what you see in the editor and what output you get.
+
+See also :ref:`docs_usage_formatting_shortcodes_break`.

novelwriter/core/index.py

@@ -708,17 +708,19 @@ class Index:
         return 0, 0, 0
 
     def getReferences(self, tHandle: str, sTitle: str | None = None) -> dict[str, list[str]]:
-        """Extract all references made in a file, and optionally title
+        """Extract all tags and references made in a file, and
-        section.
+        optionally title section.
         """
-        tRefs = {x: [] for x in nwKeyWords.VALID_KEYS}
+        refs = {x: [] for x in nwKeyWords.VALID_KEYS}
         for rTitle, hItem in self._itemIndex.iterItemHeaders(tHandle):
             if sTitle is None or sTitle == rTitle:
                 for aTag, refTypes in hItem.references.items():
                     for refType in refTypes:
-                        if refType in tRefs:
+                        if refType in refs:
-                            tRefs[refType].append(self._tagsIndex.tagName(aTag))
+                            refs[refType].append(self._tagsIndex.tagName(aTag))
-        return tRefs
+                if tag := hItem.tag:
+                    refs[nwKeyWords.TAG_KEY] = [self._tagsIndex.tagName(tag)]
+        return refs
 
     def getReferenceForHeader(self, tHandle: str, nHead: int, keyClass: str) -> list[str]:
         """Get the display names for a tags class for insertion into a

novelwriter/core/indexdata.py

@@ -342,20 +342,26 @@ class IndexHeading:
     ##
 
     def getReferences(self) -> dict[str, list[str]]:
-        """Extract all references for this heading."""
+        """Extract all tags and references for this heading."""
         refs = {x: [] for x in nwKeyWords.VALID_KEYS}
         for tag, types in self._refs.items():
             for keyword in types:
                 if keyword in refs and (name := self._cache.tags.tagName(tag)):
                     refs[keyword].append(name)
+            if name := self._cache.tags.tagName(self._tag):
+                refs[nwKeyWords.TAG_KEY] = [name]
         return refs
 
     def getReferencesByKeyword(self, keyword: str) -> list[str]:
         """Extract all references for this heading."""
         refs = []
-        for tag, types in self._refs.items():
+        if keyword == nwKeyWords.TAG_KEY:
-            if keyword in types and (name := self._cache.tags.tagName(tag)):
+            if name := self._cache.tags.tagName(self._tag):
                 refs.append(name)
+        else:
+            for tag, types in self._refs.items():
+                if keyword in types and (name := self._cache.tags.tagName(tag)):
+                    refs.append(name)
         return refs
 
     ##

novelwriter/formats/tokenizer.py

@@ -893,31 +893,21 @@ class Tokenizer(ABC):
                 if nBlock[0] != BlockTyp.TEXT:
                     # Next block is not text, so we add the buffer to blocks
                     nLines = len(pLines)
-                    cStyle = pLines[0][4]
+                    tFmt: T_Formats = []
-                    if firstIndent and not (self._noIndent or cStyle & BlockFmt.ALIGNED):
+                    pTxt = ""
-                        # If paragraph indentation is enabled, not temporarily
+                    cStyle = BlockFmt.NONE
-                        # turned off, and the block is not aligned, we add the
-                        # text indentation flag
-                        cStyle |= BlockFmt.IND_T
 
                     if nLines == 1:
-                        # The paragraph contains a single line, so we just save
+                        # The paragraph contains a single line
-                        # that directly to the blocks list. If justify is
+                        tFmt = pLines[0][3]
-                        # enabled, and there is no alignment, we apply it.
-                        if doJustify and not cStyle & BlockFmt.ALIGNED:
-                            cStyle |= BlockFmt.JUSTIFY
-
                         pTxt = pLines[0][2].translate(transMapB)
-                        sBlocks.append((
+                        cStyle = pLines[0][4]
-                            BlockTyp.TEXT, pLines[0][1], pTxt, pLines[0][3], cStyle
-                        ))
 
                     elif nLines > 1:
                         # The paragraph contains multiple lines, so we need to
                         # join them according to the line break policy, and
                         # recompute all the formatting markers
                         tTxt = ""
-                        tFmt: T_Formats = []
                         for aBlock in pLines:
                             tLen = len(tTxt)
                             tTxt += f"{aBlock[2]}{lineSep}"
@@ -925,6 +915,18 @@ class Tokenizer(ABC):
                             cStyle |= aBlock[4]
 
                         pTxt = tTxt[:-1].translate(transMapB)
+
+                    if nLines:
+                        isAligned = cStyle & BlockFmt.ALIGNED
+                        if firstIndent and not (self._noIndent or isAligned):
+                            # If paragraph indentation is enabled, not temporarily
+                            # turned off, and the block is not aligned, we add the
+                            # text indentation flag
+                            cStyle |= BlockFmt.IND_T
+
+                        if doJustify and not isAligned:
+                            cStyle |= BlockFmt.JUSTIFY
+
                         sBlocks.append((
                             BlockTyp.TEXT, pLines[0][1], pTxt, tFmt, cStyle
                         ))

novelwriter/gui/noveltree.py

@@ -593,6 +593,7 @@ class GuiNovelTree(NTreeView):
             lines = []
             if head := SHARED.project.index.getItemHeading(tHandle, sTitle):
                 tags = head.getReferences()
+                appendTags(tags, nwKeyWords.TAG_KEY, lines)
                 appendTags(tags, nwKeyWords.POV_KEY, lines)
                 appendTags(tags, nwKeyWords.FOCUS_KEY, lines)
                 appendTags(tags, nwKeyWords.CHAR_KEY, lines)

sample/content/636b6aa9b697b.nwd

@@ -1,10 +1,11 @@
 %%~name: Making a Scene
 %%~path: 6a2d6d5f4f401/636b6aa9b697b
 %%~kind: NOVEL/DOCUMENT
-%%~hash: b85f815702763f58926edd72a2eeed1df5e1cbc9
+%%~hash: c1a75b18145e49e5e71b5284f66a261207da37d0
 %%~date: Unknown/2025-06-16 12:49:10
 ### Making a Scene
 
+@tag: Scene
 @pov: Jane
 @char: John, Jane
 @location: Earth

sample/nwProject.nwx

@@ -1,6 +1,6 @@
 <?xml version='1.0' encoding='utf-8'?>
-<novelWriterXML appVersion="2.8a1" hexVersion="0x020800a1" fileVersion="1.5" fileRevision="6" timeStamp="2025-06-16 12:51:59">
+<novelWriterXML appVersion="2.8a1" hexVersion="0x020800a1" fileVersion="1.5" fileRevision="6" timeStamp="2025-06-24 17:12:20">
-  <project id="e2be99af-f9bf-4403-857a-c3d1ac25abea" saveCount="2234" autoCount="290" editTime="98009">
+  <project id="e2be99af-f9bf-4403-857a-c3d1ac25abea" saveCount="2236" autoCount="290" editTime="98026">
     <name>Sample Project</name>
     <author>Jane Smith</author>
   </project>
@@ -58,7 +58,7 @@
       <name status="sf24ce6" import="ia857f0" active="yes">Chapter One</name>
     </item>
     <item handle="636b6aa9b697b" parent="6a2d6d5f4f401" root="7031beac91f75" order="0" type="FILE" class="NOVEL" layout="DOCUMENT">
-      <meta expanded="no" heading="H3" charCount="3003" wordCount="530" paraCount="16" cursorPos="1493" />
+      <meta expanded="no" heading="H3" charCount="3003" wordCount="530" paraCount="16" cursorPos="586" />
       <name status="s90e6c9" import="ia857f0" active="yes">Making a Scene</name>
     </item>
     <item handle="bc0cbd2a407f3" parent="6a2d6d5f4f401" root="7031beac91f75" order="1" type="FILE" class="NOVEL" layout="DOCUMENT">

tests/test_core/test_core_index.py

@@ -695,6 +695,7 @@ def testCoreIndex_ExtractData(nwGUI, fncPath, mockRnd):
     ))
     assert index.scanText(nHandle, (
         "# Hello World!\n"
+        "@tag: Scene\n"
         "@pov: Jane\n"
         "@char: Jane, John\n\n"
         "% this is a comment\n\n"
@@ -749,11 +750,13 @@ def testCoreIndex_ExtractData(nwGUI, fncPath, mockRnd):
 
     # Look up an invalid handle
     refs = index.getReferences("Not a handle")
+    assert refs["@tag"] == []
     assert refs["@pov"] == []
     assert refs["@char"] == []
 
     # The novel file should now refer to Jane as @pov and @char
     refs = index.getReferences(nHandle)
+    assert refs["@tag"] == ["Scene"]
     assert refs["@pov"] == ["Jane"]
     assert refs["@char"] == ["Jane", "John"]
 
@@ -791,7 +794,7 @@ def testCoreIndex_ExtractData(nwGUI, fncPath, mockRnd):
 
     # getKeyWordTags
     # ==============
-    assert index.getKeyWordTags("@mention") == ["Jane", "John"]
+    assert index.getKeyWordTags("@mention") == ["Jane", "John", "Scene"]
     assert index.getKeyWordTags("@char") == ["Jane", "John"]
     assert index.getKeyWordTags("@plot") == []
     assert index.getKeyWordTags("@tag") == []

tests/test_core/test_core_indexdata.py

@@ -267,6 +267,7 @@ def testCoreIndexData_IndexHeadingReferences():
     head = IndexHeading(cache, "T0001")
 
     # Add some references
+    head.setTag("Scene")
     head.addReference("Jane", "@pov")
     head.addReference("Jane", "@char")
     head.addReference("John", "@char")
@@ -290,6 +291,7 @@ def testCoreIndexData_IndexHeadingReferences():
     }
 
     # Set names
+    cache.tags.add("Scene", "Scene", "0000000000000", "T00001", "NOVEL")
     cache.tags.add("Jane", "Jane", "0000000000000", "T00001", "CHARACTER")
     cache.tags.add("John", "John", "0000000000000", "T00001", "CHARACTER")
     cache.tags.add("Main", "Main", "0000000000000", "T00001", "PLOT")
@@ -301,7 +303,7 @@ def testCoreIndexData_IndexHeadingReferences():
         "@plot": ["Main"],
         "@object": ["Gun"],
         "@story": [],
-        "@tag": [],
+        "@tag": ["Scene"],
         "@focus": [],
         "@custom": [],
         "@time": [],
@@ -316,7 +318,7 @@ def testCoreIndexData_IndexHeadingReferences():
     assert head.getReferencesByKeyword("@plot") == ["Main"]
     assert head.getReferencesByKeyword("@object") == ["Gun"]
     assert head.getReferencesByKeyword("@story") == []
-    assert head.getReferencesByKeyword("@tag") == []
+    assert head.getReferencesByKeyword("@tag") == ["Scene"]
     assert head.getReferencesByKeyword("@focus") == []
     assert head.getReferencesByKeyword("@custom") == []
     assert head.getReferencesByKeyword("@time") == []

tests/test_formats/test_fmt_tohtml.py

@@ -325,6 +325,54 @@ def testFmtToHtml_ConvertMeta(mockGUI):
     )
 
 
+@pytest.mark.core
+def testFmtToHtml_Alignment(mockGUI):
+    """Test paragraph alignment in the ToHtml class."""
+    project = NWProject()
+    html = ToHtml(project)
+    html.initDocument()
+
+    # Left
+    html._text = "This is text <<\nspanning multiple\nlines"
+    html.tokenizeText()
+    html.doConvert()
+    assert html._pages[-1] == (
+        "<p style='text-align: left;'>This is text<br>spanning multiple<br>lines</p>\n"
+    )
+
+    # Right
+    html._text = ">> This is text\nspanning multiple\nlines"
+    html.tokenizeText()
+    html.doConvert()
+    assert html._pages[-1] == (
+        "<p style='text-align: right;'>This is text<br>spanning multiple<br>lines</p>\n"
+    )
+
+    # Centre
+    html._text = ">> This is text <<\nspanning multiple\nlines"
+    html.tokenizeText()
+    html.doConvert()
+    assert html._pages[-1] == (
+        "<p style='text-align: center;'>This is text<br>spanning multiple<br>lines</p>\n"
+    )
+
+    # Left before Right
+    html._text = ">> This is text\nspanning multiple <<\nlines"
+    html.tokenizeText()
+    html.doConvert()
+    assert html._pages[-1] == (
+        "<p style='text-align: left;'>This is text<br>spanning multiple<br>lines</p>\n"
+    )
+
+    # Right before Centre
+    html._text = ">> This is text <<\n>> spanning multiple\nlines"
+    html.tokenizeText()
+    html.doConvert()
+    assert html._pages[-1] == (
+        "<p style='text-align: right;'>This is text<br>spanning multiple<br>lines</p>\n"
+    )
+
+
 @pytest.mark.core
 def testFmtToHtml_Dialog(mockGUI):
     """Test paragraph formats in the ToHtml class."""

tests/test_formats/test_fmt_tokenizer.py

@@ -1118,6 +1118,80 @@ def testFmtToken_Paragraphs(mockGUI):
     ]
 
 
+@pytest.mark.core
+def testFmtToken_BreakAlignIndent(mockGUI):
+    """Test the splitting of paragraphs with alignment."""
+    project = NWProject()
+    tokens = BareTokenizer(project)
+    tokens._handle = TMH
+
+    for text in [
+        "This is text <<\nspanning multiple\nlines",
+        "This is text\nspanning multiple <<\nlines",
+        "This is text\nspanning multiple\nlines <<",
+    ]:
+        # Preserve Breaks
+        tokens.setKeepLineBreaks(True)
+        tokens._text = text
+        tokens.tokenizeText()
+        assert tokens._blocks == [
+            (BlockTyp.TEXT, "", "This is text\nspanning multiple\nlines", [], BlockFmt.LEFT),
+        ]
+
+        # Don't Preserve Breaks
+        tokens.setKeepLineBreaks(False)
+        tokens._text = text
+        tokens.tokenizeText()
+        assert tokens._blocks == [
+            (BlockTyp.TEXT, "", "This is text spanning multiple lines", [], BlockFmt.LEFT),
+        ]
+
+        # With Justify
+        # This should disable justify
+        tokens.setKeepLineBreaks(True)
+        tokens.setJustify(True)
+        tokens._text = text
+        tokens.tokenizeText()
+        assert tokens._blocks == [
+            (BlockTyp.TEXT, "", "This is text\nspanning multiple\nlines", [], BlockFmt.LEFT),
+        ]
+
+        # With Indent
+        # This should disable indent
+        tokens.setKeepLineBreaks(True)
+        tokens.setFirstLineIndent(True, 1.0, False)
+        tokens._text = text
+        tokens.tokenizeText()
+        assert tokens._blocks == [
+            (BlockTyp.TEXT, "", "This is text\nspanning multiple\nlines", [], BlockFmt.LEFT),
+        ]
+
+
+@pytest.mark.core
+def testFmtToken_BreakJustify(mockGUI):
+    """Test the of processing of justify with breaks."""
+    project = NWProject()
+    tokens = BareTokenizer(project)
+    tokens._handle = TMH
+    tokens.setJustify(True)
+
+    # Applied to all lines when breaks are preserved
+    tokens._text = "This is text\nspanning multiple\nlines"
+    tokens.setKeepLineBreaks(True)
+    tokens.tokenizeText()
+    assert tokens._blocks == [
+        (BlockTyp.TEXT, "", "This is text\nspanning multiple\nlines", [], BlockFmt.JUSTIFY),
+    ]
+
+    # Turning off breaks should make no difference (see issue #2426)
+    tokens._text = "This is text\nspanning multiple\nlines"
+    tokens.setKeepLineBreaks(False)
+    tokens.tokenizeText()
+    assert tokens._blocks == [
+        (BlockTyp.TEXT, "", "This is text spanning multiple lines", [], BlockFmt.JUSTIFY),
+    ]
+
+
 @pytest.mark.core
 def testFmtToken_TextFormat(mockGUI):
     """Test the tokenization of text formats in the Tokenizer class."""

tests/test_formats/test_fmt_toodt.py

@@ -733,7 +733,7 @@ def testFmtToOdt_ConvertParagraphs(mockGUI):
         '<office:text>'
         '<text:h text:style-name="Heading_20_2" text:outline-level="2">Scene</text:h>'
         '<text:p text:style-name="P7">Regular paragraph</text:p>'
-        '<text:p text:style-name="Text_20_body">with<text:line-break />break</text:p>'
+        '<text:p text:style-name="P7">with<text:line-break />break</text:p>'
         '<text:p text:style-name="Text_20_body">Left Align</text:p>'
         '</office:text>'
     )