Move documentation files to keep into new structure
This commit is contained in:
Veronica Berglyd Olsen
@@ -0,0 +1,65 @@
|
||||
.. _docs_technical_locations:
|
||||
|
||||
**************
|
||||
File Locations
|
||||
**************
|
||||
|
||||
.. _QStandardPaths: https://doc.qt.io/qt-6/qstandardpaths.html
|
||||
|
||||
novelWriter will create a few files on your system outside of the application folder itself. These
|
||||
file locations are described in this chapter.
|
||||
|
||||
|
||||
.. _docs_technical_locations_conf:
|
||||
|
||||
Configuration
|
||||
=============
|
||||
|
||||
The general configuration of novelWriter, including everything that is in **Preferences**, is saved
|
||||
in one central configuration file. The location of this file depends on your operating system. The
|
||||
system paths are provided by the Qt QStandardPaths_ class and its ``ConfigLocation`` value.
|
||||
|
||||
The standard paths are:
|
||||
|
||||
* Linux: ``~/.config/novelwriter/novelwriter.conf``
|
||||
* MacOS: ``~/Library/Preferences/novelwriter/novelwriter.conf``
|
||||
* Windows: ``C:\Users\<USER>\AppData\Local\novelwriter\novelwriter.conf``
|
||||
|
||||
Here, ``~`` corresponds to the user's home directory on Linux and MacOS, and ``<USER>`` is the
|
||||
user's username on Windows.
|
||||
|
||||
.. note::
|
||||
These are the standard operating system defined locations. If your system has been set up in a
|
||||
different way, these locations may also be different.
|
||||
|
||||
|
||||
.. _docs_technical_locations_data:
|
||||
|
||||
Application Data
|
||||
================
|
||||
|
||||
novelWriter also stores a bit of data that is generated by the user's actions. This includes the
|
||||
list of recent projects form the **Welcome** dialog. Custom themes should also be saved here. The
|
||||
system paths are provided by the Qt QStandardPaths_ class and its ``AppDataLocation`` value.
|
||||
|
||||
The standard paths are:
|
||||
|
||||
* Linux: ``~/.local/share/novelwriter/``
|
||||
* MacOS: ``~/Library/Application Support/novelwriter/``
|
||||
* Windows: ``C:\Users\<USER>\AppData\Roaming\novelwriter\``
|
||||
|
||||
Here, ``~`` corresponds to the user's home directory on Linux and MacOS, and ``<USER>`` is the
|
||||
user's username on Windows.
|
||||
|
||||
.. note::
|
||||
These are the standard operating system defined locations. If your system has been set up in a
|
||||
different way, these locations may also be different.
|
||||
|
||||
The Application Data location also holds several folders:
|
||||
|
||||
``cache``
|
||||
This folder is used to save the preview data for the **Manuscript Build** tool.
|
||||
|
||||
``icons``, ``syntax`` and ``themes``
|
||||
These folders are empty by default, but this is where the user can store custom theme files.
|
||||
See :ref:`docs_more_custom` for more details.
|
||||
@@ -0,0 +1,161 @@
|
||||
.. _docs_technical_source:
|
||||
|
||||
*******************
|
||||
Running from Source
|
||||
*******************
|
||||
|
||||
.. _GitHub: https://github.com/vkbo/novelWriter/releases
|
||||
.. _PyPi: https://pypi.org/project/novelWriter/
|
||||
.. _Sphinx Docs: https://www.sphinx-doc.org/
|
||||
|
||||
This chapter describes various ways of running novelWriter directly from the source code, and how
|
||||
to build the various components like the translation files and documentation.
|
||||
|
||||
.. note::
|
||||
The text below assumes the command ``python`` corresponds to a Python 3 executable. Python 2 is
|
||||
now deprecated, but on many systems the command ``python3`` may be needed instead. Likewise,
|
||||
``pip`` may need to be replaced with ``pip3``.
|
||||
|
||||
Most of the custom commands for building packages of novelWriter, or building assets, are contained
|
||||
in the ``pkgutils.py`` script in the root of the source code. You can list the available commands
|
||||
by running:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
python pkgutils.py help
|
||||
|
||||
|
||||
.. _a_source_depend:
|
||||
|
||||
Dependencies
|
||||
============
|
||||
|
||||
novelWriter has been designed to rely on as few dependencies as possible. Only the Python wrapper
|
||||
for the Qt GUI libraries is required. The package for spell checking is optional, but recommended.
|
||||
Everything else is handled with standard Python libraries.
|
||||
|
||||
The following Python packages are needed to run all features of novelWriter:
|
||||
|
||||
* ``PyQt6`` – needed for connecting with the Qt6 libraries.
|
||||
* ``PyEnchant`` – needed for spell checking (optional).
|
||||
|
||||
If you want spell checking, you must install the ``PyEnchant`` package. The spell check library
|
||||
must be at least 3.0 to work with Windows. On Linux, 2.0 also works fine.
|
||||
|
||||
If you install from PyPi, these dependencies should be installed automatically. If you install from
|
||||
source, dependencies can still be installed from PyPi with:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install -r requirements.txt
|
||||
|
||||
.. note::
|
||||
On Linux distros, the Qt library is usually split up into multiple packages. In some cases,
|
||||
secondary dependencies may not be installed automatically. For novelWriter, the library files
|
||||
for rendering the SVG icons may be left out and needs to be installed manually. This is the
|
||||
case on for instance Arch Linux.
|
||||
|
||||
|
||||
.. _a_source_install:
|
||||
|
||||
Build and Install from Source
|
||||
=============================
|
||||
|
||||
If you want to install novelWriter directly from the source available on GitHub_, you must first
|
||||
build the package using the Python Packaging Authority's build tool. It can be installed with:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install build
|
||||
|
||||
On Debian-based systems the tool can also be installed with:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo apt install python3-build
|
||||
|
||||
With the tool installed, run the following command from the root of the novelWriter source code:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
python -m build --wheel
|
||||
|
||||
This should generate a ``.whl`` file in the ``dist/`` folder at your current location. The wheel
|
||||
file can then be installed on your system. Here with example version number 2.0.7, but yours may be
|
||||
different:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install --user dist/novelWriter-2.0.7-py3-none-any.whl
|
||||
|
||||
|
||||
.. _a_source_i18n:
|
||||
|
||||
Building the Translation Files
|
||||
==============================
|
||||
|
||||
If you installed novelWriter from a package, the translation files should be pre-built and
|
||||
included. If you're running novelWriter from the source code, you will need to generate the files
|
||||
yourself. The files you need will be written to the ``novelwriter/assets/i18n`` folder, and will
|
||||
have the ``.qm`` file extension.
|
||||
|
||||
You can build the ``.qm`` files with:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
python pkgutils.py qtlrelease
|
||||
|
||||
This requires that the Qt Linguist tool is installed on your system. On Ubuntu and Debian, the
|
||||
needed package is called ``qttools5-dev-tools``.
|
||||
|
||||
.. note::
|
||||
If you want to improve novelWriter with translation files for another language, or update an
|
||||
existing translation, instructions for how to contribute can be found in the ``README.md`` file
|
||||
in the ``i18n`` folder of the source code.
|
||||
|
||||
|
||||
.. _a_source_sample:
|
||||
|
||||
Building the Example Project
|
||||
============================
|
||||
|
||||
In order to be able to create new projects from example files, you need a ``sample.zip`` file in
|
||||
the ``assets`` folder of the source. This file can be built from the ``pkgutils.py`` script by
|
||||
running:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
python pkgutils.py sample
|
||||
|
||||
|
||||
.. _a_source_docs:
|
||||
|
||||
Building the Documentation
|
||||
==========================
|
||||
|
||||
A local copy of this documentation can be generated as HTML. This requires installing some Python
|
||||
packages from PyPi:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install -r docs/requirements.txt
|
||||
|
||||
The documentation can then be built from the root folder in the source code by running:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
make -C docs html
|
||||
|
||||
If successful, the documentation should be available in the ``docs/build/html`` folder and you can
|
||||
open the ``index.html`` file in your browser.
|
||||
|
||||
You can also build a PDF manual from the documentation using the ``pkgutils.py`` script:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
python pkgutils.py docs-pdf en
|
||||
|
||||
This will build the English documentation as a PDF using LaTeX. The file will then be copied into
|
||||
the assets folder and made available in the **Help** menu in novelWriter. Replace ``en`` with
|
||||
``all`` to build for all languages. The Sphinx build system has a few extra dependencies when
|
||||
building the PDF. Please check the `Sphinx Docs`_ for more details.
|
||||
@@ -0,0 +1,151 @@
|
||||
.. _docs_technical_storage:
|
||||
|
||||
******************
|
||||
How Data is Stored
|
||||
******************
|
||||
|
||||
This chapter contains details of how novelWriter stores and handles the project data.
|
||||
|
||||
|
||||
Project Structure
|
||||
=================
|
||||
|
||||
All novelWriter files are written with utf-8 encoding. Since Python automatically converts Unix
|
||||
line endings to Windows line endings on Windows systems, novelWriter does not make any adaptations
|
||||
to the formatting on Windows systems. This is handled entirely by the Python standard library.
|
||||
Python also handles this when working on the same files on both Windows and Unix-based operating
|
||||
systems.
|
||||
|
||||
|
||||
Main Project File
|
||||
-----------------
|
||||
|
||||
The project itself requires a dedicated folder for storing its files, where novelWriter will create
|
||||
its own "file system" where the project's folder and file hierarchy is described in a project XML
|
||||
file. This is the main project file in the project's root folder with the name ``nwProject.nwx``.
|
||||
This file also contains all the meta data required for the project (except the index data), and a
|
||||
number of related project settings.
|
||||
|
||||
If this file is lost or corrupted, the structure of the project is lost, although not the text
|
||||
itself. It is important to keep this file backed up, either through the built-in backup tool, or
|
||||
your own backup solution.
|
||||
|
||||
The project XML file is indent-formatted, and is suitable for diff tools and version control since
|
||||
most of the file will stay static, although a timestamp is set in the meta section on line 2, and
|
||||
various meta data entries incremented, on each save.
|
||||
|
||||
A full project file format specification is available under "More Documents".
|
||||
|
||||
|
||||
Project Documents
|
||||
=================
|
||||
|
||||
All the project documents are saved in a subfolder of the main project folder named ``content``.
|
||||
Each document has a file handle based on a 52 bit random number, represented as a hexadecimal
|
||||
string. The documents are saved with a filename assembled from this handle and the file extension
|
||||
``.nwd``.
|
||||
|
||||
If you wish to find the file system location of a document in the project, you can either look it
|
||||
up in the project XML file, select **Show File Details** from the **Document** menu when having the
|
||||
document open in the editor, or look in the ``ToC.txt`` file in the root of the project folder. The
|
||||
``ToC.txt`` file has a list of all documents in the project, referenced by their label, and where
|
||||
they are saved.
|
||||
|
||||
The reason for this cryptic file naming is to avoid issues with file naming conventions and
|
||||
restrictions on different operating systems, and also to have a file name that does not depend on
|
||||
what you name the document within the project, or changes it to. This is particularly useful when
|
||||
using a versioning system.
|
||||
|
||||
Each document file contains a plain text version of the text from the editor. The file can in
|
||||
principle be edited in any text editor, and is suitable for diffing and version control if so
|
||||
desired. Just make sure the file remains in utf-8 encoding, otherwise unicode characters may
|
||||
become mangled when the file is opened in novelWriter again.
|
||||
|
||||
Editing these files is generally not recommended. The reason for this is that the index will not be
|
||||
automatically updated when doing so, which means novelWriter doesn't know you've altered the file.
|
||||
If you *do* edit a file in this manner, you should rebuild the index when you next open the project
|
||||
in novelWriter.
|
||||
|
||||
The first lines of the file may contain some meta data starting with the characters ``%%~``. These
|
||||
lines are mainly there to restore some information if the file is lost from the main project file,
|
||||
and the information may be helpful if you do open the file in an external editor as it contains the
|
||||
document label and the document class and layout. The lines can be deleted without any consequences
|
||||
to the rest of the content of the file, and will be added back the next time the document is saved
|
||||
in novelWriter.
|
||||
|
||||
|
||||
The File Saving Process
|
||||
-----------------------
|
||||
|
||||
When saving the project file, or any of the documents, the data is first saved to a temporary file.
|
||||
If successful, the old data file is then removed, and the temporary file replaces it. This ensures
|
||||
that the previously saved data is only replaced when the new data has been successfully saved to
|
||||
the storage medium.
|
||||
|
||||
|
||||
Project Meta Data
|
||||
=================
|
||||
|
||||
The project folder contains a subfolder named ``meta``, containing a number of files. The meta
|
||||
folder contains semi-important files. That is, they can be lost with only minor impact to the
|
||||
project. All files in this folder are JSON or JSON Lines files, although some other files may
|
||||
remain from earlier versions of novelWriter as they haven't all been JSON files in the past.
|
||||
|
||||
If you use version control software on your project, you can exclude this folder, although you may
|
||||
want to track the session log file and the custom words list.
|
||||
|
||||
|
||||
The Project Index
|
||||
-----------------
|
||||
|
||||
Between writing sessions, the project index is saved in a JSON file in ``meta/index.json``.
|
||||
This file is not critical. If it is lost, it can be completely rebuilt from within novelWriter from
|
||||
the **Tools** menu.
|
||||
|
||||
The index is maintained and updated whenever a document or note is saved in the editor. It contains
|
||||
all references and tags in documents and notes, as well as the location of all headers in the
|
||||
project, and the word counts within each header section.
|
||||
|
||||
The integrity of the index is checked when the file is loaded. It is possible to corrupt the index
|
||||
if the file is manually edited and manipulated, so the check is important to avoid sudden crashes
|
||||
of novelWriter. If the file contains errors, novelWriter will automatically build it anew. If the
|
||||
check somehow fails and novelWriter keeps crashing, you can delete the file manually and rebuild
|
||||
the index. If this too fails, you have likely encountered a bug.
|
||||
|
||||
|
||||
Build Definitions
|
||||
-----------------
|
||||
|
||||
The build definitions from the **Manuscript Build** tool are kept in the ``meta/builds.json`` file.
|
||||
If this file is lost, all custom build definitions are lost too.
|
||||
|
||||
|
||||
Cached GUI Options
|
||||
------------------
|
||||
|
||||
A file named ``meta/options.json`` contains the latest state of various GUI buttons, switches,
|
||||
dialog window sizes, column sizes, etc, from the GUI. These are the GUI settings that are specific
|
||||
to the project. Global GUI settings are stored in the main config file.
|
||||
|
||||
The file is not critical, but if it is lost, all such GUI options will revert back to their default
|
||||
settings.
|
||||
|
||||
|
||||
Custom Word List
|
||||
----------------
|
||||
|
||||
A file named ``meta/userdict.json`` contains all the custom words you've added to the project for
|
||||
spell checking purposes. The content of the file can be edited from the **Tools** menu. If you lose
|
||||
this file, all your custom spell check words will be lost too.
|
||||
|
||||
|
||||
Session Stats
|
||||
-------------
|
||||
|
||||
The writing progress is saved in the ``meta/sessions.jsonl`` file. This file records the length
|
||||
and word counts of each writing session on the given project. The file is used by the **Writing
|
||||
Statistics** tool. If this file is lost, the history it contains is also lost, but it has otherwise
|
||||
no impact on the project.
|
||||
|
||||
Each session is recorded as a JSON object on a single line of the file. Each session record is
|
||||
appended tot he file.
|
||||
@@ -0,0 +1,84 @@
|
||||
.. _docs_technical_pytest:
|
||||
|
||||
*************
|
||||
Running Tests
|
||||
*************
|
||||
|
||||
The novelWriter source code is well covered by tests. The test framework used for the development
|
||||
is ``pytest`` with the use of an extension for Qt.
|
||||
|
||||
|
||||
Dependencies
|
||||
============
|
||||
|
||||
The dependencies for running the tests can be installed with:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install -r tests/requirements.txt
|
||||
|
||||
This will install a couple of extra packages for coverage and test management. The minimum
|
||||
requirement is ``pytest`` and ``pytest-qt``.
|
||||
|
||||
|
||||
Simple Test Run
|
||||
===============
|
||||
|
||||
To run the tests, you simply need to execute the following from the root of the source folder:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pytest
|
||||
|
||||
Since several of the tests involve opening up the novelWriter GUI, you may want to disable the GUI
|
||||
for the duration of the test run. Moving your mouse while the tests are running may otherwise
|
||||
interfere with the execution of some tests.
|
||||
|
||||
You can disable the renderring of the GUI by setting the flag ``QT_QPA_PLATFORM=offscreen``:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export QT_QPA_PLATFORM=offscreen pytest
|
||||
|
||||
|
||||
Advanced Options
|
||||
================
|
||||
|
||||
Adding the flag ``-v`` to the ``pytest`` command will increase verbosity of the test execution.
|
||||
|
||||
You can also add coverage report generation. For instance to HTML:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export QT_QPA_PLATFORM=offscreen pytest -v --cov=novelwriter --cov-report=html
|
||||
|
||||
Other useful report formats are ``xml``, and ``term`` for terminal output.
|
||||
|
||||
You can also run tests per subpackage of novelWriter with the ``-m`` command. The available
|
||||
subpackage groups are ``base``, ``core``, and ``gui``. Consider for instance:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export QT_QPA_PLATFORM=offscreen pytest -v --cov=novelwriter --cov-report=html -m core
|
||||
|
||||
This will only run the tests of the "core" package, that is, all the classes that deal with the
|
||||
project data of a novelWriter project. The "gui" tests, likewise, will run the tests for the GUI
|
||||
components, and the "base" tests cover the bits in-between.
|
||||
|
||||
You can also filter the tests with the ``-k`` switch. The following will do the same as
|
||||
``-m core``:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export QT_QPA_PLATFORM=offscreen pytest -v --cov=novelwriter --cov-report=html -k testCore
|
||||
|
||||
All tests are named in such a way that you can filter them by adding more bits of the test names.
|
||||
They all start with the word "test". Then comes the group: "Core", "Base", "Dlg", "Tool", or "Gui".
|
||||
Finally comes the name of the class or module, which generally corresponds to a single source code
|
||||
file. For instance, running the following will run all tests for the document editor:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export QT_QPA_PLATFORM=offscreen pytest -v --cov=novelwriter --cov-report=html -k testGuiEditor
|
||||
|
||||
To run a single test, simply add the full test name to the ``-k`` switch.
|
||||
Reference in New Issue
Block a user