Matrix of the authorizations

☝️ Setup the server

Refer to the file README.md available at the root directory of the application to know more about the default setup.

Running commands

By design, the most important configuration cannot be done online, so you will have to invoke the administration console. Execute the command python3 pwic_admin.py [arguments] under Linux or python pwic_admin.py [arguments] under Windows.

To simplify this syntax, just type ./pa [arguments] under Linux or pa.bat [arguments] under Windows. By convenience, the long syntax for Linux is used below.

Initialize the database

The initialization consists in creating the SQLite database and the folders that store the uploaded files. It is done once at the beginning with the command python3 pwic_admin.py init-db.

In rare cases if the Python code fails for example, the database can be locked without timeout. In this case, try first to run the command python3 pwic_admin.py unlock-db else restart the server.

Set the parameters

With the command python3 pwic_admin.py set-env --help, you can define the following parameters that will change Pwic.wiki's behavior.

The project-dependent variables are mostly changeable online in the special page, but not all for security reasons. The project-independent variables generally can be edited through the console only and it often requires the restart of the server. The users will then have to reconnect from another tab, unless you define the option keep_sessions.

To show one or all the parameters, use the command python3 pwic_admin.py show-env.

In case you need to clean the obsolete variables, run the command python3 pwic_admin.py repair-env. This could happen after Pwic.wiki is upgraded.

Federated authentication

You can connect to Pwic.wiki by using a professional third-party identity provider. The mecanism OAuth relies on secret keys and, obviously, a lot of trust. The user connects from another website and Pwic.wiki pings back the server to fetch the main email address of the user. The account is then automatically configured by default. No other personal information are used.

Several project-independent variables must be defined in the following order. Any change requires the restart of the server.

Two-factor authentication (2FA TOTP)

This feature stands for 2-factor authentication (2FA) and Time-based One Time Password (TOTP). The concept is to share a secret key with the user once. When he connects to the website, a PIN code is requested: its value changes every 30 seconds automatically and should be accepted only once. Consequently, this random password managed on a separate device acts as a second source of authentication and strengthens the security of your account.

The setup is very easy:

• Define the options base_url and totp,
• Install any TOTP-compatible application on your smartphone,
• Run the command ./pa reset-totp [username] (Linux) or pa.bat reset-totp [username] (Windows) to generate a secret key for the given user account,
• Transfer privately the generated information into the TOTP application by using the mean of your choice.

This authentication technique has the benefit to be standard (RFC 6238), offline and easy to activate. It cannot act as a primary authentication method. It is also incompatible with the federated authentication because some providers include built-in 2FA techniques already (especially SMS).

2FA is optional, user-specific and can be assigned, removed and updated at any moment.

Reverse proxy

The feature is supported but not enabled by default because the filter by IP address should not be sensible to some special HTTP headers.

By using Nginx or Apache, you need to modify the code of the extension named on_ip_header and to configure the format of the HTTP log.

Compress the static files

Optionally, if you use mobile devices, it is possible to compress the big static files with the command python3 pwic_admin.py compress-static. This will reduce a bit the bandwidth and the loading time.

The option must be configured as well in your reverse proxy server (if you use one).

SQL queries

To facilitate the maintenance of the database remotely, the command python3 pwic_admin.py execute-sql allows you to execute online SQL statements in the console. It is very useful to upgrade Pwic.wiki and perform special operations.

☝️ Create a backup

First, it is recommended to warn your users in advance through the option maintenance. Note that it disables the capability to attach or update new documents to the pages.

At the time of the maintenance, shut down the server with python3 pwic_admin.py shutdown-server, and run the command python3 pwic_admin.py create-backup. A new file is created with the current time stamp. The name of the backup file can be extracted with sed if you have further scripting needs. Basically, it is a copy of the database without the documents that remain in the dedicated folder of the projects.

To restore the backup, simply replace the current database with a previous backup. Any changes made meanwhile will be lost. You must also ensure that the backup file is compatible with your used version of Pwic.wiki and that the salt PWIC_SALT is the same. To resynchronize the documents with the database and vice versa, run the command python3 pwic_admin.py repair-documents --test from the console.

As an alternative, you can backup the entire folder db/ with your own administration tools.

☝️ Create a project

As per the authorizations are designed, the creation of a new project is only possible in command line. Run the command python3 pwic_admin.py create-project --help for further details.

By default, you cannot create new pages. You must extend your rights as a manager, or add new dedicated users having that profile. Refer to the matrix above for the feasible actions per profile.

To show the existing projects and their ownership, run the command: python3 pwic_admin.py show-projects.

☝️ Take over a project

A project can become orphaned if the unique administrator left or is disabled. In this case, it is not possible to assign a new role online.

Run the command python3 pwic_admin.py takeover-project --help for further details.

When you revoke a user, you are warned if a project will be orphaned.

☝️ Split a project

The operation consists in copying one or several projects into a separate database file. The copied projects are not removed from the original database. The audit log stored in its dedicated database and the documents stored on the drive are not modified.

After you copied the databases and the documents to the target server, you can clean the audit data to remove some irrelevant traces.

Run the command python3 pwic_admin.py split-project --help for further details.

It is not possible to merge two databases into one, which would be the reverse operation.

☝️ Delete a project

Deleting a project cannot be undone. The user accounts are not deleted but their rights are removed from the projects.

The action is done in the console only through the following command: python3 pwic_admin.py delete-project --help.

☝️ Create a user account

The users are created by default as readers. The default password is predetermined by the general configuration variable PWIC_DEFAULT_PASSWORD.

The users must change their password before you can change their authorizations.

To ensure the consistency of the documentation and the audit tables, it is not possible to delete a user account.

With the federated authentication, the user accounts are created automatically. You don't need to create them in advance else two password managements will exist simultaneously. To force the use of OAuth, you must reset the password in the console.

☝️ Change the own password

The form is available in your profile page.

It is recommended to setup Pwic.wiki with HTTPS support so that the passwords cannot be intercepted in clear format. The self-signed certicates cannot be used publicly because they generate bypassable errors in your browser.

☝️ Change the password of another user

The recovery of the lost passwords is an administrative task since Pwic.wiki handles neither emails, nor personal information.

This command creates the user account if it does not exist, but it grants no role on the projects.

Run the command python3 pwic_admin.py reset-password --help for further details.

☝️ Revoke a user

The operation revokes the roles of the user on every project. If this user is the unique administrator of some projects, there are displayed.

The user account is not deleted physically, and replacing it with the default ghost account is not implemented but technically possible.

Run the command python3 pwic_admin.py revoke-user --help for further details.

☝️ Grant the authorizations

The rights are project-dependent. The form is accessible from the special page.

The administrators only can grant new rights to the users if they changed their password once.

A user must have one profile at least (ex: reader) to be able to view the project. To switch from one profile to another for a user, you should first activate the new profile, then deactivate the old profile.

The profile administrator cannot be removed or disabled oneself to ensure that there is always one administrator per project.

The assignment of the rights is monitored. You should also consider the separation of duties: no one should be able to do everything.

The anonymous access is given by the special user account pwic_anonymous. The account cannot be removed. You can disable it or enhance its rights (excluding administrator).

☝️ Create a page

The creation of new pages is restricted to the managers in order to monitor the documentation efficiently. Each page is codified with an external identifier that can be listed alphabetically in each project overview.

If a page is created in reference to another page, its content is copied by default. In that way, you can dedicate a project to store the default templates. It is not possible to refer to a page that doesn't exist or for which you have no access rights.

A page cannot be renamed to ensure the consistency of the documentation and the audit tables.

Each project must contain the home page home. It is accessed by default, it is sorted in first place in the header bar, and it holds the link to the special section of the site. Thus it is not recommended to delete this particular page.

☝️ Modify a page

Text content

A page is a collection of revisions. A new revision is created each time a page is modified. A past revision cannot be updated, but deleted under some conditions.

The tags are the base to classify the pages freely. The keywords are separated by a space. You can search for a tag in the search engine by usign a hash (ex: #tag). A tag counts a lot in the evaluation of the results.

The content of the page follows the Markdown (MD) syntax and you can use HTML markups outside of a code block. The dangerous attributes are automatically removed, but beware to not break the export to OpenDocument.

If enabled by the option mathjax, you can embed mathematical expressions supported by MathJax. As a special constraints, the sign \ must be doubled. For example, the usual form \(\sqrt{x^2}\) should be written as \\(\\sqrt{x^2}\\). Because it is not possible to preview the formulas before saving, you should save your page as draft.

Saving as draft means that you are working on a page. The revision needs no particular attention, except the one of the current editor. You can delete your own drafts. They are also automatically deleted once you publish a final version.

Saving as final means that the document is ready to be reviewed by the validator. It also physically deletes all your own previous revisions marked as draft unless you define the option keep_drafts.

If you want to save a document which is neither a draft, nor a final version, just save it with none of these flags. Such a revision is a milestone and can be deleted by an administrator only.

The attribute Reason of the page is a mandatory text that summarizes the purpose of the modification. It is used in the history to search for old revisions at glance.

The attribute Milestone of the page is an optional free comment that can be used, for example, by the manager to request a delivery date, or by the editor to announce a publishing date. You can also indicate your difficulties, the current progress, etc...

The attribute Header line means that the page is displayed in the header line. It helps to structure your documentation by themes. There is no limitation in the number of header items and only a manager can set the attribute.

The attribute Protected page means that the page can be modified by a manager only.

A manager can change all these attributes without recreating a revision if the option quick_fix is enabled.

Attachments

You can upload several files for a page. In case it is a picture, you can also include it in the page as a figure, else as a link. All the files are listed on a gallery below the page.

In any case, many rules apply in the uploading process:

• No revision is created when a file is created, modified or deleted. The changes are instantly considered.
• You cannot recover the deleted files, unless you own a backup.
• The filename must be unique per project and the file is attached to a page:
  • The file is created if it doesn't exist yet.
  • The file is updated if it exists and is assigned to the current page.
  • The file is not updated if it exists and is assigned to another page.
• The filename may have to respect a naming convention if a regular expression is defined in the variable document_name_regex.
• A file size limit may apply if the administrator has defined the variables document_size_max or project_size_max.
• For security reasons, the content of the file is verified if the option magic_bytes is enabled.
• To host the uploaded files on an external storage, you need to implement a custom code in PwicExtension.on_api_document_*, populate the field documents.exturl and don't keep any local copy.

For any document stored in the supported formats, you can import its content as a Markdown text by clicking on the button 🔨, unless you defined the option no_document_conversion. The images are excluded from the processing. The embedded convertor produces a more suitable content in regards of what the renown tool Pandoc can achieve.

☝️ Move a page

This feature has two usages:

• Rename a page within the same project,
• Move a page to another project: the page is eventually renamed at the same time, and the documents are moved if the source file names don't exist in the target project already.

Because the manager is responsible for the organization of the pages, you must have this role for the source and destination projects. The existing pages are not updated, then broken links will appear.

☝️ Delete a page

You delete a revision one by one, not the entire page at once. Moreover some restrictions apply to ensure the consistency of the documentation.

A manager and an editor can delete their own drafts. An administrator can delete a revision that is neither final, nor validated.

Because of the incremental numbering, the highest identifier of a deleted page can be reallocated. And the gaps in the numbering of the revisions exist by design.

☝️ Read a page

You must have an account to be able to read the documentation, unless the option no_login and the anonymous user are enabled for the concerned projects.

The pages are saved into a cache to improve the overall performances of the server at the cost of the disk space. After certain technical operations or over the time, it might be needed to clear the cache with the command python3 pwic_admin.py clear-cache. To not wait for the on-the-fly caching of the pages, you can regenerate in mass the cache with the command python3 pwic_admin.py regenerate-cache. The cache is not used to export a page or a project.

☝️ Export a page

A page is usually read online but you can export it on your desktop in several file formats.

html - Web page

The export to HTML is a single file that can be opened in your web browser. However, it does not embed the images, the attachments and your custom styles.

By using the feature "Save as" of your web browser, you can save the page and the loaded pictures, but without the attachments.

md - Markdown

This format is the basic content used by Pwic.wiki.

The file can be edited further with a classical raw text editor, but it needs a dedicated application to be rendered nicely.

odt - OpenDocument

This file format is public but it is not a DOCX document. The file can be edited in MS Word, LibreOffice and more.

The export is designed to be very similar to the HTML rendering online, like as if you copied-pasted the text in your editor. However, slight differences of layout are expectable.

The success of the conversion can be altered if you use HTML tags in your pages.

pdf - Desktop PDF

This format is not supported by Pwic.wiki, so it cannot be disabled in the options.

By using the feature "Print" of your web browser, select a virtual printer to produce the final PDF file. When doing so, some useless elements of the page are hidden.

On Windows, you can enable the PDF printer by executing the command %windir%\System32\OptionalFeatures.exe.

☝️ Validate a revision

Several conditions are required to validate a page:

• You have changed your password once.
• You have the role "validator" on the project.
• The page to validate is marked as "final" by the editor.

☝️ Export a project

The administrators are able to download a project as a ZIP file containing all the pages and attached documents. The purposes are the creation of a backup or the preparation of a migration for example.

The export is audited and can be disabled by setting the administrative variable no_export_project.

☝️ Verify the links

Pwic.wiki can verify the internal links of the pages to ensure that:

• Each page is linked to another one at least (check for orphans).
• Each page is not linking to a non-existent page (check for broken links).

The checks apply within a given project. The links to other projects are neither reported as orphan, nor as broken.

The feature serves as a general monitoring tool for the documentation.

☝️ Graph of the links

The directed graph is a visual representation of the links between the pages. It is possible to detect the pages of high importance and the ones that are insufficiently put into the light.

In case you link to another project, only the targeted pages are displayed. The title of the page is hidden if you don't have access to the project.

The pages in red correspond to broken links.

You can download the graph as a file. After you installed GraphViz, you can regenerate the picture with the command: dot -Tsvg input.gv -o output.svg.

☝️ Search across the pages

The search works with exact literals. If you are looking for the word excellent but the page contains the word exellent (without the c), you won't find any result. Consequently, writing without mistake is a main objective not only for search purposes, but also for the general quality of the documentation.

The search engine doesn't build an index of words. It means that you are able to search within a word. For example, document will match with both document and documents, but documents will match with itself only. Consequently, you should avoid the plural forms to maximize the number of results. This feature is also very important to be able to search for partial technical keywords if the documentation contains lines of code.

You can search for individual words, but also for sentences if you use double quotes. For exemple, "my documentation" will not match with my last documentation. However, my documentation would match with it because there is no quote (my and documentation are processed separately).

You can exclude a term by adding a leading minus sign (-). For example, -interface will exclude all the pages that contain the word interface. You can combine negated terms and they will act as an OR condition. It means that if one of the negated terms is part of the page, then the page is fully excluded. In case you want to search the minus sign explicitly, consider using surrounding quotes ("-interface").

The terms that are not negated act as an AND condition. It means that all the terms must appear so that the page can be selected. That's why you should not use the keyword and explicitly. It is not possible to make OR conditions with these terms, so do separate searches instead.

Some special search terms exist:

• term searches in the content of the page, or for the exact page identifier. This is the default behavior seen above.
• author:term checks the term against the author of the latest revision.
• title:term checks the term in the title of the page.
• :latest searches for the most recent page.
• :draft searches for drafts.
• :final searches for final pages.
• :validated checks that the page is validated, whoever is the validator.
• validator:term checks the term against the validator of the latest revision. If the term is empty, the check returns a match, so do prefer the keyword :validated in that case.
• :document checks that the page has at least one document attached.
• #tag checks that the tag is assigned to the page according to your own classification logic.

At the end, the pages are sorted by the number of occurrences of the search terms. Each term that is not negated contribute to the score as a sum. Because of that, your search terms should be at least 3- or 4-character long.

The search is case-insensitive.

☝️ Search across the documents

The search applies on the name of the page, the file name and the mime of the file.

The syntax of the terms is similar to the one used for the pages. You can use inclusive and exclusive keywords, but not the special ones that contain :.

It is not possible to search within the content of the files.

☝️ View the activity log

All the activities

Pwic.wiki logs many activities to enforce the traceability of the documentation. In case of troubles, it is possible to unwind the events and have a better understanding of the situation.

Online, the administrators of a project can see the events related to the project only, like the creation of a page, a deletion, a validation... At system level, it is possible to display all the events, including the ones not related to a given project.

The listing can be filtered out by date and content with the following syntax: python3 pwic_admin.py show-audit --max 365 | grep grant would return all the modifications of the users' rights over the last year.

All the logins

Run the command python3 pwic_admin.py show-login --days 30 to show the last logins of the users over the last given days (30 by default).

If the users don't delete their cookies and if you use the option keep_sessions, the users won't need to reconnect very often. Therefore an old login date is not necessarily a sign of inactivity.

All the inactive writers

Run the command python3 pwic_admin.py show-inactivity --days 30 to highlight the inactive users who can modify the documentation.

Some projects being completed, it is possible that they have not received any recent modifications by their owners.

Statistics

General figures about the server and the projects can be summarized by the command python3 pwic_admin.py show-stats.

Archive the log

Because the audit data can grow fast, you can archive the data on a regular basis with the command python3 pwic_admin.py archive-audit --selective 30 --complete 365. In the selective horizon (ie. older than the given number of days), some generic entries are removed but all the core traces of the projects are kept. In the full horizon, all the entries are removed.

In fact, the entries from the table audit are just moved to the table audit_arch and you can dispose of the data at your convenience through SQL commands. To remove the old entries physically, you would execute: DELETE FROM audit.audit_arch.

By design, you cannot remove the entries in the short horizon of 90 days. Also note that archiving the audit data impacts the reliability of the reports above.

☝️ View the activity of a user

The user page contains a public overview online of the recently modified pages (over the last 30 days) and the uploaded documents.

Your own user page is mentioned in the footer.

☝️ Automate Pwic.wiki (API)

The web interface communicates with Pwic.wiki through an application programming interface (API) whose specifications are available online.

You can develop external robots in the limit of your permitted authorizations. Some examples are available in the folder static/api/.

You can add your own paths by extending the source code.

☝️ Report into BI

Through API remotely

The Application Programming Interface (API) is a set of custom HTTP end-points used by the web user interface to perform all the updates in Pwic.wiki via POST requests only. All the users have access to a partial set of safe features depending on their online authentication.

The API is usable for the mass automation of the public data in read and write mode. It does not integrate well with BI tools because the data model is not reflected transparently and consistently.

The API cannot be turned off but you can restrict the users by their authorizations for each project.

Through ODBC locally

If you have a direct and local access to the server, you can use an SQLite ODBC driver locally to integrate Pwic.wiki with your favorite BI tool. However, all the data will be exposed without any filter or check. It is a critical point of failure if you leak the values of the private keys stored in the table env.

There is no possible option to prevent the local use of an ODBC driver. The access to the database is restricted by the file system only.

Through OData remotely

OData is a protocol to share data with external systems. Pwic.wiki implements minimalistic features (read-only, no filter, no stream, basic authentication), so that you can query the data remotely for your reporting. The tested tool is PowerBI that is often used in the business environments.

First, you need to activate the OData interface through the global option odata. Its effect is immediate.

Second, the OData feed is available at the following addresses:

• /api/odata: the description of the service,
• /api/odata/$metadata: the schema of the data,
• /api/odata/table: the data with the name table to be replaced by the real name that you want to retrieve. The paths are provided in the root description of the service.

The data are filtered from source and the private ones remain hidden. The OData cannot be filtered out by a dollar-argument. An authentication through a challenge Basic applies at the lowest level only. You cannot use the federated authentication (OAuth). For your security within an untrusted network, you should enable HTTPS. The exposed data may differ depending on your authorizations. A basic use of OData is possible with an account having a role of reader on the projects that you wants to report on.

The mass import of the tables in PowerBI via the shortest path is not supported for an unknown technical reason. Instead, you should add each table by its full path, which will require an authentication each time.

☝️ Read the licenses

Pwic.wiki is released under the terms of the GNU Affero General Public License v3+. For more details, refer to the file LICENSE.

Pwic.wiki also relies on several external resources whose licenses are summarized in the following table: