Sophora Courier Administration

Courier manages mailing lists, subscriptions and sending newsletters.

Courier uses different Sophora document types to persist its data. Users (sophora-user-mix:user) can be subscribed to distributors (sophora-courier-mix:distributor), which are typically mailing lists.

Either Sophora editors manually subscribe users to distributors (e.g., using the Sophora DeskClient), or a web application may be used to allow users to manage their subscriptions themselves.

Messages (sophora-courier-mix:message) contain the content that is being sent to users and represent, for example, newsletters. A message contains a list of distributors which are used to determine all receivers of the message. A message is being sent when the document is published.

Courier can use different adapters to send messages. An adapter uses one kind of transport to send a message (i.e. email, SMS, an instant messaging service etc.).

For a message to be picked up by an adapter it needs to have a specific mixin which has a boolean property. If this property is true, the corresponding adapter will be used to send the message.

Example: If a newsletter should be sent as an email, the document needs to have the mixin sophora-courier-mix:emailMessage and the property sophora-courier:isMail needs to be true.

Currently Courier offers the following adapters (using the corresponding mixin):

• Email: sophora-courier-mix:emailMessage

Installation

Courier should be installed as local service. You need a directory containing the sophora-courier-[version].jar.

This jar can directly be used to start/stop/restart courier by linking it from the /etc/init.d/ directory. This symbolic link might look like this: /etc/init.d/sophora-courier -> /cms/sophora-courier/sophora-courier-2.5.0.jar

Folder Structure

In a typical installation the courier folder looks like this:

sophora-courier/
|- config/
 |- application.yml
 |- logback-spring.xml (optional, logging can also be configured in application.yml)
|- data/ (used by the Sophora Client)
|- logs/
|- mailTemplates/ (if you use local Thymeleaf templates instead of the delivery, see chapter Templating for details)
 |- mail.html
 |- mail.txt

Configuration

This section describes the configuration of Sophora nodetypes and their corresponding Java model classes, and the configuration of the Courier application using the application.yml.

Nodetypes and Models

Sophora Courier uses Spring Data Sophora to represent each Sophora nodetype with a Java class (the model classes). The basic model classes delivered with Courier represent the mixins with their properties. You may extend these classes to enrich the model by your own customer-specific properties.

Typically one would first create and configure custom nodetypes in Sophora (i.e. customer-nt:newsletter), add the sophora-courier-mix mixins and then implement a corresponding model class (i.e. public class MyCustomMessage extends Message annotated with @NodeType("customer-nt:newsletter")).

This class only needs to have the properties of the nodetype that are used when rendering a message. Spring Data Sophora then automatically chooses the most specific class available, i.e., your MyCustomMessage implementation.

Courier comes with the following nodetypes/mixins and model classes for you to extend and build your own data model.

• sophora-courier-mix:message. Basic model class: com.subshell.sophora.courier.model.Message. A document of this type represents a basic message. It contains the text in the form of copytext.
• sophora-courier-mix:emailMessage. Basic model class: com.subshell.sophora.courier.model.Message. This nodetype extends the basic message by fields needed to send a message as email. A message is only being sent as email if the property sophora-courier:isMail is true.
• sophora-courier-mix:distributor. Basic model class: com.subshell.sophora.courier.model.Distributor
• sophora-user-mix:user. Basic model class: com.subshell.sophora.courier.model.User. Users are the recipients of messages. Their properties may be used when rendering a message (i.e. as email).

A Note on Courier and ACS

It is highly recommended, but not strictly necessary, to store all user related information in a seperate Sophora repository instead of the Sophora Primary Server. The Sophora module ACS serves this purpose.

Configuring the Application

Courier uses the file config/application.yml as main configuration. The following example configuration is included in the distribution. All properties not in a comment (comment = line starts with #) are mandatory.

Logging

It is recommended to use a logback-spring.xml file to configure logging and reference it in the application.yml. But there is also the option to configure logging directly in the application.yml.

See the following example config for details.

Sophora Courier aviods logging clear text email addresses and makes them anonymous whenever they are being logged. However, when logging on loglevel TRACE everytime an email message is being sent, the whole content of the mime message will be logged, but without headers like “to”, “from” and others containing mail addresses. Anyways the rest of the message might leak personal information into the log file.

# properties listed in comments are optional
server:
 port: 8080 # The port for the webserver of this application
sophora:
 serverUrl: http://your-sophora-server:1196
 sophoraUsername: user
 sophoraPassword: password
# channel: name # If this property is set, the parameter channel will be appended to the urlServletPath (default: null).
# basicAuthUsername: user # If the two properties basicAuthUsername and basicAuthPassword are set then every request will be authenticated via the authorization header.
# basicAuthPassword: password
 # Attention: Either urlServletPath or urlServletPostfix is required to generate URLs. It is also mandatory to set one of them for the preview.
 # The URL to a live delivery servlet that returns a document's (live) URL
 # urlServletPath: http://your-delivery:8080/context-path/system/servlet/urlService.servlet
 # The URL postfix to a live delivery servlet that returns a document's (live) URL for a available delivery.
 # If urlServletPath is set, the urlServletPostfix is ignored
 # Attention: Doesn't work for servers with replication mode 'slave' or 'none'
 # urlServletPostfix: /system/servlet/urlService.servlet
 # List of all to be ignored substrings. If ignoreDeliveryWithSubstrings is set, all URLs with the given substring will be ignored for URL generation
# ignoreDeliveryWithSubstrings:
# - ignoreString1
# - ignoreString2
# proposalName: Courier Notifications # The name of a proposal used to report failed messages, default: empty
# retryIntervalInMs: 5000 # The interval in ms of retries when status code of http response is not 200 or 302
# maxRetries: 0 # The maximum number of retries when status code of http response is not 200 or 302
# disableRedirectHandling: true # Follows a redirection with a corresponding status code in http response, if false is set. The default is true.
# successCodes: # The success http response codes for which the newsletter will be send.
# - 200
# noRetryErrorCodes: # The http error codes which do not cause a retry.
# - 204
# noRetryErrorMessage: "Error during template generation. Request will not be retried." # The error message which will be shown on the preview when one of the noRetryErrorCodes occur.
# errorMessage: "Error during template generation." # The error message which will be shown on the preview when another error code occurs (not in noRetryErrorCodes).
preview:
 # Attention: Either urlServletPath or urlServletPostfix is required to generate URLs. It is also mandatory to set one of them for live.
 # The URL to a preview delivery servlet that returns a document's (working version) URL
# urlServletPath: http://your-preview:8080/context-path/system/servlet/urlService.servlet
 # The URL postfix to a preview servlet that returns a document's (working version) URL for a available delivery
 # If urlServletPath is set, the urlServletPostfix is ignored
 # Attention: Doesn't work for servers with replication mode 'slave' or 'none'
# urlServletPostfix: /system/servlet/urlService:
 # List of all to be ignored substrings. If ignoreDeliveryWithSubstrings is set, all URLs with the given substring will be ignored for URL generation
# ignoreDeliveryWithSubstrings:
# - ignoreString1
# - ignoreString2
# userExternalId: previewUserExternalId # External ID of a published user-document to be used in the preview of messages
mail:
 host: your-mail-server
# port: 25
# username: username
# password: password
 # Whether the template should be retrieved from a Sophora delivery
 # (default: false = use templates at "mailTemplates" directory)
# useDeliveryToGenerateTemplate: false 
 sendIntervalMs: 1000 # The interval between sending mails
 # Limits the number of mails that can be in the sending queue.
 # If the queue is full, further mails will be discarded (default: 0 = no limit).
# maxQueueSize: 0 
 safeMode:
 enabled: true # With safe mode activated, no mails are sent, but only logged.
 whitelist: # Mail addresses listed here are excluded from safe mode, i.e. even with safe mode active mails may be actually sent to them.
 - mail1@example.com
 - mail2@example.com
# templateTypeHtml: "html" # If useDeliveryToGenerateTemplate, this template type is used to get the HTML template for a message. Default: "html"
# templateTypeText: "text" # If useDeliveryToGenerateTemplate, this template type is used to get the text template for a message. Default: "text"
propertyNames:
 distributorName: your-namespace:name # The property that contains the (human-readable) name of a distributor
 publicDistributor: your-namespace:public # The boolean property that defines whether a distributor is public
 
# For logging you can either use logback...
#logging:
# config: "config/logback-spring.xml" # Path to logback-spring.xml config. Use this to configure rolling log files by date
# ...or configure it directly in the application.yml, like this:
logging:
 path: "/path/to/logs" # either use a folder path
 #file: "logs/sophora-courier.log" # OR use a full/relative path to a file
 file.max-size: 10000000 # Max file size in bytes.
 file.max-history: 5 # Maximum number of archived log files to keep
 pattern:
 file: "%d{dd.MM.yyyy HH:mm:ss} %5level [%10.10thread] %-40.40(%logger{39}:%L): %msg%n%wex"
 console: "%d{dd.MM.yyyy HH:mm:ss} %5level [%10.10thread] %-40.40(%logger{39}:%L): %msg%n%wex"
 level:
 ROOT: WARN
 com.subshell.sophora: INFO
 com:
 subshell:
 sophora:
 # This is an example. On a development system you might want Courier to log on DEBUG level
 courier: DEBUG
 org.springframework.web: INFO

Using Courier for Email Newsletters

This section describes how to use Courier to create, preview, and eventually send newsletters via email to distributor recipients. Furthermore, it describes how to design the layout of emails using templates.

Creating a Newsletter

Newsletter documents contain various input fields that are necessary for sending a newsletter. The properties sophora-courier:senderName, sophora-courier:senderAddress and sophora-courier:replyAdress contain important information for the receivers: Subscribers will know who is sending them an email and how they may respond.

sophora-courier:subject contains the subject for the email. sophora-courier:text is a copytext. Use it for the actual content of a newsletter.

Previewing a Message

Courier provides two previews for a newsletter message, a HTML version and a text version. You can create preview documents with the following URLs:

• HTML preview:http://[courier_host]:[courier_port]/preview/mail/${sophora:externalId}
• Text preview:http://[courier_host]:[courier_port]/preview/mail/${sophora:externalId}?asText=true

In the preview Courier will render the newsletter the same way it does when sending it. In the templates, you have access to the message object (for which the preview should be shown), but also to the receiver object. Thus, Courier needs a receiver object to render the preview.

This preview receiver document can be configured in the application.yml via preview.userExternalId. The given document needs to be a published user document that will be used to render the preview of all newsletters.

Sending Newsletters

To send a newsletter, you have to save and publish the newsletter document. Courier then automatically sends emails to the receivers subscribed to the selected distributors.

Sending Test-Newsletters

With the test-newsletter button you can send one email/newsletter to an address of your choice. This feature allows you to test the sending of an email and check the presentation of the content.

Mail Distributors

A mail distributor is a Sophora document. It has two properties: The name of the distributor and a checkbox to specify whether or not it is a public distributor.

Public distributors allow the users to sign in or out themselves. If it’s not an public distributor, it is called an internal one. Users are not allowed to register for these by themselves.

In a newsletter-document you can select a distributor in the tab “Newsletterverteiler”. To find a distributor you can either use the drop-down menu or enter the name of it. Your choice will be saved in the yellow data.

Safe-Mode

If you want to test Courier’s features and make sure you do not accidentally send emails to registered users in your distributors, Courier can be run in a safe-mode. Only users in an allowlist will receive an email then.

Configure the Sending of Mails

When publishing a newsletter document, Courier does not send all emails simultaneously to avoid triggering anti-spam measures. Instead, mails are enqueued and sent one-by-one with a configurable interval.

With “maxQueueSize” you can define how many emails can be in the queue at most. If the queue is full, further emails are rejected; thus, you should carefully choose this setting if you do not want the default of an unbounded queue.

Furthermore you can also use “sendIntervalMs” to define the interval in milliseconds between two mailing jobs.

Templating

Courier uses Thymeleaf as its template engine. Users can decide whether they want to receive emails as HTML or text format, so you need to create a HTML template as well as a text template.

Templates for messages can be either file-based templates (located in the Courier installation directory in the mailTemplates directory), or dynamic templates from a Sophora delivery.

The default is file-based templating. Dynamic delivery-based templating can be helpful if you want to reuse certain layouts from your website (e.g., copytexts) for HTML mails.

Set mail.useDeliveryToGenerateTemplate to true to use delivery templating. In this case, Courier first uses the URL servlet (sophora.urlServletPath) to retrieve the URL to the message document that should be sent.

Next, it performs two requests to this URL using the template types html and text. (You may configure the names of the template types via mail.templateTypeHtml and mail.templateTypeText.) The publicationDate of the message is attached to the url as an additional query parameter in the format DateTimeFormatter.ISO_OFFSET_DATE_TIME, i.e. 2011-12-03T10:15:30+01:00.

In the template you can compare the given publicationDate with the publicationDate of the document to return an error code if they do not match.

Courier will retry the request as configured with sophora.retryIntervalInMs and sophora.maxRetries. The default success codes are 200 and 302 and can be configured using sophora.successCodes, i.e.

sophora.successCodes: 
 - 200
 - 302
 - 204

Furthermore error codes can be defined via sophora.noRetryErrorCodes which do not cause a retry. For these error codes a custom error message can be defined with the property sophora.noRetryErrorMessage.

As default the no retry error code is 204 (no content). For all other error codes a custom error message can be defined via sophora.errorMessage. In case of a generation error the error messages is included in the preview of the newsletter.

The results of that request must be the templates for HTML and text mails. For each recipient, the result is then further processed by Thymeleaf, where, e.g., user names etc. can be set, as described in the following.

In the example, you see how to access the fields of the variables message and receivers. The syntax to access these fields is either [[${message.subject}]], where reserved characters are escaped or [(${message.subject})] if reserved characters should not be escaped. Alternatively, the variables can also be integrated via the attribute of an HTML tag:

<p th:text="${message.subject}">example1</p>

To substitute variables in the copytext (e.g. $name or $headline) you can use the string.replace-method. You can use this on the the text of the copytext paragraphs. In the example, we get each paragraph from the copytext first and then we replace the variables.

<p th:each="p: ${message.text.paragraphs}" th:utext="${p.text.replace('$AnredeFoermlich', 'Sehr geehrte/r').replace('$Name', receiver.name)}">Absatztext</p>

For the txt-file it would be something like the following:

[# th:each="p: ${message.text.paragraphs}"] [(${p.plainText.replace('$AnredeFoermlich', 'Sehr geehrte/r').replace('$Name', receiver.name)})]

Example: mail.html

This is the default HTML template shipped with Courier, which also gives further examples on how to write a template:

<html>
<head>
<title>Dummy HTML template for Mail</title>
</head> 
<body>
<h1>Example for a mail template</h1>
<h2>Create your own template</h2>
<p>If you see this template, create the file <b>mailTemplates/mail.html</b> in the Sophora Courier folder.<br />
It has to be a valid Thymeleaf template. Find out more about thymeleaf here: <a href="https://www.thymeleaf.org/doc/tutorials/3.0/usingthymeleaf.html" target="_blank">https://www.thymeleaf.org/doc/tutorials/3.0/usingthymeleaf.html</a></p>
<p>To configure this message preview in Sophora, create a preview document with the following URL: <code>http://[courier-host:courier-port]/preview/mail/${sophora:externalId}</code></p>
<h2>Variables in the template</h2>
<p>In this template you have access to the variables "<b>message</b>" and "<b>receiver</b>".</p>
<div>
<p><code>message</code> has the following fields:</p>
<ul>
<li><code>replyAddress</code></li>
<li><code>senderAddress</code></li>
<li><code>senderName</code></li>
<li><code>subject</code></li>
<li><code>text</code> (the copytext as <code>Copytext</code> object; see example below)</li>
<li><code>attachments</code> (as <code>Attachment</code> object with fields <code>filename</code>, <code>mimeType</code>, and <code>data</code>)</li>
</ul>
<p><code>receiver</code> has the following fields:</p>
<ul>
<li><code>eMail</code></li>
<li><code>name</code></li>
</ul>
<p>Additional fields may be available if you have defined customer specific nodetypes extending the basic ones. For DasErste <code>receiver</code>, these are:</p>
<ul>
<li><code>street</code></li>
<li><code>postcode</code></li>
<li><code>city</code></li>
<li><code>country</code></li>
<li><code>organisation</code></li>
<li><code>position</code></li>
<li><code>phone</code></li>
<li><code>mobile</code></li>
<li><code>fax</code></li>
<li><code>department</code></li>
<li><code>gender</code></li>
<li><code>lra</code></li>
<li><code>notes</code></li>
<li><code>title</code></li>
<li><code>website</code></li>
</ul>
</div>
<h3>Usage</h3>
<p>You can access the fields of those objects with the following syntax:<br />
<code>[['[[${message.subject}]]']]</code></p>
<p>Or as attribute of a HTML Tag:<br />
<code>&lt;span th:text=&quot;${message.subject}&quot;&gt;Text&lt;/span&gt;</code>
</p>
<p>To loop over the paragraphs of the message and get the text use the following snippet. <b>Important:</b> Use <u>th:utext</u> instead of th:text or the template will display the HTML of the text as thymeleaf escapes it.<br />
<code>&lt;p th:each=&quot;p: ${message.text.paragraphs}&quot; th:utext=&quot;${'[Style: ' + p.style +'] ' + p.text}&quot;&gt;paragraph text&lt;/p&gt;</code></p>
<h2>Example</h2>
<p>The current message has the subject <b>[[${message.subject}]]</b></p>
<p>The name of the receiver is <b>[[${receiver.name}]]</b></p>
<p>The email of the receiver is <b>[[${receiver.eMail}]]</b></p>
<p th:if="${#ctx.containsVariable('receiver.street')}">The street of the receiver is <b>[[${receiver.street}]]</b></p>
<p th:if="${#ctx.containsVariable('receiver.postcode')}">The postcode of the receiver is <b>[[${receiver.postcode}]]</b></p>
<p th:if="${#ctx.containsVariable('receiver.city')}">The city of the receiver is <b>[[${receiver.city}]]</b></p>
<p th:if="${#ctx.containsVariable('receiver.country')}">The country of the receiver is <b>[[${receiver.country}]]</b></p>
<p th:if="${#ctx.containsVariable('receiver.organisation')}">The organisation of the receiver is <b>[[${receiver.organisation}]]</b></p>
<p>The text of the [[${message.text.paragraphs.size()}]] paragraphs of this message's copytext:</p>
<p th:each="p: ${message.text.paragraphs}" th:utext="${'[Style: ' + p.style +'] ' + p.text}">Paragraph style: Paragraph text</p>
</body>
</html>

Example: mail.txt

This is the default text template shipped with Courier, which also gives further examples on how to write a template:

If you see this template, create the file mailTemplates/mail.txt in the Sophora Courier folder.
It has to be a valid Thymeleaf template. Find out more about thymeleaf here: https://www.thymeleaf.org/doc/tutorials/3.0/usingthymeleaf.html
To configure this message preview in Sophora, create a preview document with the following URL: http://[courier-host:courier-port]/preview/mail/${sophora:externalId}?asText=true
In this template you have access to the variables "message" and "receiver".
message has the following fields:
- replyAddress
- senderAddress
- senderName
- subject
- text (the copytext as Copytext object; see example below)
- attachments (as Attachment object with fields filename, mimeType, and data)</li>
receiver has the following fields:
- eMail
- name
Additional fields may be available if you have defined customer specific nodetypes extending the basic ones. For DasErste receiver, these are:
- street
- postcode
- city
- country
- organisation
- position
- phone
- mobile
- fax
- department
- gender
- lra
- notes
- title
- website
You can access the fields of those objects with the following syntax:
[['[[${message.subject}]]']]
To loop over the paragraphs of the message and get the plain text:
[['[']]# th:each="p: ${message.text.paragraphs}"[[']']]
[['[']](${p.plainText})[[']']]
[['[']]/[[']']]
Examples of using the variables:
The current message has the subject '[[${message.subject}]]'
The name of the receiver is '[[${receiver.name}]]'
The text of the [[${message.text.paragraphs.size()}]] paragraphs of this message's copytext: 
[# th:each="p: ${message.text.paragraphs}"]
[(${p.plainText})]
[/]