Delivery Framework 4

Delivery Configuration

In this section you can find all information needed to setup and configure a Sophora Delivery Infrastructure. This covers configuration files, folder structure and setup of an eclipse project.

Software Requirements

For Sophora web applications, the minimum software requirements are:

  • Java 11
  • Apache Http Server 2.4
  • Apache Tomcat 7 (we strongly recommend Apache Tomcat 9, because version 7 and 8 are end of life)

Notes on Tomcat 7

Tomcat 7 enforces naming restrictions on identifiers in JSP-Templates. The Expression Language 2.2 Specification states in chapter 1.19, that an identifier is constrained to be a Java identifier. Amongst others, keywords such as "this", "for", and "class" must not be used as identifiers. Older versions of Tomcat did not enforce this, and in older releases of Sophora, a variable named "this" was used to reference the content map of the current document. JSP templates using any of the reserved keywords as identifiers will not run on Tomcat 7.

There are two options for migrating from Tomcat 6 to Tomcat 7:

  1. Disable the identifier check in Tomcat 7 using the Tomcat system property org.apache.el.parser.SKIP_IDENTIFIER_CHECK.
  2. Ensure that your templates don't use reserved keywords as identifiers. In Sophora JSP templates, the content map for the current document is now referenced using the variables "content" (old content map implementation) and "current" (new content map implementation), though "this" is still supported for legacy templates. See also Template development.

Notes on Tomcat 9

Tomcat 9 creates files with a restricted set of permissions. This is especially relevant for the files in the cache directory. These files should be read by the apache process and thus, should be readable by all users. Otherwise the apache process can’t read the files in the cache directory and all requests are served dynamically by the tomcat.

To fix this problem, the umask for the Tomcat must be changed. This is done by an environment variable:

export UMASK=0002

Apache Webserver Configuration

The webserver is installed in the following directory: install-path/apps/apache2
The httpd.conf file of the webserver contains the hereafter listed Sophora-specific entries:

### sophora configuration
JkWorkersFile "conf/workers.properties"
JkLogFile "logs/mod_jk.log"
JkLogLevel error
JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "
JkOptions +ForwardKeySize +ForwardURICompat +ForwardDirectories
JkRequestLogFormat "%w %V %T"
LoadModule sophora_module modules/mod_sophora.so
# # VirtualHosts
Include /cms/ts/webapps/ts-preprod-live/conf/ts_httpd.conf
Include /cms/ts/webapps/ts-prod-live/conf/ts_httpd.conf

Connection between Apache and Tomcat

To connect the Tomcat with an Apache webserver it is important to set the sophora.delivery.render.isMounted property in the corresponding context to true. Additionally, the DocumentRoot from the Apache's configuration (see below) has to equal the sophora.delivery.cache.directory of the context.

Example

Excerpt from sophora.properties:

sophora.delivery.render.isMounted=true
sophora.delivery.cache.enabled=true
sophora.delivery.cache.directory=/cms/webapps/example-prod-live/cache/htdocs/example

Excerpt from Apache virtual host config:

<VirtualHost www.example.com:80>
    ...
        DocumentRoot /cms/webapps/example-prod-live/cache/htdocs/example
    ...

We recommend using mod_proxy to manage communication between Apache and Tomcat. In comparison to mod_jk and mod_ajp_proxy, mod_proxy has yielded the best results stability and performance.

mod_jk

  1. problem: mod_jk is often unavailable for the Apache and need to be translated. Older versions sometimes cause trouble.
  2. problem: Apache 1 prevented the execution of the SSI filter within the webserver after the creation of a file in the Tomcat. In Apache 2 this works.
  3. problem: If the Tomcat generates a PHP page, the Apache only delivers plain PHP code. The code isn't interpreted until a second call (at that point the file is in the docroot directory and the Tomcat isn't required).

mod_proxy

The mod_proxy configuration is similar to mod_jk solution. However, the mod_proxy is more standard like and easier to use. But the problems remain the same: SSI is only supported by the Apache 2 and PHP isn't interpreted in the first place.

Here is an exemplary Apache configuration with this context: sophora.delivery.cache.directory=/cms/webapps/example-prod-live/cache

<VirtualHost www.example.com:80>
    UseCanonicalName On
        DocumentRoot /cms/webapps/example-prod-live/cache/htdocs/example
        LogLevel warn
 
        ProxyPreserveHost On
 
        AddOutputFilter INCLUDES .html .php .jsp .xml
 
        <Location /example-prod-live>
             SetOutputFilter INCLUDES
        </Location>
 
        AddCharset UTF-8 .html
 
        ProxyRequests Off
        ProxyMaxForwards 50
        ProxyErrorOverride Off
        ProxyPass /example-prod-live/ http://localhost:8080/example-prod-live/
        ProxyPassReverse /example-prod-live/ http://localhost:8080/example-prod-live/
        # Die Verbindung per AJP macht oft Probleme
        # ProxyPass  /example-prod-live/ ajp://localhost:8009/example-prod-live/
        # ProxyPassReverse  /example-prod-live/ ajp://localhost:8009/example-prod-live/
 
        RewriteEngine on
        RewriteLog /cms/webapps/example-prod-live/logs/example_rewrite_log
        RewriteLogLevel 0
 
        <Directory />
                Options Includes ExecCGI FollowSymLinks
        </Directory>
 
        # Add missing trailing slash for directories
        RewriteCond    %{DOCUMENT_ROOT}/%{SCRIPT_FILENAME}  -d
        RewriteRule    ^(.+[Sophora Delivery^/])$ $1/  [R,L]
 
        # Redirect to homepage
        RewriteCond %{SCRIPT_FILENAME} ^/$
        RewriteRule ^/$ /index.html
 
        # Replacement for DirectoryIndex
        RewriteRule ^(.*)/$ $1/index.html
 
        # Use tomcat for dynamic content
        RewriteCond %{SCRIPT_FILENAME} !^/example-prod-live/(.*)$
        RewriteCond %{SCRIPT_FILENAME} ^/.*\.jsp$
        RewriteRule ^/(.*)$ /example-prod-live/example/$1 [PT]
 
        # If we reach this point it must be cacheable content
        RewriteCond %{DOCUMENT_ROOT}%{SCRIPT_FILENAME} !-f
        RewriteRule ^/(.*)$ /example-prod-live/example/$1 [PT]
</VirtualHost>

Tomcat Configuration

The Tomcat is located at install-path/apps/tomcat. It needs to be started by the user "sophora" using the command ./startup.sh in the directory install-path/apps/tomcat/bin. In the same directory you can stop the Tomcat using the command ./shutdown.sh. Within all servers the Tomcat is configured to listen to port 9000. The configuration of the individual context is located in the install-path/apps/tomcat/conf/Catalina/localhost directory.

The context-xml file for each webapp (tomcat/conf/Catalina/localhost/<contextname>.xml) must contain two settings:

  • The parameter localConfigDirectory: path of the directory, where the local sophora.properties file of the webapp is located.
  • The environment variable logPath: path of the directory where the logs should be written.

Example context-xml:

<?xml version='1.0' encoding='utf-8'?>
<Context docBase="/cms/ndr/webapps/ndr-prod-live/webapp" path="/ndr-prod-live" reloadable="false" override="true">
        <Parameter name="localConfigDirectory" value="/cms/ndr/webapps/ndr-prod-live/conf/"/>
        <Environment name="logPath" value="/cms/ndr/webapps/ndr-prod-live/logs" type="java.lang.String"/>
</Context>

To avoid excessive memory ussage the tag pool size should be set to zero in the tomcat web.xml:

<servlet>
	<servlet-name>jsp</servlet-name>
	<servlet-class>org.apache.jasper.servlet.JspServlet</servlet-class>
	<init-param>
 		<param-name>tagpoolMaxSize</param-name>
 		<param-value>0</param-value>
 	</init-param>
	<load-on-startup>3</load-on-startup>
</servlet>

Configuration of the Sophora Delivery Web Application(s)

Web applications are located in the server's install-path/webapps directory. There you'll find a subdirectory for every context. For example:

install-path/webapps/ts-prod-live

Every context contains the following subdirectories:

  • cache
  • conf
  • logs
  • webapp

The webapplication will be deployed in the webapp subfolder. Jenkins is recommended for the deployment process. You can find more information about Jenkins in the section Deployment.

The configuration files are located in the conf directory. There you'll find the central configuration file sophora.properties for the web appliction as well as configuration files for the virtual hosts.

Exemplary sophora.properties file:

sophora.serviceUrl=rmi://www.sophora-cms.com:1199/ContentManager
sophora.delivery.client.username=webadmin
sophora.delivery.client.password=admin
 
sophora.delivery.render.isMounted=false
sophora.delivery.render.isLive=false
 
sophora.delivery.cache.enabled=false
sophora.delivery.cache.directory=C:/repos/sophora_web/cache
 
sophora.delivery.cache.pregeneration.enabled=false
sophora.delivery.cache.pregeneration.maxAccessTimeInterval=180
sophora.delivery.cache.pregeneration.numberOfThreads=1
sophora.delivery.cache.pregeneration.httpTimeout=240
sophora.delivery.url=http://localhost:8080/sophora-ts

sophora.delivery.externalUrl=http://www.sophora-cms.com:8080/sophora-ts
sophora.delivery.cache.groups=group1
 
log4j.logger.com.subshell.sophora.delivery.cache=INFO
log4j.rootLogger=WARN,stdout,logfile

An overview of all configuration parameters can be found in section Configuration Parameters.

Local Configuration Directory

The local configuration directory contains a sophora.properties file that overwrites the settings as defined in the web project's configuration file src/web/WEB_INF/sites/sophora.properties. Thus, for properties that exist in both files, the values from the local one replace those from the global (project wide) configuration file. During development, it might be useful to disable the caching locally or to change the server connection details without changing the entire project configuration.

The delivery requires a sophora.properties file within the local configuration folder. If this file is empty, all properties are retained from the global configuration file. If the local sophora.properties files cannot be detected, the delivery startup fails. In the following example the server connection details are overwritten:

local_sophora.properties

#sophora.serviceUrl=http://cms.server.com:1199
sophora.serviceUrl=localhost:1199
sophora.delivery.client.username=admin
sophora.delivery.client.password=adminPassword

Running the web application from an IDE

When running the webapp from Eclipse (on an embedded Tomcat), The path to the local configuration has to be specified within the src/web/META-INF/context.xml as follows:

context.xml

<?xml version='1.0' encoding='utf-8'?>
<Context>
    <Parameter name="localConfigDirectory" value="c:\sophora\webapp\localConfigurationDir"/>
</Context>

Configuring Websites within the Web Project

  • For each website the directory src/web/WEB-INF/sites has to contain a file <sitename>.properties. As with the sophora.properties, the properties given in the <sitename>.properties files can be overwritten in the correspondent files that are located in the local configuration directories. Templates then apply the properties dependent on the website the user is currently viewing.
  • Also, the directory src/web/WEB-INF/templates/<sitename> has to contain a file templates.xml. Within this file, it is determined which JSP templates should be used when a certain document type is requested. For more information see the page templates.xml.

Configuration Parameters

The following table contains all possible configuration parameters, a brief description and, if non-trivial, an exemplary value.

PropertyDescription + Example
sophora.serviceUrlServer connection, e.g. rmi://www.sophora-cms.com:1199/ContentManager
sophora.delivery.client.usernameUsername for server login
sophora.delivery.client.passwordPassword for server login
sophora.delivery.client.accesstokenMay be used as an alternativ authentication method.
This property is belongs to the AddOn Sophora Cloud Delivery.
sophora.delivery.client.sessionlessIf the delivery communicates with multiple staging server behind a loadbalancer, this option should be set to true. This can only be used, if either http or https is used (default is false).
This property is belongs to the AddOn Sophora Cloud Delivery.
sophora.delivery.client.listeningToDerivedDocumentChangesDefines whether or not this client should receive DerivedDocumentChangedEvent.
Note: This property will no longer be available in Sophora Delivery versions 5.0.0, 4.0.12, and 3.0.26. It will be enabled by default and will no longer be configurable.
sophora.delivery.urlURL of this delivery. Is also used by the pre-generation. Example: http://localhost:8080/demo
sophora.delivery.externalUrlExternal URL of the delivery. This is the URL which other components (Staging Servers and other web applications) use to communicate with the delivery. It is used for cache flushes via the cache servlet and for URL generation via the URL4Id servlet. Example: http://delivery1.domain.com:8080/demo
sophora.delivery.cache.groupsVia this property multiple delivery can form a group in which external cache flushes are propagated by the cache servlet.
sophora.delivery.cache.dbWhich database should be used for caching? Possible values: derby, derbyng (since version 1.32.4 and  1.33.1), asyncderby (version 1.32.4 and 1.33.1). For the versions after 1.32.4 and 1.33.1 consider the property sophora.delivery.cache.db.async. (Default: derby)
sophora.delivery.cache.db.asyncDefines whether the database should be updated asynchronously after a request (default true). Since delivery version 1.32.5 and 1.33.2. See the section Asynchronal Update of the Cache Database for further information.
sophora.delivery.cache.derby.optimizing.enabledWhether or not the optimisation of the derby db cache should to be done (default is true).
sophora.delivery.cache.derby.optimizing.cronHow often to run the derby db cache optimisation (default: 0 0 3 1/1 * ? *)
sophora.delivery.cache.cleanup.enabledWhether or not the cleanup of the db cache should to be done (default is false).
sophora.delivery.cache.cleanup.cronHow often to run the db cache cleanup (default: 0 0 1 1/1 * ? *)
sophora.delivery.cache.filecleanup.enabledEnables the clean up of the file system cache. Removes all cached file which are nor registered in the database. (default false)
sophora.delivery.cache.filecleanup.cronCron expression for the file system clean up (default: 0 30 0 1/1 * ? *)
sophora.delivery.cache.removeFromCacheRegistryRemove the cache entries from cache db for a removed file (default: true). If set to false the cache db cleanup job should be enabled.
sophora.delivery.cache.removalQueueType of removal queue: derbyRemovalQueue (Default), cachingDerbyRemovalQueue (faster derby implementation), memoryRemovalQueue (pure memory implementation; only recommended in combination with sophora.delivery.cache.removeFromCacheRegistry=false)
sophora.delivery.cache.removal.maxParallelGenerationsBeforeThrottleMaximum number of concurrent cache generations before the removal queue worker is throttled (default 30)
sophora.delivery.cache.enabledKeep generated file in the cache (true/false)?
sophora.delivery.cache.directoryDirectory to save cached files and the cache database in (i.e. /cms/webapps/demosite/cache)
sophora.delivery.cache.thresholdOnly cache fragments, whose generation time took longer than this threshold are cached.
The value defines this threshold in milliseconds.
This configuration parameter is used, to reduce the number of cached fragments.

If a cache fragment generates very fast, the cache overhead might be more expensive than the generation of the cache fragment.
The default value is 0, so that all requests are cached.
Please consider also the other criteria, which defines whether a request is cached or not. Like the configuration parameter sophora.delivery.cache.enabled
and the list of suffixes which are excluded from the caching. Static resources are cached even if there generate time is below the threshold.
sophora.delivery.cache.trackFieldAccessWhether or not field access will be tracked (default: true). If tracking of field access is enabled only changes to fields that were relevant during the rendering of the html file of a document will force a cache invalidation. If it is disabled all changes of a document will force a cache invalidation of its html file.
sophora.delivery.cache.event.passwordPassword that protects the cache servlet, see caching section for more information
sophora.delivery.cache.requestTimeoutTimeout in minutes for cached requests. Default value is 10. When this values is set to 0 no timeout is active.
sophora.delivery.cache.sophoraclient.cacheregistration.enabledDefines whether or not documents which are accessed via the ISophoraClient during requests should be taken into account for the registration of dependent documents of the resulting cache fragment. The default value is false, so that only documents which were accessed via the sophora taglib or the ContentProvider are taken into account. The option is useful, to ensure an automatic cache invalidation even if the sophora taglib or the ContentProvider are not used to serve client requests. This may be the case when Spring Data Sophora is used. Values: true/false
sophora.delivery.cache.ephemeralEntriesThresholdInSecondsIf set, all cache fragments with a life time below this amount of seconds will be treated as ephemeral. This implies a memory-only handling and reduces stress on your cache database. Only works together with the async cache db. See the article on caching for details. (Since 2.5.26)
sophora.delivery.contentTypePropertyNameThe name of the property which stores the content types, for example sophora:contentType
sophora.delivery.render.isMountedDoes the Tomcat operate within an Apache webserver(true/false)?
sophora.delivery.forceSsiThroughTomcatIf set to true, the tomcat server will evaluate all SSIs regardless of whether it operates with an Apache webserver or not. Enable this option to fill the developer tool bar with metrics about caching and includes. Default is false.
sophora.delivery.render.isLiveIs this a live delivery? If that's the case, time schedules and offline documents are regarded always according to the current request time. Otherwise not (due to preview). Values: true/false
sophora.delivery.render.preview.includeOfflineDocumentsShow referenced documents in preview even if their document state is 'offline'. Values: true/false - Default: true
sophora.delivery.render.defaultContentMapNameThe name of the default content map that is set into the request (default: content)
sophora.delivery.showDevToolbarCorresponds the Sophora JSP tag "devToolbar". The developer toolbar will just be shown, if this property is set to true. (default: false)
sophora.delivery.client.documentCacheOverflowToDiskDefines whether documents should be persisted on disk by the document cache. Values: true/false - Default: false.
Replaces the property sophora.delivery.client.documentCacheElementsOnDisk since version 1.38.0
The cached data is stored in sophora.delivery.cache.directory/documents.
sophora.delivery.client.documentCacheElementsInMemoryMaximum number of documents that are kept in memory by the document cache. This property must not be set to an empty (or non-numeric) value. Otherwise the webapplication-context can't be created. Default is 5000 (This property replaces sophora.delivery.client.documentCacheSize).
The value 0 means that there is no limit.
sophora.delivery.client.documentCacheStatisticsActivates cache statistics which can be read via JMX. (default: false)
sophora.delivery.jmx.portUsed to communicate the JMX port to the Sophora Admin Dashboard.
This option is useful when multiple deliveries are running on the same host.
Note: The delivery will not manage this port. The application using the delivery is responsible for making JMX available for remote access.
sophora.delivery.defaultNamespacesWhich namespaces should be filtered from the content map? Values as comma separated list, like for example sophora-content, sophora-extension
sophora.delivery.nonOverridablePropertyNamesComma separated list of properties that must not be overwritten with an empty value.
sophora.delivery.ignoredSophoraIds.sizeThe size of the cache for the ignoreList (default: 1000)
sophora.delivery.cache.remove.limitDifferentiation between the priorities of cache files that should be deleted or generated in the priority queue. You can enter as much values as you want to realise smaller priorisation steps. As an example, consider the following priorisations: 100, 1000, 10000.
If less than 100 files are affected by an operation, these files will be processed with priority "3" (which is the highest).
If 100 to 999 files are affected, they are categorised with priority "2".
If 1000 to 9999 files are affected, they are assigned with priority "1".
If the operation affects more than 9999 files, they will be processed with the lowest priority (priority "0").

(Default: 20,100,500,1000,10000,50000)
sophora.delivery.cache.remove.dateFlush.limitPriority for time scheduled flushes. You can enter a static value, such as "4", or you can set a relative priority by adding a plus ("+") or minus ("-") sign in front of a priority value, i.e. "+4". The relative priority is then added to the highest priority of the remove.limit property. By default, this flush priority is left blank so that the highest priority of the remove.limit property is adopted.
sophora.delivery.cache.pregeneration.enabledEnable pre-generation (true/false)?
sophora.delivery.cache.pregeneration.urlProviderTemplateJSP template for providing the URL used for the pregeneration of a sophora document, e.g. /site/system/pregenfilename.jsp?id= //
This is needed only for generation of non-existing files. (see generateNonExistentFiles)
Use the UUID of sophora documents as ID key here.
sophora.delivery.cache.pregeneration.maxAccessTimeIntervalAccess time of pre-generated files (in seconds, e.g. 600)
sophora.delivery.cache.pregeneration.numberOfThreadsNumber of threads for the pre-generation task
sophora.delivery.cache.pregeneration.httpTimeoutTimeout for the request of the HTTP client of the pre-generation (in seconds, e.g. 30)
sophora.delivery.cache.pregeneration.generateNonExistentFilesShould a file be pre-generated if a document is published and the file does not exist in the cache, or if a cache entry is explicitly flushed by a tag or the cache servlet (true/false)? The file is determined through the JSP urlProviderTemplate (see above). Note: Pregeneration of SSIs and other references (images etc) is not supported.
sophora.delivery.cache.pregeneration.boostAccessTimeIntervalAccess time to boost priority for files to pregenerate (in seconds. Default: 86400)
sophora.delivery.cache.pregeneration.boostPriorityPriority-Boost for file pregeneration. (Default: -1)
sophora.delivery.cache.alive.logfilePath to the file which records the time when the Sophora Server was connected lastly
sophora.rmi.cache.registryPortPort for the CacheExporter-RMI
sophora.rmi.cache.servicePortPort for the RMI service
sophora.delivery.cache.resourceChangedFlushIntervalInterval size (in seconds) to check for changed resources, like JSP files (default 60sec)
sophora.delivery.cache.flushDateFlushIntervalInterval size (in seconds) to check for date flushes (default 60sec)
sophora.delivery.cache.flushPasswordPassword of the flush template for external cache keys
sophora.delivery.cache.replication.fileFilterFiles that match this regular expression should be moved to the cache. A possible value might be .*(.php)
sophora.delivery.cache.replication.periodPeriodicity of static files to be replicated into the cache directory (in milli seconds)
sophora.delivery.cache.facade.numberOfThreadsNumber of threads within the cache facade that may access the cache simultaneously
sophora.delivery.cache.event.cleanupIntervalInterval size to remove out-dated entries from the event database of the cache (in milli seconds)
sophora.delivery.cache.event.lifetimeMinutesAfter how many minutes should an event be removed from the cache's event database?
sophora.delivery.sendMailAsFormRegExpRegular Expression for validation of email adresses: ^(.+)@(.{2,)\.(.{2,4})$}}
sophora.delivery.ssi.queryStringVariableNameThe variable name of the query string for the active SSI document (default: QUERY_STRING).
sophora.delivery.templateSuffixesA comma-seperated list containing template-suffixes to be used instead of .html. The first element has the highest priority. Default is jsp, jspx
sophora.delivery.template.checkIntervalInterval to check the template.xml for changes (in seconds)
sophora.delivery.urlCodec.validateUrlParamsShall URLs with parameters get a check sum (true/false)? Default is false
sophora.delivery.urlCodec.allowForeignSiteContentIn Sophora, any content is available within all URLs that contain an existing structure path. By default, this also holds for content from other sites. By setting this property to false, requesting content within a structure path that does not belong to the URL's site will result in a 404. Default: true
sophora.delivery.urlCodec.nameDecide which urlCodec is used for delivery. Sophora only ships with the "defaultSophoraCodec", which is the default.
sophora.delivery.localeWhich Locale should be use for the date format function in the ContentMap (default: de_DE)
sophora.delivery.captcha.usedFormCheckAlso check the form ID in submitted forms (true/false)? Default is false
sophora.delivery.site.YOUR_SITE_NAME.domainThe domain prefix for the generation of URLs. If the link is created for a site other than the current one, this prefix will be prepended. For example, http://www.site1.de or http://www.site2.de
Must be configured in sophora.properties
sophora.delivery.site.YOUR_SITE_NAME.urlPrefixURL prefix for the URL generation. If the delivery runs in the "mounted" mode, this prefix will be prepended. Example: /welle1 or /welle2
Must be configured in sophora.properties
sophora.delivery.structureNodeRedirect.responseCodeHTTP response status code for redirected requests. Default is 302.
sophora.delivery.http.proxy.hostHostname of the HTTP proxy which is used for HTTP requests between several deliveries. May only be used in combination with the sophora.delivery.http.proxy.port property.
sophora.delivery.http.proxy.portPort number of the HTTP proxy which is used for HTTP requests between several deliveries. May only be used in combination with the sophora.delivery.http.proxy.host property.
sophora.delivery.http.proxy.userLogin name for HTTP proxy which is used for HTTP requests between several deliveries. (optional)
sophora.delivery.http.proxy.passwordPassword for the HTTP proxy which is used for HTTP requests between several deliveries. (optional)
sophora.delivery.logconfiguration.disabledDisable the default configuration of log4j. Default is false
sophora.delivery.client.connectRetriesNumber of retries to connect to Sophora server before giving up. Default is 50.
sophora.delivery.client.connectRetryIntervalInterval in seconds between retries to connect to Sophora server. Default is 20.
sophora.delivery.documentType.extension.[NODE_TYPE_NAME]A comma-separated list containing allowed url-suffixes for the given nodetype name. e.g.: sophora.delivery.documentType.extension.sophora-content-nt\:video=html,jsp,xml,json Only .html, .jsp, .xml and .json are allowed suffixes for requests to video documents. Please note that ':' in the fully qualified nodetype-name has to be escaped '\:'
If you want to make a restriction for all nodetypes, the placeholder allNodeTypes can be used. This extensions are used for all node types. For node types with a specific configuration the allowed suffixes are added.
That feature only work if delivery is configured as live delivery.
sophora.delivery.solr.urlWhich SolR-Server should be use. Default is: http://localhost:1196/solr (Since 2.1.16, 2.2.6, 2.3.0: should not be set because it is retrieved from the current Staging Server)
sophora.delivery.solr.userUsername for the solr server. Default: solr (Since 2.1.16, 2.2.6, 2.3.0: should not be set because it is retrieved from the current Staging Server)
sophora.delivery.solr.passwordPassword for the solr server. Default: solr (Since 2.1.16, 2.2.6, 2.3.0: should not be set because it is retrieved from the current Staging Server)
sophora.delivery.solr.query.parameter.tzThe time zone as canonical id. Default: Europe/Berlin
sophora.delivery.solr.socket.timeoutThe socket timeout for connections to the solr server measured in milli seconds. Defaults to 3000
sophora.delivery.solr.connection.timeoutThe timeout for connecting to the solr server measured in milli seconds. Defaults to 1000
sophora.delivery.solr.maxconnectionsThe number of parallel connections allowed through the deliveries Solr client. Defaults to 100
sophora.delivery.solrcloud.enabledEnable SolrCloud support. If enabled, all solr search requests are targeted at SolrCloud. Defaults to false.
sophora.delivery.solrcloud.zkHostsList of Zookeeper hosts that are part of the SolrCloud cluster.
sophora.delivery.solrcloud.usernameSolr username if the SolrCloud Basic Authentication Plugin is enabled.
sophora.delivery.solrcloud.passwordSolr password if the SolrCloud Basic Authentication Plugin is enabled.
sophora.delivery.solrcloud.connection.timeoutSolr connection timeout in ms. Defaults to 60000.
sophora.delivery.solrcloud.socker.timeoutSolr socket timeout in ms. Defaults to 600000.
sophora.client.dataDirDefines a directory which may be used by the Sophora Client Api for persisting information like the available nodes in a cluster. The directory must be specified over an absolute path. This is only reasonable for a preview delivery.
sophora.delivery.classReloading.activeAutomatic reloading of model and function classes. Default: true
sophora.delivery.classReloading.delaySecondsDelay in seconds for automatic class reloading. Default: 10
sophora.delivery.spring.additionalBasePackagesA comma-separated list of Java package names which will additionally be considered for Spring component scanning when the Delivery is initialized.

Caching

Filesystem Cache Folders

The subdirectory install-path/webapps/context/cache contains

  • a directory db and
  • a directory removal and
  • a link htdocs -> ../../../htdocs/ts-prod-live.

The directory db is the location of the registry of the cache manager. The directory removal contains the persistent data of the removal- as well as the preGenerating queue. The link htdocs ensures that the HTML cache is located at a defined position within the directory structure but may be spotted elsewhere if necessary.

Configuring the Derby Cache DB

To deactivate the optimization of storage space within the Derby DB, you have to set the property sophora.delivery.cache.derby.optimizing.enabled to false. The database optimization is executed once a day at 3am by default, but can be changed via the property sophora.delivery.cache.derby.optimizing.cron.

Logging

Log4J has been removed as the logging framework for the delivery, in favour of SLF4J. SLF4J is a logging facade, a backend has to be provided by the web-application. This allows any SLF4J-compliant logging framework to be used in the web-application, such as log4j or logback. We recommend using logback, because it supports automatic reloading of the configuration file and configuration via JMX.

A logging configuration and the .jars of the logging backend must be provided by the web-application.

To configure logging using logback, a logback.xml file has to be added to the web-application's classpath (e.g. under WEB_INF/classes). The following example shows a configuration that behaves similar to the log4j-configuration used in previous sophora releases:

<?xml version="1.0" encoding="UTF-8"?>
 
<configuration scan="true">
    <jmxConfigurator />
    <insertFromJNDI env-entry-name="java:comp/env/logPath" as="LOGPATH" />
 
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>[%thread|%thread] %-5level %.40c:%L - %msg %n</pattern>
        </encoder>
    </appender>
 
    <appender name="logfile" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <File>${LOGPATH}/delivery.log</File>
 
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>${LOGPATH}/delivery.%d{yyyy-MM-dd}.log
            </fileNamePattern>
            <maxHistory>7</maxHistory>
        </rollingPolicy>
        <encoder>
            <pattern>[%d{ISO8601}, %-5p] [%t|%t] %m%n</pattern>
        </encoder>
    </appender>
 
    <appender name="cachefile" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <File>${LOGPATH}/cache.log</File>
 
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>${LOGPATH}/cache.%d{yyyy-MM-dd}.log
            </fileNamePattern>
            <maxHistory>7</maxHistory>
        </rollingPolicy>
        <encoder>
            <pattern>[%d{ISO8601}] [%t|%t] %m%n</pattern>
        </encoder>
    </appender>
 
    <appender name="errorLog" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
            <level>ERROR</level>
        </filter>
 
        <File>${LOGPATH}/error.log</File>
 
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>${LOGPATH}/error.%d{yyyy-MM-dd}.log
            </fileNamePattern>
            <maxHistory>7</maxHistory>
        </rollingPolicy>
        <encoder>
            <pattern>[%d{ISO8601}, %-5p] [%t|%t] %.40c:%L: %m%n</pattern>
        </encoder>
    </appender>
 
    <logger name="org.apache.commons.httpclient" level="ERROR" />
    <logger name="com.subshell.sophora" level="WARN" />
    <logger name="com.subshell.sophora.delivery.filter.BenchmarkFilter" level="INFO" />
    <logger name="com.subshell.sophora.delivery.cache" level="INFO" additivity="false">
        <appender-ref ref="cachefile" />
    </logger>
 
    <root level="WARN">
        <appender-ref ref="logfile" />
        <appender-ref ref="errorLog" />
    </root>
</configuration>

Using this configuration, logback will write the logfiles:

  • delivery.log
  • cache.log
  • error.log

in the [LOGPATH] directory. This should be the same place where the former log4j default-configuration placed the logfiles. This configuration uses LOGPATH as a placeholder for the logs directory. So this configuration can be used for multiple installations.

To define the LOGPATH variable, a java.lang.String-resource has to be provided by the container. This resource can be added to the context in the container's context-configuration (e.g. context.xml):

<?xml version='1.0' encoding='utf-8'?>
<Context>
...
    <Environment name="logPath" value="/cms/portal/webapps/myportal/logs" type="java.lang.String"/>
...
</Context>

The LOGPATH variable can also be set in the application's web.xml using the following definition. This is only recommended if the configuration via the context is insufficient or not possible for any reason.

web.xml

<env-entry>
  <env-entry-name>logPath</env-entry-name>
  <env-entry-type>java.lang.String</env-entry-type>
  <env-entry-value>/cms/portal/webapps/myportal/logs</env-entry-value>
</env-entry>

In addition, this directory contains - depending on the webserver's configuration - log files for each website:

  • <site>_access_log (e.g. ts_access_log)
  • <site>_error_log
  • <site>_rewrite_log

Finally the following .jars have to be present in the web-application's classpath (WEB-INF/lib):

  • logback-classic-0.9.27.jar
  • logback-core-0.9.27.jar
  • slf4j-api-1.6.1.jar

If maven is used in the web-project the following dependency can be added to the pom:

<dependency>
 <groupId>ch.qos.logback</groupId>
 <artifactId>logback-classic</artifactId>
 <version>0.9.27</version>
 <type>jar</type>
</dependency>

web.xml

Filters

Each request is filtered by several servlet filters within the web application. Depending on the configuration filters may be omitted or disabled. The following sections describe how the filters can be defined in the web.xml and by which parameters they can be configured. For more information about filters read the section The Sophora Filter Chain.

Standard Filters

SiteEnvironmentSetupFilter
<filter>
    <filter-name>SiteEnvironmentSetupFilter</filter-name>
    <filter-class>
        com.subshell.sophora.delivery.filter.SiteEnvironmentSetupFilter
    </filter-class>
    <init-param>
        <param-name>resourcePathRegex</param-name>
        <param-value>\.servlet</param-value>
    </init-param>
</filter>
parameterdescriptionexample
resourcePathRegexA regular expression for path segments that should be handled by the tomcat as if they were directed to the system site even though there path is not directing to an existing site.
The value \.servlet is the default if nothing is specified.
It is not necessary to modify this for static resources (e.g. in common-folders) or content from web-jars since they are detected properly even without this configuration.
\.servlet
SSIFilter
<filter>
    <filter-name>SSIFilter</filter-name>
    <filter-class>com.subshell.sophora.delivery.filter.SSIFilter</filter-class>
    <init-param>
        <param-name>contentTypePrefixes</param-name>
        <param-value>text,application/xml,application/json</param-value>
    </init-param>
</filter>
parameterdescriptionexample
parameterEncodingThe encoding that should be used (by the URLUtil) to encode and decode the URL-ParametersUTF-8
contentTypePrefixesA comma-separated list of content-types that should be searched for SSI-commands.text,application/xml,application/json
httpHeaderA comma-separeted list of HTTP headers which should be copied from the initial request into the SSI requests.
By default, no headers are copied.
 
X-Forwarded-For, User-Agent
CacheFilter
<filter>
    <filter-name>CacheFilter</filter-name>
    <filter-class>
        com.subshell.sophora.delivery.cache.filter.CacheFilter
    </filter-class>
    <init-param>
        <param-name>encoding</param-name>
        <param-value>UTF-8</param-value>
    </init-param>
    <init-param>
        <param-name>noCachingSuffixes</param-name>
        <param-value>.servlet</param-value>
    </init-param>
</filter>
parameterdescriptionexample
encodingThe encoding that will be used by the cache filter for reading and writing cache fragments.UTF-8
noCachingSuffixesa comma separated list of suffixes that will be ignored by the CacheFilter. Requests with URLs ending with one of these suffixes will not lead to cache entries..servlet
SophoraSessionFilter
<filter>
    <filter-name>sophoraSessionFilter</filter-name>
    <filter-class>
        com.subshell.sophora.delivery.filter.SophoraSessionFilter
    </filter-class>
</filter>
parameterdescriptionexample
urlEncodingThe encoding that will be used to encode URLs. This encoding will also be used to generate SSIs. Default is UTF-8UTF-8
UrlDecoderFilter
<filter>
    <filter-name>URLDecoderFilter</filter-name>
    <filter-class>com.subshell.sophora.delivery.filter.URLDecoderFilter</filter-class>
    <init-param>
        <param-name>ignoreSuffixes</param-name>
        <param-value>.servlet</param-value>
    </init-param>
</filter>
parameterdescriptionexample
ignoreSuffixesA list of suffixes separated by semicolon that will be ignored by the URLDecoderFilter. This parameter is usually used for logical URLs like servlets that shouldn't be interpreted as a request to a static file nor to a sophora-document..servlet

Additional (optional) Filters

SetUtf8EncodingFilter
<filter>
    <filter-name>sophoraUtf8EncodingFilter</filter-name>
    <filter-class>
        com.subshell.sophora.delivery.filter.SetUtf8EncodingFilter
    </filter-class>
</filter>
BenchmarkFilter
<filter>
    <filter-name>benchmarkFilter</filter-name>
    <filter-class>
        com.subshell.sophora.delivery.filter.BenchmarkFilter
    </filter-class>
</filter>
MultipartFilter
<filter>
    <filter-name>multipartFilter</filter-name>
    <filter-class>com.subshell.sophora.delivery.filter.MultipartFilter</filter-class>
</filter>
parameterdescriptionexample
maxFileSizeA numeric (long) value of the maximum size (in bytes) of a multipart request.1024
ExceptionLoggingFilter
<filter>
    <filter-name>exceptionFilter</filter-name>
    <filter-class>
        com.subshell.sophora.commons.filter.ExceptionLoggingFilter
    </filter-class>
</filter>
TrimFilter
<filter>
    <filter-name>trimFilter</filter-name>
    <filter-class>
        com.subshell.sophora.commons.filter.TrimFilter
    </filter-class>
</filter>
parameterdescriptionexample
applicationSubtypesA comma separated list of subtypes of the content-type application, for which the trim filter should be applied..xml
applicationSubtypePrefixesA comma separated list of subtype prefixes of the content-type application, for which the trim filter should be applied.xml+,xhtml+,rss+,rdf+
ignoredTagsA comma seperated list of HTML-Tags in which lines should not be trimmed.pre,textarea

Sophora Initializer

The SophoraInitializer needs to be defined as a listener. It takes care of initializing the properties, templates and spring configurations.

<listener>
    <listener-class>
        com.subshell.sophora.delivery.configuration.SophoraInitializer
    </listener-class>
</listener>

Servlets

To enable servlets, the suffix of the servlets must be excluded from certain Sophora filters.
To exclude certain suffixes from caching the cache filter needs to be configured by setting the parameter noCachingSuffixes in the web.xml like in the following example (the excluded suffix is ".servlet"):

<filter>
    <filter-name>CacheFilter</filter-name>
    <filter-class>
        com.subshell.sophora.delivery.cache.filter.CacheFilter
    </filter-class>
    <init-param>
        <param-name>encoding</param-name>
        <param-value>UTF-8</param-value>
    </init-param>
    <init-param>
        <param-name>noCachingSuffixes</param-name>
        <param-value>.servlet</param-value>
    </init-param>
</filter>

To ignore suffixes in the URLDecoderFilter the ignoreSuffixes parameter has to be used:

<filter>
    <filter-name>URLDecoderFilter</filter-name>
    <filter-class>
        com.subshell.sophora.delivery.filter.URLDecoderFilter
    </filter-class>
    <init-param>
        <param-name>ignoreSuffixes</param-name>
        <param-value>.servlet</param-value>
    </init-param>
</filter>

Get the document uuid for a URL (idForUrl.servlet)

There are some configurations to do in delivery in order to provide the "Open document by URL from clipboard" action within the main menu of the Deskclient. To configure the new idForUrl.servlet execute the steps below:

1. Add to the web.xml file:

<servlet>
<servlet-name>idForUrlServlet</servlet-name>
<servlet-class>com.subshell.sophora.delivery.servlet.IdForUrlServlet</servlet-class>
</servlet>
 
<servlet-mapping>
<servlet-name>idForUrlServlet</servlet-name>
<url-pattern>/system/servlet/idForUrl.servlet</url-pattern>
</servlet-mapping>

1. Finally, the template for the preview (preview.jsp) needs to evaluate the new parameter idForUrl, and possibly forward to the servlet:

<%@ page session="false" pageEncoding="utf-8" contentType="text/html; charset=UTF-8"%>
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>
<%@ taglib tagdir="/WEB-INF/tags/sophora-commons" prefix="sc"%>
<%@ taglib uri="http://www.subshell.com/sophora/jsp" prefix="sophora" %>
 
<c:choose>
    <c:when test="${param.idForUrl eq 1}">
        <c:set var="redirectUrl" >/system/servlet/idForUrl.servlet?url=${param.url}</c:set>
        <c:import url="${redirectUrl}" />
    </c:when>
    <c:otherwise>
        <c:choose>
            <c:when test="${param.print eq 1}">
                <sophora:map sophoraid="${param.sophoraid}" doprint="${param.doprint}" admin="${param.admin}" var="params" />
                <sophora:staticUrl path="/system/print.jsp" params="${params}" var="redirectUrl"/>
            </c:when>
            <c:when test="${!empty param.path}">
                <sophora:url structureNodeUuid="${param.path}" var="redirectUrl" />
            </c:when>
            <c:otherwise>
                <sc:createLink sophoraId="${param.sophoraid}" urlOnly="true" var="redirectUrl" />
            </c:otherwise>
        </c:choose>
    <?xml version="1.0" encoding="UTF-8" ?>
    <html>
        <head>
            <c:if test="${!empty redirectUrl}">
                <meta http-equiv="refresh" content="0; URL=${redirectUrl}">
            </c:if>
        </head>
        <body>
            <c:if test="${empty redirectUrl}">
                <b>Es wird kein Inhalt für dieses Dokument ausgespielt!</b>
            </c:if>
        </body>
    </html>
    </c:otherwise>
</c:choose>

To test whether the configuration is successful, you can call the preview.jsp with a queryString ?idForUrl=1&url='AnyPreviewUrl'. The result of that request should be the UUID of the document shown in 'AnyPreviewUrl'.

Cache Servlet

Error-Sites

To configure error sites which will be rendered by the tomcat, they have to be configured in the DeskClient first. Within the property labeled 404-Error-Documents (sophora:notFoundErrorDocument) on the configuration tab of a structure node, you can define error documents which will be delivered when a document is not found under the configured structure node. By default, this property inherits its value from parent structure nodes. The resulting site will have the status code 404 (Not Found).

Then, some additional configuration is needed:

  • ProxyErrorOverride in the apache configuration has to be set to off, so the tomcat can deliver its own error sites.
  • In the web.xml you have to define jsp files which will be delivered as the default error page if there is no error document configured for an 404 request.
<error-page>
	<error-code>404</error-code>
	<location>/404.jsp</location>
</error-page>
<error-page>
	<error-code>403</error-code>
	<location>/403.jsp</location>
</error-page>
<error-page>
	<error-code>500</error-code>
	 <location>/500.jsp</location>
</error-page>
  • The jsp files have to be placed directly under the webapp directory and have the following content:
<%@ page contentType="text/html; charset=UTF-8"%>
<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core"%>
 
<c:set var="contextPath" value="${pageContext.request.scheme}://${pageContext.request.serverName}:${pageContext.request.serverPort}${pageContext.request.contextPath}" />
 
<c:import url="${contextPath}/site/404-error100.html"/>
  • The hostname, port and context path are set automatically. In this example a request resulting in a 404 error will show the sophora document 404-error100. The same has to be done for 403 and 500 error documents.

Deployment

The delivery webapplication is usually built and assembled by maven and managed over SVN or any other source code management system. The following section describes how Jenkins can be used to automatically checkout, build and deploy webapplications based on the sophora delivery infrastructure.

Jenkins

Jenkins is a java webapplication that monitors executions of jobs, such as building or deployning a software project. Jenkins integrates easily with SVN and Maven so that both technologies can be embedded in the deployment process. While using Jenkins it's possible to trigger deployment jobs either manually or based on events like a commit into the SVN. Each job can be configured and monitored by the Jenkins UI. For more information visit the Jenkis homepage.

The following section describes how to setup a simple deploy job for a delivery webapplication.

Deploy Job

  • To setup a Job just open the Jenkins UI in a browser and click on new Job in the menu on the left.
  • Enter a descriptive name for the job and choose Build a free-style software project from the selection under the Job name textfield.
  • Click ok to save your selection an go on with the configuration.
  • choose Subversion in the section Source Code Management and enter the Repository URL to the module you want to build and deploy
  • choose the event that starts the Job in the Build Triggerssection
  • choose Build after other projects are built to make the Job dependent from other jobs e.g. test runs
  • choose Poll SCM if you want to start the job after a SVN commit (This might be useful for development and test environments)
  • leave the selection blank if you want to start the job manually
  • add a build step Invoke top-level Maven targets in the Build section
  • after jenkins checked out or updated the project we want to build a war file that will be deployed on a servlet engine. So enter clean package as Goals
  • add another build step Execute shell
  • finally we have setup the deployment of the war file that has been build in the previous step. The following script copies the packaged war file to the delivery servers via rsync over ssh.
cd "${WORKSPACE}/target"
# Change into the directory of the assembled webapp.
WEBAPP_DIR="$(ls -1 *.war | sed -r -e 's|\.war$||')"
cd "${WEBAPP_DIR}"
 
# Copy webapp.
rsync --rsh "ssh -C" --delete -avz -c "." "sophora@${TARGET_HOST}:${TARGET_WEBAPP_DIR}"

where

  • TARGET_HOST is the host where the application will be deployed and
  • TARGET_WEBAPP_DIR is the full qualified path to the context root of the webapplication

Mirroring

Static content is mirrored to several delivery machines by the mirror.sh script. This script mirrors a directory from the machine on which it is run to all Staging Servers that are connected to a Sophora Primary Server. The transfer is done via rsync on SSH.

The following settings can be configured in the script's head:

KeyDescriptionExample
INTERVALInterval to mirror file changes to the Sophora Replica Servers (in seconds) 
PID_FILEPath on the filesystem where the script keeps its process ID/var/run/sophora-mirror/sophora-mirror.pid
REPLICATION_SLAVES_URL"replicationSlaves" URL of the Sophora Primary Server which is used to identify all connected Sophora Staging Servers.http://master.example.com:3696/replicationSlaves
SOURCE_DIRPath on the filesystem to the directory that should be mirrored 
DEST_DIRPath on the filesystem of the Sophora Replica Servers to the target directory. The directory is the same on every Staging Server. 

To start the script prompt ./mirror.sh. If want to stop it, you can press CTRL+C or enter kill at the process ID anytime. Before stopping the script completes the current mirroring job.

Last modified on 1/13/21

The content of this page is licensed under the CC BY 4.0 License. Code samples are licensed under the MIT License.

Icon