Media Configuration

The media configuration holds the information about media formats, file paths, and server descriptions.

The media configuration holds the information about the following parts:

document types and formats
media tagging
server descriptions and paths
YouTube global settings

The mediaconfig.xml is a Spring bean XML file.

Document type configuration

The document type of the video must be known to the AVTool. Set it in a MediaDocumentDescription for the property "nodeTypeName".

The "formats" define different resolutions or codecs for the same file. It is a free form text that is used as a list of all currently active file formats. This list may be used, for instance, to filter out the inactive formats. Any media file in a format which is not on this list will not be uploaded. Therefore, all active media file formats that appear in the media documents should be listed here.

Make sure your mediaconfig.xml contains something like the following block:

	<!-- List of document types -->
	<util:list id="documentDescriptionList">
		<!-- Video -->
		<bean class="com.subshell.sophora.avtool.api.MediaDocumentDescription">
			<property name="formats">
				<set>
					<value>h264-1080p</value> 
					<value>h264-720p</value>
				</set>
			</property>
			<!-- Only events for this document type are handled. -->
			<property name="nodeTypeName" value="example-nt:video"/>
			<!-- Optional: Name of a playout channel. The presence of a document on this channel corresponds to the visibility of
				the media files on YouTube. -->
			<property name="playoutChannelName" value="YouTube"/>
			<!-- Optional: Name of property that has the duration of the video. Format must be [[hours:]minutes:]seconds[.SSS]. Only
				used for captions if an intro/opener of this type is prepended to the video to be uploaded. -->
			<property name="durationPropertyName" value="example:duration"/>
		</bean>
	</util:list>

Media tagging

It is possible to add/extend MP3 tags in audio files and MP4 tags in videos. The MP3 tags are fully handled by the AVTool. For MP4 tagging, an external application AtomicParsley is used.

The values for the tags can be set in the configuration. You can use fixed values, property names, or Groovy scripts.

Configure MP3 tagging

For setting MP3 tags, you have to define an mp3TagsMapping property within the mediaTaggingConfig bean. The key of this mapping is the name of a tag. The value can be either a fixed value like "Example", a property name like "example:headline" (see tagging.propertyPrefix.regexp in configuration), or a script like "scriptClass:TagTitle" (see tagging.scriptClassPrefix.regexp in configuration).

For podcasts, different mappings can be specified. The mapping for podcast can use both the values from the podcast document itself and the values from the media documents.

To add chapter to the MP3 tags, you need to provide a mp3TagsChapterConfiguration. That in turn needs a mp3ChapterEmbeddedTagsMapping. The latter corresponds to a table in the document.

Configure MP4 tagging

In order to use MP4 Tagging, AtomicParsley must be installed and made executable for the AVTool.

You need to define the atomicParsleyExe, an mp4TagsMapping, and an mp4TagsAPMapping that will be used to create the command line to execute.

Media Tagging Example Configuration

	<bean id="mediaTaggingConfig" class="com.subshell.sophora.avtool.mediatagging.api.MediaTaggingConfiguration">
		<property name="mp3TagsMapping" ref="mp3TagsMapping" />
		<property name="mp3PodcastTagsMapping" ref="mp3PodcastTagsMapping" />
		<property name="mp3PodcastFromAudioTagsMapping" ref="mp3PodcastFromAudioTagsMapping" />
		<property name="mp3TagsChapterConfiguration" ref="mp3TagsChapterConfiguration" />
		<property name="mp4TagsMapping" ref="mp4TagsMapping" />
		<property name="mp4TagsAPMapping" ref="mp4TagsAPMapping" />
		<property name="formatSuffixMapping" ref="formatSuffixMapping" />
		<property name="atomicParsleyExe" value="/opt/local/bin/atomicparsley" />
		<property name="podcastImageVariant" value="podcast" />
		<property name="imageChildNodeName" value="ndr:image" />
		<property name="tagPodcastsOnly" value="true" />
	</bean>

	<!-- Optional tagging of chapters in audios. The child node must contain
		start offsets and may have addition properties for tags. -->
	<bean id="mp3TagsChapterConfiguration"
		class="com.subshell.sophora.avtool.mediatagging.api.ChapterTaggingConfiguration">
		<property name="childNodeName" value="ndr:chapterdata" />
		<property name="startTimePropertyName" value="ndr:chapterTime" />
		<property name="totalDurationPropertyName" value="ndr:duration" />
		<property name="embeddedTagsMapping" ref="mp3ChapterEmbeddedTagsMapping" />
	</bean>

	<!-- Optional tags for each chapter in audios. Map: Tag name -> expression
        Tagnamen: com.subshell.sophora.avtool.mediatagging.Mp3TagConverter -->
	<util:map id="mp3ChapterEmbeddedTagsMapping">
		<entry key="TITLE" value="ndr:chapterTitle" />
	</util:map>

	<!-- For tag names see com.subshell.sophora.avtool.mediatagging.Mp3TagConverter. -->
	<util:map id="mp3TagsMapping">
		<entry key="ARTIST" value="ndr:author" />
		<entry key="TYER" value="ndr:broadcastdate" />
		<entry key="GENRE" value="28" />
		<entry key="TRACK" value="0" />
		<entry key="TITLE" value="ndr:headline" />
		<entry key="COMMENT" value="ndr:url" />
		<entry key="ATTACHED_PICTURE" value="" />
		<entry key="COMPOSER" value="NDR" />
		<entry key="RADIO_OWNER" value="NDR" />
		<entry key="RADIO_NAME" value="ndr:broadcast" />
		<entry key="CONTENT_GROUP_DESC" value="ndr:shorttext" />
	</util:map>

	<util:map id="mp4TagsMapping">
		<entry key="TITLE" value="ndr:headline" />
		<entry key="COMMENT" value="ndr:shorttext" />
		<entry key="DESCRIPTION" value="ndr:shorttext" />
		<entry key="ARTIST" value="ndr:author" />
		<entry key="YEAR" value="ndr:broadcastdate" />
		<entry key="SHOW" value="ndr:broadcast" />
		<entry key="KEYWORD" value="ndr:keywords" />
		<entry key="COMPOSER" value="NDR" />
		<entry key="ALBUM_ARTIST" value="NDR" />
		<entry key="TV_NETWORK" value="NDR" />
		<entry key="PODCAST_FLAG" value="true" />
		<entry key="ARTWORK" value="" />
	</util:map>

	<util:map id="mp4TagsAPMapping">
		<entry key="TITLE" value="--title" />
		<entry key="COMMENT" value="--comment" />
		<entry key="DESCRIPTION" value="--description" />
		<entry key="ARTIST" value="--artist" />
		<entry key="YEAR" value="--year" />
		<entry key="SHOW" value="--TVShowName" />
		<entry key="KEYWORD" value="--keyword" />
		<entry key="COMPOSER" value="--composer" />
		<entry key="ALBUM_ARTIST" value="--albumArtist" />
		<entry key="TV_NETWORK" value="--TVNetwork" />
		<entry key="PODCAST_FLAG" value="--podcastFlag" />
		<entry key="ARTWORK" value="--artwork" />
	</util:map>

Servers

The AVTool must download the files from a server to be able to upload them to internet servers and YouTube. The source server is named "media server" because it holds all media files. The internet servers for the online files are called "streaming server" because clients stream videos from there.

There are different protocols for the transport to choose from:

SFTP (default)
FTP(S)
local file system
Akamai NetStorage Usage API

You need to configure the credentials for accessing the server. Usually, a username and a password will be set.

The media server is defined as a ServerDescription. Multiple servers can be provided. A file will be looked up in order of the media servers until found.

The transporterFactory determines the protocol for transferring files. For locating the files, a pattern can be used to form the path on the server. The full path consists of two parts:

The "basedir" set in the ServerDescription. The interpretation depends on the used protocol especially for relative paths.
The "formatDirectoryMapping" provides further parts to the directory.

If the formatDirectoryMapping for a Path does not contain the {filename} placeholder it is automatically appended.

The folder on the server which contains the files must be set as "basedir". The media files can be organised in sub folders with a pattern that contains placeholders (e.g. a date). Such patterns can be configured per format in the map "formatDirectoryMapping".

After putting the parts together the following placeholders will be resolved in a path like /data/flashmedia/streams/ndr/%1$tY/%1$tm%1$td/{sophora_id}/TV-20130603-1637-5342.mp4:

File path pattern formats
Name	Example	Description
Date pattern	`%1$tY/%1$tm%1$td`	A Java format string that formats the date contained in the sequence name. In the configuration the property "`dateRegex`" must be set to extract a date from the sequence name.
Sequence name	`{sequencename}`	The sequence name of the current file that is being processed as provided in the document.
File name	`{filename}`	The target paths or previous target paths as provided in the document. Note that the file name is automatically appended to the path on the server if the placeholder is not in the configuration.
Sophora ID	`{sophora_id}`	The ID of the Sophora document from which the current file was retrieved.

The following example of a media server list describes some options:

 <util:list id="mediaServerDescriptionList">
	<bean class="com.subshell.sophora.avtool.api.ServerDescription">
		<!-- pre-defined: defaultTransporterFactory (SFTP), ftpTransporterFactory, ftpsTransporterFactory, localTransporterFactory, akamaiNetStorageTransporterFactory (HTTP), akamaiNetStorageSymlinkTransporterFactory, akamaiNetStorageSymlinkParentTransporterFactory -->
		<property name="transporterFactory" ref="defaultTransporterFactory"/>
		<!-- Server name from which the video files should be retrieved. When local file transfer is used, this field is only
			used for logging. -->
		<property name="host" value="mediaserver.example.com"/>
		<property name="port" value="22"/>
		<property name="user" value="sophora-avtool"/>
		<property name="password" value="changeMe"/>
		<property name="keyfile" value=""/>
		<property name="basedir" value="/data/mediasource"/>
		<property name="formatDirectoryMapping">
			<map>
				<entry key="h264-1080p" value="."/>
				<entry key="h264-720p" value="h264/%1$tY/%1$tm%1$td"/> 
			</map>
		</property>
		<!-- Optional: A regex to parse the date from a file's name, which will then be applied to all entries in the formatDirecoryMapping
			that contain format specifiers, e.g. "%1$tY/%1$tm%1$td". The date in the file name must be written in the format "yyyyMMdd". -->
		<property name="dateRegex" value="#?.{3}(\\d{8}).+"/>
	</bean>
</util:list>

Akamai Symlinks

The AVTool can use the Akamai NetStorage API to create symlinks for files. A usecase for this is for example:
The encoding system uploads files to Akamai in a private folder. When the files are to be published, the AVTool creates symlinks from a public folder to the files in the private folder.

To use this functionality, com.subshell.sophora.avtool.api.AkamaiServerDescription bean with either akamaiNetStorageSymlinkTransporterFactory or akamaiNetStorageSymlinkParentTransporterFactory must be configured.
It is advised to also set ignoreFileDigestCheckBeforeUpload to true and have any servers with these transporters first in the list of servers.

Media file digest check

By default the AVTool computes a hash for every file uploaded. If the same file would be uploaded again later the hash is checked against a list of known hashes. That way unecessary uploads can be avoided.
It is possible to skip that check with the ignoreFileDigestCheckBeforeUpload parameter.
That can be helpful if the AVTool does not need actually need to upload a file e.g. when only symlinks should be created.

Full Example Configuration

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns:util="http://www.springframework.org/schema/util"
	xsi:schemaLocation="http://www.springframework.org/schema/beans
	http://www.springframework.org/schema/beans/spring-beans.xsd
	http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd">

	<!--####################################################################
		Document type and media formats
		#################################################################### -->

	<!-- List of document types -->
	<util:list id="documentDescriptionList">
		<!-- Video -->
		<bean class="com.subshell.sophora.avtool.api.MediaDocumentDescription">
			<property name="formats">
				<set>
					<value>youtube</value>
				</set>
			</property>
			<!-- Only events for this document type are handled. -->
			<property name="nodeTypeName" value="example-nt:video"/>
			<!-- Optional: Name of a playout channel. The presence of a document on this channel corresponds to the visibility of
				the media files on YouTube. -->
			<property name="playoutChannelName" value="YouTube"/>
			<!-- Optional: Name of property that has the duration of the video. Format must be [[hours:]minutes:]seconds[.SSS]. Only
				used for captions if an intro/opener of this type is prepended to the video to be uploaded. -->
			<property name="durationPropertyName" value="example:duration"/>
		</bean>
	</util:list>

	<!-- Feedback to DeskClient. Map: error type to error string (you can use a select value for readable labels). -->
	<util:map id="deskclientFeedbackErrTypeConf">
		<entry key="TRANSPORT_MEDIASERVER" value="transport_mediaserver"/>
		<entry key="YOUTUBE" value="youtube"/>
		<entry key="OTHER" value="other"/>
		<entry key="MANY" value="many"/>
		<entry key="OK" value="ok"/>
	</util:map>

	<!--####################################################################
		Sources for video files
		#################################################################### -->

	<bean id="localTransporter" class="com.subshell.sophora.avtool.transporter.LocalFSTransporterFactory"/>

	<util:list id="mediaServerDescriptionList">
		<bean class="com.subshell.sophora.avtool.api.ServerDescription">
			<!-- pre-defined: defaultTransporterFactory (SFTP), ftpTransporterFactory, ftpsTransporterFactory, localTransporterFactory,
				akamaiNetStorageTransporterFactory (HTTP) -->
			<property name="transporterFactory" ref="defaultTransporterFactory"/>
			<!-- Server name from which the video files should be retrieved. When local file transfer is used, this field is only
				used for logging. -->
			<property name="host" value="mediaserver.example.com"/>
			<property name="port" value="22"/>
			<property name="user" value="sophora-avtool"/>
			<property name="password" value="changeMe"/>
			<property name="keyfile" value=""/>
			<property name="basedir" value="/data/mediasource"/>
			<property name="formatDirectoryMapping">
				<map>
					<entry key="youtube" value="."/>
				</map>
			</property>
			<!-- Optional: A regex to parse the date from a file's name, which will then be applied to all entries in the formatDirecoryMapping
				that contain format specifiers, e.g. "%1$tY/%1$tm%1$td". The date in the file name must be written in the format "yyyyMMdd". -->
			<property name="dateRegex" value="#?.{3}(\\d{8}).+"/>
		</bean>
	</util:list>

	<!--####################################################################
		YouTube
		#################################################################### -->

	<!-- YouTube settings for all channels -->
	<bean id="youtubeGlobalConfig" class="com.subshell.sophora.avtool.api.youtube.YoutubeConfiguration">
		<!-- Property of the document that indicates which channel to use. -->
		<property name="youtubeAccountPropertyName" value="example:youtubeChannel"/>
		<!-- The YouTube action to take when a video document is deleted or set offline. Options are DELETE, SET_PRIVATE and DO_NOTHING (default). -->
		<property name="deleteEventAction" value="DELETE"/>
		<property name="offlineEventAction" value="SET_PRIVATE"/>
		<!-- Optional: Application for adding opener and closer to the beginning/end of video. The specific videos are configured
			per channel. The local script file will be executed before each YouTube upload. It will receive the following parameters:
			- path to the opener file (if present)
			- path to the original video file (downloaded to a temp folder)
			- path to the closer file (if present)
			- path to the concatenated output file that the script must create
		-->
		<property name="concatVideosScript"
		          value="/cms-install-directory/ytc_opener_und_closer/addOpenerCloser_ffmpeg.sh"/>
	</bean>

	<util:list id="streamingServerDescriptionList">

		<!--####################################################################
		Optional list of servers to upload files to. Media files can be uploaded to Akamai or similar CDN servers using ftp, sftp
		and ftps. When a video document is set offline, the corresponding media files will be removed from these servers. Also
		purging of files from Akamai can be configured.
		#################################################################### -->

		<bean class="com.subshell.sophora.avtool.api.AkamaiServerDescription">

			<property name="name" value="video-cdn"/>
			<property name="transporterFactory" ref="sftpTransporterFactory"/>
			<property name="host" value="akamai.example.net"/>
			<property name="port" value="22"/>
			<property name="user" value="alice"/>
			<property name="password" value="secret"/>
			<property name="keyfile" value=""/>

			<!-- Base directory for all uploads -->
			<property name="basedir" value="/video"/>

			<!-- Optional: A regex to parse the date from a file's name. -->
			<property name="dateRegex" value="#?.{3}(\\d{8}).+"/>

			<!-- Akamai FastPurge -->
			<property name="akamaiUrlPatterns">
				<list>
					<value>http://download.mysite.com/video/%1$tY/%1$tm%1$td/{filename}</value>
					<value>http://media.mysite.com/video/%1$tY/%1$tm%1$td/{filename}</value>
				</list>
			</property>
			<property name="akamaiTagPatterns">
				<list>
					<value>{filename}</value>
				</list>
			</property>

			<!-- Akamai HLS/HDS Purge -->
			<property name="akamaiRestPurgeConfiguration">
				<list>
					<bean class="com.subshell.sophora.avtool.api.AkamaiRestPurgeRequestConfiguration">
						<property name="requestName" value="HLS"/>
						<property name="requestPath" value="/config-media-live/v1/vod/purge/HLS"/>
						<property name="manifestUrl" value="https://mysite.akamaihd.net/i/video/%1$tY/%1$tm%1$td/{sequencename}.,webs,webm,webl,webxl,.h264.mp4.csmil/master.m3u8"/>
						<property name="fileNamePattern" value="%1$tY/%1$tm%1$td/{filename}"/>
					</bean>
				</list>
			</property>

			<!-- Mapping of format to server path. Can use date fields parsed using the dateRegex. -->
			<property name="formatDirectoryMapping">
				<map>
					<entry key="youtube" value="%1$tY/%1$tm%1$td"/>
				</map>
			</property>
		</bean>
	</util:list>

</beans>

When configuring the ServerDescription for the Akamai NetStorage Usage API, you need to use your upload account ID as user and the HTTP CMS API key as password. The Ingest directory is your basedir.

The content of this page is licensed under the CC BY 4.0 License. Code samples are licensed under the MIT License.

AVTool Media Configuration