Metadata Supplier | Version 4

Sophora Metadata Supplier for ARD Mediathek/Audiothek Developing Guide

This article describes how to develop plugins for the Sophora Metadata Supplier for ARD Mediathek/Audiothek.

The Metadata Supplier as a link between the source system and the ARD Mediathek/Audiothek does not specify the contents of the resources that will be transferred. For this purpose, custom mappers are used as plugins in the Metadata Supplier application. They define the mapping from source objects to ARD Core API resources by creating Metadata Supplier model objects and filling in all the required (and optional) fields. The finished model objects are then returned to the Metadata Supplier application which will in turn continue with the preparation and uploading process to the ARD Mediathek/Audiothek.

Sophora Metadata Supplier for ARD Mediathek/Audiothek Components
The components of the Metadata Supplier.

Create a custom mapper plugin

To create a custom mapper plugin, a Java project with the following Maven dependency to the artifact metadata-supplier-mapper has to be created. Note that we provide a metadata-supplier-plugin-example project, which can be used as a starting point for your own custom mapper (GroupId: com.subshell.sophora.metadatasupplier, ArtifactId: metadata-supplier-plugin-example).

We provide helpers for handling Sophora documents and Spring-Data-Sophora entities in the projects metadata-supplier-sophora-commons and metadata-supplier-spring-data-sophora-commons. The example project demonstrates how to check for Sophora external IDs.

<dependencyManagement>
    <dependencies>
        <!-- Metadata Supplier -->
        <dependency>
            <groupId>com.subshell.sophora.metadatasupplier</groupId>
            <artifactId>metadatasupplier-parent</artifactId>
            <version>${sophora.metadatasupplier.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

<dependencies>
    <!-- Sophora Metadata Supplier -->
    <dependency>
        <groupId>com.subshell.sophora.metadatasupplier</groupId>
        <artifactId>metadata-supplier-mapper</artifactId>
        <scope>provided</scope>
    </dependency>
    <!-- Optional helper for Sophora as source system -->
    <dependency>
	    <groupId>com.subshell.sophora.metadatasupplier</groupId>
	    <artifactId>metadata-supplier-sophora-commons</artifactId>
        <scope>provided</scope>
    </dependency>
    <!-- Optional helper for using Spring Data Sophora and Sophora as source system (includes metadata-supplier-sophora-commons) -->
    <dependency>
	    <groupId>com.subshell.sophora.metadatasupplier</groupId>
	    <artifactId>metadata-supplier-spring-data-sophora-commons</artifactId>
        <scope>provided</scope>
    </dependency>
 </dependencies>

Create a launcher project

To start the Metadata Supplier with your custom mapper plugin for development reasons from your IDE you have to use a launcher project. Note that we provide a metadata-supplier-plugin-example-launcher project, which can be used as a starting point (GroupId: com.subshell.sophora.metadatasupplier, ArtifactId: metadata-supplier-plugin-example-launcher).

1. Create a new Maven project that is used as launcher project (e.g. metadata-supplier-with-custom-plugin-launcher) with a dependency to metadata-supplier-application and your custom plugin project created above:

...
<artifactId>metadata-supplier-with-custom-plugin-launcher</artifactId>
...
<dependencies>
	<!-- Metadata Supplier Application -->
	<dependency>
 		<groupId>com.subshell.sophora.metadatasupplier</groupId>
 		<artifactId>metadata-supplier-application</artifactId>
		<!-- The released version of the Metadata Supplier -->
		<version>4.5.0</version>
	</dependency>
	<!-- Your custom Metadata Supplier plugin -->
	<dependency>
 		<groupId>com.mycompany.metadatasupplier</groupId>
		<artifactId>my-custom-metadata-supplier-plugin</artifactId>
		<!-- The development version of your plugin -->
		<version>4.x.y-SNAPSHOT</version>
	</dependency>
</dependencies>
...

2. Create (or copy) an application.yml to the root of your launcher project (next to the pom.xml) and configure it with your settings. See the administration documentation for examples.

3. In Eclipse IDE (IntelliJ IDEA should be similar) create a launch configuration for the launcher project with the following settings:

  • Name: Sophora Metadata Supplier with custom plugin (or similar)
  • Project: metadata-supplier-with-custom-plugin-launcher (your launcher project)
  • Main class: com.subshell.sophora.metadatasupplier.application.Application
  • Working Directory: ${workspace_loc:metadata-supplier-with-custom-plugin-launcher} (the launcher project root)

4. Finally run the launch configuration. The Metadata Supplier should start with your settings and your plugin.

Create custom mappers

The Metadata Supplier recognizes a mapper class by looking for two criteria. It needs to be a Spring component (with the @Component annotation) and it has to extend the IResourceMapper interface. This generic interface can be used to create any kind of resource. There are also ways to map specific kinds of resources with the interfaces that already extend from the base interface. An example would be the IShowMapper to map source objects to resources of type IShow. The following sections go into further detail as to how mappers are implemented. We also provide an ExampleShowMapper in the sources JAR file of the metadata-supplier-plugin-example project. It also determines if an unpublish or republish must be done at the ARD Core API based on the isPublished() on mapped objects that support it.

@Component
@Slf4j
public class ExampleShowMapper implements IShowMapper {

  @Override public boolean accept(SourceDescriptor sourceDescriptor, Parameters parameters) {
    // To be implemented
  }
  @Override public List<IShow> mapToResources(SourceDescriptor sourceDescriptor, IMappingContext mappingContext) {
    // To be implemented
  }
}

Implement IResourceMapper

The interface methods of the IResourceMapper will be triggered by the IMetadataSupplier (which is called by events from the source system for example). Before any mapping takes place, the accept(SourceDescriptor, Parameters) method will be called to select the correct mapper for the source object. Only if your custom mapper identifies with the given source descriptor, the mapToResources(SourceDescriptor, IMappingContext) method will be called. In the Sophora environment for example, the SourceDescriptor consists of the UUID and the node type of a Sophora document. So with the accept method you could define if the mapper is suitable to map documents of certain node types.

@Override
public boolean accept(SourceDescriptor sourceDescriptor, Parameters parameters) {
  boolean accepted;
  // For example you can check the type of the source.
  accepted = sourceDescriptor.hasSourceType(ACCEPTED_TYPE_NAME);
  // Sophora users may use the SophoraSourceDescriptor to use Sophora wordings.
  accepted = SophoraSourceDescriptor.isValidForNodeType(sourceDescriptor, ACCEPTED_TYPE_NAME);
  return accepted;
}

In the mapToResources(SourceDescriptor, IMappingContext) method, the mapping from your source objects to the Metadata Supplier model objects takes place. It provides you with the descriptor of the source object that needs to be mapped and an additional parameter of type IMappingContext which holds the Parameters and can be used for further things explained in the following sections. Since the mapper is responsible to retrieve the source object from the source system in order to map it, you may use a mechanism like Spring Data Sophora in the Sophora context. After the mapping process, the finished resource(s) have to be returned as a list, which may be empty if the mapping has failed. The Metadata Supplier automatically sorts the returned resource objects into the correct order for delivery to the ARD Mediathek/Audiothek. It also determines if an unpublish or republish must be done at the ARD Core API based on the isPublished() on mapped objects that support it.

@Override
public List<IShow> mapToResources(SourceDescriptor sourceDescriptor, IMappingContext mappingContext) {
  // The id of the source object is needed to get the source object from the source system.
  String sourceId = sourceDescriptor.getSourceId();
  // Sophora users may use the Sophora wordings of the SophoraSourceDescriptor.
  sourceId = SophoraSourceDescriptor.getExternalId(sourceDescriptor);
  IShowSourceObject myShowSourceObject = mySourceSystem.getShowSourceObject(sourceId);

  ShowBuilder<?, ?> showBuilder = Show.builder()
    .withTitle(myShowSourceObject.getTitle())
    .withExternalId(new ExternalId(sourceId))
    .withPublished(myShowSourceObject.isOnline())
    (...)
  // An image collection containing the added images will be created by the Metadata Supplier.
  for (IImage image : images) {
    showBuilder.withImage(image);
  }

  IShow show = showBuilder.build();
  return Collections.singletonList(show);
}

Unpublish/Republish without fully mapped resources

If the source object is no longer available in the source system to be fully mapped (e.g. it has been deleted) and the associated entity is to be unpublished in the ARD Mediathek/Audiothek, then a resource of type Publishable can be created instead of the full IResource object. This contains only the type, the external id and the instruction to unpublish.

@Override
public List<IResource> mapToResources(SourceDescriptor sourceDescriptor, IMappingContext mappingContext) {
	String sourceId = sourceDescriptor.getSourceId();
	ISeasonSourceObject seasonSourceObject = mySourceSystem.getSeasonSourceObject(sourceId);
	IResource season;
	if (seasonSourceObject == null) {
		season = Publishable.builder()
				.withType(Type.SEASON)
				.withExternalId(ExternalId.of(sourceId))
				.withPublished(false)
				.build();
	} else {
		// Otherwise map the full resource
		season = Season.builder()
				.withExternalId(ExternalId.of(sourceId))
				//.with...
				.build();
	}
	return List.of(season);
}

Unpublish resources that can no longer be mapped from the source object

A source object can be mapped multiple times and therefore create, update and publish several published resources in the ARD Mediathek/Audiothek, e.g. a varying number of publications. However, if the data from which the previous published resources were created is no longer available in the source object, in most cases the resources should be unpublished in the ARD Mediathek/Audiothek, too. To do this, the method IMappingContext.setUnpublishPreviouslyMappedResources(true/false) can be used. If set to true a Publishable.withPublished(false) is added for all publishable resources that were created in the previous mapping but are not included in the passed list of the current mapping and are not published for any other source object.

@Override
public List<IResource> mapToResources(SourceDescriptor sourceDescriptor, IMappingContext mappingContext) {
    ...
    List<IResource> resources = new ArrayList<>();
    resources.add(...)

    // Set that all other resources that have been mapped before from source object should be unpublished (if possible)
    mappingContext.setUnpublishPreviouslyMappedResources(true);
    return resources;
}

In addition, IMappingContext.getResourceProvider() gives you access to the data stored by the Metadata Supplier of the last successful mapping of the current source.

Update resources only if they already exist in the ARD Mediathek/Audiothek

To prevent entities from being created when they should not (and do not yet) exist in the ARD Mediathek/Audiothek set the createIfNotExisting flag to false (default is true). In other words: Set createIfNotExisting of a resource to false to update the resource only if it already exists in the ARD Mediathek/Audiothek. In combination with withPublished(false) resources can be unpublished in the ARD Mediathek/Audiothek only if they already exist there (without creating them, only to be able to unpublish them).

// Determine whether the mapped source should exist in the ARD Mediathek/Audiothek (maybe it depends on a flag in the source object that has been switched)
boolean shouldExistInMediathek = ...
// Determine whether the mapped source should be visible in the ARD Mediathek/Audiothek (maybe it depends on the state of the source object that has been switched)
boolean shouldBePublishedInMediathek = ...
Show show = Show.builder()
		.withExternalId(...)
		.withTitle(...)
		//.with...
		.withCreateIfNotExisting(shouldExistInMediathek)
		.withPublished(shouldBePublishedInMediathek)
		.build();

Give (editorial) feedback

The Metadata Supplier provides fixed feedback messages for standard situations and for technical information to the editors:

  • No resource mapper accepts the SourceDescriptor => No feedback message
  • Multiple resource mapper accept the SourceDescriptor => No feedback message
  • Exactly one resource mapper accepts the SourceDescriptor but provides no IResource => No feedback message
  • Exactly one resource mapper accepts the SourceDescriptor and returns exactly one IResource => See the following feedback messages table
  • Exactly one resource mapper accepts the SourceDescriptor and returns multiple IResource objects => Combinations of the following messages table
Feedback messages table
Metadata Supplier Resource TypewithCreateIfNotExisting(...)withPublished(...)Resource exists in ARD Mediathek/AudiothekResource is published in ARD Mediathek/AudiothekFeedback message(s) (All of FeedbackType.INFO)
Show, Season, Publication, PermanentLivestreamtruetruetruetrueErfolgreich an die ARD Mediathek/Audiothek gesendet (am dd.MM.yyyy, um HH:mm:ss Uhr) + Ein (erneutes) Veröffentlichen in der ARD Mediathek/Audiothek war nicht notwendig
Season, Publication, PermanentLivestreamtruetruetruefalseErfolgreich an die ARD Mediathek/Audiothek gesendet (am dd.MM.yyyy, um HH:mm:ss Uhr) + Erfolgreich in der ARD Mediathek/Audiothek veröffentlicht
Season, Publication, PermanentLivestreamtruetruefalse- (by default true after creation)Erfolgreich an die ARD Mediathek/Audiothek gesendet (am dd.MM.yyyy, um HH:mm:ss Uhr) + Ein (erneutes) Veröffentlichen in der ARD Mediathek/Audiothek war nicht notwendig
Season, Publication, PermanentLivestreamtruefalsetruetrueErfolgreich an die ARD Mediathek/Audiothek gesendet (am dd.MM.yyyy, um HH:mm:ss Uhr) + Erfolgreich aus der ARD Mediathek/Audiothek entfernt
Season, Publication, PermanentLivestreamtruefalsetruefalseErfolgreich an die ARD Mediathek/Audiothek gesendet (am dd.MM.yyyy, um HH:mm:ss Uhr) + Ein (erneutes) Entfernen aus der ARD Mediathek/Audiothek war nicht notwendig
Season, Publication, PermanentLivestreamtruefalsefalse- (by default true after creation)Erfolgreich an die ARD Mediathek/Audiothek gesendet (am dd.MM.yyyy, um HH:mm:ss Uhr) + Erfolgreich aus der ARD Mediathek/Audiothek entfernt
Season, Publication, PermanentLivestreamfalsetruetruetrueErfolgreich an die ARD Mediathek/Audiothek gesendet (am dd.MM.yyyy, um HH:mm:ss Uhr) + Ein (erneutes) Veröffentlichen in der ARD Mediathek/Audiothek war nicht notwendig
Season, Publication, PermanentLivestreamfalsetruetruefalseErfolgreich an die ARD Mediathek/Audiothek gesendet (am dd.MM.yyyy, um HH:mm:ss Uhr) + Erfolgreich in der ARD Mediathek/Audiothek veröffentlicht
Season, Publication, PermanentLivestreamfalsetruefalse-Ein Veröffentlichen in der ARD Mediathek/Audiothek war nicht möglich
Season, Publication, PermanentLivestreamfalsefalsetruetrueErfolgreich an die ARD Mediathek/Audiothek gesendet (am dd.MM.yyyy, um HH:mm:ss Uhr) + Erfolgreich aus der ARD Mediathek/Audiothek entfernt
Season, Publication, PermanentLivestreamfalsefalsetruefalseErfolgreich an die ARD Mediathek/Audiothek gesendet (am dd.MM.yyyy, um HH:mm:ss Uhr) + Ein (erneutes) Entfernen aus der ARD Mediathek/Audiothek war nicht notwendig
Season, Publication, PermanentLivestreamfalsefalsefalse-Ein Entfernen aus der ARD Mediathek/Audiothek war nicht möglich
Publishabletrue (not possible)--- 
PublishablefalsetruetruetrueEin (erneutes) Veröffentlichen in der ARD Mediathek/Audiothek war nicht notwendig
PublishablefalsetruetruefalseErfolgreich in der ARD Mediathek/Audiothek veröffentlicht
Publishablefalsetruefalse-Ein Veröffentlichen in der ARD Mediathek/Audiothek war nicht möglich
PublishablefalsefalsetruetrueErfolgreich aus der ARD Mediathek/Audiothek entfernt
PublishablefalsefalsetruefalseEin (erneutes) Entfernen aus der ARD Mediathek/Audiothek war nicht notwendig
Publishablefalsefalsefalse-Ein Entfernen aus der ARD Mediathek/Audiothek war nicht möglich

In addition, custom feedback messages can be added to provide assistance to the editors. By calling the given addFeedback(...) methods on the IMappingContext visitor object of IResourceMapper.mapToResources(...), it can be used to collect information, warnings and errors (enum FeedbackType) that may occur during the mapping process.

// A show cannot be mapped without images.
if (myImageSourceObjects.isEmpty()) {
  mappingContext.addFeedback(FeedbackType.ERROR, "The show has no images.");
  return Collections.emptyList();
}

After the new or updated resource has been sent to the ARD Mediathek/Audiothek (which may succeed or fail), the feedback messages from the mapper as well as additional feedback from the Metadata Supplier itself are sent to all registered IFeedbackHandlers. In case of the Sophora FeedbackHandler that feedback is displayed to the user in Sophora, but you are also able to create your own IFeedbackHandler. It needs to be a Spring component (with the @Component annotation) so that the Metadata Supplier can find it. The Metadata Supplier calls the handleFeedback(Feedback) method for each collected feedback during the mapping, saving, unpublishing and republishing process of a source. You can handle this feedback here in your own way, e.g. by logging, sending e-mails, writing back to the source system, etc.

@Component
@Slf4j
class ExampleFeedbackHandler implements IFeedbackHandler {

  @Override
  public void handleFeedback(Feedback feedback) {
    // In this example we just log the error messages
    SourceDescriptor sourceDescriptor = feedback.getSourceDescriptor();
    log.info("Feedback for {}", sourceDescriptor);
    // You can get all feedback entries or filtered by type.
    for (FeedbackEntry errorEntry : feedback.getEntries(FeedbackType.ERROR)) {
      log.error("- {}", errorEntry.getMessage());
    }
  }

}

Register and get previously used sources

The addUsedSource(...) method on the IMappingContext visitor object should be called for each additional source object that is used during the mapping of a source. The Metadata Supplier will save the connection between those objects in a database so that next time when one of the "used sources" is mapped, it will automatically trigger the mapping of all sources that have been "users" of this source object. This makes sure a source object is always up-to-date if one of its "used sources" has changed.

SourceDescriptor sourceDescriptorOfUsedSourceObject = new SourceDescriptor(mySourceObject.getId(), mySourceObject.getType());
mappingContext.addUsedSource(sourceDescriptorOfUsedSourceObject);

Sophora users may use the Sophora wordings of the SophoraSourceDescriptor.

SourceDescriptor sourceDescriptorOfUsedSourceObject = SophoraSourceDescriptor.create(mySourceObject.getId(), mySourceObject.getType());
mappingContext.addUsedSource(sourceDescriptorOfUsedSourceObject);

Exactly the used sources of the respective mapping process are saved. Usage relationships already registered from a previous mapping process of the source are removed, e.g. if less or other sources are used in this mapping process.

Sophora users must not forget to configure the document types and state changes for the used sources in the application.yml in order to trigger the Metadata Supplier when one of the used sources has been changed in Sophora.

It is also possible to access the sources used in the previous mapping of the source being mapped via IMappingContext.getPreviousUsedSources().

Install a custom mapper plugin

When the Java project with the custom mapper(s) is built, the result has to be a JAR that includes all its dependencies (jar-with-dependencies). Then place your built plugin JAR in the plugins folder which is located next to the Metadata Supplier Application JAR file.

By default the Metadata Supplier scans for new components in the package com.subshell.sophora.metadatasupplier (and all sub packages). If you created your mapper implementation in another package, you have to configure that in your application.yml.

---
# Application settings
spring:
  application:
    spring-additional-base-packages: 'my.custom.package.name'

Create a docker image

The Metadata Supplier Docker Image can be used to create your own custom docker image. The working directory of the base image is the /app folder, where the application.yml file must be copied to. The subdirectory plugins must be filled with your custom mapper plugin from the previous step. That's all and your Dockerfile should look like this:

ARG SMS_TAG

FROM docker.subshell.com/metadata-supplier/metadata-supplier:$SMS_TAG

COPY application.yml . 
COPY your-custom-mapper-plugin-jar-with-dependencies.jar ./plugins/

Now, run your own Metadata Supplier Docker Container with this image and enjoy the working Metadata Supplier Application wherever you want.

Last modified on 11/9/22

The content of this page is licensed under the CC BY 4.0 License. Code samples are licensed under the MIT License.

Icon