Smart tag your video assets video-smart-tags

The expanding need for new content calls for reduced manual efforts to deliver compelling digital experiences in no time. Adobe Experience Manager as a Cloud Service supports auto-tagging of video assets using artificial intelligence. Tagging the videos manually can be time-consuming. However, Adobe Sensei powered video smart tagging feature uses artificial intelligence models to analyze video content and add tags to the video assets. Thereby reducing time for DAM users to deliver rich experiences to their customers. Adobe’s machine learning service generates two sets of tags for a video. While, one set corresponds to objects, scenes, and attributes in that video; the other set relates to actions such as drinking, running, and jogging.

Video tagging is enabled by default in Adobe Experience Manager as a Cloud Service. However, you can opt-out of video smart tagging on a folder. Videos are auto-tagged when you upload new videos or re-process existing ones. Experience Manager also creates the thumbnails and extracts metadata of the video files. The smart tags are displayed in descending order of their confidence score in asset Properties.

Smart tagging videos on upload smart-tag-assets-on-ingestion

When you upload video assets to Adobe Experience Manager as a Cloud Service, the videos are processed. Once the processing is complete, see the Basic tab of asset Properties page. Smart tags are automatically added to the video under Smart Tags. Asset microservices uses Adobe Sensei to create these smart tags.

The applied smart tags are sorted in descending order of confidence score, combined for object and action tags, within Smart Tags.

Smart tagging existing videos in DAM smart-tag-existing-videos

The already existing video assets in DAM are not smart tagged automatically. You need to Reprocess Assets manually to generate smart tags for them.

To smart tag video assets, or folders (including subfolders) of assets that already exist in assets repository, follow these steps:

Once the process completes, navigate to the Properties page of any video asset within the folder. The automatically added tags are seen in Smart Tags section in Basic tab. These applied smart tags are sorted in descending order of confidence score.

Search for tagged videos search-smart-tagged-videos

To search for the video assets based on the auto generated smart tags, use Omnisearch:

The search results display the video assets based on the tag you specified.

Your search results are a combination of video assets with searched keywords in the metadata and the video assets that are smart tagged with the searched keywords. However, the search results that match all search terms in metadata fields are displayed first, followed by the search results that match any of the search terms in the smart tags. For more information, see Understand Experience Manager search results with smart tags.

Moderate video smart tags moderate-video-smart-tags

Adobe Experience Manager lets you curate the smart tags to:

To know more about how to moderate the smart tags for assets, see Manage smart tags.

Opt out of video smart tagging opt-out-video-smart-tagging

As the automated tagging of videos runs in parallel to other asset processing tasks like thumbnail creation and metadata extraction, it can be time consuming. To expedite the asset processing you can opt out of video smart tagging on upload at folder level.

To opt out of automated video smart tags generation for assets uploaded to specific folder:

Confidence score confidence-score-video-tag

Adobe Experience Manager applies a minimum confidence threshold for object and action smart tags to avoid having too many tags for each video asset, which slows down indexing. Your asset search results are ranked based on the confidence scores, which generally improve search results beyond what an inspection of the assigned tags of any video asset suggests. Inaccurate tags often have low confidence scores so they seldom appear at the top of the Smart Tags list for assets.

The default threshold for action and object tags in Adobe Experience Manager is 0.7 (should be value between 0 and 1). If some video assets are not tagged by a specific tag, then it indicates that the algorithm is less than 70% confident in the predicted tags. The default threshold might not always be optimal for all the users. You can, therefore, change the confidence score value in OSGI configuration.

To add the confidence score OSGI configuration to the project deployed to Adobe Experience Manager as a Cloud Service through Cloud Manager:

{
  "minVideoActionConfidenceScore":0.5,
  "minVideoObjectConfidenceScore":0.5,
}

Limitations video-smart-tagging-limitations

See also