Video Tagging

Overview

Video tagging is one of intelligence tasks in the Filestack platform. It allows you to detect general features in your videos.

This task is only available at Filestack Workflows. Read more about Workflows here.

By default, Video tags and Video SFW tasks support videos that are up to 10 minutes long. Please contact our Sales Team if you are interested in increasing this limit.

Workflows Task Configuration

Video Tagging task is available in the Intelligence tasks category in Filestack Workflows. To learn more about how to use the Workflows UI for configuring tasks (and the logic between them), visit Creating Workflows Tutorial.

Workflows Parameters


Task Name	string	Unique name of the task. It will be included in the webhook response.
Details	boolean false	Indicates if a detailed response should be returned.
As file	boolean false	Indicates if response should be returned as a file.

Webhook

Here is an example of a webhook that you will receive once your video is processed. details key is only present when Details parameter set to true:

{
    "id": 88085450,
    "action": "fs.workflow",
    "timestamp": 1570438682,
    "text": {
        "workflow": "1401680c-1234-81b4-bae3-04166d1a96ee",
        "jobid": "2c22748c-ce65-4979-8071-530b467e4d8b",
        "createdAt": "2019-10-07T08:50:15.001357914Z",
        "updatedAt": "2019-10-07T08:57:42.180333126Z",
        "sources": [
            "Yq31nO8kbj6ijRJGvTE"
        ],
        "results": {
            "video_tags_task": {
                "data": {
                    "tags": {
                        "user": null,
                        "auto": {
                            "electrical device": 94,
                            "face": 88,
                            "head": 51,
                            "human": 99,
                            "lecture": 51,
                            "man": 58,
                            "microphone": 94
                        }
                    },
                    "details": {
                        "tags": [
                            {
                                "confidence": 81,
                                "start_offset": 0,
                                "end_offset": 4571,
                                "name": "face"
                            },
                            {
                                "confidence": 92,
                                "start_offset": 1220,
                                "end_offset": 5234,
                                "name": "face"
                            }
                        ]
                    }
                }
            }
        },
        "status": "Finished",
        "ttl": 172800
    }
}

Because the response can get large for longer videos, you can receive video processing results as a file by setting As file parameter to true. In this case, webhook will look like this:

{
    "id": 88085450,
    "action": "fs.workflow",
    "timestamp": 1570438682,
    "text": {
        "workflow": "1401680c-1234-81b4-bae3-04166d1a96ee",
        "jobid": "2c22748c-ce65-4979-8071-530b467e4d8b",
        "createdAt": "2019-10-07T08:50:15.001357914Z",
        "updatedAt": "2019-10-07T08:57:42.180333126Z",
        "sources": [
            "Yq31nO8kbj6ijRJGvTE"
        ],
        "results": {
            "video_tags_task": {
                "url": "https://cdn.filestackcontent.com/YOUR_APIKEY/wf://1401680c-1234-81b4-bae3-04166d1a96ee/2c22748c-ce65-4979-8071-530b467e4d8b/acf95aff157ab3c950f1f71a682b68f5",
                "mimetype": "application/json",
                "size": 8978
            }
        },
        "status": "Finished",
        "ttl": 172800
    }
}

Please visit the webhooks documentation page to learn more.

Logic

Video Tagging can be used in Workflow logic only when it was configured with As File parameter set to false

Video Tagging task returns following response:

{
    "data": {
        "tags": {
            "auto": {
                "dog": 98,
                "sky": 82
            },
            "user": null
        },
        "details": {}
    }
}

Logic Parameters


data	dictionary	Indicates the response from this service.
tags	dictionary	Contains "auto" and "user" tags.
auto	dictionary	Includes all of detected general features from the image with their corresponding confidence numbers.
details	dictionary	This element is not useful in workflows logic.

Based on the task’s response you can build logic that tells the workflow how dependent tasks should be executed. For example, if you would like to run another task if a specific key was detected you can use the following rule:

tags.auto kex "sky"

In Workflows UI this command would look like in the example below:

Visit Creating Workflows Tutorial to learn how you can use Workflows UI to configure your tasks and logic between them.

Video Tagging Result

Below you can see how the response is structured. tags.auto key contains each tag that was detected in the video with its highest confidence.
More detailed information can be found under details.tags key. Each item contains a name, confidence and a start/end offset (time, in milliseconds from the start of the video, that the tag was detected).

{
    "tags": {
        "user": null,
        "auto": {
            "electrical device": 94,
            "face": 88,
            "head": 51,
            "human": 99,
            "lecture": 51,
            "man": 58,
            "microphone": 94
        },
    },
    "details": {
        "tags": [
            {
                "name": "human",
                "confidence": 93,
                "start_offset": 12379,
                "end_offset": 15382,
            },
            {
                "name": "microphone",
                "confidence": 89,
                "start_offset": 22134,
                "end_offset": 26233,
            }
        ]
    }
}