VCD - Video Content Description

Metadata specification format for the description of scenes and data sequences, such as videos or point clouds. It supports multi-camera and multi-sensor systems, spatio-temporal object annotation, and semantics with actions, events and relations between them.

Find more!

Automotive Industry

Ground truth description, ADAS and AD function validation. Road topology, dynamic objects and semantic scenarios

Security & Surveillance

Traffic descriptions, CCTV-related events, trajectories, re-identification of subjects, action and behaviour recognition

Biomedical Imaging

Pixel-wise and polygonal description of images, sequences and volumes

Your next application

Manufacturing, robotics, aerospace, traffic and smart city management, precise agriculture, etc

FEATURES

Annotation of spatio-temporal Objects with unlimited numeric, textual and binary formats

Scene annotations and multiple sensors (e.g. camera, lidar, etc) including calibration and synchronization metadata

Description of Events, Actions, Contexts and Relations for rich semantic description

Open format defined with JSON schemas to enable creating compliant applications and interfaces

Connection to ontologies to enable semantic reasoning using Graph databases and query languages like Cypher or SPARQL

Good storage/streaming trade-off with frame-level JSON messages

C++ and Python libraries for Online and batch processing modes

Image, binary data and matrices embedding capabilities

Converters available from popular languages and datasets, and easy-to-use API to create translators

VCD Elements

{"openlabel": {
"objects": {
"0": {"frame_intervals:
[(0,176),(200,1145)],
"name": "paola",
"object_data": {
"bbox": [{"name": "head",
"val": [10, 15, 30, 30]}],
"poly2d": [{"name": "shape",
"val":[23, 54, ...]}...}

Object

Spatio-temporal description of entities of any class (e.g. “Person”, “Car”), including Polygons, Bounding Boxes, Points, Images, etc.

{"openlabel": {
"actions": {
"0": {"frame_intervals: [(0,176)],
"name": "coding_1",
"type": "#Coding",
"ontology_uid": 0},
"ontologies": {"0": "http://www.domain.org/ont/actions"}}

Action

Temporal description of actions, with semantic content: “Overtaking”, “Pedestrian crossing”.

{"openlabel": {
"events": {
"0": {"frame_intervals: [(0,0)], "name": "coding_1",
"type": "#StartsTalking", "ontology_uid": 0},
"ontologies": {"0": "http://www.domain.org/ont/actions"}}

Event

Point in time that triggers Actions or Contexts

{"openlabel": {
"context": {
"0":  {"name": "sunny",
"type": "#Sunny",
"ontology_uid": 1}},
"ontologies":{"1": "http://www.domain.org/ont/concepts"}}

Context

Atemporal description of contextual information of the scene: “Urban”, “DayTime”, “Raining”

{"openlabel": [...], {
"relations": {
"0": {"rdf_objects": [{
"type": "action", "uid":0}], "rdf_subjects": [{"type": "event", "uid": 0}], "type": "#Starts"}}}

Relation

Connection between Objects, Actions, Events and Contexts, using RDF triplets

describe a scene
why VCD TO structurE YOUR metadata?

Annotations need to be created, transmitted, edited, stored, searched and consumed

  • Generate labels for content at the edge (e.g. in an instrumented car or near the CCTV camera)‍

  • Stream labels through IoT and 5G networks for sharing object-level metadata

  • Massive storage of annotations for querying scenes, scenarios, maneuvers

  • Interfacing between Docker containers running AI algorithms (detection, tracking, recognition)

  • Edition of labels through Web Applications

  • Interoperability with simulation engines to automate test-scenes generation

successful use cases

Surveillance - CCTV

Action recognition, individual re-identification for security applications and large critical infrastructures (e.g. airports).

Multiple object tracking

Description of objects with time-consistent IDs. Spatio-temporal information can contain 2D and 3D entities, related to specific frames for multiple-sensors.

Biometrics

Description of facial and physiological parameters from drivers, computed real-time, including headpose, blink patterns and gaze vectors.

Semantic segmentation

Pixel-level information about classes and instances, for deep learning training and scene understanding.

Biomedical imaging

Identification of shapes of elements of interest in images, sequences and volumes.

Lanes and objects labeling for ground truth generation

Precise annotation of multiple shapes in 3D point clouds, with properties and attributes for different 2D and 3D primitives (polygons, cuboids, etc.).

Lane modeling from camera systems

Identification of position of ego-vehicle within its lane, for precise localisation.

Driver Monitoring

Action recognition of drivers to identify driver awareness, distraction and driving behaviour.
See project

Ontologies

Link to ontology concepts to provide linked-data descriptions of concepts.
vcd has been used in
What they say about vcd

Suzanne Little

Insight Center for Data Analytics

Donec tincidunt nibh sed nibh ultrices finibus. Fusce vel scelerisque mi, vitae elementum orci. Morbi vitae auctor ligula. Duis lacinia at mauris nec tempus. Fusce finibus mollis mauris, nec convallis lectus fringilla sit amet. Phasellus blandit tortor ac diam dictum, ut ultrices risus ultricies. Ut vitae imperdiet justo, sit amet congue libero.

Profile

Erwin Vermassen

ERTICO

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum.Praesent ac sapien lobortis, rhoncus felis eu, tristique tellus. Integer molestie.

LinkedIn

Joachim Kreikemeier

Valeo

The VCD language enables the handling of complex structures of metadata. This is key for multi-sensor solutions in the automotive sector and autonomous driving. Especially to manage the representation of semantic dependencies between objects in different environments.

LinkedIn

Suzanne Little

Insight Center for Data Analytics

Donec tincidunt nibh sed nibh ultrices finibus. Fusce vel scelerisque mi, vitae elementum orci. Morbi vitae auctor ligula. Duis lacinia at mauris nec tempus. Fusit amet congue libero.

LinkedIn

Suzanne Little

Insight Center for Data Analytics

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum.Praesent ac sapien lobortis, rhoncus felis eu, tristique tellus. Integer molestie.

LinkedIn

Suzanne Little

Insight Center for Data Analytics

Nullam vel dictum tortor. Cras sit amet lectus non eros sagittis condimentum in id nulla. Donec in laoreet orci. Aenean condimentum dolor nec hendrerit semper. Praesent ac sapien lobortis, rhoncus felis eu, tristique tellus. Integer molestie laoreet commodo.

LinkedIn
Documentation
VCD timeline

Contact

Comments, questions and requests are welcome :)

Thank you! Your submission has been received!
Oops! Something went wrong. Try again, please.

Citation

If you wish to use VCD, add our cite to your bibliography:

M. Nieto, O. Senderos, and O. Otaegui, “Boosting AI applications: Labeling format for complex datasets,” SoftwareX, 2021, p. 100653, vol. 13 (https://doi.org/10.1016/j.softx.2020.100653).