Creating a Basic Capsule¶

Introduction¶

In this tutorial, we will walk you through the creation of a basic capsule. If you get stuck along the way or simply want to view the end-result of the tutorial, you can find the completed capsule on our GitHub repository.

Before we start, we highly recommend you to read the OpenVisionCapsules Documentation. It will give you a bit of background information about capsules.

Set Up Environment¶

To develop your own capsule, you will need to install vcap and vcap_utils, which are a set of Python libraries for encapsulating machine learning and computer vision algorithms for intelligent video analytics. They can be found on Github here.

pip3 install vcap vcap_utils

You might also want to download a few of our open-source capsules:

git clone https://github.com/aotuai/capsule_zoo.git

Creating a Basic Capsule¶

In this tutorial, we will create a fake capsule. It isn't going to perform any inference, but instead return some fake values. The purpose of this example is to help you to understand our capsule system better.

Directory structure¶

First, let's create a folder called detector_bounding_box_fake under the capsules directory under where your docker-compose file is located. Also create a meta.py and a capsule.py under this directory. The resulting structure will look like:

your_working_directory
├── docker-compose.yml
└── capsules
    └── detector_bounding_box_fake
        ├── meta.conf
        └── capsule.py

For more information about the structure, you can check the documentation here.

Capsule Metadata¶

The meta.conf file provides basic information about the capsule to BrainFrame before the rest of the capsule is loaded. In our meta.conf, we are going to define the version of the OpenVisionCapsules SDK that this capsule will be compatible with.

[about]
api_compatibility_version = 0.3

This number should be the same as the Major.Minor version from

pip3 show vcap | grep Version

Capsule¶

In capsule.py, we define a class called Capsule, which will define the actual behavior of the capsule. The Capsule class provides metadata that allows BrainFrame to understand the capabilities of the capsule and how it can be used, and must inherit from BaseCapsule. For more information about the BaseCapsule class, see the documentation here.

We'll import the dependencies from vcap first:

from vcap import BaseCapsule, NodeDescription, BaseBackend, DetectionNode

Then we'll define the Capsule class as a sub-class of BaseCapsule.

# Define the Capsule class
class Capsule(BaseCapsule):
    # Metadata of this capsule
    name = "detector_bounding_box_fake"
    description = "A fake detector that outputs a single bounding box"
    version = 1
    # Define the input type. As this is an object detector, and does not require
    # any input from other capsules, the input type will be a NodeDescription
    # with size=NONE
    input_type = NodeDescription(size=NodeDescription.Size.NONE)
    # Define the output type. In this case we are going to return a list of
    # bounding boxes, so the output type will be size=ALL
    output_type = NodeDescription(
        size=NodeDescription.Size.ALL,
        detections=["fake_box"],
    )
    # Define the backend. In this example, we are going to use a fake Backend,
    # defined below
    backend_loader = lambda capsule_files, device: Backend(
        capsule_files=capsule_files, device=device)
    options = {}

Backend¶

Now let's create a Backend class. The Backend class defines how the underlying algorithm is initialized and used. For more information about Backend classes, please refer to the OpenVisionCapsules Documentation.

# Define the Backend Class
class Backend(BaseBackend):
    # Since this is a fake Backend, we are not going to do any fancy stuff in
    # the constructor.
    def __init__(self, capsule_files, device):
        print("Loading onto device:", device)
        super().__init__()

    # In a real capsule, this function will be performing inference or running
    # algorithms. For this tutorial, we are just going to return a single, fake
    # bounding box.
    def process_frame(self, frame, detection_node: None, options, state):
        return [
            DetectionNode(
                name="fake_box",
                coords=[[10, 10], [100, 10], [100, 100], [10, 100]]
            )
        ]

    # Batch process can be used to improve the performance, we will skip it in
    # this example.
    def batch_predict(self, input_data_list):
        pass

    # This function can be implemented to perform clean-up. We can't skip it
    # for this tutorial
    def close(self) -> None:
        pass

The fake capsule is now complete. If you restart your BrainFrame server, you will be able to see it loaded.

If you load a stream, you will be able to see the inference results.