Uploading data

See how you can upload and manage your datasets on Mindkosh using the SDK

Before you attempt to upload data, make sure you've followed the steps outlined here to setup the SDK.

Uploading data of a single type

from mindkosh import Client

## Upload all image files found in the specified directory
client.upload_data(
    dataset_id=dataset_id,
    resources=['/example_images/'],
    tags=['penguin'] ## optional
)

## Or specify the files to be uploaded
client.upload_data(
    dataset_id=dataset_id,
    resources=[
        '/home/user/Downloads/test1.pcd',
        '/home/user/Downloads/test2.pcd'
    ],
    tags=['forest'] ## optional
)

If you would like to specify different tags for each file, you can use the ImageFile or PointCloudFile objects.

from mindkosh import Client, ImageFile

client.upload_imagefiles(
    dataset_id = dataset_id,
    imagefiles = [
        ImageFile(
            filepath='/path/to/image1',
            tags=["city1"]
        ),
        ImageFile(
            filepath='path/to/image2',
            tags=["city2"]
        )
    ]
)

It can take up-to 2 minutes for the uploaded files to be processed. Before you create a task with the uploaded data, make sure you've given the files some time to be processed. You can check the UI to see if all the files have been processed and are visible in the dataset.

Uploading lidar + camera data

To upload lidar point clouds + reference camera images, create point cloud file objects and specify the camera images as ImageFile objects. If available, calibration parameters can be specified as Extrinsic parameters (For Lidar to camera projection) and Intrinsic (For camera to image projection) parameters. If these parameters have been specified, Mindkosh can automatically project cuboid annotations over reference camera images.

1. intrinsic - Specified in [fx, fy, cx, cy] form.

2. extrinsic - Specified in a 4x4 transformation matrix that can transform a point in the point cloud reference frame to the camera reference frame.

3. camera_model - Mindkosh supports two camera models - PINHOLE which is the most widely used, and FISHEYE . When specifying parameters for Fisheye, the mirrorParameter also needs to be set.

4. Parameters are set separately for each camera and for each frame. Even if they stay the same across a scene.

5. device_id - A unique number that can be used to identify a camera. This can be any number as long as it is unique across the cameras, and stays the same across frames. For e.g. if you have a left and right camera, you can assign the device ID 1 and 2 to them, across all frames.

Mindkosh requires all files within a dataset to have unique names. This means that if your camera files have the same names (for e.g. the same timestamp.png for both left and right cameras), you will need to rename them by adding camera prefixes, so the names are unique.

from mindkosh import Client, PointCloudFile

pointcloudFrames = []
pcdfile1 = PointCloudFile(
    filepath = '/file/path1/pcd1.pcd',
    related_files = [
        ImageFile(
            filepath='/path/to/image1.png',
            tags=[],
            extra={
                ## Inrinsic camera parameter values specified in this order: 
                ### [fx, fy, cx, cy]
                "intrinsic": [255.520403, 449.883682, 250.828738, 255.237477],

                ## Extrinsic parameters, specifying the 4x4 projection matrix from lidar 
                ## to camera
                "extrinsic": [
                    [-0.735827, -0.65789, -0.0384775, 0],
                    [0.6567956, -0.70452916, 0.00140833, 0],
                    [-0.02255652, -0.0248216, 0.9993586, 0],
                    [0, 0, 0, 1]
                ],

                # Camera projection model - PINHOLE OR FISHEYE
                # For FISHEYE, the mirrorParameter must also be specified
                "cameraModel": "PINHOLE",
                ## Device ID must be unique for each camera.
                "device_id": 1
            }
        )
    ]
)

pointcloudFiles.append(pcdfile1)

client.upload_pointcloud_data(dataset_id=datasetId, pointcloudfiles=pointcloudFrames)