Query Images
WARNING auKsys/5 has been released as stable, but this documentation has not been fully updated with changes since auKsys/4. In case of questions or mistakes, you can use the support project.
In this tutorial, we are going to look how to query for an image at a given location, convert it to an OpenCV image and display it.
We will require a dataset store with the kDBSensing
extension (this will also enable the kDBDatasets
extension).
It can be started with the following command:
kdb store --path tuto_kdb_store --extension kDBSensing
Or it can be started programmatically, refer to the Getting Started tutorial for more information.
Running examples from this tutorial can be found at the query images examples.
Query for Image
We will use the dataset of images acquired in the water near the Gränsö castle. It can be downloaded and uncompress with the following command.
curl -o images_sea.kdb_dataset.xz "https://gitlab.liu.se/lks/tutorials_data/-/raw/main/images_sea.kdb_dataset.xz?ref_type=heads&inline=false"
xz -d images_sea.kdb_dataset.xz
The dataset can be imported into the database using the following command:
kdb datasets import --path tuto_kdb_store --filename images_sea.kdb_dataset
To query for an image at a given location, we will use a SQL query. kDB uses three tables for storing images:
cameraframes
contains the image data and associated metadataagentpositions
contains the agent positionsframetransformations
contains the frame transformations used by the platform
For this tutorial we will ignore the frame transformations, and assume that we are interested in the 5 images acquired near a given location, we will use 16.681958, 57.760420
in this example:
SELECT cf.data, cf.width, cf.height, cf.encoding, ap.longitude, ap.latitude FROM cameraframes cf
JOIN agentpositions AS ap
ON cf.timestamp >= ap.timestamp AND cf.timestamp < ap.nexttimestamp
ORDER BY ST_Distance(ST_SetSRID(ST_MakePoint(16.681958, 57.760420),4326), ST_SetSRID(ST_MakePoint(ap.longitude, ap.latitude),4326))
LIMIT 5
This will return as result the rawdata of the images (cf.data
), the width (cf.width
), the height (cv.height
), its encoding (cf.encoding
, for instance rgb8
) and the longitude (ap.longitude
) and latitude (ap.latitude
).
Display in OpenCV
The examples bellow show how to create a knowCore::Image
from the previous SQL Query, and how to convert that to a C++ cv::Mat
or a Python numpy.array
, and to display the resulting image in OpenCV.
-
#include <clog_print> #include <opencv2/opencv.hpp> #include <knowCore/Image.h> #include <knowCore/TypeDefinitions.h> #include <knowDBC/Query.h> #include <knowDBC/Result.h> #include <kDB/Repository/Connection.h> kDB::Repository::Connection connection = kDB::Repository::Connection::create("tuto_kdb_store", 1242); connection.connect(); knowDBC::Query query = connection.createSQLQuery(R"V0G0N(SELECT cf.data, cf.width, cf.height, cf.encoding, ap.longitude, ap.latitude FROM cameraframes cf JOIN agentpositions AS ap ON cf.timestamp >= ap.timestamp AND cf.timestamp < ap.nexttimestamp ORDER BY ST_Distance(ST_SetSRID(ST_MakePoint(16.681958, 57.760420),4326), ST_SetSRID(ST_MakePoint(ap.longitude, ap.latitude),4326)) LIMIT 5)V0G0N"); knowDBC::Result result = query.execute(); for(int i = 0; i < result.tuples(); ++i) { clog_print("At coordinates ({}, {}) image ({}x{}, {})", result.value(i, 4), result.value(i, 5), result.value(i, 1), result.value(i, 2), result.value(i, 3)); knowCore::Image img = knowCore::Image::fromRawData(result.value<QByteArray>(i,0).expectSuccess(), result.value<quint64>(i, 1).expectSuccess(), result.value<quint64>(i, 2).expectSuccess(), result.value<QString>(i, 3).expectSuccess()).expectSuccess(); cv::Mat cv_img(cv::Size(img.width(), img.height()), CV_8UC3, img.dataPtr(), cv::Mat::AUTO_STEP); cv::imshow("image", cv_img); cv::waitKey(0); }
-
from kDB.Repository import Connection import knowCore import numpy as np import cv2 connection = Connection.create("tuto_kdb_store") connection.connect() query = connection.createSQLQuery("""SELECT cf.data, cf.width, cf.height, cf.encoding, ap.longitude, ap.latitude FROM cameraframes cf JOIN agentpositions AS ap ON cf.timestamp >= ap.timestamp AND cf.timestamp < ap.nexttimestamp ORDER BY ST_Distance(ST_SetSRID(ST_MakePoint(16.681958, 57.760420),4326), ST_SetSRID(ST_MakePoint(ap.longitude, ap.latitude),4326)) LIMIT 5""") result = query.execute(); for i in range(0, result.tuples()): print(f"At coordinates ({result.value(i, 4)}, {result.value(i, 5)}) image ({result.value(i, 1)}x{result.value(i, 2)}, {result.value(i, 3)})") img = knowCore.Image.fromRawData(result.value(i,0), result.value(i, 1), result.value(i, 2), result.value(i, 3)) img_np = np.array(img, copy=True) cv2.imshow('image',img_np) cv2.waitKey(0)
The first image to be displayed is at coordinates (16.681914783967493, 57.76033842696137):