Introduction to pralin/compose
WARNING auKsys/5 has been released as stable, but this documentation has not been fully updated with changes since auKsys/4. In case of questions or mistakes, you can use the support project.
pralin (processing algorithms interfaces) is a C++ library that provides interfaces between algorithms used for processing data.
It has support for different processing libraries, in computer vision (such as opencv), lidar (such as pcl), deep learning (such as pytorch).
pralin
is fully integrated in kDB and is the recommended solution for processing data.
pralin
comes with an API for C++, Python and Ruby, and also, with a composition (pralin/compose
) language which allow to specify a set of processing operations in a YAML format.
pralin/compose
is the recommended approach for integration with kDB
, we will therefor only cover the part of the API for running composition in this documentation.
Running examples from this tutorial can be found at the getting started examples.
Introduction to pralin/compose
The pralin/compose
language is based on a YAML, and it defines a tree of processing operations, with a set of basic operations (sequence
, parallel
), control operations (repeat_while
, conditional
, …) and processing operations.
A composition is defined as a set of inputs
, outputs
, parameters
and process
. In the YAML file, they are defined using a map.
The full definition is provided in (Compose Language)[https://gitlab.com/cyloncore/pralin/-/blob/dev/2.0/docs/pages/ComposeLanguage.md?ref_type=heads].
For the purpose of this tutorial, to test the pralin/compose
API, we will use a simple composition that computes the addition of two numbers:
compose:
inputs:
- a
- b
outputs:
c: add[0]
process:
- pralin/arithmetic/addition:
id: add
inputs: [a, b]
It takes two inputs, called a
and b
, it will output the result of the addition add[0]
, it uses the pralin/arithmetic/addition
operation, that takes two inputs [a, b]
Run pralin/compose
-
#include <clog_print> #include <pralin/algorithms_registry> #include <pralin/compose/computation_graph> // Load the algorithms definitions pralin::algorithms_registry::load_default_definitions(); // Create a new computation graph pralin::compose::computation_graph cg; // Load from a string cg.load_from_string(R"V0G0N( compose: inputs: - a - b outputs: c: add[0] process: - pralin/arithmetic/addition: id: add inputs: [a, b] )V0G0N"); // Initialize an output value pralin::values_ptr_vector outputs_vector = cg.create_output_values(); // Run the computation with a=2 and b=3 cg.process({2, 3}, outputs_vector); // Should show 5 clog_print("Result is {}", outputs_vector[0].dereference());
-
import pralin.compose # Load the algorithms definitions pralin.AlgorithmsRegistry.load_default_definitions() # Create a new computation graph cg = pralin.compose.ComputationGraph() # Load from a string cg.load_from_string(""" compose: inputs: - a - b outputs: c: add[0] process: - pralin/arithmetic/addition: id: add inputs: [a, b] """) # Create an output value sc = pralin.Output() # Run the computation with a=2 and b=3 cg.process(2, 3, sc) # Should show 5 print(f"Result is {sc.value()}")
-
require 'pralin/compose' # Load the algorithms definitions Pralin::AlgorithmsRegistry.load_default_definitions() # Create a new computation graph cg = Pralin::Compose::ComputationGraph.new # Load from a string cg.load_from_string <<-COMPOSITION compose: inputs: - a - b outputs: c: add[0] process: - pralin/arithmetic/addition: id: add inputs: [a, b] COMPOSITION # Create an output value sc = Pralin::Output.new # Run the computation with a=2 and b=3 cg.process 2, 3, sc # Should show 5 puts "Result is #{sc.value}"
-
This composition has an input
prompt
and one outputllm[0]
:compose: inputs: [prompt] outputs: ["llm[0]"] process: - ggml/llama/inference: id: llm inputs: - prompt parameters: filename: ggml-model-q4_0.gguf source: https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v0.2-GGUF/resolve/main/ggml-model-q4_0.gguf
This can be run with the
pralin compose
command or as part of the computation server. -
# A GUI for editing and running compositions: pralin studio # Run from the command line, assuming the previously given composition is stored in a # `addition.yaml` file pralin compose --inputs "{a: 2, b: 3}" addition.yaml # Run from the command line, copy/paste the composition from above. pralin compose --inputs "{a: 2, b: 3}" -- # Run from the command line, copy/paste the composition from above, and enter the inputs # when requested pralin compose --
As an alternative, to load composition saved in file, all APIs include a load_from_file
function, which takes a filename as argument.
Documentation
pralin
has a built-in tool for accessing the documentation of available algorithms:
# List of all modules
pralin documentation
# Show the documentation of a single module, and the list of algorithms
pralin documentation module_name
# For example
pralin documentation kdb
# Show the documentation of a specific algorithm
pralin documentation module_name algorithm_name
# For example
pralin documentation kdb create_connection_handle
Next
This tutorial covers the basics of using pralin/compose
, from defining a simple composition to how to run composition.
- ROS Computation Server: provides the possibility to run
pralin/compose
as a service.
The following tutorials cover the use of pralin/compose
in connection with kDB
for processing sensor data:
- Process images datasets this tutorial shows how to process an existing image datasets
- Process Lidar Datasets this tutorial covers how to process lidar data, to generate a point cloud and downsample it.