This page is part of the documentation of auKsys/5

WARNING auKsys/5 has been released as stable, but this documentation has not been fully updated with changes since auKsys/4. In case of questions or mistakes, you can use the support project.

pralin (processing algorithms interfaces) is a C++ library that provides interfaces between algorithms used for processing data. It has support for different processing libraries, in computer vision (such as opencv), lidar (such as pcl), deep learning (such as pytorch). pralin is fully integrated in kDB and is the recommended solution for processing data. pralin comes with an API for C++, Python and Ruby, and also, with a composition (pralin/compose) language which allow to specify a set of processing operations in a YAML format. pralin/compose is the recommended approach for integration with kDB, we will therefor only cover the part of the API for running composition in this documentation.

Running examples from this tutorial can be found at the getting started examples.

Introduction to pralin/compose

The pralin/compose language is based on a YAML, and it defines a tree of processing operations, with a set of basic operations (sequence, parallel), control operations (repeat_while, conditional, …) and processing operations. A composition is defined as a set of inputs, outputs, parameters and process. In the YAML file, they are defined using a map. The full definition is provided in (Compose Language)[https://gitlab.com/cyloncore/pralin/-/blob/dev/2.0/docs/pages/ComposeLanguage.md?ref_type=heads].

For the purpose of this tutorial, to test the pralin/compose API, we will use a simple composition that computes the addition of two numbers:

compose:
  inputs:
    - a
    - b
  outputs:
    c: add[0]
  process:
    - pralin/arithmetic/addition:
        id: add
        inputs: [a, b]

It takes two inputs, called a and b, it will output the result of the addition add[0], it uses the pralin/arithmetic/addition operation, that takes two inputs [a, b]

Run pralin/compose

  • #include <clog_print>
    #include <pralin/algorithms_registry>
    #include <pralin/compose/computation_graph>
    
    // Load the algorithms definitions
    pralin::algorithms_registry::load_default_definitions();
    
    // Create a new computation graph
    pralin::compose::computation_graph cg;
    
    // Load from a string
    cg.load_from_string(R"V0G0N(
    compose:
      inputs:
        - a
        - b
      outputs:
        c: add[0]
      process:
        - pralin/arithmetic/addition:
            id: add
            inputs: [a, b]
    )V0G0N");
    
    // Initialize an output value
    pralin::values_ptr_vector outputs_vector = cg.create_output_values();
    
    // Run the computation with a=2 and b=3
    cg.process({2, 3}, outputs_vector);
    
    // Should show 5
    clog_print("Result is {}", outputs_vector[0].dereference());
    
  • import pralin.compose
    
    # Load the algorithms definitions
    pralin.AlgorithmsRegistry.load_default_definitions()
    
    # Create a new computation graph
    cg = pralin.compose.ComputationGraph()
    
    # Load from a string
    cg.load_from_string("""
    compose:
      inputs:
        - a
        - b
      outputs:
        c: add[0]
      process:
        - pralin/arithmetic/addition:
            id: add
            inputs: [a, b]
    """)
    
    # Create an output value
    sc = pralin.Output()
    
    # Run the computation with a=2 and b=3
    cg.process(2, 3, sc)
    
    # Should show 5
    print(f"Result is {sc.value()}")
    
    
  • require 'pralin/compose'
    
    # Load the algorithms definitions
    Pralin::AlgorithmsRegistry.load_default_definitions()
    
    # Create a new computation graph
    cg = Pralin::Compose::ComputationGraph.new
    
    # Load from a string
    cg.load_from_string <<-COMPOSITION
    compose:
      inputs:
        - a
        - b
      outputs:
        c: add[0]
      process:
        - pralin/arithmetic/addition:
            id: add
            inputs: [a, b]
    COMPOSITION
    
    # Create an output value
    sc = Pralin::Output.new
    
    # Run the computation with a=2 and b=3
    cg.process 2, 3, sc
    
    # Should show 5
    puts "Result is #{sc.value}"
    
  • This composition has an input prompt and one output llm[0]:

    compose:
      inputs: [prompt]
      outputs: ["llm[0]"]
      process:
        - ggml/llama/inference:
            id: llm
            inputs:
              - prompt
            parameters:
              filename: ggml-model-q4_0.gguf
              source: https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v0.2-GGUF/resolve/main/ggml-model-q4_0.gguf
    

    This can be run with the pralin compose command or as part of the computation server.

  • # A GUI for editing and running compositions:
    pralin studio
    
    # Run from the command line, assuming the previously given composition is stored in a
    # `addition.yaml` file
    pralin compose --inputs "{a: 2, b: 3}" addition.yaml
    
    # Run from the command line, copy/paste the composition from above.
    pralin compose --inputs "{a: 2, b: 3}" --
    
    # Run from the command line, copy/paste the composition from above, and enter the inputs
    # when requested
    pralin compose --
    

As an alternative, to load composition saved in file, all APIs include a load_from_file function, which takes a filename as argument.

Documentation

pralin has a built-in tool for accessing the documentation of available algorithms:

# List of all modules
pralin documentation

# Show the documentation of a single module, and the list of algorithms
pralin documentation module_name

# For example
pralin documentation kdb 

# Show the documentation of a specific algorithm
pralin documentation module_name algorithm_name

# For example
pralin documentation kdb create_connection_handle

Next

This tutorial covers the basics of using pralin/compose, from defining a simple composition to how to run composition.

The following tutorials cover the use of pralin/compose in connection with kDB for processing sensor data: