pygridgain

GridGain Community Edition thin (binary protocol) client, written in Python 3.

GridGain is a memory-centric distributed database, caching, and processing platform for transactional, analytical, and streaming workloads delivering in-memory speeds at petabyte scale.

The GridGain binary client protocol provides user applications the ability to communicate with an existing GridGain cluster without starting a full-fledged GridGain node. An application can connect to the cluster through a raw TCP socket.

Prerequisites

• Python 3.9 or above (3.9, 3.10, 3.11, 3.12 and 3.13 are tested),
• Access to GridGain node, local or remote. The current thin client version was tested on GridGain CE 8.8 and 8.9 (binary client protocol 1.7.1).

Installation

From PyPI

This is a recommended way for users. If you only want to use the pygridgain module in your project, do:

$ pip install "pygridgain<9"

Installs GridGain 8.x client. <9 bound is required to avoid GridGain 9.x client.

From sources

This way is more suitable for developers or if you install client from zip archive.

1. Download and/or unzip GridGain Python client sources to gridgain_client_path
2. Go to gridgain_client_path folder
3. Execute pip install -e .

$ cd <gridgain_client_path>
$ pip install -e .

This will install the repository version of pygridgain into your environment in so-called “develop” or “editable” mode. You may read more about editable installs in the pip manual.

Then run through the contents of requirements folder to install the additional requirements into your working Python environment using

$ pip install -r requirements/<your task>.txt

For development, it is recommended to install tests requirements

$ pip install -r requirements/tests.txt

For checking codestyle run:

$ flake8

The package metadata lives in pyproject.toml; setup.py only builds the optional C extension. You may also want to consult the Python packaging guide.

optional C extension

There is an optional C extension to speedup some computational intensive tasks. If it's compilation fails (missing compiler or CPython headers), pygridgain will be installed without this module.

• On Linux or MacOS X only C compiler is required (gcc or clang). It compiles during standard setup process.

• For building universal wheels (binary packages) for Linux, just invoke script ./scripts/create_distr.sh.

  NB! Docker is required.

• On Windows MSVC 14.x required, and it should be in path, also python versions 3.9, 3.10, 3.11, 3.12 and 3.13 both for x86 and x86-64 should be installed. You can disable some of these versions but you'd need to edit script for that.

• For building wheels for Windows, invoke script .\scripts\BuildWheels.ps1 using PowerShell. Just make sure that your execution policy allows execution of scripts in your environment.

  Ready wheels for x86 and x86-64 for different python versions (3.9, 3.10, 3.11, 3.12 and 3.13) will be located in distr directory.

• ./scripts/create_distr.sh also builds the source distribution, as a .tar.gz and a .zip, into the same distr directory.

Updating from older version

To upgrade an existing package, use the following command:

pip install --upgrade "pygridgain<9"

To install the latest version of a package:

pip install "pygridgain<9"

To install a specific version:

pip install pygridgain==1.7.0

Documentation

The package documentation is available at RTD for your convenience.

If you want to build the documentation from source, do the developer installation as described above, then run the following commands from your virtualenv environment:

$ pip install -r requirements/docs.txt
$ cd docs
$ make html

Then open docs/generated/html/index.html in your browser.

If you feel that old version is stuck, do

$ make clean
$ sphinx-apidoc -feM -o source/ ../ ../setup.py
$ make html

And that should be it.

Examples

Some examples of using pygridgain are provided in examples folder. They are extensively commented in the “Examples of usage” section of the documentation.

This code implies that it is run in the environment with pygridgain package installed, and GridGain node is running on localhost:10800, unless otherwise noted.

There is also a possibility to run examples alone with tests. For the explanation of testing, look up the Testing section.

Testing

NB! It is recommended installing pygridgain in development mode. Refer to this section for instructions.

Do not forget to install test requirements:

$ pip install -r requirements/install.txt -r requirements/tests.txt

Also, you'll need to have a binary release of GridGain with log4j2 enabled and to set IGNITE_HOME environment variable:

$ cd <gridgain_binary_release>
$ export IGNITE_HOME=$(pwd)
$ cp -r $IGNITE_HOME/libs/optional/ignite-log4j2 $IGNITE_HOME/libs/

Run basic tests

$ pytest

Run with examples

$ pytest --examples 

The --examples option runs the examples as one test. In this test assertion fails if any of the examples' processes ends with non-zero exit code. If you wish to run only the examples, supply also the name of the test function to the pytest launcher:

$ pytest --examples tests/test_examples.py::test_examples

Examples are not parameterized for the sake of simplicity. They always run with default parameters (host and port) regardless of any other pytest option.

Since failover, SSL and authentication examples are meant to be controlled by user or depend on special configuration of the GridGain cluster, they can not be automated.

Using tox

For automate running tests against different python version, it is recommended to use tox

$ pip install tox
$ tox

Licensing

This is a free software, brought to you on terms of the GridGain Community Edition License.