Package `sysbot`

SysBot - System Test Automation Library

MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

For complete documentation, see the README.md and CONTRIBUTING.md files included in this package directory.

SysBot

Overview
Installation
Quickstart
RobotFramework Usage
UnitTest Usage
Listener Usage
Polarion Integration
Additional Resources
License
Author

Overview

SysBot is a system test tool that provides a unified interface for connecting to and testing various systems through different protocols. Built with Robot Framework integration in mind, it offers a modular architecture that simplifies system automation and testing.

Key Features

Multi-protocol Support: SSH, HTTP, WinRM, Socket, and more
SSH Tunneling: Support for nested SSH tunnels with automatic management
Cross-platform: Support for Linux and Windows systems
Robot Framework Integration: Built-in support for Robot Framework automation with GLOBAL scope
Modular Architecture: Dynamic components loading and discovery (modules and plugins)
Connection Management: Robust session caching and lifecycle management
Secret Management: Secure storage and retrieval of sensitive data
Database Listeners: Store test results in SQLite, MySQL, PostgreSQL, or MongoDB
Polarion Integration: Generate Polarion-compatible xUnit reports for ALM/QA integration

Architecture

sysbot/
├── Sysbot.py           # Main SysBot class
├── connectors/         # Protocol-specific connectors
├── plugins/            # Plugins utilities (data, vault)
├── utils/
│   └── engine.py       # Engine class
└── modules/            # Modules

Installation

Prerequisites

Python 3.8 or higher
pip

Install from PyPI

pip install sysbot

Optional Dependencies

For specific features, you can install additional dependencies:

# Install with all database support
pip install sysbot[all_databases]

# Install with specific database support
pip install sysbot[mysql]        # MySQL support only
pip install sysbot[postgresql]   # PostgreSQL support only
pip install sysbot[mongodb]      # MongoDB support only

# Install with development dependency
pip install sysbot[dev]

Quickstart

Basic SSH Connection

import sysbot

bot = sysbot.Sysbot()

# Open an SSH session to a Linux system
bot.open_session(
    alias="my_linux_server",
    protocol="ssh",
    product="bash",
    host="192.168.1.100",
    port=22,
    login="username",
    password="password"
)

# Execute a command
result = bot.execute_command("my_linux_server", "ls -la")
print(result)

# Close all sessions
bot.close_all_sessions()

SSH Tunneling

# Configure nested SSH tunnels
tunnel_config = [
    {
        "ip": "192.168.1.1",
        "port": 22,
        "username": "user1",
        "password": "pass1"
    },
    {
        "ip": "192.168.2.1", 
        "port": 22,
        "username": "user2",
        "password": "pass2"
    }
]

# Open session through tunnels
bot.open_session(
    alias="tunneled_server",
    protocol="ssh", # or http / winrm / ect...
    product="bash",
    host="192.168.3.100",
    port=22,
    login="final_user",
    password="final_pass",
    tunnel_config=tunnel_config
)

Secret Management

SysBot provides a built-in secret management system for secure storage and retrieval of sensitive data like passwords, tokens, and configuration values. Secrets can be stored directly or loaded from external sources like files or HashiCorp Vault.

import sysbot

bot = sysbot.Sysbot()

# Using plugins with secret management
bot.plugins.data.csv("/path/to/file", key="my_secret")
secret_data = bot.get_secret("my_secret.0.name")

# Secret management without plugin
bot.add_secret("new_secret", "very_secret_value")
bot.get_secret("new_secret")
bot.remove_secret("new_secret")

# Using Vault plugin to dump HashiCorp Vault secrets
bot.plugins.vault.dump_engine(
    token="hvs.CAESIJ...",
    url="https://vault.example.com:8200",
    engine_name="secret",
    key="vault_secrets",
    verify_ssl=False  # Set to True for production with valid certificates
)
# Access Vault secrets using dot notation
db_url = bot.get_secret("vault_secrets.myapp/config.database_url")

Module System

# Import sysbot to access loaded components (all modules/plugins loaded by default)
import sysbot

bot = sysbot.Sysbot()

# Open an SSH session to a Linux system
bot.open_session(
    alias="my_linux_server",
    protocol="ssh",
    product="bash",
    host="192.168.1.100",
    port=22,
    login="username",
    password="password"
)

result = bot.linux.dnf.repolist("my_linux_server")

Session Management

# Close a specific session
bot.close_session("my_linux_server")

# Close all sessions (automatically handles tunnels)
bot.close_all_sessions()

Supported Protocols

SSH

Bash: Full support for bash via SSH
Powershell: Support for powershell via SSH (requires SSH server)

Local Execution

Bash: Execute bash/shell commands locally without SSH
Powershell: Execute PowerShell commands locally without SSH or WinRM

SysBot provides local execution connectors that allow running commands directly on the local machine without the overhead of SSH or WinRM connections. This is useful for: - Running commands on the local system during automation - Testing without remote systems - Avoiding connection overhead for local operations

Local Bash Execution:

import sysbot

bot = sysbot.Sysbot()

# Open a local bash session (no actual connection is made)
bot.open_session(
    alias="local_bash",
    protocol="local",
    product="bash",
    host="localhost",  # Required but not used
    port=0  # Required but not used
)

# Execute commands locally
result = bot.execute_command("local_bash", "ls -la")
print(result)

bot.close_session("local_bash")

Local PowerShell Execution:

import sysbot

bot = sysbot.Sysbot()

# Open a local PowerShell session (no actual connection is made)
bot.open_session(
    alias="local_ps",
    protocol="local",
    product="powershell",
    host="localhost",  # Required but not used
    port=0  # Required but not used
)

# Execute PowerShell commands locally
result = bot.execute_command("local_ps", "Get-Process | Select-Object -First 5")
print(result)

bot.close_session("local_ps")

HTTP/HTTPS

SysBot provides a generic HTTP/HTTPS connector with support for 9 authentication methods.

Supported Authentication Methods: 1. API Key (apikey) - API Key authentication via headers or query parameters 2. Basic Auth (basicauth) - Standard HTTP Basic Authentication 3. OAuth 1.0 (oauth1) - OAuth 1.0 authentication (RFC 5849) 4. OAuth 2.0 (oauth2) - OAuth 2.0 Bearer token authentication 5. JWT (jwt) - JSON Web Token authentication with automatic token generation 6. SAML (saml) - SAML assertion/token authentication 7. HMAC (hmac) - HMAC signature-based authentication 8. Certificate (certificate) - Client certificate authentication (mutual TLS) 9. OpenID Connect (openidconnect) - OpenID Connect authentication

Usage Examples:

Basic Authentication:

bot.open_session(
    alias="my_api",
    protocol="http",
    product="basicauth",
    host="api.example.com",
    port=443,
    login="username",
    password="password"
)

result = bot.execute_command("my_api", "/users", options={"method": "GET"})

API Key Authentication:

bot.open_session(
    alias="my_api",
    protocol="http",
    product="apikey",
    host="api.example.com",
    port=443,
    api_key="your-api-key-here",
    api_key_header="X-API-Key"  # Custom header name (optional)
)

result = bot.execute_command("my_api", "/data", options={"method": "GET"})

WinRM

Powershell: Native Windows Remote Management support

Socket

TCP: Native TCP socket with SSL if needed
UDP: Native UDP socket

RobotFramework Usage

SysBot is designed to work seamlessly with Robot Framework, providing powerful automation capabilities with a simple syntax.

Basic Robot Framework Test

*** Settings ***
Library        sysbot.Sysbot
Suite Setup       Call Components    plugins.data.yaml    /path/to/connexion.yml    key=connexion
Suite Teardown    Close All Sessions

*** Variables ***
${HOST}=       192.168.1.112
${PORT}=       22
${USER}=       sysbot
${PASSWORD}=   P@ssw0rd

*** Test Cases ***

Open Session without secret
    Open Session    target    ssh    bash    ${HOST}    ${PORT}   ${USER}    ${PASSWORD}
    Close All Sessions

Open Session with secret
    Open Session    target    ssh    bash    connexion.host    ${PORT}   connexion.username    connexion.password   is_secret=True
    Close All Sessions

Using Modules in Robot Framework

Modules can be loaded and used to perform specific operations on target systems:

*** Settings ***
Library        sysbot.Sysbot    linux.systemd    linux.dnf

*** Test Cases ***

Check System Service
    Open Session    server1    ssh    bash    ${HOST}    ${PORT}    ${USER}    ${PASSWORD}
    ${status}=    Linux Dnf Repolist    server1
    Log    ${status}
    Close All Sessions

Secret Management in Robot Framework

*** Settings ***
Library        sysbot.Sysbot

*** Test Cases ***

Using Secrets
    Add Secret    db_password    MySecretPassword
    ${password}=    Get Secret    db_password
    Log    Using password: ${password}
    Remove Secret    db_password

UnitTest Usage

SysBot can be used in Python unittest for system testing scenarios.

Module Testing with UnitTest

import unittest
import Sysbot

class TestLinuxModules(unittest.TestCase):

    @classmethod
    def setUpClass(cls):
        """Set up class fixtures."""
        cls.bot = sysbot.Sysbot("linux.systemd", "linux.dnf")
        cls.bot.open_session(
            alias="linux_server",
            protocol="ssh",
            product="bash",
            host="192.168.1.100",
            port=22,
            login="user",
            password="pass"
        )

    @classmethod
    def tearDownClass(cls):
        """Clean up class fixtures."""
        cls.bot.close_all_sessions()

    def test_systemd_service_status(self):
        """Test checking systemd service status."""
        result = self.bot.linux.systemd.status("linux_server", "sshd")
        self.assertIsNotNone(result)

    def test_dnf_repolist(self):
        """Test listing DNF repositories."""
        result = self.bot.linux.dnf.repolist("linux_server")
        self.assertIsNotNone(result)

if __name__ == '__main__':
    unittest.main()

Listener Usage

SysBot includes Robot Framework listener plugins that can store test results in various databases. Each database type has its own independent, self-contained listener. The listeners create a hierarchical structure: Campaign → Suite → Test Case → Keyword.

Available Listeners

SQLite: Lightweight file-based database, perfect for local testing
MySQL: Popular relational database for team environments
PostgreSQL: Enterprise-grade relational database
MongoDB: NoSQL document database for flexible schemas

Usage with Robot Framework

Each listener is used directly with its specific class:

# Store results in SQLite
robot --listener sysbot.utils.robot.listener.sqlite.Sqlite:results.db:MyCampaign your_tests/

# Store results in MySQL
robot --listener sysbot.utils.robot.listener.mysql.Mysql:mysql://user:pass@localhost/testdb:MyCampaign your_tests/

# Store results in PostgreSQL
robot --listener sysbot.utils.robot.listener.postgresql.Postgresql:postgresql://user:pass@localhost/testdb:MyCampaign your_tests/

# Store results in MongoDB
robot --listener sysbot.utils.robot.listener.mongodb.Mongodb:mongodb://localhost:27017/testdb:MyCampaign your_tests/

Listener Parameters

The listener accepts two parameters: 1. Database Connection: Connection string or path to database 2. Campaign Name: Name of the test campaign for organizing results

Data Structure

The listeners store test execution data in a hierarchical format:

Campaign: Top-level container for test executions
Suite: Test suite information
- Test Case: Individual test cases
- Keyword: Keywords executed within tests

Each level stores relevant metadata including: - Execution timestamps - Status (PASS/FAIL) - Error messages - Statistics

Installation Requirements

# Install with all database support
pip install sysbot[all_databases]

# Or install specific database support
pip install sysbot[mysql]        # MySQL support only
pip install sysbot[postgresql]   # PostgreSQL support only
pip install sysbot[mongodb]      # MongoDB support only

Polarion Integration

SysBot includes a Polarion plugin that enables integration with Siemens Polarion ALM/QA for test result management. The plugin provides:

xUnit Post-processor: Converts Robot Framework output to Polarion-compatible xUnit XML using rebot
Test Case Mapping: Links Robot Framework tests to Polarion test cases via tags
Custom Properties: Supports Polarion custom fields and metadata

Linking Tests to Polarion

Use tags in your Robot Framework tests to establish links with Polarion test cases:

*** Test Cases ***
Login Functionality Test
    [Documentation]    Validates user login with valid credentials
    [Tags]    polarion-id:TEST-001    polarion-title:Login Test    polarion-priority:High
    # Test steps...

User Management Test
    [Documentation]    Test user creation and deletion
    [Tags]    polarion-id:TEST-002    polarion-testEnvironment:Production
    # Test steps...

Tag Format

polarion-id:TEST-XXX - Links to Polarion test case ID (required for mapping)
polarion-title:Test Name - Sets Polarion test case title
polarion-{property}:{value} - Custom Polarion properties (e.g., polarion-priority:High, polarion-assignee:jdoe)

Generating Polarion-Compatible xUnit

Using Python API:

from sysbot.utils.robot.polarion import Polarion

polarion = Polarion()
polarion.generate_xunit(
    output_xml='output.xml',
    xunit_file='polarion_results.xml',
    project_id='MYPROJECT',
    test_run_id='RUN-001',
    custom_properties={'environment': 'test', 'version': '1.0'}
)

Using Command Line:

# Run Robot Framework tests
robot --outputdir results your_tests/

# Generate Polarion xUnit using Python
python -c "from sysbot.utils.robot.polarion import Polarion; \
    polarion = Polarion(); \
    polarion.generate_xunit('results/output.xml', 'results/polarion.xml', \
    project_id='PROJ', test_run_id='RUN-001')"

Importing into Polarion

Once you have the Polarion-compatible xUnit file:

Manual Import: Use Polarion's UI to import the xUnit file
Scheduled Import: Configure Polarion's scheduled xUnit importer
API Import: Use tools like dump2polarion or Polarion's REST API

Generated xUnit Content

The generated xUnit file includes: - Test case IDs for proper mapping to existing Polarion test cases - Test execution results (pass/fail/error) - Execution time and timestamps - Custom properties for filtering and reporting - Project and test run associations

Additional Resources

Documentation

SysBot includes comprehensive Google-style docstrings for all modules, classes, and methods.

Online Documentation

The complete documentation is available online at https://joreci2.github.io/sysbot/

Viewing Documentation Locally with pdoc3

Install pdoc3 as a development dependency:

pip install pdoc3

Generate and serve interactive HTML documentation:

# Start a local documentation server (recommended)
pdoc3 --http localhost:8080 sysbot

# Or generate static HTML files
pdoc3 --html --output-dir docs sysbot

Then open your browser and navigate to http://localhost:8080/sysbot to browse the complete API documentation.

The documentation includes: - Module-level docstrings: Purpose and overview of each module - Class documentation: Detailed class descriptions and initialization parameters - Method documentation: Comprehensive Args, Returns, and Raises sections - Package structure: Hierarchical organization of all components

Error Handling

SysBot provides comprehensive error handling:

Connection Errors: Detailed error messages for connection failures
Tunnel Management: Automatic cleanup on tunnel failures
Session Validation: Verification of session validity before operations
Module Errors: Clear error messages for module and function calls

License

This project is licensed under the MIT License - see the LICENSE file for details.

Author

Thibault SCIRE - GitHub

Sub-modules

sysbot.connectors: Connectors Package …
sysbot.modules: Modules Package …
sysbot.plugins: Plugins Package …
sysbot.utils: Utilities Package …

Classes

class Sysbot (components=None)

Main Sysbot class for managing system automation and remote connections.

This class provides a unified interface for managing remote system connections, executing commands, and interacting with various system modules and plugins. It supports multiple protocols (SSH, WinRM, Socket, Local) and can manage tunneling configurations for complex network setups.

Attributes

ROBOT_LIBRARY_SCOPE: Robot Framework library scope set to GLOBAL.
ROBOT_LIBRARY_DOC_FORMAT: Documentation format set to reST.

Initialize the Sysbot instance.

Args

components: Optional list of component paths to load. If None, automatically discovers and loads all available modules and plugins. Component paths should be in the format 'modules.name' or 'plugins.name'.

Class variables

var ROBOT_LIBRARY_DOC_FORMAT
var ROBOT_LIBRARY_SCOPE

Methods

def add_secret(self, secret_name: str, value: ) ‑> None

Add or update a secret in the secret cache.

Args

secret_name: Name of the secret to store.
value: Secret value to store (can be any type).

def call_components(self, function_path: str, *args, **kwargs) ‑>

Dynamically call a function from loaded components.

This method allows calling any function from loaded modules or plugins using a dot-notation path (e.g., 'modules.linux.systemd.is_active').

Args

function_path: Dot-separated path to the function (e.g., 'module.submodule.function'). Must contain at least module.function.
*args: Positional arguments to pass to the function.
**kwargs: Keyword arguments to pass to the function.

Returns

The result returned by the called function.

Raises

ValueError: If the function path format is invalid.
AttributeError: If the module or function is not found.
TypeError: If the target is not a callable function.
Exception: If the function call fails.

def close_all_sessions(self) ‑> None

Close all active sessions and clean up associated resources.

This method closes all open connections, stops all active tunnels, and clears the connection cache.

Raises

Exception: If any session fails to close properly.

def close_session(self, alias: str) ‑> None

Close a specific session identified by its alias.

Args

alias: Session alias identifying the connection to close.

Raises

RuntimeError: If no valid session is found for the alias.
Exception: If the session fails to close properly.

def execute_command(self, alias: str, command: str, **kwargs) ‑>

Execute a command on a remote session.

Args

alias: Session alias identifying the connection to use.
command: Command string to execute on the remote system.
**kwargs: Additional command execution options specific to the protocol.

Returns

Command execution result. The format depends on the protocol used.

Raises

ValueError: If the specified alias does not exist.
RuntimeError: If no valid session is found for the alias.
Exception: If command execution fails.

def get_secret(self, secret_name: str) ‑>

Retrieve a secret value from the secret cache.

Args

secret_name: Name of the secret to retrieve.

Returns

The secret value associated with the given name.

def open_session(self, alias: str, protocol: str, product: str, host: str, port: int, login: str = None, password: str = None, tunnel_config=None, is_secret=False, **kwargs) ‑> None

Open a new remote session with the specified connection parameters.

This method establishes a connection to a remote system using the specified protocol and credentials. It supports direct connections and tunneling through intermediate hosts for complex network configurations.

Args

alias: Unique identifier for the session.
protocol: Connection protocol to use (e.g., 'ssh', 'winrm', 'socket', 'local').
product: Product-specific implementation (e.g., 'bash', 'powershell').
host: Target host IP address or hostname.
port: Target port number.
login: Username for authentication. Optional if is_secret is True.
password: Password for authentication. Optional if is_secret is True.
tunnel_config: Optional tunnel configuration as JSON string or dict for nested tunneling through intermediate hosts.
is_secret: If True, treats host, login, and password as secret keys to retrieve actual values from the secret cache.
**kwargs: Additional protocol-specific connection options.

Raises

Exception: If the session fails to open or tunnel configuration is invalid.

def remove_secret(self, secret_name: str) ‑> None

Remove a secret from the secret cache.

Args

secret_name: Name of the secret to remove.

SysBot

Table of Contents

Overview

Key Features

Architecture

Installation

Prerequisites

Install from PyPI

Optional Dependencies

Quickstart

Basic SSH Connection

SSH Tunneling

Secret Management

Module System

Session Management

Supported Protocols

SSH

Local Execution

HTTP/HTTPS

WinRM

Socket

RobotFramework Usage

Basic Robot Framework Test

Using Modules in Robot Framework

Secret Management in Robot Framework

UnitTest Usage

Module Testing with UnitTest

Listener Usage

Available Listeners

Usage with Robot Framework

Listener Parameters

Data Structure

Installation Requirements

Polarion Integration

Linking Tests to Polarion

Tag Format

Generating Polarion-Compatible xUnit

Importing into Polarion

Generated xUnit Content

Additional Resources

Documentation

Online Documentation

Viewing Documentation Locally with pdoc3

Error Handling

License

Author

Links

Sub-modules

Classes

Attributes

Args

Class variables

Methods

Args

Args

Returns

Raises

Raises

Args

Raises

Args

Returns

Raises

Args

Returns

Args

Raises

Args