Plugin Development Guide

Extend S9S functionality with custom plugins for specialized workflows, integrations, and automation.

🔌 Overview

S9S plugins provide:

• Custom views and interfaces
• Workflow automation
• Third-party integrations
• Data processing pipelines
• Custom commands and shortcuts
• Specialized monitoring and alerting

🚀 Plugin Architecture

Plugin Types

View Plugins: Custom views and interfaces Command Plugins: New commands and operations Filter Plugins: Custom filtering logic Export Plugins: New export formats and destinations Integration Plugins: Third-party service connections Automation Plugins: Workflow and task automation

Plugin Structure

Copy
~/.s9s/plugins/
├── my-plugin/
│   ├── plugin.yaml          # Plugin manifest
│   ├── main.py              # Main plugin code
│   ├── views/               # Custom views
│   ├── commands/            # Custom commands
│   ├── templates/           # Templates and configs
│   └── README.md            # Documentation

📋 Plugin Manifest

Basic Manifest

Copy
# plugin.yaml
name: "my-plugin"
version: "1.0.0"
description: "My custom S9S plugin"
author: "Your Name <[email protected]>"
license: "MIT"

# Plugin metadata
s9s_version: ">=1.0.0"
type: "integration"
category: "monitoring"

# Plugin configuration
entry_point: "main.py"
dependencies:
  - "requests>=2.25.0"
  - "prometheus_client>=0.12.0"

# Plugin capabilities
capabilities:
  - "views"
  - "commands"
  - "filters"
  - "exports"

# Configuration schema
config_schema:
  api_url:
    type: "string"
    required: true
    description: "API endpoint URL"
  api_key:
    type: "string"
    required: true
    secret: true
    description: "API authentication key"
  refresh_interval:
    type: "integer"
    default: 60
    description: "Refresh interval in seconds"

Advanced Manifest

Copy
# Advanced plugin configuration
name: "efficiency-analyzer"
version: "2.1.0"
description: "Advanced job efficiency analysis and optimization"

# Plugin lifecycle
lifecycle:
  install: "scripts/install.sh"
  uninstall: "scripts/uninstall.sh"
  upgrade: "scripts/upgrade.sh"

# Permissions required
permissions:
  - "read:jobs"
  - "read:nodes"
  - "write:reports"
  - "execute:commands"

# Plugin hooks
hooks:
  job_completed: "hooks/job_completed.py"
  node_status_changed: "hooks/node_changed.py"
  user_login: "hooks/user_login.py"

# UI elements
ui:
  views:
    - name: "efficiency"
      title: "Job Efficiency"
      shortcut: "E"
      icon: "bar-chart"
  
  commands:
    - name: "analyze-efficiency"
      description: "Analyze job efficiency"
      shortcut: ":efficiency"
    
  menu_items:
    - section: "Analysis"
      items:
        - "Efficiency Report"
        - "Optimization Suggestions"

🛠️ Plugin Development

Basic Plugin Template

Copy
# main.py
from s9s.plugin import Plugin, View, Command
from s9s.api import JobsAPI, NodesAPI

class MyPlugin(Plugin):
    def __init__(self, config):
        super().__init__(config)
        self.jobs_api = JobsAPI()
        self.nodes_api = NodesAPI()
    
    def initialize(self):
        """Initialize plugin"""
        self.register_view(CustomView(self))
        self.register_command(CustomCommand(self))
        self.log.info("Plugin initialized")
    
    def cleanup(self):
        """Cleanup on plugin unload"""
        self.log.info("Plugin cleaned up")

class CustomView(View):
    name = "my-view"
    title = "My Custom View"
    
    def render(self, screen):
        """Render the view"""
        data = self.plugin.get_data()
        # Render custom interface
        pass
    
    def handle_key(self, key):
        """Handle key presses"""
        if key == 'r':
            self.refresh()
        return True

class CustomCommand(Command):
    name = "my-command"
    description = "Execute custom command"
    
    def execute(self, args):
        """Execute the command"""
        # Custom command logic
        return "Command executed successfully"

View Plugin Example

Copy
# views/efficiency_view.py
from s9s.plugin import View
from s9s.ui import Table, Chart, Panel

class EfficiencyView(View):
    name = "efficiency"
    title = "Job Efficiency Analysis"
    shortcut = "E"
    
    def __init__(self, plugin):
        super().__init__(plugin)
        self.data = None
        self.refresh_interval = 60
    
    def refresh(self):
        """Refresh efficiency data"""
        jobs = self.plugin.jobs_api.get_completed_jobs()
        self.data = self.calculate_efficiency(jobs)
    
    def calculate_efficiency(self, jobs):
        """Calculate job efficiency metrics"""
        efficiency_data = []
        for job in jobs:
            cpu_eff = job.cpu_time / (job.runtime * job.cpu_count)
            mem_eff = job.max_memory / job.requested_memory
            efficiency_data.append({
                'job_id': job.job_id,
                'user': job.user,
                'cpu_efficiency': cpu_eff,
                'memory_efficiency': mem_eff,
                'overall_score': (cpu_eff + mem_eff) / 2
            })
        return efficiency_data
    
    def render(self, screen):
        """Render efficiency view"""
        if not self.data:
            self.refresh()
        
        # Create efficiency table
        table = Table([
            'Job ID', 'User', 'CPU Eff%', 'Mem Eff%', 'Score'
        ])
        
        for row in self.data:
            table.add_row([
                row['job_id'],
                row['user'],
                f"{row['cpu_efficiency']:.1%}",
                f"{row['memory_efficiency']:.1%}",
                f"{row['overall_score']:.1%}"
            ])
        
        # Create efficiency chart
        chart = Chart(
            title="Efficiency Distribution",
            data=[row['overall_score'] for row in self.data],
            chart_type="histogram"
        )
        
        # Layout panels
        screen.split_horizontal([
            Panel(table, title="Job Efficiency"),
            Panel(chart, title="Distribution")
        ])
    
    def handle_key(self, key):
        """Handle key presses"""
        if key == 'r':
            self.refresh()
        elif key == 's':
            self.export_report()
        elif key == 'o':
            self.show_optimization_suggestions()
        return True

Command Plugin Example

Copy
# commands/optimization.py
from s9s.plugin import Command
from s9s.api import JobsAPI

class OptimizationCommand(Command):
    name = "optimize"
    description = "Provide job optimization suggestions"
    usage = "optimize [--user USER] [--job JOB_ID]"
    
    def __init__(self, plugin):
        super().__init__(plugin)
        self.jobs_api = JobsAPI()
    
    def add_arguments(self, parser):
        """Add command arguments"""
        parser.add_argument('--user', help='Analyze specific user')
        parser.add_argument('--job', help='Analyze specific job')
        parser.add_argument('--threshold', type=float, default=0.8,
                          help='Efficiency threshold')
    
    def execute(self, args):
        """Execute optimization analysis"""
        if args.job:
            return self.analyze_job(args.job)
        elif args.user:
            return self.analyze_user(args.user, args.threshold)
        else:
            return self.analyze_cluster(args.threshold)
    
    def analyze_job(self, job_id):
        """Analyze specific job"""
        job = self.jobs_api.get_job(job_id)
        if not job:
            return f"Job {job_id} not found"
        
        suggestions = []
        
        # CPU efficiency analysis
        cpu_eff = job.cpu_time / (job.runtime * job.cpu_count)
        if cpu_eff < 0.8:
            suggestions.append(f"CPU under-utilized ({cpu_eff:.1%}). Consider reducing core count from {job.cpu_count} to {int(job.cpu_count * cpu_eff * 1.2)}")
        
        # Memory analysis
        mem_eff = job.max_memory / job.requested_memory
        if mem_eff < 0.7:
            suggested_mem = int(job.max_memory * 1.2)
            suggestions.append(f"Memory over-allocated. Reduce from {job.requested_memory}GB to {suggested_mem}GB")
        
        return "\\n".join(suggestions) if suggestions else "Job appears well-optimized"

Integration Plugin Example

Copy
# integrations/prometheus.py
from s9s.plugin import Plugin
from prometheus_client import Gauge, Counter, push_to_gateway
import time

class PrometheusIntegration(Plugin):
    name = "prometheus"
    description = "Export S9S metrics to Prometheus"
    
    def initialize(self):
        """Initialize Prometheus metrics"""
        self.setup_metrics()
        self.start_exporter()
    
    def setup_metrics(self):
        """Setup Prometheus metrics"""
        self.metrics = {
            'cluster_utilization': Gauge('s9s_cluster_utilization_percent', 
                                        'Cluster CPU utilization percentage'),
            'jobs_total': Gauge('s9s_jobs_total', 'Total number of jobs', 
                              ['state']),
            'nodes_total': Gauge('s9s_nodes_total', 'Total number of nodes',
                               ['state']),
            'queue_wait_time': Gauge('s9s_queue_wait_seconds',
                                   'Average queue wait time in seconds')
        }
    
    def start_exporter(self):
        """Start metrics export loop"""
        self.schedule_recurring(self.export_metrics, 
                              interval=self.config.get('interval', 30))
    
    def export_metrics(self):
        """Export current metrics to Prometheus"""
        try:
            # Get cluster stats
            stats = self.api.get_cluster_stats()
            
            # Update metrics
            self.metrics['cluster_utilization'].set(stats.cpu_utilization)
            
            for state, count in stats.job_counts.items():
                self.metrics['jobs_total'].labels(state=state).set(count)
            
            for state, count in stats.node_counts.items():
                self.metrics['nodes_total'].labels(state=state).set(count)
            
            self.metrics['queue_wait_time'].set(stats.avg_wait_time)
            
            # Push to gateway if configured
            if self.config.get('pushgateway'):
                push_to_gateway(
                    self.config['pushgateway'], 
                    job='s9s_exporter',
                    registry=self.metrics
                )
                
        except Exception as e:
            self.log.error(f"Failed to export metrics: {e}")

🔧 Plugin APIs

Core APIs

Copy
from s9s.api import (
    JobsAPI,      # Job management
    NodesAPI,     # Node management  
    UsersAPI,     # User information
    PartitionsAPI, # Partition data
    MetricsAPI    # Performance metrics
)

# Job API usage
jobs_api = JobsAPI()
jobs = jobs_api.get_jobs(state='RUNNING')
job = jobs_api.get_job('12345')
jobs_api.cancel_job('12345')

# Node API usage
nodes_api = NodesAPI()
nodes = nodes_api.get_nodes(state='IDLE')
node = nodes_api.get_node('node001')
nodes_api.drain_node('node001', reason='Maintenance')

UI APIs

Copy
from s9s.ui import (
    Table,        # Data tables
    Chart,        # Charts and graphs
    Form,         # Input forms
    Dialog,       # Modal dialogs
    Panel,        # Layout panels
    StatusBar     # Status information
)

# Create interactive table
table = Table(['Column 1', 'Column 2'])
table.add_row(['Value 1', 'Value 2'])
table.set_sortable(True)
table.set_filterable(True)

# Create chart
chart = Chart(
    title="Resource Usage",
    data=[65, 78, 82, 91],
    labels=['CPU', 'Memory', 'GPU', 'Storage'],
    chart_type='bar'
)

Plugin Utilities

Copy
from s9s.plugin import (
    ConfigManager,  # Configuration handling
    Logger,         # Logging utilities
    Scheduler,      # Task scheduling
    Cache,          # Data caching
    Notifications   # User notifications
)

# Configuration
config = ConfigManager(self.config_file)
api_key = config.get('api_key')
config.set('last_updated', time.time())

# Logging
log = Logger(self.name)
log.info("Plugin started")
log.error("Something went wrong", exc_info=True)

# Scheduling
scheduler = Scheduler()
scheduler.every(60).seconds.do(self.update_data)
scheduler.every().hour.do(self.cleanup)

📦 Plugin Distribution

Plugin Repository

Submit plugins to the official repository:

Copy
# Package plugin
s9s plugin package my-plugin

# Validate plugin
s9s plugin validate my-plugin-1.0.0.tar.gz

# Publish to repository
s9s plugin publish my-plugin-1.0.0.tar.gz

Private Distribution

Host plugins privately:

Copy
# ~/.s9s/config.yaml
plugins:
  repositories:
    - url: "https://plugins.example.com/s9s"
      auth:
        type: "token"
        token: "${PLUGIN_REPO_TOKEN}"
    - url: "file:///opt/s9s-plugins"
      type: "local"

🔧 Plugin Management

Installation

Copy
# Install from repository
s9s plugin install efficiency-analyzer

# Install from file
s9s plugin install ./my-plugin-1.0.0.tar.gz

# Install from git
s9s plugin install git+https://github.com/user/s9s-plugin.git

# List available plugins
s9s plugin list --available

Configuration

Copy
# Configure plugin
s9s plugin config efficiency-analyzer
# Opens configuration editor

# Set plugin config values
s9s plugin config efficiency-analyzer --set threshold=0.8
s9s plugin config efficiency-analyzer --set api_url=https://api.example.com

Management

Copy
# List installed plugins
s9s plugin list

# Enable/disable plugins
s9s plugin enable efficiency-analyzer
s9s plugin disable old-plugin

# Update plugins
s9s plugin update efficiency-analyzer
s9s plugin update --all

# Remove plugins
s9s plugin remove old-plugin

🚀 Best Practices

Plugin Development

1. Follow conventions - Use standard naming and structure
2. Handle errors gracefully - Don't crash S9S
3. Be responsive - Don't block the UI
4. Cache appropriately - Balance freshness and performance
5. Document thoroughly - Include usage examples

Performance

1. Async operations - Use async for I/O operations
2. Efficient data structures - Choose appropriate data types
3. Minimal resource usage - Don't consume excessive memory/CPU
4. Lazy loading - Load data only when needed
5. Proper cleanup - Free resources on plugin unload

Security

1. Validate inputs - Sanitize all user inputs
2. Secure credentials - Store secrets safely
3. Minimal permissions - Request only necessary permissions
4. Safe execution - Avoid arbitrary code execution
5. Regular updates - Keep dependencies current

📚 Plugin Examples

Find example plugins at:

• Official Plugin Repository
• Community Plugins
• Plugin Templates

🔍 Debugging Plugins

Copy
# Enable plugin debugging
s9s --debug-plugins plugin-name

# View plugin logs
s9s plugin logs efficiency-analyzer

# Test plugin functionality
s9s plugin test efficiency-analyzer --interactive

# Plugin development mode
s9s --dev-mode --plugin-path ./my-plugin/

🚀 Next Steps

• Study existing plugins for examples
• Start with the plugin template
• Join the plugin development community
• Read the API reference documentation