tooling and docs
This commit is contained in:
172
CLAUDE.md
172
CLAUDE.md
@@ -9,22 +9,26 @@ This is a NiceGUI-based web platform for testing and managing AI models through
|
||||
A streamlined interface for testing AI models locally, managing Ollama models, and running various AI-related testing tools.
|
||||
|
||||
### Main Features:
|
||||
1. **Model Manager** - Complete Ollama model management interface
|
||||
1. **Comprehensive System Monitoring** - Real-time resource tracking for AI workloads
|
||||
- Live dashboard with GPU, CPU, memory, disk, and network monitoring
|
||||
- Process monitoring with real-time top processes display
|
||||
- Enhanced header with critical metrics (GPU load, VRAM, RAM, disk space)
|
||||
- Detailed tooltips showing active Ollama models
|
||||
|
||||
2. **Model Manager** - Complete Ollama model management interface
|
||||
- Download, delete, create, and test models
|
||||
- Support for Hugging Face models via Ollama pull syntax
|
||||
- Rich model metadata display
|
||||
- Quick in-app chat testing
|
||||
- Rich model metadata display with size, quantization, context length
|
||||
- Quick in-app chat testing interface
|
||||
|
||||
2. **System Monitoring** - Resource tracking for AI workloads
|
||||
- Real-time GPU monitoring (AMD/NVIDIA) to track model performance
|
||||
- CPU and memory usage during model inference
|
||||
- System metrics dashboard
|
||||
3. **Plugin-Based Tool System** - Extensible framework for AI testing tools
|
||||
- Auto-discovery of tools from `src/tools/` directory
|
||||
- Each tool can have multiple sub-pages with routing
|
||||
- Tools have access to system monitors via ToolContext
|
||||
- Enable/disable tools via simple property override
|
||||
|
||||
3. **AI Testing Tools**:
|
||||
- **Censor** - Text content filtering/censoring tool for testing AI outputs
|
||||
- Additional testing tools to be added as needed
|
||||
|
||||
4. **Settings** - Application configuration and refresh intervals
|
||||
4. **External Integrations** - Quick access to related services
|
||||
- Direct link to Open WebUI for advanced model interactions
|
||||
|
||||
## Development Commands
|
||||
|
||||
@@ -70,38 +74,56 @@ uv sync
|
||||
```
|
||||
src/
|
||||
├── main.py # Entry point, NiceGUI app configuration with all routes
|
||||
├── pages/ # Page components (inheriting NiceGUI elements)
|
||||
│ ├── dashboard.py # Main dashboard with system metrics
|
||||
│ ├── ollama_manager.py # Ollama model management interface (AsyncColumn)
|
||||
│ ├── system_overview.py # System information page
|
||||
│ └── welcome.py # Welcome/landing page
|
||||
├── pages/ # Core page components
|
||||
│ ├── dashboard.py # Comprehensive system monitoring dashboard
|
||||
│ └── ollama_manager.py # Ollama model management interface (AsyncColumn)
|
||||
├── components/ # Reusable UI components
|
||||
│ ├── circular_progress.py # Circular progress indicators
|
||||
│ ├── header.py # App header with live status
|
||||
│ ├── sidebar.py # Navigation sidebar with updated menu structure
|
||||
│ ├── header.py # Enhanced header with critical metrics and tooltips
|
||||
│ ├── sidebar.py # Navigation sidebar with auto-populated tools
|
||||
│ ├── bottom_nav.py # Mobile bottom navigation
|
||||
│ ├── ollama_downloader.py # Ollama model downloader component (AsyncCard)
|
||||
│ ├── ollama_model_creation.py # Model creation component (AsyncCard)
|
||||
│ └── ollama_quick_test.py # Model testing component (AsyncCard)
|
||||
├── tools/ # Plugin system for extensible tools
|
||||
│ ├── __init__.py # Auto-discovery and tool registry
|
||||
│ ├── base_tool.py # BaseTool and BasePage classes, ToolContext
|
||||
│ └── example_tool/ # Example tool demonstrating plugin system
|
||||
│ ├── __init__.py
|
||||
│ └── tool.py # ExampleTool with main, settings, history pages
|
||||
├── utils/ # Utility modules
|
||||
│ ├── gpu_monitor.py # GPU monitoring (AMD/NVIDIA auto-detect)
|
||||
│ ├── system_monitor.py # System resource monitoring
|
||||
│ ├── ollama_monitor.py # Ollama status monitoring (bindable dataclass)
|
||||
│ ├── system_monitor.py # Comprehensive system resource monitoring
|
||||
│ ├── ollama_monitor.py # Ollama status and active models monitoring
|
||||
│ └── ollama.py # Ollama API client functions
|
||||
└── static/ # Static assets (CSS, images)
|
||||
└── style.css # Custom dark theme styles
|
||||
```
|
||||
|
||||
### Key Design Patterns
|
||||
1. **Async Components**: Uses custom `niceguiasyncelement` framework for async page/component construction
|
||||
- `AsyncColumn`, `AsyncCard` base classes for complex components
|
||||
- `OllamaManagerPage(AsyncColumn)` for full page async initialization
|
||||
- Async component dialogs with `await component.create()` pattern
|
||||
2. **Bindable Dataclasses**: Monitor classes use `@binding.bindable_dataclass` for reactive data binding
|
||||
- `SystemMonitor`, `GPUMonitor`, `OllamaMonitor` for real-time data updates
|
||||
3. **Environment Configuration**: All app settings are managed via `.env` file and loaded with python-dotenv
|
||||
4. **Centralized Routing**: All routes defined in main.py with layout creation pattern
|
||||
5. **Real-time Updates**: Timer-based updates every 2 seconds for all monitor instances
|
||||
1. **Plugin Architecture**: Extensible tool system with auto-discovery
|
||||
- Tools are auto-discovered from `src/tools/` directory
|
||||
- Each tool inherits from `BaseTool` and defines routes for sub-pages
|
||||
- Tools can be enabled/disabled via simple property override
|
||||
- Sub-routes support: tools can have multiple pages (main, settings, etc.)
|
||||
|
||||
2. **Async Components**: Uses custom `niceguiasyncelement` framework
|
||||
- `BasePage(AsyncColumn)` for consistent tool page structure
|
||||
- `AsyncCard` base classes for complex components
|
||||
- All tool pages inherit from `BasePage` to eliminate boilerplate
|
||||
|
||||
3. **Context Pattern**: Shared resource access via ToolContext
|
||||
- `ToolContext` provides access to system monitors from any tool
|
||||
- Global context initialized in main.py and accessible via `tool.context`
|
||||
- Clean separation between tools and system resources
|
||||
|
||||
4. **Bindable Dataclasses**: Monitor classes use `@binding.bindable_dataclass`
|
||||
- Real-time UI updates with 2-second refresh intervals
|
||||
- `SystemMonitor`, `GPUMonitor`, `OllamaMonitor` for live data
|
||||
|
||||
5. **Enhanced Header**: Critical metrics display with detailed tooltips
|
||||
- GPU load, VRAM usage, system RAM, disk space badges
|
||||
- Active model tooltip with detailed model information
|
||||
- Clean metric formatting with proper units
|
||||
|
||||
## Component Architecture
|
||||
|
||||
@@ -148,7 +170,7 @@ The Ollama API client (`src/utils/ollama.py`) provides async functions:
|
||||
- `model_info()`: Get detailed model information and Modelfile
|
||||
- `stream_chat()`: Stream chat responses
|
||||
|
||||
### AI Model Testing Features:
|
||||
## Tools Plugin System\n\nThe application features an extensible plugin system for AI testing tools:\n\n### Creating a New Tool\n\n1. **Create tool directory**: `src/tools/my_tool/`\n2. **Create tool class**: `src/tools/my_tool/tool.py`\n\n```python\nfrom tools.base_tool import BaseTool, BasePage\nfrom typing import Dict, Callable, Awaitable\n\nclass MyTool(BaseTool):\n @property\n def name(self) -> str:\n return \"My Tool\"\n \n @property\n def description(self) -> str:\n return \"Description of what this tool does\"\n \n @property\n def icon(self) -> str:\n return \"build\" # Material icon name\n \n @property\n def enabled(self) -> bool:\n return True # Set to False to disable\n \n @property\n def routes(self) -> Dict[str, Callable[[], Awaitable]]:\n return {\n '': lambda: MainPage().create(self),\n '/settings': lambda: SettingsPage().create(self),\n }\n\nclass MainPage(BasePage):\n async def content(self):\n # Access system monitors via context\n cpu_usage = self.tool.context.system_monitor.cpu_percent\n active_models = self.tool.context.ollama_monitor.active_models\n \n # Your tool UI here\n ui.label(f\"CPU: {cpu_usage}%\")\n```\n\n### Tool Features:\n- **Auto-discovery**: Tools are automatically found and loaded\n- **Sub-routes**: Tools can have multiple pages (/, /settings, /history, etc.)\n- **Context Access**: Access to system monitors via `self.tool.context`\n- **Enable/Disable**: Control tool visibility via `enabled` property\n- **Consistent Layout**: `BasePage` handles standard layout structure\n\n### AI Model Testing Features:
|
||||
- **Model Discovery & Management**:
|
||||
- Browse and pull models from Ollama library
|
||||
- Support for HuggingFace models via Ollama syntax
|
||||
@@ -219,17 +241,31 @@ Custom dark theme with:
|
||||
- Live data binding for all metrics
|
||||
- Smooth transitions and animations
|
||||
|
||||
## Enhanced Dashboard Features
|
||||
|
||||
The dashboard provides comprehensive real-time monitoring specifically designed for AI workload testing:
|
||||
|
||||
### Primary Monitoring Sections:
|
||||
- **GPU Performance**: Large circular progress for GPU load, VRAM usage bar, temperature & power draw
|
||||
- **CPU & Memory**: Dual circular progress with detailed specs and frequency info
|
||||
- **Ollama Service**: Live status, version, and grid display of active models with metadata
|
||||
- **Storage & Network**: Disk usage bars and real-time network I/O monitoring
|
||||
- **Process Monitoring**: Live table of top processes with CPU%, memory usage, and status
|
||||
- **System Information**: OS details, uptime, load average, hardware specifications
|
||||
|
||||
### Header Enhancements:
|
||||
- **Critical Metrics Badges**: GPU load, VRAM usage, system RAM, disk space with live updates
|
||||
- **Active Models Tooltip**: Detailed grid showing running models with context length, size, VRAM usage
|
||||
- **Live Status Indicators**: Ollama service status with version information
|
||||
|
||||
## NiceGUI Patterns
|
||||
- **Data Binding**: Use `bind_text_from()` and `bind_value_from()` for reactive updates
|
||||
- **Page Routing**: Navigation via `ui.navigate.to(route)` with centralized route handling
|
||||
- **Async Components**: Custom `niceguiasyncelement` framework for complex async initialization
|
||||
- `AsyncColumn.create()` for async page construction
|
||||
- `AsyncCard.create()` for dialog components
|
||||
- `@ui.refreshable` decorators for dynamic content updates
|
||||
- **Timer Updates**: `app.timer()` for periodic data refresh (2-second intervals)
|
||||
- **Dialog Patterns**: Modal dialogs with `await dialog` for user interactions
|
||||
- **Component Layout**: `create_layout(route)` pattern for consistent page structure
|
||||
- **Dark Mode**: Forced dark mode with custom CSS overrides
|
||||
- **Plugin-Based Routing**: Tools auto-register their routes with sub-page support
|
||||
- **Context Pattern**: Shared monitor access via `tool.context` for all plugins
|
||||
- **BasePage Pattern**: Consistent tool page structure with `BasePage(AsyncColumn)`
|
||||
- **Data Binding**: Reactive UI updates with `bind_text_from()` and `bind_value_from()`
|
||||
- **Async Components**: `niceguiasyncelement` framework with `@ui.refreshable` decorators
|
||||
- **Timer Updates**: 2-second intervals for real-time monitoring data
|
||||
- **Dark Mode**: Comprehensive dark theme with custom metric colors
|
||||
|
||||
## Environment Variables
|
||||
Configured in `.env`:
|
||||
@@ -246,25 +282,39 @@ Configured in `.env`:
|
||||
- Use browser DevTools for WebSocket debugging
|
||||
|
||||
## Current Route Structure
|
||||
From main.py routing:
|
||||
- `/` - Dashboard (system metrics for monitoring AI workloads)
|
||||
- `/system` - System Overview (detailed resource information)
|
||||
- `/ollama` - Model Manager (primary interface for AI model testing)
|
||||
- `/censor` - Censor tool (AI output filtering/testing)
|
||||
- `/settings` - Settings (refresh intervals, app configuration)
|
||||
|
||||
### Placeholder Routes (may be repurposed for AI tools):
|
||||
- `/processes` - Reserved for future AI tools
|
||||
- `/network` - Reserved for future AI tools
|
||||
- `/packages` - Reserved for future AI tools
|
||||
- `/logs` - Reserved for future AI tools
|
||||
- `/info` - Reserved for future AI tools
|
||||
### Core Application Routes:
|
||||
- `/` - Comprehensive system monitoring dashboard
|
||||
- `/ollama` - Advanced model manager (download, test, create, manage)
|
||||
- `/settings` - Application configuration and monitoring intervals
|
||||
|
||||
### Plugin System Routes (Auto-Generated):
|
||||
- `/example-tool` - Example tool demonstrating plugin capabilities
|
||||
- `/example-tool/settings` - Tool-specific settings page
|
||||
- `/example-tool/history` - Tool-specific history page
|
||||
- **Dynamic Discovery**: Additional tool routes auto-discovered from `src/tools/` directory
|
||||
|
||||
### External Integrations:
|
||||
- Direct link to Open WebUI for advanced model interactions
|
||||
|
||||
## Tool Development Guide
|
||||
|
||||
### Quick Start:
|
||||
1. Create `src/tools/my_tool/` directory
|
||||
2. Add `tool.py` with class inheriting from `BaseTool`
|
||||
3. Define routes dictionary mapping paths to page classes
|
||||
4. Create page classes inheriting from `BasePage`
|
||||
5. Tool automatically appears in sidebar and routes are registered
|
||||
|
||||
### Advanced Features:
|
||||
- **Context Access**: Access system monitors via `self.tool.context.system_monitor`
|
||||
- **Sub-routing**: Multiple pages per tool (main, settings, config, etc.)
|
||||
- **Enable/Disable**: Control tool visibility via `enabled` property
|
||||
- **Live Data**: Bind to real-time system metrics and Ollama status
|
||||
|
||||
## Future Enhancements
|
||||
- Enhanced model chat interface with conversation history
|
||||
- Model performance benchmarking tools
|
||||
- Batch testing capabilities for multiple models
|
||||
- Output comparison tools between different models
|
||||
- Integration with more AI model formats
|
||||
- Advanced prompt testing and optimization tools
|
||||
- Model fine-tuning interface
|
||||
- Local AI model testing capabilities that prioritize privacy and security
|
||||
- Tools for testing model behaviors that external providers might restrict
|
||||
- Advanced local prompt engineering and safety testing frameworks
|
||||
- Private data processing and analysis tools using local models
|
||||
- Additional testing capabilities as needs are discovered through usage
|
||||
Reference in New Issue
Block a user