270 lines
11 KiB
Markdown
270 lines
11 KiB
Markdown
# CLAUDE.md
|
|
|
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
|
|
## Project Context
|
|
This is a NiceGUI-based web platform for testing and managing AI models through Ollama on Arch Linux systems with GPU support. The application serves as an AI model testing environment featuring:
|
|
|
|
### Core Purpose
|
|
A streamlined interface for testing AI models locally, managing Ollama models, and running various AI-related testing tools.
|
|
|
|
### Main Features:
|
|
1. **Model Manager** - Complete Ollama model management interface
|
|
- Download, delete, create, and test models
|
|
- Support for Hugging Face models via Ollama pull syntax
|
|
- Rich model metadata display
|
|
- Quick in-app chat testing
|
|
|
|
2. **System Monitoring** - Resource tracking for AI workloads
|
|
- Real-time GPU monitoring (AMD/NVIDIA) to track model performance
|
|
- CPU and memory usage during model inference
|
|
- System metrics dashboard
|
|
|
|
3. **AI Testing Tools**:
|
|
- **Censor** - Text content filtering/censoring tool for testing AI outputs
|
|
- Additional testing tools to be added as needed
|
|
|
|
4. **Settings** - Application configuration and refresh intervals
|
|
|
|
## Development Commands
|
|
|
|
### Running the Application
|
|
```bash
|
|
# Install dependencies
|
|
uv sync
|
|
|
|
# Run the development server (use port 8081 for testing as 8080 is usually occupied)
|
|
APP_PORT=8081 uv run python src/main.py
|
|
|
|
# Default port (8080) - usually already in use by main instance
|
|
uv run python src/main.py
|
|
```
|
|
|
|
### Dependency Management
|
|
```bash
|
|
# Add a new dependency
|
|
uv add <package>
|
|
|
|
# Add a dev dependency
|
|
uv add --dev <package>
|
|
|
|
# Update dependencies
|
|
uv sync
|
|
```
|
|
|
|
## Architecture Overview
|
|
|
|
### Technology Stack
|
|
- **Package Manager**: uv (version 0.8.17)
|
|
- **UI Framework**: NiceGUI (async web framework based on FastAPI/Vue.js)
|
|
- **Python Version**: 3.13+
|
|
- **Ollama API**: Running on localhost:11434
|
|
- **Dependencies**:
|
|
- `nicegui` - Main UI framework
|
|
- `niceguiasyncelement` - Custom async component framework (from git)
|
|
- `psutil` - System monitoring
|
|
- `httpx` - Async HTTP client for Ollama API
|
|
- `python-dotenv` - Environment configuration
|
|
|
|
### Project Structure
|
|
```
|
|
src/
|
|
├── main.py # Entry point, NiceGUI app configuration with all routes
|
|
├── pages/ # Page components (inheriting NiceGUI elements)
|
|
│ ├── dashboard.py # Main dashboard with system metrics
|
|
│ ├── ollama_manager.py # Ollama model management interface (AsyncColumn)
|
|
│ ├── system_overview.py # System information page
|
|
│ └── welcome.py # Welcome/landing page
|
|
├── components/ # Reusable UI components
|
|
│ ├── circular_progress.py # Circular progress indicators
|
|
│ ├── header.py # App header with live status
|
|
│ ├── sidebar.py # Navigation sidebar with updated menu structure
|
|
│ ├── bottom_nav.py # Mobile bottom navigation
|
|
│ ├── ollama_downloader.py # Ollama model downloader component (AsyncCard)
|
|
│ ├── ollama_model_creation.py # Model creation component (AsyncCard)
|
|
│ └── ollama_quick_test.py # Model testing component (AsyncCard)
|
|
├── utils/ # Utility modules
|
|
│ ├── gpu_monitor.py # GPU monitoring (AMD/NVIDIA auto-detect)
|
|
│ ├── system_monitor.py # System resource monitoring
|
|
│ ├── ollama_monitor.py # Ollama status monitoring (bindable dataclass)
|
|
│ └── ollama.py # Ollama API client functions
|
|
└── static/ # Static assets (CSS, images)
|
|
└── style.css # Custom dark theme styles
|
|
```
|
|
|
|
### Key Design Patterns
|
|
1. **Async Components**: Uses custom `niceguiasyncelement` framework for async page/component construction
|
|
- `AsyncColumn`, `AsyncCard` base classes for complex components
|
|
- `OllamaManagerPage(AsyncColumn)` for full page async initialization
|
|
- Async component dialogs with `await component.create()` pattern
|
|
2. **Bindable Dataclasses**: Monitor classes use `@binding.bindable_dataclass` for reactive data binding
|
|
- `SystemMonitor`, `GPUMonitor`, `OllamaMonitor` for real-time data updates
|
|
3. **Environment Configuration**: All app settings are managed via `.env` file and loaded with python-dotenv
|
|
4. **Centralized Routing**: All routes defined in main.py with layout creation pattern
|
|
5. **Real-time Updates**: Timer-based updates every 2 seconds for all monitor instances
|
|
|
|
## Component Architecture
|
|
|
|
### Monitor Classes (Supporting AI Testing)
|
|
- **SystemMonitor**: Tracks system resources during AI model testing
|
|
- CPU usage during model inference
|
|
- Memory consumption by loaded models
|
|
- Disk I/O for model loading
|
|
- Process statistics for Ollama and GPU processes
|
|
|
|
- **GPUMonitor**: Critical for AI workload monitoring
|
|
- Auto-detects AMD/NVIDIA GPUs
|
|
- Tracks GPU usage during model inference
|
|
- Memory usage by loaded models
|
|
- Temperature monitoring during extended testing
|
|
- Power draw under AI workloads
|
|
|
|
- **OllamaMonitor**: Core service monitoring
|
|
- Ollama service status and version
|
|
- Currently loaded/active models
|
|
- Real-time model state tracking
|
|
|
|
### UI Components
|
|
- **MetricCircle**: Small circular progress indicator with icon
|
|
- **LargeMetricCircle**: Large circular progress for primary metrics
|
|
- **ColorfulMetricCard**: Action cards with gradient backgrounds
|
|
- **Sidebar**: Navigation menu with updated structure:
|
|
- Main: Dashboard, System Overview
|
|
- Tools: Censor (content filtering)
|
|
- Bottom: Model Manager, Settings
|
|
- **Header**: Top bar with system status indicators
|
|
|
|
### Ollama-Specific Components (AsyncCard-based):
|
|
- **OllamaDownloaderComponent**: Model downloading with progress tracking (supports HF models via Ollama's pull syntax)
|
|
- **OllamaModelCreationComponent**: Custom model creation from Modelfile
|
|
- **ModelQuickTestComponent**: Interactive model testing interface
|
|
|
|
## Ollama Integration
|
|
The Ollama API client (`src/utils/ollama.py`) provides async functions:
|
|
- `status()`: Check if Ollama is online and get version
|
|
- `available_models()`: List installed models with detailed metadata
|
|
- `active_models()`: Get currently loaded/running models
|
|
- `delete_model()`: Remove a model
|
|
- `model_info()`: Get detailed model information and Modelfile
|
|
- `stream_chat()`: Stream chat responses
|
|
|
|
### AI Model Testing Features:
|
|
- **Model Discovery & Management**:
|
|
- Browse and pull models from Ollama library
|
|
- Support for HuggingFace models via Ollama syntax
|
|
- Rich metadata display (size, quantization, parameters, format)
|
|
- Time tracking for model versions
|
|
|
|
- **Testing Capabilities**:
|
|
- Quick chat interface for immediate model testing
|
|
- Model information and Modelfile inspection
|
|
- Custom model creation from Modelfiles
|
|
- Real-time resource monitoring during inference
|
|
|
|
- **Testing Tools**:
|
|
- Censor tool for output filtering analysis
|
|
- Extensible framework for adding new testing tools
|
|
|
|
API endpoints at `http://localhost:11434/api/`:
|
|
- `/api/version`: Get Ollama version
|
|
- `/api/tags`: List available models
|
|
- `/api/pull`: Download models
|
|
- `/api/delete`: Remove models
|
|
- `/api/generate`: Generate text
|
|
- `/api/chat`: Chat completion
|
|
- `/api/ps`: List running models
|
|
- `/api/show`: Show model details
|
|
|
|
## System Monitoring
|
|
|
|
### GPU Monitoring Strategy
|
|
The application uses a hierarchical approach for GPU monitoring:
|
|
|
|
1. **NVIDIA GPUs** (via `nvidia-smi`):
|
|
- Temperature, usage, memory, power draw
|
|
- CUDA version and driver info
|
|
- Multi-GPU support
|
|
|
|
2. **AMD GPUs** (multiple fallbacks):
|
|
- Primary: `rocm-smi` for full metrics
|
|
- Fallback: `/sys/class/drm` filesystem
|
|
- Reads hwmon for temperature data
|
|
- Supports both server and consumer GPUs
|
|
|
|
### CPU & System Monitoring
|
|
- Real-time CPU usage and per-core statistics
|
|
- Memory (RAM and swap) usage
|
|
- Disk usage and I/O statistics
|
|
- Network traffic monitoring
|
|
- Process tracking with top processes by CPU/memory
|
|
- System uptime and kernel information
|
|
|
|
## UI/UX Features
|
|
|
|
### Dark Theme
|
|
Custom dark theme with:
|
|
- Background: `#1a1d2e` (main), `#252837` (sidebar)
|
|
- Card backgrounds: `rgba(26, 29, 46, 0.7)` with backdrop blur
|
|
- Accent colors: Cyan (`#06b6d4`) for primary actions
|
|
- Metric colors: Purple (CPU), Green (Memory), Orange (GPU), Cyan (Temp)
|
|
|
|
### Responsive Design
|
|
- Desktop: Full sidebar navigation
|
|
- Mobile: Bottom navigation bar
|
|
- Adaptive grid layouts for different screen sizes
|
|
- Viewport-aware content scaling
|
|
|
|
### Real-time Updates
|
|
- System metrics update every 2 seconds (configurable via `MONITORING_UPDATE_INTERVAL`)
|
|
- Live data binding for all metrics
|
|
- Smooth transitions and animations
|
|
|
|
## NiceGUI Patterns
|
|
- **Data Binding**: Use `bind_text_from()` and `bind_value_from()` for reactive updates
|
|
- **Page Routing**: Navigation via `ui.navigate.to(route)` with centralized route handling
|
|
- **Async Components**: Custom `niceguiasyncelement` framework for complex async initialization
|
|
- `AsyncColumn.create()` for async page construction
|
|
- `AsyncCard.create()` for dialog components
|
|
- `@ui.refreshable` decorators for dynamic content updates
|
|
- **Timer Updates**: `app.timer()` for periodic data refresh (2-second intervals)
|
|
- **Dialog Patterns**: Modal dialogs with `await dialog` for user interactions
|
|
- **Component Layout**: `create_layout(route)` pattern for consistent page structure
|
|
- **Dark Mode**: Forced dark mode with custom CSS overrides
|
|
|
|
## Environment Variables
|
|
Configured in `.env`:
|
|
- `MONITORING_UPDATE_INTERVAL`: Update frequency in seconds (default: 2)
|
|
- `APP_PORT`: Web server port (default: 8080, use 8081 for testing)
|
|
- `APP_TITLE`: Application title
|
|
- `APP_STORAGE_SECRET`: Session storage encryption key
|
|
- `APP_SHOW`: Auto-open browser on startup
|
|
|
|
## Testing & Development
|
|
- Run on port 8081 to avoid conflicts: `APP_PORT=8081 uv run python src/main.py`
|
|
- Monitor GPU detection in console logs
|
|
- Check Ollama connectivity at startup
|
|
- Use browser DevTools for WebSocket debugging
|
|
|
|
## Current Route Structure
|
|
From main.py routing:
|
|
- `/` - Dashboard (system metrics for monitoring AI workloads)
|
|
- `/system` - System Overview (detailed resource information)
|
|
- `/ollama` - Model Manager (primary interface for AI model testing)
|
|
- `/censor` - Censor tool (AI output filtering/testing)
|
|
- `/settings` - Settings (refresh intervals, app configuration)
|
|
|
|
### Placeholder Routes (may be repurposed for AI tools):
|
|
- `/processes` - Reserved for future AI tools
|
|
- `/network` - Reserved for future AI tools
|
|
- `/packages` - Reserved for future AI tools
|
|
- `/logs` - Reserved for future AI tools
|
|
- `/info` - Reserved for future AI tools
|
|
|
|
## Future Enhancements
|
|
- Enhanced model chat interface with conversation history
|
|
- Model performance benchmarking tools
|
|
- Batch testing capabilities for multiple models
|
|
- Output comparison tools between different models
|
|
- Integration with more AI model formats
|
|
- Advanced prompt testing and optimization tools
|
|
- Model fine-tuning interface |