dupe/docs/knowledge-base.md

du.pe - Knowledge Base & Development Guide

🎯 Quick Reference

What is du.pe?

A self-hosted torrent-based file manager and media player that provides a clean web interface for managing torrents and streaming media content. Think of it as a personal Put.io alternative.

Core Features

• 🧲 Torrent management (files & magnet links)
• 📥 Real-time download progress tracking
• 🎬 Instant video streaming
• 🗂️ Media library organization
• 🔍 Search functionality
• 🗑️ Trash management
• 📱 Mobile-friendly interface

Technology Stack

• Frontend: Svelte + Vite
• Backend: Node.js + Express
• Torrent Engine: WebTorrent
• Real-time: WebSocket
• Deployment: Docker

---

🏗️ Architecture Overview

System Architecture Diagram

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   Browser UI    │ ←→ │   Express API   │ ←→ │  WebTorrent     │
│   (Svelte)      │    │   (REST + WS)   │    │   Engine        │
└─────────────────┘    └─────────────────┘    └─────────────────┘
         ↓                       ↓                       ↓
   Real-time updates      File operations        Torrent management
   (WebSocket)            (API calls)           (Download/Seed)
         ↓                       ↓                       ↓
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│  State Stores   │    │  File System    │    │  External APIs  │
│  (Media/Data)   │    │  (Downloads)    │    │  (TMDB/TVDB)    │
└─────────────────┘    └─────────────────┘    └─────────────────┘

Data Flow Patterns

Torrent Addition Flow

1. User adds torrent (file/magnet) → Frontend
2. Frontend validates input → API endpoint
3. Server creates WebTorrent instance → Torrent engine
4. Download starts → WebSocket broadcasts progress
5. Files download → Organized in /downloads/
6. Metadata fetched → Cached in /cache/
7. UI updates → Real-time progress display

Media Streaming Flow

1. User clicks play → Frontend requests stream
2. API validates access → Checks file availability
3. Server streams file → HTTP Range requests supported
4. Browser plays → Native HTML5 video player
5. Optional transcoding → For codec compatibility

---

🔧 Component Deep Dive

Frontend Components

Core Application Structure

App.svelte (Root)
├── Router (svelte-routing)
├── Layout Components
│   ├── Sidebar (Navigation)
│   └── Topbar (Actions/Menu)
├── Route Components
│   ├── Files (File browser)
│   ├── Movies (Movie library)
│   ├── TvShows (TV series library)
│   ├── Transfers (Active torrents)
│   └── Trash (Deleted items)
└── Services
    ├── WebSocket client
    ├── API client
    └── State management

State Management (Svelte Stores)

• movieStore.js - Movie library state and operations
• tvStore.js - TV show library state and operations
• searchStore.js - Search functionality and results
• trashStore.js - Trash management state

Key Features by Component

Files.svelte - File browser with:

• Directory navigation
• File type filtering
• Thumbnail previews
• Multi-select operations
• Drag & drop support

Movies.svelte - Movie library with:

• Grid/list view toggle
• Genre/year filtering
• Search functionality
• Metadata display
• Watch status tracking

Transfers.svelte - Torrent management with:

• Real-time progress bars
• Download/upload speed display
• Peer information
• Torrent control actions (pause, remove)
• File list within torrents

Backend Services

Server Architecture

server.js (Main server)
├── Express Application
│   ├── REST API endpoints
│   ├── Static file serving
│   ├── File upload handling
│   └── Error middleware
├── WebSocket Server
│   ├── Real-time progress events
│   ├── File update notifications
│   └── Connection management
├── WebTorrent Client
│   ├── Torrent management
│   ├── File downloading
│   └── Seeding functionality
└── File System Operations
    ├── Download organization
    ├── Thumbnail generation
    ├── Metadata caching
    └── Trash management

Key Server Features

Torrent Management

• Add torrents via file upload or magnet links
• Real-time progress tracking via WebSocket
• File access within torrents
• Automatic file organization

Media Processing

• Automatic metadata fetching (TMDB/TVDB)
• Thumbnail generation for videos and images
• Library categorization (movies/TV shows)
• Search indexing

File Operations

• Streaming with HTTP Range support
• File browser with directory navigation
• Trash system with restore capability
• Disk space monitoring

---

🔄 Real-time Communication

WebSocket Events

// Client-side WebSocket handling
const ws = new WebSocket('ws://localhost:3001/?token=auth');

ws.onmessage = (event) => {
  const message = JSON.parse(event.data);

  switch(message.type) {
    case 'progress':
      // Update torrent progress UI
      updateTorrentProgress(message.torrents);
      break;

    case 'fileUpdate':
      // Refresh media libraries
      refreshMediaLibrary();
      break;

    case 'error':
      // Handle errors
      showErrorMessage(message.message);
      break;
  }
};

Event Types

• progress - Torrent download/upload progress updates
• fileUpdate - Media library changes (added/removed files)
• torrentAdded - New torrent added successfully
• torrentRemoved - Torrent removed or completed
• error - Various error notifications

---

📊 Data Models

Torrent Data Model

{
  infoHash: "abc123def456...",      // Unique identifier
  name: "Example Movie 2024",       // Torrent name
  progress: 0.75,                   // Download progress (0-1)
  downloadSpeed: 2097152,          // Bytes per second
  uploadSpeed: 1048576,            // Bytes per second
  numPeers: 25,                     // Connected peers
  state: "downloading",             // Current state
  timeRemaining: 600,               // Seconds remaining
  files: [                          // File array
    {
      name: "movie.mp4",
      length: 1073741824,
      path: "/downloads/example/",
      type: "video"
    }
  ],
  magnetURI: "magnet:?xt=...",      // Magnet link
  added: "2024-01-15T10:30:00Z"     // Addition timestamp
}

Movie Data Model

{
  id: "movie_123",
  title: "Example Movie",
  year: 2024,
  runtime: 120,
  rating: 8.5,
  genres: ["Action", "Drama"],
  poster: "/cache/movie_data/123/poster.jpg",
  backdrop: "/cache/movie_data/123/backdrop.jpg",
  overview: "Movie description...",
  filePath: "/downloads/example/movie.mp4",
  fileSize: 1073741824,
  added: "2024-01-15T10:30:00Z",
  watched: false,
  thumbnail: "/cache/thumbnails/videos/123/movie.jpg"
}

TV Show Data Model

{
  id: "show_456",
  title: "Example TV Series",
  year: 2023,
  rating: 9.0,
  seasons: 2,
  poster: "/cache/tv_data/456/poster.jpg",
  backdrop: "/cache/tv_data/456/backdrop.jpg",
  overview: "Series description...",
  episodes: [
    {
      season: 1,
      episode: 1,
      title: "Pilot Episode",
      overview: "Episode description...",
      airDate: "2023-01-15",
      filePath: "/downloads/example/s01e01.mp4",
      thumbnail: "/cache/tv_data/456/episodes/s01e01.jpg"
    }
  ]
}

---

🛠️ Development Guide

Getting Started for Development

Prerequisites

• Node.js 18+
• Docker & Docker Compose
• Git

Local Development Setup

# Clone the repository
git clone <repository-url>
cd dupe

# Start backend server
cd server
npm install
npm start

# Start frontend (in separate terminal)
cd client
npm install
npm run dev

# Access the application
# Frontend: http://localhost:5173
# Backend API: http://localhost:3001

Docker Development

# Build and run with Docker Compose
docker compose up -d --build

# View logs
docker compose logs -f

# Stop containers
docker compose down

Configuration

Environment Variables

# .env file
USERNAME=admin                    # Basic auth username
PASSWORD=password                # Basic auth password
TMDB_API_KEY=your_tmdb_key      # The Movie Database API key
TVDB_API_KEY=your_tvdb_key      # The TV Database API key
FANART_TV_API_KEY=your_fanart_key # Fanart.tv API key
VIDEO_THUMBNAIL_TIME=30         # Thumbnail timestamp (seconds)
PORT=3001                       # Server port

Frontend Configuration

// client/vite.config.js
export default {
  server: {
    host: '0.0.0.0',
    port: 5173
  },
  define: {
    'VITE_API': JSON.stringify('http://localhost:3001')
  }
}

Code Structure Guidelines

Frontend (Svelte)

• Use Svelte stores for state management
• Follow component-based architecture
• Implement proper error handling
• Use CSS variables for theming
• Maintain responsive design

Backend (Node.js)

• Use async/await for asynchronous operations
• Implement proper error middleware
• Follow RESTful API conventions
• Use meaningful HTTP status codes
• Implement proper logging

File Organization

client/src/
├── components/     # Reusable UI components
├── routes/        # Page components
├── stores/        # State management
├── utils/         # Utility functions
└── styles/        # CSS/styling

server/
├── utils/         # Server utilities
├── downloads/     # Downloaded files
├── cache/         # Cached data
└── trash/         # Deleted items

Testing Guidelines

Frontend Testing

# Component testing (planned)
npm run test:unit

# E2E testing (planned)
npm run test:e2e

Backend Testing

# API endpoint testing (planned)
npm run test:api

# Integration testing (planned)
npm run test:integration

Common Development Tasks

Adding New API Endpoints

1. Add route handler in server/server.js
2. Implement error handling
3. Add WebSocket events if needed
4. Update frontend API client
5. Add state management if required

Adding New Frontend Pages

1. Create route component in client/src/routes/
2. Add route in App.svelte
3. Update navigation in Sidebar.svelte
4. Add API calls as needed
5. Implement state management

Adding New Features

1. Plan API changes first
2. Implement backend logic
3. Add WebSocket events for real-time updates
4. Build frontend components
5. Add state management
6. Test thoroughly

---

🚀 Deployment Guide

Production Deployment

Docker Compose (Recommended)

# Production environment
export USERNAME=admin
export PASSWORD=secure_password
export TMDB_API_KEY=your_key

# Deploy
docker compose -f docker-compose.prod.yml up -d

Manual Deployment

# Build frontend
cd client
npm run build

# Start server
cd ../server
npm start

Reverse Proxy (Nginx)

server {
    listen 80;
    server_name your-domain.com;

    location / {
        proxy_pass http://localhost:3001;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}

Monitoring & Maintenance

Health Checks

# Check server status
curl http://localhost:3001/api/system/info

# Check active torrents
curl http://localhost:3001/api/torrents

# Check disk usage
curl http://localhost:3001/api/system/disk

Log Management

# Docker logs
docker compose logs -f dupe

# Application logs
tail -f /var/log/dupe/app.log

Backup Strategy

# Backup downloads
tar -czf downloads-backup.tar.gz downloads/

# Backup application data
tar -czf app-backup.tar.gz cache/ trash/

---

🔍 Troubleshooting

Common Issues

Torrent Not Downloading

• Check internet connection
• Verify torrent file/magnet link validity
• Check tracker status
• Review server logs for errors

Video Not Streaming

• Verify file exists and is accessible
• Check browser codec support
• Test with different browsers
• Review network connectivity

WebSocket Connection Issues

• Check authentication token
• Verify WebSocket server is running
• Review browser console for errors
• Check firewall/proxy settings

Performance Issues

• Monitor disk space usage
• Check network bandwidth
• Review system resources
• Consider cleanup operations

Debug Commands

# Check Docker container status
docker ps

# Inspect container logs
docker logs dupe

# Test API endpoints
curl -v http://localhost:3001/api/torrents

# Monitor resource usage
docker stats dupe

---

📚 Cross-References

Documentation Links

• Project Structure - Detailed architecture overview
• API Documentation - Complete API reference
• Component Guide - Frontend component documentation
• Deployment Guide - Production deployment instructions

Key Implementation Files

• server/server.js - Main server implementation
• client/src/App.svelte - Root application component
• client/src/utils/api.js - Frontend API client
• docker-compose.yml - Container configuration

External Dependencies

• Svelte Documentation - Frontend framework
• Express.js Documentation - Backend framework
• WebTorrent Documentation - Torrent engine
• WebSocket API - Real-time communication

---

🛣️ Development Roadmap

Planned Features

• 🔐 Complete user authentication system
• 🎛️ Advanced video player controls
• 📱 Mobile app (React Native)
• 🔄 Automatic subtitle downloading
• 📊 Advanced analytics and reporting
• 🌐 Multi-language support
• ⚡ CDN integration for remote access
• 🔗 Share links for media content

Technical Improvements

• 🧪 Comprehensive test suite
• 📈 Performance monitoring
• 🔒 Enhanced security measures
• 🗂️ Advanced file organization
• 🔄 Database integration
• 📊 Usage analytics
• 🛡️ Rate limiting and DDoS protection

---

This knowledge base provides comprehensive documentation for understanding, developing, and deploying the du.pe application. For specific implementation details, refer to the cross-referenced files and external documentation.