metascraper/doc/DEPLOYMENT.md

MetaScraper Deployment Guide

📦 Package Publishing

Preparation Checklist

Before publishing, ensure:

• All tests pass: npm test
• Code is properly documented
• Version number follows semantic versioning
• CHANGELOG.md is updated
• Package.json is complete and accurate
• License file is present
• README.md is up to date

Version Management

Semantic Versioning

# Patch version (bug fixes)
npm version patch

# Minor version (new features, backward compatible)
npm version minor

# Major version (breaking changes)
npm version major

Version Numbering Rules

• MAJOR: Breaking changes (API changes, Node.js version requirements)
• MINOR: New features (new Turkish patterns, performance improvements)
• PATCH: Bug fixes (error handling, small fixes)

Package.json Configuration

{
  "name": "flixscaper",
  "version": "1.0.0",
  "description": "Netflix meta veri scraper.",
  "type": "module",
  "main": "src/index.js",
  "exports": {
    ".": "./src/index.js",
    "./parser": "./src/parser.js",
    "./headless": "./src/headless.js"
  },
  "files": [
    "src/",
    "README.md",
    "LICENSE",
    "CHANGELOG.md"
  ],
  "engines": {
    "node": ">=18"
  },
  "keywords": [
    "netflix",
    "scraper",
    "metadata",
    "turkish",
    "flixscaper"
  ],
  "repository": {
    "type": "git",
    "url": "https://github.com/username/flixscaper.git"
  }
}

Publishing Process

1. Local Testing

# Test package locally
npm pack

# Install in test project
npm install ./flixscaper-1.0.0.tgz

# Test functionality
node -e "import { scraperNetflix } from 'flixscaper'; console.log('Import successful')"

2. NPM Registry Publishing

# Login to npm
npm login

# Publish to public registry
npm publish

# Publish with beta tag
npm publish --tag beta

# Publish dry run
npm publish --dry-run

3. Private Registry Publishing

# Publish to private registry
npm publish --registry https://registry.yourcompany.com

# Configure default registry
npm config set registry https://registry.yourcompany.com

🏗️ Build & Distribution

Source Distribution

MetaScraper is distributed as source code with minimal processing:

# Files included in distribution
src/
├── index.js     # Main entry point
├── parser.js    # HTML parsing logic
├── headless.js  # Playwright integration
└── polyfill.js  # Node.js compatibility

# Documentation files
README.md
LICENSE
CHANGELOG.md

# Configuration files
package.json

Browser/Node.js Compatibility

Node.js Support Matrix

Node.js Version	Support Status	Notes
18.x	✅ Full Support	Requires polyfill
20.x	✅ Full Support	Polyfill optional
22.x	✅ Full Support	Native support
16.x	❌ Not Supported	Use older version or upgrade
<16.x	❌ Not Supported	Major compatibility issues

Compatibility Layer

// src/polyfill.js - Automatic compatibility handling
import { Blob } from 'node:buffer';

// Only apply polyfill if needed
if (typeof globalThis.File === 'undefined') {
  class PolyfillFile extends Blob {
    constructor(parts, name, options = {}) {
      super(parts, options);
      this.name = String(name);
      this.lastModified = options.lastModified ?? Date.now();
    }
  }
  globalThis.File = PolyfillFile;
}

globalThis.Blob = globalThis.Blob || Blob;

🔄 Continuous Integration/Deployment

GitHub Actions Workflow

# .github/workflows/deploy.yml
name: Deploy Package

on:
  push:
    tags:
      - 'v*'
  release:
    types: [published]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        node-version: [18.x, 20.x, 22.x]

    steps:
    - uses: actions/checkout@v3

    - name: Setup Node.js
      uses: actions/setup-node@v3
      with:
        node-version: ${{ matrix.node-version }}
        cache: 'npm'
        registry-url: 'https://registry.npmjs.org'

    - name: Install dependencies
      run: npm ci

    - name: Run tests
      run: npm test

    - name: Run linting
      run: npm run lint

    - name: Check build
      run: npm pack

  publish:
    needs: test
    runs-on: ubuntu-latest
    if: github.event_name == 'release' || startsWith(github.ref, 'refs/tags/')

    steps:
    - uses: actions/checkout@v3

    - name: Setup Node.js
      uses: actions/setup-node@v3
      with:
        node-version: '20.x'
        cache: 'npm'
        registry-url: 'https://registry.npmjs.org'

    - name: Install dependencies
      run: npm ci

    - name: Build package
      run: npm pack

    - name: Publish to NPM
      run: npm publish
      env:
        NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

Automated Testing Pipeline

# .github/workflows/test.yml
name: Test Suite

on:
  push:
    branches: [ main, develop ]
  pull_request:
    branches: [ main ]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        node-version: [18.x, 20.x, 22.x]
        os: [ubuntu-latest, windows-latest, macos-latest]

    steps:
    - uses: actions/checkout@v3

    - name: Setup Node.js ${{ matrix.node-version }}
      uses: actions/setup-node@v3
      with:
        node-version: ${{ matrix.node-version }}
        cache: 'npm'

    - name: Install dependencies
      run: npm ci

    - name: Install Playwright (if needed)
      run: npx playwright install chromium

    - name: Run tests
      run: npm test -- --coverage

    - name: Upload coverage
      uses: codecov/codecov-action@v3

🐳 Docker Deployment

Dockerfile

# Dockerfile
FROM node:18-alpine

WORKDIR /app

# Copy package files
COPY package*.json ./

# Install dependencies
RUN npm ci --only=production

# Copy source code
COPY src/ ./src/

# Create non-root user
RUN addgroup -g 1001 -S nodejs
RUN adduser -S flixscaper -u 1001

# Change ownership
RUN chown -R flixscaper:nodejs /app
USER flixscaper

# Health check
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
  CMD node -e "import('flixscaper').then(() => process.exit(0)).catch(() => process.exit(1))"

EXPOSE 3000

CMD ["node", "-e", "import('flixscaper').then(m => console.log('MetaScraper ready'))"]

Docker Compose

# docker-compose.yml
version: '3.8'

services:
  flixscaper:
    build: .
    container_name: flixscaper
    environment:
      - NODE_ENV=production
    volumes:
      - ./logs:/app/logs
    restart: unless-stopped

  flixscaper-test:
    build: .
    container_name: flixscaper-test
    command: npm test
    environment:
      - NODE_ENV=test
    volumes:
      - .:/app
      - /app/node_modules

Building Docker Images

# Build image
docker build -t flixscaper:latest .

# Build with specific version
docker build -t flixscaper:1.0.0 .

# Run container
docker run --rm flixscaper:latest node -e "
  import('flixscaper').then(async (m) => {
    const result = await m.scraperNetflix('https://www.netflix.com/title/80189685');
    console.log(result);
  })
"

🔒 Security Considerations

Package Security

Dependency Scanning

# Audit dependencies for vulnerabilities
npm audit

# Fix vulnerabilities
npm audit fix

# Generate security report
npm audit --json > security-report.json

Secure Publishing

# Use 2FA for npm account
npm profile enable-2fa

# Check package contents before publishing
npm pack --dry-run

# Verify no sensitive files included
tar -tzf flixscaper-*.tgz | grep -E "(key|secret|password|token)" || echo "No sensitive files found"

Runtime Security

Input Validation

// Ensure all inputs are validated
function validateInput(url, options = {}) {
  if (!url || typeof url !== 'string') {
    throw new Error('Invalid URL provided');
  }

  // Validate URL format
  try {
    new URL(url);
  } catch {
    throw new Error('Invalid URL format');
  }

  // Sanitize options
  const safeOptions = {
    headless: Boolean(options.headless),
    timeoutMs: Math.max(1000, Math.min(60000, Number(options.timeoutMs) || 15000)),
    userAgent: typeof options.userAgent === 'string' ? options.userAgent : undefined
  };

  return safeOptions;
}

Network Security

// Secure request configuration
const secureHeaders = {
  'User-Agent': userAgent || DEFAULT_USER_AGENT,
  'Accept': 'text/html,application/xhtml+xml',
  'Accept-Language': 'en-US,en;q=0.9',
  'Cache-Control': 'no-cache',
  'Pragma': 'no-cache'
};

// Rate limiting consideration
const requestDelay = 1000; // 1 second between requests

📊 Monitoring & Analytics

Usage Analytics

Basic Metrics Collection

// Optional analytics (user consent required)
function trackUsage(url, options, success, duration) {
  if (!options.analytics) return;

  const metrics = {
    timestamp: Date.now(),
    url: url.replace(/\/title\/\d+/, '/title/XXXXXX'), // Anonymize
    headless: options.headless,
    success: success,
    duration: duration,
    nodeVersion: process.version,
    version: require('./package.json').version
  };

  // Send to analytics service (optional)
  // analytics.track('flixscaper_usage', metrics);
}

Error Tracking

function trackError(error, context) {
  const errorInfo = {
    message: error.message,
    stack: error.stack,
    context: context,
    timestamp: Date.now(),
    nodeVersion: process.version
  };

  // Log for debugging
  console.error('MetaScraper Error:', errorInfo);

  // Optional: Send to error tracking service
  // errorTracker.captureException(error, { extra: context });
}

Performance Monitoring

// Performance metrics
class PerformanceMonitor {
  constructor() {
    this.metrics = {
      totalRequests: 0,
      successfulRequests: 0,
      averageResponseTime: 0,
      errorCounts: {}
    };
  }

  recordRequest(duration, success, error = null) {
    this.metrics.totalRequests++;

    if (success) {
      this.metrics.successfulRequests++;
    } else {
      this.metrics.errorCounts[error?.message] =
        (this.metrics.errorCounts[error?.message] || 0) + 1;
    }

    // Update average response time
    this.metrics.averageResponseTime =
      (this.metrics.averageResponseTime * (this.metrics.totalRequests - 1) + duration)
      / this.metrics.totalRequests;
  }

  getMetrics() {
    return {
      ...this.metrics,
      successRate: (this.metrics.successfulRequests / this.metrics.totalRequests) * 100
    };
  }
}

🔄 Version Management

Release Process

1. Development Release

# Create feature branch
git checkout -b feature/new-patterns

# Implement changes
# Add tests
# Update documentation

# Create development release
npm version prerelease --preid=dev
git push --tags
npm publish --tag dev

2. Production Release

# Merge to main
git checkout main
git merge develop

# Update version
npm version minor  # or patch/major

# Create GitHub release
gh release create v1.1.0 --generate-notes

# Publish to npm
npm publish

3. Hotfix Release

# Create hotfix branch from main
git checkout -b hotfix/critical-bug

# Fix issue
npm version patch

# Publish immediately
npm publish --tag latest

# Merge back to develop
git checkout develop
git merge main
git checkout main
git merge hotfix/critical-bug

Changelog Management

# CHANGELOG.md

## [1.1.0] - 2025-11-23

### Added
- New Turkish UI pattern: "yeni başlık"
- Performance monitoring API
- Docker support

### Fixed
- Memory leak in Playwright cleanup
- URL validation for Turkish Netflix domains

### Changed
- Improved error messages in Turkish
- Updated Node.js compatibility matrix

### Deprecated
- Support for Node.js 16.x (will be removed in 2.0.0)

## [1.0.1] - 2025-11-20

### Fixed
- Critical bug in title cleaning
- Missing year extraction for movies

🌐 Distribution Channels

NPM Registry

// package.json - publishing configuration
{
  "publishConfig": {
    "access": "public",
    "registry": "https://registry.npmjs.org"
  },
  "repository": {
    "type": "git",
    "url": "https://github.com/username/flixscaper.git"
  },
  "bugs": {
    "url": "https://github.com/username/flixscaper/issues"
  },
  "homepage": "https://github.com/username/flixscaper#readme"
}

CDN Distribution

// For browser usage (future enhancement)
// Available via CDN:
// https://cdn.jsdelivr.net/npm/flixscaper/dist/flixscaper.min.js

import('https://cdn.jsdelivr.net/npm/flixscaper@latest/dist/flixscaper.min.js')
  .then(module => {
    const { scraperNetflix } = module;
    // Use in browser
  });

Private Distribution

# For enterprise/internal distribution
npm config set @company:registry https://npm.company.com

# Publish to private registry
npm publish --registry https://npm.company.com

# Install from private registry
npm install @company/flixscaper

---

Deployment guide last updated: 2025-11-23