TTL & Cleanup

Torrin includes built-in mechanisms to automatically expire and clean up abandoned uploads. This prevents storage bloat from incomplete or interrupted uploads.

What is TTL?

TTL (Time To Live) is the maximum lifetime of an upload session. After the TTL expires, the upload is considered abandoned and eligible for cleanup.

Default TTL: 24 hours

How TTL Works

1. Upload Created: TTL timer starts when upload is initialized
2. Updates Reset: Each chunk upload updates the updatedAt timestamp (but not the expiration)
3. Expiration: After TTL time passes, upload status becomes "expired"
4. Cleanup: Expired uploads can be removed with cleanup methods

const service = new TorrinService({
  storage,
  store,
  uploadTtlMs: 24 * 60 * 60 * 1000, // 24 hours (default)
});

Configuring TTL

Server-Side Configuration

Express

import { createTorrinExpressRouter } from '@torrin-kit/server-express';

const router = createTorrinExpressRouter({
  storage,
  store,
  uploadTtlMs: 48 * 60 * 60 * 1000, // 48 hours
});

NestJS

import { TorrinModule } from '@torrin-kit/server-nestjs';

@Module({
  imports: [
    TorrinModule.forRoot({
      storage,
      store,
      uploadTtlMs: 48 * 60 * 60 * 1000, // 48 hours
    }),
  ],
})
export class AppModule {}

Recommended TTL Values

Use Case	Recommended TTL	Reasoning
Small files (<10MB)	1-6 hours	Quick uploads, less likely to need resume
Medium files (10MB-1GB)	12-24 hours	Balance between storage and user convenience
Large files (>1GB)	24-72 hours	Longer uploads, more interruptions possible
Development/Testing	1-2 hours	Faster cleanup for rapid iteration

Manual Cleanup Methods

Clean Expired Uploads

Removes all uploads past their TTL:

import { TorrinService } from '@torrin-kit/server';

const service = new TorrinService({ storage, store, uploadTtlMs });

const result = await service.cleanupExpiredUploads();
console.log(`Cleaned ${result.cleaned} expired uploads`);
console.log(`Errors:`, result.errors);

Returns:

interface CleanupResult {
  cleaned: number;      // Number of successfully cleaned uploads
  errors: string[];     // Error messages for failed cleanups
}

Clean Stale Uploads

Removes uploads not updated within a specific time (regardless of TTL):

// Remove uploads older than 12 hours
const maxAgeMs = 12 * 60 * 60 * 1000;
const result = await service.cleanupStaleUploads(maxAgeMs);

This is useful for:

• Cleaning up truly abandoned uploads faster than TTL
• Emergency cleanup when storage is running low
• Different cleanup policies for different file sizes

Automatic Cleanup

Periodic Cleanup (Recommended)

Run cleanup automatically at regular intervals:

// Clean up every hour
setInterval(async () => {
  try {
    const result = await service.cleanupExpiredUploads();
    if (result.cleaned > 0) {
      console.log(`[Cleanup] Removed ${result.cleaned} expired uploads`);
    }
    if (result.errors.length > 0) {
      console.error(`[Cleanup] Errors:`, result.errors);
    }
  } catch (error) {
    console.error('[Cleanup] Failed:', error);
  }
}, 60 * 60 * 1000);

Cron Job

Use a cron job for production environments:

Node.js with node-cron:

import cron from 'node-cron';
import { TorrinService } from '@torrin-kit/server';

const service = new TorrinService({ storage, store });

// Run cleanup every day at 3 AM
cron.schedule('0 3 * * *', async () => {
  console.log('[Cron] Starting upload cleanup...');
  const result = await service.cleanupExpiredUploads();
  console.log(`[Cron] Cleaned ${result.cleaned} uploads`);
});

Unix Cron:

# /etc/crontab or crontab -e
# Run cleanup script every day at 3 AM
0 3 * * * node /path/to/cleanup-script.js

cleanup-script.js:

const { TorrinService } = require('@torrin-kit/server');
const { createLocalStorageDriver } = require('@torrin-kit/storage-local');
const { createInMemoryStore } = require('@torrin-kit/server');

const service = new TorrinService({
  storage: createLocalStorageDriver({ baseDir: './uploads' }),
  store: createInMemoryStore(),
});

(async () => {
  const result = await service.cleanupExpiredUploads();
  console.log(`Cleaned ${result.cleaned} uploads`);
  process.exit(0);
})();

Cleanup Endpoint

Create an admin endpoint for manual cleanup:

Express

import express from 'express';

const app = express();

app.post('/admin/cleanup', authenticateAdmin, async (req, res) => {
  try {
    const result = await service.cleanupExpiredUploads();
    res.json({
      success: true,
      cleaned: result.cleaned,
      errors: result.errors,
    });
  } catch (error) {
    res.status(500).json({
      success: false,
      error: error.message,
    });
  }
});

NestJS

import { Controller, Post, UseGuards } from '@nestjs/common';
import { InjectTorrin, TorrinService } from '@torrin-kit/server-nestjs';
import { AdminGuard } from './guards/admin.guard';

@Controller('admin')
export class AdminController {
  constructor(
    @InjectTorrin() private readonly torrin: TorrinService
  ) {}

  @Post('cleanup')
  @UseGuards(AdminGuard)
  async cleanup() {
    const result = await this.torrin.cleanupExpiredUploads();
    return {
      success: true,
      cleaned: result.cleaned,
      errors: result.errors,
    };
  }
}

Cleanup Behavior

What Gets Cleaned

1. Upload Session: Metadata and state removed from store
2. Temporary Chunks: All chunk files deleted from storage
3. Final File: If completed but expired, removed from storage (configurable)

What Doesn't Get Cleaned

• Completed uploads within TTL
• Active uploads (recently updated)
• Uploads in progress (currently uploading chunks)

Cleanup Process

graph TD
    A[Start Cleanup] --> B[Query Expired Sessions]
    B --> C{Any Expired?}
    C -->|No| D[Return: cleaned=0]
    C -->|Yes| E[For Each Session]
    E --> F[Delete Temp Chunks]
    F --> G[Delete Session Data]
    G --> H{More Sessions?}
    H -->|Yes| E
    H -->|No| I[Return Results]

Storage-Specific Cleanup

Local Filesystem

import { createLocalStorageDriver } from '@torrin-kit/storage-local';

const storage = createLocalStorageDriver({
  baseDir: './uploads',
  tempDir: './uploads/.temp',
});

// Cleanup removes:
// - ./uploads/.temp/{uploadId}/* (all chunks)
// - Session data from store

S3 Storage

import { createS3StorageDriver } from '@torrin-kit/storage-s3';

const storage = createS3StorageDriver({
  bucket: 'my-uploads',
  region: 'us-east-1',
  keyPrefix: 'temp/',
});

// Cleanup removes:
// - s3://my-uploads/temp/{uploadId}/* (all chunks)
// - Session data from store

Note: S3 cleanup can incur API costs. Consider:

• Using S3 Lifecycle Policies to auto-delete temp objects
• Running cleanup less frequently
• Batching cleanup operations

Store Persistence

In-Memory Store

import { createInMemoryStore } from '@torrin-kit/server';

const store = createInMemoryStore();

Limitations:

• State lost on restart
• No cleanup needed on restart (all sessions gone)
• TTL checked in-memory

Persistent Store (Redis/Database)

For production, use a persistent store:

// Custom Redis store with TTL support
class RedisStore implements TorrinUploadStore {
  async createSession(init, chunkSize, ttlMs) {
    const session = { ...init, uploadId: generateId() };
    
    // Redis native TTL
    await redis.set(
      `session:${session.uploadId}`,
      JSON.stringify(session),
      'PX', // milliseconds
      ttlMs
    );
    
    return session;
  }
  
  // ... other methods
}

Benefits:

• Sessions persist across restarts
• Automatic expiration with Redis TTL
• Scalable across multiple servers

Monitoring Cleanup

Logging Cleanup Activity

async function cleanupWithLogging() {
  const startTime = Date.now();
  const result = await service.cleanupExpiredUploads();
  const duration = Date.now() - startTime;
  
  // Log to monitoring service
  logger.info('Upload cleanup completed', {
    cleaned: result.cleaned,
    errors: result.errors.length,
    duration,
    timestamp: new Date().toISOString(),
  });
  
  // Alert if too many errors
  if (result.errors.length > 10) {
    alerting.send('High cleanup error rate', result.errors);
  }
}

Metrics

Track cleanup metrics for monitoring:

import { Counter, Histogram } from 'prom-client';

const cleanupCounter = new Counter({
  name: 'torrin_cleanup_total',
  help: 'Total cleanups performed',
  labelNames: ['status'],
});

const cleanupDuration = new Histogram({
  name: 'torrin_cleanup_duration_seconds',
  help: 'Cleanup duration in seconds',
});

async function cleanupWithMetrics() {
  const end = cleanupDuration.startTimer();
  
  try {
    const result = await service.cleanupExpiredUploads();
    cleanupCounter.inc({ status: 'success' }, result.cleaned);
    return result;
  } catch (error) {
    cleanupCounter.inc({ status: 'error' });
    throw error;
  } finally {
    end();
  }
}

Best Practices

1. Set Appropriate TTL

   • Longer TTL = better UX (more time to resume)
   • Shorter TTL = less storage waste
   • Balance based on typical file sizes and upload times

2. Regular Cleanup Schedule

   • Run cleanup at least daily
   • Off-peak hours (3-4 AM) for heavy cleanup
   • More frequent for high-volume systems

3. Monitor Storage Usage

   • Track temp storage size
   • Alert on unusual growth
   • Adjust TTL if needed

4. Error Handling

   • Log cleanup errors
   • Retry failed cleanups
   • Alert on repeated failures

5. Gradual Rollout

   • Start with longer TTL (72h)
   • Monitor abandonment rates
   • Adjust based on user behavior

6. Testing Cleanup

   • Test cleanup in staging first
   • Verify temp files are removed
   • Check for storage leaks

Troubleshooting

Cleanup Not Running

Problem: Expired uploads not being removed

Solutions:

• Verify cleanup job is scheduled and running
• Check store supports listExpiredSessions()
• Ensure TTL is configured properly
• Check for errors in cleanup logs

High Storage Usage

Problem: Temp storage growing despite cleanup

Solutions:

• Reduce TTL
• Run cleanup more frequently
• Check for failed cleanups (orphaned files)
• Verify cleanup process has permissions

Cleanup Takes Too Long

Problem: Cleanup blocking other operations

Solutions:

• Batch cleanup operations
• Run cleanup during off-peak hours
• Use pagination for large cleanup jobs
• Consider background job queue

Examples

Complete Production Setup

import express from 'express';
import cron from 'node-cron';
import { TorrinService } from '@torrin-kit/server';
import { createTorrinExpressRouter } from '@torrin-kit/server-express';
import { createS3StorageDriver } from '@torrin-kit/storage-s3';
import { createRedisStore } from './stores/redis';

const service = new TorrinService({
  storage: createS3StorageDriver({
    bucket: process.env.S3_BUCKET,
    region: process.env.AWS_REGION,
  }),
  store: createRedisStore(process.env.REDIS_URL),
  uploadTtlMs: 24 * 60 * 60 * 1000, // 24 hours
});

const app = express();

// Mount router
app.use('/api/uploads', createTorrinExpressRouter({
  storage: service.storage,
  store: service.store,
  uploadTtlMs: service.uploadTtlMs,
}));

// Cleanup every day at 3 AM
cron.schedule('0 3 * * *', async () => {
  console.log('[Cleanup] Starting...');
  const result = await service.cleanupExpiredUploads();
  console.log(`[Cleanup] Removed ${result.cleaned} uploads`);
  if (result.errors.length > 0) {
    console.error('[Cleanup] Errors:', result.errors);
  }
});

// Admin endpoint for manual cleanup
app.post('/admin/cleanup', authenticateAdmin, async (req, res) => {
  const result = await service.cleanupExpiredUploads();
  res.json(result);
});

app.listen(3000);

Next Steps

• Configuration - Configure TTL and other settings
• Storage Drivers - Storage-specific cleanup behavior
• Error Handling - Handle cleanup errors
• Server API - Complete cleanup API reference