We needed to turn long-form content into TikTok/Reels/Shorts automatically. No human editors, no manual review — just drop a 10-minute video and get 5-10 vertical clips with captions, ready to post.

Architecture Overview

Video Upload → Queue → AI Analysis → FFmpeg Clip → MediaConvert → S3 → CDN → Webhook → DB

Step 1: AI Clip Selection

We use a vision model (GPT-4V / Claude 3.5) to analyze video frames and score segments for "viral potential": hook strength, visual change density, speech density, emotional peaks.

# Python service (FastAPI)
@app.post("/analyze")
async def analyze_video(video_url: str):
    frames = extract_keyframes(video_url, interval=2)  # 1 frame/2s
    scores = await vision_model.batch_score(frames, prompt=VIRAL_PROMPT)
    return select_top_clips(scores, max_duration=60, count=5)

Step 2: FFmpeg Precision Clipping

Frame-accurate cuts without re-encoding where possible:

# Lossless cut (keyframe-aware)
ffmpeg -ss {start} -i input.mp4 -t {duration} -c copy -avoid_negative_ts make_zero output.mp4

# Re-encode for vertical (9:16) with captions baked in
ffmpeg -i input.mp4 -vf "
  crop=ih*9/16:ih,
  subtitles=captions.srt:force_style='Fontsize=24,PrimaryColour=&HFFFFFF'
" -c:a copy output.mp4

Step 3: AWS MediaConvert for Multi-Platform

MediaConvert handles the heavy lifting — transcoding to H.264/HEVC, multiple bitrates, HLS/DASH packaging:

# Job template (JSON)
{
  "Role": "arn:aws:iam::123456789:role/MediaConvertRole",
  "Settings": {
    "OutputGroups": [{
      "Name": "Apple HLS",
      "OutputGroupSettings": {
        "Type": "HLS_GROUP_SETTINGS",
        "HlsGroupSettings": {"Destination": "s3://bucket/hls/", "SegmentLength": 6}
      },
      "Outputs": [
        {"Width": 1080, "Height": 1920, "Codec": "H_264", "Bitrate": 4000000},
        {"Width": 720, "Height": 1280, "Codec": "H_264", "Bitrate": 2500000},
        {"Width": 480, "Height": 854, "Codec": "H_264", "Bitrate": 1200000}
      ]
    }]
  }
}

Laravel Queue Integration

Everything runs on Laravel Horizon (Redis). Jobs are chained:

VideoUploaded::dispatch($video)
    ->chain([
        AnalyzeVideoJob::class,
        ClipVideoJob::class,
        TranscodeJob::class,
        PublishJob::class,
    ]);

Cost & Performance

  • MediaConvert: ~$0.02/min for 1080p HLS
  • FFmpeg (EC2 spot): ~$0.003/min
  • AI analysis: ~$0.10/video (GPT-4V)
  • End-to-end: ~$0.15/video, 2-3 min for 10-min source

Monitoring

Custom metrics in Laravel Pulse: queue depth, job duration, error rate, cost/video. Alerts on queue depth > 100.

Full code: GitHub.