Metadata-Version: 2.4
Name: transnetv2-pytorch
Version: 1.0.2
Summary: TransNetV2 PyTorch implementation for video scene detection
Author: TransNetV2 Contributors
License: MIT
Project-URL: Homepage, https://github.com/allenday/transnetv2_pytorch
Project-URL: Repository, https://github.com/allenday/transnetv2_pytorch
Project-URL: Issues, https://github.com/allenday/transnetv2_pytorch/issues
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Multimedia :: Video
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: ffmpeg-python
Requires-Dist: numpy
Requires-Dist: pandas
Requires-Dist: pillow
Requires-Dist: torch>=1.9.0
Requires-Dist: tqdm
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: black; extra == "dev"
Requires-Dist: flake8; extra == "dev"
Requires-Dist: mypy; extra == "dev"
Dynamic: license-file

# TransNet V2: Shot Boundary Detection Neural Network (PyTorch)

This repository contains a PyTorch implementation of [TransNet V2: An effective deep network architecture for fast shot transition detection](https://arxiv.org/abs/2008.04838).

This is a PyTorch reimplementation of the TransNetV2 model that produces identical results as the original TensorFlow version. The code is for inference only.

## Performance

Our reevaluation of other publicly available state-of-the-art shot boundary methods (F1 scores):

Model | ClipShots | BBC Planet Earth | RAI
--- | :---: | :---: | :---:
TransNet V2 | **77.9** | **96.2** | 93.9
[TransNet](https://arxiv.org/abs/1906.03363) [(github)](https://github.com/soCzech/TransNet) | 73.5 | 92.9 | **94.3**
[Hassanien et al.](https://arxiv.org/abs/1705.03281) [(github)](https://github.com/melgharib/DSBD) | 75.9 | 92.6 | 93.9
[Tang et al., ResNet baseline](https://arxiv.org/abs/1808.04234) [(github)](https://github.com/Tangshitao/ClipShots_basline) | 76.1 | 89.3 | 92.8

## Installation

```bash
pip install transnetv2-pytorch
```

Or install from source:
```bash
git clone https://github.com/allenday/transnetv2_pytorch.git
cd transnetv2_pytorch
pip install -e .
```

## Usage

### Command Line Interface

The package provides both a direct command and Python module execution:

```bash
# Direct command
transnetv2_pytorch path/to/video.mp4

# Python module execution
python -m transnetv2_pytorch path/to/video.mp4
```

#### CLI Arguments

```bash
# Basic usage
transnetv2_pytorch path/to/video.mp4

# Specify output file
transnetv2_pytorch path/to/video.mp4 --output predictions.txt

# Use specific device
transnetv2_pytorch path/to/video.mp4 --device cuda

# Get help for all options
transnetv2_pytorch --help
```

### Python API

#### Basic Usage

```python
import torch
from transnetv2_pytorch import TransNetV2

# Initialize model with automatic device detection
model = TransNetV2(device='auto')  # Automatically selects best device
model.eval()

# Load weights
state_dict = torch.load("transnetv2-pytorch-weights.pth", map_location=model.device)
model.load_state_dict(state_dict)

with torch.no_grad():
    # Basic prediction (returns raw data)
    video_frames, single_frame_pred, all_frame_pred = model.predict_video("video.mp4")
    
    # Enhanced prediction with rich scene metadata
    results = model.predict_video_with_scenes("video.mp4", threshold=0.5)
    
    print(f"Video FPS: {results['fps']}")
    print(f"Total scenes: {results['total_scenes']}")
    
    # Access rich scene data with timestamps and shot IDs
    for scene in results['scenes'][:3]:
        print(f"Shot {scene['shot_id']}: "
              f"frames {scene['start_frame']}-{scene['end_frame']} "
              f"({scene['start_time']}s-{scene['end_time']}s) "
              f"probability={scene['probability']:.4f}")
```

#### Enhanced Features for Application Developers

The TransNetV2 class now provides rich functionality previously only available in the CLI:

```python
# Automatic device detection
model = TransNetV2(device='auto')  # Chooses CUDA > MPS > CPU automatically

# Extract video FPS
fps = model.get_video_fps("video.mp4")

# Convert frame numbers to timestamps
timestamp = TransNetV2.frame_to_timestamp(frame_number=150, fps=25.0)
print(f"Frame 150 = {timestamp}s")

# Get structured scene data with metadata
scenes = model.predictions_to_scenes_with_data(
    predictions,  # numpy array or torch tensor
    fps=25.0,     # optional, can auto-extract from video
    threshold=0.5
)

# Each scene contains:
# - shot_id: Scene number (1-indexed)
# - start_frame, end_frame: Frame boundaries
# - start_time, end_time: Timestamps (if FPS available)
# - probability: Maximum probability in the scene
```

#### Advanced Usage

```python
# Custom device handling
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = TransNetV2(device=device)

# Working with existing predictions
import numpy as np
predictions = np.array([...])  # Your existing predictions

# Convert to rich scene data
scenes = model.predictions_to_scenes_with_data(
    predictions, 
    video_path="video.mp4",  # Will auto-extract FPS
    threshold=0.5
)

# Or provide FPS directly
scenes = model.predictions_to_scenes_with_data(
    predictions, 
    fps=29.97,
    threshold=0.5
)

# Comprehensive video analysis
results = model.predict_video_with_scenes("video.mp4")
# Returns: video_frames, predictions, fps, scenes, total_scenes
```

## Device Support

This implementation supports:
- **CPU**: Works on all systems
- **CUDA**: For NVIDIA GPUs
- **MPS**: For Apple Silicon Macs (automatic fallback for unsupported operations)

The model automatically detects and uses the best available device. For MPS devices, unsupported operations (like 3D convolutions) automatically fall back to CPU.

## Memory Optimization

TransNetV2 includes transparent memory optimizations that work automatically without affecting the detection algorithm:

### Automatic Memory Management

The model automatically:
- **Performs periodic memory cleanup** to prevent accumulation
- **Uses efficient tensor management** during processing
- **Applies device-specific memory optimizations** (MPS, CUDA, CPU)

```python
# Memory optimization is automatic and transparent
model = TransNetV2(device='auto')  # All optimizations work behind the scenes
```

### Handling Memory Issues

The memory optimizations are built-in and transparent. For persistent memory issues with very large videos:

1. **Reduce video resolution** before processing
2. **Split longer videos** into shorter segments  
3. **Close other memory-intensive applications**

All optimizations preserve the original algorithm parameters and accuracy!

## Original Work & Training

This PyTorch implementation is based on the original TensorFlow version. For:
- **Training code and datasets**
- **TensorFlow implementation** 
- **Weight conversion utilities**
- **Research replication**

Please visit the original repository: **[soCzech/TransNetV2](https://github.com/soCzech/TransNetV2)**

## Credits

### Original Work

This PyTorch implementation is based on the original TensorFlow TransNet V2 by Tomáš Souček and Jakub Lokoč.

If found useful, please cite the original work:

```bibtex
@article{soucek2020transnetv2,
    title={TransNet V2: An effective deep network architecture for fast shot transition detection},
    author={Sou{\v{c}}ek, Tom{\'a}{\v{s}} and Loko{\v{c}}, Jakub},
    year={2020},
    journal={arXiv preprint arXiv:2008.04838},
}
```

### PyTorch Implementation

This production-ready PyTorch package was developed by [Your Name] with significant improvements including:
- Complete PyTorch reimplementation for inference
- Cross-platform device support (CPU, CUDA, MPS)
- Command-line interface
- Package distribution and installation
- Comprehensive testing and error handling

### Related Papers

- ACM Multimedia paper of the older version: [A Framework for Effective Known-item Search in Video](https://dl.acm.org/doi/abs/10.1145/3343031.3351046)
- The older version paper: [TransNet: A deep network for fast detection of common shot transitions](https://arxiv.org/abs/1906.03363)

## License

MIT License

Original work Copyright (c) 2020 Tomáš Souček, Jakub Lokoč  
PyTorch implementation Copyright (c) 2025 Allen Day

See the original [TransNetV2 repository](https://github.com/soCzech/TransNetV2) for the original license.
