API Reference
ccBT API Reference
Comprehensive API documentation for ccBitTorrent with references to actual implementation files.
Entry Points
Main Entry Point (ccbt)
Main command-line entry point for basic torrent operations.
Implementation: ccbt/main.py:main
Features:
- Single torrent download mode
- Daemon mode for multi-torrent sessions: ccbt/main.py:52
- Magnet URI support: ccbt/main.py:73
- Tracker announcement: ccbt/main.py:89
Entry point configuration: pyproject.toml:79
Async Download Helpers
High-performance async helpers and download manager for advanced operations.
Implementation: ccbt/session/download_manager.py
Key exports:
- AsyncDownloadManager
- download_torrent()
- download_magnet()
AsyncDownloadManager
High-performance async download manager for individual torrents.
Implementation: ccbt/session/download_manager.py:AsyncDownloadManager
Methods:
- __init__(): ccbt/session/async_main.py:41 - Initialize with torrent data
- start(): ccbt/session/async_main.py:110 - Start download manager
- stop(): ccbt/session/async_main.py:115 - Stop download manager
- start_download(): ccbt/session/async_main.py:122 - Start download with peers
Features:
- Peer connection management via AsyncPeerConnectionManager: ccbt/session/async_main.py:127
- Piece management via AsyncPieceManager: ccbt/session/async_main.py:94
- Callback system for events: ccbt/session/async_main.py:103
Core Modules
TorrentParser
Parses BitTorrent torrent files and extracts metadata.
Parser for BitTorrent torrent files.
Initialize the torrent parser.
Source code in ccbt/core/torrent.py
| def __init__(self) -> None:
"""Initialize the torrent parser."""
|
get_announce_url(torrent_data: TorrentInfo) -> str
Get the announce URL for a parsed torrent.
Source code in ccbt/core/torrent.py
| def get_announce_url(self, torrent_data: TorrentInfo) -> str:
"""Get the announce URL for a parsed torrent."""
return torrent_data.announce
|
get_info_hash(torrent_data: TorrentInfo) -> bytes
Get the info hash for a parsed torrent.
Source code in ccbt/core/torrent.py
| def get_info_hash(self, torrent_data: TorrentInfo) -> bytes:
"""Get the info hash for a parsed torrent."""
return torrent_data.info_hash
|
get_num_pieces(torrent_data: TorrentInfo) -> int
Get the number of pieces for a parsed torrent.
Source code in ccbt/core/torrent.py
| def get_num_pieces(self, torrent_data: TorrentInfo) -> int:
"""Get the number of pieces for a parsed torrent."""
return torrent_data.num_pieces
|
get_piece_hash(torrent_data: TorrentInfo, piece_index: int) -> bytes
Get the SHA-1 hash for a specific piece.
Source code in ccbt/core/torrent.py
| def get_piece_hash(self, torrent_data: TorrentInfo, piece_index: int) -> bytes:
"""Get the SHA-1 hash for a specific piece."""
if piece_index < 0 or piece_index >= torrent_data.num_pieces:
msg = f"Invalid piece index: {piece_index}"
raise TorrentError(msg)
return torrent_data.pieces[piece_index]
|
get_piece_length(torrent_data: TorrentInfo) -> int
Get the piece length for a parsed torrent.
Source code in ccbt/core/torrent.py
| def get_piece_length(self, torrent_data: TorrentInfo) -> int:
"""Get the piece length for a parsed torrent."""
return torrent_data.piece_length
|
get_total_length(torrent_data: TorrentInfo) -> int
Get the total download length for a parsed torrent.
Source code in ccbt/core/torrent.py
| def get_total_length(self, torrent_data: TorrentInfo) -> int:
"""Get the total download length for a parsed torrent."""
return torrent_data.total_length
|
parse(torrent_path: str | Path) -> TorrentInfo
Parse a torrent file from a local path or URL.
Parameters:
| Name |
Type |
Description |
Default |
torrent_path
|
str | Path
|
Path to local torrent file or URL
|
required
|
Returns:
| Type |
Description |
TorrentInfo
|
TorrentInfo object containing parsed torrent data
|
Raises:
| Type |
Description |
TorrentError
|
|
Source code in ccbt/core/torrent.py
| def parse(self, torrent_path: str | Path) -> TorrentInfo:
"""Parse a torrent file from a local path or URL.
Args:
torrent_path: Path to local torrent file or URL
Returns:
TorrentInfo object containing parsed torrent data
Raises:
TorrentError: If parsing fails
"""
try:
# Read torrent data
if self._is_url(torrent_path):
torrent_data = self._read_from_url(
str(torrent_path)
) # pragma: no cover - URL torrent reading, tested via integration tests with URL torrents
else:
torrent_data = self._read_from_file(torrent_path)
# Decode bencoded data
decoded_data = decode(torrent_data)
# Validate torrent structure
self._validate_torrent(decoded_data)
# Extract and process data
return self._extract_torrent_data(decoded_data, torrent_data)
except TorrentError:
# Re-raise TorrentError as-is (validation errors)
raise
except Exception as e:
msg = f"Failed to parse torrent: {e}"
raise TorrentError(msg) from e
|
Key Methods:
Bencode Encoding/Decoding
Bencode codec for BitTorrent protocol (BEP 3).
Classes:
Functions:
- decode(): ccbt/core/bencode.py:decode - Decode bencoded bytes to Python object
- encode(): ccbt/core/bencode.py:encode - Encode Python object to bencode format
Supported Types:
- Integers: i<number>e
- Strings: <length>:<data>
- Lists: l<items>e
- Dictionaries: d<key-value pairs>e
Exceptions:
- BencodeDecodeError: ccbt/core/bencode.py:BencodeDecodeError - Decoding errors
- BencodeEncodeError: ccbt/core/bencode.py:BencodeEncodeError - Encoding errors
Magnet URI Parsing
Parses magnet URIs (BEP 9) with BEP 53 file selection support.
Functions:
- parse_magnet(): ccbt/core/magnet.py:parse_magnet - Parse magnet URI and extract components
Data Model:
- MagnetInfo: ccbt/core/magnet.py:MagnetInfo - Magnet info data model with BEP 53 support
Features:
- Info hash extraction: ccbt/core/magnet.py:_hex_or_base32_to_bytes - Supports hex (40 chars) and base32 (32 chars)
- Tracker URLs: Extracts tr parameters
- Web seeds: Extracts ws parameters
- BEP 53 file selection: ccbt/core/magnet.py:_parse_index_list - Parses so (selected) and x.pe (prioritized) parameters
- Display name: Extracts dn parameter
Helper Functions:
- build_minimal_torrent_data(): ccbt/core/magnet.py:build_minimal_torrent_data - Build minimal torrent from magnet info
- build_torrent_data_from_metadata(): ccbt/core/magnet.py:build_torrent_data_from_metadata - Build torrent from metadata exchange
Session Management
AsyncSessionManager
High-performance async session manager for multiple torrents.
High-performance async session manager for multiple torrents.
Initialize async session manager.
Source code in ccbt/session/session.py
| def __init__(self, output_dir: str = "."):
"""Initialize async session manager."""
self.config = get_config()
self.output_dir = output_dir
self.torrents: dict[bytes, AsyncTorrentSession] = {}
self.lock = asyncio.Lock()
# Global components
self.dht_client: AsyncDHTClient | None = None
self.metrics = Metrics()
self.peer_service: PeerService | None = PeerService(
max_peers=self.config.network.max_global_peers,
connection_timeout=self.config.network.connection_timeout,
)
# Background tasks
self._cleanup_task: asyncio.Task | None = None
self._metrics_task: asyncio.Task | None = None
# Callbacks
self.on_torrent_added: Callable[[bytes, str], None] | None = None
self.on_torrent_removed: Callable[[bytes], None] | None = None
self.on_torrent_complete: Callable[[bytes, str], None] | None = None
self.logger = logging.getLogger(__name__)
# Simple per-torrent rate limits (not enforced yet, stored for reporting)
self._per_torrent_limits: dict[bytes, dict[str, int]] = {}
# Optional dependency injection container
self._di: DIContainer | None = None
# Components initialized by startup functions
self.security_manager: Any | None = None
self.nat_manager: Any | None = None
self.tcp_server: Any | None = None
# CRITICAL FIX: Store reference to initialized UDP tracker client
# This ensures all torrent sessions use the same initialized socket
# The UDP tracker client is a singleton, but we store the reference
# to ensure it's accessible and to prevent any lazy initialization
self.udp_tracker_client: Any | None = None
# CRITICAL FIX: Store executor initialized at daemon startup
# This ensures executor uses the session manager's initialized components
# and prevents duplicate executor creation
self.executor: Any | None = None
# CRITICAL FIX: Store protocol manager initialized at daemon startup
# Singleton pattern removed - protocol manager is now managed via session manager
# This ensures proper lifecycle management and prevents conflicts
self.protocol_manager: Any | None = None
# CRITICAL FIX: Store WebTorrent WebSocket server initialized at daemon startup
# WebSocket server socket must be initialized once and never recreated
# This prevents port conflicts and socket recreation issues
self.webtorrent_websocket_server: Any | None = None
# CRITICAL FIX: Store WebRTC connection manager initialized at daemon startup
# WebRTC manager should be shared across all WebTorrent protocol instances
# This ensures proper resource management and prevents duplicate managers
self.webrtc_manager: Any | None = None
# CRITICAL FIX: Store uTP socket manager initialized at daemon startup
# Singleton pattern removed - uTP socket manager is now managed via session manager
# This ensures proper socket lifecycle management and prevents socket recreation
self.utp_socket_manager: Any | None = None
# CRITICAL FIX: Store extension manager initialized at daemon startup
# Singleton pattern removed - extension manager is now managed via session manager
# This ensures proper lifecycle management and prevents conflicts
self.extension_manager: Any | None = None
# CRITICAL FIX: Store disk I/O manager initialized at daemon startup
# Singleton pattern removed - disk I/O manager is now managed via session manager
# This ensures proper lifecycle management and prevents conflicts
self.disk_io_manager: Any | None = None
# Private torrents set (used by DHT client factory)
self.private_torrents: set[bytes] = set()
|
dht: Any | None
property
Get DHT instance (placeholder).
peers: list[dict[str, Any]]
property
Get list of connected peers (placeholder).
add_magnet(uri: str, resume: bool = False) -> str
async
Add a magnet link to the session.
Source code in ccbt/session/session.py
| async def add_magnet(self, uri: str, resume: bool = False) -> str:
"""Add a magnet link to the session."""
info_hash: bytes | None = None
session: AsyncTorrentSession | None = None
try:
mi = _session_mod.parse_magnet(uri)
td = _session_mod.build_minimal_torrent_data(
mi.info_hash, mi.display_name, mi.trackers
)
info_hash = td["info_hash"]
if isinstance(info_hash, str):
info_hash = bytes.fromhex(info_hash)
if not isinstance(info_hash, bytes):
error_msg = "info_hash must be bytes"
raise TypeError(error_msg)
# Check if already exists
async with self.lock:
if isinstance(info_hash, bytes) and info_hash in self.torrents:
msg = f"Magnet {info_hash.hex()} already exists"
raise ValueError(msg)
# Create session
session = AsyncTorrentSession(td, self.output_dir, self)
# Set source information for checkpoint metadata
session.magnet_uri = uri
self.torrents[info_hash] = session
# BEP 27: Track private torrents for DHT/PEX/LSD enforcement
if session.is_private:
self.private_torrents.add(info_hash)
self.logger.debug(
"Added private magnet torrent %s to private_torrents set (BEP 27)",
info_hash.hex()[:8],
)
# CRITICAL FIX: Start session with cleanup on failure
# If session.start() fails, remove the session from torrents dict
# to prevent orphaned sessions that could cause issues
try:
await session.start(resume=resume)
except Exception as start_error:
# Clean up the session from torrents dict if start failed
self.logger.exception(
"Failed to start session for magnet %s, cleaning up",
uri,
)
async with self.lock:
# Remove session from torrents dict if it's still there
if info_hash and info_hash in self.torrents:
removed_session = self.torrents.pop(info_hash, None)
if removed_session:
# Try to stop the session to clean up resources
try:
await removed_session.stop()
except Exception:
# Ignore errors during cleanup
pass
# Re-raise the original error
raise start_error
# Notify callback
if self.on_torrent_added:
await self.on_torrent_added(info_hash, session.info.name)
self.logger.info("Added magnet: %s", session.info.name)
return info_hash.hex()
except Exception:
self.logger.exception("Failed to add magnet %s", uri)
raise
|
add_torrent(path: str | dict[str, Any], resume: bool = False) -> str
async
Add a torrent file or torrent data to the session.
Source code in ccbt/session/session.py
| async def add_torrent(
self,
path: str | dict[str, Any],
resume: bool = False,
) -> str:
"""Add a torrent file or torrent data to the session."""
try:
# Handle both file paths and torrent dictionaries
if isinstance(path, dict):
td = path # Already parsed torrent data
info_hash = td.get("info_hash") if isinstance(td, dict) else None
if not info_hash:
error_msg = "Missing info_hash in torrent data"
raise ValueError(error_msg)
else:
parser = TorrentParser()
td_model = parser.parse(path)
# Accept both model objects and plain dicts from mocked parsers in tests
if isinstance(td_model, dict):
name = (
td_model.get("name")
or td_model.get("torrent_name")
or "unknown"
)
ih = td_model.get("info_hash")
if isinstance(ih, str):
ih = bytes.fromhex(ih)
if not isinstance(ih, (bytes, bytearray)):
error_msg = "info_hash must be bytes"
raise TypeError(error_msg)
td = {
"name": name,
"info_hash": bytes(ih),
"pieces_info": td_model.get("pieces_info", {}),
"file_info": td_model.get(
"file_info",
{
"total_length": td_model.get("total_length", 0),
},
),
}
else:
td = {
"name": td_model.name,
"info_hash": td_model.info_hash,
"pieces_info": {
"piece_hashes": list(td_model.pieces),
"piece_length": td_model.piece_length,
"num_pieces": td_model.num_pieces,
"total_length": td_model.total_length,
},
"file_info": {
"total_length": td_model.total_length,
},
}
info_hash = td["info_hash"]
if isinstance(info_hash, str):
info_hash = bytes.fromhex(info_hash)
if not isinstance(info_hash, bytes):
error_msg = "info_hash must be bytes"
raise TypeError(error_msg)
# Check if already exists
async with self.lock:
if isinstance(info_hash, bytes) and info_hash in self.torrents:
msg = f"Torrent {info_hash.hex()} already exists"
raise ValueError(msg)
# Create session
session = AsyncTorrentSession(td, self.output_dir, self)
# Set source information for checkpoint metadata
if isinstance(path, str):
session.torrent_file_path = path
self.torrents[info_hash] = session
# BEP 27: Track private torrents for DHT/PEX/LSD enforcement
if session.is_private:
self.private_torrents.add(info_hash)
self.logger.debug(
"Added private torrent %s to private_torrents set (BEP 27)",
info_hash.hex()[:8],
)
# Start session
await session.start(resume=resume)
# Notify callback
if self.on_torrent_added:
await self.on_torrent_added(info_hash, session.info.name)
self.logger.info("Added torrent: %s", session.info.name)
return info_hash.hex()
except Exception:
path_desc = (
getattr(path, "name", str(path)) if hasattr(path, "name") else str(path)
)
self.logger.exception("Failed to add torrent %s", path_desc)
raise
|
checkpoint_backup_torrent(info_hash_hex: str, destination: Path, compress: bool = True, encrypt: bool = False) -> bool
async
Create a checkpoint backup for a torrent to the destination path.
Source code in ccbt/session/session.py
| async def checkpoint_backup_torrent(
self,
info_hash_hex: str,
destination: Path,
compress: bool = True,
encrypt: bool = False,
) -> bool:
"""Create a checkpoint backup for a torrent to the destination path."""
try:
info_hash = bytes.fromhex(info_hash_hex)
except ValueError:
return False
try:
cp = CheckpointManager(self.config.disk)
await cp.backup_checkpoint(
info_hash,
destination,
compress=compress,
encrypt=encrypt,
)
except Exception:
return False
else:
return True
|
cleanup_completed_checkpoints() -> int
async
Remove checkpoints for completed downloads.
Source code in ccbt/session/session.py
| async def cleanup_completed_checkpoints(self) -> int:
"""Remove checkpoints for completed downloads."""
checkpoint_manager = CheckpointManager(self.config.disk)
checkpoints = await checkpoint_manager.list_checkpoints()
cleaned = 0
async def process_checkpoint(checkpoint_info):
"""Process a single checkpoint."""
try:
checkpoint = await checkpoint_manager.load_checkpoint(
checkpoint_info.info_hash,
)
if (
checkpoint
and len(checkpoint.verified_pieces) == checkpoint.total_pieces
):
# Download is complete, delete checkpoint
await checkpoint_manager.delete_checkpoint(
checkpoint_info.info_hash,
)
self.logger.info(
"Cleaned up completed checkpoint: %s",
checkpoint.torrent_name,
)
return True
except Exception as e:
self.logger.warning(
"Failed to process checkpoint %s: %s",
checkpoint_info.info_hash.hex(),
e,
)
return False
for checkpoint_info in checkpoints:
if await process_checkpoint(checkpoint_info):
cleaned += 1
return cleaned
|
export_session_state(path: Path) -> None
async
Export current session state to a JSON file.
Source code in ccbt/session/session.py
| async def export_session_state(self, path: Path) -> None:
"""Export current session state to a JSON file."""
import json
data: dict[str, Any] = {
"torrents": {},
"config": self.config.model_dump(mode="json"),
}
async with self.lock:
for ih, sess in self.torrents.items():
data["torrents"][ih.hex()] = await sess.get_status()
path.write_text(json.dumps(data, indent=2), encoding="utf-8")
|
find_checkpoint_by_name(name: str) -> TorrentCheckpoint | None
async
Find checkpoint by torrent name.
Source code in ccbt/session/session.py
| async def find_checkpoint_by_name(self, name: str) -> TorrentCheckpoint | None:
"""Find checkpoint by torrent name."""
checkpoint_manager = CheckpointManager(self.config.disk)
checkpoints = await checkpoint_manager.list_checkpoints()
for checkpoint_info in checkpoints:
try:
checkpoint = await checkpoint_manager.load_checkpoint(
checkpoint_info.info_hash,
)
if checkpoint and checkpoint.torrent_name == name:
return checkpoint
except Exception as e:
self.logger.warning(
"Failed to load checkpoint %s: %s",
checkpoint_info.info_hash.hex(),
e,
)
continue
return None
|
force_announce(info_hash_hex: str) -> bool
async
Force a tracker announce for a given torrent if possible.
Source code in ccbt/session/session.py
| async def force_announce(self, info_hash_hex: str) -> bool:
"""Force a tracker announce for a given torrent if possible."""
try:
info_hash = bytes.fromhex(info_hash_hex)
except ValueError:
return False
async with self.lock:
sess = self.torrents.get(info_hash)
if not sess:
return False
try:
td: dict[str, Any]
if isinstance(sess.torrent_data, dict):
td = sess.torrent_data # type: ignore[assignment]
else:
td = {
"info_hash": sess.info.info_hash,
"announce": "",
"name": sess.info.name,
}
await sess.tracker.announce(td)
except Exception:
return False
else:
return True
|
force_scrape(info_hash_hex: str) -> bool
async
Force tracker scrape (placeholder).
Source code in ccbt/session/session.py
| async def force_scrape(self, info_hash_hex: str) -> bool:
"""Force tracker scrape (placeholder)."""
try:
_ = bytes.fromhex(info_hash_hex)
except ValueError:
return False
return True
|
get_checkpoint_info(info_hash: bytes) -> dict[str, Any] | None
async
Get checkpoint summary information.
Source code in ccbt/session/session.py
| async def get_checkpoint_info(self, info_hash: bytes) -> dict[str, Any] | None:
"""Get checkpoint summary information."""
checkpoint_manager = CheckpointManager(self.config.disk)
checkpoint = await checkpoint_manager.load_checkpoint(info_hash)
if not checkpoint:
return None
return {
"info_hash": info_hash.hex(),
"name": checkpoint.torrent_name,
"progress": len(checkpoint.verified_pieces) / checkpoint.total_pieces
if checkpoint.total_pieces > 0
else 0,
"verified_pieces": len(checkpoint.verified_pieces),
"total_pieces": checkpoint.total_pieces,
"total_size": checkpoint.total_length,
"created_at": checkpoint.created_at,
"updated_at": checkpoint.updated_at,
"can_resume": bool(checkpoint.torrent_file_path or checkpoint.magnet_uri),
"torrent_file_path": checkpoint.torrent_file_path,
"magnet_uri": checkpoint.magnet_uri,
}
|
get_global_stats() -> dict[str, Any]
async
Aggregate global statistics across all torrents.
Source code in ccbt/session/session.py
| async def get_global_stats(self) -> dict[str, Any]:
"""Aggregate global statistics across all torrents."""
stats: dict[str, Any] = {
"num_torrents": 0,
"num_active": 0,
"num_paused": 0,
"num_seeding": 0,
"download_rate": 0.0,
"upload_rate": 0.0,
"average_progress": 0.0,
}
aggregate_progress = 0.0
async with self.lock:
stats["num_torrents"] = len(self.torrents)
for sess in self.torrents.values():
st = await sess.get_status()
s = st.get("status", "")
if s == "paused":
stats["num_paused"] += 1
elif s == "seeding":
stats["num_seeding"] += 1
else:
stats["num_active"] += 1
stats["download_rate"] += float(st.get("download_rate", 0.0))
stats["upload_rate"] += float(st.get("upload_rate", 0.0))
aggregate_progress += float(st.get("progress", 0.0))
if stats["num_torrents"]:
stats["average_progress"] = aggregate_progress / stats["num_torrents"]
return stats
|
get_peers_for_torrent(info_hash_hex: str) -> list[dict[str, Any]]
async
Return list of peers for a torrent (placeholder).
Returns an empty list until peer tracking is wired.
Source code in ccbt/session/session.py
| async def get_peers_for_torrent(self, info_hash_hex: str) -> list[dict[str, Any]]:
"""Return list of peers for a torrent (placeholder).
Returns an empty list until peer tracking is wired.
"""
try:
_ = bytes.fromhex(info_hash_hex)
except ValueError:
return []
if not self.peer_service:
return []
try:
peers = await self.peer_service.list_peers()
return [
{
"ip": p.peer_info.ip,
"port": p.peer_info.port,
"download_rate": 0.0,
"upload_rate": 0.0,
"choked": False,
"client": "?",
}
for p in peers
]
except Exception:
return []
|
get_session_for_info_hash(info_hash: bytes) -> AsyncTorrentSession | None
async
Get torrent session by info hash.
Parameters:
| Name |
Type |
Description |
Default |
info_hash
|
bytes
|
Torrent info hash (20 bytes)
|
required
|
Returns:
| Type |
Description |
AsyncTorrentSession | None
|
AsyncTorrentSession instance if found, None otherwise
|
Source code in ccbt/session/session.py
| async def get_session_for_info_hash(
self, info_hash: bytes
) -> AsyncTorrentSession | None:
"""Get torrent session by info hash.
Args:
info_hash: Torrent info hash (20 bytes)
Returns:
AsyncTorrentSession instance if found, None otherwise
"""
async with self.lock:
return self.torrents.get(info_hash)
|
get_status() -> dict[str, Any]
async
Get status of all torrents.
Source code in ccbt/session/session.py
| async def get_status(self) -> dict[str, Any]:
"""Get status of all torrents."""
status = {}
async with self.lock:
for info_hash, session in self.torrents.items():
status[info_hash.hex()] = await session.get_status()
return status
|
get_torrent_status(info_hash_hex: str) -> dict[str, Any] | None
async
Get status of a specific torrent.
Source code in ccbt/session/session.py
| async def get_torrent_status(self, info_hash_hex: str) -> dict[str, Any] | None:
"""Get status of a specific torrent."""
try:
info_hash = bytes.fromhex(info_hash_hex)
except ValueError:
return None
async with self.lock:
session = self.torrents.get(info_hash)
if session:
return await session.get_status()
return None
|
import_session_state(path: Path) -> dict[str, Any]
async
Import session state from a JSON file. Returns the parsed state.
This does not automatically start torrents.
Source code in ccbt/session/session.py
| async def import_session_state(self, path: Path) -> dict[str, Any]:
"""Import session state from a JSON file. Returns the parsed state.
This does not automatically start torrents.
"""
import json
return json.loads(path.read_text(encoding="utf-8"))
|
list_resumable_checkpoints() -> list[TorrentCheckpoint]
async
List all checkpoints that can be auto-resumed.
Source code in ccbt/session/session.py
| async def list_resumable_checkpoints(self) -> list[TorrentCheckpoint]:
"""List all checkpoints that can be auto-resumed."""
checkpoint_manager = CheckpointManager(self.config.disk)
checkpoints = await checkpoint_manager.list_checkpoints()
resumable = []
for checkpoint_info in checkpoints:
try:
checkpoint = await checkpoint_manager.load_checkpoint(
checkpoint_info.info_hash,
)
if checkpoint and (
checkpoint.torrent_file_path or checkpoint.magnet_uri
):
resumable.append(checkpoint)
except Exception as e:
self.logger.warning(
"Failed to load checkpoint %s: %s",
checkpoint_info.info_hash.hex(),
e,
)
continue
return resumable
|
load_torrent(torrent_path: str | Path) -> dict[str, Any] | None
Load torrent file and return parsed data.
Source code in ccbt/session/session.py
| def load_torrent(self, torrent_path: str | Path) -> dict[str, Any] | None:
"""Load torrent file and return parsed data."""
try:
parser = TorrentParser()
tdm = parser.parse(str(torrent_path))
return {
"name": tdm.name,
"info_hash": tdm.info_hash,
"pieces_info": {
"piece_hashes": list(tdm.pieces),
"piece_length": tdm.piece_length,
"num_pieces": tdm.num_pieces,
"total_length": tdm.total_length,
},
"file_info": {
"total_length": tdm.total_length,
},
"announce": getattr(tdm, "announce", ""),
}
except Exception:
self.logger.exception("Failed to load torrent %s", torrent_path)
return None
|
parse_magnet_link(magnet_uri: str) -> dict[str, Any] | None
Parse magnet link and return torrent data.
Source code in ccbt/session/session.py
| def parse_magnet_link(self, magnet_uri: str) -> dict[str, Any] | None:
"""Parse magnet link and return torrent data."""
try:
magnet_info = parse_magnet(magnet_uri)
return build_minimal_torrent_data(
magnet_info.info_hash,
magnet_info.display_name,
magnet_info.trackers,
)
except Exception:
self.logger.exception("Failed to parse magnet link")
return None
|
pause_torrent(info_hash_hex: str) -> bool
async
Pause a torrent download by info hash.
Returns True if paused, False otherwise.
Source code in ccbt/session/session.py
| async def pause_torrent(self, info_hash_hex: str) -> bool:
"""Pause a torrent download by info hash.
Returns True if paused, False otherwise.
"""
try:
info_hash = bytes.fromhex(info_hash_hex)
except ValueError:
return False
async with self.lock:
session = self.torrents.get(info_hash)
if not session:
return False
await session.pause()
return True
|
refresh_pex(info_hash_hex: str) -> bool
async
Refresh Peer Exchange (placeholder).
Source code in ccbt/session/session.py
| async def refresh_pex(self, info_hash_hex: str) -> bool:
"""Refresh Peer Exchange (placeholder)."""
try:
info_hash = bytes.fromhex(info_hash_hex)
except ValueError:
return False
async with self.lock:
sess = self.torrents.get(info_hash)
if not sess or not sess.pex_manager:
return False
try:
if hasattr(sess.pex_manager, "refresh"):
await sess.pex_manager.refresh() # type: ignore[attr-defined]
except Exception:
return False
else:
return True
|
rehash_torrent(info_hash_hex: str) -> bool
async
Rehash all pieces.
Source code in ccbt/session/session.py
| async def rehash_torrent(self, info_hash_hex: str) -> bool:
"""Rehash all pieces."""
try:
info_hash = bytes.fromhex(info_hash_hex)
except ValueError:
return False
# Integrate with piece manager to re-verify data
torrent = self.torrents.get(info_hash)
if (
torrent
and torrent.piece_manager
and hasattr(torrent.piece_manager, "verify_all_pieces")
):
verify_method = cast(
"Callable[[], Awaitable[bool]] | Callable[[], bool]",
torrent.piece_manager.verify_all_pieces,
)
if asyncio.iscoroutinefunction(verify_method):
return await cast("Callable[[], Awaitable[bool]]", verify_method)()
return cast("Callable[[], bool]", verify_method)()
return False
|
reload_config(new_config: Any) -> None
async
Reload configuration and update affected components.
Parameters:
| Name |
Type |
Description |
Default |
new_config
|
Any
|
New Config instance to apply
|
required
|
Source code in ccbt/session/session.py
| async def reload_config(self, new_config: Any) -> None:
"""Reload configuration and update affected components.
Args:
new_config: New Config instance to apply
"""
old_config = self.config
self.config = new_config
reloaded_components = []
try:
# Reload security manager if IP filters changed
if (
old_config.security.ip_filter.filter_files
!= new_config.security.ip_filter.filter_files
or old_config.security.ip_filter.enable_ip_filter
!= new_config.security.ip_filter.enable_ip_filter
) and self.security_manager:
try:
await self.security_manager.load_ip_filter(new_config)
reloaded_components.append("security_manager")
self.logger.info("Reloaded security manager with new IP filters")
except Exception as e:
self.logger.warning("Failed to reload security manager: %s", e)
# Reload DHT client if DHT config changed
dht_config_changed = (
old_config.discovery.enable_dht != new_config.discovery.enable_dht
or old_config.discovery.dht_port != new_config.discovery.dht_port
)
if dht_config_changed:
# Stop existing DHT client
if self.dht_client:
try:
await self.dht_client.stop()
self.dht_client = None
reloaded_components.append("dht_client (stopped)")
except Exception as e:
self.logger.warning("Failed to stop DHT client: %s", e)
# Start new DHT client if enabled
if new_config.discovery.enable_dht:
from ccbt.session.manager_startup import start_dht
try:
await start_dht(self)
reloaded_components.append("dht_client (started)")
self.logger.info("Reloaded DHT client")
except Exception as e:
self.logger.warning("Failed to start DHT client: %s", e)
# Reload NAT manager if NAT config changed
nat_config_changed = (
old_config.nat.auto_map_ports != new_config.nat.auto_map_ports
or old_config.nat.enable_nat_pmp != new_config.nat.enable_nat_pmp
or old_config.nat.enable_upnp != new_config.nat.enable_upnp
)
if nat_config_changed:
# Stop existing NAT manager
if self.nat_manager:
try:
await self.nat_manager.stop()
self.nat_manager = None
reloaded_components.append("nat_manager (stopped)")
except Exception as e:
self.logger.warning("Failed to stop NAT manager: %s", e)
# Start new NAT manager if enabled
if new_config.nat.auto_map_ports:
from ccbt.session.manager_startup import start_nat
try:
await start_nat(self)
reloaded_components.append("nat_manager (started)")
self.logger.info("Reloaded NAT manager")
except Exception as e:
self.logger.warning("Failed to start NAT manager: %s", e)
# Reload peer service if peer limits changed
peer_config_changed = (
old_config.network.max_global_peers
!= new_config.network.max_global_peers
or old_config.network.connection_timeout
!= new_config.network.connection_timeout
)
if peer_config_changed and self.peer_service:
try:
# Update peer service config
self.peer_service.max_peers = new_config.network.max_global_peers
self.peer_service.connection_timeout = (
new_config.network.connection_timeout
)
reloaded_components.append("peer_service")
self.logger.info("Reloaded peer service configuration")
except Exception as e:
self.logger.warning("Failed to reload peer service: %s", e)
# Reload TCP server if listen port changed
tcp_config_changed = (
old_config.network.listen_port != new_config.network.listen_port
or old_config.network.enable_tcp != new_config.network.enable_tcp
)
if tcp_config_changed:
# Stop existing TCP server
if hasattr(self, "tcp_server") and self.tcp_server:
try:
await self.tcp_server.stop()
self.tcp_server = None
reloaded_components.append("tcp_server (stopped)")
except Exception as e:
self.logger.warning("Failed to stop TCP server: %s", e)
# Start new TCP server if enabled
if new_config.network.enable_tcp:
from ccbt.session.manager_startup import start_tcp_server
try:
await start_tcp_server(self)
reloaded_components.append("tcp_server (started)")
self.logger.info("Reloaded TCP server")
except Exception as e:
self.logger.warning("Failed to start TCP server: %s", e)
if reloaded_components:
self.logger.info(
"Configuration reloaded successfully. Components reloaded: %s",
", ".join(reloaded_components),
)
else:
self.logger.info("Configuration updated (no component reloads needed)")
except Exception:
self.logger.exception("Error during config reload")
# Revert to old config on critical error
self.config = old_config
raise
|
remove(info_hash_hex: str) -> bool
async
Remove a torrent from the session.
Source code in ccbt/session/session.py
| async def remove(self, info_hash_hex: str) -> bool:
"""Remove a torrent from the session."""
try:
info_hash = bytes.fromhex(info_hash_hex)
except ValueError:
return False
async with self.lock:
session = self.torrents.pop(info_hash, None)
# BEP 27: Remove from private_torrents set when torrent is removed
self.private_torrents.discard(info_hash)
if session:
await session.stop()
# Notify callback
if self.on_torrent_removed:
await self.on_torrent_removed(info_hash)
self.logger.info("Removed torrent: %s", session.info.name)
return True
return False
|
resume_from_checkpoint(info_hash: bytes, checkpoint: TorrentCheckpoint, torrent_path: str | None = None) -> str
async
Resume download from checkpoint.
Parameters:
| Name |
Type |
Description |
Default |
info_hash
|
bytes
|
|
required
|
checkpoint
|
TorrentCheckpoint
|
|
required
|
torrent_path
|
str | None
|
Optional explicit torrent file path
|
None
|
Returns:
| Type |
Description |
str
|
Info hash hex string of resumed torrent
|
Raises:
| Type |
Description |
ValueError
|
If torrent source cannot be determined
|
FileNotFoundError
|
If torrent file doesn't exist
|
ValidationError
|
|
Source code in ccbt/session/session.py
| async def resume_from_checkpoint(
self,
info_hash: bytes,
checkpoint: TorrentCheckpoint,
torrent_path: str | None = None,
) -> str:
"""Resume download from checkpoint.
Args:
info_hash: Torrent info hash
checkpoint: Checkpoint data
torrent_path: Optional explicit torrent file path
Returns:
Info hash hex string of resumed torrent
Raises:
ValueError: If torrent source cannot be determined
FileNotFoundError: If torrent file doesn't exist
ValidationError: If checkpoint is invalid
"""
try:
# Validate checkpoint
if not await self.validate_checkpoint(checkpoint):
error_msg = "Invalid checkpoint data"
raise ValidationError(error_msg)
# Priority order: explicit path -> stored file path -> magnet URI
torrent_source = None
source_type = None
if torrent_path and Path(torrent_path).exists():
torrent_source = torrent_path
source_type = "file"
self.logger.info("Using explicit torrent file: %s", torrent_path)
elif (
checkpoint.torrent_file_path
and Path(checkpoint.torrent_file_path).exists()
):
torrent_source = checkpoint.torrent_file_path
source_type = "file"
self.logger.info(
"Using stored torrent file: %s",
checkpoint.torrent_file_path,
)
elif checkpoint.magnet_uri:
torrent_source = checkpoint.magnet_uri
source_type = "magnet"
self.logger.info("Using stored magnet URI: %s", checkpoint.magnet_uri)
else:
error_msg = (
f"Cannot resume torrent {info_hash.hex()}: "
"No valid torrent source found in checkpoint. "
"Please provide torrent file or magnet link."
)
raise ValueError(error_msg)
# Validate info hash matches if using explicit torrent file
if source_type == "file" and torrent_source:
parser = TorrentParser()
torrent_data_model = parser.parse(torrent_source)
if isinstance(torrent_data_model, dict):
torrent_info_hash = torrent_data_model.get("info_hash")
else:
torrent_info_hash = getattr(torrent_data_model, "info_hash", None)
torrent_data = {
"info_hash": torrent_info_hash,
}
if torrent_data["info_hash"] != info_hash:
torrent_hash_hex = (
torrent_data["info_hash"].hex()
if torrent_data["info_hash"] is not None
else "None"
)
error_msg = (
f"Info hash mismatch: checkpoint is for {info_hash.hex()}, "
f"but torrent file is for {torrent_hash_hex}"
)
raise ValueError(error_msg)
# Add torrent/magnet with resume=True
if source_type == "file":
return await self.add_torrent(torrent_source, resume=True)
return await self.add_magnet(torrent_source, resume=True)
except Exception:
self.logger.exception("Failed to resume from checkpoint")
raise
|
resume_torrent(info_hash_hex: str) -> bool
async
Resume a paused torrent by info hash.
Source code in ccbt/session/session.py
| async def resume_torrent(self, info_hash_hex: str) -> bool:
"""Resume a paused torrent by info hash."""
try:
info_hash = bytes.fromhex(info_hash_hex)
except ValueError:
return False
async with self.lock:
session = self.torrents.get(info_hash)
if not session:
return False
await session.resume()
return True
|
set_rate_limits(info_hash_hex: str, download_kib: int, upload_kib: int) -> bool
async
Set per-torrent rate limits (stored for reporting).
Currently not enforced at I/O level, but stored for future enforcement
and reporting purposes.
Parameters:
| Name |
Type |
Description |
Default |
info_hash_hex
|
str
|
Torrent info hash (hex string)
|
required
|
download_kib
|
int
|
Download limit in KiB/s (0 = unlimited)
|
required
|
upload_kib
|
int
|
Upload limit in KiB/s (0 = unlimited)
|
required
|
Returns:
| Type |
Description |
bool
|
True if limits were set, False if torrent not found
|
Note
Per-torrent limits should not exceed global limits. Validation
is performed to ensure compliance.
Source code in ccbt/session/session.py
| async def set_rate_limits(
self,
info_hash_hex: str,
download_kib: int,
upload_kib: int,
) -> bool:
"""Set per-torrent rate limits (stored for reporting).
Currently not enforced at I/O level, but stored for future enforcement
and reporting purposes.
Args:
info_hash_hex: Torrent info hash (hex string)
download_kib: Download limit in KiB/s (0 = unlimited)
upload_kib: Upload limit in KiB/s (0 = unlimited)
Returns:
True if limits were set, False if torrent not found
Note:
Per-torrent limits should not exceed global limits. Validation
is performed to ensure compliance.
"""
try:
info_hash = bytes.fromhex(info_hash_hex)
except ValueError:
return False
# Validate per-torrent limits against global limits
global_down = self.config.limits.global_down_kib
global_up = self.config.limits.global_up_kib
if download_kib > 0 and global_down > 0 and download_kib > global_down:
self.logger.warning(
"Per-torrent download limit %d KiB/s exceeds global limit %d KiB/s, "
"capping to global limit",
download_kib,
global_down,
)
download_kib = global_down
if upload_kib > 0 and global_up > 0 and upload_kib > global_up:
self.logger.warning(
"Per-torrent upload limit %d KiB/s exceeds global limit %d KiB/s, "
"capping to global limit",
upload_kib,
global_up,
)
upload_kib = global_up
async with self.lock:
if info_hash not in self.torrents:
return False
self._per_torrent_limits[info_hash] = {
"down_kib": max(0, int(download_kib)),
"up_kib": max(0, int(upload_kib)),
}
self.logger.debug(
"Set per-torrent rate limits for %s: down=%d KiB/s, up=%d KiB/s",
info_hash_hex[:8],
download_kib,
upload_kib,
)
return True
|
start() -> None
async
Start the async session manager.
Startup order:
1. NAT manager:
a. Create NAT manager
b. UPnP/NAT-PMP discovery (MUST complete first)
c. Port mapping (only after discovery completes)
2. TCP server (waits for NAT port mapping to complete)
3. UDP tracker client (waits for NAT port mapping to complete)
4. DHT client (waits for NAT port mapping to complete, especially DHT UDP port)
5. Security manager (before peer service - used for IP filtering)
6. Peer service (after NAT, TCP server, DHT, and security manager are ready)
7. Queue manager (if enabled - manages torrent priorities)
8. Background tasks
Source code in ccbt/session/session.py
| async def start(self) -> None:
"""Start the async session manager.
Startup order:
1. NAT manager:
a. Create NAT manager
b. UPnP/NAT-PMP discovery (MUST complete first)
c. Port mapping (only after discovery completes)
2. TCP server (waits for NAT port mapping to complete)
3. UDP tracker client (waits for NAT port mapping to complete)
4. DHT client (waits for NAT port mapping to complete, especially DHT UDP port)
5. Security manager (before peer service - used for IP filtering)
6. Peer service (after NAT, TCP server, DHT, and security manager are ready)
7. Queue manager (if enabled - manages torrent priorities)
8. Background tasks
"""
from ccbt.session.manager_startup import start_nat, start_tcp_server
# CRITICAL: Start NAT manager first (UPnP/NAT-PMP discovery and port mapping)
# This must happen before services that need incoming connections
try:
await start_nat(self)
except Exception:
# Best-effort: log and continue
self.logger.warning(
"NAT manager initialization failed. Port mapping may not work, which could prevent incoming connections.",
exc_info=True,
)
# Start TCP server for incoming peer connections if enabled
# TCP server waits for NAT port mapping to complete before starting
try:
await start_tcp_server(self)
except Exception:
# Best-effort: log and continue
self.logger.warning(
"TCP server initialization failed. Incoming peer connections may not work.",
exc_info=True,
)
# CRITICAL FIX: Initialize UDP tracker client during daemon startup
# This ensures the socket is created once and never recreated, preventing
# daemon/executor sync issues. The socket must be ready before the executor
# can use it, so initialize it here rather than lazily.
try:
from ccbt.session.manager_startup import start_udp_tracker_client
await start_udp_tracker_client(self)
except Exception:
# Best-effort: log and continue
self.logger.warning(
"UDP tracker client initialization failed. UDP tracker operations may not work.",
exc_info=True,
)
# Start DHT client if enabled (after NAT for better connectivity)
# CRITICAL FIX: Use start_dht() which properly waits for NAT port mapping
# and uses the correct factory method with proper bind_ip/bind_port
if self.config.discovery.enable_dht:
from ccbt.session.manager_startup import start_dht
try:
await start_dht(self)
except Exception:
# Best-effort: log and continue
self.logger.warning(
"DHT client initialization failed. DHT peer discovery may not work.",
exc_info=True,
)
# CRITICAL FIX: Initialize security manager early (before peer service)
# Security manager is used by peer managers for IP filtering and validation
# It should be initialized during daemon startup to ensure it's ready before
# any peer connections are established
try:
from ccbt.session.manager_startup import start_security_manager
await start_security_manager(self)
except Exception:
# Best-effort: log and continue
self.logger.warning(
"Security manager initialization failed. IP filtering and peer validation may not work.",
exc_info=True,
)
# Start peer service (after NAT, TCP server, and security manager are ready)
try:
from ccbt.session.manager_startup import start_peer_service
await start_peer_service(self)
except Exception:
# Best-effort: log and continue
self.logger.debug("Peer service start failed", exc_info=True)
# CRITICAL FIX: Initialize queue manager if enabled
# Queue manager manages torrent priorities and bandwidth allocation
# It should be initialized during daemon startup to ensure it's ready before
# any torrents are added or managed
try:
from ccbt.session.manager_startup import start_queue_manager
await start_queue_manager(self)
except Exception:
# Best-effort: log and continue
self.logger.warning(
"Queue manager initialization failed. Queue management may not work.",
exc_info=True,
)
# CRITICAL FIX: Initialize executor after all components are ready
# This ensures executor has access to all initialized components (UDP tracker, DHT, etc.)
# The executor will be used by IPC server and other components
# Use ExecutorManager to ensure single executor instance per session manager
try:
from ccbt.executor.manager import ExecutorManager
executor_manager = ExecutorManager.get_instance()
self.executor = executor_manager.get_executor(session_manager=self)
# CRITICAL FIX: Verify executor is properly initialized
if not hasattr(self.executor, "adapter") or self.executor.adapter is None:
raise RuntimeError("Executor adapter not initialized")
if (
not hasattr(self.executor.adapter, "session_manager")
or self.executor.adapter.session_manager is None
):
raise RuntimeError("Executor session_manager not initialized")
if self.executor.adapter.session_manager is not self:
raise RuntimeError("Executor session_manager reference mismatch")
self.logger.info(
"Command executor initialized successfully via ExecutorManager (adapter=%s, session_manager=%s)",
type(self.executor.adapter).__name__,
type(self.executor.adapter.session_manager).__name__,
)
except Exception as e:
self.logger.warning(
"Failed to initialize command executor: %s. "
"Some operations may not work correctly.",
e,
exc_info=True,
)
# Don't fail startup - executor may not be needed in all scenarios
self.executor = None
# CRITICAL FIX: Initialize disk I/O manager at daemon startup
# Singleton pattern removed - disk I/O manager is now managed via session manager
try:
from ccbt.config.config import get_config
from ccbt.storage.disk_io import DiskIOManager
config = get_config()
disk_io_manager = DiskIOManager(
max_workers=config.disk.disk_workers,
queue_size=config.disk.disk_queue_size,
cache_size_mb=getattr(config.disk, "cache_size_mb", 256),
)
await disk_io_manager.start()
self.disk_io_manager = disk_io_manager
self.logger.info(
"Disk I/O manager initialized successfully (workers: %d, queue_size: %d, cache_size_mb: %d)",
disk_io_manager.max_workers,
disk_io_manager.queue_size,
disk_io_manager.cache_size_mb,
)
except Exception as e:
self.logger.warning(
"Failed to initialize disk I/O manager: %s. "
"Disk operations may not work correctly.",
e,
exc_info=True,
)
# Don't fail startup - disk I/O may not be needed in all scenarios
self.disk_io_manager = None
# CRITICAL FIX: Initialize extension manager at daemon startup
# Singleton pattern removed - extension manager is now managed via session manager
try:
from ccbt.extensions.manager import ExtensionManager
self.extension_manager = ExtensionManager()
await self.extension_manager.start()
self.logger.info("Extension manager initialized successfully")
except Exception as e:
self.logger.warning(
"Failed to initialize extension manager: %s. "
"BitTorrent extensions may not work correctly.",
e,
exc_info=True,
)
# Don't fail startup - extensions may not be needed in all scenarios
self.extension_manager = None
# CRITICAL FIX: Initialize protocol manager at daemon startup
# Singleton pattern removed - protocol manager is now managed via session manager
try:
from ccbt.protocols.base import ProtocolManager
self.protocol_manager = ProtocolManager()
self.logger.info("Protocol manager initialized successfully")
except Exception as e:
self.logger.warning(
"Failed to initialize protocol manager: %s. "
"Protocol operations may not work correctly.",
e,
exc_info=True,
)
# Don't fail startup - protocol manager may not be needed in all scenarios
self.protocol_manager = None
# CRITICAL FIX: Initialize WebTorrent components at daemon startup if enabled
# This ensures WebSocket server and WebRTC manager are initialized once
if self.config.network.webtorrent.enable_webtorrent:
try:
from ccbt.session.manager_startup import start_webtorrent_components
await start_webtorrent_components(self)
except Exception as e:
self.logger.warning(
"Failed to initialize WebTorrent components: %s. "
"WebTorrent operations may not work correctly.",
e,
exc_info=True,
)
# Start background tasks
self._cleanup_task = asyncio.create_task(self._cleanup_loop())
self._metrics_task = asyncio.create_task(self._metrics_loop())
self.logger.info("Async session manager started")
|
start_web_interface(host: str = 'localhost', port: int = 9090) -> None
async
Start web interface (placeholder implementation).
Source code in ccbt/session/session.py
| async def start_web_interface(
self,
host: str = "localhost",
port: int = 9090,
) -> None:
"""Start web interface (placeholder implementation)."""
self.logger.info("Web interface would start on http://%s:%s", host, port)
# TODO: Implement actual web interface
await asyncio.sleep(1) # Placeholder to prevent immediate exit
|
stop() -> None
async
Stop the async session manager.
Source code in ccbt/session/session.py
| async def stop(self) -> None:
"""Stop the async session manager."""
# Clean up executor via ExecutorManager
if self.executor:
try:
from ccbt.executor.manager import ExecutorManager
executor_manager = ExecutorManager.get_instance()
executor_manager.remove_executor(session_manager=self)
self.executor = None
self.logger.debug("Removed executor from ExecutorManager")
except Exception as e:
self.logger.debug("Error removing executor: %s", e, exc_info=True)
# Stop all torrents
async with self.lock:
for session in self.torrents.values():
await session.stop()
self.torrents.clear()
# Stop background tasks
if self._cleanup_task:
self._cleanup_task.cancel()
if self._metrics_task:
self._metrics_task.cancel()
# Stop TCP server (releases TCP port)
if self.tcp_server:
try:
await self.tcp_server.stop()
self.logger.debug("TCP server stopped (port released)")
except Exception as e:
self.logger.debug("Error stopping TCP server: %s", e, exc_info=True)
# Stop UDP tracker client (releases UDP tracker port)
if self.udp_tracker_client:
try:
await self.udp_tracker_client.stop()
self.logger.debug("UDP tracker client stopped (port released)")
except Exception as e:
self.logger.debug(
"Error stopping UDP tracker client: %s", e, exc_info=True
)
# Stop DHT client (releases DHT UDP port)
if self.dht_client:
try:
await self.dht_client.stop()
self.logger.debug("DHT client stopped (port released)")
except Exception as e:
self.logger.debug("Error stopping DHT client: %s", e, exc_info=True)
# Stop NAT manager (unmaps all ports)
if self.nat_manager:
try:
await self.nat_manager.stop()
self.logger.debug("NAT manager stopped (ports unmapped)")
except Exception as e:
self.logger.debug("Error stopping NAT manager: %s", e, exc_info=True)
# Stop peer service
try:
if self.peer_service:
await self.peer_service.stop()
except Exception:
# Best-effort: log and continue
self.logger.debug("Peer service stop failed", exc_info=True)
self.logger.info("Async session manager stopped (all ports released)")
|
validate_checkpoint(checkpoint: TorrentCheckpoint) -> bool
async
Validate checkpoint integrity.
Source code in ccbt/session/session.py
| async def validate_checkpoint(self, checkpoint: TorrentCheckpoint) -> bool:
"""Validate checkpoint integrity."""
try:
# Basic validation
if (
not checkpoint.info_hash
or len(checkpoint.info_hash) != INFO_HASH_LENGTH
):
return False
if checkpoint.total_pieces <= 0 or checkpoint.piece_length <= 0:
return False
if checkpoint.total_length <= 0:
return False
# Validate verified pieces are within bounds
for piece_idx in checkpoint.verified_pieces:
if piece_idx < 0 or piece_idx >= checkpoint.total_pieces:
return False
# Validate piece states
for piece_idx, state in checkpoint.piece_states.items():
if piece_idx < 0 or piece_idx >= checkpoint.total_pieces:
return False
if not isinstance(state, PieceState):
return False
except Exception:
return False
else:
return True
|
Initialization
Constructor: ccbt/session/session.py:608
response = await self.tracker.announce(td)
if (
response.peers
and self.download_manager
and hasattr(self.download_manager, "add_peers")
and callable(self.download_manager.add_peers)
):
# Update peer list in download manager
add_peers_method = cast(
"Callable[[Any], Any] | Callable[[Any], Awaitable[Any]]",
self.download_manager.add_peers,
Lifecycle Methods
-
start(): ccbt/session/session.py:637 - Start the async session manager
def _collect_trackers(self, td: dict[str, Any]) -> list[str]:
"""Collect and deduplicate tracker URLs from torrent_data.
Args:
td: Torrent data dictionary
Returns:
List of unique tracker URLs
"""
urls: list[str] = []
# BEP 12 tiers or flat list from magnet parsing
announce_list = td.get("announce_list")
if isinstance(announce_list, list):
for item in announce_list:
if isinstance(item, list):
urls.extend([u for u in item if isinstance(u, str)])
-
stop(): ccbt/session/session.py:657 - Stop the async session manager
urls.append(item)
# Additional trackers key (magnet parsing)
trackers = td.get("trackers")
if isinstance(trackers, list):
urls.extend([u for u in trackers if isinstance(u, str)])
# Fallback to single announce
announce = td.get("announce")
if isinstance(announce, str) and announce.strip():
urls.append(announce.strip())
# Deduplicate, basic validation
seen: set[str] = set()
unique: list[str] = []
for u in urls:
if not isinstance(u, str):
continue
v = u.strip()
if v and v not in seen:
seen.add(v)
unique.append(v)
return unique
async def _status_loop(self) -> None:
Torrent Management
Status and Monitoring
Advanced Operations
AsyncTorrentSession
Individual torrent session representing one active torrent's lifecycle with async operations.
Represents one active torrent's lifecycle with async operations.
Initialize TorrentSession with torrent data and output directory.
Source code in ccbt/session/session.py
| def __init__(
self,
torrent_data: dict[str, Any] | TorrentInfoModel,
output_dir: str | Path = ".",
session_manager: AsyncSessionManager | None = None,
) -> None:
"""Initialize TorrentSession with torrent data and output directory."""
self.config = get_config()
self.torrent_data = torrent_data
self.output_dir = Path(output_dir)
self.session_manager = session_manager
# Core components
self.download_manager = AsyncDownloadManager(torrent_data, str(output_dir))
# Create a proper piece manager for checkpoint operations
from ccbt.piece.async_piece_manager import AsyncPieceManager
self._normalized_td = self._normalize_torrent_data(torrent_data)
self.piece_manager = AsyncPieceManager(self._normalized_td)
# Set the piece manager on the download manager for compatibility
self.download_manager.piece_manager = self.piece_manager
# CRITICAL FIX: Pass session_manager to AsyncTrackerClient
# This ensures it uses the daemon's initialized UDP tracker client
# instead of creating a new one, preventing WinError 10048
self.tracker = AsyncTrackerClient()
# Store session_manager reference so tracker can use initialized UDP client
if session_manager:
self.tracker._session_manager = session_manager # type: ignore[attr-defined]
self.pex_manager: PEXManager | None = None
self.checkpoint_manager = CheckpointManager(self.config.disk)
# Session state
if isinstance(torrent_data, TorrentInfoModel):
name = torrent_data.name
info_hash = torrent_data.info_hash
else:
name = torrent_data.get("name") or torrent_data.get("file_info", {}).get(
"name",
"Unknown",
)
info_hash = torrent_data["info_hash"]
self.info = TorrentSessionInfo(
info_hash=info_hash,
name=name,
output_dir=str(output_dir),
added_time=time.time(),
)
# Source tracking for checkpoint metadata
self.torrent_file_path: str | None = None
self.magnet_uri: str | None = None
# Background tasks
self._announce_task: asyncio.Task[None] | None = None
self._status_task: asyncio.Task[None] | None = None
self._checkpoint_task: asyncio.Task[None] | None = None
self._stop_event = asyncio.Event()
# Checkpoint state
self.checkpoint_loaded = False
self.resume_from_checkpoint = False
# Callbacks
self.on_status_update: Callable[[dict[str, Any]], None] | None = None
self.on_complete: Callable[[], None] | None = None
self.on_error: Callable[[Exception], None] | None = None
self.logger = get_logger(__name__)
# Extract is_private flag for DHT discovery
if isinstance(torrent_data, dict):
self.is_private = torrent_data.get("is_private", False)
elif hasattr(torrent_data, "is_private"):
self.is_private = getattr(torrent_data, "is_private", False)
else:
self.is_private = False
# Per-torrent configuration options (overrides global config for this torrent)
# These are set via UI or API and applied during session.start()
self.options: dict[str, Any] = {}
|
download_rate: float
property
downloaded_bytes: int
property
info_hash_hex: str
property
Get info hash as hex string.
peers: dict[str, Any]
property
upload_rate: float
property
uploaded_bytes: int
property
delete_checkpoint() -> bool
async
Delete checkpoint files for this torrent.
Source code in ccbt/session/session.py
| async def delete_checkpoint(self) -> bool:
"""Delete checkpoint files for this torrent."""
try:
return await self.checkpoint_manager.delete_checkpoint(self.info.info_hash)
except Exception:
self.logger.exception("Failed to delete checkpoint")
return False
|
get_status() -> dict[str, Any]
async
Get current torrent status.
Source code in ccbt/session/session.py
| async def get_status(self) -> dict[str, Any]:
"""Get current torrent status."""
status = self.download_manager.get_status()
status.update(
{
"info_hash": self.info.info_hash.hex(),
"name": self.info.name,
"status": self.info.status,
"added_time": self.info.added_time,
"uptime": time.time() - self.info.added_time,
"is_private": self.is_private, # BEP 27: Include private flag in status
},
)
return status
|
pause() -> None
async
Pause the torrent session by stopping background work and saving a checkpoint.
Resume will restart the session using existing state.
Source code in ccbt/session/session.py
| async def pause(self) -> None:
"""Pause the torrent session by stopping background work and saving a checkpoint.
Resume will restart the session using existing state.
"""
try:
# Save checkpoint before pausing
if self.config.disk.checkpoint_enabled:
try:
await self._save_checkpoint()
except Exception as e:
self.logger.warning("Failed to save checkpoint on pause: %s", e)
# Stop background tasks
self._stop_event.set()
if self._announce_task:
self._announce_task.cancel()
if self._status_task:
self._status_task.cancel()
if self._checkpoint_task:
self._checkpoint_task.cancel()
# Stop heavy components
if self.pex_manager:
await self.pex_manager.stop()
await self.tracker.stop()
await self.download_manager.stop()
self.info.status = "paused"
self.logger.info("Paused torrent session: %s", self.info.name)
except Exception:
self.logger.exception("Failed to pause torrent")
raise
|
resume() -> None
async
Resume a previously paused torrent session.
Source code in ccbt/session/session.py
| async def resume(self) -> None:
"""Resume a previously paused torrent session."""
try:
await self.start(resume=True)
self.info.status = "downloading"
self.logger.info("Resumed torrent session: %s", self.info.name)
except Exception:
self.logger.exception("Failed to resume torrent")
raise
|
start(resume: bool = False) -> None
async
Start the async torrent session.
Source code in ccbt/session/session.py
| async def start(self, resume: bool = False) -> None:
"""Start the async torrent session."""
try:
self.info.status = "starting"
# Check for existing checkpoint only if resuming
checkpoint = None
if self.config.disk.checkpoint_enabled and (
resume or self.config.disk.auto_resume
):
try:
checkpoint = await self.checkpoint_manager.load_checkpoint(
self.info.info_hash,
)
if checkpoint:
self.logger.info("Found checkpoint for %s", self.info.name)
self.resume_from_checkpoint = True
self.logger.info("Resuming from checkpoint")
except Exception as e:
self.logger.warning("Failed to load checkpoint: %s", e)
checkpoint = None
# Start tracker client
await self.tracker.start()
# Apply per-torrent configuration options (override global config)
self._apply_per_torrent_options()
# Start piece manager
self.logger.debug("Starting piece manager for torrent: %s", self.info.name)
try:
await self.piece_manager.start()
self.logger.debug("Piece manager started successfully")
except Exception as e:
self.logger.exception("Failed to start piece manager: %s", e)
raise # Re-raise - piece manager is critical
# CRITICAL FIX: Initialize peer manager early, even without peers
# This ensures _peer_manager is set on piece manager before piece selection starts
# The peer manager can wait for peers to arrive from tracker/DHT/PEX
if (
not hasattr(self.download_manager, "peer_manager")
or self.download_manager.peer_manager is None
):
from ccbt.peer.async_peer_connection import AsyncPeerConnectionManager
# Extract is_private flag
is_private = False
try:
if isinstance(self.torrent_data, dict):
is_private = self.torrent_data.get("is_private", False)
elif hasattr(self.torrent_data, "is_private"):
is_private = getattr(self.torrent_data, "is_private", False)
except Exception:
pass
# Normalize torrent_data for peer manager
if isinstance(self.torrent_data, dict):
td_for_peer = self.torrent_data
else:
# Convert to dict format
td_for_peer = {
"info_hash": getattr(self.torrent_data, "info_hash", b""),
"name": getattr(self.torrent_data, "name", "unknown"),
"pieces_info": {
"piece_hashes": getattr(self.torrent_data, "pieces", []),
"piece_length": getattr(
self.torrent_data, "piece_length", 0
),
"num_pieces": getattr(self.torrent_data, "num_pieces", 0),
"total_length": getattr(
self.torrent_data, "total_length", 0
),
},
}
try:
self.logger.debug(
"Initializing peer manager for torrent: %s", self.info.name
)
our_peer_id = getattr(self.download_manager, "our_peer_id", None)
peer_manager = AsyncPeerConnectionManager(
td_for_peer,
self.piece_manager,
our_peer_id,
)
self.logger.debug(
"Peer manager created, setting security manager and flags"
)
peer_manager._security_manager = getattr(
self.download_manager, "security_manager", None
) # type: ignore[attr-defined]
peer_manager._is_private = is_private # type: ignore[attr-defined]
# Apply per-torrent max_peers_per_torrent if set (overrides global)
if "max_peers_per_torrent" in self.options:
max_peers = self.options["max_peers_per_torrent"]
if max_peers is not None and max_peers >= 0:
# Override the config value for this peer manager
# Store original and set per-torrent value
original_max = self.config.network.max_peers_per_torrent
self.config.network.max_peers_per_torrent = max_peers
self.logger.debug(
"Applied per-torrent max_peers_per_torrent: %s (global: %s)",
max_peers,
original_max,
)
# Note: This modifies the global config object, but only for this session
# A better approach would be to pass it to peer manager directly
# For now, we'll store it and peer manager will read from config
# TODO: Refactor to pass max_peers directly to peer manager
# Wire callbacks
self.logger.debug("Wiring peer manager callbacks")
if hasattr(self.download_manager, "_on_peer_connected"):
peer_manager.on_peer_connected = (
self.download_manager._on_peer_connected
) # type: ignore[attr-defined]
if hasattr(self.download_manager, "_on_peer_disconnected"):
peer_manager.on_peer_disconnected = (
self.download_manager._on_peer_disconnected
) # type: ignore[attr-defined]
if hasattr(self.download_manager, "_on_piece_received"):
peer_manager.on_piece_received = (
self.download_manager._on_piece_received
) # type: ignore[attr-defined]
if hasattr(self.download_manager, "_on_bitfield_received"):
peer_manager.on_bitfield_received = (
self.download_manager._on_bitfield_received
) # type: ignore[attr-defined]
# Set peer manager on download manager
self.download_manager.peer_manager = peer_manager # type: ignore[assignment]
# Start peer manager
self.logger.debug("Starting peer manager")
if hasattr(peer_manager, "start"):
await peer_manager.start() # type: ignore[misc]
# CRITICAL FIX: Set _peer_manager on piece manager immediately
# This allows piece selection to work even before peers are connected
self.piece_manager._peer_manager = peer_manager # type: ignore[attr-defined]
self.logger.info(
"Peer manager initialized early (waiting for peers from tracker/DHT/PEX)"
)
# CRITICAL FIX: Start piece manager download with peer manager
# This sets is_downloading=True and allows piece selection to work
# CRITICAL FIX: For magnet links, this may set is_downloading=True even if num_pieces=0
# This is intentional - allows piece selector to be ready when metadata arrives
self.logger.debug("Starting piece manager download")
await self.piece_manager.start_download(peer_manager)
self.logger.info(
"Piece manager download started (is_downloading=%s, num_pieces=%d, waiting for peers)",
self.piece_manager.is_downloading,
self.piece_manager.num_pieces,
)
except Exception as e:
self.logger.exception(
"Failed to initialize peer manager early: %s", e
)
# Continue without early initialization - will be created when peers arrive
# Don't re-raise - allow session to start even if peer manager init fails
# Set up callbacks
self.download_manager.on_download_complete = self._on_download_complete
self.download_manager.on_piece_verified = self._on_piece_verified
# Set up checkpoint callback
if self.config.disk.checkpoint_enabled:
self.download_manager.piece_manager.on_checkpoint_save = (
self._save_checkpoint
)
# Handle resume from checkpoint
if self.resume_from_checkpoint and checkpoint:
await self._resume_from_checkpoint(checkpoint)
# Start PEX manager if enabled
if self.config.discovery.enable_pex:
self.pex_manager = PEXManager()
await self.pex_manager.start()
# CRITICAL FIX: Set up DHT peer discovery for magnet links and regular torrents
# This must happen after session manager and DHT client are ready
if self.config.discovery.enable_dht and self.session_manager:
try:
from ccbt.session.dht_setup import DHTDiscoverySetup
dht_setup = DHTDiscoverySetup(self)
await dht_setup.setup_dht_discovery()
except Exception as dht_error:
# Log but don't fail session start - DHT is best-effort
self.logger.warning(
"Failed to set up DHT peer discovery: %s (peer discovery may be limited)",
dht_error,
)
# Start background tasks with error isolation
# CRITICAL FIX: Wrap task creation to ensure exceptions don't crash the daemon
# The event loop exception handler will catch any unhandled exceptions in these tasks
try:
self._announce_task = asyncio.create_task(self._announce_loop())
self._status_task = asyncio.create_task(self._status_loop())
# Start checkpoint task if enabled
if self.config.disk.checkpoint_enabled:
self._checkpoint_task = asyncio.create_task(self._checkpoint_loop())
except Exception as task_error:
# Log error but don't fail session start - tasks will be handled by exception handler
self.logger.warning(
"Error creating background tasks (will be handled by exception handler): %s",
task_error,
)
# Re-raise only if critical - but task creation shouldn't fail
raise
self.info.status = "downloading"
self.logger.info("Started torrent session: %s", self.info.name)
except Exception as e:
self.info.status = "error"
self.logger.exception("Failed to start torrent session")
if self.on_error:
await self.on_error(e)
raise
|
stop() -> None
async
Stop the async torrent session.
Source code in ccbt/session/session.py
| async def stop(self) -> None:
"""Stop the async torrent session."""
self._stop_event.set()
# Cancel background tasks
if self._announce_task:
self._announce_task.cancel()
if self._status_task:
self._status_task.cancel()
if self._checkpoint_task:
self._checkpoint_task.cancel()
# Save final checkpoint before stopping
if (
self.config.disk.checkpoint_enabled
and not self.download_manager.download_complete
):
try:
await self._save_checkpoint()
except Exception as e:
self.logger.warning("Failed to save final checkpoint: %s", e)
# Stop components
if self.pex_manager:
await self.pex_manager.stop()
await self.download_manager.stop()
await self.piece_manager.stop()
# CRITICAL FIX: Ensure tracker is properly stopped and session is closed
# This prevents "Unclosed client session" warnings
try:
await self.tracker.stop()
except Exception as e:
self.logger.warning("Error stopping tracker: %s", e)
# Try to force close session if stop() failed
if hasattr(self.tracker, "session") and self.tracker.session:
try:
if not self.tracker.session.closed:
await self.tracker.session.close()
except Exception:
pass
self.tracker.session = None
self.info.status = "stopped"
self.logger.info("Stopped torrent session: %s", self.info.name)
|
Key Methods:
Components:
- download_manager: ccbt/session/session.py:78 - AsyncDownloadManager for piece management
- file_selection_manager: ccbt/session/session.py:86 - FileSelectionManager for multi-file torrents
- piece_manager: ccbt/session/session.py:92 - AsyncPieceManager for piece selection
- checkpoint_manager: ccbt/session/session.py:102 - CheckpointManager for resume functionality
Data Model: ccbt/session/session.py:TorrentSessionInfo
Peer Management
Peer
Represents a peer connection.
Implementation: ccbt/peer/peer.py
Properties and methods:
- Peer information: IP, port, peer ID, client identification
- Connection state: Connected, choked, interested
- Transfer rates: Download/upload speeds
AsyncPeerConnection
Async peer connection with pipelining, tit-for-tat choking, and adaptive block sizing.
Implementation Status
The AsyncPeerConnection class is currently under development. For peer connection management, see AsyncPeerConnectionManager below.
Features:
- Request pipelining for high throughput: Deep request queues (16-64 outstanding requests)
- Async message handling: Non-blocking message processing
- Tit-for-tat choking: Fair bandwidth allocation with optimistic unchoke
- Connection state management: Tracks connection lifecycle
Key Methods:
- connect(): Establish connection and perform handshake
- disconnect(): Close connection and cleanup
- request_piece(): Request piece blocks with pipelining
- send_piece(): Send piece data to peer
AsyncPeerConnectionManager
Manages multiple peer connections with connection pooling and lifecycle management.
Async peer connection manager with advanced features.
Initialize async peer connection manager.
Parameters:
| Name |
Type |
Description |
Default |
torrent_data
|
dict[str, Any]
|
|
required
|
piece_manager
|
Any
|
|
required
|
peer_id
|
bytes | None
|
|
None
|
key_manager
|
Any
|
Optional Ed25519KeyManager for cryptographic authentication
|
None
|
Source code in ccbt/peer/async_peer_connection.py
| def __init__(
self,
torrent_data: dict[str, Any],
piece_manager: Any,
peer_id: bytes | None = None,
key_manager: Any = None, # Ed25519KeyManager
):
"""Initialize async peer connection manager.
Args:
torrent_data: Parsed torrent data
piece_manager: Piece manager instance
peer_id: Our peer ID (20 bytes)
key_manager: Optional Ed25519KeyManager for cryptographic authentication
"""
self.torrent_data = torrent_data
self.piece_manager = piece_manager
self.config = get_config()
self.webtorrent_protocol = None # Will be set if WebTorrent is enabled
self.key_manager = key_manager
if peer_id is None:
peer_id = b"-CC0101-" + b"x" * 12
self.our_peer_id = peer_id
# Connection pool for connection reuse
from ccbt.peer.connection_pool import PeerConnectionPool
self.connection_pool = PeerConnectionPool(
max_connections=self.config.network.connection_pool_max_connections,
max_idle_time=self.config.network.connection_pool_max_idle_time,
health_check_interval=self.config.network.connection_pool_health_check_interval,
)
# Circuit breaker for peer connections
if self.config.network.circuit_breaker_enabled:
from ccbt.utils.resilience import PeerCircuitBreakerManager
self.circuit_breaker_manager = PeerCircuitBreakerManager(
failure_threshold=self.config.network.circuit_breaker_failure_threshold,
recovery_timeout=self.config.network.circuit_breaker_recovery_timeout,
)
else:
self.circuit_breaker_manager = None
# Connection management
self.connections: dict[str, AsyncPeerConnection] = {}
self.connection_lock = asyncio.Lock()
# Failed peer tracking with exponential backoff
# CRITICAL FIX: Track failure count for exponential backoff instead of just timestamp
# Peers will be automatically retried when:
# 1. New peer lists arrive from trackers/DHT/PEX (if backoff period has expired)
# 2. Exponential backoff ensures we don't retry too aggressively
# Backoff intervals: 30s (1st), 60s (2nd), 120s (3rd), 240s (4th), 480s (5th), 600s (max)
self._failed_peers: dict[
str, dict[str, Any]
] = {} # peer_key -> {"timestamp": float, "count": int, "reason": str}
self._failed_peer_lock = asyncio.Lock()
self._min_retry_interval = 30.0 # Initial retry interval (30 seconds)
self._max_retry_interval = 600.0 # Maximum retry interval (10 minutes)
self._backoff_multiplier = 2.0 # Exponential backoff multiplier
# CRITICAL FIX: Global connection limiter for Windows to prevent WinError 121
# Windows has a very limited OS-level semaphore for TCP connections
import sys
if sys.platform == "win32":
# Limit to 5 total simultaneous connection attempts on Windows
self._global_connection_semaphore = asyncio.Semaphore(5)
else:
# Other platforms can handle more concurrent connections
self._global_connection_semaphore = asyncio.Semaphore(20)
# Choking management
self.upload_slots: list[AsyncPeerConnection] = []
self.optimistic_unchoke: AsyncPeerConnection | None = None
self.optimistic_unchoke_time: float = 0.0
# Background tasks
self._choking_task: asyncio.Task | None = None
self._stats_task: asyncio.Task | None = None
self._reconnection_task: asyncio.Task | None = None
# Running state flag for idempotency
self._running: bool = False
# CRITICAL FIX: Debouncing for piece selection triggers from Have messages
# Prevent excessive piece selection calls from duplicate Have messages
self._last_piece_selection_trigger: float = 0.0
self._piece_selection_debounce_interval: float = 0.1 # 100ms debounce interval
self._piece_selection_debounce_lock = asyncio.Lock()
# Callbacks
self.on_peer_connected: Callable[[AsyncPeerConnection], None] | None = None
self.on_peer_disconnected: Callable[[AsyncPeerConnection], None] | None = None
self.on_bitfield_received: (
Callable[[AsyncPeerConnection, BitfieldMessage], None] | None
) = None
self.on_piece_received: (
Callable[[AsyncPeerConnection, PieceMessage], None] | None
) = None
# Message handlers
self.message_handlers: dict[
MessageType,
Callable[[AsyncPeerConnection, PeerMessage], None],
] = {
MessageType.CHOKE: self._handle_choke,
MessageType.UNCHOKE: self._handle_unchoke,
MessageType.INTERESTED: self._handle_interested,
MessageType.NOT_INTERESTED: self._handle_not_interested,
MessageType.HAVE: self._handle_have,
MessageType.BITFIELD: self._handle_bitfield,
MessageType.REQUEST: self._handle_request,
MessageType.PIECE: self._handle_piece,
MessageType.CANCEL: self._handle_cancel,
}
self.logger = logging.getLogger(__name__)
# Initialize uTP incoming connection handler if uTP is enabled
if self.config.network.enable_utp:
_task = asyncio.create_task(self._setup_utp_incoming_handler())
# Store task reference to avoid garbage collection
del _task # Task runs in background, no need to keep reference
|
accept_incoming(reader: asyncio.StreamReader, writer: asyncio.StreamWriter, handshake: Handshake, peer_ip: str, peer_port: int) -> None
async
Accept an incoming peer connection.
Called by TCP server when a peer connects to us. The handshake has
already been read and validated by the TCP server.
Parameters:
| Name |
Type |
Description |
Default |
reader
|
StreamReader
|
Stream reader for incoming data
|
required
|
writer
|
StreamWriter
|
Stream writer for outgoing data
|
required
|
handshake
|
Handshake
|
Parsed handshake object from peer
|
required
|
peer_ip
|
str
|
|
required
|
peer_port
|
int
|
|
required
|
Source code in ccbt/peer/async_peer_connection.py
| async def accept_incoming(
self,
reader: asyncio.StreamReader,
writer: asyncio.StreamWriter,
handshake: Handshake,
peer_ip: str,
peer_port: int,
) -> None:
"""Accept an incoming peer connection.
Called by TCP server when a peer connects to us. The handshake has
already been read and validated by the TCP server.
Args:
reader: Stream reader for incoming data
writer: Stream writer for outgoing data
handshake: Parsed handshake object from peer
peer_ip: Peer IP address
peer_port: Peer port
"""
# Check connection limits
async with self.connection_lock:
current_connections = len(self.connections)
max_global = self.config.network.max_global_peers
max_per_torrent = self.config.network.max_peers_per_torrent
if current_connections >= max_global:
self.logger.debug(
"Rejecting incoming connection from %s:%d: max global peers reached (%d/%d)",
peer_ip,
peer_port,
current_connections,
max_global,
)
writer.close()
await writer.wait_closed()
return
if current_connections >= max_per_torrent:
self.logger.debug(
"Rejecting incoming connection from %s:%d: max peers per torrent reached (%d/%d)",
peer_ip,
peer_port,
current_connections,
max_per_torrent,
)
writer.close()
await writer.wait_closed()
return
# Create PeerInfo from handshake and connection details
from ccbt.models import PeerInfo
peer_info = PeerInfo(
ip=peer_ip,
port=peer_port,
peer_id=handshake.peer_id,
peer_source="incoming",
)
# Check if we already have a connection to this peer
peer_key = f"{peer_ip}:{peer_port}"
async with self.connection_lock:
if peer_key in self.connections:
self.logger.debug(
"Already connected to peer %s:%d, closing incoming connection",
peer_ip,
peer_port,
)
writer.close()
await writer.wait_closed()
return
# Create peer connection
connection = AsyncPeerConnection(peer_info, self.torrent_data)
connection.reader = reader
connection.writer = writer
connection.state = ConnectionState.HANDSHAKE_RECEIVED
# Validate info_hash matches
info_hash = self.torrent_data["info_hash"]
if handshake.info_hash != info_hash:
self.logger.warning(
"Info hash mismatch from incoming peer %s:%d: expected %s, got %s",
peer_ip,
peer_port,
info_hash.hex()[:16],
handshake.info_hash.hex()[:16],
)
writer.close()
await writer.wait_closed()
return
# Send our handshake response
try:
# Create handshake with optional Ed25519 signature
ed25519_public_key = None
ed25519_signature = None
if self.key_manager:
try:
from ccbt.security.ed25519_handshake import Ed25519Handshake
ed25519_handshake = Ed25519Handshake(self.key_manager)
ed25519_public_key, ed25519_signature = (
ed25519_handshake.initiate_handshake(
info_hash, self.our_peer_id
)
)
except Exception as e:
self.logger.debug(
"Failed to create Ed25519 handshake signature: %s", e
)
our_handshake = Handshake(
info_hash,
self.our_peer_id,
ed25519_public_key=ed25519_public_key,
ed25519_signature=ed25519_signature,
)
# Configure reserved bytes based on configuration
our_handshake.configure_from_config(self.config)
handshake_data = our_handshake.encode()
writer.write(handshake_data)
await writer.drain()
self.logger.debug(
"Sent handshake response to incoming peer %s:%d", peer_ip, peer_port
)
except Exception as e:
self.logger.warning(
"Failed to send handshake response to %s:%d: %s", peer_ip, peer_port, e
)
writer.close()
await writer.wait_closed()
return
# Add to connections
async with self.connection_lock:
self.connections[peer_key] = connection
# Start connection processing
# For incoming connections, handshake is already received and we've sent our response
# Now we need to continue with the normal BitTorrent protocol flow
connection.state = ConnectionState.CONNECTED
try:
# Send bitfield and unchoke (same as outbound connections)
self.logger.info(
"Sending initial messages to incoming peer %s:%d: bitfield, unchoke",
peer_ip,
peer_port,
)
try:
await self._send_bitfield(connection)
self.logger.debug(
"Sent bitfield to incoming peer %s:%d", peer_ip, peer_port
)
except Exception as e:
error_msg = f"Failed to send bitfield to incoming peer {peer_ip}:{peer_port}: {e}"
self.logger.warning(error_msg)
raise PeerConnectionError(error_msg) from e
try:
await self._send_unchoke(connection)
self.logger.debug(
"Sent unchoke to incoming peer %s:%d", peer_ip, peer_port
)
except Exception as e:
error_msg = f"Failed to send unchoke to incoming peer {peer_ip}:{peer_port}: {e}"
self.logger.warning(error_msg)
raise PeerConnectionError(error_msg) from e
# Attempt SSL negotiation after handshake if extension protocol is supported
try:
await self._attempt_ssl_negotiation(connection)
except Exception as e:
# SSL negotiation failure shouldn't break the connection
self.logger.debug(
"SSL negotiation failed for incoming peer %s:%d (continuing with plain connection): %s",
peer_ip,
peer_port,
e,
)
# Start message handling loop
self.logger.debug(
"Starting message handling loop for incoming peer %s:%d",
peer_ip,
peer_port,
)
connection.connection_task = asyncio.create_task(
self._handle_peer_messages(connection)
)
# Notify callback
if self.on_peer_connected:
try:
self.on_peer_connected(connection)
except Exception as e:
self.logger.warning(
"Error in on_peer_connected callback for incoming peer %s:%d: %s",
peer_ip,
peer_port,
e,
)
self.logger.info(
"Accepted incoming connection from %s:%d (handshake complete, message loop started)",
peer_ip,
peer_port,
)
except Exception as e:
self.logger.exception(
"Error processing incoming connection from %s:%d: %s",
peer_ip,
peer_port,
e,
)
async with self.connection_lock:
if peer_key in self.connections:
del self.connections[peer_key]
try:
writer.close()
await writer.wait_closed()
except Exception:
pass
|
broadcast_have(piece_index: int) -> None
async
Broadcast HAVE message to all connected peers.
Source code in ccbt/peer/async_peer_connection.py
| async def broadcast_have(self, piece_index: int) -> None:
"""Broadcast HAVE message to all connected peers."""
have_msg = HaveMessage(
piece_index
) # pragma: no cover - Broadcasting requires multiple connected peers, complex to test
async with self.connection_lock: # pragma: no cover - Same context
for connection in self.connections.values(): # pragma: no cover - Broadcasting requires multiple connected peers, complex to test
if connection.is_connected(): # pragma: no cover - Same context
await self._send_message(
connection, have_msg
) # pragma: no cover - Same context
|
connect_to_peers(peer_list: list[dict[str, Any]]) -> None
async
Connect to a list of peers with rate limiting and error handling.
Parameters:
| Name |
Type |
Description |
Default |
peer_list
|
list[dict[str, Any]]
|
List of peer dictionaries with 'ip', 'port', and optionally 'peer_source'
|
required
|
Source code in ccbt/peer/async_peer_connection.py
| async def connect_to_peers(self, peer_list: list[dict[str, Any]]) -> None:
"""Connect to a list of peers with rate limiting and error handling.
Args:
peer_list: List of peer dictionaries with 'ip', 'port', and optionally 'peer_source'
"""
# CRITICAL FIX: Add detailed logging for peer connection attempts
if not peer_list:
self.logger.debug("connect_to_peers called with empty peer list")
return
# CRITICAL FIX: Enhanced logging for connection attempts
peer_sources = {}
for peer in peer_list:
source = peer.get("peer_source", "unknown")
peer_sources[source] = peer_sources.get(source, 0) + 1
source_summary = ", ".join(
[f"{count} from {source}" for source, count in peer_sources.items()]
)
self.logger.info(
"Starting connection attempts to %d peer(s) (sources: %s, info_hash: %s)",
len(peer_list),
source_summary,
self.info_hash.hex()[:16] + "..."
if hasattr(self, "info_hash") and self.info_hash
else "unknown",
)
config = self.config.network
max_connections = min(config.max_peers_per_torrent, len(peer_list))
# Filter out recently failed peers using exponential backoff
current_time = time.time()
async with self._failed_peer_lock:
# Clean up old failures (older than max retry interval)
expired_keys = [
key
for key, fail_info in self._failed_peers.items()
if current_time - fail_info.get("timestamp", 0)
> self._max_retry_interval
]
for key in expired_keys:
del self._failed_peers[key]
# Convert to PeerInfo list and filter out recently failed peers
peer_info_list = []
skipped_failed = 0
for idx, peer_data in enumerate(
peer_list[: max_connections * 2]
): # Check more peers to account for filtering
# CRITICAL FIX: Validate peer_data is a dict before accessing it
if not isinstance(peer_data, dict):
error_msg = (
f"peer_data at index {idx} is not a dict, got {type(peer_data)}. "
f"Expected dict with 'ip' and 'port' keys. "
f"peer_data value: {str(peer_data)[:200]}"
)
self.logger.error(error_msg)
continue # Skip invalid peer data
try:
peer_info = PeerInfo(
ip=peer_data["ip"],
port=peer_data["port"],
peer_source=peer_data.get(
"peer_source", "tracker"
), # Default to tracker for tracker responses
)
except (KeyError, TypeError) as e:
error_msg = (
f"Invalid peer_data at index {idx}: {e}. "
f"peer_data type: {type(peer_data)}, "
f"peer_data value: {str(peer_data)[:200]}"
)
self.logger.exception(error_msg)
continue # Skip invalid peer data
# Skip if already connected
peer_key = str(peer_info)
if peer_key in self.connections:
continue
# Skip if recently failed (using exponential backoff)
async with self._failed_peer_lock:
if peer_key in self._failed_peers:
fail_info = self._failed_peers[peer_key]
fail_time = fail_info.get("timestamp", 0)
fail_count = fail_info.get("count", 1)
# Calculate exponential backoff: min_interval * (multiplier ^ (count - 1))
# Cap at max_retry_interval
backoff_interval = min(
self._min_retry_interval
* (self._backoff_multiplier ** (fail_count - 1)),
self._max_retry_interval,
)
elapsed = current_time - fail_time
if elapsed < backoff_interval:
skipped_failed += 1
self.logger.debug(
"Skipping peer %s (failed %d times, backoff: %.1fs, elapsed: %.1fs, reason: %s)",
peer_key,
fail_count,
backoff_interval,
elapsed,
fail_info.get("reason", "unknown"),
)
continue
# Add to connection list
peer_info_list.append(peer_info)
# Limit to max_connections
if len(peer_info_list) >= max_connections:
break
if skipped_failed > 0:
# Calculate average backoff for logging
async with self._failed_peer_lock:
total_backoff = 0.0
backoff_count = 0
peer_keys_in_list = {str(p) for p in peer_info_list}
for peer_key, fail_info in self._failed_peers.items():
# Only count peers that were actually skipped (not in peer_info_list)
if peer_key not in peer_keys_in_list:
fail_count = fail_info.get("count", 1)
backoff = min(
self._min_retry_interval
* (self._backoff_multiplier ** (fail_count - 1)),
self._max_retry_interval,
)
total_backoff += backoff
backoff_count += 1
avg_backoff = (
total_backoff / backoff_count
if backoff_count > 0
else self._min_retry_interval
)
self.logger.debug(
"Skipped %d recently failed peers (using exponential backoff, avg retry after %.1fs)",
skipped_failed,
avg_backoff,
)
# Warmup connections if enabled
# CRITICAL FIX: Disable warmup on Windows to avoid WinError 121
import sys
if (
config.connection_pool_warmup_enabled
and peer_info_list
and sys.platform != "win32"
):
warmup_count = min(config.connection_pool_warmup_count, len(peer_info_list))
await self.connection_pool.warmup_connections(peer_info_list, warmup_count)
elif config.connection_pool_warmup_enabled and sys.platform == "win32":
self.logger.debug(
"Connection pool warmup disabled on Windows to avoid WinError 121 (semaphore timeout)"
)
if not peer_info_list:
self.logger.info(
"No peers to connect to after filtering (total input: %d, skipped failed: %d, already connected: %d, max_connections: %d)",
len(peer_list),
skipped_failed,
len(peer_list) - len(peer_info_list) - skipped_failed,
max_connections,
)
return
# CRITICAL FIX: Enhanced logging for connection attempt start
self.logger.info(
"Starting connection attempts to %d peer(s) (filtered from %d input, skipped %d failed, max_per_torrent: %d)",
len(peer_info_list),
len(peer_list),
skipped_failed,
max_connections,
)
# Rate limit connection attempts to avoid overwhelming the system
# CRITICAL FIX: On Windows, use smaller batches to avoid WinError 121 (semaphore timeout)
# Note: The global_connection_semaphore already limits total simultaneous attempts,
# but we still batch to avoid creating too many tasks at once
import sys
is_windows = sys.platform == "win32"
if is_windows:
# Windows is more sensitive to concurrent connections
# CRITICAL FIX: Increased batch size from 2 to 5 for better throughput
# The global semaphore already limits total concurrent connections, so we can be more aggressive
batch_size = min(
5, max_connections
) # Connect to 5 peers at a time on Windows (increased from 2)
connection_delay = (
1.0 # 1 second delay between batches on Windows (reduced from 2.0s)
)
intra_batch_delay = 0.2 # 200ms delay between connections within a batch (reduced from 0.5s)
self.logger.debug(
"Windows detected: Using optimized connection batching (batch_size=%d, delay=%.1fs, intra_batch_delay=%.1fs) - global semaphore prevents WinError 121",
batch_size,
connection_delay,
intra_batch_delay,
)
else:
batch_size = min(
10, max_connections
) # Connect to 10 peers at a time on other platforms
connection_delay = 0.5 # 500ms delay between batches
intra_batch_delay = 0.0 # No delay within batch on non-Windows
self.logger.debug(
"Non-Windows platform: Using standard connection batching (batch_size=%d, delay=%.1fs)",
batch_size,
connection_delay,
)
# CRITICAL FIX: Aggregate connection statistics for diagnostics
connection_stats = {
"successful": 0,
"failed": 0,
"timeout": 0,
"connection_refused": 0,
"winerror_121": 0,
"other_errors": 0,
"total_attempts": 0,
}
for batch_start in range(0, len(peer_info_list), batch_size):
batch = peer_info_list[batch_start : batch_start + batch_size]
# Create connection tasks for this batch
tasks = []
for i, peer_info in enumerate(
batch
): # pragma: no cover - Loop for connecting to multiple peers, tested via single peer connections
# CRITICAL FIX: Add delay between connections within batch on Windows
if i > 0 and intra_batch_delay > 0:
await asyncio.sleep(intra_batch_delay)
task = asyncio.create_task(
self._connect_to_peer(peer_info)
) # pragma: no cover - Same context
tasks.append(task) # pragma: no cover - Same context
# Wait for batch to complete
if tasks:
results = await asyncio.gather(*tasks, return_exceptions=True)
for i, result in enumerate(
results
): # pragma: no cover - Exception handling from gather results is difficult to test reliably
peer_info = batch[i]
peer_key = str(peer_info)
connection_stats["total_attempts"] += 1
if isinstance(result, Exception): # pragma: no cover - Same context
connection_stats["failed"] += 1
# CRITICAL FIX: Record failure with exponential backoff tracking
error_str = str(result)
error_type = type(result).__name__
# Determine failure reason for better retry strategy
# CRITICAL FIX: Categorize errors as temporary (should retry) vs permanent (should not retry)
failure_reason = "unknown"
is_temporary = (
True # Default to temporary - most errors are retryable
)
if (
"WinError 121" in error_str
or "semaphore timeout" in error_str.lower()
):
failure_reason = "semaphore_timeout"
connection_stats["winerror_121"] += 1
is_temporary = True # Semaphore timeout is temporary - retry after backoff
elif (
"connection refused" in error_str.lower()
or "WinError 1225" in error_str
):
failure_reason = "connection_refused"
connection_stats["connection_refused"] += 1
is_temporary = True # Connection refused is temporary - peer may be busy
elif "timeout" in error_str.lower() or isinstance(
result, asyncio.TimeoutError
):
failure_reason = "timeout"
connection_stats["timeout"] += 1
is_temporary = (
True # Timeouts are temporary - network may be slow
)
elif "connection reset" in error_str.lower():
failure_reason = "connection_reset"
connection_stats["other_errors"] += 1
is_temporary = True # Connection reset is temporary - peer may have closed connection
elif (
"info hash" in error_str.lower()
or "mismatch" in error_str.lower()
):
failure_reason = "info_hash_mismatch"
is_temporary = False # Info hash mismatch is permanent - peer has wrong torrent
elif (
"handshake" in error_str.lower()
and "invalid" in error_str.lower()
):
failure_reason = "invalid_handshake"
is_temporary = False # Invalid handshake is permanent - peer protocol incompatible
else:
failure_reason = error_type.lower()
# Default to temporary for unknown errors - better to retry than give up
is_temporary = True
# Update failure tracking with exponential backoff
# CRITICAL FIX: Only track temporary failures - permanent failures should not be retried
async with self._failed_peer_lock:
if is_temporary:
# Only track temporary failures for retry logic
if peer_key in self._failed_peers:
# Increment failure count for exponential backoff
fail_info = self._failed_peers[peer_key]
fail_info["count"] = fail_info.get("count", 1) + 1
fail_info["timestamp"] = time.time()
fail_info["reason"] = failure_reason
fail_count = fail_info["count"]
else:
# First failure
self._failed_peers[peer_key] = {
"timestamp": time.time(),
"count": 1,
"reason": failure_reason,
}
fail_count = 1
else:
# Permanent failure - don't track for retry, but log it
fail_count = 0 # No retry count for permanent failures
# Remove from failed_peers if it was there (permanent failures shouldn't be retried)
if peer_key in self._failed_peers:
del self._failed_peers[peer_key]
# Calculate backoff interval for logging (only for temporary failures)
if is_temporary and fail_count > 0:
backoff_interval = min(
self._min_retry_interval
* (self._backoff_multiplier ** (fail_count - 1)),
self._max_retry_interval,
)
else:
backoff_interval = 0 # No retry for permanent failures
# CRITICAL FIX: Handle WinError 121 (semaphore timeout) gracefully on Windows
# This is expected on Windows when many connections are attempted simultaneously
import sys
is_windows = sys.platform == "win32"
if not is_temporary:
# Permanent failure - log as warning and don't retry
self.logger.warning(
"Permanent connection failure to %s: %s (reason: %s, will not retry)",
peer_info,
result,
failure_reason,
)
elif failure_reason == "semaphore_timeout":
# Log as debug on Windows (expected), warning on other platforms
if is_windows:
self.logger.debug(
"Connection semaphore timeout (WinError 121) to %s: %s. "
"This is normal on Windows when many connections are attempted simultaneously. "
"Will retry after %.1fs (attempt %d)",
peer_info,
result,
backoff_interval,
fail_count,
)
else:
self.logger.warning(
"Connection semaphore timeout to %s: %s (will retry after %.1fs, attempt %d)",
peer_info,
result,
backoff_interval,
fail_count,
)
elif failure_reason == "connection_refused":
# Connection refused - peer not accepting connections (temporary)
self.logger.debug(
"Connection refused by peer %s (will retry after %.1fs, attempt %d)",
peer_info,
backoff_interval,
fail_count,
)
elif failure_reason in ("timeout", "connection_reset"):
# Timeout or connection reset - temporary network issues
self.logger.debug(
"Temporary connection failure to %s: %s (reason: %s, will retry after %.1fs, attempt %d)",
peer_info,
result,
failure_reason,
backoff_interval,
fail_count,
)
else:
# Log other temporary connection failures as warnings (not errors to reduce noise)
log_level = "warning" if fail_count <= 3 else "debug"
if log_level == "warning":
self.logger.warning(
"Temporary connection failure to %s: %s (will retry after %.1fs, attempt %d, reason: %s)",
peer_info,
result,
backoff_interval,
fail_count,
failure_reason,
)
else:
self.logger.debug(
"Temporary connection failure to %s: %s (will retry after %.1fs, attempt %d, reason: %s)",
peer_info,
result,
backoff_interval,
fail_count,
failure_reason,
)
else:
# CRITICAL FIX: Check if connection actually completed handshake AND bitfield exchange
# _connect_to_peer() returns after handshake completes, so if no exception,
# the connection should be in self.connections
# However, we should only count it as successful if it has completed the full protocol
# handshake (handshake + bitfield exchange), not just the initial handshake
peer_key = str(peer_info)
async with self.connection_lock:
if peer_key in self.connections:
conn = self.connections[peer_key]
# CRITICAL FIX: Only count connections as successful if they've completed
# the full protocol handshake (handshake + bitfield exchange)
# HANDSHAKE_SENT is too early - connection may not have received peer's handshake yet
# We need at least HANDSHAKE_RECEIVED (handshake complete) or better yet,
# BITFIELD_RECEIVED (bitfield exchange complete) or ACTIVE (fully active)
if conn.is_active() or conn.state in [
ConnectionState.ACTIVE,
ConnectionState.BITFIELD_RECEIVED,
ConnectionState.BITFIELD_SENT,
ConnectionState.HANDSHAKE_RECEIVED,
]:
self.logger.debug(
"Connection to %s completed successfully (state=%s, is_active=%s)",
peer_info,
conn.state.value,
conn.is_active(),
)
connection_stats["successful"] += 1
else:
# Connection exists but not in a valid state - may still be connecting
# Don't count as failed yet - it may complete later
self.logger.debug(
"Connection to %s exists but not yet active (state=%s) - may complete later",
peer_info,
conn.state.value,
)
# Count as successful if it's at least connecting (not disconnected)
if conn.state != ConnectionState.DISCONNECTED:
connection_stats["successful"] += 1
else:
connection_stats["failed"] += 1
else:
# No connection in dict - handshake must have failed
# However, this could be a race condition - connection may be added shortly
# Wait a brief moment and check again
await asyncio.sleep(0.1)
async with self.connection_lock:
if peer_key in self.connections:
conn = self.connections[peer_key]
if conn.state != ConnectionState.DISCONNECTED:
connection_stats["successful"] += 1
self.logger.debug(
"Connection to %s found after brief wait (state=%s)",
peer_info,
conn.state.value,
)
else:
connection_stats["failed"] += 1
else:
connection_stats["failed"] += 1
# Delay before next batch (except for last batch)
if batch_start + batch_size < len(peer_info_list):
await asyncio.sleep(connection_delay)
# CRITICAL FIX: Log connection summary after batch completes with detailed statistics
total_attempts = connection_stats["total_attempts"]
successful = connection_stats["successful"]
failed = connection_stats["failed"]
success_rate = (
(successful / total_attempts * 100) if total_attempts > 0 else 0.0
)
if successful > 0:
# Build detailed failure breakdown
failure_details = []
if connection_stats["timeout"] > 0:
failure_details.append(f"{connection_stats['timeout']} timeout(s)")
if connection_stats["connection_refused"] > 0:
failure_details.append(
f"{connection_stats['connection_refused']} refused"
)
if connection_stats["winerror_121"] > 0:
failure_details.append(
f"{connection_stats['winerror_121']} WinError 121"
)
if connection_stats["other_errors"] > 0:
failure_details.append(
f"{connection_stats['other_errors']} other error(s)"
)
failure_summary = (
f" ({', '.join(failure_details)})" if failure_details else ""
)
# CRITICAL FIX: Get current connection counts for logging
current_connections = len(self.connections)
active_connections = len(
[c for c in self.connections.values() if c.is_active()]
)
self.logger.info(
"Connection batch completed: %d/%d successful (%.1f%% success rate, failed: %d%s, skipped recently failed: %d, total_connections: %d, active_connections: %d)",
successful,
total_attempts,
success_rate,
failed,
failure_summary,
skipped_failed,
current_connections,
active_connections,
)
elif failed > 0:
# All connections failed - provide detailed breakdown
failure_details = []
if connection_stats["timeout"] > 0:
failure_details.append(f"{connection_stats['timeout']} timeout(s)")
if connection_stats["connection_refused"] > 0:
failure_details.append(
f"{connection_stats['connection_refused']} refused"
)
if connection_stats["winerror_121"] > 0:
failure_details.append(
f"{connection_stats['winerror_121']} WinError 121"
)
if connection_stats["other_errors"] > 0:
failure_details.append(
f"{connection_stats['other_errors']} other error(s)"
)
failure_summary = (
", ".join(failure_details) if failure_details else "unknown errors"
)
self.logger.warning(
"All %d connection attempts failed (%s). Will retry failed peers after %d seconds.",
failed,
failure_summary,
int(self._min_retry_interval),
)
elif total_attempts == 0:
self.logger.debug(
"No connection attempts made (all peers filtered out or already connected)"
)
|
disconnect_all() -> None
async
Disconnect from all peers.
Source code in ccbt/peer/async_peer_connection.py
| async def disconnect_all(self) -> None:
"""Disconnect from all peers."""
async with self.connection_lock: # pragma: no cover - Disconnect all requires multiple connections, complex to test
for connection in list(
self.connections.values()
): # pragma: no cover - Disconnect all requires multiple connections, complex to test
await self._disconnect_peer(
connection
) # pragma: no cover - Same context
|
disconnect_peer(peer_info: PeerInfo) -> None
async
Disconnect from a specific peer.
Source code in ccbt/peer/async_peer_connection.py
| async def disconnect_peer(self, peer_info: PeerInfo) -> None:
"""Disconnect from a specific peer."""
async with self.connection_lock: # pragma: no cover - Edge case: disconnecting non-existent peer, tested via existing tests
peer_key = str(peer_info)
if (
peer_key in self.connections
): # pragma: no cover - Edge case: disconnecting non-existent peer
connection = self.connections[
peer_key
] # pragma: no cover - Same context
await self._disconnect_peer(
connection
) # pragma: no cover - Same context
|
get_active_peers() -> list[AsyncPeerConnection]
Get list of active peers.
Source code in ccbt/peer/async_peer_connection.py
| def get_active_peers(self) -> list[AsyncPeerConnection]:
"""Get list of active peers."""
# CRITICAL FIX: Include peers that are connected but not yet fully active
# Also include peers that have received bitfield (ready for requests)
# Note: This is a synchronous method, so we can't use async locks
# We create a copy of connections.values() to iterate safely
active_peers = []
connections_copy = list(self.connections.values())
for conn in connections_copy:
# Include peers that are active OR have received bitfield (ready for requests)
# CRITICAL FIX: Explicitly exclude ERROR state connections
# ERROR state indicates connection is being cleaned up or has failed
if conn.state == ConnectionState.ERROR:
continue
# Also include peers that are in ACTIVE state even if not fully active yet
if conn.is_active() or conn.state in {
ConnectionState.BITFIELD_RECEIVED,
ConnectionState.ACTIVE,
}:
active_peers.append(conn)
# Debug logging for connection state distribution
if self.logger.isEnabledFor(logging.DEBUG):
states = {}
for conn in self.connections.values():
state_val = conn.state.value
states[state_val] = states.get(state_val, 0) + 1
self.logger.debug(
"Connection state distribution: %s (total: %d, active: %d)",
states,
len(self.connections),
len(active_peers),
)
return active_peers
|
get_connected_peers() -> list[AsyncPeerConnection]
Get list of connected peers.
Source code in ccbt/peer/async_peer_connection.py
| def get_connected_peers(self) -> list[AsyncPeerConnection]:
"""Get list of connected peers."""
return [
conn for conn in self.connections.values() if conn.is_connected()
] # pragma: no cover - Simple getter, tested via existing tests
|
get_peer_bitfields() -> dict[str, BitfieldMessage]
Get bitfields for all connected peers.
Source code in ccbt/peer/async_peer_connection.py
| def get_peer_bitfields(self) -> dict[str, BitfieldMessage]:
"""Get bitfields for all connected peers."""
result = {} # pragma: no cover - Simple getter with filtering, tested via existing tests
for (
peer_key,
connection,
) in (
self.connections.items()
): # pragma: no cover - Simple getter with filtering, tested via existing tests
if connection.peer_state.bitfield: # pragma: no cover - Same context
result[peer_key] = (
connection.peer_state.bitfield
) # pragma: no cover - Same context
return result # pragma: no cover - Return path for get_peer_bitfields (tested but coverage tool may not track reliably due to dict comprehension)
|
handle_v2_message(connection: AsyncPeerConnection, message: Any) -> None
async
Handle v2 protocol message (BEP 52).
Routes v2 messages (piece layer and file tree requests/responses)
to appropriate handlers.
Parameters:
| Name |
Type |
Description |
Default |
connection
|
AsyncPeerConnection
|
|
required
|
message
|
Any
|
v2 message object (PieceLayerRequest, PieceLayerResponse, etc.)
|
required
|
Source code in ccbt/peer/async_peer_connection.py
| async def handle_v2_message(
self,
connection: AsyncPeerConnection,
message: Any,
) -> None:
"""Handle v2 protocol message (BEP 52).
Routes v2 messages (piece layer and file tree requests/responses)
to appropriate handlers.
Args:
connection: Peer connection
message: v2 message object (PieceLayerRequest, PieceLayerResponse, etc.)
"""
try:
if isinstance(message, PieceLayerRequest):
await self._handle_piece_layer_request(connection, message)
elif isinstance(message, PieceLayerResponse):
await self._handle_piece_layer_response(connection, message)
elif isinstance(message, FileTreeRequest):
await self._handle_file_tree_request(connection, message)
elif isinstance(
message, FileTreeResponse
): # pragma: no cover - v2 message handling, requires v2 torrent and peer support
await self._handle_file_tree_response(connection, message)
else: # pragma: no cover - Unknown v2 message type, defensive error handling
self.logger.warning(
"Unknown v2 message type: %s from %s",
type(message).__name__,
connection.peer_info,
)
except (
Exception
): # pragma: no cover - v2 message exception handler, defensive error handling
self.logger.exception(
"Error handling v2 message from %s",
connection.peer_info,
)
|
request_piece(connection: AsyncPeerConnection, piece_index: int, begin: int, length: int) -> None
async
Request a block from a peer.
Source code in ccbt/peer/async_peer_connection.py
| async def request_piece(
self,
connection: AsyncPeerConnection,
piece_index: int,
begin: int,
length: int,
) -> None:
"""Request a block from a peer."""
if not connection.can_request():
self.logger.debug(
"Cannot request piece %d:%d:%d from %s (choking=%s, active=%s, pipeline=%d/%d)",
piece_index,
begin,
length,
connection.peer_info,
connection.peer_choking,
connection.is_active(),
len(connection.outstanding_requests),
connection.max_pipeline_depth,
)
return
# CRITICAL FIX: Ensure "interested" message is sent before requesting pieces
# According to BitTorrent protocol, we should be "interested" before requesting,
# but we don't block requests if sending fails - some peers may accept requests anyway
if not connection.am_interested:
try:
await self._send_interested(connection)
self.logger.debug(
"Sent interested message to %s (fallback before piece request)",
connection.peer_info,
)
except Exception as e:
# Log but continue - some peers may accept requests even without "interested"
self.logger.debug(
"Failed to send interested to %s before piece request: %s (continuing with request anyway)",
connection.peer_info,
e,
)
# CRITICAL FIX: Log when we actually request a piece
self.logger.debug(
"Requesting piece %d:%d:%d from %s (interested=%s, can_request=%s)",
piece_index,
begin,
length,
connection.peer_info,
connection.am_interested,
connection.can_request(),
)
if connection.can_request(): # pragma: no cover - Piece request logic requires active connection with unchoked peer, complex to test
# Calculate priority for this request
priority = await self._calculate_request_priority(
piece_index, self.piece_manager
)
request_info = RequestInfo(
piece_index, begin, length, time.time()
) # pragma: no cover - Same context
# Use priority queue if prioritization is enabled
enable_prioritization = getattr(
self.config.network, "pipeline_enable_prioritization", True
)
if enable_prioritization:
# Initialize priority queue if not exists
if connection._priority_queue is None: # noqa: SLF001 - Internal queue state
connection._priority_queue = [] # noqa: SLF001 - Internal queue state
# Add to priority queue (negative priority for max-heap via min-heap)
heappush(
connection._priority_queue, # noqa: SLF001 - Internal queue state
(-priority, time.time(), request_info),
)
else:
# Use regular queue
connection.request_queue.append(request_info)
# Process queued requests with coalescing
requests_sent = await self._process_request_queue(connection)
if requests_sent > 0:
# Log at INFO level when requests are actually sent
self.logger.info(
"Sent %d REQUEST message(s) to %s for piece %d:%d:%d (priority=%.2f, outstanding=%d/%d)",
requests_sent,
connection.peer_info,
piece_index,
begin,
length,
priority,
len(connection.outstanding_requests),
connection.max_pipeline_depth,
)
else:
self.logger.debug(
"Queued block %s:%s:%s from %s (priority=%.2f, not sent yet - queue processing)",
piece_index,
begin,
length,
connection.peer_info,
priority,
) # pragma: no cover - Same context
|
send_v2_message(connection: AsyncPeerConnection, message: Any) -> None
async
Send v2 protocol message to peer.
Parameters:
| Name |
Type |
Description |
Default |
connection
|
AsyncPeerConnection
|
|
required
|
message
|
Any
|
v2 message object (PieceLayerRequest, PieceLayerResponse, etc.)
|
required
|
Raises:
| Type |
Description |
RuntimeError
|
If connection is not ready or send fails
|
Source code in ccbt/peer/async_peer_connection.py
| async def send_v2_message(
self,
connection: AsyncPeerConnection,
message: Any,
) -> None:
"""Send v2 protocol message to peer.
Args:
connection: Peer connection
message: v2 message object (PieceLayerRequest, PieceLayerResponse, etc.)
Raises:
RuntimeError: If connection is not ready or send fails
"""
if (
connection.writer is None
): # pragma: no cover - Defensive check: writer should exist for active connection
msg = f"Cannot send v2 message: connection {connection.peer_info} has no writer"
raise RuntimeError(msg)
if not connection.is_active(): # pragma: no cover - Defensive check: connection should be active before sending
msg = f"Cannot send v2 message: connection {connection.peer_info} is not active"
raise RuntimeError(msg)
try:
message_bytes = message.serialize()
connection.writer.write(message_bytes)
await connection.writer.drain()
self.logger.debug(
"Sent %s to %s",
type(message).__name__,
connection.peer_info,
)
connection.stats.last_activity = time.time()
connection.stats.bytes_uploaded += len(message_bytes)
except (
Exception
): # pragma: no cover - v2 message send error, defensive error handling
self.logger.exception(
"Failed to send v2 message to %s",
connection.peer_info,
)
raise
|
shutdown() -> None
async
Alias for stop method for backward compatibility.
Source code in ccbt/peer/async_peer_connection.py
| async def shutdown(
self,
) -> None: # pragma: no cover - Alias method, tested via stop()
"""Alias for stop method for backward compatibility."""
await self.stop() # pragma: no cover - Same context
|
start() -> None
async
Start background tasks and initialize the peer connection manager.
This method is idempotent - calling it multiple times will only start
the manager once. It initializes:
- Connection pool
- Choking/unchoking management loop
- Peer statistics update loop
- Failed peer reconnection loop
Raises:
| Type |
Description |
RuntimeError
|
If the manager fails to start due to initialization errors
|
Source code in ccbt/peer/async_peer_connection.py
| async def start(self) -> None:
"""Start background tasks and initialize the peer connection manager.
This method is idempotent - calling it multiple times will only start
the manager once. It initializes:
- Connection pool
- Choking/unchoking management loop
- Peer statistics update loop
- Failed peer reconnection loop
Raises:
RuntimeError: If the manager fails to start due to initialization errors
"""
# Idempotency check: if already running, return early
if self._running:
self.logger.debug("Async peer connection manager already started, skipping")
return
try:
# Start connection pool first (required for connection reuse)
await self.connection_pool.start()
self.logger.debug("Connection pool started")
# Start background tasks
# CRITICAL FIX: Only create tasks if they don't already exist
# This prevents duplicate tasks if start() is called multiple times
if self._choking_task is None or self._choking_task.done():
self._choking_task = asyncio.create_task(self._choking_loop())
self.logger.debug("Choking loop task started")
if self._stats_task is None or self._stats_task.done():
self._stats_task = asyncio.create_task(self._stats_loop())
self.logger.debug("Stats loop task started")
if self._reconnection_task is None or self._reconnection_task.done():
self._reconnection_task = asyncio.create_task(self._reconnection_loop())
self.logger.debug("Reconnection loop task started")
# Mark as running after all tasks are started
self._running = True
self.logger.info(
"Async peer connection manager started (connection_pool=%s, "
"choking_task=%s, stats_task=%s, reconnection_task=%s)",
getattr(self.connection_pool, "_running", "unknown"),
self._choking_task is not None and not self._choking_task.done(),
self._stats_task is not None and not self._stats_task.done(),
self._reconnection_task is not None
and not self._reconnection_task.done(),
)
except Exception as e:
# If startup fails, clean up any partially started tasks
self.logger.exception("Failed to start async peer connection manager")
# Attempt cleanup
try:
await self.stop()
except Exception as cleanup_error:
self.logger.warning(
"Error during cleanup after failed start: %s", cleanup_error
)
# Re-raise the original error
error_msg = f"Failed to start peer connection manager: {e}"
raise RuntimeError(error_msg) from e
|
stop() -> None
async
Stop background tasks and disconnect all peers.
This method gracefully shuts down the peer connection manager by:
1. Cancelling all background tasks with timeout protection
2. Disconnecting all active peer connections
3. Stopping the connection pool
4. Cleaning up resources
This method is idempotent - calling it multiple times is safe.
Source code in ccbt/peer/async_peer_connection.py
| async def stop(self) -> None:
"""Stop background tasks and disconnect all peers.
This method gracefully shuts down the peer connection manager by:
1. Cancelling all background tasks with timeout protection
2. Disconnecting all active peer connections
3. Stopping the connection pool
4. Cleaning up resources
This method is idempotent - calling it multiple times is safe.
"""
# Idempotency check: if already stopped, return early
if not self._running:
self.logger.debug("Async peer connection manager already stopped, skipping")
return
# Mark as not running immediately to prevent new operations
self._running = False
self.logger.info("Stopping async peer connection manager...")
# Collect all tasks to cancel
tasks_to_cancel: list[asyncio.Task] = []
if self._choking_task and not self._choking_task.done():
tasks_to_cancel.append(self._choking_task)
if self._stats_task and not self._stats_task.done():
tasks_to_cancel.append(self._stats_task)
if self._reconnection_task and not self._reconnection_task.done():
tasks_to_cancel.append(self._reconnection_task)
# Cancel all background tasks with timeout protection
for task in tasks_to_cancel:
try:
task.cancel()
except Exception as e:
self.logger.warning("Error cancelling task %s: %s", task.get_name(), e)
# Wait for tasks to complete cancellation (with timeout to prevent hanging)
for task in tasks_to_cancel:
if not task.done():
try:
# Use timeout to prevent indefinite waiting
await asyncio.wait_for(task, timeout=5.0)
except asyncio.TimeoutError:
self.logger.warning(
"Task %s did not cancel within timeout, forcing cancellation",
task.get_name(),
)
except asyncio.CancelledError:
# Expected when task is cancelled
pass
except Exception as e:
self.logger.debug(
"Error waiting for task %s cancellation: %s", task.get_name(), e
)
# Clear task references
self._choking_task = None
self._stats_task = None
self._reconnection_task = None
# Disconnect all peers (with timeout protection)
try:
async with self.connection_lock:
connections_to_disconnect = list(self.connections.values())
self.logger.debug(
"Disconnecting %d peer connection(s)...",
len(connections_to_disconnect),
)
# Disconnect peers with timeout to prevent hanging
disconnect_tasks = [
asyncio.create_task(self._disconnect_peer(conn))
for conn in connections_to_disconnect
]
if disconnect_tasks:
# Wait for all disconnections with timeout
try:
await asyncio.wait_for(
asyncio.gather(*disconnect_tasks, return_exceptions=True),
timeout=10.0,
)
except asyncio.TimeoutError:
self.logger.warning(
"Some peer disconnections did not complete within timeout"
)
# Cancel remaining disconnect tasks
for task in disconnect_tasks:
if not task.done():
task.cancel()
except Exception as e:
self.logger.debug("Error during peer disconnection: %s", e)
except Exception as e:
self.logger.warning(
"Error disconnecting peers during stop: %s", e, exc_info=True
)
# Stop connection pool (with timeout protection)
try:
await asyncio.wait_for(self.connection_pool.stop(), timeout=5.0)
self.logger.debug("Connection pool stopped")
except asyncio.TimeoutError:
self.logger.warning("Connection pool stop timed out")
except Exception as e:
self.logger.warning("Error stopping connection pool: %s", e)
self.logger.info("Async peer connection manager stopped")
|
PeerConnection
Synchronous peer connection (legacy).
Implementation: ccbt/peer/peer_connection.py
ConnectionPool
Connection pool for managing peer connections.
Implementation: ccbt/peer/connection_pool.py
Features:
- Connection reuse
- Connection limits
- Connection lifecycle management
Piece Management
AsyncPieceManager
Advanced piece selection with rarest-first and endgame.
Advanced piece manager with rarest-first and endgame mode.
Initialize async piece manager.
Parameters:
| Name |
Type |
Description |
Default |
torrent_data
|
dict[str, Any]
|
Parsed torrent data from TorrentParser
|
required
|
file_selection_manager
|
Any | None
|
Optional FileSelectionManager instance
|
None
|
Note
For magnet links, torrent_data may have incomplete metadata initially.
The piece manager will need to be updated once metadata is fetched.
Source code in ccbt/piece/async_piece_manager.py
| def __init__(
self,
torrent_data: dict[str, Any],
file_selection_manager: Any | None = None,
):
"""Initialize async piece manager.
Args:
torrent_data: Parsed torrent data from TorrentParser
file_selection_manager: Optional FileSelectionManager instance
Note:
For magnet links, torrent_data may have incomplete metadata initially.
The piece manager will need to be updated once metadata is fetched.
"""
self.torrent_data = torrent_data
self.config = get_config()
# Handle magnet links with incomplete metadata
pieces_info = torrent_data.get("pieces_info")
if pieces_info is None:
# This shouldn't happen if _normalize_torrent_data worked correctly,
# but handle it defensively
pieces_info = {
"piece_hashes": [],
"piece_length": 16384,
"num_pieces": 0,
"total_length": 0,
}
self.torrent_data["pieces_info"] = pieces_info
self._metadata_incomplete = True
else:
self._metadata_incomplete = torrent_data.get("_metadata_incomplete", False)
# Ensure we have valid types (defensive programming for type safety)
num_pieces_val = pieces_info.get("num_pieces", 0)
piece_length_val = pieces_info.get("piece_length", 16384)
piece_hashes_val = pieces_info.get("piece_hashes", [])
# Type assertions to ensure we have the right types
self.num_pieces = (
int(num_pieces_val) if isinstance(num_pieces_val, (int, float)) else 0
)
self.piece_length = (
int(piece_length_val)
if isinstance(piece_length_val, (int, float))
else 16384
)
self.piece_hashes = (
list(piece_hashes_val)
if isinstance(piece_hashes_val, (list, tuple))
else []
)
# v2/hybrid torrent support (BEP 52)
# Check if torrent has v2 piece layers (for hybrid torrents)
self.piece_layers = torrent_data.get("piece_layers")
self.meta_version = torrent_data.get("meta_version", 1) # 1=v1, 2=v2, 3=hybrid
# File selection manager (optional)
self.file_selection_manager = file_selection_manager
# Piece tracking
self.pieces: list[PieceData] = []
self.completed_pieces: set[int] = set()
self.verified_pieces: set[int] = set()
self.lock = asyncio.Lock()
# Xet Merkle hash cache (piece_index -> merkle_hash)
self.xet_merkle_hashes: dict[int, bytes] = {}
self.xet_chunk_hashes: dict[
int, list[bytes]
] = {} # piece_index -> list of chunk hashes
# Per-peer availability tracking
self.peer_availability: dict[str, PeerAvailability] = {}
self.piece_frequency: Counter = Counter() # How many peers have each piece
# Endgame mode
self.endgame_mode = False
self.endgame_threshold = self.config.strategy.endgame_threshold
self.endgame_duplicates = self.config.strategy.endgame_duplicates
# Hash verification pool (adaptive if enabled)
if self.config.disk.hash_workers_adaptive:
# Work-stealing executor limitation and workaround:
# Python's ThreadPoolExecutor doesn't support work-stealing natively.
# True work-stealing requires custom implementation using deque-based task
# queues per worker, where workers can steal tasks from other workers'
# queues when idle. This would improve load balancing for hash verification.
#
# Current workaround: Double the worker count to approximate work-stealing
# benefits. More workers = better load distribution, as idle workers can
# pick up tasks from the shared queue. This provides reasonable performance
# but is not as efficient as true work-stealing.
#
# TODO: Implement custom WorkStealingExecutor class using asyncio.Queue
# with per-worker deques for efficient task stealing. This would further
# improve load balancing and reduce idle time.
effective_workers = min(
self.config.disk.hash_workers * 2,
32, # Cap at reasonable maximum
)
else: # pragma: no cover - Non-adaptive hash workers path, adaptive is default
effective_workers = self.config.disk.hash_workers
self.hash_executor = ThreadPoolExecutor(
max_workers=effective_workers,
thread_name_prefix="hash-verify",
)
# No background queue; verify hashes via scheduled tasks on completion
self.hash_queue = None # kept for backward compatibility, not used
# Initialize pieces
for i in range(self.num_pieces):
# Calculate actual piece length (last piece may be shorter)
if i == self.num_pieces - 1:
# Get total_length safely - handle different torrent_data structures
total_length = 0
if "file_info" in torrent_data and torrent_data.get("file_info"):
total_length = torrent_data["file_info"].get("total_length", 0)
elif "total_length" in torrent_data:
total_length = torrent_data["total_length"]
else:
# Fallback: calculate from pieces (approximation)
total_length = self.num_pieces * self.piece_length
piece_length = total_length - (i * self.piece_length)
# Ensure piece_length is positive
if piece_length <= 0:
piece_length = self.piece_length
else:
piece_length = self.piece_length
piece = PieceData(i, piece_length)
# Set priorities for streaming mode
if self.config.strategy.streaming_mode:
if i == 0:
piece.priority = 1000 # First piece highest priority
elif i == self.num_pieces - 1:
# Fallback: boost last piece modestly
piece.priority = 100
else:
piece.priority = max(0, 1000 - i) # Decreasing priority
# Apply file-based priorities if file selection manager exists
if self.file_selection_manager:
file_priority = self.file_selection_manager.get_piece_priority(i)
# Scale file priority to piece priority (multiply by 100 to match streaming mode scale)
piece.priority = max(piece.priority, file_priority * 100)
self.pieces.append(piece)
# Download state
self.is_downloading = False
self.download_complete = False
self.download_start_time = time.time()
self.bytes_downloaded = 0
self._current_sequential_piece: int = 0 # Track current sequential position
self._peer_manager: Any | None = None # Store peer manager for piece requests
# Callbacks
self.on_piece_completed: Callable[[int], None] | None = None
self.on_piece_verified: Callable[[int], None] | None = None
self.on_download_complete: Callable[[], None] | None = None
self.on_file_assembled: Callable[[int], None] | None = None
self.on_checkpoint_save: Callable[[], None] | None = None
# File assembler (set by download manager)
self.file_assembler: Any | None = None
# Background tasks
self._hash_worker_task: asyncio.Task | None = None
self._piece_selector_task: asyncio.Task | None = None
self._background_tasks: set[asyncio.Task] = set()
self.logger = logging.getLogger(__name__)
|
get_block(piece_index: int, begin: int, length: int) -> bytes | None
Get a block of data from a piece.
Source code in ccbt/piece/async_piece_manager.py
| def get_block(self, piece_index: int, begin: int, length: int) -> bytes | None:
"""Get a block of data from a piece."""
if (
piece_index >= len(self.pieces)
): # pragma: no cover - Defensive bounds check, tested via get_block_invalid_indices
return None
# get_block method: block lookup and extraction requires specific verified piece and block arrangement
piece = self.pieces[
piece_index
] # pragma: no cover - Piece access, tested separately
if (
piece.state != PieceState.VERIFIED
): # pragma: no cover - State check for verified piece, tested separately
return None
# Find the block that contains this range
for block in (
piece.blocks
): # pragma: no cover - Block lookup loop, requires specific block arrangement
if (
block.begin <= begin < block.begin + block.length
): # pragma: no cover - Block range matching
offset = (
begin - block.begin
) # pragma: no cover - Offset calculation in block extraction
end_offset = min(
offset + length, block.length
) # pragma: no cover - Block extraction logic
return block.data[
offset:end_offset
] # pragma: no cover - Block data extraction
return None # pragma: no cover - No matching block found, edge case
|
get_checkpoint_state(torrent_name: str, info_hash: bytes, output_dir: str) -> TorrentCheckpoint
async
Get current state for checkpointing.
Parameters:
| Name |
Type |
Description |
Default |
torrent_name
|
str
|
|
required
|
info_hash
|
bytes
|
|
required
|
output_dir
|
str
|
Output directory for files
|
required
|
Returns:
| Type |
Description |
TorrentCheckpoint
|
TorrentCheckpoint with current state
|
Source code in ccbt/piece/async_piece_manager.py
| async def get_checkpoint_state(
self,
torrent_name: str,
info_hash: bytes,
output_dir: str,
) -> TorrentCheckpoint:
"""Get current state for checkpointing.
Args:
torrent_name: Name of the torrent
info_hash: Torrent info hash
output_dir: Output directory for files
Returns:
TorrentCheckpoint with current state
"""
async with self.lock:
# Calculate download statistics
current_time = time.time()
download_time = current_time - self.download_start_time
average_speed = (
self.bytes_downloaded / download_time if download_time > 0 else 0
)
# Get piece states
piece_states = {}
for i, piece in enumerate(self.pieces):
piece_states[i] = PieceStateModel(piece.state.value)
# Get file information
files = []
if self.file_assembler: # pragma: no cover - File assembler integration, requires full session setup
file_paths = self.file_assembler.get_file_paths()
file_sizes = self.file_assembler.get_file_sizes()
files_exist = self.file_assembler.verify_files_exist()
# Map file paths to FileInfo to get BEP 47 attributes
file_info_map: dict[str, Any] = {}
if hasattr(self.file_assembler, "files") and hasattr(
self.file_assembler, "output_dir"
):
for file_info in self.file_assembler.files:
# Skip padding files in checkpoint
if file_info.is_padding:
continue
# Construct file path same way as in file_assembler
import os
if file_info.full_path:
file_path = os.path.join(
self.file_assembler.output_dir, file_info.full_path
)
elif file_info.path:
file_path = os.path.join(
self.file_assembler.output_dir, *file_info.path
)
else:
file_path = os.path.join(
self.file_assembler.output_dir, file_info.name
)
file_info_map[file_path] = file_info
# Create FileCheckpoint objects with BEP 47 attributes
for file_path in (
file_paths
): # pragma: no cover - File checkpoint creation, integration path
file_info = file_info_map.get(file_path)
files.append(
FileCheckpoint(
path=file_path,
size=file_sizes.get(file_path, 0),
exists=files_exist.get(file_path, False),
# BEP 47: Include file attributes in checkpoint
attributes=(
getattr(file_info, "attributes", None)
if file_info and hasattr(file_info, "attributes")
else None
),
symlink_path=(
getattr(file_info, "symlink_path", None)
if file_info and hasattr(file_info, "symlink_path")
else None
),
file_sha1=(
getattr(file_info, "file_sha1", None)
if file_info and hasattr(file_info, "file_sha1")
else None
),
),
)
# Create download stats
download_stats = DownloadStats(
bytes_downloaded=self.bytes_downloaded,
download_time=download_time,
average_speed=average_speed,
start_time=self.download_start_time,
last_update=current_time,
)
# Ensure info_hash is exactly 20 bytes for checkpoint schema
safe_info_hash = info_hash[:20]
# Get total_length safely - handle different torrent_data structures
total_length = 0
if self.torrent_data:
# Try normalized structure first (async_main format)
if self.torrent_data.get("file_info"):
total_length = self.torrent_data["file_info"].get("total_length", 0)
# Fallback to direct total_length key
elif "total_length" in self.torrent_data:
total_length = self.torrent_data["total_length"]
# Fallback to calculating from pieces
elif self.num_pieces > 0 and self.piece_length > 0:
# Calculate from piece info (last piece may be shorter, but this is close enough)
total_length = (self.num_pieces - 1) * self.piece_length
# Try to get last piece length if available
if hasattr(self, "pieces") and self.pieces:
last_piece = self.pieces[-1]
if hasattr(last_piece, "length"):
total_length = (
self.num_pieces - 1
) * self.piece_length + last_piece.length
# Create checkpoint
return TorrentCheckpoint(
info_hash=safe_info_hash,
torrent_name=torrent_name,
created_at=self.download_start_time,
updated_at=current_time,
total_pieces=self.num_pieces,
piece_length=self.piece_length,
total_length=total_length,
verified_pieces=list(self.verified_pieces),
piece_states=piece_states,
download_stats=download_stats,
output_dir=output_dir,
files=files,
peer_info=self._get_peer_info_summary(),
endgame_mode=self.endgame_mode,
)
|
get_completed_pieces() -> list[int]
Get list of completed piece indices.
Source code in ccbt/piece/async_piece_manager.py
| def get_completed_pieces(self) -> list[int]:
"""Get list of completed piece indices."""
return list(self.completed_pieces)
|
get_download_progress() -> float
Get download progress as a fraction (0.0 to 1.0).
Source code in ccbt/piece/async_piece_manager.py
| def get_download_progress(self) -> float:
"""Get download progress as a fraction (0.0 to 1.0)."""
# CRITICAL FIX: If num_pieces is 0, return 0.0 (not 1.0) - torrent not initialized yet
if self.num_pieces == 0:
return 0.0
# CRITICAL FIX: Ensure verified_pieces is a set and we're counting correctly
verified_count = len(self.verified_pieces) if self.verified_pieces else 0
# Validate that verified_count doesn't exceed num_pieces (shouldn't happen, but defensive)
if verified_count > self.num_pieces:
self.logger.warning(
"Verified pieces count (%d) exceeds total pieces (%d), capping at 100%%",
verified_count,
self.num_pieces,
)
return 1.0
progress = verified_count / self.num_pieces
# CRITICAL FIX: Only return 1.0 if we actually have all pieces verified
# Also check that we have pieces initialized
if progress >= 1.0 and len(self.pieces) == self.num_pieces:
# Double-check: verify that all pieces are actually verified
actual_verified = sum(
1 for piece in self.pieces if piece.state == PieceState.VERIFIED
)
if actual_verified == self.num_pieces:
return 1.0
# Some pieces aren't actually verified, recalculate
return actual_verified / self.num_pieces
return progress
|
get_download_rate() -> float
Get current download rate in bytes per second.
Returns:
| Type |
Description |
float
|
Download rate in bytes/second, or 0 if download hasn't started
|
Source code in ccbt/piece/async_piece_manager.py
| def get_download_rate(self) -> float:
"""Get current download rate in bytes per second.
Returns:
Download rate in bytes/second, or 0 if download hasn't started
"""
current_time = time.time()
download_time = current_time - self.download_start_time
if download_time > 0:
return self.bytes_downloaded / download_time
return 0.0
|
get_downloading_pieces() -> list[int]
Get list of downloading piece indices.
Source code in ccbt/piece/async_piece_manager.py
| def get_downloading_pieces(self) -> list[int]:
"""Get list of downloading piece indices."""
return [
i
for i, piece in enumerate(self.pieces)
if piece.state == PieceState.DOWNLOADING
]
|
get_missing_pieces() -> list[int]
Get list of missing piece indices.
Source code in ccbt/piece/async_piece_manager.py
| def get_missing_pieces(self) -> list[int]:
"""Get list of missing piece indices."""
# CRITICAL FIX: Handle case where pieces list is empty but num_pieces > 0
# This can happen when metadata arrives after piece manager initialization
if not self.pieces and self.num_pieces > 0:
self.logger.warning(
"Pieces list is empty but num_pieces=%d - returning all indices as missing. "
"Pieces should be initialized in start_download().",
self.num_pieces,
)
# Return all indices as missing - they will be initialized when needed
missing = list(range(self.num_pieces))
elif len(self.pieces) != self.num_pieces and self.num_pieces > 0:
# Pieces list length doesn't match num_pieces - this is a bug
self.logger.warning(
"Pieces list length (%d) doesn't match num_pieces (%d) - "
"returning missing pieces from existing list plus uninitialized indices",
len(self.pieces),
self.num_pieces,
)
# Get missing pieces from existing list with validation
missing = []
for i, piece in enumerate(self.pieces):
# CRITICAL FIX: Validate piece state - if state is COMPLETE but blocks aren't complete, treat as MISSING
if piece.state == PieceState.MISSING:
missing.append(i)
elif (
piece.state in (PieceState.COMPLETE, PieceState.VERIFIED)
and not piece.is_complete()
):
# State mismatch - reset to MISSING
self.logger.debug(
"Piece %d state is %s but not complete - treating as MISSING",
i,
piece.state.name,
)
piece.state = PieceState.MISSING
piece.hash_verified = False
missing.append(i)
# Add indices for pieces that haven't been initialized yet
missing.extend(range(len(self.pieces), self.num_pieces))
else:
# Normal case: pieces list matches num_pieces
missing = []
for i, piece in enumerate(self.pieces):
# CRITICAL FIX: Validate piece state - check both state and actual completion
if piece.state == PieceState.MISSING:
missing.append(i)
elif (
piece.state in (PieceState.COMPLETE, PieceState.VERIFIED)
and not piece.is_complete()
):
# If state says COMPLETE/VERIFIED but blocks aren't actually complete, treat as MISSING
self.logger.debug(
"Piece %d state is %s but not complete - treating as MISSING",
i,
piece.state.name,
)
piece.state = PieceState.MISSING
piece.hash_verified = False
missing.append(i)
# Filter by file selection if manager exists
if self.file_selection_manager:
missing = [
piece_idx
for piece_idx in missing
if self.file_selection_manager.is_piece_needed(piece_idx)
]
return missing
|
get_piece_availability(piece_index: int) -> int
async
Get the number of peers that have a specific piece.
Parameters:
| Name |
Type |
Description |
Default |
piece_index
|
int
|
Index of the piece to check
|
required
|
Returns:
| Type |
Description |
int
|
Number of peers that have this piece. Returns 0 if piece is not
|
int
|
available from any peer, or >0 indicating availability from N peers.
|
Note
The value represents the current count of peers that have this piece
and updates dynamically as peers connect/disconnect. This is used
for rarest-first piece selection prioritization.
Raises:
| Type |
Description |
ValueError
|
If piece_index is out of valid range
|
Source code in ccbt/piece/async_piece_manager.py
| async def get_piece_availability(self, piece_index: int) -> int:
"""Get the number of peers that have a specific piece.
Args:
piece_index: Index of the piece to check
Returns:
Number of peers that have this piece. Returns 0 if piece is not
available from any peer, or >0 indicating availability from N peers.
Note:
The value represents the current count of peers that have this piece
and updates dynamically as peers connect/disconnect. This is used
for rarest-first piece selection prioritization.
Raises:
ValueError: If piece_index is out of valid range
"""
async with self.lock:
# Validate piece_index range
if piece_index < 0 or piece_index >= len(self.pieces):
raise ValueError(
f"Piece index {piece_index} is out of range [0, {len(self.pieces)})"
)
# Return availability count from piece_frequency Counter
# Returns 0 if piece not in frequency dict (not available)
return self.piece_frequency.get(piece_index, 0)
|
get_piece_data(piece_index: int) -> bytes | None
Get complete piece data if available.
Source code in ccbt/piece/async_piece_manager.py
| def get_piece_data(self, piece_index: int) -> bytes | None:
"""Get complete piece data if available."""
if (
piece_index >= len(self.pieces)
): # pragma: no cover - Defensive bounds check, tested via get_piece_data_not_verified
return None
piece = self.pieces[piece_index]
if piece.state == PieceState.VERIFIED:
return piece.get_data()
return None
|
get_piece_status() -> dict[str, int]
Get piece status counts.
Source code in ccbt/piece/async_piece_manager.py
| def get_piece_status(self) -> dict[str, int]:
"""Get piece status counts."""
status_counts = defaultdict(int)
for piece in self.pieces:
status_counts[piece.state.value] += 1
return dict(status_counts)
|
get_stats() -> dict[str, Any]
Get piece manager statistics.
Source code in ccbt/piece/async_piece_manager.py
| def get_stats(self) -> dict[str, Any]:
"""Get piece manager statistics."""
return {
"total_pieces": self.num_pieces,
"completed_pieces": len(self.completed_pieces),
"verified_pieces": len(self.verified_pieces),
"missing_pieces": len(self.get_missing_pieces()),
"downloading_pieces": len(self.get_downloading_pieces()),
"progress": self.get_download_progress(),
"endgame_mode": self.endgame_mode,
"piece_frequency": dict(self.piece_frequency.most_common(10)),
"peer_count": len(self.peer_availability),
}
|
get_verified_pieces() -> list[int]
Get list of verified piece indices.
Source code in ccbt/piece/async_piece_manager.py
| def get_verified_pieces(self) -> list[int]:
"""Get list of verified piece indices."""
return list(self.verified_pieces)
|
handle_piece_block(piece_index: int, begin: int, data: bytes) -> None
async
Handle a received piece block.
Parameters:
| Name |
Type |
Description |
Default |
piece_index
|
int
|
|
required
|
begin
|
int
|
Starting offset of the block
|
required
|
data
|
bytes
|
|
required
|
Source code in ccbt/piece/async_piece_manager.py
| async def handle_piece_block(
self,
piece_index: int,
begin: int,
data: bytes,
) -> None:
"""Handle a received piece block.
Args:
piece_index: Index of the piece
begin: Starting offset of the block
data: Block data
"""
async with self.lock:
if (
piece_index >= len(self.pieces)
): # pragma: no cover - Bounds check in handle_piece_block, tested via out_of_range test
return
piece = self.pieces[piece_index]
# Add block to piece
if piece.add_block(begin, data):
if piece.state == PieceState.COMPLETE:
self.completed_pieces.add(piece_index)
self.logger.info(
"PIECE_MANAGER: Piece %d completed (all blocks received, state=COMPLETE)",
piece_index,
)
else:
# Log progress for incomplete pieces
missing_blocks = sum(1 for b in piece.blocks if not b.received)
total_blocks = len(piece.blocks)
self.logger.debug(
"PIECE_MANAGER: Piece %d block received (missing: %d/%d blocks, state=%s)",
piece_index,
missing_blocks,
total_blocks,
piece.state.value
if hasattr(piece.state, "value")
else str(piece.state),
)
# Update file progress if file selection manager exists
if self.file_selection_manager:
files_in_piece = self.file_selection_manager.get_files_for_piece(
piece_index,
)
for file_index in files_in_piece:
# Calculate bytes for this file in this piece
file_segments = [
(f_idx, f_off, f_len)
for f_idx, f_off, f_len in self.file_selection_manager.mapper.piece_to_files.get(
piece_index,
[],
)
if f_idx == file_index
]
bytes_for_file = sum(length for _, _, length in file_segments)
current_state = self.file_selection_manager.get_file_state(
file_index,
)
if current_state:
current_bytes = current_state.bytes_downloaded
await self.file_selection_manager.update_file_progress(
file_index,
current_bytes + bytes_for_file,
)
# Notify callback
if self.on_piece_completed:
self.on_piece_completed(piece_index)
# Schedule hash verification and keep a strong reference
_task = asyncio.create_task(
self._verify_piece_hash(piece_index, piece),
)
self._background_tasks.add(_task)
_task.add_done_callback(self._background_tasks.discard)
|
handle_streaming_seek(target_piece: int) -> None
async
Handle seek operation during streaming download.
Parameters:
| Name |
Type |
Description |
Default |
target_piece
|
int
|
|
required
|
Source code in ccbt/piece/async_piece_manager.py
| async def handle_streaming_seek(self, target_piece: int) -> None:
"""Handle seek operation during streaming download.
Args:
target_piece: Piece index to seek to
"""
async with self.lock:
# Update current sequential piece position
self._current_sequential_piece = target_piece
# Prioritize pieces around seek position
config = self.config
seek_window_start = max(0, target_piece - 2)
seek_window_end = min(
target_piece + config.strategy.sequential_window,
len(self.pieces),
)
# Add priority for seek window pieces
missing_pieces = self.get_missing_pieces()
for piece_idx in range(seek_window_start, seek_window_end):
if piece_idx in missing_pieces:
# Increase priority for pieces in seek window
self.pieces[piece_idx].priority += 500
# Trigger piece selection update
await self._select_sequential()
|
request_piece_from_peers(piece_index: int, peer_manager: Any) -> None
async
Request a piece from available peers using rarest-first or endgame logic.
Parameters:
| Name |
Type |
Description |
Default |
piece_index
|
int
|
Index of piece to request
|
required
|
peer_manager
|
Any
|
|
required
|
Source code in ccbt/piece/async_piece_manager.py
| async def request_piece_from_peers(
self,
piece_index: int,
peer_manager: Any,
) -> None:
"""Request a piece from available peers using rarest-first or endgame logic.
Args:
piece_index: Index of piece to request
peer_manager: Peer connection manager
"""
# CRITICAL FIX: Ensure pieces are initialized before requesting
# This handles the case where get_missing_pieces() returns indices but pieces list is empty
if self.num_pieces > 0 and len(self.pieces) == 0:
self.logger.warning(
"request_piece_from_peers called for piece %d but pieces list is empty (num_pieces=%d) - "
"initializing pieces now",
piece_index,
self.num_pieces,
)
# Initialize pieces on-the-fly (fallback - should have been initialized in start_download())
pieces_info = self.torrent_data.get("pieces_info", {})
if self.piece_length == 0 and "piece_length" in pieces_info:
self.piece_length = int(pieces_info.get("piece_length", 16384))
elif self.piece_length == 0:
self.piece_length = 16384
async with self.lock:
for i in range(self.num_pieces):
piece = PieceData(i, self.piece_length)
if self.config.strategy.streaming_mode:
if i == 0:
piece.priority = 1000
elif i == self.num_pieces - 1:
piece.priority = 100
else:
piece.priority = max(0, 1000 - i)
if self.file_selection_manager:
file_priority = self.file_selection_manager.get_piece_priority(
i
)
piece.priority = max(piece.priority, file_priority * 100)
self.pieces.append(piece)
self.logger.info(
"Initialized %d pieces in request_piece_from_peers (fallback)",
len(self.pieces),
)
async with self.lock:
# CRITICAL FIX: Handle case where piece_index is valid but pieces list hasn't caught up yet
if piece_index >= len(self.pieces):
if piece_index < self.num_pieces:
# Piece index is valid but piece hasn't been initialized yet
self.logger.warning(
"PIECE_MANAGER: Piece %d is valid (num_pieces=%d) but not yet initialized (pieces_list_len=%d) - "
"this should not happen if pieces were initialized properly",
piece_index,
self.num_pieces,
len(self.pieces),
)
return
piece = self.pieces[piece_index]
if piece.state != PieceState.MISSING:
self.logger.debug(
"PIECE_MANAGER: Piece %d is not MISSING (state=%s), skipping request",
piece_index,
piece.state.value
if hasattr(piece.state, "value")
else str(piece.state),
)
return
old_state = piece.state
piece.state = PieceState.REQUESTED
piece.request_count += 1
self.logger.info(
"PIECE_MANAGER: Piece %d state transition: %s -> REQUESTED (request_count=%d)",
piece_index,
old_state.value if hasattr(old_state, "value") else str(old_state),
piece.request_count,
)
# Get available peers for this piece
available_peers = await self._get_peers_for_piece(piece_index, peer_manager)
if not available_peers:
self.logger.debug(
"No available peers for piece %d (peer_manager=%s)",
piece_index,
peer_manager is not None,
)
async with self.lock:
piece.state = PieceState.MISSING
return
self.logger.info(
"Requesting piece %d from %d available peers (missing blocks: %d)",
piece_index,
len(available_peers),
len(piece.get_missing_blocks())
if hasattr(piece, "get_missing_blocks")
else 0,
)
# Get missing blocks
missing_blocks = piece.get_missing_blocks()
if not missing_blocks: # pragma: no cover - Early return when piece already complete, tested via early_returns test
self.logger.debug("Piece %d has no missing blocks, skipping", piece_index)
return
self.logger.debug(
"Piece %d has %d missing blocks, distributing among %d peers",
piece_index,
len(missing_blocks),
len(available_peers),
)
# Distribute blocks among peers
if (
self.endgame_mode
): # pragma: no cover - Endgame path tested separately, normal path is default
self.logger.debug("Requesting piece %d in endgame mode", piece_index)
await self._request_blocks_endgame(
piece_index,
missing_blocks,
available_peers,
peer_manager,
)
else:
self.logger.debug("Requesting piece %d in normal mode", piece_index)
await self._request_blocks_normal(
piece_index,
missing_blocks,
available_peers,
peer_manager,
)
async with self.lock: # pragma: no cover - Lock acquisition after request, state change tested via mark_requested
old_state = piece.state
piece.state = PieceState.DOWNLOADING
self.logger.info(
"PIECE_MANAGER: Piece %d state transition: %s -> DOWNLOADING",
piece_index,
old_state.value if hasattr(old_state, "value") else str(old_state),
)
|
restore_from_checkpoint(checkpoint: TorrentCheckpoint) -> None
async
Restore piece manager state from checkpoint.
Parameters:
| Name |
Type |
Description |
Default |
checkpoint
|
TorrentCheckpoint
|
Checkpoint data to restore from
|
required
|
Source code in ccbt/piece/async_piece_manager.py
| async def restore_from_checkpoint(
self, checkpoint: TorrentCheckpoint
) -> None: # pragma: no cover - Checkpoint restoration, requires full checkpoint integration
"""Restore piece manager state from checkpoint.
Args:
checkpoint: Checkpoint data to restore from
"""
async with self.lock: # pragma: no cover - Checkpoint restoration path
self.logger.info(
"Restoring piece manager from checkpoint: %s (total_pieces=%d, verified=%d, pieces_list_len=%d)",
checkpoint.torrent_name,
checkpoint.total_pieces,
len(checkpoint.verified_pieces),
len(self.pieces),
)
# CRITICAL FIX: Detect checkpoint corruption before restoring
# Check for impossible state: all pieces marked COMPLETE but no verified pieces and 0% downloaded
if checkpoint.piece_states:
complete_count = sum(
1
for state in checkpoint.piece_states.values()
if state in (PieceStateModel.COMPLETE, PieceStateModel.VERIFIED)
)
total_states = len(checkpoint.piece_states)
bytes_downloaded = (
checkpoint.download_stats.bytes_downloaded
if checkpoint.download_stats
else 0
)
# If all pieces are marked COMPLETE but no verified pieces and no bytes downloaded, likely corrupted
if (
complete_count == total_states
and len(checkpoint.verified_pieces) == 0
and bytes_downloaded == 0
and total_states > 0
):
self.logger.error(
"Checkpoint corruption detected: all %d pieces marked as COMPLETE but "
"0 verified pieces and 0 bytes downloaded. This checkpoint is likely corrupted. "
"Resetting all pieces to MISSING state.",
total_states,
)
# Don't restore piece states - they'll be initialized as MISSING below
checkpoint.piece_states = {}
checkpoint.verified_pieces = []
# CRITICAL FIX: Validate checkpoint data before restoring
# Ensure pieces list is initialized and matches checkpoint total_pieces
if checkpoint.total_pieces > 0 and len(self.pieces) == 0:
self.logger.warning(
"Checkpoint has total_pieces=%d but pieces list is empty - initializing pieces",
checkpoint.total_pieces,
)
# Initialize pieces if not already done
if self.num_pieces == 0:
self.num_pieces = checkpoint.total_pieces
if self.piece_length == 0:
self.piece_length = checkpoint.piece_length
# Initialize pieces
for i in range(self.num_pieces):
piece = PieceData(i, self.piece_length)
if self.config.strategy.streaming_mode:
if i == 0:
piece.priority = 1000
elif i == self.num_pieces - 1:
piece.priority = 100
else:
piece.priority = max(0, 1000 - i)
if self.file_selection_manager:
file_priority = self.file_selection_manager.get_piece_priority(
i
)
piece.priority = max(piece.priority, file_priority * 100)
self.pieces.append(piece)
self.logger.info(
"Initialized %d pieces from checkpoint", len(self.pieces)
)
# CRITICAL FIX: Ensure num_pieces is set after initializing pieces from checkpoint
# This ensures start_download() can use the pieces even if metadata isn't available
if self.num_pieces == 0 and len(self.pieces) > 0:
self.num_pieces = len(self.pieces)
self.logger.info(
"Set num_pieces=%d from checkpoint pieces (metadata not yet available)",
self.num_pieces,
)
# Validate checkpoint total_pieces matches current num_pieces
if checkpoint.total_pieces != self.num_pieces and self.num_pieces > 0:
self.logger.warning(
"Checkpoint total_pieces (%d) doesn't match current num_pieces (%d) - "
"using current num_pieces, may skip some checkpoint piece states",
checkpoint.total_pieces,
self.num_pieces,
)
# Validate checkpoint verified_pieces count is reasonable
if len(checkpoint.verified_pieces) > self.num_pieces:
self.logger.warning(
"Checkpoint has %d verified pieces but only %d total pieces - "
"truncating verified_pieces list",
len(checkpoint.verified_pieces),
self.num_pieces,
)
checkpoint.verified_pieces = [
idx
for idx in checkpoint.verified_pieces
if 0 <= idx < self.num_pieces
]
# Restore download state
# download_stats is guaranteed to be non-None by validator, but type checker doesn't know
if (
checkpoint.download_stats is None
): # pragma: no cover - Validator ensures non-None
checkpoint.download_stats = DownloadStats() # type: ignore[assignment]
self.download_start_time = (
checkpoint.download_stats.start_time
) # pragma: no cover - State restoration
self.bytes_downloaded = checkpoint.download_stats.bytes_downloaded
self.endgame_mode = checkpoint.endgame_mode
# CRITICAL FIX: Restore piece states with validation
# Only restore states for pieces that exist and are within valid range
restored_count = 0
skipped_count = 0
state_corrected_count = 0
for (
piece_idx,
piece_state,
) in (
checkpoint.piece_states.items()
): # pragma: no cover - Piece state restoration loop
if 0 <= piece_idx < len(self.pieces):
piece = self.pieces[piece_idx]
# CRITICAL FIX: Validate piece state - don't mark as verified unless in verified_pieces set
# This prevents incorrect state restoration from corrupted checkpoints
if piece_state == PieceStateModel.VERIFIED:
if piece_idx not in checkpoint.verified_pieces:
self.logger.warning(
"Checkpoint piece_states marks piece %d as VERIFIED but not in verified_pieces - "
"marking as COMPLETE instead",
piece_idx,
)
piece.state = PieceState.COMPLETE
piece.hash_verified = False
else:
piece.state = PieceState.VERIFIED
piece.hash_verified = True
else:
piece.state = PieceState(piece_state.value)
piece.hash_verified = piece_state == PieceStateModel.VERIFIED
# CRITICAL FIX: Validate that restored state matches actual block completion
# If checkpoint says COMPLETE/VERIFIED but blocks aren't received, reset to MISSING
if (
piece_state
in (PieceStateModel.COMPLETE, PieceStateModel.VERIFIED)
and not piece.is_complete()
):
self.logger.warning(
"Checkpoint marks piece %d as %s but blocks are not complete - "
"resetting to MISSING (possible checkpoint corruption)",
piece_idx,
piece_state.value,
)
piece.state = PieceState.MISSING
piece.hash_verified = False
state_corrected_count += 1
restored_count += 1
else:
skipped_count += 1
if skipped_count <= 5: # Log first 5 skipped pieces
self.logger.debug(
"Skipping checkpoint piece state for index %d (out of range: 0-%d)",
piece_idx,
len(self.pieces) - 1,
)
if skipped_count > 5:
self.logger.debug(
"Skipped %d additional checkpoint piece states (out of range)",
skipped_count - 5,
)
# CRITICAL FIX: Restore verified pieces with validation
# Only restore verified pieces that actually exist and are marked as verified in piece_states
verified_pieces_set = set(checkpoint.verified_pieces)
validated_verified = set()
for piece_idx in verified_pieces_set:
if 0 <= piece_idx < len(self.pieces):
piece = self.pieces[piece_idx]
# Only mark as verified if piece state is actually VERIFIED
if piece.state == PieceState.VERIFIED:
validated_verified.add(piece_idx)
else:
self.logger.debug(
"Piece %d in verified_pieces but state is %s - not adding to verified set",
piece_idx,
piece.state,
)
else:
self.logger.debug(
"Skipping verified piece index %d (out of range: 0-%d)",
piece_idx,
len(self.pieces) - 1,
)
self.verified_pieces = validated_verified
# Restore completed pieces (pieces that are complete but not yet verified)
self.completed_pieces = set()
for i, piece in enumerate(
self.pieces
): # pragma: no cover - Completed pieces restoration loop
if piece.state == PieceState.COMPLETE:
self.completed_pieces.add(i)
# Restore peer availability if available
if (
checkpoint.peer_info and "piece_frequency" in checkpoint.peer_info
): # pragma: no cover - Peer info restoration
self.piece_frequency = Counter(checkpoint.peer_info["piece_frequency"])
self.logger.info(
"Restored checkpoint: %d piece states, %d verified pieces (validated), "
"%d completed pieces, %d skipped states, %d state corrections",
restored_count,
len(self.verified_pieces),
len(self.completed_pieces),
skipped_count,
state_corrected_count,
)
|
start() -> None
async
Start background tasks.
Source code in ccbt/piece/async_piece_manager.py
| async def start(self) -> None:
"""Start background tasks."""
# No hash worker; schedule verifications per piece completion
self._piece_selector_task = asyncio.create_task(self._piece_selector())
self.logger.info("Async piece manager started")
|
start_download(peer_manager: Any) -> None
async
Start the download process.
Parameters:
| Name |
Type |
Description |
Default |
peer_manager
|
Any
|
|
required
|
CRITICAL FIX: Don't start downloads until metadata is available for magnet links.
Source code in ccbt/piece/async_piece_manager.py
| async def start_download(self, peer_manager: Any) -> None:
"""Start the download process.
Args:
peer_manager: Peer connection manager
CRITICAL FIX: Don't start downloads until metadata is available for magnet links.
"""
try:
# CRITICAL FIX: Re-check num_pieces from torrent_data in case metadata was updated
# This handles the case where metadata is fetched after piece manager initialization
pieces_info = self.torrent_data.get("pieces_info", {})
current_num_pieces = pieces_info.get("num_pieces", 0)
if isinstance(current_num_pieces, (int, float)):
current_num_pieces = int(current_num_pieces)
else:
current_num_pieces = 0
# Update num_pieces if it changed (metadata was fetched)
if current_num_pieces > 0 and self.num_pieces == 0:
self.logger.info(
"Metadata now available: updating num_pieces from %d to %d",
self.num_pieces,
current_num_pieces,
)
self.num_pieces = current_num_pieces
# Also update piece_length and piece_hashes if available
if "piece_length" in pieces_info:
self.piece_length = int(pieces_info.get("piece_length", 16384))
if "piece_hashes" in pieces_info:
piece_hashes_val = pieces_info.get("piece_hashes", [])
if isinstance(piece_hashes_val, (list, tuple)):
self.piece_hashes = list(piece_hashes_val)
# Re-initialize pieces if needed
if not self.pieces and self.num_pieces > 0:
self.logger.info(
"Initializing %d pieces after metadata fetch", self.num_pieces
)
# Initialize pieces (same logic as __init__)
for i in range(self.num_pieces):
# Calculate actual piece length (last piece may be shorter)
if i == self.num_pieces - 1:
# Get total_length safely - handle different torrent_data structures
total_length = 0
if (
"file_info" in self.torrent_data
and self.torrent_data.get("file_info")
):
total_length = self.torrent_data["file_info"].get(
"total_length", 0
)
elif "total_length" in self.torrent_data:
total_length = self.torrent_data["total_length"]
else:
# Fallback: calculate from pieces (approximation)
total_length = self.num_pieces * self.piece_length
piece_length = total_length - (i * self.piece_length)
# Ensure piece_length is positive
if piece_length <= 0:
piece_length = self.piece_length
else:
piece_length = self.piece_length
piece = PieceData(i, piece_length)
# Set priorities for streaming mode
if self.config.strategy.streaming_mode:
if i == 0:
piece.priority = 1000 # First piece highest priority
elif i == self.num_pieces - 1:
# Fallback: boost last piece modestly
piece.priority = 100
else:
piece.priority = max(0, 1000 - i) # Decreasing priority
# Apply file-based priorities if file selection manager exists
if self.file_selection_manager:
file_priority = (
self.file_selection_manager.get_piece_priority(i)
)
# Scale file priority to piece priority (multiply by 100 to match streaming mode scale)
piece.priority = max(piece.priority, file_priority * 100)
self.pieces.append(piece)
self.logger.info(
"start_download() called: peer_manager=%s, num_pieces=%d, pieces_count=%d, is_downloading=%s",
peer_manager is not None,
self.num_pieces,
len(self.pieces),
self.is_downloading,
)
# CRITICAL FIX: Ensure pieces are initialized if num_pieces > 0 but pieces list is empty
# This handles cases where num_pieces was updated but pieces weren't initialized
if self.num_pieces > 0 and len(self.pieces) == 0:
self.logger.warning(
"num_pieces=%d but pieces list is empty - initializing pieces now",
self.num_pieces,
)
# Initialize pieces using the same logic as __init__
pieces_info = self.torrent_data.get("pieces_info", {})
if "piece_length" in pieces_info:
self.piece_length = int(pieces_info.get("piece_length", 16384))
elif self.piece_length == 0:
self.piece_length = 16384 # Default 16KB
for i in range(self.num_pieces):
# Calculate actual piece length (last piece may be shorter)
if i == self.num_pieces - 1:
total_length = 0
if "file_info" in self.torrent_data and self.torrent_data.get(
"file_info"
):
total_length = self.torrent_data["file_info"].get(
"total_length", 0
)
elif "total_length" in self.torrent_data:
total_length = self.torrent_data["total_length"]
else:
total_length = self.num_pieces * self.piece_length
piece_length = total_length - (i * self.piece_length)
if piece_length <= 0:
piece_length = self.piece_length
else:
piece_length = self.piece_length
piece = PieceData(i, piece_length)
# Set priorities for streaming mode
if self.config.strategy.streaming_mode:
if i == 0:
piece.priority = 1000
elif i == self.num_pieces - 1:
piece.priority = 100
else:
piece.priority = max(0, 1000 - i)
# Apply file-based priorities if available
if self.file_selection_manager:
file_priority = self.file_selection_manager.get_piece_priority(
i
)
piece.priority = max(piece.priority, file_priority * 100)
self.pieces.append(piece)
self.logger.info(
"Initialized %d pieces in start_download()", len(self.pieces)
)
# CRITICAL FIX: Check if pieces were initialized from checkpoint
# If pieces list has items but num_pieces is 0, use pieces count
if self.num_pieces == 0 and len(self.pieces) > 0:
self.num_pieces = len(self.pieces)
self.logger.info(
"Inferred num_pieces=%d from restored pieces list (checkpoint had pieces but num_pieces was 0)",
self.num_pieces,
)
# CRITICAL FIX: Check if metadata is available before starting download
# For magnet links, num_pieces will be 0 until metadata is fetched
# CRITICAL FIX: Allow download start even without metadata for magnet links
# This allows peer connections to proceed while metadata is being fetched
if self.num_pieces == 0:
# CRITICAL FIX: Try to infer num_pieces from peer data if available
# This handles the case where peers are sending Have messages but metadata hasn't been fetched yet
inferred_num_pieces = 0
max_piece_index = -1
connections_checked = 0
connections_with_pieces = 0
if peer_manager is not None and hasattr(peer_manager, "connections"):
connections_dict = peer_manager.connections
total_connections = len(connections_dict) if connections_dict else 0
self.logger.info(
"Attempting to infer num_pieces from %d peer connections (num_pieces=0, metadata not available)",
total_connections,
)
# Check all peer connections for the highest piece index seen
# CRITICAL FIX: Use connection_lock if available, otherwise access connections directly
if hasattr(peer_manager, "connection_lock"):
async with peer_manager.connection_lock:
connections_to_check = list(connections_dict.values())
else:
connections_to_check = list(connections_dict.values())
for connection in connections_to_check:
connections_checked += 1
# Track if connection reported any pieces (for diagnostics only)
# Check pieces_we_have set
if hasattr(connection, "peer_state") and hasattr(
connection.peer_state, "pieces_we_have"
):
pieces_we_have = connection.peer_state.pieces_we_have
if pieces_we_have:
connections_with_pieces += 1
max_piece = max(pieces_we_have)
max_piece_index = max(max_piece_index, max_piece)
self.logger.debug(
"Connection %s has %d pieces, max piece index: %d",
connection.peer_info,
len(pieces_we_have),
max_piece,
)
# Also check bitfield if available
if (
hasattr(connection, "peer_state")
and hasattr(connection.peer_state, "bitfield")
and connection.peer_state.bitfield
):
bitfield = connection.peer_state.bitfield
bitfield_length = len(bitfield)
# Infer num_pieces from bitfield length (bitfield has 8 bits per byte, with padding)
inferred_from_bitfield = bitfield_length * 8
# Subtract padding (last byte may have unused bits)
# Conservative estimate: assume at least 1 bit is used in last byte
inferred_from_bitfield = max(1, inferred_from_bitfield - 7)
if inferred_from_bitfield > inferred_num_pieces:
inferred_num_pieces = inferred_from_bitfield
self.logger.debug(
"Inferred num_pieces=%d from bitfield length (%d bytes) from connection %s",
inferred_from_bitfield,
bitfield_length,
connection.peer_info,
)
# If we found a piece index, infer num_pieces as piece_index + 1 with safety margin
if max_piece_index >= 0:
inferred_from_have = max_piece_index + 1
# Add 20% safety margin (round up) to account for pieces we haven't seen yet
inferred_from_have = int(inferred_from_have * 1.2) + 1
inferred_num_pieces = max(
inferred_num_pieces, inferred_from_have
)
self.logger.info(
"Inferred num_pieces=%d from max piece index %d (checked %d connections, %d with pieces)",
inferred_num_pieces,
max_piece_index,
connections_checked,
connections_with_pieces,
)
else:
self.logger.debug(
"No piece indices found in %d connections (checked %d, %d with pieces)",
total_connections,
connections_checked,
connections_with_pieces,
)
# If we inferred num_pieces from peer data, use it
if inferred_num_pieces > 0:
self.logger.info(
"Inferred num_pieces=%d from peer data (Have messages/bitfields) - metadata not yet available",
inferred_num_pieces,
)
self.num_pieces = inferred_num_pieces
# Initialize pieces with inferred count
if not self.pieces:
self.logger.info(
"Initializing %d pieces from inferred count",
self.num_pieces,
)
# Try to get piece_length from torrent_data, otherwise use default
if self.piece_length == 0:
pieces_info = self.torrent_data.get("pieces_info", {})
if "piece_length" in pieces_info:
self.piece_length = int(
pieces_info.get("piece_length", 16384)
)
else:
self.piece_length = 16384 # Default 16KB
# Initialize pieces (simplified - we don't have exact piece lengths)
for i in range(self.num_pieces):
piece = PieceData(i, self.piece_length)
# Set priorities for streaming mode
if self.config.strategy.streaming_mode:
if i == 0:
piece.priority = 1000
elif i == self.num_pieces - 1:
piece.priority = 100
else:
piece.priority = max(0, 1000 - i)
# Apply file-based priorities if available
if self.file_selection_manager:
file_priority = (
self.file_selection_manager.get_piece_priority(
i
)
)
piece.priority = max(
piece.priority, file_priority * 100
)
self.pieces.append(piece)
else:
self.logger.warning(
"Cannot start download: metadata not available yet (num_pieces=0) and cannot infer from peer data "
"(checked %d connections, %d with pieces). "
"This is normal for magnet links - download will start after metadata is fetched.",
connections_checked,
connections_with_pieces,
)
# CRITICAL FIX: Store peer_manager for later use when metadata is available
# But also set is_downloading=True to allow piece selector to run
# This allows the system to be ready when metadata arrives
if peer_manager is not None:
self._peer_manager = peer_manager
# CRITICAL FIX: Set is_downloading=True even without metadata
# This allows piece selector to run and be ready when metadata arrives
# The piece selector will handle num_pieces=0 gracefully
self.is_downloading = True
self.logger.info(
"Set is_downloading=True even without metadata (num_pieces=0) to allow piece selector to run when metadata arrives"
)
return
# CRITICAL FIX: Verify peer_manager is valid
if peer_manager is None:
self.logger.error("Cannot start download: peer_manager is None")
return
# CRITICAL FIX: Check if already downloading to avoid duplicate starts
if self.is_downloading:
self.logger.debug(
"Download already started (is_downloading=True), skipping duplicate start"
)
# Still ensure _peer_manager is set in case it wasn't before
if self._peer_manager is None and peer_manager is not None:
self._peer_manager = peer_manager
self.logger.debug(
"Set _peer_manager reference in piece manager (was None)"
)
return
# CRITICAL FIX: Set _peer_manager BEFORE is_downloading to ensure it's available for piece selection
# This prevents piece selector from running with None peer_manager
self._peer_manager = peer_manager
self.logger.debug(
"Set _peer_manager reference in piece manager (peer_manager is not None)"
)
# CRITICAL FIX: Validate that _peer_manager is set before proceeding
if self._peer_manager is None:
self.logger.error(
"Cannot start download: _peer_manager is None after assignment"
)
return
# Set is_downloading to True - this must happen after _peer_manager is set
self.is_downloading = True
self.logger.info(
"Piece manager download started (is_downloading=True, _peer_manager=%s, num_pieces=%d)",
self._peer_manager is not None,
self.num_pieces,
)
# CRITICAL FIX: Trigger initial piece selection after starting download
# This ensures pieces are requested as soon as download starts
# Add a small delay to ensure peer_manager is fully ready
await asyncio.sleep(0.1) # Small delay to ensure peer_manager is ready
try:
task = asyncio.create_task(self._select_pieces())
_ = task # Store reference to avoid unused variable warning
self.logger.debug(
"Triggered initial piece selection after starting download"
)
except Exception:
self.logger.exception(
"Error triggering initial piece selection after starting download"
)
# Don't fail the entire start_download() if piece selection fails
# The piece selector loop will retry
except Exception as e:
self.logger.exception("Error starting download: %s", e)
# Only reset state if we actually set it
if self.is_downloading:
self.is_downloading = False
# Don't clear _peer_manager if it was set before - might be needed for retry
if self._peer_manager == peer_manager:
self._peer_manager = None
|
stop() -> None
async
Stop background tasks.
Source code in ccbt/piece/async_piece_manager.py
| async def stop(self) -> None:
"""Stop background tasks."""
if self._piece_selector_task:
self._piece_selector_task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await self._piece_selector_task
self.hash_executor.shutdown(wait=True)
self.logger.info("Async piece manager stopped")
|
stop_download() -> None
async
Stop the download process.
Source code in ccbt/piece/async_piece_manager.py
| async def stop_download(self) -> None:
"""Stop the download process."""
self.is_downloading = False
self.logger.info("Stopped piece download")
|
update_download_stats(bytes_downloaded: int) -> None
async
Update download statistics.
Source code in ccbt/piece/async_piece_manager.py
| async def update_download_stats(
self, bytes_downloaded: int
) -> None: # pragma: no cover - Stats update method, called by session/peer manager during downloads
"""Update download statistics."""
async with self.lock: # pragma: no cover - Stats update path
self.bytes_downloaded += bytes_downloaded
|
update_peer_availability(peer_key: str, bitfield: bytes) -> None
async
Update peer availability from bitfield.
Parameters:
| Name |
Type |
Description |
Default |
peer_key
|
str
|
|
required
|
bitfield
|
bytes
|
|
required
|
Source code in ccbt/piece/async_piece_manager.py
| async def update_peer_availability(self, peer_key: str, bitfield: bytes) -> None:
"""Update peer availability from bitfield.
Args:
peer_key: Unique key for the peer
bitfield: Bitfield data from peer
"""
self.logger.info(
"update_peer_availability called for peer %s (bitfield length: %d bytes, num_pieces: %d)",
peer_key,
len(bitfield) if bitfield else 0,
self.num_pieces,
)
async with self.lock:
# Parse bitfield
pieces = set()
for byte_idx, byte_val in enumerate(bitfield):
for bit_idx in range(8):
piece_idx = byte_idx * 8 + bit_idx
if piece_idx < self.num_pieces and (
byte_val & (1 << (7 - bit_idx))
):
pieces.add(piece_idx)
self.logger.info(
"Parsed bitfield for peer %s: %d pieces available (sample: %s)",
peer_key,
len(pieces),
sorted(pieces)[:10] if pieces else [],
)
# Update peer availability
if peer_key not in self.peer_availability:
self.peer_availability[peer_key] = PeerAvailability(peer_key)
self.logger.info("Created new peer availability entry for %s", peer_key)
old_pieces = self.peer_availability[peer_key].pieces
self.peer_availability[peer_key].pieces = pieces
self.peer_availability[peer_key].last_updated = time.time()
# Update piece frequency
for piece_idx in (
old_pieces - pieces
): # pragma: no cover - Edge case when peer loses pieces, rare in practice
self.piece_frequency[piece_idx] -= 1
for piece_idx in pieces - old_pieces:
self.piece_frequency[piece_idx] += 1
self.logger.info(
"Updated peer availability for %s: %d pieces (was %d), piece_frequency updated",
peer_key,
len(pieces),
len(old_pieces),
)
|
update_peer_have(peer_key: str, piece_index: int) -> None
async
Update peer availability for a single piece.
Source code in ccbt/piece/async_piece_manager.py
| async def update_peer_have(self, peer_key: str, piece_index: int) -> None:
"""Update peer availability for a single piece."""
async with self.lock:
if peer_key not in self.peer_availability:
self.peer_availability[peer_key] = PeerAvailability(peer_key)
old_has_piece = piece_index in self.peer_availability[peer_key].pieces
self.peer_availability[peer_key].pieces.add(piece_index)
self.peer_availability[peer_key].last_updated = time.time()
# Update piece frequency
if not old_has_piece:
self.piece_frequency[piece_index] += 1
|
Features:
- Rarest-first piece selection
- Sequential piece selection
- Round-robin piece selection
- Endgame mode with duplicate requests
- File selection integration: ccbt/piece/async_piece_manager.py - Filters pieces based on file selection state
Configuration: ccbt.toml:99-114
FileSelectionManager
Manages file selection and prioritization for multi-file torrents.
Manages file selection state and piece filtering.
Initialize file selection manager.
Parameters:
| Name |
Type |
Description |
Default |
torrent_info
|
TorrentInfo
|
Parsed torrent information
|
required
|
Source code in ccbt/piece/file_selection.py
| def __init__(self, torrent_info: TorrentInfo):
"""Initialize file selection manager.
Args:
torrent_info: Parsed torrent information
"""
self.torrent_info = torrent_info
self.mapper = PieceToFileMapper(torrent_info)
self.logger = logging.getLogger(__name__)
# File selection states - indexed by file_index
# Note: Padding files (BEP 47) are excluded from selection states
self.file_states: dict[int, FileSelectionState] = {}
# Initialize all non-padding files as selected by default
for file_index, file_info in enumerate(torrent_info.files):
# Skip padding files - they should never be downloaded
if file_info.is_padding:
self.logger.debug(
"Skipping padding file %s: %s",
file_index,
file_info.full_path or file_info.name,
)
continue
self.file_states[file_index] = FileSelectionState(
file_index=file_index,
selected=True,
priority=FilePriority.NORMAL,
bytes_total=file_info.length,
)
self.lock = asyncio.Lock()
|
deselect_all() -> None
async
Deselect all files.
Source code in ccbt/piece/file_selection.py
| async def deselect_all(self) -> None:
"""Deselect all files."""
async with self.lock:
for file_state in self.file_states.values():
file_state.selected = False
|
deselect_file(file_index: int) -> None
async
Deselect a file from download.
Parameters:
| Name |
Type |
Description |
Default |
file_index
|
int
|
Index of file to deselect
|
required
|
Source code in ccbt/piece/file_selection.py
| async def deselect_file(self, file_index: int) -> None:
"""Deselect a file from download.
Args:
file_index: Index of file to deselect
"""
async with self.lock:
if file_index in self.file_states:
self.file_states[file_index].selected = False
self.logger.info(
"Deselected file %s: %s",
file_index,
self.torrent_info.files[file_index].name,
)
|
deselect_files(file_indices: list[int]) -> None
async
Deselect multiple files.
Parameters:
| Name |
Type |
Description |
Default |
file_indices
|
list[int]
|
List of file indices to deselect
|
required
|
Source code in ccbt/piece/file_selection.py
| async def deselect_files(self, file_indices: list[int]) -> None:
"""Deselect multiple files.
Args:
file_indices: List of file indices to deselect
"""
async with self.lock:
for file_index in file_indices:
if file_index in self.file_states:
self.file_states[file_index].selected = False
|
get_all_file_states() -> dict[int, FileSelectionState]
Get all file selection states.
Returns:
| Type |
Description |
dict[int, FileSelectionState]
|
Dictionary mapping file_index to FileSelectionState
|
Source code in ccbt/piece/file_selection.py
| def get_all_file_states(self) -> dict[int, FileSelectionState]:
"""Get all file selection states.
Returns:
Dictionary mapping file_index to FileSelectionState
"""
return self.file_states.copy()
|
get_file_priority(file_index: int) -> FilePriority
Get priority for a file.
Parameters:
| Name |
Type |
Description |
Default |
file_index
|
int
|
|
required
|
Returns:
| Type |
Description |
FilePriority
|
|
Source code in ccbt/piece/file_selection.py
| def get_file_priority(self, file_index: int) -> FilePriority:
"""Get priority for a file.
Args:
file_index: Index of file
Returns:
File priority level
"""
if file_index not in self.file_states:
return FilePriority.NORMAL
return self.file_states[file_index].priority
|
get_file_state(file_index: int) -> FileSelectionState | None
Get selection state for a file.
Parameters:
| Name |
Type |
Description |
Default |
file_index
|
int
|
|
required
|
Returns:
| Type |
Description |
FileSelectionState | None
|
FileSelectionState or None if not found
|
Source code in ccbt/piece/file_selection.py
| def get_file_state(self, file_index: int) -> FileSelectionState | None:
"""Get selection state for a file.
Args:
file_index: Index of file
Returns:
FileSelectionState or None if not found
"""
return self.file_states.get(file_index)
|
get_files_for_piece(piece_index: int) -> list[int]
Get list of file indices that contain this piece.
Parameters:
| Name |
Type |
Description |
Default |
piece_index
|
int
|
|
required
|
Returns:
| Type |
Description |
list[int]
|
|
Source code in ccbt/piece/file_selection.py
| def get_files_for_piece(self, piece_index: int) -> list[int]:
"""Get list of file indices that contain this piece.
Args:
piece_index: Index of piece
Returns:
List of file indices
"""
if piece_index not in self.mapper.piece_to_files:
return []
return [
file_index for file_index, _, _ in self.mapper.piece_to_files[piece_index]
]
|
get_piece_priority(piece_index: int) -> int
Get priority for a piece based on file priorities.
Piece priority is the maximum priority of any selected file in the piece.
Parameters:
| Name |
Type |
Description |
Default |
piece_index
|
int
|
|
required
|
Returns:
| Type |
Description |
int
|
Priority value (higher = more important)
|
Source code in ccbt/piece/file_selection.py
| def get_piece_priority(self, piece_index: int) -> int:
"""Get priority for a piece based on file priorities.
Piece priority is the maximum priority of any selected file in the piece.
Args:
piece_index: Index of piece
Returns:
Priority value (higher = more important)
"""
if piece_index not in self.mapper.piece_to_files:
return 0
max_priority = FilePriority.DO_NOT_DOWNLOAD
for file_index, _, _ in self.mapper.piece_to_files[piece_index]:
if self.is_file_selected(file_index):
file_priority = self.get_file_priority(file_index)
max_priority = max(max_priority, file_priority)
return int(max_priority.value)
|
get_pieces_for_file(file_index: int) -> list[int]
Get list of piece indices that belong to this file.
Parameters:
| Name |
Type |
Description |
Default |
file_index
|
int
|
|
required
|
Returns:
| Type |
Description |
list[int]
|
|
Source code in ccbt/piece/file_selection.py
| def get_pieces_for_file(self, file_index: int) -> list[int]:
"""Get list of piece indices that belong to this file.
Args:
file_index: Index of file
Returns:
List of piece indices
"""
return self.mapper.file_to_pieces.get(file_index, [])
|
get_selected_files() -> list[int]
Get list of selected file indices.
Returns:
| Type |
Description |
list[int]
|
List of file indices that are selected
|
Source code in ccbt/piece/file_selection.py
| def get_selected_files(self) -> list[int]:
"""Get list of selected file indices.
Returns:
List of file indices that are selected
"""
return [
file_index
for file_index, state in self.file_states.items()
if state.selected
]
|
get_statistics() -> dict[str, Any]
Get file selection statistics.
Returns:
| Type |
Description |
dict[str, Any]
|
Dictionary with selection statistics
|
Source code in ccbt/piece/file_selection.py
| def get_statistics(self) -> dict[str, Any]:
"""Get file selection statistics.
Returns:
Dictionary with selection statistics
"""
# Count non-padding files only
non_padding_files = [
(idx, f)
for idx, f in enumerate(self.torrent_info.files)
if not f.is_padding
]
padding_files = [
(idx, f) for idx, f in enumerate(self.torrent_info.files) if f.is_padding
]
total_files = len(non_padding_files)
selected_files = len(self.get_selected_files())
total_size = sum(f.length for _, f in non_padding_files)
selected_size = sum(
self.torrent_info.files[file_index].length
for file_index in self.get_selected_files()
)
padding_size = sum(f.length for _, f in padding_files)
return {
"total_files": total_files,
"selected_files": selected_files,
"deselected_files": total_files - selected_files,
"padding_files": len(padding_files),
"total_size": total_size,
"selected_size": selected_size,
"deselected_size": total_size - selected_size,
"padding_size": padding_size,
}
|
is_file_selected(file_index: int) -> bool
Check if a file is selected.
Parameters:
| Name |
Type |
Description |
Default |
file_index
|
int
|
|
required
|
Returns:
Source code in ccbt/piece/file_selection.py
| def is_file_selected(self, file_index: int) -> bool:
"""Check if a file is selected.
Args:
file_index: Index of file
Returns:
True if file is selected
"""
if file_index not in self.file_states:
return False
return self.file_states[file_index].selected
|
is_piece_needed(piece_index: int) -> bool
Check if a piece is needed based on file selection.
A piece is needed if at least one of its containing files is selected.
Parameters:
| Name |
Type |
Description |
Default |
piece_index
|
int
|
|
required
|
Returns:
| Type |
Description |
bool
|
True if piece should be downloaded
|
Source code in ccbt/piece/file_selection.py
| def is_piece_needed(self, piece_index: int) -> bool:
"""Check if a piece is needed based on file selection.
A piece is needed if at least one of its containing files is selected.
Args:
piece_index: Index of piece
Returns:
True if piece should be downloaded
"""
if piece_index not in self.mapper.piece_to_files:
return True # Default to needed if not in mapping
for file_index, _, _ in self.mapper.piece_to_files[piece_index]:
if self.is_file_selected(file_index):
return True
return False
|
select_all() -> None
async
Select all files.
Source code in ccbt/piece/file_selection.py
| async def select_all(self) -> None:
"""Select all files."""
async with self.lock:
for file_state in self.file_states.values():
file_state.selected = True
|
select_file(file_index: int) -> None
async
Select a file for download.
Parameters:
| Name |
Type |
Description |
Default |
file_index
|
int
|
|
required
|
Source code in ccbt/piece/file_selection.py
| async def select_file(self, file_index: int) -> None:
"""Select a file for download.
Args:
file_index: Index of file to select
"""
async with self.lock:
if file_index in self.file_states:
self.file_states[file_index].selected = True
self.logger.info(
"Selected file %s: %s",
file_index,
self.torrent_info.files[file_index].name,
)
|
select_files(file_indices: list[int]) -> None
async
Select multiple files.
Parameters:
| Name |
Type |
Description |
Default |
file_indices
|
list[int]
|
List of file indices to select
|
required
|
Source code in ccbt/piece/file_selection.py
| async def select_files(self, file_indices: list[int]) -> None:
"""Select multiple files.
Args:
file_indices: List of file indices to select
"""
async with self.lock:
for file_index in file_indices:
if file_index in self.file_states:
self.file_states[file_index].selected = True
|
set_file_priority(file_index: int, priority: FilePriority) -> None
async
Set priority for a file.
Parameters:
| Name |
Type |
Description |
Default |
file_index
|
int
|
|
required
|
priority
|
FilePriority
|
|
required
|
Source code in ccbt/piece/file_selection.py
| async def set_file_priority(self, file_index: int, priority: FilePriority) -> None:
"""Set priority for a file.
Args:
file_index: Index of file
priority: Priority level
"""
async with self.lock:
if file_index in self.file_states:
self.file_states[file_index].priority = priority
self.logger.info(
"Set file %s priority to %s",
file_index,
priority,
)
|
update_file_progress(file_index: int, bytes_downloaded: int) -> None
async
Update download progress for a file.
Parameters:
| Name |
Type |
Description |
Default |
file_index
|
int
|
|
required
|
bytes_downloaded
|
int
|
Number of bytes downloaded for this file
|
required
|
Source code in ccbt/piece/file_selection.py
| async def update_file_progress(
self, file_index: int, bytes_downloaded: int
) -> None:
"""Update download progress for a file.
Args:
file_index: Index of file
bytes_downloaded: Number of bytes downloaded for this file
"""
async with self.lock:
if file_index in self.file_states:
self.file_states[file_index].bytes_downloaded = bytes_downloaded
|
Features:
- File selection state management: ccbt/piece/file_selection.py:FileSelectionState - Tracks selection, priority, and progress per file
- File priority system: ccbt/piece/file_selection.py:FilePriority - Priority levels (DO_NOT_DOWNLOAD, LOW, NORMAL, HIGH, MAXIMUM)
- Piece-to-file mapping: ccbt/piece/file_selection.py:PieceToFileMapper - Efficient bidirectional mapping between pieces and files
- Piece filtering: ccbt/piece/file_selection.py:is_piece_needed - Determines if a piece should be downloaded based on file selection
- Priority-based piece selection: ccbt/piece/file_selection.py:get_piece_priority - Calculates piece priority from file priorities
- Progress tracking: ccbt/piece/file_selection.py:update_file_progress - Updates download progress per file
Key Methods:
- select_file(file_index): ccbt/piece/file_selection.py:select_file - Select a file for download
- deselect_file(file_index): ccbt/piece/file_selection.py:deselect_file - Deselect a file from download
- set_file_priority(file_index, priority): ccbt/piece/file_selection.py:set_file_priority - Set file download priority
- is_piece_needed(piece_index): ccbt/piece/file_selection.py:is_piece_needed - Check if a piece is needed based on file selection
- get_piece_priority(piece_index): ccbt/piece/file_selection.py:get_piece_priority - Get priority for a piece based on file priorities
- get_statistics(): ccbt/piece/file_selection.py:get_statistics - Get file selection statistics
Integration:
- Integrated with AsyncPieceManager: ccbt/piece/async_piece_manager.py - File selection manager passed during initialization
- Integrated with AsyncTorrentSession: ccbt/session/session.py - Automatically created for multi-file torrents
- Checkpoint persistence: ccbt/session/session.py - File selection state saved/restored in checkpoints
PieceManager
Synchronous piece manager (legacy).
Implementation: ccbt/piece/piece_manager.py
Parallel metadata fetching with reliability scoring.
Implementation: ccbt/piece/async_metadata_exchange.py
Features:
- Concurrent metadata fetching from multiple peers
- Reliability scoring
- Failure handling
Synchronous metadata exchange (legacy).
Implementation: ccbt/piece/metadata_exchange.py
Protocols
BaseProtocol
Base protocol implementation.
Implementation: ccbt/protocols/base.py
Protocol types: ccbt/protocols/base.py:ProtocolType
Protocol states: ccbt/protocols/base.py:ProtocolState
BitTorrentProtocol
Standard BitTorrent protocol implementation.
Implementation: ccbt/protocols/bittorrent.py
Features:
- BitTorrent protocol message handling
- Handshake negotiation
- Piece requests and responses
HybridProtocol
Hybrid protocol supporting multiple transport methods.
Implementation: ccbt/protocols/hybrid.py
WebTorrentProtocol
WebTorrent protocol support.
Implementation: ccbt/protocols/webtorrent.py
IPFSProtocol
IPFS protocol integration for decentralized content addressing and peer-to-peer networking.
Implementation: ccbt/protocols/ipfs.py
Requirements:
- IPFS daemon must be running (default: http://127.0.0.1:5001)
- Dependencies: ipfshttpclient>=0.8.0a2, multiaddr>=0.0.9, py-multiformats>=0.2.1
Features:
- IPFS daemon integration via HTTP API
- Content addressing with CID (Content Identifier)
- Peer-to-peer messaging via IPFS pubsub
- Content discovery via DHT (Distributed Hash Table)
- Content operations: add, get, pin, unpin
- Torrent-to-IPFS conversion
- Gateway fallback support
- Automatic content pinning (configurable)
Configuration:
- API URL: config.ipfs.api_url (default: http://127.0.0.1:5001)
- Gateway URLs: config.ipfs.gateway_urls (fallback for content retrieval)
- Enable pinning: config.ipfs.enable_pinning (default: False)
- Connection timeout: config.ipfs.connection_timeout (default: 30s)
- Request timeout: config.ipfs.request_timeout (default: 30s)
- DHT enabled: config.ipfs.enable_dht (default: True)
- Discovery cache TTL: config.ipfs.discovery_cache_ttl (default: 300s)
Methods:
start(): ccbt/protocols/ipfs.py:127- Connect to IPFS daemon and initialize protocol
- Verifies connection by querying node ID
-
Sets protocol state to CONNECTED
-
stop(): ccbt/protocols/ipfs.py:151
- Disconnect from IPFS daemon and cleanup resources
- Closes all peer connections
-
Sets protocol state to DISCONNECTED
-
connect_peer(peer_info: PeerInfo) -> bool: ccbt/protocols/ipfs.py:300
- Connect to an IPFS peer using multiaddr format
- Parses peer multiaddr and validates peer ID
- Sets up message listener for peer communication
-
Returns True on success, False on failure
-
disconnect_peer(peer_id: str) -> None: ccbt/protocols/ipfs.py:450
- Disconnect from an IPFS peer
-
Cleans up message queues and listeners
-
send_message(peer_id: str, message: bytes) -> bool: ccbt/protocols/ipfs.py:470
- Send message to IPFS peer via pubsub
- Creates topic from peer_id:
/ccbt/peer/{peer_id}
- Validates message size (max 1MB)
-
Returns True on success, False on failure
-
receive_message(peer_id: str) -> bytes | None: ccbt/protocols/ipfs.py:519
- Receive message from IPFS peer
- Waits up to 1 second for message from peer queue
-
Returns message bytes or None if timeout
-
announce_torrent(torrent_info: TorrentInfo) -> list[PeerInfo]: ccbt/protocols/ipfs.py:550
- Announce torrent to IPFS network
- Converts torrent to IPFS content (CID)
- Discovers peers providing the content via DHT
-
Returns list of peer information
-
scrape_torrent(torrent_info: TorrentInfo) -> dict[str, int]: ccbt/protocols/ipfs.py:799
- Scrape torrent statistics from IPFS network
- Returns dict with
seeders, leechers, completed counts
-
Uses content statistics from IPFS object stats
-
add_content(data: bytes) -> str: ccbt/protocols/ipfs.py:882
- Add content to IPFS and return CID
- Automatically pins content if
enable_pinning is True
-
Returns CID string or empty string on failure
-
get_content(cid: str) -> bytes | None: ccbt/protocols/ipfs.py:962
- Retrieve content from IPFS by CID
- Uses IPFS daemon
cat command
- Updates content tracking with access time
-
Returns content bytes or None if not found
-
pin_content(cid: str) -> bool: ccbt/protocols/ipfs.py:1012
- Pin content in IPFS to prevent garbage collection
-
Returns True on success, False on failure
-
unpin_content(cid: str) -> bool: ccbt/protocols/ipfs.py:1035
- Unpin content from IPFS
-
Returns True on success, False on failure
-
get_ipfs_peers() -> list[str]: ccbt/protocols/ipfs.py:1058
- Get list of connected IPFS peer IDs
-
Returns list of peer ID strings
-
get_ipfs_content() -> dict[str, IPFSContent]: ccbt/protocols/ipfs.py:1065
- Get all tracked IPFS content
-
Returns dict mapping CID to IPFSContent objects
-
get_content_stats(cid: str) -> dict[str, int]: ccbt/protocols/ipfs.py:1072
- Get statistics for specific content
-
Returns dict with seeders, leechers, completed
-
get_all_content_stats() -> dict[str, dict[str, int]]: ccbt/protocols/ipfs.py:1080
- Get statistics for all tracked content
- Returns dict mapping CID to stats dicts
CID Format:
- IPFS uses Content Identifiers (CIDs) to uniquely identify content
- CIDv0 format: Base58-encoded, starts with Qm (e.g., QmYjtig7VJQ6XsnUjqqJvj7QaMcCAwtrgNdahSiFofrE7o)
- CIDv1 format: Multibase-encoded, supports different bases (e.g., bafybei...)
- Default: CIDv1 is used for new content, CIDv0 for legacy content
Example Usage:
from ccbt.protocols.ipfs import IPFSProtocol
from ccbt.models import PeerInfo
# Initialize protocol (normally done via session manager)
protocol = IPFSProtocol()
protocol.config = get_config()
# Start protocol
await protocol.start()
# Add content to IPFS
content = b"Hello, IPFS!"
cid = await protocol.add_content(content)
print(f"Content added with CID: {cid}")
# Retrieve content
retrieved = await protocol.get_content(cid)
assert retrieved == content
# Pin content
await protocol.pin_content(cid)
# Connect to peer
peer_info = PeerInfo(
ip="192.168.1.1",
port=4001,
peer_id=b"QmPeerId1234567890abcdefghijklmnopqrstuvwxyz"
)
await protocol.connect_peer(peer_info)
# Send message
await protocol.send_message(peer_info.peer_id.hex(), b"Hello from IPFS!")
# Receive message
message = await protocol.receive_message(peer_info.peer_id.hex())
# Stop protocol
await protocol.stop()
Session Manager Integration:
The IPFS protocol is automatically registered when the session manager starts (if IPFS is configured):
from ccbt.session.session import AsyncSessionManager
from ccbt.models import Config, IPFSConfig
config = Config()
config.ipfs = IPFSConfig(
api_url="http://127.0.0.1:5001",
enable_pinning=True,
enable_dht=True,
)
session = AsyncSessionManager(config)
await session.start()
# IPFS protocol is now available in session.protocols
ipfs_protocol = next(p for p in session.protocols if isinstance(p, IPFSProtocol))
Discovery
AsyncDHTClient
Enhanced DHT (BEP 5) client with full Kademlia implementation for peer discovery.
High-performance async DHT client with full Kademlia support.
Initialize DHT client.
Source code in ccbt/discovery/dht.py
| def __init__(self, bind_ip: str = "0.0.0.0", bind_port: int = 0): # nosec B104
"""Initialize DHT client."""
self.config = get_config()
# Node identity
self.node_id = self._generate_node_id()
# Network
self.bind_ip = bind_ip
self.bind_port = bind_port
self.socket: asyncio.DatagramProtocol | None = None
self.transport: asyncio.DatagramTransport | None = None
# Routing table
self.routing_table = KademliaRoutingTable(self.node_id)
# Bootstrap nodes
self.bootstrap_nodes = DEFAULT_BOOTSTRAP.copy()
# Pending queries
self.pending_queries: dict[bytes, asyncio.Future] = {}
self.query_timeout = 5.0
# Tokens for announce_peer
self.tokens: dict[bytes, DHTToken] = {}
self.token_secret = os.urandom(20)
# Background tasks
self._refresh_task: asyncio.Task | None = None
self._cleanup_task: asyncio.Task | None = None
# Callbacks
self.peer_callbacks: list[Callable[[list[tuple[str, int]]], None]] = []
# BEP 27: Callback to check if a torrent is private
self.is_private_torrent: Callable[[bytes], bool] | None = None
self.logger = logging.getLogger(__name__)
|
add_peer_callback(callback: Callable[[list[tuple[str, int]]], None]) -> None
Add callback for new peers.
Source code in ccbt/discovery/dht.py
| def add_peer_callback(
self,
callback: Callable[[list[tuple[str, int]]], None],
) -> None:
"""Add callback for new peers."""
self.peer_callbacks.append(callback)
|
announce_peer(info_hash: bytes, port: int) -> bool
async
Announce our peer to the DHT.
Parameters:
| Name |
Type |
Description |
Default |
info_hash
|
bytes
|
|
required
|
port
|
int
|
|
required
|
Returns:
| Type |
Description |
bool
|
True if announcement was successful
|
Source code in ccbt/discovery/dht.py
| async def announce_peer(self, info_hash: bytes, port: int) -> bool:
"""Announce our peer to the DHT.
Args:
info_hash: Torrent info hash
port: Our port
Returns:
True if announcement was successful
"""
# BEP 27: Private torrents must not use DHT for peer announcements
if self.is_private_torrent and self.is_private_torrent(info_hash):
self.logger.debug(
"Skipping DHT announce_peer for private torrent %s (BEP 27)",
info_hash.hex()[:8],
)
return False
# Get token for this info hash
if info_hash not in self.tokens:
# Try to get token by doing a get_peers query
await self.get_peers(info_hash, 1)
if info_hash not in self.tokens:
self.logger.debug("No token available for %s", info_hash.hex())
return False
token = self.tokens[info_hash]
# Check if token is still valid
if time.time() > token.expires_time:
del self.tokens[info_hash]
return False
# Find closest nodes to announce to
closest_nodes = self.routing_table.get_closest_nodes(info_hash, 8)
success_count = 0
for node in closest_nodes:
try:
response = await self._send_query(
(node.ip, node.port),
"announce_peer",
{
b"id": self.node_id,
b"info_hash": info_hash,
b"port": port,
b"token": token.token,
},
)
if response and response.get(b"y") == b"r":
success_count += 1
self.routing_table.mark_node_good(node.node_id)
else:
self.routing_table.mark_node_bad(node.node_id)
except Exception as e:
self.logger.debug(
"announce_peer failed for %s:%s: %s",
node.ip,
node.port,
e,
)
self.routing_table.mark_node_bad(node.node_id)
return success_count > 0
|
get_peers(info_hash: bytes, max_peers: int = 50) -> list[tuple[str, int]]
async
Get peers for an info hash using iterative lookup.
Parameters:
| Name |
Type |
Description |
Default |
info_hash
|
bytes
|
|
required
|
max_peers
|
int
|
Maximum number of peers to return
|
50
|
Returns:
| Type |
Description |
list[tuple[str, int]]
|
List of (ip, port) tuples
|
Source code in ccbt/discovery/dht.py
| async def get_peers(
self,
info_hash: bytes,
max_peers: int = 50,
) -> list[tuple[str, int]]:
"""Get peers for an info hash using iterative lookup.
Args:
info_hash: Torrent info hash
max_peers: Maximum number of peers to return
Returns:
List of (ip, port) tuples
"""
# BEP 27: Private torrents must not use DHT for peer discovery
if self.is_private_torrent and self.is_private_torrent(info_hash):
self.logger.debug(
"Skipping DHT get_peers for private torrent %s (BEP 27)",
info_hash.hex()[:8],
)
return []
peers = []
queried_nodes = set()
# Get closest nodes to info hash
closest_nodes = self.routing_table.get_closest_nodes(info_hash, 8)
# Query nodes iteratively
for node in closest_nodes:
if node.node_id in queried_nodes:
continue
queried_nodes.add(node.node_id)
try:
# Send get_peers query
response = await self._send_query(
(node.ip, node.port),
"get_peers",
{
b"id": self.node_id,
b"info_hash": info_hash,
},
)
if not response or response.get(b"y") != b"r":
continue
r = response.get(b"r", {})
# Check for peers (values)
values = r.get(b"values", [])
if isinstance(values, list):
for value in values:
if isinstance(value, bytes) and len(value) == 6:
ip = ".".join(str(b) for b in value[:4])
port = int.from_bytes(value[4:6], "big")
peers.append((ip, port))
if len(peers) >= max_peers:
break
# Check for nodes to query
nodes_data = r.get(b"nodes", b"")
if nodes_data:
# Parse and add new nodes
for i in range(0, len(nodes_data), 26):
if i + 26 <= len(nodes_data):
node_data = nodes_data[i : i + 26]
node_id = node_data[:20]
ip = ".".join(str(b) for b in node_data[20:24])
port = int.from_bytes(node_data[24:26], "big")
new_node = DHTNode(node_id, ip, port)
self.routing_table.add_node(new_node)
# Store token for announce_peer
token = r.get(b"token")
if token:
self.tokens[info_hash] = DHTToken(token, info_hash)
# Mark node as good
self.routing_table.mark_node_good(node.node_id)
except Exception as e:
self.logger.debug(
"get_peers failed for %s:%s: %s",
node.ip,
node.port,
e,
)
self.routing_table.mark_node_bad(node.node_id)
# Notify callbacks
if peers:
for callback in self.peer_callbacks:
try:
callback(peers)
except Exception:
self.logger.exception("Peer callback error")
return peers
|
get_stats() -> dict[str, Any]
Get DHT statistics.
Source code in ccbt/discovery/dht.py
| def get_stats(self) -> dict[str, Any]:
"""Get DHT statistics."""
return {
"node_id": self.node_id.hex(),
"routing_table": self.routing_table.get_stats(),
"tokens": len(self.tokens),
"pending_queries": len(self.pending_queries),
}
|
handle_response(data: bytes, _addr: tuple[str, int]) -> None
Handle incoming DHT response.
Source code in ccbt/discovery/dht.py
| def handle_response(self, data: bytes, _addr: tuple[str, int]) -> None:
"""Handle incoming DHT response."""
try:
# Decode message
decoder = BencodeDecoder(data)
message = decoder.decode()
# Check if it's a response
if message.get(b"y") != b"r":
return
# Get transaction ID
tid = message.get(b"t")
if not tid or tid not in self.pending_queries:
return
# Set response
future = self.pending_queries[tid]
if not future.done():
future.set_result(message)
except Exception as e:
self.logger.debug("Failed to parse DHT response: %s", e)
|
remove_peer_callback(callback: Callable[[list[tuple[str, int]]], None]) -> None
Remove peer callback.
Source code in ccbt/discovery/dht.py
| def remove_peer_callback(
self,
callback: Callable[[list[tuple[str, int]]], None],
) -> None:
"""Remove peer callback."""
if callback in self.peer_callbacks:
self.peer_callbacks.remove(callback)
|
start() -> None
async
Start the DHT client.
Source code in ccbt/discovery/dht.py
| async def start(self) -> None:
"""Start the DHT client."""
# Create UDP socket
loop = asyncio.get_event_loop()
try:
self.transport, self.socket = await loop.create_datagram_endpoint(
lambda: DHTProtocol(self),
local_addr=(self.bind_ip, self.bind_port),
)
except OSError as e:
# CRITICAL FIX: Enhanced port conflict error handling
error_code = e.errno if hasattr(e, "errno") else None
import sys
if sys.platform == "win32":
if error_code == 10048: # WSAEADDRINUSE
from ccbt.utils.port_checker import get_port_conflict_resolution
resolution = get_port_conflict_resolution(self.bind_port, "udp")
error_msg = (
f"DHT UDP port {self.bind_port} is already in use.\n"
f"Error: {e}\n\n"
f"{resolution}"
)
self.logger.error(error_msg)
raise RuntimeError(error_msg) from e
elif error_code == 10013: # WSAEACCES
error_msg = (
f"Permission denied binding to {self.bind_ip}:{self.bind_port}.\n"
f"Error: {e}\n\n"
f"Resolution: Run with administrator privileges or change the port."
)
self.logger.error(error_msg)
raise RuntimeError(error_msg) from e
else:
if error_code == 98: # EADDRINUSE
from ccbt.utils.port_checker import get_port_conflict_resolution
resolution = get_port_conflict_resolution(self.bind_port, "udp")
error_msg = (
f"DHT UDP port {self.bind_port} is already in use.\n"
f"Error: {e}\n\n"
f"{resolution}"
)
self.logger.error(error_msg)
raise RuntimeError(error_msg) from e
elif error_code == 13: # EACCES
error_msg = (
f"Permission denied binding to {self.bind_ip}:{self.bind_port}.\n"
f"Error: {e}\n\n"
f"Resolution: Run with root privileges or change the port to >= 1024."
)
self.logger.error(error_msg)
raise RuntimeError(error_msg) from e
# Re-raise other OSErrors as-is
raise
# Start background tasks
self._refresh_task = asyncio.create_task(self._refresh_loop())
self._cleanup_task = asyncio.create_task(self._cleanup_loop())
# Bootstrap
await self._bootstrap()
self.logger.info("DHT client started on %s:%s", self.bind_ip, self.bind_port)
|
stop() -> None
async
Stop the DHT client.
Source code in ccbt/discovery/dht.py
| async def stop(self) -> None:
"""Stop the DHT client."""
if self._refresh_task:
self._refresh_task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await self._refresh_task
if self._cleanup_task:
self._cleanup_task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await self._cleanup_task
if self.transport:
self.transport.close()
self.logger.info("DHT client stopped")
|
Features:
- Kademlia DHT implementation: ccbt/discovery/dht.py:AsyncDHTClient - Full Kademlia routing table
- Peer discovery via DHT: ccbt/discovery/dht.py:find_peers - Iterative lookup for peer discovery
- Node routing table management: ccbt/discovery/dht.py:DHTNode - IPv4/IPv6 node support with BEP 45 multi-address
- Token verification: ccbt/discovery/dht.py:DHTToken - Secure announce tokens
- Continuous refresh: ccbt/discovery/dht.py - Automatic routing table maintenance
Key Methods:
- start(): ccbt/discovery/dht.py:start - Start DHT client and bootstrap
- stop(): ccbt/discovery/dht.py:stop - Stop DHT client
- find_peers(): ccbt/discovery/dht.py:find_peers - Find peers for info hash
- announce_peer(): ccbt/discovery/dht.py:announce_peer - Announce peer to DHT
Configuration: ccbt.toml:118-125
AsyncTrackerClient
High-performance async tracker communication for peer discovery.
High-performance async client for communicating with BitTorrent trackers.
Initialize the async tracker client.
Parameters:
| Name |
Type |
Description |
Default |
peer_id_prefix
|
str
|
Prefix for generating peer IDs (default: -CC0101- for ccBitTorrent 0.1.0)
|
'-CC0101-'
|
Source code in ccbt/discovery/tracker.py
| def __init__(self, peer_id_prefix: str = "-CC0101-"):
"""Initialize the async tracker client.
Args:
peer_id_prefix: Prefix for generating peer IDs (default: -CC0101- for ccBitTorrent 0.1.0)
"""
self.config = get_config()
self.peer_id_prefix = peer_id_prefix.encode("utf-8")
self.user_agent = "ccBitTorrent/0.1.0"
# HTTP session
self.session: aiohttp.ClientSession | None = None
# Tracker sessions
self.sessions: dict[str, TrackerSession] = {}
# Background tasks
self._announce_task: asyncio.Task | None = None
# Session metrics
self._session_metrics: dict[str, dict[str, Any]] = {}
self.logger = logging.getLogger(__name__)
|
announce(torrent_data: dict[str, Any], port: int = 6881, uploaded: int = 0, downloaded: int = 0, left: int | None = None, event: str = 'started') -> TrackerResponse
async
Announce to the tracker and get peer list asynchronously.
Parameters:
| Name |
Type |
Description |
Default |
torrent_data
|
dict[str, Any]
|
Parsed torrent data from TorrentParser
|
required
|
port
|
int
|
Port the client is listening on
|
6881
|
uploaded
|
int
|
Number of bytes uploaded (0 for initial announce)
|
0
|
downloaded
|
int
|
Number of bytes downloaded (0 for initial announce)
|
0
|
left
|
int | None
|
Number of bytes left to download (defaults to total file size)
|
None
|
event
|
str
|
Event type ("started", "completed", "stopped", or "" for regular)
|
'started'
|
Returns:
| Type |
Description |
TrackerResponse
|
TrackerResponse containing tracker response data
|
Raises:
| Type |
Description |
TrackerError
|
If tracker communication fails
|
Source code in ccbt/discovery/tracker.py
| async def announce(
self,
torrent_data: dict[str, Any],
port: int = 6881,
uploaded: int = 0,
downloaded: int = 0,
left: int | None = None,
event: str = "started",
) -> TrackerResponse:
"""Announce to the tracker and get peer list asynchronously.
Args:
torrent_data: Parsed torrent data from TorrentParser
port: Port the client is listening on
uploaded: Number of bytes uploaded (0 for initial announce)
downloaded: Number of bytes downloaded (0 for initial announce)
left: Number of bytes left to download (defaults to total file size)
event: Event type ("started", "completed", "stopped", or "" for regular)
Returns:
TrackerResponse containing tracker response data
Raises:
TrackerError: If tracker communication fails
"""
if not self.session:
msg = "Tracker client not started"
raise TrackerError(msg)
try:
# CRITICAL FIX: Validate torrent_data is a dict before accessing it
# Log immediately for debugging
self.logger.debug(
"tracker.announce() called with torrent_data type=%s, is_list=%s, is_dict=%s",
type(torrent_data),
isinstance(torrent_data, list),
isinstance(torrent_data, dict),
)
if not isinstance(torrent_data, dict):
if isinstance(torrent_data, list):
error_msg = (
f"CRITICAL: torrent_data cannot be a list, got {type(torrent_data)} "
f"(length={len(torrent_data)}). Expected dict. "
f"First items: {torrent_data[:3] if len(torrent_data) > 0 else 'empty'}"
)
self.logger.exception(error_msg)
import traceback
self.logger.error(
"Stack trace when list was passed:\n%s",
"".join(traceback.format_stack()),
)
raise TrackerError(error_msg)
# If it's an object, try to convert or raise error
if not hasattr(torrent_data, "announce") and not hasattr(
torrent_data, "info_hash"
):
error_msg = f"torrent_data must be a dict or object with announce/info_hash, got {type(torrent_data)}"
self.logger.exception(error_msg)
raise TrackerError(error_msg)
# Generate peer ID if not already present (only if dict)
if isinstance(torrent_data, dict) and "peer_id" not in torrent_data:
torrent_data["peer_id"] = self._generate_peer_id()
# Set left to total file size if not specified
if left is None:
# CRITICAL FIX: Handle missing or None file_info - validate torrent_data is dict first
if isinstance(torrent_data, dict):
file_info = torrent_data.get("file_info")
if file_info and isinstance(file_info, dict):
left = file_info.get("total_length", 0)
else:
left = 0 # Default to 0 if file_info not available
elif hasattr(torrent_data, "file_info"):
# Try attribute access if it's an object
file_info = getattr(torrent_data, "file_info", None)
left = getattr(file_info, "total_length", 0) if file_info else 0
else:
left = 0 # Default to 0 if file_info not available
# CRITICAL FIX: Use large but reasonable value for magnet links without metadata
# left=0 means "completed download" to trackers, so they won't return peers
# Using max int64 (2^63-1) may confuse some trackers, so use a large reasonable value instead
# 1TB (1099511627776 bytes) is large enough to indicate "unknown size, downloading full file"
# but not so large that it causes issues with tracker implementations
if isinstance(torrent_data, dict):
file_info = torrent_data.get("file_info", {})
if isinstance(file_info, dict):
total_length = file_info.get("total_length", 0)
# If total_length is 0, this is a magnet link without metadata
# Use a large but reasonable value to indicate "unknown size, need full file" (not "completed")
if total_length == 0:
# Use 1TB (1099511627776 bytes) - large enough to indicate "unknown size"
# but reasonable enough that trackers won't reject it
# This is better than max int64 which some trackers may not handle correctly
large_left = 1099511627776 # 1 TB
if left != large_left:
self.logger.debug(
"Magnet link without metadata detected (total_length=0), using left=%d (1TB) to indicate 'unknown size, need full file' (was %d)",
large_left,
left,
)
left = large_left
# CRITICAL FIX: Validate required fields before building URL
# Handle both dict and object access patterns
announce_url = (
torrent_data.get("announce")
if isinstance(torrent_data, dict)
else getattr(torrent_data, "announce", "")
)
info_hash_raw = (
torrent_data.get("info_hash")
if isinstance(torrent_data, dict)
else getattr(torrent_data, "info_hash", None)
)
peer_id_raw = (
torrent_data.get("peer_id")
if isinstance(torrent_data, dict)
else getattr(torrent_data, "peer_id", None)
)
if not announce_url:
msg = "No announce URL in torrent data"
raise TrackerError(msg)
if not info_hash_raw:
msg = "No info_hash in torrent data"
raise TrackerError(msg)
if not peer_id_raw:
msg = "No peer_id in torrent data"
raise TrackerError(msg)
# CRITICAL FIX: Ensure info_hash and peer_id are bytes, not strings
# Convert hex strings to bytes if needed
if isinstance(info_hash_raw, str):
# Try to decode as hex string (40 chars = 20 bytes)
if len(info_hash_raw) == 40:
try:
info_hash = bytes.fromhex(info_hash_raw)
except ValueError:
msg = f"info_hash is string but not valid hex: {info_hash_raw[:20]}..."
raise TrackerError(msg) from None
else:
# Try to decode as URL-encoded bytes
try:
info_hash = urllib.parse.unquote_to_bytes(info_hash_raw)
except Exception:
msg = f"info_hash is string but cannot be decoded: {type(info_hash_raw)}"
raise TrackerError(msg) from None
elif isinstance(info_hash_raw, bytes):
info_hash = info_hash_raw
else:
msg = f"info_hash has invalid type: {type(info_hash_raw)}, expected bytes or hex string"
raise TrackerError(msg)
# Validate info_hash length (should be 20 bytes for SHA-1)
if len(info_hash) != 20:
msg = f"info_hash must be exactly 20 bytes (SHA-1), got {len(info_hash)} bytes"
self.logger.error(msg)
raise TrackerError(msg)
# Ensure peer_id is bytes
if isinstance(peer_id_raw, str):
# Try to decode as hex string or URL-encoded
try:
if len(peer_id_raw) == 40: # Hex string
peer_id = bytes.fromhex(peer_id_raw)
else:
peer_id = urllib.parse.unquote_to_bytes(peer_id_raw)
except Exception:
# Fallback: encode as UTF-8
peer_id = peer_id_raw.encode("utf-8")
elif isinstance(peer_id_raw, bytes):
peer_id = peer_id_raw
else:
msg = f"peer_id has invalid type: {type(peer_id_raw)}, expected bytes or string"
raise TrackerError(msg)
# Validate peer_id length (should be 20 bytes)
if len(peer_id) != 20:
msg = f"peer_id must be exactly 20 bytes, got {len(peer_id)} bytes"
self.logger.error(msg)
raise TrackerError(msg)
# Validate port is in valid range
if not (1 <= port <= 65535):
msg = f"port must be in range 1-65535, got {port}"
self.logger.error(msg)
raise TrackerError(msg)
# Build tracker URL with parameters
# Ensure left is not None (default to 0 if None)
left_value = left if left is not None else 0
# Log announce parameters for debugging
self.logger.debug(
"Tracker announce parameters: info_hash=%s, peer_id=%s, port=%d, uploaded=%d, downloaded=%d, left=%d, event=%s",
info_hash.hex()[:16] + "...",
peer_id.hex()[:16] + "...",
port,
uploaded,
downloaded,
left_value,
event,
)
# CRITICAL FIX: Detect UDP trackers and route to UDP client
# Normalize URL first to ensure proper format detection
normalized_url = self._normalize_tracker_url(announce_url)
# Enhanced logging: Log tracker request parameters (after normalization)
info_hash_hex = (
info_hash.hex() if isinstance(info_hash, bytes) else str(info_hash)[:40]
)
peer_id_hex = (
peer_id.hex()[:20] if isinstance(peer_id, bytes) else str(peer_id)[:20]
)
self.logger.info(
"TRACKER_REQUEST: url=%s, info_hash=%s, peer_id=%s, port=%d, uploaded=%d, downloaded=%d, left=%d, event=%s",
normalized_url[:100] if len(normalized_url) > 100 else normalized_url,
info_hash_hex,
peer_id_hex,
port,
uploaded,
downloaded,
left_value,
event,
)
is_udp = normalized_url.startswith("udp://")
if is_udp:
# Route to UDP tracker client
# CRITICAL FIX: Singleton pattern removed - use session_manager.udp_tracker_client
# Socket must be initialized during daemon startup and never recreated
# This prevents WinError 10022 on Windows and ensures proper socket lifecycle
udp_client = None
if hasattr(self, "_session_manager") and self._session_manager:
# Use session manager's initialized UDP tracker client
if (
hasattr(self._session_manager, "udp_tracker_client")
and self._session_manager.udp_tracker_client
):
udp_client = self._session_manager.udp_tracker_client
self.logger.debug(
"Using session manager's initialized UDP tracker client"
)
# CRITICAL: Require session_manager.udp_tracker_client - no fallback
# This ensures socket is initialized at daemon startup and prevents recreation
if udp_client is None:
self.logger.error(
"UDP tracker client not available from session_manager. "
"Socket must be initialized during daemon startup via start_udp_tracker_client(). "
"This indicates a serious initialization issue."
)
raise RuntimeError(
"UDP tracker client not initialized. "
"Socket must be initialized during daemon startup via start_udp_tracker_client(). "
"Singleton pattern removed - use session_manager.udp_tracker_client."
)
# CRITICAL FIX: Validate socket is ready before use
# Socket should NEVER be recreated - if invalid, fail gracefully
if (
udp_client.transport is None
or udp_client.transport.is_closing()
or not udp_client._socket_ready
):
# CRITICAL: Socket should have been initialized during daemon startup
# If it's invalid here, this indicates a serious initialization issue
self.logger.error(
"UDP tracker client socket is invalid (transport=%s, is_closing=%s, ready=%s). "
"Socket should have been initialized during daemon startup. "
"This indicates a serious initialization issue.",
udp_client.transport is not None,
udp_client.transport.is_closing()
if udp_client.transport
else None,
udp_client._socket_ready,
)
raise RuntimeError(
"UDP tracker client socket is invalid. "
"Socket should have been initialized during daemon startup and should never need recreation. "
"If socket is invalid, daemon must be restarted."
)
try:
# Convert event string to TrackerEvent enum
from ccbt.discovery.tracker_udp_client import (
TrackerEvent as UDPTrackerEvent,
)
event_map = {
"started": UDPTrackerEvent.STARTED,
"completed": UDPTrackerEvent.COMPLETED,
"stopped": UDPTrackerEvent.STOPPED,
"": UDPTrackerEvent.NONE,
}
udp_event = event_map.get(event, UDPTrackerEvent.STARTED)
# Call UDP client announce and get full response info
# For single tracker, we need to call the full method to get interval, seeders, leechers
# Extract the single tracker URL from torrent_data
tracker_url = normalized_url
if isinstance(torrent_data, dict):
# Create a copy with just this tracker URL
single_tracker_data = torrent_data.copy()
single_tracker_data["announce"] = tracker_url
else:
single_tracker_data = torrent_data
# Use the full response method to get interval, seeders, leechers
# CRITICAL FIX: Pass port parameter to UDP tracker client to use external port
udp_result = await udp_client._announce_to_tracker_full(
tracker_url,
single_tracker_data,
port=port, # Use external port from NAT manager if available
uploaded=uploaded,
downloaded=downloaded,
left=left_value,
event=udp_event,
)
if udp_result is None:
# Treat as failure rather than "success with 0 peers"
self.logger.warning(
"UDP tracker announce failed for %s (no response). This usually indicates a connection error or tracker rejection.",
normalized_url,
)
raise TrackerError(
f"UDP tracker announce failed: no response from {normalized_url}"
)
udp_peers, udp_interval, udp_seeders, udp_leechers = udp_result
# Log if we got a response but no peers - this is unusual
# CRITICAL FIX: Enhanced warning for 0 peers from trackers
# This is especially important for popular torrents where 0 peers is unusual
if (
not udp_peers
and (udp_seeders is None or udp_seeders == 0)
and (udp_leechers is None or udp_leechers == 0)
):
self.logger.warning(
"UDP tracker %s returned response but reported 0 peers, 0 seeders, 0 leechers. "
"This may indicate: (1) The torrent has no active peers (unlikely for popular torrents), "
"(2) The tracker is filtering based on firewall/reachability (most likely), "
"(3) The announce parameters are incorrect, or (4) Network connectivity issues. "
"TROUBLESHOOTING: Check Windows Firewall allows incoming connections on port %d (TCP/UDP) and %d (UDP for DHT). "
"Also verify NAT port mapping is active and UDP responses can reach your client. "
"If this is a popular torrent, this likely indicates a firewall/NAT issue preventing peer discovery.",
normalized_url,
self.config.network.listen_port,
self.config.discovery.dht_port,
)
# Convert UDP response to TrackerResponse format
# CRITICAL FIX: Convert dict peers to PeerInfo objects for type consistency
# CRITICAL FIX: Log UDP peer count before conversion
raw_peer_count = len(udp_peers) if udp_peers else 0
if raw_peer_count > 0:
self.logger.info(
"UDP tracker %s returned %d raw peer(s) before conversion (seeders=%s, leechers=%s)",
normalized_url,
raw_peer_count,
udp_seeders if udp_seeders is not None else "unknown",
udp_leechers if udp_leechers is not None else "unknown",
)
peer_info_list: list[PeerInfo] = []
conversion_errors = 0
for peer_dict in udp_peers or []:
try:
if isinstance(peer_dict, dict):
peer_info = PeerInfo(
ip=str(peer_dict.get("ip", "")),
port=int(peer_dict.get("port", 0)),
peer_id=None,
peer_source=peer_dict.get("peer_source", "tracker"),
)
# Validate peer info (PeerInfo validator will check IP/port)
if (
peer_info.port >= 1
and peer_info.port <= 65535
and peer_info.ip
):
peer_info_list.append(peer_info)
else:
self.logger.warning(
"Skipping invalid peer from UDP tracker %s: ip=%s, port=%d (valid_ip=%s, valid_port=%s)",
normalized_url,
peer_info.ip,
peer_info.port,
peer_info.port >= 1 and peer_info.port <= 65535,
bool(peer_info.ip),
)
elif isinstance(peer_dict, PeerInfo):
# Already a PeerInfo object
peer_info_list.append(peer_dict)
else:
self.logger.warning(
"Unexpected peer format from UDP tracker: type=%s",
type(peer_dict),
)
except Exception as e:
conversion_errors += 1
self.logger.warning(
"Error converting peer from UDP tracker %s: %s (peer_dict=%s)",
normalized_url,
e,
peer_dict,
)
# CRITICAL FIX: Log conversion results at INFO/WARNING level for visibility
if conversion_errors > 0:
self.logger.warning(
"Converted %d/%d peers from UDP tracker %s (skipped %d invalid)",
len(peer_info_list),
raw_peer_count,
normalized_url,
conversion_errors,
)
elif raw_peer_count > 0 and len(peer_info_list) == 0:
self.logger.warning(
"WARNING: UDP tracker %s returned %d raw peers but all were filtered out during conversion",
normalized_url,
raw_peer_count,
)
elif len(peer_info_list) > 0:
self.logger.info(
"Successfully converted %d peer(s) from UDP tracker %s",
len(peer_info_list),
normalized_url,
)
# Use actual values from UDP response
response = TrackerResponse(
interval=udp_interval if udp_interval is not None else 1800,
peers=peer_info_list,
complete=udp_seeders, # UDP seeders -> complete
incomplete=udp_leechers, # UDP leechers -> incomplete
download_url=None,
tracker_id=None,
warning_message=None,
)
# Enhanced logging with peer conversion results
self.logger.info(
"UDP tracker announce successful: %d peers (converted to %d PeerInfo objects), %d seeders, %d leechers, interval=%ds from %s",
len(udp_peers),
len(peer_info_list),
udp_seeders if udp_seeders is not None else 0,
udp_leechers if udp_leechers is not None else 0,
udp_interval if udp_interval is not None else 1800,
normalized_url,
)
except Exception as udp_error:
# Log UDP-specific error with enhanced message
error_type = type(udp_error).__name__
# Provide specific error context for UDP trackers
if isinstance(
udp_error, (ConnectionError, TimeoutError, asyncio.TimeoutError)
):
error_context = f"UDP tracker connection failed: {udp_error}"
elif "connection" in str(udp_error).lower():
error_context = f"UDP tracker connection error: {udp_error}"
else:
error_context = (
f"UDP tracker announce error ({error_type}): {udp_error}"
)
self.logger.warning(
"UDP tracker announce failed for %s - %s",
normalized_url,
error_context,
)
# Re-raise as TrackerError for consistent error handling
msg = f"UDP tracker announce failed: {error_context}"
raise TrackerError(msg) from udp_error
else:
# HTTP/HTTPS tracker - use existing HTTP client logic
# Build tracker URL with parameters
tracker_url = self._build_tracker_url(
normalized_url,
info_hash,
peer_id,
port,
uploaded,
downloaded,
left_value,
event,
)
# Make async HTTP request
response_data = await self._make_request_async(tracker_url)
# Parse response
response = self._parse_response_async(response_data)
# Update tracker session (safely get announce URL)
announce_url_for_session = (
torrent_data.get("announce")
if isinstance(torrent_data, dict)
else getattr(torrent_data, "announce", "")
)
if announce_url_for_session:
self._update_tracker_session(announce_url_for_session, response)
except Exception as e:
# Get announce URL safely for error handling
announce_url = ""
from contextlib import suppress
with suppress(Exception):
announce_url = torrent_data.get("announce") or getattr(
torrent_data, "announce", ""
)
if announce_url:
self._handle_tracker_failure(announce_url)
msg = f"Tracker announce failed: {e}"
raise TrackerError(msg) from e
else:
return response
|
announce_to_multiple(torrent_data: dict[str, Any], tracker_urls: list[str], port: int = 6881, uploaded: int = 0, downloaded: int = 0, left: int | None = None, event: str = 'started') -> list[TrackerResponse]
async
Announce to multiple trackers concurrently.
Parameters:
| Name |
Type |
Description |
Default |
torrent_data
|
dict[str, Any]
|
|
required
|
tracker_urls
|
list[str]
|
List of tracker URLs to announce to
|
required
|
port
|
int
|
Port the client is listening on
|
6881
|
uploaded
|
int
|
|
0
|
downloaded
|
int
|
Number of bytes downloaded
|
0
|
left
|
int | None
|
Number of bytes left to download
|
None
|
event
|
str
|
|
'started'
|
Returns:
| Type |
Description |
list[TrackerResponse]
|
List of successful tracker responses
|
Source code in ccbt/discovery/tracker.py
| async def announce_to_multiple(
self,
torrent_data: dict[str, Any],
tracker_urls: list[str],
port: int = 6881,
uploaded: int = 0,
downloaded: int = 0,
left: int | None = None,
event: str = "started",
) -> list[TrackerResponse]:
"""Announce to multiple trackers concurrently.
Args:
torrent_data: Parsed torrent data
tracker_urls: List of tracker URLs to announce to
port: Port the client is listening on
uploaded: Number of bytes uploaded
downloaded: Number of bytes downloaded
left: Number of bytes left to download
event: Event type
Returns:
List of successful tracker responses
"""
if not self.session:
msg = "Tracker client not started"
raise TrackerError(msg)
if not tracker_urls:
self.logger.warning("No tracker URLs provided for announce_to_multiple")
return []
# Log tracker types for debugging
udp_count = sum(1 for url in tracker_urls if url.startswith("udp://"))
http_count = len(tracker_urls) - udp_count
self.logger.info(
"Announcing to %d tracker(s) concurrently (%d UDP, %d HTTP/HTTPS)",
len(tracker_urls),
udp_count,
http_count,
)
# Create announce tasks for all trackers
tasks = []
url_to_task = {} # Map URL to task for better error reporting
for url in tracker_urls:
# Create a copy of torrent data with this tracker URL
torrent_copy = torrent_data.copy()
torrent_copy["announce"] = url
task = asyncio.create_task(
self._announce_to_tracker(
torrent_copy,
port,
uploaded,
downloaded,
left,
event,
),
)
tasks.append(task)
url_to_task[task] = url
# Wait for all announces to complete
results = await asyncio.gather(*tasks, return_exceptions=True)
# Filter successful responses and log detailed results
successful_responses = []
failed_trackers = []
total_peers = 0
for task, result in zip(tasks, results):
url = url_to_task.get(task, "unknown")
if isinstance(result, TrackerResponse):
successful_responses.append(result)
peer_count = len(result.peers) if result.peers else 0
total_peers += peer_count
tracker_type = "UDP" if url.startswith("udp://") else "HTTP/HTTPS"
self.logger.info(
"%s tracker %s: %d peer(s)",
tracker_type,
url[:80] + "..." if len(url) > 80 else url,
peer_count,
)
elif isinstance(result, Exception):
tracker_type = "UDP" if url.startswith("udp://") else "HTTP/HTTPS"
failed_trackers.append((url, result))
self.logger.debug(
"%s tracker %s failed: %s",
tracker_type,
url[:80] + "..." if len(url) > 80 else url,
str(result),
)
self.logger.info(
"Multi-tracker announce completed: %d/%d successful, %d total peer(s) discovered",
len(successful_responses),
len(tracker_urls),
total_peers,
)
if failed_trackers and len(failed_trackers) == len(tracker_urls):
# All trackers failed - log warning
self.logger.warning(
"All %d tracker(s) failed to respond",
len(tracker_urls),
)
return successful_responses
|
get_session_stats() -> dict[str, Any]
Get HTTP session statistics.
Returns:
| Type |
Description |
dict[str, Any]
|
Dictionary with session statistics per tracker host
|
Source code in ccbt/discovery/tracker.py
| def get_session_stats(self) -> dict[str, Any]:
"""Get HTTP session statistics.
Returns:
Dictionary with session statistics per tracker host
"""
stats = {}
for host, metrics in self._session_metrics.items():
request_count = metrics.get("request_count", 0)
if request_count > 0:
stats[host] = {
"request_count": request_count,
"average_request_time": (
metrics.get("total_request_time", 0.0) / request_count
),
"average_dns_time": (
metrics.get("total_dns_time", 0.0) / request_count
),
"connection_reuse_rate": (
metrics.get("connection_reuse_count", 0) / request_count * 100
),
"error_rate": (metrics.get("error_count", 0) / request_count * 100),
}
else: # pragma: no cover - Zero request count path, tested via stats with requests
stats[host] = metrics
return stats
|
normalize_tracker_url(url: str) -> str
Normalize and validate tracker URL to prevent malformed URLs (public API).
Parameters:
| Name |
Type |
Description |
Default |
url
|
str
|
Raw tracker URL from torrent
|
required
|
Returns:
Raises:
| Type |
Description |
TrackerError
|
If URL is invalid or cannot be normalized
|
Source code in ccbt/discovery/tracker.py
| def normalize_tracker_url(self, url: str) -> str:
"""Normalize and validate tracker URL to prevent malformed URLs (public API).
Args:
url: Raw tracker URL from torrent
Returns:
Normalized tracker URL
Raises:
TrackerError: If URL is invalid or cannot be normalized
"""
return self._normalize_tracker_url(url)
|
scrape(torrent_data: dict[str, Any]) -> dict[str, Any]
async
Scrape tracker for statistics asynchronously (if supported).
Note: Not all trackers support scraping.
Parameters:
| Name |
Type |
Description |
Default |
torrent_data
|
dict[str, Any]
|
|
required
|
Returns:
| Type |
Description |
dict[str, Any]
|
Scraped statistics with keys: seeders, leechers, completed
|
dict[str, Any]
|
Returns empty dict if scraping fails or is not supported
|
Source code in ccbt/discovery/tracker.py
| async def scrape(self, torrent_data: dict[str, Any]) -> dict[str, Any]:
"""Scrape tracker for statistics asynchronously (if supported).
Note: Not all trackers support scraping.
Args:
torrent_data: Parsed torrent data
Returns:
Scraped statistics with keys: seeders, leechers, completed
Returns empty dict if scraping fails or is not supported
"""
try:
# Check if session is initialized
if self.session is None:
self.logger.warning("Tracker client not started, cannot scrape")
return {}
# Extract info hash from torrent data
info_hash = torrent_data.get("info_hash")
if not info_hash:
self.logger.debug("No info_hash in torrent data")
return {}
# Build scrape URL
announce_url = torrent_data.get("announce")
if not announce_url:
self.logger.debug("No announce URL in torrent data")
return {}
scrape_url = self._build_scrape_url(info_hash, announce_url)
if not scrape_url:
self.logger.debug("Failed to build scrape URL")
return {}
# Make HTTP request using existing session
try:
async with self.session.get(scrape_url) as response:
if response.status == 200:
data = await response.read()
return self._parse_scrape_response(data, info_hash)
self.logger.debug(
"Tracker scrape failed with status %d: %s",
response.status,
response.reason,
)
return {}
except aiohttp.ClientError as e:
self.logger.debug("Network error during scrape: %s", e)
return {}
except asyncio.TimeoutError:
self.logger.debug("Timeout during scrape")
return {}
except Exception:
self.logger.exception("HTTP scrape failed")
return {}
|
start() -> None
async
Start the async tracker client.
Source code in ccbt/discovery/tracker.py
| async def start(self) -> None:
"""Start the async tracker client."""
# Create HTTP session with optimized settings
timeout = aiohttp.ClientTimeout(
total=self.config.network.connection_timeout,
connect=self.config.network.connection_timeout,
)
# Setup proxy connector if enabled
connector = self._create_connector(timeout)
self.session = aiohttp.ClientSession(
timeout=timeout,
connector=connector,
headers={"User-Agent": self.user_agent},
)
self.logger.info("Async tracker client started")
|
stop() -> None
async
Stop the async tracker client and clean up resources.
Source code in ccbt/discovery/tracker.py
| async def stop(self) -> None:
"""Stop the async tracker client and clean up resources."""
# Cancel and wait for announce task if it exists
if self._announce_task and not self._announce_task.done():
self._announce_task.cancel()
try:
await self._announce_task
except asyncio.CancelledError:
pass
except Exception as e:
self.logger.debug("Error waiting for announce task: %s", e)
# Clear task reference
self._announce_task = None
# CRITICAL FIX: Properly close HTTP session to prevent "Unclosed client session" warnings
if self.session:
try:
# CRITICAL FIX: Ensure session is fully closed before setting to None
# Use context manager pattern to ensure cleanup even if close() raises
if not self.session.closed:
# CRITICAL FIX: Close all connectors to ensure complete cleanup
await self.session.close()
# CRITICAL FIX: Wait longer for session to fully close (especially on Windows)
# This prevents "Unclosed client session" warnings
# On Windows, aiohttp sessions may need more time to fully close
import sys
if sys.platform == "win32":
await asyncio.sleep(0.2)
else:
await asyncio.sleep(0.1)
# CRITICAL FIX: Verify session is actually closed
if not self.session.closed:
self.logger.warning(
"HTTP session not fully closed after close() call"
)
except Exception as e:
self.logger.debug("Error closing HTTP session: %s", e)
# CRITICAL FIX: Even if close() fails, try to clean up
try:
if hasattr(self.session, "_connector") and self.session._connector:
await self.session._connector.close()
except Exception:
pass
finally:
# CRITICAL FIX: Always set to None even if close() fails
self.session = None
self.logger.info("Async tracker client stopped")
|
Features:
- HTTP tracker support: ccbt/discovery/tracker.py:AsyncTrackerClient - Async HTTP tracker communication
- UDP tracker support: ccbt/discovery/tracker_udp_client.py:AsyncUDPTrackerClient - Async UDP tracker communication
- Concurrent announces: ccbt/discovery/tracker.py - Multiple tracker announces in parallel
- DNS caching: ccbt/discovery/tracker.py:DNSCache - TTL-based DNS cache for tracker hostnames
- Announce and scrape operations: ccbt/discovery/tracker.py:announce - Peer discovery and statistics
Key Methods:
- announce(): ccbt/discovery/tracker.py:announce - Announce torrent to tracker
- scrape(): ccbt/discovery/tracker.py:scrape - Scrape tracker for statistics
- get_session(): ccbt/discovery/tracker.py:TrackerSession - Get or create tracker session
AsyncUDPTrackerClient
Async UDP tracker client implementation (BEP 15).
High-performance async UDP tracker client.
Initialize UDP tracker client.
Parameters:
| Name |
Type |
Description |
Default |
peer_id
|
bytes | None
|
|
None
|
Source code in ccbt/discovery/tracker_udp_client.py
| def __init__(self, peer_id: bytes | None = None):
"""Initialize UDP tracker client.
Args:
peer_id: Our peer ID (20 bytes)
"""
self.config = get_config()
if peer_id is None:
peer_id = b"-CC0101-" + b"x" * 12
self.our_peer_id = peer_id
# Tracker sessions
self.sessions: dict[str, TrackerSession] = {}
# UDP socket
self.socket: asyncio.DatagramProtocol | None = None
self.transport: asyncio.DatagramTransport | None = None
self.transaction_counter = 0
# Pending requests
self.pending_requests: dict[int, asyncio.Future] = {}
# Background tasks
self._cleanup_task: asyncio.Task | None = None
# CRITICAL FIX: Add lock to prevent concurrent socket operations
# Windows requires serialized access to UDP sockets to prevent WinError 10022
self._socket_lock: asyncio.Lock = asyncio.Lock()
self._socket_ready: bool = False
# CRITICAL FIX: Track WinError 10022 warning frequency to reduce verbosity
# Only log at WARNING level once per time period, then use DEBUG for subsequent occurrences
self._last_winerror_warning_time: float = 0.0
self._winerror_warning_interval: float = 30.0 # Log WARNING once per 30 seconds
# CRITICAL FIX: Socket health monitoring to prevent aggressive recreation
self._socket_error_count: int = 0
self._socket_last_error_time: float = 0.0
self._socket_health_check_interval: float = (
5.0 # Check socket health every 5 seconds
)
self._socket_recreation_backoff: float = (
1.0 # Exponential backoff for socket recreation
)
self._max_socket_recreation_backoff: float = 60.0 # Max backoff of 60 seconds
self._socket_recreation_count: int = 0
self._last_socket_health_check: float = 0.0
self.logger = logging.getLogger(__name__)
|
announce(torrent_data: dict[str, Any], uploaded: int = 0, downloaded: int = 0, left: int | None = None, event: TrackerEvent = TrackerEvent.STARTED) -> list[dict[str, Any]]
async
Announce to UDP trackers and get peer list.
Parameters:
| Name |
Type |
Description |
Default |
torrent_data
|
dict[str, Any]
|
|
required
|
uploaded
|
int
|
|
0
|
downloaded
|
int
|
|
0
|
left
|
int | None
|
|
None
|
event
|
TrackerEvent
|
|
STARTED
|
Returns:
| Type |
Description |
list[dict[str, Any]]
|
List of peer dictionaries
|
Source code in ccbt/discovery/tracker_udp_client.py
| async def announce(
self,
torrent_data: dict[str, Any],
uploaded: int = 0,
downloaded: int = 0,
left: int | None = None,
event: TrackerEvent = TrackerEvent.STARTED,
) -> list[dict[str, Any]]:
"""Announce to UDP trackers and get peer list.
Args:
torrent_data: Parsed torrent data
uploaded: Bytes uploaded
downloaded: Bytes downloaded
left: Bytes left to download
event: Announce event
Returns:
List of peer dictionaries
"""
if left is None:
left = torrent_data["file_info"]["total_length"]
# Get tracker URLs
tracker_urls = self._extract_tracker_urls(torrent_data)
if (
not tracker_urls
): # pragma: no cover - No trackers path, tested via trackers present
self.logger.warning("No UDP trackers found")
return []
# Announce to all trackers concurrently
tasks = []
for url in tracker_urls:
task = asyncio.create_task(
self._announce_to_tracker(
url,
torrent_data,
port=None, # Use config port (external port should be passed from AnnounceController via _announce_to_tracker_full)
uploaded=uploaded,
downloaded=downloaded,
left=left,
event=event,
),
)
tasks.append(task)
# Wait for all announces to complete
results = await asyncio.gather(*tasks, return_exceptions=True)
# Collect all peers
all_peers = []
for result in results:
if isinstance(result, list):
all_peers.extend(result)
elif isinstance(
result, Exception
): # pragma: no cover - Exception result path, tested via success paths
self.logger.debug("Tracker announce failed: %s", result)
# Deduplicate peers
peer_set = set()
unique_peers = []
for peer in all_peers:
peer_key = (peer["ip"], peer["port"])
if peer_key not in peer_set:
peer_set.add(peer_key)
unique_peers.append(peer)
self.logger.info(
"Got %s unique peers from %s trackers",
len(unique_peers),
len(tracker_urls),
)
return unique_peers
|
handle_response(data: bytes, _addr: tuple[str, int]) -> None
Handle incoming UDP response.
CRITICAL: Socket should always be ready. If not, this indicates an initialization issue.
Source code in ccbt/discovery/tracker_udp_client.py
| def handle_response(self, data: bytes, _addr: tuple[str, int]) -> None:
"""Handle incoming UDP response.
CRITICAL: Socket should always be ready. If not, this indicates an initialization issue.
"""
# If socket not ready, log warning and drop response (socket should always be ready)
if not self._socket_ready:
self.logger.warning(
"Received UDP response from %s:%d but socket not ready (length=%d bytes). "
"Socket should have been initialized during daemon startup. Dropping response.",
_addr[0] if _addr else "unknown",
_addr[1] if _addr else 0,
len(data),
)
return
# Add comprehensive logging for debugging
self.logger.debug(
"Received UDP datagram from %s:%d, length=%d bytes",
_addr[0] if _addr else "unknown",
_addr[1] if _addr else 0,
len(data),
)
try:
if len(data) < 8:
self.logger.debug(
"UDP response too short: %d bytes (minimum 8)", len(data)
)
return
# Parse response header
action = struct.unpack("!I", data[0:4])[0]
transaction_id = struct.unpack("!I", data[4:8])[0]
self.logger.debug(
"Parsed UDP response: action=%d, transaction_id=%d from %s:%d",
action,
transaction_id,
_addr[0] if _addr else "unknown",
_addr[1] if _addr else 0,
)
# Check if we have a pending request for this transaction
if transaction_id not in self.pending_requests:
# Enhanced logging for unmatched responses
# This can happen if: (1) Response arrived after timeout, (2) Transaction ID collision,
# or (3) Response from different tracker/client
self.logger.warning(
"Received UDP response with transaction_id=%d from %s:%d but no pending request found. "
"This may indicate: (1) Response arrived after timeout, (2) Transaction ID collision, "
"or (3) Response from different tracker/client. "
"Pending transaction IDs: %s (count: %d). Response action: %d",
transaction_id,
_addr[0] if _addr else "unknown",
_addr[1] if _addr else 0,
sorted(list(self.pending_requests.keys()))[
:10
], # Show first 10 for brevity
len(self.pending_requests),
action,
)
return
future = self.pending_requests[transaction_id]
if future.done():
self.logger.debug(
"Future for transaction_id=%d already done", transaction_id
)
return
# Parse response based on action
if action == TrackerAction.CONNECT.value:
if len(data) >= 16:
connection_id = struct.unpack("!Q", data[8:16])[0]
response = TrackerResponse(
action=TrackerAction.CONNECT,
transaction_id=transaction_id,
connection_id=connection_id,
)
future.set_result(response)
elif action == TrackerAction.ANNOUNCE.value:
if len(data) >= 20:
interval = struct.unpack("!I", data[8:12])[0]
leechers = struct.unpack("!I", data[12:16])[0]
seeders = struct.unpack("!I", data[16:20])[0]
# CRITICAL FIX: Add detailed logging of raw tracker response
# Always log at INFO level for visibility - this is critical for debugging
self.logger.info(
"UDP Tracker ANNOUNCE response from %s:%d: "
"interval=%d, leechers=%d, seeders=%d, response_length=%d bytes",
_addr[0] if _addr else "unknown",
_addr[1] if _addr else 0,
interval,
leechers,
seeders,
len(data),
)
# CRITICAL FIX: Log FULL raw response data at INFO level for debugging
# This helps identify if peers are in the response but not being parsed
if len(data) > 0:
# Log first 200 bytes at INFO level, full response at DEBUG
preview_len = min(200, len(data))
self.logger.info(
"Raw tracker response (first %d/%d bytes): %s",
preview_len,
len(data),
data[:preview_len].hex(),
)
if len(data) > preview_len:
self.logger.debug(
"Raw tracker response (remaining %d bytes): %s",
len(data) - preview_len,
data[preview_len:].hex(),
)
# Parse peers (compact format)
# CRITICAL FIX: Improved peer parsing with validation and logging
peers = []
invalid_peers = 0
# CRITICAL FIX: Log raw response for debugging at INFO level for visibility
self.logger.info(
"UDP Tracker response parsing: length=%d bytes, action=ANNOUNCE, seeders=%d, leechers=%d",
len(data),
seeders,
leechers,
)
self.logger.debug(
"Raw tracker response hex (first 100 bytes): %s",
data[:100].hex() if len(data) >= 100 else data.hex(),
)
if len(data) > 20:
peer_data = data[20:]
peer_count = len(peer_data) // 6
# CRITICAL FIX: Validate peer data length is multiple of 6
if len(peer_data) % 6 != 0:
self.logger.warning(
"Peer data length not multiple of 6: %d bytes (expected multiple of 6 for compact format). "
"Truncating to valid length.",
len(peer_data),
)
# Truncate to valid length
peer_data = peer_data[
: len(peer_data) - (len(peer_data) % 6)
]
peer_count = len(peer_data) // 6
# CRITICAL FIX: Enhanced logging for peer parsing
self.logger.info(
"Parsing %d peer(s) from tracker %s:%d (peer_data length: %d bytes, "
"expected peers: %d, seeders reported: %d, leechers reported: %d)",
peer_count,
_addr[0] if _addr else "unknown",
_addr[1] if _addr else 0,
len(peer_data),
peer_count,
seeders,
leechers,
)
# CRITICAL FIX: Log peer data preview for debugging
if len(peer_data) > 0:
preview_peers = min(3, peer_count) # First 3 peers
preview_bytes = preview_peers * 6
self.logger.debug(
"Peer data preview (first %d peer(s), %d bytes): %s",
preview_peers,
preview_bytes,
peer_data[:preview_bytes].hex(),
)
else:
self.logger.warning(
"Tracker response has no peer data (response length: %d bytes, "
"minimum expected: 20 bytes for header)",
len(data),
)
# CRITICAL FIX: If tracker reports seeders/leechers but no peer data, log error
if (seeders > 0 or leechers > 0) and len(data) <= 20:
self.logger.error(
"INCONSISTENCY: Tracker %s:%d reports seeders=%d, leechers=%d but no peer data! "
"Response hex: %s",
_addr[0] if _addr else "unknown",
_addr[1] if _addr else 0,
seeders,
leechers,
data.hex()[:200],
)
# Parse peers from peer_data (if available)
if len(data) > 20:
peer_data = data[20:]
for i in range(0, len(peer_data), 6):
if i + 6 <= len(peer_data):
try:
peer_bytes = peer_data[i : i + 6]
# CRITICAL FIX: Validate peer_bytes length before parsing
if len(peer_bytes) != 6:
invalid_peers += 1
self.logger.debug(
"Invalid peer bytes length at offset %d: %d bytes (expected 6)",
i,
len(peer_bytes),
)
continue
# Parse IP address (4 bytes)
ip_bytes = peer_bytes[:4]
ip = ".".join(str(b) for b in ip_bytes)
# Parse port (2 bytes, big-endian)
port_bytes = peer_bytes[4:6]
if len(port_bytes) != 2:
invalid_peers += 1
self.logger.debug(
"Invalid port bytes length at offset %d: %d bytes (expected 2)",
i,
len(port_bytes),
)
continue
port = int.from_bytes(port_bytes, "big")
# CRITICAL FIX: Validate IP and port (relaxed validation)
# Only filter obviously invalid IPs - don't filter private IPs as they might be valid
# Many valid peers use private IPs (NAT, VPN, etc.)
ip_parts = ip.split(".")
is_valid_ip = False
try:
is_valid_ip = (
len(ip_parts) == 4
and all(
p.isdigit() and 0 <= int(p) <= 255
for p in ip_parts
)
and ip != "0.0.0.0"
# CRITICAL: Don't filter 127.x.x.x, 169.254.x.x, or private IPs
# These might be valid in NAT/VPN scenarios
)
except (ValueError, AttributeError) as e:
self.logger.debug(
"Error validating IP %s: %s",
ip,
e,
)
# Check if port is valid
is_valid_port = 1 <= port <= 65535
if is_valid_ip and is_valid_port:
peer_dict = {
"ip": ip,
"port": port,
"peer_source": "tracker", # Mark peers from tracker responses (BEP 27)
}
peers.append(peer_dict)
# CRITICAL FIX: Log each parsed peer at INFO level for visibility
self.logger.info(
"Parsed peer from tracker: %s:%d (offset %d, peer %d/%d)",
ip,
port,
i,
len(peers),
peer_count,
)
else:
invalid_peers += 1
self.logger.warning(
"Skipping invalid peer from tracker: ip=%s, port=%d (valid_ip=%s, valid_port=%s, offset=%d)",
ip,
port,
is_valid_ip,
is_valid_port,
i,
)
except (
ValueError,
IndexError,
struct.error,
TypeError,
) as e:
invalid_peers += 1
self.logger.warning(
"Error parsing peer at offset %d: %s (peer_bytes=%s)",
i,
e,
peer_bytes.hex()
if "peer_bytes" in locals()
else "N/A",
)
if invalid_peers > 0:
self.logger.debug(
"Skipped %d invalid peer(s) from tracker response",
invalid_peers,
)
# CRITICAL FIX: Log at INFO level for visibility when peers are found
if len(peers) > 0:
self.logger.info(
"Parsed %d valid peer(s) from tracker %s:%d (seeders=%d, leechers=%d)",
len(peers),
_addr[0] if _addr else "unknown",
_addr[1] if _addr else 0,
seeders,
leechers,
)
else:
# CRITICAL FIX: Enhanced logging for 0 peers case
peer_data_len = (len(data) - 20) if len(data) > 20 else 0
self.logger.warning(
"Tracker %s:%d responded with 0 valid peers after parsing "
"(seeders=%d, leechers=%d, peer_data_length=%d bytes, "
"expected_peers=%d, invalid_peers_skipped=%d, response_length=%d bytes)",
_addr[0] if _addr else "unknown",
_addr[1] if _addr else 0,
seeders,
leechers,
peer_data_len,
peer_data_len // 6 if peer_data_len > 0 else 0,
invalid_peers,
len(data),
)
# If tracker reports seeders/leechers but no peer data, this is suspicious
if (seeders > 0 or leechers > 0) and peer_data_len == 0:
self.logger.error(
"INCONSISTENCY: Tracker %s:%d reports seeders=%d, leechers=%d "
"but provided NO peer data (response_length=%d bytes). "
"This may indicate a tracker response format issue.",
_addr[0] if _addr else "unknown",
_addr[1] if _addr else 0,
seeders,
leechers,
len(data),
)
response = TrackerResponse(
action=TrackerAction.ANNOUNCE,
transaction_id=transaction_id,
interval=interval,
leechers=leechers,
seeders=seeders,
peers=peers,
)
future.set_result(response)
elif action == TrackerAction.SCRAPE.value:
# Scrape response format:
# action (4) + transaction_id (4) + [complete (4) + downloaded (4) + incomplete (4) per info_hash]
if (
len(data) >= 20
): # At least action + tx_id + one set of scrape data (12 bytes)
complete = struct.unpack("!I", data[8:12])[0]
downloaded = struct.unpack("!I", data[12:16])[0]
incomplete = struct.unpack("!I", data[16:20])[0]
response = TrackerResponse(
action=TrackerAction.SCRAPE,
transaction_id=transaction_id,
complete=complete,
downloaded=downloaded,
incomplete=incomplete,
)
future.set_result(response)
elif action == TrackerAction.ERROR.value:
error_message = data[8:].decode("utf-8", errors="ignore")
response = TrackerResponse(
action=TrackerAction.ERROR,
transaction_id=transaction_id,
error_message=error_message,
)
future.set_result(response)
except Exception as e: # pragma: no cover - Exception handling in response parsing, hard to trigger reliably in tests
self.logger.debug(
"Error parsing tracker response: %s", e
) # pragma: no cover - Logging statement in exception handler
|
scrape(torrent_data: dict[str, Any]) -> dict[str, Any]
async
Scrape tracker for statistics.
Parameters:
| Name |
Type |
Description |
Default |
torrent_data
|
dict[str, Any]
|
Parsed torrent data with info_hash and announce URLs
|
required
|
Returns:
| Type |
Description |
dict[str, Any]
|
Scraped statistics with keys: seeders, leechers, completed
|
dict[str, Any]
|
Returns empty dict if scraping fails or is not supported
|
Source code in ccbt/discovery/tracker_udp_client.py
| async def scrape(self, torrent_data: dict[str, Any]) -> dict[str, Any]:
"""Scrape tracker for statistics.
Args:
torrent_data: Parsed torrent data with info_hash and announce URLs
Returns:
Scraped statistics with keys: seeders, leechers, completed
Returns empty dict if scraping fails or is not supported
"""
try:
# Check if transport is initialized
if self.transport is None:
self.logger.warning("UDP transport not initialized, cannot scrape")
return {}
# Extract info hash from torrent data
info_hash = torrent_data.get("info_hash")
if not info_hash:
self.logger.debug("No info_hash in torrent data")
return {}
# Validate info_hash length
if len(info_hash) != 20:
self.logger.debug("Invalid info_hash length: %d", len(info_hash))
return {}
# Get tracker URLs
tracker_urls = self._extract_tracker_urls(torrent_data)
if not tracker_urls:
self.logger.debug("No UDP tracker URLs found")
return {}
# Use first UDP tracker
tracker_url = tracker_urls[0]
host, port = self._parse_udp_url(tracker_url)
tracker_address = (host, port)
# Get or create tracker session
session_key = f"{host}:{port}"
if session_key not in self.sessions:
self.sessions[session_key] = TrackerSession(
url=tracker_url, host=host, port=port
)
session = self.sessions[session_key]
# Ensure connection is established
if not session.is_connected or time.time() - session.connection_time > 60.0:
try:
await self._connect_to_tracker(session)
except Exception as e:
self.logger.debug(
"Failed to connect to tracker %s:%s: %s", host, port, e
)
return {}
if not session.is_connected:
self.logger.debug(
"Not connected to tracker %s:%s", host, port
) # pragma: no cover - Connection check debug, tested via integration tests
return {} # pragma: no cover - Connection check early return, tested via integration tests
if session.connection_id is None:
self.logger.debug(
"No connection ID for tracker %s:%s", host, port
) # pragma: no cover - Connection ID check debug, tested via integration tests
return {} # pragma: no cover - Connection ID check early return, tested via integration tests
# Create scrape request
transaction_id = self._get_transaction_id()
request_data = self._encode_scrape_request(
session.connection_id, transaction_id, info_hash
)
# Send scrape request
# Validate socket is ready
self._validate_socket_ready()
# Use lock to serialize socket operations
async with self._socket_lock:
# Send scrape request (transport is guaranteed to be non-None after validation)
if self.transport is None:
raise RuntimeError("Transport is None after validation")
# CRITICAL FIX: On Windows ProactorEventLoop, ensure socket is fully ready before sendto
import sys
loop = asyncio.get_event_loop()
is_proactor = isinstance(loop, asyncio.ProactorEventLoop)
if sys.platform == "win32" and is_proactor:
# Small delay to ensure socket state is synchronized on Windows Proactor
await asyncio.sleep(0.01)
# Wrap sendto in try/except to catch WinError 10022 and other socket errors
try:
self.transport.sendto(request_data, tracker_address)
except OSError as send_error:
# Check if this is WinError 10022 (transient on Windows)
import sys
error_code = getattr(send_error, "winerror", None) or getattr(
send_error, "errno", None
)
is_winerror_10022 = (
error_code == 10022
or (hasattr(send_error, "errno") and send_error.errno == 22)
or (sys.platform == "win32" and "10022" in str(send_error))
)
if is_winerror_10022:
# WinError 10022 is transient - log and re-raise for caller to handle
self.logger.debug(
"WinError 10022 during sendto to %s:%d (will retry): %s",
tracker_address[0],
tracker_address[1],
send_error,
)
# Re-raise to be caught by caller's exception handler
raise
# Wait for response
response_data = await self._wait_for_response(transaction_id, timeout=10.0)
if response_data:
# Parse scrape response
return self._decode_scrape_response(response_data, info_hash)
self.logger.debug(
"No response from tracker for scrape"
) # pragma: no cover - No response debug, tested via integration tests with timeout
return {} # pragma: no cover - No response early return, tested via integration tests
except (
Exception
): # pragma: no cover - Scrape exception, defensive error handling
self.logger.exception("UDP scrape failed")
return {}
|
start() -> None
async
Start the UDP tracker client.
CRITICAL: Socket must be initialized during daemon startup via start_udp_tracker_client().
Socket recreation is not supported as it breaks session logic.
Source code in ccbt/discovery/tracker_udp_client.py
| async def start(self) -> None:
"""Start the UDP tracker client.
CRITICAL: Socket must be initialized during daemon startup via start_udp_tracker_client().
Socket recreation is not supported as it breaks session logic.
"""
# CRITICAL FIX: Assert socket should never be recreated during runtime
# If socket is already initialized and healthy, return immediately
# Socket recreation breaks session logic and causes WinError 10022 on Windows
if (
self._socket_ready
and self.transport is not None
and not self.transport.is_closing()
):
if self._check_socket_health():
self.logger.debug(
"UDP socket already ready and healthy, skipping start() (socket should never be recreated)"
)
# Reset error counters since socket is healthy
self._socket_error_count = 0
self._socket_recreation_backoff = 1.0
return
# Socket is marked ready but health check failed - log warning
self.logger.warning(
"UDP socket marked ready but health check failed. "
"Socket should have been initialized during daemon startup. "
"Attempting recovery (this should not happen in normal operation)."
)
# Use lock to prevent concurrent start() calls
async with self._socket_lock:
# CRITICAL FIX: Double-check socket health after acquiring lock
if (
self._socket_ready
and self.transport is not None
and not self.transport.is_closing()
):
if self._check_socket_health():
self.logger.debug(
"UDP socket already ready and healthy after lock acquisition, skipping start()"
)
return
# CRITICAL FIX: Apply exponential backoff to prevent aggressive socket recreation
current_time = time.time()
time_since_last_recreation = current_time - self._socket_last_error_time
if (
self._socket_recreation_count > 0
and time_since_last_recreation < self._socket_recreation_backoff
):
wait_time = self._socket_recreation_backoff - time_since_last_recreation
self.logger.debug(
"Socket recreation backoff active: waiting %.1fs before recreation (recreation_count: %d)",
wait_time,
self._socket_recreation_count,
)
await asyncio.sleep(wait_time)
# Increment recreation counter and update backoff
self._socket_recreation_count += 1
self._socket_recreation_backoff = min(
self._socket_recreation_backoff * 2.0,
self._max_socket_recreation_backoff,
)
self._socket_last_error_time = current_time
# CRITICAL FIX: Prevent socket recreation - fail gracefully instead
# Socket should have been initialized during daemon startup
# Recreation breaks session logic and causes WinError 10022 on Windows
if self.transport is not None and not self.transport.is_closing():
self.logger.error(
"CRITICAL: Attempted to recreate UDP tracker socket during runtime. "
"Socket should have been initialized during daemon startup and never recreated. "
"This breaks session logic. Socket state: ready=%s, transport=%s, closing=%s, error_count=%d. "
"Failing gracefully instead of recreating socket.",
self._socket_ready,
self.transport is not None,
self.transport.is_closing() if self.transport else None,
self._socket_error_count,
)
raise RuntimeError(
"UDP tracker socket recreation is not allowed. "
"Socket must be initialized during daemon startup via start_udp_tracker_client(). "
"If socket is invalid, daemon must be restarted."
)
# Mark socket as not ready before closing (lock already held)
# Only close if transport exists and is closing (cleanup scenario)
self._socket_ready = False
if self.transport is not None:
try:
self.transport.close()
# CRITICAL FIX: Wait longer on Windows Proactor for socket to fully close
import sys
loop = asyncio.get_event_loop()
is_proactor = isinstance(loop, asyncio.ProactorEventLoop)
if sys.platform == "win32" and is_proactor:
await asyncio.sleep(0.3) # Longer wait for Proactor
else:
await asyncio.sleep(0.1)
except Exception as e:
self.logger.debug("Error closing existing transport: %s", e)
finally:
self.transport = None
self.socket = None
# CRITICAL FIX: Only create new socket if transport is None or closing
# This should only happen during initial daemon startup, not during runtime
if self.transport is not None and not self.transport.is_closing():
self.logger.error(
"CRITICAL: Attempted to create new UDP socket when existing socket is valid. "
"This should never happen - socket should be initialized once at daemon startup."
)
raise RuntimeError(
"Cannot create new UDP socket - existing socket is valid"
)
# Create UDP socket
import socket as std_socket
loop = asyncio.get_event_loop()
# Create socket with proper options
sock = std_socket.socket(std_socket.AF_INET, std_socket.SOCK_DGRAM)
try:
# Set socket options
try:
sock.setsockopt(
std_socket.SOL_SOCKET, std_socket.SO_RCVBUF, 131072
) # 128KB
sock.setsockopt(
std_socket.SOL_SOCKET, std_socket.SO_SNDBUF, 131072
) # 128KB
except OSError:
pass # Continue if buffer size setting fails
# Set socket to non-blocking mode for asyncio
sock.setblocking(False)
# Bind to configured tracker UDP port
# Use tracker_udp_port if available, fallback to listen_port for backward compatibility
configured_port = (
self.config.network.tracker_udp_port
or self.config.network.listen_port
)
sock.bind(("0.0.0.0", configured_port)) # nosec B104 - Bind to all interfaces on configured port
self.logger.debug("Bound UDP tracker socket to port %d", configured_port)
except OSError as e:
sock.close()
# CRITICAL FIX: Enhanced port conflict error handling
error_code = e.errno if hasattr(e, "errno") else None
import sys
if sys.platform == "win32":
if error_code == 10048: # WSAEADDRINUSE
from ccbt.utils.port_checker import get_port_conflict_resolution
resolution = get_port_conflict_resolution(configured_port, "udp")
error_msg = (
f"UDP tracker port {configured_port} is already in use.\n"
f"Error: {e}\n\n"
f"{resolution}"
)
self.logger.error(error_msg)
raise RuntimeError(error_msg) from e
elif error_code == 10013: # WSAEACCES
from ccbt.utils.port_checker import get_permission_error_resolution
resolution = get_permission_error_resolution(
configured_port, "udp", "network.tracker_udp_port"
)
error_msg = (
f"Permission denied binding to 0.0.0.0:{configured_port}.\n"
f"Error: {e}\n\n"
f"{resolution}"
)
self.logger.error(error_msg)
raise RuntimeError(error_msg) from e
else:
if error_code == 98: # EADDRINUSE
from ccbt.utils.port_checker import get_port_conflict_resolution
resolution = get_port_conflict_resolution(configured_port, "udp")
error_msg = (
f"UDP tracker port {configured_port} is already in use.\n"
f"Error: {e}\n\n"
f"{resolution}"
)
self.logger.error(error_msg)
raise RuntimeError(error_msg) from e
elif error_code == 13: # EACCES
from ccbt.utils.port_checker import get_permission_error_resolution
resolution = get_permission_error_resolution(
configured_port, "udp", "network.tracker_udp_port"
)
error_msg = (
f"Permission denied binding to 0.0.0.0:{configured_port}.\n"
f"Error: {e}\n\n"
f"{resolution}"
)
self.logger.error(error_msg)
raise RuntimeError(error_msg) from e
# Re-raise other OSErrors as-is
self.logger.error("Failed to create UDP socket: %s", e)
raise
# Create datagram endpoint with the configured socket
try:
self.transport, self.socket = await loop.create_datagram_endpoint(
lambda: UDPTrackerProtocol(self),
sock=sock,
)
except Exception as e:
sock.close()
self.logger.error("Failed to create datagram endpoint: %s", e)
raise
# Start cleanup task
if self._cleanup_task is None or self._cleanup_task.done():
self._cleanup_task = asyncio.create_task(self._cleanup_loop())
# Verify socket is properly bound and listening
if self.transport is None:
raise RuntimeError("Transport not initialized after socket creation")
# Log socket binding information
try:
sockname = self.transport.get_extra_info("sockname")
if sockname:
self.logger.info(
"UDP socket bound to %s:%d",
sockname[0] if sockname else "unknown",
sockname[1] if sockname else 0,
)
except Exception as e:
self.logger.debug("Could not get socket name: %s", e)
# Verify protocol is registered
if not isinstance(self.socket, UDPTrackerProtocol):
self.logger.warning("Socket protocol may not be properly registered")
# CRITICAL FIX: Verify socket is actually ready before marking as ready
# Perform a health check to ensure socket can send/receive
try:
# Verify transport is not closing
if self.transport.is_closing():
raise RuntimeError("Transport is closing immediately after creation")
# Verify socket name is available (indicates socket is bound)
sockname = self.transport.get_extra_info("sockname")
if sockname is None:
raise RuntimeError("Socket name not available after creation")
# Mark socket as ready only after verification
self._socket_ready = True
# Reset error counters on successful initialization
self._socket_error_count = 0
self._socket_recreation_count = 0
self._socket_recreation_backoff = 1.0
self._last_socket_health_check = time.time()
self.logger.info(
"UDP tracker client started and ready (socket bound to %s:%d)",
sockname[0] if sockname else "unknown",
sockname[1] if sockname else 0,
)
except Exception as e:
self._socket_ready = False
self.logger.error(
"Socket initialization verification failed: %s. Socket may not be ready.",
e,
)
raise
|
stop() -> None
async
Stop the UDP tracker client.
Source code in ccbt/discovery/tracker_udp_client.py
| async def stop(self) -> None:
"""Stop the UDP tracker client."""
# Mark socket as not ready first
self._socket_ready = False
if self._cleanup_task:
self._cleanup_task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await self._cleanup_task
# CRITICAL FIX: Ensure proper cleanup of transport on Windows
# Close transport and wait for it to fully close before proceeding
if self.transport:
try:
self.transport.close()
# CRITICAL FIX: On Windows Proactor, wait for transport to fully close
# This prevents WinError 10022 when socket is reused
import sys
loop = asyncio.get_event_loop()
is_proactor = isinstance(loop, asyncio.ProactorEventLoop)
if sys.platform == "win32" and is_proactor:
# Wait longer for Proactor to release socket resources
await asyncio.sleep(0.3)
elif sys.platform == "win32":
await asyncio.sleep(0.1)
except Exception as e:
self.logger.debug("Error closing transport: %s", e)
finally:
self.transport = None
self.socket = None
# Cancel pending requests
for future in self.pending_requests.values():
if not future.done():
future.cancel()
self.logger.info("UDP tracker client stopped")
|
Features:
- BEP 15 compliant: ccbt/discovery/tracker_udp_client.py:AsyncUDPTrackerClient - Full UDP tracker protocol support
- Connection ID management: ccbt/discovery/tracker_udp_client.py:TrackerSession - Tracks connection IDs per tracker
- Transaction ID tracking: ccbt/discovery/tracker_udp_client.py - Handles concurrent requests
TrackerServerHTTP
HTTP tracker server implementation.
Implementation: ccbt/discovery/tracker_server_http.py
TrackerServerUDP
UDP tracker server implementation.
Implementation: ccbt/discovery/tracker_server_udp.py
PEX
Peer Exchange (BEP 11) for peer discovery.
Implementation: ccbt/discovery/pex.py
Features:
- Peer exchange with other clients
- Automatic peer sharing
- PEX extension support
Configuration: ccbt.toml:128-129
Services
Service Base
Base service class for service-oriented architecture.
Implementation: ccbt/services/base.py
Service states: ccbt/services/base.py:ServiceState
Service error: ccbt/services/base.py:ServiceError
Service manager: ccbt/services/base.py:ServiceManager
PeerService
Manages peer connections and communication.
Implementation: ccbt/services/peer_service.py
Service exports: ccbt/services/init.py
StorageService
Manages file system operations with high-performance chunked writes.
Implementation: ccbt/services/storage_service.py
Features:
- File creation and management
- Data read/write operations with chunked writes for large files
- File assembly coordination
- Configurable file size limits via disk.max_file_size_mb
- Integration with DiskIOManager for optimized disk I/O
Write Operations
The write_file() method implements chunked writes for optimal performance:
- Small files (≤
write_buffer_kib): Written in a single operation
- Large files (>
write_buffer_kib): Written in chunks using DiskIOManager.write_block()
- Memory efficiency: Uses
memoryview for zero-copy chunk slicing
- Size limits: Enforces
max_file_size_mb from configuration (0/None = unlimited)
Write implementation: ccbt/services/storage_service.py:_write_file
Configuration
disk.max_file_size_mb: Maximum file size in MB (0 or None = unlimited, max 1TB)disk.write_buffer_kib: Chunk size for large file writes- Default: Unlimited (0) for production, configurable for testing
Configuration: ccbt.toml:87-89
TrackerService
Handles tracker communication.
Implementation: ccbt/services/tracker_service.py
Features:
- Tracker registration
- Announce coordination
- Scrape operations
Storage
DiskIOManager
High-performance disk I/O manager with preallocation, batching, memory-mapped I/O, and async operations.
High-performance disk I/O manager with preallocation, batching, and mmap.
Initialize disk I/O manager.
Parameters:
| Name |
Type |
Description |
Default |
max_workers
|
int
|
Number of disk I/O worker threads
|
2
|
queue_size
|
int
|
Maximum size of write queue
|
200
|
cache_size_mb
|
int
|
Maximum size of mmap cache in MB
|
256
|
Source code in ccbt/storage/disk_io.py
| def __init__(
self,
max_workers: int = 2,
queue_size: int = 200,
cache_size_mb: int = 256,
):
"""Initialize disk I/O manager.
Args:
max_workers: Number of disk I/O worker threads
queue_size: Maximum size of write queue
cache_size_mb: Maximum size of mmap cache in MB
"""
self.config = get_config()
self.max_workers = max_workers
self.queue_size = queue_size
self.cache_size_mb = cache_size_mb
self.cache_size_bytes = cache_size_mb * 1024 * 1024
# Thread pool for disk I/O (will be adjusted adaptively if enabled)
self.executor = ThreadPoolExecutor(
max_workers=max_workers,
thread_name_prefix="disk-io",
)
self._worker_adjustment_task: asyncio.Task[None] | None = None
# Write batching
# Use priority queue if enabled, otherwise regular queue
if self.config.disk.write_queue_priority:
# Priority queue using heapq for ordered processing
self._write_queue_heap: list[WriteRequest] = []
self._write_queue_lock = asyncio.Lock()
self._write_queue_condition = asyncio.Condition(self._write_queue_lock)
self.write_queue: asyncio.Queue[WriteRequest] | None = (
None # Will be handled by priority queue methods
)
else: # pragma: no cover - Non-priority queue mode not tested, priority queue is default
self.write_queue: asyncio.Queue[WriteRequest] = asyncio.Queue(
maxsize=queue_size
)
self.write_requests: dict[Path, list[WriteRequest]] = {}
self.write_lock = threading.Lock()
# Memory-mapped file cache
self.mmap_cache: dict[Path, MmapCache] = {}
self.cache_lock = threading.Lock()
self.cache_size = 0
# Zero-copy staging buffer (simple ring buffer)
try:
self.ring_buffer = get_buffer_manager().create_ring_buffer(
max(
1024 * 1024,
int(self.write_buffer_kib) * 1024
if hasattr(self, "write_buffer_kib")
and isinstance(self.write_buffer_kib, (int, float))
else 1024 * 1024,
),
) # type: ignore[attr-defined]
except Exception: # pragma: no cover - Buffer manager initialization failure, defensive fallback
self.ring_buffer = None
# Thread-local staging buffers to avoid per-call allocation and avoid contention
self._thread_local: threading.local = threading.local()
# Advanced I/O features
# Respect config toggles: enable_io_uring, direct_io
try:
self.io_uring_enabled = (
bool(self.config.disk.enable_io_uring) and HAS_IO_URING
)
except (
Exception
): # pragma: no cover - Config access error handling, defensive check
self.io_uring_enabled = HAS_IO_URING
self.direct_io_enabled = bool(self.config.disk.direct_io)
self.nvme_optimized = False
self.storage_type: str = (
"hdd" # Will be detected in _detect_platform_capabilities
)
self.write_cache_enabled: bool = True
# Read pattern tracking for adaptive read-ahead
self._read_patterns: dict[Path, ReadPattern] = {}
self._read_pattern_lock = threading.Lock()
# Read buffer pool
self._read_buffer_pool: list[bytearray] = []
self._read_buffer_pool_lock = threading.Lock()
# Background tasks
self._write_batcher_task: asyncio.Task[None] | None = None
self._cache_cleaner_task: asyncio.Task[None] | None = None
self._cache_adaptive_task: asyncio.Task[None] | None = None
self._worker_adjustment_task: asyncio.Task[None] | None = None
# Flag to track if manager is running (for cancellation checks)
self._running = False
# Xet deduplication (lazy initialization)
self._xet_deduplication: Any | None = None
# Statistics
self.stats = {
"writes": 0,
"bytes_written": 0,
"cache_hits": 0,
"cache_misses": 0,
"cache_evictions": 0,
"cache_total_accesses": 0,
"cache_bytes_served": 0,
"preallocations": 0,
"queue_full_errors": 0,
"io_uring_operations": 0,
"direct_io_operations": 0,
"nvme_optimizations": 0,
}
self._cache_stats_start_time = time.time()
self.logger = get_logger(__name__)
self._detect_platform_capabilities()
|
get_cache_stats() -> dict[str, int | float]
Return mmap cache statistics with detailed metrics.
Source code in ccbt/storage/disk_io.py
| def get_cache_stats(self) -> dict[str, int | float]:
"""Return mmap cache statistics with detailed metrics."""
with self.cache_lock:
hits = self.stats.get("cache_hits", 0)
misses = self.stats.get("cache_misses", 0)
total_accesses = hits + misses
evictions = self.stats.get("cache_evictions", 0)
bytes_served = self.stats.get("cache_bytes_served", 0)
# Calculate hit rate
hit_rate = (hits / total_accesses * 100) if total_accesses > 0 else 0.0
# Calculate eviction rate (evictions per second)
elapsed = time.time() - self._cache_stats_start_time
eviction_rate = evictions / elapsed if elapsed > 0 else 0.0
# Calculate cache efficiency (bytes served from cache / total bytes read)
# Note: This is an approximation - we track bytes_served but not total bytes read
cache_efficiency = (
(bytes_served / (bytes_served + misses * 65536) * 100)
if (bytes_served + misses * 65536) > 0
else 0.0
)
return {
"entries": len(self.mmap_cache),
"total_size": sum(entry.size for entry in self.mmap_cache.values()),
"cache_hits": hits,
"cache_misses": misses,
"cache_evictions": evictions,
"hit_rate_percent": hit_rate,
"eviction_rate_per_sec": eviction_rate,
"cache_efficiency_percent": cache_efficiency,
"total_accesses": total_accesses,
"average_access_time": (
sum(entry.last_access for entry in self.mmap_cache.values())
/ len(self.mmap_cache)
if self.mmap_cache
else 0.0
),
}
|
preallocate_file(file_path: Path, size: int) -> None
async
Preallocate file space.
Parameters:
| Name |
Type |
Description |
Default |
file_path
|
Path
|
|
required
|
size
|
int
|
Size to preallocate in bytes
|
required
|
Source code in ccbt/storage/disk_io.py
| async def preallocate_file(self, file_path: Path, size: int) -> None:
"""Preallocate file space.
Args:
file_path: Path to file
size: Size to preallocate in bytes
"""
config = get_config()
strategy = config.disk.preallocate
if strategy == PreallocationStrategy.NONE:
return
try:
await asyncio.get_event_loop().run_in_executor(
self.executor,
self._preallocate_file_sync,
file_path,
size,
strategy,
)
self.stats["preallocations"] += 1
except Exception as e:
self.logger.exception("Failed to preallocate %s", file_path)
msg = f"Preallocation failed: {e}"
raise DiskIOError(msg) from e
|
read_block(file_path: Path, offset: int, length: int) -> bytes
async
Asynchronously read a block of data from a file, using mmap cache if enabled.
Parameters:
| Name |
Type |
Description |
Default |
file_path
|
Path
|
|
required
|
offset
|
int
|
|
required
|
length
|
int
|
|
required
|
Returns:
Source code in ccbt/storage/disk_io.py
| async def read_block(self, file_path: Path, offset: int, length: int) -> bytes:
"""Asynchronously read a block of data from a file, using mmap cache if enabled.
Args:
file_path: Path to file
offset: Offset in bytes to read
length: Number of bytes to read
Returns:
The read data as bytes.
"""
# Update read pattern tracking
with self._read_pattern_lock:
if file_path not in self._read_patterns:
self._read_patterns[file_path] = ReadPattern()
self._read_patterns[file_path].update(offset)
config = get_config()
if config.disk.use_mmap:
with self.cache_lock:
cache_entry = self._get_mmap_entry(file_path)
if cache_entry:
self.stats["cache_hits"] = self.stats.get("cache_hits", 0) + 1
self.stats["cache_total_accesses"] = (
self.stats.get("cache_total_accesses", 0) + 1
)
self.stats["cache_bytes_served"] = (
self.stats.get("cache_bytes_served", 0) + length
)
cache_entry.last_access = time.time()
return await asyncio.get_event_loop().run_in_executor(
self.executor,
lambda: cache_entry.mmap_obj[offset : offset + length],
)
self.stats["cache_misses"] = self.stats.get("cache_misses", 0) + 1
self.stats["cache_total_accesses"] = (
self.stats.get("cache_total_accesses", 0) + 1
)
# Fallback to direct read if mmap not used or cache miss
# Use adaptive read-ahead if enabled
read_ahead_size = self._get_read_ahead_size(file_path, offset, length)
effective_length = max(length, read_ahead_size)
return await asyncio.get_event_loop().run_in_executor(
self.executor,
self._read_block_sync,
file_path,
offset,
effective_length,
)
|
read_block_mmap(file_path: str | Path, offset: int, length: int) -> bytes
async
Read a block of data using memory mapping for better performance.
Uses an ephemeral read-only mmap to avoid persisting OS file locks.
Parameters:
| Name |
Type |
Description |
Default |
file_path
|
str | Path
|
|
required
|
offset
|
int
|
|
required
|
length
|
int
|
|
required
|
Returns:
Source code in ccbt/storage/disk_io.py
| async def read_block_mmap(
self,
file_path: str | Path,
offset: int,
length: int,
) -> bytes:
"""Read a block of data using memory mapping for better performance.
Uses an ephemeral read-only mmap to avoid persisting OS file locks.
Args:
file_path: Path to the file
offset: Byte offset in the file
length: Number of bytes to read
Returns:
The requested bytes
"""
# Use ephemeral mapping when mmap isn't normally enabled to avoid persisting locks
try:
fp = Path(file_path)
with open(fp, "rb") as f:
end = f.seek(0, os.SEEK_END)
if end == 0:
return b""
mm = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)
try:
# Count as a cache hit for stats visibility in tests
self.stats["cache_hits"] = self.stats.get("cache_hits", 0) + 1
self.stats["cache_total_accesses"] = (
self.stats.get("cache_total_accesses", 0) + 1
)
self.stats["cache_bytes_served"] = (
self.stats.get("cache_bytes_served", 0) + length
)
return mm[offset : offset + length]
finally:
mm.close()
except FileNotFoundError:
raise
except Exception: # pragma: no cover - Fallback path for mmap read errors, defensive error handling
# Fallback to normal read path
return await self.read_block(Path(file_path), offset, length)
|
read_xet_chunk(chunk_hash: bytes) -> bytes | None
async
Read chunk by hash from Xet storage.
Parameters:
| Name |
Type |
Description |
Default |
chunk_hash
|
bytes
|
|
required
|
Returns:
| Type |
Description |
bytes | None
|
Chunk data if found, None otherwise
|
Raises:
| Type |
Description |
ValueError
|
If chunk_hash is not 32 bytes
|
Source code in ccbt/storage/disk_io.py
| async def read_xet_chunk(self, chunk_hash: bytes) -> bytes | None:
"""Read chunk by hash from Xet storage.
Args:
chunk_hash: 32-byte chunk hash
Returns:
Chunk data if found, None otherwise
Raises:
ValueError: If chunk_hash is not 32 bytes
"""
if len(chunk_hash) != 32:
msg = f"Chunk hash must be 32 bytes, got {len(chunk_hash)}"
raise ValueError(msg)
if not self.config.disk.xet_enabled:
return None
dedup = self._get_xet_deduplication()
if not dedup:
return None
try:
chunk_path = await dedup.check_chunk_exists(chunk_hash)
if not chunk_path or not chunk_path.exists():
return None
# Read chunk data
return await asyncio.get_event_loop().run_in_executor(
self.executor,
chunk_path.read_bytes,
)
except Exception as e:
self.logger.warning(
"Failed to read Xet chunk %s: %s",
chunk_hash.hex()[:16],
e,
)
return None
|
start() -> None
async
Start background tasks.
Source code in ccbt/storage/disk_io.py
| async def start(self) -> None:
"""Start background tasks."""
self._running = True
self._write_batcher_task = asyncio.create_task(self._write_batcher())
self._cache_cleaner_task = asyncio.create_task(self._cache_cleaner())
if self.config.disk.mmap_cache_adaptive:
self._cache_adaptive_task = asyncio.create_task(self._adaptive_cache_size())
if self.config.disk.disk_workers_adaptive:
self._worker_adjustment_task = asyncio.create_task(self._adjust_workers())
self.logger.info("Disk I/O manager started with %s workers", self.max_workers)
|
stop() -> None
async
Stop background tasks and cleanup.
Source code in ccbt/storage/disk_io.py
| async def stop(self) -> None:
"""Stop background tasks and cleanup."""
self._running = False
# Cancel and await write batcher task to ensure cancellation handler completes
if self._write_batcher_task:
self._write_batcher_task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await self._write_batcher_task
# Give cancellation handler a moment to process
await asyncio.sleep(0.001)
if self._cache_cleaner_task:
self._cache_cleaner_task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await self._cache_cleaner_task
if self._cache_adaptive_task:
self._cache_adaptive_task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await self._cache_adaptive_task
if self._worker_adjustment_task:
self._worker_adjustment_task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await self._worker_adjustment_task
# Flush remaining writes and wait for completion (with timeout)
try:
await asyncio.wait_for(self._flush_all_writes(), timeout=5.0)
except asyncio.TimeoutError: # Tested in test_disk_io_coverage_gaps.py::TestStopCleanupPaths::test_flush_timeout_during_stop
self.logger.warning("Timeout flushing writes during stop")
# Verify all futures in write_requests are done after flush
with self.write_lock:
remaining_futures = [
req.future
for requests in self.write_requests.values()
for req in requests
if not req.future.done()
]
if remaining_futures: # Tested in test_disk_io_coverage_gaps.py::TestStopCleanupPaths::test_remaining_futures_after_flush
self.logger.warning(
"Found %d futures still pending after flush, setting exceptions",
len(remaining_futures),
)
for future in remaining_futures:
if not future.done(): # Race condition: future may complete between checks, tested via test_remaining_futures_after_flush
future.set_exception(asyncio.CancelledError())
# Collect all pending futures from queue (batcher was cancelled, so these won't be processed)
pending_futures = []
if self.config.disk.write_queue_priority:
# Collect from priority queue
async with self._write_queue_lock:
while self._write_queue_heap:
request = heapq.heappop(self._write_queue_heap)
if not request.future.done():
pending_futures.append(request.future)
# Collect from regular queue
elif self.write_queue is not None:
while not self.write_queue.empty():
try:
request = self.write_queue.get_nowait()
if not request.future.done():
pending_futures.append(request.future)
except asyncio.QueueEmpty: # Tested in test_disk_io_coverage_gaps.py::TestStopCleanupPaths::test_queue_empty_race_condition
break
# Collect any remaining futures from write_requests
with self.write_lock:
pending_futures.extend(
[
req.future
for requests in self.write_requests.values()
for req in requests
if not req.future.done()
]
)
# Explicitly set exceptions on all pending futures
for future in pending_futures:
if not future.done():
with contextlib.suppress(Exception):
# Future might already be done or in invalid state
future.set_exception(asyncio.CancelledError())
# Wait for all futures to be done using gather with timeout
if pending_futures: # Tested in test_disk_io_coverage_gaps.py::TestStopCleanupPaths::test_future_completion_polling_timeout
try:
await asyncio.wait_for(
asyncio.gather(*pending_futures, return_exceptions=True),
timeout=0.1,
)
except asyncio.TimeoutError: # Tested in test_disk_io_coverage_gaps.py::TestStopCleanupPaths::test_future_completion_polling_timeout
# Force completion of any still-pending futures
for future in pending_futures:
if not future.done():
try:
future.cancel()
# Wait briefly for cancellation
await asyncio.sleep(0.001)
except Exception: # pragma: no cover - Exception during future cancellation cleanup, defensive error handling
pass
# Final verification: ensure all collected futures are actually done
# Poll until all futures are done (handles race conditions where set_exception
# doesn't immediately make the future done, or where cancellation needs time)
max_iterations = 20
for _iteration in range(
max_iterations
): # Tested in test_disk_io_coverage_gaps.py::TestStopCleanupPaths::test_future_completion_polling_loop
all_done = True
for future in pending_futures:
if not future.done(): # Tested via test_future_completion_polling_loop
all_done = False
with contextlib.suppress(Exception):
# Try setting exception again
future.set_exception(asyncio.CancelledError())
# If still not done, try cancelling
if not future.done(): # Race condition: future state may change, tested via test_future_completion_polling_loop
future.cancel()
if all_done:
break
# Give futures time to transition to done state after exception/cancellation
await asyncio.sleep(
0.001
) # pragma: no cover - Timing-dependent cleanup path, hard to test reliably
# One final forced pass - any still-pending futures get maximum effort
for future in pending_futures:
if not future.done(): # pragma: no cover - Defensive cleanup: futures should be done by this point
with contextlib.suppress(Exception):
# Try everything to force completion
future.set_exception(asyncio.CancelledError())
if (
not future.done()
): # pragma: no cover - Race condition: future state may change
future.cancel()
# One more check after cancellation
if (
not future.done()
): # pragma: no cover - Last resort path: very rare edge case
# Last resort: try to get the result to force completion
with contextlib.suppress(
asyncio.TimeoutError, asyncio.CancelledError
):
await asyncio.wait_for(
future, timeout=0.0
) # pragma: no cover - Last resort path
# Close mmap cache (ensure handles are closed so Windows can delete files)
with self.cache_lock:
for cache_entry in self.mmap_cache.values():
self._close_cache_entry_safely(cache_entry)
self.mmap_cache.clear()
# On Windows, give the OS a brief moment to release file handles
if sys.platform == "win32":
await self._windows_cleanup_delay()
# Shutdown executor with timeout to prevent hanging
await self._shutdown_executor_safely()
self.logger.info("Disk I/O manager stopped")
|
warmup_cache(file_paths: list[Path], priority_order: list[int] | None = None) -> None
async
Warmup cache by pre-loading frequently accessed files.
Parameters:
| Name |
Type |
Description |
Default |
file_paths
|
list[Path]
|
List of file paths to warmup
|
required
|
priority_order
|
list[int] | None
|
Optional list of priorities (0 = highest priority)
|
None
|
Source code in ccbt/storage/disk_io.py
| async def warmup_cache(
self, file_paths: list[Path], priority_order: list[int] | None = None
) -> None:
"""Warmup cache by pre-loading frequently accessed files.
Args:
file_paths: List of file paths to warmup
priority_order: Optional list of priorities (0 = highest priority)
"""
if not self.config.disk.mmap_cache_warmup:
return
# Sort by priority if provided, otherwise use file order
if priority_order:
files_with_priority = list(zip(file_paths, priority_order))
files_with_priority.sort(key=lambda x: x[1])
file_paths = [fp for fp, _ in files_with_priority]
# Load files in background, limiting concurrent loads
max_concurrent = min(4, len(file_paths))
semaphore = asyncio.Semaphore(max_concurrent)
async def load_file(file_path: Path) -> None:
async with semaphore:
try:
if file_path.exists():
with self.cache_lock:
if file_path not in self.mmap_cache:
self._get_mmap_entry(file_path)
except Exception as e:
self.logger.debug("Failed to warmup cache for %s: %s", file_path, e)
# Load files concurrently
tasks = [load_file(fp) for fp in file_paths[:20]] # Limit to first 20 files
await asyncio.gather(*tasks, return_exceptions=True)
self.logger.debug("Cache warmup completed for %d files", len(tasks))
|
write_block(file_path: Path, offset: int, data: bytes, priority: int = 0) -> asyncio.Future
async
Asynchronously write a block of data to a file.
Parameters:
| Name |
Type |
Description |
Default |
file_path
|
Path
|
|
required
|
offset
|
int
|
|
required
|
data
|
bytes
|
|
required
|
priority
|
int
|
Write priority (0=regular, 50=metadata, 100=checkpoint)
|
0
|
Returns:
| Type |
Description |
Future
|
An asyncio.Future that will be set when the write is complete.
|
Source code in ccbt/storage/disk_io.py
| async def write_block(
self,
file_path: Path,
offset: int,
data: bytes,
priority: int = 0,
) -> asyncio.Future:
"""Asynchronously write a block of data to a file.
Args:
file_path: Path to file
offset: Offset in bytes to write
data: Data to write
priority: Write priority (0=regular, 50=metadata, 100=checkpoint)
Returns:
An asyncio.Future that will be set when the write is complete.
"""
future = asyncio.get_event_loop().create_future()
request = WriteRequest(file_path, offset, data, future, priority=priority)
try:
if self.config.disk.write_queue_priority:
await self._put_write_request(request)
elif self.write_queue is not None:
self.write_queue.put_nowait(request)
except asyncio.QueueFull: # pragma: no cover - Queue full error path, requires saturating queue which is hard to test deterministically
self.stats["queue_full_errors"] += 1
future.set_exception(DiskIOError("Disk I/O write queue is full"))
return future
# Let the batcher handle the write - the future will be completed by the batcher
return future
|
write_xet_chunk(chunk_hash: bytes, chunk_data: bytes, file_path: Path, offset: int) -> bool
async
Write Xet chunk with deduplication check.
Checks if chunk already exists in deduplication cache. If it does,
creates a reference link instead of storing duplicate data. Otherwise,
stores the chunk and updates the deduplication cache.
Parameters:
| Name |
Type |
Description |
Default |
chunk_hash
|
bytes
|
32-byte chunk hash (BLAKE3-256)
|
required
|
chunk_data
|
bytes
|
|
required
|
file_path
|
Path
|
Path to file where chunk should be referenced
|
required
|
offset
|
int
|
Offset in file where chunk should be referenced
|
required
|
Returns:
| Type |
Description |
bool
|
True if successful, False otherwise
|
Raises:
| Type |
Description |
ValueError
|
If chunk_hash is not 32 bytes
|
DiskIOError
|
If storage operation fails
|
Source code in ccbt/storage/disk_io.py
| async def write_xet_chunk(
self,
chunk_hash: bytes,
chunk_data: bytes,
file_path: Path,
offset: int,
) -> bool:
"""Write Xet chunk with deduplication check.
Checks if chunk already exists in deduplication cache. If it does,
creates a reference link instead of storing duplicate data. Otherwise,
stores the chunk and updates the deduplication cache.
Args:
chunk_hash: 32-byte chunk hash (BLAKE3-256)
chunk_data: Chunk data bytes
file_path: Path to file where chunk should be referenced
offset: Offset in file where chunk should be referenced
Returns:
True if successful, False otherwise
Raises:
ValueError: If chunk_hash is not 32 bytes
DiskIOError: If storage operation fails
"""
if len(chunk_hash) != 32:
msg = f"Chunk hash must be 32 bytes, got {len(chunk_hash)}"
raise ValueError(msg)
if not self.config.disk.xet_enabled:
# Xet not enabled, fall back to standard write
future = await self.write_block(file_path, offset, chunk_data)
await future
return True
dedup = self._get_xet_deduplication()
if not dedup:
# Deduplication not available, use standard write
future = await self.write_block(file_path, offset, chunk_data)
await future
return True
try:
# Check if chunk exists
existing_path = await dedup.check_chunk_exists(chunk_hash)
if existing_path:
# Chunk exists - create reference link
return await self._link_chunk_reference(
chunk_hash, existing_path, file_path, offset
)
# Store new chunk
return await self._store_new_chunk(chunk_hash, chunk_data, dedup)
except Exception as e:
self.logger.exception(
"Failed to write Xet chunk %s",
chunk_hash.hex()[:16],
)
error_msg = f"Failed to write Xet chunk: {e}"
raise DiskIOError(error_msg) from e
|
Features:
- File preallocation: ccbt/storage/disk_io.py:preallocate_file - Supports NONE, SPARSE, FULL, FALLOCATE strategies
- Write batching: ccbt/storage/disk_io.py:write_block - Priority queue for write requests
- Memory-mapped I/O: ccbt/storage/disk_io.py:MmapCache - Cached memory-mapped files for fast access
- io_uring support (Linux): ccbt/storage/disk_io.py - High-performance async I/O on Linux
- Direct I/O support: ccbt/storage/disk_io.py - Bypass page cache for large files
- Parallel hash verification: ccbt/storage/disk_io.py - Thread pool for hash verification
Key Methods:
- write_block(): ccbt/storage/disk_io.py:write_block - Write data block to file with batching
- read_block(): ccbt/storage/disk_io.py:read_block - Read data block from file
- preallocate_file(): ccbt/storage/disk_io.py:preallocate_file - Preallocate file space
- verify_piece(): ccbt/storage/disk_io.py:verify_piece - Verify piece hash
Configuration: ccbt.toml:57-96
FileAssembler
Assembles pieces into complete files.
Implementation: ccbt/storage/file_assembler.py
Features:
- Piece-to-file mapping
- File assembly coordination
- Multi-file torrent support
CheckpointManager
Checkpoint management for resume functionality.
Manages torrent download checkpoints with JSON and binary checkpoint_formats.
Initialize checkpoint manager.
Parameters:
| Name |
Type |
Description |
Default |
config
|
DiskConfig | None
|
Disk configuration with checkpoint settings
|
None
|
Source code in ccbt/storage/checkpoint.py
| def __init__(self, config: DiskConfig | None = None):
"""Initialize checkpoint manager.
Args:
config: Disk configuration with checkpoint settings
"""
self.config = config or DiskConfig()
self.logger = get_logger(__name__)
# Determine checkpoint directory
if self.config.checkpoint_dir:
self.checkpoint_dir = Path(self.config.checkpoint_dir)
else:
# Default to download_dir/.ccbt/checkpoints
self.checkpoint_dir = Path(".ccbt/checkpoints")
# Ensure checkpoint directory exists
self.checkpoint_dir.mkdir(parents=True, exist_ok=True)
# Track last checkpoint state for incremental saves and deduplication
self._last_checkpoint_hash: bytes | None = None
self._last_checkpoint: TorrentCheckpoint | None = None
self.logger.info(
"Checkpoint manager initialized with directory: %s",
self.checkpoint_dir,
)
|
backup_checkpoint(info_hash: bytes, destination: Path, *, compress: bool = True, encrypt: bool = False) -> Path
async
Create a portable backup of the checkpoint at destination.
Backup checkpoint_format is JSON optionally gzipped and optionally encrypted if cryptography is available.
Source code in ccbt/storage/checkpoint.py
| async def backup_checkpoint(
self,
info_hash: bytes,
destination: Path,
*,
compress: bool = True,
encrypt: bool = False,
) -> Path:
"""Create a portable backup of the checkpoint at destination.
Backup checkpoint_format is JSON optionally gzipped and optionally encrypted if cryptography is available.
"""
cp = await self.load_checkpoint(info_hash)
if cp is None:
msg = f"No checkpoint for {info_hash.hex()}"
raise CheckpointNotFoundError(msg)
# Serialize JSON
data = await self.export_checkpoint(info_hash, fmt="json")
# Optional compression
if compress:
data = gzip.compress(data)
# Optional encryption
if encrypt:
try:
from cryptography.fernet import Fernet
except Exception:
msg = "Encryption requested but cryptography is not installed"
raise CheckpointError(
msg,
) from None
key_path = destination.with_suffix(destination.suffix + ".key")
key = Fernet.generate_key()
f = Fernet(key)
data = f.encrypt(data)
key_path.write_bytes(key)
destination.parent.mkdir(parents=True, exist_ok=True)
destination.write_bytes(data)
self.logger.info("Wrote checkpoint backup to %s", destination)
return destination
|
cleanup_old_checkpoints(max_age_days: int = 30) -> int
async
Clean up old checkpoint files.
Parameters:
| Name |
Type |
Description |
Default |
max_age_days
|
int
|
Maximum age in days before cleanup
|
30
|
Returns:
Source code in ccbt/storage/checkpoint.py
| async def cleanup_old_checkpoints(self, max_age_days: int = 30) -> int:
"""Clean up old checkpoint files.
Args:
max_age_days: Maximum age in days before cleanup
Returns:
Number of files deleted
"""
if (
not self.checkpoint_dir.exists()
): # pragma: no cover - Directory created in __init__, defensive check
return 0
cutoff_time = time.time() - (max_age_days * 24 * 60 * 60)
deleted_count = 0
for file_path in self.checkpoint_dir.glob("*.checkpoint.*"):
try:
if file_path.stat().st_mtime < cutoff_time:
file_path.unlink()
deleted_count += 1
self.logger.debug("Cleaned up old checkpoint: %s", file_path)
except Exception as e:
self.logger.warning("Failed to cleanup checkpoint %s: %s", file_path, e)
if deleted_count > 0:
self.logger.info("Cleaned up %s old checkpoint files", deleted_count)
return deleted_count
|
Convert checkpoint between checkpoint_formats.
Parameters:
| Name |
Type |
Description |
Default |
info_hash
|
bytes
|
|
required
|
from_checkpoint_format
|
CheckpointFormat
|
|
required
|
to_checkpoint_format
|
CheckpointFormat
|
|
required
|
Returns:
| Type |
Description |
Path
|
Path to converted checkpoint file
|
Source code in ccbt/storage/checkpoint.py
| async def convert_checkpoint_checkpoint_format( # pragma: no cover - Duplicate method (typo), kept for backward compatibility
self,
info_hash: bytes,
from_checkpoint_format: CheckpointFormat,
to_checkpoint_format: CheckpointFormat,
) -> Path:
"""Convert checkpoint between checkpoint_formats.
Args:
info_hash: Torrent info hash
from_checkpoint_format: Source checkpoint_format
to_checkpoint_format: Target checkpoint_format
Returns:
Path to converted checkpoint file
"""
# Load from source checkpoint_format
checkpoint = await self.load_checkpoint(info_hash, from_checkpoint_format)
if checkpoint is None:
msg = f"No checkpoint found for {info_hash.hex()}"
raise CheckpointNotFoundError(msg)
# Save in target checkpoint_format
return await self.save_checkpoint(checkpoint, to_checkpoint_format)
|
Convert checkpoint from one format to another.
Parameters:
| Name |
Type |
Description |
Default |
info_hash
|
bytes
|
Info hash of the checkpoint
|
required
|
from_format
|
CheckpointFormat
|
|
required
|
to_format
|
CheckpointFormat
|
|
required
|
Returns:
| Type |
Description |
Path
|
Path to the converted checkpoint file
|
Raises:
| Type |
Description |
CheckpointNotFoundError
|
If source checkpoint doesn't exist
|
CheckpointError
|
|
Source code in ccbt/storage/checkpoint.py
| async def convert_checkpoint_format(
self,
info_hash: bytes,
from_format: CheckpointFormat,
to_format: CheckpointFormat,
) -> Path:
"""Convert checkpoint from one format to another.
Args:
info_hash: Info hash of the checkpoint
from_format: Source format
to_format: Target format
Returns:
Path to the converted checkpoint file
Raises:
CheckpointNotFoundError: If source checkpoint doesn't exist
CheckpointError: If conversion fails
"""
# Load checkpoint from source format
checkpoint = await self.load_checkpoint(info_hash, from_format)
if checkpoint is None:
msg = f"Checkpoint not found for info hash {info_hash.hex()}"
raise CheckpointNotFoundError(msg)
# Save checkpoint in target format
return await self.save_checkpoint(checkpoint, to_format)
|
delete_checkpoint(info_hash: bytes) -> bool
async
Delete checkpoint files for given info hash.
Parameters:
| Name |
Type |
Description |
Default |
info_hash
|
bytes
|
|
required
|
Returns:
| Type |
Description |
bool
|
True if any files were deleted, False otherwise
|
Source code in ccbt/storage/checkpoint.py
| async def delete_checkpoint(self, info_hash: bytes) -> bool:
"""Delete checkpoint files for given info hash.
Args:
info_hash: Torrent info hash
Returns:
True if any files were deleted, False otherwise
"""
deleted = False
# Delete JSON checkpoint
json_path = self._get_checkpoint_path(info_hash, CheckpointFormat.JSON)
if json_path.exists():
json_path.unlink()
deleted = True
self.logger.debug("Deleted JSON checkpoint: %s", json_path)
# Delete binary checkpoint
bin_path = self._get_checkpoint_path(info_hash, CheckpointFormat.BINARY)
if bin_path.exists():
bin_path.unlink()
deleted = True
self.logger.debug("Deleted binary checkpoint: %s", bin_path)
return deleted
|
export_checkpoint(info_hash: bytes, fmt: str = 'json') -> bytes
async
Export checkpoint in the desired checkpoint_format and return bytes.
Source code in ccbt/storage/checkpoint.py
| async def export_checkpoint(self, info_hash: bytes, fmt: str = "json") -> bytes:
"""Export checkpoint in the desired checkpoint_format and return bytes."""
cp = await self.load_checkpoint(info_hash)
if cp is None:
msg = f"No checkpoint for {info_hash.hex()}"
raise CheckpointNotFoundError(msg)
fmt = (fmt or "json").lower()
if fmt == "json":
cp_dict = cp.model_dump()
cp_dict["info_hash"] = cp.info_hash.hex()
return json.dumps(cp_dict, indent=2).encode("utf-8")
if fmt == "binary":
# Save to temp path using binary writer and read back
path = await self._save_binary_checkpoint(cp)
return path.read_bytes()
msg = f"Unsupported export checkpoint_format: {fmt}"
raise CheckpointError(msg)
|
get_checkpoint_stats() -> dict[str, Any]
Get checkpoint directory statistics.
Source code in ccbt/storage/checkpoint.py
| def get_checkpoint_stats(self) -> dict[str, Any]:
"""Get checkpoint directory statistics."""
if (
not self.checkpoint_dir.exists()
): # pragma: no cover - Directory created in __init__, defensive check
return {
"total_files": 0,
"total_size": 0,
"json_files": 0,
"binary_files": 0,
"oldest_checkpoint": None,
"newest_checkpoint": None,
}
files = list(self.checkpoint_dir.glob("*.checkpoint.*"))
total_size = sum(f.stat().st_size for f in files)
json_files = len([f for f in files if f.suffix == ".json"])
binary_files = len([f for f in files if f.suffix in [".bin", ".gz"]])
timestamps = [f.stat().st_mtime for f in files]
oldest_checkpoint = min(timestamps) if timestamps else None
newest_checkpoint = max(timestamps) if timestamps else None
return {
"total_files": len(files),
"total_size": total_size,
"json_files": json_files,
"binary_files": binary_files,
"oldest_checkpoint": oldest_checkpoint,
"newest_checkpoint": newest_checkpoint,
}
|
list_checkpoints() -> list[CheckpointFileInfo]
async
List all available checkpoints.
Returns:
| Type |
Description |
list[CheckpointFileInfo]
|
List of checkpoint file incheckpoint_formation
|
Source code in ccbt/storage/checkpoint.py
| async def list_checkpoints(self) -> list[CheckpointFileInfo]:
"""List all available checkpoints.
Returns:
List of checkpoint file incheckpoint_formation
"""
checkpoints = []
if (
not self.checkpoint_dir.exists()
): # pragma: no cover - Directory created in __init__, defensive check
return checkpoints
for file_path in self.checkpoint_dir.glob("*.checkpoint.*"):
try:
# Extract info hash from filename
filename = file_path.stem
if filename.endswith(".checkpoint"):
info_hash_hex = filename.replace(".checkpoint", "")
info_hash = bytes.fromhex(info_hash_hex)
else: # pragma: no cover - Filename pattern matching, tested via invalid files
continue
# Determine checkpoint_format
if file_path.suffix == ".json":
checkpoint_format_type = CheckpointFormat.JSON
elif file_path.suffix in [".bin", ".gz"]:
checkpoint_format_type = CheckpointFormat.BINARY
else:
continue
stat = file_path.stat()
checkpoints.append(
CheckpointFileInfo(
path=file_path,
info_hash=info_hash,
created_at=stat.st_ctime,
updated_at=stat.st_mtime,
size=stat.st_size,
checkpoint_format=checkpoint_format_type,
),
)
except Exception as e:
self.logger.warning(
"Failed to process checkpoint file %s: %s",
file_path,
e,
)
continue
return sorted(checkpoints, key=lambda x: x.updated_at, reverse=True)
|
load_checkpoint(info_hash: bytes, checkpoint_format: CheckpointFormat | None = None) -> TorrentCheckpoint | None
async
Load checkpoint from disk.
Parameters:
| Name |
Type |
Description |
Default |
info_hash
|
bytes
|
|
required
|
checkpoint_format
|
CheckpointFormat | None
|
Format to load (tries both if None)
|
None
|
Returns:
| Type |
Description |
TorrentCheckpoint | None
|
Loaded checkpoint or None if not found
|
Raises:
| Type |
Description |
CheckpointError
|
|
Source code in ccbt/storage/checkpoint.py
| async def load_checkpoint(
self,
info_hash: bytes,
checkpoint_format: CheckpointFormat | None = None,
) -> TorrentCheckpoint | None:
"""Load checkpoint from disk.
Args:
info_hash: Torrent info hash
checkpoint_format: Format to load (tries both if None)
Returns:
Loaded checkpoint or None if not found
Raises:
CheckpointError: If loading fails
"""
if not self.config.checkpoint_enabled:
return None
checkpoint_format = checkpoint_format or self.config.checkpoint_format
try:
if checkpoint_format == CheckpointFormat.JSON:
return await self._load_json_checkpoint(info_hash)
if checkpoint_format == CheckpointFormat.BINARY:
return await self._load_binary_checkpoint(info_hash)
if checkpoint_format == CheckpointFormat.BOTH:
# Try JSON first, then binary
checkpoint = await self._load_json_checkpoint(info_hash)
if checkpoint is None:
checkpoint = await self._load_binary_checkpoint(info_hash)
return checkpoint
msg = f"Invalid checkpoint checkpoint_format: {checkpoint_format}"
raise ValueError(msg)
except CheckpointNotFoundError:
return None
except Exception as e:
self.logger.exception("Failed to load checkpoint")
msg = f"Failed to load checkpoint: {e}"
raise CheckpointError(msg) from e
|
restore_checkpoint(backup_file: Path, *, info_hash: bytes | None = None) -> TorrentCheckpoint
async
Restore a checkpoint from a backup file. Returns the restored checkpoint model.
Source code in ccbt/storage/checkpoint.py
| async def restore_checkpoint(
self,
backup_file: Path,
*,
info_hash: bytes | None = None,
) -> TorrentCheckpoint:
"""Restore a checkpoint from a backup file. Returns the restored checkpoint model."""
data = backup_file.read_bytes()
# Try decrypt if a .key file exists
key_file = backup_file.with_suffix(backup_file.suffix + ".key")
if key_file.exists():
try:
from cryptography.fernet import Fernet
key = key_file.read_bytes()
f = Fernet(key)
data = f.decrypt(data)
except Exception as e:
msg = f"Failed to decrypt backup: {e}"
raise CheckpointError(msg) from e
# Try decompress
with contextlib.suppress(OSError):
data = gzip.decompress(data)
try:
cp_dict = json.loads(data.decode("utf-8"))
except Exception as e:
msg = f"Invalid backup content: {e}"
raise CheckpointError(msg) from e
# Convert back types
cp_dict["info_hash"] = bytes.fromhex(cp_dict["info_hash"])
if info_hash and cp_dict["info_hash"] != info_hash:
msg = "Backup info hash does not match provided info hash"
raise CheckpointError(msg)
if "piece_states" in cp_dict:
cp_dict["piece_states"] = {
int(k): PieceState(v) for k, v in cp_dict["piece_states"].items()
}
cp = TorrentCheckpoint(**cp_dict)
# Clear checkpoint state to force save (in case file was deleted)
# This ensures restore always writes the file even if deduplication would skip it
self._last_checkpoint_hash = None
self._last_checkpoint = None
# Save to disk using configured checkpoint_format(s)
await self.save_checkpoint(cp, self.config.checkpoint_format)
return cp
|
save_checkpoint(checkpoint: TorrentCheckpoint, checkpoint_format: CheckpointFormat | None = None) -> Path
async
Save checkpoint to disk.
Parameters:
| Name |
Type |
Description |
Default |
checkpoint
|
TorrentCheckpoint
|
|
required
|
checkpoint_format
|
CheckpointFormat | None
|
Format to save in (uses config default if None)
|
None
|
Returns:
| Type |
Description |
Path
|
Path to saved checkpoint file
|
Raises:
| Type |
Description |
CheckpointError
|
|
Source code in ccbt/storage/checkpoint.py
| async def save_checkpoint(
self,
checkpoint: TorrentCheckpoint,
checkpoint_format: CheckpointFormat | None = None,
) -> Path:
"""Save checkpoint to disk.
Args:
checkpoint: Checkpoint data to save
checkpoint_format: Format to save in (uses config default if None)
Returns:
Path to saved checkpoint file
Raises:
CheckpointError: If saving fails
"""
if not self.config.checkpoint_enabled: # pragma: no cover - Checkpoint disabled path, tested but not all branches covered
msg = "Checkpointing is disabled"
raise CheckpointError(msg)
# Check for deduplication
if self.config.checkpoint_deduplication:
current_hash = self._calculate_checkpoint_hash(checkpoint)
if self._last_checkpoint_hash == current_hash:
self.logger.debug("Checkpoint unchanged, skipping save")
# Return existing path
checkpoint_format = checkpoint_format or self.config.checkpoint_format
return self._get_checkpoint_path(
checkpoint.info_hash, checkpoint_format
)
self._last_checkpoint_hash = current_hash
checkpoint_format = checkpoint_format or self.config.checkpoint_format
try:
if checkpoint_format == CheckpointFormat.JSON:
path = await self._save_json_checkpoint(checkpoint)
elif checkpoint_format == CheckpointFormat.BINARY:
path = await self._save_binary_checkpoint(checkpoint)
elif (
checkpoint_format == CheckpointFormat.BOTH
): # pragma: no cover - Both format path, tested but not all branches covered
# Save both checkpoint_formats
json_path = await self._save_json_checkpoint(checkpoint)
bin_path = await self._save_binary_checkpoint(checkpoint)
self.logger.debug(
"Saved checkpoint in both checkpoint_formats: %s, %s",
json_path,
bin_path,
)
path = json_path # Return JSON path as primary
else:
msg = f"Invalid checkpoint checkpoint_format: {checkpoint_format}"
raise ValueError(msg)
# Store last checkpoint for incremental saves
self._last_checkpoint = checkpoint
return path
except Exception as e:
self.logger.exception("Failed to save checkpoint")
msg = f"Failed to save checkpoint: {e}"
raise CheckpointError(msg) from e
|
verify_checkpoint(info_hash: bytes) -> bool
async
Verify that a checkpoint exists and is structurally valid.
Source code in ccbt/storage/checkpoint.py
| async def verify_checkpoint(self, info_hash: bytes) -> bool:
"""Verify that a checkpoint exists and is structurally valid."""
cp = await self.load_checkpoint(info_hash)
if cp is None:
return False
# basic invariants
return not len(cp.verified_pieces) > cp.total_pieces
|
Features:
- Checkpoint save/load
- Checkpoint validation
- Checkpoint cleanup
- Multiple format support (JSON, binary)
Configuration: ccbt.toml:88-96
Checkpoint Model: ccbt/models.py:TorrentCheckpoint
Buffers
Storage buffers for I/O operations.
Implementation: ccbt/storage/buffers.py
Features:
- Ring buffers
- Write buffers
- Read buffers
Monitoring
MetricsCollector
Advanced metrics collection system.
Advanced metrics collector.
Initialize metrics collector.
Source code in ccbt/monitoring/metrics_collector.py
| def __init__(self):
"""Initialize metrics collector."""
self.metrics: dict[str, Metric] = {}
self.alert_rules: dict[str, AlertRule] = {}
self.collectors: dict[str, Callable] = {}
# System metrics
self.system_metrics: MetricsCollector._SystemMetrics = {
"cpu_usage": 0.0,
"memory_usage": 0.0,
"disk_usage": 0.0,
"network_io": {"bytes_sent": 0, "bytes_recv": 0},
"process_count": 0,
}
# Performance tracking
self.performance_data = {
"peer_connections": 0,
"download_speed": 0.0,
"upload_speed": 0.0,
"pieces_completed": 0,
"pieces_failed": 0,
"tracker_requests": 0,
"tracker_responses": 0,
# DHT metrics
"dht_nodes_discovered": 0,
"dht_queries_sent": 0,
"dht_queries_received": 0,
"dht_response_rate": 0.0,
# Queue metrics
"queue_length": 0,
"queue_wait_time": 0.0,
"priority_distribution": {},
# Disk I/O metrics
"disk_write_throughput": 0.0,
"disk_read_throughput": 0.0,
"disk_queue_depth": 0,
# Tracker metrics
"tracker_announce_success_rate": 0.0,
"tracker_scrape_success_rate": 0.0,
"tracker_average_response_time": 0.0,
"tracker_error_count": 0,
# Connection health metrics
"connection_success_rate": 0.0,
"connection_timeout_count": 0,
"connection_refused_count": 0,
"connection_winerror_121_count": 0,
"connection_other_errors_count": 0,
"total_connection_attempts": 0,
"active_peer_connections": 0,
"queued_peers_count": 0,
# NAT mapping metrics
"nat_active_protocol": "",
"nat_external_ip": "",
"nat_mappings_count": 0,
"nat_tcp_mapped": False,
"nat_udp_mapped": False,
"nat_dht_mapped": False,
"nat_tracker_udp_mapped": False,
}
# Session reference for accessing DHT, queue, disk I/O, and tracker services
self._session: Any | None = None
# Collection interval
self.collection_interval = 5.0 # seconds
self.collection_task: asyncio.Task | None = None
self.running = False
# HTTP server for Prometheus endpoint (if enabled)
self._http_server: Any | None = None
self._http_server_thread: Any | None = None
# Statistics
self.stats = {
"metrics_collected": 0,
"alerts_triggered": 0,
"collection_errors": 0,
}
|
add_alert_rule(name: str, metric_name: str, condition: str, severity: str = 'warning', description: str = '', cooldown_seconds: int = 300) -> None
Add an alert rule.
Source code in ccbt/monitoring/metrics_collector.py
| def add_alert_rule(
self,
name: str,
metric_name: str,
condition: str,
severity: str = "warning",
description: str = "",
cooldown_seconds: int = 300,
) -> None:
"""Add an alert rule."""
self.alert_rules[name] = AlertRule(
name=name,
metric_name=metric_name,
condition=condition,
severity=severity,
description=description,
cooldown_seconds=cooldown_seconds,
)
|
cleanup_old_metrics(max_age_seconds: int = 3600) -> None
Clean up old metric values.
Source code in ccbt/monitoring/metrics_collector.py
| def cleanup_old_metrics(self, max_age_seconds: int = 3600) -> None:
"""Clean up old metric values."""
current_time = time.time()
cutoff_time = current_time - max_age_seconds
for metric in self.metrics.values():
# Remove old values
while metric.values and metric.values[0].timestamp < cutoff_time:
metric.values.popleft()
|
export_metrics(format_type: str = 'json') -> str
Export metrics in specified format.
Source code in ccbt/monitoring/metrics_collector.py
| def export_metrics(self, format_type: str = "json") -> str:
"""Export metrics in specified format."""
if format_type == "json": # pragma: no cover - Tested via get_all_metrics
return json.dumps(
self.get_all_metrics(), indent=2
) # pragma: no cover - Tested via get_all_metrics
if format_type == "prometheus":
return self._export_prometheus_format()
msg = f"Unsupported format: {format_type}" # pragma: no cover - Error path for unsupported format
raise ValueError(msg) # pragma: no cover - Error path for unsupported format
|
get_all_metrics() -> dict[str, Any]
Get all metrics with their values.
Source code in ccbt/monitoring/metrics_collector.py
| def get_all_metrics(self) -> dict[str, Any]:
"""Get all metrics with their values."""
result = {}
for name, metric in self.metrics.items():
result[name] = {
"type": metric.metric_type.value,
"description": metric.description,
"labels": [
{"name": label.name, "value": label.value}
for label in metric.labels
],
"aggregation": metric.aggregation.value,
"retention_seconds": metric.retention_seconds,
"value_count": len(metric.values),
"current_value": self.get_metric_value(name),
"aggregated_value": self.get_metric_value(name, metric.aggregation),
}
return result
|
get_metric(name: str) -> Metric | None
Get a metric by name.
Source code in ccbt/monitoring/metrics_collector.py
| def get_metric(self, name: str) -> Metric | None:
"""Get a metric by name."""
return self.metrics.get(name) # pragma: no cover
|
get_metric_value(name: str, aggregation: AggregationType | None = None) -> int | float | str | None
Get aggregated metric value.
Source code in ccbt/monitoring/metrics_collector.py
| def get_metric_value(
self,
name: str,
aggregation: AggregationType | None = None,
) -> int | float | str | None:
"""Get aggregated metric value."""
if name not in self.metrics: # pragma: no cover
return None
metric = self.metrics[name]
if not metric.values:
return None
agg_type = aggregation or metric.aggregation
if agg_type == AggregationType.SUM:
return sum(
v.value for v in metric.values if isinstance(v.value, (int, float))
)
if agg_type == AggregationType.AVG:
values = [
v.value for v in metric.values if isinstance(v.value, (int, float))
]
return sum(values) / len(values) if values else 0
if agg_type == AggregationType.MIN:
values = [
v.value for v in metric.values if isinstance(v.value, (int, float))
]
return min(values) if values else 0
if agg_type == AggregationType.MAX:
values = [
v.value for v in metric.values if isinstance(v.value, (int, float))
]
return max(values) if values else 0
if agg_type == AggregationType.COUNT:
return len(metric.values)
# Return latest value
return metric.values[-1].value if metric.values else None # pragma: no cover
|
get_metrics_statistics() -> dict[str, Any]
Get metrics collection statistics.
Source code in ccbt/monitoring/metrics_collector.py
| def get_metrics_statistics(self) -> dict[str, Any]:
"""Get metrics collection statistics."""
return { # pragma: no cover
"metrics_collected": self.stats["metrics_collected"],
"alerts_triggered": self.stats["alerts_triggered"],
"collection_errors": self.stats["collection_errors"],
"registered_metrics": len(self.metrics),
"alert_rules": len(self.alert_rules),
"collection_interval": self.collection_interval,
"running": self.running,
}
|
Get performance metrics.
Source code in ccbt/monitoring/metrics_collector.py
| def get_performance_metrics(self) -> dict[str, Any]:
"""Get performance metrics."""
return self.performance_data.copy() # pragma: no cover
|
get_system_metrics() -> dict[str, Any]
Get system metrics.
Source code in ccbt/monitoring/metrics_collector.py
| def get_system_metrics(self) -> dict[str, Any]:
"""Get system metrics."""
return { # pragma: no cover
"cpu_usage": self.system_metrics["cpu_usage"],
"memory_usage": self.system_metrics["memory_usage"],
"disk_usage": self.system_metrics["disk_usage"],
"network_io": self.system_metrics["network_io"],
"process_count": self.system_metrics["process_count"],
}
|
increment_counter(name: str, value: int = 1, labels: list[MetricLabel] | None = None) -> None
Increment a counter metric.
Source code in ccbt/monitoring/metrics_collector.py
| def increment_counter(
self,
name: str,
value: int = 1,
labels: list[MetricLabel] | None = None,
) -> None:
"""Increment a counter metric."""
if name not in self.metrics: # pragma: no cover
self.register_metric(
name, MetricType.COUNTER, f"Counter: {name}"
) # pragma: no cover
metric = self.metrics[name] # pragma: no cover
if metric.values: # pragma: no cover
current_value = metric.values[-1].value # pragma: no cover
new_value = current_value + value # pragma: no cover
else: # pragma: no cover
new_value = value # pragma: no cover
self.record_metric(name, new_value, labels) # pragma: no cover
|
record_histogram(name: str, value: float, labels: list[MetricLabel] | None = None) -> None
Record a histogram value.
Source code in ccbt/monitoring/metrics_collector.py
| def record_histogram(
self,
name: str,
value: float,
labels: list[MetricLabel] | None = None,
) -> None:
"""Record a histogram value."""
if name not in self.metrics:
self.register_metric(name, MetricType.HISTOGRAM, f"Histogram: {name}")
self.record_metric(name, value, labels)
|
record_metric(name: str, value: float | str, labels: list[MetricLabel] | None = None) -> None
Record a metric value.
Source code in ccbt/monitoring/metrics_collector.py
| def record_metric(
self,
name: str,
value: float | str,
labels: list[MetricLabel] | None = None,
) -> None:
"""Record a metric value."""
if name not in self.metrics:
# Auto-register metric if it doesn't exist
self.register_metric(
name,
MetricType.GAUGE,
f"Auto-registered metric: {name}",
)
metric = self.metrics[name]
metric_value = MetricValue(
value=value,
timestamp=time.time(),
labels=labels or [],
)
metric.values.append(metric_value)
# Check alert rules
self._check_alert_rules(name, value)
# Also forward to global AlertManager (if available) so shared rules can trigger
try:
# Lazy import to avoid circular imports during module load
from ccbt.monitoring import (
get_alert_manager,
)
am = get_alert_manager()
# Only attempt numeric evaluation for shared rules
v_any: float | str = value
if isinstance(value, str):
# simple numeric parse; ignore parse errors
with contextlib.suppress(Exception): # pragma: no cover
if value.replace(".", "", 1).isdigit(): # pragma: no cover
v_any = float(value) # pragma: no cover
# Schedule async processing if running in an event loop
import asyncio as _asyncio
with contextlib.suppress(RuntimeError): # pragma: no cover
loop = _asyncio.get_event_loop() # pragma: no cover
if loop.is_running(): # pragma: no cover
task = loop.create_task(am.process_alert(name, v_any)) # type: ignore[arg-type] # pragma: no cover
task.add_done_callback(
lambda _t: None
) # Discard task reference # pragma: no cover
except Exception: # pragma: no cover
# If alert manager not available, skip silently
logger.debug( # pragma: no cover
"Alert manager not available for metric processing", exc_info=True
)
# Update statistics
self.stats["metrics_collected"] += 1
|
register_custom_collector(name: str, collector: Callable) -> None
Register a custom metrics collector.
Source code in ccbt/monitoring/metrics_collector.py
| def register_custom_collector(self, name: str, collector: Callable) -> None:
"""Register a custom metrics collector."""
self.collectors[name] = collector
|
register_metric(name: str, metric_type: MetricType, description: str, labels: list[MetricLabel] | None = None, aggregation: AggregationType = AggregationType.SUM, retention_seconds: int = 3600) -> None
Register a new metric.
Source code in ccbt/monitoring/metrics_collector.py
| def register_metric(
self,
name: str,
metric_type: MetricType,
description: str,
labels: list[MetricLabel] | None = None,
aggregation: AggregationType = AggregationType.SUM,
retention_seconds: int = 3600,
) -> None:
"""Register a new metric."""
self.metrics[name] = Metric(
name=name,
metric_type=metric_type,
description=description,
labels=labels or [],
aggregation=aggregation,
retention_seconds=retention_seconds,
)
|
set_gauge(name: str, value: float, labels: list[MetricLabel] | None = None) -> None
Set a gauge metric value.
Source code in ccbt/monitoring/metrics_collector.py
| def set_gauge(
self,
name: str,
value: float,
labels: list[MetricLabel] | None = None,
) -> None:
"""Set a gauge metric value."""
if name not in self.metrics:
self.register_metric(name, MetricType.GAUGE, f"Gauge: {name}")
self.record_metric(name, value, labels)
|
set_session(session: Any) -> None
Set session reference for accessing DHT, queue, disk I/O, and tracker services.
Parameters:
| Name |
Type |
Description |
Default |
session
|
Any
|
AsyncSessionManager instance
|
required
|
Source code in ccbt/monitoring/metrics_collector.py
| def set_session(self, session: Any) -> None:
"""Set session reference for accessing DHT, queue, disk I/O, and tracker services.
Args:
session: AsyncSessionManager instance
"""
self._session = session
|
start() -> None
async
Start metrics collection.
Source code in ccbt/monitoring/metrics_collector.py
| async def start(self) -> None:
"""Start metrics collection."""
if self.running: # pragma: no cover
return
self.running = True # pragma: no cover
self.collection_task = asyncio.create_task(
self._collection_loop()
) # pragma: no cover
# Start Prometheus HTTP server if enabled and available
await self._start_prometheus_server()
# Emit start event
await emit_event( # pragma: no cover
Event(
event_type=EventType.MONITORING_STARTED.value,
data={
"collection_interval": self.collection_interval,
"timestamp": time.time(),
},
),
)
|
stop() -> None
async
Stop metrics collection.
Source code in ccbt/monitoring/metrics_collector.py
| async def stop(self) -> None:
"""Stop metrics collection."""
if not self.running: # pragma: no cover
return
self.running = False # pragma: no cover
if self.collection_task: # pragma: no cover
self.collection_task.cancel()
with contextlib.suppress(asyncio.CancelledError): # pragma: no cover
await self.collection_task
# Stop Prometheus HTTP server if running
await self._stop_prometheus_server()
# Emit stop event
await emit_event( # pragma: no cover
Event(
event_type=EventType.MONITORING_STOPPED.value,
data={
"timestamp": time.time(),
},
),
)
|
unregister_custom_collector(name: str) -> None
Unregister a custom metrics collector.
Source code in ccbt/monitoring/metrics_collector.py
| def unregister_custom_collector(self, name: str) -> None:
"""Unregister a custom metrics collector."""
if name in self.collectors:
del self.collectors[name]
|
Features:
- System metrics collection: ccbt/monitoring/metrics_collector.py:394
- Performance metrics tracking: ccbt/monitoring/metrics_collector.py:404
- Custom metrics registration: ccbt/monitoring/metrics_collector.py:190
- Prometheus metrics export: ccbt/utils/metrics.py:134
See the MetricsCollector section below for detailed usage.
AlertManager
Rule-based alert system.
Alert management system.
Initialize alert manager.
Source code in ccbt/monitoring/alert_manager.py
| def __init__(self):
"""Initialize alert manager."""
self.alert_rules: dict[str, AlertRule] = {}
self.active_alerts: dict[str, Alert] = {}
self.alert_history: deque = deque(maxlen=10000)
self.notification_configs: dict[NotificationChannel, NotificationConfig] = {}
self.notification_handlers: dict[NotificationChannel, Callable] = {}
# Alert suppression
self.suppression_rules: dict[str, dict[str, Any]] = {}
self.suppressed_alerts: dict[str, float] = {}
# Statistics
self.stats = {
"alerts_triggered": 0,
"alerts_resolved": 0,
"notifications_sent": 0,
"notification_failures": 0,
"suppressed_alerts": 0,
}
# Initialize default notification handlers
self._initialize_notification_handlers()
|
add_alert_rule(rule: AlertRule) -> None
Add an alert rule.
Source code in ccbt/monitoring/alert_manager.py
| def add_alert_rule(self, rule: AlertRule) -> None:
"""Add an alert rule."""
self.alert_rules[rule.name] = rule
|
add_suppression_rule(name: str, rule: dict[str, Any]) -> None
Add a suppression rule.
Source code in ccbt/monitoring/alert_manager.py
| def add_suppression_rule(self, name: str, rule: dict[str, Any]) -> None:
"""Add a suppression rule."""
self.suppression_rules[name] = rule
|
cleanup_old_alerts(max_age_seconds: int = 86400) -> None
Clean up old alerts from history.
Source code in ccbt/monitoring/alert_manager.py
| def cleanup_old_alerts(self, max_age_seconds: int = 86400) -> None:
"""Clean up old alerts from history."""
current_time = time.time()
cutoff_time = current_time - max_age_seconds
# Remove old alerts from history
while self.alert_history and self.alert_history[0].timestamp < cutoff_time:
self.alert_history.popleft()
# Clean up suppressed alerts
to_remove = []
for alert_id, suppress_time in self.suppressed_alerts.items():
if suppress_time < cutoff_time:
to_remove.append(alert_id)
for alert_id in to_remove:
del self.suppressed_alerts[alert_id]
|
Configure notification channel.
Source code in ccbt/monitoring/alert_manager.py
| def configure_notification(
self,
channel: NotificationChannel,
config: NotificationConfig,
) -> None:
"""Configure notification channel."""
self.notification_configs[channel] = config
|
export_rules() -> list[dict[str, Any]]
Export alert rules as a serializable list of dicts.
Source code in ccbt/monitoring/alert_manager.py
| def export_rules(self) -> list[dict[str, Any]]:
"""Export alert rules as a serializable list of dicts."""
return [
{
"name": rule.name,
"metric_name": rule.metric_name,
"condition": rule.condition,
"severity": rule.severity.value,
"description": rule.description,
"enabled": rule.enabled,
"cooldown_seconds": rule.cooldown_seconds,
"escalation_seconds": rule.escalation_seconds,
"notification_channels": [c.value for c in rule.notification_channels],
"suppression_rules": list(rule.suppression_rules),
# omit dynamic fields last_triggered/trigger_count on export
}
for rule in self.alert_rules.values()
]
|
get_active_alerts() -> dict[str, Alert]
Get all active alerts.
Source code in ccbt/monitoring/alert_manager.py
| def get_active_alerts(self) -> dict[str, Alert]:
"""Get all active alerts."""
return self.active_alerts.copy()
|
get_alert_history(limit: int = 100) -> list[Alert]
Get alert history.
Source code in ccbt/monitoring/alert_manager.py
| def get_alert_history(self, limit: int = 100) -> list[Alert]:
"""Get alert history."""
return list(self.alert_history)[-limit:]
|
get_alert_rules() -> dict[str, AlertRule]
Get all alert rules.
Source code in ccbt/monitoring/alert_manager.py
| def get_alert_rules(self) -> dict[str, AlertRule]:
"""Get all alert rules."""
return self.alert_rules.copy()
|
get_alert_statistics() -> dict[str, Any]
Get alert statistics.
Source code in ccbt/monitoring/alert_manager.py
| def get_alert_statistics(self) -> dict[str, Any]:
"""Get alert statistics."""
return {
"alerts_triggered": self.stats["alerts_triggered"],
"alerts_resolved": self.stats["alerts_resolved"],
"notifications_sent": self.stats["notifications_sent"],
"notification_failures": self.stats["notification_failures"],
"suppressed_alerts": self.stats["suppressed_alerts"],
"active_alerts": len(self.active_alerts),
"alert_rules": len(self.alert_rules),
"suppression_rules": len(self.suppression_rules),
}
|
get_suppression_rules() -> dict[str, dict[str, Any]]
Get all suppression rules.
Source code in ccbt/monitoring/alert_manager.py
| def get_suppression_rules(self) -> dict[str, dict[str, Any]]:
"""Get all suppression rules."""
return self.suppression_rules.copy()
|
import_rules(rules: list[dict[str, Any]]) -> int
Import alert rules from list of dicts; returns number loaded.
Source code in ccbt/monitoring/alert_manager.py
| def import_rules(self, rules: list[dict[str, Any]]) -> int:
"""Import alert rules from list of dicts; returns number loaded."""
loaded = 0
for data in rules:
try:
sev = AlertSeverity(str(data.get("severity", "warning")))
except Exception:
sev = AlertSeverity.WARNING
channels = []
for c in data.get("notification_channels", []):
try:
channels.append(NotificationChannel(str(c)))
except Exception as e:
logger.debug("Failed to parse alert channel: %s", e)
continue
rule = AlertRule(
name=str(data.get("name")),
metric_name=str(data.get("metric_name")),
condition=str(data.get("condition")),
severity=sev,
description=str(data.get("description", "")),
enabled=bool(data.get("enabled", True)),
cooldown_seconds=int(data.get("cooldown_seconds", 300)),
escalation_seconds=int(data.get("escalation_seconds", 0)),
notification_channels=channels,
suppression_rules=list(data.get("suppression_rules", [])),
)
self.alert_rules[rule.name] = rule
loaded += 1
return loaded
|
load_rules_from_file(path: Path) -> int
Load alert rules from JSON file; returns number loaded.
Source code in ccbt/monitoring/alert_manager.py
| def load_rules_from_file(self, path: Path) -> int:
"""Load alert rules from JSON file; returns number loaded."""
try:
if not path.exists():
return 0
payload = json.loads(path.read_text(encoding="utf-8"))
rules = payload.get("rules", [])
return self.import_rules(rules)
except Exception:
return 0
|
process_alert(metric_name: str, value: Any, timestamp: float | None = None) -> None
async
Process an alert for a metric.
Source code in ccbt/monitoring/alert_manager.py
| async def process_alert(
self,
metric_name: str,
value: Any,
timestamp: float | None = None,
) -> None:
"""Process an alert for a metric."""
if timestamp is None:
timestamp = time.time()
# Check all alert rules for this metric
for rule in self.alert_rules.values():
if rule.metric_name != metric_name or not rule.enabled:
continue
# Check cooldown
if timestamp - rule.last_triggered < rule.cooldown_seconds:
continue
# Evaluate condition
if self._evaluate_condition(rule.condition, value):
await self._trigger_alert(rule, value, timestamp)
|
register_notification_handler(channel: NotificationChannel, handler: Callable) -> None
Register custom notification handler.
Source code in ccbt/monitoring/alert_manager.py
| def register_notification_handler(
self,
channel: NotificationChannel,
handler: Callable,
) -> None:
"""Register custom notification handler."""
self.notification_handlers[channel] = handler
|
remove_alert_rule(rule_name: str) -> None
Remove an alert rule.
Source code in ccbt/monitoring/alert_manager.py
| def remove_alert_rule(self, rule_name: str) -> None:
"""Remove an alert rule."""
if rule_name in self.alert_rules:
del self.alert_rules[rule_name]
|
remove_suppression_rule(name: str) -> None
Remove a suppression rule.
Source code in ccbt/monitoring/alert_manager.py
| def remove_suppression_rule(self, name: str) -> None:
"""Remove a suppression rule."""
if name in self.suppression_rules:
del self.suppression_rules[name]
|
resolve_alert(alert_id: str, timestamp: float | None = None) -> bool
async
Resolve an alert.
Source code in ccbt/monitoring/alert_manager.py
| async def resolve_alert(
self,
alert_id: str,
timestamp: float | None = None,
) -> bool:
"""Resolve an alert."""
if timestamp is None:
timestamp = time.time()
if alert_id not in self.active_alerts:
return False
alert = self.active_alerts[alert_id]
alert.resolved = True
alert.resolved_timestamp = timestamp
# Move to history
self.alert_history.append(alert)
del self.active_alerts[alert_id]
# Update statistics
self.stats["alerts_resolved"] += 1
# Emit alert resolved event
await emit_event(
Event(
event_type=EventType.ALERT_RESOLVED.value,
data={
"alert_id": alert_id,
"rule_name": alert.rule_name,
"metric_name": alert.metric_name,
"duration": timestamp - alert.timestamp,
"timestamp": timestamp,
},
),
)
return True
|
resolve_alerts_for_metric(metric_name: str, timestamp: float | None = None) -> int
async
Resolve all alerts for a specific metric.
Source code in ccbt/monitoring/alert_manager.py
| async def resolve_alerts_for_metric(
self,
metric_name: str,
timestamp: float | None = None,
) -> int:
"""Resolve all alerts for a specific metric."""
if timestamp is None:
timestamp = time.time()
resolved_count = 0
alerts_to_resolve = [
alert_id
for alert_id, alert in self.active_alerts.items()
if alert.metric_name == metric_name
]
for alert_id in alerts_to_resolve:
if await self.resolve_alert(alert_id, timestamp):
resolved_count += 1
return resolved_count
|
save_rules_to_file(path: Path) -> None
Save alert rules to JSON file.
Source code in ccbt/monitoring/alert_manager.py
| def save_rules_to_file(self, path: Path) -> None:
"""Save alert rules to JSON file."""
try:
path.parent.mkdir(parents=True, exist_ok=True)
payload = {"rules": self.export_rules()}
path.write_text(json.dumps(payload, indent=2), encoding="utf-8")
except (
OSError,
ValueError,
TypeError,
) as e: # pragma: no cover - Defensive exception handling for file I/O errors that are difficult to reliably trigger in tests
logger.debug(
"Failed to save alert rules: %s", e
) # pragma: no cover - Error logging path
|
update_alert_rule(rule_name: str, updates: dict[str, Any]) -> None
Update an alert rule.
Source code in ccbt/monitoring/alert_manager.py
| def update_alert_rule(self, rule_name: str, updates: dict[str, Any]) -> None:
"""Update an alert rule."""
if rule_name in self.alert_rules:
rule = self.alert_rules[rule_name]
for key, value in updates.items():
if hasattr(rule, key):
setattr(rule, key, value)
|
Features:
- Alert rule engine: ccbt/monitoring/alert_manager.py:AlertRule
- Notification channels: ccbt/monitoring/alert_manager.py:NotificationChannel
- Alert escalation: [ccbt/monitoring/alert_manager.py]
- Alert suppression: [ccbt/monitoring/alert_manager.py]
Alert Severity: ccbt/monitoring/alert_manager.py:AlertSeverity
DashboardManager
Dashboard management system.
Implementation: ccbt/monitoring/dashboard.py:DashboardManager
Features:
- Dashboard creation: ccbt/monitoring/dashboard.py:156
- Grafana export: ccbt/monitoring/dashboard.py:366
- Widget system: ccbt/monitoring/dashboard.py:WidgetType
Dashboard types: ccbt/monitoring/dashboard.py:DashboardType
TracingManager
Distributed tracing for performance analysis.
Implementation: ccbt/monitoring/tracing.py
Features:
- Span management: ccbt/monitoring/tracing.py:Span
- Trace correlation: ccbt/monitoring/tracing.py:Trace
- Performance profiling
- OpenTelemetry integration
Span status: ccbt/monitoring/tracing.py:SpanStatus
Span kind: ccbt/monitoring/tracing.py:SpanKind
Security
SecurityManager
Security management system.
Implementation: ccbt/security/security_manager.py
Features:
- Security policy enforcement
- Threat detection
- Security event handling
Encryption
Protocol encryption support.
Implementation: ccbt/security/encryption.py
Configuration: ccbt.toml:174
PeerValidator
Validates peer connections and behavior.
Implementation: ccbt/security/peer_validator.py
Configuration: ccbt.toml:175
RateLimiter
Adaptive rate limiting for bandwidth management.
Implementation: ccbt/security/rate_limiter.py
Configuration: ccbt.toml:176
AnomalyDetector
Detects anomalous behavior patterns.
Implementation: ccbt/security/anomaly_detector.py
Features:
- Behavior pattern analysis
- Anomaly detection algorithms
- Threat scoring
Machine Learning
PeerSelector
ML-based peer selection.
Implementation: ccbt/ml/peer_selector.py
Configuration: ccbt.toml:181
PiecePredictor
ML-based piece prediction.
Implementation: ccbt/ml/piece_predictor.py
Configuration: ccbt.toml:182
AdaptiveLimiter
ML-based adaptive rate limiting.
Implementation: ccbt/ml/adaptive_limiter.py
Features:
- Adaptive bandwidth allocation
- Performance-based adjustment
- Learning from usage patterns
Extensions
ExtensionManager
Manages BitTorrent protocol extensions (BEP 10) with automatic negotiation and feature detection.
Implementation: ccbt/extensions/manager.py
Features:
- Extension negotiation: ccbt/extensions/protocol.py - BEP 10 extension handshake
- Extension registration: Register custom extensions
- Message routing: Route extension messages to handlers
- Feature detection: Detect peer capabilities
Supported Extensions:
- Fast Extension (BEP 6): ccbt/extensions/fast.py - Reject requests for pieces we don't have
- Peer Exchange (BEP 11): ccbt/extensions/pex.py - Exchange peer lists
- DHT Extension (BEP 5): ccbt/extensions/dht.py - DHT port announcement
- Compact Extension: ccbt/extensions/compact.py - Compact peer format
- WebSeed Extension: ccbt/extensions/webseed.py - HTTP seeding support
FastExtension
Fast extension (BEP 6) support.
Implementation: ccbt/extensions/fast.py
WebSeedExtension
Web seed extension support.
Implementation: ccbt/extensions/webseed.py
PEXExtension
Peer Exchange extension (BEP 11) support.
Implementation: ccbt/extensions/pex.py
DHTExtension
DHT extension (BEP 5) support.
Implementation: ccbt/extensions/dht.py
CompactExtension
Compact peer format extension support.
Implementation: ccbt/extensions/compact.py
Utilities
Events
Event system for asynchronous component communication.
Implementation: ccbt/utils/events.py
Event priority: ccbt/utils/events.py:EventPriority
Event types: ccbt/utils/events.py:EventType
Event model: ccbt/utils/events.py:Event
Functions:
- emit_event(): ccbt/utils/events.py - Emit event to subscribers
- subscribe_to_event(): ccbt/utils/events.py - Subscribe to event type
- unsubscribe_from_event(): ccbt/utils/events.py - Unsubscribe from event type
Event-driven architecture supports decoupled component communication across the entire codebase.
Exceptions
Exception hierarchy for error handling.
Implementation: ccbt/utils/exceptions.py
Exception types:
- CCBTException: Base exception class
- NetworkError: Network-related errors
- DiskError: Disk I/O errors
- ProtocolError: Protocol violations
- ValidationError: Data validation errors
- ConfigurationError: Configuration errors
- TorrentError: Torrent-related errors
LoggingConfig
Logging configuration and setup.
Implementation: ccbt/utils/logging_config.py
Configuration: ccbt.toml:156-160
Metrics Utils
Metrics utility functions.
Implementation: ccbt/utils/metrics.py
Prometheus integration: ccbt/utils/metrics.py:134
NetworkOptimizer
Network optimization utilities.
Implementation: ccbt/utils/network_optimizer.py
Features:
- Network parameter optimization
- Connection tuning
- Performance analysis
Resilience
Resilience and fault tolerance utilities.
Implementation: ccbt/utils/resilience.py
Features:
- Retry logic
- Circuit breaker patterns
- Error recovery
Configuration
ConfigManager
Configuration management with hot-reload.
Manages configuration loading, validation, and hot-reload.
Initialize configuration manager.
Parameters:
| Name |
Type |
Description |
Default |
config_file
|
str | Path | None
|
Path to TOML config file. If None, searches for ccbt.toml
|
None
|
Source code in ccbt/config/config.py
| def __init__(self, config_file: str | Path | None = None):
"""Initialize configuration manager.
Args:
config_file: Path to TOML config file. If None, searches for ccbt.toml
"""
# internal
self._hot_reload_task: asyncio.Task | None = None
self._encryption_key: bytes | None = None
self.config_file = self._find_config_file(config_file)
self.config = self._load_config()
self._setup_logging()
|
export(fmt: str = 'toml', encrypt_passwords: bool = True) -> str
Export current configuration as a string in the given format.
Parameters:
| Name |
Type |
Description |
Default |
fmt
|
str
|
one of "toml", "json", or "yaml"
|
'toml'
|
encrypt_passwords
|
bool
|
If True, encrypt proxy passwords before export
|
True
|
Source code in ccbt/config/config.py
| def export(self, fmt: str = "toml", encrypt_passwords: bool = True) -> str:
"""Export current configuration as a string in the given format.
Args:
fmt: one of "toml", "json", or "yaml"
encrypt_passwords: If True, encrypt proxy passwords before export
"""
data = self.config.model_dump(mode="json")
# Encrypt proxy password before export if enabled
if (
encrypt_passwords
and "proxy" in data
and data["proxy"].get("proxy_password")
):
password = data["proxy"]["proxy_password"]
if password and not self._is_encrypted(password):
try:
encrypted = self._encrypt_proxy_password(password)
data["proxy"]["proxy_password"] = encrypted
except Exception as e:
logging.warning("Failed to encrypt proxy password: %s", e)
# Continue with plaintext (not recommended)
fmt = (fmt or "toml").lower()
if fmt == "toml": # pragma: no cover
try: # pragma: no cover
return toml.dumps(data) # pragma: no cover
except Exception as e: # pragma: no cover
msg = f"Failed to export TOML: {e}" # pragma: no cover
raise ConfigurationError(msg) from e # pragma: no cover
if fmt == "json": # pragma: no cover
import json # pragma: no cover
return json.dumps(data, indent=2) # pragma: no cover
if fmt == "yaml": # pragma: no cover
try: # pragma: no cover
import yaml # pragma: no cover
except Exception as e: # pragma: no cover
msg = "PyYAML not installed; cannot export YAML" # pragma: no cover
raise ConfigurationError(msg) from e # pragma: no cover
return yaml.safe_dump(data, sort_keys=False) # pragma: no cover
msg = f"Unsupported export format: {fmt}" # pragma: no cover
raise ConfigurationError(msg) # pragma: no cover
|
export_schema(format_type: str = 'json') -> str
Export configuration schema in specified format.
Parameters:
| Name |
Type |
Description |
Default |
format_type
|
str
|
Output format ("json" or "yaml")
|
'json'
|
Returns:
| Type |
Description |
str
|
Schema as string in specified format
|
Source code in ccbt/config/config.py
| def export_schema(self, format_type: str = "json") -> str:
"""Export configuration schema in specified format.
Args:
format_type: Output format ("json" or "yaml")
Returns:
Schema as string in specified format
"""
from ccbt.config.config_schema import ConfigSchema
return ConfigSchema.export_schema(format_type)
|
Get metadata for a specific configuration option.
Parameters:
| Name |
Type |
Description |
Default |
key_path
|
str
|
Dot-separated path to the option
|
required
|
Returns:
| Type |
Description |
dict[str, Any] | None
|
Metadata for the option or None if not found
|
Source code in ccbt/config/config.py
| def get_option_metadata(self, key_path: str) -> dict[str, Any] | None:
"""Get metadata for a specific configuration option.
Args:
key_path: Dot-separated path to the option
Returns:
Metadata for the option or None if not found
"""
from ccbt.config.config_schema import ConfigDiscovery
return ConfigDiscovery.get_option_metadata(key_path)
|
get_schema() -> dict[str, Any]
Get configuration schema.
Returns:
| Type |
Description |
dict[str, Any]
|
JSON Schema for the configuration
|
Source code in ccbt/config/config.py
| def get_schema(self) -> dict[str, Any]:
"""Get configuration schema.
Returns:
JSON Schema for the configuration
"""
from ccbt.config.config_schema import ConfigSchema
return ConfigSchema.generate_full_schema()
|
get_section_schema(section_name: str) -> dict[str, Any] | None
Get schema for a specific configuration section.
Parameters:
| Name |
Type |
Description |
Default |
section_name
|
str
|
Name of the configuration section
|
required
|
Returns:
| Type |
Description |
dict[str, Any] | None
|
Schema for the section or None if not found
|
Source code in ccbt/config/config.py
| def get_section_schema(self, section_name: str) -> dict[str, Any] | None:
"""Get schema for a specific configuration section.
Args:
section_name: Name of the configuration section
Returns:
Schema for the section or None if not found
"""
from ccbt.config.config_schema import ConfigSchema
return ConfigSchema.get_schema_for_section(section_name)
|
list_options() -> list[dict[str, Any]]
List all configuration options with metadata.
Returns:
| Type |
Description |
list[dict[str, Any]]
|
List of configuration options with metadata
|
Source code in ccbt/config/config.py
| def list_options(self) -> list[dict[str, Any]]:
"""List all configuration options with metadata.
Returns:
List of configuration options with metadata
"""
from ccbt.config.config_schema import ConfigDiscovery
return ConfigDiscovery.list_all_options()
|
start_hot_reload() -> None
async
Start hot-reload monitoring.
Source code in ccbt/config/config.py
| async def start_hot_reload(self) -> None:
"""Start hot-reload monitoring."""
if not self.config_file: # pragma: no cover
return # pragma: no cover
logger = get_logger(
__name__
) # pragma: no cover - Hot reload loop, difficult to test
logger.info("Starting configuration hot-reload monitoring") # pragma: no cover
try: # pragma: no cover
# track current task so stop_hot_reload can cancel it
self._hot_reload_task = asyncio.current_task() # pragma: no cover
except Exception: # pragma: no cover
self._hot_reload_task = None # pragma: no cover
while await self._hot_reload_loop_step(logger): # pragma: no cover
pass # pragma: no cover
|
stop_hot_reload() -> None
Stop hot-reload monitoring.
Source code in ccbt/config/config.py
| def stop_hot_reload(self) -> None:
"""Stop hot-reload monitoring."""
if (
hasattr(self, "_hot_reload_task") and self._hot_reload_task
): # pragma: no cover
self._hot_reload_task.cancel() # pragma: no cover
|
validate_detailed() -> tuple[bool, list[str]]
Validate configuration with detailed error messages.
Returns:
| Type |
Description |
tuple[bool, list[str]]
|
Tuple of (is_valid, list_of_errors)
|
Source code in ccbt/config/config.py
| def validate_detailed(self) -> tuple[bool, list[str]]:
"""Validate configuration with detailed error messages.
Returns:
Tuple of (is_valid, list_of_errors)
"""
from ccbt.config.config_schema import ConfigValidator
config_data = self.config.model_dump(mode="json")
# Basic validation
is_valid, errors = ConfigValidator.validate_with_details(config_data)
# Cross-field validation
if is_valid:
cross_field_errors = ConfigValidator.validate_cross_field_rules(config_data)
errors.extend(cross_field_errors)
is_valid = len(cross_field_errors) == 0
return is_valid, errors
|
validate_option(key_path: str, value: Any) -> tuple[bool, str]
Validate a single configuration option.
Parameters:
| Name |
Type |
Description |
Default |
key_path
|
str
|
Dot-separated path to the option
|
required
|
value
|
Any
|
|
required
|
Returns:
| Type |
Description |
tuple[bool, str]
|
Tuple of (is_valid, error_message)
|
Source code in ccbt/config/config.py
| def validate_option(self, key_path: str, value: Any) -> tuple[bool, str]:
"""Validate a single configuration option.
Args:
key_path: Dot-separated path to the option
value: Value to validate
Returns:
Tuple of (is_valid, error_message)
"""
from ccbt.config.config_schema import ConfigValidator
return ConfigValidator.validate_option(key_path, value)
|
Features:
- Configuration loading: ccbt/config/config.py:_load_config
- File discovery: ccbt/config/config.py:_find_config_file
- Environment variable parsing: ccbt/config/config.py:_get_env_config
- Hot reload support: ccbt/config/config.py:ConfigManager
Config Models
Pydantic-based configuration models.
Implementation: ccbt/models.py
Configuration sections:
- NetworkConfig: ccbt/models.py:NetworkConfig
- DiskConfig: ccbt/models.py:DiskConfig
- StrategyConfig: ccbt/models.py:StrategyConfig
- DiscoveryConfig: ccbt/models.py:DiscoveryConfig
- LimitsConfig: ccbt/models.py:LimitsConfig
- ObservabilityConfig: ccbt/models.py:ObservabilityConfig
- SecurityConfig: ccbt/models.py:SecurityConfig
- MLConfig: ccbt/models.py:MLConfig
- DashboardConfig: ccbt/models.py:DashboardConfig
Main config: ccbt/models.py:Config
ConfigSchema
Configuration schema and validation.
Implementation: ccbt/config/config_schema.py
ConfigTemplates
Predefined configuration templates.
Implementation: ccbt/config/config_templates.py
Templates:
- High-performance setup
- Low-resource setup
- Security-focused setup
- Development setup
ConfigMigration
Configuration migration utilities.
Implementation: ccbt/config/config_migration.py
ConfigBackup
Configuration backup utilities.
Implementation: ccbt/config/config_backup.py
ConfigDiff
Configuration diff utilities.
Implementation: ccbt/config/config_diff.py
ConfigCapabilities
Feature detection and capabilities.
Implementation: ccbt/config/config_capabilities.py
ConfigConditional
Conditional configuration support.
Implementation: ccbt/config/config_conditional.py
Plugins
Plugin Base
Base plugin class for extensibility.
Implementation: ccbt/plugins/base.py
Plugin states: ccbt/plugins/base.py:PluginState
Plugin error: ccbt/plugins/base.py:PluginError
MetricsPlugin
Metrics collection plugin.
Implementation: ccbt/plugins/metrics_plugin.py
LoggingPlugin
Logging plugin.
Implementation: ccbt/plugins/logging_plugin.py
Observability
Profiler
Performance profiler for function-level, async, memory, and I/O profiling.
Implementation: ccbt/observability/profiler.py:Profiler
Profile types: ccbt/observability/profiler.py:ProfileType
Profile entry model: ccbt/observability/profiler.py:ProfileEntry
Profile report model: ccbt/observability/profiler.py:ProfileReport
Methods:
- start(): ccbt/observability/profiler.py:93 - Start profiling
- stop(): ccbt/observability/profiler.py - Stop profiling
- profile_function(): ccbt/observability/profiler.py - Profile a function
- profile_async(): ccbt/observability/profiler.py - Profile async operations
- get_report(): ccbt/observability/profiler.py - Get profiling report
Features:
- Function-level profiling with cProfile integration
- Async operation profiling
- Memory usage tracking
- I/O operation profiling
- Bottleneck detection
Interface
Terminal Dashboard (Bitonic)
Textual-based terminal dashboard for real-time monitoring.
Implementation: ccbt/interface/terminal_dashboard.py:TerminalDashboard
Initialization: ccbt/interface/terminal_dashboard.py:299
Layout composition: ccbt/interface/terminal_dashboard.py:321
Key bindings: ccbt/interface/terminal_dashboard.py:337
Widgets:
- Overview: Global statistics overview
- SpeedSparklines: Real-time speed graphs
- TorrentsTable: Active torrents table
- PeersTable: Connected peers table
- RichLog: Logging output
Methods:
- compose(): ccbt/interface/terminal_dashboard.py:321 - Compose dashboard layout
- on_mount(): ccbt/interface/terminal_dashboard.py:346 - Initialize dashboard
- _poll_once(): ccbt/interface/terminal_dashboard.py - Poll session for updates
- _schedule_poll(): ccbt/interface/terminal_dashboard.py - Schedule periodic polling
Entry point: ccbt/interface/terminal_dashboard.py:main
Entry point configuration: pyproject.toml:81
CLI Components
Interactive CLI
Interactive command-line interface.
Implementation: ccbt/cli/interactive.py:InteractiveCLI
Features:
- Interactive command processing
- Command history
- Auto-completion
- Session management integration
CLI Progress Display
Progress bar and status display utilities.
Implementation: ccbt/cli/progress.py
Features:
- Download progress bars
- Speed indicators
- ETA calculations
- Multi-torrent progress display
Checkpoint Management
CheckpointManager
Comprehensive checkpoint management for resume functionality with JSON and binary format support.
Key Methods:
- save_checkpoint(): ccbt/storage/checkpoint.py:save_checkpoint - Save checkpoint with format selection (JSON, binary, or both)
- load_checkpoint(): ccbt/storage/checkpoint.py:load_checkpoint - Load checkpoint from disk
- list_checkpoints(): ccbt/storage/checkpoint.py:list_checkpoints - List all available checkpoints
- delete_checkpoint(): ccbt/storage/checkpoint.py:delete_checkpoint - Delete checkpoint file
- validate_checkpoint(): ccbt/storage/checkpoint.py:validate_checkpoint - Validate checkpoint integrity
Checkpoint Data:
- Piece states: Tracks which pieces are verified, complete, or missing
- File progress: Per-file download progress for multi-file torrents
- Download statistics: Bytes downloaded, uploaded, speed, etc.
- Torrent metadata: Info hash, name, file paths
Checkpoint Models
Checkpoint data models.
Implementation: ccbt/models.py:TorrentCheckpoint
Properties:
- info_hash: Torrent info hash
- torrent_name: Torrent name
- verified_pieces: List of verified piece indices
- piece_states: Piece state mapping
- torrent_file_path: Original torrent file path
- magnet_uri: Original magnet URI
See the CheckpointManager section below for detailed usage.
Session Resume Methods
Resume functionality methods in AsyncSessionManager:
CLI checkpoint commands: ccbt/cli/main.py:checkpoints
CLI Integration
All API functionality is accessible via the CLI:
See btbt CLI Reference for complete CLI documentation.
Data Models
Comprehensive data models for all components with Pydantic validation.
Implementation: ccbt/models.py
Enumerations
class LogLevel(str, Enum):
"""Logging levels."""
DEBUG = "DEBUG"
INFO = "INFO"
WARNING = "WARNING"
ERROR = "ERROR"
CRITICAL = "CRITICAL"
class PieceSelectionStrategy(str, Enum):
"""Piece selection strategies."""
ROUND_ROBIN = "round_robin"
RAREST_FIRST = "rarest_first"
SEQUENTIAL = "sequential"
class PreallocationStrategy(str, Enum):
"""File preallocation strategies."""
NONE = "none"
SPARSE = "sparse"
FULL = "full"
FALLOCATE = "fallocate"
class PieceState(str, Enum):
"""Piece download states."""
MISSING = "missing"
REQUESTED = "requested"
DOWNLOADING = "downloading"
COMPLETE = "complete"
VERIFIED = "verified"
class ConnectionState(str, Enum):
"""Peer connection states."""
DISCONNECTED = "disconnected"
CONNECTING = "connecting"
HANDSHAKE_SENT = "handshake_sent"
HANDSHAKE_RECEIVED = "handshake_received"
CONNECTED = "connected"
BITFIELD_SENT = "bitfield_sent"
BITFIELD_RECEIVED = "bitfield_received"
ACTIVE = "active"
CHOKED = "choked"
ERROR = "error"
class CheckpointFormat(str, Enum):
"""Checkpoint file format options."""
JSON = "json"
BINARY = "binary"
BOTH = "both"
class TorrentPriority(str, Enum):
"""Torrent priority levels for queue management."""
PAUSED = "paused" # Do not download
LOW = "low" # Lowest priority
NORMAL = "normal" # Default priority
HIGH = "high" # High priority
MAXIMUM = "maximum" # Highest priority
Core Models
-
PeerInfo: ccbt/models.py:PeerInfo - Peer information with IP, port, peer_id
PROPORTIONAL = "proportional" # Allocate by priority weight ratio
EQUAL = "equal" # Equal share to all active torrents
FIXED = "fixed" # Fixed KiB/s per priority level
MANUAL = "manual" # User-specified per torrent
class MessageType(int, Enum):
"""BitTorrent message types."""
CHOKE = 0
UNCHOKE = 1
INTERESTED = 2
NOT_INTERESTED = 3
HAVE = 4
BITFIELD = 5
REQUEST = 6
PIECE = 7
CANCEL = 8
class PeerInfo(BaseModel):
"""Peer information."""
ip: str = Field(..., description="Peer IP address")
port: int = Field(..., ge=1, le=65535, description="Peer port number")
peer_id: bytes | None = Field(None, description="Peer ID")
peer_source: str | None = Field(
default="tracker",
description="Source of peer discovery (tracker/dht/pex/lsd/manual)",
)
@field_validator("ip")
@classmethod
def validate_ip(cls, v):
-
TrackerResponse: ccbt/models.py:TrackerResponse - Tracker announce response
"""Validate IP address format."""
# Basic IP validation - could be enhanced with proper IP address validation
if not v or len(v) == 0:
msg = "IP address cannot be empty"
raise ValueError(msg)
return v
def str(self) -> str:
"""String representation of peer info."""
return f"{self.ip}:{self.port}"
def hash(self) -> int:
-
PieceInfo: ccbt/models.py:PieceInfo - Piece information with index, length, hash, state
"""Hash peer info for use as dictionary key."""
return hash((self.ip, self.port))
def eq(self, other) -> bool:
"""Equality comparison for peer info."""
if not isinstance(other, PeerInfo):
return False
return self.ip == other.ip and self.port == other.port
model_config = {"arbitrary_types_allowed": True}
class TrackerResponse(BaseModel):
"""Tracker response data."""
interval: int = Field(..., ge=0, description="Announce interval in seconds")
-
FileInfo: ccbt/models.py:FileInfo - File information with name, length, path
peers: list[PeerInfo] = Field(default_factory=list, description="List of peers")
complete: int | None = Field(None, ge=0, description="Number of seeders")
incomplete: int | None = Field(None, ge=0, description="Number of leechers")
download_url: str | None = Field(None, description="Download URL")
tracker_id: str | None = Field(None, description="Tracker ID")
warning_message: str | None = Field(None, description="Warning message")
class PieceInfo(BaseModel):
-
TorrentInfo: ccbt/models.py:TorrentInfo - Complete torrent metadata
"""Piece information."""
index: int = Field(..., ge=0, description="Piece index")
length: int = Field(..., gt=0, description="Piece length in bytes")
hash: bytes = Field(
...,
min_length=20,
max_length=20,
description="Piece SHA-1 hash",
)
state: PieceState = Field(default=PieceState.MISSING, description="Piece state")
model_config = {"arbitrary_types_allowed": True}
class FileInfo(BaseModel):
"""File information for torrents.
Supports BEP 47 padding files and extended file attributes:
- Padding files (attr='p'): Used for piece alignment, skipped during download
- Symlinks (attr='l'): Symbolic links with target path
- Executable (attr='x'): Files with executable permission
- Hidden (attr='h'): Hidden files (Windows)
"""
Configuration Models
-
NetworkConfig: ccbt/models.py:NetworkConfig - Network settings with validation
name: str = Field(..., description="File name")
length: int = Field(..., ge=0, description="File length in bytes")
path: list[str] | None = Field(None, description="File path components")
full_path: str | None = Field(None, description="Full file path")
# BEP 47: Padding Files and Attributes
attributes: str | None = Field(
None,
description="File attributes string from BEP 47 (e.g., 'p', 'x', 'h', 'l')",
)
symlink_path: str | None = Field(
None,
description="Symlink target path (required when attr='l')",
)
file_sha1: bytes | None = Field(
None,
description="SHA-1 hash of file contents (optional BEP 47 sha1 field, 20 bytes)",
)
@property
def is_padding(self) -> bool:
"""Check if file is a padding file (BEP 47 attr='p')."""
return self.attributes is not None and "p" in self.attributes
@property
def is_symlink(self) -> bool:
"""Check if file is a symlink (BEP 47 attr='l')."""
return self.attributes is not None and "l" in self.attributes
@property
def is_executable(self) -> bool:
"""Check if file is executable (BEP 47 attr='x')."""
return self.attributes is not None and "x" in self.attributes
@property
def is_hidden(self) -> bool:
"""Check if file is hidden (BEP 47 attr='h')."""
return self.attributes is not None and "h" in self.attributes
@field_validator("symlink_path")
@classmethod
def validate_symlink_path(cls, v: str | None, _info: Any) -> str | None:
"""Validate symlink_path is provided when attr='l'."""
# Note: This validator runs before model_validator, so we can't check attributes here
# The model_validator below handles the cross-field validation
return v
@field_validator("file_sha1")
@classmethod
def validate_file_sha1(cls, v: bytes | None, _info: Any) -> bytes | None:
"""Validate file_sha1 is 20 bytes (SHA-1 length) if provided."""
if v is not None and len(v) != 20:
msg = f"file_sha1 must be 20 bytes (SHA-1), got {len(v)} bytes"
raise ValueError(msg)
return v
@model_validator(mode="after")
def validate_symlink_requirements(self) -> FileInfo:
"""Validate symlink_path is provided when attr='l'."""
if self.attributes and "l" in self.attributes and not self.symlink_path:
msg = "symlink_path is required when attributes contains 'l' (symlink)"
raise ValueError(msg)
return self
-
DiskConfig: ccbt/models.py:DiskConfig - Disk I/O settings
-
StrategyConfig: ccbt/models.py:StrategyConfig - Piece selection strategy
-
DiscoveryConfig: ccbt/models.py:DiscoveryConfig - Tracker and DHT settings
-
LimitsConfig: ccbt/models.py:LimitsConfig - Rate limiting configuration
-
ObservabilityConfig: ccbt/models.py:ObservabilityConfig - Monitoring and logging
-
SecurityConfig: ccbt/models.py:SecurityConfig - Security features
-
MLConfig: ccbt/models.py:MLConfig - Machine learning features
-
DashboardConfig: ccbt/models.py:DashboardConfig - Dashboard settings
-
Config: ccbt/models.py:Config - Main configuration aggregating all sections
Checkpoint Models
Validation
All models use Pydantic field validators: ccbt/models.py
Field constraints include:
- Range validation (ge, le, gt, lt)
- String length validation
- IP address format validation
- Type coercion and validation
Module Exports
Public API exports: ccbt/init.py
Key exports:
- AsyncSessionManager: ccbt/init.py:94
- ConfigManager: [ccbt/init.py]
- TorrentParser: [ccbt/init.py]
- Utility modules
Best Practices
Resource Management
Use async context managers where available. See ccbt/session/session.py:AsyncSessionManager
Error Handling
Handle exceptions appropriately:
- ccbt/utils/exceptions.py:CCBTException - Base exception
- ccbt/utils/exceptions.py:NetworkError - Network errors
- ccbt/utils/exceptions.py:DiskError - Disk errors
- ccbt/utils/exceptions.py:ProtocolError - Protocol errors
Async Operations
All I/O operations are asynchronous. Always use await:
- Session operations: ccbt/session/session.py:AsyncSessionManager
- Peer operations: ccbt/peer/async_peer_connection.py
- Piece operations: ccbt/piece/async_piece_manager.py
- Storage operations: ccbt/storage/disk_io.py
Configuration
Access configuration via ConfigManager: ccbt/config/config.py:ConfigManager
Configuration file: ccbt.toml
Environment variables: env.example
Monitoring
Enable monitoring for production use:
- Metrics: ccbt.toml:164
- Alerts: ccbt.toml:170
- Tracing: ccbt.toml:168
See the Monitoring section below for detailed setup.
Helper Functions and Utilities
Torrent Builder Functions
Configuration Helpers
Service Helpers
Module Structure
Package Exports
Public API: ccbt/init.py
Key exports defined in __all__: ccbt/init.py:108
Includes:
- Core classes: AsyncSessionManager, TorrentParser, BencodeEncoder, BencodeDecoder
- Configuration: Config, ConfigManager
- Models: MagnetInfo
- Modules: All utility and component modules
Lazy attribute access: ccbt/init.py:160 - Supports dynamic imports
Type Safety
Type marker file: ccbt/py.typed - Indicates package supports type checking
All modules use comprehensive type hints with:
- Type annotations for all functions and methods
- Generic types where appropriate
- Pydantic models for runtime validation
- Protocol definitions for interfaces
BitTorrent Protocol v2 (BEP 52) API
TorrentV2Parser
Main class for BitTorrent Protocol v2 operations.
Implementation: ccbt/core/torrent_v2.py:TorrentV2Parser
Methods
parse_v2(info_dict: dict, torrent_data: dict) -> TorrentV2Info
Parse v2-only torrent metadata.
- Parameters:
info_dict: Bencoded info dictionary from torrent filetorrent_data: Complete torrent data dictionary- Returns:
TorrentV2Info object with parsed metadata
- Raises:
ValueError if parsing fails or metadata is invalid
parse_hybrid(info_dict: dict, torrent_data: dict) -> tuple[TorrentInfo, TorrentV2Info]
Parse hybrid torrent (both v1 and v2 metadata).
- Returns: Tuple of (v1 TorrentInfo, v2 TorrentV2Info)
- Raises:
ValueError if metadata is incomplete or invalid
generate_v2_torrent(...) -> bytes
Generate v2-only torrent file.
Parameters:
- source: Path - Source file or directory
- output: Path | None = None - Output torrent file path
- trackers: list[str] | None = None - Tracker announce URLs
- web_seeds: list[str] | None = None - WebSeed URLs
- comment: str | None = None - Torrent comment
- created_by: str = "ccBitTorrent" - Creator name
- piece_length: int | None = None - Piece length (auto-calculated if None)
- private: bool = False - Private torrent flag
Returns: Bencoded torrent file as bytes
generate_hybrid_torrent(...) -> bytes
Generate hybrid torrent compatible with both v1 and v2.
Parameters: Same as generate_v2_torrent()
Returns: Bencoded hybrid torrent file as bytes
TorrentV2Info
Data model for v2 torrent metadata.
Implementation: ccbt/core/torrent_v2.py:TorrentV2Info
Attributes
name: str - Torrent nameinfo_hash_v2: bytes - 32-byte SHA-256 info hashinfo_hash_v1: bytes | None - 20-byte SHA-1 info hash (hybrid only)announce: str - Primary tracker URLannounce_list: list[list[str]] | None - Tracker tierscomment: str | None - Torrent commentcreated_by: str | None - Creator namecreation_date: int | None - Unix timestampencoding: str | None - Character encodingis_private: bool - Private torrent flagfile_tree: dict[str, FileTreeNode] - Hierarchical file structurepiece_layers: dict[bytes, PieceLayer] - Piece layer hashespiece_length: int - Piece length in bytesfiles: list[FileInfo] - List of files in torrenttotal_length: int - Total size in bytesnum_pieces: int - Total number of pieces
Methods
get_file_paths() -> list[str]
Get list of all file paths in torrent.
get_piece_layer(pieces_root: bytes) -> PieceLayer | None
Get piece layer for a specific file by its pieces root hash.
Protocol Communication
Implementation: ccbt/protocols/bittorrent_v2.py
Protocol Version Detection
detect_protocol_version(handshake: bytes) -> ProtocolVersion
Detect BitTorrent protocol version from handshake.
- Returns:
ProtocolVersion.V1, ProtocolVersion.V2, or ProtocolVersion.HYBRID
- Raises:
ProtocolVersionError if handshake is invalid
parse_v2_handshake(data: bytes) -> dict[str, Any]
Parse v2 or hybrid handshake into components.
Returns dictionary with keys:
- protocol: bytes - Protocol string
- reserved_bytes: bytes - Reserved bytes
- info_hash_v1: bytes | None - v1 hash (if present)
- info_hash_v2: bytes - v2 hash
- peer_id: bytes - Peer ID
- version: ProtocolVersion - Detected version
Handshake Creation
create_v2_handshake(info_hash_v2: bytes, peer_id: bytes) -> bytes
Create v2 handshake (80 bytes).
- Parameters:
info_hash_v2: 32-byte SHA-256 hashpeer_id: 20-byte peer ID- Raises:
ProtocolVersionError if lengths are invalid
create_hybrid_handshake(info_hash_v1: bytes, info_hash_v2: bytes, peer_id: bytes) -> bytes
Create hybrid handshake (100 bytes).
- Parameters:
info_hash_v1: 20-byte SHA-1 hashinfo_hash_v2: 32-byte SHA-256 hashpeer_id: 20-byte peer ID
Protocol Negotiation
negotiate_protocol_version(handshake: bytes, supported: list[ProtocolVersion]) -> ProtocolVersion | None
Negotiate protocol version with peer.
- Parameters:
handshake: Peer's handshake bytessupported: List of versions we support (in priority order)- Returns: Negotiated version or None if incompatible
Async Communication
async send_v2_handshake(writer: StreamWriter, info_hash_v2: bytes, peer_id: bytes) -> None
Send v2 handshake asynchronously.
async send_hybrid_handshake(writer: StreamWriter, info_hash_v1: bytes, info_hash_v2: bytes, peer_id: bytes) -> None
Send hybrid handshake asynchronously.
async handle_v2_handshake(reader: StreamReader, writer: StreamWriter, our_info_hash_v2: bytes | None = None, our_info_hash_v1: bytes | None = None, timeout: float = 30.0) -> tuple[ProtocolVersion, bytes, dict]
Receive and validate v2 handshake.
Returns: (protocol_version, peer_id, parsed_handshake)
async upgrade_to_v2(connection: Any, info_hash_v2: bytes) -> bool
Attempt to upgrade v1 connection to v2.
Returns: True if upgrade successful, False otherwise
V2 Messages
PieceLayerRequest (Message ID 20)
Request piece layer hashes for a file.
request = PieceLayerRequest(pieces_root)
data = request.serialize() # Returns bytes with length prefix
PieceLayerResponse (Message ID 21)
Respond with piece layer hashes.
response = PieceLayerResponse(pieces_root, piece_hashes)
data = response.serialize()
FileTreeRequest (Message ID 22)
Request complete file tree structure.
request = FileTreeRequest()
data = request.serialize()
FileTreeResponse (Message ID 23)
Send file tree structure (bencoded).
response = FileTreeResponse(file_tree_bencoded)
data = response.serialize()
SHA-256 Hashing
Implementation: ccbt/piece/hash_v2.py
Piece Hashing
hash_piece_v2(data: bytes) -> bytes
Hash piece data using SHA-256.
Returns: 32-byte hash
hash_piece_v2_streaming(data_source: bytes | IO) -> bytes
Hash piece data from file or stream.
verify_piece_v2(data: bytes, expected_hash: bytes) -> bool
Verify piece hash.
verify_piece_v2_streaming(data_source: bytes | IO, expected_hash: bytes) -> bool
Verify piece hash from stream.
Merkle Tree Hashing
hash_piece_layer(piece_hashes: list[bytes]) -> bytes
Build Merkle tree from piece hashes.
Returns: 32-byte root hash (pieces_root)
verify_piece_layer(piece_hashes: list[bytes], expected_root: bytes) -> bool
Verify piece layer against expected root.
File Tree Hashing
hash_file_tree(file_tree: dict[str, FileTreeNode]) -> bytes
Hash file tree structure.
Returns: 32-byte file tree root hash
Configuration
Protocol v2 settings in ProtocolV2Config:
Implementation: ccbt/models.py:ProtocolV2Config
Attributes:
- enable_protocol_v2: bool = True - Enable v2 support
- prefer_protocol_v2: bool = False - Prefer v2 over v1
- support_hybrid: bool = True - Support hybrid torrents
- v2_handshake_timeout: float = 30.0 - Handshake timeout
Access via: config.network.protocol_v2
Environment variables:
- CCBT_PROTOCOL_V2_ENABLE
- CCBT_PROTOCOL_V2_PREFER
- CCBT_PROTOCOL_V2_SUPPORT_HYBRID
- CCBT_PROTOCOL_V2_HANDSHAKE_TIMEOUT
CLI Commands
Create v2 torrent:
ccbt create-torrent file.mp4 --v2 --output file.torrent --tracker http://tracker.example.com/announce
Create hybrid torrent:
ccbt create-torrent directory/ --hybrid --output directory.torrent
Enable v2 protocol:
ccbt download file.torrent --protocol-v2
See BEP 52 Guide for comprehensive documentation and examples.
Additional Resources