two computers meet on the internet. what do they say first?

2 computers have never met.

they do not share a language, a timezone, a reason to trust each other, or any prior relationship. 1 wants some bytes. the other may have them. the only thing they know is that they are both interested in the same 20-byte hash.

what do they say first?

if you guessed "a 68-byte binary handshake followed by length-prefixed messages over a raw tcp connection," congratulations. you already understand bittorrent's peer wire protocol.

if you guessed "probably http and json," honestly, fair. that is how most software introduces itself now.

bittorrent does not.

trackers may help peers find one another over http. the dht performs peer discovery through its own udp-based rpc. but once 2 peers actually connect and begin exchanging file data, they speak something much smaller and much stranger: the peer wire protocol.

no request headers. no json objects. no status codes. just bytes with very strict meanings.

this is the story of building that protocol for styx, my bittorrent client in rust. you do not need to know bittorrent, networking, or rust to follow along. if you understand that computers need rules before they can exchange data, you already know enough.

a tiny glossary before we start

you can read the article without memorizing any of this. this is just a map for the words that would otherwise appear out of nowhere.

the only relationship worth remembering right now is:

torrent
  → split into pieces
      → split into blocks
          → requested from peers

the first 68 bytes

the peer wire protocol begins with exactly one moment of ceremony: the handshake.

after that, nearly every message uses the same format: 4 bytes describing a length, followed by that many bytes of payload. but the handshake comes first, before the regular message stream begins.

it is always 68 bytes:

byte 0:      1 byte   protocol string length
bytes 1–19:  19 bytes "BitTorrent protocol"
bytes 20–27: 8 bytes  reserved extension bits
bytes 28–47: 20 bytes torrent info hash
bytes 48–67: 20 bytes peer id

that is the entire introduction.

in human terms:

• the protocol string says, "hello, this is a bittorrent conversation."
• the info hash says, "i am here for this specific torrent."
• the peer id says, "this is the identity i am using for this session."
• the reserved bytes say, very quietly, "i may know a few extra tricks."

if the protocol string is wrong, the conversation ends.

if the info hash does not match the torrent this connection belongs to, the conversation ends.

there is no awkward recovery. no "perhaps you meant another endpoint." you have approached the wrong table. please leave.

why 19?

because "BitTorrent protocol" contains 19 bytes.

the 1st byte tells the receiver how long the protocol identifier is, and every normal bittorrent implementation sends the same 19-byte string. changing it would mean speaking a different protocol to peers expecting bittorrent, which is a wonderfully efficient way to make no friends.

this is one of those tiny protocol decisions that becomes permanent through success. a number chosen in 2001 now sits at the beginning of connections made by modern clients decades later.

the reserved bytes

the 8 reserved bytes are a small parking lot for future capabilities.

styx currently understands 2 important flags:

• bep 5 dht support
• bep 10 extended messaging support

the dht flag means the peer participates in bittorrent's distributed peer-discovery network. once advertised, the remote side may send a port message containing its dht udp port.

the extended-messaging flag opens a more general escape hatch. peers can send an extension handshake through message id 20, then advertise locally assigned ids for features such as metadata exchange or peer exchange.

the important design idea is not the exact bit positions. it is that old peers can ignore bits they do not understand while newer peers still discover extra capabilities.

the handshake stays fixed. the protocol grows around it. that is a very good trick.

the info hash is a boundary, not an identity card

the 20-byte v1 info hash identifies the torrent. styx compares the received value against the torrent associated with the connection and rejects a mismatch immediately.

this does not authenticate the remote peer.

it does not prove they possess valid data. it does not make their blocks trustworthy. it simply establishes which torrent this conversation concerns.

think of it as checking the event name on someone's ticket, not performing a background check.

actual data integrity happens later, when completed pieces are verified against the hashes from the torrent metadata.

this distinction matters because protocol code becomes dangerous when 1 check is allowed to carry more meaning than it really has. the handshake selects the swarm. piece verification protects the content. different jobs, different layers.

the peer id

the peer id is 20 bytes of session identification.

some clients use a recognizable prefix that advertises their brand and version. styx does not do that by default. its default privacy configuration has no brand prefix, and its primary runtime path generates peer ids through a cryptographically secure random-number generator.

the identity manager also remembers ids it has already produced and retries if it generates a duplicate. that makes reuse extraordinarily unlikely in the normal path without turning the peer id into a persistent identity.

styx's persisted torrent state contains things such as the torrent source, destination, status, and timestamps. it does not store the peer id. restart the client and the old identity is gone.

the peer id is merely an introduction, not a passport.

tcp gives you bytes, not messages

once the handshake is complete, the regular message stream begins.

every frame looks like this:

[4-byte big-endian payload length][payload]

the payload normally begins with a 1-byte message id, followed by fields specific to that message.

this framing exists because tcp is a stream protocol.

that sentence sounds boring until it ruins your afternoon.

when you send one message over tcp, the receiver is not promised 1 matching read. a message may arrive in pieces. multiple messages may arrive together. the network is allowed to hand you whatever chunking was convenient, because tcp promises an ordered stream of bytes — not the boundaries your application imagined.

so the protocol creates its own boundaries.

the 4-byte prefix says exactly how many bytes belong to the next message. the receiver reads the length, then reads precisely that many bytes, then interprets the result.

keepalive: the empty envelope

the smallest valid frame is 4 zero bytes.

00 00 00 00

a payload length of zero means keepalive. there is no message id and no body.

it says:

i am still connected. i simply have nothing useful to contribute right now.

which, to be fair, describes a meaningful percentage of college club group chats.

styx can encode and decode keepalive frames. scheduling them is deliberately outside the wire codec — and at the current stage of the project, styx does not yet contain automatic keepalive generation. the wire layer knows what the message is. a future runtime policy will decide when to send it.

that difference between "can represent" and "currently schedules" is exactly the kind of boundary this project tries to keep honest.

the vocabulary of a bittorrent peer

the original peer protocol defines nine numbered messages, ids 0 through 8, plus the zero-length keepalive frame.

styx also understands several extension messages:

• id 9: dht port
• id 20: the extension-protocol container
• ids 21, 22, and 23: bittorrent v2 hash messages

but the original nine are enough to understand the social dynamics of a swarm.

1st, peers describe what they have.

then they decide whether they are interested.

then choking policy determines who may request data.

then requests and pieces begin moving.

it is a dating ritual designed by people who prefer finite-state machines.

bittorrent solved mixed signals by assigning them integer ids.

the request/piece dance

the most important exchange is request → piece.

a peer asks for a block by specifying:

• the piece index
• the offset inside that piece
• the desired block length

the other peer responds with:

• the same piece index
• the same offset
• the block bytes

here is a request for the 1st 16 kib block of piece 5:

length prefix:  00 00 00 0d   13-byte payload
message id:     06            request
piece index:    00 00 00 05   piece 5
begin offset:   00 00 00 00   start of piece
block length:   00 00 40 00   16,384 bytes

17 bytes total.

the encoder is correspondingly unromantic:

PeerMessage::Request {
    index,
    begin,
    length,
} => {
    bytes.put_u8(6);
    put_request_payload(&mut bytes, *index, *begin, *length);
}

write the message id. write 3 big-endian integers. done.

there are places where abstractions clarify a system, and places where they merely put a waistcoat on 12 bytes. this was the second kind.

before reading the message, distrust the frame

every byte in a peer-to-peer protocol comes from a machine you do not control.

the remote peer may be correct, buggy, confused, ancient, experimental, or actively hostile. the decoder must behave sensibly in every case.

styx therefore validates the frame before it tries to understand the message inside it.

the complete-frame decoder follows this order:

fn complete_payload(
    input: &[u8],
    max_payload_len: u32,
) -> Result<&[u8], PeerWireError> {
    if input.len() < 4 {
        return Err(TruncatedLengthPrefix {
            actual: input.len(),
        });
    }
 
    let payload_len =
        u32::from_be_bytes([input[0], input[1], input[2], input[3]]);
 
    if payload_len > max_payload_len {
        return Err(FrameTooLarge {
            length: payload_len,
            max: max_payload_len,
        });
    }
 
    let available = input.len() - 4;
    let declared = payload_len as usize;
 
    if available < declared {
        return Err(TruncatedFrame {
            declared: payload_len,
            actual: available,
        });
    }
 
    if available > declared {
        return Err(TrailingFrameBytes {
            declared: payload_len,
            actual: available,
        });
    }
 
    Ok(&input[4..])
}

before the message id is inspected, 4 questions are answered:

1. is there a complete length prefix?
2. is the declared payload below our configured cap?
3. did the peer actually provide that many bytes?
4. did the peer provide exactly that many bytes?

the club-bouncer translation:

1. no invitation? no entry.
2. claiming you brought 4 gb of friends? the venue has a capacity.
3. said 5 people, arrived with three? sus.
4. said 5 people, arrived with seven? also sus.

only after the frame passes does the decoder ask what kind of message it contains.

reject absurd sizes before allocating memory

the order matters most in the async reader.

let payload_len = u32::from_be_bytes(len_prefix);
 
if payload_len > max_payload_len {
    return Err(PeerWireError::FrameTooLarge {
        length: payload_len,
        max: max_payload_len,
    });
}
 
let mut payload = vec![0; payload_len as usize];
reader.read_exact(&mut payload).await?;

allocation happens only after the declared size passes the cap.

without that check, a malicious peer could claim an enormous frame and trick the client into attempting an enormous allocation before sending any useful data.

the principle is broader than bittorrent:

never let untrusted input choose a resource cost before you have bounded it.

that applies to message lengths, decompression ratios, recursion depth, uploaded image dimensions, database query limits, and almost every parser that has ever become a security advisory.

structural validation

after framing comes message shape.

fixed-size messages must contain exactly the expected body length.

a have message contains one 4-byte piece index after its id. a dht port message contains one 2-byte port. request and cancel each contain 3 4-byte integers.

styx rejects deviations with typed errors.

fn require_len(
    message: &'static str,
    body: &[u8],
    expected: usize,
) -> Result<(), PeerWireError> {
    if body.len() == expected {
        return Ok(());
    }
 
    Err(PeerWireError::InvalidMessageLength {
        message,
        expected: fixed_len_label(expected),
        actual: body.len() + 1,
    })
}

this catches malformed traffic before higher layers receive a misleading value.

semantic validation

some frames are structurally valid but still meaningless.

a request for 0 bytes has all the correct fields, yet requests nothing. a piece message with an empty block has a correct header, yet delivers no data.

styx rejects both.

if length == 0 {
    return Err(PeerWireError::ZeroBlockLength { message });
}

and:

if body.len() < 9 {
    return Err(PeerWireError::InvalidMessageLength {
        message: "piece",
        expected: "at least 10",
        actual: body.len() + 1,
    });
}

the validation stack is therefore:

• framing: is this one bounded, complete frame?
• structure: does this message contain the right fields?
• semantics: do those fields describe something meaningful?

each layer narrows the set of inputs allowed to travel deeper into the client.

the decoder should never get emotionally overwhelmed

rust prevents many forms of memory unsafety, but rust code can still panic.

an unchecked index can panic. an unwrap() can panic. an expect() attached to a false assumption can panic.

inside a wire decoder, a remotely reachable panic is a denial-of-service opportunity wearing a friendly error message.

the styx peer decoder avoids unwrap() and expect() along its parsing path. malformed input becomes PeerWireError, not process termination.

the test suite also feeds it arbitrary short byte vectors and checks that decoding returns normally:

proptest! {
    #[test]
    fn arbitrary_frames_never_panic(
        input in prop_vec(any::<u8>(), 0..128)
    ) {
        let _ = decode_message_frame(&input);
    }
}

this does not mathematically prove safety for every possible byte sequence. proptest executes a finite number of generated cases, and this particular strategy covers vectors shorter than 128 bytes.

what it does provide is a continuously checked invariant:

strange bytes are allowed to be rejected. they are not allowed to surprise the process into dying.

that is a useful mindset for any boundary parser.

the decoder is not supposed to like every input. it is supposed to remain composed.

networking will split your beautiful message wherever it pleases

the synchronous frame decoder receives 1 complete byte slice, which makes exact-length validation straightforward.

the async reader has a different problem: real stream reads may be partial.

styx uses read_exact for both handshake and message bodies. tests place the reader and writer on deliberately tiny in-memory duplex streams, forcing data to move in fragments.

let (mut client, mut server) = duplex(3);
 
let writer = tokio::spawn(async move {
    write_message(&mut client, &outbound).await
});
 
let decoded =
    read_message(&mut server, DEFAULT_MAX_PEER_FRAME_LEN).await?;

the duplex stream has a capacity of only three bytes. the encoded piece message is larger than that, so the writer cannot deposit it in one convenient operation. the reader must reconstruct it across multiple partial transfers.

this test is valuable because local development has a dangerous habit of making the network look polite. it is not.

bytes in one crate, decisions in another

the most important decision in this implementation was not how to encode request.

it was deciding what the wire module should not know.

the peer codec lives in styx-proto. it knows how to:

• encode and decode handshakes
• encode and decode peer messages
• read and write those values over async byte streams
• reject malformed wire representations

it does not know:

• whether a peer should be choked
• which block should be requested next
• whether an incoming piece was actually requested
• where downloaded bytes are stored
• whether a completed piece passes its content hash
• when to reconnect
• which peers deserve another chance

those decisions belong elsewhere.

the runtime owns sockets and orchestration. it connects peers, drains decoded messages, and passes them into the core manager.

the core manager owns peer behavior and transfer policy. it receives a structured PeerMessage, not raw bytes.

when a piece message arrives, the core manager reconstructs the corresponding block request and completes it against that peer's in-flight request pipeline:

let request = BlockRequest::new(
    PieceIndex::new(index),
    BlockOffset::new(begin),
    length,
);
 
let pipeline = self
    .pipelines
    .get_mut(&peer)
    .ok_or(CoreError::UnknownPeer { peer })?;
 
pipeline.complete(request)?;

if the block was never requested — or does not match an in-flight request — the pipeline rejects it.

only after that succeeds does the session record the downloaded byte count and emit an AcceptBlock action:

if let Some(session) = self.peers.get_mut(&peer) {
    session.record_download(now, block.len() as u64);
}
 
Ok(vec![PeerAction::AcceptBlock {
    peer,
    request,
    bytes: block,
}])

the runtime then executes that action by handing the block to the piece manager. once all blocks for a piece arrive, the disk layer verifies and commits the completed piece.

there is a clean chain of responsibility:

tcp bytes
    ↓
styx-proto: decode a valid peer message
    ↓
styx-core: decide whether that message is acceptable
    ↓
styx-runtime: execute the resulting action
    ↓
styx-disk: store and verify the data

each layer knows enough to do its job and not enough to quietly steal another layer's job.

why this separation feels so good later

a narrow wire layer is easy to reason about because it is mostly transformation:

bytes → message
message → bytes

there is no peer reputation, disk state, retry policy, or piece-selection strategy hiding inside the parser.

that gives 3 practical benefits.

testing

wire tests do not need a complete torrent runtime. they can construct bytes, decode them, and inspect the result.

they can construct messages, encode them, decode them again, and verify round-trip fidelity.

they can simulate fragmented streams without loading files or contacting a tracker.

transport independence

the framing rules describe the bittorrent conversation, not the transport carrying it.

styx later added utp support without modifying the peer-wire module. the utp commit touched no files in styx-proto.

that is the boundary paying rent.

tcp changed to another reliable byte-stream transport. the wire messages did not care.

extension without demolition

styx added extended messaging and bittorrent v2 hash messages later. those additions expanded the enum and decoder match arms, but the fundamental frame-validation path remained unchanged.

the outer grammar stayed stable:

length prefix → bounded payload → message decoding

new vocab entered the conversation without forcing everyone to rebuild the room.

testing the wire

the test suite is small enough to understand and broad enough to catch the mistakes that matter.

exact-layout tests

the handshake test checks that encoding produces exactly 68 bytes, with the correct protocol string and info-hash position.

that is important because binary protocols do not reward being "basically correct." byte 28 is byte 28. the peer on the other side is not interested in your intentions.

rejection tests

individual tests cover malformed inputs:

• wrong handshake length
• wrong protocol string
• mismatched info hash
• truncated length prefix
• truncated payload
• trailing bytes
• oversized frame
• unknown message id
• incorrect fixed-size body
• zero-length request
• empty piece block
• malformed extension message

these tests read almost like requirements:

an oversized declared frame must be rejected before allocation. a request with a zero block length must be rejected. a complete frame may not contain undeclared trailing bytes.

a good rejection test documents the edge of the accepted language.

async stream tests

tiny duplex buffers verify that handshakes and messages survive fragmented delivery.

this is less glamorous than a giant integration test and more likely to catch the exact bug the network will eventually show you at 2:17 a.m.

round-trip property tests

proptest generates valid peer messages from a strategy, encodes them, decodes the result, and verifies that the same message comes back.

#[test]
fn peer_messages_roundtrip(message in arb_peer_message()) {
    let encoded = encode_message(&message)?;
    let decoded = decode_message_frame(&encoded)?;
    prop_assert_eq!(decoded, message);
}

this catches disagreement between encoder and decoder: wrong field order, wrong lengths, forgotten variants, or accidental asymmetry.

what does not exist yet

styx currently has a fuzz target for its bencode decoder, but not for the peer-wire decoder.

that is now an obvious next step.

the entry point is already excellent for fuzzing:

fn fuzz_peer_frame(bytes: &[u8]) {
    let _ = decode_message_frame(bytes);
}

simple input. no setup. clear success boundary. malformed bytes should produce an error, never a crash or hang.

what this module deliberately does not do

the peer-wire layer is complete only because its scope is narrow.

it does not interpret whether a bit in a bitfield corresponds to a valid piece for the current torrent. it preserves the bytes; the core layer applies torrent context.

it does not decide when to send keepalives. it only represents them.

it does not reconnect dropped peers. it reports the i/o failure.

it does not score peers, quarantine bad sources, or decide whether a peer is worth another request.

it does not verify downloaded pieces. content verification happens after the full piece is assembled.

it does not negotiate every extension internally. message id 20 carries an extension id and opaque payload upward, where extension-specific modules can interpret it.

this is not missing functionality inside the codec. this is the codec refusing to become the entire client.

simple protocols are not simple implementations

the peer wire protocol is tiny enough to explain in 1 article:

• 1 fixed-size handshake
• 1 length-prefixed frame format
• 9 original numbered messages
• a handful of extensions
• 1 request-and-piece exchange that moves the actual data

but "small protocol" does not mean "nothing can go wrong."

you can still:

• trust a declared length before bounding it
• confuse tcp reads with message boundaries
• accept malformed fixed-size messages
• let invalid pieces bypass the request pipeline
• panic on hostile input
• mix raw parsing with peer policy until neither can be tested alone

the real elegance is not that the protocol uses few bytes.

it is that those bytes can pass through a sequence of narrow, understandable boundaries:

is the frame complete?
is it bounded?
does the message have the right shape?
does it make semantic sense?
was this block actually requested?
does the completed piece match its hash?

each question belongs somewhere specific.

each answer lets the next layer work with stronger assumptions.

that is how a pile of untrusted network bytes slowly becomes a verified file on disk.

this post is part of the styx series. the previous posts covered simulating a bittorrent swarm and building the bencode parser behind torrent metadata. next: trackers, the part where a client finally asks the internet, "okay, where is everyone?"