This is the second article in a series on Lightning Network routing gossip. The introduction article explained the purpose of Lightning Network routing gossip.  

This article will discuss the next topic, a frequently used primitive in the Lightning Network gossip, the short_channel_id. The short_channel_id is a compact way to identify a channel by pointing to the on-chain unspent transaction output (UTXO) that was used to fund the channel. short_channel_id is defined in Bolt #7.

A short_channel_id is 8-bytes and includes three pieces of information that point to the funding UTXO.

  1. The height of the block (3 bytes)
  2. The transaction index in the block (3 bytes)
  3. The output index in the transaction (2 bytes)

Given these three pieces of information, you can obtain a block based on its height. Obtain the transaction in the block based on its index. Then find the output of that transaction using the output index. We'll get to how this relates to a channel when we discuss channel_announcement messages validation in a subsequent article.

Astute readers may be wondering why it's acceptable to use a blockheight as a reference given reorganizations. One of the parameters used during channel creation specifies the "safe" number of blocks that both parties must wait until the channel can be considered "ready for use". short_channel_ids and the subsequence gossip messages are only used after this block depth has been reached making it statistically unlikely that a channel identified by height will change.  And even if it did happen, at worse our view of the network might contain some stale information that would eventually be pruned out of the view.

Also using 8-bytes to accomplish our UTXO goal imposes a few practical limits on the data we can retrieve. Block height is limited to 24-bytes (16,777,215) which certainly won't be a problem for Bitcoin in the foreseeable future. The transaction index is also impossible to reach due to the block weight restrictions (at least at present). Same goes for the output index limits of 65535.

Without futher ado, the short_channel_id can take three forms: human readable encoding, binary encoded, and unsigned-integer encoded. We'll also talk about how we can find the UTXO that the short_channel_id represents.

Human Readable Encoding

A human readable short_channel_id looks like 539268x845x1.  This example has block 539268, transaction index 845, and output index 1. The three values are combined with an x between each value. Pretty straightforward.

The human readable version of an short_channel_id is mostly used when printing information or displaying a channel to a user. Data is usually not transmitted in this format.

The human readable portion is easily created from the component pieces of a short_channel_id using a string formatting function which is pretty self-explanatory. On to more interesting things.

Binary Encoding

When a short_channel_id is included in a wire messages it is encoded in binary format. Values are encoded in an 8-byte representation using the format:

  • 3-byte uint big-endian block height as the first three bytes
  • 3-byte uint big-endian transaction index in the next three bytes
  • 2-byte uint big-endian output index last two bytes

A short_channel_id for block 539268, transaction index 845, and output index 1 would be encoded as

> 539268x845x1        human readable
> 0x083a8400034d0001  binary encoded (hex)

083a84 - block (539268)
00034d - tx index (845)
0001   - output index (1)

Many implementations convert the binary format into their component pieces and store this as an object. Extracting these values into their component pieces is relatively straightforward using a byte stream reader.

type ShortChannelId = {  block: number, txIdx: number, outIdx: number };

function scidFromBuf(buf: Buffer): ShortChannelId {
    const block = buf.readUIntBE(0, 3);  // index=0, read=3
    const txIdx = buf.readUIntBE(3, 3);  // index=3, read=3
    const outIdx = buf.readUIntBE(6, 2); // index=6, read=2
    return { block, txIdx, outIdx };
}

Similarly, these values can be encoded by writing into a byte stream accordingly:

type ShortChannelId = {  block: number, txIdx: number, outIdx: number };

function scidToBuffer(scid: ShortChannelId): Buffer {
    const buf = Buffer.alloc(8);
    buf.writeUIntBE(scid.block, 0, 3);  // index=0, write=3
    buf.writeUIntBE(scid.txIdx, 3, 3);  // index=3, write=3
    buf.writeUIntBE(scid.outIdx, 6, 2); // index=6, write=2
    return buf;
}

Unsigned-Integer Encoding

Some implementations of the Lightning Network encode and display a short_channel_id as a unsigned integer, specifically uint64. It is efficient to store the value as such since it only takes 8-bytes. This is compared to an object (8-byte pointer) with two u32 and one u16 for a best case of 18-bytes. In JavaScript, this is a whopping 32-bytes since all numbers are stored as f64!

We can see the same short_channel_id encoded in all three formats:

> 539268x845x1         human readable
> 0x083a8400034d0001   binary encoding (hex)
> 592931436542885889   numeric encoding

We can easily encode a uint64 into its big-endian byte representation. For example, a really simple example using the Node.js Buffer class looks like:

function scidToBuf(scid: bigint): Buffer {
    const buf = Buffer.alloc(8);
    buf.writeBigUInt64BE(scid);
    return buf;
}

Using this function results the binary encoding we would expect 

> scidToBuf(592931436542885889n);
<Buffer 08 3a 84 00 03 4d 00 01>

This function is just for example though. In reality, we wouldn't want to allocate a new Buffer just for the short_channel_id. We would be writing to a stream or a larger Buffer containing the entire message to prevent excessive memory garbage.

We can easily decode from a Buffer using a similar function:

function scidFromBuf(buf: Buffer): bigint {
    return buf.readBigUInt64BE();
}

Again this function assumes the Buffer only contains 8-bytes containing the short_channel_id. In reality this would likely be consumed from a stream or a Buffer at a specific index position.

It is also easy to extract the component pieces from an short_channel_id encoded as a uint64 using some helper functions.  

Extracting the block is easy, we just remove the lower 5-bytes by right shifting them. We are then left with the topmost 3-bytes (24-bits) which just happens to the block.

function extractScidBlock(scid: bigint): number {
    return scid >> 40n;
}

Extracting the transaction index is the hardest. We must mask off the middle 3-bytes. We then left shift the lower bits to extract the numeric value of the index correctly.

function extractScidTxIdx(scid: bigint): number {
    return (scid & 0x000000_ffffff_0000n) >> 16n;
}

Finally, extracting the output index just requires us to mask the lower 16 bits.

function extractScidOutIdx(scid: bigint): number {
    return scid & 0x000000_000000_ffffn;
}

We now have some simple helpers we can use to convert a short_channel_id into its component pieces.

We can use these to create the human readable version by taking the component pieces and using a string formatter:

function humanReadableScid(scid: bigint): string {
    const block = extractScidBlock(scid);
    const txIdx = extractScidTxIdx(scid);
    const outIdx = extractScidOutIdx(scid);
    return `${block}x${txIdx}x${outIdx}`;
}

So now you should have a solid understanding of the ways a short_channel_id can be represented. The last piece to discuss is how the short_channel_id can be used to look up a UTXO.

short_channel_id to UTXO

As we've discussed, the short_channel_id is really a compact way to represent the UTXO that was used to fund a channel. There is no magic bullet to obtaining the UTXO information. But using the component pieces of the short_channel_id we can make a series of calls to take a peek at the output.  

We'll use a real channel example (on testnet) with short_channel_id=1413847x29x0. Using bitcoind we'll get to the root of the UTXO data!

First step is finding the block hash where block=1413847. We can use the getblockhash command to obtain the block hash:

> bitcoin-cli -testnet getblockhash 1413847
000000003e2df2ae6afd9d95997ceb7cbd21de3c3582171d61dd4142f15668c5

Next we need to find the transaction with index 29 in the block. We can obtain the block metadata using our recently obtained block hash value. The block metadata includes the transaction identifiers in the the order they were included in the block.  We use the getblock command to make this happen:

bitcoin-cli -testnet getblock 000000003e2df2ae6afd9d95997ceb7cbd21de3c3582171d61dd4142f15668c5
{
  "hash": "000000003e2df2ae6afd9d95997ceb7cbd21de3c3582171d61dd4142f15668c5",
  "confirmations": 524274,
  "strippedsize": 45665,
  "size": 64190,
  "weight": 201185,
  "height": 1413847,
  "version": 536870912,
  "versionHex": "20000000",
  "merkleroot": "4e7a51a097974c0970d58d95ee3b2faeea432168f144a29320418db2cfca493c",
  "tx": [
    "c3b2c7150d8d551a8ab8bf50ad4145b2cd1fb50a19e73bf3dc03b5e87a3185c7",
    "8fd19d2e3e55652b993c180288828edaea2ab31f2e2685a56f916974463924c1",
    ...
    "2bd60e409731dccc0e760e4036fde7a2c2eb5f89e224e0c69d4f14580c67a7b0",
    ...
    "e048d088e0709fe13deb6cb2864e0817f0cb0b556c96d8f84b2f35382cab57ce",
    "30b67d420b2a49a5464a0e7163338a29fbad162e1a688bf704a926ee0563774f"
  ],
  "time": 1537380158,
  "mediantime": 1537374703,
  "nonce": 2566288397,
  "bits": "1d00ffff",
  "difficulty": 1,
  "chainwork": "0000000000000000000000000000000000000000000000c4ab37c2d6806a142c",
  "nTx": 196,
  "previousblockhash": "000000004be67ed462c438b3304257f53d324c94ea754d1c1905cade88eb211f",
  "nextblockhash": "000000000000000c5dbc3aad8cfd247d43ccc3602db83f9eeca89a2398699e93"
}

If we find the 30th transaction (remember indexes are zero-based) we can obtains txid: 2bd60e409731dccc0e760e4036fde7a2c2eb5f89e224e0c69d4f14580c67a7b0.

Finally, we know that we want output 0 for this transaction. We can obtain the output information using the gettxout command that will retrieve UTXOs from bitcoind.

> bitcoin-cli -testnet gettxout 2bd60e409731dccc0e760e4036fde7a2c2eb5f89e224e0c69d4f14580c67a7b0 0
{
  "bestblock": "0000000000000011843e1bb62a28d4c904175cff3add3a832ebbad63bc0d7f7e",
  "confirmations": 524274,
  "value": 0.16540784,
  "scriptPubKey": {
    "asm": "0 6ae4a633af96e275789f062d4bb8ecf95ea21bc191ef26279a2228694cdd7720",
    "hex": "00206ae4a633af96e275789f062d4bb8ecf95ea21bc191ef26279a2228694cdd7720",
    "reqSigs": 1,
    "type": "witness_v0_scripthash",
    "addresses": [
      "tb1qdtj2vva0jm3827ylqck5hw8vl902yx7pj8hjvfu6yg5xjnxawusq6qvk5n"
    ]
  },
  "coinbase": false
}

We can do a quick gut check and make sure that it's a P2WSH output and we are good to go!

If the value isn't returned by gettxout then the UTXO may be spent, meaning the the channel has closed. In this case, you can still look up the value using getrawtx and looking at the first output!

So there you have it. Everything you could possibly want to know about short_channel_id. In the next article we'll dig into these last few steps a bit deeper as they bleed into the the process used to validate channel_announcement messages. Once we have validated channel_announcement messages we can start to construct a graph of the network!