Bitcoin Optech

Bitcoin Optech Newsletter #425 Recap Podcast

2026-10-06T00:00:00+00:00

Gustavo Flores Echaiz and Brandon Black are joined by Matt Morehouse to discuss Newsletter #425.

The Bitcoin Optech Podcast and transcription content is licensed Creative Commons CC BY-SA 2.0

Download audio

News

● Disclosure of two DoS vulnerabilities in Eclair (1:01)
● Proposal for wallet label synchronization (15:25)

Changing consensus

● Continued discussion of PQC output types (21:36)
● Block-wide signature aggregation via SNARKs (27:17)
● Bounds on chain length with BIP54 time warp fixes (35:19)
● Comparing covenant proposals for vaults (39:23)
● Depots for probabilistic Lightning channels (43:18)

Releases and release candidates

● LND v0.21.4-beta.rc1 (48:40)
● LND v0.20.5-beta.rc1 (50:29)

Notable code and documentation changes

● Bitcoin Core #29278 (51:55)
● Bitcoin Core #35984 (53:43)
● Bitcoin Core #35301 (56:45)
● Bitcoin Core #36312 (58:51)
● Bitcoin Core #36284 (1:01:59)
● Bitcoin Core #35752 (1:07:47)
● Bitcoin Core #35813 (1:11:46)
● BIPs #2276 (1:13:07)
● LDK #4993 (1:16:39)
● LND #11173 (1:19:40)
● LND #11190 (1:22:12)
● LND #11212 (1:27:37)
● LND #11258 (1:29:58)

Transcription

Gustavo Flores Echaiz: Hi everyone and welcome for another episode of the Bitcoin Optech podcast. Today, we’re covering Newsletter #425, and I’m pleased to be joined by Brandon, Rearden Code, and Matt Morehouse. Brandon, would you like to introduce yourself; and later on, Matt, please?

Brandon Black: Sure. I’m Brandon Black or Rearden Code, I’ve been working in the Bitcoin industry for almost ten years now on various things, self-custody, whatnot. Most recently, I started my own Bitcoin node implementation because I got sniped by the vibe-coding vibe.

Matt Morehouse: And I’m Matt Morehouse, and I work on LN security.

Gustavo Flores Echaiz: Awesome. Thank you guys. Today, I’ll be hosting with Brandon since Murch and Mike couldn’t make it. So, this week we have two News items and a lot in the Changing consensus section; and later on, two Releases and multiple Notable code and documentation changes.

Disclosure of two DoS vulnerabilities in Eclair

We’re going to start with the first News item. So, “Disclosure of two DoS vulnerabilities in Eclair”. Matt, you posted on Delving Bitcoin about a responsible disclosure of two DoS vulnerabilities affecting Eclair v0.13.1. And both of these vulnerabilities were fixed back in May 2026 in v0.14.0. Any users that are still running an older version should upgrade now. So, Matt, would you like to give us maybe an intro to what these vulnerabilities were about?

Matt Morehouse: Sure. Both of the vulnerabilities have similar impact, where they could basically cause Eclair to use an extreme amount of resources and eventually crash. The first vulnerability is related to Eclair’s feature bit parsing. So, in the Lightning protocol, there’s this concept of feature vectors that are used to advertise the different features that your node supports. So, there’s different messages you send to peers. In those messages are these feature vectors, which are a bit vector where each bit corresponds to one feature. So, if you set the bit to 1, it means, “I support this feature”, and if you set it to 0, it means, “I don’t support this feature”. Now, Eclair’s problem was that the way they parsed these feature vectors was extremely inefficient. So, they would actually iterate the entire bit vector bit-by-bit six different times, and they would allocate multiple heap objects for each bit in the vector. And so, the maximum length feature vector you can have in the protocol is 64 KB. And if you sent one of these maximum length feature vectors, Eclair would allocate over 300 MB of data on the heap. Now that’s not all at once; it would allocate some data and then free some data. But cumulative over the course of processing the message, it would allocate 300 MB of memory, and it would take up to 300 milliseconds to process one of these messages. So, an attacker could simply send a whole bunch of messages that have these max length feature vectors, and Eclair would run out of memory within a few minutes and crash.

The fix that Eclair did for this was basically to optimize their feature vector parsing. So, instead of iterating the peer’s feature vector, what they do now is they simply check the certain feature bits that Eclair knows about in the feature vector. So, Eclair has a list of around 30 features they support, and so they just check the offsets in the peer’s feature vector to see if the peer supports those too. And then, they pretty much don’t look at any other part of the feature vector, and they’ll just store it as a bit vector directly instead of allocating data on the heap for each bit. So, after the fix, Eclair processes a single message in one to two milliseconds and the heap churn is much lower; I think it was around 200 KB, or something like that, and the attack no longer works.

The second vulnerability is similar in that it causes Eclair to allocate a lot of memory, but the mechanism is different. So, in the Lightning protocol, in the gossip part of the protocol, there’s a couple of messages where you encode a list of short channel IDs (SCIDs). And these are basically to tell another node, “Hey, I’d like some data or some information about these channels; do you have any?” And then, if the node does, then they can respond and send you some gossip messages to help you update your view of the Lightning graph. And in the early spec, there were two different ways to encode these lists of SCIDs. The first way is just a simple, plain array, where each SCID gets 8 bytes in the array, and you just line them up next to each other and send them on the wire. The second way was using zlib encoding. So, you could have a long list of SCIDs that gets compressed into a much shorter payload and then gets sent on the wire that way.

Now, in 2022, this second encoding, the zlib encoding, was actually removed from the spec, I believe because at the time LDK was trying to implement this part of the spec and they didn’t want to take a dependency on zlib, because zlib had had security issues in the past. And all the implementations agreed at the time that we don’t actually really need the zlib encoding, so let’s just remove it, and the plain array encoding works good enough. So, within a month of that change to the spec, Eclair removed support for sending zlib-encoded data. But they kept in the ability to receive zlib-encoding data, because it would take a while for the network to update. And so, they still wanted to be able to communicate with the rest of the network with nodes that hadn’t upgraded yet. And they had planned to have this second phase happen sometime in the near future and remove the ability to receive zlib-encoded data as well, but they just never got around to doing that. And so, as late as a few months ago, they still allow receiving zlib-encoded data.

What this means for an attacker is that they can send a small payload that decompresses to a large amount of memory. The maximum compression ratio for zlib is about 1,000 to 1. So, if an attacker creates a pathological payload, a 64-KB message on the wire could turn into 64 MB in Eclair’s memory. And so, I did some experiments with this, and sure enough, by sending an all-zero payload, it decompresses to 64 MB. And then, to make matters worse, Eclair would allocate additional heap memory for each 8 bytes of that list. So, each 8 bytes corresponds to one SCID, and Eclair would allocate 48 bytes total for each one of those. So, we end up with a total of, like, 450 MB of memory that Eclair would allocate. So, an attacker just sends a few of these zlib bombs, and Eclair runs out of memory and crashes.

Their fix was to just remove zlib support entirely. It’s been gone from the spec since 2022. No one’s even sending it anymore, and that fixed that. But both of these, again, were fixed in Eclair 0.14.0, so make sure you upgrade. There’s also several newer security releases on top of that. So, I would upgrade to the latest one if you haven’t yet.

Brandon Black: I had two questions for you about this. One, were you specifically looking for vulnerabilities when you found these, or was it kind of happenstance from other work you were doing?

Matt Morehouse: Yeah, I was looking specifically. So, the first vulnerability, the feature-parsing one, was found by fuzzing. We have a new Lightning-specific fuzzer called Smite that basically acts like a malicious node, and will connect to a target Lightning node and will send random stuff on the wire to it. And in this particular case, the scenario that was being fuzzed was Smite would connect to Eclair, it would complete the BOLT8-encrypted handshake, and then it would just send random bytes over the encrypted connection. And then, it would send a ping and wait for the pong from Eclair, and if the pong took too long to come back, then it flagged the input as potentially causing a crash or a hang or just being slow. And so, one of these inputs happened to be an init message that had a very large feature vector in it. And so, that kind of tipped me off to the issue here. And then, looking further, figured out the code was very inefficient and tried exploiting it and came to the crash.

The second one was found by telling an LLM to basically look for the similar pattern in Eclair. So, I gave it the text of the first vulnerability and I said, “Here’s a vulnerability where Eclair allocates magnitudes more resources than it takes an attacker to send the message. Are there any similar possible ways an attacker could do this?” And it came up with a list of potential things, and then I just worked down the list, and the zlib bomb was the one that ended up working.

Brandon Black: Very cool. My other question was, what are the consequences to a node if it gets taken offline by this in the LN? I know we’ve talked about that before on this show, but it’s worth reiterating why these DoSs really matter for Lightning nodes.

Matt Morehouse: Yeah, a Lightning node needs to be online in order to protect its funds. If the Lightning node is offline and your peer force-closes, they could force-close with an old commitment that’s been revoked. And if your node isn’t online to see that and punish that, then you can lose funds that way. Now, there’s watchtowers which can help with that, but if your node’s not online, even if you have a watchtower, you can still lose HTLC (Hash Time Locked Contract) values. So, there’s ways to steal funds if you can’t handle an HTLC onchain quick enough, then your upstream peer can time it out and get a refund on it. So, it’s very important for all Lightning nodes to be online all the time. These particular vulnerabilities, they would crash your node once, and then you could restart your node, and hopefully your node would be able to process onchain events in the small window between restart and when the attacker could crash you again. But you’d have to keep restarting and there’s no guarantee. So, to be safe, you’d rather just not have your node crash at all.

Brandon Black: That’s good advice right there. Yeah, also, I mean I think Lightning, we’ve learned over the many years now of Lightning that when possible, it’s best to have some degree of relationship with your channel partners, because if you have a channel with someone and they find something like this, they can cause you pain. If you have a little bit of trust of just not malice with your channel partners, the LN works quite a bit better. So, yeah, awesome stuff.

Gustavo Flores Echaiz: Totally. Thank you so much, Matt. So, I had a question. So, do you know the reason why, in April 2022, BOLT7 removed the zlib encoding? Was it because this type of vulnerability was feared, or for another reason?

Matt Morehouse: All I can speculate, well, I can only tell you what the PR that removes it says. I wasn’t aware of whatever conversations were happening outside of the PR at the time. So, it’s possible that they knew of this attack vector already, and that was another reason for removing it from the spec that was not stated in the PR. But what the PR basically says is LDK didn’t want to take a zlib dependency, and that the zlib-encoding type wasn’t needed anyways. And then, everybody commented, “Concept ACK sounds good”, and then it got merged.

Gustavo Flores Echaiz: Understood, okay. So, it wasn’t like this was already known as a potential vulnerability. Maybe it was understood by some, but it wasn’t disclosed and presented publicly. Awesome. So, besides upgrading to v0.14.0, do you have any other advice for listeners, any action items they should pick up from this item?

Matt Morehouse: I think in general, just stay updated. There’s a flood of all kinds of vulnerabilities being reported lately with LLMs. And so, if you’re running a Lightning node, you should be watching releases like a hawk and upgrading as soon as possible.

Gustavo Flores Echaiz: Totally. Well, thank you so much, Matt. Brandon, would you have any other questions for Matt? On my end, that would be it. Awesome. So, you’re free to drop, but you can also stay in the podcast if you wish to. Thank you so much, Matt.

Matt Morehouse: Yeah, thanks, folks.

Proposal for wallet label synchronization

Gustavo Flores Echaiz: Awesome. So, now we’re going to go to the next item in the News section, which is, “Proposal for wallet label synchronization”. So, I don’t know how much you looked into this one, Matt, but from my understanding, Jakub posted to the Bitcoin-Dev mailing list proposing a standardization spec of how wallets can basically communicate between themselves or synchronize wallet labels between themselves. So, I believe there are two open questions here. How can they do this in a manner where only they can decrypt the information? And also, how can they transport this information? So, my understanding is that Jakub proposes that the descriptor data, which is technically considered not fully private data, but in most cases it is data that only wallet holders have, that the descriptor data would be used to sort of decrypt and find the information for wallet label synchronization. And Jakub also proposed Nostr as a reference transfer protocol. But the protocol only requires a service that can store and return data.

I saw some major pushback from Craig Raw, where he basically says that this is not a bad idea in itself, it’s just that label synchronization should be part of a broader inter-wallet communication, which should include other use cases such as PSBTs, multisig setups, and payment confirmations. Because he does bring a good point that wallets could not just have the need to synchronize on wallet labels, but on other types of data. Craig Raw also pushed back on the use of Nostr as the reference transfer protocol, since exchange of financial data should optimize for privacy rather than censorship-resistance, and noting that he’s working on a BIP for canonical output descriptors.

I think this is an interesting proposal. It does sound similar or at least inspired partially by something we covered in the previous newsletter, which was BIP138, which was a compact encryption scheme for non-seed wallet data. But this is more applied to wallet labels, or not just encrypting or decrypting information, but rather creating a sort of synchronization specification system that works through a transport protocol and also has techniques around how to find the data and decrypt the data. So, would you have any thoughts you would like to add here, Brandon? I would just say that if listeners are more interested, they can look at the original Bitcoin-Dev mailing list comment. They can also reply to it since Jakub is looking for feedback here. But please, Brandon, if you have any thoughts, I would love to hear them.

Brandon Black: I think my biggest thing is, as you said, there’s recent talk about other kind of related parts of the wallet ecosystem. I think we’re seeing, as Bitcoin matures here and people have wallets that live a long time that have multiple signers in them, obviously companies that support multisig wallets already have somewhat of an interchange that they’re doing of this kind of data between the different signers in that wallet. But when you’re doing it in an open-source way, there isn’t a standard yet for doing that. And so, I think this is reflecting a need for standardization of things that right now you only get when you use a vendor for your multisig. And I hope this kind of stuff goes forward. There’s been lots and lots of talk about this kind of thing in terms of how you encrypt and retrieve descriptor data, and how you exchange PSBTs, and even payjoin essentially relates to this, because payjoin uses Nostr as an option for how to exchange in-progress payjoins. So, I think hopefully we’ll see kind of a coalescing of these different proposals and get a general interop library essentially, where wallets can make payjoins and label synchronization and all that stuff work together, as Craig kind of said.

Gustavo Flores Echaiz: Totally. Yes, I was recently thinking about this and, for example, I use Nunchuk, and Nunchuk has a sort of internal transport protocol where we can basically share data with other multisig participants. But yeah, it would be really cool to have a sort of standard that wallets can apply and wallets can become interoperable this way. And I also think this is particularly related to coin selection. So, for example, yes, BIP329 allows for a standardized way of exporting labels from a wallet and importing them into another wallet. But let’s say you are two signers in the same multisig wallet trying to coordinate signing and also selecting specific UTXOs. Well, synchronizing wallet labels in real time would be really helpful in order to both have the same information on the specific UTXOs that will be chosen for this wallet.

So, yeah, once again listeners, if you are curious to learn more about this, Jakub shared this a few weeks ago, on September 13, on the Bitcoin-Dev mailing list. You can reach out to him, you can also join the conversation for further feedback, and we’ll also be patiently waiting for Craig Raw’s work, because he said he’s working on a BIP for canonical output descriptors. So, he seems to have some ideas on this topic and potentially maybe brings on a solution that will solve it. That’s it for this section, and we can jump in directly to the Changing consensus section. Brandon, the floor is yours.

Continued discussion of PQC output types

Brandon Black: All right. Yes, we have several interesting items in Changing consensus. As has been common, a chunk of it is about post-quantum stuff, with the first item being a continuation of the discussion we’ve covered twice already on Optech Changing consensus, about post-quantum output types. This is a big topic, which is why it’s taking up a lot of bandwidth, let’s say, on the mailing list and on Delving as well. The basic question here is if we’re going to make a new output type that’s for post quantum particularly, what does that look like? And how do we, as an ecosystem, create the right incentives for users to move to that type when it’s right for them? Because Bitcoin isn’t about forcing people to change how they do things, we’ve never done that kind of thing. We always want to look at, I think, a Bitcoin ecosystem in avoiding confiscation, as was worked hard on in past soft forks and past changes. And so, there’s a lot of debate here.

The most recent conversation is really about the incentives, where if we were to create a new output type that supports both a post-quantum signature scheme, but also cross-input signature aggregation (CISA) for existing secp256k1-based signatures, that would create a situation where moving to the post-quantum supporting output type would also give you a fee savings on your existing elliptic curve secp signatures. And so, the debate is, is that the right thing to do? Antoine Poinsot posted about how that’s kind of an opposite-direction pull. You’re kind of pulling some people that are fee-sensitive, but then some people that aren’t fee-sensitive into the same output type, and is that the right incentive? So, yeah, it’s a huge conversation. We’ve covered it three times here for a reason. It’s worth users following up on this and commenting, because this is one of the biggest conversations happening in Bitcoin right now.

As everyone probably knows by now, I don’t think quantum is necessarily coming to attack Bitcoin, but I think this is a very, very important conversation, because we are going to want additional crypto systems in Bitcoin, regardless of quantum. We might want zero-knowledge proofs (ZKPs) of some sort, we might want a hash-based signature for various reasons, especially as we think about Bitcoin existing for 20, 50, 100 years. We don’t want to be relying on a single crypto system forever. So, really worth diving in here and considering the shape of the next output type after segwitv1 P2TR, (Pay-to-Taproot), whether that might be P2MR (Pay-to-Merkle-Root), P2TRv2, or something else that’s not proposed yet. Really worth thinking about this stuff.

Gustavo Flores Echaiz: Thank you, Brandon, that was really good coverage. Could you just remind for listeners what P2TRv2 is? Maybe some didn’t listen to the previous episode where we covered it.

Brandon Black: Yeah, so I’ll cover both P2MR and P2TRv2. So, P2MR is basically taking P2TR and taking out the internal key. So, existing taproot has an internal key and then a script tree, and you can spend either using a keypath or a scriptpath. P2MR is a pretty, in a certain way, simple change. It just takes out that internal key. So, it’s just a merkle root of script tree; there’s only a script spend. And so, that makes it kind of the root spend path is now quantum-resistant because it’s a hash-based spend path not vulnerable to quantum. And so, users could then put still an elliptic curve secp signature in one of those scriptpaths, but it’s hidden behind a hash and therefore resists quantum attack, as long as you don’t reuse addresses. On the flip side, P2TRv2 basically takes taproot exactly as it is, copies it, but says, if you’re using P2TRv2, you expect that at some point, your keypath will become unspendable. So, when you use P2TRv2, you as the user are accepting you cannot always rely on keypath. So, you should either expect to rotate your UTXOs frequently, or have a scriptpath that you can fall back on. And that’s kind of, if we needed to turn off secp for some reason in an urgency, without confiscating anyone’s coins who didn’t opt into that potential confiscation, we could turn off the keypath spend on P2TRv2.

Now, inside either of these, you could hide – hide is maybe the wrong word, but you could put a quantum-resistant signing scheme as well. And so, the details could be similar under the hood. The question here is about what the root level looks like. Is it P2TR with an internal key that could be activated, or is it just P2MR?

Gustavo Flores Echaiz: Right, totally. Very, very interesting. So, it seems that, I mean, this research has had a little while going already and we still haven’t settled, but it’s great to see a discussion here. So, if listeners want to learn more about it, the discussion is still ongoing, very actively on Delving. It started a few months ago, I believe, in July. But now, it still has recent communication updates. So, everyone’s welcome to join the conversation, but also to learn more about the details directly on Delving.

Brandon Black: Yeah, it’s a huge conversation, and I think it’s going to keep going because there’s no perfect answer. And we’ll find, I think, the right compromise. I’m glad that the minds like Pieter Wuille and Antoine Poinsot and Conduition are in there thinking hard about it, because a lot of it goes above my pay grade for sure.

Block-wide signature aggregation via SNARKs

All right, moving along, speaking of ZKPs that I mentioned, as we’re talking about post quantum again, we’re facing the very real possibility that post-quantum signatures might be much larger than our existing secp256k1 signatures. And we’re talking potentially an order of magnitude larger. So, we either accept a much lower throughput on Bitcoin transactions, or we need to do something. Some conversations have kind of looked around at what it would look like to separately price post-quantum signature data, so that we would have larger blocks that could potentially have, I think Conduition cited, a terabyte per year of blockchain growth. I mentioned an order of magnitude; that kind of matches, right? An order of magnitude faster blockchain growth to facilitate a similar transaction throughput with post-quantum signatures.

I think for most bitcoiners, that’s probably not something we really want to do. That would be a significantly higher cost for node runners, and we want to keep the network decentralized, all these things we talked about. So, what’s the alternative? And that’s where this post comes in. Conduition posted about really a sketch at this point, it’s not a fully fleshed out proposal, but for a way that we could have a block-wide proof that all the signatures were validated, rather than having every signature embedded in the blocks that every node stores. So, nodes would relay all the big signatures, so we’d have a higher relay bandwidth. But once the blocks are mined, they would only include a compact proof that all the signatures were validated and were valid at the time the block was created. And then, nodes would only store that smaller proof durably. And that proof could be in the shape of a SNARK.

So, Conduition did some proof-of-concept work on this, looking at how big the proof would be, and it came up with about 500 KB probably. So, we’d end up with this interesting kind of block structure, where the main transaction data might be up to a megabyte, and then you’d have this one big proof over all of that transaction data that those signatures were valid that would take up another half megabyte, and blocks would be about the same size they are today when you total it all up. And then, I think he made a valid point similar to the incentives conversation we were having about post-quantum output types, that if we have the option for a ZKP of all the signatures. we should not also have the block format allow individual signatures, because that creates kind of a weird incentive distortion thing, where some users can disproportionately use resources by kind of forcing their full signature into the block, or a miner could kind of overuse node resources by not doing a SNARK when they should have. So, if we’re going to do this, we would want all the signatures to go into the single SNARK.

Then, the other thing he brought up is that unfortunately, anytime you’re adding new cryptography to a system, there’s a risk that the new cryptography is worse than the old cryptography in some way. And that’s a big part of the whole quantum conversation as well, and part of the reason that P2TRv2 exists; let’s use the existing stuff we know works until we know it doesn’t so that we’re not over-reliant on the new stuff. And so, he suggested that we would have a way to fall back out of this SNARK-proof block format if they were to be found vulnerable. It wouldn’t be a permanent addition initially, it would be, I’d say potentially, a temporary soft fork that eventually forks itself out if needed, similar to how we would fork out secp if needed for a quantum vulnerability. I know that’s a whole other conversation we’ve talked about on this podcast before. So, just worth saying, it’s similar that conversation of how you would handle adding new cryptography, and how Bitcoin as a whole system would handle the potential need to someday disable any cryptography it relies on.

There were some other kind of posters that replied to this about the quality of the proofs that are required for this, and how when you’re SNARK-proving this block, you’re essentially recursive on the SNARK proof of the previous block, and it’s not absolutely clear that that’s proven secure yet. And then also, that an abuse of a signature could now potentially delay a miner’s ability to produce block templates by slowing down the proving of the SNARK that needs to be embedded in the block template before it can be mined. So, that’s something else that has to be thought about if we’re going to add what’s called another layer to what miners have to do, right? Right now, miners have this pretty quick job they do, where they pre-select transactions, and then when it’s time to make a new block, they quickly grab the top block chunk to start mining on it. If we have a SNARK they have to produce, now they have to do work that might take several seconds before they can start mining on a new block. And if some user’s signature can make that work long, now you’re attacking a miner by sending them a transaction that delays their ability to start mining. All things to think about as we go down new cryptographic roads. Questions?

Gustavo Flores Echaiz: Very interesting. Well, thank you so much for explaining all that. Yeah, I have a few questions. So, I mean, even on the best-case scenario, you still have some major trade-offs right? So, you add an extra job to a miner, but you also complicate more the work of individual nodes that verify these SNARK proofs. Is my understanding correct that those are the trade-offs, or are there other trade-offs even in the best-case scenario?

Brandon Black: The main one is on the miner side. For the nodes, this actually will end up being a quicker verification, I think, even than secp signatures. The single SNARK proof that covers all of the post-quantum signatures of the whole block, verification of SNARKs is very quick. So, the nodes are going to actually get more efficient, it’s really on the miner. So, we essentially trade some miner work for some node work, which that, to me at least, seems like a reasonable trade-off. For the nodes, it’s really about a trust trade-off, where now the nodes can’t see directly, when they get a SNARK, all of the transactions that were in it in the same way they could. Now, a big powerful node could actually do that, hold all of the signatures, and rebuild the SNARK on its own. But I would expect most people running most nodes would only validate the SNARK, not re-prove it to themselves.

Gustavo Flores Echaiz: Right, understood. Okay, and that’s assuming that everything goes well with the SNARK cryptography, which as you mentioned, has some potential risks. I believe other cryptocurrencies have had issues implementing such cryptography, right, which has led to, like, unable to verify the supply of those coins; is that correct?

Brandon Black: Yeah, and of course, just to keep perspective here, Bitcoin would not be using a STARK or a SNARK, either of these zero-knowledge structures to prove the shape of the transactions themselves. This would only be proof of ownership, proof of authorization. And so, we wouldn’t have any kind of supply vulnerability, even if the SNARK was totally broken. We would have potentially coins moved, not by their owner, but we wouldn’t have any new coins or any double-spend or anything like that. So, this is a lower-risk thing than, like, a full Zcash implementation style thing. But there is still a trusted setup involved in some of ZKP structures, I think, that was talked about in this thread, if you want to get down into the details and read the whole Delving conversation.

Gustavo Flores Echaiz: Right, understood. Thank you for making that clarification. I think that wasn’t completely clear to me, but now it is. So, yeah, if listeners want to learn more about it, there’s a bunch of links and a conversation in Delving about this all. So, really cool stuff. Let’s move on to the next topic.

Bounds on chain length with BIP54 time warp fixes

Brandon Black: Yeah, keep going. So, the next one, we get out of quantum land, which makes me happy because, you know, that’s me! On to BIP54. This is a really cool result that Pieter Wuille developed. So, the topic is BIP54, the consensus cleanup. And one of the things that consensus cleanup aims to do is to fix this category of attacks on Bitcoin called time warp, where miners can manipulate the Bitcoin block time. Miners can do this because Bitcoin doesn’t have a true oracle to real time, right? The time on Bitcoin blocks is whatever the miners say it is within some bounds enforced by the protocol. In the current Bitcoin, those bounds are very wide, and miners can manipulate the timestamps significantly to affect the rate of block production. And that’s what Pieter Wuille went to look at, is right now, miners are essentially unbounded on how fast they can produce blocks by manipulating the timestamp. If we implement BIP54 and make it the consensus rules of the network, he proved that there is a provable bound on how fast they can produce blocks, even if they maximally abuse the new BIP54 rules. And that’s a very cool result, it means BIP54 works, it does what it’s supposed to. It kind of puts a bound that we can understand, in terms of the relationship between the real rate of block production and the rate specified by the protocol, of about one block every ten minutes. And it gets actually very close. I think the proof shows it’s within about 1% of the intended rate by more closely tying the timestamps to each other through the Bitcoin protocol and to the work. So, that’s basically the summary of it.

If I understand correctly, this prevents a malicious group with hashrate from creating a false headers chain that creates many millions of blocks and forces a new Bitcoin node to explore that chain in order to find out which the most-work chain is. Once BIP54 is the rule of the network above a certain height, the nodes know how many blocks a legitimate chain can possibly have and are never forced to explore many, many millions of blocks to find the tip. And so, that’s the attack that this defends against, in addition to just the general growth of the chain being a known rate versus there was actually talk some years ago that users may not have read about, where someone proposed rather than changing Bitcoin’s block size to get more transaction throughput, let’s just abuse the existing loose rules on block timestamps so we get more blocks; and that way, we get more throughput without actually changing the block size. And so, this closes both of those potential vulnerabilities In the protocol. But Pieter was focused on the headers one.

Gustavo Flores Echaiz: Right, that’s very clear.

Brandon Black: Questions on that one?

Gustavo Flores Echaiz: No. Just, so basically, this is not about a new idea. Instead, it’s about proving that the proposal is quite solid at fixing the vulnerabilities, the potential risks that could occur without them; is that correct?

Brandon Black: Yeah. And one thing I think is really cool about Pieter being able to go back and prove this is that in the development of BIP54, there was a huge amount of debate about these timestamps. And the reason there was all this debate about it was that they wanted to make sure that normal miners, without applying any BIP54 rules to their own internal stuff, wouldn’t accidentally violate it. They wanted the rules to be, okay, these protect from attacks, but they’re not going to kick off an old miner who didn’t upgrade to BIP54, who accidentally tweaks the timestamp a little bit, their clock’s off by five seconds. And so, even with all that conversation and relaxing, like, the original BIP54 had even tighter bounds. They relaxed it a little bit to make sure that we wouldn’t be kicking off miners unnecessarily, and it still does the job correctly. So, I’m very glad to see this result that matches kind of the theory that was done many months ago on BIP54.

Gustavo Flores Echaiz: Awesome. Great, thank you, Brandon.

Comparing covenant proposals for vaults

Brandon Black: All right. So now, the next one we have here is about using different covenant proposals to build reactive security vaults. I have to be a little bit careful. I realized when I wrote this, I didn’t include the words, ‘reactive security’ in the title. Unfortunately, the word, ‘vault’, for good reasons, is used everywhere in Bitcoin. In this item, we are talking about reactive security vaults, meaning a Bitcoin storage structure where if there’s an attack, an attacker gets the key that releases the funds, there’s time between when they can post a transaction and when it can actually happen, letting a user react to that before the funds are actually gone to the attacker. And so, Lillian Wang posted about this to the Bitcoin-Dev mailing list and Delving Bitcoin, about how different proposals compare in implementing these reactive security vaults. And basically, what she came up with was that for a simple vault structure, something like CTV (CHECKTEMPLATEVERIFY) or I added in TEMPLATEHASH, because they’re equivalent for this use case, it is kind of the right shape of opcode to implement simple vaults. And I think that matches what we’ve seen with both James O’Beirne’s older simple CTV vaults, and also ademan’s recent more complex CTV vaults proposals. Both of those show that for vault structures that are, let’s say, infrequently accessed, relatively simple, you want to have a low cognitive load of thinking about how they work, CTV or TEMPLATEHASH works really well for those things.

If you want to do vaults and get other stuff along with it, there are other proposals that might work better. If you need your vaults to have nuanced rules around coin access and which coins come out in which ways, then TXHASH is maybe better. And if you need your vaults to be more functional as an everyday wallet, where you can have change going back into the vault and you can deposit arbitrary amounts, and these kinds of everyday functionality for a vault, then something like CCV (CHECKCONTRACTVERIFY), which replaced OP_VAULT for this purpose, would be more suitable. But again, the overall conclusion is that simple vaults work well with CTV or TEMPLATEHASH. And that, again, matches what the rest of the ecosystem is seeing, but it’s cool to see someone going down the road and looking at each of the proposals and how they fit for that use case.

Gustavo Flores Echaiz: Totally. That’s very clear to me.

Brandon Black: One thing that came up that I added to the summary here and is worth mentioning, is there’s an interesting nuance, which is that SIGHASH_ANYPREVOUT, formerly the SIGHASH_NOINPUT proposal, can also be used for these things, but it has one additional risk versus CTV or TEMPLATEHASH, which I think is reflected in the fact that TEMPLATEHASH and CTV have had more mind share for several years now. They’re kind of tighter. They protect against more kinds of risks in the way they’re implemented.

Gustavo Flores Echaiz: Really cool ideas. It’s really nice to see the comparison between all these different covenant proposals, right, because sometimes it just feels like there’s a new one and a new one, and you kind of lose track of the multiple ones. So, if listeners are curious about how these proposals compare for building vaults, this is a great read.

Brandon Black: Yeah. And I think right now, at least, I think the mindshare in covenant proposal is around the TEMPLATEHASH plus CSFS (CHECKSIGFROMSTACK) plus INTERNALKEY proposal, BIP448. And that is, I think, where I would direct users. If you want to learn about one covenant proposal, look at BIP448. I would recommend looking at others because I love that stuff, but I’m not saying everyone has time to read everything. So, if you’re going to read just one, read BIP448 and it’s kind of three related BIPs. There’s a lot of work around that right now.

Gustavo Flores Echaiz: Makes total sense.

Depots for probabilistic Lightning channels

Brandon Black: Moving along to our last Changing consensus item, we have John Law’s latest proposal. John Law has over the years posted many remarkable proposals for Bitcoin protocol ideas, how we can use Bitcoin better as users. And this one’s called depots. And the idea here is how do we get everybody into the LN? He’s been working on that for a long time. He built timeout trees, which is in some ways a relative of Ark. So, this is another proposal that kind of works with something like BIP448 that we were just talking about. And this one allows potentially millions of Lightning channels to be produced from a single output on chain. And the way it achieves this, I think, is very clever. And that is that because each of these channels might be very small and the balance might not really be possible to resolve into an onchain output without all of it going to fees, for example – that’s one of the challenges with something like Ark or Spark right now, is that if you have a small Ark or Spark output, you try to go onchain, the whole output’s consumed by the fees to go onchain.

So, how does Depot solve this? It says, well, we don’t actually give every user an individual way to get out onchain. Instead, to protect users from a malicious depot operator, if the depot operator doesn’t honour the contract, the whole output goes to one of the users in the chain with a probability of one over the number of users. So, each user is assigned a random value, and you don’t know until the operator violates, if the operator violates, one of those will get the whole thing. And so, every user has a net expected onchain value, in the event that the operator violates, of their proper value. In reality, one user would get the whole thing. So, it’s an incentive-compatible structure where if it ever were to actually go onchain, there would be this weird thing where one user gets a windfall and the others get nothing. But this hopefully keeps the operator honest so that never actually happens. As long as the operator’s honest, everyone gets their own correct values out, and the operator rolls them forward kind of similar to an Ark, where it rolls forward over time by sweeping the old channels into a new depot. According to Law, this structure would let us keep everyone, the whole world, in Lightning for the cost of about 1 or 2 vbytes per user per year, which if you think about it, that’s enough that we could really actually put every person on the LN. It’s very inexpensive to do it this way.

Then, yeah, there’s one last little item that was talked about, which is that there’s a difference here between Ark or timeout trees, which is that the users can’t be automatically rolled into a new depot like they can in a timeout tree or an Ark, they need to participate. And so, if users lose their stuff, they might lose their value in an old depot, rather than being able to be recovered into a new one. So, just a little tidbit of difference. I think it’s a very interesting construction. It’s another way where if we have some relatively constrained covenant addition to Bitcoin, it’ll make a material difference in our ability to serve users with self-custody on Bitcoin. So, great to see more applications for these different covenant proposals coming forward.

Gustavo Flores Echaiz: Really cool stuff. So, is my understanding correct that this is similar to Ark, where a user has to come online, for example, every two weeks, every 28 days?

Brandon Black: Yes, it has similar properties to Ark there where they will roll over periodically. If I understood it correctly, however, if users don’t roll over, they can be just automatically put onchain for relatively low cost compared to Ark. So, there’s like a trade-off where you can’t be automatically rolled into a new one, but you can be put onchain more affordably, if I understood correctly. Don’t quote me on that though, it’s a little complicated.

Gustavo Flores Echaiz: Right. But also, at the same time, it also says that the balances are typically too small to be worth claiming onchain, right, that’s the whole point?

Brandon Black: Fair, yeah.

Gustavo Flores Echaiz: Right. This is a sort of like, I cannot claim back my funds because they’re too small to be claimed back, but I can have a chance at winning big, right, at getting the whole value.

Brandon Black: If the operator is malicious, you have a chance of winning big, yeah.

Gustavo Flores Echaiz: Interesting proposal. Definitely something I had never thought of. So, definitely cool to see the depots idea. So, we’ll see where this goes. Hopefully, there’s feedback to the conversation. I think people have already left some feedback. So, if listeners are interested, they can also jump in into Delving to drop some additional feedback.

Brandon Black: Absolutely. And that’s all for Changing consensus this month.

LND v0.21.4-beta.rc1

Gustavo Flores Echaiz: Awesome. Well, thank you so much, Brandon, for being here and helping us out with this section. Very cool stuff. Now, we’re going to jump into the Release sections and we’re going to finish with the Notable code and documentation changes. So, this week, we’ve got two release candidates on the LND side, v0.21.4 and v0.20.5. Both of these are maintenance releases. So, actually, when we wrote the newsletter, they were both at the RC level. But now, since precisely the day of where we published the newsletter, now they’re fully in beta. So, these are official maintenance releases. There are no longer testable RCs. Users are recommended to upgrade for security reasons, and they include multiple fixes. We cover some in the Notable code section below, specifically the ones related to channel announcement synchronization and restriction on new legacy channels. This release, v0.21.4, also requires explicit channel types when opening channels. So, you can no longer assume a channel type, you always have to specify one when opening a channel.

There’s other fixes related to pending HTLCs, unintended cancellation of AMP invoices, and SQL graph migration failures. So, if listeners want to learn more about the pending HTLCs issues or the AMP invoices, we covered that precisely in Newsletter #424, so just the previous one last week. And finally, v0.21.4 also has a new feature where WalletKit can now reserve outputs until their spending transactions reach a specified confirmation depth. So, this is a mechanism to block other transactions from selecting these UTXOs.

LND v0.20.5-beta.rc1

And v0.20.5-beta is very similar to v0.21.4. It backports several fixes, including those for pending HTLCs, AMP invoice cancellation, and channel synchronization. But it also adds bounds on onion payload parsing. And the bounds on onion payload parsing was already a fix that was included in version v0.21, so this was simply backported to v0.20.5. So, once again here, the action item is to go and update your LND node if you are using LND and haven’t updated yet, similar to the action item that Matt told us about Eclair, and in general, right? Lightning is a very complex system, so it potentially can have multiple vulnerabilities. So, it’s always very important to stay up-to-date across all implementations, particularly in this moment in time we are where it seems that AI models are becoming more and more powerful and more able to find new vulnerabilities.

Bitcoin Core #29278

We move to the notable code and documentation changes section. So, here, Brandon, if something’s not clear, please let me know; if you have anything to add as well. So, we have multiple items in the Bitcoin Core repo, about seven of them. The first one, #29278, this is a new configuration option called -maxfeerate that caps the fee rate of wallet transactions at 10,000 sat/vB (satoshis per vbyte) by default, but this is of course configurable. It was previously thought that this was possible, or at least there was some confusion that -maxtxfee, the existing config option, although it was documented as an absolute fee cap, so not a per-vB fee cap, but rather just an absolute fee, some code was already interpreting that existing option as a fee per 1,000 vB. So, -maxtxfee, the existing option, was playing both roles accidentally. So, now, this PR not only introduces the new config option, but also ensures that the previous option only serves its intended purpose of being an absolute fee cap rather than a feerate cap. And the new option separates the feerate limit from the total fee limit and applies to transaction creation, fee bumping, CPFP, and regular wallet broadcasts.

Brandon Black: This is such an important change to avoid fat-finger loss of funds. We’ve seen many cases over the years in Bitcoin of massive fees. There’s one notable case where I think a miner returned like something like 80 bitcoin worth of fees to the accidental spender. We shouldn’t be counting on miner goodwill, we should have better software, so this is great.

Bitcoin Core #35984

Gustavo Flores Echaiz: Totally agree with you. Let’s jump on to the next item #35984. So, this is a bug when signing PSBTs, when using SIGHASH_SINGLE. So, SIGHASH_SINGLE signs an input to make it commit to the output at the same index as the input being signed. So, if I sign input number two, it will commit to output number two; and if I was later to change output number two, then the signature would simply be invalid then. However, if the corresponding output is missing, legacy signing, so pre-segwit signing, produces a signature over a constant hash. The problem is that this signature can be reused to spend other UTXOs controlled by the same key. So, when legacy signing with SIGHASH_SINGLE when there’s no corresponding output, I can then later on switch the UTXO if it’s controlled by the same key and the signature will still be valid. So, it doesn’t specifically commit to the UTXO being signed, it only can commits to the key being signed and the position in the absence of the corresponding output. Very important to say that this requires both the corresponding output to be missing and also legacy signing, because this doesn’t affect legacy segwit v0 or taproot. So, the problem was that anybody could change the UTXO and the signature would still be valid.

So, now, this bug is now fixed, where Bitcoin Core now leaves such inputs unsigned. So, it will refuse to sign if there’s no corresponding output, because the whole point of SIGHASH_SINGLE is to commit to the corresponding output. So, Bitcoin Core will refuse to sign precisely this input. However, it will still sign the PSBT’s other inputs. And this was a check that was already present in raw transaction signing, but it has now been extended to PSBT signing as well. If someone wants to learn more about the SIGHASH_SINGLE bug in legacy signing, we covered in Newsletter #207 how Rust Bitcoin fixed the occasion of that bug. So, you can also learn more of how Rust Bitcoin implemented it back then. But that specific item is also linked to a coinspect.com blog article that explains further how this bug or incident worked.

Bitcoin Core #35301

Next item, Bitcoin Core #35301. This is a very exciting item, because it’s the beginning of the implementation of BIP352, also known as silent payments, inside Bitcoin Core. So, now, this doesn’t mean that you as a user can already use it through the RPC or through the GUI. But now, Bitcoin Core has support for encoding and decoding silent payment addresses; it can derive taproot payment outputs from eligible transaction inputs; and it can also scan transactions for payments to a recipient. This also adds support for labels, which are a way to distinguish payments to different derived addresses and identify change when using silent payments. So, this of course builds on libsecp256k1, a silent payment module that we covered in Newsletter #415. So, it’s really cool to see Bitcoin Core finally implement silent payments after the module was added to libsecp256k1 a few weeks ago. So, really exciting news, and we’re hopeful that this will conclude into usable features soon. But if I look at the BIP352 tracking issue on the Bitcoin Core repository, I can see that the pre-works and the refactors have all been done, and now this major one was implemented. So, there remains a few steps. But we’re definitely getting closer to full implementation after the tracking of that issue was started more than three years ago. So, this has been a long way coming, and it’s now finally getting to Bitcoin Core.

Bitcoin Core #36312

Next item, Bitcoin Core #36312. This is about a fix in the experimental opt-in private transaction broadcasting feature. So, we covered the announcement or the addition of that in Bitcoin Core in Newsletter #388. To remind listeners, this is a feature that allows a user to broadcast transactions to short-lived Tor or I2P connections, or to the Tor proxy to IPv4 and IPv6 peers, when using sendrawtransaction. So, this allows you to temporarily conceal your IP address and it protects the privacy of you as the transaction originator when your node broadcasts your transaction to your network peers and to the Bitcoin Network. However, there was a bug when if there was a misbehaving peer that you would like to discourage from your node, so you would like to disconnect from this misbehaving peer, you would previously disconnect both the private broadcast connection and your regular Bitcoin Core node connection. However, the problem was that a malicious peer could deliberately trigger these disconnections to make sure they can observe that both of these connections disconnected at the same time. So, they can then assume that you are the one behind both connections, and that weakens considerably transaction origin privacy.

So, now, in the case where you would discourage misbehaving peer connection, in the regular use, it will leave private broadcast connections intact, so it wouldn’t disconnect them separately. However, if you’re in the private broadcast mode and you want to disconnect the peer, it would do so without discouraging their address from your general connection. So, it either happens in one, it never happens in both at the same time to avoid a malicious peer from correlating both addresses back to your node. And if listeners want to know more about what it means to discourage a misbehaving peer, the Newsletter attaches a reference back to Newsletter #106, where a documentation change in Bitcoin Core was made to clarify the distinction between manual peer banning and automated peer discouragement. So, you can learn more about that. But basically, this is what your node does when it basically finds that a user is misbehaving, and it will not ban it, but it will discourage it by simply disconnecting once, which eventually, the peer could connect again to your node, so it doesn’t fully ban it; it simply disconnects.

Bitcoin Core #36284

Next item, #36284. This is another bug fix in Bitcoin Core where, when using the option -avoidpartialspends or the wallet’s avoid_reuse flag, your wallet could reject the payment despite you having enough funds to pay for this. So, what is -avoidpartialspends? When you use the -avoidpartialspends function, your wallet will by default spend all outputs received to the same address. So, even if you don’t need to select all the UTXOs that were received to the same address because the amount can just be paid with simply one, it will still select all these UTXOs so that they can be consolidated and they can be moved to a change address. So, this is a privacy feature. However, when using this feature or the wallet avoid_reuse flag, where the wallet avoid_reuse flag prevents the wallet from spending bitcoins received to an address that the wallet had already used for spending, with this one, it’s not your wallet spending multiple UTXOs; when choosing coins for a transaction, your wallet will make sure to never select a UTXO that was received in an address that you had already previously spent from. And this is once again a privacy protection, but potentially it could even have further implications when we consider quantum computing as a potential attack. This is also a protection around creating multiple signatures.

Anyways, the point is that when using this flag or the previous described option, if a group failed eligibility check, for example limit on unconfirmed ancestors, its value was subtracted twice. So, for example, you have 4 UTXOs of 1 bitcoin each. You need to spend 3.5 bitcoins. You have 3 bitcoins that are fully confirmed, so you select those. Excuse me, that’s not it. If you want to spend, for example, 2.5 bitcoins, and you have 3 bitcoins that are fully confirmed, but you have one that is not confirmed yet, so it fails the eligibility check, it gets subtracted twice. So, from having 4 and from having 3 available, instead it gets reduced to 2 available, because the one that was unavailable gets subtracted twice, where it shouldn’t, it should only be subtracted once. So, the fix is to make sure that it only gets subtracted once when using this specific flag or options. That was the only occurrence where such a bug could occur, and the worst outcome was simply your wallet rejecting a payment despite having enough eligible funds. And if, for example, that UTXO would confirm, then the issue would have disappeared, so this was a temporary issue and not that consequential.

Brandon Black: It almost seems like we should be just not allowing spends with -avoidpartialspends if there are any candidate UTXOs that match the set that are unconfirmed, because now we have a UTXO that is going to the same address and isn’t getting included in the full spend of that address. I feel like the subtraction is wrong. I don’t know, I almost want to make a new PR to Core that kills off this fix by saying we just don’t allow that.

Gustavo Flores Echaiz: You think? Maybe you could potentially propose that. But why do you think that? Can you share more?

Brandon Black: So, the idea of -avoidpartialspends is that when you spend coins that were sent to an address, you spend all the funds sent to that address. If there’s an incoming unconfirmed payment to that address, you shouldn’t just subtract that from the candidates and see if the group still fits. You should say if there’s unconfirmed, this whole group is not eligible for -avoidpartialspend, because now you’re going to end up with a UTXO, sort of, at that address again.

Gustavo Flores Echaiz: Right. I think that’s a misunderstanding, okay, so maybe I mis-explained myself. But it’s not a UTXO that is related to the group targeted by -avoidpartialspends, it’s a completely unrelated UTXO. It’s a UTXO that is in a completely different address.

Brandon Black: Oh, I see, it’s because the group failed eligibility checks?

Gustavo Flores Echaiz: Exactly. So, it doesn’t have anything to do with that. It’s just a completely random group that was unconfirmed and its value was subtracted twice.

Brandon Black: Okay. Definitely no problems and good bug fix. Got it now. Thank you.

Gustavo Flores Echaiz: Right, okay. Yeah, no, that was good. Thank you for saying that because I did run into that doubt when I was writing this. So, it’s good that you brought that up so that it’s fully clear for listeners.

Bitcoin Core #35752

Next item, #35752. This is another bug fix. This is actually quite an interesting and big one because it’s about eight different potential bugs in one PR item. So, this PR fixes error handling when encrypting a wallet, changing its encryption passphrase, or adding private keys. So, the recurring problem here is that errors weren’t properly handled. So, for example, you would think that you change your passphrase because it would appear as successful, but in reality the database write would fail. However, Bitcoin Core wouldn’t give you that feedback of that error, it would simply make you think that the passphrase update was successful, and later on, only the old passphrase would work. So, your passphrase change completely failed, and Bitcoin Core didn’t properly inform you about it. It made you think it was successful, and you could have potentially destroyed the previous passphrase and you would have gotten locked out of your funds. I want to be precise here, when I use the word, ‘encryption passphrase’, I’m talking about specifically the password that you use on Bitcoin Core when encrypting or decrypting a wallet file. So, I’m not talking about BIP39 passphrases, so it’s important to make that distinction.

So, that’s the first case where it would fail. But there’s multiple others, such as when encrypting your wallet for the first time, it could also report success, despite missing required key records or leaving plain text private keys in the database. So, once again, the encryption process makes you think it’s successful when in reality, it isn’t. And there’s multiple edge cases like that. So, the next one, for example, when you add a private key, failed private key insertions could leave keys only in memory, causing them to disappear from the wallet upon reloading. So, you thought you added a private key to your wallet, but it gave you some feedback that it was properly added, although it wasn’t. And there’s multiple other edge cases. The point here is that this PR now prevents basically these errors from silently happening, right? It will give the feedback, the correct feedback, and it will properly handle those errors so that users are fully aware of what was successful and what wasn’t when setting up wallet encryption or making a passphrase change, or even adding a private key to their private key set.

Brandon Black: Are we aware of any loss of funds from any of these issues, or were these all found by doing some code analysis?

Gustavo Flores Echaiz: Yeah, I don’t think we’ve seen any. There wasn’t a reported issue of a user. This was the developer himself, l0rinc, if I’m spelling correctly, finding this himself. Hopefully, no loss of funds have happened. And probably this required multiple things to occur at the same time, right? It is potentially possible that users haven’t run into this bug themselves, but I guess we’ll know more eventually. Sometimes these things are not fully disclosed at first and only later we know the full details. But yeah, potentially this was a consequential bug, so I’m really happy this was now fixed.

Bitcoin Core #35813

The next item, Bitcoin Core #35813. So, this is a new wallet RPC, called listrawtransactions, that lists every transaction known to the wallet and it returns one entry per transaction. So, the difference with listtransactions, that was an existing RPC, is that listtransactions returns accounting entries. So, for example, a self-transfer to a receiving address appears as both a send and a receive, so it’s a double entry. It got sent and it came back into my wallet, but in reality it’s just one transaction that appears twice, while a transfer entirely to a change address would simply not appear in listtransactions. It would be considered as fully an internal operation that shouldn’t be presented with the listtransactions RPC. So, now, listrawtransactions will actually deliver every known transaction for the wallet and return one entry per transaction, not potentially two or zero. And you can also use the count and skip parameters for pagination, while the verbose argument adds decoded transaction details.

BIPs #2276 and #2277

The next item, now we’re getting into the BIPs repository. So, this was a fix in PSBT finalization rules that could discard information needed for later signing or transaction extraction. So, the first one is in BIP376, there was a requirement to delete PSBT_IN_WITNESS_UTXO. The BIP376 is the specification about how to spend silent payment outputs with PSBTs. And there was an instruction that when finalizing a silent payment input, you would delete PSBT_IN_WITNESS_UTXO, which references the prevouts, or the previous outputs, that are being spent in the transaction, so all the inputs of the transaction. However, the problem was that if you would discard that information, other taproot inputs would be unable to compute their signatures, because their signatures required the information of all inputs of the transactions, or of all previous outputs that are being spent. So, now, that instruction is now removed from spending silent payments in PSBT’s specification, BIP376, so that other taproot inputs in the same transaction are able to compute their signatures from the information of all previous outputs. So, that was the main fix, and that’s in PR #2276.

The second one, PR #2277, updates BIP370, so a different BIP specification, which is the one defining PSBTv2, which is now the standard version, as of Bitcoin Core v32, which we’ve only seen a RC of, not yet the full release. So, in this, the second PR updates this specification BIP370 to retain previous output identifiers, sequence numbers, and required locktimes after a PSBTv2 input is finalized. So, previously, when finalizing a PSBTv2 input, those fields could get discarded, although deleting those fields could have invalidated the PSBT or altered the extracted transaction. So now, the specification is updated to avoid that issue. The second PR also updates BIP371, which is the specification for taproot fields for PSBT. And basically, what happens is that an erroneous instruction to delete output taproot derivation data during input finalization is also removed from BIP371, probably for similar reasons as the previous issues, where it could either invalidate the PSBT or block signing of other inputs. Anything to add here? No? Perfect.

LDK #4993

We jump in now into the Lightning implementations. We have one from LDK and multiple ones from LND. So, the first one is about LDK. I want to say that LDK PRs are now on a different repository, no longer on a GitHub, but instead on git.rustbitcoin.org. So now, those PRs will link to this new repo. So, PR #4993. This is a bug that could cause a wallet to spend a reserved input in a transaction conflicting with an unconfirmed splice. So, this occurred if LDK automatically would reject an RBF fee bump because the channel had already been force-closed, it could instruct the wallet to release inputs still needed by the original splice. So, for example, imagine you’re in a channel, you want to splice the channel, but at the same time, you probably even broadcast the splicing transaction, but then you see on the blockchain that your peer has broadcasted a force-closed transaction for that channel. So, while these transactions are in the mempool and haven’t yet confirmed, an RBF fee-bump attempt might trigger on your splice transaction. However, LDK will reject that RBF fee-bump contribution because it will observe that the channel is already in force-close progress.

However, this means that LDK would then instruct the wallets to release inputs still needed by the original splice. And then, another transaction could have grabbed the inputs that the original splice was using for the splice-in, the additional inputs that were going to be added into the channel. And even if there were successive fee-bump attempts, this could also receive duplicate release instructions, or even keep reservations after they were no longer needed. So, one RBF attempt made clear that it would release the inputs of the splice, and other transactions could potentially try to grab them. But if there were multiple successive fee-bump attempts, then the behavior could become even weirder, where duplicate release instructions could occur, or some reservations could apply for longer after even all these transactions had confirmed or been discarded.

So, now, each funding contribution records which inputs and outputs it inherited from earlier attempts. So, failure tells the wallet to release only the contributions on reservations. You never have to worry about your wallet accidentally releasing inputs that an unconfirmed transaction is using with this bug fix.

LND #11173

Next item, and now we get into the LND repo, which like I mentioned in the release section, many of these items were included in the latest release. So, the first one, #11173, this is a bug fix where an invalid or oversized channel range response could stall the initial sync of channel announcements until the next scheduled attempt. So, what does that mean? In Newsletter #417, we covered how LND added a protection, amongst what other implementations have been doing, adding more and more protections lately. And LND basically added a limit of a maximum of 100,000 SCIDs per reply channel range message. So, the goal of that was simply to bound the size of that message. So, a user that was attempting to send a very large number of SCIDs would simply get blocked at 100,000, because I believe the LN today is around 60,000. So, someone sending you more than 100,000 is obviously sending you incorrect data. The attempted goal was simply to block that message. However, the bug was that LND would receive, for example, that message with more than 100,000 SCIDs, and it would simply stop syncing. It wouldn’t look for another eligible peer, it would simply stop syncing, but LND would have a scheduled attempt system where an hour later, it would resume syncing. So, it wasn’t like it would forever stop syncing. It would simply stop syncing for maybe like one hour or less.

However, now, the fix is if there’s another eligible peer, which there almost always is, LND will simply retry continuing the synchronization with the other eligible peer and temporarily exclude the failed peer from selection. And the existing connection to the failed peer would still remain open, so it doesn’t disconnect from the peer, it simply stops selecting him for concluding the channel announcement synchronization.

LND #11190

Next item, LND #11190. Here, LND is updated to reject BOLT11 invoices containing multiple payment hash fields, even when the hashes are identical. Okay, so this is a very interesting one, because this was actually a major bug, and so I didn’t write it here precisely. But I found out about this news later when making my BitDevs presentation in the monthly BitDevs meetup we have in Mexico City. And the thing is, I don’t know if you remember, Brandon, when lnp2pBot declared they had shut down about a month ago, they disclosed that it was because of this issue. So, the invoices in a library by Alex Bosworth had an issue that was possible because of this missing part in the spec. So, first I’m going to talk about the issue.

So, LND now will reject BOLT11 invoices containing multiple payment hash fields. Before, it wouldn’t, okay? So, as a receiver, I could send a sender a BOLT11 invoice with two payment hashes, and LND previously would accept it, and it would use the first payment hash and would ignore the rest, as it’s recommended by the BOLT11 spec already. However, other invoice parsers, such as the library invoices that I mentioned earlier, would choose a different hash. And there would be a sort of incompatibility between LND and the other invoice parser, where a Lightning node could basically get fooled into thinking that it didn’t pay a payment because it would say that, “Well, I paid this”. Well, the user would say, “Well, this is the payment hash of the transaction”, because he would choose the second one, and LND would only identify the payment to the first one. And if a system was using both LND and the other library, it could create a sort of issue where it would think it hadn’t paid it, but it had, and it could potentially trigger a second payment to that user.

So, now, LND simply rejects duplicates to remove fully that ambiguity. And roasbeef, the one behind this PR, also opened a BOLTs PR #1357, where he requests to modify the spec to reject an invoice with duplicate payment hashes because, like he says, it was a recent hack of a popular Telegram p2pBot that exploited this ambiguity, which is lnp2pBot. Some libraries’ invoices, the library that I mentioned, took the first payment hash while some took the last. So, LND took the first and it seems invoices took the last. And if you were running a system that exhibited both variants of the behavior, then your system could get tricked into thinking it never paid out a withdrawal. So now, hopefully, the specification will get updated to include this change, and we won’t be having these type of issues eventually. But also, even if that isn’t added to the BOLTs specification, LND has already covered themselves by rejecting any BOLT11 invoice that contains multiple payment hashes. So, at first, it seemed not that consequential, but now that I’ve looked into the details, it was pretty consequential. Anything you want to add here, Brandon?

Brandon Black: No, I mean, it’s interesting. Do you know why the invoices allow multiple payment hash fields in the first place?

Gustavo Flores Echaiz: Right. It probably seems like this was a misunderstanding, because it does allow. But it always said, like, a writer shouldn’t include, but it never said a reader must fail the payment if more than one p field is present. So, how the BOLT specification works is that sometimes it will define things for a writer different than for a reader, and this was one of those cases where it was doing that. And also, for example, it was saying a reader should use the first p field as the payment hash, it didn’t say ‘must’ use the p field as the payment hash, right?

Brandon Black: Oh, that good old RFC language distinction, ‘should’ versus ‘must’? Absolutely.

Gustavo Flores Echaiz: Exactly. So, yeah, unfortunately, the language used and the omission on the reader side led to this issue.

LND #11212

Next item, LND #11212. This is about removing support for opening or accepting new channels using the legacy commitment format. So, this is what I described earlier in the Release section. So, in Newsletter #305, we covered how this was already removed from the BOLT specification, and LND simply had kept support ongoing and now has fully aligned with the BOLT specification by removing support for opening or accepting new channels. However, if you have existing legacy channels, they remain usable with LND. The problem with legacy commitment formats is that they derive the key for the output paying the counterparty, using a per-commitment point, which means that if your node loses its channel state, recovering funds from its peer commitment transaction therefore requires that peer to supply the missing point. New channels, static remote key channels, keep the output key unchanged across channel updates. So, from your wallet BIP39 seed phrase, you can recover the output key, because it’s unchanged across channel updates. You don’t need the dependency of the per-commitment point. And in Newsletter #67, which is referenced here, we covered how LND added support for the option static remote key, which is the preferred commitment format.

So, LND had had already, for a long time, support for this, but it had kept ongoing legacy support for new channels for longer than necessary. So, now, this PR fully makes that change. And also, Eclair previously, earlier this last year, so almost a year ago, on October 31, 2025, we covered in Newsletter #378 how Eclair had also made that change. So now, LND does as well to avoid users to potentially lose funds by using the legacy commitment format.

LND #11258

Finally, the last item of the section in the newsletter is #11258, which fixes a bug where forwarded HTLCs could remain unresolved if LND restarted after the outgoing channel was closed and its records cleaned up. So, here, it’s a classic situation of an HTLC getting forwarded. The downstream node forwards one HTLC to me, I forward a similar value HTLC to my upstream node, however my upstream node fails the HTLC and closes the channel immediately. And if, for example, I was unable to fail back the HTLC into the incoming channel because, for example, the downstream node was offline during that time, for example, so I was unable to fail back the HTLC, if LND would restart as well – so, it requires multiple things to happen at the same time. The upstream node closes the channel, my LND node restarts, and the downstream node is unavailable to receive the HTLC failed message, that leads to a situation where LND simply kind of forgets that it has to fail back the HTLC to the downstream node, because it has sort of cleaned up the record of the channel that is now closed. So, it kind of doesn’t remember that the upstream peer failed back the HTLC, so it cannot tell the incoming channel of that failure and also fail back the HTLC. That would leave the incoming HTLC stuck, and it would force the downstream node to force-close the channel in order to be able to claim back the funds onchain.

So, now, the fix is for LND to save pending responses separately from the closed channel state. So, even if a channel gets closed, it will remember that it still has this response pending, and it still has to fail back the HTLC to the downstream node. So, it now retains the information needed to match them to incoming HTLCs, and can also replay them on startup, so it doesn’t keep this in memory, but actually in disk, so it persists across startups.

That is the final item of this section and that completes the whole newsletter. Anything you want to add, Brandon, before we finish?

Brandon Black: I’m just going to echo what Matt said earlier, which is, update your software. It’s a wild time in security-sensitive software and we all deal with money regularly. Probably a lot of listeners maintain Bitcoin nodes or Lightning nodes, or whatever. Update your software. That’s my closing message for this Optech podcast.

Gustavo Flores Echaiz: That’s a great message, Brandon, completely agree. Thank you so much, Brandon and Matt, for being present. Thank you, listeners, and we’ll see you next week.

Bitcoin Optech Newsletter #424 Recap Podcast

2026-09-29T00:00:00+00:00

Gustavo Flores Echaiz and Mike Schmidt are joined by Ahmet Kurt and Jan B to discuss Newsletter #424.

The Bitcoin Optech Podcast and transcription content is licensed Creative Commons CC BY-SA 2.0

Download audio

News

● Proposal for a post-quantum Lightning Network (1:10)

Selected Q&A from Bitcoin Stack Exchange

● What would be a drawback if sum instead of SHA256 of amounts was used in the taproot signature message? (36:18)
● Post-BIP110 fork is it necessary to resync from block 0? (39:32)
● Can a Bitcoin node build a partial UTXO set from only the most recent blocks and use it to validate new transactions? (42:16)

Releases and release candidates

● Bitcoin Core 32.0rc2 (21:13)
● Core Lightning 26.06.8 (45:36)
● LDK v0.3-rc2 (47:15)

Notable code and documentation changes

● Bitcoin Core #34566 (51:31)
● BIPs #1951 (54:14)
● BIPs #2224 (57:45)
● Core Lightning #9507 (1:02:00)
● Core Lightning #9508 (1:07:35)
● Core Lightning #9509 (1:12:02)
● Core Lightning #9510 (1:16:03)
● Core Lightning #9513 (1:20:01)
● LND #11198 (1:21:47)
● LND #11146 (1:24:15)
● LND #11132 (1:27:20)

Transcription

Mike Schmidt: Welcome, everyone, to Bitcoin Optech Newsletter #424 Recap. This week, we’re going to be talking about PQLN, which is a proposal for making offchain parts of Lightning Quantum secure; we have Bitcoin Core 32.0 release candidates, we covered RC2 this week and its associated testing guide; we have a monthly segment on selected Q&A from the Bitcoin Stack Exchange; and our Notable code and Release segments. This week, it’s Gustavo and I, no Murch, and we are joined by two special guests. We have Ahmet Kurt, who will be joining in a first-for-Optech pre-recorded segment that we’ll put in when this is published and edited as a podcast, and we have Jan B. Jan B, who are you?

Jan B: Hi, thanks for having me. I’m janb84, I’m a Bitcoin Core contributor, among other Bitcoin-related stuff. My main focus is reviewing PRs and working on kernel-related stuff.

Proposal for a post-quantum Lightning Network

Mike Schmidt: Awesome. Well, thank you for taking the time to join us today. We’re going to jump into our only News item titled, “Proposal for a post-quantum Lightning Network”. And as I mentioned, we’re trying something new this week. Ahmet Kurt couldn’t make the live recording, but I thought it would be interesting to try a recorded segment for us ahead of time, walking through his proposal in his own words. So, here’s Ahmet.

Ahmet Kurt: Hello, everyone. My name is Ahmet Kurt. I am an assistant professor at East Texas A&M University. I would like to thank Mike for inviting me to the podcast. Unfortunately, though, I’m not able to attend it live tomorrow. Instead, I’m doing a recording. The reason is, I am at a workshop in Washington, DC, and my return flight to Texas is coinciding with the podcast. So, Mike suggested that I record it instead and then we go from there. So, I’m excited to be here first of all, and I would like to introduce myself a little bit for people that do not know me. And then, I’m going to introduce the PQLN and all that stuff.

So, my Bitcoin journey started around 2017 with me hearing about Bitcoin, the exchanges, like, you can make money with it, and all that. But my official academic journey started in 2019 with my PhD. My PhD is from Florida International University, which is in Miami, Florida. When I first started, really I wasn’t working in Bitcoin or Lightning. My PhD advisor actually knew about Lightning surprisingly, and he suggested that I do a small implementation for covert communication using LN, which I did, and it was successful. Then he was like, “Hey, do you want to change your topic to LN stuff instead? Do you feel more excited about it?” I was like, “Yeah, for sure, I’m more excited about Lightning”. So, that’s how it started. Then, my PhD took four-and-a-half years, and during this time I did some cool projects. I’m really proud of most of them, fun stuff like designing and evaluating these LN applications. You’re actually welcome to look at them on my Google Scholar. To find my Google Scholar, probably you can just type my name there, or you can go to my website, which is ahmet-kurt.github.io, then you can find all sorts of information there. So, right after my PhD, I got this assistant professor job at East Texas A&M, and I am still continuing to do active LN/Bitcoin/cybersecurity research.

Today’s topic is about PQLN, which stands for Post-Quantum security for Bitcoin LN’s offchain surfaces. So, this particular one exploded on X basically, which is quite a bit of a shock to me, because I put the paper on arXiv first, then I posted on Delving Bitcoin about it. Nothing really happened, but I guess people picked it up later on. I actually DMd a couple people to get some feedback, then it just became actually trending on X. And Mike and his team on Bitcoin Optech also featured the work on their newsletter. Okay, I think this should be enough introduction and probably, I should now start talking about PQLN itself. This idea, making Lightning post quantum came to our heads around February 2025 with my co-author Abdul. He’s a post-quantum researcher, you know, very experienced guy. He’s an assistant professor as well. He was like, “Why don’t we try to make Lightning post quantum? Your expertise is in Lightning and my expertise is in post quantum. Maybe we can join them and do something”. I was like, “Yeah, this sounds great. Let’s try it”. Well, we tried, but we figured out, oh my God, this is too much and we might not be able to do this at all.

At first, I felt overwhelmed. I worked on modifying the Lightning protocol before, but it wasn’t as heavy as this one. This is particularly like there’s just too many moving components in Lightning that has to be made PQ. So, around December 2025, Abdul and I started implementing it using Rust-Lightning. We figured Rust Lightning is kind of the most suitable one to start such an implementation. Yeah, I think it’s pretty modular and there are some benefits overall using Rust-Lightning, I think. And around April, I actually kind of started working on it more heavily, like spending every day maybe a couple of hours actively working on it. It took around four months, I think, the total implementation. Then, you’re going to write the paper, because I’m an assistant professor, I have to publish papers, that is part of our job. We cannot just put the GitHub repo out there and not publish a paper. Probably writing the paper took, I think, like one-and-a-half months or something, which is also not an easy thing to do for a complicated project like this. We are targeting journals and conferences for works like this. They usually have a page limit. And in this particular work, the venue that we submitted it to had a page limit of 13. That’s why the paper is 13 pages by the way, in case you’re wondering. It could have easily been extended to 30 pages, I think, or I don’t know, 20 pages at least. We just cannot do that. Unfortunately, I have to follow their page limit. This is also not necessarily a cryptography paper. I know in cryptography papers, people just publish 30-, 40-page papers and put it on IACR website, I think. But in our field in computer science, we usually put it on arXiv and you all probably know about these things, but I just wanted to kind of go over them regardless.

Yeah, all right. So, let me actually dive into the technical part a little bit. Okay. So, why make Lightning PQ? You will ask, “Hey, we cannot make Bitcoin PQ. How can you make Lightning PQ without making Bitcoin PQ?” Yeah, you’re right, but you also need to think about the fact that even if, let’s say, Bitcoin gets its post-quantum upgrade today, Lightning won’t necessarily be fully PQ, right, because Lightning has its P2P messages, gossip, transport, invoices, offers, payment onions. These are Lightning’s offchain services which won’t necessarily benefit it from Bitcoin’s PQ upgrade. Therefore, we could make Lightning PQ right now. Also, another motivation to make Lightning PQ right now is that people can record Lightning traffic to some extent, maybe not the whole network itself, but they could record some portion of it with the hopes to decrypt it later when there’s a sufficiently powerful quantum computer. These motivations are I think good enough to start working on it as a prototype. I have to let you know that the implementation and the approach might have issues, but we tried our best. We tried to audit the whole thing and I believe personally it is a sound and a good implementation overall. There shouldn’t be any big obvious flaws, in my opinion, but there could be, so you’re most welcome to give us feedback on that either on our GitHub repository or in the Delving-Bitcoin post. Maybe reviewers will also find something. But I will tell regardless about our approach.

It starts with the gossip, okay? So, we had to distribute the PQ keys. And because now the nodes will have their classical node identities plus the PQ node identities, and the distribution of PQ node identities, we figured could be happening through gossip. I think it’s a sound approach. People will know each other, their PQ node identities over gossip messages. In gossip, we made node announcement, and what else is the other one? Well, there are three gossip messages. We made two of them PQ and we left the channel announcement classical on purpose, because channel announcement has two of its signatures dependent on the onchain funding keys. Therefore, making two of them PQ and leaving the other two non-PQ, or classical, would probably not bring much benefit, so we leave the channel announcement alone. Next one is the transport layer, which is BOLT8. Basically, this is kind of, I think, straightforward and probably the easiest one. I believe this implementation is correct, because we just add PQ primitives next to the classical ones. You are welcome to look at the diagrams also in the paper. I think they have a clearer way of showing things instead of me explaining it verbally.

Okay, then we got payment onion, invoices, and offers. Let me start with invoices. So, making invoices PQ, the main issue is the size. You put a PQ signature in the invoice, which blows its size up, right, and it almost exceeds the QR code limit. But if you use ML-DSA-44 signatures, which are 2,420 bytes, you still make the QR code size limit by leaving only ten characters to spare. So, it kind of works. The BOLT11 can be made PQ; you can fit a ML-DSA-44 signature in it, though you have to split it, since there’s a size limit in the Lightning implementation, which only allows you to hold at most up to 639 bytes. We divided the signature across four tagged fields to be able to carry it over the network. If you use Falcon, for example, Falcon will allow you to decrease the size of the signature significantly, and the whole invoice will drop to about 1,500 characters. For BOLT12, the gossip approach that we had, the gossip pin, which lets other nodes learn your PQ identity, it doesn’t help much here because the node behind an offer is often unannounced and sits behind a blinded path. So, the offer has to carry its own anchor. We commit afresh the ML-DSA key for each offer, and then the payer checks the return invoice against exactly that key before you send any HTLC (Hash Time Locked Contract). So, that’s kind of the summary of the approach with BOLT 12 offers.

Finally, BOLT 4. To be honest, this is the hardest part, I think making the onions PQ. I spent a good amount of time on this one, because really the problem is that onion packets are only 1,300 bytes and each payment needs one ML-KEM ciphertext per hop. This is per hop. And one ML-KEM ciphertext is 1,088 bytes. And if you have two hops, you just blow the limit up, right, so it’s not going to work. But what we came up with here is that we keep the onion exactly as it is, and make its per-hop secrets hybrid instead. For each hop, the sender encapsulates the hop’s ML-KEM key and mixes that secret into the classical Sphinx secret. This way, the ciphertext basically, they travel next to the onion in an updated HTLC message in a fixed list of 20 slots. So, this is the approach. It’s an interesting approach. I would like to hear feedback on this one. I do believe this should be correct, but let me know.

By the way, you might ask, “Why don’t we just increase the size limits of the Lightning packages so you don’t need to worry about these limits?” The reason we are not doing that is because if you do that, the interoperability between the PQ nodes that are upgraded and the classical ones will not be happening. They will not be able to talk to each other and not crash, or not quit the communication. If we want to have interoperability, we want to have the Lightning protocol working as it is and find workarounds. And the reason I wanted to have interoperability in the implementation was that we could just drop a PQ node in the network today and it would still be able to talk to all the unmodified nodes. But if it sees a PQ node, it can upgrade to PQ both sides. Since both sides are supporting PQ they can upgrade. I think this is a pretty necessary thing to do instead of modifying the Lightning altogether and just break the interoperability. I think this is what we should do at first, and then maybe the specs will change. The Lightning community will adopt the PQ upgrades slowly, and then everyone will maybe start running their nodes on PQ. Or it could start as an optional mode, and then become required. I think this could be worked out, so I would love to hear feedback on this one as well.

Okay, doing all of this in Rust-Lightning, we added around 11,000 lines of code over 52 files. We added like 100 tests in total, 99 or 100. It covers lots of different scenarios, adversarial scenarios, functional scenarios. I would really love Rust-Lightning developers to look at it. I am not a software developer, I am an assistant professor, I’m an academician. I might not have written the best code, or I might not have approached it in the best way possible, but I think this is a good start. These tests should be covering most obvious attacks against the changes we did. So, doing all of this resulted in some overhead, right, because PQ messages are big and it will cost us something in return. The computation, due to the increased sizes and everything, is really negligible. I think the most expensive operation, the ML-DSA signing, adds around only 0.33 milliseconds. The real overhead is the gossip bandwidth. When we did the tests, a fresh PQLN node that’s joining the network downloaded ten times more gossip than a regular node. This is with ML-DSA-44, by the way. If we use Falcon instead, it drops to about four times more than unmodified nodes, which is not too bad, in my opinion. Only issue with Falcon is that it’s not NIST standardized yet, right? ML-DSA and ML-KEM is NIST standardized in 2024, but not Falcon. So, maybe to use Falcon, we need to delay a little bit more to be making sure that it’s secure.

I also talked to Matt Corallo today on Twitter. We just tweeted out a few tweets. He was saying one of the other important things is to use secure Rust crates. We used fips203, fips204. I don’t know if they are audited, if they can be used in a production environment. I don’t know. Experts would probably be able to tell the answer, but that’s what we used in our implementation. We did some interoperability tests against an unmodified node and a PQ node. We just exercised mainline functionality, such as sending payments, receiving payments, opening and closing channels, asynchronous payments. I think we might have tested even splicing, I’m not sure, but all of that works, nothing breaks. They’re able to do these operations without nothing absurd going on between the nodes. They follow the protocol completely, which is great. I want to emphasize, I really spent so much time making sure it’s interoperable. If I hadn’t had that constraint, I think the implementation would have taken less time overall.

Okay that’s all I’m going to say. I exceeded the ten-minute limit Mike had given me. I’m sorry if it’s too long. But the paper is on arXiv, the GitHub repo is ahmet-kurt/pq-rust-lightning. You’re welcome to go have a look at it, let us know what you think. You can open issues, PRs even. I would really be happy to look at them. It kind of is a new thing to me. I haven’t maintained a GitHub repository before, but it might be a good start with this project. Again, thanks to Mike and the Bitcoin Optech team for this podcast opportunity and for the feature on their newsletter. Hope to see you guys some other time. Bye.

Mike Schmidt: So, that was Ahmet Kurt on PQLN. We want to thank him for recording that segment for us. We have links to the Delving post, the paper, and the implementation as well. Those are all in the newsletter. That wraps up our News segment, and we’re going to jump to Releases and release candidates, so a little bit out of order.

Bitcoin Core 32.0rc2

We’re going to skip the Stack Exchange for the moment and jump to Bitcoin Core 32.0rc2. Jan, 32.0 is the next major version of Bitcoin Core, and we have RC2 out. I don’t know since we published on Friday if there’s anything new there, but you wrote the testing guide, which is up on the Bitcoin Core Dev Wiki, and perhaps you can walk us through two components. I know you’ve been on the show talking about this in the past, so maybe you have a way that you’d like to portray it, but I think we want to have the audience walking away with maybe both knowledge of what’s in 32, as well as maybe some details on the testing guide and things you have in there, or things you’re working on, or feedback you’re looking for in particular. So, I’ll turn it over to you, Jan.

Jan B: Yeah, no worries. So, once every six months, there is a new major release of the Bitcoin Core software. And right now, we are on 32 RC2, and the final release is targeted for October 10th. Releases are time-based, not feature-based. So, every release has not the same amount of new or improved features. And this one has a lot of new features or improvements. Not everything is in the release notes, that’s also good to know, and the notes cover the most notable changes. But there are also a lot of bug fixes for minor issues that don’t get their own line. Before every release, the project puts out one or more RCs. Currently, we are on RC2, and that means that after the first RC, there were some minor issues found, and the project has released a new RC with some bug fixes. The RCs are feature-complete, so the community can test their own setup against it. For this testing campaign, the project also writes a testing guide, and that’s intentionally incomplete. And so, why an RC needs the community? Bitcoin Core has already an extensive test suite, so every PR gets tested with unit tests, functional tests, fuzzing, continuous integration on many platforms for every PR. On top of that, there’s also now an effort by the Red Team that tests with AI for security issues, etc, but the project cannot test on every downstream setup. So, every Lightning node, block explorer, or wallets app, hardware wallet, mining pool, operating system, etc, or for instance a Raspberry Pi in someone’s closet, every setup is a little bit different. We cannot test that for every setup.

So, that’s where the community comes in. You run it in your way, on your hardware with your software around it, and we need your feedback. So, we have built the most feature-complete release, and we really have put some effort in it to test it really well, and now we want some community feedback if it also runs on your hardware in your setup in your way.

Mike Schmidt: Jan, you mentioned this, but I think we’ll just bat the point home for listeners as well. Obviously, there’s new and different feature/component settings, etc, that will be in 32. But even if folks grab their RC and just use that RC Bitcoin Core in the same way that they have previously, even if it doesn’t necessarily touch any of those features directly, it could be valuable because it could surface some other bug that maybe was behind the scenes and not one of the headline features, right, maybe certain operating system compatibility, etc?

Jan B: Certainly. So, you do not have to use the testing guide. Just run it as you would run it. So, the testing guide is nothing more than a starting point for experimentation, and it’s in no way comprehensive. But also, the feedback in the way you run it at home is pretty important.

Mike Schmidt: And where should folks put that feedback if they find something interesting or notable for the team?

Jan B: Well, there are two points where you can give feedback. There is an issue for the testing guide if you want to give some feedback on things you miss or you want to get changed or just have feedback, Well, I love the testing guide, but next time I would like to see this, please. And there is also an issue on GitHub for explicit release 32 issues. Both are linked in also the testing guide. So, like I said, it’s a starting point for experimentation and it’s in no way comprehensive. But what we try to do with the testing guide is walk you through the steps. And it now covers, like, ten tests in four groups, some new RPCs, some updated RPCs, some wallet-related issues, and it also covers the HTTP server rewrite. Everything runs on regtest, and that’s your own private test chain, so you can mine your own blocks, and you don’t have to download the whole blockchain. It just runs instantly and it will take up only a few megabytes of storage on your own machine.

Mike Schmidt: So, what kind of things are in there? What’s in the testing guide? And if there’s anything interesting that’s not in the testing guide, what can listeners expect in 32?

Jan B: So, we cover a lot of stuff that’s also in the release notes. The most notable are, for instance, there is now a new RPC command, getopenrpcinfo. And that’s a machine-readable description of every RPC that Bitcoin Core has. And you can use that, for instance, in your own downstream client. And this will also help in the future, because you can easily diff between the two outputs, and then you can discover changes in the future more easily so that you can also change your downstream clients in a more convenient way. One other thing that is there is now also a new RPC, exportwatchonlywallet. And so, if you have a wallet on Bitcoin Core and you want to have a way to see your balance but not to be able to make transactions, so for instance you have an additional wallet on your phone and you only want to see if you get paid or what’s your balance, you can export with that command the xpub. And then, you can restore that on your phone. We test that in the RC testing guide. There’s also some updated RPCs that we test, for instance the PSBT is now v2 by default. And it’s good to confirm that it works, especially if you use PSBTs in a different way in your own setup. So, we use it to confirm, sign and finalize the broadcast. But also, if you use it in your own setup, please test it.

This release also has a new HTTP server that was rewritten from scratch to remove libevent. So, anything that talks to your node over RPC or REST uses this new feature. One thing is that it is deliberately rewritten to be more strict on some RFC rules. So, if you use your own RPC or REST client, you are advised to test against it. There has been a lot of effort to keep it as compatible as possible, but we wanted to be more strict for the RFC that are used on the internet for HTTP servers.

Mike Schmidt: For listeners, we had Matthew Zipkin on in Podcast #411 to talk about all the work that went into that replacing of libevent. If you’re curious for more, it’s an interesting story.

Jan B: Certainly. Yes, so those are the main topics of the testing guide. So, we also do some derivehdkey, addhdkey. It has to do with the option to use multisig in the new Bitcoin Core wallet. If you want to have a deep dive, you can use the release testing guide as a start, and there is extensive documentation in the documentation of Bitcoin Core. And there’s also a new fee estimator; we test that. And there is also an option to dumptxoutset to a named pipe. That’s a little bit more advanced feature. So, you can stream UTXO set to a file and then use that in a database or something else.

Mike Schmidt: I made a few notes. I think Jan covered most, if not all, of these, but I did pull out which newsletter we covered them in. And I think these are all in 32, but I’ll just list them off. And then, Jan, if there’s anything you want to elaborate on, we can get into that. But the exportwatchonlywallet, we discussed in Newsletter and Podcast #413; the libevent, I mentioned we had Zipkin on for #411; there’s the parallel coin fetching for faster block validation, which was #414; smaller and faster -txindex, #419; Jan mentioned the mempool-based fee estimator in #420; migrating the wallet without loading the wallet, I think we actually talked about that in a Stack Exchange answer in Newsletter #416; and then, there’s the global transaction-relay limits, and the 200 connection default from 416 as well; and then, the last one I noted here, and I think this is in 32, but the BIP434 peer feature negotiation, that was in Newsletter #410.

Jan B: Yes. For instance, those small -txindexes, that’s pretty hard to test in a RC on regtest. So, we are interested in the experiences of the community, but it isn’t mentioned in the RC testing guide because for that feature, you need a bigger data set. So, you need to copy mainnet data, and that’s outside of the scope. But we are interested in your experiences, so please test that. And, yeah, the parallel fetching, that should help with syncing. We’re also pretty excited about that and to see what kind of in-the-field experiences are with that. But again, too hard to test in a regtest situation, so those are also not in the RC guide.

Mike Schmidt: Well, we link to the RC, we link to the guide. I think the only other thing that you mentioned, Jan, that folks may have to crawl around for would be the draft release notes that we don’t have a URL handy in the newsletter for just yet.

Jan B: It’s also in the release testing guide.

Mike Schmidt: Oh, good, you link to it from there, great.

Jan B: Yes.

Mike Schmidt: Okay, perfect. So, check that out. If you don’t want to do the RC testing, you can at least see what’s coming in this v32.

Jan B: I want to say one thing. So, we are also interested in if everything just ran and it worked. Those mentions are also useful. And send your OS and your hardware, please, for that.

Mike Schmidt: Excellent. Jan thanks for joining us, thanks for putting the testing guide together, and we appreciate your time.

Jan B: Thanks for having me.

What would be a drawback if sum instead of SHA256 of amounts was used in the taproot signature message?

Mike Schmidt: Cheers. We’re going to go back up now to our monthly Selected QA from the Bitcoin Stack Exchange segment. First question, “What would be a drawback if sum instead of SHA256 of amounts was used in the taproot signature message?” Maybe a bit of background on this one first. When you have a hardware signing device that signs a transaction, it doesn’t have the blockchain on it. It only knows what the computer it’s plugged into is telling it. So, the fee in a transaction is just inputs minus the outputs. So, if you are connected to a computer that is compromised, or for whatever reason lies about how much an input is worth, the device could potentially be fooled about the fee. So, this is the whole idea of a fee-overpayment attack. So, if your input is really worth 10 bitcoin, but a malicious piece of software on your computer that your hardware signing device is connected to tells that device that it’s worth 1 bitcoin. Then you approve sending 0.99 BTC to the recipient, thinking the fee is 0.01, when in reality it’s 9.01 bitcoin that goes to the miner. So, that’s a little bit of a background on the fee attack. Taproot fixed this and it makes every signature actually commit to the amounts of all of the inputs. So, if that computer that’s malicious lies about any of the amount, the signature would just be invalid and that transaction can’t be broadcast.

So, maybe back to the question. Taproot hashes all of the amounts together. This person asking this question asks, “Why not just commit to that total?” He notes that there’s potentially the same protection against fee theft with less work. And the answer here is that committing to the total actually does still stop that fee attack, since the total in minus the total out is fixed, but the malicious computer in our example here could actually still shuffle the amounts between the different inputs around as long as that total stayed the same. And so, then you can picture something maybe like a coinjoin where you put in 1 bitcoin, someone else puts in 0.1 bitcoin, and the outputs pay you 1 bitcoin, to them 0.1 bitcoin. Now, this malicious software and computer lies to your device and says that your input is 0.1 and theirs is 1, so the total is still 1.1. So, you’re satisfying that total and the fee check passes, but now the device is going to happily approve a version where you get 0.1 BTC and the other party gets that 1. So, that’s why you want to keep the actual taproot design that has your device actually committing to your input being that 1 bitcoin. So, I was going to say, “Murch,” I’m so used to it! Gustavo, anything you’d add to that?

Gustavo Flores Echaiz: No. I have no idea about how that works. But yeah, I think you explained very clearly, so that’s interesting.

Post-BIP110 fork is it necessary to resync from block 0?

Mike Schmidt: Next question from the Stack Exchange, “Post-BIP110 fork is it necessary to resync from block 0?” Maybe a little bit about where this person might be coming from. They were running Bitcoin Knots with BIP110 enforcement turned on. And then, when the activation attempt failed back in August, those nodes started rejecting that first block that didn’t signal and split off into two different chains. We covered some of this BIP110 discussion in Newsletter #418. But that chain, as it was, never got past really a few blocks before stalling out. There’s been a few different forks of that now that have come into play, but that’s not what we’re talking about, just maybe more of a conceptual question using that as an example. This person now wants to get back on Bitcoin mainchain by switching to Bitcoin Core. But their worry was that they were running a pruned node, which throws away old blocks, and then a pruned node can’t rewind past what was already previously deleted. So, they were asking, “Hey, do I have to download the whole blockchain from scratch now?” And our co-host, Murch answered this, “Probably not”. And he notes that probably not, because that split happened recently, at least in the eyes of his Bitcoin Knots node. And because the BIP110 chain, with those rules at the time, barely added any blocks before that node was last run, then the pruned node will almost certainly have that last block that both the chains shared, because there would only be a few blocks, even if the Bitcoin mainchain had thousands of blocks since then.

So, he, Murch, advocated that he thinks you could probably install Bitcoin Core in place of Knots and it should just reorg back to the mainchain on its own. And maybe one fallback he mentioned that I should mention here is that you could also run reconsiderblock on the first block the Knots node rejected that wasn’t mandatory signaling, and that would tell the node to stop treating that block as invalid, and then re-evaluate the chain from there. I don’t think Murch actually did this himself, but I think he’s just speaking about how the mechanisms within Bitcoin Core work to recover from such a situation. Any feedback, Gustavo?

Gustavo Flores Echaiz: It makes total sense. I think if someone got stuck in this situation, it isn’t as hard to recover because there was not so much work done in this new chain. So, no need to start from the beginning.

Can a Bitcoin node build a partial UTXO set from only the most recent blocks and use it to validate new transactions?

Mike Schmidt: Last question from the Stack Exchange, “Can a Bitcoin node build a partial UTXO set from only the most recent blocks and use it to validate new transactions?” Kind of an odd ask, but I think what the person asking this question is getting at is that, “Hey, I don’t want to go back to every block since 2019. That’s a lot. What if I just skip all that old history and only process the last few months’ of blocks”, I think is what he was asking, “and then build a UTXO set out of that and then validate new transactions against that?” I think they’re hypothesizing that that could be like a cheap, lightweight node. But our question answerer, Pieter Wuille, who answers so many questions on the Stack Exchange, actually did take a crack at this and explained why it wouldn’t work. And I think probably our listeners are familiar with why this might not work. You obviously need to check that every coin in that transaction that is being spent actually exists and hasn’t been spent yet. And obviously, with only a partial UTXO set, you could be spending one that the node doesn’t have. And in that case, that coin could have been created years ago and it’s perfectly valid. But your hyper-pruned partial-UTXO-set node doesn’t know about that coin even existing at all. So, interesting that people are thinking about these kinds of things, but obviously not possible.

Gustavo Flores Echaiz: Right, yeah, I saw the guy was trying to make a sort of new design. He said, “I’m working on a lightweight-node design for improving independent verification of new/unconfirmed Bitcoin transactions under limited storage”. So, it’s a good initiative, but yeah, this definitely doesn’t work because you can never know for sure if you have UTXOs and spent transaction outputs that date very long ago in the early days of the of the Bitcoin Network. So, because of that, you do need to validate the whole transaction history to build a UTXO set.

Mike Schmidt: Yeah, good energy. Probably put that energy into something that we’ve been talking about, did we talk about it last week or the week before? Yeah, Utreexo, Floresta gets you sort of that compact representation of the UTXO set while taking into account all blocks and not just some blocks like this proposal was getting at. That’s it for the Stack Exchange. We only pulled out three this month. So, we can go back to the Releases and release candidates segment to pick up where we left off. Gustavo, I’ll let you take it from here.

Gustavo Flores Echaiz: Yes. So, we already covered the RC of Bitcoin Core v32. So, as Jan said at the beginning, there’s a testing guide. Please go check it out if you’re interested in testing this work. Quite a heavy release. We’ll have probably a follow-up RC in the next few weeks, and we’ll probably also have more information on this release in the next episodes and newsletters.

Core Lightning 26.06.8

The other two releases that are part of this newsletter, the first one is Core Lightning 26.06.8. This is another security release from Core Lightning (CLN). Once again, the notes are not that self-explanatory. So, the source code is however released, unlike the previous version. But on purpose, a small number of tests are withheld temporarily to make it more difficult for a prospective attacker to identify and exploit the underlying vulnerabilities. In the previous release, it was even more extreme. It wasn’t just the test suites, it was the source code as a whole. And by the way, in the Notable code and documentation changes, I’m going to talk about what was in the previous release of CLN, because we finally got the source code release. So, yeah, so users of CLN are strongly recommended to upgrade to the new version 26.06.8. Also, important to note, if you had been experimenting with the development build on master, you cannot downgrade to this release, because the database schema that you were using on experimental development build was newer. So, also be careful if you have run the development build; if you would like to downgrade to this version, you cannot. So, worth noting for operators.

LDK v0.3-rc2

The next item in this release section is LDK v0.3-rc2. So, once again, this is the second RC of a release we covered in a previous newsletter. It has multiple new features, such as RBF fee bumping. When a splicing transaction gets stuck and cannot confirm onchain, you can now finally RBF it to increase its fee and to get it actually to confirm. It also allows you to add or remove funds in that splice. So, for example, you could remove or add additional funds in that splice. Also, anchor channels are now negotiated by default, and applications that use LDK are explicitly required to accept incoming channels. The upgrade also invalidates previously issued BOLT11 invoices, because there was a format update in how BOLT11 invoices were constructed where the payment data was restructured. So, if you were to upgrade to this RC, your previous BOLT11 invoices that have not yet expired would become sort of incompatible with this new version. So, also worth looking into that. And there are multiple API and backward-compatibility changes. So, operators should take a look at that as well. However, this is still just an RC. So, don’t yet fully consider it as a production release. It’s mostly released for testing purposes.

I also want to note that the LDK repo has moved away from GitHub to a self-hosted mirror, git.rustbitcoin.org, instead of the GitHub repo. So, the PRs, for example, and the consistent work can no longer be found on GitHub. It is found on that mirror website. However, you can easily find the mirror website through the GitHub repo; it’s in the ‘about’ description of that repo.

Mike Schmidt: Gustavo, you mentioned it in the previous item, because it wasn’t a RC. So, obviously, test your RC if you have a setup and provide feedback to those project maintainers. But obviously, for full versions that we’re covering of software that’s coming out, we’ve covered it for the last few months here, there’s just the amount of bug fixes that these pieces of software are going through due to Red Team and other groups. There’s a loop and there’s actually hackers out there attacking software. So, I encourage folks to update to the latest version of their software. In fact, for this coming week’s newsletter, one of the items that I’m doing a summary of is the disclosure of two DoS vulnerabilities in Lightning software. So, stay safe out there, keep updated with the latest versions when you can.

Gustavo Flores Echaiz: Thank you, Mike. And yeah, Lightning is particularly vulnerable to these types of attacks because of how complex the protocol works. We’re going to cover the CLN previous vulnerabilities in a bit, but yeah, particularly Lightning is affected. So, once there’s a security release, it’s strongly recommended to upgrade. And if you are a tester, please take a look at the RC and the testing guides.

Bitcoin Core #34566

Now we move forward with the Notable code and documentation changes section. We have one item from Bitcoin Core, two new BIP proposals, and multiple items from CLN and LND. So, let’s start with the Bitcoin Core item #34566. This is a very simple one that follows what I had covered in Newsletter #412. This is about adding default multi-signet data directory support in Bitcoin Core, which means that previously, if you were to use multiple signet networks, which is not the most common use case of Bitcoin Core, but it’s still an allowed use case, you could previously manually build multiple subdirectory data directories for your different multiple signet networks. However, it was not supported by default or it wouldn’t automatically arrange multiple data directories on your behalf. You would have to do it manually. Now, Bitcoin Core is updated to allow users to have multiple custom signets that share a base data directory but all have subdirectories that are tagged with their four-byte network identifier. And this allows the chain data not to conflict, as previously you could run into conflict scenarios.

So, now, if you were a user that had already set up multiple signets on your device, to avoid resynchronizing after upgrading, you should manually rename your directories to the new format to use the four-byte network identifier in the subdirectory name, so that you avoid unnecessary resynchronization after upgrading. Yes, Mike?

Mike Schmidt: Yeah, maybe just a common point of confusion that I see is that people treat signet as the signet, there is just one signet. I think this obviously bats home the point that you can have many signets that you’re operating on. You can operate your own signet if you choose. I think the fact that there is a default one that is easier to use is just sort of a convenience thing for people. But I think it’s sort of put in people’s minds that there’s just one of those. But you can run your own. Reminder.

Gustavo Flores Echaiz: That’s great. Right, exactly. Great feedback. And that’s exactly for those types of users who would like to run their own. This helps set it up easier. Now the next two items are from the BIPs repository.

BIPs #1951

The first one, #1951, adds BIP138. So, BIP138 is an idea that has been previously discussed in its nascent form in Bitcoin Optech, particularly in Newsletter #351. In the News section, we discussed and we probably invited to the episode, standardized backups for wallet descriptors. Salvatore Ingala had posted to Delving Bitcoin back in April 2025 about this idea of creating a standard for backing up wallet descriptors. The idea is if I’m setting up, for example, a multisig wallet with other signers, we have a descriptor of our multisig wallet, or sometimes called just a wallet file, that even if we had two keys out of three in a 2-of-3 multisig, we would still need the descriptor or the wallet file in order to recover the funds. Even if the multisig says that two keys out of three are sufficient, the third xpub is required to reconstruct the wallet. So, the idea here is how can we create a specification or a standard for creating backups of descriptors and other non-seed wallet data, for example, BIP388 wallet policies too, that a user could safely upload, for example, in their computer or their Google Drive folder, and only him could decrypt that file and nobody else that had access to that folder could decrypt it? And this is where BIP138 comes in.

The idea here is that if each different signer has their xpub, what if their xpub could allow it to decrypt the encrypted wallet descriptor? So, the analogy that I would say here is sort of if there’s one encrypted safe holding the descriptor, but beside it there’s three small lockers containing the same key that will decrypt the safe holding the descriptor, and the small lockers are each specific to each signer. So, I, as the first signer, I have my xpub. I can open, for example, the first locker, and that gets me the key to open the main safe to obtain the descriptor. So, each signer can access the same decryption key that will then allow it to decrypt the descriptor. So, this is now fully standardized. And it allows any cosigner to recover a descriptor from their own seed without needing the other cosigners’ keys. So, I think this was very interesting. We’ll see how it gets adopted. And yeah, there’s additional notes in the Bitcoin Optech Newsletter about this, and also you can check out directly the BIP specification.

BIPs #2224

The next item introduces BIP461, so PR #2224. This is about introducing a single deterministic ECDSA signing algorithm. So, the problem that we want to mitigate here is basically exfiltration, or we want to create exfiltration-resistant signing. If you had malicious software that would choose the nonce value when signing with an ECDSA signing algorithm, if your malicious software is the one choosing the nonce value, it could technically leak private key material that only the attacker could decrypt through the nonce value. And it would require only two signatures for enough private key material to be leaked that could allow an attacker to decrypt that private key material and obtain access to your wallet. So, in Newsletter #315, we covered the Dark Skippy attack, which is an improved method for key exfiltration from a Bitcoin signing device that works in the manner that I just described, where an attacker leaks private key material through the nonce value.

So, now that we understand the problem, BIP461 comes up with a standard specification with a single deterministic ECDSA signing algorithm that ensures that any given private key and message will always produce the same signature, because it uses RFC 6979, so a specific standard for how to derive the nonces. And it also applies low-r grinding, which we have this whole section on Bitcoin Optech that explains what that is, but it’s an optimization for wallets where they keep generating new ECDSA signatures for the same transaction until they find a signature whose r value is on the lower half of the range, allowing it to be encoded with one fewer byte than a signature on the top half of the range. The point is that BIP461 has a specific standard for how it derives nonces, it always applies low-r grinding by retrying until r encodes without a leading zero byte, and also normalizes s to its low form. So, now, because the output is deterministic, any software that supports BIP461, that supports this single deterministic ECDSA signing algorithm, should always have the same result. If you compare two different signing softwares or devices and you don’t arrive to the same result, but both claim to support BIP461, then either they have failed at implementing the specification or potentially there’s a potential attack attempt to leak key material through nonce selection.

So, this allows a user or an operator, a developer, to detect whether this was properly implemented, and if not, you could be suspicious that there’s a potential attack here to attempt to leak your key material through nonce selection. So, an important BIP, I believe, and an interesting background to how the Dark Skippy attack works. If you want to know more about it, you can look at Newsletter #315.

Core Lightning #9507

So, the next five items are from the Core Lightning repo, and we finally now know what were the vulnerabilities that weren’t previously disclosed. So, I’m going to go through all of them one by one, starting with PR #9507. So, each of these items has multiple issues and multiple bug fixes in them. The first one mostly comes to feerate bounds and feerate values that could overflow. So, previously when a peer proposed the splice or when you try to RBF a dual-funding opening or funding transaction, there was never an upper feerate bound. So, your peer could propose an extremely high feerate for a splice or an RBF attempt on a dual-funding opening, and you would simply accept it. You would completely lack upper bounds. The dual-funded opens lacked both lower and upper check bounds, while the RBF of a splice or the RBF of a dual-funded opening lacked simply the upper feerate bound. Also, if you had stored a feerate that was a very large value, this could potentially overflow. It could basically reset to zero, so you could either pay a very low fee that simply wouldn’t get the transaction to be added into a block, or you could also trigger an assertion when calling the command listpeerchannels RPC. But the problem here is that every plugin calls this RPC command at startup. So, if your node would accidentally store a very large value for an RBF feerate calculation, then you would basically hit a cycle of restarts, because every time the node would restart, the plugins would call listpeerchannels RPC and that would hit an assertion failure and crash the node continuously.

So, three bug scenarios here. The first one, you stored a zero value, you’re not paying enough fees, your transactions are not getting confirmed. The other one, you stored a super-high value that hits an assertion failure, and this simply repeats on all restarts. And the third bug, which is the one that I started with first, is when you are missing feerate bounds, so potentially you’re paying excessively high fees when, for example, your peer proposes a super-high fee. So, the fixes here are to introduce a new absolute ceiling of 4,000 sat/vB (satoshis per vbyte) on peer proposals and also backend fee estimates. And this absolute ceiling always applies. Even if you use the command –ignore-fee-limits, this ceiling would still apply. However, there’s also another ceiling that’s introduced that is ten times lower at 400 sat/vB, that caps RBFs, commitment updates, splices, and your own opens. So, CLN differentiates between the fees that come manually from you, caps those at 400 sat/vB. However, you can override those with the –ignore-fee-limits setting. But if a fee comes from a peer proposal or from a backend fee estimate, that is always capped at 4,000 sat/vB. And also, if you had stored values that would hit the assertion failures, the overflow bugs, that code is also clean. So, those out-of-range stored feerates are also repaired on upgrade.

Mike Schmidt: Pretty crazy. Yeah, there’s a lot going on there. But it sounds like the ceiling should be enough for the foreseeable future, with 4,000 sat/vB, given our current fee climate. But yeah, good to understand some of the details of this one, although it sounds like there’s more.

Gustavo Flores Echaiz: Right. And I don’t know if you remember, but I had talked about this in a previous newsletter, actually the last newsletter, the Eclair implementation had similar fixes. So, it seems that Eclair kind of saw that CLN dealt with this, so they made the fixes themselves on their repo too. So, I’m actually observing a lot of similarities between CLN, LND, and Eclair. They all seem to have similar gaps that could be exploited and are fixing them, which brings me to the next item.

Core Lightning #9508

So, #9508 fixes several splicing and channel-opening issues. So, once again, in the last newsletter, we also talked about the splicing issues within Eclair, which now are apparent within CLN too. So, there’s multiple scenarios here. So, the main one was that before CLN received a peer’s splice_locked message, but CLN might have already sent its signatures to the peer, then the peer could technically broadcast the splicing transaction or the commitment transaction for a pending splice. But if CLN had not received the splice_locked message, it would simply miss these peer broadcasts. So, now, even if CLN doesn’t receive the technically accurate message from the peer confirming that the splice has been locked, as soon as CLN sends its signatures, it should and it will start monitoring the funding output of every pending splice and recognize that spend if it sees it onchain. So, now, a peer can no longer fool CLN into thinking that the splice hasn’t been completed when it has actually been completed, and the peer could technically broadcast these transactions by himself. So, that’s the first scenario.

The second one is if a peer sends you a tx_abort message during a splice, you technically have to abort the splice. However, if CLN has already sent its signatures once again to the peer and the peer sends a tx_abort message, you cannot accept that message because you have already sent your signatures, so the peer could already broadcast the transaction. So, now, CLN properly force-closes the channel if the peer sends tx_abort after CLN has sent its signatures. And the check previously would fail because CLN would just simply fail to recognize the splice that had already been signed. It would look for another splice and it would just see, like, “Okay, I haven’t signed it yet”, but in reality, it was just looking at the wrong splice, and also the flag indicating that the signature had been sent was not preserved across restarts. So, if I restarted my node, I would lose the memory that had already sent my signature to my peer, and the peer could technically make me think that the splicing was abandoned when actually, I had sent him all my signatures.

Mike Schmidt: Crazy!

Gustavo Flores Echaiz: Yeah, crazy scenario.

Mike Schmidt: And there’s a third one! And it’s like, yeah, there’s three in the last one, three in this one.

Gustavo Flores Echaiz: There’s so many different scenarios in Lightning. This is why I think Lightning was particularly vulnerable to these latest rounds of attacks because of all these so many different edge cases, particularly in the splicing protocol. You’ve got so many different steps and messages, right? You’ve first got to go suspend the channel with the quiescence protocol, and then you’ve got to start negotiating. So, all of those edge cases are kind of getting exploited and now getting fixed.

Mike Schmidt: The timing is a little bit poor as well, right? You know, when dual-funding; when splicing? We’ve been asking for years, and then they sort of both kind of come out in experimental ways right around when LLMs start finding edge cases and complex schemes like this. So, it’s unfortunate timing, I guess, as well.

Gustavo Flores Echaiz: Right, exactly. Finally, the last part of this PR is that if my peer keeps sending me dual-funding opening requests, I will now disconnect the peer if I already have three ongoing channel-opening negotiations with him. So, this basically puts a threshold on how many dual-funding opening requests a peer can make to me.

Core Lightning #9509

The next item, #9509, this one’s actually, I think, probably the most important one, because there are issues related to onchain channel resolution. So, the first part is that an attacker could trick CLN into treating a force-closed as a cooperative close. So, an attacker could force-close a channel, but CLN would not react to the force-closing. Even if the peer had sent a revoked commitment transaction and CLN was supposed to reply with a penalty transaction, it would get fooled into thinking it was a cooperative close and it had no reaction to be given. And how was that done? First, the peer that doesn’t commit to an upfront shutdown script – so, when you open a channel, you can commit to an upfront shutdown script, which means, “I want to get paid to this address”. So, you don’t have to do that, that’s an option. So, the first requirement for this attack is the peer doesn’t commit to an upfront shutdown script. The second step is the peer sends a shutdown message as if it was initiating a cooperative close. But in the shutdown message, it tells him, “I want to get paid to an address that was part of a revoked commitment”. However, the attacker, as soon as it sends the shutdown message and CLN registers the shutdown script or the output that’s getting paid, then the attacker aborts the cooperative close and broadcasts that commitment. But CLN would basically think that because the shutdown message had been sent, the cooperative close was still ongoing, so it would simply not react to the broadcast of a revoked commitment with a penalty transaction.

Now, CLN simply does a better job at identifying commitment transactions by their locktime and their sequence encoding before checking outputs. Previously, it was simply checking outputs. And if the outputs match what it was expecting for a cooperative close, it would simply not react as if it was a force-close. Now, it looks at better identifying information to detect what the commitment transaction is really about, and it immediately notices that it’s a revoked commitment, which is what allows it to reply with a penalty transaction. Also, so that is probably the main fund-theft case that we’ve explored in these notes. But for example, other edge cases that are also present is if I have a commitment transaction or a descendant of a commitment transaction confirms but gets reorged after, then CLN would stop monitoring the outputs of the commitment transaction. So, another transaction that would spend the commitment transaction would simply fail to be monitored until the node restarted. So, now, CLN ensures that if the descendant of a commitment transaction gets reorged, it would keep on monitoring those outputs for any new transactions.

Yeah, I think that’s the main part of this PR. There are additional edge cases that we could also discuss, but they’re present in the notes. And if listeners want to know more about it, they can jump into that item.

Core Lightning #9510 and #9511

The next one is two different PRs put together, #9510 and #9511. This is about hardening input parsing and logging. So, not as critical as previous ones, where fund theft was even possible. This is more about potential crashes or data leaking. For example, if a node announced a very long DNS hostname, CLN simply had no limits on it before. So, we could try to connect through a proxy to a node that announced a very long DNS hostname, and that could crash the connectd of CLN and therefore crash CLN. So, now, CLN simply rejects DNS addresses from peers in received node announcements that are invalid. It will simply ignore invalid DNS node announcement addresses. But it will also reject DNS addresses that are not valid hostnames, even if it comes from its own configuration. So, now, two protections are added from its own configuration and from node announcements by network peers.

Also, JSON nesting is now limited to 256 levels. So, this was also a potential other crash if JSON information had too many nestings that could also potentially crash CLN, or request bodies and REST calls that were higher than 2 MiB; that was also another crash trigger that’s also been capped. And also, a TLV parsing bug for a BOLT12 offers that could basically let an attacker create a malformed message that could crash CLN, that is also fixed. So, multiple crash scenarios in this one.

Mike Schmidt: It’s crazy how many Lightning bugs, at least recently, including I think one or both of the ones that I’m summarizing for next week’s newsletter, but other ones that we’ve covered in the past, is just unbounded growth of things or not having length-checking on certain things. It seems like that’s what a lot of what that PR did. Obviously, it doesn’t solve everything, but it’s obviously some low-hanging fruit to put some limitations on how long certain strings or things can be, and how large a queue can grow. I think we had the LND gossip that grew infinitely. So, yeah, it’s just good to see that it’s being fixed, but it’s crazy how many are around that sort of genre of bug.

Gustavo Flores Echaiz: Yeah, implementing Lightning is hard and all nodes require so many different entries on communication systems, because in the end there’s simply no shared data like the blockchain in the Bitcoin Network. So, nodes in the LN, they have to agree about everything and communicate everything. So, that creates so many different entries for these types of vulnerabilities, from my understanding.

The final part of this PR is that I/O logs, input/output logs, are removed from getlog, because it could contain authentication tokens that grant restricted RPC access. So, also, a logging cleanup to not disclose unnecessary credentials.

Core Lightning #9513

The next item, CLN #9513, this is a simpler one. This fixes amount validation when fetching an invoice with xpay for BOLT12 offers. Basically, previously, after making an invoice request to a node and fetching an invoice, CLN would simply not check the amount, it would allow the recipient to request a larger payment. Now, the invoice must either match the requested amount or it cannot exceed the offer amount, so it is now capped by the requested amount or the offer amount if no amount was requested. Previously, you could simply get a node to tell you, “Pay me this invoice,” of whichever amount, and if you were not careful enough, you might just pay a higher amount than the amount you requested. Also, onion messages that contained a reply path with no hops are also now prevented. Any node could technically send you this and it would stop your node, because a reply path with no hops simply makes no sense. So, now, that is blocked too.

So, yeah, that’s the last item of the five CLN items. So, now we know what happened better in the past few weeks with the whole security issues in CLN. But now, like I mentioned earlier, there’s an even newer release, so users should upgrade to that newer release.

LND #11198

The next three items come from the LND repo. The first one is related to AMP. So, AMP, Atomic Multipath Payments, are the implementation in the LND repo, not to be confused with multipath payments that are a BOLT standard. If I’m not mistaken, I believe MPP is the standardized version between multiple implementations, while AMP is the implementation of LND itself. I think LND has support for both, but AMP is only in LND. So, the point here is, how AMP works differently from MPP is that as a receiver, I generate an AMP invoice. So, it’s not like this is just a regular BOLT11 invoice. In this scenario, you can also do Keysend AMP. You don’t have to generate an invoice, you could also do spontaneous payments with AMP. But in this scenario, I, as a receiver, generate an AMP invoice.

An AMP invoice can be paid multiple times. And like MPP, an AMP payment gets split into multiple parts, into different routes, and arrives to the recipient. However, like I said, a generated AMP invoice can be paid multiple times. So, here, the problem was if I, as a receiver, I could potentially cancel the invoice and interfere with all potential payments if one payment failed. So, if one part of one payment attempt failed, I would simply cancel both my AMP invoice, and if other people were trying to pay me at the same time to that invoice, their payments would also fail. However, I should only fail the attempt that failed, not all attempts. So, now, LND fails the arriving HTLCs, cancels the previously accepted HTLCs belonging to the failing set, but the invoice remains payable, so other accepted sets can still be completed and settled.

LND #11146

The next item, LND #11146, this is about the latest in the BOLT12 offers implementation in LND. So, we’ve been covering this over the past few newsletters. In Newsletter #422, we covered support for signing and verifying invoice requests and invoices for BOLT12. And now, we have validated string encoders and decoders for offers, invoice requests, and invoices. So, we’re getting closer. There’s still a ton of work, but we’re getting closer. Now there’s a new function, ValidateInvoiceForPayment, which checks an invoice against the originating request, but also the node that payer expected to sign it. So, it has to match the receiver signature, it cannot just be issued by any node. Technically, any node along the blinded path could otherwise return an invoice signed with its own key, so it’s very important to verify that this was signed by the receiver, the real receiver.

Mike Schmidt: I was curious, since we’ve seen this progression of BOLT12 in PRs, do they have a tracking issue? I always find those good to put things in context of what’s done and what’s coming and what’s being worked on and needs review. Do you know if they have one for BOLT12 in LND?

Gustavo Flores Echaiz: Yes, they do have one. It’s issue #10736. That’s the issue number that has all the related steps to get to the full BOLT12 implementation. And I’m looking at it right now. I’m seeing that the codec part, so the BOLT12 package for messages and data structures, that is almost done. There’s simply one left to merge, finalize the codec package. But about ten items have already been merged for the basics for the message and data structures. And now, the next work, which already has an open PR, is a receiver side. However, that’s only the first item of the receiver side. So, there’s still a lot of work to be done, right? Then, once the codec part, the message and data structures part, which is almost done, once that is done, that is milestone A, then there’s milestone B: receiver side; milestone C: sender side; multi-hop is D; pay offer is E, and RPC parity is F. So, still a very long way to go, but the first milestone is almost complete.

Mike Schmidt: Okay. Yeah, cool. I have that up now, I’m taking a look at it. Thanks, Gustavo.

LND #11132

Gustavo Flores Echaiz: Pleasure. I’m going back to the newsletter now. And the last item that we have in this newsletter is LND #11132. Here, this is a follow-up to something we had discussed in Newsletter #421. So, the goal of this new item is to restore BOLT1 compliance by replying to every valid ping message admitted from peers. Why does LND need to restore that BOLT1 compliance? It’s because in Newsletter #421, we covered that LND had added rate limits for inbound ping messages, and also request buckets and reply buckets for ping messages and pong replies. So, it was rate-limiting those messages and it had kind of dropped compliance with BOLT1 that requires LND to reply to every valid ping message. So, now that compliance is fully restored. So, now LND has at the same time the rate limits, but it will also properly reply to every valid ping message that is admitted by its flood policy. So, that’s the final item in this newsletter.

Mike Schmidt: Awesome. Thanks, Gustavo. We also want to thank Jan for hanging on throughout the remainder of the newsletter and joining us for the Bitcoin Core 32 RC and testing guide item. We also want to thank Ahmet for recording and sending us his thoughts and progress on post-quantum Lightning work. And we want to thank you all for listening. Cheers.

Bitcoin Optech Newsletter #423 Recap Podcast

2026-09-22T00:00:00+00:00

Mark “Murch” Erhardt, Gustavo Flores Echaiz, and Mike Schmidt are joined by Davidson Souza, Eric Price, and PortlandHODL to discuss Newsletter #423.

The Bitcoin Optech Podcast and transcription content is licensed Creative Commons CC BY-SA 2.0

Download audio

News

● Vardiff controllers that strand slowing miners (23:06)
● Improvements in Utreexo initial block download (1:18)
● New BIP draft for unspendable internal keys (1:17:51)

Changes to services and client software

● BitBoxApp adds Spark-based Lightning payments (1:24:09)
● Covenants.diy script editor (1:28:40)
● EntropyLab offline key calculator (59:46)

Releases and release candidates

● Eclair 0.14.3 (1:31:10)

Notable code and documentation changes

● Bitcoin Core #35445 (1:33:34)
● Bitcoin Core #36076 (1:38:51)
● Bitcoin Core #36150 (1:42:18)
● Bitcoin Core #36174 (1:44:37)
● Bitcoin Core #34743 (1:46:13)
● Bitcoin Core #36081 (1:48:07)
● Bitcoin Core #35975 (1:48:59)
● BIPs #2241 (1:50:04)
● BIPs #2258 (1:53:09)
● Eclair #3380 (1:56:33)
● Eclair #3376 (1:57:37)
● Eclair #3372 (2:04:34)
● LND #11163 (2:05:57)
● BDK #2246 (2:07:44)

Transcription

Mike Schmidt: Welcome everyone to Bitcoin Optech Newsletter #423 Recap. This week, we’re going to be talking about mining pool difficulty and stranding potentially stranded miners; we’re also going to talk about improvements to Utreexo’s IBD (Initial Block Download); there’s a draft BIP for unspendable taproot internal keys that we’ll get to; and then we have our monthly segment on Changes to client and service ecosystem software, including one update from a new group working on EntropyLab; and then we have our weekly Notable code and documentation, and Release segments. This week, Murch, Gustavo and I are joined by a few guests. We’ll have them introduce themselves briefly. Davidson, who are you and what are you working on?

Davidson Souza: I’m Davidson, I work on Utreexo in Floresta, and thanks for having me.

Mike Schmidt: Thanks for joining. Eric?

Eric Price: Hi, I’m Eric, I work at MARA on Pool mining pool, and I’ve been doing some work on vardiffs.

Mike Schmidt: Thanks for joining. Portland?

PortlandHODL: PortlandHODL, I work for AnchorWatch and am a Bitcoin educator on Twitter, possibly.

Improvements in Utreexo initial block download

Mike Schmidt: Thank you all for taking your time to walk through some of your items with us. For listeners, we’re going to start with the first item this time, “Improvements in Utreexo initial block download”, and we have Davidson, who mentioned he’s working on that, Utreexo as well as Floresta. Maybe a quick setup for listeners. Do you want to, Davidson, explain maybe just normal full node and keeping UTXO set on disk and maybe how Utreexo fits into that, and then we can get into Floresta as well maybe at the end, but I want to hear about improvements to IBD; but maybe we have this background information for people first?

Davidson Souza: Yeah, one of the main rules that a Bitcoin node actually checks when validating your block is whether the UTXOs existed, meaning the outputs were created by a previous transaction and not spent by that time on the current best chain. And usually, we use something called the UTXO set, which is an abstraction where only live TxOuts are kept. We learned in recent years that that’s not the only way to do that. Libbitcoin, for instance, doesn’t keep that. But at the end of the day, you have implicitly that idea that a UTXO exists and it’s still unspent, like a TxOut is still live in the chain. And this tends to be on the main bottleneck for running nodes, especially for low-power devices and a device that doesn’t have good SSDs or doesn’t have enough memory, because the I/O operations need to fetch those UTXOs; as the UTXO set grows, the I/O operations tend to mount. And right now, we have several people complaining that their setup, which usually is one of those things, it either doesn’t have a good SSD or doesn’t have enough memory, it takes a while to IBD. I’ve seen people that took like a month to IBD now, using basically old SSDs.

Utreexo tries to alleviate that by instead of having the entire set of UTXOs of TxOuts that are still unspent, we keep an accumulator, which is just like a tiny representation of that, where it’s less than 1 kB, currently it’s half of that, I think, and you only need to keep that information, around 1 kB, that’s nothing, you can keep that in the CPU cache. And instead of storing it, whenever you see a transaction or a block, you must receive as well a proof that this transaction was like those outputs, they exist in this accumulator. So, when you’re trading off bandwidth, like you’re using more bandwidth and a little bit more CPU, but you don’t need to keep the full set anymore, don’t need to keep, like I think nowadays, Bitcoin Core’s UTXO set is like 11-ish GB; you don’t need to keep that. And you don’t need particularly fast SSDs or lots of memory. The current code already runs on super-tiny devices. We can run on, like, smartphones, we can run on single-board computers without any problem, because the I/O bottlenecks no longer apply to what we are doing.

Mike Schmidt: All right, great background. Maybe you can walk us through maybe what happens when the UTXO is added and spent, and then you talk about this in your latest post, this 200 GB, where did that come from, and maybe explain the trick you had here around your proposal, that the deletion overhead gets down to almost nothing during sync?

Mark Erhardt: Maybe can I just jump in very quickly?

Mike Schmidt: Yeah, go ahead, Murch.

Mark Erhardt: I think I heard that you said Libbitcoin doesn’t have a UTXO set, and I’ve heard that a few times lately, so I wanted to clarify. The model that Libbitcoin uses does not have an explicit UTXO set because it has the whole transaction output set. So, yes, it has a UTXO set, but it’s a subset of the entire transaction output set; it just keeps everything. So, I think it is incorrect to say that Libbitcoin does not have a UTXO set. It has way more data than the UTXO set, but also has the UTXO set. So, I just wanted to clarify that. And Utreexo, on the other hand, does not have a UTXO set. So, it is actually a distinguishing feature here.

Davidson Souza: Yeah, sure. I think we should say that it doesn’t have an explicit UTXO set abstraction, but yeah, it’s basically the difference between the outputs versus inputs tables, you get the UTXO set. That’s fair. So, before this trick, Utreexo, every time you receive a block, you must receive as well the proof that every single element in that block existed. And you also need to do a lot of hashing to update the accumulator and produce the newer version of the accumulator with those inputs spent. So, you remove them from the accumulator. And this is quite a lot of data and quite a lot of hashing that you need to do. For instance, the worst case we can get in terms of overhead is 100%. So, if the chain is 700 GB, the worst case the proofs would be also 700 GB, so you end up downloading 1.4 TB of data during IBD, which is not great. Bandwidth is also, in some places, a very big problem, and over 1 TB is gonna take a while.

So, the idea or the question was, can you remove this proof-checking during IBD; can we remove proofs at all from IBD and not require in doing that? The first attempt was basically caching, which is a natural thing to think. Bitcoin has this nice property of UTXOs, they’re exponentially more likely to get spent right after being created. So, if you keep just, say, the last 100 blocks’ worth of UTXOs, you already save, like, currently it’s almost 80% of all these spends happening, recreating the last few hundred blocks. So, you already save a lot. I think in the experimentations we had, you could reduce the amount of data required to like 30% to 40% of the chain size, which is improvement, but still 200 GB to 300 GB of extra data that we need to download. So, that’s still not great. And to get more than that, you need to have even more RAM available to cache. So, finally getting to a point where it doesn’t make sense anymore to use Utreexo. But last year we had an in-person get together with a few people, we had the Floresta team, but we also had Tadge, Calvin and Ruben Somsen. And we tried to get rid of this somehow, when you add things to the accumulator, can we already update it in a way that the deletion is already taken into account? Because Utreexo is just a bunch of trees, and the way you delete things is just moving things up. So, if you’re deleting an older, take the sibling and move the sibling to where the parent is.

Mark Erhardt: Yeah, so because, of course, in a hash tree, two siblings hash together to make an inner node. So, if one of the two siblings is deleted, the other sibling moves up and becomes the parent, and that way you can make the tree smaller, yeah.

Davidson Souza: And then, we’re trying to figure out if I know the UTXO is gonna be spent ahead of time, so it’s an assumption I have, I know when I add that UTXO to the accumulator, I know at some block height, it is already spent. So, if I know this, can I already apply that operation in a way that it already goes to where it will end up after all those operations are made, like all the additions are made and the deletions? And the answer is yes, I can do that. And the nice thing is that the additional operations, they only require the root sets. So, they only require the roots, which is the less than 1 kB set of data that you need. So, if you have the roots and you know the spent-ness of a TxOut when you are adding it to the accumulator, you can already move it up to the positions it moves, and then you end up where it would be if you did explicitly; so, explicitly just do the additions and then the deletions as usual.

Mark Erhardt: Okay, sorry, I have a question here. So, if you change the content of the tree, wouldn’t the hash of the inner node change and therefore the root change? So, if you already delete things that will only be used later, wouldn’t the intermittent proofs become invalid?

Davidson Souza: Yes, the intermediate states are invalid, but we don’t need them because we’re not checking proofs here. And then, that’s the next thing that we need to talk about. For this to work, you need SwiftSync, because SwiftSync already tells you whether the spends were valid or not. You don’t need to check the proofs if you’re doing SwiftSync, because the aggregate will tell you exactly the same thing that validating proofs would give you. So, the intermediate spends, they are invalid. You cannot validate a proof, like your block 100, and you have an accumulator built this way of adding and removing things implicitly. That proof that was computed before doing this is invalid for that particular accumulator, because that accumulator is different. But the thing we need is by the end of the IBD, when I reach the last block in that range, that accumulator is valid. That accumulator is equal to the one you would obtain during the normal operations. And during this process, we explicitly want to eliminate proofs. So, there is no reason to validate proofs in the meantime.

Mark Erhardt: Okay, I think I get the gist here. So, usually, if you naively did Utreexo, you would process the entire blockchain, but you wouldn’t store the UTXO set along the way. And therefore, you would have to process all the proofs along the blockchain, which would double the bandwidth roughly and require you to download all these proofs. So, if you instead do a vanilla SwiftSync on the part until you reach a predetermined height, you can check whether inputs and outputs canceled out for the ones that have been hinted to you, and you just build the forest of merkle trees that is the accumulator, for the height that you want to reach with the SwiftSync. And you depend on SwiftSync along the way to check that all the other outputs were created and consumed, because that accumulator cancels out.

Davidson Souza: Yeah, and I need SwiftSync as well, because I have that assumption that I know the spent-ness of an output when I create it. So, I get that from the hintsfile from SwiftSync. And we just changed the addition algorithm slightly to take that information in and already compute the updates when I add things. And the way I know that range is valid when it comes to removal and addition of UTXOs, and that there is no double-spends, and all of that, it’s not from Utreexo, it’s from SwiftSync. I’ll get that from Utreexo after I finish SwiftSync when I reach steady state. Then, it’s Utreexo that’s telling me that, but for IBD, you want SwiftSync.

Mark Erhardt: Right, I think I get it. Cool. So, that saves you about half the bandwidth. SwiftSync is also pretty quick. It might just be faster in IBD.

Davidson Souza: It’s way faster. I’ve just tested on a crazy-fast machine or implementation. We have an implementation of assumevalid SwiftSync that’s almost ready. It’s fully parallel, like everything’s concurrent. I had a VPS that’s, like, crazy internet. I finished in 19 minutes 45 seconds. And I think I can get better results, but at this time I’m like, I’m not going to bother. But yeah, it’s really fast. We are saturating every single network link we can come up with. The guy who is making it is Jose. He was already here explaining Witnessless Sync. So, he did the Witnessless Sync implementation in Floresta as well. So, he’s got an internet that’s like 800 Mb/s top. He was able to finish in 1 hour 30 minutes. My internet’s kind of bad because I live in the middle of nowhere, but it can finish in three hours, and very bad machines. So, we are basically going as fast as you can download things and it’s really fast and it’s super-lightweight. We already finished on like a Pi 3, all sorts of low-end. Like, I finished on my Pixel phone. I recently had an Android binding and in four or five hours, I finished on like a Pixel running over Wi-Fi.

So, we are getting some pretty low bounds to where you can run this. So, at some point, you can even go to sleep, leave your phone syncing, and when you get up it’s already synced, and you’re just validating; at least inside the assumevalid assumptions, you just did a sync. So, it’s really interesting results we’re getting. I’m very excited to have this ship in the next couple of weeks and see people trying it out in different setups.

Mark Erhardt: That does sound pretty awesome. So, this is a thin client, but due to the proofs it can actually validate unconfirmed transactions and can validate blocks fully, once it has reached the steady state? And you say it runs on your mobile phone? So, that’s pretty impressive.

Davidson Souza: Yeah. And now, I’m starting to work on the non-assumevalid version of it. This one is going to require more hardware. It’s very hard to make it, especially memory. It’s very hard to make it with low memory requirements. But already, we have the architectural means of making it also fairly concurrent, at least like Libbitcoin-level concurrent, for processing blocks in the non-assumevalid as well. So, that should also give us some interesting results, but that’s very preliminary work. I’m still figuring things out.

Mark Erhardt: All right. Is this discovery the reason why there hasn’t been an update to the SwiftSync BIP in a year?!

Davidson Souza: Yes! Sorry about that actually, sorry about that. But I wrote that blog as a first version of what I will eventually write in the BIP, because I have some updates. I still need some reviews from the team, but I have some updates for the 183 and 182. But 181, I’ll have to rewrite a lot of stuff there because of the changes. So, these are also like a brainstorming of what I need to update in the BIP for it to be compatible. But yeah, that’s largely the reason why we haven’t updated anything, is we are redoing things from the ground.

Mark Erhardt: Okay, so the old style would have worked, right? Would anybody still run the old style, for example the full nodes, the translation nodes that translate from the open network to the proof network?

Davidson Souza: You mean the bridge nodes? They don’t have to. And we are even working with the assumption that no one will request old proofs. So, we are building faster bridge nodes. I have one that can sync in like 20-something minutes. So, Bitcoin Core plugin uses libbitcoinkernel to get blocks from Core. It builds the forest, and then you can spin up Core again and it gets blocks from RPC and serves them over P2P. So, if you have Core and you want to use Utreexo stuff, you can just use this. I should announce this in the next couple of weeks because I’m finishing some stuff. But we are working with the assumption that historical proofs will no longer be required because they are quite big. You require terabytes of data to store them, and that’s kind of off-putting for most people. So, now, we’re just saying, like, 100 GB is already more than enough to keep recent-ish UTXO block proofs. So, you can serve that and you’re already helping a lot. Historical proofs, we’re thinking about dropping them.

Mark Erhardt: So, there is a total rewrite going on, because you’re actually incorporating this assumption throughout the proposal. Instead of it being sort of an alternative way of syncing, it actually will become the new go-to.

Davidson Souza: It’s slow. It’s better to do that way, because you don’t gain anything by doing the old way. Because validation-wise, SwiftSync also gives you full validation, the non-assumevalid SwiftSync. And you’re literally just spending more bandwidth. There is no other improvement for doing the old way, so we think we can just deprecate it since it makes things so much faster and so much easier infrastructure-wise.

Mark Erhardt: More bandwidth and more computation and more I/O.

Davidson Souza: Yes. Because bridge nodes for actually proving everything from genesis is super-slow. But we can just skip through and view the forest as if it were, like, block 900,000. So, you skip all the way to 900,000. This is insanely quick. I synced up to, like, 960,000, and I was using an old, one of those off-the-shelf desktops, like mini-computers that Tadge got me. He bought four for $90, like super-small machine, and I was able to sync in 30 minutes using that Core plugin thing. So, that’s super-fast, and if I can do that, it’s just way better, and there’s no reason to support anything old.

Mark Erhardt: Okay, okay. Great.

Mike Schmidt: So, there’s a lot of exciting tech that we just talked about, and I can imagine there’s a couple classes of listeners who maybe one wants to learn a little bit more about the details and maybe contribute. Obviously, we could post them to the Delving-Bitcoin thread, Davidson, that you posted about that we covered this week. But there also may be folks who want to start maybe building tooling around this or play with it a little bit deeper. Where would people go who want to start contributing to the project and maybe not just understanding the technicals at this level?

Davidson Souza: The SwiftSync stuff and all the client-wise stuff, we have an implementation in Floresta. There’s a PR with that, if you want to play around, test it, and give your results and maybe give some feedback. It’s on GitHub, it’s getfloresta/Floresta. There is a PR by Jose that implements all the assumevalid part. The non-assumevalid should open a draft PR in the next weeks maybe, just for people to review. And the Bitcoin Core plugin thing that I mentioned is also on the org. I think it’s called rpc-utreexo-bridge, something like this. So, there is an open PR there with the fast bridge building. You can check it out as well. And we’re planning in the next couple of months to revamp our blog. We have a blog in getfloresta.org, where we should publish some technical stuff of Floresta. So, I think Jose is already preparing one for his experiments with the assumevalid SwiftSync. I should also post some about some experiments I’m doing. So, check out the blog, keep an eye on it. We should update this with some cool stuff as well. I think that’s the main ways people can get in touch with the latest changes.

Mike Schmidt: Awesome, Davidson, we appreciate your time joining us today. You’re free to drop and move to other things.

Davidson Souza: Yeah, thank you for inviting me. See you all.

Vardiff controllers that strand slowing miners

Mike Schmidt: Cheers. Okay, so, I lied and that wasn’t the first item in the newsletter. We actually went a little bit out of order, but I think that also benefited Davidson and his schedule. So, we’ll go to the actual first News item from the newsletter, “Vardiff controllers that strand slowing miners”. Eric, I don’t know if you listen to our show or read our newsletter, but we don’t do a ton of stuff in the mining world. So, maybe you can summarize or maybe contextualize for folks on the software side of things, like exactly what are the components in play here; what’s the scenario and issue; and maybe we can get into the Delving thread then?

Eric Price: Yeah, sure. Can you hear me?

Mike Schmidt: Yeah.

Eric Price: So, the basic context for this is the mining pool, and it goes to something called a controller basically. So, the mining pool is trying to produce blocks for the blockchain and you’ve got a bunch of contributors working on small pieces of work that could be a potential block. And so, the basic loop is we’re trying to produce blocks and the miners are solving these jobs and they’re, let’s see, you’ve got a stream of shares coming from the miner and we’re receiving them. So, we’ve got a sensor and the pool’s trying to create a regular cadence of shares from all of its contributors. And so, it has a target share rate that it wants them all to produce output at. So, different miners have different efficiencies and capabilities, and so there’s this layer of indirection, called the difficulty, that lets it adjust how long it takes to produce the solution. And so, it measures how quickly it’s getting answers from each miner and compares that to the desired frequency that it wants, and adjusts the problem difficulty in accordance to the difference between what it wants and what it’s receiving. And so, that’s the basic feedback loop.

Mark Erhardt: Right, so the actual Bitcoin difficulty is way higher, of course. So, the mining pool gives each participant in the pool, let’s call them hashers, a custom difficulty that they participate on, and they will give shares whenever they beat that custom difficulty. Is that the vardiff that we’re talking about?

Eric Price: Yeah, within the mining pool. There’s a similar problem at the chain level, where you have all the miners basically contributing blocks to the chain. And then, every two weeks, the chain looks at how many it got and compares it to a target that it wants them to contribute at and adjusts the network difficulty according to the error, the difference between what it got and what it wanted. So, it’s a similar context in the mining pool environment, where basically it’s just the miners connected directly to the pool. It’s like an infrastructure thing. It’s like, how do we control the amount of traffic that’s coming into the entire server so that we don’t fall over.

Mark Erhardt: Right. You want to measure how much work each hasher is doing, but you don’t want to overwhelm the controller. So, if you gave everybody the same difficulty, the faster hashers would produce tons of shares, the slower hashers would not be able to give a signal at all. So, each hasher gets a difficulty that is custom to them. And then, what are you aiming for, like one share every ten seconds or something?

Eric Price: Yeah, so this is a pool-configured property. The pool basically writes down whether they want six or typically it’s in the neighborhood of six shares a second. But at the chain level, we have what a block every ten minutes, so that’s how the problems map. So, I set out this project just trying to see if I could – and let me say, we’re working primarily in Stratum, Sv2 world, so a lot of my context is from that. I had some prior experience in Sv1 world, but a lot of things are different. So, just to give you context.

Mark Erhardt: Yeah, talking about context, could you tell us again what problem you’re trying to solve? So, we have different speeds, we have introduced the concept of shares, and now what is happening and causing an issue?

Eric Price: So, originally, I was just trying to optimize the vardiff that exists in Sv2. I wasn’t necessarily trying, I was just optimizing, looking to improve it. But as I poked around at the algorithm, looking at ways to like – when you first connect, sometimes the machine, the miner, it starts at a low difficulty so the share rate is very high. And then, it waits until it sees some shares before it has enough information to adjust that down. So, one of the first problems was trying to keep network traffic from spiking. But I soon realized that there wasn’t a lot of wiggle room in optimizing the vardiff because there’s this wall. The problem is sort of information bound by the stream of data that comes in itself. And if you try to make it react faster, you lose precision; and if you try to make it more precise, it becomes less responsive. So, you’re really bounded by the rate of data that comes in, by the share rate; the target rate that you’re setting, it is a proxy for that. It may not be that value when you first connect, but as it converges to the target, six shares a second, that’s the rate of data that you get in, and you can’t get around that anyway except changing that rate itself.

So, I was getting frustrated trying to do this optimization problem, because every time I improved something, I lost something else. And then I noticed something. I was using AI and doing a lot of runs, changing different parameters, and I stumbled on something where I was playing with the adjustment rules. And so, you can either adjust it up to produce shares faster, or you can adjust it down to have shares go slower. And I started playing with an asymmetric rule here, where one side is adjusted harder than the other side. And I noticed that if you increase the share rate more than you decrease the share rate, if the rule is to push the share rates higher more easily than to push them lower, the machines happen to live longer, they just stay up longer. And I thought this was very weird.

Mark Erhardt: So, let me try to paraphrase that. So, you found that if you allow them to do more shares, they stay live longer. So, there was an issue with liveness if the share rate was too low?

Eric Price: Yes. If for whatever reason your share rate is lower, I’ll just say it’s generally in a statistical sense, you’re not likely to live as long as a connection to a mining pool than if your share rate is above the target.

Mark Erhardt: So, when would this be happening? If a miner is spinning down or if it’s suddenly running slower, the mining pool would just sort of lose track of it and disconnect it?

Eric Price: Yeah. If it’s below the target, it’s more difficult to deal with generally. It’s more difficult to see, it’s more difficult to know if something changes. Basically, it’s like this, and I’ll just kind of reason through it. For whatever reason, say there’s some sort of event that happens, maybe you have a power outage or, you know, it’s solar powered and the power goes down, or for some reason there’s some disruption and you’re hashing at a reduced rate, maybe you had a hardware issue or something. You had a certain difficulty and you were producing supposedly at rate, so your target was fine. And now, something’s happened and you’re producing shares at a lower rate. The controller sees less shares, it has less information about what is happening. So, let me look at my notes here just so I get the sequence right. Yeah, the shares slow down due to some initial disturbance. The controller, the estimator has this window where it’s watching the shares as they come in. Well, it’s taking longer for shares to come in, so it’s seeing less data, so it has less certainty about what the true hashrate actually is. And so, the vardiff controller can’t push you back up, can’t adjust your difficulty to save you from the situation, because it has less information.

Mark Erhardt: So, reading our write-up of what you posted about, it sounds like the vardiff is often adjusted at a specific number of shares. So, when the shares start coming in much slower, the time until the next adjustment becomes greater. So, if some hasher is in a continuous decline for hashrate, they might run into a situation where the number of shares they have to bring in order to provide enough information for an adjustment is hard to reach and they get disconnected before they produce enough shares. So, I think you had an idea how to improve the adjustment of the vardiff?

Eric Price: Yeah. So, the issue here is that this whole loop is event-driven. You need the events to drive the loop to get you out of the problem you’re in, but the problem is that you’re getting less events. So, that’s the vicious cycle. And the way out of it is to perceive the passage of time within this cycle, okay, and there’s different ways to do that. But you need to know that your events are spreading apart. And the only way to do that is to inject emptiness into this window of measurements. You need to see that there’s gaps of time between things, and take the absence of events as an event in itself. And there’s parallels to this on the other side; the blockchain problem. And you can look as recently as this last fork, the Luke fork. When the chain split, his fork was sitting there at a high difficulty and the miners needed to produce blocks, but they couldn’t reach the difficulty. And so, they weren’t able to trigger the difficulty calculation to lower the difficulty in the first place.

So, we’ve seen this before, and Bitcoin’s own control loop is not particularly sophisticated. And you can look all the way back to our first major chain split with Bitcoin Cash. There’s actually academic literature that followed that, where they went and tried to add a timing aspect to the difficulty controller so that they could actually get blocks into the chain in the first place. And they made a couple of different adjustments over several years. The first, and this is actually really interesting and I’m going to go into it when I talk at TABConf, but their first attempt was called the emergency difficulty adjustment. I think it was a hard fork; it was a fork. And it made it so that when you wait a certain amount of, time the difficulty automatically increased. But there’s a big difference between a mining pool and a blockchain.

Mark Erhardt: Sorry, the target increased or the difficulty got lower, right?

Eric Price: Yeah, I’m sorry, I forgot what I said.

Mark Erhardt: No, you said the difficulty increased when the time was too long.

Eric Price: Sorry, no, it got easier for the miners to put blocks on the chain.

Mark Erhardt: Sorry, let me jump in a little more. You described the problem that if the difficulty is way too high but it takes a fixed number of blocks in the blockchain for the adjustment to come, the chain might starve. So, for example, you brought up the Bitcoin Luke’s Vision fork recently. So, the difficulty was way too high and we were projecting some two years or so until it would adjust, and it would have cost millions of dollars of hashing to get there. But we also have the same issue on testnet. So, on testnet, we had a different difficulty adjustment rule; we had an exception. If the time was more than 20 minutes, we would allow a low-difficulty block to be found. But we had to remove that recently because it caused other issues. And now, we’re a little worried with the proposed testnet5, if some miner wants to troll us and points hashrate at the testnet5, it might run into the situation where we starve, because the difficulty was run up by the miner that pointed their hashrate at it, and then we would need to get a certain number of blocks to reduce the difficulty again to the actual hashrate on the testnet.

So, we have the same situation here for hashers in a mining pool, where for some reason, the difficulty adjustment is based on the number of shares that have been produced. But when they are winding down and are finding fewer shares, they might starve before they reach it. So, you’re saying instead of counting or waiting until all the shares have been found for the next adjustment, you’re injecting emptiness in between. You’re just counting when you get fewer shares, how much time has passed, and you realize that they are way below the expected rate. And so, it sounds like once you get a low number of shares in a period of time, like the difficulty adjustment that Bitcoin Cash used, where after six hours, I think the difficulty would halve, or go down even more drastically. Or was it, I think, six blocks in six hours, or something, the difficulty would halve. But it also had some interesting consequences. It caused miners to deliberately not find blocks on Bitcoin Cash and had the effect of the oscillating hashrate. In this case we wouldn’t have an adversarial situation, because the hasher and the mining pool are actually trying to work together.

But so, the solution is just to look at the shares are coming in way slower, and now we adjust based on time instead of hashrate, is that right?

Eric Price: Yes, conceptually based on time. The final evolution of the Bitcoin Cash saga was an algorithm they called ASERT, which was basically the final evolution of their estimator. In the paper, I kind of talk about vardiff as being composed of three parts: the estimator, whose job is to have a guess at what the true hashrate of the miner is; and then, the boundary which decides whether that hashrate needs to be adjusted or not; and then, the last part is the magnitude of the adjustment. So, what the BCH literature looked at was the estimator part, where they said, “Of all the ways we could inject time into the system, what seems to work best for us is to use an exponentially-weighted moving average”, and it solves some of the problems you were mentioning before. So, with just a regular average, you have a window of a certain size, and we’ve been talking about this wall which governs what happens when you go between a short estimator and a long estimator. The long estimator, you have a lot more data and so you’re not bouncing off of yourself, you’re not seeing these kind of reverberation effects; whereas the shorter window, you can respond quickly to changes, but you don’t know what the source of the changes are, you might be responding to yourself. Go ahead, Murch.

Mark Erhardt: Right. So, if you, for example, just used an average, and then you found 50 fast blocks and 50 slow blocks. Before that, you had fast blocks. As you find the 50 slow blocks, it keeps slowing down a little bit. But then, let’s say the energy goes back up and your issue is resolved and you go back to the regular speed. Now, it also takes the 50 next shares to move back up. And also, the adjustment would be very slow, because first it’s just one slow block and 99 fast blocks; two slow blocks, 98 fast blocks, and so on. Whereas you want it to be more reactive if something changes, but you want the context, the broader context of the entire situation, right?

Eric Price: Right. So, this exponential mechanism takes the benefits of the short window, and the benefits of the long window and combines them by basically attenuating the impact of the older samples. So, the window grows in a way that doesn’t cause other issues. The content within the window grows unbounded, but the size of the window itself stays fixed, which prevents some other issues. There’s an issue that’s live in the Sv2 vardiff today that has to deal with an unbounded window, and the exponential version fixes those problems, and you can see it through the history of what happened to BCH. So, that’s why I think it’s very interesting.

Mark Erhardt: Right. So, you basically were able to use BCH as a study of the problem that you were trying to solve and look at their solution space, evaluate it. And what you came up with eventually was that you put a higher weight on the most recent share latency and a lower weight on the older, but it still factors in. And that way, you get a faster adaptation to the changed circumstances, but don’t lose the context completely.

Eric Price: Yeah, so we don’t see as much churn or chop from the window having a finite size. Like, it’s hard to explain what the effect is, but it messes with the payout variance.

Mark Erhardt: Yeah, it removes the overcorrections, right? If you have just the average, you under-correct and catch up for a long time. But then, once the situation is improved, you suddenly have way too many shares. So, you might overwhelm the controller with additional data that you wouldn’t have to send because there’s too much information now. So, you sort of adjust more quickly, but you don’t lose the entire context. That sounds like a good solution for the issue.

Eric Price: So, now we’re looking at ways to kind of bound and control the excess of shares now. We’ve sort of fixed one problem and made an easier problem that needs a little bit of work. But one interesting thing I wanted to bring up, in comparing BCH to mining pools, not BCH, but chain difficulty versus pool difficulty with – well, I didn’t get into the asymmetry part. Well, we started to touch it, but when you refuse to increase difficulty as much as you would increase the share rate, the asymmetry, where you are reluctant to tighten up the difficulty and eager to increase it, more share rate, that creates this kind of permanent offset where the shares are going faster than they should by kind of a regular offset. In the chain context, that causes the miners to receive their mining subsidy faster. So, the fix that Bitcoin Cash made caused the emission rate of the chain to increase, which sort of violates consensus.

Mark Erhardt: Well, more like the reward schedule that had been communicated earlier. Adjusting the difficulty calculation is already a consensus change, right?

Eric Price: Yeah, true. But it was a no-go situation to have the emission schedule accelerated on the chain. But in the pool context, almost all of the ways we do share accounting is with proportional-valued shares, like your higher-difficulty share is worth more than a lower-difficulty share. So, the same solution of being asymmetric about how you adjust the pool difficulty doesn’t have the same consequence as it does in the chain context, because we don’t have an emission schedule, we don’t have a fixed value per share. So, we don’t need to worry about that concern. We can apply this asymmetric technique to our controller problem without worrying about the chain concerns. So, the same technique that was initially tried and rolled back in the Bitcoin Cash context applies fine in the mining pool context. And so, that’s sort of the gist of the paper, that asymmetry kind of fixes this kind of intrinsic problem with making your miners hash with harder problems. You lose sight of them because it takes longer for them to get – if they’re not actually efficient enough to produce shares with that difficulty, if they’re going below the difficulty that they’re capable of producing, then you lose sight of them and you can’t see what happens if something else changes. That’s the gist.

Mark Erhardt: To summarize it, the solution didn’t work for Bitcoin Cash perfectly, because the shares basically were also what produced the block rewards. So, their emission schedule was changed and that was an unintended side effect. But in our case, in the mining pool, each share only counts at the weight of the difficulty that it is mined at. So, there’s no fixed reward per share, but rather the shares are weighted in their own context, and thereby the asymmetric solution solves all the issues here, but it wasn’t the right solution in the context of Bitcoin Cash. Basically, if they had adjusted the reward to be based on the difficulty or something, well, now we’re getting outlandish and too much into the details. But thank you for covering your thought process. I think we got into all the details of how mining pools can now adjust the vardiff calculation.

So, where is this going live? So, the interesting thing of course is that mining pools don’t have consensus. So, they need to run on software that they know what it’s doing, but it is client-server relationship. So, you can update software and just have a different calculation scheme. You said you’re doing this in the context of Sv2?

Eric Price: Yeah. But I think to your question, the impact of this I think is more responsive miners, less jankiness when something happens to your mining fleet. You’ll be able to see it under adverse situations and keep on chugging no matter what happens. That’s what I think is the end goal of this.

Mike Schmidt: I saw Portland has his hand up and he also did 20 minutes ago. Sorry Portland!

PortlandHODL: Oh, no, it’s no problem. I had two quick thoughts on this working, specifically at MARA at one point, the vardiff was more problematic in our cases during startup. So, it would start, like you said, at a low value, and especially when you have a site with a ton of miners, you would completely flood the traffic controllers, etc. So, that’s very interesting that this can possibly be solved on especially the downward side, but still, that is the primary pain point there. The second question I had is, the vardiff, from my understanding, is like a miner reports in and you say like, “Hey, okay, you’re reporting really quick, I’m going to give you a harder job”. And in the case of some miners, am I correct that it is an exponential function, like you can do powers of two, or is it a number that you can actually set any value to?

Eric Price: So, this is interesting.

PortlandHODL: Because I had this issue with the Antminer specifically. You could do powers of two, but you couldn’t set it like, “I just want this exact target”. So, you kind of had to guess which one is the best.

Eric Price: Yeah, so I’ve noticed this, kind of poking around in some of the Sv1 pools. A lot of them are always setting difficulties to powers of two. And I’ve noticed this in some Sv2 pools, and I’m trying to dig out to the origin of this. And it seems like it started in CGMiner, an open-source mining software that actually gets run inside of the firmware. And so, I think it started on the firmware side, and then the pool started adapting to that and locking it in, and it’s bad for vardiff. It removes variability. And now, I think the firmware has moved on largely, but some of the pools still have that in there. So, it’s sort of this legacy bug. It’s bad for vardiff, it’s bad for the visibility of your miner, and I’ve seen it in our pool and some others. So, it’s something that we need to eradicate. I don’t see it in Sv2, so I think the future is going to be absent of that artifact of the past.

Mark Erhardt: So, there’s no hardware requirement for the specific hashers’ difficulty to be two?

PortlandHODL: I would research that very closely. Most documentation and firmwares I’ve seen for older Bitmain machines, S19 and previous, seem to require a power of two. Please do not hold me to that though; that’s just from my own experience. The next question I had specifically is, in my experience with vardiff, the machine would start off just absolutely blasting shares off to your pool like crazy when it fires up. But once the machine was stable, it typically hit like, okay, it wasn’t exactly – the target was 30 seconds, but it was still good enough. What would actually cause a machine to basically stop reporting at that specific rate? I’m going to give my answers to what I think could happen. You could lose a hash board, you could lose chips, or you could downclock. But how common is that? Like, how common is it to find a miner that is now slowing down, or am I missing something there?

Eric Price: Well, I’m trying to do some experiments now to catch them live. It’s hard. You’re trying to catch a rare event. But I mean, we could induce one. I mean, I can have somebody go into a lab and pull a board out, and I might have to do that to do some of these experiments. But what was the other part of your question? Yeah, it is hard to find them.

PortlandHODL: Yeah, I was wondering if there’s other use cases that would cause, like, a Stratum connection to appear to be more variable, because in my experience, usually once you hit that, it’s hashing, unless it blows a hashboard, it’s pretty stable overall, the vardiff is constant. And then, a proxy was my last one. Like, if you have an entire pool proxying machines together to send over shares, that is the case where I think this would be incredibly useful.

Mark Erhardt: But also, in the case of a whole pool, wouldn’t you have a much more gradual reduction, because you wouldn’t turn off the whole pool at once? You might lose one of the machines or maybe downclock slowly through; no?

PortlandHODL: Curtailments in certain sites mean lever gets flipped, you lose several thousand machines on proxy, right? So, the proxy aggregates all these machines into one connection. So, it’ll appear like, “Hey, we just lost a significant portion of our hashrate very suddenly”. And then, that can at any moment be flipped back on.

Mark Erhardt: Right, but that wouldn’t be 90% usually, it would be more; or would you actually run into the 30-second limit that AJ was proposing?

PortlandHODL: I don’t believe so. But the curtailments are variable in the size, like it can be 5%. You’re given a power target you’ve got to hit and you’re supposed to turn off machines till you hit it. It can be 90%, it can be 95%.

Mark Erhardt: I see. Eric?

Eric Price: But I think the promise here is a partial. Like, what happens today is there’s no way to do a partial curtailment. You just turn them all off and then turn them back on. But we could have partial. And to the point about the share storm, first of all, when you first turn on the miner, with Sv2, that problem is done. There’s a protocol carve out. On the initial connection, you can parse in your nominal hashrate, like, your sticker hashrate. And so, that’s basically feeding forward. You don’t even have to wait for the convergence time at all. It comes out of the box for free. But if there is this decline and a cutoff, then you’re ending up reconnecting multiple times. So, if we’re finding a vardiff issue that induces reconnects, then we’re back to having this share storm problem. But if we can seed in the natural hashrate, then we don’t have to worry about this anymore, at least for native Sv2 firmware.

PortlandHODL: Thank you, that was pretty cool. Awesome.

Eric Price: If I missed any questions, let me know.

Mike Schmidt: Well, we can direct folks to your post as well, which is available in the newsletter, and folks can follow up there. Eric, thanks for diving deep with us on that one. We appreciate it. You’re free to drop if you have other things to do.

Eric Price: Yeah, no problem. Okay, thanks.

EntropyLab offline key calculator

Mike Schmidt: “EntropyLab offline key calculator”. We have Portland to talk about this. Portland, you guys created (checks notes) an HTML file. Why is this cool? What is this HTML file?

PortlandHODL: This HTML file is actually pretty cool. There’s going to be some caveats here in a moment. But basically, the problem that we were trying to solve at EntropyLab was essentially how do you validate and verify the outputs of various hardware wallets and entropy generation solutions for the deterministic side. So, for example, we have hardware wallets that can offer dice rolls and stuff. For their users, how can you prove that those solutions are being honest, or as it expanded, because there was scope creep throughout the project, how can you convert other forms of physical entropy into a seed phrase essentially? This was essentially started by MrHodl on X, who is not a programmer or a dev by any means, and a great guy, but he just used AI, specifically Claude, and generated an HTML file to solve his issue of converting dice rolls to a seed phrase. And then, it kind of just took off, like, okay, what else can we verify, down to PSBTs, different forms of entropy, later on generating a wallet.dat file from that entropy. Like, direct imports from that HTML file creates a wallet.dat that you can check your pubkeys to see like, “Hey, is there a balance here?” replacing kind of the BlueWallet as a recovery tool.

My notes, before I get too sidetracked into this, this is 100% not designed for any mainnet bitcoin. I don’t want to be responsible for anything like that. It calculates tpubs, xpubs, but this is really only for verification of existing things. Do not put in seed phrases or anything involving real bitcoin into this EntropyLab HTML file. I’m just going to be very clear that this is not the tool to put real bitcoin with. If you have maybe a brand new hardware wallet and you want to test some dice rolls without any real funds first to see, like, “Hey, does it generate the correct seed phrase given the dice rolls?” that’s probably okay. Just don’t use the same seed phrase again, wipe it, etc.

Mike Schmidt: So, I maybe just want to spit back the context. So, we had this COLDCARD incident and then everyone was like, “Well don’t trust the device’s randomness”, was the natural backlash. And so then, memes about dice-rolling and, you know, “You’ve got to do that”. And I guess the natural reaction was then like, “Well, how do I know if I put in my dice rolls, what if that conversion within the hardware signing device is incorrect and it’s actually showing me seed words that aren’t corresponding to my dice, and maybe that’s a bad thing?” So, in that instance, as well as all the other use cases you mentioned, it’s a way to validate that that device in that instance is behaving.

PortlandHODL: Absolutely.

Mike Schmidt: Yeah, okay.

PortlandHODL: There’s other things as well. For example, like RFC6979, which is when you sign your transactions, you have to generate this nonce. And there are very specific properties of a nonce that it cannot be reused for the same message, like, that you need to verify it’s deterministically generating on that device. So, we have a tool like, “Hey, you can check your PSBT signatures to make sure that with this specific key, this lines up”. It’s deterministically generating, it’s not leaking information. Dark Skippy was an example of that. You can also validate anti-exfil, all these different functions of a hardware wallet. This is supposed to be the one-stop shop to test, to validate that, “Hey, this is my expected output from the device”, and it gives just another reference point to validate these devices.

Mark Erhardt: Sorry, I have to get nitpicky about something you said. You said you’re not allowed to use a nonce for the same message. That is safe. Using the same nonce for exactly the same message will produce the same signature because it’s deterministic. If you use the same nonce for a different message, you leak your private key.

PortlandHODL: Correct, and that is a very bad outcome. So, yeah, going forward, it quickly became that even things like BIP85, all these things need verification. And the tools available to do these kinds of tasks, from the libraries to AI specifically, are all present. Like, Rust Bitcoin exists, right? We have the ability to derive things, we have the ability to convert entropy to seed phrases, follow these standards, BIP32, BIP39. How do you put them all in one place? Ian Coleman previously existed; Jameson Lopp has a site where you can convert your xpubs and get the little parts and bits; how do you put all this together in one place? And that’s kind of the next thing that happened, was not just how can we get a seed phrase into an xpriv, or whatnot, or wallet.dat, it’s how do you put all the things together? And the answer very quickly, just hang out on Twitter Spaces, use AI models and burn tokens, and you will end up with a probably pretty overall good solution to your problem.

So, the nuts and bolts of the whole thing became Rust Bitcoin was the baseline for everything. It was the framework everything was built on, because Rust Bitcoin could be compiled into WASM, WebAssembly, essentially. So, you got all the libsecp256k1 code Jonas Nick and everybody wrote that is, in my opinion, beyond excellent. And you get to just kind of bolt that directly into your website. And then the AI essentially was able to link the pieces together, take these bindings, follow the specifications and standards, and produce very usable input frameworks so that the user could put in their dice rolls, put in their entropy, give them a live hash, etc, and combine all these pieces together to form EntropyLab.

Some nice little tidbits of things that were actually – one new thing that came out of this whole thing is there is now a direct Bitcoin Core wallet.dat js library. So, you can convert a mnemonic directly into a wallet.dat file, or give it descriptors, and you can just build these things out. That’s a part of EntropyLab. I think that’s pretty neat. And from there, what kind of also came about is how did we really come up with the use of AI being acceptable for this kind of tooling, right? This is private key material; this is applied cryptography essentially. And that decision came about because of, one, cost and accessibility. So, EntropyLab, I think, given the number of features it has, like you have everything from like multisig, descriptor creators, PSBT editors, validators that it’s consensus valid, BIP85 child derivation, silent payments, vanity address generators, the time to build all these features out by hand, if I had to sit there and just type it out, I think it would have been on the order of months. Maybe if I was really motivated a little less, but this happened in two weeks, we were able to create all these functions and tools because of AI.

The cost overall was incredibly low. For the entire suite of EntropyLab, I could probably say it’s around $2,000 in AI credits. Given the cost of a developer, that if they were to take their own time, let’s say they’re senior, because this kind of work is typically senior, that would be maybe like one to three days’ worth of their work, and they still wouldn’t have it done. AI was able to complete all these tasks for that same cost of one to three days’ worth of work. The next component is what the Bitcoin Red Team, I think, also showed incredibly clearly, is that AI and its ability to analyze code from a security perspective, I think is outperforming humans, and not by a little bit, but by a lot. I think that projects with longstanding, very talented developers, like Bitcoin Core, secp256k1, that have had tons of brain power, tons of time to review, fared incredibly well against AI. But in terms of more niche, less reviewed code bases that don’t get access to the top talent, those projects don’t fare as well, in my opinion.

Mark Erhardt: I wanted to jump in there. Review is one thing, but also, Bitcoin Core especially has built out a core set of all sorts of different testing strategies that I think other projects should look more into, and that might also be way easier to bootstrap now with the assistance of AI. So, if you haven’t been doing fuzz testing yet, I think it is way easier to get started with that now. Maybe, I don’t know, mutation testing is probably similar, and so forth. We’ve been doing a lot of these things. We’ve probably been on the forefront on some of these strategies for testing software. And I think that is a big contribution. Niklas Gögge had a great blogpost about this the other week. So, yeah, I think getting more creative with your testing strategies is another way where you don’t have to throw just money into the AI oven to test every time a new model comes out, but you can actually be proactive instead of reactive.

PortlandHODL: Absolutely. And, for example, fuzzing, EntropyLab uses fuzzing as a method, like even during CI, and we have some runners externally to fuzz different components of it. And that process, just from using, for example, an open-weights model like Kimi K3, that is literally like $3 of credits to build an entire fuzzing suite, obviously human review is necessary to kind of say like, “Hey, this is legitimately doing what it should be doing”. But my point more so about a project like EntropyLab, which is open source, is we don’t get access to Pieter Wuille to double-check our solutions. So, with that stated, AI has provided, I believe, a sufficient alternative for security review and in the current and test framework generation.

So, yeah, that’s kind of the interesting thing, is that this project is the first example of a project where instead of leaning on the humans as the primary backbone for code generation, review, and validation, we moved towards a direction of realizing the speed at which these tools should be delivered. The accuracy of AI can present, in terms of its review, from a security perspective and finding holes, bugs, logical inconsistencies, deviating from specs, is incredibly good. And so, with that, we just, as a project, leaned into it as we kind of nicknamed it Team Oogabooga, because we have become the Oogaboogas, the simple primitive beings.

Mark Erhardt: So, we’re now proud of being meat proxies, is that what you’re saying?! No, sorry, I’m just teasing. But yes, I’m not sure if you are the first. Rearden Code has been working on his rbitcoin.org thing.

PortlandHODL: And Ibis Wallet.

Mark Erhardt: Yeah, Ibis Wallet too, okay. But yeah, this is definitely, I guess, a new advent of how to develop software.

PortlandHODL: Yeah, and I just want to close out that I’m not saying that this model is right for every path. Like, a production path, probably not maybe the best idea to just 100% go all-in on AI and just burn thousands of dollars of tokens and just hope the result is right if you don’t have any technical competence on your team at all. But at the same time, a project like EntropyLab, where your budget and time constraints are very low and you’re trying to achieve a great output, which is to create a validator for basically any type of function on a hardware wallet, you’re kind of forced to use these tools to produce those outputs. I can’t spend my entire day building out a BIP85 derivation tool for when I have a regular job I’ve got to maintain. And so, that became the rest of the team as well, is we all just leaned into it and then built on top of these AI tools to create EntropyLab.

Mike Schmidt: Portland, if I’m using this tool as instructed and I somehow validate my download, I’m doing this offline AirGap, all this good stuff, and I’m using it for verification, other than legal concerns, what is the concern with using it in, I don’t know, a production mainnet setting; and by legal concerns, I mean the developer’s legal concerns?

PortlandHODL: So, yeah, I’m going to say this very in my opinion. I have used EntropyLab for my mainnet bitcoins before, specifically the act of splitting, because I was dumping the Luke Dashjr fork for some extra bitcoin and I wanted to validate that my PSBT’s input belonged, the txid was from the new chain, not my previous input, because that would cause a fee issue and a bunch of other problems. It works just fine and it told me, “Hey, you had a problem”, to which I resolved that, which I’m very thankful that I had this tool because it was very easy to go into the PSBT Editor and see. I think that the main thing I have a concern with anybody using this for mainnet bitcoins is I just don’t want to be responsible if there is a problem. I’m not willing to take on that liability. Because I think, like, Rob Hamilton, for example, made a post that a computer can never be responsible, or summarizing it, and accountable.

At the end of the day, it’s not the LLM that’s responsible for the bad code, it’s the person who authored and pushed that code that has to be their head’s on the line, basically, and that’s going to be in any environment. There’s got to be some ability to point to something and go, “That was the problem”. I’m stating very clearly that I don’t want anybody using this tool for mainnet bitcoins because of the fact that I don’t trust this software in its current state. Another issue as well to say why you shouldn’t use this, there hasn’t been a formal 0.anything released. This is still just a master branch constantly evolving every day. Things are in flux.

Mike Schmidt: That’s fair. Yeah, I think there’s like a 1970’s IBM engineer quote about the computer can’t make a decision because he can’t be accountable, or something like this, yeah. That’s fair.

PortlandHODL: Yeah. But at the end of the day, I am fairly confident, in my opinion, on the validation testing of the frameworks that it’s built on and the ability to link those together to form the outputs that are needed to verify. So, basically, we’re using Rust Bitcoin for all the internals, and then really it’s just gluing it all together. And as long as the glue logic is correct and adheres to the specifications, you should be fine.

Mark Erhardt: Yeah, if only. I mean, the COLDCARD thing was also a problem across glue boundaries, right?

PortlandHODL: Yeah, that is correct. They did not glue on the entropy generation.

Mark Erhardt: The good thing is it’s a fairly small project. I think the other thing that I want to point out is I assume it’s licensed under some open-source license, and almost all of the open-source licenses are no warranty, you run it at your own risk. And that is on purpose, because you’re also not paying for it. While the developers are definitely going to try to do their best in general, there’s no company to sue. It’s provided as is, no warranty. And I think more people need to remember that that is how open source works. And while certainly mistakes are made on the developer side, it’s also partially the responsibility of the users to help review and audit and make sure that things are verified sufficiently.

PortlandHODL: Yeah. We essentially used the Unlicense. We did have a rewrite of it called the Oogabooga license, just for a little bit of thematic elements essentially, and we did say that this includes all the previous conditions of the Unlicense. But yeah, just it goes no warranty, not responsible for anything. Just yeah, you’re using this software 100% at your own risk completely.

New BIP draft for unspendable internal keys

Mike Schmidt: Portland, thanks for your time and joining us. I do want to offer, if you want to stay for the remaining News item, that if perhaps in your capacity at AnchorWatch and, I guess, even previous involvement in your Bitcoin endeavors, you want to talk about the draft BIP for unspendable internal keys? Sounds like he’s sticking around.

PortlandHODL: Yeah, I’ll stick around. I’m going to note I have not reviewed that specific draft BIP.

Mike Schmidt: Okay, no problem. Well, maybe Murch has anyway.

Mark Erhardt: Well, it hasn’t even hit the inbox of the BIPs repository yet. It’s so far just a mailing-list post that announced that someone had picked up a previous, well, we previously had one or two attempts where people were trying to define how you would make a descriptor or a wallet policy in which the internal key is not spendable. So, a nothing-up-my-sleeve (NUMS) point, which means that there’s an unknown discrete logarithm you cannot actually, as the creator of that output script, spend with the internal key. So, basically you have a P2TR output for which the keypath is human-disabled. Now, maybe also the caveat right away, quantum computers could still, if and when they exist, revert that logarithm and then spend. And funnily enough, that would also give the owner of that output script a proof. They can show that it was a NUMS point and they could prove that someone broke the discrete logarithm assumption.

So, yeah, basically from what I saw, there’s a new write-up, a draft that is being proposed, how to use the unspendable internal key in descriptors and wallet policies, and I’ll have a stronger opinion on this once it hits the BIPs repository.

Mike Schmidt: So, yeah, we did talk about this a while back. It’s been a couple of years we talked about it. There was a delving discussion. It had Salvatore, Pieter Wuille; that was back in #283. And then, Andrew Toth took a crack at it, and we talked about that in #338, if people want to go back and review that discussion. Maybe just to recap, I think, Murch, you said it, essentially disable the keypath spend. Two questions. One, why would you want to do that? What are the use cases there? And two, isn’t there this sort of built-in NUMS point, and why not use that?

Mark Erhardt: So, the built-in NUMS point is present, but then you don’t want to exactly use the NUMS point, because when you reveal your scriptpath, you have to show the internal key as the hashing partner of the merkle root, and thereby you would show that it was a NUMS point. So, I think it is always a tweaked NUMS point. But the question is not whether or not to use the NUMS point, but rather, how this would work in descriptors, how you could express the absence of an internal key in a descriptor or a wallet policy. So, the concept is available, has been discussed since 2022 at least. The idea would be you only want scriptpaths to be available. So, you have a script tree with maybe multiple alternative spending conditions, but the keypath is excluded, because you only want script leaves to be able to spend that UTXO. So, did I cover all of your questions?

Mike Schmidt: I think so. Would there be a privacy implication then too if everyone’s just using the NUMS point?

Mark Erhardt: Yeah, if everybody were using the NUMS point untweaked, every time someone spent from a scriptpath, well, I’m going out on a limb here, I’m not 100% sure, but from what I remember, in BIP341, a NUMS point was proposed that is used to construct the internal key, and you would tweak it in order to hide which one it is actually. Because if you just use the same NUMS point all the time, once you do a scriptpath in the control block, you have to reveal, “Okay, here are the hashing partners for the merkle tree”, and then you also have to reveal the internal key. So, everybody that uses the NUMS point, if it were just the NUMS point, would reveal the same internal key. So, every scriptpath spent that forbids the keypath would become identifiable by that. So, it is tweaked. And by tweaking a number into it, you don’t learn the key. So, it is still a NUMS point, and you can prove later that you knew the tweak, and thereby it was derived from the NUMS point, if someone needed that proof. But it is no longer discernible for other people that you used the NUMS point unless you prove it.

Mike Schmidt: Portland, any thoughts, or pass?

PortlandHODL: I’m going to pass on that. I don’t have enough insight or information towards this specific proposal.

Mike Schmidt: For listeners, jump into the summary on the newsletter, and there’s also then the underlying post by NTL that is reinvigorating this discussion, for more information and to participate yourselves. Portland, thanks for joining us. Oh, go ahead, Murch?

Mark Erhardt: Also, if I said something wrong, feel free to correct me below this show somewhere.

Mike Schmidt: Yell at Murch on Twitter!

Mark Erhardt: Yeah. Flame me and correct me so I learn.

BitBoxApp adds Spark-based Lightning payments

Mike Schmidt: Thanks, Portland. We’ll see you. We have two other Changes to services and client software that we covered, in addition to the Oogabooga guys’ EntropyLab work. There’s BitBoxApp adding Spark-based Lightning payments. So, I’m emphasizing a couple things there. One is this is the BitBox application, so this will be on, I think it’s their mobile app, and it’s version 4.52.0, adding support for Lightning payments through Spark. So, it’s built on the Breez SDK and it’s using the Spark statechain. So, you can do Lightning payments. It’s not necessarily using that hardware device that people know BitBox for; it’s in the app and it’s a mobile app. So, the funds are held in a hot wallet on the phone rather than on the hardware device, from my notes.

Mark Erhardt: Oh, really? So, not even the signatures are being done with the BitBox?

Mike Schmidt: I do not think so. Now that you ask me, now I’m anxious about making that statement.

Mark Erhardt: I haven’t looked into that but my underlying assumption would be that they use the BitBox for the secret management, but then the mobile wallet app for the liveness. But, well, I haven’t looked at it, so I might well be wrong. I wanted to point out a couple more things about statechains. Statechains are semi-trusted, in the sense that the statechain operator could cheat with any previous owner that held the same state coin. When funds are transferred in a statechain, the secret is re-sharded by the statechain operator and the new owner, I guess. So, you’re reusing a different split of the same secret so other people can spend the same coin in a statechain if they go back. So, you’re relying on the statechain operator to faithfully execute the protocol.

The other thing that you rely on is the statechain operator gets full insight on your payments. So, this is apparently a way of making Lightning payments from something that you have semi-custodial control over, but I just want to point out that the trade-offs, both privacy and security, are slightly different than vanilla Lightning. It is of course a trade-off of convenience, where running your own Lightning service or Lightning node has other issues. Especially on a mobile phone, the liveness is still a difficult problem; we’ve reported numerous times on async payments, not the company, ACINQ, but asynchronous payments. So, I just want to point that out because I believe some marketing material called it just outright non-custodial, and that is kind of offensive.

Mike Schmidt: Yes, I think this has come up several times in discussions I’ve seen online, in that the nuance either gets glazed over on the marketing, and then there’s a fight on the back end about the technical. So, appreciate you clarifying that, Murch.

Gustavo Flores Echaiz: I want to add that Murch was right about the secret being derived from the hardware wallet using BIP85, but the Lightning or Spark wallet used is a hot wallet derived from the secret of the hardware wallet.

Mike Schmidt: So, I think that’s for recoverability, right? So, the hardware device will have that secret and it’s derived, but it’s obviously advantageous to have a hot Lightning wallet.

Mark Erhardt: Oh, okay. So, it’s a derived key that the hardware wallet produces and then shares with the mobile wallet, and the mobile wallet uses that to participate in the statechain, which means that the hardware wallet can be plugged into a different device to recover the statechain access.

Gustavo Flores Echaiz: Exactly.

Mark Erhardt: I see, that’s cool. Well, kind of scary that you can export private keys, but…

Covenants.diy script editor

Mike Schmidt: Our last piece of software from this month was Covenants.diy script editor. And we mentioned this one briefly, I think it was in the instagibbs segment a couple of weeks ago when we were talking about Changing consensus items, and sort of the ecosystem around OP_TEMPLATEHASH and covenants, and things like that, sort of proof of concepts, things like this, and tooling. We mentioned it briefly in that newsletter, and I think maybe in that podcast as well, but I figured we’d highlight it here as well for folks to jump in. So, this is a browser-based, I guess you could call it editor, for building covenant scripts, and then sort of stepping through how they might execute. So, it includes a bundle of things we’ve been talking about as prospective opcodes, including OP_CTV (CHECKTEMPLATEVERIFY), OP_CSFS (CHECKSIGFROMSTACK), OP_CAT even, OP_TEMPLATEHASH, OP_INTERNALKEY, OP_PAIRCOMMIT, OP_TXHASH, and then also APO (ANYPREVOUT) as well. And user beware, this is for test networks only, obviously, since these things aren’t activated.

But yeah, I thought it was interesting. People keep talking, they want covenants, so part of that is getting little tooling websites like this, little editors to step through how things might work, and I think it might be interesting for listeners to play around with. Murch?

Mark Erhardt: Yeah, so it looks like both LNHANCE and re-bindable signatures are completely covered here. OP_TXHASH is part of the Great Script Restoration proposal, or also a standalone. And OP_CAT is separate, but has had some demand in the past. APO is BIP118, of course, which I think is a little more superseded idea-wise, because re-bindable signatures, the BIP448 proposal, sort of replicates the things that APO was asked for, as in LN-Symmetry. But if you’re into covenant research, it sounds like this is something you might want to grab and go wild with on Inquisition.

Mike Schmidt: Thanks, Murch. That’s it for the News and for the Changes to client and service software segments. We have Releases and release candidates and Notable code changes with Gustavo.

Eclair 0.14.3

Gustavo Flores Echaiz: Yes, thank you guys. So, this week, we only have one release from the Eclair repo, v0.14.3. I’m going to talk more in detail about the changes that are brought into this release, because we also cover them as Notable code changes. But briefly, there’s a new configuration parameter, called max-funding-feerate, which is the maximum feerate that your Eclair node would pay for funding or splicing transactions. Even if your fee estimator were to say you should pay a higher fee, this configuration parameter basically allows you to put a cap on the feerate that you would pay for those transactions. This is a protection against potential manipulation of the fee estimator, which would make you pay excessive fees. Of course, there’s a trade-off, because in some real scenarios, you would actually want to pay higher fees, but this configuration parameter allows you to cap the feerate you will pay for funding and splicing transactions.

There’s also other changes that are more security-related that I’m going to cover later on. They’re mostly related to channel closing scenarios, splicing scenarios, and one important potential fund loss scenario on on-the-fly funding, but we’ll get to that more in a few minutes. This release is a patch release that contains important security fixes, so users are recommended to upgrade in order to avoid running an Eclair node that has issues that could be potentially exploitable by malicious nodes.

So, now we get into the Notable code and documentation changes section. We have a big, heavy week this one, about seven items from the Bitcoin Core repo. Most of them are bug fixes. And in addition, we have a few ones from the Lightning implementations that are mainly bug fixes, but also a new BIP and an update in an existent BIP specification.

Bitcoin Core #35445

So, let’s start with Bitcoin Core items. The first one, #35445. Here, there was a bug if you had a descriptor wallet that was using a miniscript expression that was using the h syntax for hardened derivation markers, instead of the apostrophe syntax. So, this was really a syntax issue that if you had a wallet that used the h syntax to define hardened derivation, after upgrading to v31, you wouldn’t have been able to load your wallet. Yes, Murch?

Mark Erhardt: Yeah, so maybe a little background on why the syntax was changed. A lot of the interactions with the RPC in Bitcoin Core, especially if you do it through the console in the app, require a lot of escaping. So, you would frequently enclose parts like arguments to your calls with quotation marks or apostrophes, the single quote marks, I should say, because the apostrophe is actually a different symbol. But anyway, the issue here is now of course, if you use the single quotation mark as a hardened derivation marker, you have to do extra escaping on that, whereas h is just a regular character. And I think that caused some usability issues in the past. So, we changed the expression, or the syntax, for hardened derivation in Bitcoin Core over the last few versions. Sorry, do you want me to also talk a little bit about how this bug came to pass, or do you want to cover it?

Gustavo Flores Echaiz: Please go ahead.

Mark Erhardt: Okay. So, the descriptors in themselves have a bit of a challenge, in that you could create multiple descriptors that derive equivalent output scripts, right? And so, one attempt to notice whether we have a descriptor already was to derive an ID based on the text of the descriptor, like the textual representation of the descriptor. If you put in exactly the same text, we would notice, “This is the other descriptor that we already have in the wallet with the same descriptor string”. And now, of course, between one and the other versions, it was previously the single quotation mark for hardened derivation, and then it was the h instead as the hardened derivation marker. So, the string changed, and therefore the descriptor ID that was calculated from the descriptor string would have calculated a different ID, and this caused the compatibility issue. Now, the fix for this was to instead treat the already stored descriptor ID as an identifier that is not derived from the string directly, but just as a random number that was stored to identify this descriptor. And now, I think was this where we introduced normalization so we always compare the strings? Like, if you put in a descriptor with a quotation mark versus a descriptor with an h, it would recognize that these are both the same, even though the previously calculated descriptor IDs would change. It now compares the strings and will realize, “Yes, this is intended to refer to the same descriptor”.

The descriptor ID was only used in importdescriptors and createwalletdescriptor, the two RPCs. So, these now are based on a string comparison rather than the descriptor ID. We do think about using the descriptor IDs internally still, now just as an identifier inside of a scriptPubKey manager. For example, we were just talking about how we would be using that in the context of multisig setups, which we’re working on here in Localhost Research, to identify which key expressions participate in a descriptor. But yeah, so we still need sort of a way of disambiguating descriptors internally, even though at the surface level, the RPC level, we no longer use this identifier.

Gustavo Flores Echaiz: Exactly. Thank you, Murch. So, the issue here was treating the identifier as a value that had to be recomputed from the string. So, if you would change the syntax, then the identifier wouldn’t match anymore. Now instead, the identifiers aren’t treated as such, simply are treated as links between related wallet records. And now the RPCs, importdescriptors and createwalletdescriptor, compare the strings instead of the identifiers.

Bitcoin Core #36076

Next item, Bitcoin Core #36076. This is another bug fix when using the combinepsbt RPC. When you’re combining two different PSBTs, you could discard the sighash (signature hash) type of the second PSBT file. So, if your first PSBT file didn’t have the sighash type defined in the PSBT_IN_SIGHASH_TYPE field but the second one did, and the second one also was properly signed, then because you would first put a PSBT that didn’t have that field, Bitcoin Core would simply erase the field. And then, there would be a mismatch between the signature and the absence of the sighash field. Actually, it would get overridden. So, if you were using a non-default sighash type, this is when this problem would occur. Yes, Murch?

Mark Erhardt: Yeah, so the thing is that when you create a signature, you get to pick which sighash you use, so what parts of the transaction do you commit to with your signature. You can commit to all the inputs or the one input that you’re signing, you can commit to all the outputs or just the one output at the same position that you’re signing at in the input list, or you can commit to none of the inputs or none of the outputs, and combinations thereof. But the person that signs, or the user that signs, gets to decide the sighash type of their signature. So, this problem would occur when someone creates a PSBT, it doesn’t make any determination about the sighash type, and then someone else adds a valid signature with a sighash type. And now, the conflict was between no sighash type and the provided sighash type with this signature. Is that right? Yeah, okay.

So, yeah, we talked about how we combine fields when they disagree, I think a few weeks ago. And obviously, if there are two different values for the same field, that is an issue. But if there’s only one value, it should override. So, it sounds like the absence of a type here was equivocated with the default type, and that’s where the conflict came.

Gustavo Flores Echaiz: Precisely. And it was also depending on the order of the PSBT files, right? It had to be that the first file didn’t have the field. That’s where the conflict would arise. So, now, it doesn’t matter in which order you put them, it will retain the field if it’s not a non-default sighash type.

Mark Erhardt: Basically, the difference between null and zero, or null and the default value. You have to explicitly realize whether the default value has been picked or no value has been picked.

Bitcoin Core #36150

Gustavo Flores Echaiz: Right. Next item, Bitcoin Core #36150. Here, this is a bug where if you have an archival node, just a regular node that is unpruned, and you also haven’t configured yet either a compact block filter index (-blockfilterindex) or UTXO set statistics index (-coinstatsindex), so you haven’t configured any of those, but you decide to prune your node to enable pruning, at the same time, you decide to enable one of those indexes. Then, you restart your node and the pruning begins before the index. So, it starts pruning all block files before the new index put a lock into those block files, so that the index could be built before those block files were eliminated. So, you would find yourself in a moment where the process couldn’t even continue, because the index would simply return an error because the block files were absent. So, now, the fix is to put a lock at height zero, so the index builds and the pruning can begin after those files have been used.

Mark Erhardt: I didn’t read this PR, but does it throw an error or does it try to re-sync? Because if you turn on -blockfilterindex, I think it would have to recover the block files in order to reintroduce it. So, if you had a pruned node and turned it on, it would probably just process the entire blockchain again, or does it just error that the files are not there? I haven’t checked.

Gustavo Flores Echaiz: From my understanding, the index simply fails to start syncing and tells you to re-index.

Mark Erhardt: Right, you would need to re-index in order to get the data, but it doesn’t do it automatically.

Gustavo Flores Echaiz: Exactly.

Mark Erhardt: Yeah. The first time I read this News item, I thought it was always when you turned on compact block filter and were already pruning, that it would, on restart, first prune and then update the index. I’m glad it’s only when you go from an unpruned node to a pruned node, but that’s an important thing that I missed the first time I looked at this.

Bitcoin Core #36174

Gustavo Flores Echaiz: Right, good clarification. Next item, Bitcoin Core #36174. This is about the replacement HTTP server, which we covered in Newsletter #411, which is now replacing the libevent-based HTTP server, one of the last remaining dependencies in Bitcoin Core. In Newsletter #422, we covered some fixes to this server that were complementing the receive-side protection. And now, the new item is about send-side protection, or backpressure. So, in Newsletter #422, we covered a client is making too many requests; we’ve got to limit those requests. Here instead, the problem is that the client that has made many requests isn’t reading the responses that our node is giving back to it. So, the queued response data is growing indefinitely. So, now, the fix is simply to pause processing of further requests when the send buffer exceeds 32 MiB. And it resumes once the client starts draining the responses. So, it will actually start reading the responses, then the send buffer drains or reduces in size, and then we can start processing new requests again.

Bitcoin Core #34743

The next item, Bitcoin Core #34743, this is about how, if you manually selected some peers, either through the -addnode, -connect options, or the addnode RPC, if these manually selected peers would stall block downloading during IBD, previously your node would simply disconnect the peer. Now, what it does instead is that it will simply request the same blocks that were getting stalled from other peers and pause new block requests from the stalling peer for two minutes. So, instead of disconnecting the peer, you simply stop requesting blocks from it for two minutes. And this is specifically when the peer is stalling block download during IBD or during, let’s say, another moment of you need to sync multiple blocks. So, other peers are already giving you the next blocks, but this peer is unable to complete this block request. And that’s what we called ‘stalling block download’, which is different from just timeouts set for header syncs or separate block download timeouts, right?

In regular timeouts, you will actually disconnect from this peer. That scenario remains as it was previously working. It’s simply in the scenario where your peer is stalling block download. That’s when you won’t disconnect from it if you had manually selected that peer. You will simply pause new block requests for two minutes.

Bitcoin Core #36081

The next item, Bitcoin Core #36081. This is about adding a new field to the getmininginfo RPC response, called bestblockhash. So, the problem here is that there could be a sort of race condition where you want to obtain the next object, so you want to know what’s going to be the difficulty target for the next object, and you want to build on the current tip hash. But because you had to make two requests separately, there could be a sort of race condition where the current tip hash could change while you were making the second request. So, now, that field that defines the current tip hash is added to the same RPC response.

Bitcoin Core #35975

Next one, Bitcoin Core #35975. This fixes a potential wallet crash when, if you had in your wallet two malleated versions of the same transaction but you bumped the fee on simply one, it didn’t immediately mark the other one as replaced. So, if you were to attempt to bump the other one too, that could trigger an assertion failure and would crash the node. The fix is simple. If you bump the fee of one of the transaction versions and that one gets replaced, the other one has to be marked as replaced too. And if you were trying to bump the fee of the variant, that would simply hit an error instead of hitting an assertion failure and crashing the node.

BIPs #2241

The next items are from the BIPs repository. So, BIPs #2241 adds BIP332. So maybe, Murch, I don’t know if you want to jump in here, but this is about the specification of a new message that is using the BIP434 peer negotiating support feature, or called opt-in relay of recent stale chain tips.

Mark Erhardt: Right, so this introduces the capability for nodes to opt into participating in stale tip relay, and that’s also the title of the BIP. The idea here is that occasionally, the Bitcoin Network has two competing blocks at the chain tip. And when that happens, you would usually only learn about one in most cases, because once you hear about the first one of the two competing blocks, the second one you would download but not forward. So, it sort of creates a rift through the network, where on one side the nodes have received one block, and on the other side they have received the other block first; and on the borderline, they know about both blocks, but they do not forward the information about the second chain tip. So, this is interesting when there’s a reorg, because if you know about the other chain tip already, you would be faster at reorging to the chain tip that wins if you’re on the other one. And it is especially also interesting for researchers that are trying to track the stale rate. So, how often stale blocks occur as a measure of how much latency there is in the block relay system, and it can also indicate selfish mining attacks and other deviant behavior, where the stale rate goes up for malicious reasons.

So, this is a proposal for telling other peers that want to hear about it when you have seen more chain tips. It will also propagate, on first connection, up to ten chain tips from the last 1,000 blocks, so in the last week. Usually, we only have one of those every other week. So, it shouldn’t usually be a lot of data. But I think it is also already implemented in Bitcoin. No, sorry, there’s only a draft for Bitcoin Core. So, if it were coming to Bitcoin Core, it would be not in the upcoming release v32, but in v33 or later.

BIPs #2258

Gustavo Flores Echaiz: Thank you, Murch. The next item is also from the BIPs repository, PR #2258. Here, BIP93 specification, which defines Codex32, which are a way of encoding BIP32 seeds different from the BIP39 protocol. So, here, I don’t know, Murch, if you know much about this, but from my understanding, this is about fixing checksum length limits. So, the prefix was not being considered. The checks were only counting the data portion, so some strings would exceed the lengths of the error-detection guarantees of the checksum. And also now, the encodings have a specific size. They used to be very flexible. I think it was from 16 to 64 bytes. But now, they have specific defined byte sizes, either 16, 20, 24, 28, 32 or 64, to avoid user error. So, if, let’s say, you made a transcription error where you added or removed a digit, then this would allow you to know better whether you made that transcription error or you didn’t. Anything you want to add here, Murch?

Mark Erhardt: Yeah, this is part of an effort to make a bunch of improvements to BIP93, which is Codex32, which is an extremely work-intensive way of manually, literally with pen and paper, creating your own entropy – no, not the entropy, but to shard keys. And, yeah, so this one contributor has been digging into BIP93 for a few months, and I think the overall intention or motivation there is that BenWestgate seems to be working on introducing BIP93 support into BIP85, so like a dedicated derivation path. And yeah, this has been going on, and it started as one very big change to BIP93, and now changed into, I think, six PRs that are all more focused and smaller. But anyway, there’s a bunch of cleanup coming to BIP93. BIP93 is still in draft status, even though we have maybe already played around with volvelles and with, well, big books of tables to fill out at various workshops in the last, whatever, six years or so. This is a draft BIP, and it looks like there’s, at the edges, still improvements being made.

If that sort of thing, calculating key shards by hand, is your thing, you might want to check out BIP93 and the recent changes to it. There are still several open PRs.

Eclair #3380

Gustavo Flores Echaiz: Great, thank you, Murch. The next three items are from the Eclair repo. The first one, a pretty simple one, #3380. Here, Eclair adds rejection of API requests that contain an Origin header. Usually, the API requests that contain an Origin header come from a browser. So, basically now this doesn’t allow any more browser-based front end to call Eclair directly. It has to call its own backend that then its own backend would call Eclair directly. This is to prevent cross-site request forgery using cached HTTP Basic authentication credentials. If you’re using command-line clients, such as curl or eclair-cli, this remains unaffected, unless you explicitly set this header. But if you are making these API request calls directly from a browser, then the Origin header is automatically set. And that is now forbidden by Eclair too for security reasons.

Eclair #3376

The next item, #3376. This is the main security finding that was included in the latest Eclair release. There’s multiple fixes being packaged here, the first one is in a closing fee negotiation scenario. So, if Eclair pays the fees, let’s say your node pays the fees, but your peer proposes a fee that exceeds what you have configured as your maximum closing feerate, in a scenario where your peer didn’t support fee ranges, which is a part of an advanced version of the fee negotiation protocol – so, very old nodes don’t support fee ranges – then if you were negotiating fees with a node that didn’t support fee ranges, then your node could accidentally accept the fee being proposed by your peer, even if it was above your maximum closing feerate. So, an attacker running an old version of a Lightning node could convince your node to pay an excessive amount of fees. And now, Eclair simply makes sure that it will never accept a fee range that is outside of its fee range or that is above its configured maximum closing feerate. That’s the first issue.

The second one is if in your node, Eclair is involved in a splicing negotiation, but it has to force-close the channel during an incomplete splice, and it had already exchanged commitment signatures for the post-splice state, but it hadn’t yet completed the splice, previously Eclair would try to broadcast the commitment transaction after the splice state. But like I said, the splicing was incomplete. So, it would simply make no sense for Eclair to broadcast that transaction that’s dependent on the splice that was incomplete. So, now Eclair will broadcast the latest commitment whose funding transaction has been fully signed. So, in the scenario where the splicing is incomplete, it won’t broadcast the commitment transaction that depended on the splicing transaction. It will broadcast the previous commitment transaction that depended on the initial funding one before the splice even took place. So, this is a fix when force-closing, Eclair would simply broadcast the correct transaction instead of broadcasting one that doesn’t have a finalized state yet.

However, in this specific scenario where you broadcasted that commitment transaction, if your peer who has received your splice and signature, so he has both signatures for the splice, if he broadcasts the splice and the splice confirms first, the other addition to this PR is that Eclair will instead close using the commitment that followed the splice. So, I want to take a step back. The first part of this PR was about the closing scenario that I covered. Now, we’re talking about the splicing scenario, and there’s two parts to this splicing scenario. First, Eclair broadcasts the correct commitment transaction when it wants to force close. But if ever the peer broadcasted the splicing transaction, then Eclair will properly respond with the commitment transaction that followed the splicing one, which will allow Eclair to properly respond to the peer. That’s the second part of the PR.

The third part is a fund-loss scenario when executing an on-the-fly funding payment via blinded paths. So, basically, Eclair will always make sure that the outgoing HTLC (Hash Time Locked Contract) has a lower CLTV (CHECKLOCKTIMEVERIFY) expiry delta than the incoming HTLC, except in a scenario of on-the-fly funding via blinded paths. So, in that specific scenario, yes, Eclair constructs the outgoing HTLC and defines the CLTV expiry delta, but the outgoing peer constructs the blinded path in his invoice, and those instructions specify a CLTV expiry delta. Eclair is not supposed to trust those instructions, it’s supposed to decide on its own what CLTV expiry delta it will use. However, in this specific edge case, it wasn’t properly looking for that. So, it would simply accept any CLTV expiry delta that was instructed in the blinded path of the invoice. And that could lead to a fund-loss scenario because at the same time, it would be an unsafe forwarding where the downstream peer could claim the payment, but the incoming HTLC would expire before Eclair was able to recover that. So, it would lose funds in both edges. So, that’s the third part of the PR.

The fourth part is adding the configuration setting that I discussed earlier on in the release section, which is called on-chain-fees.max-funding-feerate, which defines the maximum feerate that Eclair will ever pay. Even if its fee estimator says to pay a higher fee, Eclair will refuse to pay a higher fee than the configured setting. And this is a protection against manipulation of fee estimators. Of course, it has a trade-off, because in some scenarios you do want to pay that high fee, but it allows users to define a maximum feerate.

Eclair #3372

The next item, Eclair #3372. This is when Eclair is acting as a trampoline node, an issue was that Eclair, as a trampoline node, was retaining too much of a fee, which was making the payment fail eventually, because the fee budget reserved for downstream routing wasn’t sufficient, specifically in the case if the receiver was using an LSP (Lightning Service Provider) and the LSP was charging a forwarding fee. So, now there’s a new setting that allows an Eclair node that is acting as a trampoline node to define the minimum fee that Eclair retains, so that Eclair can minimize the fee that it retains and allow the budget to cover the downstream fees, including those charged by the recipient’s LSP. At the same time, the PR also increases the default minimum total fee budget. So, on one end, there’s a new setting that allows Eclair to retain less fees as a trampoline node, but at the same time, Eclair now asks for a larger minimum total budget.

LND #11163

The next item is from the LND repo #11163. This one fixes the handling of replayed HTLCs when using the forward interceptor. So, the forward interceptor was covered as early as Newsletter #104. It adds the ability to delay forwarding a payment by giving an external process the ability to review and decide the forwarding of the HTLC. So, here, the issue was if your peer would disconnect and reconnect, or your node would restart, LND might reprocess the incoming HTLC, even if it has already forwarded it. So, that means that LND could treat this as a new interception, and if the expiry was too close, it would simply reject it, even though it had previously already forwarded the HTLC. So, it would think that it’s a sort of new HTLC when it’s simply just one old one being replayed. So, now, LND simply checks existing forwarding records. It allows the replay to continue through the original payment’s resolution. It doesn’t take a new decision here, and it avoids the scenario where it would simply reject the HTLC because it thought the expiry was too close, even though it had previously accepted.

BDK #2246 and #2263

The final item from this list comes from the BDK repo, two items, #2246 and #2263. This is about how wallet balances are classified. Previously, for example, if you had change that came from spending an unconfirmed incoming payment, that change, because it was change, it could be considered trusted as if it came from your wallet. However, the change was coming from an unconfirmed incoming payment. So, BDK was not properly classifying wallet balances, because it wasn’t properly checking the output’s unsettled transaction ancestry, right? So, now BDK ensures to check the ancestry of each UTXO to make sure to properly classify it either as trusted or as untrusted. And there’s a new API, called classify_outpoints, which allows a user to classify the outpoints himself. And there’s even another function being added, called confirmations_lower_bound, which helps applications define settlement rules. For example, this should require six confirmations before it’s considered settled, so that’s also part of this addition. And that’s the final PR item and this completes the newsletter. Thank you.

Mike Schmidt: Awesome. Thanks, Gustavo. And we also want to thank our guests today, Davidson, Eric, and Portland for joining, and Murch for co-hosting, and you all for listening. We’ll hear you next week. Cheers.

Mark Erhardt: We would have made a shorter News Recap if we had had more time, right?

Mike Schmidt: Yeah, that old saying. Was that Mark Twain?! Yeah, cheers.

Bitcoin Optech Newsletter #422 Recap Podcast

2026-09-15T00:00:00+00:00

Mark “Murch” Erhardt and Mike Schmidt are joined by Adam Gibson and Rob Segers to discuss Newsletter #422.

The Bitcoin Optech Podcast and transcription content is licensed Creative Commons CC BY-SA 2.0

Download audio

News

● A protocol for probabilistic coinjoin and covert betting (1:14)
● Update on silent payments light clients (43:26)

Releases and release candidates

● LDK v0.3-rc1 (1:04:06)
● LDK v0.2.6 (1:05:58)
● BTCPay Server 2.4.4 (1:07:44)

Notable code and documentation changes

● Bitcoin Core #35949 (1:13:04)
● Bitcoin Core #34931 (1:15:11)
● Bitcoin Core #36048 (1:18:14)
● Bitcoin Core #36123 (1:21:12)
● Bitcoin Core #36176 (1:22:32)
● Core Lightning #9434 (1:24:49)
● LND #11061 (1:26:29)
● LND #11125 (1:27:23)
● LND #11064 (1:28:33)
● BTCPay Server #7561 (1:29:31)
● BTCPay Server #7559 (1:31:03)

Transcription

Mike Schmidt: Welcome everyone to Bitcoin Optech Newsletter #422 Recap. Today, we’re going to be talking about a protocol for coinjoins disguised as covert bets; we have a discussion about benchmarks for silent payment light clients; and we have our usual weekly segments for Notable code and Releases. This week Murch and I are joined by two guests. We’ll have them introduce themselves briefly. Waxwing?

Adam Gibson: Oh, hello, yes, thank you for having me. I look forward to it.

Mike Schmidt: Thanks for joining. And, Rob, who are you and what do you do?

Rob Segers: Hi, Rob from Bitsaga. What do I do? I help people secure their Bitcoin, not as much as a low-level engineer as a usual guest, I guess. But did some work on silent payments, unblocked, I guess, some discussion that was going on in 2024. I guess we’ll dive into some findings and measurements and perhaps where silent payments could be going.

A protocol for probabilistic coinjoin and covert betting

Mike Schmidt: Cool, thank you both for joining. We’re actually going to go in order for once in the newsletter, since both of our guests are part of the News segment this week. So, we will start with Babilonia, which is a probabilistic coinjoin and covert betting system. Adam, maybe folks are aware, but obviously coinjoins have some issues, in that they’re recognizable and there’s obviously a variety of attributes people can use to sort of identify these and they can get flagged, and in some places such things are maybe unnecessarily criminalized, and so this privacy tool sort of marks yourself. You have this idea for Babilonia that takes sort of a different angle on that. Maybe you can elaborate on the issue and then also how you came to what you came to with Babilonia and what it solves in this regard?

Adam Gibson: Okay, sure, thanks. Yeah, I think you summarize the starting point pretty well. So, I guess there’s two starting points, right, because the other starting point is a line of discussion that started on Delving Bitcoin maybe a year ago or two years ago, I can’t remember, and it was called, “Emulating OP_RAND”, and the idea there was there’s this clever cryptographic trick that a guy, Oleksandr Kurbatov, I think is his name – well, that is his name, but I think I pronounced it correctly – and he had this idea about within Bitcoin’s existing consensus, so not proposing some new opcode or something, that you could get a sort of trustless arrival at a random number that the two parties or the end parties couldn’t predict in advance and actually have that enforced onchain. And it was a cool idea and some people were discussing it and seeing pros and cons. And the obvious application is that whole idea of being able to make probabilistic payments on Lightning, because this is an old problem that if you want to make payments that are so small that they’re not sort of economic to be broadcast onchain, could you still do that within an offchain protocol? And it might be a good time to mention that it goes back a lot further than that.

I’m guilty, as well as I think they were in that discussion, I’m also guilty of not referring to the original history here, which is quite interesting. But way back, 30 years ago, 1997, Ron Rivest published a paper basically discussing the same thing. And what he was thinking was this, that if we have these new digital cash payment systems, they’re cool and everything, but there will be some flaw, some minimum cost to do such a transaction. You know, it might be 30 cents or a dollar or something, right? And so, he was thinking, “Well, how could we have payments that were really small, micropayments between customers and merchants, and yet deal with the fact that if the customer only wants to pay 10 cents, well, that’s just not economically viable?” And so, he had the idea of a sort of a lottery approach, where essentially the merchant would be receiving like lottery tickets instead of actual payments. And that, at some point later, and because in his point of view of the world, this is way before Bitcoin, we were thinking about payments that were mediated through banks or through some kind of central party.

So, within that context, the problem being solved is a little different, but it’s the same basic motivation that you might be able to, over a period of payments that were statistical in nature, but nevertheless trustless, at least the statistics of them were something that you didn’t have to trust in, the merchant could end up claiming the appropriate amount of money over a long period of time. And then, the problem repeats in Lightning, like I don’t know how many years ago it was, and I think, somebody correct me if I’m wrong, but I think it was Tadge Dryja that originally came up with the idea that you could have a probabilistic payment to replace a single satoshi payment. And then, over time, you might end up with 10,000 sats, which is a reasonable kind of payment, one of those instead of several thousand 1-sat payments, which are not economical. But of course, it’s a little different in Lightning, because there isn’t any central bank entity to rely on, so it’s a little bit more difficult. So, I actually don’t know all the history of that discussion, I know there was a lot of discussion, but this new OP_RAND idea is kind of cool, because it makes a sort of purely trustless way for someone to basically roll a dice or flip a coin within the payment.

Okay, so that’s kind of the background, and if you go back to that, “Emulating OP_RAND”, thread in Delving Bitcoin, you’ll see that there wasn’t a huge discussion, but there was some discussion of that, and whether this idea might be applicable to that. And then, there was a kind of a back and forth of, well, but if this new protocol we’re coming up with involves zero-knowledge proofs (ZKPs), which is the thing I haven’t mentioned yet, in other words, how do you get the scenario where there’s essentially a coin flip onchain that neither side can predict? And the answer is that both sides do some kind of preparatory work, and then prove that they’ve done it honestly with a ZKP. And I think it was AJ Towns who was saying that, “Yeah, this could work, but if you have to do thousands of ZKP creations and verifications during the flow of updating HTLCs (Hash Time Locked Contracts), it might become quite impractical.

So, there’s a whole area of discussion there, and there was a paper by Gerhart, Taylor, and a third name I can’t pronounce, shortly after Kurbatov’s original short paper on this OP_RAND idea, and they sort of developed the idea a bit further, and they made it possible to make a more arbitrary continuous value out of a set of possible payments, instead of just, say, a coin flip, which obviously is a bit limiting if you can only create two possible outcomes. And then, I haven’t even started talking about my idea yet! So, my idea sort of continues on in that thesis, and I said to myself, well, the nice thing about this, going back to Rivest’s original idea, a user pays a merchant, how do we try to address that in Bitcoin? Well, we came up with this idea of payjoin. We said, well, look, we can actually improve the privacy of actual real-world payments by creating a situation where the two parties collaborate a little bit, and most specifically, of course, in payjoin, the idea is that the collaboration takes the form of both the user and the merchant contributing to the inputs of the transaction, thus violating the most natural heuristic and the central heuristic in blockchain analysis, which is that one party is producing the inputs and therefore you can use that to deduce traces of where the coin ownership is going.

So, I thought to myself, well, the nice thing about payjoin is, well, how can I say this? Actually, it’s better to look at the other extreme. Look at the most normal concept of coinjoin as we imagine it today, which is that n participants, possibly a large n, are all coming together. And yes, they are violating common input ownership heuristic, because they all provide inputs and then they all get outputs. And we try to improve privacy very specifically with equal-sized outputs. But we realized early on that one of the big problems with that construction, albeit it does achieve a specific anonymity set goal, one of the biggest problems with that construction is that this large transaction, suppose it’s 100 participants, each participant is paying themselves the amount that they put in. So, each participant comes along with 1 bitcoin or 4 bitcoins and they get out 1 bitcoin or 4 bitcoins. And the implication of that is that even if some outputs are equal-sized, therefore creating an anonymity set in coinjoin outputs, the other outputs, the change outputs, do not have that property. And so, it’s actually quite easy to track everything which doesn’t go through in this exact size.

So, let’s say you put in 4 bitcoins and you get out 1 bitcoin as an equal-sized coinjoin output. That 1 bitcoin might have an anonymity set of 100, but you’re remaining 3 bitcoin, your change, does not have any increase in anonymity set in general because of what’s called subset sum. Because somebody looks at the 4 on the input side and they know that the equal output side is 1, and then all they have to do is look for 3 on the output side and they know that’s the same person who was the 4 on the input side.

So, in the discussions of payjoin, which occurred back in 2018 in London, we realized that what would be really nice is if we could break that subset-sum property. And we also realized that the only way that you can be sure that subset sum doesn’t occur, in other words the coinjoin transaction can simply be broken down into a subset of individual transactions, the only way to avoid that is to have payments within the coinjoin. And that was one of the motivating ideas behind payjoin itself.

Mark Erhardt: Sorry, may I jump in briefly? So, in your example just now, you said that someone puts in a 4-bitcoin UTXO and splits it into 1 and 3 because the shared output size, the standard output size, is 1 bitcoin. But of course, they could just pay 4 times 1 bitcoin to themselves. So, wouldn’t the same also translate if you had multiple different stratified values, like say you can make 1-bitcoin outputs, you can make 0.5-bitcoin outputs, 0.2, 0.1, and so forth, like some series of allowed sizes, and then multiple people create multiple of those?

Adam Gibson: Yes, 100%. That’s exactly the idea of Wasabi v2. Well, WabiSabi has extra cryptography in there, but that was part of what they did, is that they broke into many denominations instead of just the crude example that we used back in the day, and that we still use, is a single denomination; so, everyone agrees to one denomination, everything else is change. You can instead create multiple denominations, but I’m sure it’s obvious that the limiting factor of that argument is scale, right? Obviously, if I have an output of 0.423765 bitcoin, there’s only so far you can go in doing that. And in some sense you could argue – well, I don’t know, obviously it’s a good idea to have multiple denominations, I think that was a good innovation.

Mark Erhardt: Okay, so basically, yes, but it doesn’t work if you have a very uneven amount, and you were going to tell us how to fix that.

Adam Gibson: Yeah, so in payjoin, obviously it’s addressing that problem, but it’s obviously a completely orthogonal direction of trying to improve privacy, right? It focuses on completely different things. It doesn’t focus on this concept of a quantifiable anonymity set. The great thing about this style of these simple, large coordination of multiple-participant coinjoins is that you can say, at least at the point of that transaction, I have created anonymity set K, you know, 100 or 50 or whatever it is. Payjoin isn’t even trying to do that, but it is introducing this whole other angle, which I think is really important, which is this sort of steganographic footprint, the idea that you are making transactions which do have a privacy effect, but which look the same as things which don’t attempt to create a privacy effect. And that doesn’t create some simple quantifiability.

I mean, a better example in a way is coinswap. Coinswap is really pushing hard on that concept. It’s saying, “I’m going to make swaps where there are transactions, perhaps even on different chains, but let’s just say on Bitcoin itself, two different transactions, and they’re not connected in the transaction graph”. And therefore, assuming you don’t actually print the preimage on chain, in which case they’re trivially linked, right, but if you don’t, you’re creating two completely separate transactions. And if you address things like amount correlation, which is extremely difficult, and timing correlation, which is less difficult, then if you fix those problems magically, then coinswaps have this property that, no, you can’t give them an anonymity set of 100, but in some random sets they have an anonymity set of the entire blockchain.

Mark Erhardt: Could we maybe just briefly explain what a coinswap is? I think maybe some listeners might not be familiar with that one.

Adam Gibson: Sure. The most general concept is atomic swap. And I think it’s become quite like, what’s the word, pedestrian now. It’s quite sort of an everyday thing nowadays, because people are doing things like swapping their Lightning to onchain, or swapping their onchain to Lightning, or swapping their Bitcoin to Tether, or swapping their Bitcoin to Litecoin, or I don’t know. There’s so many different swap services out there, especially outside of Bitcoin, but even within Bitcoin, what’s sometimes called a submarine swap, where you move funds between an onchain output and an offchain Lightning balance. So, all of them have the same basic principle involved, which is that the atomicity, the fact that two different payments are guaranteed to happen, both or neither, both of them happen or neither of them happen, is coming from the idea that within one of the payments, within activating one payment, you’re revealing a secret which the other side can use to enact the second payment. I’m trying to be as general as possible, because there’s so many different variants, right? And when we say coinswap, let’s say in the Bitcoin privacy community, we’re referring specifically to a slight finesse on that idea.

This was published by Greg Maxwell around the same time as he published coinjoin, back in 2013. And the idea here is that instead of publishing that preimage or that secret, which enables the other transaction onchain, which would make it obvious that a swap is happening, and which would tie the two transactions together, instead of doing that, you swap the secrets offchain sort of optimistically, and then you broadcast the two payments without ever revealing the tied-together secret itself onchain. So, that’s what we call a coinswap.

Mark Erhardt: So, let me try to recap that. Generally, the concept here is that two parties move funds into a shared custody, and then while they might have a recovery option, they tie together two transactions so that either none of the two or both of them happen at the same time. And this allows them to create two transactions that pay each other that seem completely unlinked. This happens, for example, with an HTLC, or well maybe an adaptor signature is maybe more the standard example, where one of the signatures that they exchange is tweaked, and the tweak is missing, but the other signature uses the same tweak. And then, when one of the two transactions is broadcast, the other one can be broadcast too. And that way, because the funds are held by both owners together in a 2-of-2 situation, they can ensure atomicity, either both or none.

Adam Gibson: Yeah, that’s a very good explanation. I think the only thing that was missing there is that it’s two separate shared custody amounts needed, not just one, but apart from that, I think that was a perfect description. And I think that might be a nice lead-in, because we haven’t really talked about Babilonia itself, that might be a nice lead-in there, which is that you said at the end, the two transactions look totally disconnected. And of course, the obvious problem with that is if the two parties are deciding to swap, let’s say, 1.000 bitcoin, then both of those transactions are broadcast onchain without any connecting secret, but they both have exactly the same amount; or perhaps there’s a fee difference, but basically exactly the same amount. And this is a problem that is always going to crop up and is really difficult. And Chris Belcher spent a lot of time designing a system to try and at least partly address that by having essentially networks of swaps to break amounts into constituent parts.

So, you keep seeing this pattern repeating of people trying to find ways to break the connections between amounts of funds by sort of finessing what the amount is, whether it be in denominations, whether it be splitting into sub-amounts, and so on. But often, those solutions have the problem that they end up creating more and more transactions. Whereas notice with payjoin or a payjoin-type design, you partly don’t have that problem, well you don’t have that problem, but that problem doesn’t really arise because all you’re really trying to do is to make the payment hide not the amounts, but the fact that there was a collaboration which mixes history.

Mark Erhardt: Where payjoin, of course, is a way of paying the recipient, and the recipient can also add an input to the transaction. So, the recipient can combine that with a payment themselves or with a consolidation, for example.

Adam Gibson: Yeah, absolutely. And I just realized I said something slightly wrong there, which is it is true that payjoin is trying to hide the amount. Whether it does or not is a kind of an interesting question. It kind of does and it kind of doesn’t. So, with the Babilonia concept, what I’m saying is, “Oh, look, this old history and recent history of developing a way for there to be a statistical outcome to the payment creates a situation where there is actually a payment, and we can make it like a payment of a random amount within something like a payjoin, where the two parties are collaborating anyway”. So, I’m saying add this extra layer of cryptography which makes it so that instead of me paying you a fixed amount, like 1 bitcoin, I’m creating a situation where there’s a distribution where the expected payment I make is, let’s say, 1 bitcoin, but I mean it could be zero perhaps, more realistically; but the actual payment I make could be anything from -0.5, well it could be anything. Okay, I haven’t talked about like the mechanics of it, but does that kind of describe what it’s doing?

Mark Erhardt: So, let me try to phrase it back to you. Babilonia proposes that you overlay a coinswap situation, where two people are just trying to move around funds in order to obfuscate the ownership with a probabilistic payment, and in our News item we speak about a bet, and this probabilistic payment would, in the long run, as people keep participating in that, end up being roughly net zero. But it would obfuscate the amounts more efficiently, because now the amounts actually diverge based on this random bet.

Adam Gibson: Yeah, that’s sort of half correct, because the thing is, my proposal is actually coinjoins, not coinswaps. But in the paper, I am sort of observing rather lazily that actually, this could be even more interesting in the form of a coinswap.

Mark Erhardt: Well, sorry for the idea!

Adam Gibson: No, no, I mean I actually suggested it already to the Citadel guys. I said, “Look, you should really look into it, maybe!” I think it’s interesting. But I thought just to limit the scope of what I’m trying to write out, because it’s already quite complicated, I thought let me write out a version of it where, like payjoin, you try to make the payment look, so to speak, non-suspicious. And then, I mean you mentioned the idea of bet versus probabilistic payment, and I think that’s important as well, is that you could just read the protocol as I put it out and say, “Well, this is a design for a trustless casino”, because it actually is. I mentioned early on in the paper, I say, well, the earliest kind of example of this was Satoshi Dice, right? And it’s interesting to compare, because what they did with Satoshi Dice is they had a specific phrase they used. They didn’t call it trustless because it isn’t, they called it sort of provable after the fact.

Mark Erhardt: “Provably fair”, I think was the term.

Adam Gibson: Provably fair, thank you. So, the idea was that obviously, you’re not in an actual real casino and you can’t watch somebody rolling a dice or spinning a roulette wheel, but what you can do is watch blockhashes. And so, they say, “Well, look, we’re going to pay out according to a formula based on the next blockhash. And you can check, because you can see the blockhash, whether we did it fairly or not”.

Mark Erhardt: Right. I think they pre-committed to some sort of nonce or something, and then the second part of entropy was coming from the blockhash. So, later they would reveal what the preimage was that they used for their commitment. And then obviously, they couldn’t have known the hash, I think the suffix of a blockhash, and because they wouldn’t know the blockhash, now they had two sources of entropy that were unrelated, that weren’t both controlled by them. So, after the fact, by revealing the preimage and by people knowing the blockhash, they knew all the inputs.

Adam Gibson: Yeah, exactly. There’s a lot of sort of finesse around it, and there’s caveats you could look into, “Is it fair?” etc. Point is, the general idea is pretty clear, provably fair. You can check after the fact. But what you can’t prevent them from doing is just taking your money. They could just take your money.

Mark Erhardt: One of the big problems was, of course, that because it was based on the blockhash and because you could reorganize if you had a lot of hashrate, there were some suspicions that some big miners might, or I don’t know if it actually ever happened, but someone could have reorg’d a block after losing a very big bet, and that could have put off the mining incentives, like skewed them.

Adam Gibson: Yeah, absolutely. As I said in the paper, there are caveats and there are things you can look into. But I think just at a high level, it’s clear that if you can create a scenario where the betting outcome is provably underivable by any party, that’s what you actually want.

Mark Erhardt: Sorry, I have a thought jumping into my mind. Would there be a way maybe to make it only be possible to find out who won the bet a few blocks later, so that the commitment were onchain and it couldn’t be reorg’d out, but then you only find out later?

Adam Gibson: Are you talking about a finesse on the Satoshi Dice idea?

Mark Erhardt: Well, maybe a finesse on your idea, that maybe one contributing factor could be sort of the third block after your transaction that locks in the bet confirms, but then of course, you still have the reorg problem. But yeah, I don’t know if there were a way to make it so that nobody could go back in time to change the outcome. But sorry, we’re getting really in the weeds here!

Adam Gibson: Yeah, sure. I mean, yeah, you could always make it more expensive, right, to attack the system. But I think this is sort of qualitatively what was originally Kurbatov’s idea, is sort of more fundamental, which is that what I think it really hinges on is this concept of atomicity, just as we were earlier speaking about coinswap. The most elegant way to do it, which is the way I do it, is with adaptors, and indeed the way Gerhart’s paper does it. It’s interesting that Kurbatov’s original paper did it using the fact that when you pay an address, you’re paying a hash, and so that when you spend from an address, you’re revealing the preimage of the hash. But it’s the same basic principle. That’s just a little bit less clean way to do it than adaptors. So, this idea that when you spend atomically, you have to reveal a secret, which occurs in coinswap or atomic swaps, also is the heart of the idea here, because otherwise you have this kind of unresolvable standoff between two parties who need to know a secret and have to reveal a secret, and you have to make it so that neither of them can predict what the outcome will be. But the heart of the trick that Kurbatov’s OP_RAND emulation had, which has now been extended, is just that, using atomicity to make it so that both parties…

But maybe I can sort of very high level describe the sequence of steps, which is that basically, if you think of it as Alice and Bob, Alice being the sort of proposer or dealer of the bet, and Bob being the player, so that the dealer sort of commits upfront to a set of possibilities. In my case, that’s specifically a list of curve points and the proof of knowledge that they actually own those curve points. And they’re going to prove that an adaptor that will be revealed when they give their co-signature will reveal a secret key which decrypts one of those list of possibilities, one of those list of private keys. And on the other side, the player also has to give a ZKP that their output address, the one that’s going to be the payout address, is formed from their own key plus one of the original curve points they were given as possibilities, without revealing which. So, with those two ZKPs in hand on both sides, it’s safe for – well, the dealer will give the adaptor, of course; you have to verify an adaptor before the secret gets revealed. So, when both sides have seen the proofs, the dealer can pay into the address knowing that with a probability 1 out of n, the player will win.

So, remember that the dealer had n curve points that they sort of handed over to the player. The player secretly chose one of them, constructed his output address, including that secret choice of one of those points. Both sides ZKP that they did it correctly. And then, when the dealer makes the payment, they reveal the preimage behind the adaptor. The player takes that adaptor and uses it as a secret key to decrypt one of the points. And if it’s the right one, then they get the money; and if it’s not, then after a time delay, the dealer can swipe the money instead. And then, you use the kind of typical optimistic taproot style approach, where if the other guy is lost, well, neither of them want to lose privacy, so they’ll just agree to sign out to an ordinary address instead of to a timelocked address.

Mark Erhardt: I see. I was wondering how the money was recovered after the commitment. So, they construct an output together, where both parties have committed to secret information. And in the dealer’s side, they have a set of choices that they commit to; and on the player’s side, they commit to one of those choices. If they guess right, they can immediately spend; if they guess wrong, the dealer can take back the money after some time. The recovery is pre-known, right? So, you could just say, after 10 blocks, or whatever, it can be spent. Oh, right, you don’t want to reveal onchain that you have the locktime construction there.

Adam Gibson: Yeah, you’ll put a CSV branch in the output. So, if when the player looks at the adaptor, they see, “Oh, damn, I lost. I can’t actually spend this key”, he’ll do nothing. And after a CSV of, I don’t know, 20, 100 blocks, whatever it is, then the dealer could take that money. But instead, this is the case where if the two parties are interested in privacy, then they’ll agree to cosign a spend-out without a CSV branch, which just has the same effect.

Mark Erhardt: I was wondering whether you could just maybe have the recovery transaction already encoded with a locktime and signed by both. But just by it having a locktime that is set to the future, you could have the recovery already pre-committed by both sides, and then you don’t need the on-script footprint.

Adam Gibson: Yeah. We avoid it anyway in what I’ve written in the paper by – I actually can’t remember if we do it the way you just said, which sounds definitely the right way to do it, or whether I had it done where – that’s a detail, but I think you’re right that that is the most optimal and efficient way to do it. Yeah. I’ll check that.

Mark Erhardt: Because then you don’t need an onchain script that would reveal that something special was going on. Anyway, so yeah, that’s pretty cool. And so, what amount would they be playing for? Just a small portion of the funds in order to mix them around and move them around and obfuscate the ownership graph?

Adam Gibson: Yeah. So, that’s left open, in terms of what fraction of the input that you choose to bet. I guess I should take this opportunity to mention the second layer, which I found was necessary. I didn’t realize originally, and I had to update the paper, because I realized that because this has a structure of two parties funding a MuSig and then spending out of it, there was actually a privacy leak, which was because, let’s say you choose to make a pot for the bet of 1 bitcoin. The problem is that if you just make a single simple bet of 1 bitcoin, well, let’s say both parties contribute 1 bitcoin, so the pot is 2 bitcoins, the problem is it’s quite trivial to see that such a bet has occurred, because there’s two transactions here, not just one. There’s the pay into the MuSig, and then there’s the pay out of the MuSig. And I mean, avoiding the details, basically even though there’s change outputs and there might be some slight details here, there will be a fingerprint, which will say, “Oh, look, there’s an amount there and an amount there”, a bit like how in basic payjoin, even in basic payjoin, there can be cases, depending how you set it up, where you kind of are giving away the amount.

But I realized that actually, you can avoid that problem, admittedly at some cost, which is this. You say, instead of just making one bet, offchain we’re effectively making five bets. And you presign the five bets, the five payouts for the different bets. But the nice thing about that is it’s not just a case of, “Oh, let’s make five bets of 1 bitcoin”, because the problem then is a blockchain analyst can still say to himself, “Well, I think this is a bet. I’m going to look at the possibility if it’s one or two or three or four, divided into an integer number of equal-sized bets”. But they don’t have to be equal-sized. All you have to do is split the bets into private and unknown partition. So, like, the first bet will have 72% of the amount; the second bet will have 19% of the amount; and the third bet will have whatever that is.

Mark Erhardt: 9%!

Adam Gibson: Thank you very much. You’re very good. But that split will be completely secret. Nobody else will know it. The result will be, because there’ll be a certain probability of win on each one of these bets, which itself can be different, it’ll be not just too computationally difficult, but it will actually be impossible for somebody to deduce that the specific payment in the second output is correlated to the payment in the first transaction. I just wanted to mention that because I thought it was a cute little extra.

Mark Erhardt: So, basically, this would look now like a deposit into, like, two UTXOs combining into one output and then a branching out payment into multiple different outputs.

Adam Gibson: Actually, the first transaction structure will be almost always, it could n in, 2 out, but generally perhaps 2 in, 2 out; both participants contribute something. In the funding transaction, let’s say the dealer participant, one of the participants will receive change, and the other party’s change is deferred to the second transaction. But because of presigning, it’s safe in the normal way. And so, you can actually spread the change out to make both the transactions look like very typical payments, or at least I’m claiming that; it’s maybe not exactly true. But the negative of my splitting idea is that if the two parties fall out of agreement, then they’re suddenly broadcasting five payouts onchain and not one. But you’ve already kind of lost, if the two parties don’t agree, you’ve completely lost the privacy properties anyway, because you’ll have scripts in there, or in this case, you’ll have five payouts. It’ll just look very weird. Well, that was pretty quick, but I mean, I think we’ve pretty much covered the idea, more or less.

Mark Erhardt: I’m a little wondering still, obviously you need to split the commitment from the payout, because otherwise, if you did it all in one transaction, one of the two parties could renege as soon as they know the outcome, and they know the outcome as soon as they see. Would it be possible to set it all up in one go, and then the reveal would be separate in some fashion?

Adam Gibson: I’m actually forgetting now whether I had it where the single adaptors is acting as the secret key for all of the reveals, or I can’t remember. I’d have to look back at the paper. It’s been a while. I’d have to get back to you on that, like, how the splitting affects the reveal of the secret behind the commitment, is what you’re asking, I think?

Mark Erhardt: No, I think I’m asking, so one of the unattractive things of how I understood it works now is that you have two transactions, right?

Adam Gibson: Yes, absolutely.

Mark Erhardt: So, I was considering whether it would be possible to have the setup transaction immediately create the payouts, but then it’s the outcome only being looked at afterward. Anyway, if there were a way maybe to make it all in one transaction, but the outcome still being revealed afterwards, you could still have a recovery where people just get back their money after some time, but if they don’t agree… anyway.

Adam Gibson: Can I just weigh in there, because I suddenly obviously thought about, wouldn’t this whole thing be way better if it was just one transaction? I think the simple answer is you could only get that by – I mean, think about Lightning, right? The way Lightning works is you enter into joint control. Once you’re in joint control, you can do a lot of things offchain, right? So, the way I see it is you could have someone or two parties creating a stream of bets by doing one setup, and then each individual bet could be done from that. So, in other words, if you’re starting fresh with two participants who’ve never talked to each other, and it has to be because you have to enter into joint control to start contracting with each other, I don’t think you can avoid that from fresh, which is what we are thinking about. I’m claiming it’s impossible, but maybe I’m wrong.

Mark Erhardt: Right. I’m thinking a little bit about the coinjoin in, what was it, in Phoenix? ACINQ does this where every onchain payment that you receive to the Phoenix wallet is immediately –

Adam Gibson: Yes, spliced in.

Mark Erhardt: No, it is possible to splice it in by revealing something to the counterparty, and then you immediately get a Lightning channel. But if you just wait, it becomes unilateral control.

Mike Schmidt: Is that the Swap-in-Potentiam?

Mark Erhardt: Thank you, Swap-in-Potentiam. So, I was wondering maybe if there are known dealers or something, you could set up the control in-potentiam when you get paid yourself, and then you could enter the bet from there.

Adam Gibson: I’ll have to believe you on that one. I have to read in Swap-in-Potentiam.

Mark Erhardt: Yeah, maybe. Yeah, sorry, I’m really getting into the weeds here.

Mike Schmidt: Adam, where do we go from here? You have a v2 that you made reference to around the amounts of additional obfuscation; is there a v3? Is there a group working on this? You mentioned the Citadel folks, maybe, and CoinSwap?

Adam Gibson: I certainly shouldn’t claim on their behalf that they’re working on it. I just told them about it, and they said, “Well, we’re kind of busy, but we might look at it later”! But no, there isn’t really. I set up an example website, thimbly.org, where I just put, “If you wanted to try it on signet, you could just enter it”. The interesting thing about that, which I found from a kind of UI perspective is a problem, is I was saying, right, somebody wants to come along with their wallet and make this bet with this dealer. Obviously, this isn’t really the idea. The idea is it should be more P2P, and you could do it literally on Bitcoin Core nodes. I mean, it’s set up that you can do that. But I thought, just for fun, here’s a website. You can literally try a bet out on signet, and it does work. But the interesting thing about the UI is that you have to sign a PSBT, because the dealer’s engaging in a MuSig with you. And I found that I could only do it with Sparrow and Core. It’s pretty easy, I suppose, for a technical person to do it. But I think that’s probably an example of many other projects where there’s this attempt to create some kind of collaborative, offchain negotiation of something, is that your coins are in your wallet, but somebody’s coming along and saying, “Oh, please cosign this weird transaction with me”.

So, yeah, it’s set up, and you can try it out. But yeah, you need to use either Sparrow or Core. I don’t think anything else would work. Hello? Are we here? Am I there?

Mike Schmidt: We’re here. You’re there. We’re here.

Adam Gibson: Okay, so I’ve made my point, yeah. So, there we go. But that’s it, there’s nothing else. A v2, that’s just the second version of the paper. I don’t really have any other specific plans.

Mike Schmidt: Okay, cool. Maybe I’ll just drop one piece of Optech history that waxwing mentioned earlier. We did have Oleksandr on in Podcast #340 to talk about, “Emulating OP_RAND”, which was the Delving discussion that Adam mentioned earlier. So, if folks are curious about that piece of the history that we went through, folks can look back at that newsletter or that podcast and transcription as well. Murch, anything else on this item?

Mark Erhardt: No, I think I have gotten all my brainstorming in!

Mike Schmidt: Okay, great. Adam, thanks for joining us.

Adam Gibson: Thank you.

Update on silent payments light clients

Mike Schmidt: We appreciate your time. You’re free to drop a few other things. Moving to our second news item titled, “Update on silent payments light clients”. We have our second guest, Rob, to join us to talk about this one. Rob, with silent payments, you obviously have this static address or this identifier that doesn’t leak anything onchain. But as part of the mechanics there, there is this sort of cost to the receiver that the receiver has to scan every block to find their money essentially, which is cumbersome for a desktop, but pretty brutal for something like a phone wallet, let’s say. There was this discussion that we covered in Newsletter #305 about how light clients should or could get scanning data, and it was sort of an open question on Delving that we talked about back in June 2024. And now, you’ve sort of revived this discussion, and you’ve done some benchmarking and whatnot. So, maybe you can talk about your motivation, what you did, and how you started picking up this conversation?

Rob Segers: Yeah, so I’m coming at this from a totally different perspective, which is UX and UI and more the application layer. And so, we have sort of accepted that creating a new address every time is normal, but actually, from a UX standpoint, it’s absolutely horrible. Imagine you have to recreate a new email address every time to receive an email, because or else everyone else can see what every email is you got. It sounds absolutely absurd. And so, from the UX perspective, silent payments is total magic, and you have to get there. So, the arc that I went, and I’ll just say that arc that I took, and you sort of glanced over that. So, the problem that silent payments solve is indeed that static address, but it creates a new problem, because the silent payment address is two public keys glued together, and – I’m going through my notes here – the sender is deriving things. So, you can’t predict your own payments anymore, you’ve got to find your own money. That’s the problem, it’s shifting a problem to the receiver. And so, that is a costly thing to do.

Mark Erhardt: The sender basically creates a shared secret based on the inputs that they’re using with your silent payments payment instructions. And then, you need to scan for the shared secret based on your scan key.

Rob Segers: Yes, the inputs that someone used. Yes, so a silent payment address, SP1, Sparrow supports it today, it’s really not great today. It’s really two parts. It’s a scan key and a spend key. And each of those keys is a keypair, so there’s four keys. And so, you need to find your own money where money could have been sent, because you need to derive it. And so, for every transaction, there’s this 33-byte value called the tweak, which is built from public information. And every wallet downloads that same tweak. And the scan key is what turns a tweak into an actual address that you can find out, “Okay, one of my addresses was paid”. And so, the question really is, who does the work? Someone has to produce these tweaks, your phone can’t, that requires a full node because you have to go back all the outputs. But checking the tweak is free. So, I did some measurements, I did a lot, the discussion will get there. But checking a tweak’s like 65 microseconds. If you need to do a day of blocks, 144 blocks, you’re at a couple of seconds.

Mark Erhardt: Sorry, when you say checking a tweak, do you mean checking that the tweak was correctly derived from the transactions, or checking whether the tweak with your secret creates a payment?

Rob Segers: It’s money that was pointed at you. So, the tweaks are the same for everyone.

Mark Erhardt: Okay, so once you have the tweaks, yes.

Rob Segers: There’s a tweak for a transaction that’s the same for everyone. And then, you have your scan key that derives an address for you, so that you know, “Okay, this money was intended for me”. So, the sender derives something, and the receiver also has to derive something. And the deriving at the receiver part is a computation that needs to happen, which is the framing of the problem at hand with silent payments. Because the UX is great, you got this static payment address, and the sender deriving, it’s trivial because it’s just its outputs. There’s like a sum and a hash going on. And then, the receiver side has to find his money, literally.

So, there’s two solutions proposed. So, if you hand over your scan key to an index server, that server becomes trusted. And that’s the current design with Frigate, which is the thing that Sparrow connects with, which is also by the same guy, Craig Raw, if I’m right. And so, it’s better than your privacy being laid bare onchain, but that server still sees every payment that you will ever receive, including all the old ones. And then, the other version is where every wallet gets identical data, but the server has no idea which transaction you cared about. So, if there’s a tweak that’s intended for you, you just get the full block. And so, that’s two propositions. Now we’re in the Delving Bitcoin discussion. And the question was, “Well, we need to do some measurements, which design is feasible here”. And so, there’s filters, which is by setavenger, 2024 spec, which is saying it’s just filtering based on taproot outputs. So, it only sends you the taproot outputs. So, okay, your payment might be in this block, in these taproot outputs. So, they’re cheap because they filter, but they produce false alarms, apparently false positives, and that hasn’t ever been built yet, as I’ve found. Or sending the full payload, sending the full block, if there’s a tweak for you, which is what BlindBit Oracle v2 does, which is setavenger’s index server software.

Adam Gibson: Sorry, can I ask a question? Is that kind of filtering the same as the kind of filtering we have in BIP157? Is it that kind of filtering? Do we know? I’m just curious what the technology is.

Rob Segers: What kind of filtering? I’m not sure, but it’s filtering only the taproot outputs.

Adam Gibson: Right, that makes it easier, yeah.

Rob Segers: It’s sending not the full block, it’s sending only those outputs, because only those could be silent payments.

Mike Schmidt: I think it’s BIP158 compact block filters.

Adam Gibson: It’s the same type, is it? Okay, thanks, cool.

Mark Erhardt: Yeah, basically, so BIP158 allows a client to pull the filter with their own wallet data to see if they got paid and they might get a false positive. But for silent payments, really you only care about P2TR outputs, because only a P2TR output could be a silent payment output, because it’ll always create a P2TR output. So, if you reduce the filter to only P2TR outputs, you can make much smaller filters, because there’s fewer of them.

Rob Segers: Exactly, yeah. And so, there’s two proposed solutions there. Either the filters send less over the line, less bandwidth. So, two proposed solutions there to determining what the client sends, either the full block or the filters. So, I did some measurements, every block since taproot, 255,000 blocks, no sampling. And so, the numbers, let me peek, yeah, the difference is not as big as expected, because it boils down to you still need the tweaks anyway. And the tweaks are 6 GB. So, the first one with a silent payment is 6 times smaller. The second finding was that it’s about 2x, right? A wallet on the filter route still needs, like, 6 GB of tweaks, and it’s 1 GB of filters. So, it’s 7 GB, and the full payload is 15 GB. So, that ratio is only 2x, which isn’t super huge. What was interesting was that the ratio dropped, the full payload is nearer to 3 MB daily today. And so, the discussion was framed on bandwidth. Like, okay, is the filter going to save us so much bandwidth that that’s going to be the better solution?

But perhaps what we gain with sending full blocks is not communicating to the server, because a second request has to happen to fetch the actual transaction. You send more data over the line, but you get more privacy. And so, does a hit force your wallet to go back and ask the index server for something extra? That’s a signal in and of itself. And so, that could be framing of the discussion going forward, where that bandwidth question might not matter as much. And then, we could go, okay, perhaps we can make this a setting. But then, there’s the history of BIP37 and Bloom filters, where light wallets could tune privacy against bandwidth, which is a bit of a similar discussion, apparently. And then, every wallet chose bandwidth, because that’s the UI, the UX, I guess, angle again, where I guess I’m usually coming from. And that got turned off for a DoS thing, not for privacy. So, perhaps the better thing is to settle a default now, okay, sending the full block anyway, which tells the server nothing about you. That’s a little bit perhaps what these measurements could indicate.

Mike Schmidt: What has reception been, Rob, to the, I guess, refreshed data, whether public feedback or private feedback that you’ve gotten from folks on what you’ve done here?

Rob Segers: What I’ve done here, the more feedback I got was more on the application layer side of things, and not specifically the measurements itself. The Delving Bitcoin thread is still where it’s at, because I wanted a new account, I couldn’t join the actual discussion. It’s posted in the Bitcoin mailing list, and some PRs and things happened related, but the reaction was more towards the application layer. Silentpayments.net was a website that I made, where I combined silent payments and Lightning and Cashu actually into one name. Again, the UX side of things, which did have quite positive feedback, because that’s where I could see this going, is you just have one name, and it’s both onchain and Lightning, and you can just send money to it depending on the sender’s preference. So, that’s some more things on top. Also, DNSSEC proving on SeedSigner, that was something interesting, but that’s not been actually run on actual SeedSigner, it’s all in a simulator that’s live. But on the measurements themselves, nothing yet so far, more on the application layer. I’ve been building more on the application side of things, and UI and UX, and not so much these lower-level technical things, because I’m not all too comfortable in these lower-level technical things.

Mike Schmidt: Well, I guess the next steps here sounds like, audience, keep an eye on the mailing-list posts as well as that potentially revived Delving thread.

Rob Segers: Exactly. So, there’s perhaps one thing more to add, is a condensed pre-draft, something that merges the existing specs with the measurements, and that’s up for, I guess, the best case is someone to disagree with, to replicate, refute, and just start a discussion. And then, where this could be going, the silent payments server that I’ve started running is, I like the provably fair framing, where I’m publishing hashes to nostr, where you can check if it’s all correct, because what can happen is not a silent payments server loudly lying to you, but silently leaving tweaks out, which is what actually happened in 2024 with the Cake Wallet users, that they didn’t see payments coming in. And so, silent omission is something that is still a problem, and so publishing hashes is something that I threw on top. But then, down the road, this could go even further, and it’s on the silentpayments.net roadmap also.

Mark Erhardt: So, I’m a little confused at the trade-off. So, the two options for serving the silent payment data, one was the approach to have a separate filter just for taproot outputs that a light client would be able to scan against. And then, if they have a false positive or a real positive, they would download the full block from someone, but not necessarily the one that served them the filter. And then, the other one would be to just stream the tweak data directly for each P2TR output, which then would enable the recipient to not just scan for their scan key in the filter, but directly calculate whether anything paid them. But in either case, I’m not quite sure how one or the other would be a loss of privacy. In either case, you sort of reveal that you’re interested in potential silent payments, because you’re downloading either tweak data or a filter specifically for P2TR outputs. But in neither case can the server tell which transaction you actually were interested in.

Rob Segers: Fair, but in the filter case, because there is a conditional request, that again happens. And that conditional request in and of itself is a signal to the server compared to just getting it all at once.

Mark Erhardt: Oh, I see, the conditional request of how many blocks do you want the filter for, and you can calculate the filter, “Only give me new P2TR outputs that weren’t spent in some range of blocks”, right?

Rob Segers: So, you’re getting only the filter, and then you still have to do the follow-up request to get the full thing. And that request, that’s what it’s about.

Mark Erhardt: Right, but you can pull a block from any peer. You don’t have to pull it from the person that you got the filter from.

Rob Segers: That’s true, yes.

Mark Erhardt: If you just get a block from someone, that is one of the most common P2P messages.

Rob Segers: You don’t have to ask that server.

Adam Gibson: But, Murch, if you’re a node, yeah, then what you’re saying is true, but aren’t we talking about non-nodes, a light client?

Mark Erhardt: Oh, if you have only a single server, yes. Yeah, but many light clients have multiple connections. But yeah, if you’re connected to a server that serves you the data and you would only be talking to them, yes, now I get it. Now I get the concern. Thank you.

Mike Schmidt: Rob, thanks for your time. Thanks for joining us today. You’re free to stay on or drop if you have other things you’re working on.

Rob Segers: Thanks for having me, yeah.

Mike Schmidt: Cheers. Releases and release candidates. We have three this week. And if you’re wondering why I’m talking about them, we do not have Gustavo today. He’s out. So, Murch and I are leading the Releases and Notable code segment.

LDK v0.3-rc1

LDK v0.3-rc1. This is kind of a big one. I’ll list a few of the items from this RC. Obviously, read the notes before you test. But RC1 includes the ability for LDK users to fee bump a splice that’s waiting to confirm, and you can do that using RBF. And you can also add and remove funds in the same splice. Also, and this surprised me that it’s taken so long, but anchor channels are now the default when opening channels in v0.3. And also, apps built on LDK now have to explicitly say, “Yes”, to an incoming channel instead of automatically accepting it, which was the previous behavior. And, Murch?

Mark Erhardt: Well, that was a slippery slope there, Mike. A year ago, nobody was doing anchor channels. Earlier this year, we were like, “Yeah, some people are doing anchor channels”. And now you’re like, “LDK, why did this take so long?”!

Mike Schmidt: All right, maybe a little harsh. When anchor channels… There is a heads up here for developers. If your app issued a BOLT11 invoice, we talked about this back in #405 and #407, but if you’re issuing a BOLT11 invoice with payment metadata on an older version, upgrading to this version makes those invoices stop working. We talked about some of the weird payment metadata in Podcasts #405 and #407 as well. And I guess the general disclaimer for users is read the API and compatibility notes before you test this particular release.

LDK v0.2.6

Second LDK release, LDK v0.2.6. This is a security fix for the current stable line of releases, so not anything related to the v0.3. There’s two bugs that are addressed here. The first one is a crash bug. If your LDK node forwarded two HTLCs with the same payment hash and the second one got rejected, the LDK node’s channel state could get saved in a form that it can’t load back. So, the node goes down and can’t come back up, at least not cleanly. And the second security fix is one that could result in loss of funds. If a peer starts a splice with you, they could trick your LDK node into putting too much toward fees, and that extra doesn’t necessarily go to miners, it actually goes into the peer’s own output. So, the peer can basically take money through some of the fee math. So, if you’re on v0.2, upgrade and grab these two fixes, among others.

Mark Erhardt: I think if you’re on any Bitcoin software this year, you just want to keep your eyes peeled and watch carefully what’s happening in the corresponding repositories. Obviously, it’s also gotten easier than ever to check what code changes went into releases. So, if you subscribe to ‘trust but verify’, you should maybe point some of your compute at making sure that you want to upgrade, but you kind of do want to be on top of things these days.

BTCPay Server 2.4.4

Mike Schmidt: Yes, and not surprisingly, the last release is in a similar vein to Murch’s energy here. This is a BTCPay Server 2.4.4 release. It’s a follow-up security release after a 2.4.2 incident. I believe we talked about in #418, where attackers stole LND macaroons and potentially funds as well. This release removes BTCPay’s old BitPay-style API keys entirely. So, they have this notion of these Greenfield API keys, but also these old API-style keys, and it removes the old-style keys. Existing Greenfield keys still work, but if you have integrated using the BitPay-style keys, you have to move those to the new Greenfield API keys that have been around for a while. So, no more BitPay-style API keys. Also, in BTCPay Server, changing an invoice’s state now requires, I guess, proper authentication or authorization, and a restricted API key cannot create unrestricted one anymore. API keys are now stored in hash instead of plain text. There’s more on that in the code section that we’ll get to. There’s some changes to the Docker setup, which tightens some access and enforces a little bit more permissions, I guess, when you’re setting LND up in Docker.

So, why are all these LND changes there, because we’re talking about BTCPay? Well, BTCPay says attackers are actively hitting LND’s password-change endpoints on servers. There’s been some chatter about this. And the idea here is that people ended up reopening LND API access after the 2.4.2 fix for BTCPay Server, and that was causing issues. So, if you did do that, close that gap and check out the release notes for some of the details there. Two last points here. If you use the web-hosting billing plugin for BTCPay Server, you need v4.0.0 and a new Greenfield key. There’s a migration guide around that particular plugin. And also, back to, I guess, Murch’s point earlier, everyone running BTCPay should upgrade and read these breaking changes as well.

Mark Erhardt: I don’t remember who said it, but there was this tweet recently, “We’re watching the future being born, and in the transition period, we live in the time of monsters”. And it was more pithy in the written word, but yeah, I must say it is quite fascinating what we can do with AI assist now, and it has really changed how people work with code and what they can do in security research, and it is a transition period.

Adam Gibson: Guys, I want to say you went one hour and a half without being an AI podcast; congratulations!

Mike Schmidt: And now a moment from our sponsors!

Mark Erhardt: Well, it’s the topic of the time.

Mike Schmidt: Well, it’s not something that we normally cover, which is, I mean, we have covered some things here and there, but obviously the Liquid thing, we had Core Lightning (CLN) that we got into pretty deep with Níckolas, and their thinking about rolling out binary-only, which immediately get decompiled. So, yeah, it’s quite a crazy, crazy time right now. And so, it’s not surprising that not only these three releases that we’re covering this week, but other recent releases that we’ve covered, in addition to some of the PRs that we’ll get to later, are around security-related work. And I actually think that, was it Nicolas from BTCPay, that had some tweet in the last weeks or so that, “Hey, we’re not adding any features. We’re spending the next X months”, or versions or whatever it was, “just on security and hardening or moving old stuff”. So, we see some of that here, even with the BitPay-style API keys being removed. So, some legacy-type stuff that could have issues.

So, I guess we can move to Notable code and documentation changes. Murch is going to take us through a handful of Bitcoin Core ones, and then I’ll pick up with the remaining items. Murch, why are there so many Bitcoin Core PRs?

Mark Erhardt: I have no idea. Oh, well, yesterday was branch off. So, presumably a bunch of stuff got merged yesterday and today, but that would have been after our, what is it?

Mike Schmidt: Cut off, yeah, publication.

Bitcoin Core #35949

Mark Erhardt: Yes, thank you, publication cut off. So, yeah, I’m actually a little surprised that there are so many, but let’s get into it. So, Bitcoin Core #35949 updates how miners build blocks. If you’re building block templates, your Bitcoin Core node will now start to be forward-compliant to a rule that is being proposed by BIP54, the consensus cleanup, which mitigates one of the time warp issues that was identified before BIP54 was written. This is the Murch-Zawy attack. So, if you’re familiar with the time warp attack, people can basically change how the time is progressing in blocks in order to fudge with the difficulty. In the Murch-Zawy attack, you could still do some time fudging, even if you stuck to the originally proposed BIP54 rules. Zawy had a Delving thread on that a while back, and I had an idea too, how to make it more potent.

So, the main issue that you need to fix is you need difficulty periods to have a positive amount of time. So, the last block in a difficulty period has to have a higher timestamp than the first block in the difficulty period, and that fixes this issue. Otherwise, you can sort of make the time go forward a lot and then rewind back a lot, and that allows you to still drive down difficulty and start producing blocks much faster. So, our new block templates will always have a positive time on the last block compared to the first block in a difficulty period, which honestly should happen anyway, because 2,015 blocks later, you should have had at least one second pass, right?

Bitcoin Core #34931

Anyway, that was that one. Second one is Bitcoin Core #34931. This one fixes an issue in the LevelDB wrapper. And in the LevelDB wrapper, there was no distinction between whether you could not deserialize a UTXO from disk or whether a UTXO was not present in your chainstate. So, if you’re missing a UTXO that you are spending in a transaction, obviously that transaction is invalid; and if it’s in a block, that block is invalid, because you can’t spend funds that don’t exist. But if there were some deserialization issue with one UTXO, it would have also shown up as the UTXO not being present, rather than just, “I failed to read something”. And then, it would mark the block as invalid and that would fork you off the chain. As far as I understand, there is no such deserialization issue. Someone just found that these two situations should be properly distinguished, probably by throwing a lot of compute at our repository. So, this is just a bug fix. As far as I know, there’s no security issue here, okay? All right. Stop me if you have comments.

Adam Gibson: Sorry, could I just pick up on that one? That’s slightly mind-breaking. What could be the actual functional difference if a person has a database from which they cannot read the entry, or that it was invalid? I’m not following how there would be a functional difference.

Mark Erhardt: Sorry, whether a UTXO was non-existent, so that it was not found in the database, or whether you could not read the database entry, is the functional difference. And yes, in both cases, you’re probably in a state where you need to re-sync or fix your database. Well, either you have found an invalid block and everybody else seems to believe it, which probably also should make your head scratch.

Adam Gibson: Yeah, it’s fine. As you said, there was no actual…

Mark Erhardt: Yeah, I think it’s probably never going to be an issue unless you ever have an issue in deserializing stuff from the database, and that would be a pretty big issue by itself.

Adam Gibson: Okay.

Mark Erhardt: I think it’s sort of the thing that you find when you spend tens of thousands of dollars on compute.

Adam Gibson: Yeah, true!

Bitcoin Core #36048

Mark Erhardt: All right. The next one is an issue where you could shoot yourself in the foot. So, if you used the -walletnotify configuration option and used a wallet name as a regular expression, essentially, the -walletnotify could be used to run commands on your machine. So, you already have access to a node, you can create wallets, and then you are also running -walletnotify and now you can use wallet names to run commands on your own computer that you already have access to. So, this was fixed by properly escaping wallet names in the -walletnotify messages, but also kind of an interesting problem.

Adam Gibson: I’m not sure, maybe I’m wrong here, but I’m not sure that it’s quite as trivial as, “Well, you already have access”, because, I mean at least theoretically, this is an authentication system. So, somebody could be saying to someone else, “You are authenticated to do this”, and I expect them to be able to use the wallet, but I don’t expect to be able to execute arbitrary code on their server, right? So, I mean, maybe nobody’s using Bitcoin like that, but I don’t know. It seems like a non-trivial bug.

Mark Erhardt: Fair enough. You could do little Bobby Tables, if you have RPC access to a server that’s running wallets already. So, I guess if you’re running an Uncle Joe setup where people can create their own Bitcoin Core wallets on your Bitcoin Core node that you have given them VPN access to or something, they could break out of it.

Adam Gibson: I think Uncle Joe is usually Stalin. I think you meant to say ‘Uncle Jim’, but that’s fine!

Mark Erhardt: Thank you, but I didn’t quite catch it. What is Uncle Joe usually?

Mike Schmidt: It’s Uncle Jim, I think.

Adam Gibson: Uncle Joe used to be Stalin, but not nowadays. But never mind!

Mark Erhardt: Oh, shit, okay! Never mind! Okay, Uncle Jim is what I meant anyway. I think I’m just too young for that reference, or something!

Adam Gibson: Very, very much so. But yeah, I do think the Uncle Jim scenario is relevant, or a corporate environment, or something like that.

Mark Erhardt: All right. But either way, well, okay, maybe I’m over-trivializing, but it is a very special setup, and then you need people that you trust very heavily trying to be malicious to you, and I don’t know. It seems hard to exploit, let’s say it that way.

Bitcoin Core #36123 and #36169

All right, we have two more here, Bitcoin Core #36123 and #36169. This fixes an unbounded memory growth issue. When you’re running an HTTP server, you could make a server send messages slower than they can process them, and that would make the buffer eventually bloat, and I think you could probably crash the server. And yeah, you basically introduce some TCP backpressure here in order to not allow them to send requests faster than you can process them. And yeah, basically just fixing a DoS issue. Sorry, I learned this morning that I’m responsible for these, so cut me some slack here!

Mike Schmidt: Wait till we get to the Lightning ones!

Bitcoin Core #36176

Mark Erhardt: All right, the last one that I have is Bitcoin Core #36176. So, this fixes an error when you’re trying to load a wallet while also running with the -nosettings option, and then when you’re loading the wallet, are trying to make sure that it’ll load on the next startup too. So, it’s sort of an odd situation. You’re forbidding your node to update the settings, but at the same time you’re importing a wallet and you want to start loading it every single time. This used to cause a crash because of the conflict where it wasn’t allowed to update the settings, but was requested to start loading this wallet every time. And now, it instead completes the operation and warns that the preference couldn’t be saved because -nosettings was on. So, again, a bug that would crash your node if you do some maybe conflicting things at the same time.

Mike Schmidt: It’s nice that we have all of these totally human testers testing all of these crazy edge cases that you’re announcing fixes for, Murch.

Mark Erhardt: Yeah, I mean, it’s good that all these little issues are being found, but these are good fixes. So obviously, we wouldn’t be talking about them if they were super-serious, then they would be fixed and later disclosed. I think this is just a series of bugs of, well, a lot of testing going into Bitcoin Core, where because computers can do a lot of things in parallel and serialize things and try a ton of scenarios that nobody would ever come up with, we find these and it’s great they’re being fixed, but the sky is not falling.

Core Lightning #9434 and #9473

Mike Schmidt: We’ll move out of the Bitcoin Core notable PRs and we’ll move on to CLN with Core Lightning #9434 and Core Lightning #9473. These are related because they’re two fixes related to the askrene plugin, which is routing engine. Askrene has this feature where you can set biases, meaning that you can tell askrene that you prefer to connect to certain nodes or avoid certain nodes or channels, and you can actually save those biases so that they can survive a restart of CLN. And so, that’s maybe the preface to the two bugs. The first bug is that when you’re loading one of these saved biases and it had a description attached, the code used a piece of memory that had already been freed. So, essentially that causes memory error, it crashes askrene as a plugin. But since CLN treats askrene as an important plugin, it actually takes the whole node down. So, that was the first bug around askrene.

Then, the second bug was if you removed one of these biases by setting it to zero, the code actually deleted it from memory before saving it, so then it crashed on a null pointer issue, because the removal never got saved, the old bias came back after a restart. So, the fix there is now it saves that zeroed-out bias first. More bugs.

LND #11061

LND #11061 is more work on LND’s effort towards BOLT12 offers. This particular PR adds signing and verifying invoice requests and invoices using Schnorr signatures. So, the way BOLT12 signatures work is you build this merkle tree out of the messages fields and then sign the root. Before this, LND only checked that a signature was there, and now LND will actually check the signature if that is valid and it will reject bad ones. This builds on some of the invoice request encoding that we talked about back in #413, which we link to. Good to see more progress on BOLT12 offers from the LND folks.

LND #11125

LND #11125 lets you lock up wallet UTXOs in LND until the transaction spending them gets a certain number of confirmations you pick. So, LND now lets you do that. But before this, you had two options and both of them had issues. You could do time-based locks that could expire before your transaction was actually confirmed; or the lock released at one confirmation, but then you’re at a position where a potential reorg could unlock the coins while your transaction was sort of still up in the air, so the wallet might actually spend them somewhere else. So now, there’s this LeaseOutput and FundPsbt that take a confirmation count, and the lock actually survives reorgs and doesn’t expire on a timer. So, you still have to release the lock yourself if you abandon the transaction, it’s noted in the write-up, and then also timelocks are still the default.

LND #11064

LND #11064. This is a change so that LND will now always say explicitly which channel type it wants when opening a channel, which is actually what the spec requires. It puts the channel_type in the open_channel and then spits it back out in accept_channel, and then it also rejects incoming opens that leave out that field. So, maybe one thing to note here is that if you’re using LND and you open a channel through the RPC without picking a type, LND picks one based on what it knows both sides support. And this is sort of tied into the fix that we talked about back in Newsletter #407, where LND was picking a taproot channel implicitly and then failing on it. So, this is sort of related to that line of work and fixes.

BTCPay Server #7561 and #7542

And we’ll round out with some BTCPay Server PRs, some of which we sort of talked about earlier with the Releases for BTCPay Server. But BTCPay Server #7561 and #7542, these are API key storage changes that shipped in that 2.4.4 release that we talked about earlier. The #7561 change made a change such that BTCPay Server API keys are now stored as a hash and then a key ID, and not the actual secret. And then, there’s also a cleanup job in 2.4.4 that wipes any of that leftover plain text once it’s more than five minutes old. Your existing keys will keep working but you can’t view the secret in the server anymore. The idea here is if someone dumps the database, that they don’t get usable keys. And finally, revoking a key now uses the key ID instead of the secret. So, that was the #7561.

For #7542, this is again around API key storage issues. When you create a new key, the secret is no longer in the URL that sends you to the key management page, for reasons that URLs end up in browser history, server logs and all kinds of things, so you shouldn’t have secrets in them. And so, this fixes that, this #7542.

BTCPay Server #7559

Last PR this week, BTCPay Server #7559. Maybe some quick background. In BTCPay, the invoice has two different clocks associated with it. There is a clock for the payment deadline, which is 15 minutes by default, and that locks in the exchange rate. So, using BTCPay Server, you have these fiat exchange rates. But there’s also this monitoring window that’s separate from the payment deadline window that runs after that. So, after a full day by default, BTCPay Server will stop watching for late payments. And a merchant can configure both of these different windows. The bug here is that onchain payments in BTCPay Server were watched through that whole monitoring default period, or the whole period, whatever you had that configured to; but the Lightning side stopped after the 15 minute payment deadline. So, a customer could pay the Lightning invoice a couple of minutes late and then the payment would go through on Lightning, but BTCPay Server would never record it, and the merchant had to sort this whole fiasco out by hand. Now, Lightning is watched through that same monitoring window as the onchain and late payments still get recorded. They get recorded as being paid late but they get recorded. And the payment deadline itself doesn’t change in any of this. It just doesn’t watch forever. So, that’s a fix there. And I think that’s it, Murch.

Mark Erhardt: Yeah, the dog barking in the background just reminded me of times when we sometimes had roosters crowing in the background.

Mike Schmidt: Ah, yes, the good old days.

Mark Erhardt: Still miss those days. Yeah, all right, I think we did it.

Mike Schmidt: Well, Adam, we want to thank you for joining us and also for hanging on, we want to thank Rob for joining us earlier for this week’s episode. And, Murch, thank you for co-hosting and for all of the audience for making it through. Cheers.