<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en"><generator uri="https://jekyllrb.com/" version="4.3.2">Jekyll</generator><link href="https://wavey.info/feed.xml" rel="self" type="application/atom+xml" /><link href="https://wavey.info/" rel="alternate" type="text/html" hreflang="en" /><updated>2026-11-15T20:37:26+00:00</updated><id>https://wavey.info/feed.xml</id><title type="html">blank</title><subtitle>A site for posts by wavey.
</subtitle><entry><title type="html">Customizing Reth for Up to 16× Faster SlotScan Tracing</title><link href="https://wavey.info/posts/2026/customizing-reth-to-improve-slotscan-performance/" rel="alternate" type="text/html" title="Customizing Reth for Up to 16× Faster SlotScan Tracing" /><published>2026-07-29T04:00:00+00:00</published><updated>2026-07-29T04:00:00+00:00</updated><id>https://wavey.info/posts/2026/customizing-reth-to-improve-slotscan-performance</id><content type="html" xml:base="https://wavey.info/posts/2026/customizing-reth-to-improve-slotscan-performance/"><![CDATA[<p>I built <a href="https://slotscan.info/">SlotScan</a> to decode contract storage layouts and let users trace every storage write in a transaction back to a human-readable variable. Until now, no standard RPC method could produce the view we wanted, so building it required an awkward combination of expensive trace calls.</p>

<p>In <a href="https://wavey.info/posts/2025/reverse-engineering-evm-storage/">Part 1</a>, I wrote that multiple trace passes were necessary to collect the state diff, ordered writes, call-frame attribution, and hash preimages SlotScan needs.</p>

<p>That was true at the JSON-RPC boundary: none of Reth’s default methods returned all of that evidence in one trace. But what if I moved my custom tracing logic into Reth itself?</p>

<p>Using Reth’s NodeBuilder, I built a custom binary that handles SlotScan’s tracing in one RPC call backed by one replay. The transaction executes once, and every trace view is built from that execution.</p>

<p>Across four sample transactions, I measured a <strong>2.03× to 16.07× speedup</strong>. Trace processing time for the largest fell from 10.5 seconds to 656 milliseconds.</p>

<figure class="slotscan-trace-flow">
  <svg viewBox="0 0 680 350" role="img" aria-labelledby="slotscan-flow-title slotscan-flow-description">
    <title id="slotscan-flow-title">SlotScan tracing before and after the native Reth integration</title>
    <desc id="slotscan-flow-description">The legacy SlotScan flow called debug trace transaction twice in sequence, once with the prestate tracer and once with a JavaScript tracer. The native SlotScan RPC returns both evidence views from one replay.</desc>

    <defs>
      <marker id="slotscan-flow-arrow" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="5" markerHeight="5" orient="auto">
        <path class="arrow-head" d="M 0 0 L 10 5 L 0 10 z" />
      </marker>
    </defs>

    <text class="heading" x="20" y="28">BEFORE</text>
    <text class="count" x="660" y="28" text-anchor="end">2 replays</text>

    <rect class="node" x="20" y="57" width="82" height="48" rx="6" />
    <text class="label" x="61" y="86" text-anchor="middle">SlotScan</text>
    <path class="flow" d="M 102 81 H 120" />

    <rect class="node" x="122" y="52" width="220" height="58" rx="6" />
    <text class="label" x="232" y="77" text-anchor="middle">debug_traceTransaction</text>
    <text class="detail" x="232" y="96" text-anchor="middle">prestateTracer</text>
    <path class="flow" d="M 342 81 H 360" />

    <rect class="node" x="362" y="57" width="92" height="48" rx="6" />
    <text class="label" x="408" y="86" text-anchor="middle">replay</text>
    <path class="flow" d="M 454 81 H 472" />

    <rect class="node" x="474" y="57" width="186" height="48" rx="6" />
    <text class="label" x="567" y="86" text-anchor="middle">state diff</text>

    <path class="flow" d="M 567 105 V 124 H 232 V 139" />

    <rect class="node" x="122" y="141" width="220" height="58" rx="6" />
    <text class="label" x="232" y="166" text-anchor="middle">debug_traceTransaction</text>
    <text class="detail" x="232" y="185" text-anchor="middle">JavaScript tracer</text>
    <path class="flow" d="M 342 170 H 360" />

    <rect class="node" x="362" y="146" width="92" height="48" rx="6" />
    <text class="label" x="408" y="175" text-anchor="middle">replay</text>
    <path class="flow" d="M 454 170 H 472" />

    <rect class="node" x="474" y="146" width="186" height="48" rx="6" />
    <text class="label" x="567" y="175" text-anchor="middle">ordered evidence</text>

    <line class="divider" x1="20" y1="229" x2="660" y2="229" />

    <text class="heading" x="20" y="262">AFTER</text>
    <text class="count" x="660" y="262" text-anchor="end">1 replay</text>

    <rect class="node native" x="20" y="281" width="82" height="48" rx="6" />
    <text class="label" x="61" y="310" text-anchor="middle">SlotScan</text>
    <path class="flow" d="M 102 305 H 120" />

    <rect class="node native" x="122" y="281" width="220" height="48" rx="6" />
    <text class="label" x="232" y="310" text-anchor="middle">slotscan_traceTransaction</text>
    <path class="flow" d="M 342 305 H 360" />

    <rect class="node native" x="362" y="281" width="92" height="48" rx="6" />
    <text class="label" x="408" y="310" text-anchor="middle">replay</text>
    <path class="flow" d="M 454 305 H 472" />

    <rect class="node native" x="474" y="281" width="186" height="48" rx="6" />
    <text class="label" x="567" y="300" text-anchor="middle">state diff +</text>
    <text class="label" x="567" y="318" text-anchor="middle">ordered evidence</text>
  </svg>

  <figcaption class="caption">Two sequential trace requests became one native replay.</figcaption>
</figure>

<style>
  .slotscan-trace-flow {
    margin: 1.5rem auto;
    max-width: 680px;
  }

  .slotscan-trace-flow svg {
    display: block;
    height: auto;
    width: 100%;
  }

  .slotscan-trace-flow text {
    fill: var(--global-text-color);
    font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
  }

  .slotscan-trace-flow .heading {
    font-size: 15px;
    font-weight: 600;
    letter-spacing: 0.04em;
  }

  .slotscan-trace-flow .count,
  .slotscan-trace-flow .detail {
    fill: var(--global-text-color-light);
  }

  .slotscan-trace-flow .count {
    font-size: 14px;
  }

  .slotscan-trace-flow .label {
    font-size: 15px;
    font-weight: 500;
  }

  .slotscan-trace-flow .detail {
    font-size: 13px;
  }

  .slotscan-trace-flow .node {
    fill: var(--global-bg-color);
    stroke: var(--global-text-color-light);
    stroke-opacity: 0.35;
    stroke-width: 1.25;
  }

  .slotscan-trace-flow .node.native {
    fill: rgba(91, 145, 103, 0.08);
    stroke: rgba(91, 145, 103, 0.7);
    stroke-opacity: 1;
  }

  .slotscan-trace-flow .flow {
    fill: none;
    marker-end: url(#slotscan-flow-arrow);
    stroke: var(--global-text-color-light);
    stroke-width: 1.5;
  }

  .slotscan-trace-flow .arrow-head {
    fill: var(--global-text-color-light);
  }

  .slotscan-trace-flow .divider {
    stroke: var(--global-divider-color);
  }
</style>

<h2 id="the-problem-one-tracer-per-request-meant-two-executions">The problem: one tracer per request meant two executions</h2>

<p>Reth’s execution engine could produce the evidence SlotScan needed. The limitation was its default external API: <code class="language-plaintext highlighter-rouge">debug_traceTransaction</code> runs one selected tracer per request, and that tracer determines what the execution returns. No default method combined Reth’s pre/post state diff with SlotScan’s custom ordered evidence.</p>

<p>SlotScan needed two kinds of evidence:</p>

<ul>
  <li>an authoritative diff of committed state changes; and</li>
  <li>an ordered timeline of every write attempted, including repeated, restored, transient, and reverted writes.</li>
</ul>

<p>The first request used Reth’s built-in <code class="language-plaintext highlighter-rouge">prestateTracer</code> in diff mode to produce the committed state diff. The second used a compact JavaScript tracer to record the ordered execution evidence and the hash preimages needed to resolve mappings and dynamic storage.</p>

<p>Each request performed its own heavyweight replay. To trace a historical transaction, Reth reconstructs the block’s starting state, processes earlier transactions in the same block, and then executes the target while the selected tracer observes it.</p>

<p>Putting both calls into one JSON-RPC batch would save a network round trip. It would not save either replay or execution.</p>

<h2 id="the-implementation-extending-reth">The implementation: extending Reth</h2>

<p>I wanted one canonical replay without losing either evidence view. I also wanted to reuse Reth’s state database, execution environment, and tracing controls instead of reimplementing historical state reconstruction or maintaining a permanent source fork.</p>

<p>Luckily, <a href="https://reth.rs/docs/reth/builder/struct.NodeBuilder.html">Reth’s NodeBuilder</a> made this possible. Its <a href="https://reth.rs/sdk/examples/modify-node/"><code class="language-plaintext highlighter-rouge">extend_rpc_modules</code></a> hook registers custom methods before the servers launch, keeping the SlotScan integration narrow and Reth upgrades manageable.</p>

<p>The node-level change is small:</p>

<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="n">handle</span> <span class="o">=</span> <span class="n">builder</span>
    <span class="nf">.node</span><span class="p">(</span><span class="nn">EthereumNode</span><span class="p">::</span><span class="nf">default</span><span class="p">())</span>
    <span class="nf">.extend_rpc_modules</span><span class="p">(|</span><span class="n">ctx</span><span class="p">|</span> <span class="p">{</span>
        <span class="k">let</span> <span class="n">adapter</span> <span class="o">=</span> <span class="nn">RethTraceAdapter</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="n">ctx</span><span class="py">.registry</span><span class="nf">.eth_api</span><span class="p">()</span><span class="nf">.clone</span><span class="p">());</span>
        <span class="n">ctx</span><span class="py">.modules</span>
            <span class="nf">.merge_configured</span><span class="p">(</span><span class="nn">SlotScanRpc</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="n">adapter</span><span class="p">)</span><span class="nf">.into_rpc</span><span class="p">())</span><span class="o">?</span><span class="p">;</span>
        <span class="nf">Ok</span><span class="p">(())</span>
    <span class="p">})</span>
    <span class="nf">.launch_with_debug_capabilities</span><span class="p">()</span>
    <span class="k">.await</span><span class="o">?</span><span class="p">;</span>
</code></pre></div></div>

<p>Reth still handles networking, consensus, the database, the transaction pool, and its standard RPC methods; SlotScan adds one <code class="language-plaintext highlighter-rouge">slotscan</code> namespace.</p>

<p>Inside <code class="language-plaintext highlighter-rouge">slotscan_traceTransaction</code>, the adapter invokes Reth’s canonical <code class="language-plaintext highlighter-rouge">spawn_trace_transaction_in_block_with_inspector</code> helper and supplies a custom REVM inspector. When the replay finishes, the completion closure receives the inspector output, execution result, and historical database view—before any JSON serialization.</p>

<p>During execution, <code class="language-plaintext highlighter-rouge">SlotScanInspector</code> records ordered <code class="language-plaintext highlighter-rouge">SSTORE</code> and <code class="language-plaintext highlighter-rouge">TSTORE</code> operations, call-frame outcomes, relevant <code class="language-plaintext highlighter-rouge">KECCAK256</code> preimages, and a bounded set of storage reads. REVM’s journal supplies the immediate old value for each write, while its call hooks preserve storage attribution and rollback semantics.</p>

<p>After execution, <code class="language-plaintext highlighter-rouge">GethTraceBuilder</code> derives the committed state diff from the same result. The replay database supplies the transaction-start storage values.</p>

<h2 id="the-result-2-to-16-faster">The result: 2× to 16× faster</h2>

<p>Before measuring performance, I verified that both paths returned equivalent state diffs, ordered writes, frame attribution, hash evidence, storage observations, step counts, and transaction identity. I then warmed both modes and ran eight paired samples with alternating order against the same Reth v2.4.1 node.</p>

<div class="slotscan-benchmark-table">
  <table>
    <thead>
      <tr>
        <th>Transaction</th>
        <th>Steps</th>
        <th>Writes</th>
        <th>Legacy</th>
        <th>Native</th>
        <th>Speedup</th>
      </tr>
    </thead>
    <tbody>
      <tr>
        <td>ERC-20 transfer</td>
        <td>571</td>
        <td>2</td>
        <td>1,101.5 ms</td>
        <td>543.7 ms</td>
        <td>2.03×</td>
      </tr>
      <tr>
        <td>Reverted writes</td>
        <td>318,375</td>
        <td>468</td>
        <td>2,360.9 ms</td>
        <td>487.4 ms</td>
        <td>4.84×</td>
      </tr>
      <tr>
        <td>Proxy voting</td>
        <td>167,759</td>
        <td>99</td>
        <td>1,587.1 ms</td>
        <td>479.4 ms</td>
        <td>3.31×</td>
      </tr>
      <tr>
        <td><a href="https://etherscan.io/tx/0x0fe2542079644e107cbf13690eb9c2c65963ccb79089ff96bfaf8dced2331c92">High-fanout delegation</a></td>
        <td>2,324,323</td>
        <td>511</td>
        <td>10,545.8 ms</td>
        <td>656.4 ms</td>
        <td>16.07×</td>
      </tr>
    </tbody>
  </table>
</div>

<style>
  .slotscan-benchmark-table {
    margin: 1.5rem 0;
    overflow-x: auto;
    width: 100%;
  }

  .slotscan-benchmark-table table {
    background: transparent;
    border-collapse: collapse;
    color: var(--global-text-color);
    margin: 0;
    min-width: 760px;
    width: 100%;
  }

  .slotscan-benchmark-table th,
  .slotscan-benchmark-table td {
    border-bottom: 1px solid var(--global-divider-color);
    color: var(--global-text-color);
    padding: 0.55rem 0.65rem;
    text-align: right;
    white-space: nowrap;
  }

  .slotscan-benchmark-table th:first-child,
  .slotscan-benchmark-table td:first-child {
    text-align: left;
  }

  .slotscan-benchmark-table th {
    font-weight: 500;
  }

  .slotscan-benchmark-table td:not(:first-child) {
    font-variant-numeric: tabular-nums;
  }
</style>

<p>Native response sizes stayed within -3.2% to +2.5% of the legacy responses, so the speedup did not come from dropping evidence.</p>

<p>I expected something close to a 2× improvement from removing one of two replays. That is roughly what happened for the small ERC-20 transfer. The 2.32-million-step transaction improved by 16× because the old path also paid for a JavaScript callback on every opcode.</p>

<p>These numbers cover uncached evidence acquisition, not total SlotScan page latency. Historical tracing is still expensive. This change only removes the work SlotScan was doing twice.</p>

<p>You can try SlotScan at <a href="https://slotscan.info/">slotscan.info</a>.</p>

<h2 id="sources">Sources</h2>

<ul>
  <li><a href="https://github.com/wavey0x/slotscan/blob/ae3af0e/reth-slotscan/src/main.rs">Custom node entrypoint</a></li>
  <li><a href="https://github.com/wavey0x/slotscan/blob/ae3af0e/reth-slotscan/src/reth_adapter.rs">Native replay adapter</a></li>
  <li><a href="https://github.com/wavey0x/slotscan/blob/ae3af0e/reth-slotscan/src/inspector.rs">SlotScan REVM inspector</a></li>
</ul>]]></content><author><name></name></author><category term="Reth" /><category term="EVM" /><category term="Tracing" /><category term="Performance" /><summary type="html"><![CDATA[How a custom Reth RPC method reduced SlotScan tracing from two transaction replays to one, delivering up to a 16× speedup]]></summary></entry><entry><title type="html">Reverse Engineering EVM Storage</title><link href="https://wavey.info/posts/2025/reverse-engineering-evm-storage/" rel="alternate" type="text/html" title="Reverse Engineering EVM Storage" /><published>2025-12-10T00:00:00+00:00</published><updated>2025-12-10T00:00:00+00:00</updated><id>https://wavey.info/posts/2025/reverse-engineering-evm-storage</id><content type="html" xml:base="https://wavey.info/posts/2025/reverse-engineering-evm-storage/"><![CDATA[<p>Ethereum storage is deceptively simple: 32-byte slots holding 32-byte values. But mapping those slots back to meaningful variable names is where things get interesting. Particularly when the goal is to go beyond generating a simple layout to reverse engineering SSTOREs within transaction traces. This was my objective with <a href="https://slotscan.info">SlotScan.info</a>, and forms the background for the learnings that I will share here.</p>

<div style="display: flex; justify-content: center; gap: 16px; margin: 20px 0;">
    <div style="text-align: center; flex: 1; max-width: 48%;">
        <img src="/assets/img/slotscan_transaction.png" alt="SlotScan transaction view" data-zoomable="" style="width: 100%; aspect-ratio: 1/1.18; object-fit: cover; object-position: top; box-shadow: 0 4px 12px rgba(0, 0, 0, 0.15); border-radius: 4px; cursor: zoom-in;" />
        <em style="color: #808080; display: block; margin-top: 8px; font-size: 0.9em;">
            Example "Transaction" view from SlotScan.info
        </em>
    </div>
    <div style="text-align: center; flex: 1; max-width: 48%;">
        <img src="/assets/img/slotscan_layout.png" alt="SlotScan layout view" data-zoomable="" style="width: 100%; aspect-ratio: 1/1.18; object-fit: cover; object-position: top; box-shadow: 0 4px 12px rgba(0, 0, 0, 0.15); border-radius: 4px; cursor: zoom-in;" />
        <em style="color: #808080; display: block; margin-top: 8px; font-size: 0.9em;">
            Example "Layout" view from SlotScan.info
        </em>
    </div>
</div>

<p><strong>Contents</strong></p>

<ul>
  <li><a href="#background">Background</a></li>
  <li><a href="#tracing-transactions">Tracing Transactions</a></li>
  <li><a href="#resolving-mapping-keys">Resolving Mapping Keys</a></li>
  <li><a href="#decoding-storage-patterns">Decoding Storage Patterns</a></li>
  <li><a href="#proxy-detection">Proxy Detection</a></li>
  <li><a href="#storage-layouts-where-they-come-from">Storage Layouts</a></li>
  <li><a href="#edge-cases">Edge Cases</a></li>
  <li><a href="#key-takeaways">Key Takeaways</a></li>
  <li><a href="#what-i-built">What I Built</a></li>
</ul>

<hr />

<h2 id="background">Background</h2>

<p>The EVM has no concept of variable names, types, or struct members. That information exists only in compiler metadata. And since mapping slots are computed as <code class="language-plaintext highlighter-rouge">keccak256(key || baseSlot)</code>, you’re dealing with one-way functions: given a slot hash, the key that produced it is mathematically unrecoverable.</p>

<p>This post covers the techniques I learned and utilized to reverse engineer EVM storage: capturing hash preimages at runtime, decoding complex storage patterns, tracing transactions through DELEGATECALL chains, and detecting proxies.</p>

<hr />

<h2 id="tracing-transactions">Tracing Transactions</h2>

<p>To decode storage changes within a transaction, I need three things:</p>

<ol>
  <li><strong>What changed</strong>: Which slots, with before/after values</li>
  <li><strong>Mapping keys</strong>: The preimages that produced hashed slots</li>
  <li><strong>Write order</strong>: The sequence of changes (a slot might be written multiple times)</li>
</ol>

<p>No single RPC call provides all three. Here’s the approach I landed on.</p>

<h3 id="ground-truth-prestatetracer">Ground Truth: prestateTracer</h3>

<p><code class="language-plaintext highlighter-rouge">debug_traceTransaction</code> is an RPC method that replays a transaction and returns trace data. With the <code class="language-plaintext highlighter-rouge">prestateTracer</code> option (in diff mode), it returns a compact summary: exactly which storage slots changed, with their before and after values.</p>

<p>This is the ground truth for <em>what</em> changed. But it only shows the final state. If a slot is written three times, you only see the first and last values.</p>

<h3 id="execution-order-and-preimages-structlogs">Execution Order and Preimages: structLogs</h3>

<p>For the full picture, I need the execution trace. <code class="language-plaintext highlighter-rouge">structLogs</code> is a trace format that returns every EVM step: the opcode, stack, memory, and program counter. From this, I extract:</p>

<ul>
  <li><strong>SSTORE operations</strong> in execution order (capturing intermediate writes)</li>
  <li><strong>SHA3 operations</strong> with their memory inputs (the preimages for mapping slots)</li>
</ul>

<p>The SHA3 capture is critical. When the EVM computes <code class="language-plaintext highlighter-rouge">keccak256(key || slot)</code>, the preimage sits in memory. I grab it before the hash is computed, building a lookup table of hash → preimage.</p>

<h3 id="why-both">Why Both?</h3>

<p><code class="language-plaintext highlighter-rouge">prestateTracer</code> is authoritative but loses intermediate writes. <code class="language-plaintext highlighter-rouge">structLogs</code> captures everything but is massive (a complex transaction can produce a 2GB+ trace). I use prestateTracer to identify the slots of interest, then structLogs to get execution order and preimages.</p>

<p>For very large transactions, I fall back to a custom JS tracer that captures only SHA3 operations, keeping the response under 100KB instead of gigabytes.</p>

<h3 id="handling-delegatecall">Handling DELEGATECALL</h3>

<p>When contract A delegatecalls contract B, B’s code executes but writes to A’s storage. Getting the correct address attribution requires care.</p>

<p>structLogs doesn’t include an address field on each step, so I track a call stack manually. For DELEGATECALL, storage stays with the caller while only the code context changes. For regular CALL, both change. This lets me attribute each SSTORE to the correct contract.</p>

<p>The prestateTracer serves as validation. It knows exactly which slots changed in which contract, so I can cross-check my attribution.</p>

<hr />

<h2 id="resolving-mapping-keys">Resolving Mapping Keys</h2>

<p>The SHA3 preimage capture mentioned above deserves a deeper look. Mapping slots are computed as <code class="language-plaintext highlighter-rouge">keccak256(key || baseSlot)</code>. For <code class="language-plaintext highlighter-rouge">balances[0xABC...]</code> at slot 5, the EVM hashes the address concatenated with the slot number to produce the final storage location:</p>

<table>
  <thead>
    <tr>
      <th>Step</th>
      <th>Value</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Variable</td>
      <td><code class="language-plaintext highlighter-rouge">mapping(address =&gt; uint256) balances</code> at slot 5</td>
    </tr>
    <tr>
      <td>Key</td>
      <td><code class="language-plaintext highlighter-rouge">0xABC...123</code></td>
    </tr>
    <tr>
      <td>Hash Input</td>
      <td><code class="language-plaintext highlighter-rouge">0xABC...123</code> ‖ <code class="language-plaintext highlighter-rouge">0x05</code> (64 bytes total)</td>
    </tr>
    <tr>
      <td>Final Slot</td>
      <td><code class="language-plaintext highlighter-rouge">keccak256(input)</code> → <code class="language-plaintext highlighter-rouge">0x8a3f7b2c9d...</code></td>
    </tr>
  </tbody>
</table>

<h3 id="the-reverse-engineering-problem">The Reverse Engineering Problem</h3>

<p>When digging through trace data, this final slot is all we see. Since hashes are one-way functions, given just the slot hash, the key used to compute it is not recoverable through computation alone.</p>

<p>When I see an SSTORE to slot <code class="language-plaintext highlighter-rouge">0x8a3f7b2c9d...</code>, I need to figure out which mapping key produced that hash. I can’t reverse the hash itself.</p>

<h3 id="the-solution-capturing-preimages-at-runtime">The Solution: Capturing Preimages at Runtime</h3>

<p>The EVM has to compute these hashes at runtime. If I’m tracing execution, I can capture them as they happen.</p>

<p>When the EVM executes SHA3, I grab:</p>

<ul>
  <li>The memory region being hashed (the preimage, containing the key and base slot)</li>
  <li>The resulting hash (from the next step’s stack)</li>
</ul>

<p>This gives me a lookup table: hash → preimage. When I see an SSTORE to a hashed slot, I check the table. The key that produced that hash is right there.</p>

<h3 id="the-compile-time-optimization-problem">The Compile-Time Optimization Problem</h3>

<p>This approach works well, until it doesn’t. I hit a case with no SHA3 in the trace. The slot was clearly a mapping (huge hash value), but no preimage to be found.</p>

<p>The culprit: compile-time optimization.</p>

<pre><code class="language-solidity">address constant REWARD_TOKEN = 0xD533a949740bb3306d119CC777fa900bA034cd52;
mapping(address =&gt; uint256) rewards;

function setReward() external {
    rewards[REWARD_TOKEN] = 100;  // No runtime SHA3!
}
</code></pre>

<p>There are some cases when No preceding SHA3 opcode can be found! This is a result of optimization-time logic where the compiler sees a constant used as a mapping key, it pre-computes the hash and embeds it in bytecode. At runtime: <code class="language-plaintext highlighter-rouge">CODECOPY → SSTORE</code>. This is a nice gas savings trick by the compiler which forces us to handle a new edge case.</p>

<p>My solution: parse the source code for constant addresses and pre-compute their mapping hashes myself. If runtime capture fails, the source-derived lookup fills the gap.</p>

<hr />

<h2 id="decoding-storage-patterns">Decoding Storage Patterns</h2>

<p>Solidity hides a surprising amount of complexity inside 32-byte slots. Here are the patterns you must handle to correctly decode storage.</p>

<h3 id="packed-variables">Packed Variables</h3>

<p>Variables smaller than 32 bytes can share a slot. An <code class="language-plaintext highlighter-rouge">address</code> (20 bytes) + <code class="language-plaintext highlighter-rouge">bool</code> (1 byte) + <code class="language-plaintext highlighter-rouge">uint32</code> (4 bytes) all fit in one slot.</p>

<p>This means a single storage write can change multiple variables. You can’t just decode “the value that changed.” You need to decode the entire slot into its component parts, using the layout’s offset and size information for each variable.</p>

<h3 id="dynamic-strings">Dynamic Strings</h3>

<p>Solidity’s string encoding has a quirk that tripped me up.</p>

<p><strong>Short strings (&lt; 32 bytes)</strong> store content and length in the same slot. The lowest byte holds <code class="language-plaintext highlighter-rouge">length * 2</code>.</p>

<p><strong>Long strings (&gt;= 32 bytes)</strong> work completely differently. The base slot stores <code class="language-plaintext highlighter-rouge">length * 2 + 1</code>. The actual content lives at <code class="language-plaintext highlighter-rouge">keccak256(baseSlot)</code>, potentially spanning multiple consecutive slots.</p>

<p>You have to detect which encoding is in use before decoding. I check the lowest bit of the base slot value: if it’s set, it’s a long string.</p>

<h3 id="nested-mappings-and-arrays-of-structs">Nested Mappings and Arrays of Structs</h3>

<p>These compound types are where slot resolution gets genuinely complex.</p>

<p>For <code class="language-plaintext highlighter-rouge">mapping(address =&gt; mapping(uint256 =&gt; Data))</code>:</p>

<ul>
  <li>Outer lookup: <code class="language-plaintext highlighter-rouge">keccak256(outerKey || baseSlot)</code> → intermediate slot</li>
  <li>Inner lookup: <code class="language-plaintext highlighter-rouge">keccak256(innerKey || intermediateSlot)</code> → final slot</li>
</ul>

<p>You need to chain preimage lookups to recover both keys.</p>

<p>Dynamic arrays of structs add another dimension:</p>

<ul>
  <li>Length stored at base slot</li>
  <li>Data starts at <code class="language-plaintext highlighter-rouge">keccak256(baseSlot)</code></li>
  <li>Element N lives at <code class="language-plaintext highlighter-rouge">dataStart + N * slotsPerElement</code></li>
  <li>Individual struct fields offset within each element</li>
</ul>

<p>When I see a write to slot <code class="language-plaintext highlighter-rouge">0x8f3a...</code>, I might need to trace back through several layers: “this is offset 2 within element 7 of the <code class="language-plaintext highlighter-rouge">proposals</code> array, which means it’s the <code class="language-plaintext highlighter-rouge">votesFor</code> field.”</p>

<hr />

<h2 id="proxy-detection">Proxy Detection</h2>

<p>Proxies break naïve storage decoders because the layout belongs to the implementation, not the proxy. The contract you’re looking at isn’t the contract with the logic.</p>

<h3 id="three-proxy-patterns">Three Proxy Patterns</h3>

<p><strong>EIP-1167 minimal proxies</strong> embed the implementation address directly in bytecode. The pattern <code class="language-plaintext highlighter-rouge">363d3d373d3d3d363d73</code> followed by 20 bytes is the address. No storage read needed.</p>

<p><strong>EIP-1967 proxies</strong> store the implementation at a standard slot: <code class="language-plaintext highlighter-rouge">0x360894a13ba1a3210667c828492db98dca3e2076cc3735a920a3ca505d382bbc</code>. Read that slot, get the implementation.</p>

<p><strong>EIP-1822 (older UUPS)</strong> uses a different slot: <code class="language-plaintext highlighter-rouge">0xc5f16f0fcc639fa48a6947836d9850f504798523bf8c9a3a87d5876cf622bcf7</code>.</p>

<h3 id="the-bytecode-cache-trap">The Bytecode Cache Trap</h3>

<p>I initially tried caching storage layouts by bytecode hash. Same bytecode, same layout, right?</p>

<p>Wrong. For proxies.</p>

<p>EIP-1167 minimal proxies work because the implementation address is in the bytecode. Same bytecode means same implementation means same layout.</p>

<p>But EIP-1967 and EIP-1822 proxies all share identical bytecode. The implementation address is in storage, not bytecode. Different instances point to different implementations with different layouts.</p>

<p>I now only use bytecode caching for non-proxies and EIP-1167 minimal proxies.</p>

<hr />

<h2 id="storage-layouts-where-they-come-from">Storage Layouts: Where They Come From</h2>

<p>Storage layouts don’t exist on-chain. They’re a compile-time artifact.</p>

<p>The EVM only knows about 32-byte slots and 32-byte values. Variable names, types, and struct members exist only in compiler metadata that lives off-chain.</p>

<p>This means you need verified source code to decode storage meaningfully. But not all “verified” contracts are equal.</p>

<p><strong>Sourcify</strong> stores full compiler metadata, including storage layouts when available. Usually Solidity only.</p>

<p><strong>Etherscan</strong> stores source code but typically not the layout. Many contracts require local recompilation just to understand their storage.</p>

<p>And recompilation is fragile. Use the wrong compiler version and you get different slot assignments. Miss an optimizer setting and the packing changes. Target the wrong contract in a multi-file project and you get someone else’s layout entirely.</p>

<p>Exact reproduction of compiler settings isn’t optional. It’s the difference between correct decoding and complete nonsense.</p>

<hr />

<h2 id="edge-cases">Edge Cases</h2>

<h3 id="constructor-storage">Constructor Storage</h3>

<p>Contract creation transactions need special handling. The EVM knows the new contract’s address during constructor execution, but the trace data doesn’t include it. Every SSTORE during the constructor is missing its target address, so we need to reverse engineer the attribution.</p>

<p>The sequence:</p>

<ul>
  <li>CREATE/CREATE2 opcode executes</li>
  <li>Constructor (initcode) runs, SSTOREs occur</li>
  <li>Trace data omits the contract address on these operations</li>
  <li>Created address appears on the stack only when the frame returns</li>
</ul>

<p>I handle this by scanning for CREATE/CREATE2 opcodes, then tracking when the call depth returns to the parent level. At that point, the created address appears on the stack. I map which step ranges belong to each constructor so I can retroactively attribute those SSTOREs to the correct contract.</p>

<h3 id="vyper-differences">Vyper Differences</h3>

<p>Vyper uses different type names (<code class="language-plaintext highlighter-rouge">HashMap</code> instead of <code class="language-plaintext highlighter-rouge">mapping</code>), different packing rules, and importantly, doesn’t include storage layouts in standard compilation output. I had to use an experimental compiler flag to get layout information.</p>

<p>String encoding differs too. Vyper stores length as raw byte count, not Solidity’s <code class="language-plaintext highlighter-rouge">length * 2 + 1</code> encoding. The decoder needs to know which compiler produced the contract.</p>

<hr />

<h2 id="key-takeaways">Key Takeaways</h2>

<p><strong>The compiler is the source of truth.</strong> Without exact compiler metadata, storage decoding is guesswork.</p>

<p><strong>Multiple trace passes are necessary.</strong> prestateTracer gives you ground truth for what changed; structLogs gives you execution order and SHA3 preimages. Neither alone is sufficient.</p>

<p><strong>Optimizations create blind spots.</strong> Compile-time hash precomputation breaks naive trace analysis. If your tool relies on runtime behavior, watch out for compile-time shortcuts.</p>

<p><strong>Storage is more complex than it looks.</strong> Packed variables, dynamic encoding, nested structures. The simple cases are simple, but the edge cases compound.</p>

<hr />

<h2 id="what-i-built">What I Built</h2>

<p>I built <a href="https://slotscan.info">SlotScan</a>, a tool that applies these techniques to make EVM storage readable.</p>

<p>Paste any verified contract address to see its full storage layout with decoded values. Enter a transaction hash to see exactly what storage changed, with before/after values and variable names.</p>

<hr />

<h2 id="closing">Closing</h2>

<p>The techniques here (SHA3 preimage capture, call stack tracking, multi-pass tracing) apply to any EVM storage analysis tool. The patterns are consistent across Solidity and Vyper, mainnet and L2s.</p>

<p>If you’re building something similar: capture SHA3 preimages, parse source for constants when runtime fails, track the call stack for DELEGATECALL, and don’t cache layouts for upgradeable proxies.</p>

<p>SlotScan is available at <a href="https://slotscan.info">slotscan.info</a>. The unexpected complexity of EVM storage turned a weekend project into something much deeper. Hopefully these notes help others navigating the same terrain.</p>]]></content><author><name></name></author><category term="EVM" /><category term="Storage" /><category term="Compilers" /><category term="Solidity" /><summary type="html"><![CDATA[Techniques for mapping EVM storage slots back to variable names and decoding transaction traces]]></summary></entry><entry><title type="html">Noop, Not my Safe</title><link href="https://wavey.info/posts/2025/noop-not-my-safe/" rel="alternate" type="text/html" title="Noop, Not my Safe" /><published>2025-01-29T00:00:00+00:00</published><updated>2025-01-29T00:00:00+00:00</updated><id>https://wavey.info/posts/2025/noop-not-my-safe</id><content type="html" xml:base="https://wavey.info/posts/2025/noop-not-my-safe/"><![CDATA[<p>If you’re like me, you get a kick out of deploying Safe multisigs deterministically to <a href="https://github.com/nlordell/deadbeef">vanity addresses</a> and across multiple networks.</p>

<p>But how safe is this practice?</p>

<p>We all remember when <a href="https://banteg.mirror.xyz/iZAsBNL3j_5NIAY2Erav1r7Q4ecc7SC76AfMjyScs34">Wintermute lost 20M OP</a> because an attacker replicated their Safe address on Optimism with malicious owners. Lesson learned: nobody deploys multichain Safes from the factory with the <code class="language-plaintext highlighter-rouge">CREATE</code> opcode anymore— <code class="language-plaintext highlighter-rouge">CREATE2</code> is the way to go. And since my deployments depend on an initializer containing my owner addresses, nobody can steal my Safe on new networks, right?</p>

<p><strong>Wrong.</strong></p>

<p>Today, I stumbled upon a way to “steal” a Safe address even when the owner deployed with <code class="language-plaintext highlighter-rouge">CREATE2</code> and a proper initializer containing owners. Thus, Safe users should <strong>not</strong> rely on their ability to control a particular Safe address on new networks.</p>

<p><em>(Note: This was responsibly disclosed to the Safe team, and they confirmed it was a known issue, fixed in v1.4.1.)</em></p>

<hr />

<h3 id="how-normal-safe-deployments-work"><strong>How Normal Safe Deployments Work</strong></h3>

<p>Safe follows the <strong>EIP-1167 minimal proxy pattern</strong>. The proxy deployments come from a factory that requires the following data:</p>

<ul>
  <li>A list of owners</li>
  <li>A threshold</li>
  <li>Other optional data</li>
</ul>

<p>This data is hashed into the <code class="language-plaintext highlighter-rouge">CREATE2</code> salt, ensuring deterministic deployment across chains. Normally, deployment looks like this:</p>

<p align="center">
    <img src="/assets/img/safe_deploy1.png" alt="Alt text" width="650" />
    <em style="color: #808080;"><br />
        Safe deployment happy path.
    </em>
</p>
<p>But what if the Safe <strong>implementation contract</strong> doesn’t exist on the network yet?</p>

<hr />

<h3 id="how-an-attacker-steals-your-safe"><strong>How an Attacker Steals Your Safe</strong></h3>

<p>Let’s say you deploy on a new chain, <strong>Sonic</strong>, which at the time of observation doesn’t yet have a Safe implementation contract. Here’s what happens:</p>

<ol>
  <li>The proxy factory deploys your Safe proxy.</li>
  <li>The proxy calls <code class="language-plaintext highlighter-rouge">setup()</code>, but since the implementation contract <strong>doesn’t exist yet</strong>, the delegate call does not revert, but nothing is written to storage.</li>
  <li>The proxy is now deployed at its expected deterministic address, but it’s <strong>uninitialized</strong>.</li>
  <li>An attacker <strong>deploys the implementation contract</strong>, then calls <code class="language-plaintext highlighter-rouge">setup()</code> themselves—setting their own owners and threshold.</li>
</ol>

<p align="center">
    <img src="/assets/img/safe_deploy2.png" alt="Alt text" width="650" />
    <em style="color: #808080;"><br />
        Safe address theft.
    </em>
</p>

<p><strong>Boom.</strong> They now control your Safe address before you even get the chance to initialize it.</p>

<hr />

<h3 id="how-to-prevent-this"><strong>How to Prevent This</strong></h3>

<p>Good news—this attack is mitigated in several ways:</p>

<ul>
  <li><strong>Safe v1.4.1+ fixes this issue</strong> by <a href="https://github.com/safe-global/safe-smart-account/blob/v1.4.1/contracts/proxies/SafeProxyFactory.sol#L27">explicitly checking</a> for code at the implementation address before deployment.</li>
  <li><strong>OP chains are safe</strong> because they <a href="https://docs.optimism.io/builders/chain-operators/features/preinstalls">preinstall Safe implementations</a>, blocking attackers from front-running deployments.</li>
</ul>]]></content><author><name></name></author><category term="Security" /><category term="Gnosis" /><category term="Safe" /><category term="Disclosures" /><summary type="html"><![CDATA[Stealing a Safe Address — Even with CREATE2]]></summary></entry><entry><title type="html">Decommissioning Prisma Finance - A Turbulent but Ultimately Soft-Landing</title><link href="https://wavey.info/posts/2025/decomissioning-prisma-finance/" rel="alternate" type="text/html" title="Decommissioning Prisma Finance - A Turbulent but Ultimately Soft-Landing" /><published>2025-01-17T00:00:00+00:00</published><updated>2025-01-17T00:00:00+00:00</updated><id>https://wavey.info/posts/2025/decomissioning-prisma-finance</id><content type="html" xml:base="https://wavey.info/posts/2025/decomissioning-prisma-finance/"><![CDATA[<p>As the Resupply team prepares for an exciting launch, one pesky task has occupied the focus of the team: shutting down Prisma Finance, a <a href="https://rekt.news/prismafi-rekt/">hacked</a> Liquity fork that has since become a ghost ship. Once supported by Yearn and Convex liquid locker products, Prisma Finance was abandoned by its team, leaving it in a fragile state. Resupply was in fact originally conceived as a replacement for Prisma, which governance approved with <a href="https://gov.prismafinance.com/t/pip-46-shutdown-prisma-finance-introduce-resupply/232">PIP-46</a>. But unwinding the system turned out to be a trickier task than expected.</p>

<p>In this post I’ll highlight how we:</p>

<ol>
  <li>Discovered and worked around a critical bug found in Trove Manager accounting, allowing users to regain access to their collateral and avoided introducing bad debt into the system.</li>
  <li>Discovered a bug in the ULTRA stability pool that allowed an attacker to extract approximately 13.92 ETH that did not belong to them.</li>
  <li>Created a creative solution to combat upward depegs by allowing users to close their loans using a separate stablecoin.</li>
</ol>

<h3 id="exposing-a-trove-manager-accounting-bug">Exposing a Trove Manager Accounting Bug</h3>

<p>As Prisma’s TVL dwindled (thanks to the PSMs which I will describe in a later section), a subtle accounting issue was brought to light. I noticed a sizeable mismatch between the sum of user debt and what the trove thinks its total debt is.</p>

<p align="center">
    <img src="/assets/img/active_debt_mismatch.png" alt="Alt text" width="550" />
    <em style="color: #808080;"><br />
        Sum of borrows exceeds total active debt.
    </em>
</p>

<p>Clearly there is some sort of accounting bug at play here, and the search for the culprit was on. After meticulous code inspection and help from Tenderly, I was able to trace the issue to the <code class="language-plaintext highlighter-rouge">openTrove()</code> function.</p>

<p align="center">
    <img src="/assets/img/open_trove.png" alt="Alt text" width="750" />
    <em style="color: #808080;">
        The bug in the Trove Manager's `openTrove()` function.
    </em>
</p>

<p>As you can see, the code incorrectly writes a state update to <code class="language-plaintext highlighter-rouge">totalActiveDebt</code> using a stale value that doesn’t include the interest accrued since the prior checkpoint. During normal operations, this bug was largely transparent because <code class="language-plaintext highlighter-rouge">totalActiveDebt</code> was high enough that there was no underflow risk when subtracting a repayment amount. But if the discrepancy is big enough, and as TVL drops, even a modest sized repayments can trigger it. In fact, the bug would render repayments, redemptions, and even liquidations impossible because when the final borrower repays, say $1,000 the trove manager’s attempt to subtract that amount from <code class="language-plaintext highlighter-rouge">totalActiveDebt</code> (some number less than $1,000) will underflow. These reverting transactions effectively trap users’ collateral.</p>

<p>Come to find out later, a patch was introduced by Prisma in one of the later versions of their Trove Manager code. So this bug is only present in older versions of trove managers.</p>

<p>Unfortunately, there is no way to manipulate this value back in line with where it should be. Additionally, at this point, governance had already placed all Trove Managers in “sunsetting” mode, preventing any obvious methods of intervention. However, we discovered a loophole: the ability for governance to upgrade the oracle was still available.</p>

<p>By setting a custom oracle that priced collateral at <code class="language-plaintext highlighter-rouge">type(uint).max</code> (effectively infinite), users could withdraw their collateral, minus a small dust amount. Essentially, given the manipulated price, users need just a few wei of collateral would be used to collateralize the outstanding debt. This maneuver created about $35,000 in bad debt as stuck users were relieved of their remaining debt.</p>

<p>In a nice piece of news, we found more than enough money to pay that down via unclaimed crvUSD earned by Prisma’s veCRV voter. Transferring this crvUSD to the PSM now means that all mkUSD is backed 1:1, and the protocol is fully solvency.</p>

<h3 id="identifying-the-stability-pool-exploit">Identifying the Stability Pool Exploit</h3>

<p align="center">
    <img src="/assets/img/prisma-sp-claim-issue.png" alt="Alt text" width="600" />
    <em style="color: #808080;">
    </em>
</p>

<p>In late December, a Discord user pastelfork flagged a new issue during the shutdown: collateral gains from the ULTRA stability pool depositors couldn’t be claimed. After some review, we discovered a bug introduced to ULTRA’s stability pool. An errant line in <code class="language-plaintext highlighter-rouge">claimCollateralGains()</code> resets a value that should have been zeroed out, incorrectly allowing repeated claims of the same amount until the pool is fully drained.</p>

<p align="center">
    <img src="/assets/img/prisma-claim-collateral-bug.png" alt="Alt text" width="750" />
    <em style="color: #808080;">
    </em>
</p>
<p>…</p>
<p align="center">
    <img src="/assets/img/prisma-repeat-claims.png" alt="Alt text" width="750" />
    <em style="color: #808080;">
    </em>
</p>

<p>A user was able to extract approximately 13.92 ETH that did not belong to him. In all likelihood, he didn’t set out to find this bug, but rather stumbled across it.</p>

<p>Looking at Etherscan, you can see just a sample of the many repeated claims he made to drain the pool.</p>

<p align="center">
    <img src="/assets/img/ultra_sp.png" alt="Alt text" width="750" />
    <em style="color: #808080;">
    </em>
</p>

<p>To prevent further abuse, a call to users to withdraw from the ULTRA stability pool was issued.</p>

<h3 id="stablecoin-hoarders-trigger-an-upward-depeg">Stablecoin Hoarders Trigger an Upward Depeg</h3>

<p>Shutting down Prisma began with reducing the debt ceiling to zero, preventing new mkUSD and ULTRA loans. However, this move triggered a destructive market dynamic: mkUSD and ULTRA, Prisma’s stablecoins, deviated from their $1 peg, climbing as high as $1.45. This happened because these stablecoins are required for borrowers to close their loans. Opportunistic traders saw this as a chance to hoard these tokens, driving up the prices, allowing an eventual sale at a profit.</p>

<p>Our solution was announced in <a href="https://x.com/wavey0x/status/1870629576294158463">PIP-47: Operation Rainbow Pegger</a> which introduced the concept of a custom Peg Stability Module (PSM) disguised as a regular Trove Manager, giving it permissions to mint new stablecoins. By design, the PSMs offer two-way purchasing of stables via key functions:</p>

<ul>
  <li><code class="language-plaintext highlighter-rouge">repayDebt</code>: Accepts crvUSD, mints corresponding amount of mkUSD/ULTRA at a 1:1 rate, and closes the user’s loan atomically.</li>
  <li><code class="language-plaintext highlighter-rouge">sellDebtToken</code>: Burns mkUSD/ULTRA and returns crvUSD at a 1:1 rate.</li>
</ul>

<p>This innovation got the protocol out of the jam by providing users with liquidity they needed, and eliminated the price gouging. The PSMs will exist perpetually to facilitate repayments and serve as a facility to redeem mkUSD and ULTRA.</p>

<p>Custom UI development work was done to provide users with an app to interact with the PSM.</p>

<ul>
  <li><a href="https://prisma-psm.yearn.space/">https://prisma-psm.yearn.space/</a></li>
</ul>

<p>As of writing the PSM balances are as follows:</p>

<ul>
  <li><a href="https://etherscan.io/address/0x9d7634b2B99c2684611c0Ac3150bAF6AEEa4Ed77">mkUSD PSM</a>: <code class="language-plaintext highlighter-rouge">1,115,770</code> crvUSD</li>
  <li><a href="https://etherscan.io/address/0xAe21Fe5B61998b143f721CE343aa650a6d5EadCe">ULTRA PSM</a>: <code class="language-plaintext highlighter-rouge">180,835</code> crvUSD</li>
</ul>

<h3 id="wrapping-up">Wrapping Up</h3>

<p>Shutting down Prisma is still in progress, with less than $80k debt remaining, but it was far from straightforward. Between economic exploits and protocol bugs, the process required custom tools, clever engineering, and delicate maneuvering to ensure users exited with minimal harm. The experience highlights the complexity of unwinding DeFi protocols and the importance of testing.</p>]]></content><author><name></name></author><category term="Resupply" /><category term="Prisma" /><summary type="html"><![CDATA[Working around a Trove Manager accounting bug, closing a stability pool exploit, and handling an upward depeg while winding down Prisma Finance.]]></summary></entry><entry><title type="html">Using Convex Just Got Cheaper for Everyone</title><link href="https://wavey.info/posts/2023/convex-just-got-cheaper/" rel="alternate" type="text/html" title="Using Convex Just Got Cheaper for Everyone" /><published>2023-12-28T00:00:00+00:00</published><updated>2023-12-28T00:00:00+00:00</updated><id>https://wavey.info/posts/2023/convex-just-got-cheaper</id><content type="html" xml:base="https://wavey.info/posts/2023/convex-just-got-cheaper/"><![CDATA[<h3 id="overview">Overview</h3>
<p>If you’re like me, you’ve noticed gas costs becoming inordinately expensive when interacting with Convex. So I decided to investigate what was going on.</p>

<p>What I found was an extremely inefficient code path which actually had nothing to do with Convex contracts at all - despite it having an outsized impact on their users specifically. In fact, the gas guzzling code turned out to be related to a niche feature in Curve’s LP gauges.</p>

<p>To visualize it, I analyzed data from every Convex deposit over the last ~14 months, and the gas consumption of each.</p>

<p align="center">
    <img src="/assets/img/gas_chart0.png" alt="Alt text" width="800" />
    <em style="color: #808080;">
        Gas cost of each deposit transaction sent to Convex in the last 14 months.
    </em>
</p>

<p>As you can see, the average cost of a deposit had been growing steadily for more than a year from late 2022, increasing by 123% to nearly 1.25M gas per transaction by December this year.</p>

<p>This post will provide a technical overview of what caused this ramping of gas costs, why it affected Convex more than others, and what the fix was. We’ll finish with some suggestions for how Curve might mitigate this in the future.</p>

<h3 id="background">Background</h3>
<p>Every Convex deposit routes a user’s LPs into their respective Curve gauge. The process is routine. However, as execution is handed over to the Curve gauge via <code class="language-plaintext highlighter-rouge">deposit</code>, <code class="language-plaintext highlighter-rouge">withdraw</code> or <code class="language-plaintext highlighter-rouge">claim</code>, it is possible for a significant jump in gas costs when certain conditions are in place.</p>

<p>Curve invented a concept of “boost” which allows people who lock more CRV to earn higher emissions on the same sized LP position. A couple years ago, they took this concept further by introducing a clever mechanism called “boost delegation”, which allows users who may have unutilized boost to delegate it to others. This opened the possibility for boost marketplaces like <a href="https://app.warden.vote/boosts/">Warden</a> to emerge.</p>

<p>This boost delegation feature was built into new gauges and requires a few additional operations during each user interaction. In the context of Convex’s gas issue we’re discussing today, two key contracts played a role:</p>
<ol>
  <li><a href="https://etherscan.io/address/0xd0921691c7debc698e6e372c6f74dc01fc9d3778#code"><strong>Boost Delegation V2</strong></a>. This contract tracks all boost delegations. Under the hood, when a delegation is made, this contract saves the data and performs the math to apply a modified veCRV balance to the users involved. The <code class="language-plaintext highlighter-rouge">adjusted_balance_of(_user)</code> function it exposes returns this modified amount.</li>
  <li><a href="https://etherscan.io/address/0xe7330c73b373a50B95D3Beb25C13D2BEDB6FcE7E"><strong>veBoost Patch</strong></a>. As the name suggests, this contract is a patch. It was introduced at block 17,964,967 to fix an unrelated issue (<a href="https://hackmd.io/WKwRKhEfTiCeWY8qFxiMJA">outlined here</a>), but as we’ll see, it also contributed to the problem.</li>
</ol>

<h3 id="the-problem">The Problem</h3>
<p>When a gauge checks for a user’s adjusted balance, the call gets routed through multiple contracts. Importantly, the call reaches <em>Boost Delegation V2</em> and arrives at a function called <code class="language-plaintext highlighter-rouge">_checkpoint_read()</code> on each check for adjusted balance. By glancing at this code, you can probably guess where this is going…</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nd">@view</span>
<span class="nd">@internal</span>
<span class="k">def</span> <span class="nf">_checkpoint_read</span><span class="p">(</span><span class="n">_user</span><span class="p">:</span> <span class="n">address</span><span class="p">,</span> <span class="n">_delegated</span><span class="p">:</span> <span class="nb">bool</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">Point</span><span class="p">:</span>
    <span class="bp">...</span>

    <span class="n">ts</span><span class="p">:</span> <span class="n">uint256</span> <span class="o">=</span> <span class="p">(</span><span class="n">point</span><span class="p">.</span><span class="n">ts</span> <span class="o">/</span> <span class="n">WEEK</span><span class="p">)</span> <span class="o">*</span> <span class="n">WEEK</span>
    <span class="k">for</span> <span class="n">_</span> <span class="ow">in</span> <span class="nf">range</span><span class="p">(</span><span class="mi">255</span><span class="p">):</span>
        <span class="n">ts</span> <span class="o">+=</span> <span class="n">WEEK</span>

        <span class="n">dslope</span><span class="p">:</span> <span class="n">uint256</span> <span class="o">=</span> <span class="mi">0</span>
        <span class="k">if</span> <span class="n">block</span><span class="p">.</span><span class="n">timestamp</span> <span class="o">&lt;</span> <span class="n">ts</span><span class="p">:</span>
            <span class="n">ts</span> <span class="o">=</span> <span class="n">block</span><span class="p">.</span><span class="n">timestamp</span>
        <span class="k">else</span><span class="p">:</span>
            <span class="k">if</span> <span class="n">_delegated</span><span class="p">:</span>
                <span class="n">dslope</span> <span class="o">=</span> <span class="n">self</span><span class="p">.</span><span class="n">delegated_slope_changes</span><span class="p">[</span><span class="n">_user</span><span class="p">][</span><span class="n">ts</span><span class="p">]</span>
            <span class="k">else</span><span class="p">:</span>
                <span class="n">dslope</span> <span class="o">=</span> <span class="n">self</span><span class="p">.</span><span class="n">received_slope_changes</span><span class="p">[</span><span class="n">_user</span><span class="p">][</span><span class="n">ts</span><span class="p">]</span>

        <span class="n">point</span><span class="p">.</span><span class="n">bias</span> <span class="o">-=</span> <span class="n">point</span><span class="p">.</span><span class="n">slope</span> <span class="o">*</span> <span class="p">(</span><span class="n">ts</span> <span class="o">-</span> <span class="n">point</span><span class="p">.</span><span class="n">ts</span><span class="p">)</span>
        <span class="n">point</span><span class="p">.</span><span class="n">slope</span> <span class="o">-=</span> <span class="n">dslope</span>
        <span class="n">point</span><span class="p">.</span><span class="n">ts</span> <span class="o">=</span> <span class="n">ts</span>
    <span class="bp">...</span>

    <span class="k">return</span> <span class="n">point</span>
</code></pre></div></div>

<p>The <code class="language-plaintext highlighter-rouge">for</code> loop must be entered twice on each gauge interaction as it checks for each user’s boost sent, and again for his boost received. Certainly this threatens to be expensive, but a close look at the surrounding code shows that some unique conditions are required in order for a user to find themselves stuck in this loop.</p>

<p>Specifically, a user must have an update (or <code class="language-plaintext highlighter-rouge">Point</code>) written in the past via them sending or receiving boost from another user. The problem grows more pronounced as the number of weeks since the last <code class="language-plaintext highlighter-rouge">Point</code> write increases, forcing the view function to compute the data for each missing week. Each week that passes adds an extra two SLOADs (<code class="language-plaintext highlighter-rouge">2 * 2,100 gas</code>) plus a few math operations.</p>

<p>At the time of my research, it had been 71 weeks (!!!) since the last update had ocurred for both Yearn and Convex’s veCRV position. And sure enough, as we can see in the chart above, the average cost to transact is trending upwards as new weeks add iterations to the loop.</p>

<p>But the problem became even more acute for Convex users in late August 2023, as you can see in the chart above where the smooth trend upwards in gas prices suddenly becomes disjointed, leaping upward. This is when the patch contract mentioned above was implemented. Taking a quick look at the code, you’ll notice why immediately.</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nd">@view</span>
<span class="nd">@external</span>
<span class="k">def</span> <span class="nf">adjusted_balance_of</span><span class="p">(</span><span class="n">_user</span><span class="p">:</span> <span class="n">address</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">uint256</span><span class="p">:</span>
    <span class="k">if</span> <span class="n">_user</span> <span class="o">==</span> <span class="n">CONVEX_WALLET</span><span class="p">:</span>
        <span class="k">return</span> <span class="nc">ERC20</span><span class="p">(</span><span class="n">BOOST_V2</span><span class="p">).</span><span class="nf">balanceOf</span><span class="p">(</span><span class="n">CONVEX_WALLET</span><span class="p">)</span> <span class="o">-</span> <span class="nc">BoostV2</span><span class="p">(</span><span class="n">BOOST_V2</span><span class="p">).</span><span class="nf">delegated_balance</span><span class="p">(</span><span class="n">YEARN_WALLET</span><span class="p">)</span>
    
    <span class="k">if</span> <span class="n">_user</span> <span class="o">==</span> <span class="n">YEARN_WALLET</span><span class="p">:</span>
        <span class="k">return</span> <span class="nc">ERC20</span><span class="p">(</span><span class="n">VE</span><span class="p">).</span><span class="nf">balanceOf</span><span class="p">(</span><span class="n">YEARN_WALLET</span><span class="p">)</span>

    <span class="k">return</span> <span class="nc">ERC20</span><span class="p">(</span><span class="n">BOOST_V2</span><span class="p">).</span><span class="nf">balanceOf</span><span class="p">(</span><span class="n">_user</span><span class="p">)</span>
</code></pre></div></div>

<p>As you can see, it forces each query for Convex’s <code class="language-plaintext highlighter-rouge">adjusted_balance()</code> to also involve an additional query to Yearn’s boost, effectively multiplying the amount of work needing to be done as Yearn’s position <em>also</em> must traverse a large number of weeks.</p>

<p>In this example transaction, <a href="https://dashboard.tenderly.co/tx/mainnet/0x5b93c34cd96150443682aa0cc8a652711155022b4918f5825c92fde65e806955/gas-usage">we can profile its gas usage on Tenderly</a>. It’s almost difficult to see, but you can notice over 100 SLOADs along the bottom row which should be a huge red flag.</p>

<p><img src="/assets/img/tenderly_gas.png" alt="Alt text" width="800" /></p>

<h3 id="the-fix">The Fix</h3>
<p>Luckily I found that within the Boost Delegation V2 contract, Curve exposes a handy <code class="language-plaintext highlighter-rouge">checkpoint_user</code> function which is totally permissionless to call. It does exactly what we need in order to fix the problem: loops through all the unfilled weeks, and populates them with data.</p>

<p>And just like that, two ~$85 transactions (one for Convex, and one for Yearn) fixed our problem! Several weeks later, we can see the impact it’s had…</p>

<p align="center">
    <img src="/assets/img/gas_chart_checkpoint_fix.png" alt="Alt text" width="800" />
    <em style="color: #808080;">
        Gas cost of each deposit transaction sent to Convex, highlighting patch and checkpoint blocks.
    </em>
</p>

<p>Notice:</p>
<ul>
  <li>The less frequent dots tha appear along the bottom, even before the checkpoint, represent deposits to older gauges which do not support delegation. These transactions bypass the issue entirely.</li>
  <li>We can now get gas on each transaction significantly below 1M again to an average of 0.6M.</li>
</ul>

<h3 id="proposed-long-term-solutions">Proposed Long Term Solutions</h3>
<p>Because the problem is multi-faceted, there are several things Curve might consider to permanently address this.</p>
<ol>
  <li>First and foremost, the patch contract should be deprecated in favor of a new Boost Delegation V3 contract which properly addresses the issue it was attempting to solve in the first place. This can be done by simply redeploying the contract.</li>
  <li>In all future gauge codebases, create a “write” counterpart for the <code class="language-plaintext highlighter-rouge">adjusted_balance_of()</code> function, which should populate any unfilled weeks on each user interaction. This will enforce that no two users in the same week will have to traverse many iterations of the <code class="language-plaintext highlighter-rouge">for</code> loop.</li>
</ol>]]></content><author><name></name></author><category term="Curve" /><category term="Convex" /><category term="data" /><summary type="html"><![CDATA[How I was able to permissionlessly resolve a code inefficiency and drop costs by over 50%.]]></summary></entry></feed>