How to put custom messages into Bitcoin blockchain – OP_RETURN

I have spent last few days looking very closely at OP_RETURN features. This blogpost summarizes my findings and thoughts.

Bitcoin transactions – recap

Every bitcoin transaction creates outputs (called transaction outputs, sometimes called txout) from one or more transaction inputs. All except one type of these transaction output types will create spendable outputs (called unspent transaction outputs – UTXO), the type that behaves differently is OP_RETURN
The UTXO are tracked and stored by every full node (btcd or bitcoin-core/satoshi client). You can say, that transactions consume inputs and create new outputs. One transaction can contain one or more inputs and one or more outputs.
Transaction outputs have 2 parts:
  • value – how much money you are sending
  • locking script – this can be understand as a set of conditions that allow you to “spend” this output. For example this can be an ownership of a specific private key.

Historical context

There were multiple attempts to store custom messages in blockchain, one very popular was to just to send coins to “meaningful address” (also called “vanity” addresses), for example to 1TodayIsFridayatpoGgzqxD7r2BMLr9dG – these addresses were incorrect, meaning that no one was able to spend coins there.
It was also possible to create CHECKMULTISIG transactions which have similar implications.
Almost all of those attempts resulted in a “blockchain bloat” – all UTXOs are stored in memory in satoshi client forever!
Moreover, in most cases the coins used to “write messages” were destroyed.
Bitcoin version 0.9 introduced new operator OP_RETURN which addresses those problems.

What is OP_RETURN?

OP_RETURN is a custom transaction locking script that can store data, but nothing beyond that, the execution of the script will be terminated immediately when bitcoin nodes encounter UTXOs that contain it – this means that you won’t be able to spend that transaction output.
When using OP_RETURN properly you will also avoid destroying coins.

Limitations

  • OP_RETURN scripts are limited to 40 bytes, this is enough for a SHA-256 checksum (32 bytes) with 8 bytes of prefix or for a shortened URL
  • There can be only one OP_RETURN per transaction
OP_RETURNs are stored in blockchain, but they are not UTXOs so they will consume disk space but not RAM.

Current status – bitcoin-core/satoshi client

Bitcoin-core, currently handles OP_RETURN without any problems.
An OP_RETURN is a standard transaction, this means it’s safe to use them and all mines will process those transactions.
It only differs in one case – OP_RETURNs won’t be treated as dust in any case
  • Transactions that transfer 0 coins are allowed – verification
  • This if statement is the one that confirms that OP_RETURN won’t be dust ever – OP_RETURN is a type TX_NULL_DATA in the bitcoin core client.
Not treating OP_RETURN as dust provides a way to send transaction that transfers 0 BTC (but will require fee) – this means that coins won’t be destroyed, but relayed to the miner that will verify that transaction.

Use cases

Multiple use cases exist, these are few examples:

How to spot OP_RETURN transactions in blockchain

These are few examples, both on main network and on testnet (try to decode them using hex editor):
More examples can be viewed using this explorer: https://www.coinprism.info/

How to send “blockchain messages”

At the moment it’s quite complicated to do so. You should consult following sources:
More user friendly way to send OP_RETURN is coming soon to bitcoinj

Add code coverage metrics to your maven project

While working on the bitcoinj project I have added support for code coverage collection and tracking. This post summarizes steps you have to take to add that for your own maven projects.

1. Figure out what type of code coverage you want

For the bitcoinj library it is important to have a “unit test coverage” – it works by verifying which parts of the source code are reached during unit testing phase during build process.
This type of code coverage is the one we usually mean when we say “code coverage”, but it’s important to know that you can also instrument a live application and see which parts of application are used and how frequently – you should consult for example wikipedia here:http://en.wikipedia.org/wiki/Code_coverage or here: http://en.wikipedia.org/wiki/Java_Code_Coverage_Tools and choose the tool you want to use.
Because for libraries we have to focus on “unit test coverage” I have choosen JaCoCo

2. Add JaCoCo maven plugin

Adding JaCoCo maven plugin is quite straightforward, those are the parts you have to add to your pom.xml:
<!-- Code coverage plugin, generates coverage report to target/site/jacoco/
     To skip coverage generation add -Djacoco.skip=true
-->
<plugin>
   <groupId>org.jacoco</groupId>
   <artifactId>jacoco-maven-plugin</artifactId>
   <version>0.7.2.201409121644</version>
   <configuration>
      <excludes>
         <exclude>**/Protos*.class</exclude>         <!-- Exclude generated protobuf classes -->
         <exclude>org/bitcoinj/jni/*</exclude>         <!-- Exclude JNI classes -->
         <exclude>org/bitcoin/*</exclude>         <!-- Exclude native classes -->
      </excludes>
   </configuration>
   <executions>
      <execution>
         <id>pre-unit-test</id>
         <goals>
            <goal>prepare-agent</goal>
         </goals>
         <configuration>
            <destFile>${project.build.directory}/coverage-reports/jacoco.exec</destFile>
            <propertyName>surefireArgLine</propertyName>
         </configuration>
      </execution>
      <execution>
         <id>post-unit-test</id>
         <phase>test</phase>
         <goals>
            <goal>report</goal>
         </goals>
         <configuration>
            <dataFile>${project.build.directory}/coverage-reports/jacoco.exec</dataFile>
            <outputDirectory>${project.reporting.outputDirectory}/jacoco</outputDirectory>
         </configuration>
      </execution>
      <execution>
         <id>default-report</id>
         <phase>prepare-package</phase>
         <goals>
            <goal>report</goal>
         </goals>
      </execution>
   </executions>
</plugin>
Important things to note here:
  • in excludes we can exclude files from being included in the report – it’s worth to exclude automatically generated files
  • the jacoco.exec file will be useful in the next steps as well
Additionally you have to update configuration for maven-surefire-plugin:
<!-- Unit tests plugin, to skip runing test add -Dmaven.test.skip -->
<plugin>
   <groupId>org.apache.maven.plugins</groupId>
   <artifactId>maven-surefire-plugin</artifactId>
   <version>2.12.4</version>
   <configuration>
      <argLine>${surefireArgLine}</argLine>      <!-- This is required for code coverage to work. -->
   </configuration>
</plugin>
These 2 plugins work together – JaCoCo will be used as a Java agent during the unit testing phase.
From now on you can just run
mvn clean test
Which will rerun all tests and recreate coverage report.
to skip running tests and generating coverage report run maven commands like this:
mvn clean package -Dmaven.test.skip=true -Djacoco.skip=true

3. Setup coveralls.io support

As a additional step (especially for projects using travis-ci) you can configure http://coveralls.io to keep track of your code coverage over time.
It’s enough to add this maven plugin:
<!-- Coveralls is a online code coverage tool -->
<plugin>
   <groupId>org.eluder.coveralls</groupId>
   <artifactId>coveralls-maven-plugin</artifactId>
   <version>3.0.1</version>
</plugin>
And then in the .travis.yml add this:
after_success:
  - mvn jacoco:report coveralls:report

4. Add Intellij IDEA integration

Following guide works for Intellij Community edition (I have tested this on version 14 but it should work on earlier versions as well)
To add Intellij integration you have to do following things:
  1. Hit Ctrl+Alt+6
  2. Select previously generated file jacoco.exec
  3. The code coverage report should refresh every time you run mvn test
This is how Intellij looks with code coverage enabled Imgur