The Dark Side of the Elliptic Curve - Signing Ethereum Transactions with AWS KMS in JavaScript

8 min readNov 5, 2020

Effective key management is one of the core challenges of blockchain adoption among enterprises. Every blockchain transaction relies on the unconditional secrecy of the private key. A compromised key can lead to permanent financial losses as well as legal and regulatory ramifications. Enterprises need to solve this in order to use blockchain.

This article explains how to overcome the key management challenge by signing an Ethereum transaction with one of the most widely adopted solutions for enterprise key management, Amazon’s Key Management Services (AWS KMS). At the end of this article, we will have generated an Ethereum transaction signed with a key held by Amazon. This key will never leave Amazon’s infrastructure, allowing us to reach enterprise-level security, including the ability to use FIPS 140–2 certified hardware security modules for Ethereum transactions.

At SUKU Ecosystem, we use the outlined strategy to manage cryptographic keys in the most secure and resilient way. As a blockchain company, we strive to be as decentralized as possible which is why we encourage our users and partners to use their own keys with a similar key management strategy on their end.

Caution: Make sure to follow each step of this guide carefully. Missing a single step or flipping a single byte will likely result in an invalid signature. If you do follow along, you will understand the differences between conventional elliptic curve keys and key pairs on Ethereum. I guarantee there are challenges and strong distinctions you would not have expected.

1. Setting up the Curve

Just like Bitcoin, Ethereum uses Elliptic Curve Digital Signing Algorithm (ECDSA). More specifically, the elliptic curve being used for transaction signing is secp256k1. You do not need to understand ECDSA to follow this guide but if you want to know more about it I recommend this article.

Luckily, AWS KMS offers the ECC_SECG_P256K1 key spec which is precisely what we need for ECDSA secp256k1 signatures. The first step is to create a new asymmetric key pair in KMS. Follow the steps in the KMS key configuration wizard and finish by obtaining your key pair ID.

If everything is setup correctly, you can retrieve your public key using the KMS part of the AWS SDK.

The result of this is your ECDSA public key. It will look somewhat like this (hex):

3056301006072a8648ce3d020106052b8104000a034200043d471f65fb7066ef3656c90fc262d14fecd637adb5d1a369427ebb342340badd791ec332ee985b7ec5af6d8ee83e1237342805c219de34fa2b42e753358cd3f5

2. Calculating the Ethereum Address

You have probably already realized that the public key that you retrieved from AWS doesn’t look anything like an Ethereum address. Well, you are absolutely right. In order to get your address we need to:

Decode the DER-encoded public key
Calculate the Ethereum address

2.1 Decoding the public key

Before calculating the Ethereum address, we need to get the raw value of the public key. Looking at AWS’s documentation, the getPublicKey() function returns a DER-encoded X.509 public key, also known asSubjectPublicKeyInfo (SPKI), as defined in RFC 5280.

As it turns out, the SubjectPublicKeyInfo format is actually defined in section 2 of RFC 5480. It looks like this:

SubjectPublicKeyInfo  ::=  SEQUENCE  {
       algorithm         AlgorithmIdentifier,
       subjectPublicKey  BIT STRING
     }

I spent hours trying to parse this manually (don’t even bother, there’s plenty of room for error). A much better option is to use an ASN1 library that allows us to define this as a schema. I did that in the code below.

The resulting bit string will look like this (in hex):

043d471f65fb7066ef3656c90fc262d14fecd637adb5d1a369427ebb342340badd791ec332ee985b7ec5af6d8ee83e1237342805c219de34fa2b42e753358cd3f5

2.2 Calculating the address

According to section 2.2 of RFC 5480, the first byte,0x04 indicates that this is an uncompressed key. We need to remove this byte for the public key to be correct (line 14 below). Once we delete the first byte, we get the raw public key that can be used to calculate our Ethereum address.

As explained in chapter 2 of Mastering Ethereum, the address is the last 20 bytes of the keccak256 hash of the public key. Thus, the code to generate the Ethereum address is as follows.

The result of this is a valid Ethereum address and should look like this:

0x3b62a92b8873a89d8c1e487fe8258f0360a97037

3. Signing

With the Ethereum address in the back pocket, we can start signing the transaction. The signing process can be divided into four separate steps, each of which I will explain below.

Sign a keccak256 hash of the message with AWS KMS
Decode the DER-encoded signature
Calculate r and s
Find the right v value to complete your Ethereum signature

3.1 Signing with AWS KMS

Make sure to specify MessageType: 'DIGEST' in the KMS.SignRequest, otherwise, AWS will try to hash your payload again which will generate an invalid signature. The payload needs to be a keccak256 hash of your transaction object.

We will get to the real payload later. For now, you can just sign an empty keccak256 hash that will be passed to this function as msgHash.

3.2 Decoding the Signature

The kms.sign() returns a DER-encoded object as defined by ANS X9.62–2005. I’ve created a parse function for this according to section 2.2.3 of RFC 3279. Taking a closer look at it, you will see that this function expects to find two integers r and s in the signature that will be returned as two BigNumber (BN.js) objects.

The result represents a point on the elliptic curve where r represents the x coordinate and s represents y.

3.3 Calculating the Ethereum Signature

According to EIP-2, allowing transactions with any s value (from 0 to the max number on the secp256k1n curve), opens a transaction malleability concern. This is why a signature with a value of s > secp256k1n / 2 (greater than half of the curve) is invalid, i.e. it is a valid ECDSA signature but from an Ethereum perspective the signature is on the dark side of the curve.

Caution: Not every ECDSA signature is a valid Ethereum signature!

The code above solves this by checking if the value of s is greater than secp256k1n / 2 (line 21). If that’s the case, we’re on the dark side of the curve. We need to invert s (line 26) in order to get a valid Ethereum signature. This works because the value of s does not define a distinct point on the curve. The value can be +s or -s, either signature is valid from an ECDSA perspective.

3.4 Finding the right v value

We calculated r and s , but another value v is missing for a valid Ethereum signature. v is the recovery id and it can be one of two possible values: 27 or 28.

v is typically created during Ethereum’s signing process and stored alongside the signature. Unfortunately, we did not use an Ethereum function to generate the signature which is why we do not know the value of v yet.

Using Ethereum’s ecrecover(sig, v, r, s) function, we can recover the public key from an Ethereum signature. Since we calculated the Ethereum address earlier, we already know what the outcome of this equation needs to be. All we have to do is call this function twice, once with v=27 , and in case that does not give us the right Ethereum address, a second time with v=28. One of the two calls should result in the Ethereum address that we calculated earlier.

Caution: v may vary from one signature to another. Therefore, you may need to call the recover function twice in order to find the right v. If neither v=27 nor v=28 give you the previously calculated public key, you should double check every step. Missing a single byte or a single conversion will result in an invalid signature.

4. Sending the Transaction

If the recovered public key matches your previously calculated public key, you’re good to go, you can send your first Ethereum transaction.

Make sure to fund your wallet first since you will need to pay the transaction fee. I’d recommend Kovan or Rinkeby for this but it will work on any testnet. Once your account is funded, you can go ahead and send your transaction.

There’s another gotcha when it comes to transaction signing. Remember, you are signing a hash of the transaction object. If you do not set any value for r, s and v before signing, Ethereum will not be able to determine the from address of the transaction. In turn, it will end up calculating the wrong public key and it won’t be able to send your transaction due to insufficient funds. This is a chicken-egg problem since you do not know the rsv values prior to signing.

There are multiple ways to solve this but I think the easiest way is to set a temporary value for rsv before hashing the transaction object. Replace the temporary values with the real signature before sending and web3 should calculate the from address correctly, resulting in a valid transaction if your account is funded. By doing this, you will be signing twice but you avoid hard coded values and hick-ups. And even though you are technically signing twice to make web3 come up with the right from value, you’re only sending once.

5. Next Steps

If you have made it this far, congratulations! You should have been able to KMS-sign your first Ethereum transaction. Feel free to post your transaction hash in the comments and let me know what challenges you were facing along the way. This was my first transaction and it felt quite rewarding after going through all this.

For your convenience, I have published a minimal working example of this code here. For an enterprise use-case, this should be wrapped into a transaction signing library. I recommend implementing your own signing provider for web3 or ethers.js but that is beyond the scope of this document.

At SUKU, we have been using an KMS/HSM-based signing strategy for a while and it has enabled us to sign Ethereum transactions without ever storing a single key. For us this strategy turned out to be of tremendous value from a security perspective. Not only does it allow us to use our AWS infrastructure for transaction signing, it also unburdens each individual by leaving key management to the HSM.

6. Conclusion

The mere fact that Ethereum is based on ECDSA secp256k1 signatures does not mean it is fully supported by existing key management solutions. Custom requirements such as EIP-2 mandate additional rules on top of the standard and make it hard to use existing protocol layers. Additional work and going all the way down to the byte level is required to make sure you are on the right side of the curve. This leaves plenty of room for error and makes the development experience very cumbersome.

With enough pressure from the community and an increasing number of companies adopting blockchain technology, cloud providers and infrastructure firms will eventually support blockchain standards like this. Until then, blockchain engineers will have to fill the gap by #buidling libraries and tools that make blockchain technology more compatible with conventional platforms.

Resources:

The entire code shown in this article is available here
AWS Key Management Service Documentation
Using AWS CloudHSM to sign transactions [Ethereum StackOverflow]