Skip to content
Murali Krishnan A
All writing

14 Jun 2020

I rewrote SSL in my first year of college. Here is what it taught me.

Hedge won a Facebook productization grant in 2017. It captured browsing activity in internet cafes, scored it with NLP, and moved it over a handshake I built myself. Most of what I learned came from the parts I got wrong.

9 min read · security, nlp, learning

In my first year of engineering college I built a system called Hedge with my team, and we kept working on it through the year after. It won the Digital Masala Challenge, a hackathon run by Facebook, now Meta, and Youth Ki Awaaz. We were selected from over two hundred teams and were given a productization grant to take it further.

I had my first reliable internet connection in 2017. The same year I wrote a key exchange protocol from scratch because I did not know TLS already existed in a form I could just use.

We stopped working on it two years ago. Going back through the repository now is an unusual experience, because I can see exactly what I understood at the time and exactly what I did not. This is a post about both.

What Hedge did

Internet cafes and college computer labs have a problem nobody wants to own. The machines are public, the browsing is anonymous, and if somebody uses one of them to plan something serious, the owner is the one who has to answer for it.

Hedge sat on those machines. It watched what was typed and read in the browser, scored the content, and gave the cafe administrator a dashboard. A privacy notice appeared before monitoring started, because the whole point was that this ran on cafe-owned machines with the customer's knowledge.

Four pieces:

The extension collected. The client on each machine forwarded. The agent gave the owner a view. The brain scored and stored. That separation was the one piece of architecture I would still defend today.

Collecting was easy. Collecting the right thing was not.

The first version of the content script is four lines of idea and one bad decision.

var fname = window.location.hostname;
var oldcontent = '';

setInterval(function() {
  var newcontent = document.body.innerText;
  if (oldcontent != newcontent) {
    var msg = {
      "Hostname": fname,
      "URL": window.location.href,
      "Content": newcontent,
      "UID": script_id
    };
    chrome.runtime.sendMessage(msg, function(id) { /* ... */ });
    oldcontent = newcontent;
  }
}, 100);

Every hundred milliseconds, read the entire visible text of the page. If it changed at all, send the whole thing.

It worked. It was also enormously wasteful, and worse, it was wrong for the problem. Someone typing a message into a forum produces a new full-page snapshot on every keystroke. What we actually wanted was the sentence they were writing, not two hundred copies of the page it was being written on.

That is the moment the project got interesting for me.

Sending the difference instead of the document

The fix was to keep two versions of every page. One I called Sync, the content already accounted for. One I called aSync, the newest snapshot. When a new snapshot came in, diff it against the settled one and keep only what was added.

function comparator(p1, p2) {
  var dmp = new diff_match_patch();
  dmp.Diff_Timeout = 0;
  var dif = dmp.diff_main(p1, p2, false);
  dmp.diff_cleanupSemantic(dif);

  for (v of dif) {
    if (v[0] == 1) {                       // 1 means inserted
      return { "stat": "update", "adata": v[1] };
    }
  }
}

function preprocess(msg) {
  aSync[msg["UID"]] = msg["Content"];

  if ((msg["UID"] in Sync) && (msg["UID"] != 0)) {
    specimen = comparator(Sync[msg["UID"]], aSync[msg["UID"]]);

    if (specimen.stat == "update") {
      if (aSyncData[msg["UID"]] != specimen.adata) {
        if (ajax_data.includes(specimen.adata)) { }
        else {
          aSyncData[msg["UID"]] = specimen.adata;
          ajax_data.push(specimen.adata);    // only the new text goes on
        }
      }
      Sync[msg["UID"]] += aSync[msg["UID"]];
    }
  }
  else if (!(msg["UID"] in Sync) && (msg.UID != 0)) {
    Sync[msg["UID"]] = msg["Content"];
  }
}

I used Google's diff-match-patch for the comparison. The filter is the whole trick: v[0] == 1 keeps only the segments marked as insertions and throws away everything that was already there.

I did not have the vocabulary while I was writing it, but I have since learned this pattern has a name. It is change data capture. Snapshots in, deltas out. It turns out to be a common shape in systems that have nothing to do with browsers.

Two things in that code make me wince now. The empty if branch with the real work in the else is backwards, and I should have inverted the condition. And ajax_data grows forever, because nothing ever removes from it. On a machine left running all day that is a slow memory leak. Nobody caught it because nobody left it running all day.

The NLP, and the thing I did not understand

The scoring ran in Python, called from Node as a subprocess with the payload passed as a JSON argument.

def analyser(uid, data):
    sents = nltk.sent_tokenize(data)      # split into sentences
    sid = SentimentIntensityAnalyzer()    # VADER
    com = 0

    for sentence in sents:
        ss = sid.polarity_scores(sentence)
        com = com + ss['compound']

    sentiment_score = (float)((com) / (len(sents) + 0.0001))
    rounded_score = round(sentiment_score, 2)

    if (sentiment_score < 0.0):
        feedback = "negative"
    elif (sentiment_score > 0.1):
        feedback = "positive"
    else:
        feedback = "neutral"

    if (sentiment_score < -0.5):
        indicator = "violent"
    else:
        indicator = "non violent"

    return UserSentiment(uid, rounded_score, feedback, indicator)

NLTK splits the text into sentences. VADER gives each sentence a compound score between -1 and 1. Average them, and you have a number for the whole passage.

The + 0.0001 in the denominator is there because I divided by zero on empty input and did not want to write the guard properly. It is the most first-year line in the entire repository and I am leaving it in this post exactly as it was.

Now the real problem.

Look at that last branch. A score below -0.5 gets labeled violent.

Sentiment is not intent, and I had built the whole detection layer on the assumption that it was. VADER measures whether language is positive or negative. It does not measure whether someone is planning to hurt anyone. A student writing a furious review of a terrible movie scores strongly negative. A person calmly researching something genuinely dangerous, in flat neutral prose, scores around zero.

So the system was close to backwards on the exact cases it existed for. It would flag anger and miss calm. And because the output was a confident word on a dashboard rather than a probability with a caveat, it invited a cafe owner to trust it.

I did not see any of this while I was building it. I saw a working pipeline, a number coming out of the other end, and a demo that got applause. It took someone asking me what the label actually meant before I understood that I could not answer.

If I built this today I would not put a label on a person at all. I would surface the passage to a human and let them decide, and I would be very careful about what the interface implied.

Rewriting SSL

The pieces of Hedge had to talk to each other across a network, and the traffic included what people had typed. It needed to be encrypted.

I knew what TLS was. I did not understand that I could just use it. So I read about how key exchange works and built one.

var generateSymKey = function(length = keyLength) {
  var chars = '0123456789abcdefghijklmnopqrstuvwxyz...';
  var key = '';
  for (var i = length; i > 0; --i)
    key += chars[Math.floor(Math.random() * chars.length)];
  return key.toString();
};

var encryptMessage = function(specimen) {
  var key = generateSymKey();
  var iv = crypto.randomBytes(ivLength);
  var enc = crypto.createCipheriv(encType, new Buffer(key), iv)
                  .update(specimen, fromEnc, toEnc);
  return { message: enc, key: key, iv: iv };
};

this.encrypt = function(msg, pubKeyRemote, privKeyLocal) {
  var pubKeyC  = ursa.createPublicKey(pubKeyRemote);
  var privKeyM = ursa.createPrivateKey(privKeyLocal);

  var encryptedMessage = encryptMessage(msg);
  var key = encryptedMessage.key;

  var enc = pubKeyC.encrypt(key, 'utf8', 'base64');                 // seal the key
  var sig = privKeyM.hashAndSign('sha256', key, 'utf8', 'base64');  // sign the key

  return {
    message: encryptedMessage.message,
    signature: sig,
    key: enc,
    iv: encryptedMessage.iv
  };
};

The receiving side unwraps it in the reverse order.

this.decrypt = function(toDec, key, iv, sig, pubKeyRemote, privKeyLocal) {
  var privKeyC = ursa.createPrivateKey(privKeyLocal);
  var pubKeyM  = ursa.createPublicKey(pubKeyRemote);

  var decryptedKey = privKeyC.decrypt(key, 'base64', 'utf8');
  rcv = new Buffer(decryptedKey).toString('base64');

  if (pubKeyM.hashAndVerify('sha256', rcv, sig, 'base64')) {
    verified = true;
    msg = crypto.createDecipheriv(encType, new Buffer(decryptedKey), iv)
                .update(toDec, toEnc, fromEnc);
  } else {
    verified = false;
    msg = null;
  }

  resolve({ verified: verified, message: msg });
};

Read past the variable names and the shape is right. Generate a fresh symmetric key per message. Encrypt the payload with AES. Encrypt the symmetric key with the recipient's RSA public key so only they can open it. Sign it with your own private key so they know it came from you. That is hybrid encryption, and it is genuinely what TLS does.

I was proud of that. It ran, the messages came out the other side, and the signature check caught tampering when I tested it by hand.

Four ways it was broken

I found out about every one of these after the fact, mostly by reading things I should have read first. Together they are why "never roll your own crypto" is advice rather than snobbery.

The keys came from Math.random(). This is the serious one. Math.random() is not a cryptographically secure generator. Its output is predictable from enough samples, and it is not designed to resist anyone trying. I was generating the actual encryption key with it, three lines above a call to crypto.randomBytes for the IV. I had the right function in the same file and used it for the less important value.

RSA at 1024 bits. Already below the recommended minimum in 2017. NIST had moved to 2048 years earlier. I picked 1024 because it generated faster on my laptop and I was impatient during testing.

The signature covers the key, not the message. Look carefully at what gets signed: hashAndSign(..., key, ...). Not the ciphertext. So an attacker who cannot read the message can still modify the ciphertext in transit, and the signature still verifies, because the key it signed is untouched. AES-CTR is malleable by design, which means flipping a bit in the ciphertext flips exactly that bit in the decrypted plaintext. The verification I trusted was checking something adjacent to the thing that mattered.

A hardcoded key in the source. There is a literal encryption key in one of the files, committed to the repository, used to encrypt client credentials. It is still there. I am not reproducing it here, but I know exactly which line it is on.

None of these would have been caught by testing, and that is the lesson. The system passed every test I could think of, because I could only think of tests for the failures I already understood. Cryptography fails in ways you have to be told about.

What it actually taught me

The grant paid for a project that, judged as a security product, was not one. The detection logic measured the wrong thing and the transport had a predictable key. If somebody deployed it today I would tell them to stop.

But I would build it again.

The diff pipeline taught me to think in changes instead of states, and I have reached for that pattern constantly since. Splitting the system into four processes that each did one job taught me more about architecture than any course did. Writing the handshake meant that when I finally read the TLS specification properly, I already knew what problem each part of it was solving.

And getting the NLP wrong taught me the most valuable thing on the list, which is that a confident output is not a correct one. I built something that produced a clean answer for every input, and I never asked whether the answer meant what the label claimed. Two years later I think that question is the most useful thing the whole project gave me.

If you are building something too ambitious with tools you half understand, keep going. Just write down what you were unsure about while you were unsure about it. I suspect that list gets more interesting the further you get from it.

Read next