How do you understand big numbers?
OpenDNS does ~3 billion DNS requests daily, with around 450 billion all-time so far. Big numbers, but tough to comprehend.
Recently, we changed the stats number at the top of every page of our website from the all-time number to requests/per second. This number moves around, but recently has swung between 37,000 - 41,000 requests per second.
OK… sounds impressive, but again, what should you compare the number to?
How about the volume of transactions on the New York Stock Exchange (NYSE)?
Tuesday’s Wall Street Journal ran “After Crash, NYSE Got the Message(s)” on the front page of the Money & Investing section (C1 in print).
In reading the article (in case it’s not available when you click… WSJ.com requires payment at some point), we learn:
On Feb. 27, 2007, messages flowed in at a rate of 15,000 per second. The exchange quickly thereafter doubled its capacity to 38,000 messages a second. As markets fell in August when credit markets seized up, the NYSE was getting as many as 28,000 messages per second. This time, systems held up without a major hitch, but the volume of messages prompted [NYSE CEO]Mr. Thain to call for an increase in capacity to 64,000 by year’s end. [Emphasis added]
So, even at its top volume in August, the NYSE volume of messages wasn’t matching the volume of DNS requests our customers make each day. And its capacity currently falls short of ours.
Maybe I’m comparing apples and oranges, but we like to think that your DNS requests are as important (almost?!) as those buy-sell messages.
OpenDNS has plenty of headroom, and we’re adding more to support our growing customer base… and to stay ahead of the NYSE! 
James D Kirk
You mention apples and oranges, but I think not. It’s all just data, correct? So the real question of being able to compare is just what sort of volume size are the requests from openDNS and the messages from the NYSE? And if variable, then what is the total amount of data that is being dealt with “per second”? That would allow you to compare apples to apples (or oranges to oranges if you prefer!)
posted on October 19th, 2007 at 4:06 pm
John Roberts
Yup, all just data. Whether a lookup for yahoo.com is as valuable as a “buy” message for YHOO was really my point there.
I doubt the size of each individual request is very different. I know DNS requests are small. Need to find someone who knows more about the nature of the NYSE messages to know for sure.
posted on October 19th, 2007 at 4:09 pm
habtool
Thats amazing numbers!!
Well done guys, i hope it now or in the future equates into big profits for you company.
S
tay well
posted on October 21st, 2007 at 4:27 am
jb
The key difference (and hence the oranges), is that the NYSE messages require more work than a DNS req. A lot more work.
Let me explain.
Each NYSE order has a few data params - symbol, price, volume, whole_lot? (do we only execute if we can trade all of the specified volume), party_id, etc. When you send an order, it gets put in the “book” for that symbol. We’ll ignore orders that say “buy/sell at the current price” - those are uninteresting.
The “book” is an array of fifos. Elems in the array are keyed off of the price. Elems in the fifo are based on time of submission and order type (buy vs sell).
Then, a timer goes off, and the trading engine matches orders that match on price (and sometimes size) based on the last trade price, whole_lot?, time of submission, etc. This is a logical puzzle for those who care.
The easy part of the NYSE’s system is that it’s easy to parallelize - one machine per book - but also quite a logical mess. Add to that, the need for audit logs, and well, it’s a much bigger job to handle 1 NYSE transaction vs 1 DNS transaction.
In other words, the NYSE’s volume is a little more complex and much more computationally complex.
Not to say the OpenDNS is not impressive, but it’s a different type of problem. =)
jb
posted on December 18th, 2007 at 7:31 pm
David Ulevitch
JB,
Yeah yeah, a few of my friends who work in the financial markets told me the same.
We were saying mostly in jest… Losing a DNS packet stinks, but not as much as a trade, I’m sure.
posted on December 18th, 2007 at 7:35 pm