A Quick Analysis of Facebook Bug Bounty Program

(v2, updated 10/15/2014)

Nowadays, Web companies have been relying on vulnerability reward programs (VRP, also called bug bounty programs) to discover vulnerabilities in their products. Basically, a white hat (good hacker) can submit a vulnerability discovery report and then get some money back. We have written a preliminary paper analyzing a related program called Wooyun, and please take a look if you are in general interested in this new paradigm of improving security.

Facebook is one of the companies that embrace this idea, although Facebook is generous sometime (see this and this), :P. FB also hides information about what vulnerabilities have been discovered, or the details of each white hat's accomplishment (e.g. how many vulnerabilities one has discovered, and when). FB only provides a list of white hats who have contributed to Facebook security every year, at this page.

Anyway, we can start with this page and do some quick analysis. The data is obtained by 9/20/2014. First, there are 670 names on the list (there are several cases when multiple names appear in one line and separated by commas, and we will count each name alone). Quite a lot, isn't it? But it is possible that some enthusiastic white hats contributed every year and leave their name multiple times, so we also count the number of unique names, which is 516.

Next, we count the number of white hats each year, shown in the following table:

Time	White Hat Count
2014 (up to 9.20)	191
2013	255
2012	126
2011	55
Prior to 2011	43

We clearly see the trend: more and more players are joining this game, and the number roughly doubles every year:) I guess VRP is really a promising idea (please see our paper for more discussions).

There is also an interesting fact: a lot of white hats are only active in one year. To show this, we create another table counting the white hats based on number of years being active:

Number Years being Active	White Hat Count
1	402
2	82
3	26
4	5
>=5	1

So far, there are 402 who have only appeared in one year's thank list. And we can see that the white hat count distribution is highly skewed. Much few white hats are active for more than one year. And there is only one person who has been thanked all the time! This probably shows that the value of this kind of VRP not only lies in a few experts, but also in a large number of people. But since we don't know how many vulnerabilities each white hat contributes and the severity of them, the conclusion is hard to make. Still, this observation is consistent with what we claim in our paper.

You might wonder who is the "all the time" person, and the answer is: Szymon Gruszecki. You can access his personal page here.

Please feel free to discuss by leaving a message. Thank you for your time!


Update:

Facebook has released some interesting statistics of its bounty program here:
https://www.facebook.com/notes/facebook-bug-bounty/bug-bounty-highlights-and-updates/818902394790655

Some interesting points:

• From the statics we see that there is a huge number of invalid reports. The valid rate is only 4.7%. Why?
• It says that "One of the most encouraging trends we've observed is that repeat submitters usually improve over time. It's not uncommon for a researcher who has submitted non-security or low-severity issues to later find valuable bugs that lead to higher rewards." Actually, we plan to investigate this issue further in our data set.
• The country rank: Russia -> India -> USA -> Brazil  -> UK

References

[1] The picture. http://america.aljazeera.com/content/dam/ajam/images/shows/Real%20Money%20with%20Ali%20Velshi/SG_FB2_1460.jpg

Bugs and Patches for Papers

In an earlier article, Writing Like Compiling, I have made some connections between programs and papers. This article makes a connection from a different perspective.

For publications in Computer Science (and possibly other domains), there is a problem. A paper could contain bugs: errors, unclear sentences, missing backgrounds, etc. These bugs might caused by the knowledge gap between the authors and the readers. Or they simply arise due to the conference-driven publication paradigm. Such paradigm puts researchers on a fast race and leave them less time to ponder and polish their work [2]. These bugs inflict readers minds and eats up their time. Some smart readers might find ways to fix those bugs, just like an advanced user finds a bug in a program and makes a patch for it. However, since there is not good way to share the fix, and a paper is usually fixed after the camera-ready version, this paper patches only stay in a paper copy as some red marks...

Programs, too, are not perfect after release. However, software developers and users will constantly discover new bugs and apply corresponding patches. And this model generally works well. After all, there seems to be no alternative way. Therefore, I think we need to treat papers as programs, and creates ways for reporting bugs and sharing patches. A first step is to store these paper bugs and patches in some database and enable readers to search for them. However, we want to avoid the detachment between the papers and the patches, so we could allow some energetic readers to fork a paper and make version 2.0 of that paper. This fork ability is very common in the opensource software community [3].

In general, isn't it a bit ironic that Computer Science, the field that aims to digitalize paper-based information, still record its cutting-edge findings in papers?


References

[1] The picture. http://www.rebeccaheflin.com/wordpress/wp-content/uploads/2013/08/rejected-writing.jpg

[2] Fortnow, Lance. "Viewpoint Time for computer science to grow up." Communications of the ACM 52.8 (2009): 33-35.

[3] Raymond, Eric. "The cathedral and the bazaar." Knowledge, Technology & Policy 12.3 (1999): 23-49.

Two Downsides of Privacy

v0.2

These days people are all talking about privacy, partly thanks to Mr. Snowden's effort. While I definitely support the the individual right of privacy, I want to talk about two downsides of privacy here. But again, I strongly agree that each individual should have full control of her or his information. I just think sometime we might want to trade privacy for more important things.

The first downside is that privacy could lead to distrust. An often used example for supporting privacy is: you got drunk one night and shared a photo of drinking on Facebook. While your friends might like it, your boss doesn't. So people have been designing advanced access control technologies to keep your boss away from your "little secrets". Some people might even avoid using Facebook, considering that some companies require employees' Facebook passwords [2]. However, such personal information disclosure could help others understand you more and thus enhance the relationship. On the other hand, if a person cannot be traced at all on the Internet, he or she will be a mystery in others' eyes. And will you trust a mysterious figure?

The second downside is that privacy might obliviate a person. From East to West, from past to present, an eternal pursue of the mankind is immortality. At least, a person wants to leave something to this world after death, which could only be achieved by a small group of people in the past. This digital age enables immortality to everyone, in the sense that their words and activities on the public Internet can be recorded and kept almost forever. Search engine could retrieve one's words and return to somebody in the future, and we could imagine it as a kind of conversation between the dead and the live. However, privacy would make these information secret to only a few people or even one person, through technologies like encryption. If that person happened to pass away, then his words also gone with him, if no one else knows the password. What a pity if Einstein II encrypted his remarkable theory and then passed away accidentally. Would it be better to publish it on a blog, like this blog?

Update: we all feel really sad about the MH17 tragedy. It has been said that there are more than 100 AIDS researchers on board. And I hope we can rescue their ideas and thoughts as much as possible.



References:

[1] The picture. http://blog.static.abine.com/blog/wp-content/uploads/2011/10/privacy.jpg?e835a1

[2] http://www.usatoday.com/story/money/business/2014/01/10/facebook-passwords-employers/4327739/

Heartbleed and the Paradox of Security Professionals

The recent Heartbleed vulnerability of OpenSSL shaken the whole Internet. Yet such incident might not be a total surprise because OpenSSL only had one full time employee and received $2000 a year as a donation [1] before the incident. Such support is by no means enough for the developers to produce high quality code and test the software product comprehensively. I guess the even hackers who keep searching for vulnerabilities inside OpenSSL have much more funding.

But the situation is changed now, as tech giants agreed to fund OpenSSL for at least 3.9 millions in three years. Even a Chinese mobile company, Smartisan, announced a donation of 1 million yuan ($160,000) [2]. You might already feel the strangeness of this event: a mistake makes millions of money!

The more astonishing thought is that if OpenSSL developers did a better job by not introducing the vulnerability, then they will still starving and suffering in poverty! This paradox does not only apply to OpenSSL, but probably to every company that needs security. For such a company, if the security team is doing a good job, then the company's CEO might feel the security team is redundant because nothing bad happens. Although the CEO might not that dumb to fire the security team, nonetheless the CEO could not appreciate the effort of the security team and might not raise their salary. Thus for the security team, there is hardly any incentives to do better. Rather, they might just want to meet the minimum requirements, or even accept some security incidents to attract the attention from company managers.

Do you know how to break this paradox?




References:

[1] Tech giants, chastened by Heartbleed, finally agree to fund OpenSSL. http://arstechnica.com/information-technology/2014/04/tech-giants-chastened-by-heartbleed-finally-agree-to-fund-openssl/

[2] http://www.ithome.com/html/android/86232.htm

[3] The picture. http://www.paradoxproductions.com/pics/tritwo.gif

Research vs. Learning

A PhD student has two roles. One is a research assistant that strives to produce good research papers. Another is a learner that keeps improving oneself. Actually, even after obtaining PhD, a researcher still need to learn, and might need to do so for one's lifetime. Research result is explicit while learning result is implicit. That why some students and faculties (e.g. [1]) ignore the later role. However, I would like to use a simple model to show that such ignorance leads to inefficiency.

Let's consider making research progress as a random sampling from a normal distribution, which represents the capability of the PhD student. And higher value of the sample corresponds to higher quality of the research. No matter how many samples are drawn, the average of the samples is the mean of the distribution. And only a few of them could have high value.

Now, student A and B both start with a normal distribution of mean = 1 and variance = 3. And we further assume that sample x > 5 means a very good paper and x < 0 means a failed research project. The figure is:

Now, A decide to solely focus on research. So A keep sampling from this distribution. However, the probability to produce a good pare is only 1.0%. So 100 trials lead to one good paper. And the chance of failure is 28.2%, so more than 1/4 of the trials result in failure.

On the other hand, B focus on learning, by which B pushed the mean of the distribution to 3:

For B, the probability of generating a good paper is 12.4%, which is better than A's in an order of magnitude. Moreover, the probability of failure now drops to 2.8%.

I don't want to draw any conclusion because it is just a very rough model. However, I think you can see the point. And I am also not saying that a PhD student should only focus on learning without any research responsibility. I personally think a PhD student should definitely spend more time on research than on learning. And doing research is actually another very important way of learning, that's why I put the Taiji graph in the beginning of this article, because they boost each other. The point I want to make is that during the journey, sometime there will be a stagnant period during research and we might feel sad. However, we should smile because as long as we keep learning and keep pushing the mean of our normal distribution to the right, things will be fine:)


References:

[1] The picture. http://www.acuherb.us/image/taiji01.png

[2] http://blog.liyiwei.org/?p=1429

Learn the Upstream

v0.1

It seems that learning the upstream of your research field would be very helpful for your research. This claim assumes that a field has a upstream field, or all fields are constructed in a hierarchical structure. I guess most people would agree with this. For example, the upstream field of Computer Science is mainly Mathematics. Mathematics provide language, tools and theorems to build the foundation of Computer Science, and many people (e.g. [1]) believe that the prerequisite of a good Computer Scientist is a solid knowledge of Mathematics. This article will expand this point to other fields by presenting several examples.

Most sub fields in computer security are more or less rooted in cryptography, which is no doubt the earliest sub field in computer security and the most rigorous one. This explains that several famous security researchers such as Ross Anderson and Bruce Schneier, started their career in cryptography, and then "invaded" many other sub fields. Andrew Yao might be another example. He switched from Physics to Computer Science, and got Turing Awards.

More examples can be found. Yin Wang has been criticizing many important products in computer science, such as SQL, Unix, Go language, ... While the validity of his criticisms are always in debate, I think they do have some value. And I further realize that it is because Yin Wang is from the programming language field, which is more or less the upstream of many other computer science subfields, such as database and OS. My roommate also serves an interesting example. Once a Math major undergraduate student, he switched to the field of Deep Learning now. Compared with researchers in CS background, his knowledge in Math helped him understand the problem deeper.

People always talk about jumping out of the box is the way towards creativity. Well I guess learning the upstream is the way towards the outside of the box. Isn't it?

References:

[1] The picture. http://www.nolandalla.com/wp-content/uploads/2014/02/salmon.jpg

[2] How to do Research At the MIT AI Lab. David Chapman. 1988

Technological Niche

v0.1


The Facebook purchase of Oculus VR  in 2 billion dollar is still quite shocking. Particularly, it is because the company is only 2-year-old and the owner is only 21. And for people who are more familiar with technologies, there are two more reasons to be shocked. One, the legend graphics programmer, John Carmack, joined the company. And two, Virtual Reality (VR), a once hot technology, was considered a mistake and pretty much dead.

I still remember when I was an intern in Microsoft Research Asia, 2011, I attended a talk given by Professor James A. Landay. During the Q&A session, a student asked Dr. Landay about the prospect of VR. Dr. Landay said something like "many people believe VR is a mistake, and augment reality is the right way". Indeed, in the first fad of VR back in late 1990s and early 2000s, companies spent a lot of money for VR but the return was frustrating [1]. However, the founder of Oculus VR just loves VR all the time and collected most of available VR equipment. This love gives him motivation and this collection gives him inspiration. I guess most of us could regret for what unique interests we have abandoned in order to cater to others, and what large amount of money we have lost...

Let's wipe our tears and move on. Despite the lesson of perseverance, we could also find some other interesting things. For me, I would say a unique thing will never be a mistake and every unique thing will have a position and purpose in the world. And this is why I am able to write this article and you are able to read it. We know that human are mammals. But in 65 million years ago, the lord of the earth, dinosaurs, might not know mammals, which at that time, were just some tiny creatures eating bugs to survive. An alien who visited earth at that point could say that mammals are "a mistake" compared with dinosaurs. However, we all know the story afterwards, dinosaurs got wiped out in a disaster and mammals control the world. We could say that mammals win this time, but the ultimate winner is Nature who keeps the diversity of the ecosystem to combat disasters.

Same principle applies for technology. VR might not be the mainstream technology in the past 20 years and possibly not the mainstream technology in the next 20 years, as Oculus VR might fail. However, VR does have the value to exist, because it might play a critical role in the future. Every unique technology fits a technological niche, and is prepared for its day.



References:

[1] http://time.com/39577/facebook-oculus-vr-inside-story/

[2] The picture. http://media.pcgamer.com/files/2013/04/Eve-Oculus-RIft.jpg