In this article I will describe the situation in the security department with my opinion on this.
Callisto Network as a security platform.
Here you can find an article describing Callisto fundamentals. One of the main goals of Callisto was to serve as a third party security expert for those who want ensure their investments safety.
Fundamentally, there is a number of differences between Callisto and regular auditing companies. The main clients of audit companies are contract developers themselves who want to make sure that the contracts they have developed are secure. It was assumed that the main customers of Callisto will be investors who want to ensure the security of smart contracts in which they want to invest their money so as not to become victims of hackers.
Main issues in the development of the Callisto Security Department.
1. Structure of the Security Department.
Since the very start we relied on the full time team of security auditors. We hired employees based on the results of test tasks that we gave them. However, in this approach there is a problem of scalability — test items must be compiled and updated. If we are going to support more networks (for example ETH, ETC, TRX, EOS and others) then we need to maintain these test tasks which could be time-consuming in quickly evolving crypto industry.
We also have a goal to form an open market for audits and audit services. This means that we must establish a structure which will allow auditors to contribute in “freelance style”.
However, such a decision may affect the quality of audits. Currently, at least three full time auditors conduct each audit. By allowing third parties to participate in audits we risk getting lower quality reports. Therefore, it was decided that each audit should be conducted by at least two full-time auditors + one more full-time auditor or a third party auditor. Thus, at least two trusted auditors will participate in each audit.
Current stage: we have a proposal to implement an experimental hiring / contributing system which should solve this goal.
2. Reward calculation.
Auditor rewards are calculated in proportion to the scope and effectiveness of the work they perform. The remuneration system is faced with the task of ensuring that auditors work with maximum efficiency by sacrificing the amount of work done. On the other hand, it is undesirable to make it so that “beginner auditors” or auditors who do not have sufficient experience in performing these tasks receive too little salary, which would not cost their time.
On the one hand, we need to achieve high quality of audits, on the other hand, it is necessary to allow auditors with little experience to participate and get paid.
The work efficiency can be increased by increasing the payments to auditors for finding errors and increasing penalties for not finding errors. However, not all contracts contain errors and auditors have no clue about whether they can theoretically find errors in contract they audit or not. Thus it is necessary to maintain payment for the work performed at a high level even without premiums for error reporting.
On the other hand, this approach opens up a strategy for auditors to maximize the amount of work performed by sacrificing the quality. In this case, auditors theoretically can accept audit requests and report no mistakes without even spending significant amount of time on contract code check. By “hoping that everything is fine and providing a report as if it is so” auditors could earn significant score.
It is necessary to find the right balance between the reward for finding errors, penalties for missing errors and the amount of work performed in the absence of reported errors.
Current stage: proposal #59 is intended to resolve this problem, however the right balance is difficult to find in absence of competitiveness between auditors.
3. Competitiveness and scalability.
One of the main problems of the Callisto Security Department is the small number of participants and, as a result, the lack of competition. However, we cannot hire more auditors or offer higher salaries than now because the work of auditors does not directly affect the price growth rate.
The same can be said about scalability. We could form the EOS Audit Department, but this will require extra expenses.
Someone might say that we can start charging fees for audits. I am absolutely sure that this will not bring any benefit to the project.
- First, our main clients should be investors, not contract developers. Investors most likely will not pay for an audit of a project in which they have not yet invested their funds.
- Secondly, we will engage into competition with ordinary audit companies. The difference is that ordinary audit companies do not have a whole network with its own emissions and value. The entire income from company audits is spent on maintaining employees, while in Callisto, in addition to auditors, there are still many aspects of the project.
- Callisto will not be able to fulfill its main mission if we start charging for audits, as in this case there will be projects that no one will audit or request an audit for and this projects will possibly cause damage to the entire crypto industry in the same way as it was with TheDAO or Parity Multisigs.
Current stage: there are 4 full-time auditors in Callisto. You can observe the workflow and statistics in an open google table or at github.
4. The issue of incentivisation.
There is another fundamental problem in Callisto. Auditors are not interested in following the rules of the Callisto Security Department if they find a serious mistake. Callisto Auditing Department system is designed so that auditors should not know each other. The only incentive for auditors to follow the rules of Callisto is not knowing what mistakes other auditors find. Penalties for failure to find errors may exceed the chance of success if the auditor decides not to report an error and exploit it after the audit completion.
In real terms, there are not so many auditors and they know each other, which is a consequence of the problem (3) “Competitiveness and scalability”.