Should developers be responsible for potentially harmful software?

With the recent news concerning the arrest of Marcus Hutchins — the security researchers who stopped the biggest-ever ransomware attack — I’ve been thinking (more than usual) about the ethics of software development. But before we go into that, who’s Marcus Hutchins and what did he really do?

Hutchins, is a so-called security researcher which means he’s testing and developing software that will help with understanding and stopping computer viruses, malware, ransomware etc. When the Wannacry ransomware spread across the globe Hutchins was the first one to stop it. He is now arrested by US authorities because he developed another malware that someone else used in attacks against banking software. Hutchins supposedly developed the malware for research purposes, with clear and good intentions. But despite this he’s now being arrested.

The case around malware is obviously a grey area, because the primary intention of malware is to cause harm (although there are malware used with good intentions). Should it be allowed to develop malware? It’s a reasonable question to ask, however, my answer to that question is yes, as long as there’s no ill intent. But where do you draw the line? How do you define purpose and intentions? These are mostly subjective questions, so it depends on who’s asking and who’s answering.

I applied this question to something relevant to myself. Because I’m a core maintainer of Drupal, should I be held responsible for criminal use of Drupal, e.g. distribution of illegal drugs or child pornography? In line with my answer above, I should probably not be held responsible. But should criminals be allowed to use Drupal for criminal purpose? Obviously not.

But as a contributor to Ethereum, the next questions became a bit harder to answer for myself. Ethereum by design makes anonymity and financial transactions easy, which in turn lends itself really well to illegal use. Tax evasion and money laundering, just to mention two obvious examples, could have dramatic consequences to our society if blockchain technology like Ethereum become mainstream. Vlad Zamfir’s blog post titled “Blockchains Considered (Potentially) Harmful” is a very interesting read on this subject.

The conversation around harmful use of blockchain technology needs to be approached slightly differently to the general conversation around privacy and encryption on the Internet. Transport Layer Security works on such a low infrastructure level that it’s not reasonable to consider it harmful in my opinion. But blockchain technology such as Ethereum works on a higher level with very direct and obvious applications for illegal use.

I will obviously continue to contribute to both Drupal and Ethereum because I believe that both of these projects ultimately are enablers of digital freedom and privacy, rights that otherwise are being taken away from us by corporations and governments.

But this is still an important conversation to be had. How do we define and communicate intended use of the software we distribute? Should we even need to? Where do we draw the line on subjective matters like these? When robotics and artificial intelligence comes into the picture the potential impact of software development can have even more drastic consequences.

4 thoughts on “Should developers be responsible for potentially harmful software?

  1. We don’t sue gun manufacturers for murder. We don’t sue car manufacturers after an accident. I think we have to consider the manufacturer of the good blameless from a legal perspective. It’s up to the operator to use responsibly. So I think ethereal and drupal makers are safe and blameless. The malware maker is slightly more problematic.

  2. In my opinion, responsibility is the personal response to a demanded behaviour, based on a generally accepted value system in a society.
    To this extent, it is determined by the desire of a person to belong to it.
    But who defines these values and their benefits? Who has the sovereignty of interpretation? What is the benefit of these values, for whom and for how long?
    Responsibility is, on the one hand, retrospective (for past acts), prospectively (for future acts), as well as rectification. Responsibility therefore includes always the estimation of the consequences of action.
    And yet, not everyone is responsible for everything, because of their different competences and the therewith associated possibilities to estimate the consequences of acting.
    On a micro level, at the level of personal activities, one also has to bear a direct responsibility and to balance it with regard to the fulfillment of responsibilities and avoiding risks.
    In the case of the programmer mentioned by you, I am not sure whether he could not have foreseen the possibility to use its software for not so “nice” purposes, if he did not estimated the consequences of its development adequately.

Leave a Reply