Previously I discussed concerns over developing ethical AI systems and used Microsoft’s chatbot Tay as an example. I argued that an algorithm learning from humans will likely result in unethical behavior. Coincidentally my colleagues who wrote the report “The Frankenstein Factor” also used this example, however in their view internet trolls were to blame for Tay’s racist slurs.
A similar disagreement could be seen in the media coverage of the WannaCry attack. While many blame Microsoft for not providing the security update that would have fixed the vulnerability free of charge for out-of-support Windows versions, Microsoft blames the NSA for not communicating these vulnerabilities and exploiting them for intelligence purposes. Others choose to blame operations personnel for not installing the security update in time. The affected users however cannot access their files and do their work, despite the fact that they’ve paid good money for their OS and its maintenance.
These examples highlight a radical difference in the way responsibility for IT solutions is viewed. IT companies tend to view the responsibility for the way their products or services are used as external. They provide a solution and stand for its quality, but then it’s up to clients to use those solutions responsibly. However clients, both on business side and within society, tend to expect suppliers to take additional responsibility to prevent misuse of their solutions.
Another great example of this can be seen in ongoing media coverage on digital advertising. Both in the US and in The Netherlands internet users have publicly called out organizations advertising on sites containing material supposedly not in line with the company’s image. As a result multiple organizations have revised their online advertising strategy because of this. It’s conspicuous that it took attentive and vocal internet users to come to this.
Clients regard online advertising as similar to offline advertising, where visibility usually is the only factor in play. Traditional advertising companies would make sure not to advertise in places that may cause a liability to the reputation of their clients. Online advertisements however may often be placed on any website, including those dedicated to content that clients may not approve of. Online advertising companies generally do not feel responsible for where they place their clients’ ads, beyond providing clients with an opt out for specified sites.
This example shows more clearly where the root of the problem lies. Confidence is based on implicit expectations of a mutual trust, however those expectations are radically different in the offline world. Traditional companies are used to operating within an environment where the law places restrictions on how businesses operate. They take full responsibility for the products and services they deliver, otherwise they may be subject to litigation. Localized companies and family operated companies may even go one step further and concern themselves with supporting the community they operate from. These companies expect their digital partners to operate the same way. They consider them fully responsible for delivering their products or services, whatever those may be, and the consequences that may follow from that delivery.
However, like the internet, the IT sector has never known much regulation. It has traditionally been rooted in a culture of innovation, sharing and freedom. The flipside of this coin is that in a business relationship, each party is considered responsible for their own activities. They consider the customer responsible for specifying exactly what their requirements are regarding the products or services delivered. If those requirements turn out to be insufficient, that’s not the suppliers fault.
This difference in expectations often comes to surface only when things start going wrong, as we’ve seen with the examples above. It then becomes necessary to clean up after the fact, which is what Microsoft, ad companies and their clients have had to do. Unfortunately the knee-jerk reaction from IT companies is often to deny responsibility. They hold clients and users responsible for the way they make use of IT products and services, even if that caused major problems. This causes major damage to the mutual trust between clients and users and their IT suppliers.
Consider also Facebooks initial reaction to claims that fake news may have influenced the US presidential election of 2016. While Facebook denied responsibility for the content placed by their users and expressed reluctance to make attempts at resolving this problem on grounds of free speech (and technical complexity), polls amongst voters showed a significant percentage using Facebook as their primary source of news.
Excessive media coverage of such incidents may lead to a general loss of trust in the IT sector within our society. Companies may even lose confidence in their own digital strategy because of this. In my view all this can be avoided if all involved parties share their expectations about IT towards each other. This is exactly where testing can make the difference.
After all it is our goal as test professionals to establish confidence in IT based on concrete results. Expectations of all involved need to be aligned until no gaps in responsibilities exist. Testing for those expectations must involve users and stakeholders alike so a shared mutual confidence can be created. This requires assurance and testing activities in all phases of a product lifecycle, from the inception phase right up to and including operation and maintenance.
It starts with requirements lifecycle management. Assuring that business requirements are clear, complete and actual helps to create clarity on the expectations placed on IT suppliers. Another critical activity is the product risk analysis. This not only provides invaluable input for any test strategy, it also helps all involved think about what may go wrong when the product or service is live. When everything goes well people like to take responsibility, however when things go wrong we’d rather avoid to do so. It is therefore imperative to consider this before delivery to make sure those responsibilities are clear should an incident occur afterwards.
Test plans and specifications help make those situations less abstract. When rainy day scenarios are described including expectations of what behavior may occur, all involved can see for themselves what the consequences of requirements or design decisions can be. Such insight in the way a solution may work out in real life enables productive discussion on preventing or mitigating potential incidents before they occur. Test execution and reporting will then establish the confidence that the product or service delivered meets the expectations shared throughout the project .
In my work testing a solution for executing queries on an aggregated customer information database we helped the project to considered a vast amount of rainy day scenario’s in the design phase. For example information could be missing, invalid or inconsistent between different tables. We shared our test scenario’s and result expectations with the project and the users. This lead to an agreed expectation of how these scenario’s should be presented to the user. After test execution we shared our test results with the users for inspection and approval. As a result they rarely encountered unexpected situations after go-live. When these did happen we all agreed we would help them analyze those results and provide a tested change if necessary.
From our unique perspective as testing professionals we can thus help IT take the responsibility that clients expect us to take and work together towards a fulfilling digital society.
Niek Fraanje is a Test manager working for Sogeti in the Netherlands
Further reading: https://www.ict-books.com/topics/vint-report-mi3-en-info