Congratulations to Sam Hepenstal, Leishi Zhang and William Wong, whose presentation titled ‘An Evaluation of System Transparency to Support Intellectual Judgements’ was awarded ‘Outstanding Oral Presentation’ at the Middlesex Research Student Summer Conference (RSSC) 2021! Below is the presentation abstract:
Artificial Intelligence (AI) systems can aid criminal intelligence analysis, for example by speeding up the retrieval and processing of large amounts of information. However, AI systems are typically opaque, obscuring the underlying processes that inform outputs. This is a problem when analysts make critical decisions and need to be held accountable. Therefore, past research on system transparency has emphasised ethical concerns, such as the needs for justifications and audit. In this paper, we demonstrate that transparency is also critical to supporting intellectual choice, with significant implications for insight. We report on a qualitative study with 10 intelligence analysts who performed an investigation using an AI conversational agent system. Half were provided with system transparency. Here we analyse differences in insight, understanding, and requirements expressed. Our results show how transparency is needed for tasks involving intellectual judgements. We find that, unless clearly presented, the importance of transparency is underappreciated by users.