This article addresses an important challenge in artificial intelligence research in the humanities, which has impeded progress with supervised methods. It introduces a novel method to creating test collections from smaller subsets. This method is based on what we will introduce as distant supervisiontextquoteright and will allow us to improve computational modelling in the digital humanities by including new methods of supervised learning. Using recurrent neural networks, we generated a training corpus and were able to train a highly accurate model that qualitatively and quantitatively improved a baseline model. To demonstrate our new approach experimentally, we employ a real-life research question based on existing humanities collections. We use neural network based sentiment analysis to decode Holocaust memories and present a methodology to combine supervised and unsupervised sentiment analysis to analyse the oral history interviews of the United States Holocaust Memorial Museum. Finally, we employed three advanced methods of computational semantics. These helped us decipher the decisions by the neural network and understand, for instance, the complex sentiments around family memories in the testimonies.