Any views expressed within media held on this service are those of the contributors, should not be taken as approved or endorsed by the University, and do not necessarily reflect the views of the University in respect of any particular issue.
Between the 18th and 26th October we were exhibiting our VR experience as part of the Manchester Science Festival, one of the biggest science festivals in the UK. We were situated in the magnificent Central Library between the music library, 7000 feet and an exhibition of images from Manchester’s rock history.
The main experience we were showing the public was a virtual reality reconstruction of St Cecilia’s Hall, Edinburgh, in the years 2018 and 1769. Prior to putting the headset on, participants were explained the cultural heritage and significance of the hall, giving them a deeper understanding of the purpose of our research.
The diverse number of people who were interested in trying the VR experience was truly staggering. One of my favourites was a young man of about 19 who hadn’t used VR before and was exceedingly sceptical that he would enjoy our exhibition. After 5 minutes, first in 2018 and then travelling back to 1769 he was a full convert to virtual reality, lifting the headset up and saying, “This is the future…you need to tell people about this!”
During our time in Manchester we had well over 250 people experience the VR reconstruction. The majority of people had never user VR previously and were intrigued to learn more. Before the headset was placed on each participant, they were asked to familiarise themselves with the environment and in particular listen to the music. Once comfortable, the virtual scene was changed from St Cecilia’s Hall in its current state to how it is believed it would have looked in 1769.
Participants were asked to focus on the music and see if they could hear a difference between the past and present. The majority did. They were also asked if they could describe the difference. Words like ‘colder’, ‘clearer’ and ‘brighter’ were used to describe the 2018 model and ‘warmer’ and ‘natural’ for the 1769 model. Participants were asked to complete a questionnaire following their experience. The results of this is given below:
I enjoyed the experience – 96% agreed
I would use the virtual reality experience if it was available within a historical venue – 93% agree
This has enhanced my understanding of how the space(s) have changed over the years – 86% agree
I could hear the difference between the present and the past in virtual reality – 81% agree
I enjoyed hearing the music and appreciate its historical context more now I have heard it within a historically accurate virtual reality reconstruction – 88% agreed
I would like to see more virtual reality reconstructions of spaces in heritage sites – 95% agreed
I would recommend using this system to others – 95% agree
Clearly, looking at these results, it wasn’t just us that were enjoying ourselves at the Science Festival! The responses show an overwhelming desire to see more virtual reality reconstruction of heritage sites, experiencing music of the past as it was intended to be heard and it gave them a heightened appreciation of the venues.
Our experience at the Manchester Science Festival was an unbelievable success! Everyone seemed to enjoy hearing about our research, the venues we are looking at, the music we have chosen, how it was recorded and how it was presented – and we enjoyed sharing it with everyone who attended!
At 6am, Wednesday 10th October 2018 I had taken on board coffee and porridge and was stood waiting for my train to Dundee, where I was getting a connection to Stirling. The DigiDoc 2018 Conference Research and Innovation Day was due to start at 9am at the Engine Shed in Stirling; James and I were presenting our work. The Engine Shed is the home of all the digital innovators of Historic Environment Scotland, one of our partners in our project recreating musical performances of the past.
We were in the first session and immediately before us was our colleague Daniel Wheeldon presenting his work on 3D Digital Modelling for the Reproduction of Historical Musical Instruments. It was fantastic to see his work recreating a guitar-like instrument that is played by keys with a similar mechanism to a piano.
On next, we presented our work on recreating two historical spaces – Linlithgow Palace Chapel in the early 16th Century and St Cecilia’s Hall in Edinburgh in 1769. The significance and importance of these places is described in our website.
Our talk was well received and, judging by the response on twitter, two of the biggest take away points from the audience seems to be the concept that “sound and image together is important to recreate the experience” and my question of “what does a tallow candle sound like?”!
The day that followed was full of fascinating talks including another Edinburgh University project to create a digital record of the Mahābhārata – beautiful scroll is one of the longest poems ever written, containing a staggering 200,000 verses spread along 72 meters of richly decorated silk backed paper. Scott McGibbon from Heriot-Watt University presented his work on digitally documenting buildings for the purpose of maintaining and repairing historic sites.
After lunch Bob Marshall wowed us all with his graphical reconstructions of Windsor Castle throughout the ages. Bob created all his impressive 3D models using the open source programme Blender, with textures from Substance. Even with the high-quality textures from Substance, Bob’s artistic touch raised the models to an even greater level.
Chris Landorf presented her work on the Windmill Tower in Brisbane, Australia. This was a really enlightening presentation where we learned that there is very little wind in Brisbane and it seems that before too long it was the convicts who were powering the mill. The importance of emotion in history and how this helps connect us to the people and places of the past was presented by Maria Economou and Hilary Young from Glasgow University. Their case study was based on a soldier posted to the Antonine Wall and the decisions he would have had to make when the wall was being abandoned.
In the final session of the day Fergus Bruce talked us through the great work being undertaken by the Soluis Group. Li Sou described her PhD research where she has been carrying out investigations into Shetland’s Iron Age archaeology using digital documentation and historic survey records of a broch.
It was around this time an empty train derailed at Stonehaven – I don’t know why… more on this later…
After the final talk from Lyn Wilson from Historic Environment Scotland discussing digital developments over their vast and impressive array of sites, there was a drinks reception. Sadly, I had to rush off to see if I could get home on the train. James forced himself to stay for the wine and got to chat to all the great people who were attending.
The next day started reasonably smoothly. No trains were running north of Dundee due to the derailment so my wife had driven me there to catch the 8.15am train back to Stirling. Imagine the feeling of resignation that flowed through Dundee train station, the only station in the world which has undergone a £38M redevelopment yet has less customer facilities than before.
I got to Stirling eventually but sadly missed some of the opening session. But I did get there to see Greg Downing’s awesome presentation about lightfields. The 3D images were spell binding and like nothing I had seen before.
The second morning session focussed on digitising and documenting artefacts. The Fraunhofer Institute were successfully capturing 3D images of historical artefacts on an industrial scale. A conveyer belt with numerous scanners and cameras recorded every detail. The presentation by Vince Rossi from the Smithsonian reminded us just how big the Smithsonian is and the amount of documentation required to keep track of all their commodities.
The first session of the afternoon started with Steven Ramsey from 4D Max asking everyone to put their hands up in the air. His first direction was to lower your hands if you had seen Iron Man 3. The majority had. Then a few more well-known titles were mentioned, Lucy, Gravity, and after about 4 films all hands were done. They had worked on them all — and about another 100 titles that flashed up on the screen! It was great to gain an insight into how all these visual effects are added and how many more effects there are than I had realised.
The second afternoon session contained some of the presentations that had the biggest impact on me and the work we are carrying out. Catherine Allen’s presentation was an excellent insight into how to present VR experiences to the public. Try to have a chair, try to keep the participants to adults, have groups book in advance — great advice just before we head off to the Manchester Science Festival.
Paul Chapman from Glasgow School of Art was next, and he had some great take away points including the argument that: “I hear and I forget”, “I see and I remember”, “I do and I understand”. He then went on to explain the great work he is involved in training people in industry using VR technology to assist them, saving time and giving access to expert knowledge when required.
After the final session and a bit of setting up, the highly anticipated whisky tasting session started. James, who had to teach back in Edinburgh during the day, found it in himself to attend the evening and join in [the lengths I go to for this research project… James Cook].
There were four separate malts available to taste — a Highland, Speyside, Glaswegian and Islay. The whisky master, Jock, was keen for us to try them all and to educate us on the history of each. Mid-evening, Paul Chapman launched the excellent book, ‘Art of the Point Cloud‘, edited by him, David Mitchell, Chris McGregor, Lyn Wilson, FSA Scott and Alastair Rawlinson. By this time we were getting fairly well educated on the whisky while enjoying the point cloud projections on the side of the building.
After an insightful conversation with Jock about whisky, James IV and our VR project, Jock informed us that this was the first event he had attended where the delegates had drunk all his whisky. With an over-inflated sense of accomplishment, we headed to the ceilidh to try to remember the moves. As the only Scot out of the three of us the pressure was on….
Friday morning arrived without having to fight my way to Stirling via the train lines. A bonus since the line was still shut north of Dundee. The highlight of the morning session for me was Martin McDonnell of Soluis asking the audience what superpower they would most like to have. A number of days later I’m still thinking about this and still not made my mind up. Martin then showed us a number of models they had designed and created which had then been built in reality. Incredibly accurate in their detail, the difference between the models and what was then actually built was minimal.
Martin then went on to discuss the options of shared VR experiences. One of the main pieces of hardware used for this is their dome. In this, a 3D video is projected onto the inside of the dome and a number of participants can enter the space to experience it together. I was wondering if they would use some sort of ambisonic sound system to capture the spatialisation of their 3D worlds but never got the chance to find out. I will though!
After lunch Emily Smith from Atlantic productions brought to our attention all the great VR experiences they have been working on over the past couple of years. This included an underwater experience with David Attenborough, returning to earth with Tim Peak, and entering black holes with Stephen Hawking. All of these looked amazing and on the list of must-do experiences!
Emily was followed by Ben Von Zastrow from Tippett Studio. Ben is a visual effects artist who has worked on a number of cool films, beginning his career on Star Wars. In the presentation, he discussed how he has linked two pieces of film footage together using 3D modelling. He also showed graphical footage of a tourist attraction in China and how it compared to the actual images captured when he visited the location.
The final presentations of the day were by Maxime Durand, a historian with Ubisoft and Thomas Flynn from Sketchfab. Maxine is a historian and discussed the historic details placed inside the game Assassins Creed. Thomas gave examples of the array of 3D models in Sketchfab and how this is used by 3D artists.
At the end of the conference it is fair to say we were brimming with inspiration. Each talk was unique, fascinating and accessible for everyone. We put faces to names and names to faces; hopefully we will continue to collaborate with a number of outstanding innovators who we saw. I now can’t wait until DigiDoc 2019…..
The conductor on the train home informed me the trains were still not running north of Dundee and replacement busses would be offered. They weren’t.
One of the most exciting aspects of working with The Binchois Consort has always been the necessity to make a decision, to come to a conclusion, and to present something complete and coherent to the public in concert or CD recordings. It is a challenge – do doubt – but one that can be refreshing for someone used to working in a world of scholarly caution. To varying degrees, the same challenges are present here in this project: how can we present something which is engaging, exciting, and yet still founded in the historical record?
For Linlithgow Palace (for more on our choice of venues, see here) the problem is particularly intense. We are seeking to reproduce the acoustic and visual environment at the turn of the 16th century. This means that we are focusing on the Scottish sacred repertoire of the fifteenth century for our core repertory. Fifteenth-century repertory is, of course, varied and exciting. Those of you who enjoy Early Music may be wetting your lips at the thought of Dunstaple, Frye, Du Fay, Binchois… some of the biggest names in Early Music and all from this period. But none of them are Scottish. If you are struggling to name their Scottish contemporaries, don’t be embarrassed. I am too. Indeed, almost all that survives of Scottish polyphony from this period can be found on a slate dug up in a drain in Paisley abbey. Twenty notes. One voice. No text. Hardly an inspiring selection of music around which to build a VR experience. And yet, we know that music took place frequently in the chapel at Linlithgow palace, provided by members of the Chapel Royal.
For our reconstruction, we initially based our musical choices around a known historical event. In Easter of 1512, James IV stayed in Linlithgow. It was here that his son, later James V, was born. James IV celebrated Mass on Easter Sunday in the chapel and, following this, James V was baptised in the same place. Thankfully, we know what liturgy Scotland followed in this period. Like much of England, it was based on the Sarum Rite, though the sources of the chant themselves were often imported from France rather than South of the border. Whilst this was later replaced by the Aberdeen Rite (itself almost indistinguishable from Sarum), it was followed exactly during the period we seek to reproduce. This means that we know exactly what chant James IV would have heard during Easter Sunday in 1512. Of course, Easter Sunday is an important event, and royalty are used to the finer things in life. It is very probable that much of the Mass would have been adorned with magnificent polyphony, which is now sadly lost to us. Our approach to this issue has been two-fold. Rather than add obviously incorrect polyphony we have instead chosen to omit the sections, such as the Kyrie, where this would have been heard. What we present instead are some of the sections that would, in all probability, have remained as chant. We have also used a practice known as ‘fauxbourdon’, as described by the ‘Scottish anonymous’, to create a form of quasi-improvised polyphonic scaffold around some of the chant. We know this practice took place in Scotland and was a way of elaborating chant, without the need for specifically composed polyphony. This is presumably the approach that would have been taken by someone finding themselves in our position in fifteenth-century Scotland!
We wanted to give some indication, though, of how the chapel would sound adorned with the kind of magnificent polyphony that must have taken place there. For this, we turned to the Carver Choirbook. This source was owned by the very same singers that we know came to Linlithgow palace chapel to sing, and it clearly contains, within its oldest sections, some music from the turn of the century. Its most famous passages come from the pen of Robert Carver, but these are largely too new for our purposes. Nonetheless, within its older sections we are presented, for instance, with a collection of music that probably made its way to Scotland with Margaret Tudor as she travelled for her wedding in 1503. It is all English, with many concordances with the Eton Choirbook. Our attention was drawn to a section that must be older still, however, and particularly two anonymous Mass cycles that must come from the 1460s-1470s. These have been accorded English provenance, but they clearly formed part of the Scottish Chapel Royal’s repertory. For our recording, we chose the Sine nomine Mass with troped Kyrie ‘Ave Rex Virginum’, which would have been appropriate for any Marian celebration in Linlithgow. Having got to know the piece better, I am now increasingly of the opinion that this work is not English after all and that it, and its twin preserved alongside it, are instead Scottish in origin. This would make them the earliest Mass cycles in Scotland by some 50 years and increase the Scottish repertoire of this period by several thousand percent. The evidence for this deserves its own special consideration and I won’t go into it further here, but watch this space!
The music for Linlithgow palace was recorded, in our anechoic chamber, by our friends The Binchois Consort. We made a valiant attempt to reproduce the pronunciation of fifteenth-century Scottish Latin for our recording. See below for a photo of the consort in the interrogation anechoic chamber.
St Cecilia’s Hall is, to an extent, a somewhat easier prospect. The Edinburgh Musical Society, who built the hall and ran its concert series, very kindly preserved records of the concerts and repertoire. We have taken the conceit of reproducing a typical first act of one of their concerts. These began with a periodical overture and were followed by a couple of songs by visiting singers. Whilst we have a list of the periodical overtures in use by the society at the period, the songs were chosen by the visiting singers and we have less idea of what these may have been. We have chosen, from their plan books, the periodical overture XIII by Lord Kellie. This piece was performed more times than any other periodical overture in the decade in which we are situating our reconstruction and it was composed by the lead violinist of the orchestra during this period. Following the overture, we asked Dr Jenny Nex, who is also the curator of St Cecilia’s, to sing two songs for us accompanied by Dr John Kitchen on Harpsichord. She chose to sing Johann Christian Bach’s, Blest with thee my Soul’s Dear Treasure which was published in Edinburgh shortly after the period we are reconstructing, noting on its first page ‘Sung by Mrs Stuart in St Cecilia’s Hall’ and ‘The Lass of Peaty’s Mill’, from The Scots Musical Museum, since Scots songs were also popular material in the hall at the time.
We recorded each of these pieces with attention to historically informed performance practice, using historical pitch, approaches to ornamentation, and instruments. The effect is quite striking, and particularly effective within our reconstruction of the acoustics of the time.
This project brings together cross-disciplinary expertise from a range of academic, industry and cultural partners to explore the point where performance practice, gaming and virtual reality (VR), technology, and culture and heritage meet. We use 3D imaging, binaural and surround sound, and room-impulse responses to create a software application that allows users to experience the performance of Early Music in an accurately-modelled historic space.
Listening to music is an experiential activity that connects listeners to their surroundings and to those around them. In part, the recent growth of the live performance industry is a direct consequence of this need to connect and share musical experiences in a communal space.
But the transient nature of live performance presents real challenges. The physical properties and locations of spaces impose constraints on the nature of events and the geographical reach of performances.
While recordings can capture the sound of performance, they also stop short of allowing the listener to feel a sense of presence and participation.
This is a challenge that is only amplified when one considers Early Music performances: even a curated performance in a modern venue loses much of the detail that characterises historic performance.
