Didn’t Catch That

In my second year of study I completed a semester of work supported by Microsoft. They were interested in CUIs (Conversational User Interfaces). CUIs go beyond just ‘voice recognition’ or ‘speech activated’, they attempt to emulate the human flow of conversation. This can be focused down to query and response. The same way I can ask a friend, ‘Hey, what is the time?’ and they respond with ‘Oh, it’s half three.’. Microsoft were interested in the applications of this kind of technology. We formed groups and got to work.

My group had a common interest in recording data. Whether it be notes in a meeting, a voice recording of a summary of an event or event just the ambient sound in a place at a time. The first thing we decided to do was to see what people write in their journals and a nice way to get a baseline for this was to journal ourselves. So for the next week we each took a daily journal. From this we realised that there are a few different types of journalling.

From this we made a set of notebooks designed to be left around the University, Dundee and beyond. They were to be picked up by a passer by, written in, then left somewhere else for another person to find. When the pages were full or when a certain date was reached the books were to be returned to the university for analysis. In reality we got one notebook back but the entries were amazingly varied and valuable.

Alongside this serendipitous research we also filmed interviews with various people asking them about how they record, store and recall memories. The best insight from this research was how this lady stored memories of her wedding. This prompted the question, how could we record the experience of planning a wedding and getting married in the near future with CUI. We began to imagine a product that would be worn on the body that would allow easy recording of experiences to be recalled later.

We created a prototype glove that focused on how the user would ‘start’ the recording or ‘save’ a moment. We identified that one of the best places to do this was the first knuckle on your index finger as it’s almost always accessible and is rarely accidentally touched. This came off the back of some wacky prototypes like little rubber finger sheathes designed to see how comfortable people would be with wearing something intrusive on their index finger. Eventually we decided on a ring as it’s familiar, subtle, portable and can be designed to appear in different ways.


Once we had decided on what the product that recorded the information was going to be we then moved our attention to how this information would be recalled, the same day or ten years from now. After some wild experimentation we settled on using existing filesystems but changing the way we navigate them. This is my favorite part of the project and something I thought would appeal to Microsoft. I created a visual filesystem that you could navigate using voice queries. We have been using the mouse and keyboard, almost unchanged from when they were first conceived, yet the complexity of data we navigate daily has increased exponentially. I find this unacceptable and a new solution should be made.

Imagine you could ask your PC/Laptop/Phone, ‘Show me all the photos I took with my Husband in Belgium last time we were there’ and bam, those photos instantly appear. It would make showing off photos at parties much easier, no more searching through your whole phone for five minutes while the conversation moves on and no one is interested anymore. Imagine a filesystem that could help you. ‘When was the last time I saw Kevin’ ‘three months ago, July 22nd.’ The metadata already exists to make this possible, we just haven’t designed a way to make it useful to the user yet.

We then set about creating a short video presentation to present such a complex idea. We were still struggling how to explain what our product actually was even in week 11/12. It was a big task to consolidate what we were actually trying to do or say. I still think we were unsuccessful to this day, I’m struggling to explain what it is in a text only format. It’s something that is easily understood when experienced but that’s not really good enough as a short presentation.

We decided that the easiest way to explain our product was to give an example of it in use. We managed to get a friend of ours to take part in not only our film but to share her story as well. We compared how she was currently planning her wedding to the process one of our older research participants went through. We displayed them side by side as if they were happening simultaneously, highlighting how this process was done in the past and how our product changed that in the future. I think the final product comes across as quite cheesy but if you can look past that it does it’s job well.

The presentation day was one of the first big presentations we had done as year group. There was a great supportive atmosphere, everyone wanted everyone else to do well and impress Microsoft. I think it was the best work we had done up until that point and everyone exceeded my expectations. It was a very positive day and our product, ‘Catch’ came third overall. Very proud of what we had achieved as second years. The only downside was that the Microsoft staff did not have time to give feedback which was quite deflating after 12 weeks of work and presenting a product you really believe in.

It’s a project that is always in my mind. It would change so much about the way we interact with technologies. The amount of people who struggle with mouse and keyboard input from motor issues or other reasons. It just excites me so much, It fits into any electronic device with a filesystem. It could make lots of process easier, faster and less stressful. I believe that a lot of negative feelings exist because of how frustrating our technologies are to use. As we make these devices not only more important to the function of our society, but also as part of our personal identity, this archaic interface of mouse and keyboard must be changed.

I want to develop the prototype further but I don’t know if I’ll ever get the opportunity to present it to the correct people at the correct time. I may enter another entrepreneur competition in the future with this product.