As a part of COMP204P/COMP205P System Engineering modules in my 2nd year of study in UCL I've been given a chance to work with Microsoft HoloLens and I became a part of the PEACH project, namely the Reality group. In this article I'll be talking about PEACH Reality itself and the things I've done during my team working on it.

PEACH Reality

PEACH stands for Platform for Enhanced Analytics and Computational Healthcare, and it's aim, quite unsurprisingly, is to bring some wonders of computer science and software engineering into healthcare. PEACH Reality, the project I was working on, has several major goals.

First of all, we're planning to define a specification and an appropriate file format for a holographic patient case. Such cases will be used to store models, DICOM files and relevant meta data, such as coordinates of models in 3D space or voice recordings attached to various parts of the model (which is done using data markers).

Secondly, we're developing a robust Microsoft HoloLens application that will be paired with a web app to make use of the specifications defined above. Using our web app, medical specialists can upload 3D models, CT scans and various other files which can the be viewed and interacted with on HoloLens. Once the user is done making changes to objects on HoloLens, all information will be serialised and can later be exported using the file format described above.

Last but not least, we're working on a seamless pipeline that will automatically convert CT scans into 3D model. Thanks to PhD students from UCL Computer Science and Innersight Labs we have access to a neural network that can do exactly that. The resultant 3D models are not 100% accurate but the accuracy is high enough for a proof of concept.

As the leader of the team working on this project I'm responsible for supervising the whole development process. Although this project requires quite a bit of effort it's been very interesting so far and I got a chance to meet a lot of great people, ranging from experienced surgeons to the corporate VP of Microsoft. I'll be posting updated in this blog as the project progresses further.

Demos

Here are some videos showing off various features I've been adding to our app prototype. The most recent videos are at the top.