System Architecture

SARAS’ workplan is decomposed into four clusters of WorkPackages (WP1-2, WP3-6, WP7 and WP8-9), visible in the Pert diagram.

The clusters are the following:

  • Surgical knowledge, specification and validation (WP1-2), where medical specifications and requirements are settled and translated into engineering specifications;
  • System architecture (WP3-6), in which the technologies enabling the abilities needed by the SARAS system are developed;
  • System integration (WP7): here SARAS’ innovative tools, algorithms, methodologies and hardware are integrated in a seamless way, and validated by surgeons in real-world clinical scenarios.
  • Communication/Exploitation (WP8) and Management (WP9) complete the picture.

 

 

 

Work package descriptions

 

WP1 Surgical procedure specification and validation

Objectives

• To define the clinical, technical and safety requirements for radical prostatectomy and partial/complete nephrectomy
• To develop and validate the workflow models for the surgical operations
• To identify the main risks related to the specific tasks and strategies for their mitigation
• To identify the benchmark tests to perform and demonstrate the surgical procedures
• To validate the SARAS system on the synthetic phantom models and on Thiel soft embalmed human cadaver models

 

 


 

WP2 Anatomical modelling, artificial and Thiel embalmed human phantoms

Objectives

3D artificial phantom models:
• To develop anatomically accurate complex 3D virtual reconstructions, as computational models of features of body, organs, bones, vessels and soft tissues
• To develop anatomically accurate complex 3D synthetic tissue mimicking material (TMM) physical phantoms of features of the body, organs, bones, vessels and soft tissues
• To develop appropriate anatomical abdomen phantom assemblies for MIS radical prostatectomy and partial/complete nephrectomy based on clinical imaging data with respiratory motion and pulsatile flow in arterial perfusion of organs
• To develop artificial free breathing and perfusion models of the abdomen that accomplishes the requirements of testing the SARAS robotic system

Thiel soft embalmed human cadaver models:
• To develop synthetic abdominal phantom carcases with artificially perfused explanted porcine organs embedded

• To develop Thiel soft embalmed human cadaver models with re-perfused kidney and respiratory motion

 


WP3 Multi-modal human-robot interface and bilateral teleoperation

Objectives

Multi-modal human-robot interface:
• To design the assistant surgeon’s interface for the MULTIROBOTS-SURGERY platform
• To improve the da Vinci master console by integrating multi-modal feedback (e.g. force feedback, virtual fixtures, interventional checklist, vocal commands) and by allowing a more natural interaction with the environment and with the SARAS assistive robotic arms in the SOLO-SURGERY platform
• To develop an easy-to-use interface for the main surgeon using traditional laparoscopic tools and the SARAS assistive robotic arms in the LAPARO2.0-SURGERY platform

Bilateral teleoperation:
• To develop the bilateral teleoperation system for allowing the assistant surgeon to teleoperate the SARAS assistive robotic arms
• To develop the multi-master multi-slave (MMMS) bilateral teleoperation system for allowing the main and assistant surgeons to perform R-MIS interventions together with the SARAS MULTIROBOTS-SURGERY platform


WP4 Multi-robot Cooperation and Task Planning

Objectives

Task planning:
• To develop novel algorithms for robot motion planning in a dynamic and deformable environment, taking into account moving obstacles and kinematics constraints related to the MIS domain
• To implement the motion planning algorithms within a reconfigurable control system, able to cope with online specification of obstacles and constraints (e.g. trocars)

Multi-robot cooperation:
• To develop a Multi-Robot Cooperation Platform (MRCP) for Task Oriented Teleoperation to assure an optimal cooperation between the main surgeon and the SARAS assistive robotic arms by addressing context awareness, safety issues and by enhancing virtual reality techniques
• To model the task to be performed autonomously by the SARAS system by assigning subtasks to the assistive robotic arms and to monitor the execution to prevent interferences with the main surgeon tasks
• To develop repulsion strategies (in automated mode) and forces (in teleoperated mode) around the main task working volume in order to inhibit or modify trajectories that may lead to incompatibilities or dangerous configurations between robots actions
• To create areas of interest, like potential points of attraction, to determine priority operating areas


WP5 Cognitive control, Task supervision and Computer vision

Objectives

Computer vision:
• To produce an up-to-date 3D reconstruction of the anatomical scene involved in the laparoscopic procedure using active SLAM algorithms
• To register the position and orientation of the surgical tools and the SARAS robotic arms with respect to the reconstructed anatomical scene
• To track the motion of the rigid objects involved (tools and robotic arms) within the scene, and implement a fine tracking of the deformable anatomical surfaces present
• To recognise, detect and label all scene elements based on prior knowledge on the tools, learned appearance models acquired during training, and direct vocal messages from the surgeon

Cognitive control and Task supervision:
• To develop the models of the surgical knowledge for the control of the SARAS system by translating workflow description into control-based actions
• To design the supervision architecture to control and monitor the execution of the task by the SARAS assistive robotic arms taking into account human-robot and robot-robot interactions


WP6 Reasoning and situation awareness

Objectives

Reasoning:
• To learn a flexible spatiotemporal representation of complex laparoscopic procedures, and estimate at each time instant the stage the procedure is in, based on present tools and surgeon activity
• To recognise in real-time what activity the main surgeon is conducting at each given moment
• To foresee, given the current stage, what action(s) the surgeon will likely perform next

Situation Awareness:
• To detect possible deviations from the normal unfolding of the procedure (nervousness of the surgeon, unexpected presence of blood, dangerous contacts) in order to issue prompt warnings
• To select the appropriate action(s) for the SARAS assistive arms to take based on procedure stage, surgeon’s current behaviour and future intentions and contingent alerts

 

 


WP7 System integration

Objectives

Development of SARAS Assistive Robotic Arms:
• To translate the medical specifications and requirements defined in WP1 into technical specifications for the SARAS assistive robotic arms
• To define and realise HW/SW interfaces for off-the-shelf laparoscopic tools for the SARAS assistive robotic arms
Software-Hardware integration:
• To develop and implement a component based software-architecture
• To monitor the development of components for cognitive assistance (e.g. force/tactile feedback, vocal commands, interventional checklist), registration, task planning, and virtual fixtures in other WPs
• To integrate the developed software (cognitive control, situation awareness, speech recognition, image analysis, etc.) into one full-fledged surgical suite.

 

 


WP8 Communication and exploitation

Objectives

This WP aims at
• organizing the envisaged communication activities;
• monitoring the progress of the dissemination activities;
• defining the exploitation strategy of the project’s results.

Dissemination, communication and exploitation of project results will run in parallel to the other SARAS activities throughout the whole project. Exploitation activities will be maintained as long as possible beyond the project’s conclusion.

 

 


WP9  Management

Objectives

The project management work-package is responsible for the coordination of the project in both administrative and technical terms aiming towards achieving effective operation of the project as well as timely delivery of quality results.

Its activities include:
• Establishment of appropriate project relations with and reporting to the European Commission.
• Organization and running of project meetings and achieving common understanding in the project.
• Setting up services for electronic documentation storage and communication.
• Establishment of appropriate review and assessment procedures within the project.
• Establishment of a quality management system according to ISO 13485:2016.