View source for DDS-UI dashboards

This section explains the internal dashboards used to monitor data flows in to and out of DDS. These dashboards have been built into an application called DDS-UI which, in addition to these dashboards, provides many back-end DDS management utilities, such as those for setting up new publisher feeds.
__TOC__

'''Included'''

The DDS-UI dashboards monitor the majority of DDS components that are responsible for the processing of published data, transforming raw data into the standardised FHIR-based intermediary model, and then transforming again for the individual subscriber databases. The dashboards cover almost every element of this data flow, although there are currently some exceptions to this, explained in the Exceptions section.

'''Not included'''

This section does not include:

* Slack alerts that report progress or to raise alerts about errors or failures; in many cases these alerts are simply to draw attention to an issue that the DDS-UI dashboards will display.

* Any aspect of hardware or infrastructure; these metrics are separately captured and monitored by the infrastructure team who have their own configured alerts to detect issues. 

* Other applications that form part of the DDS, such as [[Data Sharing Manager]] or User Manager.

== Overview ==
The DDS is made up of a large number of applications, running on many individual servers, interacting with many databases, all with different purposes. Due to this wide range of components, multiple dashboards have been implemented, to each monitor a specific part of the DDS.

This diagram shows a high-level component-based diagram of the DDS (only including components related to the processing of data).

Each box represents one or more instances of an application, API or process.  

Some of the major DDS databases are also shown. 
[[File:Overview.png|frameless|800x800px|alt=|center]]
  {| class="wikitable sortable" border="1" style="border-collapse:collapse text-align:right width:100%"
! scope="col" width="20%" |Component
! scope="col" width="80%" |Description
|-
|File-based Publishers
|Represents all DDS publishers providing flat-file data to via an SFTP push or pull (or via the DDS Uploader application in the case of TPP
|-
|SFTP Reader
|Set of DDS applications that receives flat-file data, performs some initial processing (such as decryption) and posts the data into the Messaging API for onwards transformation.
|-
|MLLP Publishers
|Represents the two DDS publishers sending HL7v2 ADT real-time messages via MLLP.
|-
|HL7 Receiver
|DDS application that receives HL7v2 ADT messages over MLLP, performs some processing, and then posts those into the Messaging API for onwards transformation.
|-
|HL7v2 Rest Publishers
|Represents the DDS publishers sending HL7v2 ADT and ORU real-time messages via a REST API.
|-
|HL7v2 Receiver
|DDS application that receives HL7v2 ADT and ORU messages via REST, and then posts them into the Messaging API for onwards transformation.
|-
|Messaging API
|API that acts as the gateway into the DDS transformation pipeline for all published data.
|-
|Inbound Transformation
|Represents the set of Queue Reader applications used to transform raw published data into the standardised FHIR format and then   store that in the core FHIR record databases.
|-
|Outbound Transformation
|Represents the set of Queue Reader applications used to transform the FHIR data into the CSV for DDS subscriber databases. For internally hosted subscriber databases, the CSV data is written directly to them. For remotely hosted databases, the CSV data is written to a staging database for remote sending.
|-
|UPRN API
|API used in the outbound transformation process to look up various attributes relating to addresses.
|-
|Information Model API
|API used in the outbound transformation process to look up concept identifiers for various clinical coding schemes.
|-
|Remote Sending
|Represents the process of writing the staged CSV data for remote subscriber databases (output of the outbound transformation process) to the DDS SFTP server, and that data then being collected by the Remote Filer application and applied to the remote subscriber databases.
|-
|National Reference Updates
|Represents various processes run regularly (typically monthly or quarterly) to update DDS reference databases with new content from national sources, such as TRUD or ONS.
|-
|Local Code Updates
|Represents processes run regularly (daily) to push new local codes received from EMIS and TPP into the Information Model.
|-
|Scheduled Report Extracts
|Represents various clinical extracts generated from internal subscriber databases.
|-
|Frailty API
|API provided to Adastra (via Redwood Technologies) to allow discovery of if a patient is considered “frail”.
|-
|Know Diabetes API
|Feed to the Know Diabetes API, sending patient record updates to the remotely hosted API.
|-
|Clinical Dashboards
|Represents the processing of data from the internal subscriber databases into the format used by the DDS clinical dashboards.
|- valign="top"
|}
The following sections describe each dashboard, and provide a similar diagram, highlighting the components that the specific dashboard monitors.

== Publishers ==
The Publishers page shows the current state of each discrete publishing service to DDS, irrespective of how their data is published. It shows details on when data was last published to DDS and the progress of the inbound transform (raw data to FHIR record store) for each one.
[[File:Publishers.png|center|frameless|800x800px]]
The primary use for this dashboard, from a monitoring perspective, is to detect services where no data has been received into DDS. The page is also used in the set up and configuration of DDS publishers, and controlling how published data is routed through the parallel processing queues.
[[File:Publisher dashboard.png|border|center|frameless|800x800px]]
<br />
== Publisher errors ==

== Subscribers ==

== Queue readers ==

== Frailty API ==

== HL7 Reciever ==

== SFTP Reader ==

== Schedules tasks ==

== Exceptions ==
<br />