How to read this page
- Every step is one screen, or one moment, in a real run. The label under the title names where it happens, and the step reads in the same order each time: what is happening, the evidence, and then what to look for in it.
- Screenshots are real pictures of the Munitas console, the web app that people open in a browser. Click any picture to enlarge it.
- Every name of a person, patient or card holder in the example data is invented, and every screenshot is unedited apart from being cropped to the part that matters.
Words used on this page
- Organisation
- A company or hospital group that uses Munitas. Each organisation sees only its own data.
- Department
- A team inside an organisation that owns some of its data, such as Cardiology.
- Dataset
- A named collection of records, such as a table of patient admissions.
- Custodian
- A person who holds the data custodian role, which allows deciding who may read data. A custodian decides for the departments they are an approver for.
- Department approver
- A custodian who is listed for one department and may approve access to its data and confirm claims about it. A department can have several, and any one of them can act on their own.
- Sealed
- Finished and closed for good, so that it can never be edited.
- Version
- One sealed state of a dataset. Changing the contents means making a new version, never editing an old one.
- Pipeline
- A series of steps that Munitas runs on a dataset version, each step after the ones it depends on. The last step is a gate decision.
- Run
- One execution of an agent or a pipeline, started for a stated purpose.
- Custom step
- A step of a pipeline whose script was written by the person who registered the pipeline, instead of one of the steps Munitas provides. It runs in a closed container with no internet, no key and no storage address, and can only ask the platform for the recordings it is allowed to read and send back the files it makes.
- Stage
- Where a version of a pipeline is in its life. In development means the version runs only on development data, which is a small sample or recordings with no real person in them. In production means it runs on any data the department allows. Retired means it has ended for good and cannot be run again.
- Under review
- The access level of something that no person has reviewed. Nobody may use it until a reviewer decides that it may be released.
- Fingerprint
- A value worked out from the exact contents of a file. It changes if even one byte changes, so it shows that a file was not altered.
- Access level
- How restricted a dataset is. Raw is the most restricted, and nobody may read it without permission. Published is the most open.
- Container
- An isolated environment, with no network access of its own, where Munitas runs an agent.
- Locked recording
- A recording stored scrambled under a key made for the one patient it is about. Reading it back needs the platform to open it, and destroying the patient’s key later makes it unreadable for good.
- Development sample
- A few recordings drawn by the platform from a sealed version of locked recordings, for building and testing a pipeline version that is still in development. It names the same locked recordings as the original and copies none, and it ends on its own.
- Synthetic
- Made up, with no real person in it, such as sentences read aloud by volunteers from invented scripts.
Devi uploads a pipeline with a step of their own, and is stopped
A pipeline is a series of steps that Munitas runs on a version of a dataset. Most steps are provided by Munitas. A custom step is one whose script was written by the person who registered the pipeline. A step like that is code that the platform did not write, so when it asks to read recordings or to write files, a second person has to agree first. This part shows Devi uploading such a pipeline and the platform refusing to run it.
Devi is a data engineer in the Cardiology department of the Health organisation. A department is the team inside an organisation that owns some of its data. Devi has registered a pipeline called silence-the-start, and now uploads its first version, which is one sealed state of the pipeline that can never be edited. The version is made of a YAML file, which is a plain text file that names each step, and a zip file that holds the script of each step. The first step is a custom step. The YAML file says that the step is given the recordings and may write files, and its script makes the first half second of each recording silent and writes the result back.
Uploading seals the version. Sealed means finished and closed for good, so that it can never be edited, only replaced by a next version. Every version of a pipeline has a stage, which is where it is in its life: in development, which means it runs only on development data, in production, which means it runs on any data the department allows, or retired, which means it has ended for good. Development data is recordings declared synthetic, which means made up with no real person in them, or a small sample drawn by the platform for the person who asks. A version with a step like this one starts in development. The line under the pipeline’s name explains the three stages. The page lists the two steps and then shows a yellow box. The box says that the first step reads the locked recordings and writes files, which become a new version that a reviewer checks before anyone can use it. The version runs on any other data only when an approver of the owning department moves it to production, and the approver cannot be the person who uploaded it. An approver is a data custodian, a person who may decide who reads a department’s data, who is listed for that department.
Devi opens the version of the recordings that was sealed, which means finished and closed for good, and chooses the new pipeline in the list of pipelines that can be run on it. The platform checks the stage of the pipeline version before it starts anything, and refuses because the version is in development and these are real recordings. It does not wait until the step reads a recording to find out, and nothing is left behind: no run, no file and no record of an attempt to read. Devi could instead run the version on a small sample of these recordings, which the platform draws for the person who asks.
Hartley moves it to production
Moving a version to production is a decision by the department that owns the pipeline, made by a person other than the one who uploaded the version. It is recorded with who made it and when, and it can be reversed.
Hartley is the data custodian for Cardiology. Hartley opens the same page and sees the same yellow box, and this time it has a button to move the version out of development, because Hartley is an approver of the department that owns the pipeline and did not upload the version. Moving it to production is the department’s statement that the script has been looked at and may be given the recordings and may write files.
Hartley presses the button, and the move is recorded with Hartley’s name and the time. Returning the version to development stops a step that is already running at its next request, because the platform checks the stage again on every request that the step makes and not only when the run starts.
Devi runs it, and finds what it made
With the version in production the same run starts. This part shows what the run records and where the files that the step wrote end up.
Devi starts the same run again, and this time it works. The script runs in a container, which is an isolated environment. This one has no internet, runs as a user without special rights, and holds no key and no storage address. It can only ask the platform for the recordings it is allowed to read, and send back the files it makes, and the platform checks on every request that the version is still in production. When the script ends, the platform seals the files it finished, which closes them for good, as one new version, and the page of the run links to it.
Devi follows the link. The files are in a new dataset, a named collection of data, whose name is the name of the first dataset followed by the name of the step. The version is marked Under review, the access level of something that no person has reviewed. Nobody may use it until a reviewer decides. The platform made this version itself from what the script sent, so the script was never given any way to write into storage. The page also says that the version is stored as a table of 2 rows, one for each file it holds.
Further down the same page, Where this came from names what produced the version: the step, the identity that the platform runs pipelines as, the code version and the environment. The code version is a fingerprint of the exact pipeline version that was moved to production. A fingerprint is a value worked out from the exact contents of a file, so it changes if the contents change. Made from links to the version of recordings that the step read, so erasing a patient later lists this version among what was made from their recordings.
Hartley retires the version
A version is retired when something replaces it or when it should no longer run. Retiring is final: the version is never brought back, and a later version takes its place.
Hartley retires the version of the pipeline. Retiring ends the life of a version for good. Nothing can run it again, and a step that is running at the moment it is retired is refused at its next request, so the files that step was writing are discarded. An approver of the owning department may retire a version, and so may the person who uploaded it, because retiring only makes things stricter.
Devi opens the version of the recordings again and chooses the retired pipeline version in the list of pipelines that can be run on it. The platform refuses before it starts anything and says that the version was retired.
