MunitasFeature tourAll walkthroughs

All walkthroughs / Health organisation

Health organisation · Regular data

Bring in a table, and see that nobody can read it yet

A research programme at a hospital group needs the quarterly figures for hospital admissions. Devi, a data engineer, brings in the table, and nobody can read it yet, not even Devi. Devi also brings in a small public list of diagnosis codes and claims that it is open to everybody. Hartley, who answers for the Cardiology department, agrees with the claim.

8Steps
2People
about 6Minutes to read
Who is involved
Devi
Data engineer in the Health organisation
Hartley
Data custodian for the Cardiology department

How to read this page

Words used on this page

Organisation
A company or hospital group that uses Munitas. Each organisation sees only its own data.
Department
A team inside an organisation that owns some of its data, such as Cardiology.
Dataset
A named collection of records, such as a table of patient admissions.
Access level
How restricted a dataset is. Raw is the most restricted, and nobody may read it without permission. Published is the most open.
Lease
Permission, given by the custodian, to read one dataset for one stated purpose and for a limited time.
Custodian
A person who holds the data custodian role, which allows deciding who may read data. A custodian decides for the departments they are an approver for.
Privacy contract
A file that says, column by column, how identifying each column of a table is and what is done to it when the table is sealed, and how to find the person each row is about.
Sealed
Finished and closed for good, so that it can never be edited.
Version
One sealed state of a dataset. Changing the contents means making a new version, never editing an old one.
Fingerprint
A value worked out from the exact contents of a file. It changes if even one byte changes, so it shows that a file was not altered.
Claim
A statement, made by the person who brings a dataset in, about how sensitive it is. A different approver of the owning department has to confirm it.
PART ONE

A table of admissions is brought in

A hospital group has a research programme that needs the quarterly figures for hospital admissions. Devi is a data engineer in the Health organisation, and bringing data in is part of the job. The table has one row for each hospital stay and nothing in it names a person, so Munitas treats it as an ordinary table.

1
Devi
Devi registers a dataset and names the department that owns it
Munitas console · Bring a dataset in · /datasets/register

A dataset is a named collection of data that Munitas keeps. Before any data is uploaded, the dataset is registered with a name and an owning department, so that from the first moment somebody is answerable for deciding who may read it. Devi names this dataset admissions and chooses the Cardiology department. The access level, which says how restricted the data is, is left at its safe default. Raw is the most restricted level, and nobody may read data at that level without a lease, which is permission from the owning department to read one version of one dataset for one stated purpose and for a limited time.

Screenshot: Devi registers a dataset and names the department that owns it
The Name box holds admissions, Owned by says Cardiology, and “What access level?” says Raw (the safe default). Beside the Register button the page says Nothing is readable yet. This only creates the record and its owner.
2
Devi
Devi chooses a table and gives Munitas two files: a contract and the rows
Munitas console · Bring a dataset in · /datasets/register, after Register

Once the dataset is registered, the page asks whether anybody will ever need to be erased from this data. Devi answers no, because a row of this table is a hospital stay and not a person. Devi then chooses a table, which needs two files. The first is the contract, which names every column and its type, marks the one column whose value is different in every row, and states that the table cannot be erased and why. The second holds the rows. Here it is a CSV file, which is a plain text file with one row per line and the values separated by commas. The admissions file has 400 rows, one for each stay, with the columns admission id, date admitted, age, diagnosis code, days in hospital and whether the patient was admitted again within 30 days.

Screenshot: Devi chooses a table and gives Munitas two files: a contract and the rows
The answer “No. Store what I bring as it is.” is chosen, and under it Nothing in it is locked. The choice A table is selected. The two file boxes hold admissions.contract.yaml and admissions.csv, and the reading options say Commas and The first line holds the column names.
3
Devi
Munitas checks every row against the contract and seals the table
Munitas console · Bring a dataset in · /datasets/register, after Seal as a table

When Devi presses the button, Munitas reads every row and checks it against the contract. A row with a value of the wrong type is refused with its row and column named, and nothing is sealed. Here every row passes. A sealed version is one state of a dataset that is finished and closed for good: it can never be edited, and a later table becomes a new version instead. The platform also writes the rows as a table that a standard tool such as DuckDB, a free tool for analysing tables, can read once somebody is allowed to.

Screenshot: Munitas checks every row against the contract and seals the table
The green box says Sealed as version 1: 400 rows. It goes on: Nothing in it is locked. It starts at the most restricted level, so nobody can read it until the person who answers for it allows it.
4
Devi
Even Devi cannot read the table: it starts closed
Munitas console · Version · /versions/<id>

The version page shows the sealed table as the platform recorded it. Two things are worth reading. The platform recorded who brought the data in and the exact file that arrived, with a fingerprint, which is a value worked out from the contents of a file and changes if even one byte changes. And the access level is Raw, so the person who brought the table in is in the same position as everybody else: to read it, Devi would have to ask the department's custodian, a person who holds the role that decides who may read a department's data.

Screenshot: Even Devi cannot read the table: it starts closed
Access level when sealed and Access level now both say Raw. Brought in says admissions.csv (15 KB) by eng-devi, where eng-devi is the name Devi signs in with. Stored as a table says Yes, as a table and 400 rows. Under Your access the page tells Devi This is more sensitive than your role reads, so Hartley decides.
PART TWO

A public lookup, and a claim that its custodian agrees with

Analysts who work with admissions need to know what each diagnosis code means. That list is public and about no person, so it should be open to everybody in the organisation. Only the owning department can agree that data is that open.

5
Devi
Devi registers a second table and claims that it is Published
Munitas console · Bring a dataset in · /datasets/register

The second table is a small public list of ten diagnosis codes and what each one means, with a column that says whether the condition is long lasting. Devi registers it in Cardiology and, this time, chooses the access level Published, the most open level, which every signed-in person of the organisation may read without asking. Choosing anything other than the safe default is a claim, a statement about how sensitive the data is, and it is recorded under the name of the person who makes it. The owning department's custodian has to agree with a claim before it takes effect.

Screenshot: Devi registers a second table and claims that it is Published
The name is diagnosis_codes, the department is Cardiology and the access level is Published. A yellow box says You are claiming this is Published. The claim is recorded under your name. It adds Until the owning custodian agrees, the data stays at the most restrictive setting.
6
Devi
The lookup is sealed in the same way, and the claim waits
Munitas console · Bring a dataset in · /datasets/register, after Seal as a table

The lookup goes through the same two files and the same check, and is sealed as version 1. Its claim has not been agreed yet, so for now it is no more open than the admissions table. A script checked this, and there is no picture of it: signed in as Sam, a researcher in the same organisation, the version page of the lookup still offered the form for asking to read it.

Screenshot: The lookup is sealed in the same way, and the claim waits
The green box says Sealed as version 1: 10 rows. Under the box, the dataset is listed as diagnosis_codes, owned by Cardiology, with the level Published. The page does not say that the claim has been agreed, because it has not.
7
Hartley
Hartley sees the claim waiting
Munitas console · Home · /

Hartley is the data custodian for Cardiology and signs in. The home page of a custodian lists what is waiting for a decision. A claim about how open some data is has its own list, shown first, because it is more basic than a request to read: until the level is agreed, nothing else about the dataset can be decided.

Screenshot: Hartley sees the claim waiting
The panel Arrivals waiting for your confirmation lists diagnosis_codes with the label Published and the line Claimed by Devi, with the date and time. The button says Agree with this, and the tile Need your confirmation shows 1. The admissions table is not listed, because Devi made no claim about it.
8
Hartley
Hartley agrees, and the lookup opens to everybody in the organisation
Munitas console · Home · /

Hartley presses Agree with this. The decision is recorded under Hartley's name, and the lookup is now Published. The same check as before, made by script as Sam, found that the form for asking to read the lookup is gone: it is open. The admissions table is untouched and stays Raw. Hartley is still the person who decides who may read it. Asking to read it is the subject of the next walkthrough, Find a table, ask to read it, and the custodian decides.

Screenshot: Hartley agrees, and the lookup opens to everybody in the organisation
A message at the bottom says Agreed with diagnosis_codes's claim. The panel of arrivals has gone, and the tile Need your confirmation now shows 0. Under What you are answerable for, the page says 5 datasets belong to Cardiology, which now includes both tables Devi brought in.