Data Review is a comprehensive set of features for the visual review, flagging, and in-line editing of data all from within the graphing tool. From any time-series graph, or from your saved graphs, you have the ability to edit the data in a calculated channel, allowing you to quickly remove, replace, or alter one or more datasets without needing to leave your graph.
A flagging mode provides quality assurance and quality control functionality, allowing datasets to be visually classified with colors and numbers. These flag values can then be incorporated into FACE channels for filtering, scrubbing, and other reporting purposes.
Analysts can submit their comments and notes with their changes which are stored as a channel alongside their dataset.
Enabling Data Review Mode
After plotting a time-series graph, press the 'Data Review' button in the graph toolbar to activate the 'Data Review Mode'. If you don't see this button, your account does not have the correct permissions to use this feature. Contact your account administrator for help.
Pressing this button will toggle on the data review mode and changes the behavior of the mouse so that when you click and drag a box on the graph it will select and highlight this range of data.
When working on data review, you can toggle the 'Data Review' button on and off to switch between the normal zoom mouse mode and the data selection mode required to edit or flag data. You can freely switch back to zoom into a smaller date range and then switch back to continue editing or flagging. Your changes will be remembered.
Once data review mode is active, use the mouse to highlight a range of data, and then right-click this range to choose from a set of menu options:
Editing Data
Modifying the data in your graph is an alternative to using FACE or the Edit Data tools. Internally, it functions the same way, relying on FACE under the hood to take care of the calculations, auditing, and transparency. Raw data can never be modified and changes made during the edit process can be reverted, replaced, or discarded.
After selecting a range of data, right-click on the selection and choose 'Edit Data' from the context menu. The 'Edit Data' window will provide the options for modifying the range of data that has been selected.
All edits must be applied to a calculated channel. Edits to a raw channel will be saved into a new calculated channel, which can be named using the 'Calculated Channel' text box.
The date range that was selected by mouse can be manually changed using the Start and End Range calendar pickers.
Edit Actions
-
Remove/Delete Data
This action will remove the data during the selected period. It will create a gap in the calculated channel.
-
Replace with data from
This action will replace the data during the selected period with the data from another channel. The user can pick which channel they want to use as the source of the replacement data.
-
Set to constant value & sync with channel
This action will replace the data during the selected period with a constant numeric value. This value will occur at the same frequency as the data in the selected sync channel. For example, if the constant value is 3, and the sync channel contains 5-minute fixed interval data, the result will be a repeating value of 3 every 5 minutes.
-
Scale/Offset all values
This action allows to scale or offset the data during the selected period by a constant amount and operator. The available operators include: plus (+), minus (-), multiply (*), and divide (/). This allows the user to shift data by a fixed amount.
Notes & Comments
With any edit, the user has the option to add a comment or note about the action they have taken. This note is saved into a Notes channel on the site. The user can pick from an existing notes channel, or create a new one. Notes channels can be plotted on a graph to visualize where an analyst has made changes or otherwise commented on the data.
Saving Changes
When you press 'Ok', the action will only impact the graph in your browser and you can continue with more edit actions. The changes are not saved permanently until the 'Save Review Changes' button in the toolbar is pressed. Failing to save your changes means any editing work will be lost.
Flagging Data
Visually flagging data provides the ability to classify data with a color and number. These colors and numbers can be used to represent whatever the analyst chooses. For example, 3 may represent good data; 2 may mean questionable data needing further analysis; and 1 means bad data unsuitable for analysis. The meaning is up to the analyst.
Any channels that have been flagged will show their flagged ranges and colors when enabling the 'Data Review' mode. This can be used to communicate confidence or problems in the data; or simply whether the data has been reviewed already.
The flag values are saved in a special calculated channel which can be accessed in FACE. A common use for these flag values would be for filtering or scrubbing your data.
How To Flag Data
After selecting a range of data, right-click on the selection and choose 'Modify Flag' from the context menu. The 'Flag Data' window will provide the options for flagging the range of data that has been selected.
The date range that was selected by mouse can be manually changed using the Start and End Range calendar pickers.
Choose a value to assign to this selected range, or assign another value
Adding a note to this flag allows the analyst to add a comment or reason for the flag. This will be saved with the flag and can be read whenever looking at the flag.
Saving Changes
When you press 'Ok', the flag will only be visible in the graph in your browser and you can continue with more flagging. The changes are not saved permanently until the 'Save Review Changes' button in the toolbar is pressed. Failing to save your changes means any flagging work will be lost.
Example use of Flagging for Cleaning Data
An analyst is required to review one or more raw level channels every week. The result of this process is publishing a 'Final Level' channel that is based on the approved data from the raw level channel collected from the sensor.
Initial one-time setup:
- Create a new calculated channel in FACE called: 'Final Level'
- Set the formula in this channel to: IF([Raw Level - Data Flag]==3, [Raw Level], NULL)
Note: you will have had to flag the raw level channel at least one time for this [Raw Level - Data Flag] channel to exist.
Weekly flagging process:
- Open your saved graph containing the Raw Level and Final Level channels for the last week.
- Press 'Data Review'. You will see any previously saved flags.
- Start by selecting all of the data and marking it as a 3 to indicate 'Good'.
- Then selectively mark any bad or questionable data as a 2 or 1.
- Save your changes. Replot your graph.
In our graph above, we can see where the dropouts that we flagged with the value of 1 have been removed from the Final Level channel. The rest of the data flagged with a value of 3 has been retained in the final channel as it is considered good quality.
This simple example can be extended to any number of filtering, scrubbing, or replacement scenarios you may have using FACE.
0 Comments