Current Status
Dashboard website on hypocentre fully working.
Dashboard database is located on hypocentre at /nesi/project/nesi00213/dashboard.db
Sample command to run data collection into db
yzh231@hypocentre: python run_data_collection.py melody.zhu /nesi/project/nesi00213/dashboard.db
Sample command to run website:
python run_dashboard_app.py /nesi/project/nesi00213/dashboard.db
Sample command to collect old data from 365 days ago
python run_old_data_collection.py /nesi/project/nesi00213/dashboard/dashboard.db --hpc maui --days_shift 365
Background
To be able to see a snapshot of our current HPC usage and to see it over time.
Notes
Realtime
Current time
Core Hour Usage Maui/ Mahuika - nn_corehour_usage
Disk utilisation for nesi00213 - nn_check_quota
Running Jobs for nesi00213 Maui/Mahuika - squeue
Capacity Used Maui (How many nodes are being used as a percentage of how many is available) – https://support.nesi.org.nz/hc/en-gb/articles/360000204116-M%C4%81ui-Slurm-Partitions # mahuika may be difficult as have multiple partitions and partially filled nodes
Historic - (start date, end date, time step)
For (end - start) / step periods
Display Core Hour usage per person Maui/Mahuika – ch_report.sh (there is an updated version in the master branch)
### Investigate if ch_report.sh needs adjusting for jobs on partitions that charge more than 1 ch per ch.
Tasks
- From a local (UC) computer poll the above scripts to get the data
- Print (and log) the data in a useful / readable format
- Create a cronjob to poll data (and display it on the laptop Jira screen)
Progress
Task 1-3 done, hpc core_hour usage etc are now refreshed every 60s on Jira Screen
Todo
1.Plot core hour usage
2. Create a UI ,eg website to display/download plot/data