Command-line Interface (CLI)
Now that the configuration file has been edited and saved, you can run GRACE-HPC from the command-line to calculate the energy use and carbon footprint of your HPC jobs:
gracehpc run [ARGUMENT-OPTIONS]
Running this command without any options will execute the analysis with default values for all arguments specified below.
Argument Options
You can customise the output with many arguments and options to enhance the carbon footprint analysis:
usage: gracehpc run [-h] [--StartDate STARTDATE] [--EndDate ENDDATE] [--JobIDs JOBIDS]
[--Region REGION] [--Scope3 SCOPE3] [--CSV CSV]
options:
-h, --help show this help message and exit
--StartDate STARTDATE The first date of the range to process jobs for, in YYYY-MM-DD.
Default: 2025-01-01 (January 1st of the current year)
--EndDate ENDDATE The final date of the range to process jobs for, in YYYY-MM-DD.
Default: Current date
--JobIDs JOBIDS Comma-separated list (no spaces) of all the HPC job IDs to filter on.
E.g. 'id1234,id5678'
Default: 'all_jobs'
--Region REGION UK region of the HPC cluster needed for realtime carbon intensity data.
Choices: 'North Scotland', 'South Scotland', 'North West England', 'North East England',
'Yorkshire', 'North Wales', 'South Wales', 'West Midlands', 'East Midlands', 'East England',
'South West England', 'South England', 'London', 'South East England'.
Default: 'UK_average'
E.g. 'South West England' for Isambard systems and 'South Scotland' for Archer2.
--Scope3 SCOPE3 Include scope 3 emissions for either: 'Isambard3', 'IsambardAI', 'Archer2',
or a custom numeric value in gCO2e/node-hour for other HPC systems (e.g. '51').
Default: 'no_scope3'. If 'no_scope3' is selected, only scope 2 emissions will be calculated.
--CSV CSV Save the final datasets to CSV files for further analysis.
Choices: 'full': entire dataset (all jobs) with all columns,
'full_summary': entire dataset with summary columns only,
'daily': dataset aggregated by day with all columns,
'daily_summary': dataset aggregated by day with summary columns only,
'total': dataset aggregated over all total jobs with all columns,
'total_summary': dataset aggregated over all total jobs with summary columns only,
'all' : all of the above datasets saved to CSV files.
Default: 'no_save'
Example Commands
Estimating the energy use and carbon emissions for
all jobsran by the user onIsambard 3between May and June withscope 3emissions included.
gracehpc run --StartDate 2025-05-01 --EndDate 2025-06-01 --Region 'South West England' --Scope3 'Isambard3'
Estimating energy use and carbon emissions for job
121109ran onIsambard-AIwithoutscope 3emissions included, saving thefulldataset to CSV for further analysis.
gracehpc run --JobIDs '121109' --Region 'South West England' --CSV 'full'
Estimating energy use and carbon emissions for jobs
119363 and 119364ran onArcher2withscope 3included and saving thetotalaggregated dataset to CSV, combining both jobs.
gracehpc run --JobIDs '119363,119364' --Region 'South Scotland' --Scope3 'Archer2' --CSV 'total'
Example Terminal Output
Here is an example terminal output for GRACE-HPC run on Isambard-AI command line.
(ghpc_env) username.project@nid01:~/dir> gracehpc run --StartDate 2025-05-01 --EndDate 2025-07-01 --Region 'South West England' --Scope3 'IsambardAI'
╭─ 🌱 GRACE-HPC ─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ Carbon Footprint Estimation for HPC Jobs ran on Isambard-AI │
│ │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
This tool estimates the energy consumption, scope 2 and scope 3 carbon emissions of your SLURM HPC jobs. If
energy counters are available, it will use them. Otherwise it will estimate energy and emissions from usage
data and cluster-specific TDP values.
Carbon intensity for scope 2 emissions (operational) is retrieved from the regional Carbon Intensity API
(https://carbonintensity.org.uk) at the time of job submission. Scope 3 emissions (embodied) are estimated
from the node-hours used by the job, and the scope 3 emissions factor. For Isambard systems and Archer2, these
scope 3 factors are calculated from the total lifecycle scope 3 emissions for each system divided by the total
node-hours available over the systems projected lifetime.
The results below are calculated using SLURM accounting data for jobs submitted to the Isambard-AI cluster,
including information such as runtime, resource allocation, resource usage, hardware-level energy counters
(if available), etc. For a detailed explanation of all methodologies used, please refer to the GRACE-HPC
documentation.
Note: The results presented here are estimates based on the available data and methodologies with assumptions
and limitations. Hence this tool should be used for informational purposes only, not as a definitive energy
and carbon cluster monitoring tool.
────────────────────────────────────────────────── OVERVIEW ──────────────────────────────────────────────────
HPC System: Isambard-AI
User Name: eayliffe.d5c
Date Range: 2025-05-01 to 2025-07-01
Jobs: Processing all jobs in the selected date range
System PUE: 1.1
System Energy Counters: ❌ Not available for all jobs — hardware energy counters were not used in calculations.
Usage-based estimates were used instead.
─────────────────────────────────────────── ⚡️ ENERGY CONSUMPTION ────────────────────────────────────────────
Total Energy Used (estimated): 50.2017 kWh
- CPUs: 1.0510 kWh
- GPUs: 40.4767 kWh
- Memory: 4.1101 kWh
- Data Centre Overheads (PUE): 4.5638 kWh
Compute Energy Use (estimated): 45.6379 kWh
Compute Energy Use (measured by system counters): N/A (not all jobs had energy counters available)
──────────────────────────────────────────── 🌿 CARBON FOOTPRINT ─────────────────────────────────────────────
Scope 2 Emissions (usage-based): 6.1486 kgCO2e
Scope 2 Emissions (system counter-based): N/A (not all jobs had energy counters available)
Scope 3 Emissions: 2.7534 kgCO2e (114 gCO2e/node-hour)
Total Emissions: 8.9020 kgCO2e (usage-based)
Average Carbon Intensity: 122.0 gCO2e/kWh (South West England)
CI distribution (gCO2e/kWh): Q1: 83.0, Median: 121.0 , Q3: 152.0
Note: For Isambard systems and Archer2, market-based Scope 2 emissions = 0 gCO₂e due to 100% certified
zero-carbon electricity contracts.
The estimates above are based on the UK national grid carbon intensity and are provided for informational
purposes, representing what the emissions would be if Isambard systems were not powered by renewable energy
(the grid only).
─────────────────────────────────────────── THIS IS EQUIVALENT TO: ───────────────────────────────────────────
🚗 Driving 42.15 miles (0.21 kgCO2e/mile average UK car, 2023)
🌲 Tree absorption: 10.7 tree-months (0.83 kgCO2e/month average UK tree carbon sequestration rate)
✈️ Flying 0.063 times from Bristol to Paris (140 kgCO2e/passenger)
🏠 UK Households: Daily emissions from 9.7 households electricity use (UK average)
Approximate electricity cost: £12.92 (at 0.2573 GBP/kWh)
See documentation for sources and assumptions of these estimates.
───────────────────────────────────────────── ⚙️ USAGE STATISTICS ─────────────────────────────────────────────
Number of Jobs: 178 (129 successful)
First → Last Job Submitted: 2025-06-10 → 2025-06-21
Total Runtime: 0 days 21:38:40
Total CPU Usage Time: 12 days 14:43:02.495000 (303 hours)
Total GPU Usage Time: 2 days 09:49:26 (58 hours)
Memory Requested: 93,388.0 GB
Node Hours: 24.2
CPU Hours: 0.0
GPU Hours: 57.8
─────────────────────────────────── ❌ FAILED JOBS & WASTED MEMORY IMPACT ────────────────────────────────────
Failed Jobs: 49 (27.5%)
Wasted Scope 2 Emissions: 1.7687 kgCO2e (Usage-based estimate)
Note: Failed HPC jobs are a significant source of wasted computational resources and unnecessary carbon
emissions.
Every failed job still consumes electricity for scheduling, startup, and partial execution—without producing
useful results.
Reducing failed jobs is a simple yet impactful way to lower your carbon footprint on HPC systems.
Memory overallocation is a common source of energy waste and excess carbon emissions.
On most HPC systems, power draw depends on the amount of memory requested, not the memory actually used.
If all jobs had been submitted with only the memory they truly required, approximately:
598.4678 gCO2e could have been saved (Usage-based estimate)
────────────────────────────────────────── DOCUMENTATION & FEEDBACK ──────────────────────────────────────────
Find the methodology including assumptions and limitations of this tool outlined in the documentation:
https://github.com/Elliot-Ayliffe/GRACE-HPC/tree/main
See also what other features are available with the package/API including an interactive Jupyter Notebook
interface.
If you find any bugs, have questions, or suggestions for improvements, please post these on the GitHub
repository.