Server

Key hardware specifications of the server executing this step.

OS family
Linux
Allocation
Indicates whether this server appears to be dedicated to this Metaflow step or shared with other tasks. This assessment is based on comparing system-level resource usage with task-specific resource usage, analyzing patterns in CPU, memory, GPU, and VRAM utilization against both percentage thresholds and absolute values.
Shared
vCPUs
4
Memory
15,788 MiB
Storage
73 GiB
GPUs
1 Tesla T4
VRAM
15,360 MiB

Cloud

Network discovery indicates the following cloud environment was utilized for this step.

Cloud Provider
aws
Region/Datacenter
us-west-2
Instance Type
g4dn.xlarge
Compute Costs
The cost of running this cloud server for the duration of this step, not including storage, network traffic, IPV4 prices, startup time or any discounts.
$0.001575

Usage Statistics

Current and historical (including up-to the last five successful runs) averages, peaks and other summaries on resource usage.

CPU
0.81 avg | 0.86 hist avg | 1.11 peak
Memory
324 MiB avg | 488 MiB peak | 812 MiB hist peak
Duration
10.78 sec
GPU
0.55 avg | 0.59 hist avg | 0.99 peak
VRAM
664 MiB avg | 906 MiB peak | 906 MiB hist peak
Disk Space
23 GiB peak
Traffic
0 GB in | 0 GB out

Recommendations

Based on recent average CPU usage, historical peak memory and GPU utilization.

Recommended Resources for Next Run
The Metaflow @resources decorator is limited to specifying the number of vCPUs, memory, and number of GPUs, so e.g. no way to specify the minimum amount of VRAM.
@resources(cpu=1, memory=1024, gpu=1)
Automated Tuning of Resources
Learn more at sparecores.com
Cheapest Cloud Server to Run This Step
Evaluated 2000+ servers options accross AWS, GCP, Azure, Hetzner and UpCloud by filtering for the required number of vCPUs, memory, GPUs and min VRAM, then ordered descending by ondemand price, and selected the first one. The price per execution is based on the current best ondemand price of the server and the current duration of the step, and does not include any storage, network traffic, IPV4 prices, the startup time or any discounts. If interested in more advanced recommendations, please get in touch!
  1. g2-standard-4 for $0.000438/execution
Potential Cost Savings
This calculation assumes the current cloud server is dedicated to running this step and that the recommended cloud server would provide comparable performance. Savings are based on the best available on-demand pricing in supported regions and don't account for any existing discounts you may have.
72.19% | $0.001137/execution

CPU Usage

CPU usage for both the system and specific tasks is calculated by summing user+nice and system CPU times (in clock ticks), normalized by dividing by the total elapsed time and ticks per second. Task CPU usage encompasses all child processes.

Memory Usage

On Linux, the used server memory is calculated by total - free - buffers - cached, while it depends on psutil for other systems. Task memory usage is measured by summing PSS (on Linux), USS (on MacOS and Windows), or RSS rollups of all subprocesses.

Disk I/O Usage

Task-specific disk usage tracking is unreliable; therefore, it is recommended to monitor disk usage at the system level, encompassing all mounted disks.

Disk Space Usage

System-level disk space usage on all mounted disks.

Network Usage

Network usage is monitored solely at the system level across all interfaces.

GPU Usage

nvidia-smi reported ratios standardized between 0 and GPU count, proxying how many GPUs have been 100% utilized. Note that task-specific GPU usage is not as reliable as system-level GPU usage and limited up to 4 GPUs.

GPUs in Use

nvidia-smi reported number of GPUs with a utilization greater than 0. Note that task-specific GPU usage is not as reliable as system-level GPU usage and limited up to 4 GPUs.

VRAM Usage

nvidia-smi reported, summed up VRAM usage for all GPUs. Note that task-specific GPU usage is not as reliable as system-level GPU usage and limited up to 4 GPUs.