perun.data_model.data

Storage Module.

Attributes

log

Classes

`NodeType`	DataNode type enum.
`MetricType`	Metric Type enum.
`AggregateType`	Types of data aggregation.
`Metric`	Struct with resulting metrics and the metadata.
`Stats`	Collects statistics based on multiple metrics of the same type.
`RawData`	Contains timesteps and recorded values from sensors, including information on the values.
`LocalRegions`	Stores local region data while an application is being monitored.
`Region`	Stores region data from all MPI ranks.
`DataNode`	Recursive data structure that contains all the information of a monitored application.

Module Contents

perun.data_model.data.log

class perun.data_model.data.NodeType(*args, **kwds)

Bases: enum.Enum

DataNode type enum.

APP

MULTI_RUN

RUN

NODE

DEVICE_GROUP

SENSOR

class perun.data_model.data.MetricType

Bases: str, enum.Enum

Metric Type enum.

RUNTIME = 'runtime'

POWER = 'power'

CPU_POWER = 'cpu_power'

GPU_POWER = 'gpu_power'

DRAM_POWER = 'dram_power'

OTHER_POWER = 'other_power'

CPU_UTIL = 'cpu_util'

GPU_UTIL = 'gpu_util'

OTHER_UTIL = 'other_util'

DRAM_MEM = 'dram_mem'

GPU_MEM = 'gpu_mem'

NET_READ = 'net_read'

NET_WRITE = 'net_write'

DISK_READ = 'disk_read'

DISK_WRITE = 'disk_write'

ENERGY = 'energy'

CPU_ENERGY = 'cpu_energy'

GPU_ENERGY = 'gpu_energy'

DRAM_ENERGY = 'dram_energy'

OTHER_ENERGY = 'other_energy'

OTHER_MEM = 'other_memory'

CPU_CLOCK = 'cpu_clock'

GPU_CLOCK = 'gpu_clock'

N_RUNS = 'n_runs'

MONEY = 'money'

CO2 = 'co2'

__str__(): Return string representation of MetricType.

__repr__(): Return string representation of MetricType.

fromString(value: str)

Create MetricType from string.

Parameters

valuestr: MetricType value.

Returns

MetricType: MetricType object.

class perun.data_model.data.AggregateType

Bases: str, enum.Enum

Types of data aggregation.

SUM = 'sum'

MEAN = 'mean'

MAX = 'max'

MIN = 'min'

class perun.data_model.data.Metric

Struct with resulting metrics and the metadata.

type: MetricType

value: numpy.number

metric_md: perun.data_model.measurement_type.MetricMetaData

agg: AggregateType

classmethod fromDict(metricDict: Dict): Create RawData object from a dictionary.

copy()

Create copy metric object.

Returns

_type_: Copy of object.

class perun.data_model.data.Stats

Collects statistics based on multiple metrics of the same type.

type: MetricType

metric_md: perun.data_model.measurement_type.MetricMetaData

sum: numpy.number

mean: numpy.number

std: numpy.number

max: numpy.number

min: numpy.number

classmethod fromMetrics(metrics: List[Metric])

Create stats object from list of metrics with the same type.

Parameters

metricsList[Metric]: List of metrics with the same type.

Returns

_type_: Stats object.

Raises

Exception: If metrics are not from the same type.

property value

Value property (mean).

For compatibility with Metric dataclass.

Returns

_type_: Return the mean value of the stats object.

classmethod fromDict(statsDict: Dict): Stats constructor from a dictionory.

class perun.data_model.data.RawData

Contains timesteps and recorded values from sensors, including information on the values.

timesteps: numpy.ndarray

values: numpy.ndarray

t_md: perun.data_model.measurement_type.MetricMetaData

v_md: perun.data_model.measurement_type.MetricMetaData

alt_values: numpy.ndarray | None = None

alt_v_md: perun.data_model.measurement_type.MetricMetaData | None = None

classmethod fromDict(rawDataDict: Dict)

Create RawData object from a dictionary.

Parameters

rawDataDictDict: Dictionary with same keys as RawData object.

Returns

_type_: RawData object.

class perun.data_model.data.LocalRegions

Stores local region data while an application is being monitored.

_regions: Dict[str, List[int]]

addEvent(region_name: str) → None

Mark a new event for the named region.

Parameters

region_namestr: Region to mark the event from.

isEmpty() → bool

Check if there are any regions marked.

Returns

bool: True if there are no regions marked.

__str__() → str: Return string representation of LocalRegions object.

class perun.data_model.data.Region

Stores region data from all MPI ranks.

For each marked region (decorated function), an numpy array with timestamps indicating function starts and ends.

id: str = ''

raw_data: Dict[int, numpy.ndarray]

runs_per_rank: Stats | None = None

metrics: Dict[MetricType, Stats]

processed: bool = False

toDict() → Dict[str, Any]

Convert regions to a python dictionary.

Returns

Dict[str, Dict[int, np.ndarray]]: Dictionary with region data.

classmethod fromDict(regionDictionary: Dict[str, Any])

Create Regions object from a dictionary.

Parameters

regionsDict[str, Dict[int, np.ndarray]]: Regions dictionary.

Returns

Regions: Regions object.

class perun.data_model.data.DataNode(id: str, type: NodeType, metadata: Dict = {}, nodes: Dict[str, Any] | None = None, metrics: Dict[MetricType, Metric | Stats] | None = None, deviceType: perun.data_model.sensor.DeviceType | None = None, raw_data: RawData | None = None, regions: Dict[str, Region] | None = None, processed: bool = False)

Recursive data structure that contains all the information of a monitored application.

id

type

metadata: Dict[str, Any]

nodes: Dict[str, Any]

metrics: Dict[MetricType, Metric | Stats]

deviceType: perun.data_model.sensor.DeviceType | None = None

raw_data: RawData | None = None

regions: Dict[str, Region] | None = None

processed = False

addRegionData(localRegions: List[LocalRegions], start_time: int)

Add region information to to data node.

Parameters

localRegionsList[LocalRegions]: Gathered local regions from all MPI ranks
start_timeint: ‘Official’ start time of the run.

toDict(include_raw_data: bool = True) → Dict: Transform object to dictionary.

classmethod fromDict(resultsDict: Dict)

Create dataNode from python dictionary.

Parameters

resultsDictDict: Dictionary with data node attributes.

Returns

_type_: DataNode object.