perun.data_model.data ===================== .. py:module:: perun.data_model.data .. autoapi-nested-parse:: Storage Module. Attributes ---------- .. autoapisummary:: perun.data_model.data.log Classes ------- .. autoapisummary:: perun.data_model.data.NodeType perun.data_model.data.MetricType perun.data_model.data.AggregateType perun.data_model.data.Metric perun.data_model.data.Stats perun.data_model.data.RawData perun.data_model.data.LocalRegions perun.data_model.data.Region perun.data_model.data.DataNode Module Contents --------------- .. py:data:: log .. py:class:: NodeType(*args, **kwds) Bases: :py:obj:`enum.Enum` DataNode type enum. .. py:attribute:: APP .. py:attribute:: MULTI_RUN .. py:attribute:: RUN .. py:attribute:: NODE .. py:attribute:: DEVICE_GROUP .. py:attribute:: SENSOR .. py:class:: MetricType Bases: :py:obj:`str`, :py:obj:`enum.Enum` Metric Type enum. .. py:attribute:: RUNTIME :value: 'runtime' .. py:attribute:: POWER :value: 'power' .. py:attribute:: CPU_POWER :value: 'cpu_power' .. py:attribute:: GPU_POWER :value: 'gpu_power' .. py:attribute:: DRAM_POWER :value: 'dram_power' .. py:attribute:: OTHER_POWER :value: 'other_power' .. py:attribute:: CPU_UTIL :value: 'cpu_util' .. py:attribute:: GPU_UTIL :value: 'gpu_util' .. py:attribute:: OTHER_UTIL :value: 'other_util' .. py:attribute:: DRAM_MEM :value: 'dram_mem' .. py:attribute:: GPU_MEM :value: 'gpu_mem' .. py:attribute:: NET_READ :value: 'net_read' .. py:attribute:: NET_WRITE :value: 'net_write' .. py:attribute:: DISK_READ :value: 'disk_read' .. py:attribute:: DISK_WRITE :value: 'disk_write' .. py:attribute:: ENERGY :value: 'energy' .. py:attribute:: CPU_ENERGY :value: 'cpu_energy' .. py:attribute:: GPU_ENERGY :value: 'gpu_energy' .. py:attribute:: DRAM_ENERGY :value: 'dram_energy' .. py:attribute:: OTHER_ENERGY :value: 'other_energy' .. py:attribute:: OTHER_MEM :value: 'other_memory' .. py:attribute:: CPU_CLOCK :value: 'cpu_clock' .. py:attribute:: GPU_CLOCK :value: 'gpu_clock' .. py:attribute:: N_RUNS :value: 'n_runs' .. py:attribute:: MONEY :value: 'money' .. py:attribute:: CO2 :value: 'co2' .. py:method:: __str__() Return string representation of MetricType. .. py:method:: __repr__() Return string representation of MetricType. .. py:method:: fromString(value: str) Create MetricType from string. Parameters ---------- value : str MetricType value. Returns ------- MetricType MetricType object. .. py:class:: AggregateType Bases: :py:obj:`str`, :py:obj:`enum.Enum` Types of data aggregation. .. py:attribute:: SUM :value: 'sum' .. py:attribute:: MEAN :value: 'mean' .. py:attribute:: MAX :value: 'max' .. py:attribute:: MIN :value: 'min' .. py:class:: Metric Struct with resulting metrics and the metadata. .. py:attribute:: type :type: MetricType .. py:attribute:: value :type: numpy.number .. py:attribute:: metric_md :type: perun.data_model.measurement_type.MetricMetaData .. py:attribute:: agg :type: AggregateType .. py:method:: fromDict(metricDict: Dict) :classmethod: Create RawData object from a dictionary. .. py:method:: copy() Create copy metric object. Returns ------- _type_ Copy of object. .. py:class:: Stats Collects statistics based on multiple metrics of the same type. .. py:attribute:: type :type: MetricType .. py:attribute:: metric_md :type: perun.data_model.measurement_type.MetricMetaData .. py:attribute:: sum :type: numpy.number .. py:attribute:: mean :type: numpy.number .. py:attribute:: std :type: numpy.number .. py:attribute:: max :type: numpy.number .. py:attribute:: min :type: numpy.number .. py:method:: fromMetrics(metrics: List[Metric]) :classmethod: Create stats object from list of metrics with the same type. Parameters ---------- metrics : List[Metric] List of metrics with the same type. Returns ------- _type_ Stats object. Raises ------ Exception If metrics are not from the same type. .. py:property:: value Value property (mean). For compatibility with Metric dataclass. Returns ------- _type_ Return the mean value of the stats object. .. py:method:: fromDict(statsDict: Dict) :classmethod: Stats constructor from a dictionory. .. py:class:: RawData Contains timesteps and recorded values from sensors, including information on the values. .. py:attribute:: timesteps :type: numpy.ndarray .. py:attribute:: values :type: numpy.ndarray .. py:attribute:: t_md :type: perun.data_model.measurement_type.MetricMetaData .. py:attribute:: v_md :type: perun.data_model.measurement_type.MetricMetaData .. py:attribute:: alt_values :type: Optional[numpy.ndarray] :value: None .. py:attribute:: alt_v_md :type: Optional[perun.data_model.measurement_type.MetricMetaData] :value: None .. py:method:: fromDict(rawDataDict: Dict) :classmethod: Create RawData object from a dictionary. Parameters ---------- rawDataDict : Dict Dictionary with same keys as RawData object. Returns ------- _type_ RawData object. .. py:class:: LocalRegions Stores local region data while an application is being monitored. .. py:attribute:: _regions :type: Dict[str, List[int]] .. py:method:: addEvent(region_name: str) -> None Mark a new event for the named region. Parameters ---------- region_name : str Region to mark the event from. .. py:method:: isEmpty() -> bool Check if there are any regions marked. Returns ------- bool True if there are no regions marked. .. py:method:: __str__() -> str Return string representation of LocalRegions object. .. py:class:: Region Stores region data from all MPI ranks. For each marked region (decorated function), an numpy array with timestamps indicating function starts and ends. .. py:attribute:: id :type: str :value: '' .. py:attribute:: raw_data :type: Dict[int, numpy.ndarray] .. py:attribute:: runs_per_rank :type: Optional[Stats] :value: None .. py:attribute:: metrics :type: Dict[MetricType, Stats] .. py:attribute:: processed :type: bool :value: False .. py:method:: toDict() -> Dict[str, Any] Convert regions to a python dictionary. Returns ------- Dict[str, Dict[int, np.ndarray]] Dictionary with region data. .. py:method:: fromDict(regionDictionary: Dict[str, Any]) :classmethod: Create Regions object from a dictionary. Parameters ---------- regions : Dict[str, Dict[int, np.ndarray]] Regions dictionary. Returns ------- Regions Regions object. .. py:class:: DataNode(id: str, type: NodeType, metadata: Dict = {}, nodes: Optional[Dict[str, Any]] = None, metrics: Optional[Dict[MetricType, Union[Metric, Stats]]] = None, deviceType: Optional[perun.data_model.sensor.DeviceType] = None, raw_data: Optional[RawData] = None, regions: Optional[Dict[str, Region]] = None, processed: bool = False) Recursive data structure that contains all the information of a monitored application. .. py:attribute:: id .. py:attribute:: type .. py:attribute:: metadata :type: Dict[str, Any] .. py:attribute:: nodes :type: Dict[str, Any] .. py:attribute:: metrics :type: Dict[MetricType, Union[Metric, Stats]] .. py:attribute:: deviceType :type: Optional[perun.data_model.sensor.DeviceType] :value: None .. py:attribute:: raw_data :type: Optional[RawData] :value: None .. py:attribute:: regions :type: Optional[Dict[str, Region]] :value: None .. py:attribute:: processed :value: False .. py:method:: addRegionData(localRegions: List[LocalRegions], start_time: int) Add region information to to data node. Parameters ---------- localRegions : List[LocalRegions] Gathered local regions from all MPI ranks start_time : int 'Official' start time of the run. .. py:method:: toDict(include_raw_data: bool = True) -> Dict Transform object to dictionary. .. py:method:: fromDict(resultsDict: Dict) :classmethod: Create dataNode from python dictionary. Parameters ---------- resultsDict : Dict Dictionary with data node attributes. Returns ------- _type_ DataNode object.