skripts.helpers package

Submodules

skripts.helpers.check_gpus module

skripts.helpers.combine_dfs module

Methods for working with DataFrames from the command line.

skripts.helpers.combine_dfs.combine_dc(path_combs, outpath, target_format='parquet', framework=<module 'polars' from '/mnt/lustre/groups/link/linca945/.conda/envs/VAE/lib/python3.11/site-packages/polars/__init__.py'>, bins: int = 2)[source]

Combine dataframes in divide and conquer approach, if they are too big to be loaded into RAM at current compression.

Parameters:

  • path_combs (list[str]) – All path combinations

  • outpath (path-like) – Output path

  • target_format (str, optional) – Target format [parquet, feather, tsv], defaults to “parquet”

  • framework (polars | pandas, optional) – Processing framework [pandas, polars], defaults to pd

  • bins (int, optional) – Number of dataframes that are loaded at a time, defaults to 2

skripts.helpers.combine_dfs.concat_dfs(dfs, framework=<module 'pandas' from '/mnt/lustre/groups/link/linca945/.conda/envs/VAE/lib/python3.11/site-packages/pandas/__init__.py'>)[source]

Combine dataframes.

Parameters:

  • dfs (list) – List of dataframes

  • framework (polars | pandas, optional) – Processing framework [pandas, polars], defaults to pd

Returns:

Concatted dataframe

Return type:

polars.DataFrame | pandas.DataFrame

skripts.helpers.combine_dfs.main(args)[source]

Execution from shell script. For more info run python combine_dfs.py –help.

Parameters:

args (strings) – Arguments form command line

skripts.helpers.combine_dfs.read_df(path, framework=<module 'pandas' from '/mnt/lustre/groups/link/linca945/.conda/envs/VAE/lib/python3.11/site-packages/pandas/__init__.py'>)[source]

Read a dataframe in different formats.

Parameters:

  • path (path-like) – Path to DataFrame

  • framework (polars | pandas, optional) – Processing framework [pandas, polars], defaults to pd

Returns:

Dataframe

Return type:

polars.DataFrame | pandas.DataFrame

skripts.helpers.combine_dfs.write_df(df, path, framework=<module 'pandas' from '/mnt/lustre/groups/link/linca945/.conda/envs/VAE/lib/python3.11/site-packages/pandas/__init__.py'>)[source]

Write dataframe in different formats.

Parameters:

  • df (polars.DataFrame | pandas.DataFrame) – Dataframe

  • path (path-like) – Path to write DataFrame to

  • framework (polars | pandas, optional) – Processing framework [pandas, polars], defaults to pd

skripts.helpers.file_handling module

skripts.helpers.file_handling.convert_to_utf8(file)[source]

Convert file from iso-8859-15 to UTF-8 encoding.

Parameters:

file (path-like) – Path to file

skripts.helpers.file_handling.mat_to_tsv(folder, file)[source]

Saves mat file as tsv in the same folder.

Parameters:

  • folder (path-like) – path to folder with .mat files

  • file (str) – name of .mat file

skripts.helpers.file_handling.mat_to_tsv_batch(folder: str)[source]

Saves mat files as tsv in the same folder.

Parameters:

folder (path-like) – path to folder with .mat files

skripts.helpers.file_handling.parse_folder(path)[source]

List all paths in directory.

Parameters:

path (path-like) – Path of base directory

Returns:

List of files and directories at path

Return type:

list

skripts.helpers.file_handling.remove_by_filename(directory_path, str: str)[source]

Remove files at a dirctory by part of file name.

Parameters:

  • directory_path (path-like) – Path of base directory

  • str (str) – String to filter for

skripts.helpers.mailing module

skripts.helpers.normalization module

skripts.helpers.normalization.standard_normalization(df, axis=1)[source]

Perform standard normalization (centered around 0, scaled to variance) on dataframe.

Parameters:

  • df (pandas.DataFrame) – Dataframe

  • axis (int, optional) – Axis to perform normalization on, defaults to 0

Returns:

Normalized Dataframe

Return type:

pandas.DataFrame

skripts.helpers.normalization.total_ion_count_normalization(df, axis=0)[source]

Perform TIC normalization on dataframe.

Parameters:

  • df (pandas.DataFrame) – Dataframe

  • axis (int, optional) – Axis to perform normalization on, defaults to 0

Returns:

Normalized Dataframe

Return type:

pandas.DataFrame

skripts.helpers.pc_stats module

skripts.helpers.pc_stats.bits_to_bytes(bits, factor)[source]

Coverts a number of bits to a number of bytes.

Parameters:

  • bits (int) – Bits to be converted

  • factor (int) – / 10**factor (e.g. use 9 for GB)

Returns:

Number of bytes

Return type:

int

skripts.helpers.pc_stats.print_available_gpus()[source]

Print all available GPUs.

skripts.helpers.pc_stats.print_utilization(gpu: bool = False)[source]

Print the GPU, CPU and RAM utilization at the moment.

Parameters:

gpu (bool, optional) – Whether to check for GPU too, defaults to False

Module contents