API Reference

CoderData Object

coderdata.download.downloader.download(name='all', local_path=PosixPath('/home/runner/work/coderdata/coderdata'), exist_ok=False)

Download the most recent version of files from a Figshare dataset, filtered by a specific prefix or all files.

This function queries the Figshare API to retrieve details of a dataset and then downloads files from it. Files can be filtered by a specified prefix such as hcmi, beataml, etc. If ‘all’, an empty string, or None is passed as the prefix, all files in the dataset are downloaded. The function identifies the most recent version of a file by selecting the one with the highest ID among duplicates with the same name.

Parameters:

dataset_prefix (str, optional) – The prefix of the dataset to download (e.g., ‘hcmi’). If ‘all’, an empty string, or None, all files in the dataset are downloaded. Default is None.

Returns:

The function downloads files to the local repository and does not return any value.

Return type:

None

Collection of small utility and helper functions.

coderdata.utils.utils.list_datasets(raw=False)

Hepler function that returns a list of available datasets including a short description and additional information available.

Parameters:

raw (bool, default=False) – If set to True returns a yaml dictionary containing all available datasets including additional information. If set to false prints information to stdout and returns None.

Returns:

Returns a dict containing the information if raw==True, otherwise prints information to stdout and returns None.

Return type:

dict | None

coderdata.utils.utils.version()

Helper function that returns the version strings for the package and the dataset build.

Returns:

Contains package and dataset build version.

Return type:

dict

Command Line Interface to retrieve coderdata datasets.

coderdata.cli.check_folder(path)

Helper function to check if a defined folder exists.

Returns:

Cleaned path object with the absolute path to the folder passed to the function.

Return type:

Path

Raises:

  • TypeError – If passed path argument is not of the requested type.

  • OSError – If the passed path argument does not link to a valid existing folder.

coderdata.cli.info(args)

Helper function that takes the parsed command line arguments and prints either verison information or information on the available datasets depending on the arguments in args.

Parameters:

args (Namespace) – A Namespace object that contains commandline arguments parsed by ArgumentParser.parse_args().

Dataset Object

class coderdata.dataset.dataset.Dataset(name=None, transcriptomics=None, proteomics=None, mutations=None, copy_number=None, samples=None, drugs=None, drug_descriptors=None, mirna=None, experiments=None, methylation=None, metabolomics=None, genes=None, combinations=None)
coderdata.dataset.dataset.Dataset.save(self, path)

Save the dataset to a file.

Parameters:

path (Path) – The file path where the dataset will be saved.

Return type:

None

_summary_

coderdata.dataset.dataset.format(data, data_type, remove_na=False, **kwargs)

Format the dataset according to the specified type.

Parameters:

  • data_type (str) – The type of data to format (e.g., ‘transcriptomics’, ‘mutations’).

  • remove_na (bool, optional) – Whether to remove rows with missing values, by default False.

  • **kwargs (dict) – Additional arguments for customization.

Return type:

Formatted data based on the requested type.

coderdata.dataset.dataset.load(name, local_path=PosixPath('/home/runner/work/coderdata/coderdata'), from_pickle=False)

Load a dataset from local files.

This function allows loading either from raw data files (e.g., CSV, TSV) or from a pickled file. The raw data is parsed and indexed into a Dataset object based on predefined types. If pickled data is available, it can be directly loaded for faster access.

Parameters:

  • name (str) – The name of the dataset to load (used as a filename prefix).

  • local_path (str | Path, optional) – The local directory where the dataset files are located, by default the current working directory.

  • from_pickle (bool, optional) – If True, attempts to load the dataset from a pickled file, by default False.

Returns:

An object containing the loaded dataset with attributes for specific data types like ‘transcriptomics’, ‘proteomics’, ‘mutations’, etc.

Return type:

Dataset

Raises:

  • OSError – If the specified directory does not exist.

  • TypeError – If the provided path is not a valid path.

  • FileNotFoundError – If no suitable pickled file is found when from_pickle=True.

Notes

  • When loading from raw files, supported file formats are .csv, .tsv, .csv.gz, .tsv.gz.

  • The genes dataset is subsetted to include only genes relevant to other subdatasets (‘transcriptomics’, ‘proteomics’, etc.).

  • When loading from pickle, the function looks for files with extensions .pkl or .pickle.

Examples

Load a dataset from raw files: >>> dataset = load(name=’my_dataset’, local_path=’/data/datasets’)

Load a dataset from a pickled file: >>> dataset = load(name=’my_dataset’, local_path=’/data/datasets’, from_pickle=True)

coderdata.dataset.dataset.split_train_other(data, split_type='mixed-set', ratio=(8, 2), stratify_by=None, balance=False, random_state=None, **kwargs)

Split the dataset into training and other subsets.

Parameters:

  • split_type ({'mixed-set', 'drug-blind', 'cancer-blind'}, optional) – The type of splitting to perform, by default ‘mixed-set’.

  • ratio (tuple[int, int], optional) – Ratio of train to other split sizes, by default (8, 2).

  • stratify_by (str, optional) – Column to use for stratification, if any, by default None.

  • balance (bool, optional) – Whether to balance the split data, by default False.

  • random_state (int | RandomState | None, optional) – Random seed for reproducibility, by default None.

  • **kwargs (dict) – Additional arguments for customization.

Returns:

The resulting datasets in training and other split.

Return type:

TwoWaySplit

coderdata.dataset.dataset.split_train_test_validate(data, split_type='mixed-set', ratio=(8, 1, 1), stratify_by=None, balance=False, random_state=None, **kwargs)

Split the dataset into training, testing, and validation subsets.

Parameters:

  • split_type ({'mixed-set', 'drug-blind', 'cancer-blind'}, optional) – The type of splitting strategy to use, by default ‘mixed-set’.

  • ratio (tuple[int, int, int], optional) – Ratio for train, test, and validation sizes, by default (8,1,1).

  • stratify_by (str, optional) – Column for stratification, if any, by default None.

  • balance (bool, optional) – Whether to balance the splits, by default False.

  • random_state (int | RandomState | None, optional) – Random seed for reproducible splits, by default None.

  • **kwargs (dict) – Additional arguments for customization.

Returns:

A Split object with train, test, and validation datasets.

Return type:

Split

coderdata.dataset.dataset.train_test_validate(data, split_type='mixed-set', ratio=(8, 1, 1), stratify_by=None, balance=False, random_state=None, **kwargs)

Splits a CoderData object (see also coderdata.load.loader.DatasetLoader) into three subsets for training, testing and validating machine learning algorithms.

The size of the splits can be adjusted to be different from 80:10:10 (the default)for train:test:validate. The function also allows for additional optional arguments, that define the type of split that is performed (‘mixed-set’, ‘drug-blind’, ‘cancer-blind’), if the splits should be stratified (and which drug response metric to use), as well as a random seed to enable the creation of reproducable splits. Furhermore, a list of keyword arguments can be defined that will be passed to the stratification function if so desired.

Parameters:

  • data (DatasetLoader) – CoderData object containing a full dataset either downloaded from the CoderData repository (see also coderdata.download.downloader.download_data_by_prefix) or built locally via the build_all process. The object must first be loaded via coderdata.load.loader.DatasetLoader.

  • split_type ({'mixed-set', 'drug-blind', 'cancer-blind'}, default='mixed-set') –

    Defines the type of split that should be generated:

    • mixed-set: Splits randomly independent of drug / cancer

      association of the samples. Individual drugs or cancer types can appear in all three splits

    • drug-blind: Splits according to drug association. Any sample

      associated with a drug will be unique to one of the splits. For example samples with association to drug A will only be present in the train split, but never in test or validate.

    • cancer-blind: Splits according to cancer association.

      Equivalent to drug-blind, except cancer types will be unique to splits.

  • ratio (tuple[int, int, int], default=(8,1,1)) – Defines the size ratio of the resulting test, train and validation sets.

  • stratify_by (str | None, default=None) – Defines if the training, testing and validation sets should be stratified. Any value other than None indicates stratification and defines which drug response value should be used as basis for the stratification. _None_ indicates that no stratfication should be performed.

  • random_state (int | RandomState | None, defaul=None) – Defines a seed value for the randomization of the splits. Will get passed to internal functions. Providing the seed will enable reproducability of the generated splits.

  • **kwargs – Additional keyword arguments that will be passed to the function that generates classes for the stratification (see also _create_classes).

Returns:

Splits – A Split object that contains three Dataset objects as attributes (Split.train, Split.test, Split.validate)

Return type:

Split

Raises:

  • ValueError : –

  • If supplied split_type is not in the list of accepted values. –

Collection of helper scripts to generate general statistics on the data contained in a CoderData Object.

coderdata.utils.stats.plot_2d_respones_metric(data, metric1, metric2, **kwargs)

Plot a 2D histogram of two response metrics from a dataset.

Parameters:

  • data (cd.Dataset) – The dataset containing experiment data.

  • metric1 (str) – The first response metric to plot on the y-axis.

  • metric2 (str) – The second response metric to plot on the x-axis.

  • **kwargs (dict) – Additional keyword arguments for customizing the plot: - joint_bins (int): Number of bins for the joint histogram. Default is 50. - marginal_bins (int): Number of bins for the marginal histograms. Default is 50.

Returns:

Displays the 2D histogram plot.

Return type:

None

coderdata.utils.stats.plot_response_metric(data, metric='auc', ax=None, **kwargs)

Creates a histogram detailing the distribution of dose response values for a given dose respones metric.

If used in conjunction with matplotlib.pyplot.subplot or matplotlib.pyplot.subplots and the axes object is passed to the function, the function populates the axes object with the generated plot.

Parameters:

  • data (coderdata.DataLoader) – A full CoderData object of a dataset

  • metric (str, default='auc') – A string that defines the response metric that should be plotted

  • ax (matplotlib.axes.Axes, default=None) – An Axes object can be defined. This is uesful if a multipannel subplot has been defined prior via matplotlib.pyplot.subplots. Passing the location of the axes to the function will then populate the subplot at the given location with the generated plot.

  • **kwargs (dict, optional) – Additional keyword arguments that can be passed to the function - bins : int - sets the number of bins; passed to seaborn.histplot - title : str - sets the title of the axes - kde : bool - adds a kernel density estimate plot into the histogram

Return type:

None

Example

In a Jupyter Notebook environment the following snippet can be used to display a histgram detailing the distribution of drug response AUC measures in the beataml dataset.

>>> import coderdata as cd
>>> beataml = cd.DataLoader('beataml')
>>> cd.plot_response_metric(data=beataml, metric='auc', bin=10)

For generating multipanel plots we can make use of matplotlib and the ax parameter of this function. Furthermore, other features / parameters of the cerated figure can be changed (e.g. the title of the figure via suptitle()). Finally it can be saved.

>>> import coderdata as cd
>>> import matplotlib.pyplot as plt
>>> beataml = cd.DataLoader('beataml')
>>> fig, axs = plt.subplots(ncols=2, figsize=(10, 5))
>>> plot_response_metric(
...     data=beataml,
...     metric='auc',
...     bins=10,
...     ax=axs[0]
...     )
>>> plot_response_metric(
...     data=beataml,
...     metric='aac',
...     bins=10,
...     ax=axs[0]
...     )
>>> fig.set_layout_engine('tight')
>>> fig.suptitle('Distribution of drug response values')
>>> fig.savefig('figure.png')
coderdata.utils.stats.summarize_response_metric(data)

Helper function to extract basic statistics for the experiments object in a CoderData object. Uses pandas.DataFrame.describe() internally to generate count, mean, standard deviation, minimum, 25-, 50- and 75-percentile as well as maximum for dose_response_value for each dose_response_metric present in experiments.

Parameters:

data (coderdata.cd.Dataset) – A full CoderData object of a dataset

Returns:

A pandas.DataFrame containing basic statistics for each dose response metric.

Return type:

pandas.DataFrame

Example

The Example assumes that a dataset with the prefix ‘beataml’ has been downloaded previously. See also coderdata.download()

>>> import coderdata as cd
>>> beataml = cd.DataLoader('beataml')
>>> summary_stats = summarize_response_metric(data=beataml)
>>> summary_stats
                        count          mean           std
dose_response_metric
aac                   23378.0  3.028061e-01  1.821265e-01  ...
auc                   23378.0  6.971939e-01  1.821265e-01  ...
dss                   23378.0  3.218484e-01  5.733492e-01  ...
...                   ...      ...           ...           ...