Skip to content

Data Module Reference

This section provides a detailed API reference for all modules related to managing the datasets.

Core Data Utilities

These modules define the common utilities used by all splitters.

set_column_name(columns, value, rename=True, default_name=None)

Identifies a column by its name or index and optionally renames it.

This utility function provides a flexible way to handle DataFrame columns. It can find a column based on its current name (string) or its position (integer). If rename is True, it replaces the found column name in the list of columns with a default_name.

Parameters:

Name Type Description Default
columns list

The list of current column names in the DataFrame.

 required
value Union[str, int]

The identifier for the column, either its name or its integer index.

 required
rename bool

If True, the identified column's name is changed to default_name in the returned list. Defaults to True.

 True
default_name str

The new name for the column if rename is True. Defaults to None.

 None

Returns:

Type Description
tuple[list, str]

A tuple containing: - The (potentially modified) list of column names. - The final name of the selected column (either the original or the default_name if renamed).

Raises:

Type Description
ValueError

If the value is not a valid column name or index, or if it is not a string or integer.
Source code in datarec/data/utils.py 
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
def set_column_name(columns: list, value: Union[str, int], rename=True, default_name=None) -> (list, str):
    """
    Identifies a column by its name or index and optionally renames it.

    This utility function provides a flexible way to handle DataFrame columns. It
    can find a column based on its current name (string) or its position (integer).
    If `rename` is True, it replaces the found column name in the list of columns
    with a `default_name`.

    Args:
        columns (list): The list of current column names in the DataFrame.
        value (Union[str, int]): The identifier for the column, either its name
            or its integer index.
        rename (bool, optional): If True, the identified column's name is
            changed to `default_name` in the returned list. Defaults to True.
        default_name (str, optional): The new name for the column if `rename` is
            True. Defaults to None.

    Returns:
        (tuple[list, str]): A tuple containing:
            - The (potentially modified) list of column names.
            - The final name of the selected column (either the original or the
              `default_name` if renamed).

    Raises:
        ValueError: If the `value` is not a valid column name or index, or if
            it is not a string or integer.
    """
    columns = list(columns)

    if isinstance(value, str):
        if value not in columns:
            raise ValueError(f'column \'{value}\' is not a valid column name.')
        selected_column = value

    elif isinstance(value, int):
        if value in columns:
            selected_column = value
        else:
            if value not in range(len(columns)):
                raise ValueError(f'column int \'{value}\' is out of range ({len(columns)} columns).')
            selected_column = columns[value]
    else:
        raise ValueError(f'column value must be either a string (column name) or an integer (column index)')

    if rename is True:
        columns[columns.index(selected_column)] = default_name
        return columns, default_name

    return columns, selected_column

quartiles(count)

Assigns quartile indices (0-3) to items based on their frequency counts.

The function divides the input values into four quartiles using the median and quantiles. Each item is assigned an integer: 0: long tail (lowest quartile) 1: common 2: popular 3: most popular (highest quartile)

Parameters:

Name Type Description Default
count dict

A dictionary mapping items to numeric counts or frequencies.

 required

Returns:

Type Description
dict

A dictionary mapping each item to its quartile index (0-3).
Source code in datarec/data/utils.py 
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
def quartiles(count: dict):
    """ 
    Assigns quartile indices (0-3) to items based on their frequency counts.

    The function divides the input values into four quartiles using the 
    median and quantiles. Each item is assigned an integer:
        0: long tail (lowest quartile)
        1: common
        2: popular
        3: most popular (highest quartile)

    Args:
        count (dict): A dictionary mapping items to numeric counts or frequencies.

    Returns:
        (dict): A dictionary mapping each item to its quartile index (0-3).
    """
    q1, q2, q3 = statistics.quantiles(count.values())

    def assign(value):
        if value <= q2:
            if value <= q1:
                return 0
            else:
                return 1
        else:
            if value <= q3:
                return 2
            else:
                return 3

    return {k: assign(f) for k, f in count.items()}

popularity(quartiles)

Categorizes items based on their quartile indices.

Converts quartile indices (0-3) into descriptive popularity categories: 0 -> 'long tail' 1 -> 'common' 2 -> 'popular' 3 -> 'most popular'

Parameters:

Name Type Description Default
quartiles dict

A dictionary mapping items to quartile indices (0-3).

 required

Returns:

Type Description
dict

A dictionary mapping each popularity category to a list of items.
Source code in datarec/data/utils.py 
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
def popularity(quartiles: dict):
    """ 
    Categorizes items based on their quartile indices.

    Converts quartile indices (0-3) into descriptive popularity categories:
        0 -> 'long tail'
        1 -> 'common'
        2 -> 'popular'
        3 -> 'most popular'

    Args:
        quartiles (dict): A dictionary mapping items to quartile indices (0-3).

    Returns:
        (dict): A dictionary mapping each popularity category to a list of items.
    """
    categories_map = \
        {3: 'most popular',
         2: 'popular',
         1: 'common',
         0: 'long tail'}

    categories = \
        {'most popular': [],
         'popular': [],
         'common': [],
         'long tail': []}

    for k, q in quartiles.items():
        categories[categories_map[q]].append(k)

    return categories

verify_checksum(file_path, checksum)

Verifies the MD5 checksum of a file.

This function computes the MD5 hash of the file at the given path and compares it to the expected checksum. If the file does not exist, a FileNotFoundError is raised. If the checksum does not match, a RuntimeError is raised indicating possible corruption or version mismatch.

Parameters:

Name Type Description Default
file_path str

The path to the file to verify.

 required
checksum str

The expected MD5 checksum.

 required

Raises:

Type Description
FileNotFoundError

If the specified file does not exist.
RuntimeError

If the computed checksum does not match the expected value.
Source code in datarec/data/utils.py 
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
def verify_checksum(file_path: str, checksum: str) -> None:
    """
    Verifies the MD5 checksum of a file.

    This function computes the MD5 hash of the file at the given path and
    compares it to the expected checksum. If the file does not exist, a
    FileNotFoundError is raised. If the checksum does not match, a RuntimeError
    is raised indicating possible corruption or version mismatch.

    Args:
        file_path (str): The path to the file to verify.
        checksum (str): The expected MD5 checksum.

    Raises:
        FileNotFoundError: If the specified file does not exist.
        RuntimeError: If the computed checksum does not match the expected value.
    """

    if not os.path.isfile(file_path):
        raise FileNotFoundError(f'File \'{file_path}\ not found.')

    md5 = hashlib.md5()
    with open(file_path, "rb") as f:
        for chunck in iter(lambda: f.read(65536), b""):
            md5.update(chunck)

    digest = md5.hexdigest()
    if not digest == checksum:
        raise RuntimeError(f"Checksum mismatch for '{file_path}': expected {checksum}, but got {digest}. "
                           f"The file may be corrupted or a new version has been downloaded.")

    print(f'Checksum verified.')

Dataset wrappers

BaseTorchDataset

Bases: Dataset

Base class for Torch datasets wrapping a DataRec dataset.

Source code in datarec/data/torch_dataset.py 
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
class BaseTorchDataset(Dataset):
    """
    Base class for Torch datasets wrapping a DataRec dataset.
    """
    def __init__(self, datarec, copy_data=False):
        """
        Initializes the BaseTorchDataset object.    

        Args:
            datarec (DataRec): An instance of a DataRec dataset.
            copy_data (bool): Whether to copy the dataset or use it by reference.
        """
        self.df = datarec.data.copy() if copy_data else datarec.data
        self.user_col = datarec.user_col
        self.item_col = datarec.item_col

__init__(datarec, copy_data=False)

Initializes the BaseTorchDataset object.

Parameters:

Name Type Description Default
datarec DataRec

An instance of a DataRec dataset.

 required
copy_data bool

Whether to copy the dataset or use it by reference.

 False
Source code in datarec/data/torch_dataset.py 
10
11
12
13
14
15
16
17
18
19
20
def __init__(self, datarec, copy_data=False):
    """
    Initializes the BaseTorchDataset object.    

    Args:
        datarec (DataRec): An instance of a DataRec dataset.
        copy_data (bool): Whether to copy the dataset or use it by reference.
    """
    self.df = datarec.data.copy() if copy_data else datarec.data
    self.user_col = datarec.user_col
    self.item_col = datarec.item_col

PointwiseTorchDataset

Bases: BaseTorchDataset

Torch dataset for pointwise recommendation tasks.

Source code in datarec/data/torch_dataset.py 
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
class PointwiseTorchDataset(BaseTorchDataset):
    """
    Torch dataset for pointwise recommendation tasks.
    """
    def __init__(self, datarec, copy_data=False):
        """
        Initializes the PointwiseTorchDataset object.

        Args:
            datarec (DataRec): An instance of a DataRec dataset.
            copy_data (bool): Whether to copy the dataset or use it by reference.
        """
        super().__init__(datarec, copy_data)
        self.rating_col = datarec.rating_col

    def __len__(self):
        """
        Returns the total number of samples in the dataset.

        This is required by PyTorch's DataLoader to iterate over the dataset.

        Returns:
            (int): Number of samples in the dataset.
        """
        return len(self.df)

    def __getitem__(self, idx):
        """
        Returns a sample with user, item, and rating.

        Args:
            idx (int): Sample index to be returned.

        Returns:
            (dict): Sample with user, item, and rating.
        """
        row = self.df.iloc[idx]
        return {
            "user": row[self.user_col],
            "item": row[self.item_col],
            "rating": row.get(self.rating_col, 1.0)
        }

__init__(datarec, copy_data=False)

Initializes the PointwiseTorchDataset object.

Parameters:

Name Type Description Default
datarec DataRec

An instance of a DataRec dataset.

 required
copy_data bool

Whether to copy the dataset or use it by reference.

 False
Source code in datarec/data/torch_dataset.py 
27
28
29
30
31
32
33
34
35
36
def __init__(self, datarec, copy_data=False):
    """
    Initializes the PointwiseTorchDataset object.

    Args:
        datarec (DataRec): An instance of a DataRec dataset.
        copy_data (bool): Whether to copy the dataset or use it by reference.
    """
    super().__init__(datarec, copy_data)
    self.rating_col = datarec.rating_col

__len__()

Returns the total number of samples in the dataset.

This is required by PyTorch's DataLoader to iterate over the dataset.

Returns:

Type Description
int

Number of samples in the dataset.
Source code in datarec/data/torch_dataset.py 
38
39
40
41
42
43
44
45
46
47
def __len__(self):
    """
    Returns the total number of samples in the dataset.

    This is required by PyTorch's DataLoader to iterate over the dataset.

    Returns:
        (int): Number of samples in the dataset.
    """
    return len(self.df)

__getitem__(idx)

Returns a sample with user, item, and rating.

Parameters:

Name Type Description Default
idx int

Sample index to be returned.

 required

Returns:

Type Description
dict

Sample with user, item, and rating.
Source code in datarec/data/torch_dataset.py 
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
def __getitem__(self, idx):
    """
    Returns a sample with user, item, and rating.

    Args:
        idx (int): Sample index to be returned.

    Returns:
        (dict): Sample with user, item, and rating.
    """
    row = self.df.iloc[idx]
    return {
        "user": row[self.user_col],
        "item": row[self.item_col],
        "rating": row.get(self.rating_col, 1.0)
    }

PairwiseTorchDataset

Bases: BaseTorchDataset

Torch dataset for pairwise recommendation tasks with negative sampling.

Source code in datarec/data/torch_dataset.py 
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
class PairwiseTorchDataset(BaseTorchDataset):
    """
    Torch dataset for pairwise recommendation tasks with negative sampling.
    """
    def __init__(self, datarec, num_negatives=1, item_pool=None, copy_data=False):
        """ 
        Initializes the PairwiseTorchDataset object.

        Args:
            datarec (DataRec): An instance of a DataRec dataset.
            num_negatives (int): Number of negative samples to generate per interaction.
            item_pool (array-like): Pool of items to sample from. Defaults to all items in the dataset.
            copy_data (bool): Whether to copy the dataset or use it by reference.
        """
        super().__init__(datarec, copy_data)
        self.num_negatives = num_negatives
        self.item_pool = item_pool or self.df[self.item_col].unique()
        self.user_pos_items = self.df.groupby(self.user_col)[self.item_col].apply(set).to_dict()

    def sample_negatives(self, user):
        """
        Samples negative items for a given user, avoiding known positive items.

        This method is designed to be overridden to implement custom negative
        sampling strategies (e.g., popularity-based, adversarial, or
        distribution-aware sampling). The default implementation draws
        uniformly from the item pool, excluding items the user has already interacted with.

        Args:
            user: The user ID for which to sample negatives.

        Returns:
            (List): List of sampled negative item IDs.
        """
        neg_items = []
        user_positives = self.user_pos_items.get(user, set())
        while len(neg_items) < self.num_negatives:
            candidate = np.random.choice(self.item_pool)
            if candidate not in user_positives:
                neg_items.append(candidate)
        return neg_items

    def __len__(self):
        """
        Returns the total number of samples in the dataset.

        This is required by PyTorch's DataLoader to iterate over the dataset.

        Returns:
            (int): number of samples in the dataset.
        """
        return len(self.df)

    def __getitem__(self, idx):
        """
        Returns a sample with user, positive item, and negative items.

        Args:
            idx (int): Sample index to be returned.

        Returns:
            (dict): Sample with user, positive item, and negative items.
        """
        row = self.df.iloc[idx]
        user = row[self.user_col]
        pos_item = row[self.item_col]
        neg_items = self.sample_negatives(user)
        return {
            "user": user,
            "pos_item": pos_item,
            "neg_items": neg_items
        }

__init__(datarec, num_negatives=1, item_pool=None, copy_data=False)

Initializes the PairwiseTorchDataset object.

Parameters:

Name Type Description Default
datarec DataRec

An instance of a DataRec dataset.

 required
num_negatives int

Number of negative samples to generate per interaction.

 1
item_pool array - like

Pool of items to sample from. Defaults to all items in the dataset.

 None
copy_data bool

Whether to copy the dataset or use it by reference.

 False
Source code in datarec/data/torch_dataset.py 
71
72
73
74
75
76
77
78
79
80
81
82
83
84
def __init__(self, datarec, num_negatives=1, item_pool=None, copy_data=False):
    """ 
    Initializes the PairwiseTorchDataset object.

    Args:
        datarec (DataRec): An instance of a DataRec dataset.
        num_negatives (int): Number of negative samples to generate per interaction.
        item_pool (array-like): Pool of items to sample from. Defaults to all items in the dataset.
        copy_data (bool): Whether to copy the dataset or use it by reference.
    """
    super().__init__(datarec, copy_data)
    self.num_negatives = num_negatives
    self.item_pool = item_pool or self.df[self.item_col].unique()
    self.user_pos_items = self.df.groupby(self.user_col)[self.item_col].apply(set).to_dict()

sample_negatives(user)

Samples negative items for a given user, avoiding known positive items.

This method is designed to be overridden to implement custom negative sampling strategies (e.g., popularity-based, adversarial, or distribution-aware sampling). The default implementation draws uniformly from the item pool, excluding items the user has already interacted with.

Parameters:

Name Type Description Default
user

The user ID for which to sample negatives.

 required

Returns:

Type Description
List

List of sampled negative item IDs.
Source code in datarec/data/torch_dataset.py 
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
def sample_negatives(self, user):
    """
    Samples negative items for a given user, avoiding known positive items.

    This method is designed to be overridden to implement custom negative
    sampling strategies (e.g., popularity-based, adversarial, or
    distribution-aware sampling). The default implementation draws
    uniformly from the item pool, excluding items the user has already interacted with.

    Args:
        user: The user ID for which to sample negatives.

    Returns:
        (List): List of sampled negative item IDs.
    """
    neg_items = []
    user_positives = self.user_pos_items.get(user, set())
    while len(neg_items) < self.num_negatives:
        candidate = np.random.choice(self.item_pool)
        if candidate not in user_positives:
            neg_items.append(candidate)
    return neg_items

__len__()

Returns the total number of samples in the dataset.

This is required by PyTorch's DataLoader to iterate over the dataset.

Returns:

Type Description
int

number of samples in the dataset.
Source code in datarec/data/torch_dataset.py 
109
110
111
112
113
114
115
116
117
118
def __len__(self):
    """
    Returns the total number of samples in the dataset.

    This is required by PyTorch's DataLoader to iterate over the dataset.

    Returns:
        (int): number of samples in the dataset.
    """
    return len(self.df)

__getitem__(idx)

Returns a sample with user, positive item, and negative items.

Parameters:

Name Type Description Default
idx int

Sample index to be returned.

 required

Returns:

Type Description
dict

Sample with user, positive item, and negative items.
Source code in datarec/data/torch_dataset.py 
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
def __getitem__(self, idx):
    """
    Returns a sample with user, positive item, and negative items.

    Args:
        idx (int): Sample index to be returned.

    Returns:
        (dict): Sample with user, positive item, and negative items.
    """
    row = self.df.iloc[idx]
    user = row[self.user_col]
    pos_item = row[self.item_col]
    neg_items = self.sample_negatives(user)
    return {
        "user": user,
        "pos_item": pos_item,
        "neg_items": neg_items
    }

RankingTorchDataset

Bases: BaseTorchDataset

Torch dataset for full softmax-style ranking tasks.

Source code in datarec/data/torch_dataset.py 
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
class RankingTorchDataset(BaseTorchDataset):
    """
    Torch dataset for full softmax-style ranking tasks.
    """
    def __init__(self, datarec, copy_data=False):
        """
        Initializes the RankingTorchDataset object.

        Args:
            datarec (DataRec): An instance of a DataRec dataset.
            copy_data (bool): Whether to copy the dataset or use it by reference.
        """
        super().__init__(datarec, copy_data)
        # Could prepare user->items mapping here for evaluation

    def __len__(self):
        """
        Returns the total number of samples in the dataset.

        This is required by PyTorch's DataLoader to iterate over the dataset.

        Returns:
            (int): Number of samples in the dataset.
        """
        return len(self.df)

    def __getitem__(self, idx):
        """
        Returns a sample with user and item.

        Args:
            idx (int): Sample index to be returned.

        Returns:
            (dict): Sample with user and item data.
        """
        row = self.df.iloc[idx]
        return {
            "user": row[self.user_col],
            "item": row[self.item_col]
            # No target — implicit ranking
        }

__init__(datarec, copy_data=False)

Initializes the RankingTorchDataset object.

Parameters:

Name Type Description Default
datarec DataRec

An instance of a DataRec dataset.

 required
copy_data bool

Whether to copy the dataset or use it by reference.

 False
Source code in datarec/data/torch_dataset.py 
145
146
147
148
149
150
151
152
153
def __init__(self, datarec, copy_data=False):
    """
    Initializes the RankingTorchDataset object.

    Args:
        datarec (DataRec): An instance of a DataRec dataset.
        copy_data (bool): Whether to copy the dataset or use it by reference.
    """
    super().__init__(datarec, copy_data)

__len__()

Returns the total number of samples in the dataset.

This is required by PyTorch's DataLoader to iterate over the dataset.

Returns:

Type Description
int

Number of samples in the dataset.
Source code in datarec/data/torch_dataset.py 
156
157
158
159
160
161
162
163
164
165
def __len__(self):
    """
    Returns the total number of samples in the dataset.

    This is required by PyTorch's DataLoader to iterate over the dataset.

    Returns:
        (int): Number of samples in the dataset.
    """
    return len(self.df)

__getitem__(idx)

Returns a sample with user and item.

Parameters:

Name Type Description Default
idx int

Sample index to be returned.

 required

Returns:

Type Description
dict

Sample with user and item data.
Source code in datarec/data/torch_dataset.py 
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
def __getitem__(self, idx):
    """
    Returns a sample with user and item.

    Args:
        idx (int): Sample index to be returned.

    Returns:
        (dict): Sample with user and item data.
    """
    row = self.df.iloc[idx]
    return {
        "user": row[self.user_col],
        "item": row[self.item_col]
        # No target — implicit ranking
    }

The DataRec class

These modules define the core part of the framework: the DataRec class.

BaseDataRecBuilder

Bases: ABC

Abstract base class for building DataRec datasets.

This class defines the interface for preparing, downloading, and loading datasets into DataRec objects.

Source code in datarec/data/datarec_builder.py 
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
class BaseDataRecBuilder(ABC):
    """
    Abstract base class for building `DataRec` datasets.

    This class defines the interface for preparing, downloading, and loading 
    datasets into `DataRec` objects. 
    """

    @abstractmethod
    def prepare(self) -> None:
        """Download and process the dataset, without loading it into memory."""
        pass

    @abstractmethod
    def load(self) -> DataRec:
        """Load the processed dataset into a DataRec object."""
        pass

    def prepare_and_load(self) -> DataRec:
        """
        A convenience method that runs the full prepare and load pipeline.

        Returns:
            (DataRec): The fully prepared and loaded dataset.
        """
        self.prepare()
        return self.load()

    @abstractmethod
    def download(self) -> str:
        """Download the raw dataset files."""
        pass

prepare() abstractmethod

Download and process the dataset, without loading it into memory.

Source code in datarec/data/datarec_builder.py 
13
14
15
16
@abstractmethod
def prepare(self) -> None:
    """Download and process the dataset, without loading it into memory."""
    pass

load() abstractmethod

Load the processed dataset into a DataRec object.

Source code in datarec/data/datarec_builder.py 
18
19
20
21
@abstractmethod
def load(self) -> DataRec:
    """Load the processed dataset into a DataRec object."""
    pass

prepare_and_load()

A convenience method that runs the full prepare and load pipeline.

Returns:

Type Description
DataRec

The fully prepared and loaded dataset.
Source code in datarec/data/datarec_builder.py 
23
24
25
26
27
28
29
30
31
def prepare_and_load(self) -> DataRec:
    """
    A convenience method that runs the full prepare and load pipeline.

    Returns:
        (DataRec): The fully prepared and loaded dataset.
    """
    self.prepare()
    return self.load()

download() abstractmethod

Download the raw dataset files.

Source code in datarec/data/datarec_builder.py 
33
34
35
36
@abstractmethod
def download(self) -> str:
    """Download the raw dataset files."""
    pass

DataRec

Core data structure for recommendation datasets in the DataRec framework.

This class wraps a Pandas DataFrame and standardizes common columns (user, item, rating, timestamp) to provide a consistent interface for recommendation tasks. It supports data preprocessing, user/item remapping (public vs private IDs), frequency analysis, sparsity/density metrics, Gini coefficients, and conversion into PyTorch datasets for training.

Source code in datarec/data/dataset.py 
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
class DataRec:

    """
    Core data structure for recommendation datasets in the DataRec framework.

    This class wraps a Pandas DataFrame and standardizes common columns
    (user, item, rating, timestamp) to provide a consistent interface
    for recommendation tasks. It supports data preprocessing, 
    user/item remapping (public vs private IDs), frequency analysis, 
    sparsity/density metrics, Gini coefficients, and conversion into 
    PyTorch datasets for training.
    """

    def __init__(
            self,
            rawdata: RawData = None,
            copy: bool = False,
            dataset_name: str = 'datarec',
            version_name: str = 'no_version_provided',
            pipeline: Optional[Pipeline] = None,
            *args,
            **kwargs
    ):
        """
        Initializes the DataRec object.

        Args:
            rawdata (RawData): The input dataset wrapped in a RawData object.
                If None, the DataRec is initialized empty.
            copy (bool): Whether to copy the input DataFrame to avoid 
                modifying the original RawData.
            dataset_name (str): A name to identify the dataset.
            version_name (str): A version identifier 
                for the dataset.
            pipeline (Pipeline): A pipeline object to track preprocessing steps.

        """



        self.path = None
        self._data = None
        self.dataset_name = dataset_name
        self.version_name = version_name

        if pipeline:
            self.pipeline = pipeline
        else:
            self.pipeline = Pipeline()
            self.pipeline.add_step("load", self.dataset_name, {'version': self.version_name})

        if rawdata is not None:
            if copy:
                self._data: pd.DataFrame = rawdata.data.copy()
            else:
                self._data: pd.DataFrame = rawdata.data

        # ------------------------------------
        # --------- STANDARD COLUMNS ---------
        # if a column is None it means that the DataRec does not have that information
        self.__assigned_columns = []

        self._user_col = None
        self._item_col = None
        self._rating_col = None
        self._timestamp_col = None

        if rawdata:
            self.set_columns(rawdata)

        # dataset is assumed to be the public version of the dataset
        self._is_private = False
        self.__implicit = False

        # ------------------------------
        # --------- PROPERTIES ---------
        self._sorted_users = None
        self._sorted_items = None

        # map users and items with a 0-indexed mapping
        self._public_to_private_users = None
        self._public_to_private_items = None
        self._private_to_public_users = None
        self._private_to_public_items = None

        # metrics
        self._transactions = None
        self._space_size = None
        self._space_size_log = None
        self._shape = None
        self._shape_log = None
        self._density = None
        self._density_log = None
        self._gini_item = None
        self._gini_user = None
        self._ratings_per_user = None
        self._ratings_per_item = None

        # more analyses
        self.metrics = ['transactions', 'space_size', 'space_size_log', 'shape', 'shape_log', 'density', 'density_log',
                        'gini_item', 'gini_user', 'ratings_per_user', 'ratings_per_item']

    def __str__(self):
        """
        Returns 'self.data' as a string variable.

        Returns:
            (str): 'self.data' as a string variable.
        """
        return self.data.__str__()

    def __repr__(self):
        """
        Returns the official string representation of the internal DataFrame.
        """
        return self.data.__repr__()

    def _repr_html_(self):
        """
        Returns an HTML representation of the internal DataFrame for rich displays.
        """
        return self.data._repr_html_()

    def __len__(self):
        """
        Returns the total number of samples in the dataset.

        Returns:
            (int): number of samples in the dataset.
        """
        return len(self.data)

    def set_columns(self, rawdata):
        """
        Assign dataset column names from a RawData object and reorder the data accordingly.

        Args:
            rawdata (RawData): A RawData object containing the column names for 
                user, item, rating, and timestamp.
        """
        if rawdata.user is not None:
            self.user_col = rawdata.user
            self.__assigned_columns.append(self.user_col)
        if rawdata.item is not None:
            self.item_col = rawdata.item
            self.__assigned_columns.append(self.item_col)
        if rawdata.rating is not None:
            self.rating_col = rawdata.rating
            self.__assigned_columns.append(self.rating_col)
        if rawdata.timestamp is not None:
            self.timestamp_col = rawdata.timestamp
            self.__assigned_columns.append(self.timestamp_col)

        # re-order columns
        self._data = self.data[self.__assigned_columns]

    def reset(self):

        """
        Reset cached statistics and assigned columns of the DataRec object.

        This method clears all precomputed dataset statistics (e.g., sorted users, 
        density, Gini indices, shape, ratings per user/item) and empties the list 
        of assigned columns. It is automatically called when the underlying data is changed.
        """
        self._sorted_users = None
        self._sorted_items = None
        self._transactions = None
        self._space_size = None
        self._space_size_log = None
        self._shape = None
        self._shape_log = None
        self._density = None
        self._density_log = None
        self._gini_item = None
        self._gini_user = None
        self._ratings_per_user = None
        self._ratings_per_item = None

        self.__assigned_columns = []

    @property
    def data(self):
        """
        The underlying pandas DataFrame holding the interaction data.
        """
        return self._data

    @data.setter
    def data(self, value: RawData):
        """
        Sets the internal DataFrame from a RawData object and resets stats.
        """
        if (value is not None and
                not isinstance(value, RawData)):
            raise ValueError(f'Data must be RawData or None if empty. Found {type(value)}')
        value = value if value is not None else pd.DataFrame()

        self._data = value.data
        self.reset()
        self.set_columns(value)

    @property
    def user_col(self):
        """
        The name of the user ID column.
        """
        return self._user_col

    @user_col.setter
    def user_col(self, value: Union[str, int]):
        """
        Sets and renames the user column to the internal standard name.
        """
        self.set_user_col(value, rename=True)

    def set_user_col(self, value: Union[str, int] = DATAREC_USER_COL, rename=True):
        """
        Identifies and optionally renames the user column.

        Args:
            value (Union[str, int]): The current name or index of the user column.
            rename (bool): If True, renames the column to the standard internal name.
        """
        self.data.columns, self._user_col = set_column_name(columns=list(self.data.columns),
                                                            value=value,
                                                            default_name=DATAREC_USER_COL,
                                                            rename=rename)

    @property
    def item_col(self):
        """
        The name of the item ID column.
        """
        return self._item_col

    @item_col.setter
    def item_col(self, value: Union[str, int]):
        """
        Sets and renames the item column to the internal standard name.
        """
        self.set_item_col(value, rename=True)

    def set_item_col(self, value: Union[str, int] = DATAREC_ITEM_COL, rename=True):
        """
        Identifies and optionally renames the item column.

        Args:
            value (Union[str, int]): The current name or index of the item column.
            rename (bool): If True, renames the column to the standard internal name.
        """
        self.data.columns, self._item_col = set_column_name(columns=list(self.data.columns),
                                                            value=value,
                                                            default_name=DATAREC_ITEM_COL,
                                                            rename=rename)

    @property
    def rating_col(self):
        """
        The name of the rating column.
        """
        return self._rating_col

    @rating_col.setter
    def rating_col(self, value: Union[str, int]):
        """
        Sets and renames the rating column to the internal standard name.
        """
        self.set_rating_col(value, rename=True)

    def set_rating_col(self, value: Union[str, int] = DATAREC_RATING_COL, rename=True):
        """
        Identifies and optionally renames the rating column.

        Args:
            value (Union[str, int]): The current name or index of the rating column.
            rename (bool): If True, renames the column to the standard internal name.
        """
        self.data.columns, self._rating_col = set_column_name(columns=list(self.data.columns),
                                                              value=value,
                                                              default_name=DATAREC_RATING_COL,
                                                              rename=rename)

    @property
    def timestamp_col(self):
        """
        The name of the timestamp column.
        """
        return self._timestamp_col

    @timestamp_col.setter
    def timestamp_col(self, value: Union[str, int]):
        """
        Sets and renames the timestamp column to the internal standard name.
        """
        self.set_timestamp_col(value, rename=True)

    def set_timestamp_col(self, value: Union[str, int] = DATAREC_TIMESTAMP_COL, rename=True):
        """
        Identifies and optionally renames the timestamp column.

        Args:
            value (Union[str, int]): The current name or index of the timestamp column.
            rename (bool): If True, renames the column to the standard internal name.
        """
        self.data.columns, self._timestamp_col = set_column_name(columns=list(self.data.columns),
                                                                 value=value,
                                                                 default_name=DATAREC_TIMESTAMP_COL,
                                                                 rename=rename)

    @property
    def users(self):
        """
        Returns a list of unique user IDs in the dataset.
        """
        return self.data[self.user_col].unique().tolist()

    @property
    def items(self):
        """
        Returns a list of unique item IDs in the dataset.
        """
        return self.data[self.item_col].unique().tolist()

    @property
    def n_users(self):
        """
        Returns the number of unique users.
        """
        return int(self.data[self.user_col].nunique())

    @property
    def n_items(self):
        """
        Returns the number of unique items.
        """
        return int(self.data[self.item_col].nunique())

    @property
    def columns(self):
        """
        Returns the list of column names of the internal DataFrame.
        """
        return self.data.columns

    @columns.setter
    def columns(self, columns):
        """
        Sets the column names of the internal DataFrame.
        """
        self.data.columns = columns

    @property
    def sorted_items(self):
        """
        Returns a dictionary of items sorted by their interaction count.
        """
        if self._sorted_items is None:
            count_items = self.data.groupby(self.item_col).count().sort_values(by=[self.user_col])
            self._sorted_items = dict(zip(count_items.index, count_items[self.user_col]))
        return self._sorted_items

    @property
    def sorted_users(self):
        """
        Returns a dictionary of users sorted by their interaction count.
        """
        if self._sorted_users is None:
            count_users = self.data.groupby(self.user_col).count().sort_values(by=[self.item_col])
            self._sorted_users = dict(zip(count_users.index, count_users[self.item_col]))
        return self._sorted_users

    # --- MAPPING FUNCTIONS ---
    @property
    def transactions(self):
        """
        Returns the total number of interactions (rows) in the dataset.
        """
        if self._transactions is None:
            self._transactions = len(self.data)
        return self._transactions

    @staticmethod
    def public_to_private(lst, offset=0):
        """
        Creates a mapping from public (original) IDs to private (integer) IDs.

        Args:
            lst (list): A list of public IDs.
            offset (int): The starting integer for the private IDs.

        Returns:
            (dict): A dictionary mapping public IDs to private integer IDs.
        """
        return dict(zip(lst, range(offset, offset + len(lst))))

    @staticmethod
    def private_to_public(pub_to_prvt: dict):
        """
        Creates a reverse mapping from private IDs back to public IDs.

        Args:
            pub_to_prvt (dict): A dictionary mapping public IDs to private IDs.

        Returns:
            (dict): A dictionary mapping private IDs to public IDs.
        """
        mapping = {el: idx for idx, el in pub_to_prvt.items()}
        if len(pub_to_prvt) != len(mapping):
            print('WARNING: private to public mapping could be incorrect. Please, check your code.')
        return mapping

    def map_users_and_items(self, offset=0, items_shift=False):
        """
        Generates the public-to-private and private-to-public ID mappings.

        This method creates the dictionaries needed to convert user and item IDs
        to a dense, zero-indexed integer range suitable for machine learning models.

        Args:
            offset (int): The starting integer for the ID mappings. Defaults to 0.
            items_shift (bool): If True, item private IDs will start after the last
                user private ID, creating a single contiguous ID space. Defaults to False.
        """
        # map users and items with a 0-indexed mapping
        users_offset = offset
        items_offset = offset

        # users
        self._public_to_private_users = self.public_to_private(self.users, offset=users_offset)
        self._private_to_public_users = self.private_to_public(self._public_to_private_users)

        # items
        if items_shift:
            items_offset = offset + self.n_users
        self._public_to_private_items = self.public_to_private(self.items, offset=items_offset)
        self._private_to_public_items = self.private_to_public(self._public_to_private_items)

    def map_dataset(self, user_mapping, item_mapping):
        """
        Applies ID mappings to the user and item columns of the DataFrame.

        This is an in-place operation that modifies the internal DataFrame.

        Args:
            user_mapping (dict): The dictionary to map user IDs.
            item_mapping (dict): The dictionary to map item IDs.
        """
        self.data[self.user_col] = self.data[self.user_col].map(user_mapping)
        self.data[self.item_col] = self.data[self.item_col].map(item_mapping)

    def to_public(self):
        """
        Converts user and item IDs back to their original (public) values.
        """
        if self._is_private:
            self.map_dataset(self._private_to_public_users, self._private_to_public_items)
        self._is_private = False

    def to_private(self):
        """
        Converts user and item IDs to their dense, zero-indexed (private) integer values.
        """
        if not self._is_private:
            self.map_dataset(self._public_to_private_users, self._public_to_private_items)
        self._is_private = True

    # -- METRICS --
    def get_metric(self, metric):
        """
        Retrieves a calculated dataset metric by name.

        Args:
            metric (str): The name of the metric to compute (e.g., 'density', 'gini_user').

        Returns:
            The value of the computed metric.
        """
        assert metric in self.metrics, f'{self.__class__.__name__}: metric \'{metric}\' not found.'
        func = getattr(self, metric)
        return func()

    @property
    def space_size(self):
        """
        Calculates the scaled square root of the user-item interaction space.
        """
        if self._space_size is None:
            scale_factor = 1000
            self._space_size = math.sqrt(self.n_users * self.n_items) / scale_factor
        return self._space_size

    @property
    def space_size_log(self):
        """
        Calculates the log10 of the space_size metric.
        """
        if self._space_size_log is None:
            self._space_size_log = math.log10(self.space_size)
        return self._space_size_log

    @property
    def shape(self):
        """
        Calculates the shape of the interaction matrix (n_users / n_items).
        """
        if self._shape is None:
            self._shape = self.n_users / self.n_items
        return self._shape

    @property
    def shape_log(self):
        """
        Calculates the log10 of the shape metric.
        """
        if self._shape_log is None:
            self._shape_log = math.log10(self.shape)
        return self._shape_log

    @property
    def density(self):
        """
        Calculates the density of the user-item interaction matrix.
        """
        if self._density is None:
            self._density = self.transactions / (self.n_users * self.n_items)
        return self._density

    @property
    def density_log(self):
        """
        Calculates the log10 of the density metric.
        """
        if self._density_log is None:
            self._density_log = math.log10(self.density)
        return self._density_log

    @staticmethod
    def gini(x):
        """
        Calculates the Gini coefficient for a numpy array.

        Args:
            x (np.ndarray): An array of non-negative values.

        Returns:
            (float): The Gini coefficient, a measure of inequality.
        """
        x = np.sort(x)  # O(n log n)
        n = len(x)
        cum_index = np.arange(1, n + 1)
        return (np.sum((2 * cum_index - n - 1) * x)) / (n * np.sum(x))


    @property
    def gini_item(self):
        """
        Calculates the Gini coefficient for item popularity.
        """
        if self._gini_item is None:
            self._gini_item = self.gini(np.array(list(self.sorted_items.values())))
        return self._gini_item

    @property
    def gini_user(self):
        """
        Calculates the Gini coefficient for user activity.
        """
        if self._gini_user is None:
            self._gini_user = self.gini(np.array(list(self.sorted_users.values())))
        return self._gini_user

    @property
    def ratings_per_user(self):
        """
        Calculates the average number of ratings per user.
        """
        if self._ratings_per_user is None:
            self._ratings_per_user = self.transactions / self.n_users
        return self._ratings_per_user

    @property
    def ratings_per_item(self):
        """
        Calculates the average number of ratings per item.
        """
        if self._ratings_per_item is None:
            self._ratings_per_item = self.transactions / self.n_items
        return self._ratings_per_item

    def users_frequency(self):
        """
        Computes the absolute frequency of each user in the dataset.

        Returns:
            (dict): A dictionary mapping user IDs to the number of interactions, 
                sorted in descending order of frequency.
        """
        fr = dict(Counter(self.data[self.user_col]))
        return dict(sorted(fr.items(), key=lambda item: item[1], reverse=True))

    def users_relative_frequency(self):
        """
        Computes the relative frequency of each user in the dataset.

        Returns:
            (dict): A dictionary mapping user IDs to their relative frequency 
                (fraction of total transactions).
        """
        return {u: (f / self.transactions) for u, f in self.users_frequency().items()}

    def items_frequency(self):
        """
        Computes the absolute frequency of each item in the dataset.

        Returns:
            (dict): A dictionary mapping item IDs to the number of interactions, 
                sorted in descending order of frequency.
        """
        fr = dict(Counter(self.data[self.item_col]))
        return dict(sorted(fr.items(), key=lambda item: item[1], reverse=True))

    def items_relative_frequency(self):
        """
        Computes the relative frequency of each item in the dataset.

        Returns:
            (dict): A dictionary mapping item IDs to their relative frequency 
                (fraction of total transactions).
        """
        return {u: (f / self.transactions) for u, f in self.items_frequency().items()}

    def users_quartiles(self):
        """
        Assigns quartile indices to users based on their frequency.

        Returns:
            (dict): A dictionary mapping each user ID to a quartile index (0-3),
                where 0 = lowest, 3 = highest frequency.
        """ 
        return quartiles(self.users_frequency())

    def items_quartiles(self):
        """
        Assigns quartile indices to items based on their frequency.

        Returns:
            (dict): A dictionary mapping each item ID to a quartile index (0-3),
                where 0 = lowest, 3 = highest frequency.
        """
        return quartiles(self.items_frequency())

    def users_popularity(self):
        """
        Categorizes users into descriptive popularity groups based on quartiles.

        Returns:
            (dict): A dictionary mapping popularity categories ('long tail', 
                'common', 'popular', 'most popular') to lists of user IDs.
        """
        return popularity(self.users_quartiles())

    def items_popularity(self):
        """
        Categorizes items into descriptive popularity groups based on quartiles.

        Returns:
            (dict): A dictionary mapping popularity categories ('long tail', 
                'common', 'popular', 'most popular') to lists of item IDs.
        """
        return popularity(self.items_quartiles())

    def copy(self):
        """
        Create a deep copy of the current DataRec object.

        This method duplicates the DataRec instance, including its data,
        metadata (user, item, rating, timestamp columns), pipeline, and internal
        state such as privacy settings and implicit flags.

        Returns:
            (DataRec): A new DataRec object that is a deep copy of the current instance.
        """
        pipeline = self.pipeline.copy()

        new_dr = DataRec(rawdata=self.to_rawdata(),
                         pipeline=pipeline,
                         copy=True)

        new_dr.__implicit = self.__implicit
        new_dr._user_col = self.user_col
        new_dr._item_col = self.item_col
        new_dr._rating_col = self.rating_col
        new_dr._timestamp_col = self.timestamp_col
        new_dr._is_private = self._is_private
        return new_dr

    def to_rawdata(self):
        """
        Convert the current DataRec object into a RawData object.

        This method creates a RawData instance containing the same data and
        metadata (user, item, rating, timestamp columns) as the DataRec object.

        Returns:
            (RawData): A new RawData object containing the DataRec's data and column information.
        """
        raw = RawData(self.data)
        raw.user = self.user_col
        raw.item = self.item_col
        raw.rating = self.rating_col
        raw.timestamp = self.timestamp_col
        return raw

    def save_pipeline(self, filepath: str) -> None:
        """
        Save the current processing pipeline to a YAML file.

        Args:
            filepath (str): The path (including filename) where the pipeline 
                YAML file will be saved.
        """
        print(f'Saving pipeline to {filepath}')

        self.pipeline.to_yaml(filepath)

        print(f'Pipeline correctly saved to {filepath}')

    def to_torch_dataset(self, task="pointwise", autoprepare=True, **kwargs):
        """
        Converts the current DataRec object into a PyTorch-compatible dataset.

        This method prepares the dataset (e.g., remaps user/item IDs to a dense index space)
        and returns a `torch.utils.data.Dataset` object suitable for training with PyTorch.

        Args:
            task (str): The recommendation task type. Must be one of:
                - "pointwise": returns PointwiseTorchDataset
                - "pairwise": returns PairwiseTorchDataset
                - "ranking": returns RankingTorchDataset
            autoprepare (bool): If True, automatically applies user/item remapping
                and switches the dataset to private IDs. If False, assumes the dataset
                is already properly prepared.
            **kwargs: Additional arguments passed to the specific torch dataset class.

        Returns:
            (torch.utils.data.Dataset): A PyTorch dataset instance corresponding to the selected task.

        Raises:
            ImportError: If PyTorch is not installed.
            ValueError: If an unknown task name is provided.
        """

        try:
            import torch
        except ImportError:
            raise ImportError(
                "PyTorch is required to use the to_torch_dataset() method. "
                "Please install it with `pip install torch`."
            )

        # Preparazione automatica del dataset
        if autoprepare:
            self.map_users_and_items()
            self.to_private()
        else:
            warnings.warn(
                "Autoprepare is set to False. "
                "Ensure that the dataset is prepared correctly before using it with PyTorch."
            )

        # Selezione del dataset PyTorch
        if task == "pointwise":
            from datarec.data.torch_dataset import PointwiseTorchDataset
            return PointwiseTorchDataset(self, **kwargs)
        elif task == "pairwise":
            from datarec.data.torch_dataset import PairwiseTorchDataset
            return PairwiseTorchDataset(self, **kwargs)
        elif task == "ranking":
            from datarec.data.torch_dataset import RankingTorchDataset
            return RankingTorchDataset(self, **kwargs)
        else:
            raise ValueError(f"Unknown task: {task}")

data property writable

The underlying pandas DataFrame holding the interaction data.

user_col property writable

The name of the user ID column.

item_col property writable

The name of the item ID column.

rating_col property writable

The name of the rating column.

timestamp_col property writable

The name of the timestamp column.

users property

Returns a list of unique user IDs in the dataset.

items property

Returns a list of unique item IDs in the dataset.

n_users property

Returns the number of unique users.

n_items property

Returns the number of unique items.

columns property writable

Returns the list of column names of the internal DataFrame.

sorted_items property

Returns a dictionary of items sorted by their interaction count.

sorted_users property

Returns a dictionary of users sorted by their interaction count.

transactions property

Returns the total number of interactions (rows) in the dataset.

space_size property

Calculates the scaled square root of the user-item interaction space.

space_size_log property

Calculates the log10 of the space_size metric.

shape property

Calculates the shape of the interaction matrix (n_users / n_items).

shape_log property

Calculates the log10 of the shape metric.

density property

Calculates the density of the user-item interaction matrix.

density_log property

Calculates the log10 of the density metric.

gini_item property

Calculates the Gini coefficient for item popularity.

gini_user property

Calculates the Gini coefficient for user activity.

ratings_per_user property

Calculates the average number of ratings per user.

ratings_per_item property

Calculates the average number of ratings per item.

__init__(rawdata=None, copy=False, dataset_name='datarec', version_name='no_version_provided', pipeline=None, *args, **kwargs)

Initializes the DataRec object.

Parameters:

Name Type Description Default
rawdata RawData

The input dataset wrapped in a RawData object. If None, the DataRec is initialized empty.

 None
copy bool

Whether to copy the input DataFrame to avoid modifying the original RawData.

 False
dataset_name str

A name to identify the dataset.

 'datarec'
version_name str

A version identifier for the dataset.

 'no_version_provided'
pipeline Pipeline

A pipeline object to track preprocessing steps.

 None
Source code in datarec/data/dataset.py 
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
def __init__(
        self,
        rawdata: RawData = None,
        copy: bool = False,
        dataset_name: str = 'datarec',
        version_name: str = 'no_version_provided',
        pipeline: Optional[Pipeline] = None,
        *args,
        **kwargs
):
    """
    Initializes the DataRec object.

    Args:
        rawdata (RawData): The input dataset wrapped in a RawData object.
            If None, the DataRec is initialized empty.
        copy (bool): Whether to copy the input DataFrame to avoid 
            modifying the original RawData.
        dataset_name (str): A name to identify the dataset.
        version_name (str): A version identifier 
            for the dataset.
        pipeline (Pipeline): A pipeline object to track preprocessing steps.

    """



    self.path = None
    self._data = None
    self.dataset_name = dataset_name
    self.version_name = version_name

    if pipeline:
        self.pipeline = pipeline
    else:
        self.pipeline = Pipeline()
        self.pipeline.add_step("load", self.dataset_name, {'version': self.version_name})

    if rawdata is not None:
        if copy:
            self._data: pd.DataFrame = rawdata.data.copy()
        else:
            self._data: pd.DataFrame = rawdata.data

    # ------------------------------------
    # --------- STANDARD COLUMNS ---------
    # if a column is None it means that the DataRec does not have that information
    self.__assigned_columns = []

    self._user_col = None
    self._item_col = None
    self._rating_col = None
    self._timestamp_col = None

    if rawdata:
        self.set_columns(rawdata)

    # dataset is assumed to be the public version of the dataset
    self._is_private = False
    self.__implicit = False

    # ------------------------------
    # --------- PROPERTIES ---------
    self._sorted_users = None
    self._sorted_items = None

    # map users and items with a 0-indexed mapping
    self._public_to_private_users = None
    self._public_to_private_items = None
    self._private_to_public_users = None
    self._private_to_public_items = None

    # metrics
    self._transactions = None
    self._space_size = None
    self._space_size_log = None
    self._shape = None
    self._shape_log = None
    self._density = None
    self._density_log = None
    self._gini_item = None
    self._gini_user = None
    self._ratings_per_user = None
    self._ratings_per_item = None

    # more analyses
    self.metrics = ['transactions', 'space_size', 'space_size_log', 'shape', 'shape_log', 'density', 'density_log',
                    'gini_item', 'gini_user', 'ratings_per_user', 'ratings_per_item']

__str__()

Returns 'self.data' as a string variable.

Returns:

Type Description
str

'self.data' as a string variable.
Source code in datarec/data/dataset.py 
120
121
122
123
124
125
126
127
def __str__(self):
    """
    Returns 'self.data' as a string variable.

    Returns:
        (str): 'self.data' as a string variable.
    """
    return self.data.__str__()

__repr__()

Returns the official string representation of the internal DataFrame.

Source code in datarec/data/dataset.py 
129
130
131
132
133
def __repr__(self):
    """
    Returns the official string representation of the internal DataFrame.
    """
    return self.data.__repr__()

__len__()

Returns the total number of samples in the dataset.

Returns:

Type Description
int

number of samples in the dataset.
Source code in datarec/data/dataset.py 
141
142
143
144
145
146
147
148
def __len__(self):
    """
    Returns the total number of samples in the dataset.

    Returns:
        (int): number of samples in the dataset.
    """
    return len(self.data)

set_columns(rawdata)

Assign dataset column names from a RawData object and reorder the data accordingly.

Parameters:

Name Type Description Default
rawdata RawData

A RawData object containing the column names for user, item, rating, and timestamp.

 required
Source code in datarec/data/dataset.py 
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
def set_columns(self, rawdata):
    """
    Assign dataset column names from a RawData object and reorder the data accordingly.

    Args:
        rawdata (RawData): A RawData object containing the column names for 
            user, item, rating, and timestamp.
    """
    if rawdata.user is not None:
        self.user_col = rawdata.user
        self.__assigned_columns.append(self.user_col)
    if rawdata.item is not None:
        self.item_col = rawdata.item
        self.__assigned_columns.append(self.item_col)
    if rawdata.rating is not None:
        self.rating_col = rawdata.rating
        self.__assigned_columns.append(self.rating_col)
    if rawdata.timestamp is not None:
        self.timestamp_col = rawdata.timestamp
        self.__assigned_columns.append(self.timestamp_col)

    # re-order columns
    self._data = self.data[self.__assigned_columns]

reset()

Reset cached statistics and assigned columns of the DataRec object.

This method clears all precomputed dataset statistics (e.g., sorted users, density, Gini indices, shape, ratings per user/item) and empties the list of assigned columns. It is automatically called when the underlying data is changed.

Source code in datarec/data/dataset.py 
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
def reset(self):

    """
    Reset cached statistics and assigned columns of the DataRec object.

    This method clears all precomputed dataset statistics (e.g., sorted users, 
    density, Gini indices, shape, ratings per user/item) and empties the list 
    of assigned columns. It is automatically called when the underlying data is changed.
    """
    self._sorted_users = None
    self._sorted_items = None
    self._transactions = None
    self._space_size = None
    self._space_size_log = None
    self._shape = None
    self._shape_log = None
    self._density = None
    self._density_log = None
    self._gini_item = None
    self._gini_user = None
    self._ratings_per_user = None
    self._ratings_per_item = None

    self.__assigned_columns = []

set_user_col(value=DATAREC_USER_COL, rename=True)

Identifies and optionally renames the user column.

Parameters:

Name Type Description Default
value Union[str, int]

The current name or index of the user column.

 DATAREC_USER_COL
rename bool

If True, renames the column to the standard internal name.

 True
Source code in datarec/data/dataset.py 
234
235
236
237
238
239
240
241
242
243
244
245
def set_user_col(self, value: Union[str, int] = DATAREC_USER_COL, rename=True):
    """
    Identifies and optionally renames the user column.

    Args:
        value (Union[str, int]): The current name or index of the user column.
        rename (bool): If True, renames the column to the standard internal name.
    """
    self.data.columns, self._user_col = set_column_name(columns=list(self.data.columns),
                                                        value=value,
                                                        default_name=DATAREC_USER_COL,
                                                        rename=rename)

set_item_col(value=DATAREC_ITEM_COL, rename=True)

Identifies and optionally renames the item column.

Parameters:

Name Type Description Default
value Union[str, int]

The current name or index of the item column.

 DATAREC_ITEM_COL
rename bool

If True, renames the column to the standard internal name.

 True
Source code in datarec/data/dataset.py 
261
262
263
264
265
266
267
268
269
270
271
272
def set_item_col(self, value: Union[str, int] = DATAREC_ITEM_COL, rename=True):
    """
    Identifies and optionally renames the item column.

    Args:
        value (Union[str, int]): The current name or index of the item column.
        rename (bool): If True, renames the column to the standard internal name.
    """
    self.data.columns, self._item_col = set_column_name(columns=list(self.data.columns),
                                                        value=value,
                                                        default_name=DATAREC_ITEM_COL,
                                                        rename=rename)

set_rating_col(value=DATAREC_RATING_COL, rename=True)

Identifies and optionally renames the rating column.

Parameters:

Name Type Description Default
value Union[str, int]

The current name or index of the rating column.

 DATAREC_RATING_COL
rename bool

If True, renames the column to the standard internal name.

 True
Source code in datarec/data/dataset.py 
288
289
290
291
292
293
294
295
296
297
298
299
def set_rating_col(self, value: Union[str, int] = DATAREC_RATING_COL, rename=True):
    """
    Identifies and optionally renames the rating column.

    Args:
        value (Union[str, int]): The current name or index of the rating column.
        rename (bool): If True, renames the column to the standard internal name.
    """
    self.data.columns, self._rating_col = set_column_name(columns=list(self.data.columns),
                                                          value=value,
                                                          default_name=DATAREC_RATING_COL,
                                                          rename=rename)

set_timestamp_col(value=DATAREC_TIMESTAMP_COL, rename=True)

Identifies and optionally renames the timestamp column.

Parameters:

Name Type Description Default
value Union[str, int]

The current name or index of the timestamp column.

 DATAREC_TIMESTAMP_COL
rename bool

If True, renames the column to the standard internal name.

 True
Source code in datarec/data/dataset.py 
315
316
317
318
319
320
321
322
323
324
325
326
def set_timestamp_col(self, value: Union[str, int] = DATAREC_TIMESTAMP_COL, rename=True):
    """
    Identifies and optionally renames the timestamp column.

    Args:
        value (Union[str, int]): The current name or index of the timestamp column.
        rename (bool): If True, renames the column to the standard internal name.
    """
    self.data.columns, self._timestamp_col = set_column_name(columns=list(self.data.columns),
                                                             value=value,
                                                             default_name=DATAREC_TIMESTAMP_COL,
                                                             rename=rename)

public_to_private(lst, offset=0) staticmethod

Creates a mapping from public (original) IDs to private (integer) IDs.

Parameters:

Name Type Description Default
lst list

A list of public IDs.

 required
offset int

The starting integer for the private IDs.

 0

Returns:

Type Description
dict

A dictionary mapping public IDs to private integer IDs.
Source code in datarec/data/dataset.py 
400
401
402
403
404
405
406
407
408
409
410
411
412
@staticmethod
def public_to_private(lst, offset=0):
    """
    Creates a mapping from public (original) IDs to private (integer) IDs.

    Args:
        lst (list): A list of public IDs.
        offset (int): The starting integer for the private IDs.

    Returns:
        (dict): A dictionary mapping public IDs to private integer IDs.
    """
    return dict(zip(lst, range(offset, offset + len(lst))))

private_to_public(pub_to_prvt) staticmethod

Creates a reverse mapping from private IDs back to public IDs.

Parameters:

Name Type Description Default
pub_to_prvt dict

A dictionary mapping public IDs to private IDs.

 required

Returns:

Type Description
dict

A dictionary mapping private IDs to public IDs.
Source code in datarec/data/dataset.py 
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
@staticmethod
def private_to_public(pub_to_prvt: dict):
    """
    Creates a reverse mapping from private IDs back to public IDs.

    Args:
        pub_to_prvt (dict): A dictionary mapping public IDs to private IDs.

    Returns:
        (dict): A dictionary mapping private IDs to public IDs.
    """
    mapping = {el: idx for idx, el in pub_to_prvt.items()}
    if len(pub_to_prvt) != len(mapping):
        print('WARNING: private to public mapping could be incorrect. Please, check your code.')
    return mapping

map_users_and_items(offset=0, items_shift=False)

Generates the public-to-private and private-to-public ID mappings.

This method creates the dictionaries needed to convert user and item IDs to a dense, zero-indexed integer range suitable for machine learning models.

Parameters:

Name Type Description Default
offset int

The starting integer for the ID mappings. Defaults to 0.

 0
items_shift bool

If True, item private IDs will start after the last user private ID, creating a single contiguous ID space. Defaults to False.

 False
Source code in datarec/data/dataset.py 
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
def map_users_and_items(self, offset=0, items_shift=False):
    """
    Generates the public-to-private and private-to-public ID mappings.

    This method creates the dictionaries needed to convert user and item IDs
    to a dense, zero-indexed integer range suitable for machine learning models.

    Args:
        offset (int): The starting integer for the ID mappings. Defaults to 0.
        items_shift (bool): If True, item private IDs will start after the last
            user private ID, creating a single contiguous ID space. Defaults to False.
    """
    # map users and items with a 0-indexed mapping
    users_offset = offset
    items_offset = offset

    # users
    self._public_to_private_users = self.public_to_private(self.users, offset=users_offset)
    self._private_to_public_users = self.private_to_public(self._public_to_private_users)

    # items
    if items_shift:
        items_offset = offset + self.n_users
    self._public_to_private_items = self.public_to_private(self.items, offset=items_offset)
    self._private_to_public_items = self.private_to_public(self._public_to_private_items)

map_dataset(user_mapping, item_mapping)

Applies ID mappings to the user and item columns of the DataFrame.

This is an in-place operation that modifies the internal DataFrame.

Parameters:

Name Type Description Default
user_mapping dict

The dictionary to map user IDs.

 required
item_mapping dict

The dictionary to map item IDs.

 required
Source code in datarec/data/dataset.py 
456
457
458
459
460
461
462
463
464
465
466
467
def map_dataset(self, user_mapping, item_mapping):
    """
    Applies ID mappings to the user and item columns of the DataFrame.

    This is an in-place operation that modifies the internal DataFrame.

    Args:
        user_mapping (dict): The dictionary to map user IDs.
        item_mapping (dict): The dictionary to map item IDs.
    """
    self.data[self.user_col] = self.data[self.user_col].map(user_mapping)
    self.data[self.item_col] = self.data[self.item_col].map(item_mapping)

to_public()

Converts user and item IDs back to their original (public) values.

Source code in datarec/data/dataset.py 
469
470
471
472
473
474
475
def to_public(self):
    """
    Converts user and item IDs back to their original (public) values.
    """
    if self._is_private:
        self.map_dataset(self._private_to_public_users, self._private_to_public_items)
    self._is_private = False

to_private()

Converts user and item IDs to their dense, zero-indexed (private) integer values.

Source code in datarec/data/dataset.py 
477
478
479
480
481
482
483
def to_private(self):
    """
    Converts user and item IDs to their dense, zero-indexed (private) integer values.
    """
    if not self._is_private:
        self.map_dataset(self._public_to_private_users, self._public_to_private_items)
    self._is_private = True

get_metric(metric)

Retrieves a calculated dataset metric by name.

Parameters:

Name Type Description Default
metric str

The name of the metric to compute (e.g., 'density', 'gini_user').

 required

Returns:

Type Description

The value of the computed metric.
Source code in datarec/data/dataset.py 
486
487
488
489
490
491
492
493
494
495
496
497
498
def get_metric(self, metric):
    """
    Retrieves a calculated dataset metric by name.

    Args:
        metric (str): The name of the metric to compute (e.g., 'density', 'gini_user').

    Returns:
        The value of the computed metric.
    """
    assert metric in self.metrics, f'{self.__class__.__name__}: metric \'{metric}\' not found.'
    func = getattr(self, metric)
    return func()

gini(x) staticmethod

Calculates the Gini coefficient for a numpy array.

Parameters:

Name Type Description Default
x ndarray

An array of non-negative values.

 required

Returns:

Type Description
float

The Gini coefficient, a measure of inequality.
Source code in datarec/data/dataset.py 
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
@staticmethod
def gini(x):
    """
    Calculates the Gini coefficient for a numpy array.

    Args:
        x (np.ndarray): An array of non-negative values.

    Returns:
        (float): The Gini coefficient, a measure of inequality.
    """
    x = np.sort(x)  # O(n log n)
    n = len(x)
    cum_index = np.arange(1, n + 1)
    return (np.sum((2 * cum_index - n - 1) * x)) / (n * np.sum(x))

users_frequency()

Computes the absolute frequency of each user in the dataset.

Returns:

Type Description
dict

A dictionary mapping user IDs to the number of interactions, sorted in descending order of frequency.
Source code in datarec/data/dataset.py 
608
609
610
611
612
613
614
615
616
617
def users_frequency(self):
    """
    Computes the absolute frequency of each user in the dataset.

    Returns:
        (dict): A dictionary mapping user IDs to the number of interactions, 
            sorted in descending order of frequency.
    """
    fr = dict(Counter(self.data[self.user_col]))
    return dict(sorted(fr.items(), key=lambda item: item[1], reverse=True))

users_relative_frequency()

Computes the relative frequency of each user in the dataset.

Returns:

Type Description
dict

A dictionary mapping user IDs to their relative frequency (fraction of total transactions).
Source code in datarec/data/dataset.py 
619
620
621
622
623
624
625
626
627
def users_relative_frequency(self):
    """
    Computes the relative frequency of each user in the dataset.

    Returns:
        (dict): A dictionary mapping user IDs to their relative frequency 
            (fraction of total transactions).
    """
    return {u: (f / self.transactions) for u, f in self.users_frequency().items()}

items_frequency()

Computes the absolute frequency of each item in the dataset.

Returns:

Type Description
dict

A dictionary mapping item IDs to the number of interactions, sorted in descending order of frequency.
Source code in datarec/data/dataset.py 
629
630
631
632
633
634
635
636
637
638
def items_frequency(self):
    """
    Computes the absolute frequency of each item in the dataset.

    Returns:
        (dict): A dictionary mapping item IDs to the number of interactions, 
            sorted in descending order of frequency.
    """
    fr = dict(Counter(self.data[self.item_col]))
    return dict(sorted(fr.items(), key=lambda item: item[1], reverse=True))

items_relative_frequency()

Computes the relative frequency of each item in the dataset.

Returns:

Type Description
dict

A dictionary mapping item IDs to their relative frequency (fraction of total transactions).
Source code in datarec/data/dataset.py 
640
641
642
643
644
645
646
647
648
def items_relative_frequency(self):
    """
    Computes the relative frequency of each item in the dataset.

    Returns:
        (dict): A dictionary mapping item IDs to their relative frequency 
            (fraction of total transactions).
    """
    return {u: (f / self.transactions) for u, f in self.items_frequency().items()}

users_quartiles()

Assigns quartile indices to users based on their frequency.

Returns:

Type Description
dict

A dictionary mapping each user ID to a quartile index (0-3), where 0 = lowest, 3 = highest frequency.
Source code in datarec/data/dataset.py 
650
651
652
653
654
655
656
657
658
def users_quartiles(self):
    """
    Assigns quartile indices to users based on their frequency.

    Returns:
        (dict): A dictionary mapping each user ID to a quartile index (0-3),
            where 0 = lowest, 3 = highest frequency.
    """ 
    return quartiles(self.users_frequency())

items_quartiles()

Assigns quartile indices to items based on their frequency.

Returns:

Type Description
dict

A dictionary mapping each item ID to a quartile index (0-3), where 0 = lowest, 3 = highest frequency.
Source code in datarec/data/dataset.py 
660
661
662
663
664
665
666
667
668
def items_quartiles(self):
    """
    Assigns quartile indices to items based on their frequency.

    Returns:
        (dict): A dictionary mapping each item ID to a quartile index (0-3),
            where 0 = lowest, 3 = highest frequency.
    """
    return quartiles(self.items_frequency())

users_popularity()

Categorizes users into descriptive popularity groups based on quartiles.

Returns:

Type Description
dict

A dictionary mapping popularity categories ('long tail', 'common', 'popular', 'most popular') to lists of user IDs.
Source code in datarec/data/dataset.py 
670
671
672
673
674
675
676
677
678
def users_popularity(self):
    """
    Categorizes users into descriptive popularity groups based on quartiles.

    Returns:
        (dict): A dictionary mapping popularity categories ('long tail', 
            'common', 'popular', 'most popular') to lists of user IDs.
    """
    return popularity(self.users_quartiles())

items_popularity()

Categorizes items into descriptive popularity groups based on quartiles.

Returns:

Type Description
dict

A dictionary mapping popularity categories ('long tail', 'common', 'popular', 'most popular') to lists of item IDs.
Source code in datarec/data/dataset.py 
680
681
682
683
684
685
686
687
688
def items_popularity(self):
    """
    Categorizes items into descriptive popularity groups based on quartiles.

    Returns:
        (dict): A dictionary mapping popularity categories ('long tail', 
            'common', 'popular', 'most popular') to lists of item IDs.
    """
    return popularity(self.items_quartiles())

copy()

Create a deep copy of the current DataRec object.

This method duplicates the DataRec instance, including its data, metadata (user, item, rating, timestamp columns), pipeline, and internal state such as privacy settings and implicit flags.

Returns:

Type Description
DataRec

A new DataRec object that is a deep copy of the current instance.
Source code in datarec/data/dataset.py 
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
def copy(self):
    """
    Create a deep copy of the current DataRec object.

    This method duplicates the DataRec instance, including its data,
    metadata (user, item, rating, timestamp columns), pipeline, and internal
    state such as privacy settings and implicit flags.

    Returns:
        (DataRec): A new DataRec object that is a deep copy of the current instance.
    """
    pipeline = self.pipeline.copy()

    new_dr = DataRec(rawdata=self.to_rawdata(),
                     pipeline=pipeline,
                     copy=True)

    new_dr.__implicit = self.__implicit
    new_dr._user_col = self.user_col
    new_dr._item_col = self.item_col
    new_dr._rating_col = self.rating_col
    new_dr._timestamp_col = self.timestamp_col
    new_dr._is_private = self._is_private
    return new_dr

to_rawdata()

Convert the current DataRec object into a RawData object.

This method creates a RawData instance containing the same data and metadata (user, item, rating, timestamp columns) as the DataRec object.

Returns:

Type Description
RawData

A new RawData object containing the DataRec's data and column information.
Source code in datarec/data/dataset.py 
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
def to_rawdata(self):
    """
    Convert the current DataRec object into a RawData object.

    This method creates a RawData instance containing the same data and
    metadata (user, item, rating, timestamp columns) as the DataRec object.

    Returns:
        (RawData): A new RawData object containing the DataRec's data and column information.
    """
    raw = RawData(self.data)
    raw.user = self.user_col
    raw.item = self.item_col
    raw.rating = self.rating_col
    raw.timestamp = self.timestamp_col
    return raw

save_pipeline(filepath)

Save the current processing pipeline to a YAML file.

Parameters:

Name Type Description Default
filepath str

The path (including filename) where the pipeline YAML file will be saved.

 required
Source code in datarec/data/dataset.py 
732
733
734
735
736
737
738
739
740
741
742
743
744
def save_pipeline(self, filepath: str) -> None:
    """
    Save the current processing pipeline to a YAML file.

    Args:
        filepath (str): The path (including filename) where the pipeline 
            YAML file will be saved.
    """
    print(f'Saving pipeline to {filepath}')

    self.pipeline.to_yaml(filepath)

    print(f'Pipeline correctly saved to {filepath}')

to_torch_dataset(task='pointwise', autoprepare=True, **kwargs)

Converts the current DataRec object into a PyTorch-compatible dataset.

This method prepares the dataset (e.g., remaps user/item IDs to a dense index space) and returns a torch.utils.data.Dataset object suitable for training with PyTorch.

Parameters:

Name Type Description Default
task str

The recommendation task type. Must be one of: - "pointwise": returns PointwiseTorchDataset - "pairwise": returns PairwiseTorchDataset - "ranking": returns RankingTorchDataset

 'pointwise'
autoprepare bool

If True, automatically applies user/item remapping and switches the dataset to private IDs. If False, assumes the dataset is already properly prepared.

 True
**kwargs

Additional arguments passed to the specific torch dataset class.

 {}

Returns:

Type Description
Dataset

A PyTorch dataset instance corresponding to the selected task.

Raises:

Type Description
ImportError

If PyTorch is not installed.
ValueError

If an unknown task name is provided.
Source code in datarec/data/dataset.py 
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
def to_torch_dataset(self, task="pointwise", autoprepare=True, **kwargs):
    """
    Converts the current DataRec object into a PyTorch-compatible dataset.

    This method prepares the dataset (e.g., remaps user/item IDs to a dense index space)
    and returns a `torch.utils.data.Dataset` object suitable for training with PyTorch.

    Args:
        task (str): The recommendation task type. Must be one of:
            - "pointwise": returns PointwiseTorchDataset
            - "pairwise": returns PairwiseTorchDataset
            - "ranking": returns RankingTorchDataset
        autoprepare (bool): If True, automatically applies user/item remapping
            and switches the dataset to private IDs. If False, assumes the dataset
            is already properly prepared.
        **kwargs: Additional arguments passed to the specific torch dataset class.

    Returns:
        (torch.utils.data.Dataset): A PyTorch dataset instance corresponding to the selected task.

    Raises:
        ImportError: If PyTorch is not installed.
        ValueError: If an unknown task name is provided.
    """

    try:
        import torch
    except ImportError:
        raise ImportError(
            "PyTorch is required to use the to_torch_dataset() method. "
            "Please install it with `pip install torch`."
        )

    # Preparazione automatica del dataset
    if autoprepare:
        self.map_users_and_items()
        self.to_private()
    else:
        warnings.warn(
            "Autoprepare is set to False. "
            "Ensure that the dataset is prepared correctly before using it with PyTorch."
        )

    # Selezione del dataset PyTorch
    if task == "pointwise":
        from datarec.data.torch_dataset import PointwiseTorchDataset
        return PointwiseTorchDataset(self, **kwargs)
    elif task == "pairwise":
        from datarec.data.torch_dataset import PairwiseTorchDataset
        return PairwiseTorchDataset(self, **kwargs)
    elif task == "ranking":
        from datarec.data.torch_dataset import RankingTorchDataset
        return RankingTorchDataset(self, **kwargs)
    else:
        raise ValueError(f"Unknown task: {task}")

share_info(datarec_source, datarec_target)

Copy dataset metadata and mappings from one DataRec object to another

This function transfers core attributes from a source DataRec to a target DataRec.

Parameters:

Name Type Description Default
datarec_source DataRec

The source DataRec from which information is copied.

 required
datarec_target DataRec

The target DataRec that will be updated with the source information.

 required
Source code in datarec/data/dataset.py 
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
def share_info(datarec_source: DataRec, datarec_target: DataRec) -> None:
    """
    Copy dataset metadata and mappings from one DataRec object to another

    This function transfers core attributes from a source DataRec
    to a target DataRec. 

    Args:
        datarec_source (DataRec): The source DataRec from which information 
            is copied.
        datarec_target (DataRec): The target DataRec that will be updated 
            with the source information.

    """
    ds = datarec_source
    dt = datarec_target

    dt._is_private = ds._is_private
    dt.__implicit = ds.__implicit

    dt.dataset_name = ds.dataset_name
    dt.version_name = ds.version_name
    dt.user_col = ds.user_col
    dt.item_col = ds.item_col
    dt.rating_col = ds.rating_col
    dt.timestamp_col = ds.timestamp_col

    dt._sorted_users = ds._sorted_users
    dt._sorted_items = ds._sorted_users
    dt._public_to_private_users = ds._public_to_private_users
    dt._public_to_private_items = ds._public_to_private_items
    dt._private_to_public_users = ds._private_to_public_users
    dt._private_to_public_items = ds._private_to_public_items