As I've been discussing over the last months, I personally believe measuring non-accuracy aspects of intelligent systems is one of the most important challenges to make algorithmic recommendations ethical.

• "Diversity" Means More Than What We Typically Think
• Validate, Validate, and Validate Data. But, in terms of what?
• Recommender Diversity is NOT Inversion of Similarity

Now, let's dive deep into the most ambiguous type of non-accuracy recommender metric: Serendipity.

By definition, we can say recommendation is serendipitous if and only if it's (1) relevant to a user AND (2) unexpected for them.

First, a serendipitous item should be not yet discovered and not be expected by the user [unexpected]; secondly, the item should also be interesting, relevant and useful to the user [relevant].

Source: "Beyond Accuracy: Evaluating Recommender Systems by Coverage and Serendipity" (2010)

Hence, the practitioners need to consider what defines relevance and unexpectedness of a recommended item, on a case-by-case basis. This is a big open-ended question, and that's why the concept of serendipity is rarely discussed both in academia and industry.

It should be noted that Serendipity is not the only metric that goes "beyond accuracy". For instance, Novelty and Diversity can also be employed to measure the goodness of recommendation as you can find at a dedicated section in the Recommender Systems Handbook.

Importantly, regardless of the measurements, capturing the user's taste/desire in the form of relevance is a must. That is, it is not a good idea to naively show random irrelevant contents without taking user-centricity into account for the novelty and diversity sake.

Meanwhile, novelty and diversity are essentially different from serendipity in terms of unexpectedness i.e., how "obvious" the recommendation is. Here is the citation from the "Beyond Accuracy" paper again:

[...] movie is then called novel recommendation instead of serendipitous recommendation because he might discover this movie by himself. [...] It is then called a diverse recommendation instead of a serendipitous recommendation as he might be not surprised about the recommendation.

• If the recommendation is novel, a user hasn't seen the content yet. However, they will eventually have an opportunity to interact with the item soon-ish, even if the recommender doesn't help anything.
• If the recommendation is diverse, it definitely helps users to break their unconscious bias and expand their "solution space", but the recommended items are still within the range of predictable outcomes.

Therefore, going beyond user's expectation is a unique, foundational requirement for serendipitous recommenders, and such a candidate must be in a tiny fraction of the entire relevant contents.

Notice that, since serendipitous recommendation makes users surprised, it's important to consider how to display the recommendation e.g., along with proper explanation. Otherwise, users would have no idea of what the recommendation is, and it unlikely becomes a positive "aha moment" for them.

Last but not least, there are several variations of the definition of serendipity and its relationship to the other metrics. To give an example, "A Survey of Serendipity in Recommender Systems" illustrates the problem spaces as follows, which extends the discussion to e.g., irrelevant unexpected items and familiar items that aren't new to a user.

* Image source: Kotkov, et al. "A Survey of Serendipity in Recommender Systems" Fig. 1.

  Share

Tweet

  Categories

Recommender Systems Data Science & Analytics Machine Learning

  See also

2022-04-03

Cross Validation for Recommender Systems in Julia

2022-02-27

Recommender Diversity is NOT Inversion of Similarity

2021-07-15

Reviewing Ethical Challenges in Recommender Systems

  More

Last updated: 2022-05-07

  Author: Takuya Kitazawa

Takuya Kitazawa is a freelance software developer, previously working at a Big Tech and Silicon Valley-based start-up company where he wore multiple hats as a full-stack software developer, machine learning engineer, data scientist, and product manager. At the intersection of technological and social aspects of data-driven applications, he is passionate about promoting the ethical use of information technologies through his mentoring, business consultation, and public engagement activities. See CV for more information, or contact at [email protected].

  Support by donation   Gift a cup of coffee

  Disclaimer

• Opinions are my own and do not represent the views of organizations I am/was belonging to.
• I am doing my best to ensure the accuracy and fair use of the information. However, there might be some errors, outdated information, or biased subjective statements because the main purpose of this blog is to jot down my personal thoughts as soon as possible before conducting an extensive investigation. Visitors understand the limitations and rely on any information at their own risk.
• That said, if there is any issue with the content, please contact me so I can take the necessary action.