Recommend using Gorse

Each components and concepts of Gorse will be introduced in this section.

Users, Items and Feedback

A recommender system is expected to recommend items to users. To learn the peference of each user, feedbacks between users and items are feed to recommender system. In Gorse, there are three types of entities.

  • User: A user is identified by a string identifier.
type User struct {
	UserId    string
}
  • Item: A item is identified by a string identifier. A timestamp is used to record the freshness of this item. The timestamp could be last update time, release time, etc. Labels are used to describe characters of this item, eg., tags of a movie.
type Item struct {
	ItemId    string
	Timestamp time.Time
	Labels    []string
}
  • Feedback: A feedback is identified a triple: feedback type, user ID and item ID. The type of feedback can be positive (like), negative (dislike) or neutural (read). The timestamp record the time that this feedback happened.
type Feedback struct {
	FeedbackType string
	UserId       string
	ItemId       string
	Timestamp   time.Time
}

Types of feedbacks are classified to three categories:

  1. positive_feedback_types mean a user favors a item.
  2. click_feedback_types mean a user favors a recommended item. This item must be recommended by Gorse.
  3. read_feedback_type means a user read a item. However, the real feedback this user has in his/her mind is never known.

The difference between positive_feedback_types and click_feedback_types is that the item of click_feedback_types must come from recommendations of Gorse. The item of positive_feedback_types could be found by a user through other approaches such as search, direct access, etc. read_feedback_type is a neutral event. Negative feedback can be conduct by {read_feedback_type items} - {positive_feedback_types items}.

There might be extra field in the defined structure. They are preserved for future usage.

Workflow

The main workflow of Gorse is as follows:

  1. Feedbacks generated by users are collected to data store.
  2. Archived feedbacks are pulled to train recommender model. There are two type of models (ranking model and CTR model) in Gorse, they are treated as one here.
  3. Offline recommendations are generated in background from all items and cached.
  4. Online recommendations are returned to users in real-time based on cached offline recommendations.

Data Storage

There are two types of storage used in Gorse: data store and cache store.

Data Store

The data_store is used to store items, users, feedbacks and measurements. Currently, MySQL and MongoDB are supprted as data store. Other database will be avaiable once its interface is implemented.

Unfortunately, there are two challenges in data storage:

  1. What if a feedback with unknown user or item is inserted? There are two options auto_insert_user and auto_insert_item to control feedback insertion. If new users or items insertion is forbided, a feedback with new user or item will be ignored.

  2. How to address stale feedback and items? Some items and its feedbacks are short-lived such as news. positive_feedback_ttl and item_ttl are used to ignore stale feedback and items when pulling dataset from data store.

Cache Store

The cache_store is used to store offline recommendation and temp variables. Only Redis is supported. Latest items, popular items, similar items and recommended items are cached in Redis. The length of each cached list is cache_size.

Recommendation

Recommended items come from multiple sources through multiple stages. Non-personalized recommendations (popular/latest/similar) are generated by the master node. Offline personalized recommendations are generated by worker nodes while online personalized recommendations are generated by server nodes.

Items with maximum number of users will are collected. To avoid popular items resist on the top list, popular_window restricts that timestamps of collected items must be after popular_window days ago. There will be no timestamp restriction if popular_window is 0.

Latest Items

Items with latest timestamps are collected. Items won't be added to latest items collection if their timestamp is empty.

Similar Items

For each item, top n (n equals cache_size) similar items are collected. In current implementation, the similarity between items are the number of common users of two items1.

Offline Recommendation

Worker nodes collect top n items from all items and save them to cache. Besides, latest items are added to address the cold-start problem in recommender system. When labels of items exists, the CTR prediction model is enabled, vice versa. The procedure of offline recommendatin is different depend on whether the CTR model is enabled.

If the CTR model is enabled:

  1. Collect top cache_size items from unseen items of currrent user using ranking model.
  2. Append explore_latest_num latest items to the collection.
  3. Rerank collected items using the CTR prediction model.

If the CTR model is disabled:

  1. Collect top cache_size items from unseen items of currrent user using ranking model.
  2. Insert explore_latest_num latest items to random positions in the collection.

Offline recommendation cache will be consumed by users and fashion will change. Offline recommendation will be refreshed under one of these two conditions:

  • The timestamp of offline recommendation has been refresh_recommend_period days ago.
  • New feedbacks have been inserted since the timestamp of offline recommendation.

There are 4 ranking models (BPR2/ALS3/CCD4) and 1 CTR model (factorization machines5) in Gorse. They will be applied automatically by model searcher. In ranking models, items and users are represented as embedding vectors. Since the dot product between two vectors is fast, ranking models are used to find top N items among all items. In CTR models, features from users or items are used in prediction. It's expensive to use CTR models to predict scores of all items.

Online Recommendation

The online recommendation in the server node consists of three stages:

  1. Load offline recommendation from cache, remove read items.
  2. If the number of offline recommendations is less than required, collect items similar to these items in the user's histrical feedbacks. Read items are removed as well.
  3. If the number of recommendations is still less than required, collect items from fallback_recommend (latest items or popular items). Read items are removed.

Model Update

There are two kinds of models in Gorse, but the training and hyperparameters optimization procedures are quite the same.

Model Training

Model training are done by the master node, as well as model search. The master node pull data from database and fit ranking model and CTR model periodically.

  • For every fit_jobs minutes:
    • Pull data from database.
      • Train model with hyperparameters found by model search using fit_jobs jobs.

There are many hyperparameters for each recommendation model in Gorse. However, it is hard to configuare these hyperparameters manually even for machine learning experts. To help users get rid of hyperparameters tuning, Gorse integrates random search6 for hyperparameters optimization. The procedure of model search is as following:

  • For every search_period minutes:
    • Pull data from database.
    • For every recommender models:
      • For search_trials trials:
        • Sample a hyperparameter combination.
        • Train model with sampled hyperparameters by search_epoch epoches and search_jobs jobs.
        • Update best model.

Online Evaluation

The only method to estimate recommendation performance is online evaluation. The metric of online evaluation in Gorse is click-through-rate: click feedback / read feedback.

1

Zhang, Zhenghao, et al. "SANS: Setwise Attentional Neural Similarity Method for Few-Shot Recommendation." DASFAA (3). 2021.

2

Rendle, Steffen, et al. "BPR: Bayesian personalized ranking from implicit feedback." Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence. 2009.

3

Hu, Yifan, Yehuda Koren, and Chris Volinsky. "Collaborative filtering for implicit feedback datasets." 2008 Eighth IEEE International Conference on Data Mining. Ieee, 2008.

4

He, Xiangnan, et al. "Fast matrix factorization for online recommendation with implicit feedback." Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. 2016.

5

Rendle, Steffen. "Factorization machines." 2010 IEEE International Conference on Data Mining. IEEE, 2010.

6

Bergstra, James, and Yoshua Bengio. "Random search for hyper-parameter optimization." Journal of machine learning research 13.2 (2012).