Home / Taboola Push / Potential Inventory, aka Content Moderation Tool

Potential Inventory, aka Content Moderation Tool

OVERVIEW

Potential Inventory, AKA the Content Moderation Tool, is an application that allows publishers to review what articles, galleries, blogs, videos etc are currently eligible for recommendation by Homepage For You or automated personalized push on sites or native apps. Its primary use is around taking actions like blocking an article or shortening its timeframe for recommendation. For HP4U, it also has custom algorithms designed to ensure content freshness and that only the most recent versions of articles are recommendable.

The guidelines for recommendations are set by policies which are created in Policy Builder — such as: Include only staff articles from category sports in the last 36 hours. For HP4U, these are attached to regions on pages via the Taboola Newsroom Chrome Extension.

The CMT, however, is the portal which allows an editor to either review the totality of that site’s inventory, or specifically examine what content is available for recommendation in a specific region, like Top News, or policy, like the one used to manage automated personalized push.

Beyond its pragmatic uses, the true purpose of the Content Moderation Tool is to ensure AI-assisted recommendations are not a black box. Publishers should be able to not only control and understand how the recommendations are made, but have complete transparency into what could be recommended and exactly what has been recommended.

ACCESS

The Content Moderation Tool is built within Taboola Newsroom, Taboola’s real time engagement and insights platform. Anyone with a Newsroom account can access the CMT, but a user must have the Newsroom roles of Supervisor, Supervisor with Homepage Edit or Admin access to take actions like blocking content.

To access the Content Moderation Tool, open Newsroom and mouse over the left rail navigation bar. The tool is nested under the Homepage For You or Push Notification headings labeled “Potential Inventory.”

PRIMARY DASHBOARD

The primary interface of the Content Moderation Tool is a spreadsheet-like grid displaying the most recently published content at the top. The interface is divided into a series columns based on metadata or engagement data on each piece of content.

Those columns are as follows:

  • Media: The mapped thumbnail image.
  • Title/Description: The mapped headline is visible. Hover over the I icon to review the description. A user also can launch the content-level page in a new tab by hovering on the headline and clicking on the open icon (below).

  • Status:This column denotes whether an item is Active or User Blocked. For HP4U, an item also can be Timeliness Blocked or Similarity Blocked

  • Publish Date: The publish date or last updated timestamp of the article, including a time-since-publication counter.
  • Expiration: The anticipated time and date that the article will no longer be in the potential recommendation inventory. This expiration date could change if the article is updated.
  • Category: The section mapped in the metadata.
  • Action An editor block or unblock an item from this view. Blocking an item will immediately remove it from inventory across all homepage regions or a push policy. To unblock a blocked item, reclick the block icon to undo the action.

In addition to blocking a single item, a user can bulk block items by clicking on the multiselect icons to the left of the thumbnail image in each row. When at least one content item is selected, a block icon will appear above the image column. Clicking that will allow a user to either confirm or cancel the bulk block action.

FILTERS AND NAVIGATION

The Content Moderation Tool by default shows includes content from all regions or policies on the initial load. The default view hides blocked items, but you can enable showing blocked items in the top right navigation bar.

The two primary ways to navigate inventories of recommendation eligible content is either by region (HP4U) or by policy (Taboola Push).

To filter by page or region, click the funnel icon in the top right of the window. Then, select either Region or Policy and then select either to review the articles which are eligible for recommendation.

The following flow shows how to review the articles in the homepage’s Awesome Gagets region.

The search function, on the upper right, is limited to the inventory being queried (not just displayed on screen) and will check all data in all columns. A search of “Lakers” will search headline, category etc. However, if a user is searching for an article which has been blocked but you are not including blocked content in your query, they will fail to find the article.

ADJUSTING EXPIRATION

While the expiration is based on published time and the policy rules, as a user, you can make adjustments. If you want an article to expire from inventory ahead of the expiration date, hover over the time and date in the Expiration column and an edit symbol will indicate you can make an adjustment. It also will display a tooltip of time zone in which the publisher is operating. Once time/date is clicked, a UI opens allowing the user to shorten the expiration time to within 1 hour. For more urgent issues, simply block the article. Currently, a user cannot extend the expiration time and date.

Once the user clicks Add Expiration, the inventory will be refreshed and the new expiration time and date will be displayed, along with a blue dot indicating that this field has been adjusted. When that time and date are reached in the publisher’s time zone, that content will be flagged as expired and will no longer be eligible for recommendation.

If you want to clear the manually shortened expiration time and date and revert to the original expiration, click on the time/date and again to open the edit UI and select Clear Expiration. The article will once again be eligible for recommendation.

AUTOBLOCKING ALGORITHMS

In order to recommend the freshest content, the Content Moderation Tools also features two unique algorithms. These are currently only available to English language publishers and only for Homepage For You.

  • The first looks for heuristic context in the title or description to determine if there’s a time-sensitive nature. Example: An article about what to watch on TV this weekend shouldn’t be recommendable on Monday.
  • Another focuses on identifying articles which have been outdated by developments in newer separate articles. Example: An article about opening statements of a court case should not be served if a jury already has convicted the defendant.

If an item fails any of these checks, which run every 30 minutes, it is marked with a blocked status and is no longer in the recommendable inventory.

BEST PRACTICES

Reviewing the recommendation inventory requires a very different mindset than curating positions on a homepage. An editor should not block an item just because it was in inventory yesterday, for example. The recommendation algorithm already works to optimize a personalized experience and mitigate a single user seeing old content.

In practice, the decisions should be based more on the following questions:

  • Is the information in the article still correct at this moment? Take no action
  • Are there still a reasonable number of users for whom this content is still compelling or relevant? Take no action
  • Is there already a newer item that makes the current item not relevant? Block the item
  • Is this a complementary item, like a sidebar or an opinion piece, related to another content item? Take no action

It’s useful to remember that many readers are not following news on a minute-to-minute basis, so articles that may seem old to highly-informed editors could still be of interest to many readers. The larger the inventory of available items, the greater the site can be personalized. And this results in not only the biggest engagement gains but the best overall reader experience.