Home / Homepage For You / HP4U Content Moderation Tool (Potential Inventory)

HP4U Content Moderation Tool (Potential Inventory)

OVERVIEW

The Content Moderation Tool (CMT), aka Potential Inventory, is an application that allows publishers to review what articles, galleries, blogs, videos etc are currently eligible for recommendation by Homepage For You on sites or native apps. In addition to taking actions like blocking an article or marking it safe, the CMT includes real time engagement metrics. It also has custom algorithms designed to ensure content freshness and that only the most recent versions of articles are recommendable.

The guidelines for recommendations are set by policies which are created in Policy Builder — such as: Include only staff articles from category sports in the last 36 hours. These are attached to regions on pages via the Taboola Newsroom Chrome Extension. The CMT, however, is the portal which allows an editor to either review the totality of that site’s inventory or specifically what content is available for recommendation in a specific region, like Top News.

Beyond its pragmatic uses, the true purpose of the Content Moderation Tool is to ensure AI-assisted recommendations are not a black box. Publishers should be able to not only control and understand how the recommendations are made, but have complete transparency into what could be recommended and exactly what has been recommended.

 

ACCESS

The Content Moderation Tool is built within Taboola Newsroom, Taboola’s real time engagement and insights platform. Anyone with a Newsroom account can access the CMT, but a user must have the Newsroom roles of Supervisor, Supervisor with Homepage Edit or Admin access to take actions like blocking content.

To access the Content Moderation Tool, open Newsroom and mouse over the left rail navigation bar. The tool is nested under the Homepage For You heading labeled Potential Inventory.

 

 

PRIMARY DASHBOARD

The primary interface of the Content Moderation Tool is a spreadsheet-like grid displaying the most recently published content at the top. The interface is divided into a series columns based on metadata or engagement data on each piece of content.

 

Those columns are as follows:

  • Media: The mapped thumbnail image.
  • Title/Description: The mapped headline is visible. Hover over the I icon to review the description. A user also can launch the content-level page in a new tab by hovering on the headline and clicking on the open icon (below).

  • Publish Date: The publish date or last updated timestamp of the article, including a time-since-publication counter.
  • Expiration: The anticipated time and date that the article will no longer be in the potential recommendation inventory. This expiration date could change if the article is updated.
  • Impressions: The number of times this article was recommended and served on a page in the last three hours. This includes visible and non-visible impressions. An article that is eligible for recommendation but has not been recommended will have dashes in the Impressions, Clicks and CTR columns.

  • Clicks: The number of times a user clicked on a recommended article.
  • CTR: The number of clicks divided by the number impressions.
  • Tags: Keywords mapped in the metadata.
  • Category: The section mapped in the metadata.
  • Moderation tools: An editor can mark an item as safe, block an item or mark an item “Needs Attention” for later review. Needs attention and safe are only an internal marker for moderators. Blocking an item will immediately remove it from inventory across all homepage regions. To unblock a blocked item, reclick the block icon to undo the action.

In addition to blocking a single item, a user can bulk block items by clicking on the multiselect icons to the left of the thumbnail image in each row. When at least one content item is selected, a block icon will appear above the image column. Clicking that will allow a user to either confirm or cancel the bulk block action.

 

NAVIGATION

The Content Moderation Tool’s default dashboard view includes content from all regions. It appears to be unfiltered but it actually includes only two categories — “Needs attention” and new items where no action has been taken. Statuses like “Blocked items” or “Safe” don’t appear in the default view. To review those, a user needs to add filters, which can be activated from the funnel icon in the top right.

A. Filters

The CMT features two types of filters — one based on page/region and one based on status. Filters also can be stacked, so if you want to show only “blocked” items from the region “sports”, an editor can do so by adding both filters.

Page/Region filters: To review the inventory in a region, select Page: Home and then the desired region.

Status filters: A user can either include or exclude by status by applying a “Show only” or “Hide” filter and then selecting one of four statuses.

  • Needs attention
  • Safe
  • Blocked (manual block action taken)
  • Blocked AI (blocked by an autoblocking AI)

Below is an example of reviewing inventory in the Business region and then filtering only on blocked items.

 

B. Search

The search function, on the upper right, is limited to the inventory being queried (not just displayed on screen) and will check all data in all columns. A search of “Lakers” will search headline, category, tags etc. However, if a user is searching for an article which has been blocked but you are not including blocked content in your query, they will fail to find the article.

 

ADJUSTING EXPIRATION

While the expiration is based on published time and the policy rules, as a user, you can make adjustments. If you want an article to expire from inventory ahead of the expiration date, hover over the time and date in the Expiration column and an edit symbol will indicate you can make an adjustment. It also will display a tooltip of time zone in which the publisher is operating. Once time/date is clicked, a UI opens allowing the user to shorten the expiration time to within 1 hour. For more urgent issues, simply block the article. Currently, a user cannot extend the expiration time and date.

 

Once the user clicks Add Expiration, the inventory will be refreshed and the new expiration time and date will be displayed, along with a blue dot indicating that this field has been adjusted. When that time and date are reached in the publisher’s time zone, that content will be flagged as expired and will no longer be eligible for recommendation.

If you want to clear the manually shortened expiration time and date and revert to the original expiration, click on the time/date and again to open the edit UI and select Clear Expiration. The article will once again be eligible for recommendation.

 

AUTOBLOCKING ALGORITHMS

In order to recommend the freshest content, the Content Moderation Tools also features two unique algorithms. These are currently only available to English language publishers.

  • The first looks for heuristic context in the title or description to determine if there’s a time-sensitive nature. Example: An article about what to watch on TV this weekend shouldn’t be recommendable on Monday.
  • Another focuses on identifying articles which have been outdated by developments in newer separate articles. Example: An article about opening statements of a court case should not be served if a jury already has convicted the defendant.

If an item fails any of these checks, which run every 30 minutes, it is marked with a “Blocked items (beta)” status and is no longer in the recommendable inventory.

 

BEST PRACTICES

Reviewing the recommendation inventory requires a very different mindset than curating positions on a homepage. An editor should not block an item just because it was in inventory yesterday, for example. The recommendation algorithm already works to optimize a personalized experience and mitigate a single user seeing old content.

In practice, the decisions should be based more on the following questions:

  • Is the information in the article still correct at this moment? Take no action or mark as safe
  • Are there still a reasonable number of users for whom this content is still compelling or relevant? Take no action or mark as safe
  • Is there already a newer item that makes the current item not relevant? Block the item
  • Is this a complimentary item, like a sidebar or an opinion piece, related to another content item? Take no action or mark as safe
  • Will this item need review shortly? Mark as needs attention

It’s useful to remember that many readers are not following news on a minute-to-minute basis, so articles that may seem old to highly-informed editors could still be of interest to many readers. The larger the inventory of available items, the greater the site can be personalized. And this results in not only the biggest engagement gains but the best overall reader experience.